2024-09-10T10:32:08.618 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T10:32:08.640 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:32:08.738 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045 branch: wip-rishabh-testing-20240909.165402-squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} email: null first_in_suite: false job_id: '7899045' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only - with standby daemon mds - MDS abort because newly corrupt dentry sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE05KKoqvorJC3LwqhYbF1M4Qu4oLB2lI++HVNTrmGs7EPzpOl87/THOrywtJbeYLBC1lXf3LWYXzQcuZ6IitEg= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRzMU+BWBryOKI1ABWbsEP1IbvGqek2HnOJKavGiuVVOURDVYOTuXJ/1hSAwYmZUErENqDrE4pAhv5lw+zMyGE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T10:32:08.738 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T10:32:08.739 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T10:32:08.739 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T10:32:08.740 INFO:teuthology.task.internal:Checking packages... 2024-09-10T10:32:08.767 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T10:32:08.767 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T10:32:08.767 INFO:teuthology.packaging:ref: None 2024-09-10T10:32:08.767 INFO:teuthology.packaging:tag: None 2024-09-10T10:32:08.767 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:32:08.767 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:32:08.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T10:32:08.916 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T10:32:08.918 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T10:32:08.925 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T10:32:08.926 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T10:32:08.935 INFO:teuthology.task.internal:Saving configuration 2024-09-10T10:32:08.957 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T10:32:08.963 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T10:32:08.988 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:27:19.700411', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE05KKoqvorJC3LwqhYbF1M4Qu4oLB2lI++HVNTrmGs7EPzpOl87/THOrywtJbeYLBC1lXf3LWYXzQcuZ6IitEg='} 2024-09-10T10:32:09.014 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 10:27:19.698622', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRzMU+BWBryOKI1ABWbsEP1IbvGqek2HnOJKavGiuVVOURDVYOTuXJ/1hSAwYmZUErENqDrE4pAhv5lw+zMyGE='} 2024-09-10T10:32:09.014 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T10:32:09.020 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T10:32:09.021 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T10:32:09.021 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T10:32:09.084 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f758011fbe0>, signals=[15]) 2024-09-10T10:32:09.084 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T10:32:09.089 INFO:teuthology.task.internal:Opening connections... 2024-09-10T10:32:09.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:32:09.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:32:09.172 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:32:09.173 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:32:09.247 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T10:32:09.254 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-09-10T10:32:09.272 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-09-10T10:32:09.272 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-09-10T10:32:09.330 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-09-10T10:32:09.330 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-09-10T10:32:09.330 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:32:09.331 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:32:09.332 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-09-10T10:32:09.356 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-09-10T10:32:09.375 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-09-10T10:32:09.375 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-09-10T10:32:09.432 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T10:32:09.433 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T10:32:09.434 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T10:32:09.434 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-09-10T10:32:09.456 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T10:32:09.465 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T10:32:09.475 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T10:32:09.475 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:32:09.478 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-09-10T10:32:09.494 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T10:32:09.499 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T10:32:09.500 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:32:09.535 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-09-10T10:32:09.562 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T10:32:09.608 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T10:32:09.622 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T10:32:09.622 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T10:32:09.622 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T10:32:09.623 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:32:09.623 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T10:32:09.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:32:09.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T10:32:09.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:32:09.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T10:32:09.867 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:09.867 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T10:32:09.868 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-10T10:32:09.869 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:09.869 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T10:32:09.869 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-10T10:32:09.887 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:32:09.887 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:09.887 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T10:32:09.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-09-10T10:32:09.889 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-503.el9.x86_64 2024-09-10T10:32:09.889 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:09.889 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T10:32:09.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum upgrade -y linux-firmware 2024-09-10T10:32:12.223 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-10T10:32:12.240 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-10T10:32:14.699 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 15 MB/s | 20 MB 00:01 2024-09-10T10:32:16.991 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-10T10:32:19.250 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T10:32:21.101 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 19 kB 00:00 2024-09-10T10:32:22.197 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T10:32:22.441 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-10T10:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-10T10:32:25.995 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T10:32:27.870 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T10:32:30.063 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-10T10:32:30.064 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-09-10T10:32:30.065 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-10T10:32:30.180 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T10:32:30.180 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:30.337 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T10:32:30.338 DEBUG:teuthology.orchestra.run.smithi028:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T10:32:30.340 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi028.front.sepia.ceph.com to complete... 2024-09-10T10:32:31.409 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T10:32:33.509 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-10T10:32:33.509 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-10T10:32:33.510 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-10T10:32:33.619 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T10:32:33.619 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T10:32:33.768 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T10:32:33.768 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T10:32:33.771 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-09-10T10:32:38.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10 10:32:38 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T10:32:38.504 INFO:teuthology.orchestra.run.smithi028.stderr:FINISHED --2024-09-10 10:32:38-- 2024-09-10T10:32:38.504 INFO:teuthology.orchestra.run.smithi028.stderr:Total wall clock time: 8.1s 2024-09-10T10:32:38.504 INFO:teuthology.orchestra.run.smithi028.stderr:Downloaded: 1 files, 343M in 7.8s (43.7 MB/s) 2024-09-10T10:32:38.507 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T10:32:38.507 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T10:32:40.720 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying... ######################################## 2024-09-10T10:32:40.842 INFO:teuthology.orchestra.run.smithi028.stdout:Preparing... ######################################## 2024-09-10T10:32:40.954 INFO:teuthology.orchestra.run.smithi028.stdout:Updating / installing... 2024-09-10T10:32:42.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10 10:32:42 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T10:32:42.565 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-09-10 10:32:42-- 2024-09-10T10:32:42.565 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 8.8s 2024-09-10T10:32:42.565 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 343M in 8.3s (41.1 MB/s) 2024-09-10T10:32:42.568 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T10:32:42.569 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T10:32:44.669 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-09-10T10:32:44.813 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-09-10T10:32:44.929 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-09-10T10:32:49.266 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T10:32:52.874 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T10:33:28.875 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi028.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T10:33:28.876 DEBUG:teuthology.orchestra.run.smithi028:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T10:33:28.969 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T10:33:28.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T10:33:28.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T10:33:28.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T10:33:28.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T10:33:28.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T10:33:28.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T10:33:28.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T10:33:28.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T10:33:28.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T10:33:28.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T10:33:28.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T10:33:28.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T10:33:28.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T10:33:28.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T10:33:28.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T10:33:28.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T10:33:28.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T10:33:28.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T10:33:28.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T10:33:28.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T10:33:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T10:33:28.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T10:33:28.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T10:33:28.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T10:33:28.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T10:33:28.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T10:33:28.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T10:33:28.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T10:33:28.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T10:33:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T10:33:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T10:33:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T10:33:28.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T10:33:28.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T10:33:29.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T10:33:29.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T10:33:29.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T10:33:29.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T10:33:29.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T10:33:29.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T10:33:29.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T10:33:29.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T10:33:29.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T10:33:29.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T10:33:29.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T10:33:29.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T10:33:29.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T10:33:29.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T10:33:29.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T10:33:29.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T10:33:29.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T10:33:29.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T10:33:29.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T10:33:29.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T10:33:29.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T10:33:29.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T10:33:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T10:33:29.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T10:33:29.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T10:33:29.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T10:33:29.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T10:33:29.027 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T10:33:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T10:33:29.029 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T10:33:29.030 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T10:33:29.031 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T10:33:29.032 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T10:33:29.033 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T10:33:29.034 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T10:33:29.035 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T10:33:29.036 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T10:33:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T10:33:29.038 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T10:33:29.039 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T10:33:29.040 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T10:33:29.041 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T10:33:29.042 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T10:33:29.043 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T10:33:29.044 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T10:33:29.045 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T10:33:29.046 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T10:33:29.047 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T10:33:29.048 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T10:33:29.049 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T10:33:29.050 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T10:33:29.051 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T10:33:29.052 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T10:33:29.053 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T10:33:29.054 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T10:33:29.054 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T10:33:29.054 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T10:33:29.054 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T10:33:29.054 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T10:33:29.056 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T10:33:29.057 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T10:33:29.058 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T10:33:29.059 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T10:33:29.060 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T10:33:29.061 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T10:33:29.062 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T10:33:29.063 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T10:33:29.064 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T10:33:29.065 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T10:33:29.066 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T10:33:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T10:33:29.068 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T10:33:29.069 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T10:33:29.070 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T10:33:29.071 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T10:33:29.072 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T10:33:29.073 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T10:33:29.074 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T10:33:29.075 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T10:33:29.076 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T10:33:29.077 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T10:33:29.078 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T10:33:29.079 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T10:33:29.080 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T10:33:29.081 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T10:33:29.082 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T10:33:29.083 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T10:33:29.084 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T10:33:29.085 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T10:33:29.086 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T10:33:29.087 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T10:33:29.088 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T10:33:29.089 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T10:33:29.090 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T10:33:29.091 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T10:33:29.092 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T10:33:29.093 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T10:33:29.094 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T10:33:29.095 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T10:33:29.096 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T10:33:29.097 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T10:33:29.098 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T10:33:29.099 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T10:33:29.100 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T10:33:29.101 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T10:33:29.102 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T10:33:29.103 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T10:33:29.104 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T10:33:29.105 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T10:33:29.106 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T10:33:29.107 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T10:33:29.108 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T10:33:29.109 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T10:33:29.110 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T10:33:29.111 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T10:33:29.112 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T10:33:29.113 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T10:33:29.114 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T10:33:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T10:33:29.116 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T10:33:29.117 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T10:33:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T10:33:29.119 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T10:33:29.120 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T10:33:29.121 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T10:33:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T10:33:29.123 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T10:33:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T10:33:29.125 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T10:33:29.126 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T10:33:29.127 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T10:33:29.128 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T10:33:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T10:33:29.130 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T10:33:29.131 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T10:33:29.132 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T10:33:29.133 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T10:33:29.134 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T10:33:29.135 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T10:33:29.136 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T10:33:29.137 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T10:33:29.138 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T10:33:29.139 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T10:33:29.140 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T10:33:29.141 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T10:33:29.142 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T10:33:29.143 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T10:33:29.144 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T10:33:29.145 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T10:33:29.146 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T10:33:29.147 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T10:33:29.151 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:29.151 DEBUG:teuthology.orchestra.run.smithi028:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T10:33:29.175 INFO:teuthology.orchestra.run.smithi028.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T10:33:29.175 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T10:33:29.175 INFO:teuthology.orchestra.run.smithi028.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T10:33:29.175 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:for file in vmlinuz System.map config; do 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: fi 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:done 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:29.176 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-09-10T10:33:29.177 INFO:teuthology.orchestra.run.smithi028.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T10:33:29.177 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T10:33:29.177 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:29.177 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-09-10T10:33:29.178 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -qi grub2-tools 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Name : grub2-tools 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Epoch : 1 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Version : 2.06 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Release : 82.el9 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Architecture: x86_64 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:33:29.252 INFO:teuthology.orchestra.run.smithi028.stdout:Group : Unspecified 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Size : 8274670 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:License : GPLv3+ 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Packager : builder@centos.org 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Vendor : CentOS 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Summary : Support tools for GRUB. 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout:Description : 2024-09-10T10:33:29.253 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout:hardware devices. 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:33:29.254 INFO:teuthology.orchestra.run.smithi028.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:33:29.255 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:33:29.255 INFO:teuthology.task.kernel:Updating grub on smithi028 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:29.255 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:33:29.748 INFO:teuthology.orchestra.run.smithi028.stderr:Generating grub configuration file ... 2024-09-10T10:33:31.143 INFO:teuthology.orchestra.run.smithi028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:33:31.170 INFO:teuthology.orchestra.run.smithi028.stderr:done 2024-09-10T10:33:31.172 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-09-10T10:33:31.187 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.eSc6bXHUiJ 2024-09-10T10:33:31.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eSc6bXHUiJ 2024-09-10T10:33:31.253 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.eSc6bXHUiJ 2024-09-10T10:33:31.401 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.eSc6bXHUiJ is 6KB 2024-09-10T10:33:31.448 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.eSc6bXHUiJ 2024-09-10T10:33:31.463 DEBUG:teuthology.orchestra.run.smithi028:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:33:31.535 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:33:31.535 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:33:31.535 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:33:31.536 INFO:teuthology.orchestra.run.smithi028.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T10:33:31.538 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:31.716 DEBUG:teuthology.orchestra.run.smithi028:> sudo shutdown -r now 2024-09-10T10:33:33.789 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T10:33:33.790 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T10:33:33.865 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T10:33:33.866 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T10:33:33.867 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T10:33:33.868 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T10:33:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T10:33:33.870 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T10:33:33.871 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T10:33:33.872 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T10:33:33.873 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T10:33:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T10:33:33.875 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T10:33:33.876 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T10:33:33.877 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T10:33:33.878 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T10:33:33.879 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T10:33:33.880 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T10:33:33.881 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T10:33:33.882 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T10:33:33.883 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T10:33:33.884 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T10:33:33.885 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T10:33:33.886 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T10:33:33.887 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T10:33:33.888 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T10:33:33.889 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T10:33:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T10:33:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T10:33:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T10:33:33.893 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T10:33:33.894 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T10:33:33.895 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T10:33:33.896 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T10:33:33.897 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T10:33:33.898 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T10:33:33.899 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T10:33:33.900 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T10:33:33.901 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T10:33:33.902 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T10:33:33.903 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T10:33:33.904 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T10:33:33.905 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T10:33:33.906 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T10:33:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T10:33:33.908 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T10:33:33.909 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T10:33:33.910 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T10:33:33.911 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T10:33:33.912 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T10:33:33.913 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T10:33:33.914 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T10:33:33.915 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T10:33:33.916 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T10:33:33.917 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T10:33:33.918 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T10:33:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T10:33:33.920 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T10:33:33.921 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T10:33:33.922 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T10:33:33.923 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T10:33:33.924 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T10:33:33.925 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T10:33:33.926 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T10:33:33.927 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T10:33:33.928 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T10:33:33.929 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T10:33:33.930 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T10:33:33.931 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T10:33:33.932 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T10:33:33.933 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T10:33:33.934 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T10:33:33.935 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T10:33:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T10:33:33.937 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T10:33:33.938 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T10:33:33.939 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T10:33:33.940 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T10:33:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T10:33:33.942 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T10:33:33.943 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T10:33:33.944 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T10:33:33.945 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T10:33:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T10:33:33.947 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T10:33:33.948 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T10:33:33.949 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T10:33:33.950 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T10:33:33.951 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T10:33:33.952 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T10:33:33.953 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T10:33:33.954 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T10:33:33.955 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T10:33:33.956 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T10:33:33.957 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T10:33:33.958 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T10:33:33.959 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T10:33:33.960 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T10:33:33.961 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T10:33:33.962 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T10:33:33.963 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T10:33:33.964 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T10:33:33.965 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T10:33:33.966 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T10:33:33.967 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T10:33:33.968 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T10:33:33.969 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T10:33:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T10:33:33.971 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T10:33:33.972 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T10:33:33.973 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T10:33:33.974 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T10:33:33.975 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T10:33:33.975 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T10:33:33.976 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T10:33:33.977 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T10:33:33.978 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T10:33:33.979 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T10:33:33.980 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T10:33:33.981 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T10:33:33.982 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T10:33:33.983 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T10:33:33.984 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T10:33:33.985 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T10:33:33.986 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T10:33:33.987 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T10:33:33.988 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T10:33:33.989 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T10:33:33.990 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T10:33:33.991 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T10:33:33.992 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T10:33:33.993 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T10:33:33.994 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T10:33:33.995 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T10:33:33.996 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T10:33:33.997 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T10:33:33.998 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T10:33:33.999 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T10:33:34.000 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T10:33:34.001 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T10:33:34.002 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T10:33:34.003 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T10:33:34.004 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T10:33:34.005 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T10:33:34.006 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T10:33:34.007 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T10:33:34.008 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T10:33:34.009 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T10:33:34.010 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T10:33:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T10:33:34.012 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T10:33:34.013 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T10:33:34.014 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T10:33:34.015 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T10:33:34.016 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T10:33:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T10:33:34.018 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T10:33:34.019 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T10:33:34.019 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T10:33:34.019 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T10:33:34.019 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T10:33:34.019 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T10:33:34.022 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:34.022 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T10:33:34.047 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout:for file in vmlinuz System.map config; do 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T10:33:34.048 INFO:teuthology.orchestra.run.smithi138.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: fi 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-09-10T10:33:34.049 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T10:33:34.050 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T10:33:34.050 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:34.050 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-09-10T10:33:34.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 82.el9 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-09-10T10:33:34.127 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8274670 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T10:33:34.128 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T10:33:34.129 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T10:33:34.129 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-09-10T10:33:34.129 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:33:34.129 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T10:33:34.130 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T10:33:34.130 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:34.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T10:33:34.734 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-09-10T10:33:36.131 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T10:33:36.149 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-09-10T10:33:36.151 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-10T10:33:36.167 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.1BGWfCLc7C 2024-09-10T10:33:36.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1BGWfCLc7C 2024-09-10T10:33:36.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.1BGWfCLc7C 2024-09-10T10:33:36.374 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.1BGWfCLc7C is 6KB 2024-09-10T10:33:36.422 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.1BGWfCLc7C 2024-09-10T10:33:36.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T10:33:36.504 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T10:33:36.505 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T10:33:36.505 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T10:33:36.505 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T10:33:36.506 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:33:36.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-09-10T10:34:01.742 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T10:34:01.743 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:34:01.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:34:06.670 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T10:34:06.670 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:34:06.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:34:20.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-09-10T10:34:25.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-09-10T10:34:29.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:34:29.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:34:34.217 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:34:34.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:34:47.489 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-09-10T10:34:59.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:34:59.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:35:02.561 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.28 2024-09-10T10:35:17.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:35:17.576 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:35:18.021 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-10T10:35:18.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:35:18.764 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T10:35:18.764 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-09-10T10:35:18.846 INFO:teuthology.orchestra.run.smithi028.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:35:18.847 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:35:18.847 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:35:18.847 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:35:19.848 DEBUG:teuthology.parallel:result is None 2024-09-10T10:35:34.220 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T10:35:46.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:35:46.222 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:35:46.582 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-10T10:35:47.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:35:47.098 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T10:35:47.098 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-10T10:35:47.132 INFO:teuthology.orchestra.run.smithi138.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:35:47.132 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T10:35:47.133 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T10:35:47.133 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T10:35:48.136 DEBUG:teuthology.parallel:result is None 2024-09-10T10:35:48.136 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T10:35:48.144 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T10:35:48.144 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:35:48.149 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T10:35:48.186 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T10:35:48.193 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T10:35:48.199 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T10:35:48.200 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:35:48.228 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T10:35:48.285 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T10:35:48.291 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T10:35:48.291 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:35:48.295 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T10:35:48.391 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:35:48.423 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:35:48.426 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:35:48.454 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:35:48.457 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T10:35:48.464 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T10:35:48.464 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:35:48.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T10:35:48.524 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T10:35:48.534 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T10:35:48.534 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:35:48.538 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T10:35:48.572 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:35:48.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:35:48.776 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:35:48.839 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:35:48.910 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:35:48.910 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:35:48.965 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:35:49.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T10:35:49.112 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:35:49.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T10:35:49.247 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:35:49.247 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T10:35:49.301 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-09-10T10:35:49.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-09-10T10:35:49.389 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:35:49.420 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:35:49.873 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T10:35:49.882 INFO:teuthology.task.internal:Starting timer... 2024-09-10T10:35:49.882 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T10:35:49.891 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T10:35:49.900 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-10T10:35:49.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-09-10T10:35:49.982 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-09-10T10:35:49.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-09-10T10:35:50.072 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-09-10T10:35:50.075 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T10:35:50.075 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-09-10T10:35:50.141 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-09-10T10:35:50.141 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-09-10T10:35:50.185 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-09-10T10:35:50.185 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-09-10T10:35:50.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:35:50.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:35:50.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:35:50.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:35:50.303 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T10:35:50.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:35:50.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-09-10T10:35:50.437 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T10:35:50.449 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T10:35:50.463 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T10:35:50.464 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T10:40:32.364 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-09-10T10:40:32.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:40:32.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:40:32.457 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-09-10T10:40:32.567 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-09-10T10:40:32.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:40:32.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T10:40:32.653 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-10T10:40:32.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-10T10:40:32.763 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T10:40:32.776 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T10:40:32.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:40:32.777 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:40:32.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T10:40:32.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:40:32.837 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:40:32.863 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T10:40:32.867 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:40:32.893 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T10:40:32.918 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-09-10T10:40:32.942 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-09-10T10:40:32.944 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-09-10T10:40:32.968 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-09-10T10:40:32.972 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:40:32.998 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T10:40:33.002 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:40:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T10:40:33.093 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:40:33.099 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:40:33.099 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-10T10:40:33.100 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.100 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.100 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.100 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.122 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:40:33.128 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:40:33.128 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-10T10:40:33.128 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.129 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.129 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.129 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:40:33.130 INFO:teuthology.run_tasks:Running task install... 2024-09-10T10:40:33.138 DEBUG:teuthology.task.install:project ceph 2024-09-10T10:40:33.139 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T10:40:33.139 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T10:40:33.139 INFO:teuthology.task.install:Using flavor: default 2024-09-10T10:40:33.145 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T10:40:33.145 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T10:40:33.145 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:40:33.146 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:40:33.148 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T10:40:33.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:40:33.295 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:40:33.295 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:40:33.305 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T10:40:33.305 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T10:40:33.413 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:40:33.413 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:40:33.413 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:40:33.419 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T10:40:33.419 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:40:33.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T10:40:33.470 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:40:33.471 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:40:33.476 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T10:40:33.477 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T10:40:33.569 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:40:33.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T10:40:33.648 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:40:33.653 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T10:40:33.704 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-09-10T10:40:33.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-09-10T10:40:33.708 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-09-10T10:40:33.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-09-10T10:40:33.970 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-09-10T10:40:33.990 INFO:teuthology.orchestra.run.smithi028.stdout:45 files removed 2024-09-10T10:40:34.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:40:34.023 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T10:40:34.711 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 269 kB/s | 84 kB 00:00 2024-09-10T10:40:34.732 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 283 kB/s | 84 kB 00:00 2024-09-10T10:40:35.013 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-10T10:40:35.041 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 45 kB/s | 12 kB 00:00 2024-09-10T10:40:35.296 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-10T10:40:35.333 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-10T10:40:37.450 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-10T10:40:38.313 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.2 MB 00:02 2024-09-10T10:40:40.314 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-10T10:40:46.385 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-10T10:40:47.359 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 20 MB 00:08 2024-09-10T10:40:48.166 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T10:40:49.216 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-10T10:40:52.059 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-10T10:40:53.685 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-10T10:40:54.675 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T10:40:55.577 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-10T10:40:58.073 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:40:58.074 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:40:58.085 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:40:58.086 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:40:58.087 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:40:58.088 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:40:58.089 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:40:58.092 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:40:58.093 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:40:58.157 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-10T10:40:58.187 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-10T10:40:58.187 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:40:58.188 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:40:58.189 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:40:58.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:40:58.191 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:40:58.192 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:40:58.193 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:40:58.194 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:40:58.195 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:40:58.196 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:40:58.197 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:40:58.198 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:40:58.199 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:40:58.200 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-10T10:40:58.201 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-09-10T10:40:58.202 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-09-10T10:40:58.202 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 8 Packages 2024-09-10T10:40:58.202 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:40:58.202 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 226 M 2024-09-10T10:40:58.202 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-10T10:40:58.762 INFO:teuthology.orchestra.run.smithi138.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-10T10:40:58.937 INFO:teuthology.orchestra.run.smithi138.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 5.0 MB/s | 897 kB 00:00 2024-09-10T10:40:59.004 INFO:teuthology.orchestra.run.smithi138.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 2.1 MB/s | 145 kB 00:00 2024-09-10T10:40:59.121 INFO:teuthology.orchestra.run.smithi138.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-10T10:40:59.197 INFO:teuthology.orchestra.run.smithi138.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-10T10:40:59.247 INFO:teuthology.orchestra.run.smithi138.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-10T10:40:59.572 INFO:teuthology.orchestra.run.smithi138.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-10T10:40:59.931 INFO:teuthology.orchestra.run.smithi138.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 16 MB/s | 21 MB 00:01 2024-09-10T10:40:59.973 INFO:teuthology.orchestra.run.smithi138.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 608 kB/s | 25 kB 00:00 2024-09-10T10:41:00.257 INFO:teuthology.orchestra.run.smithi138.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 17 MB/s | 17 MB 00:01 2024-09-10T10:41:00.415 INFO:teuthology.orchestra.run.smithi138.stdout:(11/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 13 MB/s | 11 MB 00:00 2024-09-10T10:41:00.449 INFO:teuthology.orchestra.run.smithi138.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.2 MB/s | 228 kB 00:00 2024-09-10T10:41:00.474 INFO:teuthology.orchestra.run.smithi138.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 461 kB/s | 27 kB 00:00 2024-09-10T10:41:00.500 INFO:teuthology.orchestra.run.smithi138.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 632 kB/s | 32 kB 00:00 2024-09-10T10:41:00.533 INFO:teuthology.orchestra.run.smithi138.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 751 kB 00:00 2024-09-10T10:41:00.558 INFO:teuthology.orchestra.run.smithi138.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.7 MB/s | 164 kB 00:00 2024-09-10T10:41:00.584 INFO:teuthology.orchestra.run.smithi138.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T10:41:00.626 INFO:teuthology.orchestra.run.smithi138.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 7.4 MB/s | 506 kB 00:00 2024-09-10T10:41:00.685 INFO:teuthology.orchestra.run.smithi138.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 776 kB/s | 45 kB 00:00 2024-09-10T10:41:00.777 INFO:teuthology.orchestra.run.smithi138.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.4 MB/s | 136 kB 00:00 2024-09-10T10:41:00.902 INFO:teuthology.orchestra.run.smithi138.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-10T10:41:00.927 INFO:teuthology.orchestra.run.smithi138.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.1 MB/s | 161 kB 00:00 2024-09-10T10:41:00.961 INFO:teuthology.orchestra.run.smithi138.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.3 MB/s | 320 kB 00:00 2024-09-10T10:41:01.003 INFO:teuthology.orchestra.run.smithi138.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.9 MB/s | 301 kB 00:00 2024-09-10T10:41:01.028 INFO:teuthology.orchestra.run.smithi138.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.4 MB/s | 99 kB 00:00 2024-09-10T10:41:01.062 INFO:teuthology.orchestra.run.smithi138.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-10T10:41:01.163 INFO:teuthology.orchestra.run.smithi138.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-10T10:41:01.256 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-10T10:41:01.263 INFO:teuthology.orchestra.run.smithi138.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 319 kB/s | 32 kB 00:00 2024-09-10T10:41:01.338 INFO:teuthology.orchestra.run.smithi138.stdout:(29/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-10T10:41:01.363 INFO:teuthology.orchestra.run.smithi138.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.4 MB/s | 146 kB 00:00 2024-09-10T10:41:01.689 INFO:teuthology.orchestra.run.smithi138.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 10 MB/s | 3.6 MB 00:00 2024-09-10T10:41:01.756 INFO:teuthology.orchestra.run.smithi138.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 3.7 MB/s | 249 kB 00:00 2024-09-10T10:41:01.806 INFO:teuthology.orchestra.run.smithi138.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 987 kB/s | 49 kB 00:00 2024-09-10T10:41:01.856 INFO:teuthology.orchestra.run.smithi138.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 330 kB/s | 16 kB 00:00 2024-09-10T10:41:01.916 INFO:teuthology.orchestra.run.smithi138.stdout:(35/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.8 MB/s | 285 kB 00:00 2024-09-10T10:41:01.999 INFO:teuthology.orchestra.run.smithi138.stdout:(36/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 9.0 MB/s | 766 kB 00:00 2024-09-10T10:41:02.066 INFO:teuthology.orchestra.run.smithi138.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 274 kB/s | 18 kB 00:00 2024-09-10T10:41:02.175 INFO:teuthology.orchestra.run.smithi138.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 737 kB/s | 80 kB 00:00 2024-09-10T10:41:02.292 INFO:teuthology.orchestra.run.smithi138.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 6.7 MB/s | 801 kB 00:00 2024-09-10T10:41:02.342 INFO:teuthology.orchestra.run.smithi138.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-10T10:41:02.434 INFO:teuthology.orchestra.run.smithi138.stdout:(41/156): ceph-mgr-diskprediction-local-19.1.1- 6.9 MB/s | 7.4 MB 00:01 2024-09-10T10:41:02.460 INFO:teuthology.orchestra.run.smithi138.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-10T10:41:02.502 INFO:teuthology.orchestra.run.smithi138.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T10:41:02.544 INFO:teuthology.orchestra.run.smithi138.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-10T10:41:02.586 INFO:teuthology.orchestra.run.smithi138.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-10T10:41:02.645 INFO:teuthology.orchestra.run.smithi138.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-09-10T10:41:02.670 INFO:teuthology.orchestra.run.smithi138.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T10:41:02.712 INFO:teuthology.orchestra.run.smithi138.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-09-10T10:41:02.779 INFO:teuthology.orchestra.run.smithi138.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-09-10T10:41:02.804 INFO:teuthology.orchestra.run.smithi138.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-09-10T10:41:02.829 INFO:teuthology.orchestra.run.smithi138.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-10T10:41:02.855 INFO:teuthology.orchestra.run.smithi138.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-10T10:41:02.880 INFO:teuthology.orchestra.run.smithi138.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-10T10:41:02.923 INFO:teuthology.orchestra.run.smithi138.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-09-10T10:41:02.981 INFO:teuthology.orchestra.run.smithi138.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-10T10:41:02.998 INFO:teuthology.orchestra.run.smithi138.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-10T10:41:03.441 INFO:teuthology.orchestra.run.smithi138.stdout:(57/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 14 MB/s | 49 MB 00:03 2024-09-10T10:41:03.466 INFO:teuthology.orchestra.run.smithi138.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 97 kB/s | 45 kB 00:00 2024-09-10T10:41:03.491 INFO:teuthology.orchestra.run.smithi138.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 314 kB/s | 160 kB 00:00 2024-09-10T10:41:03.525 INFO:teuthology.orchestra.run.smithi138.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-09-10T10:41:03.550 INFO:teuthology.orchestra.run.smithi138.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-10T10:41:03.575 INFO:teuthology.orchestra.run.smithi138.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-10T10:41:03.601 INFO:teuthology.orchestra.run.smithi138.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T10:41:03.626 INFO:teuthology.orchestra.run.smithi138.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T10:41:03.651 INFO:teuthology.orchestra.run.smithi138.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-10T10:41:03.685 INFO:teuthology.orchestra.run.smithi138.stdout:(66/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-09-10T10:41:03.794 INFO:teuthology.orchestra.run.smithi138.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-10T10:41:03.836 INFO:teuthology.orchestra.run.smithi138.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T10:41:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:41:03.871 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T10:41:03.878 INFO:teuthology.orchestra.run.smithi138.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-10T10:41:03.882 INFO:teuthology.orchestra.run.smithi028.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T10:41:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T10:41:03.884 INFO:teuthology.orchestra.run.smithi028.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T10:41:03.885 INFO:teuthology.orchestra.run.smithi028.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T10:41:03.887 INFO:teuthology.orchestra.run.smithi028.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T10:41:03.890 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T10:41:03.891 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T10:41:03.903 INFO:teuthology.orchestra.run.smithi138.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T10:41:03.961 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-09-10T10:41:03.970 INFO:teuthology.orchestra.run.smithi138.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T10:41:03.981 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-10T10:41:03.981 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-09-10T10:41:03.981 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-10T10:41:03.982 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-09-10T10:41:03.982 INFO:teuthology.orchestra.run.smithi028.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T10:41:03.983 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T10:41:03.983 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T10:41:03.983 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T10:41:03.983 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T10:41:03.984 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T10:41:03.984 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T10:41:03.984 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T10:41:03.984 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T10:41:03.985 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T10:41:03.986 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T10:41:03.986 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T10:41:03.986 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T10:41:03.986 INFO:teuthology.orchestra.run.smithi028.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T10:41:03.986 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T10:41:03.987 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T10:41:03.988 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T10:41:03.989 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T10:41:03.989 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T10:41:03.989 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T10:41:03.989 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T10:41:03.989 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-09-10T10:41:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T10:41:03.991 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T10:41:03.992 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T10:41:03.993 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T10:41:03.994 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T10:41:03.995 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T10:41:03.996 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T10:41:03.997 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T10:41:03.998 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T10:41:03.999 INFO:teuthology.orchestra.run.smithi028.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout:====================================================================================== 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout:Install 148 Packages 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 8 Packages 2024-09-10T10:41:04.000 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:41:04.001 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 226 M 2024-09-10T10:41:04.001 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-09-10T10:41:04.012 INFO:teuthology.orchestra.run.smithi138.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-10T10:41:04.037 INFO:teuthology.orchestra.run.smithi138.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T10:41:04.088 INFO:teuthology.orchestra.run.smithi138.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T10:41:04.172 INFO:teuthology.orchestra.run.smithi138.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-10T10:41:04.197 INFO:teuthology.orchestra.run.smithi138.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T10:41:04.222 INFO:teuthology.orchestra.run.smithi138.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T10:41:04.247 INFO:teuthology.orchestra.run.smithi138.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-10T10:41:04.273 INFO:teuthology.orchestra.run.smithi138.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-10T10:41:04.298 INFO:teuthology.orchestra.run.smithi138.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T10:41:04.324 INFO:teuthology.orchestra.run.smithi138.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 689 kB/s | 35 kB 00:00 2024-09-10T10:41:04.408 INFO:teuthology.orchestra.run.smithi138.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-10T10:41:04.441 INFO:teuthology.orchestra.run.smithi138.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-10T10:41:04.466 INFO:teuthology.orchestra.run.smithi138.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T10:41:04.492 INFO:teuthology.orchestra.run.smithi138.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T10:41:04.517 INFO:teuthology.orchestra.run.smithi138.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-10T10:41:04.551 INFO:teuthology.orchestra.run.smithi138.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T10:41:04.576 INFO:teuthology.orchestra.run.smithi138.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-10T10:41:04.610 INFO:teuthology.orchestra.run.smithi138.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-10T10:41:04.635 INFO:teuthology.orchestra.run.smithi138.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-09-10T10:41:04.682 INFO:teuthology.orchestra.run.smithi138.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 745 kB/s | 54 kB 00:00 2024-09-10T10:41:04.686 INFO:teuthology.orchestra.run.smithi028.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 17 kB/s | 6.5 kB 00:00 2024-09-10T10:41:04.777 INFO:teuthology.orchestra.run.smithi138.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-09-10T10:41:04.803 INFO:teuthology.orchestra.run.smithi138.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 346 kB/s | 42 kB 00:00 2024-09-10T10:41:04.828 INFO:teuthology.orchestra.run.smithi138.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-10T10:41:04.853 INFO:teuthology.orchestra.run.smithi138.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T10:41:04.879 INFO:teuthology.orchestra.run.smithi138.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 711 kB/s | 36 kB 00:00 2024-09-10T10:41:04.904 INFO:teuthology.orchestra.run.smithi138.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-10T10:41:04.929 INFO:teuthology.orchestra.run.smithi138.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T10:41:04.955 INFO:teuthology.orchestra.run.smithi138.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T10:41:04.980 INFO:teuthology.orchestra.run.smithi138.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T10:41:05.006 INFO:teuthology.orchestra.run.smithi138.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T10:41:05.031 INFO:teuthology.orchestra.run.smithi138.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-09-10T10:41:05.123 INFO:teuthology.orchestra.run.smithi138.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-10T10:41:05.332 INFO:teuthology.orchestra.run.smithi138.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-10T10:41:05.382 INFO:teuthology.orchestra.run.smithi138.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-09-10T10:41:05.416 INFO:teuthology.orchestra.run.smithi138.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 617 kB/s | 19 kB 00:00 2024-09-10T10:41:05.466 INFO:teuthology.orchestra.run.smithi138.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-09-10T10:41:05.500 INFO:teuthology.orchestra.run.smithi138.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 782 kB/s | 26 kB 00:00 2024-09-10T10:41:05.534 INFO:teuthology.orchestra.run.smithi138.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-10T10:41:05.559 INFO:teuthology.orchestra.run.smithi138.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.6 MB/s | 863 kB 00:00 2024-09-10T10:41:05.566 INFO:teuthology.orchestra.run.smithi028.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 1.0 MB/s | 897 kB 00:00 2024-09-10T10:41:05.584 INFO:teuthology.orchestra.run.smithi138.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T10:41:05.660 INFO:teuthology.orchestra.run.smithi138.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-10T10:41:05.685 INFO:teuthology.orchestra.run.smithi138.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-10T10:41:05.702 INFO:teuthology.orchestra.run.smithi138.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.5 MB/s | 43 kB 00:00 2024-09-10T10:41:05.709 INFO:teuthology.orchestra.run.smithi028.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.0 MB/s | 145 kB 00:00 2024-09-10T10:41:05.769 INFO:teuthology.orchestra.run.smithi138.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-10T10:41:05.795 INFO:teuthology.orchestra.run.smithi138.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-09-10T10:41:05.820 INFO:teuthology.orchestra.run.smithi138.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-10T10:41:05.845 INFO:teuthology.orchestra.run.smithi138.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T10:41:05.870 INFO:teuthology.orchestra.run.smithi138.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-09-10T10:41:05.896 INFO:teuthology.orchestra.run.smithi138.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-10T10:41:05.921 INFO:teuthology.orchestra.run.smithi138.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T10:41:05.946 INFO:teuthology.orchestra.run.smithi138.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-10T10:41:05.972 INFO:teuthology.orchestra.run.smithi138.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-10T10:41:05.997 INFO:teuthology.orchestra.run.smithi138.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-10T10:41:06.022 INFO:teuthology.orchestra.run.smithi138.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-10T10:41:06.047 INFO:teuthology.orchestra.run.smithi138.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-10T10:41:06.073 INFO:teuthology.orchestra.run.smithi138.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-09-10T10:41:06.098 INFO:teuthology.orchestra.run.smithi138.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-10T10:41:06.132 INFO:teuthology.orchestra.run.smithi138.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-10T10:41:06.157 INFO:teuthology.orchestra.run.smithi138.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.0 MB/s | 79 kB 00:00 2024-09-10T10:41:06.158 INFO:teuthology.orchestra.run.smithi028.stdout:(4/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 5.3 MB/s | 2.4 MB 00:00 2024-09-10T10:41:06.183 INFO:teuthology.orchestra.run.smithi138.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-10T10:41:06.217 INFO:teuthology.orchestra.run.smithi138.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2024-09-10T10:41:06.242 INFO:teuthology.orchestra.run.smithi138.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-10T10:41:06.267 INFO:teuthology.orchestra.run.smithi138.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-10T10:41:06.293 INFO:teuthology.orchestra.run.smithi138.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-10T10:41:06.326 INFO:teuthology.orchestra.run.smithi138.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-09-10T10:41:06.352 INFO:teuthology.orchestra.run.smithi138.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-09-10T10:41:06.377 INFO:teuthology.orchestra.run.smithi138.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-10T10:41:06.407 INFO:teuthology.orchestra.run.smithi138.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T10:41:06.425 INFO:teuthology.orchestra.run.smithi028.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 5.7 MB/s | 1.5 MB 00:00 2024-09-10T10:41:06.465 INFO:teuthology.orchestra.run.smithi138.stdout:(140/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-09-10T10:41:06.490 INFO:teuthology.orchestra.run.smithi138.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 317 kB/s | 36 kB 00:00 2024-09-10T10:41:06.516 INFO:teuthology.orchestra.run.smithi138.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 683 kB/s | 74 kB 00:00 2024-09-10T10:41:06.574 INFO:teuthology.orchestra.run.smithi138.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-09-10T10:41:06.625 INFO:teuthology.orchestra.run.smithi138.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-10T10:41:06.650 INFO:teuthology.orchestra.run.smithi138.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-10T10:41:06.684 INFO:teuthology.orchestra.run.smithi138.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-09-10T10:41:06.709 INFO:teuthology.orchestra.run.smithi138.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T10:41:06.768 INFO:teuthology.orchestra.run.smithi138.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T10:41:06.952 INFO:teuthology.orchestra.run.smithi138.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-10T10:41:07.000 INFO:teuthology.orchestra.run.smithi028.stdout:(6/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 8.2 MB/s | 4.7 MB 00:00 2024-09-10T10:41:07.019 INFO:teuthology.orchestra.run.smithi138.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-10T10:41:07.069 INFO:teuthology.orchestra.run.smithi138.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 821 kB/s | 41 kB 00:00 2024-09-10T10:41:07.119 INFO:teuthology.orchestra.run.smithi138.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-10T10:41:07.178 INFO:teuthology.orchestra.run.smithi138.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 23 MB/s | 1.4 MB 00:00 2024-09-10T10:41:07.253 INFO:teuthology.orchestra.run.smithi138.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-10T10:41:07.362 INFO:teuthology.orchestra.run.smithi138.stdout:(155/156): librados2-19.1.1-305.g1cf819b7.el9.x 5.0 MB/s | 3.4 MB 00:00 2024-09-10T10:41:07.412 INFO:teuthology.orchestra.run.smithi138.stdout:(156/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 4.5 MB/s | 3.2 MB 00:00 2024-09-10T10:41:07.417 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:41:07.417 INFO:teuthology.orchestra.run.smithi138.stdout:Total 25 MB/s | 226 MB 00:09 2024-09-10T10:41:07.425 INFO:teuthology.orchestra.run.smithi028.stdout:(7/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 5.4 MB 00:03 2024-09-10T10:41:08.476 INFO:teuthology.orchestra.run.smithi028.stdout:(8/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86_ 11 MB/s | 17 MB 00:01 2024-09-10T10:41:08.550 INFO:teuthology.orchestra.run.smithi028.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 338 kB/s | 25 kB 00:00 2024-09-10T10:41:09.529 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-10T10:41:09.715 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-10T10:41:09.715 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-10T10:41:11.359 INFO:teuthology.orchestra.run.smithi028.stdout:(10/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 17 MB/s | 49 MB 00:02 2024-09-10T10:41:11.418 INFO:teuthology.orchestra.run.smithi028.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 3.9 MB/s | 228 kB 00:00 2024-09-10T10:41:11.485 INFO:teuthology.orchestra.run.smithi028.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 406 kB/s | 27 kB 00:00 2024-09-10T10:41:11.535 INFO:teuthology.orchestra.run.smithi028.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 638 kB/s | 32 kB 00:00 2024-09-10T10:41:11.638 INFO:teuthology.orchestra.run.smithi028.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 7.1 MB/s | 751 kB 00:00 2024-09-10T10:41:11.694 INFO:teuthology.orchestra.run.smithi028.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.9 MB/s | 164 kB 00:00 2024-09-10T10:41:11.744 INFO:teuthology.orchestra.run.smithi028.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T10:41:11.803 INFO:teuthology.orchestra.run.smithi028.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 8.5 MB/s | 506 kB 00:00 2024-09-10T10:41:11.982 INFO:teuthology.orchestra.run.smithi028.stdout:(18/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 2.3 MB/s | 11 MB 00:04 2024-09-10T10:41:12.045 INFO:teuthology.orchestra.run.smithi028.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 727 kB/s | 45 kB 00:00 2024-09-10T10:41:12.084 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-10T10:41:12.084 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-10T10:41:12.145 INFO:teuthology.orchestra.run.smithi028.stdout:(20/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-10T10:41:12.170 INFO:teuthology.orchestra.run.smithi028.stdout:(21/156): python3-ceph-common-19.1.1-305.g1cf81 1.1 MB/s | 136 kB 00:00 2024-09-10T10:41:12.196 INFO:teuthology.orchestra.run.smithi028.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 3.2 MB/s | 161 kB 00:00 2024-09-10T10:41:12.246 INFO:teuthology.orchestra.run.smithi028.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 4.2 MB/s | 320 kB 00:00 2024-09-10T10:41:12.271 INFO:teuthology.orchestra.run.smithi028.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.9 MB/s | 301 kB 00:00 2024-09-10T10:41:12.296 INFO:teuthology.orchestra.run.smithi028.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T10:41:12.347 INFO:teuthology.orchestra.run.smithi028.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-10T10:41:12.397 INFO:teuthology.orchestra.run.smithi028.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-10T10:41:12.464 INFO:teuthology.orchestra.run.smithi028.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 478 kB/s | 32 kB 00:00 2024-09-10T10:41:12.515 INFO:teuthology.orchestra.run.smithi028.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.9 MB/s | 146 kB 00:00 2024-09-10T10:41:12.706 INFO:teuthology.orchestra.run.smithi028.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 19 MB/s | 3.6 MB 00:00 2024-09-10T10:41:12.873 INFO:teuthology.orchestra.run.smithi028.stdout:(31/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 5.4 MB/s | 3.1 MB 00:00 2024-09-10T10:41:13.022 INFO:teuthology.orchestra.run.smithi028.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 23 MB/s | 7.4 MB 00:00 2024-09-10T10:41:13.048 INFO:teuthology.orchestra.run.smithi028.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.4 MB/s | 249 kB 00:00 2024-09-10T10:41:13.091 INFO:teuthology.orchestra.run.smithi028.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 723 kB/s | 49 kB 00:00 2024-09-10T10:41:13.132 INFO:teuthology.orchestra.run.smithi028.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 197 kB/s | 16 kB 00:00 2024-09-10T10:41:13.166 INFO:teuthology.orchestra.run.smithi028.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 3.7 MB/s | 285 kB 00:00 2024-09-10T10:41:13.233 INFO:teuthology.orchestra.run.smithi028.stdout:(37/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 275 kB/s | 18 kB 00:00 2024-09-10T10:41:13.308 INFO:teuthology.orchestra.run.smithi028.stdout:(38/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 4.3 MB/s | 766 kB 00:00 2024-09-10T10:41:13.409 INFO:teuthology.orchestra.run.smithi028.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 457 kB/s | 80 kB 00:00 2024-09-10T10:41:13.517 INFO:teuthology.orchestra.run.smithi028.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-10T10:41:13.601 INFO:teuthology.orchestra.run.smithi028.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-10T10:41:13.626 INFO:teuthology.orchestra.run.smithi028.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-10T10:41:13.709 INFO:teuthology.orchestra.run.smithi028.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-09-10T10:41:13.760 INFO:teuthology.orchestra.run.smithi028.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-09-10T10:41:13.818 INFO:teuthology.orchestra.run.smithi028.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T10:41:13.869 INFO:teuthology.orchestra.run.smithi028.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-09-10T10:41:13.894 INFO:teuthology.orchestra.run.smithi028.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T10:41:14.136 INFO:teuthology.orchestra.run.smithi028.stdout:(48/156): ceph-common-19.1.1-305.g1cf819b7.el9. 2.2 MB/s | 21 MB 00:09 2024-09-10T10:41:14.386 INFO:teuthology.orchestra.run.smithi028.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 428 kB/s | 107 kB 00:00 2024-09-10T10:41:14.478 INFO:teuthology.orchestra.run.smithi028.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 236 kB/s | 22 kB 00:00 2024-09-10T10:41:14.512 INFO:teuthology.orchestra.run.smithi028.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-10T10:41:14.587 INFO:teuthology.orchestra.run.smithi028.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-09-10T10:41:14.730 INFO:teuthology.orchestra.run.smithi028.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.2 MB/s | 313 kB 00:00 2024-09-10T10:41:14.763 INFO:teuthology.orchestra.run.smithi028.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-10T10:41:14.821 INFO:teuthology.orchestra.run.smithi028.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-09-10T10:41:14.847 INFO:teuthology.orchestra.run.smithi028.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-10T10:41:14.922 INFO:teuthology.orchestra.run.smithi028.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 605 kB/s | 45 kB 00:00 2024-09-10T10:41:14.947 INFO:teuthology.orchestra.run.smithi028.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T10:41:15.205 INFO:teuthology.orchestra.run.smithi028.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 2.2 MB/s | 581 kB 00:00 2024-09-10T10:41:15.231 INFO:teuthology.orchestra.run.smithi028.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-10T10:41:15.281 INFO:teuthology.orchestra.run.smithi028.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-10T10:41:15.298 INFO:teuthology.orchestra.run.smithi028.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-09-10T10:41:15.423 INFO:teuthology.orchestra.run.smithi028.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-09-10T10:41:15.448 INFO:teuthology.orchestra.run.smithi028.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-10T10:41:15.499 INFO:teuthology.orchestra.run.smithi028.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-10T10:41:15.832 INFO:teuthology.orchestra.run.smithi028.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.4 MB/s | 3.0 MB 00:01 2024-09-10T10:41:16.349 INFO:teuthology.orchestra.run.smithi028.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.1 MB/s | 1.6 MB 00:00 2024-09-10T10:41:16.658 INFO:teuthology.orchestra.run.smithi028.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 2.9 MB/s | 910 kB 00:00 2024-09-10T10:41:16.733 INFO:teuthology.orchestra.run.smithi028.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-10T10:41:17.025 INFO:teuthology.orchestra.run.smithi028.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-09-10T10:41:17.108 INFO:teuthology.orchestra.run.smithi028.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.0 MB/s | 4.9 MB 00:01 2024-09-10T10:41:17.200 INFO:teuthology.orchestra.run.smithi028.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 958 kB/s | 87 kB 00:00 2024-09-10T10:41:17.409 INFO:teuthology.orchestra.run.smithi028.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-09-10T10:41:17.526 INFO:teuthology.orchestra.run.smithi028.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 525 kB/s | 61 kB 00:00 2024-09-10T10:41:17.634 INFO:teuthology.orchestra.run.smithi028.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T10:41:17.760 INFO:teuthology.orchestra.run.smithi028.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-09-10T10:41:17.818 INFO:teuthology.orchestra.run.smithi028.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 442 kB/s | 26 kB 00:00 2024-09-10T10:41:17.885 INFO:teuthology.orchestra.run.smithi028.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-09-10T10:41:17.952 INFO:teuthology.orchestra.run.smithi028.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-10T10:41:18.011 INFO:teuthology.orchestra.run.smithi028.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 599 kB/s | 35 kB 00:00 2024-09-10T10:41:18.327 INFO:teuthology.orchestra.run.smithi028.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.6 MB/s | 5.8 MB 00:01 2024-09-10T10:41:18.453 INFO:teuthology.orchestra.run.smithi028.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-10T10:41:18.570 INFO:teuthology.orchestra.run.smithi028.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-10T10:41:18.635 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-10T10:41:18.637 INFO:teuthology.orchestra.run.smithi028.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-10T10:41:18.712 INFO:teuthology.orchestra.run.smithi028.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 565 kB/s | 42 kB 00:00 2024-09-10T10:41:18.796 INFO:teuthology.orchestra.run.smithi028.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-09-10T10:41:18.801 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:41:18.912 INFO:teuthology.orchestra.run.smithi028.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.3 MB/s | 269 kB 00:00 2024-09-10T10:41:18.938 INFO:teuthology.orchestra.run.smithi028.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-10T10:41:19.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:41:19.030 INFO:teuthology.orchestra.run.smithi028.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-10T10:41:19.055 INFO:teuthology.orchestra.run.smithi028.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 437 kB/s | 51 kB 00:00 2024-09-10T10:41:19.138 INFO:teuthology.orchestra.run.smithi028.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 495 kB/s | 54 kB 00:00 2024-09-10T10:41:19.139 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:41:19.188 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:41:19.219 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:41:19.222 INFO:teuthology.orchestra.run.smithi028.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 500 kB/s | 42 kB 00:00 2024-09-10T10:41:19.282 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:41:19.297 INFO:teuthology.orchestra.run.smithi028.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-09-10T10:41:19.423 INFO:teuthology.orchestra.run.smithi028.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-09-10T10:41:19.448 INFO:teuthology.orchestra.run.smithi028.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-10T10:41:19.490 INFO:teuthology.orchestra.run.smithi028.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 540 kB/s | 36 kB 00:00 2024-09-10T10:41:19.539 INFO:teuthology.orchestra.run.smithi028.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 519 kB/s | 47 kB 00:00 2024-09-10T10:41:19.615 INFO:teuthology.orchestra.run.smithi028.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-10T10:41:19.660 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:41:19.682 INFO:teuthology.orchestra.run.smithi028.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 406 kB/s | 58 kB 00:00 2024-09-10T10:41:19.708 INFO:teuthology.orchestra.run.smithi028.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-10T10:41:19.770 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:41:19.774 INFO:teuthology.orchestra.run.smithi028.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-09-10T10:41:19.799 INFO:teuthology.orchestra.run.smithi028.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 544 kB/s | 64 kB 00:00 2024-09-10T10:41:19.900 INFO:teuthology.orchestra.run.smithi028.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-10T10:41:19.914 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:41:19.964 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:41:19.983 INFO:teuthology.orchestra.run.smithi028.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.5 MB/s | 548 kB 00:00 2024-09-10T10:41:20.025 INFO:teuthology.orchestra.run.smithi028.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-10T10:41:20.051 INFO:teuthology.orchestra.run.smithi028.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-10T10:41:20.093 INFO:teuthology.orchestra.run.smithi028.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-09-10T10:41:20.118 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:41:20.143 INFO:teuthology.orchestra.run.smithi028.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-10T10:41:20.157 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:41:20.176 INFO:teuthology.orchestra.run.smithi028.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-10T10:41:20.243 INFO:teuthology.orchestra.run.smithi028.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-10T10:41:20.277 INFO:teuthology.orchestra.run.smithi028.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-10T10:41:20.369 INFO:teuthology.orchestra.run.smithi028.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-10T10:41:20.370 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:41:20.403 INFO:teuthology.orchestra.run.smithi028.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-10T10:41:20.433 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:41:20.436 INFO:teuthology.orchestra.run.smithi028.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T10:41:20.460 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:41:20.577 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:41:20.603 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:41:20.603 INFO:teuthology.orchestra.run.smithi028.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 199 kB/s | 32 kB 00:00 2024-09-10T10:41:20.629 INFO:teuthology.orchestra.run.smithi028.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-10T10:41:20.662 INFO:teuthology.orchestra.run.smithi028.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-10T10:41:20.678 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:41:20.687 INFO:teuthology.orchestra.run.smithi028.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-09-10T10:41:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:41:20.721 INFO:teuthology.orchestra.run.smithi028.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T10:41:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:41:20.753 INFO:teuthology.orchestra.run.smithi028.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.6 MB/s | 303 kB 00:00 2024-09-10T10:41:20.786 INFO:teuthology.orchestra.run.smithi028.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.3 MB/s | 220 kB 00:00 2024-09-10T10:41:20.820 INFO:teuthology.orchestra.run.smithi028.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T10:41:20.845 INFO:teuthology.orchestra.run.smithi028.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T10:41:20.870 INFO:teuthology.orchestra.run.smithi028.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 720 kB/s | 18 kB 00:00 2024-09-10T10:41:20.904 INFO:teuthology.orchestra.run.smithi028.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 696 kB/s | 23 kB 00:00 2024-09-10T10:41:20.954 INFO:teuthology.orchestra.run.smithi028.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-10T10:41:20.980 INFO:teuthology.orchestra.run.smithi028.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 797 kB/s | 20 kB 00:00 2024-09-10T10:41:21.038 INFO:teuthology.orchestra.run.smithi028.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-09-10T10:41:21.055 INFO:teuthology.orchestra.run.smithi028.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-09-10T10:41:21.089 INFO:teuthology.orchestra.run.smithi028.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T10:41:21.122 INFO:teuthology.orchestra.run.smithi028.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-10T10:41:21.133 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:41:21.139 INFO:teuthology.orchestra.run.smithi028.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T10:41:21.156 INFO:teuthology.orchestra.run.smithi028.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-10T10:41:21.181 INFO:teuthology.orchestra.run.smithi028.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-10T10:41:21.198 INFO:teuthology.orchestra.run.smithi028.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-10T10:41:21.224 INFO:teuthology.orchestra.run.smithi028.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 677 kB/s | 28 kB 00:00 2024-09-10T10:41:21.274 INFO:teuthology.orchestra.run.smithi028.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-09-10T10:41:21.299 INFO:teuthology.orchestra.run.smithi028.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T10:41:21.319 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:41:21.324 INFO:teuthology.orchestra.run.smithi028.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T10:41:21.358 INFO:teuthology.orchestra.run.smithi028.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-10T10:41:21.383 INFO:teuthology.orchestra.run.smithi028.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-10T10:41:21.409 INFO:teuthology.orchestra.run.smithi028.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T10:41:21.434 INFO:teuthology.orchestra.run.smithi028.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T10:41:21.459 INFO:teuthology.orchestra.run.smithi028.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-10T10:41:21.493 INFO:teuthology.orchestra.run.smithi028.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-10T10:41:21.497 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:41:21.527 INFO:teuthology.orchestra.run.smithi028.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-10T10:41:21.610 INFO:teuthology.orchestra.run.smithi028.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T10:41:21.620 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:21.794 INFO:teuthology.orchestra.run.smithi028.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-10T10:41:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:22.162 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:41:22.270 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:41:22.315 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:41:22.327 INFO:teuthology.orchestra.run.smithi028.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 4.4 MB/s | 2.3 MB 00:00 2024-09-10T10:41:22.357 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:41:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:41:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:41:22.519 INFO:teuthology.orchestra.run.smithi028.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 3.5 MB/s | 3.2 MB 00:00 2024-09-10T10:41:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:41:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:41:22.633 INFO:teuthology.orchestra.run.smithi028.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 359 kB/s | 41 kB 00:00 2024-09-10T10:41:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:41:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:41:22.753 INFO:teuthology.orchestra.run.smithi028.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.0 MB/s | 249 kB 00:00 2024-09-10T10:41:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:41:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:41:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:41:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:41:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:41:23.061 INFO:teuthology.orchestra.run.smithi028.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-10T10:41:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:41:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:41:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:41:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:41:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:41:23.329 INFO:teuthology.orchestra.run.smithi028.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-10T10:41:23.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:41:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:41:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:41:23.854 INFO:teuthology.orchestra.run.smithi028.stdout:(155/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.4 MB/s | 16 MB 00:04 2024-09-10T10:41:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:41:24.371 INFO:teuthology.orchestra.run.smithi028.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-09-10T10:41:24.376 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-09-10T10:41:24.376 INFO:teuthology.orchestra.run.smithi028.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-10T10:41:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:41:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:41:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:41:24.609 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:41:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:41:24.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:41:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:41:24.990 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:41:25.075 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:41:25.122 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:41:25.159 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:41:25.217 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:41:25.258 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:41:25.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:41:25.354 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:41:25.524 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:41:25.580 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:41:25.622 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:41:25.741 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:41:25.809 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:41:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:41:25.891 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:41:25.915 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:41:26.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:41:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:41:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:41:26.286 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:41:26.324 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:41:26.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:41:26.418 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:41:26.455 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:41:26.467 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-09-10T10:41:26.490 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:41:26.523 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:41:26.566 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:41:26.600 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:41:26.665 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-09-10T10:41:26.665 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-09-10T10:41:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:41:26.777 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:41:26.820 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:41:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:41:27.000 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:41:27.870 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:41:27.934 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:41:27.968 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:41:28.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:41:28.142 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:41:28.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:41:28.309 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:41:28.355 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:41:28.588 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:41:28.627 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:41:28.750 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:41:29.122 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-09-10T10:41:29.124 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-09-10T10:41:29.546 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:41:29.725 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:41:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:41:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:41:31.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:41:31.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:41:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:41:31.843 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:41:31.955 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:41:31.991 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:41:32.042 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:41:32.566 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:41:32.613 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:41:32.705 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:41:32.732 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:41:35.226 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:35.272 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:35.942 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:41:36.084 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:41:36.085 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:41:36.085 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:41:36.085 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:36.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:41:36.429 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-09-10T10:41:36.576 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T10:41:36.627 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T10:41:36.693 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T10:41:36.736 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T10:41:36.768 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T10:41:36.827 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T10:41:37.219 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T10:41:37.326 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:41:37.516 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:41:37.544 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:41:37.658 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:41:37.716 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:41:37.947 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T10:41:38.005 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T10:41:38.057 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:41:38.184 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:41:38.215 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:41:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T10:41:38.316 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:41:38.355 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T10:41:38.610 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:41:38.807 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T10:41:39.023 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T10:41:39.153 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:39.241 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:39.525 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T10:41:39.670 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T10:41:39.788 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T10:41:39.830 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T10:41:39.875 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T10:41:39.960 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T10:41:40.012 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T10:41:40.050 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T10:41:40.131 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T10:41:40.168 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T10:41:40.202 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T10:41:40.293 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T10:41:40.355 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T10:41:40.392 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T10:41:40.461 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T10:41:40.552 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T10:41:40.592 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T10:41:40.637 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T10:41:40.664 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T10:41:40.760 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T10:41:40.817 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T10:41:40.911 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T10:41:41.092 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T10:41:41.163 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T10:41:41.833 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T10:41:41.939 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T10:41:41.981 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T10:41:42.025 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T10:41:42.063 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T10:41:42.138 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T10:41:42.343 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T10:41:42.399 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T10:41:42.500 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T10:41:42.598 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T10:41:42.641 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T10:41:42.678 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T10:41:42.737 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T10:41:42.807 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T10:41:42.859 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T10:41:42.910 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T10:41:43.090 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T10:41:43.151 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T10:41:43.202 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T10:41:43.334 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T10:41:43.375 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T10:41:43.417 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T10:41:43.447 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T10:41:43.505 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T10:41:43.622 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T10:41:43.788 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T10:41:43.871 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T10:41:43.907 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T10:41:43.950 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T10:41:43.991 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T10:41:44.039 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T10:41:44.075 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T10:41:44.110 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T10:41:44.151 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T10:41:44.219 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T10:41:44.269 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T10:41:44.370 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T10:41:44.426 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T10:41:44.462 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T10:41:44.590 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T10:41:44.650 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T10:41:45.588 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T10:41:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T10:41:45.685 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T10:41:45.804 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T10:41:45.868 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T10:41:45.947 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T10:41:46.055 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T10:41:46.095 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T10:41:46.342 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T10:41:46.375 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T10:41:46.473 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T10:41:47.342 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T10:41:47.493 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T10:41:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:41:49.088 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-09-10T10:41:49.088 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-09-10T10:41:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:49.154 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T10:41:49.288 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T10:41:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:41:49.323 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T10:41:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:41:49.335 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:49.335 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:41:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:49.356 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T10:41:49.712 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T10:41:49.758 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:41:49.880 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T10:41:49.915 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T10:41:49.991 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T10:41:50.543 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T10:41:50.586 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:41:50.694 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T10:41:50.721 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T10:41:50.929 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:41:50.987 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:41:50.987 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:50.987 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:41:50.987 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:41:50.987 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:41:50.988 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:51.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:41:51.280 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:51.764 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:41:51.814 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:41:51.815 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:51.815 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:41:51.815 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:41:51.815 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:41:51.815 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:51.939 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:41:52.000 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:41:52.072 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:41:52.297 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:41:52.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:41:52.447 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:41:52.580 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:41:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:41:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:41:52.835 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:41:52.861 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:41:53.389 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:41:53.434 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:53.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:41:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:41:53.570 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:41:53.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:41:54.094 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T10:41:54.145 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:41:54.280 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T10:41:54.280 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T10:41:54.280 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T10:41:54.280 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:41:54.322 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:41:54.827 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:41:54.878 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:41:55.963 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:41:55.996 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:41:56.133 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:41:56.303 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:41:56.340 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:41:56.392 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:56.441 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:41:56.467 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:41:56.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:41:56.642 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:41:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:41:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:41:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:41:57.719 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:57.720 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:41:57.720 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:41:57.720 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:41:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:41:57.819 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:41:57.819 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:57.819 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:41:57.820 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:57.871 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:41:57.921 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:41:57.921 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:57.921 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:41:57.922 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:41:57.922 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:41:57.922 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:41:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:41:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:41:58.315 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:41:58.315 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:41:58.315 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:41:58.315 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:41:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:03.261 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:42:03.311 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:42:03.355 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:42:03.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:42:03.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:42:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:42:03.710 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:42:03.740 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:42:03.771 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:42:03.815 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:42:05.035 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:05.090 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:05.131 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:06.394 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:42:06.508 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:42:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:42:06.668 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:42:06.702 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:42:06.728 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:42:06.773 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:42:06.773 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:42:06.931 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:42:06.931 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:42:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:42:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:42:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:07.367 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:07.367 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:42:07.477 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:42:07.558 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:07.779 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T10:42:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:08.079 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T10:42:08.079 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-09-10T10:42:08.079 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-09-10T10:42:08.079 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-09-10T10:42:08.080 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-09-10T10:42:08.080 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-09-10T10:42:08.080 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-09-10T10:42:08.080 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-09-10T10:42:08.080 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:08.300 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:42:08.337 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T10:42:08.338 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:08.338 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T10:42:08.338 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:10.028 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T10:42:10.086 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:10.353 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:42:10.404 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T10:42:10.404 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:10.405 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T10:42:10.405 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:42:10.405 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T10:42:10.405 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:10.912 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:42:10.963 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T10:42:10.963 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:10.963 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T10:42:10.963 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:42:10.964 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T10:42:10.964 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:11.096 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T10:42:11.157 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T10:42:11.230 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T10:42:11.467 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T10:42:11.525 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T10:42:11.613 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T10:42:11.757 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T10:42:11.945 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T10:42:12.030 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T10:42:12.072 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T10:42:12.097 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T10:42:12.613 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:42:12.638 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T10:42:12.715 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T10:42:12.792 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T10:42:12.818 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:42:13.786 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:42:13.786 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:42:13.787 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:42:13.788 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:42:13.789 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:42:13.790 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:42:13.791 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:42:13.792 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:42:13.793 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:42:13.794 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:42:13.795 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:42:13.796 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:42:13.797 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:42:13.798 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:42:13.799 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:42:14.113 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T10:42:14.154 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:42:15.305 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T10:42:15.337 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:42:15.482 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T10:42:15.668 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T10:42:15.734 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T10:42:15.786 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:15.848 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:42:15.875 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T10:42:15.985 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T10:42:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:16.012 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:42:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:42:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:42:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:42:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:42:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:42:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:42:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:42:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:42:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:42:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:42:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:16.025 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-10T10:42:16.051 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T10:42:16.604 DEBUG:teuthology.parallel:result is None 2024-09-10T10:42:17.134 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T10:42:17.176 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:42:17.227 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T10:42:17.227 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:17.227 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T10:42:17.227 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:42:17.227 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T10:42:17.228 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:17.285 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:42:17.335 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T10:42:17.335 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:17.335 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T10:42:17.335 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:17.393 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T10:42:17.445 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:17.831 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T10:42:17.884 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:23.133 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T10:42:23.200 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T10:42:23.244 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T10:42:23.280 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T10:42:23.316 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T10:42:23.561 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T10:42:23.612 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T10:42:23.642 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T10:42:23.673 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T10:42:23.717 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T10:42:24.484 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:24.552 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:24.595 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T10:42:25.984 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T10:42:26.094 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T10:42:26.144 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T10:42:26.270 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T10:42:26.313 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T10:42:26.338 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T10:42:26.375 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T10:42:26.375 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:42:26.482 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T10:42:26.482 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:42:26.617 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T10:42:26.640 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T10:42:26.657 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:26.657 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:26.936 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T10:42:26.936 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:42:27.053 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T10:42:27.129 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:27.340 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:27.340 INFO:teuthology.orchestra.run.smithi028.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T10:42:27.340 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:32.899 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T10:42:32.899 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T10:42:32.900 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T10:42:32.901 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T10:42:32.902 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T10:42:32.913 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T10:42:32.914 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T10:42:32.915 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T10:42:32.916 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T10:42:32.917 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T10:42:32.918 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T10:42:32.919 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T10:42:32.920 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T10:42:32.921 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T10:42:32.922 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T10:42:32.923 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T10:42:35.213 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T10:42:35.213 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T10:42:35.214 INFO:teuthology.orchestra.run.smithi028.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.215 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.216 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T10:42:35.217 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.218 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T10:42:35.219 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T10:42:35.220 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T10:42:35.221 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T10:42:35.222 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T10:42:35.223 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T10:42:35.224 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T10:42:35.225 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T10:42:35.226 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:35.227 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-09-10T10:42:35.832 DEBUG:teuthology.parallel:result is None 2024-09-10T10:42:35.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:42:35.974 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:42:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:42:36.019 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:42:36.019 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:42:36.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T10:42:36.162 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T10:42:36.214 INFO:teuthology.orchestra.run.smithi138.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T10:42:36.214 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T10:42:36.214 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T10:42:36.216 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T10:42:36.217 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:36.217 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:42:36.276 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:36.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T10:42:36.338 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T10:42:36.338 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:36.338 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:42:36.394 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:42:36.493 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:36.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T10:42:36.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T10:42:36.650 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T10:42:36.650 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:36.650 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:42:36.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:42:36.814 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:36.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T10:42:36.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T10:42:36.965 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T10:42:36.966 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:36.966 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:42:37.024 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:42:37.126 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:37.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T10:42:37.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T10:42:37.276 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T10:42:37.362 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T10:42:37.362 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /var/log/ceph 2024-09-10T10:42:37.366 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-09-10T10:42:37.421 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T10:42:37.422 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T10:42:37.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T10:42:37.482 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T10:42:37.482 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T10:42:37.486 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T10:42:37.551 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T10:42:37.551 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T10:42:37.551 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T10:42:37.552 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:37.552 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T10:42:37.588 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T10:42:37.769 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T10:42:37.833 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:37.833 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T10:42:37.868 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T10:42:38.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T10:42:38.208 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T10:42:38.208 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T10:42:38.209 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045', 'branch': 'wip-rishabh-testing-20240909.165402-squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage}', 'email': None, 'first_in_suite': False, 'job_id': '7899045', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240909.165402-squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240909.165402-squid', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9893, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'sleep_before_teardown': 0, 'subset': '197/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240909.165402-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE05KKoqvorJC3LwqhYbF1M4Qu4oLB2lI++HVNTrmGs7EPzpOl87/THOrywtJbeYLBC1lXf3LWYXzQcuZ6IitEg=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJRzMU+BWBryOKI1ABWbsEP1IbvGqek2HnOJKavGiuVVOURDVYOTuXJ/1hSAwYmZUErENqDrE4pAhv5lw+zMyGE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only', 'with standby daemon mds', 'MDS abort because newly corrupt dentry'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_06:58:10', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T10:42:38.209 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T10:42:38.214 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T10:42:38.250 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T10:42:38.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T10:42:38.355 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:38.356 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:42:38.391 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:42:38.392 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-09-10T10:42:38.469 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:42:27.701435279 +0000 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:40:07.922753189 +0000 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:40:07.922753189 +0000 2024-09-10T10:42:38.470 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:40:07.922753189 +0000 2024-09-10T10:42:38.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:42:38.565 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-10T10:42:38.565 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-10T10:42:38.565 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000209175 s, 2.4 MB/s 2024-09-10T10:42:38.568 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:42:38.606 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:42:27.702435252 +0000 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:40:08.361741333 +0000 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:40:08.361741333 +0000 2024-09-10T10:42:38.683 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:40:08.361741333 +0000 2024-09-10T10:42:38.684 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:42:38.778 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-10T10:42:38.778 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-10T10:42:38.778 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000164642 s, 3.1 MB/s 2024-09-10T10:42:38.780 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:42:38.818 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:42:27.702435252 +0000 2024-09-10T10:42:38.895 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:40:08.855727991 +0000 2024-09-10T10:42:38.896 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:40:08.855727991 +0000 2024-09-10T10:42:38.896 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:40:08.855727991 +0000 2024-09-10T10:42:38.896 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:42:38.988 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-10T10:42:38.989 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-10T10:42:38.989 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000176943 s, 2.9 MB/s 2024-09-10T10:42:38.992 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:42:39.032 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-09-10T10:42:39.107 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:42:39.107 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-10T10:42:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:42:27.703435225 +0000 2024-09-10T10:42:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:40:09.336715000 +0000 2024-09-10T10:42:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:40:09.336715000 +0000 2024-09-10T10:42:39.108 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:40:09.336715000 +0000 2024-09-10T10:42:39.108 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:42:39.201 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-09-10T10:42:39.201 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-09-10T10:42:39.201 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000175801 s, 2.9 MB/s 2024-09-10T10:42:39.204 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:42:39.241 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:39.241 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:39.241 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T10:42:39.279 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T10:42:39.279 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-09-10T10:42:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-10 10:42:08.127308387 +0000 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-10 10:40:05.623033706 +0000 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-10 10:40:05.623033706 +0000 2024-09-10T10:42:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-10 10:40:05.623033706 +0000 2024-09-10T10:42:39.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T10:42:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-10T10:42:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-10T10:42:39.450 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000274714 s, 1.9 MB/s 2024-09-10T10:42:39.453 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T10:42:39.490 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-09-10T10:42:39.564 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T10:42:39.564 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-10 10:42:08.128308365 +0000 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-10 10:40:06.071023852 +0000 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-10 10:40:06.071023852 +0000 2024-09-10T10:42:39.565 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-10 10:40:06.071023852 +0000 2024-09-10T10:42:39.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T10:42:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-10T10:42:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-10T10:42:39.657 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000276334 s, 1.9 MB/s 2024-09-10T10:42:39.660 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T10:42:39.695 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 676 Links: 1 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-10 10:42:08.129308344 +0000 2024-09-10T10:42:39.773 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-10 10:40:06.517014043 +0000 2024-09-10T10:42:39.774 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-10 10:40:06.517014043 +0000 2024-09-10T10:42:39.774 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-10 10:40:06.517014043 +0000 2024-09-10T10:42:39.774 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T10:42:39.865 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-10T10:42:39.866 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-10T10:42:39.866 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000326758 s, 1.6 MB/s 2024-09-10T10:42:39.868 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T10:42:39.904 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 696 Links: 1 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-10 10:42:08.129308344 +0000 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-10 10:40:06.978003904 +0000 2024-09-10T10:42:39.981 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-10 10:40:06.978003904 +0000 2024-09-10T10:42:39.982 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-10 10:40:06.978003904 +0000 2024-09-10T10:42:39.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T10:42:40.073 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-10T10:42:40.074 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-10T10:42:40.074 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000280259 s, 1.8 MB/s 2024-09-10T10:42:40.077 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T10:42:40.114 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:40.114 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T10:42:40.114 INFO:tasks.ceph:Generating config... 2024-09-10T10:42:40.116 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-10T10:42:40.116 INFO:tasks.ceph:[client] debug client = 20 2024-09-10T10:42:40.117 INFO:tasks.ceph:[client] debug ms = 1 2024-09-10T10:42:40.117 INFO:tasks.ceph:[client] pid file = 2024-09-10T10:42:40.117 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-10T10:42:40.117 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-10T10:42:40.117 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T10:42:40.117 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-10T10:42:40.118 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T10:42:40.119 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T10:42:40.120 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T10:42:40.120 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T10:42:40.120 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-10T10:42:40.120 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T10:42:40.120 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T10:42:40.121 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-10T10:42:40.121 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T10:42:40.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T10:42:40.242 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T10:42:40.248 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T10:42:40.336 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:42:40.395 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.28'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2024-09-10T10:42:40.396 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.28,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-10T10:42:40.398 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:40.398 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T10:42:40.433 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.28 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: generated fsid f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = quincy 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 0 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:fsid f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:last_changed 2024-09-10T10:42:40.550526+0000 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-09-10T10:42:40.550526+0000 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:min_mon_release 17 (quincy) 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:election_strategy: 1 2024-09-10T10:42:40.551 INFO:teuthology.orchestra.run.smithi028.stdout:0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-09-10T10:42:40.552 INFO:teuthology.orchestra.run.smithi028.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-10T10:42:40.552 INFO:teuthology.orchestra.run.smithi028.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2024-09-10T10:42:40.552 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T10:42:40.554 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T10:42:40.588 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f658987a-dd14-4573-bcd3-ee2a383c9fe0... 2024-09-10T10:42:40.592 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T10:42:40.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T10:42:40.726 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-09-10T10:42:40.726 INFO:teuthology.orchestra.run.smithi028.stdout: chdir = "" 2024-09-10T10:42:40.726 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T10:42:40.726 INFO:teuthology.orchestra.run.smithi028.stdout: auth supported = cephx 2024-09-10T10:42:40.726 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: filestore xattr use omap = true 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: mon clock drift allowed = 1.000 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: osd crush chooseleaf type = 0 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: auth debug = true 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on old message = true 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on bug = true 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.727 INFO:teuthology.orchestra.run.smithi028.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon pg warn max object skew = 0 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default size = 2 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow primary affinity = true 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow pg remap = true 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on legacy crush tunables = false 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on crush straw calc version zero = false 2024-09-10T10:42:40.728 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on no sortbitwise = false 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on osd down out interval zero = false 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on too few osds = false 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon_allow_pool_size_one = true 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: osd default data pool replay window = 5 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon allow pool delete = true 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: mon cluster log file level = debug 2024-09-10T10:42:40.729 INFO:teuthology.orchestra.run.smithi028.stdout: debug asserts on shutdown = true 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: mon health detail to clog = false 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: mon host = "172.21.15.28,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on skipped message = False 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout:[osd] 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd journal size = 100 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub load threshold = 5.0 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub max interval = 600 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd mclock profile = high_recovery_ops 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd recover clone overlap = true 2024-09-10T10:42:40.730 INFO:teuthology.orchestra.run.smithi028.stdout: osd recovery max chunk = 1048576 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug shutdown = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug op order = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify stray on activate = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug trim objects = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd open classes on start = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug pg log writeout = true 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd deep scrub update digest min age = 30 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: osd map max advance = 10 2024-09-10T10:42:40.731 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: journal zero on create = true 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: filestore ondisk finisher threads = 3 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: filestore apply finisher threads = 3 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: bdev debug aio = true 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug misdirected ops = true 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore block size = 96636764160 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore fsck on mount = True 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluefs = 20 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluestore = 20 2024-09-10T10:42:40.732 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: debug osd = 20 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: debug rocksdb = 10 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd full ratio = 0.9 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd nearfull ratio = 0.8 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: osd failsafe full ratio = 0.95 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: osd objectstore = bluestore 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr] 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: debug mgr = 20 2024-09-10T10:42:40.733 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mgr/telemetry/nag = false 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug client = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout:[mon] 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug paxos = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mon data avail warn = 5 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mon mgr mkfs grace = 240 2024-09-10T10:42:40.734 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd reporter subtree level = osd 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd prime pg temp = true 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: # 1m isn't quite enough 2024-09-10T10:42:40.735 INFO:teuthology.orchestra.run.smithi028.stdout: mon_down_mkfs_grace = 2m 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_filestore_osds = false 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: mon down mkfs grace = 300 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: mon op complaint time = 120 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on pool no app = False 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout:[client] 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: rgw cache enabled = true 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable ops log = true 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable usage log = true 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T10:42:40.736 INFO:teuthology.orchestra.run.smithi028.stdout: client mount timeout = 600 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout: debug client = 20 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = "" 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.a] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.b] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.c] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2024-09-10T10:42:40.737 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout:[mds] 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds = 20 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds balancer = 20 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug frag = True 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug scatterstat = True 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: mds op complaint time = 180 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: mds verify scatter = True 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2024-09-10T10:42:40.738 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-09-10T10:42:40.739 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.740 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.741 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.28,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on skipped message = False 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.742 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-09-10T10:42:40.743 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-09-10T10:42:40.744 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-09-10T10:42:40.745 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-10T10:42:40.746 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.747 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on pool no app = False 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-09-10T10:42:40.748 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-09-10T10:42:40.749 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-09-10T10:42:40.750 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-09-10T10:42:40.751 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-09-10T10:42:40.759 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T10:42:40.759 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T10:42:40.856 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T10:42:40.857 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:40.857 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T10:42:40.891 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:40.892 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T10:42:40.968 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:42:40.968 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:40.968 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T10:42:40.968 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:42:41.086 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:42:41.087 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:41.162 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:42:41.162 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:41.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T10:42:41.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T10:42:41.239 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:42:41.239 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:41.315 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T10:42:41.315 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T10:42:41.315 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T10:42:41.419 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T10:42:41.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T10:42:41.545 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T10:42:41.552 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T10:42:41.552 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T10:42:41.657 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T10:42:41.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T10:42:41.718 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T10:42:41.821 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T10:42:41.826 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T10:42:41.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T10:42:41.986 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T10:42:41.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T10:42:42.047 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T10:42:42.151 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T10:42:42.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T10:42:42.211 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T10:42:42.211 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T10:42:42.293 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T10:42:42.321 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T10:42:42.402 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T10:42:42.429 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T10:42:42.510 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T10:42:42.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T10:42:42.617 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T10:42:42.646 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T10:42:42.646 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T10:42:42.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T10:42:42.703 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:42.703 INFO:tasks.ceph:role: osd.0 2024-09-10T10:42:42.704 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:42:42.704 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T10:42:42.759 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:42.760 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:42.799 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-10T10:42:42.803 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-10T10:42:42.803 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T10:42:42.883 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T10:42:42.945 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T10:42:43.002 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:43.002 INFO:tasks.ceph:role: osd.1 2024-09-10T10:42:43.002 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:42:43.002 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:43.102 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-10T10:42:43.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-10T10:42:43.105 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T10:42:43.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T10:42:43.247 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T10:42:43.303 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:43.304 INFO:tasks.ceph:role: osd.2 2024-09-10T10:42:43.304 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:42:43.304 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:43.359 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:43.399 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-10T10:42:43.402 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-10T10:42:43.402 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T10:42:43.483 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T10:42:43.544 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T10:42:43.599 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:43.599 INFO:tasks.ceph:role: osd.3 2024-09-10T10:42:43.600 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:42:43.600 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T10:42:43.655 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:43.655 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:43.656 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:43.695 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-09-10T10:42:43.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-09-10T10:42:43.699 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T10:42:43.779 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T10:42:43.840 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:43.960+0000 7f6c52217640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T10:42:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:43.961+0000 7f6c52217640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T10:42:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:43.961+0000 7f6c52217640 -1 bdev(0x56438f199800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T10:42:43.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:43.961+0000 7f6c52217640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T10:42:46.977 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T10:42:47.039 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:47.130+0000 7f3e4280f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T10:42:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:47.130+0000 7f3e4280f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T10:42:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:47.131+0000 7f3e4280f640 -1 bdev(0x55651ea4b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T10:42:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:47.131+0000 7f3e4280f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T10:42:50.145 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T10:42:50.204 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:50.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:50.294+0000 7f5db640d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T10:42:50.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:50.295+0000 7f5db640d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T10:42:50.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:50.295+0000 7f5db640d640 -1 bdev(0x5603c45a3800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T10:42:50.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:50.295+0000 7f5db640d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T10:42:53.339 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T10:42:53.398 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:53.488+0000 7f75daf29640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T10:42:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:53.489+0000 7f75daf29640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T10:42:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:53.489+0000 7f75daf29640 -1 bdev(0x555ebb885800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T10:42:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:42:53.489+0000 7f75daf29640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T10:42:56.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T10:42:56.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T10:42:56.664 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:56.664 INFO:tasks.ceph:role: osd.4 2024-09-10T10:42:56.664 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:42:56.664 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T10:42:56.728 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:56.728 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:56.729 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:56.730 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:56.730 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:57.135 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-10T10:42:57.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-10T10:42:57.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T10:42:57.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T10:42:57.323 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T10:42:57.379 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:57.379 INFO:tasks.ceph:role: osd.5 2024-09-10T10:42:57.379 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:42:57.379 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T10:42:57.441 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:57.441 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:57.441 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:57.442 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:57.802 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-10T10:42:57.806 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-10T10:42:57.806 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T10:42:57.891 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T10:42:57.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T10:42:58.008 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:58.008 INFO:tasks.ceph:role: osd.6 2024-09-10T10:42:58.009 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:42:58.009 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T10:42:58.070 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:58.070 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:58.070 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:58.071 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:58.460 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-10T10:42:58.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-10T10:42:58.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T10:42:58.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T10:42:58.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T10:42:58.679 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T10:42:58.679 INFO:tasks.ceph:role: osd.7 2024-09-10T10:42:58.679 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:42:58.679 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T10:42:58.742 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T10:42:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T10:42:58.743 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T10:42:59.126 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-10T10:42:59.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-10T10:42:59.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T10:42:59.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T10:42:59.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:42:59.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:42:59.442+0000 7fdcb0958640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T10:42:59.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:42:59.444+0000 7fdcb0958640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T10:42:59.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:42:59.444+0000 7fdcb0958640 -1 bdev(0x55f584d03800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T10:42:59.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:42:59.444+0000 7fdcb0958640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T10:43:02.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T10:43:02.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:43:02.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:02.586+0000 7f33c782f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T10:43:02.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:02.586+0000 7f33c782f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T10:43:02.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:02.586+0000 7f33c782f640 -1 bdev(0x5607d3eab800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T10:43:02.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:02.587+0000 7f33c782f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T10:43:05.658 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T10:43:05.716 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:43:05.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:05.806+0000 7fc89d414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T10:43:05.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:05.806+0000 7fc89d414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T10:43:05.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:05.806+0000 7fc89d414640 -1 bdev(0x5584bd0a5800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T10:43:05.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:05.806+0000 7fc89d414640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T10:43:08.210 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:43:08.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:43:09.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T10:43:09.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:43:09.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:09.199+0000 7f4b5641a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T10:43:09.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:09.200+0000 7f4b5641a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T10:43:09.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:09.200+0000 7f4b5641a640 -1 bdev(0x562d45117800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T10:43:09.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:09.200+0000 7f4b5641a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T10:43:12.222 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T10:43:12.282 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T10:43:12.282 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.282 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T10:43:12.345 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.345 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T10:43:12.444 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T10:43:12.541 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.541 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T10:43:12.637 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.637 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T10:43:12.733 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.733 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T10:43:12.828 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:12.828 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T10:43:12.923 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:12.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T10:43:12.986 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:12.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T10:43:13.085 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T10:43:13.182 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T10:43:13.278 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.278 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T10:43:13.372 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T10:43:13.467 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T10:43:13.563 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:13.563 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T10:43:13.600 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:13.601 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T10:43:13.677 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.677 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T10:43:13.711 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:13.711 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T10:43:13.785 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T10:43:13.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T10:43:13.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T10:43:13.839 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.y] 2024-09-10T10:43:13.839 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA== 2024-09-10T10:43:13.839 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2024-09-10T10:43:13.839 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.0] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.1] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.2] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.3] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA== 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.x] 2024-09-10T10:43:13.840 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.4] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.5] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.6] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.7] 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww== 2024-09-10T10:43:13.841 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout:[client.2] 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg== 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout:[client.1] 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w== 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout:[client.3] 2024-09-10T10:43:13.842 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg== 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA== 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw== 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ== 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw== 2024-09-10T10:43:13.878 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA== 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-09-10T10:43:13.879 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w== 2024-09-10T10:43:13.880 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-09-10T10:43:13.881 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg== 2024-09-10T10:43:13.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:43:13.887 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:43:14.016 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.116 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.313 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.318 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.410 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.509 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.607 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:43:14.611 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T10:43:14.705 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.805 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.810 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T10:43:14.906 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:14.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.005 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T10:43:15.297 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.398 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.498 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T10:43:15.699 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T10:43:15.699 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T10:43:15.755 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:43:15.862 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T10:43:15.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T10:43:15.974 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:43:16.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T10:43:16.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T10:43:16.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T10:43:16.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T10:43:16.354 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:43:16.358 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T10:43:16.392 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T10:43:16.393 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T10:43:16.393 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T10:43:16.435 INFO:tasks.ceph.mon.a:Started 2024-09-10T10:43:16.435 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T10:43:16.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T10:43:16.439 INFO:tasks.ceph.mon.b:Started 2024-09-10T10:43:16.439 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T10:43:16.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T10:43:16.444 INFO:tasks.ceph.mon.c:Started 2024-09-10T10:43:16.444 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T10:43:16.444 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T10:43:16.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T10:43:16.448 INFO:tasks.ceph.mgr.y:Started 2024-09-10T10:43:16.448 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T10:43:16.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T10:43:16.451 INFO:tasks.ceph.mgr.x:Started 2024-09-10T10:43:16.451 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T10:43:16.452 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T10:43:16.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.607+0000 7f9c61600640 1 Processor -- start 2024-09-10T10:43:16.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.607+0000 7f9c61600640 1 -- start start 2024-09-10T10:43:16.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c61600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:16.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c61600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c15ff70 con 0x7f9c5c0cf280 2024-09-10T10:43:16.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c61600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c0d4d10 con 0x7f9c5c162e30 2024-09-10T10:43:16.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c61600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9c5c1632a0 con 0x7f9c5c162340 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:58174/0 (socket says 172.21.15.28:58174) 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 learned_addr learned my addr 172.21.15.28:0/2669734262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.608+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:43:16.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.809+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:16.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.809+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:16.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.809+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:16.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.809+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:16.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:16.809+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:43:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:17.210+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:17.210+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:17.211+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:17.211+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:17.211+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:43:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:18.012+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:18.012+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:18.012+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:18.012+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:18.012+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:43:19.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c5c06d740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c5c06d740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 >> v1:172.21.15.138:6789/0 conn(0x7f9c5c162e30 legacy=0x7f9c5c0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 >> v1:172.21.15.28:6789/0 conn(0x7f9c5c0cf280 legacy=0x7f9c5c162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9c3c008d20 con 0x7f9c3c0036e0 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c3c008ed0 con 0x7f9c3c006180 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.608+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:19.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.609+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:43:19.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:19.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:19.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:19.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:19.810+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:20.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:20.211+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:43:21.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.011+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-10T10:43:21.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5aa00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:43:21.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:21.012+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:43:21.767 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:21.766+0000 7f408593b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:43:21.792 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:21.790+0000 7f9f9f520040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T10:43:21.816 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:21.815+0000 7f408593b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:43:21.842 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:21.840+0000 7f9f9f520040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T10:43:21.984 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:21.982+0000 7f408593b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:43:22.030 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.028+0000 7f9f9f520040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T10:43:22.041 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.040+0000 7f408593b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:43:22.092 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.090+0000 7f9f9f520040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T10:43:22.221 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.220+0000 7f408593b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:43:22.280 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.278+0000 7f9f9f520040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:43:22.346 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.345+0000 7f408593b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:43:22.412 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.410+0000 7f9f9f520040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T10:43:22.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 msgr2=0x7f9c3c008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:43:22.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c3c006180 0x7f9c3c008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:43:22.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c5b400640 1 -- 172.21.15.28:0/2669734262 reap_dead start 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c3c00ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c5c0d4ff0 con 0x7f9c5c162e30 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9c3c008d20 con 0x7f9c3c0036e0 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.607+0000 7f9c53e00640 1 -- 172.21.15.28:0/2669734262 --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9c3c008ed0 con 0x7f9c5c162340 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.28:3300/0 conn(0x7f9c3c0036e0 0x7f9c3c005b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5b400640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c3c00ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 msgr2=0x7f9c3c00ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c3c00ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 >> v1:172.21.15.138:6789/0 conn(0x7f9c5c162e30 legacy=0x7f9c3c00b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.608+0000 7f9c5a000640 1 -- 172.21.15.28:0/2669734262 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c3c007660 con 0x7f9c3c0036e0 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.609+0000 7f9c5a000640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c3c0036e0 0x7f9c3c005b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c08d460 tx=0x7f9c4802a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c2968c494bcd39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.609+0000 7f9c59600640 1 -- 172.21.15.28:0/2669734262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48037070 con 0x7f9c3c0036e0 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.609+0000 7f9c59600640 1 -- 172.21.15.28:0/2669734262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4802c040 con 0x7f9c3c0036e0 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.609+0000 7f9c59600640 1 -- 172.21.15.28:0/2669734262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48007290 con 0x7f9c3c0036e0 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 -- 172.21.15.28:0/2669734262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c3c0036e0 msgr2=0x7f9c3c005b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c3c0036e0 0x7f9c3c005b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c08d460 tx=0x7f9c4802a9a0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 -- 172.21.15.28:0/2669734262 shutdown_connections 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 --2- 172.21.15.28:0/2669734262 >> v2:172.21.15.138:3300/0 conn(0x7f9c5c162340 0x7f9c3c00ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 --2- 172.21.15.28:0/2669734262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c3c0036e0 0x7f9c3c005b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 -- 172.21.15.28:0/2669734262 >> 172.21.15.28:0/2669734262 conn(0x7f9c5c08bfc0 msgr2=0x7f9c5c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:22.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 -- 172.21.15.28:0/2669734262 shutdown_connections 2024-09-10T10:43:22.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.610+0000 7f9c61600640 1 -- 172.21.15.28:0/2669734262 wait complete. 2024-09-10T10:43:22.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.612+0000 7f9c61600640 1 Processor -- start 2024-09-10T10:43:22.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.612+0000 7f9c61600640 1 -- start start 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c5c162340 0x7f9c5c0b8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c5c162e30 0x7f9c5c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c0d52d0 con 0x7f9c5c162e30 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c5c06de90 con 0x7f9c3c0036e0 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c61600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c5c0cf6c0 con 0x7f9c5c162340 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c5aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:58260/0 (socket says 172.21.15.28:58260) 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c5aa00640 1 -- 172.21.15.28:0/1005423192 learned_addr learned my addr 172.21.15.28:0/1005423192 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:22.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c5a000640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c5c162340 0x7f9c5c0b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.613+0000 7f9c5b400640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c5c162e30 0x7f9c5c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 -- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c5c162340 msgr2=0x7f9c5c0b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c5c162340 0x7f9c5c0b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 -- 172.21.15.28:0/1005423192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c5c162e30 msgr2=0x7f9c5c0b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c5c162e30 0x7f9c5c0b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 -- 172.21.15.28:0/1005423192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5c0b8aa0 con 0x7f9c3c0036e0 2024-09-10T10:43:22.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c5aa00640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c016720 tx=0x7f9c4c016be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c00f040 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c4c015940 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5c0b8d80 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c015d30 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.614+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c5c0b9340 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.615+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c4c02a400 con 0x7f9c3c0036e0 2024-09-10T10:43:22.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.615+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c4c021820 con 0x7f9c3c0036e0 2024-09-10T10:43:22.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.615+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5c06d740 con 0x7f9c3c0036e0 2024-09-10T10:43:22.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.619+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9c4c02f020 con 0x7f9c3c0036e0 2024-09-10T10:43:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.687+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9c5c08e640 con 0x7f9c3c0036e0 2024-09-10T10:43:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.691+0000 7f9c52a00640 1 -- 172.21.15.28:0/1005423192 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9c4c021a20 con 0x7f9c3c0036e0 2024-09-10T10:43:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 msgr2=0x7f9c5c0b6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c016720 tx=0x7f9c4c016be0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 shutdown_connections 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c5c162e30 0x7f9c5c0b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c5c162340 0x7f9c5c0b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 --2- 172.21.15.28:0/1005423192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c3c0036e0 0x7f9c5c0b6a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 >> 172.21.15.28:0/1005423192 conn(0x7f9c5c08bfc0 msgr2=0x7f9c5c0d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 shutdown_connections 2024-09-10T10:43:22.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:22.692+0000 7f9c61600640 1 -- 172.21.15.28:0/1005423192 wait complete. 2024-09-10T10:43:22.709 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T10:43:22.709 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T10:43:22.710 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:22.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T10:43:22.735 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.734+0000 7f408593b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:43:22.765 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:22.766 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T10:43:22.793 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.792+0000 7f408593b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:43:22.794 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.792+0000 7f9f9f520040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T10:43:22.849 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:22.850 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T10:43:22.866 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:22.864+0000 7f9f9f520040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T10:43:22.921 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:22.921 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T10:43:22.962 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:22.961+0000 7f408593b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:43:23.019 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:23.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T10:43:23.048 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:23.046+0000 7f9f9f520040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T10:43:23.075 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:23.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T10:43:23.172 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:23.172 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T10:43:23.267 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:43:23.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T10:43:23.365 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 897214ac-815c-46c9-bd71-6557ec7b9859 0 2024-09-10T10:43:23.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.556+0000 7fc0d4c00640 1 Processor -- start 2024-09-10T10:43:23.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.556+0000 7fc0d4c00640 1 -- start start 2024-09-10T10:43:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.557+0000 7fc0d4c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0d0162e30 0x7fc0d006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.557+0000 7fc0d4c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0d00cf280 0x7fc0d00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.557+0000 7fc0d4c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d015ff70 con 0x7fc0d0162340 2024-09-10T10:43:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.557+0000 7fc0d4c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d006de90 con 0x7fc0d00cf280 2024-09-10T10:43:23.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.557+0000 7fc0d4c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d0163270 con 0x7fc0d0162e30 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0d0162e30 0x7fc0d006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc0d0162e30 0x7fc0d006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45350/0 (socket says 172.21.15.138:45350) 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0cde00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc0d0162340 0x7fc0d0162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47282/0 (socket says 172.21.15.138:47282) 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 -- 172.21.15.138:0/1302400602 learned_addr learned my addr 172.21.15.138:0/1302400602 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0ce800640 1 --2- 172.21.15.138:0/1302400602 >> v2:172.21.15.28:3300/0 conn(0x7fc0d00cf280 0x7fc0d00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 -- 172.21.15.138:0/1302400602 >> v2:172.21.15.28:3300/0 conn(0x7fc0d00cf280 msgr2=0x7fc0d00d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 --2- 172.21.15.138:0/1302400602 >> v2:172.21.15.28:3300/0 conn(0x7fc0d00cf280 0x7fc0d00d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 -- 172.21.15.138:0/1302400602 >> v1:172.21.15.138:6789/0 conn(0x7fc0d0162340 legacy=0x7fc0d0162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.558+0000 7fc0c7e00640 1 -- 172.21.15.138:0/1302400602 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d00d4d10 con 0x7fc0d0162e30 2024-09-10T10:43:23.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.559+0000 7fc0c7e00640 1 --2- 172.21.15.138:0/1302400602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d006d740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00a23d0 tx=0x7fc0b802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f1174ae09c27e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.559+0000 7fc0cd400640 1 -- 172.21.15.138:0/1302400602 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004030 con 0x7fc0d0162e30 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.559+0000 7fc0cd400640 1 -- 172.21.15.138:0/1302400602 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b80041d0 con 0x7fc0d0162e30 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.559+0000 7fc0cd400640 1 -- 172.21.15.138:0/1302400602 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8003610 con 0x7fc0d0162e30 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1302400602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 msgr2=0x7fc0d006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1302400602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d006d740 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00a23d0 tx=0x7fc0b802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1302400602 shutdown_connections 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1302400602 >> v2:172.21.15.28:3300/0 conn(0x7fc0d00cf280 0x7fc0d00d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1302400602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d006d740 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1302400602 >> 172.21.15.138:0/1302400602 conn(0x7fc0d008bfc0 msgr2=0x7fc0d00d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1302400602 shutdown_connections 2024-09-10T10:43:23.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.560+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1302400602 wait complete. 2024-09-10T10:43:23.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.562+0000 7fc0d4c00640 1 Processor -- start 2024-09-10T10:43:23.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.562+0000 7fc0d4c00640 1 -- start start 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 0x7fc0d00cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0162340 0x7fc0d015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d00cf6f0 con 0x7fc0d0162340 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d00d2470 con 0x7fc0d0162e30 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0d4c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d00d52d0 con 0x7fc0d00cf280 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0cde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 0x7fc0d00cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0ce800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0ce800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45356/0 (socket says 172.21.15.138:45356) 2024-09-10T10:43:23.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0ce800640 1 -- 172.21.15.138:0/1148651841 learned_addr learned my addr 172.21.15.138:0/1148651841 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.563+0000 7fc0c7e00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0162340 0x7fc0d015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 -- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 msgr2=0x7fc0d00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 0x7fc0d00cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 -- 172.21.15.138:0/1148651841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0162340 msgr2=0x7fc0d015cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0162340 0x7fc0d015cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 -- 172.21.15.138:0/1148651841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d00b6040 con 0x7fc0d0162e30 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0cde00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 0x7fc0d00cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0ce800640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800efc0 tx=0x7fc0c800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8018020 con 0x7fc0d0162e30 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c8014480 con 0x7fc0d0162e30 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d00b6320 con 0x7fc0d0162e30 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014870 con 0x7fc0d0162e30 2024-09-10T10:43:23.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.564+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d00b68e0 con 0x7fc0d0162e30 2024-09-10T10:43:23.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.565+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc0c801b020 con 0x7fc0d0162e30 2024-09-10T10:43:23.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.565+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc0c80133e0 con 0x7fc0d0162e30 2024-09-10T10:43:23.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.566+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d006d740 con 0x7fc0d0162e30 2024-09-10T10:43:23.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.570+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc0c8022030 con 0x7fc0d0162e30 2024-09-10T10:43:23.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.642+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "897214ac-815c-46c9-bd71-6557ec7b9859", "id": 0} v 0) -- 0x7fc0d008e640 con 0x7fc0d0162e30 2024-09-10T10:43:23.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.647+0000 7fc0c6000640 1 -- 172.21.15.138:0/1148651841 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "897214ac-815c-46c9-bd71-6557ec7b9859", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc0c8010040 con 0x7fc0d0162e30 2024-09-10T10:43:23.648 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-09-10T10:43:23.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 msgr2=0x7fc0d00b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800efc0 tx=0x7fc0c800c490 comp rx=0 tx=0).stop 2024-09-10T10:43:23.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 shutdown_connections 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d0162e30 0x7fc0d00b58a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0d0162340 0x7fc0d015cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 --2- 172.21.15.138:0/1148651841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0d00cf280 0x7fc0d00cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 >> 172.21.15.138:0/1148651841 conn(0x7fc0d008bfc0 msgr2=0x7fc0d01616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 shutdown_connections 2024-09-10T10:43:23.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.648+0000 7fc0d4c00640 1 -- 172.21.15.138:0/1148651841 wait complete. 2024-09-10T10:43:23.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 300c9348-8f79-4a46-a010-b71ef4e61396 1 2024-09-10T10:43:23.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.815+0000 7f4df8a00640 1 Processor -- start 2024-09-10T10:43:23.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.816+0000 7f4df8a00640 1 -- start start 2024-09-10T10:43:23.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4df40d1100 0x7f4df40cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4df40cf3c0 0x7f4df40cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df8a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4df40b81d0 con 0x7f4df40b4730 2024-09-10T10:43:23.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df8a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4df40d20e0 con 0x7f4df40cf3c0 2024-09-10T10:43:23.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df8a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4df40d23c0 con 0x7f4df40d1100 2024-09-10T10:43:23.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4df40d1100 0x7f4df40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4df40d1100 0x7f4df40cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45362/0 (socket says 172.21.15.138:45362) 2024-09-10T10:43:23.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 -- 172.21.15.138:0/223253269 learned_addr learned my addr 172.21.15.138:0/223253269 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:23.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df3e00640 1 --2- 172.21.15.138:0/223253269 >> v2:172.21.15.28:3300/0 conn(0x7f4df40cf3c0 0x7f4df40cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 -- 172.21.15.138:0/223253269 >> v2:172.21.15.28:3300/0 conn(0x7f4df40cf3c0 msgr2=0x7f4df40cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 --2- 172.21.15.138:0/223253269 >> v2:172.21.15.28:3300/0 conn(0x7f4df40cf3c0 0x7f4df40cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 -- 172.21.15.138:0/223253269 >> v1:172.21.15.28:6789/0 conn(0x7f4df40b4730 legacy=0x7f4df40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.817+0000 7f4df2a00640 1 -- 172.21.15.138:0/223253269 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df40d4c20 con 0x7f4df40d1100 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.818+0000 7f4df2a00640 1 --2- 172.21.15.138:0/223253269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40d1100 0x7f4df40cee80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de002a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8ab69e518bf95cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df2000640 1 -- 172.21.15.138:0/223253269 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0037070 con 0x7f4df40d1100 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df2000640 1 -- 172.21.15.138:0/223253269 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de0032440 con 0x7f4df40d1100 2024-09-10T10:43:23.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df2000640 1 -- 172.21.15.138:0/223253269 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0032810 con 0x7f4df40d1100 2024-09-10T10:43:23.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 -- 172.21.15.138:0/223253269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40d1100 msgr2=0x7f4df40cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 --2- 172.21.15.138:0/223253269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40d1100 0x7f4df40cee80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009870 tx=0x7f4de002a2a0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 -- 172.21.15.138:0/223253269 shutdown_connections 2024-09-10T10:43:23.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 --2- 172.21.15.138:0/223253269 >> v2:172.21.15.28:3300/0 conn(0x7f4df40cf3c0 0x7f4df40cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 --2- 172.21.15.138:0/223253269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40d1100 0x7f4df40cee80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 -- 172.21.15.138:0/223253269 >> 172.21.15.138:0/223253269 conn(0x7f4df408bfc0 msgr2=0x7f4df40d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:23.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 -- 172.21.15.138:0/223253269 shutdown_connections 2024-09-10T10:43:23.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.819+0000 7f4df8a00640 1 -- 172.21.15.138:0/223253269 wait complete. 2024-09-10T10:43:23.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.821+0000 7f4df8a00640 1 Processor -- start 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.821+0000 7f4df8a00640 1 -- start start 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 0x7f4df41519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df40d1100 0x7f4df40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4df40b5060 con 0x7f4df40d1100 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4df40b4ee0 con 0x7f4df40cf3c0 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4df40b4b10 con 0x7f4df40b4730 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 0x7f4df41519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45378/0 (socket says 172.21.15.138:45378) 2024-09-10T10:43:23.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df2a00640 1 -- 172.21.15.138:0/3645968107 learned_addr learned my addr 172.21.15.138:0/3645968107 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df3e00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df40d1100 0x7f4df40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df2a00640 1 -- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 msgr2=0x7f4df41519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.822+0000 7f4df2a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 0x7f4df41519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df2a00640 1 -- 172.21.15.138:0/3645968107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df40d1100 msgr2=0x7f4df40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df2a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df40d1100 0x7f4df40b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df2a00640 1 -- 172.21.15.138:0/3645968107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df4151f30 con 0x7f4df40cf3c0 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df3400640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 0x7f4df41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df2a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000c00 tx=0x7f4de00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:23.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0041020 con 0x7f4df40cf3c0 2024-09-10T10:43:23.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4de00306c0 con 0x7f4df40cf3c0 2024-09-10T10:43:23.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0037040 con 0x7f4df40cf3c0 2024-09-10T10:43:23.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df40ce4d0 con 0x7f4df40cf3c0 2024-09-10T10:43:23.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df41ea420 con 0x7f4df40cf3c0 2024-09-10T10:43:23.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.823+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4de0044020 con 0x7f4df40cf3c0 2024-09-10T10:43:23.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.824+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4de0046560 con 0x7f4df40cf3c0 2024-09-10T10:43:23.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.824+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4df40d1570 con 0x7f4df40cf3c0 2024-09-10T10:43:23.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.827+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4de004b030 con 0x7f4df40cf3c0 2024-09-10T10:43:23.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.899+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "300c9348-8f79-4a46-a010-b71ef4e61396", "id": 1} v 0) -- 0x7f4df40d1920 con 0x7f4df40cf3c0 2024-09-10T10:43:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.903+0000 7f4deb400640 1 -- 172.21.15.138:0/3645968107 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "300c9348-8f79-4a46-a010-b71ef4e61396", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4df4151f30 con 0x7f4df40cf3c0 2024-09-10T10:43:23.904 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-09-10T10:43:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 msgr2=0x7f4df40cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4de0000c00 tx=0x7f4de00046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 shutdown_connections 2024-09-10T10:43:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df40d1100 0x7f4df40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4df40cf3c0 0x7f4df40cdf90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 --2- 172.21.15.138:0/3645968107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4df40b4730 0x7f4df41519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:23.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 >> 172.21.15.138:0/3645968107 conn(0x7f4df408bfc0 msgr2=0x7f4df406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:23.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 shutdown_connections 2024-09-10T10:43:23.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:23.904+0000 7f4df8a00640 1 -- 172.21.15.138:0/3645968107 wait complete. 2024-09-10T10:43:23.920 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new c87c3518-ad7b-47e2-8405-81c0509fd89d 2 2024-09-10T10:43:24.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.069+0000 7f505a600640 1 Processor -- start 2024-09-10T10:43:24.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.069+0000 7f505a600640 1 -- start start 2024-09-10T10:43:24.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f50540b4730 0x7f50540d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 0x7f50540b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f50540d20e0 con 0x7f50540b4730 2024-09-10T10:43:24.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f50540d23c0 con 0x7f50540b71c0 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f505a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50540d4c20 con 0x7f50540d0f30 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f5053e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.070+0000 7f5059c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 0x7f50540b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60398/0 (socket says 172.21.15.138:60398) 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 -- 172.21.15.138:0/2345408441 learned_addr learned my addr 172.21.15.138:0/2345408441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5059200640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.28:3300/0 conn(0x7f50540b4730 0x7f50540d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 -- 172.21.15.138:0/2345408441 >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 msgr2=0x7f50540b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 0x7f50540b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 -- 172.21.15.138:0/2345408441 >> v2:172.21.15.28:3300/0 conn(0x7f50540b4730 msgr2=0x7f50540d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.28:3300/0 conn(0x7f50540b4730 0x7f50540d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5053e00640 1 -- 172.21.15.138:0/2345408441 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50540d5140 con 0x7f50540d0f30 2024-09-10T10:43:24.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.071+0000 7f5059c00640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 0x7f50540b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:24.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.072+0000 7f5053e00640 1 --2- 172.21.15.138:0/2345408441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f505408e340 tx=0x7f504002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb9819e10f401d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.072+0000 7f5053400640 1 -- 172.21.15.138:0/2345408441 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040037070 con 0x7f50540d0f30 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.072+0000 7f5053400640 1 -- 172.21.15.138:0/2345408441 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5040032440 con 0x7f50540d0f30 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.072+0000 7f5053400640 1 -- 172.21.15.138:0/2345408441 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5040032810 con 0x7f50540d0f30 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 -- 172.21.15.138:0/2345408441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 msgr2=0x7f50540d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 --2- 172.21.15.138:0/2345408441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f505408e340 tx=0x7f504002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 -- 172.21.15.138:0/2345408441 shutdown_connections 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.138:3300/0 conn(0x7f50540b71c0 0x7f50540b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 --2- 172.21.15.138:0/2345408441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540d0f30 0x7f50540d13a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 --2- 172.21.15.138:0/2345408441 >> v2:172.21.15.28:3300/0 conn(0x7f50540b4730 0x7f50540d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 -- 172.21.15.138:0/2345408441 >> 172.21.15.138:0/2345408441 conn(0x7f505408bfc0 msgr2=0x7f50540d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 -- 172.21.15.138:0/2345408441 shutdown_connections 2024-09-10T10:43:24.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.073+0000 7f505a600640 1 -- 172.21.15.138:0/2345408441 wait complete. 2024-09-10T10:43:24.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.075+0000 7f505a600640 1 Processor -- start 2024-09-10T10:43:24.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.075+0000 7f505a600640 1 -- start start 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 0x7f50540cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50540d0f30 0x7f50540cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50540b4c90 con 0x7f50540d0f30 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50540b4b10 con 0x7f50540b71c0 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f505a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50540b7b70 con 0x7f50540b4730 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5053e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 0x7f50540cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5053e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45396/0 (socket says 172.21.15.138:45396) 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5053e00640 1 -- 172.21.15.138:0/873329516 learned_addr learned my addr 172.21.15.138:0/873329516 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059c00640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50540d0f30 0x7f50540cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 -- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 msgr2=0x7f505415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 -- 172.21.15.138:0/873329516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50540d0f30 msgr2=0x7f50540cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50540d0f30 0x7f50540cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5059200640 1 -- 172.21.15.138:0/873329516 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505415dc70 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.076+0000 7f5053e00640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5059200640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 0x7f50540cae80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f504800e8a0 tx=0x7f504800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5048017070 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f504800ee90 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505415df30 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504800cb40 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f505415ef50 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f504801a070 con 0x7f50540b4730 2024-09-10T10:43:24.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.077+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f5048006020 con 0x7f50540b4730 2024-09-10T10:43:24.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.078+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50540b9120 con 0x7f50540b4730 2024-09-10T10:43:24.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.081+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f504800cd40 con 0x7f50540b4730 2024-09-10T10:43:24.140 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.139+0000 7f408593b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:43:24.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.153+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c87c3518-ad7b-47e2-8405-81c0509fd89d", "id": 2} v 0) -- 0x7f50540d4150 con 0x7f50540b4730 2024-09-10T10:43:24.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.158+0000 7f5050c00640 1 -- 172.21.15.138:0/873329516 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c87c3518-ad7b-47e2-8405-81c0509fd89d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f50480065a0 con 0x7f50540b4730 2024-09-10T10:43:24.160 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 msgr2=0x7f50540cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 0x7f50540cae80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f504800e8a0 tx=0x7f504800eca0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 shutdown_connections 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50540d0f30 0x7f50540cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50540b71c0 0x7f505415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 --2- 172.21.15.138:0/873329516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50540b4730 0x7f50540cae80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.159+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 >> 172.21.15.138:0/873329516 conn(0x7f505408bfc0 msgr2=0x7f505406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.160+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 shutdown_connections 2024-09-10T10:43:24.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.160+0000 7f505a600640 1 -- 172.21.15.138:0/873329516 wait complete. 2024-09-10T10:43:24.176 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new ea9fa499-68d2-496b-8b75-8d0e15cc4be3 3 2024-09-10T10:43:24.209 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.207+0000 7f408593b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:43:24.221 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:24.219+0000 7f9f9f520040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T10:43:24.286 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:24.284+0000 7f9f9f520040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T10:43:24.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.326+0000 7fe613000640 1 Processor -- start 2024-09-10T10:43:24.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.326+0000 7fe613000640 1 -- start start 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.327+0000 7fe613000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe60c15e550 0x7fe60c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.327+0000 7fe613000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60c06d270 con 0x7fe60c0d2920 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.327+0000 7fe613000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60c0cee50 con 0x7fe60c0b3400 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.327+0000 7fe613000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe60c0d4a50 con 0x7fe60c15e550 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.327+0000 7fe611c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe60c15e550 0x7fe60c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe60c15e550 0x7fe60c06cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45398/0 (socket says 172.21.15.138:45398) 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611c00640 1 -- 172.21.15.138:0/1139615478 learned_addr learned my addr 172.21.15.138:0/1139615478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611200640 1 --1- 172.21.15.138:0/1139615478 >> v1:172.21.15.138:6789/0 conn(0x7fe60c0b3400 0x7fe60c0d1f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47288/0 (socket says 172.21.15.138:47288) 2024-09-10T10:43:24.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611c00640 1 -- 172.21.15.138:0/1139615478 >> v1:172.21.15.138:6789/0 conn(0x7fe60c0b3400 legacy=0x7fe60c0d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611c00640 1 -- 172.21.15.138:0/1139615478 >> v1:172.21.15.28:6789/0 conn(0x7fe60c0d2920 legacy=0x7fe60c0d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.328+0000 7fe611c00640 1 -- 172.21.15.138:0/1139615478 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c0d4ca0 con 0x7fe60c15e550 2024-09-10T10:43:24.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.329+0000 7fe611c00640 1 --2- 172.21.15.138:0/1139615478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c15e550 0x7fe60c06cd30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b037c59d7a4bfd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.329+0000 7fe60be00640 1 -- 172.21.15.138:0/1139615478 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600037070 con 0x7fe60c15e550 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.329+0000 7fe60be00640 1 -- 172.21.15.138:0/1139615478 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe600032440 con 0x7fe60c15e550 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.329+0000 7fe60be00640 1 -- 172.21.15.138:0/1139615478 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600032810 con 0x7fe60c15e550 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 -- 172.21.15.138:0/1139615478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c15e550 msgr2=0x7fe60c06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 --2- 172.21.15.138:0/1139615478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c15e550 0x7fe60c06cd30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 -- 172.21.15.138:0/1139615478 shutdown_connections 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 --2- 172.21.15.138:0/1139615478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c15e550 0x7fe60c06cd30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 -- 172.21.15.138:0/1139615478 >> 172.21.15.138:0/1139615478 conn(0x7fe60c08bfc0 msgr2=0x7fe60c0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 -- 172.21.15.138:0/1139615478 shutdown_connections 2024-09-10T10:43:24.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.330+0000 7fe613000640 1 -- 172.21.15.138:0/1139615478 wait complete. 2024-09-10T10:43:24.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.331+0000 7fe613000640 1 Processor -- start 2024-09-10T10:43:24.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 -- start start 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 0x7fe60c0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe60c15e550 0x7fe60c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0cf610 con 0x7fe60c15e550 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0cf180 con 0x7fe60c0d2920 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.332+0000 7fe613000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe60c0cf790 con 0x7fe60c0b3400 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45402/0 (socket says 172.21.15.138:45402) 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 -- 172.21.15.138:0/4078081906 learned_addr learned my addr 172.21.15.138:0/4078081906 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611c00640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 0x7fe60c0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 -- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 msgr2=0x7fe60c0b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 0x7fe60c0b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 -- 172.21.15.138:0/4078081906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe60c15e550 msgr2=0x7fe60c0b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe60c15e550 0x7fe60c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 -- 172.21.15.138:0/4078081906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c15cfc0 con 0x7fe60c0d2920 2024-09-10T10:43:24.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611c00640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 0x7fe60c0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:24.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.333+0000 7fe611200640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c500 tx=0x7fe5f800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8018070 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f8007500 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80049e0 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c0b7750 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c0b7d10 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe5f8004b80 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.334+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe5f80128d0 con 0x7fe60c0d2920 2024-09-10T10:43:24.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.335+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c0d18f0 con 0x7fe60c0d2920 2024-09-10T10:43:24.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.337+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe5f8012440 con 0x7fe60c0d2920 2024-09-10T10:43:24.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.409+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ea9fa499-68d2-496b-8b75-8d0e15cc4be3", "id": 3} v 0) -- 0x7fe60c0b2db0 con 0x7fe60c0d2920 2024-09-10T10:43:24.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.413+0000 7fe609600640 1 -- 172.21.15.138:0/4078081906 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ea9fa499-68d2-496b-8b75-8d0e15cc4be3", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe5f80126e0 con 0x7fe60c0d2920 2024-09-10T10:43:24.414 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-09-10T10:43:24.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 msgr2=0x7fe60c15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c500 tx=0x7fe5f800c9c0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 shutdown_connections 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe60c15e550 0x7fe60c0b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe60c0d2920 0x7fe60c15ca80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 --2- 172.21.15.138:0/4078081906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe60c0b3400 0x7fe60c0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 >> 172.21.15.138:0/4078081906 conn(0x7fe60c08bfc0 msgr2=0x7fe60c15d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 shutdown_connections 2024-09-10T10:43:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.414+0000 7fe613000640 1 -- 172.21.15.138:0/4078081906 wait complete. 2024-09-10T10:43:24.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 9d933671-197e-4151-ad30-dacffb5143bf 4 2024-09-10T10:43:24.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.619+0000 7f7d09c00640 1 Processor -- start 2024-09-10T10:43:24.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.620+0000 7f7d09c00640 1 -- start start 2024-09-10T10:43:24.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d09c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 0x7f7d0406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d09c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d040cf280 0x7f7d040d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d09c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d0415ff70 con 0x7f7d04162340 2024-09-10T10:43:24.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d09c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0406de90 con 0x7f7d04162e30 2024-09-10T10:43:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d09c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7d04163270 con 0x7f7d040cf280 2024-09-10T10:43:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d03400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d040cf280 0x7f7d040d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7cfbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 0x7f7d0406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d02a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7d04162340 0x7f7d04162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.138:37212/0 (socket says 172.21.15.138:37212) 2024-09-10T10:43:24.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d02a00640 1 -- 172.21.15.138:0/1701087289 learned_addr learned my addr 172.21.15.138:0/1701087289 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d03400640 1 -- 172.21.15.138:0/1701087289 >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 msgr2=0x7f7d0406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.621+0000 7f7d03400640 1 --2- 172.21.15.138:0/1701087289 >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 0x7f7d0406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.622+0000 7f7d03400640 1 -- 172.21.15.138:0/1701087289 >> v1:172.21.15.28:6789/0 conn(0x7f7d04162340 legacy=0x7f7d04162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.622+0000 7f7d02000640 1 -- 172.21.15.138:0/1701087289 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3651290552 0 0) 0x7f7d0415ff70 con 0x7f7d04162340 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.622+0000 7f7d03400640 1 -- 172.21.15.138:0/1701087289 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d040d4d10 con 0x7f7d040cf280 2024-09-10T10:43:24.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.622+0000 7f7cfbe00640 1 --2- 172.21.15.138:0/1701087289 >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 0x7f7d0406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:24.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.623+0000 7f7d03400640 1 --2- 172.21.15.138:0/1701087289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d040cf280 0x7f7d040d2990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406c3d0 tx=0x7f7cfc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d604b7ca478e40c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.623+0000 7f7d02000640 1 -- 172.21.15.138:0/1701087289 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc007740 con 0x7f7d040cf280 2024-09-10T10:43:24.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.623+0000 7f7d02000640 1 -- 172.21.15.138:0/1701087289 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0078e0 con 0x7f7d040cf280 2024-09-10T10:43:24.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.623+0000 7f7d02000640 1 -- 172.21.15.138:0/1701087289 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc007cb0 con 0x7f7d040cf280 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 -- 172.21.15.138:0/1701087289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d040cf280 msgr2=0x7f7d040d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1701087289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d040cf280 0x7f7d040d2990 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406c3d0 tx=0x7f7cfc02a5f0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 -- 172.21.15.138:0/1701087289 shutdown_connections 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1701087289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d040cf280 0x7f7d040d2990 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1701087289 >> v2:172.21.15.28:3300/0 conn(0x7f7d04162e30 0x7f7d0406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 -- 172.21.15.138:0/1701087289 >> 172.21.15.138:0/1701087289 conn(0x7f7d0408bfc0 msgr2=0x7f7d040d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 -- 172.21.15.138:0/1701087289 shutdown_connections 2024-09-10T10:43:24.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.624+0000 7f7d09c00640 1 -- 172.21.15.138:0/1701087289 wait complete. 2024-09-10T10:43:24.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.626+0000 7f7d09c00640 1 Processor -- start 2024-09-10T10:43:24.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.626+0000 7f7d09c00640 1 -- start start 2024-09-10T10:43:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d040cf280 0x7f7d0415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d04162340 0x7f7d04159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7d040cf6f0 con 0x7f7d040cf280 2024-09-10T10:43:24.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d040d2470 con 0x7f7d04162340 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d09c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d040d52d0 con 0x7f7d04162e30 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7cfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d04162340 0x7f7d04159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60434/0 (socket says 172.21.15.138:60434) 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 -- 172.21.15.138:0/1664433096 learned_addr learned my addr 172.21.15.138:0/1664433096 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d02a00640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d040cf280 0x7f7d0415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 -- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d04162340 msgr2=0x7f7d04159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d04162340 0x7f7d04159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 -- 172.21.15.138:0/1664433096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d040cf280 msgr2=0x7f7d0415e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d040cf280 0x7f7d0415e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.627+0000 7f7d03400640 1 -- 172.21.15.138:0/1664433096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041f8eb0 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7d03400640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009e50 tx=0x7f7cfc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc040070 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cfc031bf0 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc009450 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041f8bd0 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.628+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0415d970 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.629+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cfc039040 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.629+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f7cfc045970 con 0x7f7d04162e30 2024-09-10T10:43:24.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.629+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0406d740 con 0x7f7d04162e30 2024-09-10T10:43:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.632+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7cfc045dc0 con 0x7f7d04162e30 2024-09-10T10:43:24.672 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.671+0000 7f408593b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:43:24.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.703+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9d933671-197e-4151-ad30-dacffb5143bf", "id": 4} v 0) -- 0x7f7d0414f0a0 con 0x7f7d04162e30 2024-09-10T10:43:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.710+0000 7f7cfaa00640 1 -- 172.21.15.138:0/1664433096 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d933671-197e-4151-ad30-dacffb5143bf", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f7cfc007840 con 0x7f7d04162e30 2024-09-10T10:43:24.711 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 msgr2=0x7f7d04159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009e50 tx=0x7f7cfc004a40 comp rx=0 tx=0).stop 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 shutdown_connections 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d04162e30 0x7f7d04159cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d04162340 0x7f7d04159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 --2- 172.21.15.138:0/1664433096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d040cf280 0x7f7d0415e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 >> 172.21.15.138:0/1664433096 conn(0x7f7d0408bfc0 msgr2=0x7f7d040d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 shutdown_connections 2024-09-10T10:43:24.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.711+0000 7f7d09c00640 1 -- 172.21.15.138:0/1664433096 wait complete. 2024-09-10T10:43:24.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 53157853-f5c4-47c2-b63c-09dd6c2a964d 5 2024-09-10T10:43:24.745 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:24.743+0000 7f9f9f520040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T10:43:24.776 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.775+0000 7f408593b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:43:24.847 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:24.845+0000 7f9f9f520040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T10:43:24.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.875+0000 7f1b60a00640 1 Processor -- start 2024-09-10T10:43:24.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.875+0000 7f1b60a00640 1 -- start start 2024-09-10T10:43:24.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b60a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b5c15e550 0x7f1b5c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b60a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0b3400 0x7f1b5c0d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b60a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b5c06d270 con 0x7f1b5c0d2920 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b60a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5c0d4a50 con 0x7f1b5c15e550 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b60a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0cf180 con 0x7f1b5c0b3400 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b5b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b5c15e550 0x7f1b5c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b5aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0b3400 0x7f1b5c0d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b5b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b5c15e550 0x7f1b5c06cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45436/0 (socket says 172.21.15.138:45436) 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.876+0000 7f1b5b400640 1 -- 172.21.15.138:0/2191553480 learned_addr learned my addr 172.21.15.138:0/2191553480 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.877+0000 7f1b5b400640 1 -- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0b3400 msgr2=0x7f1b5c0d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.877+0000 7f1b5b400640 1 --2- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0b3400 0x7f1b5c0d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.877+0000 7f1b5b400640 1 -- 172.21.15.138:0/2191553480 >> v1:172.21.15.28:6789/0 conn(0x7f1b5c0d2920 legacy=0x7f1b5c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.877+0000 7f1b5b400640 1 -- 172.21.15.138:0/2191553480 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c0cee50 con 0x7f1b5c15e550 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.878+0000 7f1b5b400640 1 --2- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c15e550 0x7f1b5c06cd30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009870 tx=0x7f1b4c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a021cd4995d4a531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.878+0000 7f1b5a000640 1 -- 172.21.15.138:0/2191553480 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c037070 con 0x7f1b5c15e550 2024-09-10T10:43:24.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.878+0000 7f1b5a000640 1 -- 172.21.15.138:0/2191553480 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4c032440 con 0x7f1b5c15e550 2024-09-10T10:43:24.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.878+0000 7f1b5a000640 1 -- 172.21.15.138:0/2191553480 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c032810 con 0x7f1b5c15e550 2024-09-10T10:43:24.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 -- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c15e550 msgr2=0x7f1b5c06cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 --2- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c15e550 0x7f1b5c06cd30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009870 tx=0x7f1b4c02a130 comp rx=0 tx=0).stop 2024-09-10T10:43:24.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 -- 172.21.15.138:0/2191553480 shutdown_connections 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 --2- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0b3400 0x7f1b5c0d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 --2- 172.21.15.138:0/2191553480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c15e550 0x7f1b5c06cd30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 -- 172.21.15.138:0/2191553480 >> 172.21.15.138:0/2191553480 conn(0x7f1b5c08bfc0 msgr2=0x7f1b5c0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 -- 172.21.15.138:0/2191553480 shutdown_connections 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.879+0000 7f1b60a00640 1 -- 172.21.15.138:0/2191553480 wait complete. 2024-09-10T10:43:24.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.880+0000 7f1b60a00640 1 Processor -- start 2024-09-10T10:43:24.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.880+0000 7f1b60a00640 1 -- start start 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c0b3400 0x7f1b5c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b5c15e550 0x7f1b5c0b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d2450 con 0x7f1b5c15e550 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d4f80 con 0x7f1b5c0b3400 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.881+0000 7f1b60a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0d2db0 con 0x7f1b5c0d2920 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c0b3400 0x7f1b5c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60450/0 (socket says 172.21.15.138:60450) 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 -- 172.21.15.138:0/3928637464 learned_addr learned my addr 172.21.15.138:0/3928637464 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5be00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b5c15e550 0x7f1b5c0b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 -- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c0b3400 msgr2=0x7f1b5c158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c0b3400 0x7f1b5c158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 -- 172.21.15.138:0/3928637464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b5c15e550 msgr2=0x7f1b5c0b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b5c15e550 0x7f1b5c0b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 -- 172.21.15.138:0/3928637464 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c1e5d80 con 0x7f1b5c0d2920 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.882+0000 7f1b5aa00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e980 tx=0x7f1b4800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48017070 con 0x7f1b5c0d2920 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4800ca00 con 0x7f1b5c0d2920 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4800cdd0 con 0x7f1b5c0d2920 2024-09-10T10:43:24.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c1e6020 con 0x7f1b5c0d2920 2024-09-10T10:43:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1e3890 con 0x7f1b5c0d2920 2024-09-10T10:43:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b4801a070 con 0x7f1b5c0d2920 2024-09-10T10:43:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.883+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f1b48006610 con 0x7f1b5c0d2920 2024-09-10T10:43:24.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.884+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c1e36a0 con 0x7f1b5c0d2920 2024-09-10T10:43:24.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.886+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1b48006810 con 0x7f1b5c0d2920 2024-09-10T10:43:24.906 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.904+0000 7f408593b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:43:24.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.959+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "53157853-f5c4-47c2-b63c-09dd6c2a964d", "id": 5} v 0) -- 0x7f1b5c06ccb0 con 0x7f1b5c0d2920 2024-09-10T10:43:24.962 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:24.961+0000 7f408593b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:43:24.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.965+0000 7f1b53400640 1 -- 172.21.15.138:0/3928637464 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "53157853-f5c4-47c2-b63c-09dd6c2a964d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f1b48005ac0 con 0x7f1b5c0d2920 2024-09-10T10:43:24.966 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-09-10T10:43:24.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 msgr2=0x7f1b5c0b9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800e980 tx=0x7f1b4800ee40 comp rx=0 tx=0).stop 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 shutdown_connections 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b5c15e550 0x7f1b5c0b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b5c0d2920 0x7f1b5c0b9160 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 --2- 172.21.15.138:0/3928637464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b5c0b3400 0x7f1b5c158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 >> 172.21.15.138:0/3928637464 conn(0x7f1b5c08bfc0 msgr2=0x7f1b5c0b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 shutdown_connections 2024-09-10T10:43:24.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:24.966+0000 7f1b60a00640 1 -- 172.21.15.138:0/3928637464 wait complete. 2024-09-10T10:43:24.975 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:24.973+0000 7f9f9f520040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T10:43:24.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 7a541dc7-d8ae-4619-991d-49c2a3caffd4 6 2024-09-10T10:43:25.032 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.030+0000 7f9f9f520040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T10:43:25.101 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.099+0000 7f408593b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:43:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.133+0000 7fd33ee00640 1 Processor -- start 2024-09-10T10:43:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.133+0000 7fd33ee00640 1 -- start start 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd33815e550 0x7fd33806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3380b3400 0x7fd3380d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd33806d270 con 0x7fd3380d2920 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd3380d4a50 con 0x7fd33815e550 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd3380cf180 con 0x7fd3380b3400 2024-09-10T10:43:25.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3380b3400 0x7fd3380d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd3380b3400 0x7fd3380d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45462/0 (socket says 172.21.15.138:45462) 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33d000640 1 -- 172.21.15.138:0/2212202309 learned_addr learned my addr 172.21.15.138:0/2212202309 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.134+0000 7fd33da00640 1 --2- 172.21.15.138:0/2212202309 >> v2:172.21.15.28:3300/0 conn(0x7fd33815e550 0x7fd33806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.135+0000 7fd33d000640 1 -- 172.21.15.138:0/2212202309 >> v2:172.21.15.28:3300/0 conn(0x7fd33815e550 msgr2=0x7fd33806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.135+0000 7fd33d000640 1 --2- 172.21.15.138:0/2212202309 >> v2:172.21.15.28:3300/0 conn(0x7fd33815e550 0x7fd33806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.135+0000 7fd33d000640 1 -- 172.21.15.138:0/2212202309 >> v1:172.21.15.28:6789/0 conn(0x7fd3380d2920 legacy=0x7fd3380d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.135+0000 7fd33d000640 1 -- 172.21.15.138:0/2212202309 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3380cee50 con 0x7fd3380b3400 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.136+0000 7fd33d000640 1 --2- 172.21.15.138:0/2212202309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380b3400 0x7fd3380d1f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd3380d1e40 tx=0x7fd32402a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96473369e019052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.136+0000 7fd337e00640 1 -- 172.21.15.138:0/2212202309 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324037070 con 0x7fd3380b3400 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.136+0000 7fd337e00640 1 -- 172.21.15.138:0/2212202309 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd324032440 con 0x7fd3380b3400 2024-09-10T10:43:25.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.136+0000 7fd337e00640 1 -- 172.21.15.138:0/2212202309 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324032810 con 0x7fd3380b3400 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 -- 172.21.15.138:0/2212202309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380b3400 msgr2=0x7fd3380d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 --2- 172.21.15.138:0/2212202309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380b3400 0x7fd3380d1f10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd3380d1e40 tx=0x7fd32402a160 comp rx=0 tx=0).stop 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 -- 172.21.15.138:0/2212202309 shutdown_connections 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 --2- 172.21.15.138:0/2212202309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380b3400 0x7fd3380d1f10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 --2- 172.21.15.138:0/2212202309 >> v2:172.21.15.28:3300/0 conn(0x7fd33815e550 0x7fd33806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 -- 172.21.15.138:0/2212202309 >> 172.21.15.138:0/2212202309 conn(0x7fd33808bfc0 msgr2=0x7fd3380d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 -- 172.21.15.138:0/2212202309 shutdown_connections 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.137+0000 7fd33ee00640 1 -- 172.21.15.138:0/2212202309 wait complete. 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.138+0000 7fd33ee00640 1 Processor -- start 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.138+0000 7fd33ee00640 1 -- start start 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 0x7fd3380cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380d2920 0x7fd33815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd3380d2450 con 0x7fd3380b3400 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3380d4f80 con 0x7fd3380d2920 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.139+0000 7fd33ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3380cf610 con 0x7fd33815e550 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380d2920 0x7fd33815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 0x7fd3380cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60466/0 (socket says 172.21.15.138:60466) 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 -- 172.21.15.138:0/3267645089 learned_addr learned my addr 172.21.15.138:0/3267645089 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 -- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380d2920 msgr2=0x7fd33815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380d2920 0x7fd33815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 -- 172.21.15.138:0/3267645089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 msgr2=0x7fd3380cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 0x7fd3380cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 -- 172.21.15.138:0/3267645089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3380cdb80 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33da00640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 0x7fd3380cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.140+0000 7fd33e400640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd33000c7d0 tx=0x7fd33000cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330003b70 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd330003d30 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33000aa90 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3380cd000 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3380cd5c0 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd33000ac30 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.141+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fd3300056c0 con 0x7fd33815e550 2024-09-10T10:43:25.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.142+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3380d1f10 con 0x7fd33815e550 2024-09-10T10:43:25.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.144+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd3300050d0 con 0x7fd33815e550 2024-09-10T10:43:25.166 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.164+0000 7f9f9f520040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T10:43:25.168 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.167+0000 7f408593b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:43:25.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.216+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7a541dc7-d8ae-4619-991d-49c2a3caffd4", "id": 6} v 0) -- 0x7fd3380d18f0 con 0x7fd33815e550 2024-09-10T10:43:25.223 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.222+0000 7f408593b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:43:25.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.224+0000 7fd335600640 1 -- 172.21.15.138:0/3267645089 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a541dc7-d8ae-4619-991d-49c2a3caffd4", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fd3300052b0 con 0x7fd33815e550 2024-09-10T10:43:25.226 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 msgr2=0x7fd33815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd33000c7d0 tx=0x7fd33000cc90 comp rx=0 tx=0).stop 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 shutdown_connections 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd33815e550 0x7fd33815c5c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3380d2920 0x7fd33815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.225+0000 7fd33ee00640 1 --2- 172.21.15.138:0/3267645089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b3400 0x7fd3380cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.226+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 >> 172.21.15.138:0/3267645089 conn(0x7fd33808bfc0 msgr2=0x7fd3380b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.226+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 shutdown_connections 2024-09-10T10:43:25.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.226+0000 7fd33ee00640 1 -- 172.21.15.138:0/3267645089 wait complete. 2024-09-10T10:43:25.232 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.229+0000 7f9f9f520040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T10:43:25.241 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new bb5c2ad5-c112-419b-8ad5-e415abfc4b4c 7 2024-09-10T10:43:25.287 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.285+0000 7f9f9f520040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T10:43:25.361 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.359+0000 7f408593b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:43:25.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.392+0000 7f8076800640 1 Processor -- start 2024-09-10T10:43:25.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.392+0000 7f8076800640 1 -- start start 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8076800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80700b3400 0x7f80700d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8076800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8076800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80700cee50 con 0x7f807015e550 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8076800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f80700d32b0 con 0x7f80700b3400 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8076800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80700d4a50 con 0x7f80700d2920 2024-09-10T10:43:25.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f807015e550 0x7f807006cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:47302/0 (socket says 172.21.15.138:47302) 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075400640 1 -- 172.21.15.138:0/370139821 learned_addr learned my addr 172.21.15.138:0/370139821 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60474/0 (socket says 172.21.15.138:60474) 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8074a00640 1 --2- 172.21.15.138:0/370139821 >> v2:172.21.15.28:3300/0 conn(0x7f80700b3400 0x7f80700d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 -- 172.21.15.138:0/370139821 >> v2:172.21.15.28:3300/0 conn(0x7f80700b3400 msgr2=0x7f80700d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 --2- 172.21.15.138:0/370139821 >> v2:172.21.15.28:3300/0 conn(0x7f80700b3400 0x7f80700d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 -- 172.21.15.138:0/370139821 >> v1:172.21.15.138:6789/0 conn(0x7f807015e550 legacy=0x7f807006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f8075e00640 1 -- 172.21.15.138:0/370139821 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80700d4ca0 con 0x7f80700d2920 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.393+0000 7f806fe00640 1 -- 172.21.15.138:0/370139821 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306765646 0 0) 0x7f80700cee50 con 0x7f807015e550 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.394+0000 7f8075e00640 1 --2- 172.21.15.138:0/370139821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8068009d90 tx=0x7f806802a530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7552c3a8c40dca71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f806fe00640 1 -- 172.21.15.138:0/370139821 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068007740 con 0x7f80700d2920 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f806fe00640 1 -- 172.21.15.138:0/370139821 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80680078e0 con 0x7f80700d2920 2024-09-10T10:43:25.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f806fe00640 1 -- 172.21.15.138:0/370139821 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8068007cb0 con 0x7f80700d2920 2024-09-10T10:43:25.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 -- 172.21.15.138:0/370139821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 msgr2=0x7f80700d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 --2- 172.21.15.138:0/370139821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8068009d90 tx=0x7f806802a530 comp rx=0 tx=0).stop 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 -- 172.21.15.138:0/370139821 shutdown_connections 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 --2- 172.21.15.138:0/370139821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f80700d2d70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 --2- 172.21.15.138:0/370139821 >> v2:172.21.15.28:3300/0 conn(0x7f80700b3400 0x7f80700d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 -- 172.21.15.138:0/370139821 >> 172.21.15.138:0/370139821 conn(0x7f807008bfc0 msgr2=0x7f80700d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 -- 172.21.15.138:0/370139821 shutdown_connections 2024-09-10T10:43:25.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.395+0000 7f8076800640 1 -- 172.21.15.138:0/370139821 wait complete. 2024-09-10T10:43:25.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.397+0000 7f8076800640 1 Processor -- start 2024-09-10T10:43:25.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.397+0000 7f8076800640 1 -- start start 2024-09-10T10:43:25.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 0x7f80701f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 0x7f80700cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80700d2450 con 0x7f80700b3400 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80700cf180 con 0x7f807015e550 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8076800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f80700cf610 con 0x7f80700d2920 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8074a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8075e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 0x7f80700cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8074a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:60486/0 (socket says 172.21.15.138:60486) 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8074a00640 1 -- 172.21.15.138:0/317534887 learned_addr learned my addr 172.21.15.138:0/317534887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-10T10:43:25.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.398+0000 7f8075400640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 0x7f80701f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 -- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 msgr2=0x7f80700cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 0x7f80700cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 -- 172.21.15.138:0/317534887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 msgr2=0x7f80701f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 0x7f80701f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 -- 172.21.15.138:0/317534887 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80701f7fd0 con 0x7f80700d2920 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8075e00640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 0x7f80700cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8075400640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 0x7f80701f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:25.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8074a00640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9b0 tx=0x7f805c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c017070 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f805c00c9a0 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80701f82b0 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.399+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c00cd90 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.400+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80701f8870 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.400+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f805c01a070 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.400+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f805c013ad0 con 0x7f80700d2920 2024-09-10T10:43:25.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.400+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80700d1f10 con 0x7f80700d2920 2024-09-10T10:43:25.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.403+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f805c013470 con 0x7f80700d2920 2024-09-10T10:43:25.419 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.417+0000 7f9f9f520040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T10:43:25.424 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.423+0000 7f408593b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:43:25.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.474+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bb5c2ad5-c112-419b-8ad5-e415abfc4b4c", "id": 7} v 0) -- 0x7f80700b8530 con 0x7f80700d2920 2024-09-10T10:43:25.480 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.479+0000 7f9f9f520040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T10:43:25.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.483+0000 7f806d600640 1 -- 172.21.15.138:0/317534887 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb5c2ad5-c112-419b-8ad5-e415abfc4b4c", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f805c013650 con 0x7f80700d2920 2024-09-10T10:43:25.484 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-09-10T10:43:25.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 msgr2=0x7f8070155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e9b0 tx=0x7f805c00ee70 comp rx=0 tx=0).stop 2024-09-10T10:43:25.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 shutdown_connections 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f807015e550 0x7f80700cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80700d2920 0x7f8070155130 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 --2- 172.21.15.138:0/317534887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80700b3400 0x7f80701f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 >> 172.21.15.138:0/317534887 conn(0x7f807008bfc0 msgr2=0x7f807006d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 shutdown_connections 2024-09-10T10:43:25.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:43:25.484+0000 7f8076800640 1 -- 172.21.15.138:0/317534887 wait complete. 2024-09-10T10:43:25.486 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.485+0000 7f408593b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:43:25.500 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T10:43:25.500 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T10:43:25.504 INFO:tasks.ceph.osd.0:Started 2024-09-10T10:43:25.504 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T10:43:25.504 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T10:43:25.508 INFO:tasks.ceph.osd.1:Started 2024-09-10T10:43:25.508 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T10:43:25.508 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T10:43:25.512 INFO:tasks.ceph.osd.2:Started 2024-09-10T10:43:25.512 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T10:43:25.512 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T10:43:25.517 INFO:tasks.ceph.osd.3:Started 2024-09-10T10:43:25.517 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T10:43:25.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T10:43:25.523 INFO:tasks.ceph.osd.4:Started 2024-09-10T10:43:25.523 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T10:43:25.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T10:43:25.527 INFO:tasks.ceph.osd.5:Started 2024-09-10T10:43:25.527 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T10:43:25.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T10:43:25.531 INFO:tasks.ceph.osd.6:Started 2024-09-10T10:43:25.531 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T10:43:25.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T10:43:25.537 INFO:tasks.ceph.osd.7:Started 2024-09-10T10:43:25.538 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:25.555 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.553+0000 7f9f9f520040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T10:43:25.577 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.576+0000 7f408593b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:43:25.661 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.660+0000 7f408593b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:43:25.663 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.661+0000 7f9f9f520040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T10:43:25.722 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:25.721+0000 7f408593b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:43:25.739 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.737+0000 7f9f9f520040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T10:43:25.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.742+0000 7f1c95000640 1 Processor -- start 2024-09-10T10:43:25.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.742+0000 7f1c95000640 1 -- start start 2024-09-10T10:43:25.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c95000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1c900d2aa0 0x7f1c900d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c95000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c90161d00 con 0x7f1c90161920 2024-09-10T10:43:25.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c95000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c900cf3c0 con 0x7f1c900d4ac0 2024-09-10T10:43:25.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c95000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1c900d3450 con 0x7f1c900d2aa0 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c8f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1c90161920 0x7f1c900d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60558/0 (socket says 172.21.15.28:60558) 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c8f400640 1 -- 172.21.15.28:0/456367828 learned_addr learned my addr 172.21.15.28:0/456367828 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.743+0000 7f1c8fe00640 1 --2- 172.21.15.28:0/456367828 >> v2:172.21.15.28:3300/0 conn(0x7f1c900d2aa0 0x7f1c900d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.744+0000 7f1c8fe00640 1 -- 172.21.15.28:0/456367828 >> v1:172.21.15.138:6789/0 conn(0x7f1c900d4ac0 legacy=0x7f1c900d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.744+0000 7f1c8fe00640 1 -- 172.21.15.28:0/456367828 >> v1:172.21.15.28:6789/0 conn(0x7f1c90161920 legacy=0x7f1c900d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.744+0000 7f1c8e000640 1 -- 172.21.15.28:0/456367828 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235264748 0 0) 0x7f1c900cf3c0 con 0x7f1c900d4ac0 2024-09-10T10:43:25.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.744+0000 7f1c8fe00640 1 -- 172.21.15.28:0/456367828 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c900cf6a0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.745+0000 7f1c8fe00640 1 --2- 172.21.15.28:0/456367828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d2aa0 0x7f1c900d2f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009d90 tx=0x7f1c8802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281c061d6f3b8978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.745+0000 7f1c8e000640 1 -- 172.21.15.28:0/456367828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88007740 con 0x7f1c900d2aa0 2024-09-10T10:43:25.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.745+0000 7f1c8e000640 1 -- 172.21.15.28:0/456367828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c880078e0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.745+0000 7f1c8e000640 1 -- 172.21.15.28:0/456367828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88007cb0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 -- 172.21.15.28:0/456367828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d2aa0 msgr2=0x7f1c900d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 --2- 172.21.15.28:0/456367828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d2aa0 0x7f1c900d2f10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009d90 tx=0x7f1c8802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 -- 172.21.15.28:0/456367828 shutdown_connections 2024-09-10T10:43:25.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 --2- 172.21.15.28:0/456367828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d2aa0 0x7f1c900d2f10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 -- 172.21.15.28:0/456367828 >> 172.21.15.28:0/456367828 conn(0x7f1c9009d770 msgr2=0x7f1c9008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 -- 172.21.15.28:0/456367828 shutdown_connections 2024-09-10T10:43:25.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.746+0000 7f1c95000640 1 -- 172.21.15.28:0/456367828 wait complete. 2024-09-10T10:43:25.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.748+0000 7f1c95000640 1 Processor -- start 2024-09-10T10:43:25.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.748+0000 7f1c95000640 1 -- start start 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c90161920 0x7f1c900a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900cfc60 con 0x7f1c900d4ac0 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900cffd0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c95000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c900cf070 con 0x7f1c90161920 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35326/0 (socket says 172.21.15.28:35326) 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8ea00640 1 -- 172.21.15.28:0/4208600207 learned_addr learned my addr 172.21.15.28:0/4208600207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:49332/0 (socket says 172.21.15.28:49332) 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8fe00640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c90161920 0x7f1c900a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 -- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c90161920 msgr2=0x7f1c900a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c90161920 0x7f1c900a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 -- 172.21.15.28:0/4208600207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 msgr2=0x7f1c90160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.749+0000 7f1c8f400640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c8f400640 1 -- 172.21.15.28:0/4208600207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c900b7310 con 0x7f1c900d2aa0 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c8ea00640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c8f400640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c520 tx=0x7f1c8000c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:25.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80018070 con 0x7f1c900d2aa0 2024-09-10T10:43:25.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c80007560 con 0x7f1c900d2aa0 2024-09-10T10:43:25.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800049e0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c900ba0e0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c900b75b0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.750+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c8001b070 con 0x7f1c900d2aa0 2024-09-10T10:43:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.751+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1c80012ad0 con 0x7f1c900d2aa0 2024-09-10T10:43:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.751+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c900d2f10 con 0x7f1c900d2aa0 2024-09-10T10:43:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.754+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1c80022020 con 0x7f1c900d2aa0 2024-09-10T10:43:25.796 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:25.794+0000 7f9f9f520040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T10:43:25.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.817+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c900b6850 con 0x7f1c900d2aa0 2024-09-10T10:43:25.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.818+0000 7f1c87400640 1 -- 172.21.15.28:0/4208600207 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1c80004d90 con 0x7f1c900d2aa0 2024-09-10T10:43:25.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:25.820 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":10,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:25.476785+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.819+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 msgr2=0x7f1c9009eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.819+0000 7f1c95000640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000c520 tx=0x7f1c8000c9e0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 shutdown_connections 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c90161920 0x7f1c900a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c900d4ac0 0x7f1c90160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 --2- 172.21.15.28:0/4208600207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c900d2aa0 0x7f1c9009eda0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 >> 172.21.15.28:0/4208600207 conn(0x7f1c9009d770 msgr2=0x7f1c900cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 shutdown_connections 2024-09-10T10:43:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:25.820+0000 7f1c95000640 1 -- 172.21.15.28:0/4208600207 wait complete. 2024-09-10T10:43:25.837 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T10:43:25.837 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T10:43:26.468 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:26.466+0000 7f408593b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:43:26.580 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-09-10T10:43:26.578+0000 7f9f9f520040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T10:43:26.643 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:26.779 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.777+0000 7f282d600640 1 Processor -- start 2024-09-10T10:43:26.779 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.777+0000 7f282d600640 1 -- start start 2024-09-10T10:43:26.780 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f282d600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28280cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:26.780 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f282d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f28280b6960 0x7f28280b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:26.780 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f282d600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f282815e840 con 0x7f28280b5e10 2024-09-10T10:43:26.780 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f282d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f28280d3a40 con 0x7f28280b6960 2024-09-10T10:43:26.780 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f282d600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28280cf940 con 0x7f282815e460 2024-09-10T10:43:26.781 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.778+0000 7f2827400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f28280b5e10 0x7f28280b6250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60562/0 (socket says 172.21.15.28:60562) 2024-09-10T10:43:26.781 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827400640 1 -- 172.21.15.28:0/766453168 learned_addr learned my addr 172.21.15.28:0/766453168 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:26.781 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827e00640 1 --2- 172.21.15.28:0/766453168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28280cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:26.781 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f282cc00640 1 --2- 172.21.15.28:0/766453168 >> v2:172.21.15.138:3300/0 conn(0x7f28280b6960 0x7f28280b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:26.782 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2826a00640 1 -- 172.21.15.28:0/766453168 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3896421539 0 0) 0x7f282815e840 con 0x7f28280b5e10 2024-09-10T10:43:26.782 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827e00640 1 -- 172.21.15.28:0/766453168 >> v2:172.21.15.138:3300/0 conn(0x7f28280b6960 msgr2=0x7f28280b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:26.782 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827e00640 1 --2- 172.21.15.28:0/766453168 >> v2:172.21.15.138:3300/0 conn(0x7f28280b6960 0x7f28280b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.782 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827e00640 1 -- 172.21.15.28:0/766453168 >> v1:172.21.15.28:6789/0 conn(0x7f28280b5e10 legacy=0x7f28280b6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.782 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.779+0000 7f2827e00640 1 -- 172.21.15.28:0/766453168 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28280d3110 con 0x7f282815e460 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.780+0000 7f2827e00640 1 --2- 172.21.15.28:0/766453168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28280cf360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f28180097a0 tx=0x7f281802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ac0f5ef79a102d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.780+0000 7f2826a00640 1 -- 172.21.15.28:0/766453168 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818037070 con 0x7f282815e460 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.780+0000 7f2826a00640 1 -- 172.21.15.28:0/766453168 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2818032440 con 0x7f282815e460 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.780+0000 7f2826a00640 1 -- 172.21.15.28:0/766453168 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818032810 con 0x7f282815e460 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 -- 172.21.15.28:0/766453168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 msgr2=0x7f28280cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.783 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 --2- 172.21.15.28:0/766453168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28280cf360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f28180097a0 tx=0x7f281802a180 comp rx=0 tx=0).stop 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 -- 172.21.15.28:0/766453168 shutdown_connections 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 --2- 172.21.15.28:0/766453168 >> v2:172.21.15.138:3300/0 conn(0x7f28280b6960 0x7f28280b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 --2- 172.21.15.28:0/766453168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28280cf360 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 -- 172.21.15.28:0/766453168 >> 172.21.15.28:0/766453168 conn(0x7f2828073760 msgr2=0x7f2828073b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 -- 172.21.15.28:0/766453168 shutdown_connections 2024-09-10T10:43:26.784 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.781+0000 7f282d600640 1 -- 172.21.15.28:0/766453168 wait complete. 2024-09-10T10:43:26.785 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.783+0000 7f282d600640 1 Processor -- start 2024-09-10T10:43:26.785 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.783+0000 7f282d600640 1 -- start start 2024-09-10T10:43:26.786 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28280b6960 0x7f2828156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28281f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28280d2180 con 0x7f28280b5e10 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28280d36d0 con 0x7f28280b6960 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282d600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28280b7cb0 con 0x7f282815e460 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f2827e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f2827e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35330/0 (socket says 172.21.15.28:35330) 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f2827e00640 1 -- 172.21.15.28:0/377994317 learned_addr learned my addr 172.21.15.28:0/377994317 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f282cc00640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28281f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.784+0000 7f2827400640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28280b6960 0x7f2828156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 -- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 msgr2=0x7f28281f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.787 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28281f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.788 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 -- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28280b6960 msgr2=0x7f2828156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.788 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28280b6960 0x7f2828156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.788 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 -- 172.21.15.28:0/377994317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28280ccf20 con 0x7f28280b5e10 2024-09-10T10:43:26.788 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f2827e00640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28180098c0 tx=0x7f28180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818037040 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2818042070 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28281f47a0 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2818030920 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.785+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28281f4440 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.786+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28280b7770 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.787+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 2) ==== 45274+0+0 (secure 0 0 0) 0x7f2818030ac0 con 0x7f28280b5e10 2024-09-10T10:43:26.789 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.787+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f281806ad10 con 0x7f28280b5e10 2024-09-10T10:43:26.791 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.789+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2818047980 con 0x7f28280b5e10 2024-09-10T10:43:26.854 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.852+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28280b62d0 con 0x7f28280b5e10 2024-09-10T10:43:26.855 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.853+0000 7f281fe00640 1 -- 172.21.15.28:0/377994317 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2818047460 con 0x7f28280b5e10 2024-09-10T10:43:26.855 INFO:teuthology.misc.health.smithi028.stdout: 2024-09-10T10:43:26.855 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":10,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:25.476785+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:26.856 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.853+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 msgr2=0x7f28281f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:26.856 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.853+0000 7f282d600640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f28180098c0 tx=0x7f28180046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.856 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 shutdown_connections 2024-09-10T10:43:26.856 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f282815e460 0x7f28281f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.856 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28280b6960 0x7f2828156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.857 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 --2- 172.21.15.28:0/377994317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28280b5e10 0x7f28281f8020 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:26.857 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 >> 172.21.15.28:0/377994317 conn(0x7f2828073760 msgr2=0x7f282815c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:26.857 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 shutdown_connections 2024-09-10T10:43:26.857 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:26.854+0000 7f282d600640 1 -- 172.21.15.28:0/377994317 wait complete. 2024-09-10T10:43:26.868 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T10:43:27.029 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-10T10:43:27.028+0000 7f8089a1c640 -1 Falling back to public interface 2024-09-10T10:43:27.054 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-10T10:43:27.053+0000 7fad33c31640 -1 Falling back to public interface 2024-09-10T10:43:27.054 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-10T10:43:27.052+0000 7f121a40e640 -1 Falling back to public interface 2024-09-10T10:43:27.056 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-09-10T10:43:27.055+0000 7fe8b57a2640 -1 Falling back to public interface 2024-09-10T10:43:27.059 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-10T10:43:27.058+0000 7f2fbc520640 -1 Falling back to public interface 2024-09-10T10:43:27.124 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-10T10:43:27.123+0000 7f0a51dfd640 -1 Falling back to public interface 2024-09-10T10:43:27.371 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-10T10:43:27.369+0000 7f2fc2a5e640 -1 Falling back to public interface 2024-09-10T10:43:27.491 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:27.489+0000 7f4025c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T10:43:27.491 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-10T10:43:27.490+0000 7f4025c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T10:43:27.621 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-10T10:43:27.619+0000 7fbe55bc3640 -1 Falling back to public interface 2024-09-10T10:43:30.699 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-10T10:43:30.698+0000 7f8089a1c640 -1 osd.5 0 log_to_monitors true 2024-09-10T10:43:30.781 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-10T10:43:30.779+0000 7f121a40e640 -1 osd.1 0 log_to_monitors true 2024-09-10T10:43:31.034 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-09-10T10:43:31.032+0000 7fe8b57a2640 -1 osd.0 0 log_to_monitors true 2024-09-10T10:43:31.074 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-10T10:43:31.073+0000 7f0a51dfd640 -1 osd.7 0 log_to_monitors true 2024-09-10T10:43:31.246 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-10T10:43:31.244+0000 7fad33c31640 -1 osd.4 0 log_to_monitors true 2024-09-10T10:43:31.280 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-10T10:43:31.279+0000 7f2fbc520640 -1 osd.6 0 log_to_monitors true 2024-09-10T10:43:31.558 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-10T10:43:31.556+0000 7fbe55bc3640 -1 osd.2 0 log_to_monitors true 2024-09-10T10:43:31.845 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-10T10:43:31.843+0000 7f2fc2a5e640 -1 osd.3 0 log_to_monitors true 2024-09-10T10:43:32.678 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-10T10:43:32.676+0000 7f8084200640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T10:43:32.684 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-10T10:43:32.683+0000 7f807a200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:33.674 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:33.910 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.909+0000 7f676be00640 1 Processor -- start 2024-09-10T10:43:33.914 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.909+0000 7f676be00640 1 -- start start 2024-09-10T10:43:33.915 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6764005a60 0x7f6764053f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.915 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f676404eb60 0x7f676401d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.915 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6764052330 con 0x7f67640e7120 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f67640520e0 con 0x7f676404eb60 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6764006c80 con 0x7f6764005a60 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f676404eb60 0x7f676401d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f676404eb60 0x7f676401d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:42204/0 (socket says 172.21.15.28:42204) 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.912+0000 7f676b400640 1 -- 172.21.15.28:0/2998094875 learned_addr learned my addr 172.21.15.28:0/2998094875 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.913+0000 7f676b400640 1 -- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6764005a60 msgr2=0x7f6764053f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.913+0000 7f676b400640 1 --2- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6764005a60 0x7f6764053f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.913+0000 7f676b400640 1 -- 172.21.15.28:0/2998094875 >> v1:172.21.15.28:6789/0 conn(0x7f67640e7120 legacy=0x7f6764005e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.913+0000 7f676b400640 1 -- 172.21.15.28:0/2998094875 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6764006f60 con 0x7f676404eb60 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.914+0000 7f676b400640 1 --2- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f676401d530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f676c000b60 tx=0x7f676c0a63e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4aba84d10c0d3c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:33.916 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.914+0000 7f6769600640 1 -- 172.21.15.28:0/2998094875 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0b2070 con 0x7f676404eb60 2024-09-10T10:43:33.917 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.914+0000 7f6769600640 1 -- 172.21.15.28:0/2998094875 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676c0ad640 con 0x7f676404eb60 2024-09-10T10:43:33.917 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.914+0000 7f6769600640 1 -- 172.21.15.28:0/2998094875 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0adab0 con 0x7f676404eb60 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.915+0000 7f676be00640 1 -- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 msgr2=0x7f676401d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.915+0000 7f676be00640 1 --2- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f676401d530 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f676c000b60 tx=0x7f676c0a63e0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.916+0000 7f676be00640 1 -- 172.21.15.28:0/2998094875 shutdown_connections 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.916+0000 7f676be00640 1 --2- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f676401d530 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.916+0000 7f676be00640 1 --2- 172.21.15.28:0/2998094875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6764005a60 0x7f6764053f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.916+0000 7f676be00640 1 -- 172.21.15.28:0/2998094875 >> 172.21.15.28:0/2998094875 conn(0x7f6764036090 msgr2=0x7f67640364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:33.918 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.917+0000 7f676be00640 1 -- 172.21.15.28:0/2998094875 shutdown_connections 2024-09-10T10:43:33.923 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.920+0000 7f676be00640 1 -- 172.21.15.28:0/2998094875 wait complete. 2024-09-10T10:43:33.923 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.922+0000 7f676be00640 1 Processor -- start 2024-09-10T10:43:33.923 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.922+0000 7f676be00640 1 -- start start 2024-09-10T10:43:33.925 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.924+0000 7f676be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.925 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.924+0000 7f676be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f67640373d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.926 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.925+0000 7f676be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 0x7f676401d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.926 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.925+0000 7f676be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6764047b50 con 0x7f6764005a60 2024-09-10T10:43:33.926 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.925+0000 7f676aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.925+0000 7f676aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34986/0 (socket says 172.21.15.28:34986) 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.925+0000 7f676aa00640 1 -- 172.21.15.28:0/2561718628 learned_addr learned my addr 172.21.15.28:0/2561718628 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.926+0000 7f676b400640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 0x7f676401d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.926+0000 7f676a000640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f67640373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.926+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6764053a10 con 0x7f676404eb60 2024-09-10T10:43:33.927 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.926+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6764047cd0 con 0x7f67640e7120 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.927+0000 7f676b400640 1 -- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 msgr2=0x7f67640373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.927+0000 7f676b400640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f67640373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.927+0000 7f676b400640 1 -- 172.21.15.28:0/2561718628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 msgr2=0x7f67640e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.927+0000 7f676b400640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.928+0000 7f676b400640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6764038ec0 con 0x7f67640e7120 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.928+0000 7f676a000640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f67640373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.928+0000 7f676aa00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:33.929 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.928+0000 7f676b400640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 0x7f676401d060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f676c06afe0 tx=0x7f676c075170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:33.930 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.928+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0bd070 con 0x7f67640e7120 2024-09-10T10:43:33.930 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.929+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676c0ac9e0 con 0x7f67640e7120 2024-09-10T10:43:33.930 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.929+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c0b2040 con 0x7f67640e7120 2024-09-10T10:43:33.930 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.929+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67640cdf50 con 0x7f67640e7120 2024-09-10T10:43:33.930 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.929+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67640ce4d0 con 0x7f67640e7120 2024-09-10T10:43:33.931 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.930+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764047600 con 0x7f67640e7120 2024-09-10T10:43:33.933 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.932+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f676c0c0070 con 0x7f67640e7120 2024-09-10T10:43:33.934 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.933+0000 7f6762a00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 0x7f674c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:33.934 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.933+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f676c126a30 con 0x7f67640e7120 2024-09-10T10:43:33.935 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.934+0000 7f676aa00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 0x7f674c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:33.937 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.936+0000 7f676aa00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 0x7f674c06e330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f675c006fd0 tx=0x7f675c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:33.942 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:33.941+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f676c0c7030 con 0x7f67640e7120 2024-09-10T10:43:34.314 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.313+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6764008ad0 con 0x7f67640e7120 2024-09-10T10:43:34.319 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.318+0000 7f6762a00640 1 -- 172.21.15.28:0/2561718628 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7f6764038ec0 con 0x7f67640e7120 2024-09-10T10:43:34.321 INFO:teuthology.misc.health.smithi028.stdout: 2024-09-10T10:43:34.322 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":14,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:33.765716+0000","last_up_change":"2024-09-10T10:43:32.761426+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:34.322 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.321+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 msgr2=0x7f674c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.322 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.321+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 0x7f674c06e330 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f675c006fd0 tx=0x7f675c009190 comp rx=0 tx=0).stop 2024-09-10T10:43:34.323 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.321+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 msgr2=0x7f676401d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:34.323 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.321+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 0x7f676401d060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f676c06afe0 tx=0x7f676c075170 comp rx=0 tx=0).stop 2024-09-10T10:43:34.323 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 shutdown_connections 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f674c06be70 0x7f674c06e330 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67640e7120 0x7f676401d060 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f676404eb60 0x7f67640373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 --2- 172.21.15.28:0/2561718628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6764005a60 0x7f67640e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.322+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 >> 172.21.15.28:0/2561718628 conn(0x7f6764036090 msgr2=0x7f676404f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.323+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 shutdown_connections 2024-09-10T10:43:34.324 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:34.323+0000 7f676be00640 1 -- 172.21.15.28:0/2561718628 wait complete. 2024-09-10T10:43:34.345 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-10T10:43:34.401 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-10T10:43:34.400+0000 7fbe50a00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T10:43:34.411 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-09-10T10:43:34.410+0000 7fbe46a00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.423 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-10T10:43:34.423+0000 7f1215600640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T10:43:34.433 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-09-10T10:43:34.432+0000 7f120ac00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.441 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-09-10T10:43:34.440+0000 7fe8b0600640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T10:43:34.455 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-09-10T10:43:34.454+0000 7fe8a6600640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.581 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-10T10:43:34.579+0000 7fad2ea00640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T10:43:34.598 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-10T10:43:34.597+0000 7fad24a00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.601 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-10T10:43:34.600+0000 7f2fb7e00640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T10:43:34.617 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-10T10:43:34.616+0000 7f2fad400640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.627 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-10T10:43:34.626+0000 7f0a4cc00640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T10:43:34.635 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-10T10:43:34.634+0000 7f0a42c00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:34.977 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-10T10:43:34.976+0000 7f2fbe400640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T10:43:34.984 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-09-10T10:43:34.983+0000 7f2fb3a00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T10:43:38.364 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:43:38.368 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:43:41.207 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:41.353 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.352+0000 7fe4f8200640 1 Processor -- start 2024-09-10T10:43:41.353 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.352+0000 7fe4f8200640 1 -- start start 2024-09-10T10:43:41.355 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f8200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f01403b0 0x7fe4f00cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.355 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f8200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f00d0610 0x7fe4f015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.355 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f8200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f00cf160 con 0x7fe4f00d01c0 2024-09-10T10:43:41.355 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f8200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f00cf400 con 0x7fe4f01403b0 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f8200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f00cf6e0 con 0x7fe4f00d0610 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.353+0000 7fe4f5800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f01403b0 0x7fe4f00cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f01403b0 0x7fe4f00cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34990/0 (socket says 172.21.15.28:34990) 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 -- 172.21.15.28:0/3787329083 learned_addr learned my addr 172.21.15.28:0/3787329083 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f6200640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f00d0610 0x7fe4f015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 -- 172.21.15.28:0/3787329083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f00d0610 msgr2=0x7fe4f015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f00d0610 0x7fe4f015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 -- 172.21.15.28:0/3787329083 >> v1:172.21.15.138:6789/0 conn(0x7fe4f00d01c0 legacy=0x7fe4f0140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.356 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.354+0000 7fe4f5800640 1 -- 172.21.15.28:0/3787329083 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f00cf9c0 con 0x7fe4f01403b0 2024-09-10T10:43:41.357 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.355+0000 7fe4f5800640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f01403b0 0x7fe4f00cd6a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009870 tx=0x7fe4e402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8d656627c43706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.355+0000 7fe4efe00640 1 -- 172.21.15.28:0/3787329083 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4037070 con 0x7fe4f01403b0 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.355+0000 7fe4efe00640 1 -- 172.21.15.28:0/3787329083 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e4032440 con 0x7fe4f01403b0 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.355+0000 7fe4efe00640 1 -- 172.21.15.28:0/3787329083 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4032810 con 0x7fe4f01403b0 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 -- 172.21.15.28:0/3787329083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f01403b0 msgr2=0x7fe4f00cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f01403b0 0x7fe4f00cd6a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009870 tx=0x7fe4e402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 -- 172.21.15.28:0/3787329083 shutdown_connections 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f00d0610 0x7fe4f015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 --2- 172.21.15.28:0/3787329083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f01403b0 0x7fe4f00cd6a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 -- 172.21.15.28:0/3787329083 >> 172.21.15.28:0/3787329083 conn(0x7fe4f00735c0 msgr2=0x7fe4f006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 -- 172.21.15.28:0/3787329083 shutdown_connections 2024-09-10T10:43:41.358 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.356+0000 7fe4f8200640 1 -- 172.21.15.28:0/3787329083 wait complete. 2024-09-10T10:43:41.359 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.358+0000 7fe4f8200640 1 Processor -- start 2024-09-10T10:43:41.359 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.358+0000 7fe4f8200640 1 -- start start 2024-09-10T10:43:41.361 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.358+0000 7fe4f8200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.361 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f8200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 0x7fe4f0159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.361 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f8200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 0x7fe4f013cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.361 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f8200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f015eab0 con 0x7fe4f00d01c0 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f8200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0140ea0 con 0x7fe4f00d0610 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f8200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f015ec30 con 0x7fe4f01403b0 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35002/0 (socket says 172.21.15.28:35002) 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 -- 172.21.15.28:0/1566798679 learned_addr learned my addr 172.21.15.28:0/1566798679 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f6200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 0x7fe4f013cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f4e00640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 0x7fe4f0159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 -- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 msgr2=0x7fe4f013cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 0x7fe4f013cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 -- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 msgr2=0x7fe4f0159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.362 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 0x7fe4f0159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 -- 172.21.15.28:0/1566798679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f0138750 con 0x7fe4f00d01c0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f6200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 0x7fe4f013cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f4e00640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 0x7fe4f0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.359+0000 7fe4f5800640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4000c00 tx=0x7fe4e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.360+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4041020 con 0x7fe4f00d01c0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.360+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e40306c0 con 0x7fe4f00d01c0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.360+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4037040 con 0x7fe4f00d01c0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.360+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f0138a30 con 0x7fe4f00d01c0 2024-09-10T10:43:41.363 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.360+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f0138ff0 con 0x7fe4f00d01c0 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.361+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f015e570 con 0x7fe4f00d01c0 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.362+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7fe4e4044020 con 0x7fe4f00d01c0 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.363+0000 7fe4ed600640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 0x7fe4d006e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.363+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe4e40ab220 con 0x7fe4f00d01c0 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.364+0000 7fe4f4e00640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 0x7fe4d006e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.365 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.364+0000 7fe4f4e00640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 0x7fe4d006e260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00b4af0 tx=0x7fe4dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.369 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.368+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4e404b030 con 0x7fe4f00d01c0 2024-09-10T10:43:41.484 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.483+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4e402ad10 con 0x7fe4f00d01c0 2024-09-10T10:43:41.561 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.560+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4f00b0c90 con 0x7fe4f00d01c0 2024-09-10T10:43:41.561 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.560+0000 7fe4ed600640 1 -- 172.21.15.28:0/1566798679 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12920 (secure 0 0 0) 0x7fe4e40a9e20 con 0x7fe4f00d01c0 2024-09-10T10:43:41.562 INFO:teuthology.misc.health.smithi028.stdout: 2024-09-10T10:43:41.562 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":19,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:39.461790+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:41.564 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 msgr2=0x7fe4d006e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.564 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 0x7fe4d006e260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00b4af0 tx=0x7fe4dc009040 comp rx=0 tx=0).stop 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 msgr2=0x7fe4f00b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4000c00 tx=0x7fe4e40046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 shutdown_connections 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4d006bda0 0x7fe4d006e260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4f01403b0 0x7fe4f013cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4f00d0610 0x7fe4f0159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.565 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 --2- 172.21.15.28:0/1566798679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f00d01c0 0x7fe4f00b0750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.568 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 >> 172.21.15.28:0/1566798679 conn(0x7fe4f00735c0 msgr2=0x7fe4f015c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:41.568 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 shutdown_connections 2024-09-10T10:43:41.568 INFO:teuthology.misc.health.smithi028.stderr:2024-09-10T10:43:41.563+0000 7fe4f8200640 1 -- 172.21.15.28:0/1566798679 wait complete. 2024-09-10T10:43:41.577 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T10:43:41.577 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T10:43:41.578 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:43:41.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:43:41.582 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:43:41.582 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:43:41.582 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:43:41.586 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:43:41.586 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:43:41.587 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:43:41.591 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:43:41.591 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:43:41.591 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:43:41.595 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:43:41.596 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T10:43:41.596 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:43:41.713 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:43:41.715 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:43:41.721 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:43:41.722 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:43:41.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.759+0000 7fb3ade00640 1 Processor -- start 2024-09-10T10:43:41.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.759+0000 7fb3ade00640 1 -- start start 2024-09-10T10:43:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.760+0000 7fb3ade00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb3a80d1f90 0x7fb3a80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.760+0000 7fb3ade00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a80cf000 con 0x7fb3a815e660 2024-09-10T10:43:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.760+0000 7fb3ade00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a80d29a0 con 0x7fb3a80d0dd0 2024-09-10T10:43:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.760+0000 7fb3ade00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a806d3e0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3a7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb3a815e660 0x7fb3a806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40090/0 (socket says 172.21.15.28:40090) 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3a7e00640 1 -- 172.21.15.28:0/4079080858 learned_addr learned my addr 172.21.15.28:0/4079080858 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3aca00640 1 --1- 172.21.15.28:0/4079080858 >> v1:172.21.15.138:6789/0 conn(0x7fb3a80d0dd0 0x7fb3a80d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:33124/0 (socket says 172.21.15.28:33124) 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3ad400640 1 --2- 172.21.15.28:0/4079080858 >> v2:172.21.15.28:3300/0 conn(0x7fb3a80d1f90 0x7fb3a80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3a7400640 1 -- 172.21.15.28:0/4079080858 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425315007 0 0) 0x7fb3a80d29a0 con 0x7fb3a80d0dd0 2024-09-10T10:43:41.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3ad400640 1 -- 172.21.15.28:0/4079080858 >> v1:172.21.15.138:6789/0 conn(0x7fb3a80d0dd0 legacy=0x7fb3a80d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3ad400640 1 -- 172.21.15.28:0/4079080858 >> v1:172.21.15.28:6789/0 conn(0x7fb3a815e660 legacy=0x7fb3a806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.761+0000 7fb3ad400640 1 -- 172.21.15.28:0/4079080858 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a80d2c60 con 0x7fb3a80d1f90 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.762+0000 7fb3ad400640 1 --2- 172.21.15.28:0/4079080858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a80cf7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009d90 tx=0x7fb3a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6347a10dda20247b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.762+0000 7fb3a7400640 1 -- 172.21.15.28:0/4079080858 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0007740 con 0x7fb3a80d1f90 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.762+0000 7fb3a7400640 1 -- 172.21.15.28:0/4079080858 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a00078e0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.762+0000 7fb3a7400640 1 -- 172.21.15.28:0/4079080858 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0007cb0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 -- 172.21.15.28:0/4079080858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 msgr2=0x7fb3a80cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 --2- 172.21.15.28:0/4079080858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a80cf7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009d90 tx=0x7fb3a002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 -- 172.21.15.28:0/4079080858 shutdown_connections 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 --2- 172.21.15.28:0/4079080858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a80cf7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 -- 172.21.15.28:0/4079080858 >> 172.21.15.28:0/4079080858 conn(0x7fb3a808bfc0 msgr2=0x7fb3a80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 -- 172.21.15.28:0/4079080858 shutdown_connections 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.763+0000 7fb3ade00640 1 -- 172.21.15.28:0/4079080858 wait complete. 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.764+0000 7fb3ade00640 1 Processor -- start 2024-09-10T10:43:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.765+0000 7fb3ade00640 1 -- start start 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.765+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3a80d0dd0 0x7fb3a80b98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.765+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.765+0000 7fb3ade00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3a815e660 0x7fb3a80b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3ade00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a80cf330 con 0x7fb3a80d1f90 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3ade00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a80d3220 con 0x7fb3a815e660 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3ade00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a80d33a0 con 0x7fb3a80d0dd0 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35048/0 (socket says 172.21.15.28:35048) 2024-09-10T10:43:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 -- 172.21.15.28:0/1945646744 learned_addr learned my addr 172.21.15.28:0/1945646744 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3aca00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3a80d0dd0 0x7fb3a80b98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3ad400640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3a815e660 0x7fb3a80b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 -- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3a80d0dd0 msgr2=0x7fb3a80b98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3a80d0dd0 0x7fb3a80b98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 -- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3a815e660 msgr2=0x7fb3a80b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3a815e660 0x7fb3a80b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 -- 172.21.15.28:0/1945646744 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a8154810 con 0x7fb3a80d1f90 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.766+0000 7fb3a7e00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c3c0 tx=0x7fb39400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.767+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394017070 con 0x7fb3a80d1f90 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.767+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb394005ce0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.767+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3940072d0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.767+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a8161000 con 0x7fb3a80d1f90 2024-09-10T10:43:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.767+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a8154d90 con 0x7fb3a80d1f90 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.768+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a80d3520 con 0x7fb3a80d1f90 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.769+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb39401a070 con 0x7fb3a80d1f90 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.770+0000 7fb3a4c00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 0x7fb38806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.770+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fb394089d10 con 0x7fb3a80d1f90 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.770+0000 7fb3aca00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 0x7fb38806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:41.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.771+0000 7fb3aca00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 0x7fb38806e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007f70 tx=0x7fb39c007f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.775+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3a80d3520 con 0x7fb3a80d1f90 2024-09-10T10:43:41.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.997+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3a815dea0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.997+0000 7fb3a4c00640 1 -- 172.21.15.28:0/1945646744 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fb3a815dea0 con 0x7fb3a80d1f90 2024-09-10T10:43:41.998 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:41.999 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":2,"btime":"2024-09-10T10:43:41:721462+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T10:43:41.999 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 2 2024-09-10T10:43:42.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 msgr2=0x7fb38806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 0x7fb38806e320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007f70 tx=0x7fb39c007f00 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 msgr2=0x7fb3a81552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c3c0 tx=0x7fb39400c880 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 shutdown_connections 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb38806be60 0x7fb38806e320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3a815e660 0x7fb3a80b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3a80d1f90 0x7fb3a81552b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 --2- 172.21.15.28:0/1945646744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3a80d0dd0 0x7fb3a80b98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 >> 172.21.15.28:0/1945646744 conn(0x7fb3a808bfc0 msgr2=0x7fb3a806e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:42.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 shutdown_connections 2024-09-10T10:43:42.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:41.999+0000 7fb3ade00640 1 -- 172.21.15.28:0/1945646744 wait complete. 2024-09-10T10:43:44.017 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:43:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.182+0000 7fed8de00640 1 Processor -- start 2024-09-10T10:43:44.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.182+0000 7fed8de00640 1 -- start start 2024-09-10T10:43:44.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.183+0000 7fed8de00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed88072320 0x7fed880cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.183+0000 7fed8de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fed88072de0 0x7fed8806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.183+0000 7fed8de00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed88146140 con 0x7fed88072700 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.183+0000 7fed8de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fed880cd740 con 0x7fed88072de0 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.183+0000 7fed8de00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed880cda20 con 0x7fed88072320 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed87400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fed88072700 0x7fed8806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34004/0 (socket says 172.21.15.28:34004) 2024-09-10T10:43:44.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed87400640 1 -- 172.21.15.28:0/474874824 learned_addr learned my addr 172.21.15.28:0/474874824 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed86000640 1 --2- 172.21.15.28:0/474874824 >> v2:172.21.15.138:3300/0 conn(0x7fed88072de0 0x7fed8806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed86a00640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed88072320 0x7fed880cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 442221992 0 0) 0x7fed88146140 con 0x7fed88072700 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.184+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed6c003520 con 0x7fed88072700 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4290869717 0 0) 0x7fed6c003520 con 0x7fed88072700 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed88072320 msgr2=0x7fed880cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed88072320 0x7fed880cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 >> v2:172.21.15.138:3300/0 conn(0x7fed88072de0 msgr2=0x7fed8806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 --2- 172.21.15.28:0/474874824 >> v2:172.21.15.138:3300/0 conn(0x7fed88072de0 0x7fed8806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880cdd00 con 0x7fed88072700 2024-09-10T10:43:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fed800019f0 con 0x7fed88072700 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 >> v1:172.21.15.28:6789/0 conn(0x7fed88072700 legacy=0x7fed8806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed6c0039b0 0x7fed6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed6c0043a0 0x7fed6c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed880cd740 con 0x7fed6c004f90 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed880cda20 con 0x7fed6c0043a0 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed6c00bf30 con 0x7fed6c0039b0 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed86a00640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed88072320 0x7fed880cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed86000640 1 --2- 172.21.15.28:0/474874824 >> v2:172.21.15.138:3300/0 conn(0x7fed88072de0 0x7fed8806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed86a00640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed6c0039b0 0x7fed6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed86000640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed6c0043a0 0x7fed6c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 -- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed6c0039b0 msgr2=0x7fed6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed6c0039b0 0x7fed6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 -- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed6c0043a0 msgr2=0x7fed6c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed6c0043a0 0x7fed6c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.185+0000 7fed87400640 1 -- 172.21.15.28:0/474874824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c003520 con 0x7fed6c004f90 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.186+0000 7fed87400640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed6c00b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed8000a920 tx=0x7fed6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45492d4279516a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8000a4c0 con 0x7fed6c004f90 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8002f070 con 0x7fed6c004f90 2024-09-10T10:43:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed85600640 1 -- 172.21.15.28:0/474874824 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8002bd30 con 0x7fed6c004f90 2024-09-10T10:43:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 -- 172.21.15.28:0/474874824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 msgr2=0x7fed6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 --2- 172.21.15.28:0/474874824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed6c00b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed8000a920 tx=0x7fed6c005830 comp rx=0 tx=0).stop 2024-09-10T10:43:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed87400640 1 -- 172.21.15.28:0/474874824 reap_dead start 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 -- 172.21.15.28:0/474874824 shutdown_connections 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 -- 172.21.15.28:0/474874824 >> 172.21.15.28:0/474874824 conn(0x7fed88067d70 msgr2=0x7fed8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 -- 172.21.15.28:0/474874824 shutdown_connections 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.187+0000 7fed8de00640 1 -- 172.21.15.28:0/474874824 wait complete. 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.189+0000 7fed8de00640 1 Processor -- start 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.189+0000 7fed8de00640 1 -- start start 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 0x7fed881e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 0x7fed88140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806ee80 con 0x7fed6c004f90 2024-09-10T10:43:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed8806ed00 con 0x7fed881f3d70 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed8de00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed8815e1f0 con 0x7fed881401d0 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed86a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed86a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39796/0 (socket says 172.21.15.28:39796) 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.190+0000 7fed86a00640 1 -- 172.21.15.28:0/1840818188 learned_addr learned my addr 172.21.15.28:0/1840818188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86000640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 0x7fed881e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed87400640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 0x7fed88140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 -- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 msgr2=0x7fed88140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 0x7fed88140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 -- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 msgr2=0x7fed881e3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 0x7fed881e3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 -- 172.21.15.28:0/1840818188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed88143010 con 0x7fed6c004f90 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86000640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 0x7fed881e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed87400640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 0x7fed88140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed86a00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed780119a0 tx=0x7fed78011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7800fbb0 con 0x7fed6c004f90 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7800fd50 con 0x7fed6c004f90 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78014640 con 0x7fed6c004f90 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed881432f0 con 0x7fed6c004f90 2024-09-10T10:43:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.191+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed881e4c90 con 0x7fed6c004f90 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.193+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed880cce60 con 0x7fed6c004f90 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.194+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed78012040 con 0x7fed6c004f90 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.195+0000 7fed7f400640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 0x7fed6806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.195+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fed7808b2d0 con 0x7fed6c004f90 2024-09-10T10:43:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.195+0000 7fed86000640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 0x7fed6806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.196+0000 7fed86000640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 0x7fed6806e340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed88140180 tx=0x7fed74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.200+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed78019030 con 0x7fed6c004f90 2024-09-10T10:43:44.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.424+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed88143750 con 0x7fed6c004f90 2024-09-10T10:43:44.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.425+0000 7fed7f400640 1 -- 172.21.15.28:0/1840818188 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7fed7805a040 con 0x7fed6c004f90 2024-09-10T10:43:44.426 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:44.426 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":3,"btime":"2024-09-10T10:43:42:722860+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4284,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T10:43:44.426 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 3 2024-09-10T10:43:44.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 msgr2=0x7fed6806e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 0x7fed6806e340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fed88140180 tx=0x7fed74008040 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 msgr2=0x7fed881e7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed780119a0 tx=0x7fed78011e60 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 shutdown_connections 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6806be80 0x7fed6806e340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed881401d0 0x7fed88140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed881f3d70 0x7fed881e3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 --2- 172.21.15.28:0/1840818188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed6c004f90 0x7fed881e7420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 >> 172.21.15.28:0/1840818188 conn(0x7fed88067d70 msgr2=0x7fed880cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 shutdown_connections 2024-09-10T10:43:44.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.427+0000 7fed8de00640 1 -- 172.21.15.28:0/1840818188 wait complete. 2024-09-10T10:43:44.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:43:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.603+0000 7fa327e00640 1 Processor -- start 2024-09-10T10:43:44.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.603+0000 7fa327e00640 1 -- start start 2024-09-10T10:43:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa327e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa328169340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa327e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32815e800 con 0x7fa32806e190 2024-09-10T10:43:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa327e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3280cff30 con 0x7fa3280d4320 2024-09-10T10:43:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa327e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280d3230 con 0x7fa328166f50 2024-09-10T10:43:44.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa31e000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa32806e190 0x7fa3280d2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34010/0 (socket says 172.21.15.28:34010) 2024-09-10T10:43:44.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.604+0000 7fa31e000640 1 -- 172.21.15.28:0/248674911 learned_addr learned my addr 172.21.15.28:0/248674911 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa327400640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa328169340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 763182358 0 0) 0x7fa32815e800 con 0x7fa32806e190 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa30c003520 con 0x7fa32806e190 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204263684 0 0) 0x7fa3280cff30 con 0x7fa3280d4320 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa32815e800 con 0x7fa3280d4320 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 290545816 0 0) 0x7fa30c003520 con 0x7fa32806e190 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 msgr2=0x7fa328169340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa328169340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 >> v1:172.21.15.138:6789/0 conn(0x7fa3280d4320 legacy=0x7fa3280d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3280d3510 con 0x7fa32806e190 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa310001770 con 0x7fa32806e190 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 >> v1:172.21.15.28:6789/0 conn(0x7fa32806e190 legacy=0x7fa3280d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30c003b70 0x7fa30c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa30c004800 0x7fa30c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa30c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280d3230 con 0x7fa30c004800 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c00baa0 con 0x7fa30c003b70 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c00bca0 con 0x7fa328166f50 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.605+0000 7fa31e000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa30c004800 0x7fa30c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa326a00640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30c003b70 0x7fa30c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa327400640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa30c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa31e000640 1 -- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 msgr2=0x7fa30c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa31e000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa30c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa31e000640 1 -- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30c003b70 msgr2=0x7fa30c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa31e000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30c003b70 0x7fa30c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa31e000640 1 -- 172.21.15.28:0/248674911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c003520 con 0x7fa30c004800 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa327400640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa328166f50 0x7fa30c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.606+0000 7fa326a00640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa30c003b70 0x7fa30c003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.607+0000 7fa31e000640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa30c004800 0x7fa30c004c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa310007f60 tx=0x7fa30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35374a612d0c1e92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.607+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310007b80 con 0x7fa30c004800 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.607+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310007270 con 0x7fa30c004800 2024-09-10T10:43:44.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.607+0000 7fa326000640 1 -- 172.21.15.28:0/248674911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100057a0 con 0x7fa30c004800 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 -- 172.21.15.28:0/248674911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa30c004800 msgr2=0x7fa30c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 --2- 172.21.15.28:0/248674911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa30c004800 0x7fa30c004c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa310007f60 tx=0x7fa30c005830 comp rx=0 tx=0).stop 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327400640 1 -- 172.21.15.28:0/248674911 reap_dead start 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 -- 172.21.15.28:0/248674911 shutdown_connections 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 -- 172.21.15.28:0/248674911 >> 172.21.15.28:0/248674911 conn(0x7fa32809d810 msgr2=0x7fa32808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 -- 172.21.15.28:0/248674911 shutdown_connections 2024-09-10T10:43:44.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.608+0000 7fa327e00640 1 -- 172.21.15.28:0/248674911 wait complete. 2024-09-10T10:43:44.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.609+0000 7fa327e00640 1 Processor -- start 2024-09-10T10:43:44.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 -- start start 2024-09-10T10:43:44.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa30c004800 0x7fa3280a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 0x7fa32815d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280b1070 con 0x7fa3280cdd80 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280d3ad0 con 0x7fa3281589b0 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.610+0000 7fa327e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280b11f0 con 0x7fa30c004800 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39808/0 (socket says 172.21.15.28:39808) 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 -- 172.21.15.28:0/41907456 learned_addr learned my addr 172.21.15.28:0/41907456 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa326a00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa30c004800 0x7fa3280a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa327400640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 0x7fa32815d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 -- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa30c004800 msgr2=0x7fa3280a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa30c004800 0x7fa3280a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 -- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 msgr2=0x7fa32815d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 0x7fa32815d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 -- 172.21.15.28:0/41907456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3280ce870 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa327400640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 0x7fa32815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.611+0000 7fa31e000640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c1d0 tx=0x7fa310001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.612+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310041020 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.612+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310038040 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.612+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100059a0 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.612+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32809f2b0 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.612+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32809e290 con 0x7fa3280cdd80 2024-09-10T10:43:44.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.613+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3280d4700 con 0x7fa3280cdd80 2024-09-10T10:43:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.614+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa310044020 con 0x7fa3280cdd80 2024-09-10T10:43:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.615+0000 7fa31f400640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 0x7fa30406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.615+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa3100aa870 con 0x7fa3280cdd80 2024-09-10T10:43:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.615+0000 7fa326a00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 0x7fa30406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.616+0000 7fa326a00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 0x7fa30406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3280cfc30 tx=0x7fa318009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.619+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa31004c020 con 0x7fa3280cdd80 2024-09-10T10:43:44.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.805+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa3280690e0 con 0x7fa3280cdd80 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.806+0000 7fa31f400640 1 -- 172.21.15.28:0/41907456 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fa310079440 con 0x7fa3280cdd80 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:43:44.807 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:43:44.808 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:43:44.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 msgr2=0x7fa30406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 0x7fa30406e320 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3280cfc30 tx=0x7fa318009040 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 msgr2=0x7fa32809ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c1d0 tx=0x7fa310001a70 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 shutdown_connections 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30406be60 0x7fa30406e320 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3281589b0 0x7fa32815d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280cdd80 0x7fa32809ed70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 --2- 172.21.15.28:0/41907456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa30c004800 0x7fa3280a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 >> 172.21.15.28:0/41907456 conn(0x7fa32809d810 msgr2=0x7fa3280b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 shutdown_connections 2024-09-10T10:43:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.808+0000 7fa327e00640 1 -- 172.21.15.28:0/41907456 wait complete. 2024-09-10T10:43:44.824 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:43:44.824 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:43:44.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.982+0000 7f8426600640 1 Processor -- start 2024-09-10T10:43:44.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.982+0000 7f8426600640 1 -- start start 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.983+0000 7f8426600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f84200d0940 0x7f842015d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.983+0000 7f8426600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 0x7f84200cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8426600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84200d0730 con 0x7f84200cff50 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8426600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f84200cda90 con 0x7f84200d0940 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8426600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84200cd1a0 con 0x7f842015c500 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f841fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f84200cff50 0x7f84200d0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34016/0 (socket says 172.21.15.28:34016) 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f841fe00640 1 -- 172.21.15.28:0/3997772699 learned_addr learned my addr 172.21.15.28:0/3997772699 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f841f400640 1 --2- 172.21.15.28:0/3997772699 >> v2:172.21.15.138:3300/0 conn(0x7f84200d0940 0x7f842015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8417e00640 1 --2- 172.21.15.28:0/3997772699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 0x7f84200cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f841ea00640 1 -- 172.21.15.28:0/3997772699 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3909401643 0 0) 0x7f84200d0730 con 0x7f84200cff50 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f841ea00640 1 -- 172.21.15.28:0/3997772699 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8404003520 con 0x7f84200cff50 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8417e00640 1 -- 172.21.15.28:0/3997772699 >> v2:172.21.15.138:3300/0 conn(0x7f84200d0940 msgr2=0x7f842015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8417e00640 1 --2- 172.21.15.28:0/3997772699 >> v2:172.21.15.138:3300/0 conn(0x7f84200d0940 0x7f842015d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8417e00640 1 -- 172.21.15.28:0/3997772699 >> v1:172.21.15.28:6789/0 conn(0x7f84200cff50 legacy=0x7f84200d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.984+0000 7f8417e00640 1 -- 172.21.15.28:0/3997772699 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84200cd440 con 0x7f842015c500 2024-09-10T10:43:44.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.985+0000 7f8417e00640 1 --2- 172.21.15.28:0/3997772699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 0x7f84200cdf20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f84080097d0 tx=0x7f840802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74313290c34f225 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f841ea00640 1 -- 172.21.15.28:0/3997772699 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408037070 con 0x7f842015c500 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f841ea00640 1 -- 172.21.15.28:0/3997772699 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8408032440 con 0x7f842015c500 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f841ea00640 1 -- 172.21.15.28:0/3997772699 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84080327f0 con 0x7f842015c500 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 -- 172.21.15.28:0/3997772699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 msgr2=0x7f84200cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 --2- 172.21.15.28:0/3997772699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 0x7f84200cdf20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f84080097d0 tx=0x7f840802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 -- 172.21.15.28:0/3997772699 shutdown_connections 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 --2- 172.21.15.28:0/3997772699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f842015c500 0x7f84200cdf20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 --2- 172.21.15.28:0/3997772699 >> v2:172.21.15.138:3300/0 conn(0x7f84200d0940 0x7f842015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.986+0000 7f8426600640 1 -- 172.21.15.28:0/3997772699 >> 172.21.15.28:0/3997772699 conn(0x7f84200d2380 msgr2=0x7f84200d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.987+0000 7f8426600640 1 -- 172.21.15.28:0/3997772699 shutdown_connections 2024-09-10T10:43:44.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.987+0000 7f8426600640 1 -- 172.21.15.28:0/3997772699 wait complete. 2024-09-10T10:43:44.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.988+0000 7f8426600640 1 Processor -- start 2024-09-10T10:43:44.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.988+0000 7f8426600640 1 -- start start 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 0x7f842014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84200d0940 0x7f84200c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f842009aa30 con 0x7f842015c500 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84200d0400 con 0x7f84200d0940 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f8426600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f842009abb0 con 0x7f84200cff50 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.989+0000 7f841fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39822/0 (socket says 172.21.15.28:39822) 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 -- 172.21.15.28:0/4201171134 learned_addr learned my addr 172.21.15.28:0/4201171134 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841f400640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 0x7f842014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f8417e00640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84200d0940 0x7f84200c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 -- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 msgr2=0x7f842014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 0x7f842014b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 -- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84200d0940 msgr2=0x7f84200c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84200d0940 0x7f84200c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 -- 172.21.15.28:0/4201171134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842014bcf0 con 0x7f842015c500 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841f400640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 0x7f842014b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:44.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f841fe00640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f841800cb40 tx=0x7f841800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418017070 con 0x7f842015c500 2024-09-10T10:43:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.990+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841800ac40 con 0x7f842015c500 2024-09-10T10:43:44.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.991+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8418012400 con 0x7f842015c500 2024-09-10T10:43:44.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.991+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84200b55b0 con 0x7f842015c500 2024-09-10T10:43:44.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.991+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84200b5e50 con 0x7f842015c500 2024-09-10T10:43:44.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.992+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84200b3b00 con 0x7f842015c500 2024-09-10T10:43:44.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.992+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f841801a070 con 0x7f842015c500 2024-09-10T10:43:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.994+0000 7f8417400640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.994+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f841808a990 con 0x7f842015c500 2024-09-10T10:43:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.994+0000 7f841f400640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:44.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.995+0000 7f841f400640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 0x7f83fc06e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8410009b90 tx=0x7f8410008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:44.998+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f842014bcf0 con 0x7f842015c500 2024-09-10T10:43:45.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.182+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f84200d3290 con 0x7f842015c500 2024-09-10T10:43:45.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.494+0000 7f8417400640 1 -- 172.21.15.28:0/4201171134 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f84180595f0 con 0x7f842015c500 2024-09-10T10:43:45.495 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:43:45.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 msgr2=0x7f83fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 0x7f83fc06e2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8410009b90 tx=0x7f8410008040 comp rx=0 tx=0).stop 2024-09-10T10:43:45.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 msgr2=0x7f842015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f841800cb40 tx=0x7f841800a060 comp rx=0 tx=0).stop 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 shutdown_connections 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f83fc06be30 0x7f83fc06e2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f842015c500 0x7f842015f2b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84200d0940 0x7f84200c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 --2- 172.21.15.28:0/4201171134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84200cff50 0x7f842014b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 >> 172.21.15.28:0/4201171134 conn(0x7f84200d2380 msgr2=0x7f84200d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 shutdown_connections 2024-09-10T10:43:45.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.497+0000 7f8426600640 1 -- 172.21.15.28:0/4201171134 wait complete. 2024-09-10T10:43:45.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T10:43:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.683+0000 7f43e4800640 1 Processor -- start 2024-09-10T10:43:45.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.684+0000 7f43e4800640 1 -- start start 2024-09-10T10:43:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e4800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e4800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43dc0adce0 0x7f43dc0ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e4800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43dc0ae670 con 0x7f43dc15e2e0 2024-09-10T10:43:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e4800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f43dc0aec30 con 0x7f43dc0adce0 2024-09-10T10:43:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e4800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc067ff0 con 0x7f43dc067ae0 2024-09-10T10:43:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e1e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f43dc15e2e0 0x7f43dc15f3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34018/0 (socket says 172.21.15.28:34018) 2024-09-10T10:43:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e1e00640 1 -- 172.21.15.28:0/3187736534 learned_addr learned my addr 172.21.15.28:0/3187736534 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e2800640 1 --2- 172.21.15.28:0/3187736534 >> v2:172.21.15.28:3300/0 conn(0x7f43dc0adce0 0x7f43dc0ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.685+0000 7f43e1400640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e0a00640 1 -- 172.21.15.28:0/3187736534 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437612727 0 0) 0x7f43dc0ae670 con 0x7f43dc15e2e0 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e2800640 1 -- 172.21.15.28:0/3187736534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 msgr2=0x7f43dc0b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e2800640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc0b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e2800640 1 -- 172.21.15.28:0/3187736534 >> v1:172.21.15.28:6789/0 conn(0x7f43dc15e2e0 legacy=0x7f43dc15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e2800640 1 -- 172.21.15.28:0/3187736534 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc0af390 con 0x7f43dc0adce0 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.686+0000 7f43e1400640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e2800640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc0adce0 0x7f43dc0ae130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009d90 tx=0x7f43d802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17ca1d274d8f621 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e0a00640 1 -- 172.21.15.28:0/3187736534 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8007740 con 0x7f43dc0adce0 2024-09-10T10:43:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e0a00640 1 -- 172.21.15.28:0/3187736534 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d80078e0 con 0x7f43dc0adce0 2024-09-10T10:43:45.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e0a00640 1 -- 172.21.15.28:0/3187736534 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8007cb0 con 0x7f43dc0adce0 2024-09-10T10:43:45.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 -- 172.21.15.28:0/3187736534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc0adce0 msgr2=0x7f43dc0ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc0adce0 0x7f43dc0ae130 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009d90 tx=0x7f43d802a660 comp rx=0 tx=0).stop 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 -- 172.21.15.28:0/3187736534 shutdown_connections 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc0adce0 0x7f43dc0ae130 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 --2- 172.21.15.28:0/3187736534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc0b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.687+0000 7f43e4800640 1 -- 172.21.15.28:0/3187736534 >> 172.21.15.28:0/3187736534 conn(0x7f43dc073590 msgr2=0x7f43dc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.688+0000 7f43e4800640 1 -- 172.21.15.28:0/3187736534 shutdown_connections 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.688+0000 7f43e4800640 1 -- 172.21.15.28:0/3187736534 wait complete. 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.689+0000 7f43e4800640 1 Processor -- start 2024-09-10T10:43:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.689+0000 7f43e4800640 1 -- start start 2024-09-10T10:43:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 0x7f43dc1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0afc30 con 0x7f43dc15e2e0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0af950 con 0x7f43dc0adce0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e4800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc15e6c0 con 0x7f43dc067ae0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39850/0 (socket says 172.21.15.28:39850) 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.690+0000 7f43e2800640 1 -- 172.21.15.28:0/3391161290 learned_addr learned my addr 172.21.15.28:0/3391161290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 -- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 msgr2=0x7f43dc1497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e1e00640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e1400640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 0x7f43dc1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc1497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 -- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 msgr2=0x7f43dc1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 0x7f43dc1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 -- 172.21.15.28:0/3391161290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc15ea20 con 0x7f43dc15e2e0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e1400640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 0x7f43dc1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43e2800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009eb0 tx=0x7f43d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8040070 con 0x7f43dc15e2e0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43d8031bc0 con 0x7f43dc15e2e0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.691+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8009450 con 0x7f43dc15e2e0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.692+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc15ed00 con 0x7f43dc15e2e0 2024-09-10T10:43:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.692+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc15f2c0 con 0x7f43dc15e2e0 2024-09-10T10:43:45.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.693+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43dc15f4c0 con 0x7f43dc15e2e0 2024-09-10T10:43:45.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.694+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43d8039040 con 0x7f43dc15e2e0 2024-09-10T10:43:45.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.694+0000 7f43d6000640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 0x7f43c006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:45.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.695+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f43d80abb80 con 0x7f43dc15e2e0 2024-09-10T10:43:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.695+0000 7f43e1e00640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 0x7f43c006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.696+0000 7f43e1e00640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 0x7f43c006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000f60 tx=0x7f43d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.699+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43dc15f4c0 con 0x7f43dc15e2e0 2024-09-10T10:43:45.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:45.886+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f43dc0b0150 con 0x7f43dc15e2e0 2024-09-10T10:43:46.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.497+0000 7f43d6000640 1 -- 172.21.15.28:0/3391161290 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7f43d804b070 con 0x7f43dc15e2e0 2024-09-10T10:43:46.499 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.500+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 msgr2=0x7f43c006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.500+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 0x7f43c006e320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000f60 tx=0x7f43d0009040 comp rx=0 tx=0).stop 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.500+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 msgr2=0x7f43dc158ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.500+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009eb0 tx=0x7f43d8004a40 comp rx=0 tx=0).stop 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 shutdown_connections 2024-09-10T10:43:46.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43c006be60 0x7f43c006e320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43dc15e2e0 0x7f43dc158ef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43dc0adce0 0x7f43dc1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 --2- 172.21.15.28:0/3391161290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43dc067ae0 0x7f43dc1497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 >> 172.21.15.28:0/3391161290 conn(0x7f43dc073590 msgr2=0x7f43dc0b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 shutdown_connections 2024-09-10T10:43:46.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.501+0000 7f43e4800640 1 -- 172.21.15.28:0/3391161290 wait complete. 2024-09-10T10:43:46.536 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:43:46.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.699+0000 7fdf8cc00640 1 Processor -- start 2024-09-10T10:43:46.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.699+0000 7fdf8cc00640 1 -- start start 2024-09-10T10:43:46.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf8cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf880d02d0 0x7fdf8806e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf8cc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880d3140 con 0x7fdf880d0a90 2024-09-10T10:43:46.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf8cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf88163170 con 0x7fdf880d0e70 2024-09-10T10:43:46.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf8cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880d12c0 con 0x7fdf880d02d0 2024-09-10T10:43:46.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf87400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdf880d0a90 0x7fdf88162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34022/0 (socket says 172.21.15.28:34022) 2024-09-10T10:43:46.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.700+0000 7fdf87400640 1 -- 172.21.15.28:0/2424290644 learned_addr learned my addr 172.21.15.28:0/2424290644 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 838256935 0 0) 0x7fdf78001110 con 0x7fdf880d0a90 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf6c003520 con 0x7fdf880d0a90 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86a00640 1 --2- 172.21.15.28:0/2424290644 >> v2:172.21.15.138:3300/0 conn(0x7fdf880d02d0 0x7fdf8806e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2931769233 0 0) 0x7fdf6c003520 con 0x7fdf880d0a90 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 >> v2:172.21.15.138:3300/0 conn(0x7fdf880d02d0 msgr2=0x7fdf8806e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 --2- 172.21.15.28:0/2424290644 >> v2:172.21.15.138:3300/0 conn(0x7fdf880d02d0 0x7fdf8806e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 >> v1:172.21.15.138:6789/0 conn(0x7fdf880d0e70 legacy=0x7fdf88162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8806dd20 con 0x7fdf880d0a90 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fdf78001840 con 0x7fdf880d0a90 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 >> v1:172.21.15.28:6789/0 conn(0x7fdf880d0a90 legacy=0x7fdf88162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf6c003a00 0x7fdf6c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c005210 0x7fdf6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c00c130 con 0x7fdf6c003a00 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c00c2e0 con 0x7fdf6c004800 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.701+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c00c4e0 con 0x7fdf6c005210 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf87400640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf6c003a00 0x7fdf6c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf87e00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c005210 0x7fdf6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 -- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c005210 msgr2=0x7fdf6c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:46.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c005210 0x7fdf6c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 -- 172.21.15.28:0/2424290644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf6c003a00 msgr2=0x7fdf6c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf6c003a00 0x7fdf6c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf86a00640 1 -- 172.21.15.28:0/2424290644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c003520 con 0x7fdf6c004800 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf87e00640 1 -- 172.21.15.28:0/2424290644 reap_dead start 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.702+0000 7fdf87400640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf6c003a00 0x7fdf6c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:46.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.703+0000 7fdf86a00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf8806ef00 tx=0x7fdf6c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a66c337ef9a3979c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:46.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.703+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74036070 con 0x7fdf6c004800 2024-09-10T10:43:46.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.703+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74031400 con 0x7fdf6c004800 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.703+0000 7fdf86000640 1 -- 172.21.15.28:0/2424290644 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74009690 con 0x7fdf6c004800 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 -- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 msgr2=0x7fdf6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf8806ef00 tx=0x7fdf6c005830 comp rx=0 tx=0).stop 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 -- 172.21.15.28:0/2424290644 shutdown_connections 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/2424290644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf6c004800 0x7fdf6c004c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 -- 172.21.15.28:0/2424290644 >> 172.21.15.28:0/2424290644 conn(0x7fdf8808bfc0 msgr2=0x7fdf880d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 -- 172.21.15.28:0/2424290644 shutdown_connections 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.704+0000 7fdf8cc00640 1 -- 172.21.15.28:0/2424290644 wait complete. 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.706+0000 7fdf8cc00640 1 Processor -- start 2024-09-10T10:43:46.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.706+0000 7fdf8cc00640 1 -- start start 2024-09-10T10:43:46.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c004800 0x7fdf881f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 0x7fdf880b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880d17d0 con 0x7fdf881f79e0 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880d14f0 con 0x7fdf880cd630 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf8cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf880d5180 con 0x7fdf6c004800 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39868/0 (socket says 172.21.15.28:39868) 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf87e00640 1 -- 172.21.15.28:0/1808967335 learned_addr learned my addr 172.21.15.28:0/1808967335 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf87400640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c004800 0x7fdf881f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.707+0000 7fdf86a00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 0x7fdf880b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 -- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c004800 msgr2=0x7fdf881f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c004800 0x7fdf881f3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 -- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 msgr2=0x7fdf880b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 0x7fdf880b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 -- 172.21.15.28:0/1808967335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf881f7200 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf86a00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 0x7fdf880b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf87e00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf880d5440 tx=0x7fdf8000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80003b80 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf80018070 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800050d0 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf881f44f0 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.708+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881f4ab0 con 0x7fdf881f79e0 2024-09-10T10:43:46.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.709+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8806e9c0 con 0x7fdf881f79e0 2024-09-10T10:43:46.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.710+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf80005270 con 0x7fdf881f79e0 2024-09-10T10:43:46.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.712+0000 7fdf7f400640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:46.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.712+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fdf8008ace0 con 0x7fdf881f79e0 2024-09-10T10:43:46.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.712+0000 7fdf87400640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:46.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.713+0000 7fdf87400640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf780012f0 tx=0x7fdf78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:46.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.717+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf8001a310 con 0x7fdf881f79e0 2024-09-10T10:43:46.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:46.938+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdf8815f090 con 0x7fdf881f79e0 2024-09-10T10:43:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.525+0000 7fdf7f400640 1 -- 172.21.15.28:0/1808967335 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fdf80059650 con 0x7fdf881f79e0 2024-09-10T10:43:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.537+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 msgr2=0x7fdf6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.537+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf780012f0 tx=0x7fdf78009040 comp rx=0 tx=0).stop 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.537+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 msgr2=0x7fdf880cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.537+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf880d5440 tx=0x7fdf8000fd40 comp rx=0 tx=0).stop 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 shutdown_connections 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf780012f0 tx=0x7fdf78009040 comp rx=0 tx=0).stop 2024-09-10T10:43:47.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf881f79e0 0x7fdf880cb410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf880cd630 0x7fdf880b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1808967335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf6c004800 0x7fdf881f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.538+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 >> 172.21.15.28:0/1808967335 conn(0x7fdf8808bfc0 msgr2=0x7fdf880b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:47.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.541+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 shutdown_connections 2024-09-10T10:43:47.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.541+0000 7fdf8cc00640 1 -- 172.21.15.28:0/1808967335 wait complete. 2024-09-10T10:43:47.573 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T10:43:47.574 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T10:43:47.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.734+0000 7f4296200640 1 Processor -- start 2024-09-10T10:43:47.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.734+0000 7f4296200640 1 -- start start 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.735+0000 7f4296200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4290162340 0x7f429006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f4296200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f429015ff70 con 0x7f42900cf280 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f4296200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42900d4d10 con 0x7f4290162e30 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f4296200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f42901632a0 con 0x7f4290162340 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4290162340 0x7f429006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4290162340 0x7f429006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39878/0 (socket says 172.21.15.28:39878) 2024-09-10T10:43:47.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 -- 172.21.15.28:0/3256091142 learned_addr learned my addr 172.21.15.28:0/3256091142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:47.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 -- 172.21.15.28:0/3256091142 >> v1:172.21.15.138:6789/0 conn(0x7f4290162e30 legacy=0x7f42900d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 -- 172.21.15.28:0/3256091142 >> v1:172.21.15.28:6789/0 conn(0x7f42900cf280 legacy=0x7f4290162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428f400640 1 -- 172.21.15.28:0/3256091142 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42900d4ff0 con 0x7f4290162340 2024-09-10T10:43:47.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.736+0000 7f428ea00640 1 -- 172.21.15.28:0/3256091142 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2756146257 0 0) 0x7f429015ff70 con 0x7f42900cf280 2024-09-10T10:43:47.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.737+0000 7f428f400640 1 --2- 172.21.15.28:0/3256091142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162340 0x7f429006d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4280009870 tx=0x7f428002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490fae67a4c8b1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.737+0000 7f428ea00640 1 -- 172.21.15.28:0/3256091142 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280037070 con 0x7f4290162340 2024-09-10T10:43:47.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.737+0000 7f428ea00640 1 -- 172.21.15.28:0/3256091142 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4280032440 con 0x7f4290162340 2024-09-10T10:43:47.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f428ea00640 1 -- 172.21.15.28:0/3256091142 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280032810 con 0x7f4290162340 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 -- 172.21.15.28:0/3256091142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162340 msgr2=0x7f429006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 --2- 172.21.15.28:0/3256091142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162340 0x7f429006d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4280009870 tx=0x7f428002a270 comp rx=0 tx=0).stop 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 -- 172.21.15.28:0/3256091142 shutdown_connections 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 --2- 172.21.15.28:0/3256091142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162340 0x7f429006d740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 -- 172.21.15.28:0/3256091142 >> 172.21.15.28:0/3256091142 conn(0x7f429008bfc0 msgr2=0x7f42900d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 -- 172.21.15.28:0/3256091142 shutdown_connections 2024-09-10T10:43:47.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.738+0000 7f4296200640 1 -- 172.21.15.28:0/3256091142 wait complete. 2024-09-10T10:43:47.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.740+0000 7f4296200640 1 Processor -- start 2024-09-10T10:43:47.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.740+0000 7f4296200640 1 -- start start 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42900cf280 0x7f42901559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 0x7f42900b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42900d52d0 con 0x7f4290162e30 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f429006de90 con 0x7f42900cf280 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4296200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f42900cf6c0 con 0x7f4290162340 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39890/0 (socket says 172.21.15.28:39890) 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 -- 172.21.15.28:0/2963612958 learned_addr learned my addr 172.21.15.28:0/2963612958 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4287e00640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 0x7f42900b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428f400640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42900cf280 0x7f42901559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 -- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 msgr2=0x7f42900b8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 0x7f42900b8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 -- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42900cf280 msgr2=0x7f42901559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42900cf280 0x7f42901559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f428fe00640 1 -- 172.21.15.28:0/2963612958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4290155f30 con 0x7f4290162e30 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.741+0000 7f4287e00640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 0x7f42900b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f428fe00640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f428800c6e0 tx=0x7f428800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288003b70 con 0x7f4290162e30 2024-09-10T10:43:47.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4288003d30 con 0x7f4290162e30 2024-09-10T10:43:47.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288005310 con 0x7f4290162e30 2024-09-10T10:43:47.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42900b4060 con 0x7f4290162e30 2024-09-10T10:43:47.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.742+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42900b4620 con 0x7f4290162e30 2024-09-10T10:43:47.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.743+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42900b4950 con 0x7f4290162e30 2024-09-10T10:43:47.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.744+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f42880054b0 con 0x7f4290162e30 2024-09-10T10:43:47.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.745+0000 7f4287400640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 0x7f426c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:47.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.745+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f428808ac20 con 0x7f4290162e30 2024-09-10T10:43:47.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.745+0000 7f428f400640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 0x7f426c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:47.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.746+0000 7f428f400640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 0x7f426c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4280034000 tx=0x7f428002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:47.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.749+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4290155f30 con 0x7f4290162e30 2024-09-10T10:43:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:47.945+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f429006d740 con 0x7f4290162e30 2024-09-10T10:43:48.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.514+0000 7f4287400640 1 -- 172.21.15.28:0/2963612958 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7f42880595c0 con 0x7f4290162e30 2024-09-10T10:43:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 msgr2=0x7f426c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 0x7f426c06e2f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4280034000 tx=0x7f428002e000 comp rx=0 tx=0).stop 2024-09-10T10:43:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 msgr2=0x7f42900b6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f428800c6e0 tx=0x7f428800cba0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 shutdown_connections 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f426c06be30 0x7f426c06e2f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4290162e30 0x7f42900b6230 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4290162340 0x7f42900b8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 --2- 172.21.15.28:0/2963612958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42900cf280 0x7f42901559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 >> 172.21.15.28:0/2963612958 conn(0x7f429008bfc0 msgr2=0x7f42900d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:48.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.518+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 shutdown_connections 2024-09-10T10:43:48.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.519+0000 7f4296200640 1 -- 172.21.15.28:0/2963612958 wait complete. 2024-09-10T10:43:48.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:43:48.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.707+0000 7fe3da600640 1 Processor -- start 2024-09-10T10:43:48.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.707+0000 7fe3da600640 1 -- start start 2024-09-10T10:43:48.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3d40d2950 0x7fe3d40d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3d40b56a0 0x7fe3d40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2d30 0x7fe3d40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d40b1c00 con 0x7fe3d40d2950 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d40b1030 con 0x7fe3d40b56a0 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.708+0000 7fe3da600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40b12d0 con 0x7fe3d40d2d30 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3d40d2950 0x7fe3d40d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3d40d2950 0x7fe3d40d4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39898/0 (socket says 172.21.15.28:39898) 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 -- 172.21.15.28:0/1628811136 learned_addr learned my addr 172.21.15.28:0/1628811136 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d3e00640 1 --2- 172.21.15.28:0/1628811136 >> v2:172.21.15.138:3300/0 conn(0x7fe3d40b56a0 0x7fe3d40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9c00640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2d30 0x7fe3d40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 -- 172.21.15.28:0/1628811136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2d30 msgr2=0x7fe3d40b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2d30 0x7fe3d40b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 -- 172.21.15.28:0/1628811136 >> v2:172.21.15.138:3300/0 conn(0x7fe3d40b56a0 msgr2=0x7fe3d40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:48.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 --2- 172.21.15.28:0/1628811136 >> v2:172.21.15.138:3300/0 conn(0x7fe3d40b56a0 0x7fe3d40b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.709+0000 7fe3d9200640 1 -- 172.21.15.28:0/1628811136 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40b15b0 con 0x7fe3d40d2950 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.710+0000 7fe3d9200640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40d2950 0x7fe3d40d4c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d408dcf0 tx=0x7fe3c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b1d2eb5c583e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.710+0000 7fe3d3400640 1 -- 172.21.15.28:0/1628811136 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8037070 con 0x7fe3d40d2950 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.710+0000 7fe3d3400640 1 -- 172.21.15.28:0/1628811136 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8032440 con 0x7fe3d40d2950 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.710+0000 7fe3d3400640 1 -- 172.21.15.28:0/1628811136 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8032810 con 0x7fe3d40d2950 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 -- 172.21.15.28:0/1628811136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40d2950 msgr2=0x7fe3d40d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40d2950 0x7fe3d40d4c40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d408dcf0 tx=0x7fe3c802a140 comp rx=0 tx=0).stop 2024-09-10T10:43:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 -- 172.21.15.28:0/1628811136 shutdown_connections 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2d30 0x7fe3d40b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 --2- 172.21.15.28:0/1628811136 >> v2:172.21.15.138:3300/0 conn(0x7fe3d40b56a0 0x7fe3d40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 --2- 172.21.15.28:0/1628811136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40d2950 0x7fe3d40d4c40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 -- 172.21.15.28:0/1628811136 >> 172.21.15.28:0/1628811136 conn(0x7fe3d408bfc0 msgr2=0x7fe3d40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 -- 172.21.15.28:0/1628811136 shutdown_connections 2024-09-10T10:43:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.711+0000 7fe3da600640 1 -- 172.21.15.28:0/1628811136 wait complete. 2024-09-10T10:43:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.713+0000 7fe3da600640 1 Processor -- start 2024-09-10T10:43:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.713+0000 7fe3da600640 1 -- start start 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2950 0x7fe3d40c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 0x7fe3d415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40b46d0 con 0x7fe3d40b56a0 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d40d5180 con 0x7fe3d40d2d30 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3da600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d40b4850 con 0x7fe3d40d2950 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39910/0 (socket says 172.21.15.28:39910) 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 -- 172.21.15.28:0/3556415167 learned_addr learned my addr 172.21.15.28:0/3556415167 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9c00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 0x7fe3d415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d3e00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2950 0x7fe3d40c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 -- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2950 msgr2=0x7fe3d40c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2950 0x7fe3d40c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 -- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 msgr2=0x7fe3d415b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 0x7fe3d415b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9200640 1 -- 172.21.15.28:0/3556415167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d40c9410 con 0x7fe3d40b56a0 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.714+0000 7fe3d9c00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 0x7fe3d415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:48.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3d9200640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802a180 tx=0x7fe3c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8041020 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c80306c0 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d40c96f0 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8037040 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.715+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d40c9cb0 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.716+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d40d4c40 con 0x7fe3d40b56a0 2024-09-10T10:43:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.717+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3c8044020 con 0x7fe3d40b56a0 2024-09-10T10:43:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.718+0000 7fe3d0c00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 0x7fe3b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.718+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7fe3c80ab7a0 con 0x7fe3d40b56a0 2024-09-10T10:43:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.718+0000 7fe3d3e00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 0x7fe3b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.719+0000 7fe3d3e00640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 0x7fe3b406e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40d13d0 tx=0x7fe3c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:48.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.722+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3c804b030 con 0x7fe3d40b56a0 2024-09-10T10:43:48.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:48.906+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3d40b29d0 con 0x7fe3d40b56a0 2024-09-10T10:43:50.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.542+0000 7fe3d0c00640 1 -- 172.21.15.28:0/3556415167 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7fe3c807a010 con 0x7fe3d40b56a0 2024-09-10T10:43:50.544 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data_ec' created 2024-09-10T10:43:50.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 msgr2=0x7fe3b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 0x7fe3b406e320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40d13d0 tx=0x7fe3c0008040 comp rx=0 tx=0).stop 2024-09-10T10:43:50.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 msgr2=0x7fe3d4157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c802a180 tx=0x7fe3c80046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 shutdown_connections 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be60 0x7fe3b406e320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d40d2d30 0x7fe3d415b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d40d2950 0x7fe3d40c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 --2- 172.21.15.28:0/3556415167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d40b56a0 0x7fe3d4157b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.546+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 >> 172.21.15.28:0/3556415167 conn(0x7fe3d408bfc0 msgr2=0x7fe3d40d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.547+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 shutdown_connections 2024-09-10T10:43:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.547+0000 7fe3da600640 1 -- 172.21.15.28:0/3556415167 wait complete. 2024-09-10T10:43:50.574 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T10:43:50.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.737+0000 7f02bf200640 1 Processor -- start 2024-09-10T10:43:50.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.737+0000 7f02bf200640 1 -- start start 2024-09-10T10:43:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bf200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 0x7f02b80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bf200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 0x7f02b80d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bf200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02b80cf1e0 con 0x7f02b80d48c0 2024-09-10T10:43:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bf200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02b80d3690 con 0x7f02b80d2500 2024-09-10T10:43:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bf200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80d2a80 con 0x7f02b815e350 2024-09-10T10:43:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02be800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f02b80d48c0 0x7f02b80d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34052/0 (socket says 172.21.15.28:34052) 2024-09-10T10:43:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02be800640 1 -- 172.21.15.28:0/4271769252 learned_addr learned my addr 172.21.15.28:0/4271769252 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bde00640 1 --2- 172.21.15.28:0/4271769252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 0x7f02b80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.738+0000 7f02bd400640 1 --2- 172.21.15.28:0/4271769252 >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 0x7f02b80d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bca00640 1 -- 172.21.15.28:0/4271769252 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893136784 0 0) 0x7f02b80cf1e0 con 0x7f02b80d48c0 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bca00640 1 -- 172.21.15.28:0/4271769252 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02a0003520 con 0x7f02b80d48c0 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bde00640 1 -- 172.21.15.28:0/4271769252 >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 msgr2=0x7f02b80d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bde00640 1 --2- 172.21.15.28:0/4271769252 >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 0x7f02b80d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bde00640 1 -- 172.21.15.28:0/4271769252 >> v1:172.21.15.28:6789/0 conn(0x7f02b80d48c0 legacy=0x7f02b80d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bde00640 1 -- 172.21.15.28:0/4271769252 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b80d2d60 con 0x7f02b815e350 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.739+0000 7f02bd400640 1 --2- 172.21.15.28:0/4271769252 >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 0x7f02b80d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.740+0000 7f02bde00640 1 --2- 172.21.15.28:0/4271769252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 0x7f02b80d1fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097a0 tx=0x7f02ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8943380d13502e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.740+0000 7f02bca00640 1 -- 172.21.15.28:0/4271769252 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac037070 con 0x7f02b815e350 2024-09-10T10:43:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.740+0000 7f02bca00640 1 -- 172.21.15.28:0/4271769252 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ac032440 con 0x7f02b815e350 2024-09-10T10:43:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.740+0000 7f02bca00640 1 -- 172.21.15.28:0/4271769252 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ac032810 con 0x7f02b815e350 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 -- 172.21.15.28:0/4271769252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 msgr2=0x7f02b80d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 --2- 172.21.15.28:0/4271769252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 0x7f02b80d1fc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097a0 tx=0x7f02ac02a180 comp rx=0 tx=0).stop 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 -- 172.21.15.28:0/4271769252 shutdown_connections 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 --2- 172.21.15.28:0/4271769252 >> v2:172.21.15.138:3300/0 conn(0x7f02b80d2500 0x7f02b80d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 --2- 172.21.15.28:0/4271769252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b815e350 0x7f02b80d1fc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 -- 172.21.15.28:0/4271769252 >> 172.21.15.28:0/4271769252 conn(0x7f02b809d710 msgr2=0x7f02b808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 -- 172.21.15.28:0/4271769252 shutdown_connections 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.741+0000 7f02bf200640 1 -- 172.21.15.28:0/4271769252 wait complete. 2024-09-10T10:43:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.743+0000 7f02bf200640 1 Processor -- start 2024-09-10T10:43:50.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.743+0000 7f02bf200640 1 -- start start 2024-09-10T10:43:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 0x7f02b81603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b80d48c0 0x7f02b80b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 0x7f02b81508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80b3620 con 0x7f02b815e350 2024-09-10T10:43:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80d3320 con 0x7f02b80d2500 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bf200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02b815e730 con 0x7f02b80d48c0 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 0x7f02b81508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 0x7f02b81603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 0x7f02b81603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36024/0 (socket says 172.21.15.28:36024) 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bd400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b80d48c0 0x7f02b80b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02bde00640 1 -- 172.21.15.28:0/3988052538 learned_addr learned my addr 172.21.15.28:0/3988052538 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 -- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b80d48c0 msgr2=0x7f02b80b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b80d48c0 0x7f02b80b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 -- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 msgr2=0x7f02b81603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 0x7f02b81603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.744+0000 7f02be800640 1 -- 172.21.15.28:0/3988052538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b8160ab0 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02be800640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 0x7f02b81508c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02b400cad0 tx=0x7f02b400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4017070 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02b400ab70 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4012400 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b8160d90 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.745+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b8161350 con 0x7f02b815e350 2024-09-10T10:43:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.746+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b80ba370 con 0x7f02b815e350 2024-09-10T10:43:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.747+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02b401a070 con 0x7f02b815e350 2024-09-10T10:43:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.748+0000 7f02b2000640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 0x7f029c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.748+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f02b408add0 con 0x7f02b815e350 2024-09-10T10:43:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.748+0000 7f02bde00640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 0x7f029c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.749+0000 7f02bde00640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 0x7f029c06e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097a0 tx=0x7f02ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:50.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.753+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02b4058d70 con 0x7f02b815e350 2024-09-10T10:43:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:50.938+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f02b80b39f0 con 0x7f02b815e350 2024-09-10T10:43:51.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.545+0000 7f02b2000640 1 -- 172.21.15.28:0/3988052538 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7f02b405d070 con 0x7f02b815e350 2024-09-10T10:43:51.546 INFO:teuthology.orchestra.run.smithi028.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T10:43:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.548+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 msgr2=0x7f029c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.548+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 0x7f029c06e2f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097a0 tx=0x7f02ac02e000 comp rx=0 tx=0).stop 2024-09-10T10:43:51.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 msgr2=0x7f02b81508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 0x7f02b81508c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02b400cad0 tx=0x7f02b400ced0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 shutdown_connections 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f029c06be30 0x7f029c06e2f0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0097a0 tx=0x7f02ac02e000 comp rx=0 tx=0).stop 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02b815e350 0x7f02b81508c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02b80d48c0 0x7f02b80b9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 --2- 172.21.15.28:0/3988052538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b80d2500 0x7f02b81603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.549+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 >> 172.21.15.28:0/3988052538 conn(0x7f02b809d710 msgr2=0x7f02b80b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:51.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.550+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 shutdown_connections 2024-09-10T10:43:51.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.551+0000 7f02bf200640 1 -- 172.21.15.28:0/3988052538 wait complete. 2024-09-10T10:43:51.592 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T10:43:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.751+0000 7f964ac00640 1 Processor -- start 2024-09-10T10:43:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.751+0000 7f964ac00640 1 -- start start 2024-09-10T10:43:51.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96440d2920 0x7f96440d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96440cee50 con 0x7f964415e550 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f96440d4a50 con 0x7f96440d2920 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f96440cf180 con 0x7f96440b3400 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964a200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96440d2920 0x7f96440d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964a200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96440d2920 0x7f96440d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39926/0 (socket says 172.21.15.28:39926) 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f964a200640 1 -- 172.21.15.28:0/286378222 learned_addr learned my addr 172.21.15.28:0/286378222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.752+0000 7f9648e00640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.753+0000 7f964a200640 1 -- 172.21.15.28:0/286378222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 msgr2=0x7f96440d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.753+0000 7f964a200640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440d1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.753+0000 7f964a200640 1 -- 172.21.15.28:0/286378222 >> v1:172.21.15.138:6789/0 conn(0x7f964415e550 legacy=0x7f964406cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.753+0000 7f964a200640 1 -- 172.21.15.28:0/286378222 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96440d4ca0 con 0x7f96440d2920 2024-09-10T10:43:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.753+0000 7f9643e00640 1 -- 172.21.15.28:0/286378222 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584613250 0 0) 0x7f96440cee50 con 0x7f964415e550 2024-09-10T10:43:51.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964a200640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96440d2db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f963c009e50 tx=0x7f963c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87eaf6e48675df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:51.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f9643e00640 1 -- 172.21.15.28:0/286378222 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c036070 con 0x7f96440d2920 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f9643e00640 1 -- 172.21.15.28:0/286378222 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963c031600 con 0x7f96440d2920 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f9643e00640 1 -- 172.21.15.28:0/286378222 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0319d0 con 0x7f96440d2920 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 -- 172.21.15.28:0/286378222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 msgr2=0x7f96440d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96440d2db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f963c009e50 tx=0x7f963c02a5f0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 -- 172.21.15.28:0/286378222 shutdown_connections 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96440d2db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 --2- 172.21.15.28:0/286378222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440d1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.754+0000 7f964ac00640 1 -- 172.21.15.28:0/286378222 >> 172.21.15.28:0/286378222 conn(0x7f964408bfc0 msgr2=0x7f96440d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.755+0000 7f964ac00640 1 -- 172.21.15.28:0/286378222 shutdown_connections 2024-09-10T10:43:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.755+0000 7f964ac00640 1 -- 172.21.15.28:0/286378222 wait complete. 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.756+0000 7f964ac00640 1 Processor -- start 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.756+0000 7f964ac00640 1 -- start start 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f964415e550 0x7f96440b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f96440cf610 con 0x7f96440d2920 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96440d2450 con 0x7f964415e550 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f964ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f96440cf790 con 0x7f96440b3400 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f9648e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f9648e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39932/0 (socket says 172.21.15.28:39932) 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.757+0000 7f9648e00640 1 -- 172.21.15.28:0/4039591914 learned_addr learned my addr 172.21.15.28:0/4039591914 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9649800640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f964a200640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f964415e550 0x7f96440b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 -- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 msgr2=0x7f96440b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 -- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f964415e550 msgr2=0x7f96440b9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f964415e550 0x7f96440b9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 -- 172.21.15.28:0/4039591914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441f58a0 con 0x7f96440d2920 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9648e00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f963000e980 tx=0x7f963000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630017070 con 0x7f96440d2920 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f963000ca00 con 0x7f96440d2920 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963000cdd0 con 0x7f96440d2920 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.758+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96440b7600 con 0x7f96440d2920 2024-09-10T10:43:51.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.759+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96440b7bc0 con 0x7f96440d2920 2024-09-10T10:43:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.760+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964404ffb0 con 0x7f96440d2920 2024-09-10T10:43:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.760+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f963001a070 con 0x7f96440d2920 2024-09-10T10:43:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.761+0000 7f9641600640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 0x7f962406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.761+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7f963008b1c0 con 0x7f96440d2920 2024-09-10T10:43:51.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.762+0000 7f9649800640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 0x7f962406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:51.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.763+0000 7f9649800640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 0x7f962406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9638007fd0 tx=0x7f9638009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:51.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.766+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f964404ffb0 con 0x7f96440d2920 2024-09-10T10:43:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:51.986+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f96440d18f0 con 0x7f96440d2920 2024-09-10T10:43:52.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.565+0000 7f9641600640 1 -- 172.21.15.28:0/4039591914 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f96441f58a0 con 0x7f96440d2920 2024-09-10T10:43:52.566 INFO:teuthology.orchestra.run.smithi028.stderr:added data pool 4 to fsmap 2024-09-10T10:43:52.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 msgr2=0x7f962406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 0x7f962406e2f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9638007fd0 tx=0x7f9638009040 comp rx=0 tx=0).stop 2024-09-10T10:43:52.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 msgr2=0x7f96441f5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f963000e980 tx=0x7f963000ee40 comp rx=0 tx=0).stop 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 shutdown_connections 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f962406be30 0x7f962406e2f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f964415e550 0x7f96440b9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96440d2920 0x7f96441f5360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 --2- 172.21.15.28:0/4039591914 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96440b3400 0x7f96440b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 >> 172.21.15.28:0/4039591914 conn(0x7f964408bfc0 msgr2=0x7f96441e8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.568+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 shutdown_connections 2024-09-10T10:43:52.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.569+0000 7f964ac00640 1 -- 172.21.15.28:0/4039591914 wait complete. 2024-09-10T10:43:52.588 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:43:52.732 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:43:52.731+0000 7fb88b400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:43:52.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.747+0000 7fa999800640 1 Processor -- start 2024-09-10T10:43:52.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.747+0000 7fa999800640 1 -- start start 2024-09-10T10:43:52.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa999800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa99406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa999800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa9940d1f90 0x7fa9940cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa999800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9940cf000 con 0x7fa9940d0dd0 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa999800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa9940b37d0 con 0x7fa9940d1f90 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa999800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9940cf330 con 0x7fa99415e660 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa993e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa9940d0dd0 0x7fa9940d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34058/0 (socket says 172.21.15.28:34058) 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.748+0000 7fa993e00640 1 -- 172.21.15.28:0/741787589 learned_addr learned my addr 172.21.15.28:0/741787589 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa993400640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa99406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa998e00640 1 --2- 172.21.15.28:0/741787589 >> v2:172.21.15.138:3300/0 conn(0x7fa9940d1f90 0x7fa9940cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1318108473 0 0) 0x7fa9940cf000 con 0x7fa9940d0dd0 2024-09-10T10:43:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa978003520 con 0x7fa9940d0dd0 2024-09-10T10:43:52.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa998e00640 1 -- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 msgr2=0x7fa99406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:52.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa998e00640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa99406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 852308074 0 0) 0x7fa978003520 con 0x7fa9940d0dd0 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa998e00640 1 -- 172.21.15.28:0/741787589 >> v1:172.21.15.28:6789/0 conn(0x7fa9940d0dd0 legacy=0x7fa9940d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.749+0000 7fa998e00640 1 -- 172.21.15.28:0/741787589 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9940d29a0 con 0x7fa9940d1f90 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.750+0000 7fa998e00640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d1f90 0x7fa9940cf7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009d90 tx=0x7fa98c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421db1aa4e80a38e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.750+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984001670 con 0x7fa9940d1f90 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.750+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c007540 con 0x7fa9940d1f90 2024-09-10T10:43:52.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.750+0000 7fa992a00640 1 -- 172.21.15.28:0/741787589 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c007910 con 0x7fa9940d1f90 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 -- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d1f90 msgr2=0x7fa9940cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d1f90 0x7fa9940cf7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009d90 tx=0x7fa98c02a5f0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 -- 172.21.15.28:0/741787589 shutdown_connections 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d1f90 0x7fa9940cf7c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 --2- 172.21.15.28:0/741787589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa99406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 -- 172.21.15.28:0/741787589 >> 172.21.15.28:0/741787589 conn(0x7fa99408bfc0 msgr2=0x7fa9940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 -- 172.21.15.28:0/741787589 shutdown_connections 2024-09-10T10:43:52.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.751+0000 7fa999800640 1 -- 172.21.15.28:0/741787589 wait complete. 2024-09-10T10:43:52.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.753+0000 7fa999800640 1 Processor -- start 2024-09-10T10:43:52.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.753+0000 7fa999800640 1 -- start start 2024-09-10T10:43:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d0dd0 0x7fa9941617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa9940b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940d3220 con 0x7fa9940d1f90 2024-09-10T10:43:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940d2f40 con 0x7fa9940d0dd0 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa999800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9940d33a0 con 0x7fa99415e660 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39942/0 (socket says 172.21.15.28:39942) 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 -- 172.21.15.28:0/3453470201 learned_addr learned my addr 172.21.15.28:0/3453470201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa998e00640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa9940b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993e00640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d0dd0 0x7fa9941617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 -- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 msgr2=0x7fa9940b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa9940b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 -- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d0dd0 msgr2=0x7fa9941617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d0dd0 0x7fa9941617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.754+0000 7fa993400640 1 -- 172.21.15.28:0/3453470201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa994154d40 con 0x7fa9940d1f90 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa993400640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e870 tx=0x7fa98000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980017070 con 0x7fa9940d1f90 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98000ee60 con 0x7fa9940d1f90 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000cb40 con 0x7fa9940d1f90 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa994154fe0 con 0x7fa9940d1f90 2024-09-10T10:43:52.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.755+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941555a0 con 0x7fa9940d1f90 2024-09-10T10:43:52.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.756+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940d2440 con 0x7fa9940d1f90 2024-09-10T10:43:52.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.757+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa98001a070 con 0x7fa9940d1f90 2024-09-10T10:43:52.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.758+0000 7fa98be00640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 0x7fa97406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:52.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.758+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa98008b260 con 0x7fa9940d1f90 2024-09-10T10:43:52.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.758+0000 7fa993e00640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 0x7fa97406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:52.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.759+0000 7fa993e00640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 0x7fa97406e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa984005880 tx=0x7fa9840057f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:52.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.762+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa980059270 con 0x7fa9940d1f90 2024-09-10T10:43:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.988+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9940d1b20 con 0x7fa9940d1f90 2024-09-10T10:43:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.988+0000 7fa98be00640 1 -- 172.21.15.28:0/3453470201 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5222 (secure 0 0 0) 0x7fa98005d570 con 0x7fa9940d1f90 2024-09-10T10:43:52.990 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:52.990 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":7,"btime":"2024-09-10T10:43:52:559012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:43:51.988095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:43:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 7 2024-09-10T10:43:52.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 msgr2=0x7fa97406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 0x7fa97406e2f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa984005880 tx=0x7fa9840057f0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 msgr2=0x7fa9941e89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:52.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa98000e870 tx=0x7fa98000ed30 comp rx=0 tx=0).stop 2024-09-10T10:43:52.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 shutdown_connections 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97406be30 0x7fa97406e2f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa99415e660 0x7fa9940b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9940d1f90 0x7fa9941e89e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 --2- 172.21.15.28:0/3453470201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9940d0dd0 0x7fa9941617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 >> 172.21.15.28:0/3453470201 conn(0x7fa99408bfc0 msgr2=0x7fa9940b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 shutdown_connections 2024-09-10T10:43:52.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:52.991+0000 7fa999800640 1 -- 172.21.15.28:0/3453470201 wait complete. 2024-09-10T10:43:53.008 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:53.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.164+0000 7fdb10c00640 1 Processor -- start 2024-09-10T10:43:53.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.164+0000 7fdb10c00640 1 -- start start 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb10c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 0x7fdb0c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb10c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0c0cf000 con 0x7fdb0c15e660 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb10c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0c0d16f0 con 0x7fdb0c0d1f90 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb10c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c06d3e0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb03e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdb0c15e660 0x7fdb0c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34068/0 (socket says 172.21.15.28:34068) 2024-09-10T10:43:53.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb03e00640 1 -- 172.21.15.28:0/3413803777 learned_addr learned my addr 172.21.15.28:0/3413803777 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:53.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.165+0000 7fdb0b400640 1 --2- 172.21.15.28:0/3413803777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 0x7fdb0c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247381749 0 0) 0x7fdb0c0cf000 con 0x7fdb0c15e660 2024-09-10T10:43:53.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaf0003520 con 0x7fdb0c15e660 2024-09-10T10:43:53.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022662654 0 0) 0x7fdb0c0d16f0 con 0x7fdb0c0d1f90 2024-09-10T10:43:53.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb0c0cf000 con 0x7fdb0c0d1f90 2024-09-10T10:43:53.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0b400640 1 -- 172.21.15.28:0/3413803777 >> v1:172.21.15.138:6789/0 conn(0x7fdb0c0d1f90 legacy=0x7fdb0c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0b400640 1 -- 172.21.15.28:0/3413803777 >> v1:172.21.15.28:6789/0 conn(0x7fdb0c15e660 legacy=0x7fdb0c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.166+0000 7fdb0b400640 1 -- 172.21.15.28:0/3413803777 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0d29a0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.167+0000 7fdb0b400640 1 --2- 172.21.15.28:0/3413803777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 0x7fdb0c0d11b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0097d0 tx=0x7fdafc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31bc1c8122f6f69a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.167+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc037070 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.167+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc032440 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.167+0000 7fdb0aa00640 1 -- 172.21.15.28:0/3413803777 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0327f0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 -- 172.21.15.28:0/3413803777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 msgr2=0x7fdb0c0d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 --2- 172.21.15.28:0/3413803777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 0x7fdb0c0d11b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0097d0 tx=0x7fdafc02a210 comp rx=0 tx=0).stop 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 -- 172.21.15.28:0/3413803777 shutdown_connections 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 --2- 172.21.15.28:0/3413803777 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d0dd0 0x7fdb0c0d11b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 -- 172.21.15.28:0/3413803777 >> 172.21.15.28:0/3413803777 conn(0x7fdb0c08bfc0 msgr2=0x7fdb0c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 -- 172.21.15.28:0/3413803777 shutdown_connections 2024-09-10T10:43:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.168+0000 7fdb10c00640 1 -- 172.21.15.28:0/3413803777 wait complete. 2024-09-10T10:43:53.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.170+0000 7fdb10c00640 1 Processor -- start 2024-09-10T10:43:53.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.170+0000 7fdb10c00640 1 -- start start 2024-09-10T10:43:53.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d1f90 0x7fdb0c1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb0c15e660 0x7fdb0c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0cf330 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0c0d2f40 con 0x7fdb0c15e660 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb10c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0c0d3220 con 0x7fdb0c0d1f90 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39956/0 (socket says 172.21.15.28:39956) 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 -- 172.21.15.28:0/541663776 learned_addr learned my addr 172.21.15.28:0/541663776 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 -- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d1f90 msgr2=0x7fdb0c1552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d1f90 0x7fdb0c1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 -- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb0c15e660 msgr2=0x7fdb0c154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb0c15e660 0x7fdb0c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 -- 172.21.15.28:0/541663776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0c0cd6c0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.171+0000 7fdb0b400640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdafc035040 tx=0x7fdafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.172+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc037040 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.172+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0044c0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.172+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0325f0 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.172+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0c0b9140 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.172+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0c0b9700 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.173+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0c0d2440 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.174+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdafc044020 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.174+0000 7fdb03400640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 0x7fdae806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.175+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdafc0ab140 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.175+0000 7fdb03e00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 0x7fdae806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.176+0000 7fdb03e00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 0x7fdae806e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005880 tx=0x7fdaf4007950 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.179+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdafc04b020 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.365+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb0c0d1b20 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.367+0000 7fdb03400640 1 -- 172.21.15.28:0/541663776 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7fdafc030e20 con 0x7fdb0c0d0dd0 2024-09-10T10:43:53.368 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:53.369 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 msgr2=0x7fdae806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 0x7fdae806e320 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4005880 tx=0x7fdaf4007950 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 msgr2=0x7fdb0c1e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdafc035040 tx=0x7fdafc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 shutdown_connections 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdae806be60 0x7fdae806e320 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb0c15e660 0x7fdb0c154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb0c0d1f90 0x7fdb0c1552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 --2- 172.21.15.28:0/541663776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0c0d0dd0 0x7fdb0c1e6590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 >> 172.21.15.28:0/541663776 conn(0x7fdb0c08bfc0 msgr2=0x7fdb0c0b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 shutdown_connections 2024-09-10T10:43:53.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.369+0000 7fdb10c00640 1 -- 172.21.15.28:0/541663776 wait complete. 2024-09-10T10:43:53.386 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:53.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.541+0000 7f794d400640 1 Processor -- start 2024-09-10T10:43:53.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.541+0000 7f794d400640 1 -- start start 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.542+0000 7f794d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7948072de0 0x7f794815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f794d400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7948146140 con 0x7f7948072700 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f794d400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79480cd740 con 0x7f7948072320 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f794d400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f79480cda20 con 0x7f7948072de0 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7948072de0 0x7f794815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7948072de0 0x7f794815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39968/0 (socket says 172.21.15.28:39968) 2024-09-10T10:43:53.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 -- 172.21.15.28:0/1173219596 learned_addr learned my addr 172.21.15.28:0/1173219596 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 -- 172.21.15.28:0/1173219596 >> v1:172.21.15.138:6789/0 conn(0x7f7948072320 legacy=0x7f79480cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 -- 172.21.15.28:0/1173219596 >> v1:172.21.15.28:6789/0 conn(0x7f7948072700 legacy=0x7f794815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f793fe00640 1 -- 172.21.15.28:0/1173219596 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79480cdd00 con 0x7f7948072de0 2024-09-10T10:43:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.543+0000 7f7945e00640 1 -- 172.21.15.28:0/1173219596 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2977115370 0 0) 0x7f7948146140 con 0x7f7948072700 2024-09-10T10:43:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.544+0000 7f793fe00640 1 --2- 172.21.15.28:0/1173219596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072de0 0x7f794815d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79300097f0 tx=0x7f793002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80386907ee3a136 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f7945e00640 1 -- 172.21.15.28:0/1173219596 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930037070 con 0x7f7948072de0 2024-09-10T10:43:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f7945e00640 1 -- 172.21.15.28:0/1173219596 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793002ae20 con 0x7f7948072de0 2024-09-10T10:43:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f7945e00640 1 -- 172.21.15.28:0/1173219596 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930032660 con 0x7f7948072de0 2024-09-10T10:43:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 -- 172.21.15.28:0/1173219596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072de0 msgr2=0x7f794815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 --2- 172.21.15.28:0/1173219596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072de0 0x7f794815d1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79300097f0 tx=0x7f793002a110 comp rx=0 tx=0).stop 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 -- 172.21.15.28:0/1173219596 shutdown_connections 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 --2- 172.21.15.28:0/1173219596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072de0 0x7f794815d1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 -- 172.21.15.28:0/1173219596 >> 172.21.15.28:0/1173219596 conn(0x7f7948067d70 msgr2=0x7f794806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 -- 172.21.15.28:0/1173219596 shutdown_connections 2024-09-10T10:43:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.545+0000 7f794d400640 1 -- 172.21.15.28:0/1173219596 wait complete. 2024-09-10T10:43:53.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.547+0000 7f794d400640 1 Processor -- start 2024-09-10T10:43:53.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.547+0000 7f794d400640 1 -- start start 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7948072320 0x7f79480a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 0x7f79480b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f794815dfb0 con 0x7f7948072700 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f794815de30 con 0x7f7948072de0 2024-09-10T10:43:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f794d400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f794815e130 con 0x7f7948072320 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39970/0 (socket says 172.21.15.28:39970) 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 -- 172.21.15.28:0/87314145 learned_addr learned my addr 172.21.15.28:0/87314145 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f7946800640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7948072320 0x7f79480a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f7947200640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 0x7f79480b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 -- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7948072320 msgr2=0x7f79480a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7948072320 0x7f79480a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 -- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 msgr2=0x7f79480b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.548+0000 7f793fe00640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 0x7f79480b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f793fe00640 1 -- 172.21.15.28:0/87314145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79480b3d10 con 0x7f7948072700 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f7947200640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 0x7f79480b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f793fe00640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79300097f0 tx=0x7f79300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930042070 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793003e460 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930037040 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79480b3ff0 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.549+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79480b45b0 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.550+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794806f8a0 con 0x7f7948072700 2024-09-10T10:43:53.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.551+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7930045070 con 0x7f7948072700 2024-09-10T10:43:53.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.552+0000 7f793ea00640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 0x7f792406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.552+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f79300ab8d0 con 0x7f7948072700 2024-09-10T10:43:53.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.552+0000 7f7946800640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 0x7f792406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.553+0000 7f7946800640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 0x7f792406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7938001920 tx=0x7f7938009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.556+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f794806f8a0 con 0x7f7948072700 2024-09-10T10:43:53.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.745+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79481447b0 con 0x7f7948072700 2024-09-10T10:43:53.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.747+0000 7f793ea00640 1 -- 172.21.15.28:0/87314145 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7f793002ae20 con 0x7f7948072700 2024-09-10T10:43:53.748 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:53.748 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 msgr2=0x7f792406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 0x7f792406e2f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7938001920 tx=0x7f7938009040 comp rx=0 tx=0).stop 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 msgr2=0x7f794815ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79300097f0 tx=0x7f79300046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 shutdown_connections 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f792406be30 0x7f792406e2f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7948072de0 0x7f79480b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7948072700 0x7f794815ba70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 --2- 172.21.15.28:0/87314145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7948072320 0x7f79480a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 >> 172.21.15.28:0/87314145 conn(0x7f7948067d70 msgr2=0x7f794806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 shutdown_connections 2024-09-10T10:43:53.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.749+0000 7f794d400640 1 -- 172.21.15.28:0/87314145 wait complete. 2024-09-10T10:43:53.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:43:53.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.936+0000 7fb6f3600640 1 Processor -- start 2024-09-10T10:43:53.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.936+0000 7fb6f3600640 1 -- start start 2024-09-10T10:43:53.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.937+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6ec0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.937+0000 7fb6f3600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ec146140 con 0x7fb6ec072de0 2024-09-10T10:43:53.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.937+0000 7fb6f3600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ec06eed0 con 0x7fb6ec15e1f0 2024-09-10T10:43:53.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.937+0000 7fb6f3600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ec0cd740 con 0x7fb6ec072320 2024-09-10T10:43:53.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6ebe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb6ec072de0 0x7fb6ec06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50336/0 (socket says 172.21.15.28:50336) 2024-09-10T10:43:53.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6ebe00640 1 -- 172.21.15.28:0/1391532625 learned_addr learned my addr 172.21.15.28:0/1391532625 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6ec0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 886772460 0 0) 0x7fb6ec146140 con 0x7fb6ec072de0 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d4003520 con 0x7fb6ec072de0 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1622799410 0 0) 0x7fb6d4003520 con 0x7fb6ec072de0 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 msgr2=0x7fb6ec0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6ec0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 >> v1:172.21.15.138:6789/0 conn(0x7fb6ec15e1f0 legacy=0x7fb6ec06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec0cda20 con 0x7fb6ec072de0 2024-09-10T10:43:53.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fb6d8001670 con 0x7fb6ec072de0 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.938+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 >> v1:172.21.15.28:6789/0 conn(0x7fb6ec072de0 legacy=0x7fb6ec06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d4004d40 0x7fb6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec0cd740 con 0x7fb6d40039e0 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d400ba70 con 0x7fb6d4004d40 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d400bc70 con 0x7fb6ec072320 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6ebe00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f1600640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d4004d40 0x7fb6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 -- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 msgr2=0x7fb6d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec072320 0x7fb6d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 -- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d4004d40 msgr2=0x7fb6d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d4004d40 0x7fb6d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f0c00640 1 -- 172.21.15.28:0/1391532625 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d4003520 con 0x7fb6d40039e0 2024-09-10T10:43:53.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.939+0000 7fb6f1600640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6d4004d40 0x7fb6d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:53.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.940+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6d4003dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e75d8661bcd6207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.940+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0036070 con 0x7fb6d40039e0 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.940+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0031400 con 0x7fb6d40039e0 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.940+0000 7fb6eb400640 1 -- 172.21.15.28:0/1391532625 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0009690 con 0x7fb6d40039e0 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 -- 172.21.15.28:0/1391532625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 msgr2=0x7fb6d4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 --2- 172.21.15.28:0/1391532625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6d4003dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009ac0 tx=0x7fb6d4005830 comp rx=0 tx=0).stop 2024-09-10T10:43:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f1600640 1 -- 172.21.15.28:0/1391532625 reap_dead start 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 -- 172.21.15.28:0/1391532625 shutdown_connections 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 -- 172.21.15.28:0/1391532625 >> 172.21.15.28:0/1391532625 conn(0x7fb6ec067d70 msgr2=0x7fb6ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 -- 172.21.15.28:0/1391532625 shutdown_connections 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.941+0000 7fb6f3600640 1 -- 172.21.15.28:0/1391532625 wait complete. 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.943+0000 7fb6f3600640 1 Processor -- start 2024-09-10T10:43:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.943+0000 7fb6f3600640 1 -- start start 2024-09-10T10:43:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec140120 0x7fb6ec1e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec1405a0 0x7fb6ec1e5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec072700 con 0x7fb6d40039e0 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec15c9e0 con 0x7fb6ec1405a0 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f3600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ec072880 con 0x7fb6ec140120 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38396/0 (socket says 172.21.15.28:38396) 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f0c00640 1 -- 172.21.15.28:0/4132603405 learned_addr learned my addr 172.21.15.28:0/4132603405 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6f1600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec1405a0 0x7fb6ec1e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.944+0000 7fb6ebe00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec140120 0x7fb6ec1e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 -- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec140120 msgr2=0x7fb6ec1e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec140120 0x7fb6ec1e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 -- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec1405a0 msgr2=0x7fb6ec1e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec1405a0 0x7fb6ec1e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 -- 172.21.15.28:0/4132603405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec141f50 con 0x7fb6d40039e0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f0c00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009be0 tx=0x7fb6e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0040020 con 0x7fb6d40039e0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0003940 con 0x7fb6d40039e0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0036040 con 0x7fb6d40039e0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec142230 con 0x7fb6d40039e0 2024-09-10T10:43:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.945+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1427f0 con 0x7fb6d40039e0 2024-09-10T10:43:53.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.947+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec0cce60 con 0x7fb6d40039e0 2024-09-10T10:43:53.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.947+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6e0043020 con 0x7fb6d40039e0 2024-09-10T10:43:53.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.948+0000 7fb6e8c00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 0x7fb6cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:53.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.948+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb6e00ab1a0 con 0x7fb6d40039e0 2024-09-10T10:43:53.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.949+0000 7fb6ebe00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 0x7fb6cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:53.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.950+0000 7fb6ebe00640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 0x7fb6cc06e320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b19f0 tx=0x7fb6d8000fe0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:53.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:53.953+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6e004a070 con 0x7fb6d40039e0 2024-09-10T10:43:54.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.175+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6ec0b2e60 con 0x7fb6d40039e0 2024-09-10T10:43:54.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.175+0000 7fb6e8c00640 1 -- 172.21.15.28:0/4132603405 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5222 (secure 0 0 0) 0x7fb6e00791c0 con 0x7fb6d40039e0 2024-09-10T10:43:54.177 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:54.177 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":7,"btime":"2024-09-10T10:43:52:559012+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:43:51.988095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:43:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 7 2024-09-10T10:43:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 msgr2=0x7fb6cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 0x7fb6cc06e320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b19f0 tx=0x7fb6d8000fe0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 msgr2=0x7fb6ec142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009be0 tx=0x7fb6e0004730 comp rx=0 tx=0).stop 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 shutdown_connections 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6cc06be60 0x7fb6cc06e320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6ec1405a0 0x7fb6ec1e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6ec140120 0x7fb6ec1e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 --2- 172.21.15.28:0/4132603405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6d40039e0 0x7fb6ec142d10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.177+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 >> 172.21.15.28:0/4132603405 conn(0x7fb6ec067d70 msgr2=0x7fb6ec0b4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:54.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.178+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 shutdown_connections 2024-09-10T10:43:54.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.178+0000 7fb6f3600640 1 -- 172.21.15.28:0/4132603405 wait complete. 2024-09-10T10:43:54.194 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:43:54.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.349+0000 7f4898a00640 1 Processor -- start 2024-09-10T10:43:54.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.349+0000 7f4898a00640 1 -- start start 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4898a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48940d0dd0 0x7f48940d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4898a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 0x7f489406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4898a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48940cf000 con 0x7f48940d1f90 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4898a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f48940b37d0 con 0x7f48940d0dd0 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4898a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48940cf330 con 0x7f489415e660 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4893400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48940d0dd0 0x7f48940d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4893400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48940d0dd0 0x7f48940d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38398/0 (socket says 172.21.15.28:38398) 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4893400640 1 -- 172.21.15.28:0/2999860495 learned_addr learned my addr 172.21.15.28:0/2999860495 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.350+0000 7f4893e00640 1 --1- 172.21.15.28:0/2999860495 >> v1:172.21.15.28:6789/0 conn(0x7f48940d1f90 0x7f48940cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50346/0 (socket says 172.21.15.28:50346) 2024-09-10T10:43:54.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4892a00640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 0x7f489406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4893400640 1 -- 172.21.15.28:0/2999860495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 msgr2=0x7f489406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4893400640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 0x7f489406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4893400640 1 -- 172.21.15.28:0/2999860495 >> v1:172.21.15.28:6789/0 conn(0x7f48940d1f90 legacy=0x7f48940cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4893400640 1 -- 172.21.15.28:0/2999860495 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48940d16f0 con 0x7f48940d0dd0 2024-09-10T10:43:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.351+0000 7f4892a00640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 0x7f489406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4893400640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48940d11b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4884009870 tx=0x7f488402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b85c0e326661ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4892000640 1 -- 172.21.15.28:0/2999860495 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884037070 con 0x7f48940d0dd0 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4892000640 1 -- 172.21.15.28:0/2999860495 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4884032440 con 0x7f48940d0dd0 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4892000640 1 -- 172.21.15.28:0/2999860495 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884032810 con 0x7f48940d0dd0 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4898a00640 1 -- 172.21.15.28:0/2999860495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 msgr2=0x7f48940d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.352+0000 7f4898a00640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48940d11b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4884009870 tx=0x7f488402a270 comp rx=0 tx=0).stop 2024-09-10T10:43:54.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 -- 172.21.15.28:0/2999860495 shutdown_connections 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f489415e660 0x7f489406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 --2- 172.21.15.28:0/2999860495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48940d11b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 -- 172.21.15.28:0/2999860495 >> 172.21.15.28:0/2999860495 conn(0x7f489408bfc0 msgr2=0x7f48940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 -- 172.21.15.28:0/2999860495 shutdown_connections 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.353+0000 7f4898a00640 1 -- 172.21.15.28:0/2999860495 wait complete. 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.354+0000 7f4898a00640 1 Processor -- start 2024-09-10T10:43:54.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.354+0000 7f4898a00640 1 -- start start 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48940d1f90 0x7f4894150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 0x7f489414d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48940d2f40 con 0x7f48940d0dd0 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48940d2c60 con 0x7f489415e660 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4898a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48940d3220 con 0x7f48940d1f90 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4893400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4893400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38402/0 (socket says 172.21.15.28:38402) 2024-09-10T10:43:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4893400640 1 -- 172.21.15.28:0/3414894448 learned_addr learned my addr 172.21.15.28:0/3414894448 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.355+0000 7f4892a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48940d1f90 0x7f4894150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893e00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 0x7f489414d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 -- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48940d1f90 msgr2=0x7f4894150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48940d1f90 0x7f4894150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 -- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 msgr2=0x7f489414d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 0x7f489414d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 -- 172.21.15.28:0/3414894448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489414e100 con 0x7f48940d0dd0 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893e00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 0x7f489414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4893400640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4884009870 tx=0x7f48840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884041070 con 0x7f48940d0dd0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f488403c870 con 0x7f48940d0dd0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884037040 con 0x7f48940d0dd0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.356+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48940cd2d0 con 0x7f48940d0dd0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.357+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48940cd890 con 0x7f48940d0dd0 2024-09-10T10:43:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.358+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48940d2440 con 0x7f48940d0dd0 2024-09-10T10:43:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.358+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4884044070 con 0x7f48940d0dd0 2024-09-10T10:43:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.359+0000 7f488b400640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 0x7f487406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.359+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f48840abcd0 con 0x7f48940d0dd0 2024-09-10T10:43:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.360+0000 7f4892a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 0x7f487406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.360+0000 7f4892a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 0x7f487406e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48940b9b00 tx=0x7f4880008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.363+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f489414e100 con 0x7f48940d0dd0 2024-09-10T10:43:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.552+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48940d1b20 con 0x7f48940d0dd0 2024-09-10T10:43:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.553+0000 7f488b400640 1 -- 172.21.15.28:0/3414894448 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7f4884079c50 con 0x7f48940d0dd0 2024-09-10T10:43:54.554 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:54.554 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:43:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 msgr2=0x7f487406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 0x7f487406e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48940b9b00 tx=0x7f4880008040 comp rx=0 tx=0).stop 2024-09-10T10:43:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 msgr2=0x7f48941f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4884009870 tx=0x7f48840046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 shutdown_connections 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f487406be30 0x7f487406e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f489415e660 0x7f489414d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48940d1f90 0x7f4894150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 --2- 172.21.15.28:0/3414894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48940d0dd0 0x7f48941f4120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:54.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 >> 172.21.15.28:0/3414894448 conn(0x7f489408bfc0 msgr2=0x7f4894157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:54.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 shutdown_connections 2024-09-10T10:43:54.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:54.555+0000 7f4898a00640 1 -- 172.21.15.28:0/3414894448 wait complete. 2024-09-10T10:43:54.975 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:43:46.939715+0000', 'modified': '2024-09-10T10:43:51.988095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6835/112106318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 112106318}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 112106318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4284, 'qdb_cluster': [4284]} 2024-09-10T10:43:54.976 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:43:54.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:43:55.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.133+0000 7f54ef600640 1 Processor -- start 2024-09-10T10:43:55.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.133+0000 7f54ef600640 1 -- start start 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ef600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 0x7f54e80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ef600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e8146140 con 0x7f54e815e1f0 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ef600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e806eed0 con 0x7f54e8072de0 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ef600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54e80cd740 con 0x7f54e8072320 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ed600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54e815e1f0 0x7f54e806e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50358/0 (socket says 172.21.15.28:50358) 2024-09-10T10:43:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ed600640 1 -- 172.21.15.28:0/3430808376 learned_addr learned my addr 172.21.15.28:0/3430808376 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:55.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.134+0000 7f54ecc00640 1 --2- 172.21.15.28:0/3430808376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 0x7f54e80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716017595 0 0) 0x7f54e8146140 con 0x7f54e815e1f0 2024-09-10T10:43:55.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54cc003520 con 0x7f54e815e1f0 2024-09-10T10:43:55.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3100050082 0 0) 0x7f54e806eed0 con 0x7f54e8072de0 2024-09-10T10:43:55.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54e8146140 con 0x7f54e8072de0 2024-09-10T10:43:55.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54ecc00640 1 -- 172.21.15.28:0/3430808376 >> v1:172.21.15.138:6789/0 conn(0x7f54e8072de0 legacy=0x7f54e806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54ecc00640 1 -- 172.21.15.28:0/3430808376 >> v1:172.21.15.28:6789/0 conn(0x7f54e815e1f0 legacy=0x7f54e806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.135+0000 7f54ecc00640 1 -- 172.21.15.28:0/3430808376 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e80cda20 con 0x7f54e8072320 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.136+0000 7f54ecc00640 1 --2- 172.21.15.28:0/3430808376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 0x7f54e80cce60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097d0 tx=0x7f54d802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5916e4087b1ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.136+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8037070 con 0x7f54e8072320 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.136+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d8032440 con 0x7f54e8072320 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.136+0000 7f54e7e00640 1 -- 172.21.15.28:0/3430808376 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8032810 con 0x7f54e8072320 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 -- 172.21.15.28:0/3430808376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 msgr2=0x7f54e80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 --2- 172.21.15.28:0/3430808376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 0x7f54e80cce60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097d0 tx=0x7f54d802a180 comp rx=0 tx=0).stop 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 -- 172.21.15.28:0/3430808376 shutdown_connections 2024-09-10T10:43:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 --2- 172.21.15.28:0/3430808376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e8072320 0x7f54e80cce60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 -- 172.21.15.28:0/3430808376 >> 172.21.15.28:0/3430808376 conn(0x7f54e8067d70 msgr2=0x7f54e806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 -- 172.21.15.28:0/3430808376 shutdown_connections 2024-09-10T10:43:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.137+0000 7f54ef600640 1 -- 172.21.15.28:0/3430808376 wait complete. 2024-09-10T10:43:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.139+0000 7f54ef600640 1 Processor -- start 2024-09-10T10:43:55.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.139+0000 7f54ef600640 1 -- start start 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8072de0 0x7f54e80aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e815e1f0 0x7f54e80cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54e80cd3a0 con 0x7f54e8072320 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54e815c9e0 con 0x7f54e8072de0 2024-09-10T10:43:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ef600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54e80cd520 con 0x7f54e815e1f0 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38412/0 (socket says 172.21.15.28:38412) 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 -- 172.21.15.28:0/530887049 learned_addr learned my addr 172.21.15.28:0/530887049 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ed600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e815e1f0 0x7f54e80cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54dfe00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8072de0 0x7f54e80aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 -- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e815e1f0 msgr2=0x7f54e80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e815e1f0 0x7f54e80cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 -- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8072de0 msgr2=0x7f54e80aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8072de0 0x7f54e80aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.140+0000 7f54ecc00640 1 -- 172.21.15.28:0/530887049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e81e5cc0 con 0x7f54e8072320 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54ecc00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097d0 tx=0x7f54d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54e5600640 1 -- 172.21.15.28:0/530887049 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8041020 con 0x7f54e8072320 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54e5600640 1 -- 172.21.15.28:0/530887049 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d803d6e0 con 0x7f54e8072320 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54e5600640 1 -- 172.21.15.28:0/530887049 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8037040 con 0x7f54e8072320 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e80a9610 con 0x7f54e8072320 2024-09-10T10:43:55.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.141+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e80a9bd0 con 0x7f54e8072320 2024-09-10T10:43:55.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.143+0000 7f54e5600640 1 -- 172.21.15.28:0/530887049 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54d8044020 con 0x7f54e8072320 2024-09-10T10:43:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.144+0000 7f54e5600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 0x7f54c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.145+0000 7f54e5600640 1 -- 172.21.15.28:0/530887049 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f54d80ab9a0 con 0x7f54e8072320 2024-09-10T10:43:55.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.145+0000 7f54dfe00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 0x7f54c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.145+0000 7f54f2645740 1 Processor -- start 2024-09-10T10:43:55.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 -- start start 2024-09-10T10:43:55.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54dfe00640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 0x7f54c406e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f54e80b4610 tx=0x7f54d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616baafcf0 0x55616baf5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baad4b0 0x55616bab00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55616ba0e3b0 con 0x55616bab1480 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55616ba13410 con 0x55616baafcf0 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.146+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55616b9bb620 con 0x55616baad4b0 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38414/0 (socket says 172.21.15.28:38414) 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 -- 172.21.15.28:0/4179195179 learned_addr learned my addr 172.21.15.28:0/4179195179 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 -- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baad4b0 msgr2=0x55616bab00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baad4b0 0x55616bab00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 -- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616baafcf0 msgr2=0x55616baf5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616baafcf0 0x55616baf5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 -- 172.21.15.28:0/4179195179 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616b7262d0 con 0x55616bab1480 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54ed600640 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54e0002080 tx=0x7f54e000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fdcd71db812ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54dd200640 1 -- 172.21.15.28:0/4179195179 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0019070 con 0x55616bab1480 2024-09-10T10:43:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54dd200640 1 -- 172.21.15.28:0/4179195179 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0014480 con 0x55616bab1480 2024-09-10T10:43:55.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.147+0000 7f54dd200640 1 -- 172.21.15.28:0/4179195179 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0014830 con 0x55616bab1480 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 -- 172.21.15.28:0/4179195179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 msgr2=0x55616baaa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54e0002080 tx=0x7f54e000e530 comp rx=0 tx=0).stop 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 -- 172.21.15.28:0/4179195179 shutdown_connections 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bab1480 0x55616baaa570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baad4b0 0x55616bab00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 --2- 172.21.15.28:0/4179195179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616baafcf0 0x55616baf5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 -- 172.21.15.28:0/4179195179 >> 172.21.15.28:0/4179195179 conn(0x55616baac740 msgr2=0x55616baf4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 -- 172.21.15.28:0/4179195179 shutdown_connections 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.148+0000 7f54f2645740 1 -- 172.21.15.28:0/4179195179 wait complete. 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.149+0000 7f54f2645740 1 -- start start 2024-09-10T10:43:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.149+0000 7f54f2645740 1 Processor -- start 2024-09-10T10:43:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 0x55616bace5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 0x55616bacf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55616b9ce780 con 0x55616baafcf0 2024-09-10T10:43:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55616ba0f3d0 con 0x55616bb07f80 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55616baabbe0 con 0x55616bab1480 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54f2645740 10 client.0 _ll_register_callbacks cb 0x55616ba0c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38424/0 (socket says 172.21.15.28:38424) 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.150+0000 7f54ecc00640 1 -- 172.21.15.28:0/977077812 learned_addr learned my addr 172.21.15.28:0/977077812 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54dfe00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 0x55616bace5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ed600640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 0x55616bacf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 -- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 msgr2=0x55616bace5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 0x55616bace5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 -- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 msgr2=0x55616bacf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 0x55616bacf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 -- 172.21.15.28:0/977077812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616b7f06b0 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ed600640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 0x55616bacf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54dfe00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 0x55616bace5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54ecc00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009720 tx=0x7f54d8079f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8295a74cd3e0d31a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54d5600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d807dc30 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f54d8081fd0 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.151+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e0014480 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f54d8082460 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 10 client.4407 fetch_fsmap learned FSMap version 7 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55616baaab00 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e0014480 con 0x55616baafcf0 2024-09-10T10:43:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f54d803a030 con 0x55616baafcf0 2024-09-10T10:43:55.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 10 client.4407 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 20 client.4407 populate_metadata read hostname 'smithi028' 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bad5ea0 0x55616bad0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 4 client.4407 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 --> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616ba20b40 con 0x55616bad5ea0 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.152+0000 7f54dfe00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bad5ea0 0x55616bad0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.153+0000 7f54dfe00640 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bad5ea0 0x55616bad0cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875ef36a3803f10a server_cookie=cd1c2ae1b4192c90 in_seq=0 out_seq=0 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.153+0000 7f54d5600640 10 client.4407 ms_handle_connect on v2:172.21.15.28:6834/112106318 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54d5600640 1 -- 172.21.15.28:0/977077812 <== mds.0 v2:172.21.15.28:6834/112106318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55616ba20b40 con 0x55616bad5ea0 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54d5600640 10 client.4407 handle_command_reply: tid=0 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 client.4407 shutdown 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 msgr2=0x55616baa9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009720 tx=0x7f54d8079f80 comp rx=0 tx=0).stop 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 shutdown_connections 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bb07f80 0x55616bacf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bad5ea0 0x55616bad0cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bab1480 0x55616bace5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 --2- 172.21.15.28:0/977077812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616baa9ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54d5600640 0 client.4407 ms_handle_reset on v2:172.21.15.28:6834/112106318 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 >> 172.21.15.28:0/977077812 conn(0x55616baad4b0 msgr2=0x55616baf29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 shutdown_connections 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 1 -- 172.21.15.28:0/977077812 wait complete. 2024-09-10T10:43:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.154+0000 7f54f2645740 20 client.4407 trim_cache size 0 max 16384 2024-09-10T10:43:55.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.173+0000 7f54f2645740 1 Processor -- start 2024-09-10T10:43:55.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.173+0000 7f54f2645740 1 -- start start 2024-09-10T10:43:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baafcf0 0x55616bad0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616baaaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54e82010c0 con 0x55616bb07f80 2024-09-10T10:43:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54e82005a0 con 0x55616bab1480 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54e8200aa0 con 0x55616baafcf0 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38432/0 (socket says 172.21.15.28:38432) 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 -- 172.21.15.28:0/788993221 learned_addr learned my addr 172.21.15.28:0/788993221 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 -- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baafcf0 msgr2=0x55616bad0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ecc00640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baafcf0 0x55616bad0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54dfe00640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616baaaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baafcf0 0x55616bad0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 -- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 msgr2=0x55616baaaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616baaaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54ed600640 1 -- 172.21.15.28:0/788993221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616bad6590 con 0x55616bb07f80 2024-09-10T10:43:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.174+0000 7f54dfe00640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616baaaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54ed600640 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54e0001580 tx=0x7f54e000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57fb564b0e5878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54d7e00640 1 -- 172.21.15.28:0/788993221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0024070 con 0x55616bb07f80 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54d7e00640 1 -- 172.21.15.28:0/788993221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e001a040 con 0x55616bb07f80 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54d7e00640 1 -- 172.21.15.28:0/788993221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0019070 con 0x55616bb07f80 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 -- 172.21.15.28:0/788993221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 msgr2=0x55616bad8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f54e0001580 tx=0x7f54e000c000 comp rx=0 tx=0).stop 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 -- 172.21.15.28:0/788993221 shutdown_connections 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616bb07f80 0x55616bad8f40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616baaaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 --2- 172.21.15.28:0/788993221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616baafcf0 0x55616bad0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 -- 172.21.15.28:0/788993221 >> 172.21.15.28:0/788993221 conn(0x55616baad4b0 msgr2=0x55616baf2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.175+0000 7f54f2645740 1 -- 172.21.15.28:0/788993221 shutdown_connections 2024-09-10T10:43:55.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.176+0000 7f54f2645740 1 -- 172.21.15.28:0/788993221 wait complete. 2024-09-10T10:43:55.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.176+0000 7f54f2645740 1 -- start start 2024-09-10T10:43:55.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.176+0000 7f54f2645740 1 Processor -- start 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616bb5ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bb07f80 0x55616bad6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55616bad5d20 con 0x55616baafcf0 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55616bacfc80 con 0x55616bab1480 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55616bad6370 con 0x55616bb07f80 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54f2645740 10 client.0 _ll_register_callbacks cb 0x7f54d802a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38440/0 (socket says 172.21.15.28:38440) 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.177+0000 7f54ecc00640 1 -- 172.21.15.28:0/3498942833 learned_addr learned my addr 172.21.15.28:0/3498942833 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ed600640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bb07f80 0x55616bad6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54dfe00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616bb5ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 -- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bb07f80 msgr2=0x55616bad6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bb07f80 0x55616bad6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 -- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 msgr2=0x55616bb5ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616bb5ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 -- 172.21.15.28:0/3498942833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616bad8fe0 con 0x55616baafcf0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54ecc00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54d8079530 tx=0x7f54d80aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc914cc01e309b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54d5600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d807dc30 con 0x55616baafcf0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f54d807adc0 con 0x55616baafcf0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e001a040 con 0x55616baafcf0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f54d8031bc0 con 0x55616baafcf0 2024-09-10T10:43:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54f2645740 10 client.4413 fetch_fsmap learned FSMap version 7 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.178+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55616bb5d200 con 0x55616baafcf0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e001a040 con 0x55616baafcf0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f54d80a30d0 con 0x55616baafcf0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 10 client.4413 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 20 client.4413 populate_metadata read hostname 'smithi028' 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bb17ba0 0x55616bb73c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 4 client.4413 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 --> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] -- command(tid 0: {"prefix": "status"}) -- 0x55616ba1ae90 con 0x55616bb17ba0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.179+0000 7f54dfe00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bb17ba0 0x55616bb73c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54dfe00640 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bb17ba0 0x55616bb73c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8499e0ed12969f11 server_cookie=7f742ecbcb89b5f3 in_seq=0 out_seq=0 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54d5600640 10 client.4413 ms_handle_connect on v2:172.21.15.28:6834/112106318 2024-09-10T10:43:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54d5600640 1 -- 172.21.15.28:0/3498942833 <== mds.0 v2:172.21.15.28:6834/112106318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55616ba1ae90 con 0x55616bb17ba0 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54d5600640 10 client.4413 handle_command_reply: tid=0 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 client.4413 shutdown 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 msgr2=0x55616bad4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f54d8079530 tx=0x7f54d80aa000 comp rx=0 tx=0).stop 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 shutdown_connections 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x55616bb17ba0 0x55616bb73c40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55616bb07f80 0x55616bad6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55616bab1480 0x55616bb5ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 --2- 172.21.15.28:0/3498942833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55616baafcf0 0x55616bad4340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 >> 172.21.15.28:0/3498942833 conn(0x55616baad4b0 msgr2=0x55616baaaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54d5600640 0 client.4413 ms_handle_reset on v2:172.21.15.28:6834/112106318 2024-09-10T10:43:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 shutdown_connections 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 1 -- 172.21.15.28:0/3498942833 wait complete. 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.180+0000 7f54f2645740 20 client.4413 trim_cache size 0 max 16384 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4284, 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:43:55.186 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 7.6493572160000003, 2024-09-10T10:43:55.187 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 7, 2024-09-10T10:43:55.187 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 27, 2024-09-10T10:43:55.187 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 27, 2024-09-10T10:43:55.187 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 13.458887197999999 2024-09-10T10:43:55.187 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 msgr2=0x7f54c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 0x7f54c406e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f54e80b4610 tx=0x7f54d0009040 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 msgr2=0x7f54e81e5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54d80097d0 tx=0x7f54d80046c0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 shutdown_connections 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54c406be30 0x7f54c406e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54e815e1f0 0x7f54e80cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54e8072de0 0x7f54e80aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 --2- 172.21.15.28:0/530887049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54e8072320 0x7f54e81e5780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 >> 172.21.15.28:0/530887049 conn(0x7f54e8067d70 msgr2=0x7f54e80cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 shutdown_connections 2024-09-10T10:43:55.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.181+0000 7f54ef600640 1 -- 172.21.15.28:0/530887049 wait complete. 2024-09-10T10:43:55.198 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:43:55.198 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:55.198 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:43:55.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.353+0000 7f3e35600640 1 Processor -- start 2024-09-10T10:43:55.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.353+0000 7f3e35600640 1 -- start start 2024-09-10T10:43:55.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.354+0000 7f3e35600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3006f8d0 0x7f3e30165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.354+0000 7f3e35600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e300cdbe0 con 0x7f3e30072d30 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.354+0000 7f3e35600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e300cd900 con 0x7f3e3015b300 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.354+0000 7f3e35600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e300728a0 con 0x7f3e3006f8d0 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2e000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3e30072d30 0x7f3e3006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50368/0 (socket says 172.21.15.28:50368) 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2e000640 1 -- 172.21.15.28:0/557637425 learned_addr learned my addr 172.21.15.28:0/557637425 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:43:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2f400640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3006f8d0 0x7f3e30165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264354449 0 0) 0x7f3e300cdbe0 con 0x7f3e30072d30 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e14005730 con 0x7f3e30072d30 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556513014 0 0) 0x7f3e300cd900 con 0x7f3e3015b300 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e300cdbe0 con 0x7f3e3015b300 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416679276 0 0) 0x7f3e14005730 con 0x7f3e30072d30 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3006f8d0 msgr2=0x7f3e30165de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3006f8d0 0x7f3e30165de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 >> v1:172.21.15.138:6789/0 conn(0x7f3e3015b300 legacy=0x7f3e300cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e30072250 con 0x7f3e30072d30 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f3e1c001670 con 0x7f3e30072d30 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 >> v1:172.21.15.28:6789/0 conn(0x7f3e30072d30 legacy=0x7f3e3006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e14006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e14006a10 0x7f3e14006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e140073d0 0x7f3e1400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3e300728a0 con 0x7f3e14005d80 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1400e240 con 0x7f3e140073d0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.355+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1400e440 con 0x7f3e14006a10 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2f400640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3006f8d0 0x7f3e30165de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e14006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2e000640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e14006a10 0x7f3e14006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2f400640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e140073d0 0x7f3e1400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 -- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e14006a10 msgr2=0x7f3e14006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e14006a10 0x7f3e14006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 -- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e140073d0 msgr2=0x7f3e1400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:55.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e140073d0 0x7f3e1400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.356+0000 7f3e2ea00640 1 -- 172.21.15.28:0/557637425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e14005730 con 0x7f3e14005d80 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.357+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e14006180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e200018c0 tx=0x7f3e14007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bbf9a4ef165216 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.357+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200014e0 con 0x7f3e14005d80 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.357+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e200052f0 con 0x7f3e14005d80 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.357+0000 7f3e2d600640 1 -- 172.21.15.28:0/557637425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2002ac90 con 0x7f3e14005d80 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 -- 172.21.15.28:0/557637425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 msgr2=0x7f3e14006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 --2- 172.21.15.28:0/557637425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e14006180 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e200018c0 tx=0x7f3e14007a40 comp rx=0 tx=0).stop 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e2f400640 1 -- 172.21.15.28:0/557637425 reap_dead start 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 -- 172.21.15.28:0/557637425 shutdown_connections 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 -- 172.21.15.28:0/557637425 >> 172.21.15.28:0/557637425 conn(0x7f3e30067d70 msgr2=0x7f3e3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 -- 172.21.15.28:0/557637425 shutdown_connections 2024-09-10T10:43:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.358+0000 7f3e35600640 1 -- 172.21.15.28:0/557637425 wait complete. 2024-09-10T10:43:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.360+0000 7f3e35600640 1 Processor -- start 2024-09-10T10:43:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.360+0000 7f3e35600640 1 -- start start 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3014ff00 0x7f3e30156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 0x7f3e30152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3e300cffe0 con 0x7f3e14005d80 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e300cd5c0 con 0x7f3e301f02e0 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e35600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e300d0160 con 0x7f3e3014ff00 2024-09-10T10:43:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38460/0 (socket says 172.21.15.28:38460) 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 -- 172.21.15.28:0/3717400546 learned_addr learned my addr 172.21.15.28:0/3717400546 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2e000640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3014ff00 0x7f3e30156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2f400640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 0x7f3e30152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 -- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3014ff00 msgr2=0x7f3e30156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3014ff00 0x7f3e30156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 -- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 msgr2=0x7f3e30152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 0x7f3e30152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2ea00640 1 -- 172.21.15.28:0/3717400546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e30157350 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.361+0000 7f3e2f400640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 0x7f3e30152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e2ea00640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005040 tx=0x7f3e200012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20040020 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e200054a0 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e20007470 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e30150380 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.362+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e30155010 con 0x7f3e14005d80 2024-09-10T10:43:55.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.363+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e300b2fa0 con 0x7f3e14005d80 2024-09-10T10:43:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.364+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e20043020 con 0x7f3e14005d80 2024-09-10T10:43:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.365+0000 7f3e26a00640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 0x7f3e1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.365+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3e200ab160 con 0x7f3e14005d80 2024-09-10T10:43:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.365+0000 7f3e2e000640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 0x7f3e1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.366+0000 7f3e2e000640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 0x7f3e1006e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c001200 tx=0x7f3e1c001e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:55.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.369+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e2004a030 con 0x7f3e14005d80 2024-09-10T10:43:55.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.593+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e3006e530 con 0x7f3e14005d80 2024-09-10T10:43:55.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.594+0000 7f3e26a00640 1 -- 172.21.15.28:0/3717400546 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f3e20079850 con 0x7f3e14005d80 2024-09-10T10:43:55.595 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 msgr2=0x7f3e1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 0x7f3e1006e320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c001200 tx=0x7f3e1c001e30 comp rx=0 tx=0).stop 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 msgr2=0x7f3e300b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e20005040 tx=0x7f3e200012d0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 shutdown_connections 2024-09-10T10:43:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e1006be60 0x7f3e1006e320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e301f02e0 0x7f3e30152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e3014ff00 0x7f3e30156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 --2- 172.21.15.28:0/3717400546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e14005d80 0x7f3e300b29d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 >> 172.21.15.28:0/3717400546 conn(0x7f3e30067d70 msgr2=0x7f3e3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 shutdown_connections 2024-09-10T10:43:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55.596+0000 7f3e35600640 1 -- 172.21.15.28:0/3717400546 wait complete. 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:43:55.613 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:43:55.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:55.614 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:55.649 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42669sec preferred_lft 42669sec 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:43:55.650 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:43:55.651 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:43:55.651 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:43:55.651 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:43:55.651 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:43:55.652 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:43:55.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:56.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:56.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:56.070 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:43:56.127 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:43:56.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:56.130 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:43:56.175 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:43:56.175 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:43:56.175 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:43:56.175 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:43:56.175 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:43:56.176 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:43:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:56.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:56.793 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:43:56.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:56.828 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-09-10T10:43:56.903 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:56.903 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:43:56.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:56.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T10:43:56.904 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:43:57.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:57.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:57.081 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T10:43:57.081 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T10:43:57.082 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T10:43:57.082 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:43:57.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:57.328 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:43:57.328 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:43:57.328 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:43:57.329 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:43:57.329 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:43:57.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:43:57.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:57.495 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T10:43:57.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:43:57.495 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:57.531 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T10:43:57.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:43:57.531 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:57.607 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-09-10T10:43:57.704 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:43:57.740 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-09-10T10:43:57.740 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-09-10T10:43:57.741 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-09-10T10:43:57.742 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:57.743 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:57.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:57.744 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-09-10T10:43:57.819 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T10:43:57.819 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T10:43:57.863 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-09-10T10:43:57.919 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T10:43:57.955 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-09-10T10:43:57.956 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T10:43:57.957 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:57.958 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:43:57.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:57.959 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-09-10T10:43:58.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:2024-09-10T10:43:58.019+0000 7f361c321480 -1 init, newargv = 0x55d051bebf10 newargc=15 2024-09-10T10:43:58.022 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[41325]: starting ceph client 2024-09-10T10:43:58.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[41325]: starting fuse 2024-09-10T10:43:58.037 INFO:teuthology.orchestra.run.smithi028.stdout:42 2024-09-10T10:43:58.037 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> sudo stdin-killer -- python3 -c ' 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> import re 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2024-09-10T10:43:58.038 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> return f 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.admin")) 2024-09-10T10:43:58.039 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:43:58.203 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.admin.41325.asok 2024-09-10T10:43:58.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:58.217 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41325.asok 2024-09-10T10:43:58.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:58.217 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41325.asok status 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "admin", 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T10:43:58.402 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "41325", 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4425, 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4425 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-09-10T10:43:58.403 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2965240349 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2965240349 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4425 192.168.0.1:0/2965240349", 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/2965240349", 2024-09-10T10:43:58.404 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 7, 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 27, 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2024-09-10T10:43:58.405 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:43:58.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:58.414 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2024-09-10T10:43:58.414 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:58.450 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2024-09-10T10:43:58.450 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:58.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:58.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:58.547 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T10:43:58.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi028.stdout:# file: . 2024-09-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi028.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:43:58.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:43:58.658 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:58.658 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:43:58.693 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:43:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.888+0000 7fbb24a00640 1 Processor -- start 2024-09-10T10:43:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.888+0000 7fbb24a00640 1 -- start start 2024-09-10T10:43:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb24a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb200d0dd0 0x7fbb200d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb24a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb2015e660 0x7fbb2006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb24a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb200cf000 con 0x7fbb200d1f90 2024-09-10T10:43:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb24a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbb200b37d0 con 0x7fbb200d0dd0 2024-09-10T10:43:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb24a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbb200cf330 con 0x7fbb2015e660 2024-09-10T10:43:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb1f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb200d0dd0 0x7fbb200d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb1f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb200d0dd0 0x7fbb200d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38466/0 (socket says 172.21.15.28:38466) 2024-09-10T10:43:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.889+0000 7fbb1f400640 1 -- 172.21.15.28:0/3219574167 learned_addr learned my addr 172.21.15.28:0/3219574167 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb17e00640 1 --2- 172.21.15.28:0/3219574167 >> v2:172.21.15.138:3300/0 conn(0x7fbb2015e660 0x7fbb2006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb1f400640 1 -- 172.21.15.28:0/3219574167 >> v2:172.21.15.138:3300/0 conn(0x7fbb2015e660 msgr2=0x7fbb2006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3219574167 >> v2:172.21.15.138:3300/0 conn(0x7fbb2015e660 0x7fbb2006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb1f400640 1 -- 172.21.15.28:0/3219574167 >> v1:172.21.15.28:6789/0 conn(0x7fbb200d1f90 legacy=0x7fbb200cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb1f400640 1 -- 172.21.15.28:0/3219574167 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb200d16f0 con 0x7fbb200d0dd0 2024-09-10T10:43:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.890+0000 7fbb1ea00640 1 -- 172.21.15.28:0/3219574167 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969888207 0 0) 0x7fbb200cf000 con 0x7fbb200d1f90 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.891+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3219574167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d0dd0 0x7fbb200d11b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb200b3980 tx=0x7fbb1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52942945b01b162 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.891+0000 7fbb1ea00640 1 -- 172.21.15.28:0/3219574167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10037070 con 0x7fbb200d0dd0 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.891+0000 7fbb1ea00640 1 -- 172.21.15.28:0/3219574167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb10032440 con 0x7fbb200d0dd0 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.891+0000 7fbb1ea00640 1 -- 172.21.15.28:0/3219574167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10032810 con 0x7fbb200d0dd0 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 -- 172.21.15.28:0/3219574167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d0dd0 msgr2=0x7fbb200d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3219574167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d0dd0 0x7fbb200d11b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb200b3980 tx=0x7fbb1002a270 comp rx=0 tx=0).stop 2024-09-10T10:43:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 -- 172.21.15.28:0/3219574167 shutdown_connections 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3219574167 >> v2:172.21.15.138:3300/0 conn(0x7fbb2015e660 0x7fbb2006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3219574167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d0dd0 0x7fbb200d11b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 -- 172.21.15.28:0/3219574167 >> 172.21.15.28:0/3219574167 conn(0x7fbb2008bfc0 msgr2=0x7fbb200d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 -- 172.21.15.28:0/3219574167 shutdown_connections 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.892+0000 7fbb24a00640 1 -- 172.21.15.28:0/3219574167 wait complete. 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.893+0000 7fbb24a00640 1 Processor -- start 2024-09-10T10:43:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.894+0000 7fbb24a00640 1 -- start start 2024-09-10T10:43:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 0x7fbb200c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb2015e660 0x7fbb200c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200d2f40 con 0x7fbb200d1f90 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb200d2c60 con 0x7fbb200d0dd0 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb24a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb200d3220 con 0x7fbb2015e660 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38468/0 (socket says 172.21.15.28:38468) 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 -- 172.21.15.28:0/3264228299 learned_addr learned my addr 172.21.15.28:0/3264228299 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 0x7fbb200c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb1fe00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb2015e660 0x7fbb200c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 -- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb2015e660 msgr2=0x7fbb200c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb2015e660 0x7fbb200c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 -- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 msgr2=0x7fbb200c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 0x7fbb200c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb17e00640 1 -- 172.21.15.28:0/3264228299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb200ca380 con 0x7fbb200d1f90 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.895+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 0x7fbb200c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb17e00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d7d0 tx=0x7fbb0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800b6c0 con 0x7fbb200d1f90 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08019070 con 0x7fbb200d1f90 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800bac0 con 0x7fbb200d1f90 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb200ca660 con 0x7fbb200d1f90 2024-09-10T10:43:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.896+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201e4b70 con 0x7fbb200d1f90 2024-09-10T10:43:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.897+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb200d2420 con 0x7fbb200d1f90 2024-09-10T10:43:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.898+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb0800bc60 con 0x7fbb200d1f90 2024-09-10T10:43:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.899+0000 7fbb17400640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.899+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbb0808b710 con 0x7fbb200d1f90 2024-09-10T10:43:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.899+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.900+0000 7fbb1f400640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 0x7fbafc06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034000 tx=0x7fbb1002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:58.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:58.903+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb08059ce0 con 0x7fbb200d1f90 2024-09-10T10:43:59.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.101+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb200d1b20 con 0x7fbb200d1f90 2024-09-10T10:43:59.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.102+0000 7fbb17400640 1 -- 172.21.15.28:0/3264228299 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fbb080595e0 con 0x7fbb200d1f90 2024-09-10T10:43:59.104 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:43:59.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 msgr2=0x7fbafc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 0x7fbafc06e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb10034000 tx=0x7fbb1002e000 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 msgr2=0x7fbb200c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d7d0 tx=0x7fbb0800dc90 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 shutdown_connections 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbafc06be30 0x7fbafc06e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb2015e660 0x7fbb200c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb200d1f90 0x7fbb200c79d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 --2- 172.21.15.28:0/3264228299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb200d0dd0 0x7fbb200c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 >> 172.21.15.28:0/3264228299 conn(0x7fbb2008bfc0 msgr2=0x7fbb200b51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 shutdown_connections 2024-09-10T10:43:59.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.104+0000 7fbb24a00640 1 -- 172.21.15.28:0/3264228299 wait complete. 2024-09-10T10:43:59.122 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:43:59.122 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:43:59.158 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:43:59.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.352+0000 7fe0d9a00640 1 Processor -- start 2024-09-10T10:43:59.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.353+0000 7fe0d9a00640 1 -- start start 2024-09-10T10:43:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0d4072940 0x7fe0d4072da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d9a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d40b3570 con 0x7fe0d415ba30 2024-09-10T10:43:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d9a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d40732e0 con 0x7fe0d4072940 2024-09-10T10:43:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d40b4180 con 0x7fe0d40b4880 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:41802/0 (socket says 172.21.15.28:41802) 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d3400640 1 -- 172.21.15.28:0/3699024993 learned_addr learned my addr 172.21.15.28:0/3699024993 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d2a00640 1 --1- 172.21.15.28:0/3699024993 >> v1:172.21.15.138:6789/0 conn(0x7fe0d415ba30 0x7fe0d406f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:36106/0 (socket says 172.21.15.28:36106) 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.354+0000 7fe0d2000640 1 --2- 172.21.15.28:0/3699024993 >> v2:172.21.15.138:3300/0 conn(0x7fe0d4072940 0x7fe0d4072da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.355+0000 7fe0d3400640 1 -- 172.21.15.28:0/3699024993 >> v2:172.21.15.138:3300/0 conn(0x7fe0d4072940 msgr2=0x7fe0d4072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:43:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.355+0000 7fe0d3400640 1 --2- 172.21.15.28:0/3699024993 >> v2:172.21.15.138:3300/0 conn(0x7fe0d4072940 0x7fe0d4072da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.355+0000 7fe0d1600640 1 -- 172.21.15.28:0/3699024993 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697086608 0 0) 0x7fe0d40b3570 con 0x7fe0d415ba30 2024-09-10T10:43:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.355+0000 7fe0d3400640 1 -- 172.21.15.28:0/3699024993 >> v1:172.21.15.138:6789/0 conn(0x7fe0d415ba30 legacy=0x7fe0d406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.355+0000 7fe0d3400640 1 -- 172.21.15.28:0/3699024993 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d40b3850 con 0x7fe0d40b4880 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.356+0000 7fe0d3400640 1 --2- 172.21.15.28:0/3699024993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009e50 tx=0x7fe0cc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe970790e0c157 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.356+0000 7fe0d1600640 1 -- 172.21.15.28:0/3699024993 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc036070 con 0x7fe0d40b4880 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.356+0000 7fe0d1600640 1 -- 172.21.15.28:0/3699024993 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc031600 con 0x7fe0d40b4880 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.356+0000 7fe0d1600640 1 -- 172.21.15.28:0/3699024993 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc0319d0 con 0x7fe0d40b4880 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 -- 172.21.15.28:0/3699024993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 msgr2=0x7fe0d40b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/3699024993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009e50 tx=0x7fe0cc02a710 comp rx=0 tx=0).stop 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 -- 172.21.15.28:0/3699024993 shutdown_connections 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/3699024993 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d40b4d10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/3699024993 >> v2:172.21.15.138:3300/0 conn(0x7fe0d4072940 0x7fe0d4072da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 -- 172.21.15.28:0/3699024993 >> 172.21.15.28:0/3699024993 conn(0x7fe0d4067d70 msgr2=0x7fe0d406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 -- 172.21.15.28:0/3699024993 shutdown_connections 2024-09-10T10:43:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.357+0000 7fe0d9a00640 1 -- 172.21.15.28:0/3699024993 wait complete. 2024-09-10T10:43:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.358+0000 7fe0d9a00640 1 Processor -- start 2024-09-10T10:43:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.358+0000 7fe0d9a00640 1 -- start start 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d4148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d415ba30 0x7fe0d4168df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d406ff20 con 0x7fe0d4072940 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d40b3e10 con 0x7fe0d415ba30 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.359+0000 7fe0d9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d40ce810 con 0x7fe0d40b4880 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38484/0 (socket says 172.21.15.28:38484) 2024-09-10T10:43:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 -- 172.21.15.28:0/712077640 learned_addr learned my addr 172.21.15.28:0/712077640 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2000640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d4148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d3400640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d415ba30 0x7fe0d4168df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 -- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 msgr2=0x7fe0d4148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d4148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 -- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d415ba30 msgr2=0x7fe0d4168df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d415ba30 0x7fe0d4168df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 -- 172.21.15.28:0/712077640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d414c430 con 0x7fe0d4072940 2024-09-10T10:43:59.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.360+0000 7fe0d2a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c430 tx=0x7fe0c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.361+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c40074e0 con 0x7fe0d4072940 2024-09-10T10:43:59.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.361+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c40047e0 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.361+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4004bd0 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.361+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d4167d50 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.361+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d4168310 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.362+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d40ccfe0 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.363+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0c4011070 con 0x7fe0d4072940 2024-09-10T10:43:59.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.364+0000 7fe0caa00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:43:59.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.364+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe0c408ad00 con 0x7fe0d4072940 2024-09-10T10:43:59.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.364+0000 7fe0d2000640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:43:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.365+0000 7fe0d2000640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 0x7fe0b406e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0d414cb30 tx=0x7fe0c0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:43:59.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.368+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe0c4058c80 con 0x7fe0d4072940 2024-09-10T10:43:59.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.565+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0d41530e0 con 0x7fe0d4072940 2024-09-10T10:43:59.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.565+0000 7fe0caa00640 1 -- 172.21.15.28:0/712077640 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe0c405cf80 con 0x7fe0d4072940 2024-09-10T10:43:59.568 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:43:59.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 msgr2=0x7fe0b406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 0x7fe0b406e320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0d414cb30 tx=0x7fe0c0009190 comp rx=0 tx=0).stop 2024-09-10T10:43:59.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 msgr2=0x7fe0d414ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:43:59.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe0c400c430 tx=0x7fe0c400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 shutdown_connections 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b406be60 0x7fe0b406e320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d415ba30 0x7fe0d4168df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d40b4880 0x7fe0d4148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 --2- 172.21.15.28:0/712077640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d4072940 0x7fe0d414ded0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 >> 172.21.15.28:0/712077640 conn(0x7fe0d4067d70 msgr2=0x7fe0d40b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 shutdown_connections 2024-09-10T10:43:59.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:43:59.568+0000 7fe0d9a00640 1 -- 172.21.15.28:0/712077640 wait complete. 2024-09-10T10:43:59.588 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2024-09-10T10:43:59.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:43:59.589 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T10:43:59.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[41325]: fuse finished with error 0 and tester_r 0 2024-09-10T10:43:59.646 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:44:05.747 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:44:05.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:05.748 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T10:44:05.801 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:05.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.961+0000 7fb98fe00640 1 Processor -- start 2024-09-10T10:44:05.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.961+0000 7fb98fe00640 1 -- start start 2024-09-10T10:44:05.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 0x7fb9900cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9900cf000 con 0x7fb99015e660 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9900d29a0 con 0x7fb9900d0dd0 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb99006d3e0 con 0x7fb9900d1f90 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98e000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb99015e660 0x7fb99006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41126/0 (socket says 172.21.15.28:41126) 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.962+0000 7fb98e000640 1 -- 172.21.15.28:0/611714654 learned_addr learned my addr 172.21.15.28:0/611714654 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98f400640 1 --2- 172.21.15.28:0/611714654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 0x7fb9900cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1349287659 0 0) 0x7fb9900cf000 con 0x7fb99015e660 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb974003520 con 0x7fb99015e660 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538759455 0 0) 0x7fb9900d29a0 con 0x7fb9900d0dd0 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9900cf000 con 0x7fb9900d0dd0 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98f400640 1 -- 172.21.15.28:0/611714654 >> v1:172.21.15.138:6789/0 conn(0x7fb9900d0dd0 legacy=0x7fb9900d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98f400640 1 -- 172.21.15.28:0/611714654 >> v1:172.21.15.28:6789/0 conn(0x7fb99015e660 legacy=0x7fb99006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.963+0000 7fb98f400640 1 -- 172.21.15.28:0/611714654 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900d2c60 con 0x7fb9900d1f90 2024-09-10T10:44:05.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.964+0000 7fb98f400640 1 --2- 172.21.15.28:0/611714654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 0x7fb9900cf7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb988009dc0 tx=0x7fb98802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941b920eebb848cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988007740 con 0x7fb9900d1f90 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9880078e0 con 0x7fb9900d1f90 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98d600640 1 -- 172.21.15.28:0/611714654 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988007cb0 con 0x7fb9900d1f90 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 -- 172.21.15.28:0/611714654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 msgr2=0x7fb9900cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 --2- 172.21.15.28:0/611714654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 0x7fb9900cf7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb988009dc0 tx=0x7fb98802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 -- 172.21.15.28:0/611714654 shutdown_connections 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 --2- 172.21.15.28:0/611714654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9900d1f90 0x7fb9900cf7c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 -- 172.21.15.28:0/611714654 >> 172.21.15.28:0/611714654 conn(0x7fb99008bfc0 msgr2=0x7fb9900d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 -- 172.21.15.28:0/611714654 shutdown_connections 2024-09-10T10:44:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.965+0000 7fb98fe00640 1 -- 172.21.15.28:0/611714654 wait complete. 2024-09-10T10:44:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.967+0000 7fb98fe00640 1 Processor -- start 2024-09-10T10:44:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.967+0000 7fb98fe00640 1 -- start start 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9900d0dd0 0x7fb9900cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99015e660 0x7fb9900a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900cf330 con 0x7fb9900d1f90 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9900d3220 con 0x7fb9900d0dd0 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9900d33a0 con 0x7fb99015e660 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36336/0 (socket says 172.21.15.28:36336) 2024-09-10T10:44:05.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98e000640 1 -- 172.21.15.28:0/868064620 learned_addr learned my addr 172.21.15.28:0/868064620 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98ea00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9900d0dd0 0x7fb9900cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.968+0000 7fb98f400640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99015e660 0x7fb9900a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 -- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99015e660 msgr2=0x7fb9900a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99015e660 0x7fb9900a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 -- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9900d0dd0 msgr2=0x7fb9900cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9900d0dd0 0x7fb9900cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 -- 172.21.15.28:0/868064620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9900a0210 con 0x7fb9900d1f90 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98e000640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c3e0 tx=0x7fb97c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c017070 con 0x7fb9900d1f90 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c005ce0 con 0x7fb9900d1f90 2024-09-10T10:44:05.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c0072d0 con 0x7fb9900d1f90 2024-09-10T10:44:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9900a04f0 con 0x7fb9900d1f90 2024-09-10T10:44:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.969+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9900a0d90 con 0x7fb9900d1f90 2024-09-10T10:44:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.970+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9900a08e0 con 0x7fb9900d1f90 2024-09-10T10:44:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.971+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb97c01a070 con 0x7fb9900d1f90 2024-09-10T10:44:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.972+0000 7fb986a00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 0x7fb97006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.972+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb97c08a460 con 0x7fb9900d1f90 2024-09-10T10:44:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.973+0000 7fb98ea00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 0x7fb97006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.973+0000 7fb98ea00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 0x7fb97006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9800011e0 tx=0x7fb980009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:05.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:05.977+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb97c058470 con 0x7fb9900d1f90 2024-09-10T10:44:06.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.163+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9900d2560 con 0x7fb9900d1f90 2024-09-10T10:44:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.165+0000 7fb986a00640 1 -- 172.21.15.28:0/868064620 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7fb97c05c770 con 0x7fb9900d1f90 2024-09-10T10:44:06.167 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:06.167 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 msgr2=0x7fb97006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 0x7fb97006e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb9800011e0 tx=0x7fb980009040 comp rx=0 tx=0).stop 2024-09-10T10:44:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 msgr2=0x7fb99009e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00c3e0 tx=0x7fb97c00c8a0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 shutdown_connections 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb97006be30 0x7fb97006e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb99015e660 0x7fb9900a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900d1f90 0x7fb99009e190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 --2- 172.21.15.28:0/868064620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9900d0dd0 0x7fb9900cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 >> 172.21.15.28:0/868064620 conn(0x7fb99008bfc0 msgr2=0x7fb9900b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 shutdown_connections 2024-09-10T10:44:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.167+0000 7fb98fe00640 1 -- 172.21.15.28:0/868064620 wait complete. 2024-09-10T10:44:06.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:06.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.341+0000 7fdcab200640 1 Processor -- start 2024-09-10T10:44:06.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.341+0000 7fdcab200640 1 -- start start 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdcab200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 0x7fdca406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdcab200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 0x7fdca40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdcab200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdca40cf000 con 0x7fdca40d0dd0 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdcab200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdca40b37d0 con 0x7fdca415e660 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdcab200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdca40cf330 con 0x7fdca40d1f90 2024-09-10T10:44:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdca9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdca40d0dd0 0x7fdca40d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41128/0 (socket says 172.21.15.28:41128) 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.342+0000 7fdca9e00640 1 -- 172.21.15.28:0/1316866269 learned_addr learned my addr 172.21.15.28:0/1316866269 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdcaa800640 1 --2- 172.21.15.28:0/1316866269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 0x7fdca40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdca9400640 1 --2- 172.21.15.28:0/1316866269 >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 0x7fdca406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdca8a00640 1 -- 172.21.15.28:0/1316866269 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1816311149 0 0) 0x7fdca40cf000 con 0x7fdca40d0dd0 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdca8a00640 1 -- 172.21.15.28:0/1316866269 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc8c003520 con 0x7fdca40d0dd0 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdcaa800640 1 -- 172.21.15.28:0/1316866269 >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 msgr2=0x7fdca406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdcaa800640 1 --2- 172.21.15.28:0/1316866269 >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 0x7fdca406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdcaa800640 1 -- 172.21.15.28:0/1316866269 >> v1:172.21.15.28:6789/0 conn(0x7fdca40d0dd0 legacy=0x7fdca40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdcaa800640 1 -- 172.21.15.28:0/1316866269 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca40d29a0 con 0x7fdca40d1f90 2024-09-10T10:44:06.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.343+0000 7fdca9400640 1 --2- 172.21.15.28:0/1316866269 >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 0x7fdca406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.344+0000 7fdcaa800640 1 --2- 172.21.15.28:0/1316866269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 0x7fdca40cf7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7652e6e2a01e6696 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.344+0000 7fdca8a00640 1 -- 172.21.15.28:0/1316866269 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0007740 con 0x7fdca40d1f90 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.344+0000 7fdca8a00640 1 -- 172.21.15.28:0/1316866269 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca00078e0 con 0x7fdca40d1f90 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.344+0000 7fdca8a00640 1 -- 172.21.15.28:0/1316866269 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0007cb0 con 0x7fdca40d1f90 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 -- 172.21.15.28:0/1316866269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 msgr2=0x7fdca40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 --2- 172.21.15.28:0/1316866269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 0x7fdca40cf7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009d90 tx=0x7fdca002a660 comp rx=0 tx=0).stop 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 -- 172.21.15.28:0/1316866269 shutdown_connections 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 --2- 172.21.15.28:0/1316866269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca40d1f90 0x7fdca40cf7c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 --2- 172.21.15.28:0/1316866269 >> v2:172.21.15.138:3300/0 conn(0x7fdca415e660 0x7fdca406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 -- 172.21.15.28:0/1316866269 >> 172.21.15.28:0/1316866269 conn(0x7fdca408bfc0 msgr2=0x7fdca40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 -- 172.21.15.28:0/1316866269 shutdown_connections 2024-09-10T10:44:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.345+0000 7fdcab200640 1 -- 172.21.15.28:0/1316866269 wait complete. 2024-09-10T10:44:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.347+0000 7fdcab200640 1 Processor -- start 2024-09-10T10:44:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.347+0000 7fdcab200640 1 -- start start 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 0x7fdca41f8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca415e660 0x7fdca40cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40d3220 con 0x7fdca40d0dd0 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40d2f40 con 0x7fdca40d1f90 2024-09-10T10:44:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcab200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdca40d33a0 con 0x7fdca415e660 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36348/0 (socket says 172.21.15.28:36348) 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 -- 172.21.15.28:0/2499365501 learned_addr learned my addr 172.21.15.28:0/2499365501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdcaa800640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca415e660 0x7fdca40cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9400640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 0x7fdca41f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 -- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca415e660 msgr2=0x7fdca40cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca415e660 0x7fdca40cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 -- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 msgr2=0x7fdca41f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 0x7fdca41f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9e00640 1 -- 172.21.15.28:0/2499365501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca41619d0 con 0x7fdca40d0dd0 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.348+0000 7fdca9400640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 0x7fdca41f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:06.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdca9e00640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c430 tx=0x7fdc9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98017070 con 0x7fdca40d0dd0 2024-09-10T10:44:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc98005ce0 con 0x7fdca40d0dd0 2024-09-10T10:44:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc980072f0 con 0x7fdca40d0dd0 2024-09-10T10:44:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca415ebc0 con 0x7fdca40d0dd0 2024-09-10T10:44:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.349+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca415ee60 con 0x7fdca40d0dd0 2024-09-10T10:44:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.350+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca40d2440 con 0x7fdca40d0dd0 2024-09-10T10:44:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.351+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc9801a070 con 0x7fdca40d0dd0 2024-09-10T10:44:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.352+0000 7fdc9e000640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.352+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdc9808a630 con 0x7fdca40d0dd0 2024-09-10T10:44:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.352+0000 7fdca9400640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.353+0000 7fdca9400640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 0x7fdc8806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdca415ea60 tx=0x7fdc94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.356+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc98058c90 con 0x7fdca40d0dd0 2024-09-10T10:44:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.542+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca40d1b20 con 0x7fdca40d0dd0 2024-09-10T10:44:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.543+0000 7fdc9e000640 1 -- 172.21.15.28:0/2499365501 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7fdc98058590 con 0x7fdca40d0dd0 2024-09-10T10:44:06.545 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:06.546 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 msgr2=0x7fdc8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 0x7fdc8806e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdca415ea60 tx=0x7fdc94008040 comp rx=0 tx=0).stop 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 msgr2=0x7fdca40b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c430 tx=0x7fdc9800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 shutdown_connections 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdca415e660 0x7fdca40cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdca40d1f90 0x7fdca41f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 --2- 172.21.15.28:0/2499365501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdca40d0dd0 0x7fdca40b7de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.545+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 >> 172.21.15.28:0/2499365501 conn(0x7fdca408bfc0 msgr2=0x7fdca40cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.546+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 shutdown_connections 2024-09-10T10:44:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.546+0000 7fdcab200640 1 -- 172.21.15.28:0/2499365501 wait complete. 2024-09-10T10:44:06.564 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:44:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.722+0000 7f9bf5e00640 1 Processor -- start 2024-09-10T10:44:06.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.722+0000 7f9bf5e00640 1 -- start start 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bf5e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bf5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9bf0072050 0x7f9bf0072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bf5e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00b4620 con 0x7f9bf00b4a20 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bf5e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf00ce100 con 0x7f9bf0072050 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bf5e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00ce3a0 con 0x7f9bf015e900 2024-09-10T10:44:06.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bef400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9bf0072050 0x7f9bf0072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bef400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9bf0072050 0x7f9bf0072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36358/0 (socket says 172.21.15.28:36358) 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9bef400640 1 -- 172.21.15.28:0/2083010243 learned_addr learned my addr 172.21.15.28:0/2083010243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.723+0000 7f9beea00640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.724+0000 7f9bef400640 1 -- 172.21.15.28:0/2083010243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 msgr2=0x7f9bf00b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.724+0000 7f9bef400640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.724+0000 7f9bef400640 1 -- 172.21.15.28:0/2083010243 >> v1:172.21.15.138:6789/0 conn(0x7f9bf00b4a20 legacy=0x7f9bf00ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.724+0000 7f9bef400640 1 -- 172.21.15.28:0/2083010243 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00ce680 con 0x7f9bf0072050 2024-09-10T10:44:06.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.725+0000 7f9bef400640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf0072500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009870 tx=0x7f9be802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ea087da2027c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.725+0000 7f9bed600640 1 -- 172.21.15.28:0/2083010243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037070 con 0x7f9bf0072050 2024-09-10T10:44:06.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.725+0000 7f9bed600640 1 -- 172.21.15.28:0/2083010243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be8032440 con 0x7f9bf0072050 2024-09-10T10:44:06.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.725+0000 7f9bed600640 1 -- 172.21.15.28:0/2083010243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be8032810 con 0x7f9bf0072050 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 -- 172.21.15.28:0/2083010243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 msgr2=0x7f9bf0072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf0072500 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009870 tx=0x7f9be802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 -- 172.21.15.28:0/2083010243 shutdown_connections 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf0072500 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/2083010243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 -- 172.21.15.28:0/2083010243 >> 172.21.15.28:0/2083010243 conn(0x7f9bf006bdf0 msgr2=0x7f9bf006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:06.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 -- 172.21.15.28:0/2083010243 shutdown_connections 2024-09-10T10:44:06.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.726+0000 7f9bf5e00640 1 -- 172.21.15.28:0/2083010243 wait complete. 2024-09-10T10:44:06.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.727+0000 7f9bf5e00640 1 Processor -- start 2024-09-10T10:44:06.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.727+0000 7f9bf5e00640 1 -- start start 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf00b4a20 0x7f9bf014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00cee80 con 0x7f9bf0072050 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf00b4e20 con 0x7f9bf00b4a20 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.728+0000 7f9bf5e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf00cf000 con 0x7f9bf015e900 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36362/0 (socket says 172.21.15.28:36362) 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 -- 172.21.15.28:0/1139784449 learned_addr learned my addr 172.21.15.28:0/1139784449 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:06.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9bee000640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf00b4a20 0x7f9bf014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9bef400640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 -- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 msgr2=0x7f9bf00b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 -- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf00b4a20 msgr2=0x7f9bf014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf00b4a20 0x7f9bf014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:06.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 -- 172.21.15.28:0/1139784449 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf00b19f0 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.729+0000 7f9beea00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9be000efc0 tx=0x7f9be000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.730+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0018020 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.730+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0014480 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.730+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0014850 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.730+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf009a1a0 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.730+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf009a760 con 0x7f9bf0072050 2024-09-10T10:44:06.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.731+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf00d05d0 con 0x7f9bf0072050 2024-09-10T10:44:06.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.732+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9be0005d10 con 0x7f9bf0072050 2024-09-10T10:44:06.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.733+0000 7f9be7400640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 0x7f9bd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:06.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.733+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9be008bb10 con 0x7f9bf0072050 2024-09-10T10:44:06.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.733+0000 7f9bee000640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 0x7f9bd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:06.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.734+0000 7f9bee000640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 0x7f9bd006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0159ef0 tx=0x7f9bdc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.737+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9be0022070 con 0x7f9bf0072050 2024-09-10T10:44:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:06.963+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9bf015f820 con 0x7f9bf0072050 2024-09-10T10:44:07.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.601+0000 7f9be7400640 1 -- 172.21.15.28:0/1139784449 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f9be0010040 con 0x7f9bf0072050 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 msgr2=0x7f9bd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 0x7f9bd006e2f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0159ef0 tx=0x7f9bdc009040 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 msgr2=0x7f9bf009af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9be000efc0 tx=0x7f9be000c490 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 shutdown_connections 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9bd006be30 0x7f9bd006e2f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bf015e900 0x7f9bf00b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bf00b4a20 0x7f9bf014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 --2- 172.21.15.28:0/1139784449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bf0072050 0x7f9bf009af60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 >> 172.21.15.28:0/1139784449 conn(0x7f9bf006bdf0 msgr2=0x7f9bf0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:07.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.604+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 shutdown_connections 2024-09-10T10:44:07.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.605+0000 7f9bf5e00640 1 -- 172.21.15.28:0/1139784449 wait complete. 2024-09-10T10:44:07.626 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-10T10:44:07.626 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:44:07.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.788+0000 7f0737000640 1 Processor -- start 2024-09-10T10:44:07.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.788+0000 7f0737000640 1 -- start start 2024-09-10T10:44:07.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0737000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0730161ac0 0x7f073006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0737000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07300b3640 0x7f07300d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0737000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07300d2d00 con 0x7f07300d28f0 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0737000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f07300ceba0 con 0x7f0730161ac0 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0737000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f073006dfb0 con 0x7f07300b3640 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.789+0000 7f0735c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0730161ac0 0x7f073006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0730161ac0 0x7f073006e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36376/0 (socket says 172.21.15.28:36376) 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 -- 172.21.15.28:0/3535543039 learned_addr learned my addr 172.21.15.28:0/3535543039 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735200640 1 --2- 172.21.15.28:0/3535543039 >> v2:172.21.15.138:3300/0 conn(0x7f07300b3640 0x7f07300d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 -- 172.21.15.28:0/3535543039 >> v2:172.21.15.138:3300/0 conn(0x7f07300b3640 msgr2=0x7f07300d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 --2- 172.21.15.28:0/3535543039 >> v2:172.21.15.138:3300/0 conn(0x7f07300b3640 0x7f07300d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 -- 172.21.15.28:0/3535543039 >> v1:172.21.15.138:6789/0 conn(0x7f07300d28f0 legacy=0x7f073006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.790+0000 7f0735c00640 1 -- 172.21.15.28:0/3535543039 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07300b1170 con 0x7f0730161ac0 2024-09-10T10:44:07.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.791+0000 7f0735c00640 1 --2- 172.21.15.28:0/3535543039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f073006e6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f073008d810 tx=0x7f072402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6788dbd3760e7815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.791+0000 7f072fe00640 1 -- 172.21.15.28:0/3535543039 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724037070 con 0x7f0730161ac0 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.791+0000 7f072fe00640 1 -- 172.21.15.28:0/3535543039 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072402ae50 con 0x7f0730161ac0 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.791+0000 7f072fe00640 1 -- 172.21.15.28:0/3535543039 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724032860 con 0x7f0730161ac0 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 -- 172.21.15.28:0/3535543039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 msgr2=0x7f073006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 --2- 172.21.15.28:0/3535543039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f073006e6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f073008d810 tx=0x7f072402a300 comp rx=0 tx=0).stop 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 -- 172.21.15.28:0/3535543039 shutdown_connections 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 --2- 172.21.15.28:0/3535543039 >> v2:172.21.15.138:3300/0 conn(0x7f07300b3640 0x7f07300d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 --2- 172.21.15.28:0/3535543039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f073006e6c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 -- 172.21.15.28:0/3535543039 >> 172.21.15.28:0/3535543039 conn(0x7f073008bfc0 msgr2=0x7f07300d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 -- 172.21.15.28:0/3535543039 shutdown_connections 2024-09-10T10:44:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.792+0000 7f0737000640 1 -- 172.21.15.28:0/3535543039 wait complete. 2024-09-10T10:44:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.794+0000 7f0737000640 1 Processor -- start 2024-09-10T10:44:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.794+0000 7f0737000640 1 -- start start 2024-09-10T10:44:07.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07300b3640 0x7f0730158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07300d28f0 0x7f07301f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07300b1a10 con 0x7f0730161ac0 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07300b1730 con 0x7f07300d28f0 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0737000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0730161ea0 con 0x7f07300b3640 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36388/0 (socket says 172.21.15.28:36388) 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 -- 172.21.15.28:0/2352741510 learned_addr learned my addr 172.21.15.28:0/2352741510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0735200640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07300d28f0 0x7f07301f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0735c00640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07300b3640 0x7f0730158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 -- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07300b3640 msgr2=0x7f0730158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07300b3640 0x7f0730158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 -- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07300d28f0 msgr2=0x7f07301f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:07.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07300d28f0 0x7f07301f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:07.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.795+0000 7f0736600640 1 -- 172.21.15.28:0/2352741510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07300cb660 con 0x7f0730161ac0 2024-09-10T10:44:07.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f0736600640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f072800c730 tx=0x7f072800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728003b90 con 0x7f0730161ac0 2024-09-10T10:44:07.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728003d30 con 0x7f0730161ac0 2024-09-10T10:44:07.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072800aa90 con 0x7f0730161ac0 2024-09-10T10:44:07.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07300cb940 con 0x7f0730161ac0 2024-09-10T10:44:07.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.796+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07300cbf00 con 0x7f0730161ac0 2024-09-10T10:44:07.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.797+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07300d23b0 con 0x7f0730161ac0 2024-09-10T10:44:07.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.798+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f072800ac30 con 0x7f0730161ac0 2024-09-10T10:44:07.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.799+0000 7f072d600640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 0x7f071006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:07.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.799+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f072808afe0 con 0x7f0730161ac0 2024-09-10T10:44:07.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.800+0000 7f0735c00640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 0x7f071006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:07.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.800+0000 7f0735c00640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 0x7f071006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f072402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:07.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:07.803+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07300d2d00 con 0x7f0730161ac0 2024-09-10T10:44:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.021+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f07300d0c80 con 0x7f0730161ac0 2024-09-10T10:44:08.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:44:08.481 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:44:08.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.612+0000 7f072d600640 1 -- 172.21.15.28:0/2352741510 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f0728058f60 con 0x7f0730161ac0 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 msgr2=0x7f071006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 0x7f071006e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0724034000 tx=0x7f072402e000 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 msgr2=0x7f07300b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f072800c730 tx=0x7f072800cbf0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 shutdown_connections 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f071006be30 0x7f071006e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0730161ac0 0x7f07300b5930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07300d28f0 0x7f07301f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 --2- 172.21.15.28:0/2352741510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07300b3640 0x7f0730158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 >> 172.21.15.28:0/2352741510 conn(0x7f073008bfc0 msgr2=0x7f07300b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:08.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.614+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 shutdown_connections 2024-09-10T10:44:08.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.615+0000 7f0737000640 1 -- 172.21.15.28:0/2352741510 wait complete. 2024-09-10T10:44:08.634 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:44:08.634 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:08.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.791+0000 7fb46ea00640 1 Processor -- start 2024-09-10T10:44:08.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.791+0000 7fb46ea00640 1 -- start start 2024-09-10T10:44:08.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb4680cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb46ea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb468146140 con 0x7fb468072de0 2024-09-10T10:44:08.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb46ea00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46806eed0 con 0x7fb46815e1f0 2024-09-10T10:44:08.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb46ea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4680cd740 con 0x7fb468072320 2024-09-10T10:44:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb45fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb468072de0 0x7fb46806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41144/0 (socket says 172.21.15.28:41144) 2024-09-10T10:44:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.792+0000 7fb45fe00640 1 -- 172.21.15.28:0/3989921037 learned_addr learned my addr 172.21.15.28:0/3989921037 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:08.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467e00640 1 --2- 172.21.15.28:0/3989921037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb4680cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513461411 0 0) 0x7fb468146140 con 0x7fb468072de0 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb44c003520 con 0x7fb468072de0 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 282909771 0 0) 0x7fb46806eed0 con 0x7fb46815e1f0 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb468146140 con 0x7fb46815e1f0 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467e00640 1 -- 172.21.15.28:0/3989921037 >> v1:172.21.15.138:6789/0 conn(0x7fb46815e1f0 legacy=0x7fb46806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467e00640 1 -- 172.21.15.28:0/3989921037 >> v1:172.21.15.28:6789/0 conn(0x7fb468072de0 legacy=0x7fb46806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.793+0000 7fb467e00640 1 -- 172.21.15.28:0/3989921037 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4680cda20 con 0x7fb468072320 2024-09-10T10:44:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.794+0000 7fb467e00640 1 --2- 172.21.15.28:0/3989921037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb4680cce60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4580097d0 tx=0x7fb45802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2308182b44deac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.794+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458037070 con 0x7fb468072320 2024-09-10T10:44:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb458032440 con 0x7fb468072320 2024-09-10T10:44:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb467400640 1 -- 172.21.15.28:0/3989921037 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb458032810 con 0x7fb468072320 2024-09-10T10:44:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 -- 172.21.15.28:0/3989921037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 msgr2=0x7fb4680cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 --2- 172.21.15.28:0/3989921037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb4680cce60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb4580097d0 tx=0x7fb45802a180 comp rx=0 tx=0).stop 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 -- 172.21.15.28:0/3989921037 shutdown_connections 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 --2- 172.21.15.28:0/3989921037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb4680cce60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 -- 172.21.15.28:0/3989921037 >> 172.21.15.28:0/3989921037 conn(0x7fb468067d70 msgr2=0x7fb46806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 -- 172.21.15.28:0/3989921037 shutdown_connections 2024-09-10T10:44:08.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.795+0000 7fb46ea00640 1 -- 172.21.15.28:0/3989921037 wait complete. 2024-09-10T10:44:08.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.797+0000 7fb46ea00640 1 Processor -- start 2024-09-10T10:44:08.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.797+0000 7fb46ea00640 1 -- start start 2024-09-10T10:44:08.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb468130770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb468072de0 0x7fb468131df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb4680cd3a0 con 0x7fb46815e1f0 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb46815c9e0 con 0x7fb468072de0 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4680cd520 con 0x7fb468072320 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb467e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb468130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36404/0 (socket says 172.21.15.28:36404) 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb45fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb468072de0 0x7fb468131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.798+0000 7fb46ca00640 1 -- 172.21.15.28:0/2489053038 learned_addr learned my addr 172.21.15.28:0/2489053038 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 -- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 msgr2=0x7fb468130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb468130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 -- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb468072de0 msgr2=0x7fb468131df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb468072de0 0x7fb468131df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 -- 172.21.15.28:0/2489053038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4681310c0 con 0x7fb46815e1f0 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ca00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c7d0 tx=0x7fb46000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460003b70 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb460003d10 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000aa90 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4681313a0 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.799+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb468131960 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.801+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb468133310 con 0x7fb46815e1f0 2024-09-10T10:44:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.802+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb46000ac30 con 0x7fb46815e1f0 2024-09-10T10:44:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.802+0000 7fb464c00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.803+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb46008afc0 con 0x7fb46815e1f0 2024-09-10T10:44:08.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.803+0000 7fb467e00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:08.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.804+0000 7fb467e00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 0x7fb44406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb458034000 tx=0x7fb45802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:08.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:08.807+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb460058fd0 con 0x7fb46815e1f0 2024-09-10T10:44:09.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.030+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb468145360 con 0x7fb46815e1f0 2024-09-10T10:44:09.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.030+0000 7fb464c00640 1 -- 172.21.15.28:0/2489053038 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fb46005d2d0 con 0x7fb46815e1f0 2024-09-10T10:44:09.033 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:09.033 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:09.033 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 msgr2=0x7fb44406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 0x7fb44406e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb458034000 tx=0x7fb45802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 msgr2=0x7fb468132330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c7d0 tx=0x7fb46000cc90 comp rx=0 tx=0).stop 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 shutdown_connections 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44406be30 0x7fb44406e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46815e1f0 0x7fb468132330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb468072de0 0x7fb468131df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 --2- 172.21.15.28:0/2489053038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb468072320 0x7fb468130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 >> 172.21.15.28:0/2489053038 conn(0x7fb468067d70 msgr2=0x7fb4681448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 shutdown_connections 2024-09-10T10:44:09.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.033+0000 7fb46ea00640 1 -- 172.21.15.28:0/2489053038 wait complete. 2024-09-10T10:44:09.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.209+0000 7f4e51c00640 1 Processor -- start 2024-09-10T10:44:09.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.209+0000 7f4e51c00640 1 -- start start 2024-09-10T10:44:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e51c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e4c147e90 0x7f4e4c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e51c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 0x7f4e4c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e51c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e4c072a00 con 0x7f4e4c147230 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e51c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4e4c0b34d0 con 0x7f4e4c147e90 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e51c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4e4c0ae9e0 con 0x7f4e4c0725f0 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e4c147e90 0x7f4e4c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e4c147e90 0x7f4e4c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36412/0 (socket says 172.21.15.28:36412) 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 -- 172.21.15.28:0/4029708025 learned_addr learned my addr 172.21.15.28:0/4029708025 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e4aa00640 1 --1- 172.21.15.28:0/4029708025 >> v1:172.21.15.28:6789/0 conn(0x7f4e4c147230 0x7f4e4c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41148/0 (socket says 172.21.15.28:41148) 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e4b400640 1 --2- 172.21.15.28:0/4029708025 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 0x7f4e4c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 -- 172.21.15.28:0/4029708025 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 msgr2=0x7f4e4c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 --2- 172.21.15.28:0/4029708025 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 0x7f4e4c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 -- 172.21.15.28:0/4029708025 >> v1:172.21.15.28:6789/0 conn(0x7f4e4c147230 legacy=0x7f4e4c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e43e00640 1 -- 172.21.15.28:0/4029708025 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c06f480 con 0x7f4e4c147e90 2024-09-10T10:44:09.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.211+0000 7f4e4b400640 1 --2- 172.21.15.28:0/4029708025 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 0x7f4e4c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.212+0000 7f4e43e00640 1 --2- 172.21.15.28:0/4029708025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0720b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c148320 tx=0x7f4e3402a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7041f7f1b4f3fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e4a000640 1 -- 172.21.15.28:0/4029708025 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34037070 con 0x7f4e4c147e90 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e4a000640 1 -- 172.21.15.28:0/4029708025 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e34032440 con 0x7f4e4c147e90 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e4a000640 1 -- 172.21.15.28:0/4029708025 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34032810 con 0x7f4e4c147e90 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 -- 172.21.15.28:0/4029708025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147e90 msgr2=0x7f4e4c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 --2- 172.21.15.28:0/4029708025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0720b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c148320 tx=0x7f4e3402a280 comp rx=0 tx=0).stop 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 -- 172.21.15.28:0/4029708025 shutdown_connections 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 --2- 172.21.15.28:0/4029708025 >> v2:172.21.15.138:3300/0 conn(0x7f4e4c0725f0 0x7f4e4c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 --2- 172.21.15.28:0/4029708025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0720b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 -- 172.21.15.28:0/4029708025 >> 172.21.15.28:0/4029708025 conn(0x7f4e4c067d70 msgr2=0x7f4e4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 -- 172.21.15.28:0/4029708025 shutdown_connections 2024-09-10T10:44:09.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.213+0000 7f4e51c00640 1 -- 172.21.15.28:0/4029708025 wait complete. 2024-09-10T10:44:09.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.215+0000 7f4e51c00640 1 Processor -- start 2024-09-10T10:44:09.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.215+0000 7f4e51c00640 1 -- start start 2024-09-10T10:44:09.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c0725f0 0x7f4e4c140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c06eec0 con 0x7f4e4c147230 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c06ebe0 con 0x7f4e4c147e90 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e51c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c06f1a0 con 0x7f4e4c0725f0 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.216+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36416/0 (socket says 172.21.15.28:36416) 2024-09-10T10:44:09.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 -- 172.21.15.28:0/532481839 learned_addr learned my addr 172.21.15.28:0/532481839 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:09.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e4b400640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e4aa00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c0725f0 0x7f4e4c140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 -- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c0725f0 msgr2=0x7f4e4c140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c0725f0 0x7f4e4c140690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 -- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c147e90 msgr2=0x7f4e4c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 -- 172.21.15.28:0/532481839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c0c6930 con 0x7f4e4c147230 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.217+0000 7f4e43e00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e3402a2c0 tx=0x7f4e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.218+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34037040 con 0x7f4e4c147230 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.218+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e34042070 con 0x7f4e4c147230 2024-09-10T10:44:09.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.218+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34030bb0 con 0x7f4e4c147230 2024-09-10T10:44:09.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.218+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1f5450 con 0x7f4e4c147230 2024-09-10T10:44:09.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.218+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c0c5b70 con 0x7f4e4c147230 2024-09-10T10:44:09.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.219+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c0ae2e0 con 0x7f4e4c147230 2024-09-10T10:44:09.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.220+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4e34047460 con 0x7f4e4c147230 2024-09-10T10:44:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.221+0000 7f4e42a00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 0x7f4e2806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.221+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4e340abc90 con 0x7f4e4c147230 2024-09-10T10:44:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.221+0000 7f4e4aa00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 0x7f4e2806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.222+0000 7f4e4aa00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 0x7f4e2806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007fd0 tx=0x7f4e3c007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.225+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4e4c0ae2e0 con 0x7f4e4c147230 2024-09-10T10:44:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.411+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e4c0757f0 con 0x7f4e4c147230 2024-09-10T10:44:09.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.413+0000 7f4e42a00640 1 -- 172.21.15.28:0/532481839 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7f4e4c0757f0 con 0x7f4e4c147230 2024-09-10T10:44:09.415 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:09.416 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 msgr2=0x7f4e2806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 0x7f4e2806e320 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c007fd0 tx=0x7f4e3c007e80 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 msgr2=0x7f4e4c1f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4e3402a2c0 tx=0x7f4e340046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 shutdown_connections 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4e2806be60 0x7f4e2806e320 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e4c147e90 0x7f4e4c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e4c147230 0x7f4e4c1f4f10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 --2- 172.21.15.28:0/532481839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4e4c0725f0 0x7f4e4c140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.415+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 >> 172.21.15.28:0/532481839 conn(0x7f4e4c067d70 msgr2=0x7f4e4c06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.416+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 shutdown_connections 2024-09-10T10:44:09.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.416+0000 7f4e51c00640 1 -- 172.21.15.28:0/532481839 wait complete. 2024-09-10T10:44:09.434 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:44:09.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:44:09.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.590+0000 7fdb7ca00640 1 Processor -- start 2024-09-10T10:44:09.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.590+0000 7fdb7ca00640 1 -- start start 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.591+0000 7fdb7ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb78161c80 0x7fdb7806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.591+0000 7fdb7ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb780b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb7ca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb7806e870 con 0x7fdb780b64b0 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb7ca00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdb7806d9f0 con 0x7fdb78161c80 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb7ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb7806dcd0 con 0x7fdb78162060 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb78161c80 0x7fdb7806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb78161c80 0x7fdb7806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36420/0 (socket says 172.21.15.28:36420) 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 -- 172.21.15.28:0/900553382 learned_addr learned my addr 172.21.15.28:0/900553382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:09.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb6fe00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb780b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 -- 172.21.15.28:0/900553382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 msgr2=0x7fdb780b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:09.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb780b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 -- 172.21.15.28:0/900553382 >> v1:172.21.15.138:6789/0 conn(0x7fdb780b64b0 legacy=0x7fdb78162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.592+0000 7fdb75e00640 1 -- 172.21.15.28:0/900553382 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7806dfb0 con 0x7fdb78161c80 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.593+0000 7fdb75e00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb7806e330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009870 tx=0x7fdb6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5a1e146374a3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb75400640 1 -- 172.21.15.28:0/900553382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68037070 con 0x7fdb78161c80 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb75400640 1 -- 172.21.15.28:0/900553382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb68032440 con 0x7fdb78161c80 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb75400640 1 -- 172.21.15.28:0/900553382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68032810 con 0x7fdb78161c80 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 -- 172.21.15.28:0/900553382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 msgr2=0x7fdb7806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb7806e330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009870 tx=0x7fdb6802a270 comp rx=0 tx=0).stop 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 -- 172.21.15.28:0/900553382 shutdown_connections 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb780b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/900553382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb7806e330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 -- 172.21.15.28:0/900553382 >> 172.21.15.28:0/900553382 conn(0x7fdb780d44f0 msgr2=0x7fdb780d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 -- 172.21.15.28:0/900553382 shutdown_connections 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.594+0000 7fdb7ca00640 1 -- 172.21.15.28:0/900553382 wait complete. 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.596+0000 7fdb7ca00640 1 Processor -- start 2024-09-10T10:44:09.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.596+0000 7fdb7ca00640 1 -- start start 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb780b64b0 0x7fdb78155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb7815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdb780d2a10 con 0x7fdb78161c80 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb780d2890 con 0x7fdb780b64b0 2024-09-10T10:44:09.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.597+0000 7fdb7ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb780b6960 con 0x7fdb78162060 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36436/0 (socket says 172.21.15.28:36436) 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 -- 172.21.15.28:0/3490235401 learned_addr learned my addr 172.21.15.28:0/3490235401 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb76800640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb7815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb75e00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb780b64b0 0x7fdb78155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 -- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 msgr2=0x7fdb7815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb7815e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 -- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb780b64b0 msgr2=0x7fdb78155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb780b64b0 0x7fdb78155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 -- 172.21.15.28:0/3490235401 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb780c87e0 con 0x7fdb78161c80 2024-09-10T10:44:09.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.598+0000 7fdb6fe00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e870 tx=0x7fdb6000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.599+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60017070 con 0x7fdb78161c80 2024-09-10T10:44:09.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.599+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb6000ee60 con 0x7fdb78161c80 2024-09-10T10:44:09.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.599+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000cb40 con 0x7fdb78161c80 2024-09-10T10:44:09.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.599+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb780c8a80 con 0x7fdb78161c80 2024-09-10T10:44:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.599+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb780c9040 con 0x7fdb78161c80 2024-09-10T10:44:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.600+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb7806e330 con 0x7fdb78161c80 2024-09-10T10:44:09.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.601+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb6001a070 con 0x7fdb78161c80 2024-09-10T10:44:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.602+0000 7fdb6e000640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 0x7fdb5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.602+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdb6008b260 con 0x7fdb78161c80 2024-09-10T10:44:09.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.602+0000 7fdb75e00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 0x7fdb5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:09.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.603+0000 7fdb75e00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 0x7fdb5406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb68034000 tx=0x7fdb6802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:09.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.611+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb60059270 con 0x7fdb78161c80 2024-09-10T10:44:09.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.795+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb780ba470 con 0x7fdb5406be30 2024-09-10T10:44:09.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.806+0000 7fdb6e000640 1 -- 172.21.15.28:0/3490235401 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7fdb780ba470 con 0x7fdb5406be30 2024-09-10T10:44:09.812 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:09.814 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 msgr2=0x7fdb5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 0x7fdb5406e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb68034000 tx=0x7fdb6802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 msgr2=0x7fdb780b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb6000e870 tx=0x7fdb6000ed30 comp rx=0 tx=0).stop 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 shutdown_connections 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb5406be30 0x7fdb5406e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb78162060 0x7fdb7815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb78161c80 0x7fdb780b8e00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 --2- 172.21.15.28:0/3490235401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb780b64b0 0x7fdb78155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:09.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 >> 172.21.15.28:0/3490235401 conn(0x7fdb780d44f0 msgr2=0x7fdb7806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:09.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 shutdown_connections 2024-09-10T10:44:09.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:09.811+0000 7fdb7ca00640 1 -- 172.21.15.28:0/3490235401 wait complete. 2024-09-10T10:44:09.829 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-10T10:44:08.487222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223844,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750876,"statfs":{"total":773094113280,"available":772864897024,"internally_reserved":0,"allocated":3514368,"data_stored":2081516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003961"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870128+0000","last_change":"2024-09-10T10:43:46.542097+0000","last_active":"2024-09-10T10:43:52.870128+0000","last_peered":"2024-09-10T10:43:52.870128+0000","last_clean":"2024-09-10T10:43:52.870128+0000","last_became_active":"2024-09-10T10:43:46.541830+0000","last_became_peered":"2024-09-10T10:43:46.541830+0000","last_unstale":"2024-09-10T10:43:52.870128+0000","last_undegraded":"2024-09-10T10:43:52.870128+0000","last_fullsized":"2024-09-10T10:43:52.870128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709072+0000","last_change":"2024-09-10T10:43:47.561686+0000","last_active":"2024-09-10T10:43:52.709072+0000","last_peered":"2024-09-10T10:43:52.709072+0000","last_clean":"2024-09-10T10:43:52.709072+0000","last_became_active":"2024-09-10T10:43:47.561406+0000","last_became_peered":"2024-09-10T10:43:47.561406+0000","last_unstale":"2024-09-10T10:43:52.709072+0000","last_undegraded":"2024-09-10T10:43:52.709072+0000","last_fullsized":"2024-09-10T10:43:52.709072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.700092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563827+0000","last_change":"2024-09-10T10:43:51.575030+0000","last_active":"2024-09-10T10:43:52.563827+0000","last_peered":"2024-09-10T10:43:52.563827+0000","last_clean":"2024-09-10T10:43:52.563827+0000","last_became_active":"2024-09-10T10:43:51.570182+0000","last_became_peered":"2024-09-10T10:43:51.570182+0000","last_unstale":"2024-09-10T10:43:52.563827+0000","last_undegraded":"2024-09-10T10:43:52.563827+0000","last_fullsized":"2024-09-10T10:43:52.563827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:21.179072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709156+0000","last_change":"2024-09-10T10:43:46.528230+0000","last_active":"2024-09-10T10:43:52.709156+0000","last_peered":"2024-09-10T10:43:52.709156+0000","last_clean":"2024-09-10T10:43:52.709156+0000","last_became_active":"2024-09-10T10:43:46.527955+0000","last_became_peered":"2024-09-10T10:43:46.527955+0000","last_unstale":"2024-09-10T10:43:52.709156+0000","last_undegraded":"2024-09-10T10:43:52.709156+0000","last_fullsized":"2024-09-10T10:43:52.709156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:23.630718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708899+0000","last_change":"2024-09-10T10:43:47.536706+0000","last_active":"2024-09-10T10:43:52.708899+0000","last_peered":"2024-09-10T10:43:52.708899+0000","last_clean":"2024-09-10T10:43:52.708899+0000","last_became_active":"2024-09-10T10:43:47.536347+0000","last_became_peered":"2024-09-10T10:43:47.536347+0000","last_unstale":"2024-09-10T10:43:52.708899+0000","last_undegraded":"2024-09-10T10:43:52.708899+0000","last_fullsized":"2024-09-10T10:43:52.708899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:57.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563851+0000","last_change":"2024-09-10T10:43:51.569149+0000","last_active":"2024-09-10T10:43:52.563851+0000","last_peered":"2024-09-10T10:43:52.563851+0000","last_clean":"2024-09-10T10:43:52.563851+0000","last_became_active":"2024-09-10T10:43:51.568748+0000","last_became_peered":"2024-09-10T10:43:51.568748+0000","last_unstale":"2024-09-10T10:43:52.563851+0000","last_undegraded":"2024-09-10T10:43:52.563851+0000","last_fullsized":"2024-09-10T10:43:52.563851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869713+0000","last_change":"2024-09-10T10:43:46.525149+0000","last_active":"2024-09-10T10:43:52.869713+0000","last_peered":"2024-09-10T10:43:52.869713+0000","last_clean":"2024-09-10T10:43:52.869713+0000","last_became_active":"2024-09-10T10:43:46.524877+0000","last_became_peered":"2024-09-10T10:43:46.524877+0000","last_unstale":"2024-09-10T10:43:52.869713+0000","last_undegraded":"2024-09-10T10:43:52.869713+0000","last_fullsized":"2024-09-10T10:43:52.869713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:40.800800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564917+0000","last_change":"2024-09-10T10:43:47.547350+0000","last_active":"2024-09-10T10:43:52.564917+0000","last_peered":"2024-09-10T10:43:52.564917+0000","last_clean":"2024-09-10T10:43:52.564917+0000","last_became_active":"2024-09-10T10:43:47.547041+0000","last_became_peered":"2024-09-10T10:43:47.547041+0000","last_unstale":"2024-09-10T10:43:52.564917+0000","last_undegraded":"2024-09-10T10:43:52.564917+0000","last_fullsized":"2024-09-10T10:43:52.564917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563358+0000","last_change":"2024-09-10T10:43:51.587632+0000","last_active":"2024-09-10T10:43:52.563358+0000","last_peered":"2024-09-10T10:43:52.563358+0000","last_clean":"2024-09-10T10:43:52.563358+0000","last_became_active":"2024-09-10T10:43:51.587202+0000","last_became_peered":"2024-09-10T10:43:51.587202+0000","last_unstale":"2024-09-10T10:43:52.563358+0000","last_undegraded":"2024-09-10T10:43:52.563358+0000","last_fullsized":"2024-09-10T10:43:52.563358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:57.684586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869700+0000","last_change":"2024-09-10T10:43:46.529521+0000","last_active":"2024-09-10T10:43:52.869700+0000","last_peered":"2024-09-10T10:43:52.869700+0000","last_clean":"2024-09-10T10:43:52.869700+0000","last_became_active":"2024-09-10T10:43:46.529170+0000","last_became_peered":"2024-09-10T10:43:46.529170+0000","last_unstale":"2024-09-10T10:43:52.869700+0000","last_undegraded":"2024-09-10T10:43:52.869700+0000","last_fullsized":"2024-09-10T10:43:52.869700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870421+0000","last_change":"2024-09-10T10:43:47.550037+0000","last_active":"2024-09-10T10:43:52.870421+0000","last_peered":"2024-09-10T10:43:52.870421+0000","last_clean":"2024-09-10T10:43:52.870421+0000","last_became_active":"2024-09-10T10:43:47.549678+0000","last_became_peered":"2024-09-10T10:43:47.549678+0000","last_unstale":"2024-09-10T10:43:52.870421+0000","last_undegraded":"2024-09-10T10:43:52.870421+0000","last_fullsized":"2024-09-10T10:43:52.870421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.860584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563399+0000","last_change":"2024-09-10T10:43:51.577274+0000","last_active":"2024-09-10T10:43:52.563399+0000","last_peered":"2024-09-10T10:43:52.563399+0000","last_clean":"2024-09-10T10:43:52.563399+0000","last_became_active":"2024-09-10T10:43:51.576818+0000","last_became_peered":"2024-09-10T10:43:51.576818+0000","last_unstale":"2024-09-10T10:43:52.563399+0000","last_undegraded":"2024-09-10T10:43:52.563399+0000","last_fullsized":"2024-09-10T10:43:52.563399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869003+0000","last_change":"2024-09-10T10:43:47.528998+0000","last_active":"2024-09-10T10:43:52.869003+0000","last_peered":"2024-09-10T10:43:52.869003+0000","last_clean":"2024-09-10T10:43:52.869003+0000","last_became_active":"2024-09-10T10:43:47.528761+0000","last_became_peered":"2024-09-10T10:43:47.528761+0000","last_unstale":"2024-09-10T10:43:52.869003+0000","last_undegraded":"2024-09-10T10:43:52.869003+0000","last_fullsized":"2024-09-10T10:43:52.869003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:27.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538367+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538043+0000","last_became_peered":"2024-09-10T10:43:46.538043+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:27.129655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870256+0000","last_change":"2024-09-10T10:43:51.589628+0000","last_active":"2024-09-10T10:43:52.870256+0000","last_peered":"2024-09-10T10:43:52.870256+0000","last_clean":"2024-09-10T10:43:52.870256+0000","last_became_active":"2024-09-10T10:43:51.588470+0000","last_became_peered":"2024-09-10T10:43:51.588470+0000","last_unstale":"2024-09-10T10:43:52.870256+0000","last_undegraded":"2024-09-10T10:43:52.870256+0000","last_fullsized":"2024-09-10T10:43:52.870256+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:51.784026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564115+0000","last_change":"2024-09-10T10:43:47.539521+0000","last_active":"2024-09-10T10:43:52.564115+0000","last_peered":"2024-09-10T10:43:52.564115+0000","last_clean":"2024-09-10T10:43:52.564115+0000","last_became_active":"2024-09-10T10:43:47.539187+0000","last_became_peered":"2024-09-10T10:43:47.539187+0000","last_unstale":"2024-09-10T10:43:52.564115+0000","last_undegraded":"2024-09-10T10:43:52.564115+0000","last_fullsized":"2024-09-10T10:43:52.564115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870141+0000","last_change":"2024-09-10T10:43:46.520278+0000","last_active":"2024-09-10T10:43:52.870141+0000","last_peered":"2024-09-10T10:43:52.870141+0000","last_clean":"2024-09-10T10:43:52.870141+0000","last_became_active":"2024-09-10T10:43:46.519954+0000","last_became_peered":"2024-09-10T10:43:46.519954+0000","last_unstale":"2024-09-10T10:43:52.870141+0000","last_undegraded":"2024-09-10T10:43:52.870141+0000","last_fullsized":"2024-09-10T10:43:52.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:51.570565+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:51.570165+0000","last_became_peered":"2024-09-10T10:43:51.570165+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.439716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869301+0000","last_change":"2024-09-10T10:43:47.534339+0000","last_active":"2024-09-10T10:43:52.869301+0000","last_peered":"2024-09-10T10:43:52.869301+0000","last_clean":"2024-09-10T10:43:52.869301+0000","last_became_active":"2024-09-10T10:43:47.533413+0000","last_became_peered":"2024-09-10T10:43:47.533413+0000","last_unstale":"2024-09-10T10:43:52.869301+0000","last_undegraded":"2024-09-10T10:43:52.869301+0000","last_fullsized":"2024-09-10T10:43:52.869301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869263+0000","last_change":"2024-09-10T10:43:46.535278+0000","last_active":"2024-09-10T10:43:52.869263+0000","last_peered":"2024-09-10T10:43:52.869263+0000","last_clean":"2024-09-10T10:43:52.869263+0000","last_became_active":"2024-09-10T10:43:46.534954+0000","last_became_peered":"2024-09-10T10:43:46.534954+0000","last_unstale":"2024-09-10T10:43:52.869263+0000","last_undegraded":"2024-09-10T10:43:52.869263+0000","last_fullsized":"2024-09-10T10:43:52.869263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564514+0000","last_change":"2024-09-10T10:43:51.574146+0000","last_active":"2024-09-10T10:43:52.564514+0000","last_peered":"2024-09-10T10:43:52.564514+0000","last_clean":"2024-09-10T10:43:52.564514+0000","last_became_active":"2024-09-10T10:43:51.573681+0000","last_became_peered":"2024-09-10T10:43:51.573681+0000","last_unstale":"2024-09-10T10:43:52.564514+0000","last_undegraded":"2024-09-10T10:43:52.564514+0000","last_fullsized":"2024-09-10T10:43:52.564514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:13.180965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868735+0000","last_change":"2024-09-10T10:43:47.532923+0000","last_active":"2024-09-10T10:43:52.868735+0000","last_peered":"2024-09-10T10:43:52.868735+0000","last_clean":"2024-09-10T10:43:52.868735+0000","last_became_active":"2024-09-10T10:43:47.532644+0000","last_became_peered":"2024-09-10T10:43:47.532644+0000","last_unstale":"2024-09-10T10:43:52.868735+0000","last_undegraded":"2024-09-10T10:43:52.868735+0000","last_fullsized":"2024-09-10T10:43:52.868735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:44.518256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868884+0000","last_change":"2024-09-10T10:43:46.525829+0000","last_active":"2024-09-10T10:43:52.868884+0000","last_peered":"2024-09-10T10:43:52.868884+0000","last_clean":"2024-09-10T10:43:52.868884+0000","last_became_active":"2024-09-10T10:43:46.523492+0000","last_became_peered":"2024-09-10T10:43:46.523492+0000","last_unstale":"2024-09-10T10:43:52.868884+0000","last_undegraded":"2024-09-10T10:43:52.868884+0000","last_fullsized":"2024-09-10T10:43:52.868884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564250+0000","last_change":"2024-09-10T10:43:51.580518+0000","last_active":"2024-09-10T10:43:52.564250+0000","last_peered":"2024-09-10T10:43:52.564250+0000","last_clean":"2024-09-10T10:43:52.564250+0000","last_became_active":"2024-09-10T10:43:51.580050+0000","last_became_peered":"2024-09-10T10:43:51.580050+0000","last_unstale":"2024-09-10T10:43:52.564250+0000","last_undegraded":"2024-09-10T10:43:52.564250+0000","last_fullsized":"2024-09-10T10:43:52.564250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869809+0000","last_change":"2024-09-10T10:43:47.532764+0000","last_active":"2024-09-10T10:43:52.869809+0000","last_peered":"2024-09-10T10:43:52.869809+0000","last_clean":"2024-09-10T10:43:52.869809+0000","last_became_active":"2024-09-10T10:43:47.532407+0000","last_became_peered":"2024-09-10T10:43:47.532407+0000","last_unstale":"2024-09-10T10:43:52.869809+0000","last_undegraded":"2024-09-10T10:43:52.869809+0000","last_fullsized":"2024-09-10T10:43:52.869809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.139356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870804+0000","last_change":"2024-09-10T10:43:46.537450+0000","last_active":"2024-09-10T10:43:52.870804+0000","last_peered":"2024-09-10T10:43:52.870804+0000","last_clean":"2024-09-10T10:43:52.870804+0000","last_became_active":"2024-09-10T10:43:46.537129+0000","last_became_peered":"2024-09-10T10:43:46.537129+0000","last_unstale":"2024-09-10T10:43:52.870804+0000","last_undegraded":"2024-09-10T10:43:52.870804+0000","last_fullsized":"2024-09-10T10:43:52.870804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:47.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563273+0000","last_change":"2024-09-10T10:43:51.580449+0000","last_active":"2024-09-10T10:43:52.563273+0000","last_peered":"2024-09-10T10:43:52.563273+0000","last_clean":"2024-09-10T10:43:52.563273+0000","last_became_active":"2024-09-10T10:43:51.579803+0000","last_became_peered":"2024-09-10T10:43:51.579803+0000","last_unstale":"2024-09-10T10:43:52.563273+0000","last_undegraded":"2024-09-10T10:43:52.563273+0000","last_fullsized":"2024-09-10T10:43:52.563273+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:22.089401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870018+0000","last_change":"2024-09-10T10:43:47.532014+0000","last_active":"2024-09-10T10:43:52.870018+0000","last_peered":"2024-09-10T10:43:52.870018+0000","last_clean":"2024-09-10T10:43:52.870018+0000","last_became_active":"2024-09-10T10:43:47.531662+0000","last_became_peered":"2024-09-10T10:43:47.531662+0000","last_unstale":"2024-09-10T10:43:52.870018+0000","last_undegraded":"2024-09-10T10:43:52.870018+0000","last_fullsized":"2024-09-10T10:43:52.870018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:35.559824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869967+0000","last_change":"2024-09-10T10:43:46.527723+0000","last_active":"2024-09-10T10:43:52.869967+0000","last_peered":"2024-09-10T10:43:52.869967+0000","last_clean":"2024-09-10T10:43:52.869967+0000","last_became_active":"2024-09-10T10:43:46.527452+0000","last_became_peered":"2024-09-10T10:43:46.527452+0000","last_unstale":"2024-09-10T10:43:52.869967+0000","last_undegraded":"2024-09-10T10:43:52.869967+0000","last_fullsized":"2024-09-10T10:43:52.869967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869180+0000","last_change":"2024-09-10T10:43:51.574307+0000","last_active":"2024-09-10T10:43:52.869180+0000","last_peered":"2024-09-10T10:43:52.869180+0000","last_clean":"2024-09-10T10:43:52.869180+0000","last_became_active":"2024-09-10T10:43:51.571308+0000","last_became_peered":"2024-09-10T10:43:51.571308+0000","last_unstale":"2024-09-10T10:43:52.869180+0000","last_undegraded":"2024-09-10T10:43:52.869180+0000","last_fullsized":"2024-09-10T10:43:52.869180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:09.361189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870742+0000","last_change":"2024-09-10T10:43:47.549822+0000","last_active":"2024-09-10T10:43:52.870742+0000","last_peered":"2024-09-10T10:43:52.870742+0000","last_clean":"2024-09-10T10:43:52.870742+0000","last_became_active":"2024-09-10T10:43:47.549526+0000","last_became_peered":"2024-09-10T10:43:47.549526+0000","last_unstale":"2024-09-10T10:43:52.870742+0000","last_undegraded":"2024-09-10T10:43:52.870742+0000","last_fullsized":"2024-09-10T10:43:52.870742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870715+0000","last_change":"2024-09-10T10:43:46.539321+0000","last_active":"2024-09-10T10:43:52.870715+0000","last_peered":"2024-09-10T10:43:52.870715+0000","last_clean":"2024-09-10T10:43:52.870715+0000","last_became_active":"2024-09-10T10:43:46.538763+0000","last_became_peered":"2024-09-10T10:43:46.538763+0000","last_unstale":"2024-09-10T10:43:52.870715+0000","last_undegraded":"2024-09-10T10:43:52.870715+0000","last_fullsized":"2024-09-10T10:43:52.870715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:53.745395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.571592+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.570946+0000","last_became_peered":"2024-09-10T10:43:51.570946+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564687+0000","last_change":"2024-09-10T10:43:47.535460+0000","last_active":"2024-09-10T10:43:52.564687+0000","last_peered":"2024-09-10T10:43:52.564687+0000","last_clean":"2024-09-10T10:43:52.564687+0000","last_became_active":"2024-09-10T10:43:47.535170+0000","last_became_peered":"2024-09-10T10:43:47.535170+0000","last_unstale":"2024-09-10T10:43:52.564687+0000","last_undegraded":"2024-09-10T10:43:52.564687+0000","last_fullsized":"2024-09-10T10:43:52.564687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708875+0000","last_change":"2024-09-10T10:43:46.521709+0000","last_active":"2024-09-10T10:43:52.708875+0000","last_peered":"2024-09-10T10:43:52.708875+0000","last_clean":"2024-09-10T10:43:52.708875+0000","last_became_active":"2024-09-10T10:43:46.521411+0000","last_became_peered":"2024-09-10T10:43:46.521411+0000","last_unstale":"2024-09-10T10:43:52.708875+0000","last_undegraded":"2024-09-10T10:43:52.708875+0000","last_fullsized":"2024-09-10T10:43:52.708875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:04.600951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:51.573047+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:51.572690+0000","last_became_peered":"2024-09-10T10:43:51.572690+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:47.541642+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:47.541174+0000","last_became_peered":"2024-09-10T10:43:47.541174+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:18.047791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869868+0000","last_change":"2024-09-10T10:43:46.532535+0000","last_active":"2024-09-10T10:43:52.869868+0000","last_peered":"2024-09-10T10:43:52.869868+0000","last_clean":"2024-09-10T10:43:52.869868+0000","last_became_active":"2024-09-10T10:43:46.532256+0000","last_became_peered":"2024-09-10T10:43:46.532256+0000","last_unstale":"2024-09-10T10:43:52.869868+0000","last_undegraded":"2024-09-10T10:43:52.869868+0000","last_fullsized":"2024-09-10T10:43:52.869868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:51.847379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870621+0000","last_change":"2024-09-10T10:43:51.571178+0000","last_active":"2024-09-10T10:43:52.870621+0000","last_peered":"2024-09-10T10:43:52.870621+0000","last_clean":"2024-09-10T10:43:52.870621+0000","last_became_active":"2024-09-10T10:43:51.570742+0000","last_became_peered":"2024-09-10T10:43:51.570742+0000","last_unstale":"2024-09-10T10:43:52.870621+0000","last_undegraded":"2024-09-10T10:43:52.870621+0000","last_fullsized":"2024-09-10T10:43:52.870621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708292+0000","last_change":"2024-09-10T10:43:47.536864+0000","last_active":"2024-09-10T10:43:52.708292+0000","last_peered":"2024-09-10T10:43:52.708292+0000","last_clean":"2024-09-10T10:43:52.708292+0000","last_became_active":"2024-09-10T10:43:47.536270+0000","last_became_peered":"2024-09-10T10:43:47.536270+0000","last_unstale":"2024-09-10T10:43:52.708292+0000","last_undegraded":"2024-09-10T10:43:52.708292+0000","last_fullsized":"2024-09-10T10:43:52.708292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:25.640976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563952+0000","last_change":"2024-09-10T10:43:46.521627+0000","last_active":"2024-09-10T10:43:52.563952+0000","last_peered":"2024-09-10T10:43:52.563952+0000","last_clean":"2024-09-10T10:43:52.563952+0000","last_became_active":"2024-09-10T10:43:46.521150+0000","last_became_peered":"2024-09-10T10:43:46.521150+0000","last_unstale":"2024-09-10T10:43:52.563952+0000","last_undegraded":"2024-09-10T10:43:52.563952+0000","last_fullsized":"2024-09-10T10:43:52.563952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:35.895649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870932+0000","last_change":"2024-09-10T10:43:51.580293+0000","last_active":"2024-09-10T10:43:52.870932+0000","last_peered":"2024-09-10T10:43:52.870932+0000","last_clean":"2024-09-10T10:43:52.870932+0000","last_became_active":"2024-09-10T10:43:51.576157+0000","last_became_peered":"2024-09-10T10:43:51.576157+0000","last_unstale":"2024-09-10T10:43:52.870932+0000","last_undegraded":"2024-09-10T10:43:52.870932+0000","last_fullsized":"2024-09-10T10:43:52.870932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869651+0000","last_change":"2024-09-10T10:43:47.563302+0000","last_active":"2024-09-10T10:43:52.869651+0000","last_peered":"2024-09-10T10:43:52.869651+0000","last_clean":"2024-09-10T10:43:52.869651+0000","last_became_active":"2024-09-10T10:43:47.562694+0000","last_became_peered":"2024-09-10T10:43:47.562694+0000","last_unstale":"2024-09-10T10:43:52.869651+0000","last_undegraded":"2024-09-10T10:43:52.869651+0000","last_fullsized":"2024-09-10T10:43:52.869651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868801+0000","last_change":"2024-09-10T10:43:46.533697+0000","last_active":"2024-09-10T10:43:52.868801+0000","last_peered":"2024-09-10T10:43:52.868801+0000","last_clean":"2024-09-10T10:43:52.868801+0000","last_became_active":"2024-09-10T10:43:46.533190+0000","last_became_peered":"2024-09-10T10:43:46.533190+0000","last_unstale":"2024-09-10T10:43:52.868801+0000","last_undegraded":"2024-09-10T10:43:52.868801+0000","last_fullsized":"2024-09-10T10:43:52.868801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:09.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708748+0000","last_change":"2024-09-10T10:43:51.583515+0000","last_active":"2024-09-10T10:43:52.708748+0000","last_peered":"2024-09-10T10:43:52.708748+0000","last_clean":"2024-09-10T10:43:52.708748+0000","last_became_active":"2024-09-10T10:43:51.582366+0000","last_became_peered":"2024-09-10T10:43:51.582366+0000","last_unstale":"2024-09-10T10:43:52.708748+0000","last_undegraded":"2024-09-10T10:43:52.708748+0000","last_fullsized":"2024-09-10T10:43:52.708748+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:10.957254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869329+0000","last_change":"2024-09-10T10:43:47.520783+0000","last_active":"2024-09-10T10:43:52.869329+0000","last_peered":"2024-09-10T10:43:52.869329+0000","last_clean":"2024-09-10T10:43:52.869329+0000","last_became_active":"2024-09-10T10:43:47.520409+0000","last_became_peered":"2024-09-10T10:43:47.520409+0000","last_unstale":"2024-09-10T10:43:52.869329+0000","last_undegraded":"2024-09-10T10:43:52.869329+0000","last_fullsized":"2024-09-10T10:43:52.869329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538924+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538640+0000","last_became_peered":"2024-09-10T10:43:46.538640+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:57.368590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564215+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:43:52.564215+0000","last_peered":"2024-09-10T10:43:52.564215+0000","last_clean":"2024-09-10T10:43:52.564215+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:43:52.564215+0000","last_undegraded":"2024-09-10T10:43:52.564215+0000","last_fullsized":"2024-09-10T10:43:52.564215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870584+0000","last_change":"2024-09-10T10:43:51.571200+0000","last_active":"2024-09-10T10:43:52.870584+0000","last_peered":"2024-09-10T10:43:52.870584+0000","last_clean":"2024-09-10T10:43:52.870584+0000","last_became_active":"2024-09-10T10:43:51.570877+0000","last_became_peered":"2024-09-10T10:43:51.570877+0000","last_unstale":"2024-09-10T10:43:52.870584+0000","last_undegraded":"2024-09-10T10:43:52.870584+0000","last_fullsized":"2024-09-10T10:43:52.870584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:59.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870468+0000","last_change":"2024-09-10T10:43:47.546420+0000","last_active":"2024-09-10T10:43:52.870468+0000","last_peered":"2024-09-10T10:43:52.870468+0000","last_clean":"2024-09-10T10:43:52.870468+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.870468+0000","last_undegraded":"2024-09-10T10:43:52.870468+0000","last_fullsized":"2024-09-10T10:43:52.870468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.887906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870800+0000","last_change":"2024-09-10T10:43:46.535842+0000","last_active":"2024-09-10T10:43:52.870800+0000","last_peered":"2024-09-10T10:43:52.870800+0000","last_clean":"2024-09-10T10:43:52.870800+0000","last_became_active":"2024-09-10T10:43:46.535586+0000","last_became_peered":"2024-09-10T10:43:46.535586+0000","last_unstale":"2024-09-10T10:43:52.870800+0000","last_undegraded":"2024-09-10T10:43:52.870800+0000","last_fullsized":"2024-09-10T10:43:52.870800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563751+0000","last_change":"2024-09-10T10:43:51.578759+0000","last_active":"2024-09-10T10:43:52.563751+0000","last_peered":"2024-09-10T10:43:52.563751+0000","last_clean":"2024-09-10T10:43:52.563751+0000","last_became_active":"2024-09-10T10:43:51.578237+0000","last_became_peered":"2024-09-10T10:43:51.578237+0000","last_unstale":"2024-09-10T10:43:52.563751+0000","last_undegraded":"2024-09-10T10:43:52.563751+0000","last_fullsized":"2024-09-10T10:43:52.563751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869654+0000","last_change":"2024-09-10T10:43:47.540794+0000","last_active":"2024-09-10T10:43:52.869654+0000","last_peered":"2024-09-10T10:43:52.869654+0000","last_clean":"2024-09-10T10:43:52.869654+0000","last_became_active":"2024-09-10T10:43:47.540360+0000","last_became_peered":"2024-09-10T10:43:47.540360+0000","last_unstale":"2024-09-10T10:43:52.869654+0000","last_undegraded":"2024-09-10T10:43:52.869654+0000","last_fullsized":"2024-09-10T10:43:52.869654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:31.926385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709016+0000","last_change":"2024-09-10T10:43:46.520898+0000","last_active":"2024-09-10T10:43:52.709016+0000","last_peered":"2024-09-10T10:43:52.709016+0000","last_clean":"2024-09-10T10:43:52.709016+0000","last_became_active":"2024-09-10T10:43:46.518998+0000","last_became_peered":"2024-09-10T10:43:46.518998+0000","last_unstale":"2024-09-10T10:43:52.709016+0000","last_undegraded":"2024-09-10T10:43:52.709016+0000","last_fullsized":"2024-09-10T10:43:52.709016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:42.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708786+0000","last_change":"2024-09-10T10:43:51.587866+0000","last_active":"2024-09-10T10:43:52.708786+0000","last_peered":"2024-09-10T10:43:52.708786+0000","last_clean":"2024-09-10T10:43:52.708786+0000","last_became_active":"2024-09-10T10:43:51.585690+0000","last_became_peered":"2024-09-10T10:43:51.585690+0000","last_unstale":"2024-09-10T10:43:52.708786+0000","last_undegraded":"2024-09-10T10:43:52.708786+0000","last_fullsized":"2024-09-10T10:43:52.708786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:41.798950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870081+0000","last_change":"2024-09-10T10:43:47.540910+0000","last_active":"2024-09-10T10:43:52.870081+0000","last_peered":"2024-09-10T10:43:52.870081+0000","last_clean":"2024-09-10T10:43:52.870081+0000","last_became_active":"2024-09-10T10:43:47.540603+0000","last_became_peered":"2024-09-10T10:43:47.540603+0000","last_unstale":"2024-09-10T10:43:52.870081+0000","last_undegraded":"2024-09-10T10:43:52.870081+0000","last_fullsized":"2024-09-10T10:43:52.870081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:48.137696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869935+0000","last_change":"2024-09-10T10:43:46.525479+0000","last_active":"2024-09-10T10:43:52.869935+0000","last_peered":"2024-09-10T10:43:52.869935+0000","last_clean":"2024-09-10T10:43:52.869935+0000","last_became_active":"2024-09-10T10:43:46.524834+0000","last_became_peered":"2024-09-10T10:43:46.524834+0000","last_unstale":"2024-09-10T10:43:52.869935+0000","last_undegraded":"2024-09-10T10:43:52.869935+0000","last_fullsized":"2024-09-10T10:43:52.869935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868574+0000","last_change":"2024-09-10T10:43:51.569746+0000","last_active":"2024-09-10T10:43:52.868574+0000","last_peered":"2024-09-10T10:43:52.868574+0000","last_clean":"2024-09-10T10:43:52.868574+0000","last_became_active":"2024-09-10T10:43:51.569289+0000","last_became_peered":"2024-09-10T10:43:51.569289+0000","last_unstale":"2024-09-10T10:43:52.868574+0000","last_undegraded":"2024-09-10T10:43:52.868574+0000","last_fullsized":"2024-09-10T10:43:52.868574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:20.122047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564540+0000","last_change":"2024-09-10T10:43:47.555824+0000","last_active":"2024-09-10T10:43:52.564540+0000","last_peered":"2024-09-10T10:43:52.564540+0000","last_clean":"2024-09-10T10:43:52.564540+0000","last_became_active":"2024-09-10T10:43:47.554957+0000","last_became_peered":"2024-09-10T10:43:47.554957+0000","last_unstale":"2024-09-10T10:43:52.564540+0000","last_undegraded":"2024-09-10T10:43:52.564540+0000","last_fullsized":"2024-09-10T10:43:52.564540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870114+0000","last_change":"2024-09-10T10:43:46.530043+0000","last_active":"2024-09-10T10:43:52.870114+0000","last_peered":"2024-09-10T10:43:52.870114+0000","last_clean":"2024-09-10T10:43:52.870114+0000","last_became_active":"2024-09-10T10:43:46.527509+0000","last_became_peered":"2024-09-10T10:43:46.527509+0000","last_unstale":"2024-09-10T10:43:52.870114+0000","last_undegraded":"2024-09-10T10:43:52.870114+0000","last_fullsized":"2024-09-10T10:43:52.870114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:08.859006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:51.577121+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:51.576021+0000","last_became_peered":"2024-09-10T10:43:51.576021+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:10.204576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868710+0000","last_change":"2024-09-10T10:43:47.562724+0000","last_active":"2024-09-10T10:43:52.868710+0000","last_peered":"2024-09-10T10:43:52.868710+0000","last_clean":"2024-09-10T10:43:52.868710+0000","last_became_active":"2024-09-10T10:43:47.562412+0000","last_became_peered":"2024-09-10T10:43:47.562412+0000","last_unstale":"2024-09-10T10:43:52.868710+0000","last_undegraded":"2024-09-10T10:43:52.868710+0000","last_fullsized":"2024-09-10T10:43:52.868710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:30.773785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869413+0000","last_change":"2024-09-10T10:43:46.525713+0000","last_active":"2024-09-10T10:43:52.869413+0000","last_peered":"2024-09-10T10:43:52.869413+0000","last_clean":"2024-09-10T10:43:52.869413+0000","last_became_active":"2024-09-10T10:43:46.525130+0000","last_became_peered":"2024-09-10T10:43:46.525130+0000","last_unstale":"2024-09-10T10:43:52.869413+0000","last_undegraded":"2024-09-10T10:43:52.869413+0000","last_fullsized":"2024-09-10T10:43:52.869413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709094+0000","last_change":"2024-09-10T10:43:51.582330+0000","last_active":"2024-09-10T10:43:52.709094+0000","last_peered":"2024-09-10T10:43:52.709094+0000","last_clean":"2024-09-10T10:43:52.709094+0000","last_became_active":"2024-09-10T10:43:51.581415+0000","last_became_peered":"2024-09-10T10:43:51.581415+0000","last_unstale":"2024-09-10T10:43:52.709094+0000","last_undegraded":"2024-09-10T10:43:52.709094+0000","last_fullsized":"2024-09-10T10:43:52.709094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:18.520413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564097+0000","last_change":"2024-09-10T10:43:47.537959+0000","last_active":"2024-09-10T10:43:52.564097+0000","last_peered":"2024-09-10T10:43:52.564097+0000","last_clean":"2024-09-10T10:43:52.564097+0000","last_became_active":"2024-09-10T10:43:47.537705+0000","last_became_peered":"2024-09-10T10:43:47.537705+0000","last_unstale":"2024-09-10T10:43:52.564097+0000","last_undegraded":"2024-09-10T10:43:52.564097+0000","last_fullsized":"2024-09-10T10:43:52.564097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.301556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869080+0000","last_change":"2024-09-10T10:43:46.539973+0000","last_active":"2024-09-10T10:43:52.869080+0000","last_peered":"2024-09-10T10:43:52.869080+0000","last_clean":"2024-09-10T10:43:52.869080+0000","last_became_active":"2024-09-10T10:43:46.539509+0000","last_became_peered":"2024-09-10T10:43:46.539509+0000","last_unstale":"2024-09-10T10:43:52.869080+0000","last_undegraded":"2024-09-10T10:43:52.869080+0000","last_fullsized":"2024-09-10T10:43:52.869080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:51.584701+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:51.584321+0000","last_became_peered":"2024-09-10T10:43:51.584321+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:07.697012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869811+0000","last_change":"2024-09-10T10:43:47.549457+0000","last_active":"2024-09-10T10:43:52.869811+0000","last_peered":"2024-09-10T10:43:52.869811+0000","last_clean":"2024-09-10T10:43:52.869811+0000","last_became_active":"2024-09-10T10:43:47.549210+0000","last_became_peered":"2024-09-10T10:43:47.549210+0000","last_unstale":"2024-09-10T10:43:52.869811+0000","last_undegraded":"2024-09-10T10:43:52.869811+0000","last_fullsized":"2024-09-10T10:43:52.869811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:00.039891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869773+0000","last_change":"2024-09-10T10:43:46.520041+0000","last_active":"2024-09-10T10:43:52.869773+0000","last_peered":"2024-09-10T10:43:52.869773+0000","last_clean":"2024-09-10T10:43:52.869773+0000","last_became_active":"2024-09-10T10:43:46.519721+0000","last_became_peered":"2024-09-10T10:43:46.519721+0000","last_unstale":"2024-09-10T10:43:52.869773+0000","last_undegraded":"2024-09-10T10:43:52.869773+0000","last_fullsized":"2024-09-10T10:43:52.869773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:23.802544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870586+0000","last_change":"2024-09-10T10:43:51.571179+0000","last_active":"2024-09-10T10:43:52.870586+0000","last_peered":"2024-09-10T10:43:52.870586+0000","last_clean":"2024-09-10T10:43:52.870586+0000","last_became_active":"2024-09-10T10:43:51.570590+0000","last_became_peered":"2024-09-10T10:43:51.570590+0000","last_unstale":"2024-09-10T10:43:52.870586+0000","last_undegraded":"2024-09-10T10:43:52.870586+0000","last_fullsized":"2024-09-10T10:43:52.870586+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870110+0000","last_change":"2024-09-10T10:43:47.546795+0000","last_active":"2024-09-10T10:43:52.870110+0000","last_peered":"2024-09-10T10:43:52.870110+0000","last_clean":"2024-09-10T10:43:52.870110+0000","last_became_active":"2024-09-10T10:43:47.546529+0000","last_became_peered":"2024-09-10T10:43:47.546529+0000","last_unstale":"2024-09-10T10:43:52.870110+0000","last_undegraded":"2024-09-10T10:43:52.870110+0000","last_fullsized":"2024-09-10T10:43:52.870110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564317+0000","last_change":"2024-09-10T10:43:46.524422+0000","last_active":"2024-09-10T10:43:52.564317+0000","last_peered":"2024-09-10T10:43:52.564317+0000","last_clean":"2024-09-10T10:43:52.564317+0000","last_became_active":"2024-09-10T10:43:46.524027+0000","last_became_peered":"2024-09-10T10:43:46.524027+0000","last_unstale":"2024-09-10T10:43:52.564317+0000","last_undegraded":"2024-09-10T10:43:52.564317+0000","last_fullsized":"2024-09-10T10:43:52.564317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.575001+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.573805+0000","last_became_peered":"2024-09-10T10:43:51.573805+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:27.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869516+0000","last_change":"2024-09-10T10:43:47.542333+0000","last_active":"2024-09-10T10:43:52.869516+0000","last_peered":"2024-09-10T10:43:52.869516+0000","last_clean":"2024-09-10T10:43:52.869516+0000","last_became_active":"2024-09-10T10:43:47.540584+0000","last_became_peered":"2024-09-10T10:43:47.540584+0000","last_unstale":"2024-09-10T10:43:52.869516+0000","last_undegraded":"2024-09-10T10:43:52.869516+0000","last_fullsized":"2024-09-10T10:43:52.869516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:08.456485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:58.645779+0000","last_change":"2024-09-10T10:43:46.531547+0000","last_active":"2024-09-10T10:43:58.645779+0000","last_peered":"2024-09-10T10:43:58.645779+0000","last_clean":"2024-09-10T10:43:58.645779+0000","last_became_active":"2024-09-10T10:43:46.530309+0000","last_became_peered":"2024-09-10T10:43:46.530309+0000","last_unstale":"2024-09-10T10:43:58.645779+0000","last_undegraded":"2024-09-10T10:43:58.645779+0000","last_fullsized":"2024-09-10T10:43:58.645779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:54.587795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869118+0000","last_change":"2024-09-10T10:43:51.578360+0000","last_active":"2024-09-10T10:43:52.869118+0000","last_peered":"2024-09-10T10:43:52.869118+0000","last_clean":"2024-09-10T10:43:52.869118+0000","last_became_active":"2024-09-10T10:43:51.578021+0000","last_became_peered":"2024-09-10T10:43:51.578021+0000","last_unstale":"2024-09-10T10:43:52.869118+0000","last_undegraded":"2024-09-10T10:43:52.869118+0000","last_fullsized":"2024-09-10T10:43:52.869118+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870009+0000","last_change":"2024-09-10T10:43:47.564353+0000","last_active":"2024-09-10T10:43:52.870009+0000","last_peered":"2024-09-10T10:43:52.870009+0000","last_clean":"2024-09-10T10:43:52.870009+0000","last_became_active":"2024-09-10T10:43:47.563999+0000","last_became_peered":"2024-09-10T10:43:47.563999+0000","last_unstale":"2024-09-10T10:43:52.870009+0000","last_undegraded":"2024-09-10T10:43:52.870009+0000","last_fullsized":"2024-09-10T10:43:52.870009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:52.338110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564851+0000","last_change":"2024-09-10T10:43:46.526185+0000","last_active":"2024-09-10T10:43:52.564851+0000","last_peered":"2024-09-10T10:43:52.564851+0000","last_clean":"2024-09-10T10:43:52.564851+0000","last_became_active":"2024-09-10T10:43:46.525887+0000","last_became_peered":"2024-09-10T10:43:46.525887+0000","last_unstale":"2024-09-10T10:43:52.564851+0000","last_undegraded":"2024-09-10T10:43:52.564851+0000","last_fullsized":"2024-09-10T10:43:52.564851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563403+0000","last_change":"2024-09-10T10:43:51.587988+0000","last_active":"2024-09-10T10:43:52.563403+0000","last_peered":"2024-09-10T10:43:52.563403+0000","last_clean":"2024-09-10T10:43:52.563403+0000","last_became_active":"2024-09-10T10:43:51.587584+0000","last_became_peered":"2024-09-10T10:43:51.587584+0000","last_unstale":"2024-09-10T10:43:52.563403+0000","last_undegraded":"2024-09-10T10:43:52.563403+0000","last_fullsized":"2024-09-10T10:43:52.563403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708986+0000","last_change":"2024-09-10T10:43:47.541815+0000","last_active":"2024-09-10T10:43:52.708986+0000","last_peered":"2024-09-10T10:43:52.708986+0000","last_clean":"2024-09-10T10:43:52.708986+0000","last_became_active":"2024-09-10T10:43:47.541530+0000","last_became_peered":"2024-09-10T10:43:47.541530+0000","last_unstale":"2024-09-10T10:43:52.708986+0000","last_undegraded":"2024-09-10T10:43:52.708986+0000","last_fullsized":"2024-09-10T10:43:52.708986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:26.960082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869496+0000","last_change":"2024-09-10T10:43:46.525990+0000","last_active":"2024-09-10T10:43:52.869496+0000","last_peered":"2024-09-10T10:43:52.869496+0000","last_clean":"2024-09-10T10:43:52.869496+0000","last_became_active":"2024-09-10T10:43:46.524901+0000","last_became_peered":"2024-09-10T10:43:46.524901+0000","last_unstale":"2024-09-10T10:43:52.869496+0000","last_undegraded":"2024-09-10T10:43:52.869496+0000","last_fullsized":"2024-09-10T10:43:52.869496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:12.069623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563676+0000","last_change":"2024-09-10T10:43:51.574109+0000","last_active":"2024-09-10T10:43:52.563676+0000","last_peered":"2024-09-10T10:43:52.563676+0000","last_clean":"2024-09-10T10:43:52.563676+0000","last_became_active":"2024-09-10T10:43:51.573718+0000","last_became_peered":"2024-09-10T10:43:51.573718+0000","last_unstale":"2024-09-10T10:43:52.563676+0000","last_undegraded":"2024-09-10T10:43:52.563676+0000","last_fullsized":"2024-09-10T10:43:52.563676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:13.623233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564443+0000","last_change":"2024-09-10T10:43:47.558084+0000","last_active":"2024-09-10T10:43:52.564443+0000","last_peered":"2024-09-10T10:43:52.564443+0000","last_clean":"2024-09-10T10:43:52.564443+0000","last_became_active":"2024-09-10T10:43:47.555632+0000","last_became_peered":"2024-09-10T10:43:47.555632+0000","last_unstale":"2024-09-10T10:43:52.564443+0000","last_undegraded":"2024-09-10T10:43:52.564443+0000","last_fullsized":"2024-09-10T10:43:52.564443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565392+0000","last_change":"2024-09-10T10:43:46.528292+0000","last_active":"2024-09-10T10:43:52.565392+0000","last_peered":"2024-09-10T10:43:52.565392+0000","last_clean":"2024-09-10T10:43:52.565392+0000","last_became_active":"2024-09-10T10:43:46.527943+0000","last_became_peered":"2024-09-10T10:43:46.527943+0000","last_unstale":"2024-09-10T10:43:52.565392+0000","last_undegraded":"2024-09-10T10:43:52.565392+0000","last_fullsized":"2024-09-10T10:43:52.565392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869189+0000","last_change":"2024-09-10T10:43:51.572294+0000","last_active":"2024-09-10T10:43:52.869189+0000","last_peered":"2024-09-10T10:43:52.869189+0000","last_clean":"2024-09-10T10:43:52.869189+0000","last_became_active":"2024-09-10T10:43:51.571796+0000","last_became_peered":"2024-09-10T10:43:51.571796+0000","last_unstale":"2024-09-10T10:43:52.869189+0000","last_undegraded":"2024-09-10T10:43:52.869189+0000","last_fullsized":"2024-09-10T10:43:52.869189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869838+0000","last_change":"2024-09-10T10:43:51.577672+0000","last_active":"2024-09-10T10:43:52.869838+0000","last_peered":"2024-09-10T10:43:52.869838+0000","last_clean":"2024-09-10T10:43:52.869838+0000","last_became_active":"2024-09-10T10:43:51.577342+0000","last_became_peered":"2024-09-10T10:43:51.577342+0000","last_unstale":"2024-09-10T10:43:52.869838+0000","last_undegraded":"2024-09-10T10:43:52.869838+0000","last_fullsized":"2024-09-10T10:43:52.869838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:59.764337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869129+0000","last_change":"2024-09-10T10:43:47.538181+0000","last_active":"2024-09-10T10:43:52.869129+0000","last_peered":"2024-09-10T10:43:52.869129+0000","last_clean":"2024-09-10T10:43:52.869129+0000","last_became_active":"2024-09-10T10:43:47.537810+0000","last_became_peered":"2024-09-10T10:43:47.537810+0000","last_unstale":"2024-09-10T10:43:52.869129+0000","last_undegraded":"2024-09-10T10:43:52.869129+0000","last_fullsized":"2024-09-10T10:43:52.869129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868780+0000","last_change":"2024-09-10T10:43:46.533440+0000","last_active":"2024-09-10T10:43:52.868780+0000","last_peered":"2024-09-10T10:43:52.868780+0000","last_clean":"2024-09-10T10:43:52.868780+0000","last_became_active":"2024-09-10T10:43:46.532814+0000","last_became_peered":"2024-09-10T10:43:46.532814+0000","last_unstale":"2024-09-10T10:43:52.868780+0000","last_undegraded":"2024-09-10T10:43:52.868780+0000","last_fullsized":"2024-09-10T10:43:52.868780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:03.561914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870537+0000","last_change":"2024-09-10T10:43:51.576277+0000","last_active":"2024-09-10T10:43:52.870537+0000","last_peered":"2024-09-10T10:43:52.870537+0000","last_clean":"2024-09-10T10:43:52.870537+0000","last_became_active":"2024-09-10T10:43:51.575888+0000","last_became_peered":"2024-09-10T10:43:51.575888+0000","last_unstale":"2024-09-10T10:43:52.870537+0000","last_undegraded":"2024-09-10T10:43:52.870537+0000","last_fullsized":"2024-09-10T10:43:52.870537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:37.271562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708364+0000","last_change":"2024-09-10T10:43:47.539801+0000","last_active":"2024-09-10T10:43:52.708364+0000","last_peered":"2024-09-10T10:43:52.708364+0000","last_clean":"2024-09-10T10:43:52.708364+0000","last_became_active":"2024-09-10T10:43:47.535922+0000","last_became_peered":"2024-09-10T10:43:47.535922+0000","last_unstale":"2024-09-10T10:43:52.708364+0000","last_undegraded":"2024-09-10T10:43:52.708364+0000","last_fullsized":"2024-09-10T10:43:52.708364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:49.905762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869706+0000","last_change":"2024-09-10T10:43:46.530425+0000","last_active":"2024-09-10T10:43:52.869706+0000","last_peered":"2024-09-10T10:43:52.869706+0000","last_clean":"2024-09-10T10:43:52.869706+0000","last_became_active":"2024-09-10T10:43:46.530158+0000","last_became_peered":"2024-09-10T10:43:46.530158+0000","last_unstale":"2024-09-10T10:43:52.869706+0000","last_undegraded":"2024-09-10T10:43:52.869706+0000","last_fullsized":"2024-09-10T10:43:52.869706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869653+0000","last_change":"2024-09-10T10:43:46.537743+0000","last_active":"2024-09-10T10:43:52.869653+0000","last_peered":"2024-09-10T10:43:52.869653+0000","last_clean":"2024-09-10T10:43:52.869653+0000","last_became_active":"2024-09-10T10:43:46.537260+0000","last_became_peered":"2024-09-10T10:43:46.537260+0000","last_unstale":"2024-09-10T10:43:52.869653+0000","last_undegraded":"2024-09-10T10:43:52.869653+0000","last_fullsized":"2024-09-10T10:43:52.869653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:53.479099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869441+0000","last_change":"2024-09-10T10:43:47.545017+0000","last_active":"2024-09-10T10:43:52.869441+0000","last_peered":"2024-09-10T10:43:52.869441+0000","last_clean":"2024-09-10T10:43:52.869441+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.869441+0000","last_undegraded":"2024-09-10T10:43:52.869441+0000","last_fullsized":"2024-09-10T10:43:52.869441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:46.823226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869679+0000","last_change":"2024-09-10T10:43:51.570454+0000","last_active":"2024-09-10T10:43:52.869679+0000","last_peered":"2024-09-10T10:43:52.869679+0000","last_clean":"2024-09-10T10:43:52.869679+0000","last_became_active":"2024-09-10T10:43:51.570088+0000","last_became_peered":"2024-09-10T10:43:51.570088+0000","last_unstale":"2024-09-10T10:43:52.869679+0000","last_undegraded":"2024-09-10T10:43:52.869679+0000","last_fullsized":"2024-09-10T10:43:52.869679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:52.179305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708948+0000","last_change":"2024-09-10T10:43:46.521135+0000","last_active":"2024-09-10T10:43:52.708948+0000","last_peered":"2024-09-10T10:43:52.708948+0000","last_clean":"2024-09-10T10:43:52.708948+0000","last_became_active":"2024-09-10T10:43:46.520815+0000","last_became_peered":"2024-09-10T10:43:46.520815+0000","last_unstale":"2024-09-10T10:43:52.708948+0000","last_undegraded":"2024-09-10T10:43:52.708948+0000","last_fullsized":"2024-09-10T10:43:52.708948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:38.222213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564462+0000","last_change":"2024-09-10T10:43:47.545783+0000","last_active":"2024-09-10T10:43:52.564462+0000","last_peered":"2024-09-10T10:43:52.564462+0000","last_clean":"2024-09-10T10:43:52.564462+0000","last_became_active":"2024-09-10T10:43:47.545265+0000","last_became_peered":"2024-09-10T10:43:47.545265+0000","last_unstale":"2024-09-10T10:43:52.564462+0000","last_undegraded":"2024-09-10T10:43:52.564462+0000","last_fullsized":"2024-09-10T10:43:52.564462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564298+0000","last_change":"2024-09-10T10:43:51.585486+0000","last_active":"2024-09-10T10:43:52.564298+0000","last_peered":"2024-09-10T10:43:52.564298+0000","last_clean":"2024-09-10T10:43:52.564298+0000","last_became_active":"2024-09-10T10:43:51.585045+0000","last_became_peered":"2024-09-10T10:43:51.585045+0000","last_unstale":"2024-09-10T10:43:52.564298+0000","last_undegraded":"2024-09-10T10:43:52.564298+0000","last_fullsized":"2024-09-10T10:43:52.564298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:59.749737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563314+0000","last_change":"2024-09-10T10:43:47.533694+0000","last_active":"2024-09-10T10:43:52.563314+0000","last_peered":"2024-09-10T10:43:52.563314+0000","last_clean":"2024-09-10T10:43:52.563314+0000","last_became_active":"2024-09-10T10:43:47.533197+0000","last_became_peered":"2024-09-10T10:43:47.533197+0000","last_unstale":"2024-09-10T10:43:52.563314+0000","last_undegraded":"2024-09-10T10:43:52.563314+0000","last_fullsized":"2024-09-10T10:43:52.563314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563761+0000","last_change":"2024-09-10T10:43:47.540401+0000","last_active":"2024-09-10T10:43:52.563761+0000","last_peered":"2024-09-10T10:43:52.563761+0000","last_clean":"2024-09-10T10:43:52.563761+0000","last_became_active":"2024-09-10T10:43:47.539677+0000","last_became_peered":"2024-09-10T10:43:47.539677+0000","last_unstale":"2024-09-10T10:43:52.563761+0000","last_undegraded":"2024-09-10T10:43:52.563761+0000","last_fullsized":"2024-09-10T10:43:52.563761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868918+0000","last_change":"2024-09-10T10:43:47.539352+0000","last_active":"2024-09-10T10:43:52.868918+0000","last_peered":"2024-09-10T10:43:52.868918+0000","last_clean":"2024-09-10T10:43:52.868918+0000","last_became_active":"2024-09-10T10:43:47.539034+0000","last_became_peered":"2024-09-10T10:43:47.539034+0000","last_unstale":"2024-09-10T10:43:52.868918+0000","last_undegraded":"2024-09-10T10:43:52.868918+0000","last_fullsized":"2024-09-10T10:43:52.868918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:08.107825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708757+0000","last_change":"2024-09-10T10:43:47.542199+0000","last_active":"2024-09-10T10:43:52.708757+0000","last_peered":"2024-09-10T10:43:52.708757+0000","last_clean":"2024-09-10T10:43:52.708757+0000","last_became_active":"2024-09-10T10:43:47.535131+0000","last_became_peered":"2024-09-10T10:43:47.535131+0000","last_unstale":"2024-09-10T10:43:52.708757+0000","last_undegraded":"2024-09-10T10:43:52.708757+0000","last_fullsized":"2024-09-10T10:43:52.708757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:38.966819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870705+0000","last_change":"2024-09-10T10:43:47.546290+0000","last_active":"2024-09-10T10:43:52.870705+0000","last_peered":"2024-09-10T10:43:52.870705+0000","last_clean":"2024-09-10T10:43:52.870705+0000","last_became_active":"2024-09-10T10:43:47.545873+0000","last_became_peered":"2024-09-10T10:43:47.545873+0000","last_unstale":"2024-09-10T10:43:52.870705+0000","last_undegraded":"2024-09-10T10:43:52.870705+0000","last_fullsized":"2024-09-10T10:43:52.870705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870373+0000","last_change":"2024-09-10T10:43:47.537092+0000","last_active":"2024-09-10T10:43:52.870373+0000","last_peered":"2024-09-10T10:43:52.870373+0000","last_clean":"2024-09-10T10:43:52.870373+0000","last_became_active":"2024-09-10T10:43:47.536730+0000","last_became_peered":"2024-09-10T10:43:47.536730+0000","last_unstale":"2024-09-10T10:43:52.870373+0000","last_undegraded":"2024-09-10T10:43:52.870373+0000","last_fullsized":"2024-09-10T10:43:52.870373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564347+0000","last_change":"2024-09-10T10:43:47.557748+0000","last_active":"2024-09-10T10:43:52.564347+0000","last_peered":"2024-09-10T10:43:52.564347+0000","last_clean":"2024-09-10T10:43:52.564347+0000","last_became_active":"2024-09-10T10:43:47.555551+0000","last_became_peered":"2024-09-10T10:43:47.555551+0000","last_unstale":"2024-09-10T10:43:52.564347+0000","last_undegraded":"2024-09-10T10:43:52.564347+0000","last_fullsized":"2024-09-10T10:43:52.564347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708691+0000","last_change":"2024-09-10T10:43:47.544639+0000","last_active":"2024-09-10T10:43:52.708691+0000","last_peered":"2024-09-10T10:43:52.708691+0000","last_clean":"2024-09-10T10:43:52.708691+0000","last_became_active":"2024-09-10T10:43:47.544357+0000","last_became_peered":"2024-09-10T10:43:47.544357+0000","last_unstale":"2024-09-10T10:43:52.708691+0000","last_undegraded":"2024-09-10T10:43:52.708691+0000","last_fullsized":"2024-09-10T10:43:52.708691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:16.174872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868699+0000","last_change":"2024-09-10T10:43:47.544034+0000","last_active":"2024-09-10T10:43:52.868699+0000","last_peered":"2024-09-10T10:43:52.868699+0000","last_clean":"2024-09-10T10:43:52.868699+0000","last_became_active":"2024-09-10T10:43:47.543677+0000","last_became_peered":"2024-09-10T10:43:47.543677+0000","last_unstale":"2024-09-10T10:43:52.868699+0000","last_undegraded":"2024-09-10T10:43:52.868699+0000","last_fullsized":"2024-09-10T10:43:52.868699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:26.220907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708358+0000","last_change":"2024-09-10T10:43:47.561856+0000","last_active":"2024-09-10T10:43:52.708358+0000","last_peered":"2024-09-10T10:43:52.708358+0000","last_clean":"2024-09-10T10:43:52.708358+0000","last_became_active":"2024-09-10T10:43:47.561518+0000","last_became_peered":"2024-09-10T10:43:47.561518+0000","last_unstale":"2024-09-10T10:43:52.708358+0000","last_undegraded":"2024-09-10T10:43:52.708358+0000","last_fullsized":"2024-09-10T10:43:52.708358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:21.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:47.545540+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:47.544916+0000","last_became_peered":"2024-09-10T10:43:47.544916+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869156+0000","last_change":"2024-09-10T10:43:47.521227+0000","last_active":"2024-09-10T10:43:52.869156+0000","last_peered":"2024-09-10T10:43:52.869156+0000","last_clean":"2024-09-10T10:43:52.869156+0000","last_became_active":"2024-09-10T10:43:47.520811+0000","last_became_peered":"2024-09-10T10:43:47.520811+0000","last_unstale":"2024-09-10T10:43:52.869156+0000","last_undegraded":"2024-09-10T10:43:52.869156+0000","last_fullsized":"2024-09-10T10:43:52.869156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:55.708127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869588+0000","last_change":"2024-09-10T10:43:47.541681+0000","last_active":"2024-09-10T10:43:52.869588+0000","last_peered":"2024-09-10T10:43:52.869588+0000","last_clean":"2024-09-10T10:43:52.869588+0000","last_became_active":"2024-09-10T10:43:47.541056+0000","last_became_peered":"2024-09-10T10:43:47.541056+0000","last_unstale":"2024-09-10T10:43:52.869588+0000","last_undegraded":"2024-09-10T10:43:52.869588+0000","last_fullsized":"2024-09-10T10:43:52.869588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:56.224306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870640+0000","last_change":"2024-09-10T10:43:47.543423+0000","last_active":"2024-09-10T10:43:52.870640+0000","last_peered":"2024-09-10T10:43:52.870640+0000","last_clean":"2024-09-10T10:43:52.870640+0000","last_became_active":"2024-09-10T10:43:47.537728+0000","last_became_peered":"2024-09-10T10:43:47.537728+0000","last_unstale":"2024-09-10T10:43:52.870640+0000","last_undegraded":"2024-09-10T10:43:52.870640+0000","last_fullsized":"2024-09-10T10:43:52.870640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:10.407503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564680+0000","last_change":"2024-09-10T10:43:47.562659+0000","last_active":"2024-09-10T10:43:52.564680+0000","last_peered":"2024-09-10T10:43:52.564680+0000","last_clean":"2024-09-10T10:43:52.564680+0000","last_became_active":"2024-09-10T10:43:47.562238+0000","last_became_peered":"2024-09-10T10:43:47.562238+0000","last_unstale":"2024-09-10T10:43:52.564680+0000","last_undegraded":"2024-09-10T10:43:52.564680+0000","last_fullsized":"2024-09-10T10:43:52.564680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870710+0000","last_change":"2024-09-10T10:43:47.548547+0000","last_active":"2024-09-10T10:43:52.870710+0000","last_peered":"2024-09-10T10:43:52.870710+0000","last_clean":"2024-09-10T10:43:52.870710+0000","last_became_active":"2024-09-10T10:43:47.548237+0000","last_became_peered":"2024-09-10T10:43:47.548237+0000","last_unstale":"2024-09-10T10:43:52.870710+0000","last_undegraded":"2024-09-10T10:43:52.870710+0000","last_fullsized":"2024-09-10T10:43:52.870710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:55.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868674+0000","last_change":"2024-09-10T10:43:47.559813+0000","last_active":"2024-09-10T10:43:52.868674+0000","last_peered":"2024-09-10T10:43:52.868674+0000","last_clean":"2024-09-10T10:43:52.868674+0000","last_became_active":"2024-09-10T10:43:47.559517+0000","last_became_peered":"2024-09-10T10:43:47.559517+0000","last_unstale":"2024-09-10T10:43:52.868674+0000","last_undegraded":"2024-09-10T10:43:52.868674+0000","last_fullsized":"2024-09-10T10:43:52.868674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:28.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.546737+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.546456+0000","last_became_peered":"2024-09-10T10:43:47.546456+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:42.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870025+0000","last_change":"2024-09-10T10:43:47.533339+0000","last_active":"2024-09-10T10:43:52.870025+0000","last_peered":"2024-09-10T10:43:52.870025+0000","last_clean":"2024-09-10T10:43:52.870025+0000","last_became_active":"2024-09-10T10:43:47.533019+0000","last_became_peered":"2024-09-10T10:43:47.533019+0000","last_unstale":"2024-09-10T10:43:52.870025+0000","last_undegraded":"2024-09-10T10:43:52.870025+0000","last_fullsized":"2024-09-10T10:43:52.870025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870439+0000","last_change":"2024-09-10T10:43:47.546293+0000","last_active":"2024-09-10T10:43:52.870439+0000","last_peered":"2024-09-10T10:43:52.870439+0000","last_clean":"2024-09-10T10:43:52.870439+0000","last_became_active":"2024-09-10T10:43:47.545952+0000","last_became_peered":"2024-09-10T10:43:47.545952+0000","last_unstale":"2024-09-10T10:43:52.870439+0000","last_undegraded":"2024-09-10T10:43:52.870439+0000","last_fullsized":"2024-09-10T10:43:52.870439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869535+0000","last_change":"2024-09-10T10:43:47.541403+0000","last_active":"2024-09-10T10:43:52.869535+0000","last_peered":"2024-09-10T10:43:52.869535+0000","last_clean":"2024-09-10T10:43:52.869535+0000","last_became_active":"2024-09-10T10:43:47.540724+0000","last_became_peered":"2024-09-10T10:43:47.540724+0000","last_unstale":"2024-09-10T10:43:52.869535+0000","last_undegraded":"2024-09-10T10:43:52.869535+0000","last_fullsized":"2024-09-10T10:43:52.869535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:55.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869473+0000","last_change":"2024-09-10T10:43:47.550017+0000","last_active":"2024-09-10T10:43:52.869473+0000","last_peered":"2024-09-10T10:43:52.869473+0000","last_clean":"2024-09-10T10:43:52.869473+0000","last_became_active":"2024-09-10T10:43:47.549763+0000","last_became_peered":"2024-09-10T10:43:47.549763+0000","last_unstale":"2024-09-10T10:43:52.869473+0000","last_undegraded":"2024-09-10T10:43:52.869473+0000","last_fullsized":"2024-09-10T10:43:52.869473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:12.076487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869399+0000","last_change":"2024-09-10T10:43:47.558985+0000","last_active":"2024-09-10T10:43:52.869399+0000","last_peered":"2024-09-10T10:43:52.869399+0000","last_clean":"2024-09-10T10:43:52.869399+0000","last_became_active":"2024-09-10T10:43:47.558695+0000","last_became_peered":"2024-09-10T10:43:47.558695+0000","last_unstale":"2024-09-10T10:43:52.869399+0000","last_undegraded":"2024-09-10T10:43:52.869399+0000","last_fullsized":"2024-09-10T10:43:52.869399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:05.319579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565366+0000","last_change":"2024-09-10T10:43:47.547413+0000","last_active":"2024-09-10T10:43:52.565366+0000","last_peered":"2024-09-10T10:43:52.565366+0000","last_clean":"2024-09-10T10:43:52.565366+0000","last_became_active":"2024-09-10T10:43:47.547008+0000","last_became_peered":"2024-09-10T10:43:47.547008+0000","last_unstale":"2024-09-10T10:43:52.565366+0000","last_undegraded":"2024-09-10T10:43:52.565366+0000","last_fullsized":"2024-09-10T10:43:52.565366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.545409+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.545121+0000","last_became_peered":"2024-09-10T10:43:47.545121+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.641844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869042+0000","last_change":"2024-09-10T10:43:47.534510+0000","last_active":"2024-09-10T10:43:52.869042+0000","last_peered":"2024-09-10T10:43:52.869042+0000","last_clean":"2024-09-10T10:43:52.869042+0000","last_became_active":"2024-09-10T10:43:47.534143+0000","last_became_peered":"2024-09-10T10:43:47.534143+0000","last_unstale":"2024-09-10T10:43:52.869042+0000","last_undegraded":"2024-09-10T10:43:52.869042+0000","last_fullsized":"2024-09-10T10:43:52.869042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:33.210661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563579+0000","last_change":"2024-09-10T10:43:47.525750+0000","last_active":"2024-09-10T10:43:52.563579+0000","last_peered":"2024-09-10T10:43:52.563579+0000","last_clean":"2024-09-10T10:43:52.563579+0000","last_became_active":"2024-09-10T10:43:47.525379+0000","last_became_peered":"2024-09-10T10:43:47.525379+0000","last_unstale":"2024-09-10T10:43:52.563579+0000","last_undegraded":"2024-09-10T10:43:52.563579+0000","last_fullsized":"2024-09-10T10:43:52.563579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869264+0000","last_change":"2024-09-10T10:43:47.546839+0000","last_active":"2024-09-10T10:43:52.869264+0000","last_peered":"2024-09-10T10:43:52.869264+0000","last_clean":"2024-09-10T10:43:52.869264+0000","last_became_active":"2024-09-10T10:43:47.544570+0000","last_became_peered":"2024-09-10T10:43:47.544570+0000","last_unstale":"2024-09-10T10:43:52.869264+0000","last_undegraded":"2024-09-10T10:43:52.869264+0000","last_fullsized":"2024-09-10T10:43:52.869264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:53.642976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564645+0000","last_change":"2024-09-10T10:43:47.522224+0000","last_active":"2024-09-10T10:43:52.564645+0000","last_peered":"2024-09-10T10:43:52.564645+0000","last_clean":"2024-09-10T10:43:52.564645+0000","last_became_active":"2024-09-10T10:43:47.521820+0000","last_became_peered":"2024-09-10T10:43:47.521820+0000","last_unstale":"2024-09-10T10:43:52.564645+0000","last_undegraded":"2024-09-10T10:43:52.564645+0000","last_fullsized":"2024-09-10T10:43:52.564645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:47.538393+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:47.537972+0000","last_became_peered":"2024-09-10T10:43:47.537972+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:30.153548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708731+0000","last_change":"2024-09-10T10:43:47.540588+0000","last_active":"2024-09-10T10:43:52.708731+0000","last_peered":"2024-09-10T10:43:52.708731+0000","last_clean":"2024-09-10T10:43:52.708731+0000","last_became_active":"2024-09-10T10:43:47.540358+0000","last_became_peered":"2024-09-10T10:43:47.540358+0000","last_unstale":"2024-09-10T10:43:52.708731+0000","last_undegraded":"2024-09-10T10:43:52.708731+0000","last_fullsized":"2024-09-10T10:43:52.708731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:30.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:47.549392+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:47.546137+0000","last_became_peered":"2024-09-10T10:43:47.546137+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:02.975066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":111561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":111437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":115281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":701875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":111493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":884736,"data_stored":701897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:09.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:44:10.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.022+0000 7f0c43200640 1 Processor -- start 2024-09-10T10:44:10.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.022+0000 7f0c43200640 1 -- start start 2024-09-10T10:44:10.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.023+0000 7f0c43200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0c440d0dd0 0x7f0c440d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.023+0000 7f0c43200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.023+0000 7f0c43200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c440cf000 con 0x7f0c440d1f90 2024-09-10T10:44:10.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.023+0000 7f0c43200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0c440b37d0 con 0x7f0c440d0dd0 2024-09-10T10:44:10.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.023+0000 7f0c43200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0c440cf330 con 0x7f0c4415e660 2024-09-10T10:44:10.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44098/0 (socket says 172.21.15.28:44098) 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 -- 172.21.15.28:0/3629914657 learned_addr learned my addr 172.21.15.28:0/3629914657 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41e00640 1 --2- 172.21.15.28:0/3629914657 >> v2:172.21.15.138:3300/0 conn(0x7f0c440d0dd0 0x7f0c440d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 -- 172.21.15.28:0/3629914657 >> v2:172.21.15.138:3300/0 conn(0x7f0c440d0dd0 msgr2=0x7f0c440d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 --2- 172.21.15.28:0/3629914657 >> v2:172.21.15.138:3300/0 conn(0x7f0c440d0dd0 0x7f0c440d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 -- 172.21.15.28:0/3629914657 >> v1:172.21.15.138:6789/0 conn(0x7f0c440d1f90 legacy=0x7f0c440cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.024+0000 7f0c41400640 1 -- 172.21.15.28:0/3629914657 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c440d16f0 con 0x7f0c4415e660 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.025+0000 7f0c41400640 1 --2- 172.21.15.28:0/3629914657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009870 tx=0x7f0c3002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=743bbb662a2b1cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.025+0000 7f0c40a00640 1 -- 172.21.15.28:0/3629914657 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30037070 con 0x7f0c4415e660 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.025+0000 7f0c40a00640 1 -- 172.21.15.28:0/3629914657 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c30032440 con 0x7f0c4415e660 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.025+0000 7f0c40a00640 1 -- 172.21.15.28:0/3629914657 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30032810 con 0x7f0c4415e660 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 -- 172.21.15.28:0/3629914657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 msgr2=0x7f0c4406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 --2- 172.21.15.28:0/3629914657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009870 tx=0x7f0c3002a250 comp rx=0 tx=0).stop 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 -- 172.21.15.28:0/3629914657 shutdown_connections 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 --2- 172.21.15.28:0/3629914657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c4415e660 0x7f0c4406ccd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 --2- 172.21.15.28:0/3629914657 >> v2:172.21.15.138:3300/0 conn(0x7f0c440d0dd0 0x7f0c440d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 -- 172.21.15.28:0/3629914657 >> 172.21.15.28:0/3629914657 conn(0x7f0c4408bfc0 msgr2=0x7f0c440d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 -- 172.21.15.28:0/3629914657 shutdown_connections 2024-09-10T10:44:10.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.026+0000 7f0c43200640 1 -- 172.21.15.28:0/3629914657 wait complete. 2024-09-10T10:44:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.028+0000 7f0c43200640 1 Processor -- start 2024-09-10T10:44:10.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.028+0000 7f0c43200640 1 -- start start 2024-09-10T10:44:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 0x7f0c44157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c4415e660 0x7f0c441e4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0c440d2f40 con 0x7f0c4415e660 2024-09-10T10:44:10.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c440d2c60 con 0x7f0c440d0dd0 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c43200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0c440d3220 con 0x7f0c440d1f90 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c41400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c41e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 0x7f0c44157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c41400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44112/0 (socket says 172.21.15.28:44112) 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c41400640 1 -- 172.21.15.28:0/4076172971 learned_addr learned my addr 172.21.15.28:0/4076172971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.029+0000 7f0c42800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c4415e660 0x7f0c441e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 -- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 msgr2=0x7f0c44157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 0x7f0c44157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 -- 172.21.15.28:0/4076172971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c4415e660 msgr2=0x7f0c441e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c4415e660 0x7f0c441e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 -- 172.21.15.28:0/4076172971 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c440b8dc0 con 0x7f0c440d1f90 2024-09-10T10:44:10.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41e00640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 0x7f0c44157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c41400640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c300090b0 tx=0x7f0c300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.030+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30041070 con 0x7f0c440d1f90 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.031+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3003c8b0 con 0x7f0c440d1f90 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.031+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30037040 con 0x7f0c440d1f90 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.031+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c440b90a0 con 0x7f0c440d1f90 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.031+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c440b9660 con 0x7f0c440d1f90 2024-09-10T10:44:10.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.032+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c440d2440 con 0x7f0c440d1f90 2024-09-10T10:44:10.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.033+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c30044070 con 0x7f0c440d1f90 2024-09-10T10:44:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.034+0000 7f0c3a000640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 0x7f0c2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.034+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0c300abcf0 con 0x7f0c440d1f90 2024-09-10T10:44:10.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.035+0000 7f0c41e00640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 0x7f0c2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.035+0000 7f0c41e00640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 0x7f0c2406e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009f30 tx=0x7f0c34009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.039+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0c30079ea0 con 0x7f0c440d1f90 2024-09-10T10:44:10.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.218+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c440d1b20 con 0x7f0c2406be60 2024-09-10T10:44:10.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.226+0000 7f0c3a000640 1 -- 172.21.15.28:0/4076172971 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f0c440d1b20 con 0x7f0c2406be60 2024-09-10T10:44:10.232 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:10.234 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:44:10.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 msgr2=0x7f0c2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 0x7f0c2406e320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009f30 tx=0x7f0c34009190 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 msgr2=0x7f0c441e3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c300090b0 tx=0x7f0c300046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 shutdown_connections 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c2406be60 0x7f0c2406e320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0c4415e660 0x7f0c441e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c440d1f90 0x7f0c441e3ff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 --2- 172.21.15.28:0/4076172971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c440d0dd0 0x7f0c44157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 >> 172.21.15.28:0/4076172971 conn(0x7f0c4408bfc0 msgr2=0x7f0c440b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 shutdown_connections 2024-09-10T10:44:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.232+0000 7f0c43200640 1 -- 172.21.15.28:0/4076172971 wait complete. 2024-09-10T10:44:10.248 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-10T10:44:08.487222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223844,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750876,"statfs":{"total":773094113280,"available":772864897024,"internally_reserved":0,"allocated":3514368,"data_stored":2081516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003961"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870128+0000","last_change":"2024-09-10T10:43:46.542097+0000","last_active":"2024-09-10T10:43:52.870128+0000","last_peered":"2024-09-10T10:43:52.870128+0000","last_clean":"2024-09-10T10:43:52.870128+0000","last_became_active":"2024-09-10T10:43:46.541830+0000","last_became_peered":"2024-09-10T10:43:46.541830+0000","last_unstale":"2024-09-10T10:43:52.870128+0000","last_undegraded":"2024-09-10T10:43:52.870128+0000","last_fullsized":"2024-09-10T10:43:52.870128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709072+0000","last_change":"2024-09-10T10:43:47.561686+0000","last_active":"2024-09-10T10:43:52.709072+0000","last_peered":"2024-09-10T10:43:52.709072+0000","last_clean":"2024-09-10T10:43:52.709072+0000","last_became_active":"2024-09-10T10:43:47.561406+0000","last_became_peered":"2024-09-10T10:43:47.561406+0000","last_unstale":"2024-09-10T10:43:52.709072+0000","last_undegraded":"2024-09-10T10:43:52.709072+0000","last_fullsized":"2024-09-10T10:43:52.709072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.700092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563827+0000","last_change":"2024-09-10T10:43:51.575030+0000","last_active":"2024-09-10T10:43:52.563827+0000","last_peered":"2024-09-10T10:43:52.563827+0000","last_clean":"2024-09-10T10:43:52.563827+0000","last_became_active":"2024-09-10T10:43:51.570182+0000","last_became_peered":"2024-09-10T10:43:51.570182+0000","last_unstale":"2024-09-10T10:43:52.563827+0000","last_undegraded":"2024-09-10T10:43:52.563827+0000","last_fullsized":"2024-09-10T10:43:52.563827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:21.179072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709156+0000","last_change":"2024-09-10T10:43:46.528230+0000","last_active":"2024-09-10T10:43:52.709156+0000","last_peered":"2024-09-10T10:43:52.709156+0000","last_clean":"2024-09-10T10:43:52.709156+0000","last_became_active":"2024-09-10T10:43:46.527955+0000","last_became_peered":"2024-09-10T10:43:46.527955+0000","last_unstale":"2024-09-10T10:43:52.709156+0000","last_undegraded":"2024-09-10T10:43:52.709156+0000","last_fullsized":"2024-09-10T10:43:52.709156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:23.630718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708899+0000","last_change":"2024-09-10T10:43:47.536706+0000","last_active":"2024-09-10T10:43:52.708899+0000","last_peered":"2024-09-10T10:43:52.708899+0000","last_clean":"2024-09-10T10:43:52.708899+0000","last_became_active":"2024-09-10T10:43:47.536347+0000","last_became_peered":"2024-09-10T10:43:47.536347+0000","last_unstale":"2024-09-10T10:43:52.708899+0000","last_undegraded":"2024-09-10T10:43:52.708899+0000","last_fullsized":"2024-09-10T10:43:52.708899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:57.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563851+0000","last_change":"2024-09-10T10:43:51.569149+0000","last_active":"2024-09-10T10:43:52.563851+0000","last_peered":"2024-09-10T10:43:52.563851+0000","last_clean":"2024-09-10T10:43:52.563851+0000","last_became_active":"2024-09-10T10:43:51.568748+0000","last_became_peered":"2024-09-10T10:43:51.568748+0000","last_unstale":"2024-09-10T10:43:52.563851+0000","last_undegraded":"2024-09-10T10:43:52.563851+0000","last_fullsized":"2024-09-10T10:43:52.563851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869713+0000","last_change":"2024-09-10T10:43:46.525149+0000","last_active":"2024-09-10T10:43:52.869713+0000","last_peered":"2024-09-10T10:43:52.869713+0000","last_clean":"2024-09-10T10:43:52.869713+0000","last_became_active":"2024-09-10T10:43:46.524877+0000","last_became_peered":"2024-09-10T10:43:46.524877+0000","last_unstale":"2024-09-10T10:43:52.869713+0000","last_undegraded":"2024-09-10T10:43:52.869713+0000","last_fullsized":"2024-09-10T10:43:52.869713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:40.800800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564917+0000","last_change":"2024-09-10T10:43:47.547350+0000","last_active":"2024-09-10T10:43:52.564917+0000","last_peered":"2024-09-10T10:43:52.564917+0000","last_clean":"2024-09-10T10:43:52.564917+0000","last_became_active":"2024-09-10T10:43:47.547041+0000","last_became_peered":"2024-09-10T10:43:47.547041+0000","last_unstale":"2024-09-10T10:43:52.564917+0000","last_undegraded":"2024-09-10T10:43:52.564917+0000","last_fullsized":"2024-09-10T10:43:52.564917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563358+0000","last_change":"2024-09-10T10:43:51.587632+0000","last_active":"2024-09-10T10:43:52.563358+0000","last_peered":"2024-09-10T10:43:52.563358+0000","last_clean":"2024-09-10T10:43:52.563358+0000","last_became_active":"2024-09-10T10:43:51.587202+0000","last_became_peered":"2024-09-10T10:43:51.587202+0000","last_unstale":"2024-09-10T10:43:52.563358+0000","last_undegraded":"2024-09-10T10:43:52.563358+0000","last_fullsized":"2024-09-10T10:43:52.563358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:57.684586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869700+0000","last_change":"2024-09-10T10:43:46.529521+0000","last_active":"2024-09-10T10:43:52.869700+0000","last_peered":"2024-09-10T10:43:52.869700+0000","last_clean":"2024-09-10T10:43:52.869700+0000","last_became_active":"2024-09-10T10:43:46.529170+0000","last_became_peered":"2024-09-10T10:43:46.529170+0000","last_unstale":"2024-09-10T10:43:52.869700+0000","last_undegraded":"2024-09-10T10:43:52.869700+0000","last_fullsized":"2024-09-10T10:43:52.869700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870421+0000","last_change":"2024-09-10T10:43:47.550037+0000","last_active":"2024-09-10T10:43:52.870421+0000","last_peered":"2024-09-10T10:43:52.870421+0000","last_clean":"2024-09-10T10:43:52.870421+0000","last_became_active":"2024-09-10T10:43:47.549678+0000","last_became_peered":"2024-09-10T10:43:47.549678+0000","last_unstale":"2024-09-10T10:43:52.870421+0000","last_undegraded":"2024-09-10T10:43:52.870421+0000","last_fullsized":"2024-09-10T10:43:52.870421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.860584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563399+0000","last_change":"2024-09-10T10:43:51.577274+0000","last_active":"2024-09-10T10:43:52.563399+0000","last_peered":"2024-09-10T10:43:52.563399+0000","last_clean":"2024-09-10T10:43:52.563399+0000","last_became_active":"2024-09-10T10:43:51.576818+0000","last_became_peered":"2024-09-10T10:43:51.576818+0000","last_unstale":"2024-09-10T10:43:52.563399+0000","last_undegraded":"2024-09-10T10:43:52.563399+0000","last_fullsized":"2024-09-10T10:43:52.563399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869003+0000","last_change":"2024-09-10T10:43:47.528998+0000","last_active":"2024-09-10T10:43:52.869003+0000","last_peered":"2024-09-10T10:43:52.869003+0000","last_clean":"2024-09-10T10:43:52.869003+0000","last_became_active":"2024-09-10T10:43:47.528761+0000","last_became_peered":"2024-09-10T10:43:47.528761+0000","last_unstale":"2024-09-10T10:43:52.869003+0000","last_undegraded":"2024-09-10T10:43:52.869003+0000","last_fullsized":"2024-09-10T10:43:52.869003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:27.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538367+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538043+0000","last_became_peered":"2024-09-10T10:43:46.538043+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:27.129655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870256+0000","last_change":"2024-09-10T10:43:51.589628+0000","last_active":"2024-09-10T10:43:52.870256+0000","last_peered":"2024-09-10T10:43:52.870256+0000","last_clean":"2024-09-10T10:43:52.870256+0000","last_became_active":"2024-09-10T10:43:51.588470+0000","last_became_peered":"2024-09-10T10:43:51.588470+0000","last_unstale":"2024-09-10T10:43:52.870256+0000","last_undegraded":"2024-09-10T10:43:52.870256+0000","last_fullsized":"2024-09-10T10:43:52.870256+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:51.784026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564115+0000","last_change":"2024-09-10T10:43:47.539521+0000","last_active":"2024-09-10T10:43:52.564115+0000","last_peered":"2024-09-10T10:43:52.564115+0000","last_clean":"2024-09-10T10:43:52.564115+0000","last_became_active":"2024-09-10T10:43:47.539187+0000","last_became_peered":"2024-09-10T10:43:47.539187+0000","last_unstale":"2024-09-10T10:43:52.564115+0000","last_undegraded":"2024-09-10T10:43:52.564115+0000","last_fullsized":"2024-09-10T10:43:52.564115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870141+0000","last_change":"2024-09-10T10:43:46.520278+0000","last_active":"2024-09-10T10:43:52.870141+0000","last_peered":"2024-09-10T10:43:52.870141+0000","last_clean":"2024-09-10T10:43:52.870141+0000","last_became_active":"2024-09-10T10:43:46.519954+0000","last_became_peered":"2024-09-10T10:43:46.519954+0000","last_unstale":"2024-09-10T10:43:52.870141+0000","last_undegraded":"2024-09-10T10:43:52.870141+0000","last_fullsized":"2024-09-10T10:43:52.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:51.570565+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:51.570165+0000","last_became_peered":"2024-09-10T10:43:51.570165+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.439716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869301+0000","last_change":"2024-09-10T10:43:47.534339+0000","last_active":"2024-09-10T10:43:52.869301+0000","last_peered":"2024-09-10T10:43:52.869301+0000","last_clean":"2024-09-10T10:43:52.869301+0000","last_became_active":"2024-09-10T10:43:47.533413+0000","last_became_peered":"2024-09-10T10:43:47.533413+0000","last_unstale":"2024-09-10T10:43:52.869301+0000","last_undegraded":"2024-09-10T10:43:52.869301+0000","last_fullsized":"2024-09-10T10:43:52.869301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869263+0000","last_change":"2024-09-10T10:43:46.535278+0000","last_active":"2024-09-10T10:43:52.869263+0000","last_peered":"2024-09-10T10:43:52.869263+0000","last_clean":"2024-09-10T10:43:52.869263+0000","last_became_active":"2024-09-10T10:43:46.534954+0000","last_became_peered":"2024-09-10T10:43:46.534954+0000","last_unstale":"2024-09-10T10:43:52.869263+0000","last_undegraded":"2024-09-10T10:43:52.869263+0000","last_fullsized":"2024-09-10T10:43:52.869263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564514+0000","last_change":"2024-09-10T10:43:51.574146+0000","last_active":"2024-09-10T10:43:52.564514+0000","last_peered":"2024-09-10T10:43:52.564514+0000","last_clean":"2024-09-10T10:43:52.564514+0000","last_became_active":"2024-09-10T10:43:51.573681+0000","last_became_peered":"2024-09-10T10:43:51.573681+0000","last_unstale":"2024-09-10T10:43:52.564514+0000","last_undegraded":"2024-09-10T10:43:52.564514+0000","last_fullsized":"2024-09-10T10:43:52.564514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:13.180965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868735+0000","last_change":"2024-09-10T10:43:47.532923+0000","last_active":"2024-09-10T10:43:52.868735+0000","last_peered":"2024-09-10T10:43:52.868735+0000","last_clean":"2024-09-10T10:43:52.868735+0000","last_became_active":"2024-09-10T10:43:47.532644+0000","last_became_peered":"2024-09-10T10:43:47.532644+0000","last_unstale":"2024-09-10T10:43:52.868735+0000","last_undegraded":"2024-09-10T10:43:52.868735+0000","last_fullsized":"2024-09-10T10:43:52.868735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:44.518256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868884+0000","last_change":"2024-09-10T10:43:46.525829+0000","last_active":"2024-09-10T10:43:52.868884+0000","last_peered":"2024-09-10T10:43:52.868884+0000","last_clean":"2024-09-10T10:43:52.868884+0000","last_became_active":"2024-09-10T10:43:46.523492+0000","last_became_peered":"2024-09-10T10:43:46.523492+0000","last_unstale":"2024-09-10T10:43:52.868884+0000","last_undegraded":"2024-09-10T10:43:52.868884+0000","last_fullsized":"2024-09-10T10:43:52.868884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564250+0000","last_change":"2024-09-10T10:43:51.580518+0000","last_active":"2024-09-10T10:43:52.564250+0000","last_peered":"2024-09-10T10:43:52.564250+0000","last_clean":"2024-09-10T10:43:52.564250+0000","last_became_active":"2024-09-10T10:43:51.580050+0000","last_became_peered":"2024-09-10T10:43:51.580050+0000","last_unstale":"2024-09-10T10:43:52.564250+0000","last_undegraded":"2024-09-10T10:43:52.564250+0000","last_fullsized":"2024-09-10T10:43:52.564250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869809+0000","last_change":"2024-09-10T10:43:47.532764+0000","last_active":"2024-09-10T10:43:52.869809+0000","last_peered":"2024-09-10T10:43:52.869809+0000","last_clean":"2024-09-10T10:43:52.869809+0000","last_became_active":"2024-09-10T10:43:47.532407+0000","last_became_peered":"2024-09-10T10:43:47.532407+0000","last_unstale":"2024-09-10T10:43:52.869809+0000","last_undegraded":"2024-09-10T10:43:52.869809+0000","last_fullsized":"2024-09-10T10:43:52.869809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.139356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870804+0000","last_change":"2024-09-10T10:43:46.537450+0000","last_active":"2024-09-10T10:43:52.870804+0000","last_peered":"2024-09-10T10:43:52.870804+0000","last_clean":"2024-09-10T10:43:52.870804+0000","last_became_active":"2024-09-10T10:43:46.537129+0000","last_became_peered":"2024-09-10T10:43:46.537129+0000","last_unstale":"2024-09-10T10:43:52.870804+0000","last_undegraded":"2024-09-10T10:43:52.870804+0000","last_fullsized":"2024-09-10T10:43:52.870804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:47.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563273+0000","last_change":"2024-09-10T10:43:51.580449+0000","last_active":"2024-09-10T10:43:52.563273+0000","last_peered":"2024-09-10T10:43:52.563273+0000","last_clean":"2024-09-10T10:43:52.563273+0000","last_became_active":"2024-09-10T10:43:51.579803+0000","last_became_peered":"2024-09-10T10:43:51.579803+0000","last_unstale":"2024-09-10T10:43:52.563273+0000","last_undegraded":"2024-09-10T10:43:52.563273+0000","last_fullsized":"2024-09-10T10:43:52.563273+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:22.089401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870018+0000","last_change":"2024-09-10T10:43:47.532014+0000","last_active":"2024-09-10T10:43:52.870018+0000","last_peered":"2024-09-10T10:43:52.870018+0000","last_clean":"2024-09-10T10:43:52.870018+0000","last_became_active":"2024-09-10T10:43:47.531662+0000","last_became_peered":"2024-09-10T10:43:47.531662+0000","last_unstale":"2024-09-10T10:43:52.870018+0000","last_undegraded":"2024-09-10T10:43:52.870018+0000","last_fullsized":"2024-09-10T10:43:52.870018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:35.559824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869967+0000","last_change":"2024-09-10T10:43:46.527723+0000","last_active":"2024-09-10T10:43:52.869967+0000","last_peered":"2024-09-10T10:43:52.869967+0000","last_clean":"2024-09-10T10:43:52.869967+0000","last_became_active":"2024-09-10T10:43:46.527452+0000","last_became_peered":"2024-09-10T10:43:46.527452+0000","last_unstale":"2024-09-10T10:43:52.869967+0000","last_undegraded":"2024-09-10T10:43:52.869967+0000","last_fullsized":"2024-09-10T10:43:52.869967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869180+0000","last_change":"2024-09-10T10:43:51.574307+0000","last_active":"2024-09-10T10:43:52.869180+0000","last_peered":"2024-09-10T10:43:52.869180+0000","last_clean":"2024-09-10T10:43:52.869180+0000","last_became_active":"2024-09-10T10:43:51.571308+0000","last_became_peered":"2024-09-10T10:43:51.571308+0000","last_unstale":"2024-09-10T10:43:52.869180+0000","last_undegraded":"2024-09-10T10:43:52.869180+0000","last_fullsized":"2024-09-10T10:43:52.869180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:09.361189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870742+0000","last_change":"2024-09-10T10:43:47.549822+0000","last_active":"2024-09-10T10:43:52.870742+0000","last_peered":"2024-09-10T10:43:52.870742+0000","last_clean":"2024-09-10T10:43:52.870742+0000","last_became_active":"2024-09-10T10:43:47.549526+0000","last_became_peered":"2024-09-10T10:43:47.549526+0000","last_unstale":"2024-09-10T10:43:52.870742+0000","last_undegraded":"2024-09-10T10:43:52.870742+0000","last_fullsized":"2024-09-10T10:43:52.870742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870715+0000","last_change":"2024-09-10T10:43:46.539321+0000","last_active":"2024-09-10T10:43:52.870715+0000","last_peered":"2024-09-10T10:43:52.870715+0000","last_clean":"2024-09-10T10:43:52.870715+0000","last_became_active":"2024-09-10T10:43:46.538763+0000","last_became_peered":"2024-09-10T10:43:46.538763+0000","last_unstale":"2024-09-10T10:43:52.870715+0000","last_undegraded":"2024-09-10T10:43:52.870715+0000","last_fullsized":"2024-09-10T10:43:52.870715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:53.745395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.571592+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.570946+0000","last_became_peered":"2024-09-10T10:43:51.570946+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564687+0000","last_change":"2024-09-10T10:43:47.535460+0000","last_active":"2024-09-10T10:43:52.564687+0000","last_peered":"2024-09-10T10:43:52.564687+0000","last_clean":"2024-09-10T10:43:52.564687+0000","last_became_active":"2024-09-10T10:43:47.535170+0000","last_became_peered":"2024-09-10T10:43:47.535170+0000","last_unstale":"2024-09-10T10:43:52.564687+0000","last_undegraded":"2024-09-10T10:43:52.564687+0000","last_fullsized":"2024-09-10T10:43:52.564687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708875+0000","last_change":"2024-09-10T10:43:46.521709+0000","last_active":"2024-09-10T10:43:52.708875+0000","last_peered":"2024-09-10T10:43:52.708875+0000","last_clean":"2024-09-10T10:43:52.708875+0000","last_became_active":"2024-09-10T10:43:46.521411+0000","last_became_peered":"2024-09-10T10:43:46.521411+0000","last_unstale":"2024-09-10T10:43:52.708875+0000","last_undegraded":"2024-09-10T10:43:52.708875+0000","last_fullsized":"2024-09-10T10:43:52.708875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:04.600951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:51.573047+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:51.572690+0000","last_became_peered":"2024-09-10T10:43:51.572690+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:47.541642+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:47.541174+0000","last_became_peered":"2024-09-10T10:43:47.541174+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:18.047791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869868+0000","last_change":"2024-09-10T10:43:46.532535+0000","last_active":"2024-09-10T10:43:52.869868+0000","last_peered":"2024-09-10T10:43:52.869868+0000","last_clean":"2024-09-10T10:43:52.869868+0000","last_became_active":"2024-09-10T10:43:46.532256+0000","last_became_peered":"2024-09-10T10:43:46.532256+0000","last_unstale":"2024-09-10T10:43:52.869868+0000","last_undegraded":"2024-09-10T10:43:52.869868+0000","last_fullsized":"2024-09-10T10:43:52.869868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:51.847379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870621+0000","last_change":"2024-09-10T10:43:51.571178+0000","last_active":"2024-09-10T10:43:52.870621+0000","last_peered":"2024-09-10T10:43:52.870621+0000","last_clean":"2024-09-10T10:43:52.870621+0000","last_became_active":"2024-09-10T10:43:51.570742+0000","last_became_peered":"2024-09-10T10:43:51.570742+0000","last_unstale":"2024-09-10T10:43:52.870621+0000","last_undegraded":"2024-09-10T10:43:52.870621+0000","last_fullsized":"2024-09-10T10:43:52.870621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708292+0000","last_change":"2024-09-10T10:43:47.536864+0000","last_active":"2024-09-10T10:43:52.708292+0000","last_peered":"2024-09-10T10:43:52.708292+0000","last_clean":"2024-09-10T10:43:52.708292+0000","last_became_active":"2024-09-10T10:43:47.536270+0000","last_became_peered":"2024-09-10T10:43:47.536270+0000","last_unstale":"2024-09-10T10:43:52.708292+0000","last_undegraded":"2024-09-10T10:43:52.708292+0000","last_fullsized":"2024-09-10T10:43:52.708292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:25.640976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563952+0000","last_change":"2024-09-10T10:43:46.521627+0000","last_active":"2024-09-10T10:43:52.563952+0000","last_peered":"2024-09-10T10:43:52.563952+0000","last_clean":"2024-09-10T10:43:52.563952+0000","last_became_active":"2024-09-10T10:43:46.521150+0000","last_became_peered":"2024-09-10T10:43:46.521150+0000","last_unstale":"2024-09-10T10:43:52.563952+0000","last_undegraded":"2024-09-10T10:43:52.563952+0000","last_fullsized":"2024-09-10T10:43:52.563952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:35.895649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870932+0000","last_change":"2024-09-10T10:43:51.580293+0000","last_active":"2024-09-10T10:43:52.870932+0000","last_peered":"2024-09-10T10:43:52.870932+0000","last_clean":"2024-09-10T10:43:52.870932+0000","last_became_active":"2024-09-10T10:43:51.576157+0000","last_became_peered":"2024-09-10T10:43:51.576157+0000","last_unstale":"2024-09-10T10:43:52.870932+0000","last_undegraded":"2024-09-10T10:43:52.870932+0000","last_fullsized":"2024-09-10T10:43:52.870932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869651+0000","last_change":"2024-09-10T10:43:47.563302+0000","last_active":"2024-09-10T10:43:52.869651+0000","last_peered":"2024-09-10T10:43:52.869651+0000","last_clean":"2024-09-10T10:43:52.869651+0000","last_became_active":"2024-09-10T10:43:47.562694+0000","last_became_peered":"2024-09-10T10:43:47.562694+0000","last_unstale":"2024-09-10T10:43:52.869651+0000","last_undegraded":"2024-09-10T10:43:52.869651+0000","last_fullsized":"2024-09-10T10:43:52.869651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868801+0000","last_change":"2024-09-10T10:43:46.533697+0000","last_active":"2024-09-10T10:43:52.868801+0000","last_peered":"2024-09-10T10:43:52.868801+0000","last_clean":"2024-09-10T10:43:52.868801+0000","last_became_active":"2024-09-10T10:43:46.533190+0000","last_became_peered":"2024-09-10T10:43:46.533190+0000","last_unstale":"2024-09-10T10:43:52.868801+0000","last_undegraded":"2024-09-10T10:43:52.868801+0000","last_fullsized":"2024-09-10T10:43:52.868801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:09.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708748+0000","last_change":"2024-09-10T10:43:51.583515+0000","last_active":"2024-09-10T10:43:52.708748+0000","last_peered":"2024-09-10T10:43:52.708748+0000","last_clean":"2024-09-10T10:43:52.708748+0000","last_became_active":"2024-09-10T10:43:51.582366+0000","last_became_peered":"2024-09-10T10:43:51.582366+0000","last_unstale":"2024-09-10T10:43:52.708748+0000","last_undegraded":"2024-09-10T10:43:52.708748+0000","last_fullsized":"2024-09-10T10:43:52.708748+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:10.957254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869329+0000","last_change":"2024-09-10T10:43:47.520783+0000","last_active":"2024-09-10T10:43:52.869329+0000","last_peered":"2024-09-10T10:43:52.869329+0000","last_clean":"2024-09-10T10:43:52.869329+0000","last_became_active":"2024-09-10T10:43:47.520409+0000","last_became_peered":"2024-09-10T10:43:47.520409+0000","last_unstale":"2024-09-10T10:43:52.869329+0000","last_undegraded":"2024-09-10T10:43:52.869329+0000","last_fullsized":"2024-09-10T10:43:52.869329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538924+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538640+0000","last_became_peered":"2024-09-10T10:43:46.538640+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:57.368590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564215+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:43:52.564215+0000","last_peered":"2024-09-10T10:43:52.564215+0000","last_clean":"2024-09-10T10:43:52.564215+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:43:52.564215+0000","last_undegraded":"2024-09-10T10:43:52.564215+0000","last_fullsized":"2024-09-10T10:43:52.564215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870584+0000","last_change":"2024-09-10T10:43:51.571200+0000","last_active":"2024-09-10T10:43:52.870584+0000","last_peered":"2024-09-10T10:43:52.870584+0000","last_clean":"2024-09-10T10:43:52.870584+0000","last_became_active":"2024-09-10T10:43:51.570877+0000","last_became_peered":"2024-09-10T10:43:51.570877+0000","last_unstale":"2024-09-10T10:43:52.870584+0000","last_undegraded":"2024-09-10T10:43:52.870584+0000","last_fullsized":"2024-09-10T10:43:52.870584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:59.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870468+0000","last_change":"2024-09-10T10:43:47.546420+0000","last_active":"2024-09-10T10:43:52.870468+0000","last_peered":"2024-09-10T10:43:52.870468+0000","last_clean":"2024-09-10T10:43:52.870468+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.870468+0000","last_undegraded":"2024-09-10T10:43:52.870468+0000","last_fullsized":"2024-09-10T10:43:52.870468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.887906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870800+0000","last_change":"2024-09-10T10:43:46.535842+0000","last_active":"2024-09-10T10:43:52.870800+0000","last_peered":"2024-09-10T10:43:52.870800+0000","last_clean":"2024-09-10T10:43:52.870800+0000","last_became_active":"2024-09-10T10:43:46.535586+0000","last_became_peered":"2024-09-10T10:43:46.535586+0000","last_unstale":"2024-09-10T10:43:52.870800+0000","last_undegraded":"2024-09-10T10:43:52.870800+0000","last_fullsized":"2024-09-10T10:43:52.870800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563751+0000","last_change":"2024-09-10T10:43:51.578759+0000","last_active":"2024-09-10T10:43:52.563751+0000","last_peered":"2024-09-10T10:43:52.563751+0000","last_clean":"2024-09-10T10:43:52.563751+0000","last_became_active":"2024-09-10T10:43:51.578237+0000","last_became_peered":"2024-09-10T10:43:51.578237+0000","last_unstale":"2024-09-10T10:43:52.563751+0000","last_undegraded":"2024-09-10T10:43:52.563751+0000","last_fullsized":"2024-09-10T10:43:52.563751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869654+0000","last_change":"2024-09-10T10:43:47.540794+0000","last_active":"2024-09-10T10:43:52.869654+0000","last_peered":"2024-09-10T10:43:52.869654+0000","last_clean":"2024-09-10T10:43:52.869654+0000","last_became_active":"2024-09-10T10:43:47.540360+0000","last_became_peered":"2024-09-10T10:43:47.540360+0000","last_unstale":"2024-09-10T10:43:52.869654+0000","last_undegraded":"2024-09-10T10:43:52.869654+0000","last_fullsized":"2024-09-10T10:43:52.869654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:31.926385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709016+0000","last_change":"2024-09-10T10:43:46.520898+0000","last_active":"2024-09-10T10:43:52.709016+0000","last_peered":"2024-09-10T10:43:52.709016+0000","last_clean":"2024-09-10T10:43:52.709016+0000","last_became_active":"2024-09-10T10:43:46.518998+0000","last_became_peered":"2024-09-10T10:43:46.518998+0000","last_unstale":"2024-09-10T10:43:52.709016+0000","last_undegraded":"2024-09-10T10:43:52.709016+0000","last_fullsized":"2024-09-10T10:43:52.709016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:42.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708786+0000","last_change":"2024-09-10T10:43:51.587866+0000","last_active":"2024-09-10T10:43:52.708786+0000","last_peered":"2024-09-10T10:43:52.708786+0000","last_clean":"2024-09-10T10:43:52.708786+0000","last_became_active":"2024-09-10T10:43:51.585690+0000","last_became_peered":"2024-09-10T10:43:51.585690+0000","last_unstale":"2024-09-10T10:43:52.708786+0000","last_undegraded":"2024-09-10T10:43:52.708786+0000","last_fullsized":"2024-09-10T10:43:52.708786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:41.798950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870081+0000","last_change":"2024-09-10T10:43:47.540910+0000","last_active":"2024-09-10T10:43:52.870081+0000","last_peered":"2024-09-10T10:43:52.870081+0000","last_clean":"2024-09-10T10:43:52.870081+0000","last_became_active":"2024-09-10T10:43:47.540603+0000","last_became_peered":"2024-09-10T10:43:47.540603+0000","last_unstale":"2024-09-10T10:43:52.870081+0000","last_undegraded":"2024-09-10T10:43:52.870081+0000","last_fullsized":"2024-09-10T10:43:52.870081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:48.137696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869935+0000","last_change":"2024-09-10T10:43:46.525479+0000","last_active":"2024-09-10T10:43:52.869935+0000","last_peered":"2024-09-10T10:43:52.869935+0000","last_clean":"2024-09-10T10:43:52.869935+0000","last_became_active":"2024-09-10T10:43:46.524834+0000","last_became_peered":"2024-09-10T10:43:46.524834+0000","last_unstale":"2024-09-10T10:43:52.869935+0000","last_undegraded":"2024-09-10T10:43:52.869935+0000","last_fullsized":"2024-09-10T10:43:52.869935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868574+0000","last_change":"2024-09-10T10:43:51.569746+0000","last_active":"2024-09-10T10:43:52.868574+0000","last_peered":"2024-09-10T10:43:52.868574+0000","last_clean":"2024-09-10T10:43:52.868574+0000","last_became_active":"2024-09-10T10:43:51.569289+0000","last_became_peered":"2024-09-10T10:43:51.569289+0000","last_unstale":"2024-09-10T10:43:52.868574+0000","last_undegraded":"2024-09-10T10:43:52.868574+0000","last_fullsized":"2024-09-10T10:43:52.868574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:20.122047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564540+0000","last_change":"2024-09-10T10:43:47.555824+0000","last_active":"2024-09-10T10:43:52.564540+0000","last_peered":"2024-09-10T10:43:52.564540+0000","last_clean":"2024-09-10T10:43:52.564540+0000","last_became_active":"2024-09-10T10:43:47.554957+0000","last_became_peered":"2024-09-10T10:43:47.554957+0000","last_unstale":"2024-09-10T10:43:52.564540+0000","last_undegraded":"2024-09-10T10:43:52.564540+0000","last_fullsized":"2024-09-10T10:43:52.564540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870114+0000","last_change":"2024-09-10T10:43:46.530043+0000","last_active":"2024-09-10T10:43:52.870114+0000","last_peered":"2024-09-10T10:43:52.870114+0000","last_clean":"2024-09-10T10:43:52.870114+0000","last_became_active":"2024-09-10T10:43:46.527509+0000","last_became_peered":"2024-09-10T10:43:46.527509+0000","last_unstale":"2024-09-10T10:43:52.870114+0000","last_undegraded":"2024-09-10T10:43:52.870114+0000","last_fullsized":"2024-09-10T10:43:52.870114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:08.859006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:51.577121+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:51.576021+0000","last_became_peered":"2024-09-10T10:43:51.576021+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:10.204576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868710+0000","last_change":"2024-09-10T10:43:47.562724+0000","last_active":"2024-09-10T10:43:52.868710+0000","last_peered":"2024-09-10T10:43:52.868710+0000","last_clean":"2024-09-10T10:43:52.868710+0000","last_became_active":"2024-09-10T10:43:47.562412+0000","last_became_peered":"2024-09-10T10:43:47.562412+0000","last_unstale":"2024-09-10T10:43:52.868710+0000","last_undegraded":"2024-09-10T10:43:52.868710+0000","last_fullsized":"2024-09-10T10:43:52.868710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:30.773785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869413+0000","last_change":"2024-09-10T10:43:46.525713+0000","last_active":"2024-09-10T10:43:52.869413+0000","last_peered":"2024-09-10T10:43:52.869413+0000","last_clean":"2024-09-10T10:43:52.869413+0000","last_became_active":"2024-09-10T10:43:46.525130+0000","last_became_peered":"2024-09-10T10:43:46.525130+0000","last_unstale":"2024-09-10T10:43:52.869413+0000","last_undegraded":"2024-09-10T10:43:52.869413+0000","last_fullsized":"2024-09-10T10:43:52.869413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709094+0000","last_change":"2024-09-10T10:43:51.582330+0000","last_active":"2024-09-10T10:43:52.709094+0000","last_peered":"2024-09-10T10:43:52.709094+0000","last_clean":"2024-09-10T10:43:52.709094+0000","last_became_active":"2024-09-10T10:43:51.581415+0000","last_became_peered":"2024-09-10T10:43:51.581415+0000","last_unstale":"2024-09-10T10:43:52.709094+0000","last_undegraded":"2024-09-10T10:43:52.709094+0000","last_fullsized":"2024-09-10T10:43:52.709094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:18.520413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564097+0000","last_change":"2024-09-10T10:43:47.537959+0000","last_active":"2024-09-10T10:43:52.564097+0000","last_peered":"2024-09-10T10:43:52.564097+0000","last_clean":"2024-09-10T10:43:52.564097+0000","last_became_active":"2024-09-10T10:43:47.537705+0000","last_became_peered":"2024-09-10T10:43:47.537705+0000","last_unstale":"2024-09-10T10:43:52.564097+0000","last_undegraded":"2024-09-10T10:43:52.564097+0000","last_fullsized":"2024-09-10T10:43:52.564097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.301556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869080+0000","last_change":"2024-09-10T10:43:46.539973+0000","last_active":"2024-09-10T10:43:52.869080+0000","last_peered":"2024-09-10T10:43:52.869080+0000","last_clean":"2024-09-10T10:43:52.869080+0000","last_became_active":"2024-09-10T10:43:46.539509+0000","last_became_peered":"2024-09-10T10:43:46.539509+0000","last_unstale":"2024-09-10T10:43:52.869080+0000","last_undegraded":"2024-09-10T10:43:52.869080+0000","last_fullsized":"2024-09-10T10:43:52.869080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:51.584701+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:51.584321+0000","last_became_peered":"2024-09-10T10:43:51.584321+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:07.697012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869811+0000","last_change":"2024-09-10T10:43:47.549457+0000","last_active":"2024-09-10T10:43:52.869811+0000","last_peered":"2024-09-10T10:43:52.869811+0000","last_clean":"2024-09-10T10:43:52.869811+0000","last_became_active":"2024-09-10T10:43:47.549210+0000","last_became_peered":"2024-09-10T10:43:47.549210+0000","last_unstale":"2024-09-10T10:43:52.869811+0000","last_undegraded":"2024-09-10T10:43:52.869811+0000","last_fullsized":"2024-09-10T10:43:52.869811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:00.039891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869773+0000","last_change":"2024-09-10T10:43:46.520041+0000","last_active":"2024-09-10T10:43:52.869773+0000","last_peered":"2024-09-10T10:43:52.869773+0000","last_clean":"2024-09-10T10:43:52.869773+0000","last_became_active":"2024-09-10T10:43:46.519721+0000","last_became_peered":"2024-09-10T10:43:46.519721+0000","last_unstale":"2024-09-10T10:43:52.869773+0000","last_undegraded":"2024-09-10T10:43:52.869773+0000","last_fullsized":"2024-09-10T10:43:52.869773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:23.802544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870586+0000","last_change":"2024-09-10T10:43:51.571179+0000","last_active":"2024-09-10T10:43:52.870586+0000","last_peered":"2024-09-10T10:43:52.870586+0000","last_clean":"2024-09-10T10:43:52.870586+0000","last_became_active":"2024-09-10T10:43:51.570590+0000","last_became_peered":"2024-09-10T10:43:51.570590+0000","last_unstale":"2024-09-10T10:43:52.870586+0000","last_undegraded":"2024-09-10T10:43:52.870586+0000","last_fullsized":"2024-09-10T10:43:52.870586+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870110+0000","last_change":"2024-09-10T10:43:47.546795+0000","last_active":"2024-09-10T10:43:52.870110+0000","last_peered":"2024-09-10T10:43:52.870110+0000","last_clean":"2024-09-10T10:43:52.870110+0000","last_became_active":"2024-09-10T10:43:47.546529+0000","last_became_peered":"2024-09-10T10:43:47.546529+0000","last_unstale":"2024-09-10T10:43:52.870110+0000","last_undegraded":"2024-09-10T10:43:52.870110+0000","last_fullsized":"2024-09-10T10:43:52.870110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564317+0000","last_change":"2024-09-10T10:43:46.524422+0000","last_active":"2024-09-10T10:43:52.564317+0000","last_peered":"2024-09-10T10:43:52.564317+0000","last_clean":"2024-09-10T10:43:52.564317+0000","last_became_active":"2024-09-10T10:43:46.524027+0000","last_became_peered":"2024-09-10T10:43:46.524027+0000","last_unstale":"2024-09-10T10:43:52.564317+0000","last_undegraded":"2024-09-10T10:43:52.564317+0000","last_fullsized":"2024-09-10T10:43:52.564317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.575001+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.573805+0000","last_became_peered":"2024-09-10T10:43:51.573805+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:27.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869516+0000","last_change":"2024-09-10T10:43:47.542333+0000","last_active":"2024-09-10T10:43:52.869516+0000","last_peered":"2024-09-10T10:43:52.869516+0000","last_clean":"2024-09-10T10:43:52.869516+0000","last_became_active":"2024-09-10T10:43:47.540584+0000","last_became_peered":"2024-09-10T10:43:47.540584+0000","last_unstale":"2024-09-10T10:43:52.869516+0000","last_undegraded":"2024-09-10T10:43:52.869516+0000","last_fullsized":"2024-09-10T10:43:52.869516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:08.456485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:58.645779+0000","last_change":"2024-09-10T10:43:46.531547+0000","last_active":"2024-09-10T10:43:58.645779+0000","last_peered":"2024-09-10T10:43:58.645779+0000","last_clean":"2024-09-10T10:43:58.645779+0000","last_became_active":"2024-09-10T10:43:46.530309+0000","last_became_peered":"2024-09-10T10:43:46.530309+0000","last_unstale":"2024-09-10T10:43:58.645779+0000","last_undegraded":"2024-09-10T10:43:58.645779+0000","last_fullsized":"2024-09-10T10:43:58.645779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:54.587795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869118+0000","last_change":"2024-09-10T10:43:51.578360+0000","last_active":"2024-09-10T10:43:52.869118+0000","last_peered":"2024-09-10T10:43:52.869118+0000","last_clean":"2024-09-10T10:43:52.869118+0000","last_became_active":"2024-09-10T10:43:51.578021+0000","last_became_peered":"2024-09-10T10:43:51.578021+0000","last_unstale":"2024-09-10T10:43:52.869118+0000","last_undegraded":"2024-09-10T10:43:52.869118+0000","last_fullsized":"2024-09-10T10:43:52.869118+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870009+0000","last_change":"2024-09-10T10:43:47.564353+0000","last_active":"2024-09-10T10:43:52.870009+0000","last_peered":"2024-09-10T10:43:52.870009+0000","last_clean":"2024-09-10T10:43:52.870009+0000","last_became_active":"2024-09-10T10:43:47.563999+0000","last_became_peered":"2024-09-10T10:43:47.563999+0000","last_unstale":"2024-09-10T10:43:52.870009+0000","last_undegraded":"2024-09-10T10:43:52.870009+0000","last_fullsized":"2024-09-10T10:43:52.870009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:52.338110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564851+0000","last_change":"2024-09-10T10:43:46.526185+0000","last_active":"2024-09-10T10:43:52.564851+0000","last_peered":"2024-09-10T10:43:52.564851+0000","last_clean":"2024-09-10T10:43:52.564851+0000","last_became_active":"2024-09-10T10:43:46.525887+0000","last_became_peered":"2024-09-10T10:43:46.525887+0000","last_unstale":"2024-09-10T10:43:52.564851+0000","last_undegraded":"2024-09-10T10:43:52.564851+0000","last_fullsized":"2024-09-10T10:43:52.564851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563403+0000","last_change":"2024-09-10T10:43:51.587988+0000","last_active":"2024-09-10T10:43:52.563403+0000","last_peered":"2024-09-10T10:43:52.563403+0000","last_clean":"2024-09-10T10:43:52.563403+0000","last_became_active":"2024-09-10T10:43:51.587584+0000","last_became_peered":"2024-09-10T10:43:51.587584+0000","last_unstale":"2024-09-10T10:43:52.563403+0000","last_undegraded":"2024-09-10T10:43:52.563403+0000","last_fullsized":"2024-09-10T10:43:52.563403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708986+0000","last_change":"2024-09-10T10:43:47.541815+0000","last_active":"2024-09-10T10:43:52.708986+0000","last_peered":"2024-09-10T10:43:52.708986+0000","last_clean":"2024-09-10T10:43:52.708986+0000","last_became_active":"2024-09-10T10:43:47.541530+0000","last_became_peered":"2024-09-10T10:43:47.541530+0000","last_unstale":"2024-09-10T10:43:52.708986+0000","last_undegraded":"2024-09-10T10:43:52.708986+0000","last_fullsized":"2024-09-10T10:43:52.708986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:26.960082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869496+0000","last_change":"2024-09-10T10:43:46.525990+0000","last_active":"2024-09-10T10:43:52.869496+0000","last_peered":"2024-09-10T10:43:52.869496+0000","last_clean":"2024-09-10T10:43:52.869496+0000","last_became_active":"2024-09-10T10:43:46.524901+0000","last_became_peered":"2024-09-10T10:43:46.524901+0000","last_unstale":"2024-09-10T10:43:52.869496+0000","last_undegraded":"2024-09-10T10:43:52.869496+0000","last_fullsized":"2024-09-10T10:43:52.869496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:12.069623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563676+0000","last_change":"2024-09-10T10:43:51.574109+0000","last_active":"2024-09-10T10:43:52.563676+0000","last_peered":"2024-09-10T10:43:52.563676+0000","last_clean":"2024-09-10T10:43:52.563676+0000","last_became_active":"2024-09-10T10:43:51.573718+0000","last_became_peered":"2024-09-10T10:43:51.573718+0000","last_unstale":"2024-09-10T10:43:52.563676+0000","last_undegraded":"2024-09-10T10:43:52.563676+0000","last_fullsized":"2024-09-10T10:43:52.563676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:13.623233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564443+0000","last_change":"2024-09-10T10:43:47.558084+0000","last_active":"2024-09-10T10:43:52.564443+0000","last_peered":"2024-09-10T10:43:52.564443+0000","last_clean":"2024-09-10T10:43:52.564443+0000","last_became_active":"2024-09-10T10:43:47.555632+0000","last_became_peered":"2024-09-10T10:43:47.555632+0000","last_unstale":"2024-09-10T10:43:52.564443+0000","last_undegraded":"2024-09-10T10:43:52.564443+0000","last_fullsized":"2024-09-10T10:43:52.564443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565392+0000","last_change":"2024-09-10T10:43:46.528292+0000","last_active":"2024-09-10T10:43:52.565392+0000","last_peered":"2024-09-10T10:43:52.565392+0000","last_clean":"2024-09-10T10:43:52.565392+0000","last_became_active":"2024-09-10T10:43:46.527943+0000","last_became_peered":"2024-09-10T10:43:46.527943+0000","last_unstale":"2024-09-10T10:43:52.565392+0000","last_undegraded":"2024-09-10T10:43:52.565392+0000","last_fullsized":"2024-09-10T10:43:52.565392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869189+0000","last_change":"2024-09-10T10:43:51.572294+0000","last_active":"2024-09-10T10:43:52.869189+0000","last_peered":"2024-09-10T10:43:52.869189+0000","last_clean":"2024-09-10T10:43:52.869189+0000","last_became_active":"2024-09-10T10:43:51.571796+0000","last_became_peered":"2024-09-10T10:43:51.571796+0000","last_unstale":"2024-09-10T10:43:52.869189+0000","last_undegraded":"2024-09-10T10:43:52.869189+0000","last_fullsized":"2024-09-10T10:43:52.869189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869838+0000","last_change":"2024-09-10T10:43:51.577672+0000","last_active":"2024-09-10T10:43:52.869838+0000","last_peered":"2024-09-10T10:43:52.869838+0000","last_clean":"2024-09-10T10:43:52.869838+0000","last_became_active":"2024-09-10T10:43:51.577342+0000","last_became_peered":"2024-09-10T10:43:51.577342+0000","last_unstale":"2024-09-10T10:43:52.869838+0000","last_undegraded":"2024-09-10T10:43:52.869838+0000","last_fullsized":"2024-09-10T10:43:52.869838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:59.764337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869129+0000","last_change":"2024-09-10T10:43:47.538181+0000","last_active":"2024-09-10T10:43:52.869129+0000","last_peered":"2024-09-10T10:43:52.869129+0000","last_clean":"2024-09-10T10:43:52.869129+0000","last_became_active":"2024-09-10T10:43:47.537810+0000","last_became_peered":"2024-09-10T10:43:47.537810+0000","last_unstale":"2024-09-10T10:43:52.869129+0000","last_undegraded":"2024-09-10T10:43:52.869129+0000","last_fullsized":"2024-09-10T10:43:52.869129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868780+0000","last_change":"2024-09-10T10:43:46.533440+0000","last_active":"2024-09-10T10:43:52.868780+0000","last_peered":"2024-09-10T10:43:52.868780+0000","last_clean":"2024-09-10T10:43:52.868780+0000","last_became_active":"2024-09-10T10:43:46.532814+0000","last_became_peered":"2024-09-10T10:43:46.532814+0000","last_unstale":"2024-09-10T10:43:52.868780+0000","last_undegraded":"2024-09-10T10:43:52.868780+0000","last_fullsized":"2024-09-10T10:43:52.868780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:03.561914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870537+0000","last_change":"2024-09-10T10:43:51.576277+0000","last_active":"2024-09-10T10:43:52.870537+0000","last_peered":"2024-09-10T10:43:52.870537+0000","last_clean":"2024-09-10T10:43:52.870537+0000","last_became_active":"2024-09-10T10:43:51.575888+0000","last_became_peered":"2024-09-10T10:43:51.575888+0000","last_unstale":"2024-09-10T10:43:52.870537+0000","last_undegraded":"2024-09-10T10:43:52.870537+0000","last_fullsized":"2024-09-10T10:43:52.870537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:37.271562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708364+0000","last_change":"2024-09-10T10:43:47.539801+0000","last_active":"2024-09-10T10:43:52.708364+0000","last_peered":"2024-09-10T10:43:52.708364+0000","last_clean":"2024-09-10T10:43:52.708364+0000","last_became_active":"2024-09-10T10:43:47.535922+0000","last_became_peered":"2024-09-10T10:43:47.535922+0000","last_unstale":"2024-09-10T10:43:52.708364+0000","last_undegraded":"2024-09-10T10:43:52.708364+0000","last_fullsized":"2024-09-10T10:43:52.708364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:49.905762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869706+0000","last_change":"2024-09-10T10:43:46.530425+0000","last_active":"2024-09-10T10:43:52.869706+0000","last_peered":"2024-09-10T10:43:52.869706+0000","last_clean":"2024-09-10T10:43:52.869706+0000","last_became_active":"2024-09-10T10:43:46.530158+0000","last_became_peered":"2024-09-10T10:43:46.530158+0000","last_unstale":"2024-09-10T10:43:52.869706+0000","last_undegraded":"2024-09-10T10:43:52.869706+0000","last_fullsized":"2024-09-10T10:43:52.869706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869653+0000","last_change":"2024-09-10T10:43:46.537743+0000","last_active":"2024-09-10T10:43:52.869653+0000","last_peered":"2024-09-10T10:43:52.869653+0000","last_clean":"2024-09-10T10:43:52.869653+0000","last_became_active":"2024-09-10T10:43:46.537260+0000","last_became_peered":"2024-09-10T10:43:46.537260+0000","last_unstale":"2024-09-10T10:43:52.869653+0000","last_undegraded":"2024-09-10T10:43:52.869653+0000","last_fullsized":"2024-09-10T10:43:52.869653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:53.479099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869441+0000","last_change":"2024-09-10T10:43:47.545017+0000","last_active":"2024-09-10T10:43:52.869441+0000","last_peered":"2024-09-10T10:43:52.869441+0000","last_clean":"2024-09-10T10:43:52.869441+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.869441+0000","last_undegraded":"2024-09-10T10:43:52.869441+0000","last_fullsized":"2024-09-10T10:43:52.869441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:46.823226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869679+0000","last_change":"2024-09-10T10:43:51.570454+0000","last_active":"2024-09-10T10:43:52.869679+0000","last_peered":"2024-09-10T10:43:52.869679+0000","last_clean":"2024-09-10T10:43:52.869679+0000","last_became_active":"2024-09-10T10:43:51.570088+0000","last_became_peered":"2024-09-10T10:43:51.570088+0000","last_unstale":"2024-09-10T10:43:52.869679+0000","last_undegraded":"2024-09-10T10:43:52.869679+0000","last_fullsized":"2024-09-10T10:43:52.869679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:52.179305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708948+0000","last_change":"2024-09-10T10:43:46.521135+0000","last_active":"2024-09-10T10:43:52.708948+0000","last_peered":"2024-09-10T10:43:52.708948+0000","last_clean":"2024-09-10T10:43:52.708948+0000","last_became_active":"2024-09-10T10:43:46.520815+0000","last_became_peered":"2024-09-10T10:43:46.520815+0000","last_unstale":"2024-09-10T10:43:52.708948+0000","last_undegraded":"2024-09-10T10:43:52.708948+0000","last_fullsized":"2024-09-10T10:43:52.708948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:38.222213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564462+0000","last_change":"2024-09-10T10:43:47.545783+0000","last_active":"2024-09-10T10:43:52.564462+0000","last_peered":"2024-09-10T10:43:52.564462+0000","last_clean":"2024-09-10T10:43:52.564462+0000","last_became_active":"2024-09-10T10:43:47.545265+0000","last_became_peered":"2024-09-10T10:43:47.545265+0000","last_unstale":"2024-09-10T10:43:52.564462+0000","last_undegraded":"2024-09-10T10:43:52.564462+0000","last_fullsized":"2024-09-10T10:43:52.564462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564298+0000","last_change":"2024-09-10T10:43:51.585486+0000","last_active":"2024-09-10T10:43:52.564298+0000","last_peered":"2024-09-10T10:43:52.564298+0000","last_clean":"2024-09-10T10:43:52.564298+0000","last_became_active":"2024-09-10T10:43:51.585045+0000","last_became_peered":"2024-09-10T10:43:51.585045+0000","last_unstale":"2024-09-10T10:43:52.564298+0000","last_undegraded":"2024-09-10T10:43:52.564298+0000","last_fullsized":"2024-09-10T10:43:52.564298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:59.749737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563314+0000","last_change":"2024-09-10T10:43:47.533694+0000","last_active":"2024-09-10T10:43:52.563314+0000","last_peered":"2024-09-10T10:43:52.563314+0000","last_clean":"2024-09-10T10:43:52.563314+0000","last_became_active":"2024-09-10T10:43:47.533197+0000","last_became_peered":"2024-09-10T10:43:47.533197+0000","last_unstale":"2024-09-10T10:43:52.563314+0000","last_undegraded":"2024-09-10T10:43:52.563314+0000","last_fullsized":"2024-09-10T10:43:52.563314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563761+0000","last_change":"2024-09-10T10:43:47.540401+0000","last_active":"2024-09-10T10:43:52.563761+0000","last_peered":"2024-09-10T10:43:52.563761+0000","last_clean":"2024-09-10T10:43:52.563761+0000","last_became_active":"2024-09-10T10:43:47.539677+0000","last_became_peered":"2024-09-10T10:43:47.539677+0000","last_unstale":"2024-09-10T10:43:52.563761+0000","last_undegraded":"2024-09-10T10:43:52.563761+0000","last_fullsized":"2024-09-10T10:43:52.563761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868918+0000","last_change":"2024-09-10T10:43:47.539352+0000","last_active":"2024-09-10T10:43:52.868918+0000","last_peered":"2024-09-10T10:43:52.868918+0000","last_clean":"2024-09-10T10:43:52.868918+0000","last_became_active":"2024-09-10T10:43:47.539034+0000","last_became_peered":"2024-09-10T10:43:47.539034+0000","last_unstale":"2024-09-10T10:43:52.868918+0000","last_undegraded":"2024-09-10T10:43:52.868918+0000","last_fullsized":"2024-09-10T10:43:52.868918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:08.107825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708757+0000","last_change":"2024-09-10T10:43:47.542199+0000","last_active":"2024-09-10T10:43:52.708757+0000","last_peered":"2024-09-10T10:43:52.708757+0000","last_clean":"2024-09-10T10:43:52.708757+0000","last_became_active":"2024-09-10T10:43:47.535131+0000","last_became_peered":"2024-09-10T10:43:47.535131+0000","last_unstale":"2024-09-10T10:43:52.708757+0000","last_undegraded":"2024-09-10T10:43:52.708757+0000","last_fullsized":"2024-09-10T10:43:52.708757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:38.966819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870705+0000","last_change":"2024-09-10T10:43:47.546290+0000","last_active":"2024-09-10T10:43:52.870705+0000","last_peered":"2024-09-10T10:43:52.870705+0000","last_clean":"2024-09-10T10:43:52.870705+0000","last_became_active":"2024-09-10T10:43:47.545873+0000","last_became_peered":"2024-09-10T10:43:47.545873+0000","last_unstale":"2024-09-10T10:43:52.870705+0000","last_undegraded":"2024-09-10T10:43:52.870705+0000","last_fullsized":"2024-09-10T10:43:52.870705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870373+0000","last_change":"2024-09-10T10:43:47.537092+0000","last_active":"2024-09-10T10:43:52.870373+0000","last_peered":"2024-09-10T10:43:52.870373+0000","last_clean":"2024-09-10T10:43:52.870373+0000","last_became_active":"2024-09-10T10:43:47.536730+0000","last_became_peered":"2024-09-10T10:43:47.536730+0000","last_unstale":"2024-09-10T10:43:52.870373+0000","last_undegraded":"2024-09-10T10:43:52.870373+0000","last_fullsized":"2024-09-10T10:43:52.870373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564347+0000","last_change":"2024-09-10T10:43:47.557748+0000","last_active":"2024-09-10T10:43:52.564347+0000","last_peered":"2024-09-10T10:43:52.564347+0000","last_clean":"2024-09-10T10:43:52.564347+0000","last_became_active":"2024-09-10T10:43:47.555551+0000","last_became_peered":"2024-09-10T10:43:47.555551+0000","last_unstale":"2024-09-10T10:43:52.564347+0000","last_undegraded":"2024-09-10T10:43:52.564347+0000","last_fullsized":"2024-09-10T10:43:52.564347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708691+0000","last_change":"2024-09-10T10:43:47.544639+0000","last_active":"2024-09-10T10:43:52.708691+0000","last_peered":"2024-09-10T10:43:52.708691+0000","last_clean":"2024-09-10T10:43:52.708691+0000","last_became_active":"2024-09-10T10:43:47.544357+0000","last_became_peered":"2024-09-10T10:43:47.544357+0000","last_unstale":"2024-09-10T10:43:52.708691+0000","last_undegraded":"2024-09-10T10:43:52.708691+0000","last_fullsized":"2024-09-10T10:43:52.708691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:16.174872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868699+0000","last_change":"2024-09-10T10:43:47.544034+0000","last_active":"2024-09-10T10:43:52.868699+0000","last_peered":"2024-09-10T10:43:52.868699+0000","last_clean":"2024-09-10T10:43:52.868699+0000","last_became_active":"2024-09-10T10:43:47.543677+0000","last_became_peered":"2024-09-10T10:43:47.543677+0000","last_unstale":"2024-09-10T10:43:52.868699+0000","last_undegraded":"2024-09-10T10:43:52.868699+0000","last_fullsized":"2024-09-10T10:43:52.868699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:26.220907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708358+0000","last_change":"2024-09-10T10:43:47.561856+0000","last_active":"2024-09-10T10:43:52.708358+0000","last_peered":"2024-09-10T10:43:52.708358+0000","last_clean":"2024-09-10T10:43:52.708358+0000","last_became_active":"2024-09-10T10:43:47.561518+0000","last_became_peered":"2024-09-10T10:43:47.561518+0000","last_unstale":"2024-09-10T10:43:52.708358+0000","last_undegraded":"2024-09-10T10:43:52.708358+0000","last_fullsized":"2024-09-10T10:43:52.708358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:21.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:47.545540+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:47.544916+0000","last_became_peered":"2024-09-10T10:43:47.544916+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869156+0000","last_change":"2024-09-10T10:43:47.521227+0000","last_active":"2024-09-10T10:43:52.869156+0000","last_peered":"2024-09-10T10:43:52.869156+0000","last_clean":"2024-09-10T10:43:52.869156+0000","last_became_active":"2024-09-10T10:43:47.520811+0000","last_became_peered":"2024-09-10T10:43:47.520811+0000","last_unstale":"2024-09-10T10:43:52.869156+0000","last_undegraded":"2024-09-10T10:43:52.869156+0000","last_fullsized":"2024-09-10T10:43:52.869156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:55.708127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869588+0000","last_change":"2024-09-10T10:43:47.541681+0000","last_active":"2024-09-10T10:43:52.869588+0000","last_peered":"2024-09-10T10:43:52.869588+0000","last_clean":"2024-09-10T10:43:52.869588+0000","last_became_active":"2024-09-10T10:43:47.541056+0000","last_became_peered":"2024-09-10T10:43:47.541056+0000","last_unstale":"2024-09-10T10:43:52.869588+0000","last_undegraded":"2024-09-10T10:43:52.869588+0000","last_fullsized":"2024-09-10T10:43:52.869588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:56.224306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870640+0000","last_change":"2024-09-10T10:43:47.543423+0000","last_active":"2024-09-10T10:43:52.870640+0000","last_peered":"2024-09-10T10:43:52.870640+0000","last_clean":"2024-09-10T10:43:52.870640+0000","last_became_active":"2024-09-10T10:43:47.537728+0000","last_became_peered":"2024-09-10T10:43:47.537728+0000","last_unstale":"2024-09-10T10:43:52.870640+0000","last_undegraded":"2024-09-10T10:43:52.870640+0000","last_fullsized":"2024-09-10T10:43:52.870640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:10.407503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564680+0000","last_change":"2024-09-10T10:43:47.562659+0000","last_active":"2024-09-10T10:43:52.564680+0000","last_peered":"2024-09-10T10:43:52.564680+0000","last_clean":"2024-09-10T10:43:52.564680+0000","last_became_active":"2024-09-10T10:43:47.562238+0000","last_became_peered":"2024-09-10T10:43:47.562238+0000","last_unstale":"2024-09-10T10:43:52.564680+0000","last_undegraded":"2024-09-10T10:43:52.564680+0000","last_fullsized":"2024-09-10T10:43:52.564680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870710+0000","last_change":"2024-09-10T10:43:47.548547+0000","last_active":"2024-09-10T10:43:52.870710+0000","last_peered":"2024-09-10T10:43:52.870710+0000","last_clean":"2024-09-10T10:43:52.870710+0000","last_became_active":"2024-09-10T10:43:47.548237+0000","last_became_peered":"2024-09-10T10:43:47.548237+0000","last_unstale":"2024-09-10T10:43:52.870710+0000","last_undegraded":"2024-09-10T10:43:52.870710+0000","last_fullsized":"2024-09-10T10:43:52.870710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:55.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868674+0000","last_change":"2024-09-10T10:43:47.559813+0000","last_active":"2024-09-10T10:43:52.868674+0000","last_peered":"2024-09-10T10:43:52.868674+0000","last_clean":"2024-09-10T10:43:52.868674+0000","last_became_active":"2024-09-10T10:43:47.559517+0000","last_became_peered":"2024-09-10T10:43:47.559517+0000","last_unstale":"2024-09-10T10:43:52.868674+0000","last_undegraded":"2024-09-10T10:43:52.868674+0000","last_fullsized":"2024-09-10T10:43:52.868674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:28.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.546737+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.546456+0000","last_became_peered":"2024-09-10T10:43:47.546456+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:42.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870025+0000","last_change":"2024-09-10T10:43:47.533339+0000","last_active":"2024-09-10T10:43:52.870025+0000","last_peered":"2024-09-10T10:43:52.870025+0000","last_clean":"2024-09-10T10:43:52.870025+0000","last_became_active":"2024-09-10T10:43:47.533019+0000","last_became_peered":"2024-09-10T10:43:47.533019+0000","last_unstale":"2024-09-10T10:43:52.870025+0000","last_undegraded":"2024-09-10T10:43:52.870025+0000","last_fullsized":"2024-09-10T10:43:52.870025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870439+0000","last_change":"2024-09-10T10:43:47.546293+0000","last_active":"2024-09-10T10:43:52.870439+0000","last_peered":"2024-09-10T10:43:52.870439+0000","last_clean":"2024-09-10T10:43:52.870439+0000","last_became_active":"2024-09-10T10:43:47.545952+0000","last_became_peered":"2024-09-10T10:43:47.545952+0000","last_unstale":"2024-09-10T10:43:52.870439+0000","last_undegraded":"2024-09-10T10:43:52.870439+0000","last_fullsized":"2024-09-10T10:43:52.870439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869535+0000","last_change":"2024-09-10T10:43:47.541403+0000","last_active":"2024-09-10T10:43:52.869535+0000","last_peered":"2024-09-10T10:43:52.869535+0000","last_clean":"2024-09-10T10:43:52.869535+0000","last_became_active":"2024-09-10T10:43:47.540724+0000","last_became_peered":"2024-09-10T10:43:47.540724+0000","last_unstale":"2024-09-10T10:43:52.869535+0000","last_undegraded":"2024-09-10T10:43:52.869535+0000","last_fullsized":"2024-09-10T10:43:52.869535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:55.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869473+0000","last_change":"2024-09-10T10:43:47.550017+0000","last_active":"2024-09-10T10:43:52.869473+0000","last_peered":"2024-09-10T10:43:52.869473+0000","last_clean":"2024-09-10T10:43:52.869473+0000","last_became_active":"2024-09-10T10:43:47.549763+0000","last_became_peered":"2024-09-10T10:43:47.549763+0000","last_unstale":"2024-09-10T10:43:52.869473+0000","last_undegraded":"2024-09-10T10:43:52.869473+0000","last_fullsized":"2024-09-10T10:43:52.869473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:12.076487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869399+0000","last_change":"2024-09-10T10:43:47.558985+0000","last_active":"2024-09-10T10:43:52.869399+0000","last_peered":"2024-09-10T10:43:52.869399+0000","last_clean":"2024-09-10T10:43:52.869399+0000","last_became_active":"2024-09-10T10:43:47.558695+0000","last_became_peered":"2024-09-10T10:43:47.558695+0000","last_unstale":"2024-09-10T10:43:52.869399+0000","last_undegraded":"2024-09-10T10:43:52.869399+0000","last_fullsized":"2024-09-10T10:43:52.869399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:05.319579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565366+0000","last_change":"2024-09-10T10:43:47.547413+0000","last_active":"2024-09-10T10:43:52.565366+0000","last_peered":"2024-09-10T10:43:52.565366+0000","last_clean":"2024-09-10T10:43:52.565366+0000","last_became_active":"2024-09-10T10:43:47.547008+0000","last_became_peered":"2024-09-10T10:43:47.547008+0000","last_unstale":"2024-09-10T10:43:52.565366+0000","last_undegraded":"2024-09-10T10:43:52.565366+0000","last_fullsized":"2024-09-10T10:43:52.565366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.545409+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.545121+0000","last_became_peered":"2024-09-10T10:43:47.545121+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.641844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869042+0000","last_change":"2024-09-10T10:43:47.534510+0000","last_active":"2024-09-10T10:43:52.869042+0000","last_peered":"2024-09-10T10:43:52.869042+0000","last_clean":"2024-09-10T10:43:52.869042+0000","last_became_active":"2024-09-10T10:43:47.534143+0000","last_became_peered":"2024-09-10T10:43:47.534143+0000","last_unstale":"2024-09-10T10:43:52.869042+0000","last_undegraded":"2024-09-10T10:43:52.869042+0000","last_fullsized":"2024-09-10T10:43:52.869042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:33.210661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563579+0000","last_change":"2024-09-10T10:43:47.525750+0000","last_active":"2024-09-10T10:43:52.563579+0000","last_peered":"2024-09-10T10:43:52.563579+0000","last_clean":"2024-09-10T10:43:52.563579+0000","last_became_active":"2024-09-10T10:43:47.525379+0000","last_became_peered":"2024-09-10T10:43:47.525379+0000","last_unstale":"2024-09-10T10:43:52.563579+0000","last_undegraded":"2024-09-10T10:43:52.563579+0000","last_fullsized":"2024-09-10T10:43:52.563579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869264+0000","last_change":"2024-09-10T10:43:47.546839+0000","last_active":"2024-09-10T10:43:52.869264+0000","last_peered":"2024-09-10T10:43:52.869264+0000","last_clean":"2024-09-10T10:43:52.869264+0000","last_became_active":"2024-09-10T10:43:47.544570+0000","last_became_peered":"2024-09-10T10:43:47.544570+0000","last_unstale":"2024-09-10T10:43:52.869264+0000","last_undegraded":"2024-09-10T10:43:52.869264+0000","last_fullsized":"2024-09-10T10:43:52.869264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:53.642976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564645+0000","last_change":"2024-09-10T10:43:47.522224+0000","last_active":"2024-09-10T10:43:52.564645+0000","last_peered":"2024-09-10T10:43:52.564645+0000","last_clean":"2024-09-10T10:43:52.564645+0000","last_became_active":"2024-09-10T10:43:47.521820+0000","last_became_peered":"2024-09-10T10:43:47.521820+0000","last_unstale":"2024-09-10T10:43:52.564645+0000","last_undegraded":"2024-09-10T10:43:52.564645+0000","last_fullsized":"2024-09-10T10:43:52.564645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:47.538393+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:47.537972+0000","last_became_peered":"2024-09-10T10:43:47.537972+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:30.153548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708731+0000","last_change":"2024-09-10T10:43:47.540588+0000","last_active":"2024-09-10T10:43:52.708731+0000","last_peered":"2024-09-10T10:43:52.708731+0000","last_clean":"2024-09-10T10:43:52.708731+0000","last_became_active":"2024-09-10T10:43:47.540358+0000","last_became_peered":"2024-09-10T10:43:47.540358+0000","last_unstale":"2024-09-10T10:43:52.708731+0000","last_undegraded":"2024-09-10T10:43:52.708731+0000","last_fullsized":"2024-09-10T10:43:52.708731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:30.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:47.549392+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:47.546137+0000","last_became_peered":"2024-09-10T10:43:47.546137+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:02.975066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":111561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":111437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":115281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":701875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":111493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":884736,"data_stored":701897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:10.253 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:44:10.253 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T10:44:10.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T10:44:10.254 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T10:44:10.254 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T10:44:10.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.445+0000 7fcdc3200640 1 Processor -- start 2024-09-10T10:44:10.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.445+0000 7fcdc3200640 1 -- start start 2024-09-10T10:44:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdc40d0dd0 0x7fcdc40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc40cf000 con 0x7fcdc415e660 2024-09-10T10:44:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc406d3e0 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40b37d0 con 0x7fcdc40d1f90 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.446+0000 7fcdc2800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44120/0 (socket says 172.21.15.28:44120) 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc2800640 1 -- 172.21.15.28:0/391197975 learned_addr learned my addr 172.21.15.28:0/391197975 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc1400640 1 --1- 172.21.15.28:0/391197975 >> v1:172.21.15.138:6789/0 conn(0x7fcdc415e660 0x7fcdc406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52446/0 (socket says 172.21.15.28:52446) 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc1e00640 1 --2- 172.21.15.28:0/391197975 >> v2:172.21.15.138:3300/0 conn(0x7fcdc40d0dd0 0x7fcdc40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc2800640 1 -- 172.21.15.28:0/391197975 >> v2:172.21.15.138:3300/0 conn(0x7fcdc40d0dd0 msgr2=0x7fcdc40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc2800640 1 --2- 172.21.15.28:0/391197975 >> v2:172.21.15.138:3300/0 conn(0x7fcdc40d0dd0 0x7fcdc40d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc2800640 1 -- 172.21.15.28:0/391197975 >> v1:172.21.15.138:6789/0 conn(0x7fcdc415e660 legacy=0x7fcdc406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc0a00640 1 -- 172.21.15.28:0/391197975 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3862775171 0 0) 0x7fcdc40cf000 con 0x7fcdc415e660 2024-09-10T10:44:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.447+0000 7fcdc2800640 1 -- 172.21.15.28:0/391197975 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc40d16f0 con 0x7fcdc40d1f90 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.448+0000 7fcdc2800640 1 --2- 172.21.15.28:0/391197975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e50 tx=0x7fcdbc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e2014f8400cb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.448+0000 7fcdc0a00640 1 -- 172.21.15.28:0/391197975 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc036070 con 0x7fcdc40d1f90 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.448+0000 7fcdc0a00640 1 -- 172.21.15.28:0/391197975 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdbc031600 con 0x7fcdc40d1f90 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.448+0000 7fcdc0a00640 1 -- 172.21.15.28:0/391197975 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0319d0 con 0x7fcdc40d1f90 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 -- 172.21.15.28:0/391197975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 msgr2=0x7fcdc40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 --2- 172.21.15.28:0/391197975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009e50 tx=0x7fcdbc02a710 comp rx=0 tx=0).stop 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 -- 172.21.15.28:0/391197975 shutdown_connections 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 --2- 172.21.15.28:0/391197975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc40d1f90 0x7fcdc40cf7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 --2- 172.21.15.28:0/391197975 >> v2:172.21.15.138:3300/0 conn(0x7fcdc40d0dd0 0x7fcdc40d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 -- 172.21.15.28:0/391197975 >> 172.21.15.28:0/391197975 conn(0x7fcdc408bfc0 msgr2=0x7fcdc40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 -- 172.21.15.28:0/391197975 shutdown_connections 2024-09-10T10:44:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.449+0000 7fcdc3200640 1 -- 172.21.15.28:0/391197975 wait complete. 2024-09-10T10:44:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.450+0000 7fcdc3200640 1 Processor -- start 2024-09-10T10:44:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.451+0000 7fcdc3200640 1 -- start start 2024-09-10T10:44:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.451+0000 7fcdc3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.451+0000 7fcdc3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdc40d1f90 0x7fcdc415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc415e660 0x7fcdc40b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40d2c60 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40cf330 con 0x7fcdc40d1f90 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40d2f40 con 0x7fcdc415e660 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36456/0 (socket says 172.21.15.28:36456) 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 -- 172.21.15.28:0/2451291717 learned_addr learned my addr 172.21.15.28:0/2451291717 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc2800640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc415e660 0x7fcdc40b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1400640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdc40d1f90 0x7fcdc415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 -- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc415e660 msgr2=0x7fcdc40b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc415e660 0x7fcdc40b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 -- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdc40d1f90 msgr2=0x7fcdc415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdc40d1f90 0x7fcdc415c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 -- 172.21.15.28:0/2451291717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc40b90a0 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.452+0000 7fcdc1e00640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e870 tx=0x7fcdb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.453+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4017070 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.453+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb400c940 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.453+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400cd30 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.453+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc40b9380 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.453+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc40b9940 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.454+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc40d3220 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.455+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdb401a070 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.455+0000 7fcdba000640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 0x7fcda406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.456+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcdb408b0f0 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.456+0000 7fcdc1400640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 0x7fcda406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.457+0000 7fcdc1400640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 0x7fcda406e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007fd0 tx=0x7fcdb0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.460+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcdb40596c0 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.697+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcdc40d1b20 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.700+0000 7fcdba000640 1 -- 172.21.15.28:0/2451291717 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172943 (secure 0 0 0) 0x7fcdb4058fc0 con 0x7fcdc40d0dd0 2024-09-10T10:44:10.703 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.703+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 msgr2=0x7fcda406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.703+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 0x7fcda406e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007fd0 tx=0x7fcdb0009040 comp rx=0 tx=0).stop 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.703+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 msgr2=0x7fcdc40c9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.703+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e870 tx=0x7fcdb400ed30 comp rx=0 tx=0).stop 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 shutdown_connections 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcda406be30 0x7fcda406e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdc415e660 0x7fcdc40b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdc40d1f90 0x7fcdc415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 --2- 172.21.15.28:0/2451291717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcdc40d0dd0 0x7fcdc40c9600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 >> 172.21.15.28:0/2451291717 conn(0x7fcdc408bfc0 msgr2=0x7fcdc40b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 shutdown_connections 2024-09-10T10:44:10.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.704+0000 7fcdc3200640 1 -- 172.21.15.28:0/2451291717 wait complete. 2024-09-10T10:44:10.720 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":980174962},{"type":"v1","addr":"172.21.15.138:6801","nonce":980174962}]},"active_addr":"172.21.15.138:6801/980174962","active_change":"2024-09-10T10:43:26.470836+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2367472067}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2304804587}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1035207416}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2725598607}]}]} 2024-09-10T10:44:10.725 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T10:44:10.725 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T10:44:10.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:10.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.881+0000 7f4512800640 1 Processor -- start 2024-09-10T10:44:10.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.881+0000 7f4512800640 1 -- start start 2024-09-10T10:44:10.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f4512800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c161c80 0x7f450c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f4512800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f450c0b6680 0x7f450c0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f4512800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450c06e870 con 0x7f450c162060 2024-09-10T10:44:10.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f4512800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f450c06d9f0 con 0x7f450c0b6680 2024-09-10T10:44:10.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f4512800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f450c06dcd0 con 0x7f450c161c80 2024-09-10T10:44:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f450be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f450c0b6680 0x7f450c0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f450be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f450c0b6680 0x7f450c0cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36468/0 (socket says 172.21.15.28:36468) 2024-09-10T10:44:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.882+0000 7f450be00640 1 -- 172.21.15.28:0/3128899607 learned_addr learned my addr 172.21.15.28:0/3128899607 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.883+0000 7f450b400640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c161c80 0x7f450c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.883+0000 7f450be00640 1 -- 172.21.15.28:0/3128899607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c161c80 msgr2=0x7f450c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:10.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.883+0000 7f450be00640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c161c80 0x7f450c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.883+0000 7f450be00640 1 -- 172.21.15.28:0/3128899607 >> v1:172.21.15.138:6789/0 conn(0x7f450c162060 legacy=0x7f450c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:10.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.883+0000 7f450be00640 1 -- 172.21.15.28:0/3128899607 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c06dfb0 con 0x7f450c0b6680 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.884+0000 7f450be00640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c0b6680 0x7f450c0cf1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4504009e50 tx=0x7f450402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7f88f41ff42a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.884+0000 7f450aa00640 1 -- 172.21.15.28:0/3128899607 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504036070 con 0x7f450c0b6680 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.884+0000 7f450aa00640 1 -- 172.21.15.28:0/3128899607 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504031600 con 0x7f450c0b6680 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.884+0000 7f450aa00640 1 -- 172.21.15.28:0/3128899607 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040319d0 con 0x7f450c0b6680 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 -- 172.21.15.28:0/3128899607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c0b6680 msgr2=0x7f450c0cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c0b6680 0x7f450c0cf1e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4504009e50 tx=0x7f450402a710 comp rx=0 tx=0).stop 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 -- 172.21.15.28:0/3128899607 shutdown_connections 2024-09-10T10:44:10.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c0b6680 0x7f450c0cf1e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 --2- 172.21.15.28:0/3128899607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c161c80 0x7f450c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 -- 172.21.15.28:0/3128899607 >> 172.21.15.28:0/3128899607 conn(0x7f450c0d44f0 msgr2=0x7f450c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 -- 172.21.15.28:0/3128899607 shutdown_connections 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.885+0000 7f4512800640 1 -- 172.21.15.28:0/3128899607 wait complete. 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.886+0000 7f4512800640 1 Processor -- start 2024-09-10T10:44:10.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 -- start start 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c0b6680 0x7f450c0b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f450c161c80 0x7f450c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0b6b30 con 0x7f450c162060 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0d2aa0 con 0x7f450c161c80 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.887+0000 7f4512800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f450c0b6cb0 con 0x7f450c0b6680 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36482/0 (socket says 172.21.15.28:36482) 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 -- 172.21.15.28:0/851116235 learned_addr learned my addr 172.21.15.28:0/851116235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450b400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c0b6680 0x7f450c0b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f4503e00640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f450c161c80 0x7f450c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 -- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c0b6680 msgr2=0x7f450c0b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c0b6680 0x7f450c0b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 -- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f450c161c80 msgr2=0x7f450c1563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f450c161c80 0x7f450c1563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.888+0000 7f450be00640 1 -- 172.21.15.28:0/851116235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c0b7920 con 0x7f450c162060 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f450be00640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4504009f70 tx=0x7f4504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450403f020 con 0x7f450c162060 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4504037040 con 0x7f450c162060 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504036070 con 0x7f450c162060 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c0b7c00 con 0x7f450c162060 2024-09-10T10:44:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.889+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c15a4a0 con 0x7f450c162060 2024-09-10T10:44:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.890+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c0ba470 con 0x7f450c162060 2024-09-10T10:44:10.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.891+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4504042020 con 0x7f450c162060 2024-09-10T10:44:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.892+0000 7f4503400640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 0x7f44e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.892+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f45040ab550 con 0x7f450c162060 2024-09-10T10:44:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.892+0000 7f450b400640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 0x7f44e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.893+0000 7f450b400640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 0x7f44e806e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0079a0 tx=0x7f44fc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:10.896+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4504049030 con 0x7f450c162060 2024-09-10T10:44:11.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.080+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f450c0b5f80 con 0x7f450c162060 2024-09-10T10:44:11.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.081+0000 7f4503400640 1 -- 172.21.15.28:0/851116235 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7f4504079560 con 0x7f450c162060 2024-09-10T10:44:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:11.082 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:11.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 msgr2=0x7f44e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 0x7f44e806e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0079a0 tx=0x7f44fc008040 comp rx=0 tx=0).stop 2024-09-10T10:44:11.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 msgr2=0x7f450c0b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4504009f70 tx=0x7f4504004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 shutdown_connections 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44e806be30 0x7f44e806e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f450c162060 0x7f450c0b20b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f450c161c80 0x7f450c1563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 --2- 172.21.15.28:0/851116235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f450c0b6680 0x7f450c0b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 >> 172.21.15.28:0/851116235 conn(0x7f450c0d44f0 msgr2=0x7f450c0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 shutdown_connections 2024-09-10T10:44:11.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.083+0000 7f4512800640 1 -- 172.21.15.28:0/851116235 wait complete. 2024-09-10T10:44:11.101 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T10:44:11.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:11.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.257+0000 7f39d6200640 1 Processor -- start 2024-09-10T10:44:11.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.257+0000 7f39d6200640 1 -- start start 2024-09-10T10:44:11.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39d6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39d015e5d0 0x7f39d00d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39d6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 0x7f39d006c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39d6200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d00d0980 con 0x7f39d00d1b70 2024-09-10T10:44:11.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39d6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f39d015e9b0 con 0x7f39d015e5d0 2024-09-10T10:44:11.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39d6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39d00d0c60 con 0x7f39d006bc10 2024-09-10T10:44:11.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39cfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f39d00d1b70 0x7f39d0073780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41154/0 (socket says 172.21.15.28:41154) 2024-09-10T10:44:11.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.258+0000 7f39cfe00640 1 -- 172.21.15.28:0/2085210510 learned_addr learned my addr 172.21.15.28:0/2085210510 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d5800640 1 --2- 172.21.15.28:0/2085210510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 0x7f39d006c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d4e00640 1 --2- 172.21.15.28:0/2085210510 >> v2:172.21.15.138:3300/0 conn(0x7f39d015e5d0 0x7f39d00d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39cf400640 1 -- 172.21.15.28:0/2085210510 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826268497 0 0) 0x7f39d00d0980 con 0x7f39d00d1b70 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39cf400640 1 -- 172.21.15.28:0/2085210510 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b8003520 con 0x7f39d00d1b70 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d5800640 1 -- 172.21.15.28:0/2085210510 >> v2:172.21.15.138:3300/0 conn(0x7f39d015e5d0 msgr2=0x7f39d00d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d5800640 1 --2- 172.21.15.28:0/2085210510 >> v2:172.21.15.138:3300/0 conn(0x7f39d015e5d0 0x7f39d00d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d5800640 1 -- 172.21.15.28:0/2085210510 >> v1:172.21.15.28:6789/0 conn(0x7f39d00d1b70 legacy=0x7f39d0073780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.259+0000 7f39d5800640 1 -- 172.21.15.28:0/2085210510 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d0072f80 con 0x7f39d006bc10 2024-09-10T10:44:11.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.260+0000 7f39d5800640 1 --2- 172.21.15.28:0/2085210510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 0x7f39d006c0c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009d90 tx=0x7f39c802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df57565a4072bb3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.260+0000 7f39cf400640 1 -- 172.21.15.28:0/2085210510 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8031600 con 0x7f39d006bc10 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39cf400640 1 -- 172.21.15.28:0/2085210510 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c80317a0 con 0x7f39d006bc10 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39cf400640 1 -- 172.21.15.28:0/2085210510 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8031b50 con 0x7f39d006bc10 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 -- 172.21.15.28:0/2085210510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 msgr2=0x7f39d006c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 --2- 172.21.15.28:0/2085210510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 0x7f39d006c0c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009d90 tx=0x7f39c802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 -- 172.21.15.28:0/2085210510 shutdown_connections 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 --2- 172.21.15.28:0/2085210510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d006bc10 0x7f39d006c0c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 --2- 172.21.15.28:0/2085210510 >> v2:172.21.15.138:3300/0 conn(0x7f39d015e5d0 0x7f39d00d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 -- 172.21.15.28:0/2085210510 >> 172.21.15.28:0/2085210510 conn(0x7f39d00d1080 msgr2=0x7f39d00d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 -- 172.21.15.28:0/2085210510 shutdown_connections 2024-09-10T10:44:11.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.261+0000 7f39d6200640 1 -- 172.21.15.28:0/2085210510 wait complete. 2024-09-10T10:44:11.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.263+0000 7f39d6200640 1 Processor -- start 2024-09-10T10:44:11.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.263+0000 7f39d6200640 1 -- start start 2024-09-10T10:44:11.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d00d1b70 0x7f39d00b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d015e5d0 0x7f39d015eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39d006cb60 con 0x7f39d006bc10 2024-09-10T10:44:11.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39d006c9e0 con 0x7f39d00d1b70 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39d0162030 con 0x7f39d015e5d0 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36496/0 (socket says 172.21.15.28:36496) 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 -- 172.21.15.28:0/3596058106 learned_addr learned my addr 172.21.15.28:0/3596058106 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d5800640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d015e5d0 0x7f39d015eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39cfe00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d00d1b70 0x7f39d00b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 -- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d015e5d0 msgr2=0x7f39d015eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d015e5d0 0x7f39d015eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 -- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d00d1b70 msgr2=0x7f39d00b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d00d1b70 0x7f39d00b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.264+0000 7f39d4e00640 1 -- 172.21.15.28:0/3596058106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00b5760 con 0x7f39d006bc10 2024-09-10T10:44:11.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39d4e00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e8a0 tx=0x7f39c400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4017070 con 0x7f39d006bc10 2024-09-10T10:44:11.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c400c940 con 0x7f39d006bc10 2024-09-10T10:44:11.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c400cd30 con 0x7f39d006bc10 2024-09-10T10:44:11.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d00b5a40 con 0x7f39d006bc10 2024-09-10T10:44:11.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.265+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d00b6000 con 0x7f39d006bc10 2024-09-10T10:44:11.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.266+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d00d0310 con 0x7f39d006bc10 2024-09-10T10:44:11.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.267+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f39c401a070 con 0x7f39d006bc10 2024-09-10T10:44:11.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.268+0000 7f39ccc00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 0x7f39b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.268+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f39c408b400 con 0x7f39d006bc10 2024-09-10T10:44:11.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.269+0000 7f39cfe00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 0x7f39b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.269+0000 7f39cfe00640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 0x7f39b006e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39d01612a0 tx=0x7f39bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.272+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39c4059a60 con 0x7f39d006bc10 2024-09-10T10:44:11.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.459+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39d0073ec0 con 0x7f39d006bc10 2024-09-10T10:44:11.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.460+0000 7f39ccc00640 1 -- 172.21.15.28:0/3596058106 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7f39c4059360 con 0x7f39d006bc10 2024-09-10T10:44:11.461 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:11.462 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 msgr2=0x7f39b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 0x7f39b006e2f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39d01612a0 tx=0x7f39bc009040 comp rx=0 tx=0).stop 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 msgr2=0x7f39d0157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e8a0 tx=0x7f39c400ed60 comp rx=0 tx=0).stop 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 shutdown_connections 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f39b006be30 0x7f39b006e2f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d015e5d0 0x7f39d015eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d00d1b70 0x7f39d00b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 --2- 172.21.15.28:0/3596058106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39d006bc10 0x7f39d0157720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.462+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 >> 172.21.15.28:0/3596058106 conn(0x7f39d00d1080 msgr2=0x7f39d00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.463+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 shutdown_connections 2024-09-10T10:44:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.463+0000 7f39d6200640 1 -- 172.21.15.28:0/3596058106 wait complete. 2024-09-10T10:44:11.479 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T10:44:11.480 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T10:44:11.480 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T10:44:11.481 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T10:44:11.481 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T10:44:11.481 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T10:44:11.482 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T10:44:11.482 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T10:44:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.758+0000 7f9587a00640 1 Processor -- start 2024-09-10T10:44:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.759+0000 7f9587a00640 1 -- start start 2024-09-10T10:44:11.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 0x7f95800b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 0x7f958006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 0x7f958006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f95800cda90 con 0x7f95800b4fc0 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f95800cdcc0 con 0x7f95800b3f30 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.761+0000 7f9587a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95800cdf60 con 0x7f95800b3af0 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f9585a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 0x7f958006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f9585a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 0x7f958006e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36030/0 (socket says 172.21.15.28:36030) 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f9585a00640 1 -- 172.21.15.28:0/623242157 learned_addr learned my addr 172.21.15.28:0/623242157 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 --2- 172.21.15.28:0/623242157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 0x7f958006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f9585000640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 0x7f95800b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.763+0000 7fbb69e00640 1 Processor -- start 2024-09-10T10:44:11.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.763+0000 7fbb69e00640 1 -- start start 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 -- 172.21.15.28:0/623242157 >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 msgr2=0x7f958006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 0x7f958006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 -- 172.21.15.28:0/623242157 >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 msgr2=0x7f95800b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 0x7f95800b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.762+0000 7f957fe00640 1 -- 172.21.15.28:0/623242157 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95800ce240 con 0x7f95800b3af0 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.763+0000 7f9585000640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 0x7f95800b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:11.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.764+0000 7f957fe00640 1 --2- 172.21.15.28:0/623242157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 0x7f958006dc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95800afbd0 tx=0x7f956c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9329f5dd02f38b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7fbb69e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb6406e370 0x7fbb64141820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7f957f400640 1 -- 172.21.15.28:0/623242157 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c037070 con 0x7f95800b3af0 2024-09-10T10:44:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7f957f400640 1 -- 172.21.15.28:0/623242157 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c032440 con 0x7f95800b3af0 2024-09-10T10:44:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7f957f400640 1 -- 172.21.15.28:0/623242157 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c02ae60 con 0x7f95800b3af0 2024-09-10T10:44:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7f7efb800640 1 Processor -- start 2024-09-10T10:44:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.765+0000 7f7efb800640 1 -- start start 2024-09-10T10:44:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7f9587a00640 1 -- 172.21.15.28:0/623242157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 msgr2=0x7f958006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7f9587a00640 1 --2- 172.21.15.28:0/623242157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 0x7f958006dc60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95800afbd0 tx=0x7f956c02a110 comp rx=0 tx=0).stop 2024-09-10T10:44:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7fbb69e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb64142550 con 0x7fbb64141d60 2024-09-10T10:44:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7fbb69e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb6413f6b0 con 0x7fbb6415d2d0 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7fbb69e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbb6413f430 con 0x7fbb6406e370 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.766+0000 7fbb63400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbb64141d60 0x7fbb64140470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41156/0 (socket says 172.21.15.28:41156) 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7fbb63400640 1 -- 172.21.15.28:0/3594275268 learned_addr learned my addr 172.21.15.28:0/3594275268 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7fbb62000640 1 --2- 172.21.15.28:0/3594275268 >> v2:172.21.15.28:3300/0 conn(0x7fbb6406e370 0x7fbb64141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7efb800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ef40cfba0 0x7f7ef4073900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f9587a00640 1 -- 172.21.15.28:0/623242157 shutdown_connections 2024-09-10T10:44:11.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f9587a00640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.138:3300/0 conn(0x7f95800b3f30 0x7f958006e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f9587a00640 1 --2- 172.21.15.28:0/623242157 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3af0 0x7f958006dc60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f9587a00640 1 --2- 172.21.15.28:0/623242157 >> v2:172.21.15.28:3300/0 conn(0x7f95800b4fc0 0x7f95800b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7efb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f9587a00640 1 -- 172.21.15.28:0/623242157 >> 172.21.15.28:0/623242157 conn(0x7f9580067950 msgr2=0x7f9580067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7efb800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef415e340 con 0x7f7ef40b3130 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7efb800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef415e620 con 0x7f7ef40cfba0 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7efb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef415e900 con 0x7f7ef406bcb0 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7ef8e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ef40cfba0 0x7f7ef4073900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44212/0 (socket says 172.21.15.28:44212) 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.767+0000 7f7eebe00640 1 -- 172.21.15.28:0/3044316043 learned_addr learned my addr 172.21.15.28:0/3044316043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f9587a00640 1 -- 172.21.15.28:0/623242157 shutdown_connections 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f9587a00640 1 -- 172.21.15.28:0/623242157 wait complete. 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 466350247 0 0) 0x7fbb64142550 con 0x7fbb64141d60 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb48003520 con 0x7fbb64141d60 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.769+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2723149949 0 0) 0x7fbb48003520 con 0x7fbb64141d60 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.769+0000 7fbb62000640 1 -- 172.21.15.28:0/3594275268 >> v1:172.21.15.138:6789/0 conn(0x7fbb6415d2d0 legacy=0x7fbb6406dc60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:44:11.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.769+0000 7fbb62000640 1 -- 172.21.15.28:0/3594275268 >> v1:172.21.15.28:6789/0 conn(0x7fbb64141d60 legacy=0x7fbb64140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.769+0000 7fbb62000640 1 -- 172.21.15.28:0/3594275268 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6413f990 con 0x7fbb6406e370 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f7eebe00640 1 -- 172.21.15.28:0/3044316043 >> v2:172.21.15.138:3300/0 conn(0x7f7ef40cfba0 msgr2=0x7f7ef4073900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f7eebe00640 1 --2- 172.21.15.28:0/3044316043 >> v2:172.21.15.138:3300/0 conn(0x7f7ef40cfba0 0x7f7ef4073900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f7eebe00640 1 -- 172.21.15.28:0/3044316043 >> v1:172.21.15.138:6789/0 conn(0x7f7ef40b3130 legacy=0x7f7ef406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.768+0000 7f7eebe00640 1 -- 172.21.15.28:0/3044316043 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef415ebe0 con 0x7f7ef406bcb0 2024-09-10T10:44:11.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.770+0000 7f9587a00640 1 Processor -- start 2024-09-10T10:44:11.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.770+0000 7f9587a00640 1 -- start start 2024-09-10T10:44:11.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.770+0000 7fbb62000640 1 --2- 172.21.15.28:0/3594275268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6406e370 0x7fbb64141820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb540097a0 tx=0x7fbb5402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baee9f56a4866116 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.771+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54037070 con 0x7fbb6406e370 2024-09-10T10:44:11.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.771+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5402abf0 con 0x7fbb6406e370 2024-09-10T10:44:11.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.771+0000 7fbb61600640 1 -- 172.21.15.28:0/3594275268 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50001670 con 0x7fbb6406e370 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.771+0000 7f7eebe00640 1 --2- 172.21.15.28:0/3044316043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009870 tx=0x7f7edc02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cafacaff027c304 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f7ef3e00640 1 -- 172.21.15.28:0/3044316043 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc037070 con 0x7f7ef406bcb0 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f7ef3e00640 1 -- 172.21.15.28:0/3044316043 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7edc032440 con 0x7f7ef406bcb0 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f9587a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95800b3af0 0x7f958015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f9587a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3f30 0x7f958016aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f9587a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 0x7f95801610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7f7ef3e00640 1 -- 172.21.15.28:0/3044316043 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7edc032810 con 0x7f7ef406bcb0 2024-09-10T10:44:11.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7fbb69e00640 1 -- 172.21.15.28:0/3594275268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6406e370 msgr2=0x7fbb64141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.772+0000 7fbb69e00640 1 --2- 172.21.15.28:0/3594275268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6406e370 0x7fbb64141820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbb540097a0 tx=0x7fbb5402a160 comp rx=0 tx=0).stop 2024-09-10T10:44:11.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9587a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9580071db0 con 0x7f95800b3af0 2024-09-10T10:44:11.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 -- 172.21.15.28:0/3044316043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 msgr2=0x7f7ef415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 --2- 172.21.15.28:0/3044316043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7edc009870 tx=0x7f7edc02a250 comp rx=0 tx=0).stop 2024-09-10T10:44:11.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9585a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 0x7f95801610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9585a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 0x7f95801610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36048/0 (socket says 172.21.15.28:36048) 2024-09-10T10:44:11.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9585a00640 1 -- 172.21.15.28:0/2196109766 learned_addr learned my addr 172.21.15.28:0/2196109766 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9587a00640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9580071c30 con 0x7f95800b4fc0 2024-09-10T10:44:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9587a00640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9580071f30 con 0x7f95800b3f30 2024-09-10T10:44:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7fbb69e00640 1 -- 172.21.15.28:0/3594275268 shutdown_connections 2024-09-10T10:44:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7fbb69e00640 1 --2- 172.21.15.28:0/3594275268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6406e370 0x7fbb64141820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7fbb69e00640 1 -- 172.21.15.28:0/3594275268 >> 172.21.15.28:0/3594275268 conn(0x7fbb64067d70 msgr2=0x7fbb6406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 -- 172.21.15.28:0/3044316043 shutdown_connections 2024-09-10T10:44:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7fbb69e00640 1 -- 172.21.15.28:0/3594275268 shutdown_connections 2024-09-10T10:44:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f9585000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95800b3af0 0x7f958015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f957fe00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3f30 0x7f958016aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 --2- 172.21.15.28:0/3044316043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef415a070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 --2- 172.21.15.28:0/3044316043 >> v2:172.21.15.138:3300/0 conn(0x7f7ef40cfba0 0x7f7ef4073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.773+0000 7f7efb800640 1 -- 172.21.15.28:0/3044316043 >> 172.21.15.28:0/3044316043 conn(0x7f7ef4067950 msgr2=0x7f7ef4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 -- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3f30 msgr2=0x7f958016aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95800b3f30 0x7f958016aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 -- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 msgr2=0x7f95801610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 0x7f95801610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9580161960 con 0x7f95800b3af0 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585a00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95800b4fc0 0x7f95801610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:11.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f7efb800640 1 -- 172.21.15.28:0/3044316043 shutdown_connections 2024-09-10T10:44:11.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f9585000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95800b3af0 0x7f958015d4e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f957800cfa0 tx=0x7f957800a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.774+0000 7f7efb800640 1 -- 172.21.15.28:0/3044316043 wait complete. 2024-09-10T10:44:11.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.776+0000 7f3a78c00640 1 Processor -- start 2024-09-10T10:44:11.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.775+0000 7fbb69e00640 1 -- 172.21.15.28:0/3594275268 wait complete. 2024-09-10T10:44:11.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578019070 con 0x7f95800b3af0 2024-09-10T10:44:11.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.776+0000 7fbb69e00640 1 Processor -- start 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95780085b0 con 0x7f95800b3af0 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578008980 con 0x7f95800b3af0 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f9587a00640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9580161c40 con 0x7f95800b3af0 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f9587a00640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9580162200 con 0x7f95800b3af0 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f7efb800640 1 Processor -- start 2024-09-10T10:44:11.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f7efb800640 1 -- start start 2024-09-10T10:44:11.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.778+0000 7fbb69e00640 1 -- start start 2024-09-10T10:44:11.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.777+0000 7f3a78c00640 1 -- start start 2024-09-10T10:44:11.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7f66ef800640 1 Processor -- start 2024-09-10T10:44:11.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.778+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9554001030 con 0x7f95800b3af0 2024-09-10T10:44:11.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 0x7fbb641f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb64141d60 0x7fbb640cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6415d2d0 0x7fbb640cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb641421d0 con 0x7fbb6415d2d0 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6413ff50 con 0x7fbb6406e370 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7fbb69e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6415d6b0 con 0x7fbb64141d60 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7fbb62a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 0x7fbb641f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7fbb62a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 0x7fbb641f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36060/0 (socket says 172.21.15.28:36060) 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7fbb62a00640 1 -- 172.21.15.28:0/1959274128 learned_addr learned my addr 172.21.15.28:0/1959274128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef414ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef40b3130 0x7f7ef41f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef40cfba0 0x7f7ef40a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415a730 con 0x7f7ef40cfba0 2024-09-10T10:44:11.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef415a5b0 con 0x7f7ef40b3130 2024-09-10T10:44:11.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.779+0000 7f66ef800640 1 -- start start 2024-09-10T10:44:11.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f7efb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef406e370 con 0x7f7ef406bcb0 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f3a78c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3a74161c80 0x7f3a7406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f3a78c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7406e870 con 0x7f3a74162060 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f3a78c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7406d9f0 con 0x7f3a740b6680 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f3a78c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7406dcd0 con 0x7f3a74161c80 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.780+0000 7f3a71e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3a74161c80 0x7f3a7406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 -- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb64141d60 msgr2=0x7fbb640cd8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb64141d60 0x7fbb640cd8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 -- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6415d2d0 msgr2=0x7fbb640cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb6415d2d0 0x7fbb640cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb641f9590 con 0x7fbb6406e370 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef414ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f7ef8e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef414ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44228/0 (socket says 172.21.15.28:44228) 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f7ef8e00640 1 -- 172.21.15.28:0/2197547915 learned_addr learned my addr 172.21.15.28:0/2197547915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef40b3130 0x7f7ef41f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb62a00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 0x7fbb641f8db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00ba50 tx=0x7fbb5c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f3a71400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3a74162060 0x7f3a740b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41160/0 (socket says 172.21.15.28:41160) 2024-09-10T10:44:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7f3a71400640 1 -- 172.21.15.28:0/3876160862 learned_addr learned my addr 172.21.15.28:0/3876160862 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c019070 con 0x7fbb6406e370 2024-09-10T10:44:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.781+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb5c00cc10 con 0x7fbb6406e370 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c014440 con 0x7fbb6406e370 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb641f5f70 con 0x7fbb6406e370 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ef800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66e80b6720 0x7f66e80d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ef800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e80d2170 con 0x7f66e80d0270 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ef800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e80d2410 con 0x7f66e815e710 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ef800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f66e80d2d20 con 0x7f66e80b6720 2024-09-10T10:44:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f7ef8e00640 1 -- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef40b3130 msgr2=0x7f7ef41f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f7ef8e00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef40b3130 0x7f7ef41f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f7ef8e00640 1 -- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef40cfba0 msgr2=0x7f7ef40a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f7ef8e00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef40cfba0 0x7f7ef40a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb641f9800 con 0x7fbb6406e370 2024-09-10T10:44:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f7ef8e00640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef414b320 con 0x7f7ef406bcb0 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ed800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66e80b6720 0x7f66e80d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ed800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66e80b6720 0x7f66e80d31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36580/0 (socket says 172.21.15.28:36580) 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f66ed800640 1 -- 172.21.15.28:0/1347419148 learned_addr learned my addr 172.21.15.28:0/1347419148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.782+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f957801c070 con 0x7f95800b3af0 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f7ef8e00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef414ade0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400d620 tx=0x7f7ee400dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f3a71e00640 1 -- 172.21.15.28:0/3876160862 >> v1:172.21.15.138:6789/0 conn(0x7f3a740b6680 legacy=0x7f3a740cf1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:44:11.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400b620 con 0x7f7ef406bcb0 2024-09-10T10:44:11.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb64141820 con 0x7fbb6406e370 2024-09-10T10:44:11.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f3a71e00640 1 -- 172.21.15.28:0/3876160862 >> v1:172.21.15.28:6789/0 conn(0x7f3a74162060 legacy=0x7f3a740b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f3a71e00640 1 -- 172.21.15.28:0/3876160862 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7406dfb0 con 0x7f3a74161c80 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f66ed800640 1 -- 172.21.15.28:0/1347419148 >> v1:172.21.15.138:6789/0 conn(0x7f66e815e710 legacy=0x7f66e80cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f66ed800640 1 -- 172.21.15.28:0/1347419148 >> v1:172.21.15.28:6789/0 conn(0x7f66e80d0270 legacy=0x7f66e80b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f66ed800640 1 -- 172.21.15.28:0/1347419148 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e80d5240 con 0x7f66e80b6720 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.783+0000 7f66e7400640 1 -- 172.21.15.28:0/1347419148 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908706310 0 0) 0x7f66e80d2410 con 0x7f66e815e710 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7f957cc00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956006bf00 0x7f956006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f957808bfe0 con 0x7f95800b3af0 2024-09-10T10:44:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7f957cc00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f9560075600 0x7f9560077a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9578011e50 con 0x7f9560075600 2024-09-10T10:44:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.784+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f95780561f0 con 0x7f95800b3af0 2024-09-10T10:44:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.785+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee4019070 con 0x7f7ef406bcb0 2024-09-10T10:44:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.785+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400b9c0 con 0x7f7ef406bcb0 2024-09-10T10:44:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.785+0000 7f66ed800640 1 --2- 172.21.15.28:0/1347419148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e80d31b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e50 tx=0x7f66e002a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5833a1f76acfd91d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.785+0000 7f957fe00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956006bf00 0x7f956006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.785+0000 7f3a71e00640 1 --2- 172.21.15.28:0/3876160862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7406e330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a640097a0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec2f572e7f8a3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f7efb800640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef40aa6f0 con 0x7f7ef406bcb0 2024-09-10T10:44:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f7efb800640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef40b1300 con 0x7f7ef406bcb0 2024-09-10T10:44:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f3a70a00640 1 -- 172.21.15.28:0/3876160862 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64037070 con 0x7f3a74161c80 2024-09-10T10:44:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f3a70a00640 1 -- 172.21.15.28:0/3876160862 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64032440 con 0x7f3a74161c80 2024-09-10T10:44:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f3a70a00640 1 -- 172.21.15.28:0/3876160862 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64032810 con 0x7f3a74161c80 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f66e7400640 1 -- 172.21.15.28:0/1347419148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0033610 con 0x7f66e80b6720 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f66e7400640 1 -- 172.21.15.28:0/1347419148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e002f070 con 0x7f66e80b6720 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f66e7400640 1 -- 172.21.15.28:0/1347419148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0033a10 con 0x7f66e80b6720 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f3a78c00640 1 -- 172.21.15.28:0/3876160862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 msgr2=0x7f3a7406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f3a78c00640 1 --2- 172.21.15.28:0/3876160862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7406e330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3a640097a0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f7efb800640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ef415a070 con 0x7f7ef406bcb0 2024-09-10T10:44:11.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f3a78c00640 1 -- 172.21.15.28:0/3876160862 shutdown_connections 2024-09-10T10:44:11.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f3a78c00640 1 --2- 172.21.15.28:0/3876160862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7406e330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f3a78c00640 1 -- 172.21.15.28:0/3876160862 >> 172.21.15.28:0/3876160862 conn(0x7f3a740d44f0 msgr2=0x7f3a740d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f0908400640 1 Processor -- start 2024-09-10T10:44:11.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.786+0000 7f0908400640 1 -- start start 2024-09-10T10:44:11.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f957fe00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956006bf00 0x7f956006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f956c034000 tx=0x7f956c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f3a78c00640 1 -- 172.21.15.28:0/3876160862 shutdown_connections 2024-09-10T10:44:11.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f66ef800640 1 -- 172.21.15.28:0/1347419148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 msgr2=0x7f66e80d31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.787+0000 7f66ef800640 1 --2- 172.21.15.28:0/1347419148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e80d31b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f66e0009e50 tx=0x7f66e002a7d0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f66ef800640 1 -- 172.21.15.28:0/1347419148 shutdown_connections 2024-09-10T10:44:11.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f66ef800640 1 --2- 172.21.15.28:0/1347419148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e80d31b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f66ef800640 1 -- 172.21.15.28:0/1347419148 >> 172.21.15.28:0/1347419148 conn(0x7f66e808bfc0 msgr2=0x7f66e80d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f9585a00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f9560075600 0x7f9560077a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.788+0000 7f3a78c00640 1 -- 172.21.15.28:0/3876160862 wait complete. 2024-09-10T10:44:11.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f66ef800640 1 -- 172.21.15.28:0/1347419148 shutdown_connections 2024-09-10T10:44:11.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0908400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09000720b0 0x7f09000724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0908400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 0x7f09000ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0908400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0900072a10 con 0x7f0900162720 2024-09-10T10:44:11.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0908400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f09000b4b50 con 0x7f09000720b0 2024-09-10T10:44:11.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0908400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f09000b4df0 con 0x7f09000ce340 2024-09-10T10:44:11.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f9585a00640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f9560075600 0x7f9560077a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0905a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0900162720 0x7f09000b3ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41190/0 (socket says 172.21.15.28:41190) 2024-09-10T10:44:11.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0905a00640 1 -- 172.21.15.28:0/2081229873 learned_addr learned my addr 172.21.15.28:0/2081229873 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7ff8c9000640 1 Processor -- start 2024-09-10T10:44:11.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0905000640 1 --2- 172.21.15.28:0/2081229873 >> v2:172.21.15.138:3300/0 conn(0x7f09000720b0 0x7f09000724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7f0906400640 1 --2- 172.21.15.28:0/2081229873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 0x7f09000ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.789+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb5c003970 con 0x7fbb6406e370 2024-09-10T10:44:11.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7ff8c9000640 1 -- start start 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f7518a00640 1 Processor -- start 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7fbb5b400640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb4406beb0 0x7fbb4406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f0906400640 1 -- 172.21.15.28:0/2081229873 >> v2:172.21.15.138:3300/0 conn(0x7f09000720b0 msgr2=0x7f09000724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f0906400640 1 --2- 172.21.15.28:0/2081229873 >> v2:172.21.15.138:3300/0 conn(0x7f09000720b0 0x7f09000724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f0906400640 1 -- 172.21.15.28:0/2081229873 >> v1:172.21.15.28:6789/0 conn(0x7f0900162720 legacy=0x7f09000b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f0906400640 1 -- 172.21.15.28:0/2081229873 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09000b50d0 con 0x7f09000ce340 2024-09-10T10:44:11.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.790+0000 7f7518a00640 1 -- start start 2024-09-10T10:44:11.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbb5c08b2b0 con 0x7fbb6406e370 2024-09-10T10:44:11.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7f66ef800640 1 -- 172.21.15.28:0/1347419148 wait complete. 2024-09-10T10:44:11.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7f3a78c00640 1 Processor -- start 2024-09-10T10:44:11.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9578011e50 con 0x7f9560075600 2024-09-10T10:44:11.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7f3a78c00640 1 -- start start 2024-09-10T10:44:11.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7fbb62000640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb4406beb0 0x7fbb4406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7fbb5b400640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7fbb44075630 0x7fbb44077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb44078140 con 0x7fbb44075630 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbb5c08b790 con 0x7fbb6406e370 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f0906400640 1 --2- 172.21.15.28:0/2081229873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 0x7f09000ce7d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009d90 tx=0x7f08f802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800f89ef78157afc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7fbb63400640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7fbb44075630 0x7fbb44077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f08ffe00640 1 -- 172.21.15.28:0/2081229873 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8007740 con 0x7f09000ce340 2024-09-10T10:44:11.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a740b6680 0x7f3a7409e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c9000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8c4162ad0 0x7ff8c40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c9000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c40b49a0 con 0x7ff8c406f6f0 2024-09-10T10:44:11.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c9000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8c40afa20 con 0x7ff8c40720b0 2024-09-10T10:44:11.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a74162060 0x7f3a740cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a740b6b30 con 0x7f3a74161c80 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c9000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8c40afce0 con 0x7ff8c4162ad0 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c1e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff8c40720b0 0x7ff8c4072490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52486/0 (socket says 172.21.15.28:52486) 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7ff8c1e00640 1 -- 172.21.15.28:0/3832382004 learned_addr learned my addr 172.21.15.28:0/3832382004 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182363086 0 0) 0x7ff8c40afa20 con 0x7ff8c40720b0 2024-09-10T10:44:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a740d2aa0 con 0x7f3a74162060 2024-09-10T10:44:11.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f3a78c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a740b6cb0 con 0x7f3a740b6680 2024-09-10T10:44:11.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8a8003520 con 0x7ff8c40720b0 2024-09-10T10:44:11.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.791+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ee400bb60 con 0x7f7ef406bcb0 2024-09-10T10:44:11.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7ef1600640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ed006be80 0x7f7ed006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7518a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75140d4ac0 0x7f75140d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7518a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f75140d2aa0 0x7f75140d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7518a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7514161d00 con 0x7f7514161920 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7518a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f75140cffd0 con 0x7f75140d2aa0 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.792+0000 7f7518a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75140d09a0 con 0x7f75140d4ac0 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f7512800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7514161920 0x7f75140d16e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41214/0 (socket says 172.21.15.28:41214) 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f7512800640 1 -- 172.21.15.28:0/566629133 learned_addr learned my addr 172.21.15.28:0/566629133 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:11.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f08ffe00640 1 -- 172.21.15.28:0/2081229873 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08f80078e0 con 0x7f09000ce340 2024-09-10T10:44:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7fbb62000640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb4406beb0 0x7fbb4406e370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb54034000 tx=0x7fbb5402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f66ef800640 1 Processor -- start 2024-09-10T10:44:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 955264155 0 0) 0x7ff8a8003520 con 0x7ff8c40720b0 2024-09-10T10:44:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 >> v2:172.21.15.138:3300/0 conn(0x7ff8c4162ad0 msgr2=0x7ff8c40cfca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 --2- 172.21.15.28:0/3832382004 >> v2:172.21.15.138:3300/0 conn(0x7ff8c4162ad0 0x7ff8c40cfca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 >> v1:172.21.15.28:6789/0 conn(0x7ff8c406f6f0 legacy=0x7ff8c40b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c40b34d0 con 0x7ff8c40720b0 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7ff8b40019d0 con 0x7ff8c40720b0 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 >> v1:172.21.15.138:6789/0 conn(0x7ff8c40720b0 legacy=0x7ff8c4072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a80039b0 0x7ff8a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7ff8c0a00640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a8004800 0x7ff8a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7513200640 1 --2- 172.21.15.28:0/566629133 >> v2:172.21.15.138:3300/0 conn(0x7f75140d2aa0 0x7f75140d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7511e00640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75140d4ac0 0x7f75140d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7fbb63400640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7fbb44075630 0x7fbb44077a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f3a71400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7513200640 1 -- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75140d4ac0 msgr2=0x7f75140d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7513200640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75140d4ac0 0x7f75140d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7513200640 1 -- 172.21.15.28:0/566629133 >> v1:172.21.15.28:6789/0 conn(0x7f7514161920 legacy=0x7f75140d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f7513200640 1 -- 172.21.15.28:0/566629133 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75140cf3c0 con 0x7f75140d2aa0 2024-09-10T10:44:11.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7ff8c0a00640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40afce0 con 0x7ff8a80039b0 2024-09-10T10:44:11.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.793+0000 7f08ffe00640 1 -- 172.21.15.28:0/2081229873 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8007cb0 con 0x7f09000ce340 2024-09-10T10:44:11.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a800c290 con 0x7ff8a80052c0 2024-09-10T10:44:11.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a800c490 con 0x7ff8a8004800 2024-09-10T10:44:11.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f66ef800640 1 -- start start 2024-09-10T10:44:11.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f0908400640 1 -- 172.21.15.28:0/2081229873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 msgr2=0x7f09000ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f3a71e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a740b6680 0x7f3a7409e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7f3a72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a74162060 0x7f3a740cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7f3a72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a74162060 0x7f3a740cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36104/0 (socket says 172.21.15.28:36104) 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7f3a72800640 1 -- 172.21.15.28:0/2075425166 learned_addr learned my addr 172.21.15.28:0/2075425166 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 -- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a8004800 msgr2=0x7ff8a8004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a8004800 0x7ff8a8004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 -- 172.21.15.28:0/3832382004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a80039b0 msgr2=0x7ff8a8003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:11.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a80039b0 0x7ff8a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.795+0000 7ff8c2800640 1 -- 172.21.15.28:0/3832382004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c40b34d0 con 0x7ff8a80052c0 2024-09-10T10:44:11.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.794+0000 7f0908400640 1 --2- 172.21.15.28:0/2081229873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 0x7f09000ce7d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009d90 tx=0x7f08f802a660 comp rx=0 tx=0).stop 2024-09-10T10:44:11.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e80d0270 0x7f66e80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71400640 1 -- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a740b6680 msgr2=0x7f3a7409e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71400640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a740b6680 0x7f3a7409e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71400640 1 -- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a74162060 msgr2=0x7f3a740cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71400640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a74162060 0x7f3a740cc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71400640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a740ccde0 con 0x7f3a74161c80 2024-09-10T10:44:11.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f3a71e00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a740b6680 0x7f3a7409e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7513200640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140d2f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f750c009dc0 tx=0x7f750c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca183dadd294fc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7ff8c2800640 1 -- 172.21.15.28:0/3832382004 reap_dead start 2024-09-10T10:44:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ed006be80 0x7f7ed006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7ee408b610 con 0x7f7ef406bcb0 2024-09-10T10:44:11.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7511400640 1 -- 172.21.15.28:0/566629133 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c007740 con 0x7f75140d2aa0 2024-09-10T10:44:11.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7511400640 1 -- 172.21.15.28:0/566629133 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c0078e0 con 0x7f75140d2aa0 2024-09-10T10:44:11.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f7511400640 1 -- 172.21.15.28:0/566629133 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c007cf0 con 0x7f75140d2aa0 2024-09-10T10:44:11.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb44078140 con 0x7fbb44075630 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f7ef1600640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f7ed00755f0 0x7f7ed0077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ee400be50 con 0x7f7ed00755f0 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ee408baf0 con 0x7f7ef406bcb0 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66e815e710 0x7f66e80b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66e806ce60 con 0x7f66e80b6720 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66e80d29b0 con 0x7f66e80d0270 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ef800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66e806cfe0 con 0x7f66e815e710 2024-09-10T10:44:11.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e8156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36610/0 (socket says 172.21.15.28:36610) 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ece00640 1 -- 172.21.15.28:0/4177041497 learned_addr learned my addr 172.21.15.28:0/4177041497 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.796+0000 7f66ed800640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66e815e710 0x7f66e80b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e80d0270 0x7f66e80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f3a71400640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7415caa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c440 tx=0x7f3a60001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60017070 con 0x7f3a74161c80 2024-09-10T10:44:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60005ce0 con 0x7f3a74161c80 2024-09-10T10:44:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60017070 con 0x7f3a74161c80 2024-09-10T10:44:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ed006be80 0x7f7ed006e340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7edc034000 tx=0x7f7edc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f7ef9800640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f7ed00755f0 0x7f7ed0077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 -- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66e815e710 msgr2=0x7f66e80b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66e815e710 0x7f66e80b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 -- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 msgr2=0x7f66e8156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e8156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.797+0000 7f66e7e00640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e80cd060 con 0x7f66e80d0270 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 -- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 msgr2=0x7f75140d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140d2f50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f750c009dc0 tx=0x7f750c02a6a0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f66ece00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66e80b6720 0x7f66e8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:11.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a740b8b90 con 0x7f3a74161c80 2024-09-10T10:44:11.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f66e7e00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e80d0270 0x7f66e80cc6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c390 tx=0x7f66d400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7415cfe0 con 0x7f3a74161c80 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7ff8c2800640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8a800b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc001750 tx=0x7ff8a8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900592e47bc9c2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc001370 con 0x7ff8a80052c0 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 -- 172.21.15.28:0/566629133 shutdown_connections 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140d2f50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 --2- 172.21.15.28:0/566629133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75140d4ac0 0x7f75140d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7518a00640 1 -- 172.21.15.28:0/566629133 >> 172.21.15.28:0/566629133 conn(0x7f751409d770 msgr2=0x7f751408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0052f0 con 0x7ff8a80052c0 2024-09-10T10:44:11.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7ff8c0a00640 1 -- 172.21.15.28:0/3832382004 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc02ac00 con 0x7ff8a80052c0 2024-09-10T10:44:11.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.798+0000 7f7ef9800640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f7ed00755f0 0x7f7ed0077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a7406e330 con 0x7f3a74161c80 2024-09-10T10:44:11.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f7518a00640 1 -- 172.21.15.28:0/566629133 shutdown_connections 2024-09-10T10:44:11.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f7518a00640 1 -- 172.21.15.28:0/566629133 wait complete. 2024-09-10T10:44:11.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4017070 con 0x7f66e80d0270 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d4003610 con 0x7f66e80d0270 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40039e0 con 0x7f66e80d0270 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f0908400640 1 -- 172.21.15.28:0/2081229873 shutdown_connections 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f0908400640 1 --2- 172.21.15.28:0/2081229873 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09000ce340 0x7f09000ce7d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f0908400640 1 --2- 172.21.15.28:0/2081229873 >> v2:172.21.15.138:3300/0 conn(0x7f09000720b0 0x7f09000724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f0908400640 1 -- 172.21.15.28:0/2081229873 >> 172.21.15.28:0/2081229873 conn(0x7f0900067b30 msgr2=0x7f090006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.799+0000 7f66ef800640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e80cd340 con 0x7f66e80d0270 2024-09-10T10:44:11.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7ff8c9000640 1 -- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 msgr2=0x7ff8a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7ff8c9000640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8a800b9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc001750 tx=0x7ff8a8005830 comp rx=0 tx=0).stop 2024-09-10T10:44:11.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ee400be50 con 0x7f7ed00755f0 2024-09-10T10:44:11.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7f66ef800640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e80cd900 con 0x7f66e80d0270 2024-09-10T10:44:11.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7f0908400640 1 -- 172.21.15.28:0/2081229873 shutdown_connections 2024-09-10T10:44:11.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.800+0000 7f0908400640 1 -- 172.21.15.28:0/2081229873 wait complete. 2024-09-10T10:44:11.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7ff8c9000640 1 -- 172.21.15.28:0/3832382004 shutdown_connections 2024-09-10T10:44:11.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7ff8c9000640 1 --2- 172.21.15.28:0/3832382004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8a800b9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7ff8c9000640 1 -- 172.21.15.28:0/3832382004 >> 172.21.15.28:0/3832382004 conn(0x7ff8c4067d70 msgr2=0x7ff8c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7f66ef800640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66e80d31b0 con 0x7f66e80d0270 2024-09-10T10:44:11.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7ff8c9000640 1 -- 172.21.15.28:0/3832382004 shutdown_connections 2024-09-10T10:44:11.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.802+0000 7f7518a00640 1 Processor -- start 2024-09-10T10:44:11.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.802+0000 7f0908400640 1 Processor -- start 2024-09-10T10:44:11.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.802+0000 7f0908400640 1 -- start start 2024-09-10T10:44:11.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66d401a070 con 0x7f66e80d0270 2024-09-10T10:44:11.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.801+0000 7ff8c9000640 1 -- 172.21.15.28:0/3832382004 wait complete. 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.802+0000 7f7518a00640 1 -- start start 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75140d4ac0 0x7f75140b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 0x7f75140b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f75140cfc60 con 0x7f75140d4ac0 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75140cf980 con 0x7f75140d2aa0 2024-09-10T10:44:11.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.803+0000 7f7518a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75140cf070 con 0x7f7514161920 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f7513200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 0x7f75140b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f7512800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f7513200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 0x7f75140b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44292/0 (socket says 172.21.15.28:44292) 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f7513200640 1 -- 172.21.15.28:0/3216909548 learned_addr learned my addr 172.21.15.28:0/3216909548 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f7511e00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75140d4ac0 0x7f75140b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f66e4c00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66c806beb0 0x7f66c806e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7ff8c9000640 1 Processor -- start 2024-09-10T10:44:11.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f66d408a3a0 con 0x7f66e80d0270 2024-09-10T10:44:11.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f7513200640 1 -- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 msgr2=0x7f75140c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7ff8c9000640 1 -- start start 2024-09-10T10:44:11.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f7513200640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f7513200640 1 -- 172.21.15.28:0/3216909548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75140d4ac0 msgr2=0x7f75140b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f7513200640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75140d4ac0 0x7f75140b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f66e4c00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f66c8075630 0x7f66c8077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f7513200640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75141f8530 con 0x7f7514161920 2024-09-10T10:44:11.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66d4015e50 con 0x7f66c8075630 2024-09-10T10:44:11.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f66d4091050 con 0x7f66e80d0270 2024-09-10T10:44:11.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.805+0000 7f66ece00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66c806beb0 0x7f66c806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f0908400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 0x7f090014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f0908400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09000ce340 0x7f090015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f0908400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0900162720 0x7f09001601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f66ed800640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f66c8075630 0x7f66c8077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f0905a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 0x7f090014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.804+0000 7f0905a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 0x7f090014b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36618/0 (socket says 172.21.15.28:36618) 2024-09-10T10:44:11.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f7512800640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75140d2aa0 0x7f75140c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:11.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f7513200640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 0x7f75140b49d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f750c009ee0 tx=0x7f750c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f66ece00640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66c806beb0 0x7f66c806e370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66dc007fd0 tx=0x7f66dc007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c03f070 con 0x7f7514161920 2024-09-10T10:44:11.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8c415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a6001a070 con 0x7f3a74161c80 2024-09-10T10:44:11.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c031bc0 con 0x7f7514161920 2024-09-10T10:44:11.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8c4157660 0x7ff8c4165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75140ccf70 con 0x7f7514161920 2024-09-10T10:44:11.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75140c9890 con 0x7f7514161920 2024-09-10T10:44:11.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7f66ed800640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f66c8075630 0x7f66c8077a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8c41564f0 0x7ff8c415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8c415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8c415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36144/0 (socket says 172.21.15.28:36144) 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c1e00640 1 -- 172.21.15.28:0/705217763 learned_addr learned my addr 172.21.15.28:0/705217763 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.807+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40b3d70 con 0x7ff8c41564f0 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40b3a90 con 0x7ff8a80052c0 2024-09-10T10:44:11.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66d4015e50 con 0x7f66c8075630 2024-09-10T10:44:11.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8c40cfca0 con 0x7ff8c4157660 2024-09-10T10:44:11.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7ff8c1400640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8c4157660 0x7ff8c4165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7ff8c2800640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8c41564f0 0x7ff8c415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.808+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c03f070 con 0x7f7514161920 2024-09-10T10:44:11.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75140d0460 con 0x7f7514161920 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7ff8c1e00640 1 -- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8c4157660 msgr2=0x7ff8c4165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7ff8c1e00640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8c4157660 0x7ff8c4165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7ff8c1e00640 1 -- 172.21.15.28:0/705217763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8c41564f0 msgr2=0x7ff8c415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f3a6a000640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a5406c030 0x7f3a5406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3a6008ab30 con 0x7f3a74161c80 2024-09-10T10:44:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f3a6a000640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f3a54075730 0x7f3a54077b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7ff8c1e00640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8c41564f0 0x7ff8c415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a60015e70 con 0x7f3a54075730 2024-09-10T10:44:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a6008afc0 con 0x7f3a74161c80 2024-09-10T10:44:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7ff8c1e00640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c41657a0 con 0x7ff8a80052c0 2024-09-10T10:44:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7ff8c2800640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8c41564f0 0x7ff8c415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f09000b45f0 con 0x7f09000720b0 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09000b5690 con 0x7f09000ce340 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0900162b00 con 0x7f0900162720 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0906400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0900162720 0x7f09001601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0905000640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09000ce340 0x7f090015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 -- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0900162720 msgr2=0x7f09001601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0900162720 0x7f09001601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 -- 172.21.15.28:0/368548082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 msgr2=0x7f090014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 0x7f090014b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090015f4e0 con 0x7f09000ce340 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.809+0000 7f0905000640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09000ce340 0x7f090015eb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00e850 tx=0x7f08ec00ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.806+0000 7f0905a00640 1 -- 172.21.15.28:0/368548082 learned_addr learned my addr 172.21.15.28:0/368548082 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f0905a00640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000720b0 0x7f090014b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec017070 con 0x7f09000ce340 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08ec00c940 con 0x7f09000ce340 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00cd10 con 0x7f09000ce340 2024-09-10T10:44:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f3a71e00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a5406c030 0x7f3a5406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f3a72800640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f3a54075730 0x7f3a54077b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7ff8c1e00640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8c415c7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400cac0 tx=0x7ff8b400cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090015f7c0 con 0x7f09000ce340 2024-09-10T10:44:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0900156100 con 0x7f09000ce340 2024-09-10T10:44:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.810+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f750c03b410 con 0x7f7514161920 2024-09-10T10:44:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4018070 con 0x7ff8a80052c0 2024-09-10T10:44:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b400a7c0 con 0x7ff8a80052c0 2024-09-10T10:44:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400ab90 con 0x7ff8a80052c0 2024-09-10T10:44:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c4165a40 con 0x7ff8a80052c0 2024-09-10T10:44:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09000cc790 con 0x7f09000ce340 2024-09-10T10:44:11.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.811+0000 7f3a72800640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f3a54075730 0x7f3a54077b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.812+0000 7f3a71e00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a5406c030 0x7f3a5406e4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a64034000 tx=0x7f3a6402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.812+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c41632c0 con 0x7ff8a80052c0 2024-09-10T10:44:11.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.812+0000 7f750aa00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f74f006c060 0x7f74f006e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.813+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8c4165d20 con 0x7ff8a80052c0 2024-09-10T10:44:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.813+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== osd.4 v2:172.21.15.138:6810/1083898337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a60015e70 con 0x7f3a54075730 2024-09-10T10:44:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.813+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f750c0ac4d0 con 0x7f7514161920 2024-09-10T10:44:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.813+0000 7f7512800640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f74f006c060 0x7f74f006e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.814+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8b401b070 con 0x7ff8a80052c0 2024-09-10T10:44:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.815+0000 7f750aa00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f74f0075750 0x7f74f0077b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.815+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f750c03bbe0 con 0x7f74f0075750 2024-09-10T10:44:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.815+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f750c0b3050 con 0x7f7514161920 2024-09-10T10:44:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.814+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f08ec01a070 con 0x7f09000ce340 2024-09-10T10:44:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7ff8ba000640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8a406be80 0x7ff8a406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7f7511e00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f74f0075750 0x7f74f0077b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff8b408b5d0 con 0x7ff8a80052c0 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7ff8ba000640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7ff8a40755f0 0x7ff8a4077a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8b4006e50 con 0x7ff8a40755f0 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8b408b980 con 0x7ff8a80052c0 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.815+0000 7f08fd600640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08e006bf00 0x7f08e006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.817+0000 7ff8c1400640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8a406be80 0x7ff8a406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.817+0000 7f7512800640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f74f006c060 0x7f74f006e520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7504007fd0 tx=0x7f7504007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.817+0000 7ff8c2800640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7ff8a40755f0 0x7ff8a4077a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.818+0000 7f7511e00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f74f0075750 0x7f74f0077b90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.815+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f08ec08b620 con 0x7f09000ce340 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7f08fd600640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f08e0075680 0x7f08e0077ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08ec001d30 con 0x7f08e0075680 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f08ec08ba10 con 0x7f09000ce340 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.816+0000 7f0905a00640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08e006bf00 0x7f08e006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.817+0000 7f0906400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f08e0075680 0x7f08e0077ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.817+0000 7f0905a00640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08e006bf00 0x7f08e006e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f4001690 tx=0x7f08f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.818+0000 7f0906400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f08e0075680 0x7f08e0077ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.818+0000 7ff8c1400640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8a406be80 0x7ff8a406e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cd960 tx=0x7ff8b0006f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.818+0000 7ff8c2800640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7ff8a40755f0 0x7ff8a4077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:11.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.819+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08ec001d30 con 0x7f08e0075680 2024-09-10T10:44:11.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.819+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f750c03bbe0 con 0x7f74f0075750 2024-09-10T10:44:11.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.819+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== osd.5 v2:172.21.15.138:6802/2049132176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8b4006e50 con 0x7ff8a40755f0 2024-09-10T10:44:11.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.832+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb640b49d0 con 0x7fbb44075630 2024-09-10T10:44:11.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.833+0000 7fbb5b400640 1 -- 172.21.15.28:0/1959274128 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb640b49d0 con 0x7fbb44075630 2024-09-10T10:44:11.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.833+0000 7f7efb800640 1 -- 172.21.15.28:0/2197547915 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ef40b1730 con 0x7f7ed00755f0 2024-09-10T10:44:11.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7f7ef1600640 1 -- 172.21.15.28:0/2197547915 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7ef40b1730 con 0x7f7ed00755f0 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7fbb44075630 msgr2=0x7fbb44077a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7fbb44075630 0x7fbb44077a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb4406beb0 msgr2=0x7fbb4406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb4406beb0 0x7fbb4406e370 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbb54034000 tx=0x7fbb5402e000 comp rx=0 tx=0).stop 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 msgr2=0x7fbb641f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb69e00640 1 --2- 172.21.15.28:0/1959274128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6406e370 0x7fbb641f8db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00ba50 tx=0x7fbb5c00bf10 comp rx=0 tx=0).stop 2024-09-10T10:44:11.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.834+0000 7fbb63400640 1 -- 172.21.15.28:0/1959274128 reap_dead start 2024-09-10T10:44:11.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 shutdown_connections 2024-09-10T10:44:11.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 >> 172.21.15.28:0/1959274128 conn(0x7fbb64067d70 msgr2=0x7fbb6406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f7ed00755f0 msgr2=0x7f7ed0077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 shutdown_connections 2024-09-10T10:44:11.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7fbb69e00640 1 -- 172.21.15.28:0/1959274128 wait complete. 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.835+0000 7f7eea000640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f7ed00755f0 0x7f7ed0077a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ed006be80 msgr2=0x7f7ed006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ed006be80 0x7f7ed006e340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7edc034000 tx=0x7f7edc02e000 comp rx=0 tx=0).stop 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 msgr2=0x7f7ef414ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 --2- 172.21.15.28:0/2197547915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ef406bcb0 0x7f7ef414ade0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400d620 tx=0x7f7ee400dae0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7ef9800640 1 -- 172.21.15.28:0/2197547915 reap_dead start 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 shutdown_connections 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.836+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 >> 172.21.15.28:0/2197547915 conn(0x7f7ef4067950 msgr2=0x7f7ef40ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.837+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 shutdown_connections 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.837+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9554002dd0 con 0x7f9560075600 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.837+0000 7f7eea000640 1 -- 172.21.15.28:0/2197547915 wait complete. 2024-09-10T10:44:11.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.838+0000 7f957cc00640 1 -- 172.21.15.28:0/2196109766 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9554002dd0 con 0x7f9560075600 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.838+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f9560075600 msgr2=0x7f9560077a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.839+0000 7f9576000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f9560075600 0x7f9560077a40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.839+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956006bf00 msgr2=0x7f956006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.839+0000 7f9576000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956006bf00 0x7f956006e3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f956c034000 tx=0x7f956c02e000 comp rx=0 tx=0).stop 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.840+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95800b3af0 msgr2=0x7f958015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.840+0000 7f9576000640 1 --2- 172.21.15.28:0/2196109766 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95800b3af0 0x7f958015d4e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f957800cfa0 tx=0x7f957800a850 comp rx=0 tx=0).stop 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.840+0000 7f9585a00640 1 -- 172.21.15.28:0/2196109766 reap_dead start 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.840+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 shutdown_connections 2024-09-10T10:44:11.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.840+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 >> 172.21.15.28:0/2196109766 conn(0x7f9580067950 msgr2=0x7f95800734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.842+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 shutdown_connections 2024-09-10T10:44:11.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.842+0000 7f9576000640 1 -- 172.21.15.28:0/2196109766 wait complete. 2024-09-10T10:44:11.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.842+0000 7f66ef800640 1 -- 172.21.15.28:0/4177041497 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f66e80cdc50 con 0x7f66c8075630 2024-09-10T10:44:11.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.843+0000 7f66e4c00640 1 -- 172.21.15.28:0/4177041497 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f66e80cdc50 con 0x7f66c8075630 2024-09-10T10:44:11.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.844+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f66c8075630 msgr2=0x7f66c8077a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.844+0000 7f66da000640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f66c8075630 0x7f66c8077a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.844+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66c806beb0 msgr2=0x7f66c806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66da000640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66c806beb0 0x7f66c806e370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f66dc007fd0 tx=0x7f66dc007d60 comp rx=0 tx=0).stop 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e80d0270 msgr2=0x7f66e80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66da000640 1 --2- 172.21.15.28:0/4177041497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e80d0270 0x7f66e80cc6d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66d400c390 tx=0x7f66d400c850 comp rx=0 tx=0).stop 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66ed800640 1 -- 172.21.15.28:0/4177041497 reap_dead start 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 shutdown_connections 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 >> 172.21.15.28:0/4177041497 conn(0x7f66e808bfc0 msgr2=0x7f66e80cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.845+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a740b4440 con 0x7f3a54075730 2024-09-10T10:44:11.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 shutdown_connections 2024-09-10T10:44:11.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8c40b4290 con 0x7ff8a40755f0 2024-09-10T10:44:11.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7f66da000640 1 -- 172.21.15.28:0/4177041497 wait complete. 2024-09-10T10:44:11.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0900156300 con 0x7f08e0075680 2024-09-10T10:44:11.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7f3a6a000640 1 -- 172.21.15.28:0/2075425166 <== osd.4 v2:172.21.15.138:6810/1083898337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3a740b4440 con 0x7f3a54075730 2024-09-10T10:44:11.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.846+0000 7f08fd600640 1 -- 172.21.15.28:0/368548082 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0900156300 con 0x7f08e0075680 2024-09-10T10:44:11.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f3a54075730 msgr2=0x7f3a54077b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f3a54075730 0x7f3a54077b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a5406c030 msgr2=0x7f3a5406e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a5406c030 0x7f3a5406e4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a64034000 tx=0x7f3a6402e000 comp rx=0 tx=0).stop 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 msgr2=0x7f3a7415caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f3a78c00640 1 --2- 172.21.15.28:0/2075425166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a74161c80 0x7f3a7415caa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a6000c440 tx=0x7f3a60001e70 comp rx=0 tx=0).stop 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f08e0075680 msgr2=0x7f08e0077ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.847+0000 7f0908400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f08e0075680 0x7f08e0077ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f3a72800640 1 -- 172.21.15.28:0/2075425166 reap_dead start 2024-09-10T10:44:11.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08e006bf00 msgr2=0x7f08e006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08e006bf00 0x7f08e006e3c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f08f4001690 tx=0x7f08f4009040 comp rx=0 tx=0).stop 2024-09-10T10:44:11.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09000ce340 msgr2=0x7f090015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 --2- 172.21.15.28:0/368548082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09000ce340 0x7f090015eb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00e850 tx=0x7f08ec00ed10 comp rx=0 tx=0).stop 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0906400640 1 -- 172.21.15.28:0/368548082 reap_dead start 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 shutdown_connections 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 >> 172.21.15.28:0/2075425166 conn(0x7f3a740d44f0 msgr2=0x7f3a740cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 shutdown_connections 2024-09-10T10:44:11.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 >> 172.21.15.28:0/368548082 conn(0x7f0900067b30 msgr2=0x7f09001ed7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 shutdown_connections 2024-09-10T10:44:11.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 shutdown_connections 2024-09-10T10:44:11.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.848+0000 7f0908400640 1 -- 172.21.15.28:0/368548082 wait complete. 2024-09-10T10:44:11.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.849+0000 7f3a78c00640 1 -- 172.21.15.28:0/2075425166 wait complete. 2024-09-10T10:44:11.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.849+0000 7ff8ba000640 1 -- 172.21.15.28:0/705217763 <== osd.5 v2:172.21.15.138:6802/2049132176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff8c40b4290 con 0x7ff8a40755f0 2024-09-10T10:44:11.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7ff8a40755f0 msgr2=0x7ff8a4077a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7ff8a40755f0 0x7ff8a4077a30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8a406be80 msgr2=0x7ff8a406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8a406be80 0x7ff8a406e340 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40cd960 tx=0x7ff8b0006f90 comp rx=0 tx=0).stop 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 msgr2=0x7ff8c415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 --2- 172.21.15.28:0/705217763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a80052c0 0x7ff8c415c7f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400cac0 tx=0x7ff8b400cf80 comp rx=0 tx=0).stop 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c2800640 1 -- 172.21.15.28:0/705217763 reap_dead start 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 shutdown_connections 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 >> 172.21.15.28:0/705217763 conn(0x7ff8c4067d70 msgr2=0x7ff8c40b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.851+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 shutdown_connections 2024-09-10T10:44:11.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.854+0000 7ff8c9000640 1 -- 172.21.15.28:0/705217763 wait complete. 2024-09-10T10:44:11.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.855+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75140ba330 con 0x7f74f0075750 2024-09-10T10:44:11.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.857+0000 7f750aa00640 1 -- 172.21.15.28:0/3216909548 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75140ba330 con 0x7f74f0075750 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.858+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f74f0075750 msgr2=0x7f74f0077b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.858+0000 7f7518a00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f74f0075750 0x7f74f0077b90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.859+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f74f006c060 msgr2=0x7f74f006e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.859+0000 7f7518a00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f74f006c060 0x7f74f006e520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7504007fd0 tx=0x7f7504007e80 comp rx=0 tx=0).stop 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.859+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 msgr2=0x7f75140b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.859+0000 7f7518a00640 1 --2- 172.21.15.28:0/3216909548 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7514161920 0x7f75140b49d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f750c009ee0 tx=0x7f750c004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:11.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.859+0000 7f7513200640 1 -- 172.21.15.28:0/3216909548 reap_dead start 2024-09-10T10:44:11.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.860+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 shutdown_connections 2024-09-10T10:44:11.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.860+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 >> 172.21.15.28:0/3216909548 conn(0x7f751409d770 msgr2=0x7f75140d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:11.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.860+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 shutdown_connections 2024-09-10T10:44:11.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:11.860+0000 7f7518a00640 1 -- 172.21.15.28:0/3216909548 wait complete. 2024-09-10T10:44:11.885 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:11.886 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T10:44:11.887 INFO:teuthology.orchestra.run.smithi028.stdout:64424509451 2024-09-10T10:44:11.887 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T10:44:11.888 INFO:teuthology.orchestra.run.smithi028.stdout:64424509449 2024-09-10T10:44:11.888 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T10:44:11.889 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:11.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T10:44:11.890 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:11.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T10:44:11.891 INFO:teuthology.orchestra.run.smithi028.stdout:64424509449 2024-09-10T10:44:11.891 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T10:44:11.892 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2024-09-10T10:44:11.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T10:44:11.893 INFO:teuthology.orchestra.run.smithi028.stdout:64424509451 2024-09-10T10:44:11.893 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T10:44:12.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.194+0000 7f1ebd400640 1 Processor -- start 2024-09-10T10:44:12.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.194+0000 7f1ebd400640 1 -- start start 2024-09-10T10:44:12.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.196+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb80d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.196+0000 7f1ebd400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80d45a0 con 0x7f1eb81624d0 2024-09-10T10:44:12.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.196+0000 7f1ebd400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80d4880 con 0x7f1eb80d2940 2024-09-10T10:44:12.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.196+0000 7f1ebd400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80d4e40 con 0x7f1eb81628b0 2024-09-10T10:44:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.197+0000 7f1eb7400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb80d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.197+0000 7f1ebca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1eb80d2940 0x7f1eb80d0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52500/0 (socket says 172.21.15.28:52500) 2024-09-10T10:44:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.197+0000 7f1ebca00640 1 -- 172.21.15.28:0/2501879890 learned_addr learned my addr 172.21.15.28:0/2501879890 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.198+0000 7f1eb7400640 1 -- 172.21.15.28:0/2501879890 >> v1:172.21.15.138:6789/0 conn(0x7f1eb80d2940 legacy=0x7f1eb80d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.198+0000 7f1eb7400640 1 -- 172.21.15.28:0/2501879890 >> v1:172.21.15.28:6789/0 conn(0x7f1eb81624d0 legacy=0x7f1eb8163800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.198+0000 7f1eb7400640 1 -- 172.21.15.28:0/2501879890 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80d4b60 con 0x7f1eb81628b0 2024-09-10T10:44:12.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.200+0000 7f1eb7400640 1 --2- 172.21.15.28:0/2501879890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb80d3ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097d0 tx=0x7f1ea802a120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b602eabe9afa1e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.201+0000 7f1eb6a00640 1 -- 172.21.15.28:0/2501879890 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037070 con 0x7f1eb81628b0 2024-09-10T10:44:12.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.202+0000 7f1eb6a00640 1 -- 172.21.15.28:0/2501879890 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea802ae30 con 0x7f1eb81628b0 2024-09-10T10:44:12.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.203+0000 7f1eb6a00640 1 -- 172.21.15.28:0/2501879890 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8032640 con 0x7f1eb81628b0 2024-09-10T10:44:12.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.204+0000 7f1ebd400640 1 -- 172.21.15.28:0/2501879890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 msgr2=0x7f1eb80d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.204+0000 7f1ebd400640 1 --2- 172.21.15.28:0/2501879890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb80d3ee0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097d0 tx=0x7f1ea802a120 comp rx=0 tx=0).stop 2024-09-10T10:44:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.204+0000 7f1ebd400640 1 -- 172.21.15.28:0/2501879890 shutdown_connections 2024-09-10T10:44:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.204+0000 7f1ebd400640 1 --2- 172.21.15.28:0/2501879890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb80d3ee0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.204+0000 7f1ebd400640 1 -- 172.21.15.28:0/2501879890 >> 172.21.15.28:0/2501879890 conn(0x7f1eb809d720 msgr2=0x7f1eb808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.205+0000 7f97d5a00640 1 Processor -- start 2024-09-10T10:44:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.205+0000 7f1ebd400640 1 -- 172.21.15.28:0/2501879890 shutdown_connections 2024-09-10T10:44:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.205+0000 7f97d5a00640 1 -- start start 2024-09-10T10:44:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.205+0000 7f1ebd400640 1 -- 172.21.15.28:0/2501879890 wait complete. 2024-09-10T10:44:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f97d5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c80535b0 0x7f97c8058340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f97d5a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f97c803ce20 0x7f97c8056d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f97d5a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c8006930 con 0x7f97c80e72f0 2024-09-10T10:44:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f97d5a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f97c8006c10 con 0x7f97c803ce20 2024-09-10T10:44:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f97d5a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97c8006ef0 con 0x7f97c80535b0 2024-09-10T10:44:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.206+0000 7f2f6be00640 1 Processor -- start 2024-09-10T10:44:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.206+0000 7f2f6be00640 1 -- start start 2024-09-10T10:44:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.207+0000 7f1ebd400640 1 Processor -- start 2024-09-10T10:44:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f97cf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f97c803ce20 0x7f97c8056d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f97cf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f97c803ce20 0x7f97c8056d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36630/0 (socket says 172.21.15.28:36630) 2024-09-10T10:44:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f97cf400640 1 -- 172.21.15.28:0/1012316706 learned_addr learned my addr 172.21.15.28:0/1012316706 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f2f6be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f6c15c9f0 0x7f2f6c06dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f2f6be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0b3650 0x7f2f6c163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f97cf400640 1 -- 172.21.15.28:0/1012316706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c80535b0 msgr2=0x7f97c8058340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:44:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f97cf400640 1 --2- 172.21.15.28:0/1012316706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c80535b0 0x7f97c8058340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f97cf400640 1 -- 172.21.15.28:0/1012316706 >> v1:172.21.15.28:6789/0 conn(0x7f97c80e72f0 legacy=0x7f97c8053990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7fbc4ae00640 1 Processor -- start 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f2f6be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c06e570 con 0x7f2f6c162cf0 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c06f610 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c06e850 con 0x7f2f6c0b3650 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0b3650 0x7f2f6c163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0b3650 0x7f2f6c163dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36154/0 (socket says 172.21.15.28:36154) 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6b400640 1 -- 172.21.15.28:0/532818513 learned_addr learned my addr 172.21.15.28:0/532818513 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6b400640 1 -- 172.21.15.28:0/532818513 >> v2:172.21.15.28:3300/0 conn(0x7f2f6c15c9f0 msgr2=0x7f2f6c06dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.209+0000 7f2f6b400640 1 --2- 172.21.15.28:0/532818513 >> v2:172.21.15.28:3300/0 conn(0x7f2f6c15c9f0 0x7f2f6c06dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f2f6b400640 1 -- 172.21.15.28:0/532818513 >> v1:172.21.15.28:6789/0 conn(0x7f2f6c162cf0 legacy=0x7f2f6c0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f2f6b400640 1 -- 172.21.15.28:0/532818513 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c06eb30 con 0x7f2f6c0b3650 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7fbc4ae00640 1 -- start start 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.208+0000 7f1ebd400640 1 -- start start 2024-09-10T10:44:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb80d2940 0x7f1eb81554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 0x7f1eb8165100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb81fb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d1640 con 0x7f1eb81624d0 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80d14c0 con 0x7f1eb80d2940 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.210+0000 7f1ebd400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80d2d70 con 0x7f1eb81628b0 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f97cf400640 1 -- 172.21.15.28:0/1012316706 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c80071d0 con 0x7f97c803ce20 2024-09-10T10:44:12.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f1eb7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 0x7f1eb8165100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f2f6b400640 1 --2- 172.21.15.28:0/532818513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b3650 0x7f2f6c163dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cfc90 tx=0x7f2f6402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5779849278f3b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f1ebca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb81fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f1ebca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb81fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44336/0 (socket says 172.21.15.28:44336) 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.211+0000 7f1ebca00640 1 -- 172.21.15.28:0/1294913644 learned_addr learned my addr 172.21.15.28:0/1294913644 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7e00640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb80d2940 0x7f1eb81554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7400640 1 -- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 msgr2=0x7f1eb81fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb81fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7400640 1 -- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb80d2940 msgr2=0x7f1eb81554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb80d2940 0x7f1eb81554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f1eb7400640 1 -- 172.21.15.28:0/1294913644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb8152740 con 0x7f1eb81624d0 2024-09-10T10:44:12.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.212+0000 7f97cf400640 1 --2- 172.21.15.28:0/1012316706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c803ce20 0x7f97c8056d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009870 tx=0x7f97bc02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa3daa90c46274 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 0x7fbc4415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 0x7fbc44163f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc49a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbc4415abe0 0x7fbc440b3b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41272/0 (socket says 172.21.15.28:41272) 2024-09-10T10:44:12.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7f2f69600640 1 -- 172.21.15.28:0/532818513 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64036070 con 0x7f2f6c0b3650 2024-09-10T10:44:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7f2f69600640 1 -- 172.21.15.28:0/532818513 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f64031600 con 0x7f2f6c0b3650 2024-09-10T10:44:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7f2f69600640 1 -- 172.21.15.28:0/532818513 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f640319d0 con 0x7f2f6c0b3650 2024-09-10T10:44:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc49a00640 1 -- 172.21.15.28:0/1506386259 learned_addr learned my addr 172.21.15.28:0/1506386259 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc44072620 con 0x7fbc4415abe0 2024-09-10T10:44:12.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbc440b4240 con 0x7fbc44073b10 2024-09-10T10:44:12.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4406ebf0 con 0x7fbc4415b500 2024-09-10T10:44:12.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.213+0000 7fbc4a400640 1 --2- 172.21.15.28:0/1506386259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 0x7fbc44163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f1eb7400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 0x7f1eb8165100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8031000 tx=0x7f1ea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f2f6be00640 1 -- 172.21.15.28:0/532818513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b3650 msgr2=0x7f2f6c163dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7fbc49000640 1 --2- 172.21.15.28:0/1506386259 >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 0x7fbc4415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f97cd600640 1 -- 172.21.15.28:0/1012316706 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc037070 con 0x7f97c803ce20 2024-09-10T10:44:12.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f97cd600640 1 -- 172.21.15.28:0/1012316706 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc032440 con 0x7f97c803ce20 2024-09-10T10:44:12.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f97cd600640 1 -- 172.21.15.28:0/1012316706 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0327f0 con 0x7f97c803ce20 2024-09-10T10:44:12.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7fbc4a400640 1 -- 172.21.15.28:0/1506386259 >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 msgr2=0x7fbc4415afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8041070 con 0x7f1eb81624d0 2024-09-10T10:44:12.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f2f6be00640 1 --2- 172.21.15.28:0/532818513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b3650 0x7f2f6c163dd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0cfc90 tx=0x7f2f6402a6f0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea803c670 con 0x7f1eb81624d0 2024-09-10T10:44:12.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8037040 con 0x7f1eb81624d0 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7fbc4a400640 1 --2- 172.21.15.28:0/1506386259 >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 0x7fbc4415afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7fbc4a400640 1 -- 172.21.15.28:0/1506386259 >> v1:172.21.15.28:6789/0 conn(0x7fbc4415abe0 legacy=0x7fbc440b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.214+0000 7fbc4a400640 1 -- 172.21.15.28:0/1506386259 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4406eed0 con 0x7fbc4415b500 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 -- 172.21.15.28:0/532818513 shutdown_connections 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 --2- 172.21.15.28:0/532818513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b3650 0x7f2f6c163dd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 --2- 172.21.15.28:0/532818513 >> v2:172.21.15.28:3300/0 conn(0x7f2f6c15c9f0 0x7f2f6c06dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 -- 172.21.15.28:0/532818513 >> 172.21.15.28:0/532818513 conn(0x7f2f6c06bdf0 msgr2=0x7f2f6c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 -- 172.21.15.28:0/532818513 shutdown_connections 2024-09-10T10:44:12.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7fbc49000640 1 --2- 172.21.15.28:0/1506386259 >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 0x7fbc4415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:12.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f97d5a00640 1 -- 172.21.15.28:0/1012316706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c803ce20 msgr2=0x7f97c8056d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb81559f0 con 0x7f1eb81624d0 2024-09-10T10:44:12.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1012316706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c803ce20 0x7f97c8056d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009870 tx=0x7f97bc02a290 comp rx=0 tx=0).stop 2024-09-10T10:44:12.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2d2e800640 1 Processor -- start 2024-09-10T10:44:12.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb80ca1d0 con 0x7f1eb81624d0 2024-09-10T10:44:12.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f97d5a00640 1 -- 172.21.15.28:0/1012316706 shutdown_connections 2024-09-10T10:44:12.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f2d2e800640 1 -- start start 2024-09-10T10:44:12.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1012316706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c803ce20 0x7f97c8056d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1012316706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c80535b0 0x7f97c8058340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f97d5a00640 1 -- 172.21.15.28:0/1012316706 >> 172.21.15.28:0/1012316706 conn(0x7f97c803a520 msgr2=0x7f97c803a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.216+0000 7f97d5a00640 1 -- 172.21.15.28:0/1012316706 shutdown_connections 2024-09-10T10:44:12.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.215+0000 7f2f6be00640 1 -- 172.21.15.28:0/532818513 wait complete. 2024-09-10T10:44:12.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2f6be00640 1 Processor -- start 2024-09-10T10:44:12.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb80d3ee0 con 0x7f1eb81624d0 2024-09-10T10:44:12.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.217+0000 7fbc4a400640 1 --2- 172.21.15.28:0/1506386259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 0x7fbc44163f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc300097a0 tx=0x7fbc3002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f4b299f021bcdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7fbc43e00640 1 -- 172.21.15.28:0/1506386259 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30037070 con 0x7fbc4415b500 2024-09-10T10:44:12.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7fbc43e00640 1 -- 172.21.15.28:0/1506386259 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3002ad10 con 0x7fbc4415b500 2024-09-10T10:44:12.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7fbc43e00640 1 -- 172.21.15.28:0/1506386259 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc30032660 con 0x7fbc4415b500 2024-09-10T10:44:12.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2d2e800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 0x7f2d280b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2d2e800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2806d9f0 con 0x7f2d280b64b0 2024-09-10T10:44:12.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2d2e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2806dcd0 con 0x7f2d28161c80 2024-09-10T10:44:12.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2d2e800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2806dfb0 con 0x7f2d28162060 2024-09-10T10:44:12.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.218+0000 7f2f6be00640 1 -- start start 2024-09-10T10:44:12.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.217+0000 7f97d5a00640 1 -- 172.21.15.28:0/1012316706 wait complete. 2024-09-10T10:44:12.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7f2d27400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f2d28161c80 0x7f2d2806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52502/0 (socket says 172.21.15.28:52502) 2024-09-10T10:44:12.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 msgr2=0x7fbc44163f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7f2d27400640 1 -- 172.21.15.28:0/1939102024 learned_addr learned my addr 172.21.15.28:0/1939102024 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/1506386259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 0x7fbc44163f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc300097a0 tx=0x7fbc3002a080 comp rx=0 tx=0).stop 2024-09-10T10:44:12.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 shutdown_connections 2024-09-10T10:44:12.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/1506386259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415b500 0x7fbc44163f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/1506386259 >> v2:172.21.15.28:3300/0 conn(0x7fbc44073b10 0x7fbc4415afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7f2d26a00640 1 --2- 172.21.15.28:0/1939102024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 0x7f2d280b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 >> 172.21.15.28:0/1506386259 conn(0x7fbc4406c240 msgr2=0x7fbc440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 shutdown_connections 2024-09-10T10:44:12.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.219+0000 7fbc4ae00640 1 -- 172.21.15.28:0/1506386259 wait complete. 2024-09-10T10:44:12.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.220+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ea8044070 con 0x7f1eb81624d0 2024-09-10T10:44:12.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.220+0000 7f97d5a00640 1 Processor -- start 2024-09-10T10:44:12.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.220+0000 7f2d26000640 1 -- 172.21.15.28:0/1939102024 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041291342 0 0) 0x7f2d2806dcd0 con 0x7f2d28161c80 2024-09-10T10:44:12.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2d26000640 1 -- 172.21.15.28:0/1939102024 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d0c003520 con 0x7f2d28161c80 2024-09-10T10:44:12.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f97d5a00640 1 -- start start 2024-09-10T10:44:12.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 0x7f2f6c15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 0x7f2f6c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.222+0000 7f2f6be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f6c162cf0 0x7f2f6c0b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.222+0000 7f2f6be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c06e1b0 con 0x7f2f6c162cf0 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.222+0000 7f2f6be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c06f0f0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.222+0000 7fbc4ae00640 1 Processor -- start 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2d26a00640 1 -- 172.21.15.28:0/1939102024 >> v1:172.21.15.138:6789/0 conn(0x7f2d28161c80 legacy=0x7f2d2806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2d26a00640 1 -- 172.21.15.28:0/1939102024 >> v1:172.21.15.28:6789/0 conn(0x7f2d280b64b0 legacy=0x7f2d280cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.221+0000 7f2d26a00640 1 -- 172.21.15.28:0/1939102024 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280d2890 con 0x7f2d28162060 2024-09-10T10:44:12.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.222+0000 7f2f6be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c06e330 con 0x7f2f6c0b3650 2024-09-10T10:44:12.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 0x7f2f6c15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2f6aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 0x7f2f6c15bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44364/0 (socket says 172.21.15.28:44364) 2024-09-10T10:44:12.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2f6aa00640 1 -- 172.21.15.28:0/2464313730 learned_addr learned my addr 172.21.15.28:0/2464313730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7fbc4ae00640 1 -- start start 2024-09-10T10:44:12.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2f6a000640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 0x7f2f6c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2d26a00640 1 --2- 172.21.15.28:0/1939102024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 0x7f2d280b5f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009dc0 tx=0x7f2d1802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17c7aeea0e3aa7af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f97d5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 0x7f97c8052430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.223+0000 7f2f6b400640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f6c162cf0 0x7f2f6c0b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f1eafe00640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 0x7f1e9406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1ea80aba30 con 0x7f1eb81624d0 2024-09-10T10:44:12.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f97d5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2f6a000640 1 -- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 msgr2=0x7f2f6c15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2f6a000640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 0x7f2f6c15bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2f6a000640 1 -- 172.21.15.28:0/2464313730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f6c162cf0 msgr2=0x7f2f6c0b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2f6a000640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f6c162cf0 0x7f2f6c0b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2f6a000640 1 -- 172.21.15.28:0/2464313730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c09b940 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2d26000640 1 -- 172.21.15.28:0/1939102024 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007740 con 0x7f2d28162060 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2d26000640 1 -- 172.21.15.28:0/1939102024 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d180078e0 con 0x7f2d28162060 2024-09-10T10:44:12.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f2d26000640 1 -- 172.21.15.28:0/1939102024 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18007cf0 con 0x7f2d28162060 2024-09-10T10:44:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f89f1a00640 1 Processor -- start 2024-09-10T10:44:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.224+0000 7f97d5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c80e72f0 0x7f97c80e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f1eb7e00640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 0x7f1e9406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36182/0 (socket says 172.21.15.28:36182) 2024-09-10T10:44:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97ce000640 1 -- 172.21.15.28:0/1675357088 learned_addr learned my addr 172.21.15.28:0/1675357088 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 -- 172.21.15.28:0/1939102024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 msgr2=0x7f2d280b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 --2- 172.21.15.28:0/1939102024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 0x7f2d280b5f70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009dc0 tx=0x7f2d1802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f97c80573d0 con 0x7f97c80e72f0 2024-09-10T10:44:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2f6a000640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 0x7f2f6c15b710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c410 tx=0x7f2f5c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 -- 172.21.15.28:0/1939102024 shutdown_connections 2024-09-10T10:44:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 --2- 172.21.15.28:0/1939102024 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28162060 0x7f2d280b5f70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 -- 172.21.15.28:0/1939102024 >> 172.21.15.28:0/1939102024 conn(0x7f2d280d44f0 msgr2=0x7f2d280d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8057250 con 0x7f97c80535b0 2024-09-10T10:44:12.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 -- 172.21.15.28:0/1939102024 shutdown_connections 2024-09-10T10:44:12.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7fbc4ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f25aea00640 1 Processor -- start 2024-09-10T10:44:12.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7fbc4ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 0x7fbc44072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2d2e800640 1 -- 172.21.15.28:0/1939102024 wait complete. 2024-09-10T10:44:12.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c017070 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c005230 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0045e0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97c8057550 con 0x7f97c803ce20 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97ce000640 1 -- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 msgr2=0x7f97c8052430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.225+0000 7f97cea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 0x7f97c8052430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97ce000640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 0x7f97c8052430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97ce000640 1 -- 172.21.15.28:0/1675357088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c80e72f0 msgr2=0x7f97c80e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97ce000640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c80e72f0 0x7f97c80e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97ce000640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8052970 con 0x7f97c80535b0 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f97cea00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 0x7f97c8052430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7fbc49a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7fbc49000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 0x7fbc44072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc49a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36674/0 (socket says 172.21.15.28:36674) 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc4ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc4415b500 0x7fbc44072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc4ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4406fdc0 con 0x7fbc44073b10 2024-09-10T10:44:12.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc4ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4406f490 con 0x7fbc4415b500 2024-09-10T10:44:12.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc4ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc4406ff40 con 0x7fbc4415abe0 2024-09-10T10:44:12.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97ce000640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f97d002dd50 tx=0x7f97d0074aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c09bc20 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c09c1e0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7fbc49000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 0x7fbc44072a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44376/0 (socket says 172.21.15.28:44376) 2024-09-10T10:44:12.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7fbc49000640 1 -- 172.21.15.28:0/3921864454 learned_addr learned my addr 172.21.15.28:0/3921864454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0074bd0 con 0x7f97c80535b0 2024-09-10T10:44:12.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.226+0000 7f89f1a00640 1 -- start start 2024-09-10T10:44:12.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d0069620 con 0x7f97c80535b0 2024-09-10T10:44:12.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0072870 con 0x7f97c80535b0 2024-09-10T10:44:12.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c800ce20 con 0x7f97c80535b0 2024-09-10T10:44:12.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c800d3e0 con 0x7f97c80535b0 2024-09-10T10:44:12.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7fbc49a00640 1 -- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 msgr2=0x7fbc44072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7fbc49a00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 0x7fbc44072a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7fbc49a00640 1 -- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc4415b500 msgr2=0x7fbc44072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7fbc49a00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc4415b500 0x7fbc44072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7fbc49a00640 1 -- 172.21.15.28:0/3921864454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc44204bc0 con 0x7fbc44073b10 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7f1eb7e00640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 0x7f1e9406e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0004ea0 tx=0x7f1eb0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7f89f1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89ec0d2aa0 0x7f89ec0d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.227+0000 7f25aea00640 1 -- start start 2024-09-10T10:44:12.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c15cef0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7f2d2e800640 1 Processor -- start 2024-09-10T10:44:12.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.228+0000 7f2d2e800640 1 -- start start 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d280b64b0 0x7f2d28155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 0x7f2d280b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 0x7f2d280ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89f1a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec161d00 con 0x7f89ec161920 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89f1a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89ec0cf3c0 con 0x7f89ec0d4ac0 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89f1a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f89ec0d3450 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89eb400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f89ec0d4ac0 0x7f89ec0d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52512/0 (socket says 172.21.15.28:52512) 2024-09-10T10:44:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89eb400640 1 -- 172.21.15.28:0/2110426923 learned_addr learned my addr 172.21.15.28:0/2110426923 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f89f1000640 1 --2- 172.21.15.28:0/2110426923 >> v2:172.21.15.138:3300/0 conn(0x7f89ec0d2aa0 0x7f89ec0d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280b6cf0 con 0x7f2d280b64b0 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d280b6b70 con 0x7f2d28162060 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d2e800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d280cf8f0 con 0x7f2d28161c80 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d27400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d280b64b0 0x7f2d28155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 0x7f2d280ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c804caf0 con 0x7f97c80535b0 2024-09-10T10:44:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 0x7f2d280ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36204/0 (socket says 172.21.15.28:36204) 2024-09-10T10:44:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d27e00640 1 -- 172.21.15.28:0/171646488 learned_addr learned my addr 172.21.15.28:0/171646488 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f2d26a00640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 0x7f2d280b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.229+0000 7fbc49a00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00c8c0 tx=0x7fbc3c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25aea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f25a80b6680 0x7f25a8169830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25a7e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f25a8161c80 0x7f25a806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52520/0 (socket says 172.21.15.28:52520) 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25a7e00640 1 -- 172.21.15.28:0/2730051713 learned_addr learned my addr 172.21.15.28:0/2730051713 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25aea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a806d9f0 con 0x7f25a8162060 2024-09-10T10:44:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a806dcd0 con 0x7f25a8161c80 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f25a806dfb0 con 0x7f25a80b6680 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f25a6a00640 1 -- 172.21.15.28:0/2730051713 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421131608 0 0) 0x7f25a806dcd0 con 0x7f25a8161c80 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f25a6a00640 1 -- 172.21.15.28:0/2730051713 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f258c003520 con 0x7f25a8161c80 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f89f1000640 1 -- 172.21.15.28:0/2110426923 >> v1:172.21.15.138:6789/0 conn(0x7f89ec0d4ac0 legacy=0x7f89ec0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f89f1000640 1 -- 172.21.15.28:0/2110426923 >> v1:172.21.15.28:6789/0 conn(0x7f89ec161920 legacy=0x7f89ec0d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.230+0000 7f89f1000640 1 -- 172.21.15.28:0/2110426923 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec0cf6a0 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f5c01a070 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2d26a00640 1 -- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 msgr2=0x7f2d280ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f25aca00640 1 --2- 172.21.15.28:0/2730051713 >> v2:172.21.15.28:3300/0 conn(0x7f25a80b6680 0x7f25a8169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2d26a00640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 0x7f2d280ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2d26a00640 1 -- 172.21.15.28:0/171646488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d280b64b0 msgr2=0x7f2d28155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2d26a00640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d280b64b0 0x7f2d28155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f89f1000640 1 --2- 172.21.15.28:0/2110426923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec0d2f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0097a0 tx=0x7f89dc02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b99005d45a7dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.231+0000 7f2d26a00640 1 -- 172.21.15.28:0/171646488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d280b98d0 con 0x7f2d28161c80 2024-09-10T10:44:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97d0072a10 con 0x7f97c80535b0 2024-09-10T10:44:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f2d26a00640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 0x7f2d280b7d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d18031000 tx=0x7f2d18004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1803f070 con 0x7f2d28161c80 2024-09-10T10:44:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d18031bc0 con 0x7f2d28161c80 2024-09-10T10:44:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18044460 con 0x7f2d28161c80 2024-09-10T10:44:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.232+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d280b8570 con 0x7f2d28161c80 2024-09-10T10:44:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1eb81559f0 con 0x7f1eb81624d0 2024-09-10T10:44:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f2f62a00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 0x7f2f4c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2f5c08a420 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f78cce00640 1 Processor -- start 2024-09-10T10:44:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f25aca00640 1 -- 172.21.15.28:0/2730051713 >> v1:172.21.15.138:6789/0 conn(0x7f25a8161c80 legacy=0x7f25a806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f78cce00640 1 -- start start 2024-09-10T10:44:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f25aca00640 1 -- 172.21.15.28:0/2730051713 >> v1:172.21.15.28:6789/0 conn(0x7f25a8162060 legacy=0x7f25a80b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f25aca00640 1 -- 172.21.15.28:0/2730051713 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a80d01f0 con 0x7f25a80b6680 2024-09-10T10:44:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d280b8b10 con 0x7f2d28161c80 2024-09-10T10:44:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f97c6a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 0x7f97b006e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.233+0000 7f2f6aa00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 0x7f2f4c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f97d0107560 con 0x7f97c80535b0 2024-09-10T10:44:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7f97cea00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 0x7f97b006e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c018070 con 0x7fbc44073b10 2024-09-10T10:44:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc3c00a640 con 0x7fbc44073b10 2024-09-10T10:44:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c00aa30 con 0x7fbc44073b10 2024-09-10T10:44:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2806dcd0 con 0x7f2d28161c80 2024-09-10T10:44:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc44200890 con 0x7fbc44073b10 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.234+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc44200e50 con 0x7fbc44073b10 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f78cce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f78c80720b0 0x7f78c80cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f78cce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c80b49a0 con 0x7f78c806f6f0 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f78cce00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c80afa20 con 0x7f78c8162ad0 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f78cce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f78c80afce0 con 0x7f78c80720b0 2024-09-10T10:44:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f25aca00640 1 --2- 172.21.15.28:0/2730051713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a80b6680 0x7f25a8169830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009d90 tx=0x7f25a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba9ea9dd22f067a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f78c5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f78c8162ad0 0x7f78c8072490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52532/0 (socket says 172.21.15.28:52532) 2024-09-10T10:44:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f78c5400640 1 -- 172.21.15.28:0/3051849436 learned_addr learned my addr 172.21.15.28:0/3051849436 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d18044600 con 0x7f2d28161c80 2024-09-10T10:44:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f78c5e00640 1 --2- 172.21.15.28:0/3051849436 >> v2:172.21.15.138:3300/0 conn(0x7f78c80720b0 0x7f78c80cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f25a6a00640 1 -- 172.21.15.28:0/2730051713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007740 con 0x7f25a80b6680 2024-09-10T10:44:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc4406e680 con 0x7fbc44073b10 2024-09-10T10:44:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.235+0000 7f2f6aa00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 0x7f2f4c06e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c1f55c0 tx=0x7f2f580091c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.236+0000 7f25a6a00640 1 -- 172.21.15.28:0/2730051713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a00078e0 con 0x7f25a80b6680 2024-09-10T10:44:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f78c5e00640 1 -- 172.21.15.28:0/3051849436 >> v1:172.21.15.138:6789/0 conn(0x7f78c8162ad0 legacy=0x7f78c8072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f78c5e00640 1 -- 172.21.15.28:0/3051849436 >> v1:172.21.15.28:6789/0 conn(0x7f78c806f6f0 legacy=0x7f78c80b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f78c5e00640 1 -- 172.21.15.28:0/3051849436 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c80b34d0 con 0x7f78c80720b0 2024-09-10T10:44:12.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f25a6a00640 1 -- 172.21.15.28:0/2730051713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007c90 con 0x7f25a80b6680 2024-09-10T10:44:12.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f97cea00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 0x7f97b006e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97c000a8b0 tx=0x7f97c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97d0091330 con 0x7f97c80535b0 2024-09-10T10:44:12.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a80b6680 msgr2=0x7f25a8169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.237+0000 7f25aea00640 1 --2- 172.21.15.28:0/2730051713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a80b6680 0x7f25a8169830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009d90 tx=0x7f25a002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f78c5e00640 1 --2- 172.21.15.28:0/3051849436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80720b0 0x7f78c80cfca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78b80097a0 tx=0x7f78b802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103f038cf109c86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f2d1f400640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc3c01b070 con 0x7fbc44073b10 2024-09-10T10:44:12.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 shutdown_connections 2024-09-10T10:44:12.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f25aea00640 1 --2- 172.21.15.28:0/2730051713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a80b6680 0x7f25a8169830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 >> 172.21.15.28:0/2730051713 conn(0x7f25a80d44f0 msgr2=0x7f25a80d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.238+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2d180ac6b0 con 0x7f2d28161c80 2024-09-10T10:44:12.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7f2d27400640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 shutdown_connections 2024-09-10T10:44:12.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7fbc41600640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 0x7fbc2406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7f25aea00640 1 -- 172.21.15.28:0/2730051713 wait complete. 2024-09-10T10:44:12.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbc3c08a8f0 con 0x7fbc44073b10 2024-09-10T10:44:12.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.240+0000 7f2d27400640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 0x7f2d0406e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d200011e0 tx=0x7f2d20007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.239+0000 7fbc49000640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 0x7fbc2406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.240+0000 7fbc49000640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 0x7fbc2406e340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc4415cfd0 tx=0x7fbc38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.241+0000 7f25aea00640 1 Processor -- start 2024-09-10T10:44:12.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.241+0000 7f25aea00640 1 -- start start 2024-09-10T10:44:12.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.241+0000 7f78c4a00640 1 -- 172.21.15.28:0/3051849436 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8037070 con 0x7f78c80720b0 2024-09-10T10:44:12.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f78c4a00640 1 -- 172.21.15.28:0/3051849436 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b8032440 con 0x7f78c80720b0 2024-09-10T10:44:12.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f78c4a00640 1 -- 172.21.15.28:0/3051849436 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8032810 con 0x7f78c80720b0 2024-09-10T10:44:12.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f89eaa00640 1 -- 172.21.15.28:0/2110426923 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc037070 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f89eaa00640 1 -- 172.21.15.28:0/2110426923 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc032440 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f89eaa00640 1 -- 172.21.15.28:0/2110426923 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc032810 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 0x7f25a80b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a8162060 0x7f25a8151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f25a80b6cb0 con 0x7f25a8161c80 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25a80b6b30 con 0x7f25a8162060 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25aea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25a8169d70 con 0x7f25a80b6680 2024-09-10T10:44:12.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f5c0584b0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f78cce00640 1 -- 172.21.15.28:0/3051849436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80720b0 msgr2=0x7f78c80cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 -- 172.21.15.28:0/2110426923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d2aa0 msgr2=0x7f89ec0d2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 --2- 172.21.15.28:0/2110426923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec0d2f10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0097a0 tx=0x7f89dc02a270 comp rx=0 tx=0).stop 2024-09-10T10:44:12.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f78cce00640 1 --2- 172.21.15.28:0/3051849436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80720b0 0x7f78c80cfca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78b80097a0 tx=0x7f78b802a230 comp rx=0 tx=0).stop 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 -- 172.21.15.28:0/2110426923 shutdown_connections 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 --2- 172.21.15.28:0/2110426923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec0d2f10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 -- 172.21.15.28:0/2110426923 >> 172.21.15.28:0/2110426923 conn(0x7f89ec09d770 msgr2=0x7f89ec08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25a7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44398/0 (socket says 172.21.15.28:44398) 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.242+0000 7f25a7e00640 1 -- 172.21.15.28:0/3552827357 learned_addr learned my addr 172.21.15.28:0/3552827357 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7e00640 1 -- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a8162060 msgr2=0x7f25a8151370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:44:12.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7e00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a8162060 0x7f25a8151370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7e00640 1 -- 172.21.15.28:0/3552827357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 msgr2=0x7f25a80b2a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:44:12.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f78cce00640 1 -- 172.21.15.28:0/3051849436 shutdown_connections 2024-09-10T10:44:12.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d280b98d0 con 0x7f2d28161c80 2024-09-10T10:44:12.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7400640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 0x7f25a80b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f89f1a00640 1 -- 172.21.15.28:0/2110426923 shutdown_connections 2024-09-10T10:44:12.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7e00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 0x7f25a80b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f25a7e00640 1 -- 172.21.15.28:0/3552827357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a80b8580 con 0x7f25a80b6680 2024-09-10T10:44:12.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f78cce00640 1 --2- 172.21.15.28:0/3051849436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c80720b0 0x7f78c80cfca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.243+0000 7f78cce00640 1 -- 172.21.15.28:0/3051849436 >> 172.21.15.28:0/3051849436 conn(0x7f78c8067d70 msgr2=0x7f78c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.244+0000 7f78cce00640 1 -- 172.21.15.28:0/3051849436 shutdown_connections 2024-09-10T10:44:12.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.244+0000 7f78cce00640 1 -- 172.21.15.28:0/3051849436 wait complete. 2024-09-10T10:44:12.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.244+0000 7f89f1a00640 1 -- 172.21.15.28:0/2110426923 wait complete. 2024-09-10T10:44:12.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.244+0000 7f25a7400640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 0x7f25a80b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:12.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.246+0000 7f25a7e00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f259800c430 tx=0x7f259800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.246+0000 7f78cce00640 1 Processor -- start 2024-09-10T10:44:12.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598017070 con 0x7f25a80b6680 2024-09-10T10:44:12.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2598005ce0 con 0x7f25a80b6680 2024-09-10T10:44:12.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980072f0 con 0x7f25a80b6680 2024-09-10T10:44:12.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a80b8860 con 0x7f25a80b6680 2024-09-10T10:44:12.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.248+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a80b8e20 con 0x7f25a80b6680 2024-09-10T10:44:12.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f78cce00640 1 -- start start 2024-09-10T10:44:12.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc3c0588f0 con 0x7fbc44073b10 2024-09-10T10:44:12.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.246+0000 7f89f1a00640 1 Processor -- start 2024-09-10T10:44:12.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.247+0000 7f89f1a00640 1 -- start start 2024-09-10T10:44:12.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 0x7f89ec0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 0x7f89ec1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0cfc60 con 0x7f89ec0d2aa0 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec0cffd0 con 0x7f89ec0d4ac0 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.249+0000 7f89f1a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f89ec0cf070 con 0x7f89ec161920 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.250+0000 7f89eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 0x7f89ec0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.250+0000 7f89eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 0x7f89ec0cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36232/0 (socket says 172.21.15.28:36232) 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.250+0000 7f89eb400640 1 -- 172.21.15.28:0/3363892170 learned_addr learned my addr 172.21.15.28:0/3363892170 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.250+0000 7f89f1000640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 0x7f89ec1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.250+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a80b5f70 con 0x7f25a80b6680 2024-09-10T10:44:12.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.251+0000 7f89f1000640 1 -- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 msgr2=0x7f89ec0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.251+0000 7f89f1000640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 0x7f89ec0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.251+0000 7f89f1000640 1 -- 172.21.15.28:0/3363892170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 msgr2=0x7f89ec158990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:44:12.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.251+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f259801a070 con 0x7f25a80b6680 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 0x7f78c815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 0x7f78c8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 0x7f78c8162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78c80b3d70 con 0x7f78c80720b0 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78c80b3a90 con 0x7f78c8162ad0 2024-09-10T10:44:12.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f78cce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f78c80ce060 con 0x7f78c806f6f0 2024-09-10T10:44:12.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.252+0000 7f89ebe00640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f259fe00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f89f1000640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f259808a520 con 0x7f25a80b6680 2024-09-10T10:44:12.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f78c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 0x7f78c8162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f25a7400640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f78c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 0x7f78c8162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:36248/0 (socket says 172.21.15.28:36248) 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f78c5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 0x7f78c8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f78c6800640 1 -- 172.21.15.28:0/1864659619 learned_addr learned my addr 172.21.15.28:0/1864659619 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5e00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 0x7f78c815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 -- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 msgr2=0x7f78c815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 0x7f78c815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 -- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 msgr2=0x7f78c8162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 0x7f78c8162eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 -- 172.21.15.28:0/1864659619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c8163700 con 0x7f78c80720b0 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5e00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 0x7f78c815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f78c5400640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 0x7f78c8157650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c440 tx=0x7f78b400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.255+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4017070 con 0x7f78c80720b0 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.255+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b4005ce0 con 0x7f78c80720b0 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.255+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40072d0 con 0x7f78c80720b0 2024-09-10T10:44:12.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.253+0000 7f89f1000640 1 -- 172.21.15.28:0/3363892170 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec0ccf30 con 0x7f89ec161920 2024-09-10T10:44:12.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.254+0000 7f89ebe00640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:12.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.256+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c81639e0 con 0x7f78c80720b0 2024-09-10T10:44:12.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.256+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c8163fa0 con 0x7f78c80720b0 2024-09-10T10:44:12.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.256+0000 7f25a7400640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2594007fd0 tx=0x7f2594009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.256+0000 7f89f1000640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 0x7f89ec1f7c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89dc031000 tx=0x7f89dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.256+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc037050 con 0x7f89ec161920 2024-09-10T10:44:12.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.257+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89dc043070 con 0x7f89ec161920 2024-09-10T10:44:12.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.257+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89dc030dc0 con 0x7f89ec161920 2024-09-10T10:44:12.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.257+0000 7f89f1a00640 1 -- 172.21.15.28:0/3363892170 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec0cd210 con 0x7f89ec161920 2024-09-10T10:44:12.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.257+0000 7f89f1a00640 1 -- 172.21.15.28:0/3363892170 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec0cd7d0 con 0x7f89ec161920 2024-09-10T10:44:12.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.259+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c80cfca0 con 0x7f78c80720b0 2024-09-10T10:44:12.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.258+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2598058c20 con 0x7f25a80b6680 2024-09-10T10:44:12.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.260+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec0d2f10 con 0x7f89ec161920 2024-09-10T10:44:12.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.261+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89dc048460 con 0x7f89ec161920 2024-09-10T10:44:12.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.263+0000 7f89e3e00640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 0x7f89cc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.263+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f89dc0abeb0 con 0x7f89ec161920 2024-09-10T10:44:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.259+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f78b401a070 con 0x7f78c80720b0 2024-09-10T10:44:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.264+0000 7f78be000640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 0x7f78a806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.265+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f78b408a2c0 con 0x7f78c80720b0 2024-09-10T10:44:12.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.265+0000 7f89ebe00640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 0x7f89cc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.266+0000 7f78c5e00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 0x7f78a806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.267+0000 7f78c5e00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 0x7f78a806e3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78b8034000 tx=0x7f78b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.267+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89dc07a500 con 0x7f89ec161920 2024-09-10T10:44:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.268+0000 7f89ebe00640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 0x7f89cc06e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89e4001cd0 tx=0x7f89e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.271+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f78b4058350 con 0x7f78c80720b0 2024-09-10T10:44:12.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.654+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2f6c165ef0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.655+0000 7f2f62a00640 1 -- 172.21.15.28:0/2464313730 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f5c05c7b0 con 0x7f2f6c15c9f0 2024-09-10T10:44:12.659 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:12.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.655+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1eb80737a0 con 0x7f1eb81624d0 2024-09-10T10:44:12.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.656+0000 7f1eafe00640 1 -- 172.21.15.28:0/1294913644 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ea8079b50 con 0x7f1eb81624d0 2024-09-10T10:44:12.660 INFO:teuthology.orchestra.run.smithi028.stdout:64424509449 2024-09-10T10:44:12.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.659+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 msgr2=0x7f2f4c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.659+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 0x7f2f4c06e3c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c1f55c0 tx=0x7f2f580091c0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.659+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 msgr2=0x7f2f6c15b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.659+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 0x7f2f6c15b710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c410 tx=0x7f2f5c00c8d0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.659+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 msgr2=0x7f1e9406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 shutdown_connections 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06bf00 0x7f2f4c06e3c0 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c1f55c0 tx=0x7f2f580091c0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f6c162cf0 0x7f2f6c0b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c15c9f0 0x7f2f6c15b710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 --2- 172.21.15.28:0/2464313730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f6c0b3650 0x7f2f6c15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 >> 172.21.15.28:0/2464313730 conn(0x7f2f6c06bdf0 msgr2=0x7f2f6c0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 0x7f1e9406e3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0004ea0 tx=0x7f1eb0009040 comp rx=0 tx=0).stop 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 msgr2=0x7f1eb8165100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 0x7f1eb8165100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8031000 tx=0x7f1ea80046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 shutdown_connections 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.660+0000 7f2f6be00640 1 -- 172.21.15.28:0/2464313730 wait complete. 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 shutdown_connections 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e9406bf30 0x7f1e9406e3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1eb81628b0 0x7f1eb81fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1eb81624d0 0x7f1eb8165100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 --2- 172.21.15.28:0/1294913644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1eb80d2940 0x7f1eb81554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 >> 172.21.15.28:0/1294913644 conn(0x7f1eb809d720 msgr2=0x7f1eb80d0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 shutdown_connections 2024-09-10T10:44:12.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.661+0000 7f1ebd400640 1 -- 172.21.15.28:0/1294913644 wait complete. 2024-09-10T10:44:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.665+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f97c803b430 con 0x7f97c80535b0 2024-09-10T10:44:12.668 INFO:teuthology.orchestra.run.smithi028.stdout:64424509451 2024-09-10T10:44:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.666+0000 7f97c6a00640 1 -- 172.21.15.28:0/1675357088 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f97d00d5b20 con 0x7f97c80535b0 2024-09-10T10:44:12.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.670+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 msgr2=0x7f97b006e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.670+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 0x7f97b006e340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97c000a8b0 tx=0x7f97c0008040 comp rx=0 tx=0).stop 2024-09-10T10:44:12.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.670+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 msgr2=0x7f97c803de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.670+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f97d002dd50 tx=0x7f97d0074aa0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 shutdown_connections 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97b006be80 0x7f97b006e340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97c80e72f0 0x7f97c80e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80535b0 0x7f97c803de80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 --2- 172.21.15.28:0/1675357088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97c803ce20 0x7f97c8052430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 >> 172.21.15.28:0/1675357088 conn(0x7f97c803a520 msgr2=0x7f97c80e9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 shutdown_connections 2024-09-10T10:44:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.671+0000 7f97d5a00640 1 -- 172.21.15.28:0/1675357088 wait complete. 2024-09-10T10:44:12.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.677+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbc4415b1d0 con 0x7fbc44073b10 2024-09-10T10:44:12.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.678+0000 7fbc41600640 1 -- 172.21.15.28:0/3921864454 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbc3c05cbf0 con 0x7fbc44073b10 2024-09-10T10:44:12.679 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:12.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.681+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 msgr2=0x7fbc2406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.681+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 0x7fbc2406e340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc4415cfd0 tx=0x7fbc38008040 comp rx=0 tx=0).stop 2024-09-10T10:44:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 msgr2=0x7fbc441fa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00c8c0 tx=0x7fbc3c00cd80 comp rx=0 tx=0).stop 2024-09-10T10:44:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 shutdown_connections 2024-09-10T10:44:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc2406be80 0x7fbc2406e340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc4415b500 0x7fbc44072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc4415abe0 0x7fbc44072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 --2- 172.21.15.28:0/3921864454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc44073b10 0x7fbc441fa5f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.682+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 >> 172.21.15.28:0/3921864454 conn(0x7fbc4406c240 msgr2=0x7fbc44071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.685+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 shutdown_connections 2024-09-10T10:44:12.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.685+0000 7fbc4ae00640 1 -- 172.21.15.28:0/3921864454 wait complete. 2024-09-10T10:44:12.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.686+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f2d280ba470 con 0x7f2d28161c80 2024-09-10T10:44:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.687+0000 7f2d1f400640 1 -- 172.21.15.28:0/171646488 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d1807a6d0 con 0x7f2d28161c80 2024-09-10T10:44:12.688 INFO:teuthology.orchestra.run.smithi028.stdout:64424509450 2024-09-10T10:44:12.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-09-10T10:44:12.690 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.691+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f25a80ba470 con 0x7f25a80b6680 2024-09-10T10:44:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.691+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 msgr2=0x7f2d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.691+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 0x7f2d0406e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2d200011e0 tx=0x7f2d20007d30 comp rx=0 tx=0).stop 2024-09-10T10:44:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.691+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 msgr2=0x7f2d280b7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.691+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 0x7f2d280b7d00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d18031000 tx=0x7f2d18004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 shutdown_connections 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d0406be30 0x7f2d0406e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d28162060 0x7f2d280ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f259fe00640 1 -- 172.21.15.28:0/3552827357 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2598058520 con 0x7f25a80b6680 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d28161c80 0x7f2d280b7d00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 --2- 172.21.15.28:0/171646488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d280b64b0 0x7f2d28155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 >> 172.21.15.28:0/171646488 conn(0x7f2d280d44f0 msgr2=0x7f2d2806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 shutdown_connections 2024-09-10T10:44:12.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.692+0000 7f2d2e800640 1 -- 172.21.15.28:0/171646488 wait complete. 2024-09-10T10:44:12.695 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2024-09-10T10:44:12.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2024-09-10T10:44:12.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.693+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f78c815d9d0 con 0x7f78c80720b0 2024-09-10T10:44:12.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.694+0000 7f78be000640 1 -- 172.21.15.28:0/1864659619 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78b405c650 con 0x7f78c80720b0 2024-09-10T10:44:12.696 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.697 INFO:teuthology.orchestra.run.smithi028.stdout:64424509451 2024-09-10T10:44:12.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.694+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f89ec1e8910 con 0x7f89ec161920 2024-09-10T10:44:12.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.695+0000 7f89e3e00640 1 -- 172.21.15.28:0/3363892170 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f89dc079e00 con 0x7f89ec161920 2024-09-10T10:44:12.697 INFO:teuthology.orchestra.run.smithi028.stdout:64424509449 2024-09-10T10:44:12.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 msgr2=0x7f258806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2594007fd0 tx=0x7f2594009040 comp rx=0 tx=0).stop 2024-09-10T10:44:12.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 msgr2=0x7f25a80b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f259800c430 tx=0x7f259800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 shutdown_connections 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a8162060 0x7f25a8151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a8161c80 0x7f25a80b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 --2- 172.21.15.28:0/3552827357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a80b6680 0x7f25a80b1a90 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f259800c430 tx=0x7f259800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.696+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 >> 172.21.15.28:0/3552827357 conn(0x7f25a80d44f0 msgr2=0x7f25a80d3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 shutdown_connections 2024-09-10T10:44:12.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f25aea00640 1 -- 172.21.15.28:0/3552827357 wait complete. 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 msgr2=0x7f78a806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 0x7f78a806e3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78b8034000 tx=0x7f78b802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 msgr2=0x7f78c8157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 0x7f78c8157650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78b400c440 tx=0x7f78b400c900 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 shutdown_connections 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f78a806bf30 0x7f78a806e3f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8162ad0 0x7f78c8162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78c80720b0 0x7f78c8157650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 --2- 172.21.15.28:0/1864659619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78c806f6f0 0x7f78c815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.697+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 >> 172.21.15.28:0/1864659619 conn(0x7f78c8067d70 msgr2=0x7f78c8161d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 shutdown_connections 2024-09-10T10:44:12.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-09-10T10:44:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f78cce00640 1 -- 172.21.15.28:0/1864659619 wait complete. 2024-09-10T10:44:12.702 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 msgr2=0x7f89cc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 0x7f89cc06e3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f89e4001cd0 tx=0x7f89e4009040 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 msgr2=0x7f89ec1f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.698+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 0x7f89ec1f7c70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89dc031000 tx=0x7f89dc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 shutdown_connections 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f89cc06bf00 0x7f89cc06e3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89ec161920 0x7f89ec1f7c70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89ec0d4ac0 0x7f89ec0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 --2- 172.21.15.28:0/3363892170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89ec0d2aa0 0x7f89ec158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 >> 172.21.15.28:0/3363892170 conn(0x7f89ec09d770 msgr2=0x7f89ec0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 shutdown_connections 2024-09-10T10:44:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.700+0000 7f89e1600640 1 -- 172.21.15.28:0/3363892170 wait complete. 2024-09-10T10:44:12.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-09-10T10:44:12.709 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-09-10T10:44:12.715 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.6 2024-09-10T10:44:12.719 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-09-10T10:44:12.724 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-09-10T10:44:12.725 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:12.725 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:44:12.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:44:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.921+0000 7fdee7a00640 1 Processor -- start 2024-09-10T10:44:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.921+0000 7fdee7a00640 1 -- start start 2024-09-10T10:44:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.922+0000 7fdee7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee0162060 0x7fdee00b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.922+0000 7fdee7a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee00b64b0 0x7fdee00b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.922+0000 7fdee7a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee006d9f0 con 0x7fdee0161c80 2024-09-10T10:44:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.922+0000 7fdee7a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdee006dcd0 con 0x7fdee00b64b0 2024-09-10T10:44:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.922+0000 7fdee7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdee006dfb0 con 0x7fdee0162060 2024-09-10T10:44:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdee0161c80 0x7fdee006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41300/0 (socket says 172.21.15.28:41300) 2024-09-10T10:44:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5000640 1 -- 172.21.15.28:0/563488931 learned_addr learned my addr 172.21.15.28:0/563488931 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:12.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdedfe00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee0162060 0x7fdee00b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5a00640 1 --2- 172.21.15.28:0/563488931 >> v2:172.21.15.28:3300/0 conn(0x7fdee00b64b0 0x7fdee00b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704529091 0 0) 0x7fdee006d9f0 con 0x7fdee0161c80 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdec8003520 con 0x7fdee0161c80 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5a00640 1 -- 172.21.15.28:0/563488931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee0162060 msgr2=0x7fdee00b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5a00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee0162060 0x7fdee00b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5a00640 1 -- 172.21.15.28:0/563488931 >> v1:172.21.15.28:6789/0 conn(0x7fdee0161c80 legacy=0x7fdee006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 482998667 0 0) 0x7fdec8003520 con 0x7fdee0161c80 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.923+0000 7fdee5a00640 1 -- 172.21.15.28:0/563488931 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00d01f0 con 0x7fdee00b64b0 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.924+0000 7fdee5a00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee00b64b0 0x7fdee00b6960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fded8009d90 tx=0x7fded802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5742ac7857bcf32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8007740 con 0x7fdee00b64b0 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded80078e0 con 0x7fdee00b64b0 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdedf400640 1 -- 172.21.15.28:0/563488931 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded8007cb0 con 0x7fdee00b64b0 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 -- 172.21.15.28:0/563488931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee00b64b0 msgr2=0x7fdee00b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee00b64b0 0x7fdee00b6960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fded8009d90 tx=0x7fded802a660 comp rx=0 tx=0).stop 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 -- 172.21.15.28:0/563488931 shutdown_connections 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee00b64b0 0x7fdee00b6960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 --2- 172.21.15.28:0/563488931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee0162060 0x7fdee00b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 -- 172.21.15.28:0/563488931 >> 172.21.15.28:0/563488931 conn(0x7fdee00d44f0 msgr2=0x7fdee00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.925+0000 7fdee7a00640 1 -- 172.21.15.28:0/563488931 shutdown_connections 2024-09-10T10:44:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.926+0000 7fdee7a00640 1 -- 172.21.15.28:0/563488931 wait complete. 2024-09-10T10:44:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.927+0000 7fdee7a00640 1 Processor -- start 2024-09-10T10:44:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.927+0000 7fdee7a00640 1 -- start start 2024-09-10T10:44:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee00b64b0 0x7fdee0159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0162060 0x7fdee0160190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00cf1e0 con 0x7fdee0161c80 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdee00d2aa0 con 0x7fdee0162060 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdee7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdee00cf360 con 0x7fdee00b64b0 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36740/0 (socket says 172.21.15.28:36740) 2024-09-10T10:44:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.928+0000 7fdedfe00640 1 -- 172.21.15.28:0/1563277721 learned_addr learned my addr 172.21.15.28:0/1563277721 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdee5000640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee00b64b0 0x7fdee0159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdee5a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0162060 0x7fdee0160190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 -- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee00b64b0 msgr2=0x7fdee0159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee00b64b0 0x7fdee0159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 -- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0162060 msgr2=0x7fdee0160190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0162060 0x7fdee0160190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 -- 172.21.15.28:0/1563277721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee00b82a0 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedfe00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e870 tx=0x7fdecc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc017070 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdecc00ee60 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00cb60 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.929+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee00b91c0 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.930+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee00ba1e0 con 0x7fdee0161c80 2024-09-10T10:44:12.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.931+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee00b5f70 con 0x7fdee0161c80 2024-09-10T10:44:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.931+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdecc01a070 con 0x7fdee0161c80 2024-09-10T10:44:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.932+0000 7fdedcc00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 0x7fdec006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.932+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdecc08b410 con 0x7fdee0161c80 2024-09-10T10:44:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.933+0000 7fdee5000640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 0x7fdec006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.934+0000 7fdee5000640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 0x7fdec006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fded4007fd0 tx=0x7fded4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:12.937+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdecc0599e0 con 0x7fdee0161c80 2024-09-10T10:44:13.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.117+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdee00d2270 con 0x7fdec006be30 2024-09-10T10:44:13.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.126+0000 7fdedcc00640 1 -- 172.21.15.28:0/1563277721 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7fdee00d2270 con 0x7fdec006be30 2024-09-10T10:44:13.130 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:13.132 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 msgr2=0x7fdec006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 0x7fdec006e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fded4007fd0 tx=0x7fded4009040 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 msgr2=0x7fdee00b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00e870 tx=0x7fdecc00ed30 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 shutdown_connections 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdec006be30 0x7fdec006e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdee0162060 0x7fdee0160190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee0161c80 0x7fdee00b9ca0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 --2- 172.21.15.28:0/1563277721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdee00b64b0 0x7fdee0159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 >> 172.21.15.28:0/1563277721 conn(0x7fdee00d44f0 msgr2=0x7fdee00b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.130+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 shutdown_connections 2024-09-10T10:44:13.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.131+0000 7fdee7a00640 1 -- 172.21.15.28:0/1563277721 wait complete. 2024-09-10T10:44:13.147 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-10T10:44:12.488407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223844,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750876,"statfs":{"total":773094113280,"available":772864897024,"internally_reserved":0,"allocated":3514368,"data_stored":2081516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870128+0000","last_change":"2024-09-10T10:43:46.542097+0000","last_active":"2024-09-10T10:43:52.870128+0000","last_peered":"2024-09-10T10:43:52.870128+0000","last_clean":"2024-09-10T10:43:52.870128+0000","last_became_active":"2024-09-10T10:43:46.541830+0000","last_became_peered":"2024-09-10T10:43:46.541830+0000","last_unstale":"2024-09-10T10:43:52.870128+0000","last_undegraded":"2024-09-10T10:43:52.870128+0000","last_fullsized":"2024-09-10T10:43:52.870128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709072+0000","last_change":"2024-09-10T10:43:47.561686+0000","last_active":"2024-09-10T10:43:52.709072+0000","last_peered":"2024-09-10T10:43:52.709072+0000","last_clean":"2024-09-10T10:43:52.709072+0000","last_became_active":"2024-09-10T10:43:47.561406+0000","last_became_peered":"2024-09-10T10:43:47.561406+0000","last_unstale":"2024-09-10T10:43:52.709072+0000","last_undegraded":"2024-09-10T10:43:52.709072+0000","last_fullsized":"2024-09-10T10:43:52.709072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.700092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563827+0000","last_change":"2024-09-10T10:43:51.575030+0000","last_active":"2024-09-10T10:43:52.563827+0000","last_peered":"2024-09-10T10:43:52.563827+0000","last_clean":"2024-09-10T10:43:52.563827+0000","last_became_active":"2024-09-10T10:43:51.570182+0000","last_became_peered":"2024-09-10T10:43:51.570182+0000","last_unstale":"2024-09-10T10:43:52.563827+0000","last_undegraded":"2024-09-10T10:43:52.563827+0000","last_fullsized":"2024-09-10T10:43:52.563827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:21.179072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709156+0000","last_change":"2024-09-10T10:43:46.528230+0000","last_active":"2024-09-10T10:43:52.709156+0000","last_peered":"2024-09-10T10:43:52.709156+0000","last_clean":"2024-09-10T10:43:52.709156+0000","last_became_active":"2024-09-10T10:43:46.527955+0000","last_became_peered":"2024-09-10T10:43:46.527955+0000","last_unstale":"2024-09-10T10:43:52.709156+0000","last_undegraded":"2024-09-10T10:43:52.709156+0000","last_fullsized":"2024-09-10T10:43:52.709156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:23.630718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708899+0000","last_change":"2024-09-10T10:43:47.536706+0000","last_active":"2024-09-10T10:43:52.708899+0000","last_peered":"2024-09-10T10:43:52.708899+0000","last_clean":"2024-09-10T10:43:52.708899+0000","last_became_active":"2024-09-10T10:43:47.536347+0000","last_became_peered":"2024-09-10T10:43:47.536347+0000","last_unstale":"2024-09-10T10:43:52.708899+0000","last_undegraded":"2024-09-10T10:43:52.708899+0000","last_fullsized":"2024-09-10T10:43:52.708899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:57.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563851+0000","last_change":"2024-09-10T10:43:51.569149+0000","last_active":"2024-09-10T10:43:52.563851+0000","last_peered":"2024-09-10T10:43:52.563851+0000","last_clean":"2024-09-10T10:43:52.563851+0000","last_became_active":"2024-09-10T10:43:51.568748+0000","last_became_peered":"2024-09-10T10:43:51.568748+0000","last_unstale":"2024-09-10T10:43:52.563851+0000","last_undegraded":"2024-09-10T10:43:52.563851+0000","last_fullsized":"2024-09-10T10:43:52.563851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869713+0000","last_change":"2024-09-10T10:43:46.525149+0000","last_active":"2024-09-10T10:43:52.869713+0000","last_peered":"2024-09-10T10:43:52.869713+0000","last_clean":"2024-09-10T10:43:52.869713+0000","last_became_active":"2024-09-10T10:43:46.524877+0000","last_became_peered":"2024-09-10T10:43:46.524877+0000","last_unstale":"2024-09-10T10:43:52.869713+0000","last_undegraded":"2024-09-10T10:43:52.869713+0000","last_fullsized":"2024-09-10T10:43:52.869713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:40.800800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564917+0000","last_change":"2024-09-10T10:43:47.547350+0000","last_active":"2024-09-10T10:43:52.564917+0000","last_peered":"2024-09-10T10:43:52.564917+0000","last_clean":"2024-09-10T10:43:52.564917+0000","last_became_active":"2024-09-10T10:43:47.547041+0000","last_became_peered":"2024-09-10T10:43:47.547041+0000","last_unstale":"2024-09-10T10:43:52.564917+0000","last_undegraded":"2024-09-10T10:43:52.564917+0000","last_fullsized":"2024-09-10T10:43:52.564917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563358+0000","last_change":"2024-09-10T10:43:51.587632+0000","last_active":"2024-09-10T10:43:52.563358+0000","last_peered":"2024-09-10T10:43:52.563358+0000","last_clean":"2024-09-10T10:43:52.563358+0000","last_became_active":"2024-09-10T10:43:51.587202+0000","last_became_peered":"2024-09-10T10:43:51.587202+0000","last_unstale":"2024-09-10T10:43:52.563358+0000","last_undegraded":"2024-09-10T10:43:52.563358+0000","last_fullsized":"2024-09-10T10:43:52.563358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:57.684586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869700+0000","last_change":"2024-09-10T10:43:46.529521+0000","last_active":"2024-09-10T10:43:52.869700+0000","last_peered":"2024-09-10T10:43:52.869700+0000","last_clean":"2024-09-10T10:43:52.869700+0000","last_became_active":"2024-09-10T10:43:46.529170+0000","last_became_peered":"2024-09-10T10:43:46.529170+0000","last_unstale":"2024-09-10T10:43:52.869700+0000","last_undegraded":"2024-09-10T10:43:52.869700+0000","last_fullsized":"2024-09-10T10:43:52.869700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870421+0000","last_change":"2024-09-10T10:43:47.550037+0000","last_active":"2024-09-10T10:43:52.870421+0000","last_peered":"2024-09-10T10:43:52.870421+0000","last_clean":"2024-09-10T10:43:52.870421+0000","last_became_active":"2024-09-10T10:43:47.549678+0000","last_became_peered":"2024-09-10T10:43:47.549678+0000","last_unstale":"2024-09-10T10:43:52.870421+0000","last_undegraded":"2024-09-10T10:43:52.870421+0000","last_fullsized":"2024-09-10T10:43:52.870421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.860584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563399+0000","last_change":"2024-09-10T10:43:51.577274+0000","last_active":"2024-09-10T10:43:52.563399+0000","last_peered":"2024-09-10T10:43:52.563399+0000","last_clean":"2024-09-10T10:43:52.563399+0000","last_became_active":"2024-09-10T10:43:51.576818+0000","last_became_peered":"2024-09-10T10:43:51.576818+0000","last_unstale":"2024-09-10T10:43:52.563399+0000","last_undegraded":"2024-09-10T10:43:52.563399+0000","last_fullsized":"2024-09-10T10:43:52.563399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869003+0000","last_change":"2024-09-10T10:43:47.528998+0000","last_active":"2024-09-10T10:43:52.869003+0000","last_peered":"2024-09-10T10:43:52.869003+0000","last_clean":"2024-09-10T10:43:52.869003+0000","last_became_active":"2024-09-10T10:43:47.528761+0000","last_became_peered":"2024-09-10T10:43:47.528761+0000","last_unstale":"2024-09-10T10:43:52.869003+0000","last_undegraded":"2024-09-10T10:43:52.869003+0000","last_fullsized":"2024-09-10T10:43:52.869003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:27.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538367+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538043+0000","last_became_peered":"2024-09-10T10:43:46.538043+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:27.129655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870256+0000","last_change":"2024-09-10T10:43:51.589628+0000","last_active":"2024-09-10T10:43:52.870256+0000","last_peered":"2024-09-10T10:43:52.870256+0000","last_clean":"2024-09-10T10:43:52.870256+0000","last_became_active":"2024-09-10T10:43:51.588470+0000","last_became_peered":"2024-09-10T10:43:51.588470+0000","last_unstale":"2024-09-10T10:43:52.870256+0000","last_undegraded":"2024-09-10T10:43:52.870256+0000","last_fullsized":"2024-09-10T10:43:52.870256+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:51.784026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564115+0000","last_change":"2024-09-10T10:43:47.539521+0000","last_active":"2024-09-10T10:43:52.564115+0000","last_peered":"2024-09-10T10:43:52.564115+0000","last_clean":"2024-09-10T10:43:52.564115+0000","last_became_active":"2024-09-10T10:43:47.539187+0000","last_became_peered":"2024-09-10T10:43:47.539187+0000","last_unstale":"2024-09-10T10:43:52.564115+0000","last_undegraded":"2024-09-10T10:43:52.564115+0000","last_fullsized":"2024-09-10T10:43:52.564115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870141+0000","last_change":"2024-09-10T10:43:46.520278+0000","last_active":"2024-09-10T10:43:52.870141+0000","last_peered":"2024-09-10T10:43:52.870141+0000","last_clean":"2024-09-10T10:43:52.870141+0000","last_became_active":"2024-09-10T10:43:46.519954+0000","last_became_peered":"2024-09-10T10:43:46.519954+0000","last_unstale":"2024-09-10T10:43:52.870141+0000","last_undegraded":"2024-09-10T10:43:52.870141+0000","last_fullsized":"2024-09-10T10:43:52.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:51.570565+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:51.570165+0000","last_became_peered":"2024-09-10T10:43:51.570165+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.439716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869301+0000","last_change":"2024-09-10T10:43:47.534339+0000","last_active":"2024-09-10T10:43:52.869301+0000","last_peered":"2024-09-10T10:43:52.869301+0000","last_clean":"2024-09-10T10:43:52.869301+0000","last_became_active":"2024-09-10T10:43:47.533413+0000","last_became_peered":"2024-09-10T10:43:47.533413+0000","last_unstale":"2024-09-10T10:43:52.869301+0000","last_undegraded":"2024-09-10T10:43:52.869301+0000","last_fullsized":"2024-09-10T10:43:52.869301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869263+0000","last_change":"2024-09-10T10:43:46.535278+0000","last_active":"2024-09-10T10:43:52.869263+0000","last_peered":"2024-09-10T10:43:52.869263+0000","last_clean":"2024-09-10T10:43:52.869263+0000","last_became_active":"2024-09-10T10:43:46.534954+0000","last_became_peered":"2024-09-10T10:43:46.534954+0000","last_unstale":"2024-09-10T10:43:52.869263+0000","last_undegraded":"2024-09-10T10:43:52.869263+0000","last_fullsized":"2024-09-10T10:43:52.869263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564514+0000","last_change":"2024-09-10T10:43:51.574146+0000","last_active":"2024-09-10T10:43:52.564514+0000","last_peered":"2024-09-10T10:43:52.564514+0000","last_clean":"2024-09-10T10:43:52.564514+0000","last_became_active":"2024-09-10T10:43:51.573681+0000","last_became_peered":"2024-09-10T10:43:51.573681+0000","last_unstale":"2024-09-10T10:43:52.564514+0000","last_undegraded":"2024-09-10T10:43:52.564514+0000","last_fullsized":"2024-09-10T10:43:52.564514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:13.180965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868735+0000","last_change":"2024-09-10T10:43:47.532923+0000","last_active":"2024-09-10T10:43:52.868735+0000","last_peered":"2024-09-10T10:43:52.868735+0000","last_clean":"2024-09-10T10:43:52.868735+0000","last_became_active":"2024-09-10T10:43:47.532644+0000","last_became_peered":"2024-09-10T10:43:47.532644+0000","last_unstale":"2024-09-10T10:43:52.868735+0000","last_undegraded":"2024-09-10T10:43:52.868735+0000","last_fullsized":"2024-09-10T10:43:52.868735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:44.518256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868884+0000","last_change":"2024-09-10T10:43:46.525829+0000","last_active":"2024-09-10T10:43:52.868884+0000","last_peered":"2024-09-10T10:43:52.868884+0000","last_clean":"2024-09-10T10:43:52.868884+0000","last_became_active":"2024-09-10T10:43:46.523492+0000","last_became_peered":"2024-09-10T10:43:46.523492+0000","last_unstale":"2024-09-10T10:43:52.868884+0000","last_undegraded":"2024-09-10T10:43:52.868884+0000","last_fullsized":"2024-09-10T10:43:52.868884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564250+0000","last_change":"2024-09-10T10:43:51.580518+0000","last_active":"2024-09-10T10:43:52.564250+0000","last_peered":"2024-09-10T10:43:52.564250+0000","last_clean":"2024-09-10T10:43:52.564250+0000","last_became_active":"2024-09-10T10:43:51.580050+0000","last_became_peered":"2024-09-10T10:43:51.580050+0000","last_unstale":"2024-09-10T10:43:52.564250+0000","last_undegraded":"2024-09-10T10:43:52.564250+0000","last_fullsized":"2024-09-10T10:43:52.564250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869809+0000","last_change":"2024-09-10T10:43:47.532764+0000","last_active":"2024-09-10T10:43:52.869809+0000","last_peered":"2024-09-10T10:43:52.869809+0000","last_clean":"2024-09-10T10:43:52.869809+0000","last_became_active":"2024-09-10T10:43:47.532407+0000","last_became_peered":"2024-09-10T10:43:47.532407+0000","last_unstale":"2024-09-10T10:43:52.869809+0000","last_undegraded":"2024-09-10T10:43:52.869809+0000","last_fullsized":"2024-09-10T10:43:52.869809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.139356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870804+0000","last_change":"2024-09-10T10:43:46.537450+0000","last_active":"2024-09-10T10:43:52.870804+0000","last_peered":"2024-09-10T10:43:52.870804+0000","last_clean":"2024-09-10T10:43:52.870804+0000","last_became_active":"2024-09-10T10:43:46.537129+0000","last_became_peered":"2024-09-10T10:43:46.537129+0000","last_unstale":"2024-09-10T10:43:52.870804+0000","last_undegraded":"2024-09-10T10:43:52.870804+0000","last_fullsized":"2024-09-10T10:43:52.870804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:47.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563273+0000","last_change":"2024-09-10T10:43:51.580449+0000","last_active":"2024-09-10T10:43:52.563273+0000","last_peered":"2024-09-10T10:43:52.563273+0000","last_clean":"2024-09-10T10:43:52.563273+0000","last_became_active":"2024-09-10T10:43:51.579803+0000","last_became_peered":"2024-09-10T10:43:51.579803+0000","last_unstale":"2024-09-10T10:43:52.563273+0000","last_undegraded":"2024-09-10T10:43:52.563273+0000","last_fullsized":"2024-09-10T10:43:52.563273+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:22.089401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870018+0000","last_change":"2024-09-10T10:43:47.532014+0000","last_active":"2024-09-10T10:43:52.870018+0000","last_peered":"2024-09-10T10:43:52.870018+0000","last_clean":"2024-09-10T10:43:52.870018+0000","last_became_active":"2024-09-10T10:43:47.531662+0000","last_became_peered":"2024-09-10T10:43:47.531662+0000","last_unstale":"2024-09-10T10:43:52.870018+0000","last_undegraded":"2024-09-10T10:43:52.870018+0000","last_fullsized":"2024-09-10T10:43:52.870018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:35.559824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869967+0000","last_change":"2024-09-10T10:43:46.527723+0000","last_active":"2024-09-10T10:43:52.869967+0000","last_peered":"2024-09-10T10:43:52.869967+0000","last_clean":"2024-09-10T10:43:52.869967+0000","last_became_active":"2024-09-10T10:43:46.527452+0000","last_became_peered":"2024-09-10T10:43:46.527452+0000","last_unstale":"2024-09-10T10:43:52.869967+0000","last_undegraded":"2024-09-10T10:43:52.869967+0000","last_fullsized":"2024-09-10T10:43:52.869967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869180+0000","last_change":"2024-09-10T10:43:51.574307+0000","last_active":"2024-09-10T10:43:52.869180+0000","last_peered":"2024-09-10T10:43:52.869180+0000","last_clean":"2024-09-10T10:43:52.869180+0000","last_became_active":"2024-09-10T10:43:51.571308+0000","last_became_peered":"2024-09-10T10:43:51.571308+0000","last_unstale":"2024-09-10T10:43:52.869180+0000","last_undegraded":"2024-09-10T10:43:52.869180+0000","last_fullsized":"2024-09-10T10:43:52.869180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:09.361189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870742+0000","last_change":"2024-09-10T10:43:47.549822+0000","last_active":"2024-09-10T10:43:52.870742+0000","last_peered":"2024-09-10T10:43:52.870742+0000","last_clean":"2024-09-10T10:43:52.870742+0000","last_became_active":"2024-09-10T10:43:47.549526+0000","last_became_peered":"2024-09-10T10:43:47.549526+0000","last_unstale":"2024-09-10T10:43:52.870742+0000","last_undegraded":"2024-09-10T10:43:52.870742+0000","last_fullsized":"2024-09-10T10:43:52.870742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870715+0000","last_change":"2024-09-10T10:43:46.539321+0000","last_active":"2024-09-10T10:43:52.870715+0000","last_peered":"2024-09-10T10:43:52.870715+0000","last_clean":"2024-09-10T10:43:52.870715+0000","last_became_active":"2024-09-10T10:43:46.538763+0000","last_became_peered":"2024-09-10T10:43:46.538763+0000","last_unstale":"2024-09-10T10:43:52.870715+0000","last_undegraded":"2024-09-10T10:43:52.870715+0000","last_fullsized":"2024-09-10T10:43:52.870715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:53.745395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.571592+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.570946+0000","last_became_peered":"2024-09-10T10:43:51.570946+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564687+0000","last_change":"2024-09-10T10:43:47.535460+0000","last_active":"2024-09-10T10:43:52.564687+0000","last_peered":"2024-09-10T10:43:52.564687+0000","last_clean":"2024-09-10T10:43:52.564687+0000","last_became_active":"2024-09-10T10:43:47.535170+0000","last_became_peered":"2024-09-10T10:43:47.535170+0000","last_unstale":"2024-09-10T10:43:52.564687+0000","last_undegraded":"2024-09-10T10:43:52.564687+0000","last_fullsized":"2024-09-10T10:43:52.564687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708875+0000","last_change":"2024-09-10T10:43:46.521709+0000","last_active":"2024-09-10T10:43:52.708875+0000","last_peered":"2024-09-10T10:43:52.708875+0000","last_clean":"2024-09-10T10:43:52.708875+0000","last_became_active":"2024-09-10T10:43:46.521411+0000","last_became_peered":"2024-09-10T10:43:46.521411+0000","last_unstale":"2024-09-10T10:43:52.708875+0000","last_undegraded":"2024-09-10T10:43:52.708875+0000","last_fullsized":"2024-09-10T10:43:52.708875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:04.600951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:51.573047+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:51.572690+0000","last_became_peered":"2024-09-10T10:43:51.572690+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:47.541642+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:47.541174+0000","last_became_peered":"2024-09-10T10:43:47.541174+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:18.047791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869868+0000","last_change":"2024-09-10T10:43:46.532535+0000","last_active":"2024-09-10T10:43:52.869868+0000","last_peered":"2024-09-10T10:43:52.869868+0000","last_clean":"2024-09-10T10:43:52.869868+0000","last_became_active":"2024-09-10T10:43:46.532256+0000","last_became_peered":"2024-09-10T10:43:46.532256+0000","last_unstale":"2024-09-10T10:43:52.869868+0000","last_undegraded":"2024-09-10T10:43:52.869868+0000","last_fullsized":"2024-09-10T10:43:52.869868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:51.847379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870621+0000","last_change":"2024-09-10T10:43:51.571178+0000","last_active":"2024-09-10T10:43:52.870621+0000","last_peered":"2024-09-10T10:43:52.870621+0000","last_clean":"2024-09-10T10:43:52.870621+0000","last_became_active":"2024-09-10T10:43:51.570742+0000","last_became_peered":"2024-09-10T10:43:51.570742+0000","last_unstale":"2024-09-10T10:43:52.870621+0000","last_undegraded":"2024-09-10T10:43:52.870621+0000","last_fullsized":"2024-09-10T10:43:52.870621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708292+0000","last_change":"2024-09-10T10:43:47.536864+0000","last_active":"2024-09-10T10:43:52.708292+0000","last_peered":"2024-09-10T10:43:52.708292+0000","last_clean":"2024-09-10T10:43:52.708292+0000","last_became_active":"2024-09-10T10:43:47.536270+0000","last_became_peered":"2024-09-10T10:43:47.536270+0000","last_unstale":"2024-09-10T10:43:52.708292+0000","last_undegraded":"2024-09-10T10:43:52.708292+0000","last_fullsized":"2024-09-10T10:43:52.708292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:25.640976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563952+0000","last_change":"2024-09-10T10:43:46.521627+0000","last_active":"2024-09-10T10:43:52.563952+0000","last_peered":"2024-09-10T10:43:52.563952+0000","last_clean":"2024-09-10T10:43:52.563952+0000","last_became_active":"2024-09-10T10:43:46.521150+0000","last_became_peered":"2024-09-10T10:43:46.521150+0000","last_unstale":"2024-09-10T10:43:52.563952+0000","last_undegraded":"2024-09-10T10:43:52.563952+0000","last_fullsized":"2024-09-10T10:43:52.563952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:35.895649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870932+0000","last_change":"2024-09-10T10:43:51.580293+0000","last_active":"2024-09-10T10:43:52.870932+0000","last_peered":"2024-09-10T10:43:52.870932+0000","last_clean":"2024-09-10T10:43:52.870932+0000","last_became_active":"2024-09-10T10:43:51.576157+0000","last_became_peered":"2024-09-10T10:43:51.576157+0000","last_unstale":"2024-09-10T10:43:52.870932+0000","last_undegraded":"2024-09-10T10:43:52.870932+0000","last_fullsized":"2024-09-10T10:43:52.870932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869651+0000","last_change":"2024-09-10T10:43:47.563302+0000","last_active":"2024-09-10T10:43:52.869651+0000","last_peered":"2024-09-10T10:43:52.869651+0000","last_clean":"2024-09-10T10:43:52.869651+0000","last_became_active":"2024-09-10T10:43:47.562694+0000","last_became_peered":"2024-09-10T10:43:47.562694+0000","last_unstale":"2024-09-10T10:43:52.869651+0000","last_undegraded":"2024-09-10T10:43:52.869651+0000","last_fullsized":"2024-09-10T10:43:52.869651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868801+0000","last_change":"2024-09-10T10:43:46.533697+0000","last_active":"2024-09-10T10:43:52.868801+0000","last_peered":"2024-09-10T10:43:52.868801+0000","last_clean":"2024-09-10T10:43:52.868801+0000","last_became_active":"2024-09-10T10:43:46.533190+0000","last_became_peered":"2024-09-10T10:43:46.533190+0000","last_unstale":"2024-09-10T10:43:52.868801+0000","last_undegraded":"2024-09-10T10:43:52.868801+0000","last_fullsized":"2024-09-10T10:43:52.868801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:09.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708748+0000","last_change":"2024-09-10T10:43:51.583515+0000","last_active":"2024-09-10T10:43:52.708748+0000","last_peered":"2024-09-10T10:43:52.708748+0000","last_clean":"2024-09-10T10:43:52.708748+0000","last_became_active":"2024-09-10T10:43:51.582366+0000","last_became_peered":"2024-09-10T10:43:51.582366+0000","last_unstale":"2024-09-10T10:43:52.708748+0000","last_undegraded":"2024-09-10T10:43:52.708748+0000","last_fullsized":"2024-09-10T10:43:52.708748+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:10.957254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869329+0000","last_change":"2024-09-10T10:43:47.520783+0000","last_active":"2024-09-10T10:43:52.869329+0000","last_peered":"2024-09-10T10:43:52.869329+0000","last_clean":"2024-09-10T10:43:52.869329+0000","last_became_active":"2024-09-10T10:43:47.520409+0000","last_became_peered":"2024-09-10T10:43:47.520409+0000","last_unstale":"2024-09-10T10:43:52.869329+0000","last_undegraded":"2024-09-10T10:43:52.869329+0000","last_fullsized":"2024-09-10T10:43:52.869329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538924+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538640+0000","last_became_peered":"2024-09-10T10:43:46.538640+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:57.368590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564215+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:43:52.564215+0000","last_peered":"2024-09-10T10:43:52.564215+0000","last_clean":"2024-09-10T10:43:52.564215+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:43:52.564215+0000","last_undegraded":"2024-09-10T10:43:52.564215+0000","last_fullsized":"2024-09-10T10:43:52.564215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870584+0000","last_change":"2024-09-10T10:43:51.571200+0000","last_active":"2024-09-10T10:43:52.870584+0000","last_peered":"2024-09-10T10:43:52.870584+0000","last_clean":"2024-09-10T10:43:52.870584+0000","last_became_active":"2024-09-10T10:43:51.570877+0000","last_became_peered":"2024-09-10T10:43:51.570877+0000","last_unstale":"2024-09-10T10:43:52.870584+0000","last_undegraded":"2024-09-10T10:43:52.870584+0000","last_fullsized":"2024-09-10T10:43:52.870584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:59.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870468+0000","last_change":"2024-09-10T10:43:47.546420+0000","last_active":"2024-09-10T10:43:52.870468+0000","last_peered":"2024-09-10T10:43:52.870468+0000","last_clean":"2024-09-10T10:43:52.870468+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.870468+0000","last_undegraded":"2024-09-10T10:43:52.870468+0000","last_fullsized":"2024-09-10T10:43:52.870468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.887906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870800+0000","last_change":"2024-09-10T10:43:46.535842+0000","last_active":"2024-09-10T10:43:52.870800+0000","last_peered":"2024-09-10T10:43:52.870800+0000","last_clean":"2024-09-10T10:43:52.870800+0000","last_became_active":"2024-09-10T10:43:46.535586+0000","last_became_peered":"2024-09-10T10:43:46.535586+0000","last_unstale":"2024-09-10T10:43:52.870800+0000","last_undegraded":"2024-09-10T10:43:52.870800+0000","last_fullsized":"2024-09-10T10:43:52.870800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563751+0000","last_change":"2024-09-10T10:43:51.578759+0000","last_active":"2024-09-10T10:43:52.563751+0000","last_peered":"2024-09-10T10:43:52.563751+0000","last_clean":"2024-09-10T10:43:52.563751+0000","last_became_active":"2024-09-10T10:43:51.578237+0000","last_became_peered":"2024-09-10T10:43:51.578237+0000","last_unstale":"2024-09-10T10:43:52.563751+0000","last_undegraded":"2024-09-10T10:43:52.563751+0000","last_fullsized":"2024-09-10T10:43:52.563751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869654+0000","last_change":"2024-09-10T10:43:47.540794+0000","last_active":"2024-09-10T10:43:52.869654+0000","last_peered":"2024-09-10T10:43:52.869654+0000","last_clean":"2024-09-10T10:43:52.869654+0000","last_became_active":"2024-09-10T10:43:47.540360+0000","last_became_peered":"2024-09-10T10:43:47.540360+0000","last_unstale":"2024-09-10T10:43:52.869654+0000","last_undegraded":"2024-09-10T10:43:52.869654+0000","last_fullsized":"2024-09-10T10:43:52.869654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:31.926385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709016+0000","last_change":"2024-09-10T10:43:46.520898+0000","last_active":"2024-09-10T10:43:52.709016+0000","last_peered":"2024-09-10T10:43:52.709016+0000","last_clean":"2024-09-10T10:43:52.709016+0000","last_became_active":"2024-09-10T10:43:46.518998+0000","last_became_peered":"2024-09-10T10:43:46.518998+0000","last_unstale":"2024-09-10T10:43:52.709016+0000","last_undegraded":"2024-09-10T10:43:52.709016+0000","last_fullsized":"2024-09-10T10:43:52.709016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:42.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708786+0000","last_change":"2024-09-10T10:43:51.587866+0000","last_active":"2024-09-10T10:43:52.708786+0000","last_peered":"2024-09-10T10:43:52.708786+0000","last_clean":"2024-09-10T10:43:52.708786+0000","last_became_active":"2024-09-10T10:43:51.585690+0000","last_became_peered":"2024-09-10T10:43:51.585690+0000","last_unstale":"2024-09-10T10:43:52.708786+0000","last_undegraded":"2024-09-10T10:43:52.708786+0000","last_fullsized":"2024-09-10T10:43:52.708786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:41.798950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870081+0000","last_change":"2024-09-10T10:43:47.540910+0000","last_active":"2024-09-10T10:43:52.870081+0000","last_peered":"2024-09-10T10:43:52.870081+0000","last_clean":"2024-09-10T10:43:52.870081+0000","last_became_active":"2024-09-10T10:43:47.540603+0000","last_became_peered":"2024-09-10T10:43:47.540603+0000","last_unstale":"2024-09-10T10:43:52.870081+0000","last_undegraded":"2024-09-10T10:43:52.870081+0000","last_fullsized":"2024-09-10T10:43:52.870081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:48.137696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869935+0000","last_change":"2024-09-10T10:43:46.525479+0000","last_active":"2024-09-10T10:43:52.869935+0000","last_peered":"2024-09-10T10:43:52.869935+0000","last_clean":"2024-09-10T10:43:52.869935+0000","last_became_active":"2024-09-10T10:43:46.524834+0000","last_became_peered":"2024-09-10T10:43:46.524834+0000","last_unstale":"2024-09-10T10:43:52.869935+0000","last_undegraded":"2024-09-10T10:43:52.869935+0000","last_fullsized":"2024-09-10T10:43:52.869935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868574+0000","last_change":"2024-09-10T10:43:51.569746+0000","last_active":"2024-09-10T10:43:52.868574+0000","last_peered":"2024-09-10T10:43:52.868574+0000","last_clean":"2024-09-10T10:43:52.868574+0000","last_became_active":"2024-09-10T10:43:51.569289+0000","last_became_peered":"2024-09-10T10:43:51.569289+0000","last_unstale":"2024-09-10T10:43:52.868574+0000","last_undegraded":"2024-09-10T10:43:52.868574+0000","last_fullsized":"2024-09-10T10:43:52.868574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:20.122047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564540+0000","last_change":"2024-09-10T10:43:47.555824+0000","last_active":"2024-09-10T10:43:52.564540+0000","last_peered":"2024-09-10T10:43:52.564540+0000","last_clean":"2024-09-10T10:43:52.564540+0000","last_became_active":"2024-09-10T10:43:47.554957+0000","last_became_peered":"2024-09-10T10:43:47.554957+0000","last_unstale":"2024-09-10T10:43:52.564540+0000","last_undegraded":"2024-09-10T10:43:52.564540+0000","last_fullsized":"2024-09-10T10:43:52.564540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870114+0000","last_change":"2024-09-10T10:43:46.530043+0000","last_active":"2024-09-10T10:43:52.870114+0000","last_peered":"2024-09-10T10:43:52.870114+0000","last_clean":"2024-09-10T10:43:52.870114+0000","last_became_active":"2024-09-10T10:43:46.527509+0000","last_became_peered":"2024-09-10T10:43:46.527509+0000","last_unstale":"2024-09-10T10:43:52.870114+0000","last_undegraded":"2024-09-10T10:43:52.870114+0000","last_fullsized":"2024-09-10T10:43:52.870114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:08.859006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:51.577121+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:51.576021+0000","last_became_peered":"2024-09-10T10:43:51.576021+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:10.204576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868710+0000","last_change":"2024-09-10T10:43:47.562724+0000","last_active":"2024-09-10T10:43:52.868710+0000","last_peered":"2024-09-10T10:43:52.868710+0000","last_clean":"2024-09-10T10:43:52.868710+0000","last_became_active":"2024-09-10T10:43:47.562412+0000","last_became_peered":"2024-09-10T10:43:47.562412+0000","last_unstale":"2024-09-10T10:43:52.868710+0000","last_undegraded":"2024-09-10T10:43:52.868710+0000","last_fullsized":"2024-09-10T10:43:52.868710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:30.773785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869413+0000","last_change":"2024-09-10T10:43:46.525713+0000","last_active":"2024-09-10T10:43:52.869413+0000","last_peered":"2024-09-10T10:43:52.869413+0000","last_clean":"2024-09-10T10:43:52.869413+0000","last_became_active":"2024-09-10T10:43:46.525130+0000","last_became_peered":"2024-09-10T10:43:46.525130+0000","last_unstale":"2024-09-10T10:43:52.869413+0000","last_undegraded":"2024-09-10T10:43:52.869413+0000","last_fullsized":"2024-09-10T10:43:52.869413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709094+0000","last_change":"2024-09-10T10:43:51.582330+0000","last_active":"2024-09-10T10:43:52.709094+0000","last_peered":"2024-09-10T10:43:52.709094+0000","last_clean":"2024-09-10T10:43:52.709094+0000","last_became_active":"2024-09-10T10:43:51.581415+0000","last_became_peered":"2024-09-10T10:43:51.581415+0000","last_unstale":"2024-09-10T10:43:52.709094+0000","last_undegraded":"2024-09-10T10:43:52.709094+0000","last_fullsized":"2024-09-10T10:43:52.709094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:18.520413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564097+0000","last_change":"2024-09-10T10:43:47.537959+0000","last_active":"2024-09-10T10:43:52.564097+0000","last_peered":"2024-09-10T10:43:52.564097+0000","last_clean":"2024-09-10T10:43:52.564097+0000","last_became_active":"2024-09-10T10:43:47.537705+0000","last_became_peered":"2024-09-10T10:43:47.537705+0000","last_unstale":"2024-09-10T10:43:52.564097+0000","last_undegraded":"2024-09-10T10:43:52.564097+0000","last_fullsized":"2024-09-10T10:43:52.564097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.301556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869080+0000","last_change":"2024-09-10T10:43:46.539973+0000","last_active":"2024-09-10T10:43:52.869080+0000","last_peered":"2024-09-10T10:43:52.869080+0000","last_clean":"2024-09-10T10:43:52.869080+0000","last_became_active":"2024-09-10T10:43:46.539509+0000","last_became_peered":"2024-09-10T10:43:46.539509+0000","last_unstale":"2024-09-10T10:43:52.869080+0000","last_undegraded":"2024-09-10T10:43:52.869080+0000","last_fullsized":"2024-09-10T10:43:52.869080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:51.584701+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:51.584321+0000","last_became_peered":"2024-09-10T10:43:51.584321+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:07.697012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869811+0000","last_change":"2024-09-10T10:43:47.549457+0000","last_active":"2024-09-10T10:43:52.869811+0000","last_peered":"2024-09-10T10:43:52.869811+0000","last_clean":"2024-09-10T10:43:52.869811+0000","last_became_active":"2024-09-10T10:43:47.549210+0000","last_became_peered":"2024-09-10T10:43:47.549210+0000","last_unstale":"2024-09-10T10:43:52.869811+0000","last_undegraded":"2024-09-10T10:43:52.869811+0000","last_fullsized":"2024-09-10T10:43:52.869811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:00.039891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869773+0000","last_change":"2024-09-10T10:43:46.520041+0000","last_active":"2024-09-10T10:43:52.869773+0000","last_peered":"2024-09-10T10:43:52.869773+0000","last_clean":"2024-09-10T10:43:52.869773+0000","last_became_active":"2024-09-10T10:43:46.519721+0000","last_became_peered":"2024-09-10T10:43:46.519721+0000","last_unstale":"2024-09-10T10:43:52.869773+0000","last_undegraded":"2024-09-10T10:43:52.869773+0000","last_fullsized":"2024-09-10T10:43:52.869773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:23.802544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870586+0000","last_change":"2024-09-10T10:43:51.571179+0000","last_active":"2024-09-10T10:43:52.870586+0000","last_peered":"2024-09-10T10:43:52.870586+0000","last_clean":"2024-09-10T10:43:52.870586+0000","last_became_active":"2024-09-10T10:43:51.570590+0000","last_became_peered":"2024-09-10T10:43:51.570590+0000","last_unstale":"2024-09-10T10:43:52.870586+0000","last_undegraded":"2024-09-10T10:43:52.870586+0000","last_fullsized":"2024-09-10T10:43:52.870586+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870110+0000","last_change":"2024-09-10T10:43:47.546795+0000","last_active":"2024-09-10T10:43:52.870110+0000","last_peered":"2024-09-10T10:43:52.870110+0000","last_clean":"2024-09-10T10:43:52.870110+0000","last_became_active":"2024-09-10T10:43:47.546529+0000","last_became_peered":"2024-09-10T10:43:47.546529+0000","last_unstale":"2024-09-10T10:43:52.870110+0000","last_undegraded":"2024-09-10T10:43:52.870110+0000","last_fullsized":"2024-09-10T10:43:52.870110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564317+0000","last_change":"2024-09-10T10:43:46.524422+0000","last_active":"2024-09-10T10:43:52.564317+0000","last_peered":"2024-09-10T10:43:52.564317+0000","last_clean":"2024-09-10T10:43:52.564317+0000","last_became_active":"2024-09-10T10:43:46.524027+0000","last_became_peered":"2024-09-10T10:43:46.524027+0000","last_unstale":"2024-09-10T10:43:52.564317+0000","last_undegraded":"2024-09-10T10:43:52.564317+0000","last_fullsized":"2024-09-10T10:43:52.564317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.575001+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.573805+0000","last_became_peered":"2024-09-10T10:43:51.573805+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:27.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869516+0000","last_change":"2024-09-10T10:43:47.542333+0000","last_active":"2024-09-10T10:43:52.869516+0000","last_peered":"2024-09-10T10:43:52.869516+0000","last_clean":"2024-09-10T10:43:52.869516+0000","last_became_active":"2024-09-10T10:43:47.540584+0000","last_became_peered":"2024-09-10T10:43:47.540584+0000","last_unstale":"2024-09-10T10:43:52.869516+0000","last_undegraded":"2024-09-10T10:43:52.869516+0000","last_fullsized":"2024-09-10T10:43:52.869516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:08.456485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:58.645779+0000","last_change":"2024-09-10T10:43:46.531547+0000","last_active":"2024-09-10T10:43:58.645779+0000","last_peered":"2024-09-10T10:43:58.645779+0000","last_clean":"2024-09-10T10:43:58.645779+0000","last_became_active":"2024-09-10T10:43:46.530309+0000","last_became_peered":"2024-09-10T10:43:46.530309+0000","last_unstale":"2024-09-10T10:43:58.645779+0000","last_undegraded":"2024-09-10T10:43:58.645779+0000","last_fullsized":"2024-09-10T10:43:58.645779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:54.587795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869118+0000","last_change":"2024-09-10T10:43:51.578360+0000","last_active":"2024-09-10T10:43:52.869118+0000","last_peered":"2024-09-10T10:43:52.869118+0000","last_clean":"2024-09-10T10:43:52.869118+0000","last_became_active":"2024-09-10T10:43:51.578021+0000","last_became_peered":"2024-09-10T10:43:51.578021+0000","last_unstale":"2024-09-10T10:43:52.869118+0000","last_undegraded":"2024-09-10T10:43:52.869118+0000","last_fullsized":"2024-09-10T10:43:52.869118+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870009+0000","last_change":"2024-09-10T10:43:47.564353+0000","last_active":"2024-09-10T10:43:52.870009+0000","last_peered":"2024-09-10T10:43:52.870009+0000","last_clean":"2024-09-10T10:43:52.870009+0000","last_became_active":"2024-09-10T10:43:47.563999+0000","last_became_peered":"2024-09-10T10:43:47.563999+0000","last_unstale":"2024-09-10T10:43:52.870009+0000","last_undegraded":"2024-09-10T10:43:52.870009+0000","last_fullsized":"2024-09-10T10:43:52.870009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:52.338110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564851+0000","last_change":"2024-09-10T10:43:46.526185+0000","last_active":"2024-09-10T10:43:52.564851+0000","last_peered":"2024-09-10T10:43:52.564851+0000","last_clean":"2024-09-10T10:43:52.564851+0000","last_became_active":"2024-09-10T10:43:46.525887+0000","last_became_peered":"2024-09-10T10:43:46.525887+0000","last_unstale":"2024-09-10T10:43:52.564851+0000","last_undegraded":"2024-09-10T10:43:52.564851+0000","last_fullsized":"2024-09-10T10:43:52.564851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563403+0000","last_change":"2024-09-10T10:43:51.587988+0000","last_active":"2024-09-10T10:43:52.563403+0000","last_peered":"2024-09-10T10:43:52.563403+0000","last_clean":"2024-09-10T10:43:52.563403+0000","last_became_active":"2024-09-10T10:43:51.587584+0000","last_became_peered":"2024-09-10T10:43:51.587584+0000","last_unstale":"2024-09-10T10:43:52.563403+0000","last_undegraded":"2024-09-10T10:43:52.563403+0000","last_fullsized":"2024-09-10T10:43:52.563403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708986+0000","last_change":"2024-09-10T10:43:47.541815+0000","last_active":"2024-09-10T10:43:52.708986+0000","last_peered":"2024-09-10T10:43:52.708986+0000","last_clean":"2024-09-10T10:43:52.708986+0000","last_became_active":"2024-09-10T10:43:47.541530+0000","last_became_peered":"2024-09-10T10:43:47.541530+0000","last_unstale":"2024-09-10T10:43:52.708986+0000","last_undegraded":"2024-09-10T10:43:52.708986+0000","last_fullsized":"2024-09-10T10:43:52.708986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:26.960082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869496+0000","last_change":"2024-09-10T10:43:46.525990+0000","last_active":"2024-09-10T10:43:52.869496+0000","last_peered":"2024-09-10T10:43:52.869496+0000","last_clean":"2024-09-10T10:43:52.869496+0000","last_became_active":"2024-09-10T10:43:46.524901+0000","last_became_peered":"2024-09-10T10:43:46.524901+0000","last_unstale":"2024-09-10T10:43:52.869496+0000","last_undegraded":"2024-09-10T10:43:52.869496+0000","last_fullsized":"2024-09-10T10:43:52.869496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:12.069623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563676+0000","last_change":"2024-09-10T10:43:51.574109+0000","last_active":"2024-09-10T10:43:52.563676+0000","last_peered":"2024-09-10T10:43:52.563676+0000","last_clean":"2024-09-10T10:43:52.563676+0000","last_became_active":"2024-09-10T10:43:51.573718+0000","last_became_peered":"2024-09-10T10:43:51.573718+0000","last_unstale":"2024-09-10T10:43:52.563676+0000","last_undegraded":"2024-09-10T10:43:52.563676+0000","last_fullsized":"2024-09-10T10:43:52.563676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:13.623233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564443+0000","last_change":"2024-09-10T10:43:47.558084+0000","last_active":"2024-09-10T10:43:52.564443+0000","last_peered":"2024-09-10T10:43:52.564443+0000","last_clean":"2024-09-10T10:43:52.564443+0000","last_became_active":"2024-09-10T10:43:47.555632+0000","last_became_peered":"2024-09-10T10:43:47.555632+0000","last_unstale":"2024-09-10T10:43:52.564443+0000","last_undegraded":"2024-09-10T10:43:52.564443+0000","last_fullsized":"2024-09-10T10:43:52.564443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565392+0000","last_change":"2024-09-10T10:43:46.528292+0000","last_active":"2024-09-10T10:43:52.565392+0000","last_peered":"2024-09-10T10:43:52.565392+0000","last_clean":"2024-09-10T10:43:52.565392+0000","last_became_active":"2024-09-10T10:43:46.527943+0000","last_became_peered":"2024-09-10T10:43:46.527943+0000","last_unstale":"2024-09-10T10:43:52.565392+0000","last_undegraded":"2024-09-10T10:43:52.565392+0000","last_fullsized":"2024-09-10T10:43:52.565392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869189+0000","last_change":"2024-09-10T10:43:51.572294+0000","last_active":"2024-09-10T10:43:52.869189+0000","last_peered":"2024-09-10T10:43:52.869189+0000","last_clean":"2024-09-10T10:43:52.869189+0000","last_became_active":"2024-09-10T10:43:51.571796+0000","last_became_peered":"2024-09-10T10:43:51.571796+0000","last_unstale":"2024-09-10T10:43:52.869189+0000","last_undegraded":"2024-09-10T10:43:52.869189+0000","last_fullsized":"2024-09-10T10:43:52.869189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869838+0000","last_change":"2024-09-10T10:43:51.577672+0000","last_active":"2024-09-10T10:43:52.869838+0000","last_peered":"2024-09-10T10:43:52.869838+0000","last_clean":"2024-09-10T10:43:52.869838+0000","last_became_active":"2024-09-10T10:43:51.577342+0000","last_became_peered":"2024-09-10T10:43:51.577342+0000","last_unstale":"2024-09-10T10:43:52.869838+0000","last_undegraded":"2024-09-10T10:43:52.869838+0000","last_fullsized":"2024-09-10T10:43:52.869838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:59.764337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869129+0000","last_change":"2024-09-10T10:43:47.538181+0000","last_active":"2024-09-10T10:43:52.869129+0000","last_peered":"2024-09-10T10:43:52.869129+0000","last_clean":"2024-09-10T10:43:52.869129+0000","last_became_active":"2024-09-10T10:43:47.537810+0000","last_became_peered":"2024-09-10T10:43:47.537810+0000","last_unstale":"2024-09-10T10:43:52.869129+0000","last_undegraded":"2024-09-10T10:43:52.869129+0000","last_fullsized":"2024-09-10T10:43:52.869129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868780+0000","last_change":"2024-09-10T10:43:46.533440+0000","last_active":"2024-09-10T10:43:52.868780+0000","last_peered":"2024-09-10T10:43:52.868780+0000","last_clean":"2024-09-10T10:43:52.868780+0000","last_became_active":"2024-09-10T10:43:46.532814+0000","last_became_peered":"2024-09-10T10:43:46.532814+0000","last_unstale":"2024-09-10T10:43:52.868780+0000","last_undegraded":"2024-09-10T10:43:52.868780+0000","last_fullsized":"2024-09-10T10:43:52.868780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:03.561914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870537+0000","last_change":"2024-09-10T10:43:51.576277+0000","last_active":"2024-09-10T10:43:52.870537+0000","last_peered":"2024-09-10T10:43:52.870537+0000","last_clean":"2024-09-10T10:43:52.870537+0000","last_became_active":"2024-09-10T10:43:51.575888+0000","last_became_peered":"2024-09-10T10:43:51.575888+0000","last_unstale":"2024-09-10T10:43:52.870537+0000","last_undegraded":"2024-09-10T10:43:52.870537+0000","last_fullsized":"2024-09-10T10:43:52.870537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:37.271562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708364+0000","last_change":"2024-09-10T10:43:47.539801+0000","last_active":"2024-09-10T10:43:52.708364+0000","last_peered":"2024-09-10T10:43:52.708364+0000","last_clean":"2024-09-10T10:43:52.708364+0000","last_became_active":"2024-09-10T10:43:47.535922+0000","last_became_peered":"2024-09-10T10:43:47.535922+0000","last_unstale":"2024-09-10T10:43:52.708364+0000","last_undegraded":"2024-09-10T10:43:52.708364+0000","last_fullsized":"2024-09-10T10:43:52.708364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:49.905762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869706+0000","last_change":"2024-09-10T10:43:46.530425+0000","last_active":"2024-09-10T10:43:52.869706+0000","last_peered":"2024-09-10T10:43:52.869706+0000","last_clean":"2024-09-10T10:43:52.869706+0000","last_became_active":"2024-09-10T10:43:46.530158+0000","last_became_peered":"2024-09-10T10:43:46.530158+0000","last_unstale":"2024-09-10T10:43:52.869706+0000","last_undegraded":"2024-09-10T10:43:52.869706+0000","last_fullsized":"2024-09-10T10:43:52.869706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869653+0000","last_change":"2024-09-10T10:43:46.537743+0000","last_active":"2024-09-10T10:43:52.869653+0000","last_peered":"2024-09-10T10:43:52.869653+0000","last_clean":"2024-09-10T10:43:52.869653+0000","last_became_active":"2024-09-10T10:43:46.537260+0000","last_became_peered":"2024-09-10T10:43:46.537260+0000","last_unstale":"2024-09-10T10:43:52.869653+0000","last_undegraded":"2024-09-10T10:43:52.869653+0000","last_fullsized":"2024-09-10T10:43:52.869653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:53.479099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869441+0000","last_change":"2024-09-10T10:43:47.545017+0000","last_active":"2024-09-10T10:43:52.869441+0000","last_peered":"2024-09-10T10:43:52.869441+0000","last_clean":"2024-09-10T10:43:52.869441+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.869441+0000","last_undegraded":"2024-09-10T10:43:52.869441+0000","last_fullsized":"2024-09-10T10:43:52.869441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:46.823226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869679+0000","last_change":"2024-09-10T10:43:51.570454+0000","last_active":"2024-09-10T10:43:52.869679+0000","last_peered":"2024-09-10T10:43:52.869679+0000","last_clean":"2024-09-10T10:43:52.869679+0000","last_became_active":"2024-09-10T10:43:51.570088+0000","last_became_peered":"2024-09-10T10:43:51.570088+0000","last_unstale":"2024-09-10T10:43:52.869679+0000","last_undegraded":"2024-09-10T10:43:52.869679+0000","last_fullsized":"2024-09-10T10:43:52.869679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:52.179305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708948+0000","last_change":"2024-09-10T10:43:46.521135+0000","last_active":"2024-09-10T10:43:52.708948+0000","last_peered":"2024-09-10T10:43:52.708948+0000","last_clean":"2024-09-10T10:43:52.708948+0000","last_became_active":"2024-09-10T10:43:46.520815+0000","last_became_peered":"2024-09-10T10:43:46.520815+0000","last_unstale":"2024-09-10T10:43:52.708948+0000","last_undegraded":"2024-09-10T10:43:52.708948+0000","last_fullsized":"2024-09-10T10:43:52.708948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:38.222213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564462+0000","last_change":"2024-09-10T10:43:47.545783+0000","last_active":"2024-09-10T10:43:52.564462+0000","last_peered":"2024-09-10T10:43:52.564462+0000","last_clean":"2024-09-10T10:43:52.564462+0000","last_became_active":"2024-09-10T10:43:47.545265+0000","last_became_peered":"2024-09-10T10:43:47.545265+0000","last_unstale":"2024-09-10T10:43:52.564462+0000","last_undegraded":"2024-09-10T10:43:52.564462+0000","last_fullsized":"2024-09-10T10:43:52.564462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564298+0000","last_change":"2024-09-10T10:43:51.585486+0000","last_active":"2024-09-10T10:43:52.564298+0000","last_peered":"2024-09-10T10:43:52.564298+0000","last_clean":"2024-09-10T10:43:52.564298+0000","last_became_active":"2024-09-10T10:43:51.585045+0000","last_became_peered":"2024-09-10T10:43:51.585045+0000","last_unstale":"2024-09-10T10:43:52.564298+0000","last_undegraded":"2024-09-10T10:43:52.564298+0000","last_fullsized":"2024-09-10T10:43:52.564298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:59.749737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563314+0000","last_change":"2024-09-10T10:43:47.533694+0000","last_active":"2024-09-10T10:43:52.563314+0000","last_peered":"2024-09-10T10:43:52.563314+0000","last_clean":"2024-09-10T10:43:52.563314+0000","last_became_active":"2024-09-10T10:43:47.533197+0000","last_became_peered":"2024-09-10T10:43:47.533197+0000","last_unstale":"2024-09-10T10:43:52.563314+0000","last_undegraded":"2024-09-10T10:43:52.563314+0000","last_fullsized":"2024-09-10T10:43:52.563314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563761+0000","last_change":"2024-09-10T10:43:47.540401+0000","last_active":"2024-09-10T10:43:52.563761+0000","last_peered":"2024-09-10T10:43:52.563761+0000","last_clean":"2024-09-10T10:43:52.563761+0000","last_became_active":"2024-09-10T10:43:47.539677+0000","last_became_peered":"2024-09-10T10:43:47.539677+0000","last_unstale":"2024-09-10T10:43:52.563761+0000","last_undegraded":"2024-09-10T10:43:52.563761+0000","last_fullsized":"2024-09-10T10:43:52.563761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868918+0000","last_change":"2024-09-10T10:43:47.539352+0000","last_active":"2024-09-10T10:43:52.868918+0000","last_peered":"2024-09-10T10:43:52.868918+0000","last_clean":"2024-09-10T10:43:52.868918+0000","last_became_active":"2024-09-10T10:43:47.539034+0000","last_became_peered":"2024-09-10T10:43:47.539034+0000","last_unstale":"2024-09-10T10:43:52.868918+0000","last_undegraded":"2024-09-10T10:43:52.868918+0000","last_fullsized":"2024-09-10T10:43:52.868918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:08.107825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708757+0000","last_change":"2024-09-10T10:43:47.542199+0000","last_active":"2024-09-10T10:43:52.708757+0000","last_peered":"2024-09-10T10:43:52.708757+0000","last_clean":"2024-09-10T10:43:52.708757+0000","last_became_active":"2024-09-10T10:43:47.535131+0000","last_became_peered":"2024-09-10T10:43:47.535131+0000","last_unstale":"2024-09-10T10:43:52.708757+0000","last_undegraded":"2024-09-10T10:43:52.708757+0000","last_fullsized":"2024-09-10T10:43:52.708757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:38.966819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870705+0000","last_change":"2024-09-10T10:43:47.546290+0000","last_active":"2024-09-10T10:43:52.870705+0000","last_peered":"2024-09-10T10:43:52.870705+0000","last_clean":"2024-09-10T10:43:52.870705+0000","last_became_active":"2024-09-10T10:43:47.545873+0000","last_became_peered":"2024-09-10T10:43:47.545873+0000","last_unstale":"2024-09-10T10:43:52.870705+0000","last_undegraded":"2024-09-10T10:43:52.870705+0000","last_fullsized":"2024-09-10T10:43:52.870705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870373+0000","last_change":"2024-09-10T10:43:47.537092+0000","last_active":"2024-09-10T10:43:52.870373+0000","last_peered":"2024-09-10T10:43:52.870373+0000","last_clean":"2024-09-10T10:43:52.870373+0000","last_became_active":"2024-09-10T10:43:47.536730+0000","last_became_peered":"2024-09-10T10:43:47.536730+0000","last_unstale":"2024-09-10T10:43:52.870373+0000","last_undegraded":"2024-09-10T10:43:52.870373+0000","last_fullsized":"2024-09-10T10:43:52.870373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564347+0000","last_change":"2024-09-10T10:43:47.557748+0000","last_active":"2024-09-10T10:43:52.564347+0000","last_peered":"2024-09-10T10:43:52.564347+0000","last_clean":"2024-09-10T10:43:52.564347+0000","last_became_active":"2024-09-10T10:43:47.555551+0000","last_became_peered":"2024-09-10T10:43:47.555551+0000","last_unstale":"2024-09-10T10:43:52.564347+0000","last_undegraded":"2024-09-10T10:43:52.564347+0000","last_fullsized":"2024-09-10T10:43:52.564347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708691+0000","last_change":"2024-09-10T10:43:47.544639+0000","last_active":"2024-09-10T10:43:52.708691+0000","last_peered":"2024-09-10T10:43:52.708691+0000","last_clean":"2024-09-10T10:43:52.708691+0000","last_became_active":"2024-09-10T10:43:47.544357+0000","last_became_peered":"2024-09-10T10:43:47.544357+0000","last_unstale":"2024-09-10T10:43:52.708691+0000","last_undegraded":"2024-09-10T10:43:52.708691+0000","last_fullsized":"2024-09-10T10:43:52.708691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:16.174872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868699+0000","last_change":"2024-09-10T10:43:47.544034+0000","last_active":"2024-09-10T10:43:52.868699+0000","last_peered":"2024-09-10T10:43:52.868699+0000","last_clean":"2024-09-10T10:43:52.868699+0000","last_became_active":"2024-09-10T10:43:47.543677+0000","last_became_peered":"2024-09-10T10:43:47.543677+0000","last_unstale":"2024-09-10T10:43:52.868699+0000","last_undegraded":"2024-09-10T10:43:52.868699+0000","last_fullsized":"2024-09-10T10:43:52.868699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:26.220907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708358+0000","last_change":"2024-09-10T10:43:47.561856+0000","last_active":"2024-09-10T10:43:52.708358+0000","last_peered":"2024-09-10T10:43:52.708358+0000","last_clean":"2024-09-10T10:43:52.708358+0000","last_became_active":"2024-09-10T10:43:47.561518+0000","last_became_peered":"2024-09-10T10:43:47.561518+0000","last_unstale":"2024-09-10T10:43:52.708358+0000","last_undegraded":"2024-09-10T10:43:52.708358+0000","last_fullsized":"2024-09-10T10:43:52.708358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:21.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:47.545540+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:47.544916+0000","last_became_peered":"2024-09-10T10:43:47.544916+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869156+0000","last_change":"2024-09-10T10:43:47.521227+0000","last_active":"2024-09-10T10:43:52.869156+0000","last_peered":"2024-09-10T10:43:52.869156+0000","last_clean":"2024-09-10T10:43:52.869156+0000","last_became_active":"2024-09-10T10:43:47.520811+0000","last_became_peered":"2024-09-10T10:43:47.520811+0000","last_unstale":"2024-09-10T10:43:52.869156+0000","last_undegraded":"2024-09-10T10:43:52.869156+0000","last_fullsized":"2024-09-10T10:43:52.869156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:55.708127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869588+0000","last_change":"2024-09-10T10:43:47.541681+0000","last_active":"2024-09-10T10:43:52.869588+0000","last_peered":"2024-09-10T10:43:52.869588+0000","last_clean":"2024-09-10T10:43:52.869588+0000","last_became_active":"2024-09-10T10:43:47.541056+0000","last_became_peered":"2024-09-10T10:43:47.541056+0000","last_unstale":"2024-09-10T10:43:52.869588+0000","last_undegraded":"2024-09-10T10:43:52.869588+0000","last_fullsized":"2024-09-10T10:43:52.869588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:56.224306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870640+0000","last_change":"2024-09-10T10:43:47.543423+0000","last_active":"2024-09-10T10:43:52.870640+0000","last_peered":"2024-09-10T10:43:52.870640+0000","last_clean":"2024-09-10T10:43:52.870640+0000","last_became_active":"2024-09-10T10:43:47.537728+0000","last_became_peered":"2024-09-10T10:43:47.537728+0000","last_unstale":"2024-09-10T10:43:52.870640+0000","last_undegraded":"2024-09-10T10:43:52.870640+0000","last_fullsized":"2024-09-10T10:43:52.870640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:10.407503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564680+0000","last_change":"2024-09-10T10:43:47.562659+0000","last_active":"2024-09-10T10:43:52.564680+0000","last_peered":"2024-09-10T10:43:52.564680+0000","last_clean":"2024-09-10T10:43:52.564680+0000","last_became_active":"2024-09-10T10:43:47.562238+0000","last_became_peered":"2024-09-10T10:43:47.562238+0000","last_unstale":"2024-09-10T10:43:52.564680+0000","last_undegraded":"2024-09-10T10:43:52.564680+0000","last_fullsized":"2024-09-10T10:43:52.564680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870710+0000","last_change":"2024-09-10T10:43:47.548547+0000","last_active":"2024-09-10T10:43:52.870710+0000","last_peered":"2024-09-10T10:43:52.870710+0000","last_clean":"2024-09-10T10:43:52.870710+0000","last_became_active":"2024-09-10T10:43:47.548237+0000","last_became_peered":"2024-09-10T10:43:47.548237+0000","last_unstale":"2024-09-10T10:43:52.870710+0000","last_undegraded":"2024-09-10T10:43:52.870710+0000","last_fullsized":"2024-09-10T10:43:52.870710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:55.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868674+0000","last_change":"2024-09-10T10:43:47.559813+0000","last_active":"2024-09-10T10:43:52.868674+0000","last_peered":"2024-09-10T10:43:52.868674+0000","last_clean":"2024-09-10T10:43:52.868674+0000","last_became_active":"2024-09-10T10:43:47.559517+0000","last_became_peered":"2024-09-10T10:43:47.559517+0000","last_unstale":"2024-09-10T10:43:52.868674+0000","last_undegraded":"2024-09-10T10:43:52.868674+0000","last_fullsized":"2024-09-10T10:43:52.868674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:28.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.546737+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.546456+0000","last_became_peered":"2024-09-10T10:43:47.546456+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:42.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870025+0000","last_change":"2024-09-10T10:43:47.533339+0000","last_active":"2024-09-10T10:43:52.870025+0000","last_peered":"2024-09-10T10:43:52.870025+0000","last_clean":"2024-09-10T10:43:52.870025+0000","last_became_active":"2024-09-10T10:43:47.533019+0000","last_became_peered":"2024-09-10T10:43:47.533019+0000","last_unstale":"2024-09-10T10:43:52.870025+0000","last_undegraded":"2024-09-10T10:43:52.870025+0000","last_fullsized":"2024-09-10T10:43:52.870025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870439+0000","last_change":"2024-09-10T10:43:47.546293+0000","last_active":"2024-09-10T10:43:52.870439+0000","last_peered":"2024-09-10T10:43:52.870439+0000","last_clean":"2024-09-10T10:43:52.870439+0000","last_became_active":"2024-09-10T10:43:47.545952+0000","last_became_peered":"2024-09-10T10:43:47.545952+0000","last_unstale":"2024-09-10T10:43:52.870439+0000","last_undegraded":"2024-09-10T10:43:52.870439+0000","last_fullsized":"2024-09-10T10:43:52.870439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869535+0000","last_change":"2024-09-10T10:43:47.541403+0000","last_active":"2024-09-10T10:43:52.869535+0000","last_peered":"2024-09-10T10:43:52.869535+0000","last_clean":"2024-09-10T10:43:52.869535+0000","last_became_active":"2024-09-10T10:43:47.540724+0000","last_became_peered":"2024-09-10T10:43:47.540724+0000","last_unstale":"2024-09-10T10:43:52.869535+0000","last_undegraded":"2024-09-10T10:43:52.869535+0000","last_fullsized":"2024-09-10T10:43:52.869535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:55.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869473+0000","last_change":"2024-09-10T10:43:47.550017+0000","last_active":"2024-09-10T10:43:52.869473+0000","last_peered":"2024-09-10T10:43:52.869473+0000","last_clean":"2024-09-10T10:43:52.869473+0000","last_became_active":"2024-09-10T10:43:47.549763+0000","last_became_peered":"2024-09-10T10:43:47.549763+0000","last_unstale":"2024-09-10T10:43:52.869473+0000","last_undegraded":"2024-09-10T10:43:52.869473+0000","last_fullsized":"2024-09-10T10:43:52.869473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:12.076487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869399+0000","last_change":"2024-09-10T10:43:47.558985+0000","last_active":"2024-09-10T10:43:52.869399+0000","last_peered":"2024-09-10T10:43:52.869399+0000","last_clean":"2024-09-10T10:43:52.869399+0000","last_became_active":"2024-09-10T10:43:47.558695+0000","last_became_peered":"2024-09-10T10:43:47.558695+0000","last_unstale":"2024-09-10T10:43:52.869399+0000","last_undegraded":"2024-09-10T10:43:52.869399+0000","last_fullsized":"2024-09-10T10:43:52.869399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:05.319579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565366+0000","last_change":"2024-09-10T10:43:47.547413+0000","last_active":"2024-09-10T10:43:52.565366+0000","last_peered":"2024-09-10T10:43:52.565366+0000","last_clean":"2024-09-10T10:43:52.565366+0000","last_became_active":"2024-09-10T10:43:47.547008+0000","last_became_peered":"2024-09-10T10:43:47.547008+0000","last_unstale":"2024-09-10T10:43:52.565366+0000","last_undegraded":"2024-09-10T10:43:52.565366+0000","last_fullsized":"2024-09-10T10:43:52.565366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.545409+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.545121+0000","last_became_peered":"2024-09-10T10:43:47.545121+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.641844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869042+0000","last_change":"2024-09-10T10:43:47.534510+0000","last_active":"2024-09-10T10:43:52.869042+0000","last_peered":"2024-09-10T10:43:52.869042+0000","last_clean":"2024-09-10T10:43:52.869042+0000","last_became_active":"2024-09-10T10:43:47.534143+0000","last_became_peered":"2024-09-10T10:43:47.534143+0000","last_unstale":"2024-09-10T10:43:52.869042+0000","last_undegraded":"2024-09-10T10:43:52.869042+0000","last_fullsized":"2024-09-10T10:43:52.869042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:33.210661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563579+0000","last_change":"2024-09-10T10:43:47.525750+0000","last_active":"2024-09-10T10:43:52.563579+0000","last_peered":"2024-09-10T10:43:52.563579+0000","last_clean":"2024-09-10T10:43:52.563579+0000","last_became_active":"2024-09-10T10:43:47.525379+0000","last_became_peered":"2024-09-10T10:43:47.525379+0000","last_unstale":"2024-09-10T10:43:52.563579+0000","last_undegraded":"2024-09-10T10:43:52.563579+0000","last_fullsized":"2024-09-10T10:43:52.563579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869264+0000","last_change":"2024-09-10T10:43:47.546839+0000","last_active":"2024-09-10T10:43:52.869264+0000","last_peered":"2024-09-10T10:43:52.869264+0000","last_clean":"2024-09-10T10:43:52.869264+0000","last_became_active":"2024-09-10T10:43:47.544570+0000","last_became_peered":"2024-09-10T10:43:47.544570+0000","last_unstale":"2024-09-10T10:43:52.869264+0000","last_undegraded":"2024-09-10T10:43:52.869264+0000","last_fullsized":"2024-09-10T10:43:52.869264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:53.642976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564645+0000","last_change":"2024-09-10T10:43:47.522224+0000","last_active":"2024-09-10T10:43:52.564645+0000","last_peered":"2024-09-10T10:43:52.564645+0000","last_clean":"2024-09-10T10:43:52.564645+0000","last_became_active":"2024-09-10T10:43:47.521820+0000","last_became_peered":"2024-09-10T10:43:47.521820+0000","last_unstale":"2024-09-10T10:43:52.564645+0000","last_undegraded":"2024-09-10T10:43:52.564645+0000","last_fullsized":"2024-09-10T10:43:52.564645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:47.538393+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:47.537972+0000","last_became_peered":"2024-09-10T10:43:47.537972+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:30.153548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708731+0000","last_change":"2024-09-10T10:43:47.540588+0000","last_active":"2024-09-10T10:43:52.708731+0000","last_peered":"2024-09-10T10:43:52.708731+0000","last_clean":"2024-09-10T10:43:52.708731+0000","last_became_active":"2024-09-10T10:43:47.540358+0000","last_became_peered":"2024-09-10T10:43:47.540358+0000","last_unstale":"2024-09-10T10:43:52.708731+0000","last_undegraded":"2024-09-10T10:43:52.708731+0000","last_fullsized":"2024-09-10T10:43:52.708731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:30.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:47.549392+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:47.546137+0000","last_became_peered":"2024-09-10T10:43:47.546137+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:02.975066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":111561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":111437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":115281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":701875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":111493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":884736,"data_stored":701897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:13.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:44:13.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.342+0000 7fa47cc00640 1 Processor -- start 2024-09-10T10:44:13.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.342+0000 7fa47cc00640 1 -- start start 2024-09-10T10:44:13.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.343+0000 7fa47cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa47815b300 0x7fa4780cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa47cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4780cd3f0 0x7fa47806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa47cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780cdbe0 con 0x7fa478072d30 2024-09-10T10:44:13.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa47cc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa4780728a0 con 0x7fa4780cd3f0 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa47cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780cd900 con 0x7fa47815b300 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4780cd3f0 0x7fa47806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4780cd3f0 0x7fa47806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36750/0 (socket says 172.21.15.28:36750) 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 -- 172.21.15.28:0/779550009 learned_addr learned my addr 172.21.15.28:0/779550009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa475e00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa47815b300 0x7fa4780cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 -- 172.21.15.28:0/779550009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa47815b300 msgr2=0x7fa4780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa47815b300 0x7fa4780cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 -- 172.21.15.28:0/779550009 >> v1:172.21.15.138:6789/0 conn(0x7fa478072d30 legacy=0x7fa4780b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.344+0000 7fa46fe00640 1 -- 172.21.15.28:0/779550009 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa478072250 con 0x7fa4780cd3f0 2024-09-10T10:44:13.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.345+0000 7fa46fe00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4780cd3f0 0x7fa47806f1c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa460009870 tx=0x7fa46002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79dd966da35ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa475400640 1 -- 172.21.15.28:0/779550009 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460037070 con 0x7fa4780cd3f0 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa475400640 1 -- 172.21.15.28:0/779550009 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa460032440 con 0x7fa4780cd3f0 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa475400640 1 -- 172.21.15.28:0/779550009 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460032810 con 0x7fa4780cd3f0 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 -- 172.21.15.28:0/779550009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4780cd3f0 msgr2=0x7fa47806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4780cd3f0 0x7fa47806f1c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa460009870 tx=0x7fa46002a270 comp rx=0 tx=0).stop 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 -- 172.21.15.28:0/779550009 shutdown_connections 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4780cd3f0 0x7fa47806f1c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 --2- 172.21.15.28:0/779550009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa47815b300 0x7fa4780cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 -- 172.21.15.28:0/779550009 >> 172.21.15.28:0/779550009 conn(0x7fa478067d70 msgr2=0x7fa47806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 -- 172.21.15.28:0/779550009 shutdown_connections 2024-09-10T10:44:13.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.346+0000 7fa47cc00640 1 -- 172.21.15.28:0/779550009 wait complete. 2024-09-10T10:44:13.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.348+0000 7fa47cc00640 1 Processor -- start 2024-09-10T10:44:13.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.348+0000 7fa47cc00640 1 -- start start 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa478072d30 0x7fa4781598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4780cd3f0 0x7fa47815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780d0160 con 0x7fa47815b300 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780cffe0 con 0x7fa4780cd3f0 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa47cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa47806f700 con 0x7fa478072d30 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.349+0000 7fa476800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36758/0 (socket says 172.21.15.28:36758) 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 -- 172.21.15.28:0/823573340 learned_addr learned my addr 172.21.15.28:0/823573340 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa475e00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa478072d30 0x7fa4781598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa46fe00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4780cd3f0 0x7fa47815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 -- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa478072d30 msgr2=0x7fa4781598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa478072d30 0x7fa4781598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 -- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4780cd3f0 msgr2=0x7fa47815d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4780cd3f0 0x7fa47815d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 -- 172.21.15.28:0/823573340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781e7820 con 0x7fa47815b300 2024-09-10T10:44:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa476800640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c7a0 tx=0x7fa47000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.350+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470003b70 con 0x7fa47815b300 2024-09-10T10:44:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.351+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470003d30 con 0x7fa47815b300 2024-09-10T10:44:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.351+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781e7b00 con 0x7fa47815b300 2024-09-10T10:44:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.351+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000aa90 con 0x7fa47815b300 2024-09-10T10:44:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.351+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781e83a0 con 0x7fa47815b300 2024-09-10T10:44:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.352+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47806eef0 con 0x7fa47815b300 2024-09-10T10:44:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.353+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa47000ac30 con 0x7fa47815b300 2024-09-10T10:44:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.354+0000 7fa46e000640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 0x7fa45406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.354+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa47008aff0 con 0x7fa47815b300 2024-09-10T10:44:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.354+0000 7fa475e00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 0x7fa45406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.355+0000 7fa475e00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 0x7fa45406e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa468009aa0 tx=0x7fa468009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.358+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa470058fb0 con 0x7fa47815b300 2024-09-10T10:44:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.537+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa47806ec10 con 0x7fa45406be60 2024-09-10T10:44:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.545+0000 7fa46e000640 1 -- 172.21.15.28:0/823573340 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7fa47806ec10 con 0x7fa45406be60 2024-09-10T10:44:13.550 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:13.551 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.549+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 msgr2=0x7fa45406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.549+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 0x7fa45406e320 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa468009aa0 tx=0x7fa468009190 comp rx=0 tx=0).stop 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 msgr2=0x7fa47814fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa47000c7a0 tx=0x7fa47000cc60 comp rx=0 tx=0).stop 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 shutdown_connections 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa45406be60 0x7fa45406e320 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa47815b300 0x7fa47814fc90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4780cd3f0 0x7fa47815d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 --2- 172.21.15.28:0/823573340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa478072d30 0x7fa4781598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 >> 172.21.15.28:0/823573340 conn(0x7fa478067d70 msgr2=0x7fa4780b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 shutdown_connections 2024-09-10T10:44:13.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.550+0000 7fa47cc00640 1 -- 172.21.15.28:0/823573340 wait complete. 2024-09-10T10:44:13.567 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-10T10:44:12.488407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223844,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750876,"statfs":{"total":773094113280,"available":772864897024,"internally_reserved":0,"allocated":3514368,"data_stored":2081516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870128+0000","last_change":"2024-09-10T10:43:46.542097+0000","last_active":"2024-09-10T10:43:52.870128+0000","last_peered":"2024-09-10T10:43:52.870128+0000","last_clean":"2024-09-10T10:43:52.870128+0000","last_became_active":"2024-09-10T10:43:46.541830+0000","last_became_peered":"2024-09-10T10:43:46.541830+0000","last_unstale":"2024-09-10T10:43:52.870128+0000","last_undegraded":"2024-09-10T10:43:52.870128+0000","last_fullsized":"2024-09-10T10:43:52.870128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709072+0000","last_change":"2024-09-10T10:43:47.561686+0000","last_active":"2024-09-10T10:43:52.709072+0000","last_peered":"2024-09-10T10:43:52.709072+0000","last_clean":"2024-09-10T10:43:52.709072+0000","last_became_active":"2024-09-10T10:43:47.561406+0000","last_became_peered":"2024-09-10T10:43:47.561406+0000","last_unstale":"2024-09-10T10:43:52.709072+0000","last_undegraded":"2024-09-10T10:43:52.709072+0000","last_fullsized":"2024-09-10T10:43:52.709072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:16.700092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563827+0000","last_change":"2024-09-10T10:43:51.575030+0000","last_active":"2024-09-10T10:43:52.563827+0000","last_peered":"2024-09-10T10:43:52.563827+0000","last_clean":"2024-09-10T10:43:52.563827+0000","last_became_active":"2024-09-10T10:43:51.570182+0000","last_became_peered":"2024-09-10T10:43:51.570182+0000","last_unstale":"2024-09-10T10:43:52.563827+0000","last_undegraded":"2024-09-10T10:43:52.563827+0000","last_fullsized":"2024-09-10T10:43:52.563827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:21.179072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709156+0000","last_change":"2024-09-10T10:43:46.528230+0000","last_active":"2024-09-10T10:43:52.709156+0000","last_peered":"2024-09-10T10:43:52.709156+0000","last_clean":"2024-09-10T10:43:52.709156+0000","last_became_active":"2024-09-10T10:43:46.527955+0000","last_became_peered":"2024-09-10T10:43:46.527955+0000","last_unstale":"2024-09-10T10:43:52.709156+0000","last_undegraded":"2024-09-10T10:43:52.709156+0000","last_fullsized":"2024-09-10T10:43:52.709156+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:23.630718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708899+0000","last_change":"2024-09-10T10:43:47.536706+0000","last_active":"2024-09-10T10:43:52.708899+0000","last_peered":"2024-09-10T10:43:52.708899+0000","last_clean":"2024-09-10T10:43:52.708899+0000","last_became_active":"2024-09-10T10:43:47.536347+0000","last_became_peered":"2024-09-10T10:43:47.536347+0000","last_unstale":"2024-09-10T10:43:52.708899+0000","last_undegraded":"2024-09-10T10:43:52.708899+0000","last_fullsized":"2024-09-10T10:43:52.708899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:19:57.758247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563851+0000","last_change":"2024-09-10T10:43:51.569149+0000","last_active":"2024-09-10T10:43:52.563851+0000","last_peered":"2024-09-10T10:43:52.563851+0000","last_clean":"2024-09-10T10:43:52.563851+0000","last_became_active":"2024-09-10T10:43:51.568748+0000","last_became_peered":"2024-09-10T10:43:51.568748+0000","last_unstale":"2024-09-10T10:43:52.563851+0000","last_undegraded":"2024-09-10T10:43:52.563851+0000","last_fullsized":"2024-09-10T10:43:52.563851+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869713+0000","last_change":"2024-09-10T10:43:46.525149+0000","last_active":"2024-09-10T10:43:52.869713+0000","last_peered":"2024-09-10T10:43:52.869713+0000","last_clean":"2024-09-10T10:43:52.869713+0000","last_became_active":"2024-09-10T10:43:46.524877+0000","last_became_peered":"2024-09-10T10:43:46.524877+0000","last_unstale":"2024-09-10T10:43:52.869713+0000","last_undegraded":"2024-09-10T10:43:52.869713+0000","last_fullsized":"2024-09-10T10:43:52.869713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:40.800800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564917+0000","last_change":"2024-09-10T10:43:47.547350+0000","last_active":"2024-09-10T10:43:52.564917+0000","last_peered":"2024-09-10T10:43:52.564917+0000","last_clean":"2024-09-10T10:43:52.564917+0000","last_became_active":"2024-09-10T10:43:47.547041+0000","last_became_peered":"2024-09-10T10:43:47.547041+0000","last_unstale":"2024-09-10T10:43:52.564917+0000","last_undegraded":"2024-09-10T10:43:52.564917+0000","last_fullsized":"2024-09-10T10:43:52.564917+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563358+0000","last_change":"2024-09-10T10:43:51.587632+0000","last_active":"2024-09-10T10:43:52.563358+0000","last_peered":"2024-09-10T10:43:52.563358+0000","last_clean":"2024-09-10T10:43:52.563358+0000","last_became_active":"2024-09-10T10:43:51.587202+0000","last_became_peered":"2024-09-10T10:43:51.587202+0000","last_unstale":"2024-09-10T10:43:52.563358+0000","last_undegraded":"2024-09-10T10:43:52.563358+0000","last_fullsized":"2024-09-10T10:43:52.563358+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:57.684586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869700+0000","last_change":"2024-09-10T10:43:46.529521+0000","last_active":"2024-09-10T10:43:52.869700+0000","last_peered":"2024-09-10T10:43:52.869700+0000","last_clean":"2024-09-10T10:43:52.869700+0000","last_became_active":"2024-09-10T10:43:46.529170+0000","last_became_peered":"2024-09-10T10:43:46.529170+0000","last_unstale":"2024-09-10T10:43:52.869700+0000","last_undegraded":"2024-09-10T10:43:52.869700+0000","last_fullsized":"2024-09-10T10:43:52.869700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:33.656083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870421+0000","last_change":"2024-09-10T10:43:47.550037+0000","last_active":"2024-09-10T10:43:52.870421+0000","last_peered":"2024-09-10T10:43:52.870421+0000","last_clean":"2024-09-10T10:43:52.870421+0000","last_became_active":"2024-09-10T10:43:47.549678+0000","last_became_peered":"2024-09-10T10:43:47.549678+0000","last_unstale":"2024-09-10T10:43:52.870421+0000","last_undegraded":"2024-09-10T10:43:52.870421+0000","last_fullsized":"2024-09-10T10:43:52.870421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:34:11.860584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563399+0000","last_change":"2024-09-10T10:43:51.577274+0000","last_active":"2024-09-10T10:43:52.563399+0000","last_peered":"2024-09-10T10:43:52.563399+0000","last_clean":"2024-09-10T10:43:52.563399+0000","last_became_active":"2024-09-10T10:43:51.576818+0000","last_became_peered":"2024-09-10T10:43:51.576818+0000","last_unstale":"2024-09-10T10:43:52.563399+0000","last_undegraded":"2024-09-10T10:43:52.563399+0000","last_fullsized":"2024-09-10T10:43:52.563399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869003+0000","last_change":"2024-09-10T10:43:47.528998+0000","last_active":"2024-09-10T10:43:52.869003+0000","last_peered":"2024-09-10T10:43:52.869003+0000","last_clean":"2024-09-10T10:43:52.869003+0000","last_became_active":"2024-09-10T10:43:47.528761+0000","last_became_peered":"2024-09-10T10:43:47.528761+0000","last_unstale":"2024-09-10T10:43:52.869003+0000","last_undegraded":"2024-09-10T10:43:52.869003+0000","last_fullsized":"2024-09-10T10:43:52.869003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:27.222446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538367+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538043+0000","last_became_peered":"2024-09-10T10:43:46.538043+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:46:27.129655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870256+0000","last_change":"2024-09-10T10:43:51.589628+0000","last_active":"2024-09-10T10:43:52.870256+0000","last_peered":"2024-09-10T10:43:52.870256+0000","last_clean":"2024-09-10T10:43:52.870256+0000","last_became_active":"2024-09-10T10:43:51.588470+0000","last_became_peered":"2024-09-10T10:43:51.588470+0000","last_unstale":"2024-09-10T10:43:52.870256+0000","last_undegraded":"2024-09-10T10:43:52.870256+0000","last_fullsized":"2024-09-10T10:43:52.870256+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:51.784026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564115+0000","last_change":"2024-09-10T10:43:47.539521+0000","last_active":"2024-09-10T10:43:52.564115+0000","last_peered":"2024-09-10T10:43:52.564115+0000","last_clean":"2024-09-10T10:43:52.564115+0000","last_became_active":"2024-09-10T10:43:47.539187+0000","last_became_peered":"2024-09-10T10:43:47.539187+0000","last_unstale":"2024-09-10T10:43:52.564115+0000","last_undegraded":"2024-09-10T10:43:52.564115+0000","last_fullsized":"2024-09-10T10:43:52.564115+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870141+0000","last_change":"2024-09-10T10:43:46.520278+0000","last_active":"2024-09-10T10:43:52.870141+0000","last_peered":"2024-09-10T10:43:52.870141+0000","last_clean":"2024-09-10T10:43:52.870141+0000","last_became_active":"2024-09-10T10:43:46.519954+0000","last_became_peered":"2024-09-10T10:43:46.519954+0000","last_unstale":"2024-09-10T10:43:52.870141+0000","last_undegraded":"2024-09-10T10:43:52.870141+0000","last_fullsized":"2024-09-10T10:43:52.870141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:51.570565+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:51.570165+0000","last_became_peered":"2024-09-10T10:43:51.570165+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:53.439716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869301+0000","last_change":"2024-09-10T10:43:47.534339+0000","last_active":"2024-09-10T10:43:52.869301+0000","last_peered":"2024-09-10T10:43:52.869301+0000","last_clean":"2024-09-10T10:43:52.869301+0000","last_became_active":"2024-09-10T10:43:47.533413+0000","last_became_peered":"2024-09-10T10:43:47.533413+0000","last_unstale":"2024-09-10T10:43:52.869301+0000","last_undegraded":"2024-09-10T10:43:52.869301+0000","last_fullsized":"2024-09-10T10:43:52.869301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869263+0000","last_change":"2024-09-10T10:43:46.535278+0000","last_active":"2024-09-10T10:43:52.869263+0000","last_peered":"2024-09-10T10:43:52.869263+0000","last_clean":"2024-09-10T10:43:52.869263+0000","last_became_active":"2024-09-10T10:43:46.534954+0000","last_became_peered":"2024-09-10T10:43:46.534954+0000","last_unstale":"2024-09-10T10:43:52.869263+0000","last_undegraded":"2024-09-10T10:43:52.869263+0000","last_fullsized":"2024-09-10T10:43:52.869263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564514+0000","last_change":"2024-09-10T10:43:51.574146+0000","last_active":"2024-09-10T10:43:52.564514+0000","last_peered":"2024-09-10T10:43:52.564514+0000","last_clean":"2024-09-10T10:43:52.564514+0000","last_became_active":"2024-09-10T10:43:51.573681+0000","last_became_peered":"2024-09-10T10:43:51.573681+0000","last_unstale":"2024-09-10T10:43:52.564514+0000","last_undegraded":"2024-09-10T10:43:52.564514+0000","last_fullsized":"2024-09-10T10:43:52.564514+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:13.180965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868735+0000","last_change":"2024-09-10T10:43:47.532923+0000","last_active":"2024-09-10T10:43:52.868735+0000","last_peered":"2024-09-10T10:43:52.868735+0000","last_clean":"2024-09-10T10:43:52.868735+0000","last_became_active":"2024-09-10T10:43:47.532644+0000","last_became_peered":"2024-09-10T10:43:47.532644+0000","last_unstale":"2024-09-10T10:43:52.868735+0000","last_undegraded":"2024-09-10T10:43:52.868735+0000","last_fullsized":"2024-09-10T10:43:52.868735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:44.518256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868884+0000","last_change":"2024-09-10T10:43:46.525829+0000","last_active":"2024-09-10T10:43:52.868884+0000","last_peered":"2024-09-10T10:43:52.868884+0000","last_clean":"2024-09-10T10:43:52.868884+0000","last_became_active":"2024-09-10T10:43:46.523492+0000","last_became_peered":"2024-09-10T10:43:46.523492+0000","last_unstale":"2024-09-10T10:43:52.868884+0000","last_undegraded":"2024-09-10T10:43:52.868884+0000","last_fullsized":"2024-09-10T10:43:52.868884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564250+0000","last_change":"2024-09-10T10:43:51.580518+0000","last_active":"2024-09-10T10:43:52.564250+0000","last_peered":"2024-09-10T10:43:52.564250+0000","last_clean":"2024-09-10T10:43:52.564250+0000","last_became_active":"2024-09-10T10:43:51.580050+0000","last_became_peered":"2024-09-10T10:43:51.580050+0000","last_unstale":"2024-09-10T10:43:52.564250+0000","last_undegraded":"2024-09-10T10:43:52.564250+0000","last_fullsized":"2024-09-10T10:43:52.564250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869809+0000","last_change":"2024-09-10T10:43:47.532764+0000","last_active":"2024-09-10T10:43:52.869809+0000","last_peered":"2024-09-10T10:43:52.869809+0000","last_clean":"2024-09-10T10:43:52.869809+0000","last_became_active":"2024-09-10T10:43:47.532407+0000","last_became_peered":"2024-09-10T10:43:47.532407+0000","last_unstale":"2024-09-10T10:43:52.869809+0000","last_undegraded":"2024-09-10T10:43:52.869809+0000","last_fullsized":"2024-09-10T10:43:52.869809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:09.139356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870804+0000","last_change":"2024-09-10T10:43:46.537450+0000","last_active":"2024-09-10T10:43:52.870804+0000","last_peered":"2024-09-10T10:43:52.870804+0000","last_clean":"2024-09-10T10:43:52.870804+0000","last_became_active":"2024-09-10T10:43:46.537129+0000","last_became_peered":"2024-09-10T10:43:46.537129+0000","last_unstale":"2024-09-10T10:43:52.870804+0000","last_undegraded":"2024-09-10T10:43:52.870804+0000","last_fullsized":"2024-09-10T10:43:52.870804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:25:47.537682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563273+0000","last_change":"2024-09-10T10:43:51.580449+0000","last_active":"2024-09-10T10:43:52.563273+0000","last_peered":"2024-09-10T10:43:52.563273+0000","last_clean":"2024-09-10T10:43:52.563273+0000","last_became_active":"2024-09-10T10:43:51.579803+0000","last_became_peered":"2024-09-10T10:43:51.579803+0000","last_unstale":"2024-09-10T10:43:52.563273+0000","last_undegraded":"2024-09-10T10:43:52.563273+0000","last_fullsized":"2024-09-10T10:43:52.563273+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:22.089401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870018+0000","last_change":"2024-09-10T10:43:47.532014+0000","last_active":"2024-09-10T10:43:52.870018+0000","last_peered":"2024-09-10T10:43:52.870018+0000","last_clean":"2024-09-10T10:43:52.870018+0000","last_became_active":"2024-09-10T10:43:47.531662+0000","last_became_peered":"2024-09-10T10:43:47.531662+0000","last_unstale":"2024-09-10T10:43:52.870018+0000","last_undegraded":"2024-09-10T10:43:52.870018+0000","last_fullsized":"2024-09-10T10:43:52.870018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:35.559824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869967+0000","last_change":"2024-09-10T10:43:46.527723+0000","last_active":"2024-09-10T10:43:52.869967+0000","last_peered":"2024-09-10T10:43:52.869967+0000","last_clean":"2024-09-10T10:43:52.869967+0000","last_became_active":"2024-09-10T10:43:46.527452+0000","last_became_peered":"2024-09-10T10:43:46.527452+0000","last_unstale":"2024-09-10T10:43:52.869967+0000","last_undegraded":"2024-09-10T10:43:52.869967+0000","last_fullsized":"2024-09-10T10:43:52.869967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869180+0000","last_change":"2024-09-10T10:43:51.574307+0000","last_active":"2024-09-10T10:43:52.869180+0000","last_peered":"2024-09-10T10:43:52.869180+0000","last_clean":"2024-09-10T10:43:52.869180+0000","last_became_active":"2024-09-10T10:43:51.571308+0000","last_became_peered":"2024-09-10T10:43:51.571308+0000","last_unstale":"2024-09-10T10:43:52.869180+0000","last_undegraded":"2024-09-10T10:43:52.869180+0000","last_fullsized":"2024-09-10T10:43:52.869180+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:09.361189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870742+0000","last_change":"2024-09-10T10:43:47.549822+0000","last_active":"2024-09-10T10:43:52.870742+0000","last_peered":"2024-09-10T10:43:52.870742+0000","last_clean":"2024-09-10T10:43:52.870742+0000","last_became_active":"2024-09-10T10:43:47.549526+0000","last_became_peered":"2024-09-10T10:43:47.549526+0000","last_unstale":"2024-09-10T10:43:52.870742+0000","last_undegraded":"2024-09-10T10:43:52.870742+0000","last_fullsized":"2024-09-10T10:43:52.870742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870715+0000","last_change":"2024-09-10T10:43:46.539321+0000","last_active":"2024-09-10T10:43:52.870715+0000","last_peered":"2024-09-10T10:43:52.870715+0000","last_clean":"2024-09-10T10:43:52.870715+0000","last_became_active":"2024-09-10T10:43:46.538763+0000","last_became_peered":"2024-09-10T10:43:46.538763+0000","last_unstale":"2024-09-10T10:43:52.870715+0000","last_undegraded":"2024-09-10T10:43:52.870715+0000","last_fullsized":"2024-09-10T10:43:52.870715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:54:53.745395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.571592+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.570946+0000","last_became_peered":"2024-09-10T10:43:51.570946+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:16.118096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564687+0000","last_change":"2024-09-10T10:43:47.535460+0000","last_active":"2024-09-10T10:43:52.564687+0000","last_peered":"2024-09-10T10:43:52.564687+0000","last_clean":"2024-09-10T10:43:52.564687+0000","last_became_active":"2024-09-10T10:43:47.535170+0000","last_became_peered":"2024-09-10T10:43:47.535170+0000","last_unstale":"2024-09-10T10:43:52.564687+0000","last_undegraded":"2024-09-10T10:43:52.564687+0000","last_fullsized":"2024-09-10T10:43:52.564687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:13.072361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708875+0000","last_change":"2024-09-10T10:43:46.521709+0000","last_active":"2024-09-10T10:43:52.708875+0000","last_peered":"2024-09-10T10:43:52.708875+0000","last_clean":"2024-09-10T10:43:52.708875+0000","last_became_active":"2024-09-10T10:43:46.521411+0000","last_became_peered":"2024-09-10T10:43:46.521411+0000","last_unstale":"2024-09-10T10:43:52.708875+0000","last_undegraded":"2024-09-10T10:43:52.708875+0000","last_fullsized":"2024-09-10T10:43:52.708875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:04.600951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:51.573047+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:51.572690+0000","last_became_peered":"2024-09-10T10:43:51.572690+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869228+0000","last_change":"2024-09-10T10:43:47.541642+0000","last_active":"2024-09-10T10:43:52.869228+0000","last_peered":"2024-09-10T10:43:52.869228+0000","last_clean":"2024-09-10T10:43:52.869228+0000","last_became_active":"2024-09-10T10:43:47.541174+0000","last_became_peered":"2024-09-10T10:43:47.541174+0000","last_unstale":"2024-09-10T10:43:52.869228+0000","last_undegraded":"2024-09-10T10:43:52.869228+0000","last_fullsized":"2024-09-10T10:43:52.869228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:18.047791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869868+0000","last_change":"2024-09-10T10:43:46.532535+0000","last_active":"2024-09-10T10:43:52.869868+0000","last_peered":"2024-09-10T10:43:52.869868+0000","last_clean":"2024-09-10T10:43:52.869868+0000","last_became_active":"2024-09-10T10:43:46.532256+0000","last_became_peered":"2024-09-10T10:43:46.532256+0000","last_unstale":"2024-09-10T10:43:52.869868+0000","last_undegraded":"2024-09-10T10:43:52.869868+0000","last_fullsized":"2024-09-10T10:43:52.869868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:51.847379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870621+0000","last_change":"2024-09-10T10:43:51.571178+0000","last_active":"2024-09-10T10:43:52.870621+0000","last_peered":"2024-09-10T10:43:52.870621+0000","last_clean":"2024-09-10T10:43:52.870621+0000","last_became_active":"2024-09-10T10:43:51.570742+0000","last_became_peered":"2024-09-10T10:43:51.570742+0000","last_unstale":"2024-09-10T10:43:52.870621+0000","last_undegraded":"2024-09-10T10:43:52.870621+0000","last_fullsized":"2024-09-10T10:43:52.870621+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:35.967995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708292+0000","last_change":"2024-09-10T10:43:47.536864+0000","last_active":"2024-09-10T10:43:52.708292+0000","last_peered":"2024-09-10T10:43:52.708292+0000","last_clean":"2024-09-10T10:43:52.708292+0000","last_became_active":"2024-09-10T10:43:47.536270+0000","last_became_peered":"2024-09-10T10:43:47.536270+0000","last_unstale":"2024-09-10T10:43:52.708292+0000","last_undegraded":"2024-09-10T10:43:52.708292+0000","last_fullsized":"2024-09-10T10:43:52.708292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:25.640976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563952+0000","last_change":"2024-09-10T10:43:46.521627+0000","last_active":"2024-09-10T10:43:52.563952+0000","last_peered":"2024-09-10T10:43:52.563952+0000","last_clean":"2024-09-10T10:43:52.563952+0000","last_became_active":"2024-09-10T10:43:46.521150+0000","last_became_peered":"2024-09-10T10:43:46.521150+0000","last_unstale":"2024-09-10T10:43:52.563952+0000","last_undegraded":"2024-09-10T10:43:52.563952+0000","last_fullsized":"2024-09-10T10:43:52.563952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:35.895649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870932+0000","last_change":"2024-09-10T10:43:51.580293+0000","last_active":"2024-09-10T10:43:52.870932+0000","last_peered":"2024-09-10T10:43:52.870932+0000","last_clean":"2024-09-10T10:43:52.870932+0000","last_became_active":"2024-09-10T10:43:51.576157+0000","last_became_peered":"2024-09-10T10:43:51.576157+0000","last_unstale":"2024-09-10T10:43:52.870932+0000","last_undegraded":"2024-09-10T10:43:52.870932+0000","last_fullsized":"2024-09-10T10:43:52.870932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869651+0000","last_change":"2024-09-10T10:43:47.563302+0000","last_active":"2024-09-10T10:43:52.869651+0000","last_peered":"2024-09-10T10:43:52.869651+0000","last_clean":"2024-09-10T10:43:52.869651+0000","last_became_active":"2024-09-10T10:43:47.562694+0000","last_became_peered":"2024-09-10T10:43:47.562694+0000","last_unstale":"2024-09-10T10:43:52.869651+0000","last_undegraded":"2024-09-10T10:43:52.869651+0000","last_fullsized":"2024-09-10T10:43:52.869651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868801+0000","last_change":"2024-09-10T10:43:46.533697+0000","last_active":"2024-09-10T10:43:52.868801+0000","last_peered":"2024-09-10T10:43:52.868801+0000","last_clean":"2024-09-10T10:43:52.868801+0000","last_became_active":"2024-09-10T10:43:46.533190+0000","last_became_peered":"2024-09-10T10:43:46.533190+0000","last_unstale":"2024-09-10T10:43:52.868801+0000","last_undegraded":"2024-09-10T10:43:52.868801+0000","last_fullsized":"2024-09-10T10:43:52.868801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:09.764908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708748+0000","last_change":"2024-09-10T10:43:51.583515+0000","last_active":"2024-09-10T10:43:52.708748+0000","last_peered":"2024-09-10T10:43:52.708748+0000","last_clean":"2024-09-10T10:43:52.708748+0000","last_became_active":"2024-09-10T10:43:51.582366+0000","last_became_peered":"2024-09-10T10:43:51.582366+0000","last_unstale":"2024-09-10T10:43:52.708748+0000","last_undegraded":"2024-09-10T10:43:52.708748+0000","last_fullsized":"2024-09-10T10:43:52.708748+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:10.957254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869329+0000","last_change":"2024-09-10T10:43:47.520783+0000","last_active":"2024-09-10T10:43:52.869329+0000","last_peered":"2024-09-10T10:43:52.869329+0000","last_clean":"2024-09-10T10:43:52.869329+0000","last_became_active":"2024-09-10T10:43:47.520409+0000","last_became_peered":"2024-09-10T10:43:47.520409+0000","last_unstale":"2024-09-10T10:43:52.869329+0000","last_undegraded":"2024-09-10T10:43:52.869329+0000","last_fullsized":"2024-09-10T10:43:52.869329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869933+0000","last_change":"2024-09-10T10:43:46.538924+0000","last_active":"2024-09-10T10:43:52.869933+0000","last_peered":"2024-09-10T10:43:52.869933+0000","last_clean":"2024-09-10T10:43:52.869933+0000","last_became_active":"2024-09-10T10:43:46.538640+0000","last_became_peered":"2024-09-10T10:43:46.538640+0000","last_unstale":"2024-09-10T10:43:52.869933+0000","last_undegraded":"2024-09-10T10:43:52.869933+0000","last_fullsized":"2024-09-10T10:43:52.869933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:57.368590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564215+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:43:52.564215+0000","last_peered":"2024-09-10T10:43:52.564215+0000","last_clean":"2024-09-10T10:43:52.564215+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:43:52.564215+0000","last_undegraded":"2024-09-10T10:43:52.564215+0000","last_fullsized":"2024-09-10T10:43:52.564215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870584+0000","last_change":"2024-09-10T10:43:51.571200+0000","last_active":"2024-09-10T10:43:52.870584+0000","last_peered":"2024-09-10T10:43:52.870584+0000","last_clean":"2024-09-10T10:43:52.870584+0000","last_became_active":"2024-09-10T10:43:51.570877+0000","last_became_peered":"2024-09-10T10:43:51.570877+0000","last_unstale":"2024-09-10T10:43:52.870584+0000","last_undegraded":"2024-09-10T10:43:52.870584+0000","last_fullsized":"2024-09-10T10:43:52.870584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:59.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870468+0000","last_change":"2024-09-10T10:43:47.546420+0000","last_active":"2024-09-10T10:43:52.870468+0000","last_peered":"2024-09-10T10:43:52.870468+0000","last_clean":"2024-09-10T10:43:52.870468+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.870468+0000","last_undegraded":"2024-09-10T10:43:52.870468+0000","last_fullsized":"2024-09-10T10:43:52.870468+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:30.887906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870800+0000","last_change":"2024-09-10T10:43:46.535842+0000","last_active":"2024-09-10T10:43:52.870800+0000","last_peered":"2024-09-10T10:43:52.870800+0000","last_clean":"2024-09-10T10:43:52.870800+0000","last_became_active":"2024-09-10T10:43:46.535586+0000","last_became_peered":"2024-09-10T10:43:46.535586+0000","last_unstale":"2024-09-10T10:43:52.870800+0000","last_undegraded":"2024-09-10T10:43:52.870800+0000","last_fullsized":"2024-09-10T10:43:52.870800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:47:46.752912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563751+0000","last_change":"2024-09-10T10:43:51.578759+0000","last_active":"2024-09-10T10:43:52.563751+0000","last_peered":"2024-09-10T10:43:52.563751+0000","last_clean":"2024-09-10T10:43:52.563751+0000","last_became_active":"2024-09-10T10:43:51.578237+0000","last_became_peered":"2024-09-10T10:43:51.578237+0000","last_unstale":"2024-09-10T10:43:52.563751+0000","last_undegraded":"2024-09-10T10:43:52.563751+0000","last_fullsized":"2024-09-10T10:43:52.563751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869654+0000","last_change":"2024-09-10T10:43:47.540794+0000","last_active":"2024-09-10T10:43:52.869654+0000","last_peered":"2024-09-10T10:43:52.869654+0000","last_clean":"2024-09-10T10:43:52.869654+0000","last_became_active":"2024-09-10T10:43:47.540360+0000","last_became_peered":"2024-09-10T10:43:47.540360+0000","last_unstale":"2024-09-10T10:43:52.869654+0000","last_undegraded":"2024-09-10T10:43:52.869654+0000","last_fullsized":"2024-09-10T10:43:52.869654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:18:31.926385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709016+0000","last_change":"2024-09-10T10:43:46.520898+0000","last_active":"2024-09-10T10:43:52.709016+0000","last_peered":"2024-09-10T10:43:52.709016+0000","last_clean":"2024-09-10T10:43:52.709016+0000","last_became_active":"2024-09-10T10:43:46.518998+0000","last_became_peered":"2024-09-10T10:43:46.518998+0000","last_unstale":"2024-09-10T10:43:52.709016+0000","last_undegraded":"2024-09-10T10:43:52.709016+0000","last_fullsized":"2024-09-10T10:43:52.709016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:40:42.158638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708786+0000","last_change":"2024-09-10T10:43:51.587866+0000","last_active":"2024-09-10T10:43:52.708786+0000","last_peered":"2024-09-10T10:43:52.708786+0000","last_clean":"2024-09-10T10:43:52.708786+0000","last_became_active":"2024-09-10T10:43:51.585690+0000","last_became_peered":"2024-09-10T10:43:51.585690+0000","last_unstale":"2024-09-10T10:43:52.708786+0000","last_undegraded":"2024-09-10T10:43:52.708786+0000","last_fullsized":"2024-09-10T10:43:52.708786+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:41.798950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870081+0000","last_change":"2024-09-10T10:43:47.540910+0000","last_active":"2024-09-10T10:43:52.870081+0000","last_peered":"2024-09-10T10:43:52.870081+0000","last_clean":"2024-09-10T10:43:52.870081+0000","last_became_active":"2024-09-10T10:43:47.540603+0000","last_became_peered":"2024-09-10T10:43:47.540603+0000","last_unstale":"2024-09-10T10:43:52.870081+0000","last_undegraded":"2024-09-10T10:43:52.870081+0000","last_fullsized":"2024-09-10T10:43:52.870081+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:48.137696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869935+0000","last_change":"2024-09-10T10:43:46.525479+0000","last_active":"2024-09-10T10:43:52.869935+0000","last_peered":"2024-09-10T10:43:52.869935+0000","last_clean":"2024-09-10T10:43:52.869935+0000","last_became_active":"2024-09-10T10:43:46.524834+0000","last_became_peered":"2024-09-10T10:43:46.524834+0000","last_unstale":"2024-09-10T10:43:52.869935+0000","last_undegraded":"2024-09-10T10:43:52.869935+0000","last_fullsized":"2024-09-10T10:43:52.869935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:30.640537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868574+0000","last_change":"2024-09-10T10:43:51.569746+0000","last_active":"2024-09-10T10:43:52.868574+0000","last_peered":"2024-09-10T10:43:52.868574+0000","last_clean":"2024-09-10T10:43:52.868574+0000","last_became_active":"2024-09-10T10:43:51.569289+0000","last_became_peered":"2024-09-10T10:43:51.569289+0000","last_unstale":"2024-09-10T10:43:52.868574+0000","last_undegraded":"2024-09-10T10:43:52.868574+0000","last_fullsized":"2024-09-10T10:43:52.868574+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:20.122047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564540+0000","last_change":"2024-09-10T10:43:47.555824+0000","last_active":"2024-09-10T10:43:52.564540+0000","last_peered":"2024-09-10T10:43:52.564540+0000","last_clean":"2024-09-10T10:43:52.564540+0000","last_became_active":"2024-09-10T10:43:47.554957+0000","last_became_peered":"2024-09-10T10:43:47.554957+0000","last_unstale":"2024-09-10T10:43:52.564540+0000","last_undegraded":"2024-09-10T10:43:52.564540+0000","last_fullsized":"2024-09-10T10:43:52.564540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870114+0000","last_change":"2024-09-10T10:43:46.530043+0000","last_active":"2024-09-10T10:43:52.870114+0000","last_peered":"2024-09-10T10:43:52.870114+0000","last_clean":"2024-09-10T10:43:52.870114+0000","last_became_active":"2024-09-10T10:43:46.527509+0000","last_became_peered":"2024-09-10T10:43:46.527509+0000","last_unstale":"2024-09-10T10:43:52.870114+0000","last_undegraded":"2024-09-10T10:43:52.870114+0000","last_fullsized":"2024-09-10T10:43:52.870114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:08.859006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:51.577121+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:51.576021+0000","last_became_peered":"2024-09-10T10:43:51.576021+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:10.204576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868710+0000","last_change":"2024-09-10T10:43:47.562724+0000","last_active":"2024-09-10T10:43:52.868710+0000","last_peered":"2024-09-10T10:43:52.868710+0000","last_clean":"2024-09-10T10:43:52.868710+0000","last_became_active":"2024-09-10T10:43:47.562412+0000","last_became_peered":"2024-09-10T10:43:47.562412+0000","last_unstale":"2024-09-10T10:43:52.868710+0000","last_undegraded":"2024-09-10T10:43:52.868710+0000","last_fullsized":"2024-09-10T10:43:52.868710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:30.773785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869413+0000","last_change":"2024-09-10T10:43:46.525713+0000","last_active":"2024-09-10T10:43:52.869413+0000","last_peered":"2024-09-10T10:43:52.869413+0000","last_clean":"2024-09-10T10:43:52.869413+0000","last_became_active":"2024-09-10T10:43:46.525130+0000","last_became_peered":"2024-09-10T10:43:46.525130+0000","last_unstale":"2024-09-10T10:43:52.869413+0000","last_undegraded":"2024-09-10T10:43:52.869413+0000","last_fullsized":"2024-09-10T10:43:52.869413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:46.997184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.709094+0000","last_change":"2024-09-10T10:43:51.582330+0000","last_active":"2024-09-10T10:43:52.709094+0000","last_peered":"2024-09-10T10:43:52.709094+0000","last_clean":"2024-09-10T10:43:52.709094+0000","last_became_active":"2024-09-10T10:43:51.581415+0000","last_became_peered":"2024-09-10T10:43:51.581415+0000","last_unstale":"2024-09-10T10:43:52.709094+0000","last_undegraded":"2024-09-10T10:43:52.709094+0000","last_fullsized":"2024-09-10T10:43:52.709094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:15:18.520413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564097+0000","last_change":"2024-09-10T10:43:47.537959+0000","last_active":"2024-09-10T10:43:52.564097+0000","last_peered":"2024-09-10T10:43:52.564097+0000","last_clean":"2024-09-10T10:43:52.564097+0000","last_became_active":"2024-09-10T10:43:47.537705+0000","last_became_peered":"2024-09-10T10:43:47.537705+0000","last_unstale":"2024-09-10T10:43:52.564097+0000","last_undegraded":"2024-09-10T10:43:52.564097+0000","last_fullsized":"2024-09-10T10:43:52.564097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:56.301556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869080+0000","last_change":"2024-09-10T10:43:46.539973+0000","last_active":"2024-09-10T10:43:52.869080+0000","last_peered":"2024-09-10T10:43:52.869080+0000","last_clean":"2024-09-10T10:43:52.869080+0000","last_became_active":"2024-09-10T10:43:46.539509+0000","last_became_peered":"2024-09-10T10:43:46.539509+0000","last_unstale":"2024-09-10T10:43:52.869080+0000","last_undegraded":"2024-09-10T10:43:52.869080+0000","last_fullsized":"2024-09-10T10:43:52.869080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:55.298818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:51.584701+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:51.584321+0000","last_became_peered":"2024-09-10T10:43:51.584321+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:07.697012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869811+0000","last_change":"2024-09-10T10:43:47.549457+0000","last_active":"2024-09-10T10:43:52.869811+0000","last_peered":"2024-09-10T10:43:52.869811+0000","last_clean":"2024-09-10T10:43:52.869811+0000","last_became_active":"2024-09-10T10:43:47.549210+0000","last_became_peered":"2024-09-10T10:43:47.549210+0000","last_unstale":"2024-09-10T10:43:52.869811+0000","last_undegraded":"2024-09-10T10:43:52.869811+0000","last_fullsized":"2024-09-10T10:43:52.869811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:30:00.039891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869773+0000","last_change":"2024-09-10T10:43:46.520041+0000","last_active":"2024-09-10T10:43:52.869773+0000","last_peered":"2024-09-10T10:43:52.869773+0000","last_clean":"2024-09-10T10:43:52.869773+0000","last_became_active":"2024-09-10T10:43:46.519721+0000","last_became_peered":"2024-09-10T10:43:46.519721+0000","last_unstale":"2024-09-10T10:43:52.869773+0000","last_undegraded":"2024-09-10T10:43:52.869773+0000","last_fullsized":"2024-09-10T10:43:52.869773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:27:23.802544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870586+0000","last_change":"2024-09-10T10:43:51.571179+0000","last_active":"2024-09-10T10:43:52.870586+0000","last_peered":"2024-09-10T10:43:52.870586+0000","last_clean":"2024-09-10T10:43:52.870586+0000","last_became_active":"2024-09-10T10:43:51.570590+0000","last_became_peered":"2024-09-10T10:43:51.570590+0000","last_unstale":"2024-09-10T10:43:52.870586+0000","last_undegraded":"2024-09-10T10:43:52.870586+0000","last_fullsized":"2024-09-10T10:43:52.870586+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:00.265915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870110+0000","last_change":"2024-09-10T10:43:47.546795+0000","last_active":"2024-09-10T10:43:52.870110+0000","last_peered":"2024-09-10T10:43:52.870110+0000","last_clean":"2024-09-10T10:43:52.870110+0000","last_became_active":"2024-09-10T10:43:47.546529+0000","last_became_peered":"2024-09-10T10:43:47.546529+0000","last_unstale":"2024-09-10T10:43:52.870110+0000","last_undegraded":"2024-09-10T10:43:52.870110+0000","last_fullsized":"2024-09-10T10:43:52.870110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564317+0000","last_change":"2024-09-10T10:43:46.524422+0000","last_active":"2024-09-10T10:43:52.564317+0000","last_peered":"2024-09-10T10:43:52.564317+0000","last_clean":"2024-09-10T10:43:52.564317+0000","last_became_active":"2024-09-10T10:43:46.524027+0000","last_became_peered":"2024-09-10T10:43:46.524027+0000","last_unstale":"2024-09-10T10:43:52.564317+0000","last_undegraded":"2024-09-10T10:43:52.564317+0000","last_fullsized":"2024-09-10T10:43:52.564317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870370+0000","last_change":"2024-09-10T10:43:51.575001+0000","last_active":"2024-09-10T10:43:52.870370+0000","last_peered":"2024-09-10T10:43:52.870370+0000","last_clean":"2024-09-10T10:43:52.870370+0000","last_became_active":"2024-09-10T10:43:51.573805+0000","last_became_peered":"2024-09-10T10:43:51.573805+0000","last_unstale":"2024-09-10T10:43:52.870370+0000","last_undegraded":"2024-09-10T10:43:52.870370+0000","last_fullsized":"2024-09-10T10:43:52.870370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:27.992889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869516+0000","last_change":"2024-09-10T10:43:47.542333+0000","last_active":"2024-09-10T10:43:52.869516+0000","last_peered":"2024-09-10T10:43:52.869516+0000","last_clean":"2024-09-10T10:43:52.869516+0000","last_became_active":"2024-09-10T10:43:47.540584+0000","last_became_peered":"2024-09-10T10:43:47.540584+0000","last_unstale":"2024-09-10T10:43:52.869516+0000","last_undegraded":"2024-09-10T10:43:52.869516+0000","last_fullsized":"2024-09-10T10:43:52.869516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:08.456485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:58.645779+0000","last_change":"2024-09-10T10:43:46.531547+0000","last_active":"2024-09-10T10:43:58.645779+0000","last_peered":"2024-09-10T10:43:58.645779+0000","last_clean":"2024-09-10T10:43:58.645779+0000","last_became_active":"2024-09-10T10:43:46.530309+0000","last_became_peered":"2024-09-10T10:43:46.530309+0000","last_unstale":"2024-09-10T10:43:58.645779+0000","last_undegraded":"2024-09-10T10:43:58.645779+0000","last_fullsized":"2024-09-10T10:43:58.645779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:54.587795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869118+0000","last_change":"2024-09-10T10:43:51.578360+0000","last_active":"2024-09-10T10:43:52.869118+0000","last_peered":"2024-09-10T10:43:52.869118+0000","last_clean":"2024-09-10T10:43:52.869118+0000","last_became_active":"2024-09-10T10:43:51.578021+0000","last_became_peered":"2024-09-10T10:43:51.578021+0000","last_unstale":"2024-09-10T10:43:52.869118+0000","last_undegraded":"2024-09-10T10:43:52.869118+0000","last_fullsized":"2024-09-10T10:43:52.869118+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:33:12.498094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870009+0000","last_change":"2024-09-10T10:43:47.564353+0000","last_active":"2024-09-10T10:43:52.870009+0000","last_peered":"2024-09-10T10:43:52.870009+0000","last_clean":"2024-09-10T10:43:52.870009+0000","last_became_active":"2024-09-10T10:43:47.563999+0000","last_became_peered":"2024-09-10T10:43:47.563999+0000","last_unstale":"2024-09-10T10:43:52.870009+0000","last_undegraded":"2024-09-10T10:43:52.870009+0000","last_fullsized":"2024-09-10T10:43:52.870009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:10:52.338110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564851+0000","last_change":"2024-09-10T10:43:46.526185+0000","last_active":"2024-09-10T10:43:52.564851+0000","last_peered":"2024-09-10T10:43:52.564851+0000","last_clean":"2024-09-10T10:43:52.564851+0000","last_became_active":"2024-09-10T10:43:46.525887+0000","last_became_peered":"2024-09-10T10:43:46.525887+0000","last_unstale":"2024-09-10T10:43:52.564851+0000","last_undegraded":"2024-09-10T10:43:52.564851+0000","last_fullsized":"2024-09-10T10:43:52.564851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563403+0000","last_change":"2024-09-10T10:43:51.587988+0000","last_active":"2024-09-10T10:43:52.563403+0000","last_peered":"2024-09-10T10:43:52.563403+0000","last_clean":"2024-09-10T10:43:52.563403+0000","last_became_active":"2024-09-10T10:43:51.587584+0000","last_became_peered":"2024-09-10T10:43:51.587584+0000","last_unstale":"2024-09-10T10:43:52.563403+0000","last_undegraded":"2024-09-10T10:43:52.563403+0000","last_fullsized":"2024-09-10T10:43:52.563403+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:59.998098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708986+0000","last_change":"2024-09-10T10:43:47.541815+0000","last_active":"2024-09-10T10:43:52.708986+0000","last_peered":"2024-09-10T10:43:52.708986+0000","last_clean":"2024-09-10T10:43:52.708986+0000","last_became_active":"2024-09-10T10:43:47.541530+0000","last_became_peered":"2024-09-10T10:43:47.541530+0000","last_unstale":"2024-09-10T10:43:52.708986+0000","last_undegraded":"2024-09-10T10:43:52.708986+0000","last_fullsized":"2024-09-10T10:43:52.708986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:26.960082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869496+0000","last_change":"2024-09-10T10:43:46.525990+0000","last_active":"2024-09-10T10:43:52.869496+0000","last_peered":"2024-09-10T10:43:52.869496+0000","last_clean":"2024-09-10T10:43:52.869496+0000","last_became_active":"2024-09-10T10:43:46.524901+0000","last_became_peered":"2024-09-10T10:43:46.524901+0000","last_unstale":"2024-09-10T10:43:52.869496+0000","last_undegraded":"2024-09-10T10:43:52.869496+0000","last_fullsized":"2024-09-10T10:43:52.869496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:12.069623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563676+0000","last_change":"2024-09-10T10:43:51.574109+0000","last_active":"2024-09-10T10:43:52.563676+0000","last_peered":"2024-09-10T10:43:52.563676+0000","last_clean":"2024-09-10T10:43:52.563676+0000","last_became_active":"2024-09-10T10:43:51.573718+0000","last_became_peered":"2024-09-10T10:43:51.573718+0000","last_unstale":"2024-09-10T10:43:52.563676+0000","last_undegraded":"2024-09-10T10:43:52.563676+0000","last_fullsized":"2024-09-10T10:43:52.563676+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:13.623233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564443+0000","last_change":"2024-09-10T10:43:47.558084+0000","last_active":"2024-09-10T10:43:52.564443+0000","last_peered":"2024-09-10T10:43:52.564443+0000","last_clean":"2024-09-10T10:43:52.564443+0000","last_became_active":"2024-09-10T10:43:47.555632+0000","last_became_peered":"2024-09-10T10:43:47.555632+0000","last_unstale":"2024-09-10T10:43:52.564443+0000","last_undegraded":"2024-09-10T10:43:52.564443+0000","last_fullsized":"2024-09-10T10:43:52.564443+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565392+0000","last_change":"2024-09-10T10:43:46.528292+0000","last_active":"2024-09-10T10:43:52.565392+0000","last_peered":"2024-09-10T10:43:52.565392+0000","last_clean":"2024-09-10T10:43:52.565392+0000","last_became_active":"2024-09-10T10:43:46.527943+0000","last_became_peered":"2024-09-10T10:43:46.527943+0000","last_unstale":"2024-09-10T10:43:52.565392+0000","last_undegraded":"2024-09-10T10:43:52.565392+0000","last_fullsized":"2024-09-10T10:43:52.565392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:16.541182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869189+0000","last_change":"2024-09-10T10:43:51.572294+0000","last_active":"2024-09-10T10:43:52.869189+0000","last_peered":"2024-09-10T10:43:52.869189+0000","last_clean":"2024-09-10T10:43:52.869189+0000","last_became_active":"2024-09-10T10:43:51.571796+0000","last_became_peered":"2024-09-10T10:43:51.571796+0000","last_unstale":"2024-09-10T10:43:52.869189+0000","last_undegraded":"2024-09-10T10:43:52.869189+0000","last_fullsized":"2024-09-10T10:43:52.869189+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:07.016675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869838+0000","last_change":"2024-09-10T10:43:51.577672+0000","last_active":"2024-09-10T10:43:52.869838+0000","last_peered":"2024-09-10T10:43:52.869838+0000","last_clean":"2024-09-10T10:43:52.869838+0000","last_became_active":"2024-09-10T10:43:51.577342+0000","last_became_peered":"2024-09-10T10:43:51.577342+0000","last_unstale":"2024-09-10T10:43:52.869838+0000","last_undegraded":"2024-09-10T10:43:52.869838+0000","last_fullsized":"2024-09-10T10:43:52.869838+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:09:59.764337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869129+0000","last_change":"2024-09-10T10:43:47.538181+0000","last_active":"2024-09-10T10:43:52.869129+0000","last_peered":"2024-09-10T10:43:52.869129+0000","last_clean":"2024-09-10T10:43:52.869129+0000","last_became_active":"2024-09-10T10:43:47.537810+0000","last_became_peered":"2024-09-10T10:43:47.537810+0000","last_unstale":"2024-09-10T10:43:52.869129+0000","last_undegraded":"2024-09-10T10:43:52.869129+0000","last_fullsized":"2024-09-10T10:43:52.869129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:09.581624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868780+0000","last_change":"2024-09-10T10:43:46.533440+0000","last_active":"2024-09-10T10:43:52.868780+0000","last_peered":"2024-09-10T10:43:52.868780+0000","last_clean":"2024-09-10T10:43:52.868780+0000","last_became_active":"2024-09-10T10:43:46.532814+0000","last_became_peered":"2024-09-10T10:43:46.532814+0000","last_unstale":"2024-09-10T10:43:52.868780+0000","last_undegraded":"2024-09-10T10:43:52.868780+0000","last_fullsized":"2024-09-10T10:43:52.868780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:03.561914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870537+0000","last_change":"2024-09-10T10:43:51.576277+0000","last_active":"2024-09-10T10:43:52.870537+0000","last_peered":"2024-09-10T10:43:52.870537+0000","last_clean":"2024-09-10T10:43:52.870537+0000","last_became_active":"2024-09-10T10:43:51.575888+0000","last_became_peered":"2024-09-10T10:43:51.575888+0000","last_unstale":"2024-09-10T10:43:52.870537+0000","last_undegraded":"2024-09-10T10:43:52.870537+0000","last_fullsized":"2024-09-10T10:43:52.870537+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:25:37.271562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708364+0000","last_change":"2024-09-10T10:43:47.539801+0000","last_active":"2024-09-10T10:43:52.708364+0000","last_peered":"2024-09-10T10:43:52.708364+0000","last_clean":"2024-09-10T10:43:52.708364+0000","last_became_active":"2024-09-10T10:43:47.535922+0000","last_became_peered":"2024-09-10T10:43:47.535922+0000","last_unstale":"2024-09-10T10:43:52.708364+0000","last_undegraded":"2024-09-10T10:43:52.708364+0000","last_fullsized":"2024-09-10T10:43:52.708364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:49.905762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869706+0000","last_change":"2024-09-10T10:43:46.530425+0000","last_active":"2024-09-10T10:43:52.869706+0000","last_peered":"2024-09-10T10:43:52.869706+0000","last_clean":"2024-09-10T10:43:52.869706+0000","last_became_active":"2024-09-10T10:43:46.530158+0000","last_became_peered":"2024-09-10T10:43:46.530158+0000","last_unstale":"2024-09-10T10:43:52.869706+0000","last_undegraded":"2024-09-10T10:43:52.869706+0000","last_fullsized":"2024-09-10T10:43:52.869706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:25.173712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869653+0000","last_change":"2024-09-10T10:43:46.537743+0000","last_active":"2024-09-10T10:43:52.869653+0000","last_peered":"2024-09-10T10:43:52.869653+0000","last_clean":"2024-09-10T10:43:52.869653+0000","last_became_active":"2024-09-10T10:43:46.537260+0000","last_became_peered":"2024-09-10T10:43:46.537260+0000","last_unstale":"2024-09-10T10:43:52.869653+0000","last_undegraded":"2024-09-10T10:43:52.869653+0000","last_fullsized":"2024-09-10T10:43:52.869653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:53.479099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869441+0000","last_change":"2024-09-10T10:43:47.545017+0000","last_active":"2024-09-10T10:43:52.869441+0000","last_peered":"2024-09-10T10:43:52.869441+0000","last_clean":"2024-09-10T10:43:52.869441+0000","last_became_active":"2024-09-10T10:43:47.544716+0000","last_became_peered":"2024-09-10T10:43:47.544716+0000","last_unstale":"2024-09-10T10:43:52.869441+0000","last_undegraded":"2024-09-10T10:43:52.869441+0000","last_fullsized":"2024-09-10T10:43:52.869441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:46.823226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869679+0000","last_change":"2024-09-10T10:43:51.570454+0000","last_active":"2024-09-10T10:43:52.869679+0000","last_peered":"2024-09-10T10:43:52.869679+0000","last_clean":"2024-09-10T10:43:52.869679+0000","last_became_active":"2024-09-10T10:43:51.570088+0000","last_became_peered":"2024-09-10T10:43:51.570088+0000","last_unstale":"2024-09-10T10:43:52.869679+0000","last_undegraded":"2024-09-10T10:43:52.869679+0000","last_fullsized":"2024-09-10T10:43:52.869679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:52.179305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708948+0000","last_change":"2024-09-10T10:43:46.521135+0000","last_active":"2024-09-10T10:43:52.708948+0000","last_peered":"2024-09-10T10:43:52.708948+0000","last_clean":"2024-09-10T10:43:52.708948+0000","last_became_active":"2024-09-10T10:43:46.520815+0000","last_became_peered":"2024-09-10T10:43:46.520815+0000","last_unstale":"2024-09-10T10:43:52.708948+0000","last_undegraded":"2024-09-10T10:43:52.708948+0000","last_fullsized":"2024-09-10T10:43:52.708948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:45.492501+0000","last_clean_scrub_stamp":"2024-09-10T10:43:45.492501+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:32:38.222213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564462+0000","last_change":"2024-09-10T10:43:47.545783+0000","last_active":"2024-09-10T10:43:52.564462+0000","last_peered":"2024-09-10T10:43:52.564462+0000","last_clean":"2024-09-10T10:43:52.564462+0000","last_became_active":"2024-09-10T10:43:47.545265+0000","last_became_peered":"2024-09-10T10:43:47.545265+0000","last_unstale":"2024-09-10T10:43:52.564462+0000","last_undegraded":"2024-09-10T10:43:52.564462+0000","last_fullsized":"2024-09-10T10:43:52.564462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564298+0000","last_change":"2024-09-10T10:43:51.585486+0000","last_active":"2024-09-10T10:43:52.564298+0000","last_peered":"2024-09-10T10:43:52.564298+0000","last_clean":"2024-09-10T10:43:52.564298+0000","last_became_active":"2024-09-10T10:43:51.585045+0000","last_became_peered":"2024-09-10T10:43:51.585045+0000","last_unstale":"2024-09-10T10:43:52.564298+0000","last_undegraded":"2024-09-10T10:43:52.564298+0000","last_fullsized":"2024-09-10T10:43:52.564298+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:50.536849+0000","last_clean_scrub_stamp":"2024-09-10T10:43:50.536849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:59.749737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563314+0000","last_change":"2024-09-10T10:43:47.533694+0000","last_active":"2024-09-10T10:43:52.563314+0000","last_peered":"2024-09-10T10:43:52.563314+0000","last_clean":"2024-09-10T10:43:52.563314+0000","last_became_active":"2024-09-10T10:43:47.533197+0000","last_became_peered":"2024-09-10T10:43:47.533197+0000","last_unstale":"2024-09-10T10:43:52.563314+0000","last_undegraded":"2024-09-10T10:43:52.563314+0000","last_fullsized":"2024-09-10T10:43:52.563314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:08:09.861745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563761+0000","last_change":"2024-09-10T10:43:47.540401+0000","last_active":"2024-09-10T10:43:52.563761+0000","last_peered":"2024-09-10T10:43:52.563761+0000","last_clean":"2024-09-10T10:43:52.563761+0000","last_became_active":"2024-09-10T10:43:47.539677+0000","last_became_peered":"2024-09-10T10:43:47.539677+0000","last_unstale":"2024-09-10T10:43:52.563761+0000","last_undegraded":"2024-09-10T10:43:52.563761+0000","last_fullsized":"2024-09-10T10:43:52.563761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868918+0000","last_change":"2024-09-10T10:43:47.539352+0000","last_active":"2024-09-10T10:43:52.868918+0000","last_peered":"2024-09-10T10:43:52.868918+0000","last_clean":"2024-09-10T10:43:52.868918+0000","last_became_active":"2024-09-10T10:43:47.539034+0000","last_became_peered":"2024-09-10T10:43:47.539034+0000","last_unstale":"2024-09-10T10:43:52.868918+0000","last_undegraded":"2024-09-10T10:43:52.868918+0000","last_fullsized":"2024-09-10T10:43:52.868918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:50:08.107825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708757+0000","last_change":"2024-09-10T10:43:47.542199+0000","last_active":"2024-09-10T10:43:52.708757+0000","last_peered":"2024-09-10T10:43:52.708757+0000","last_clean":"2024-09-10T10:43:52.708757+0000","last_became_active":"2024-09-10T10:43:47.535131+0000","last_became_peered":"2024-09-10T10:43:47.535131+0000","last_unstale":"2024-09-10T10:43:52.708757+0000","last_undegraded":"2024-09-10T10:43:52.708757+0000","last_fullsized":"2024-09-10T10:43:52.708757+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:31:38.966819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870705+0000","last_change":"2024-09-10T10:43:47.546290+0000","last_active":"2024-09-10T10:43:52.870705+0000","last_peered":"2024-09-10T10:43:52.870705+0000","last_clean":"2024-09-10T10:43:52.870705+0000","last_became_active":"2024-09-10T10:43:47.545873+0000","last_became_peered":"2024-09-10T10:43:47.545873+0000","last_unstale":"2024-09-10T10:43:52.870705+0000","last_undegraded":"2024-09-10T10:43:52.870705+0000","last_fullsized":"2024-09-10T10:43:52.870705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:46.000406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870373+0000","last_change":"2024-09-10T10:43:47.537092+0000","last_active":"2024-09-10T10:43:52.870373+0000","last_peered":"2024-09-10T10:43:52.870373+0000","last_clean":"2024-09-10T10:43:52.870373+0000","last_became_active":"2024-09-10T10:43:47.536730+0000","last_became_peered":"2024-09-10T10:43:47.536730+0000","last_unstale":"2024-09-10T10:43:52.870373+0000","last_undegraded":"2024-09-10T10:43:52.870373+0000","last_fullsized":"2024-09-10T10:43:52.870373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:17.137462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564347+0000","last_change":"2024-09-10T10:43:47.557748+0000","last_active":"2024-09-10T10:43:52.564347+0000","last_peered":"2024-09-10T10:43:52.564347+0000","last_clean":"2024-09-10T10:43:52.564347+0000","last_became_active":"2024-09-10T10:43:47.555551+0000","last_became_peered":"2024-09-10T10:43:47.555551+0000","last_unstale":"2024-09-10T10:43:52.564347+0000","last_undegraded":"2024-09-10T10:43:52.564347+0000","last_fullsized":"2024-09-10T10:43:52.564347+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.162966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708691+0000","last_change":"2024-09-10T10:43:47.544639+0000","last_active":"2024-09-10T10:43:52.708691+0000","last_peered":"2024-09-10T10:43:52.708691+0000","last_clean":"2024-09-10T10:43:52.708691+0000","last_became_active":"2024-09-10T10:43:47.544357+0000","last_became_peered":"2024-09-10T10:43:47.544357+0000","last_unstale":"2024-09-10T10:43:52.708691+0000","last_undegraded":"2024-09-10T10:43:52.708691+0000","last_fullsized":"2024-09-10T10:43:52.708691+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:16.174872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868699+0000","last_change":"2024-09-10T10:43:47.544034+0000","last_active":"2024-09-10T10:43:52.868699+0000","last_peered":"2024-09-10T10:43:52.868699+0000","last_clean":"2024-09-10T10:43:52.868699+0000","last_became_active":"2024-09-10T10:43:47.543677+0000","last_became_peered":"2024-09-10T10:43:47.543677+0000","last_unstale":"2024-09-10T10:43:52.868699+0000","last_undegraded":"2024-09-10T10:43:52.868699+0000","last_fullsized":"2024-09-10T10:43:52.868699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:26.220907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708358+0000","last_change":"2024-09-10T10:43:47.561856+0000","last_active":"2024-09-10T10:43:52.708358+0000","last_peered":"2024-09-10T10:43:52.708358+0000","last_clean":"2024-09-10T10:43:52.708358+0000","last_became_active":"2024-09-10T10:43:47.561518+0000","last_became_peered":"2024-09-10T10:43:47.561518+0000","last_unstale":"2024-09-10T10:43:52.708358+0000","last_undegraded":"2024-09-10T10:43:52.708358+0000","last_fullsized":"2024-09-10T10:43:52.708358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:21.645129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564337+0000","last_change":"2024-09-10T10:43:47.545540+0000","last_active":"2024-09-10T10:43:52.564337+0000","last_peered":"2024-09-10T10:43:52.564337+0000","last_clean":"2024-09-10T10:43:52.564337+0000","last_became_active":"2024-09-10T10:43:47.544916+0000","last_became_peered":"2024-09-10T10:43:47.544916+0000","last_unstale":"2024-09-10T10:43:52.564337+0000","last_undegraded":"2024-09-10T10:43:52.564337+0000","last_fullsized":"2024-09-10T10:43:52.564337+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:34.658822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869156+0000","last_change":"2024-09-10T10:43:47.521227+0000","last_active":"2024-09-10T10:43:52.869156+0000","last_peered":"2024-09-10T10:43:52.869156+0000","last_clean":"2024-09-10T10:43:52.869156+0000","last_became_active":"2024-09-10T10:43:47.520811+0000","last_became_peered":"2024-09-10T10:43:47.520811+0000","last_unstale":"2024-09-10T10:43:52.869156+0000","last_undegraded":"2024-09-10T10:43:52.869156+0000","last_fullsized":"2024-09-10T10:43:52.869156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:55.708127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869588+0000","last_change":"2024-09-10T10:43:47.541681+0000","last_active":"2024-09-10T10:43:52.869588+0000","last_peered":"2024-09-10T10:43:52.869588+0000","last_clean":"2024-09-10T10:43:52.869588+0000","last_became_active":"2024-09-10T10:43:47.541056+0000","last_became_peered":"2024-09-10T10:43:47.541056+0000","last_unstale":"2024-09-10T10:43:52.869588+0000","last_undegraded":"2024-09-10T10:43:52.869588+0000","last_fullsized":"2024-09-10T10:43:52.869588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:41:56.224306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870640+0000","last_change":"2024-09-10T10:43:47.543423+0000","last_active":"2024-09-10T10:43:52.870640+0000","last_peered":"2024-09-10T10:43:52.870640+0000","last_clean":"2024-09-10T10:43:52.870640+0000","last_became_active":"2024-09-10T10:43:47.537728+0000","last_became_peered":"2024-09-10T10:43:47.537728+0000","last_unstale":"2024-09-10T10:43:52.870640+0000","last_undegraded":"2024-09-10T10:43:52.870640+0000","last_fullsized":"2024-09-10T10:43:52.870640+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:10.407503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564680+0000","last_change":"2024-09-10T10:43:47.562659+0000","last_active":"2024-09-10T10:43:52.564680+0000","last_peered":"2024-09-10T10:43:52.564680+0000","last_clean":"2024-09-10T10:43:52.564680+0000","last_became_active":"2024-09-10T10:43:47.562238+0000","last_became_peered":"2024-09-10T10:43:47.562238+0000","last_unstale":"2024-09-10T10:43:52.564680+0000","last_undegraded":"2024-09-10T10:43:52.564680+0000","last_fullsized":"2024-09-10T10:43:52.564680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870710+0000","last_change":"2024-09-10T10:43:47.548547+0000","last_active":"2024-09-10T10:43:52.870710+0000","last_peered":"2024-09-10T10:43:52.870710+0000","last_clean":"2024-09-10T10:43:52.870710+0000","last_became_active":"2024-09-10T10:43:47.548237+0000","last_became_peered":"2024-09-10T10:43:47.548237+0000","last_unstale":"2024-09-10T10:43:52.870710+0000","last_undegraded":"2024-09-10T10:43:52.870710+0000","last_fullsized":"2024-09-10T10:43:52.870710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:55.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.868674+0000","last_change":"2024-09-10T10:43:47.559813+0000","last_active":"2024-09-10T10:43:52.868674+0000","last_peered":"2024-09-10T10:43:52.868674+0000","last_clean":"2024-09-10T10:43:52.868674+0000","last_became_active":"2024-09-10T10:43:47.559517+0000","last_became_peered":"2024-09-10T10:43:47.559517+0000","last_unstale":"2024-09-10T10:43:52.868674+0000","last_undegraded":"2024-09-10T10:43:52.868674+0000","last_fullsized":"2024-09-10T10:43:52.868674+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:13:28.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.546737+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.546456+0000","last_became_peered":"2024-09-10T10:43:47.546456+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:42.017705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870025+0000","last_change":"2024-09-10T10:43:47.533339+0000","last_active":"2024-09-10T10:43:52.870025+0000","last_peered":"2024-09-10T10:43:52.870025+0000","last_clean":"2024-09-10T10:43:52.870025+0000","last_became_active":"2024-09-10T10:43:47.533019+0000","last_became_peered":"2024-09-10T10:43:47.533019+0000","last_unstale":"2024-09-10T10:43:52.870025+0000","last_undegraded":"2024-09-10T10:43:52.870025+0000","last_fullsized":"2024-09-10T10:43:52.870025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:15:49.441606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.870439+0000","last_change":"2024-09-10T10:43:47.546293+0000","last_active":"2024-09-10T10:43:52.870439+0000","last_peered":"2024-09-10T10:43:52.870439+0000","last_clean":"2024-09-10T10:43:52.870439+0000","last_became_active":"2024-09-10T10:43:47.545952+0000","last_became_peered":"2024-09-10T10:43:47.545952+0000","last_unstale":"2024-09-10T10:43:52.870439+0000","last_undegraded":"2024-09-10T10:43:52.870439+0000","last_fullsized":"2024-09-10T10:43:52.870439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:22:47.705884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869535+0000","last_change":"2024-09-10T10:43:47.541403+0000","last_active":"2024-09-10T10:43:52.869535+0000","last_peered":"2024-09-10T10:43:52.869535+0000","last_clean":"2024-09-10T10:43:52.869535+0000","last_became_active":"2024-09-10T10:43:47.540724+0000","last_became_peered":"2024-09-10T10:43:47.540724+0000","last_unstale":"2024-09-10T10:43:52.869535+0000","last_undegraded":"2024-09-10T10:43:52.869535+0000","last_fullsized":"2024-09-10T10:43:52.869535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:55.956488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869473+0000","last_change":"2024-09-10T10:43:47.550017+0000","last_active":"2024-09-10T10:43:52.869473+0000","last_peered":"2024-09-10T10:43:52.869473+0000","last_clean":"2024-09-10T10:43:52.869473+0000","last_became_active":"2024-09-10T10:43:47.549763+0000","last_became_peered":"2024-09-10T10:43:47.549763+0000","last_unstale":"2024-09-10T10:43:52.869473+0000","last_undegraded":"2024-09-10T10:43:52.869473+0000","last_fullsized":"2024-09-10T10:43:52.869473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:30:12.076487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869399+0000","last_change":"2024-09-10T10:43:47.558985+0000","last_active":"2024-09-10T10:43:52.869399+0000","last_peered":"2024-09-10T10:43:52.869399+0000","last_clean":"2024-09-10T10:43:52.869399+0000","last_became_active":"2024-09-10T10:43:47.558695+0000","last_became_peered":"2024-09-10T10:43:47.558695+0000","last_unstale":"2024-09-10T10:43:52.869399+0000","last_undegraded":"2024-09-10T10:43:52.869399+0000","last_fullsized":"2024-09-10T10:43:52.869399+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:05.319579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.565366+0000","last_change":"2024-09-10T10:43:47.547413+0000","last_active":"2024-09-10T10:43:52.565366+0000","last_peered":"2024-09-10T10:43:52.565366+0000","last_clean":"2024-09-10T10:43:52.565366+0000","last_became_active":"2024-09-10T10:43:47.547008+0000","last_became_peered":"2024-09-10T10:43:47.547008+0000","last_unstale":"2024-09-10T10:43:52.565366+0000","last_undegraded":"2024-09-10T10:43:52.565366+0000","last_fullsized":"2024-09-10T10:43:52.565366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708197+0000","last_change":"2024-09-10T10:43:47.545409+0000","last_active":"2024-09-10T10:43:52.708197+0000","last_peered":"2024-09-10T10:43:52.708197+0000","last_clean":"2024-09-10T10:43:52.708197+0000","last_became_active":"2024-09-10T10:43:47.545121+0000","last_became_peered":"2024-09-10T10:43:47.545121+0000","last_unstale":"2024-09-10T10:43:52.708197+0000","last_undegraded":"2024-09-10T10:43:52.708197+0000","last_fullsized":"2024-09-10T10:43:52.708197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:19.641844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869042+0000","last_change":"2024-09-10T10:43:47.534510+0000","last_active":"2024-09-10T10:43:52.869042+0000","last_peered":"2024-09-10T10:43:52.869042+0000","last_clean":"2024-09-10T10:43:52.869042+0000","last_became_active":"2024-09-10T10:43:47.534143+0000","last_became_peered":"2024-09-10T10:43:47.534143+0000","last_unstale":"2024-09-10T10:43:52.869042+0000","last_undegraded":"2024-09-10T10:43:52.869042+0000","last_fullsized":"2024-09-10T10:43:52.869042+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:33.210661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.563579+0000","last_change":"2024-09-10T10:43:47.525750+0000","last_active":"2024-09-10T10:43:52.563579+0000","last_peered":"2024-09-10T10:43:52.563579+0000","last_clean":"2024-09-10T10:43:52.563579+0000","last_became_active":"2024-09-10T10:43:47.525379+0000","last_became_peered":"2024-09-10T10:43:47.525379+0000","last_unstale":"2024-09-10T10:43:52.563579+0000","last_undegraded":"2024-09-10T10:43:52.563579+0000","last_fullsized":"2024-09-10T10:43:52.563579+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:39.336457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869264+0000","last_change":"2024-09-10T10:43:47.546839+0000","last_active":"2024-09-10T10:43:52.869264+0000","last_peered":"2024-09-10T10:43:52.869264+0000","last_clean":"2024-09-10T10:43:52.869264+0000","last_became_active":"2024-09-10T10:43:47.544570+0000","last_became_peered":"2024-09-10T10:43:47.544570+0000","last_unstale":"2024-09-10T10:43:52.869264+0000","last_undegraded":"2024-09-10T10:43:52.869264+0000","last_fullsized":"2024-09-10T10:43:52.869264+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:53.642976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.564645+0000","last_change":"2024-09-10T10:43:47.522224+0000","last_active":"2024-09-10T10:43:52.564645+0000","last_peered":"2024-09-10T10:43:52.564645+0000","last_clean":"2024-09-10T10:43:52.564645+0000","last_became_active":"2024-09-10T10:43:47.521820+0000","last_became_peered":"2024-09-10T10:43:47.521820+0000","last_unstale":"2024-09-10T10:43:52.564645+0000","last_undegraded":"2024-09-10T10:43:52.564645+0000","last_fullsized":"2024-09-10T10:43:52.564645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:24:41.803538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708908+0000","last_change":"2024-09-10T10:43:47.538393+0000","last_active":"2024-09-10T10:43:52.708908+0000","last_peered":"2024-09-10T10:43:52.708908+0000","last_clean":"2024-09-10T10:43:52.708908+0000","last_became_active":"2024-09-10T10:43:47.537972+0000","last_became_peered":"2024-09-10T10:43:47.537972+0000","last_unstale":"2024-09-10T10:43:52.708908+0000","last_undegraded":"2024-09-10T10:43:52.708908+0000","last_fullsized":"2024-09-10T10:43:52.708908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:30.153548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.708731+0000","last_change":"2024-09-10T10:43:47.540588+0000","last_active":"2024-09-10T10:43:52.708731+0000","last_peered":"2024-09-10T10:43:52.708731+0000","last_clean":"2024-09-10T10:43:52.708731+0000","last_became_active":"2024-09-10T10:43:47.540358+0000","last_became_peered":"2024-09-10T10:43:47.540358+0000","last_unstale":"2024-09-10T10:43:52.708731+0000","last_undegraded":"2024-09-10T10:43:52.708731+0000","last_fullsized":"2024-09-10T10:43:52.708731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:30.489388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T10:43:52.869991+0000","last_change":"2024-09-10T10:43:47.549392+0000","last_active":"2024-09-10T10:43:52.869991+0000","last_peered":"2024-09-10T10:43:52.869991+0000","last_clean":"2024-09-10T10:43:52.869991+0000","last_became_active":"2024-09-10T10:43:47.546137+0000","last_became_peered":"2024-09-10T10:43:47.546137+0000","last_unstale":"2024-09-10T10:43:52.869991+0000","last_undegraded":"2024-09-10T10:43:52.869991+0000","last_fullsized":"2024-09-10T10:43:52.869991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:46.495239+0000","last_clean_scrub_stamp":"2024-09-10T10:43:46.495239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:02.975066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":111561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":111437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":115953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":286720,"data_stored":112019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":115281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":701875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":290816,"data_stored":111493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":884736,"data_stored":701897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:44:13.572 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:44:13.572 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T10:44:13.572 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T10:44:13.572 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T10:44:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.762+0000 7effac600640 1 Processor -- start 2024-09-10T10:44:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.762+0000 7effac600640 1 -- start start 2024-09-10T10:44:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.763+0000 7effac600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effa4147360 0x7effa4070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.763+0000 7effac600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.763+0000 7effac600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa406cb30 con 0x7effa40d09e0 2024-09-10T10:44:13.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.763+0000 7effac600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7effa406bfb0 con 0x7effa4147360 2024-09-10T10:44:13.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.763+0000 7effac600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effa406c290 con 0x7effa4070dd0 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effa4147360 0x7effa4070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44458/0 (socket says 172.21.15.28:44458) 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9200640 1 -- 172.21.15.28:0/2845250824 learned_addr learned my addr 172.21.15.28:0/2845250824 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9c00640 1 -- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 msgr2=0x7effa40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9c00640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa9c00640 1 -- 172.21.15.28:0/2845250824 >> v1:172.21.15.138:6789/0 conn(0x7effa40d09e0 legacy=0x7effa406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.765+0000 7effa9c00640 1 -- 172.21.15.28:0/2845250824 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa406c570 con 0x7effa4147360 2024-09-10T10:44:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.765+0000 7effa9200640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:13.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.764+0000 7effa3e00640 1 -- 172.21.15.28:0/2845250824 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219439342 0 0) 0x7effa406cb30 con 0x7effa40d09e0 2024-09-10T10:44:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effa9c00640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa4147360 0x7effa4070890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff980097a0 tx=0x7eff9802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fce0dfa3ab7566ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effa3e00640 1 -- 172.21.15.28:0/2845250824 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98037070 con 0x7effa4147360 2024-09-10T10:44:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effa3e00640 1 -- 172.21.15.28:0/2845250824 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9802ae50 con 0x7effa4147360 2024-09-10T10:44:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effa3e00640 1 -- 172.21.15.28:0/2845250824 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98032660 con 0x7effa4147360 2024-09-10T10:44:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effac600640 1 -- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa4147360 msgr2=0x7effa4070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.766+0000 7effac600640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa4147360 0x7effa4070890 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff980097a0 tx=0x7eff9802a100 comp rx=0 tx=0).stop 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 -- 172.21.15.28:0/2845250824 shutdown_connections 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4070dd0 0x7effa40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 --2- 172.21.15.28:0/2845250824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa4147360 0x7effa4070890 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 -- 172.21.15.28:0/2845250824 >> 172.21.15.28:0/2845250824 conn(0x7effa40b2f40 msgr2=0x7effa40b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 -- 172.21.15.28:0/2845250824 shutdown_connections 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.767+0000 7effac600640 1 -- 172.21.15.28:0/2845250824 wait complete. 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.768+0000 7effac600640 1 Processor -- start 2024-09-10T10:44:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.768+0000 7effac600640 1 -- start start 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 0x7effa41619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 0x7effa4140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7effa406bc10 con 0x7effa4070dd0 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7effa40d0e10 con 0x7effa40d09e0 2024-09-10T10:44:13.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effac600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effa406bd90 con 0x7effa4147360 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effa9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.769+0000 7effa9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36770/0 (socket says 172.21.15.28:36770) 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 -- 172.21.15.28:0/2754244923 learned_addr learned my addr 172.21.15.28:0/2754244923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9200640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 0x7effa41619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effaa600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 0x7effa4140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 -- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 msgr2=0x7effa4140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 0x7effa4140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 -- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 msgr2=0x7effa41619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 0x7effa41619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 -- 172.21.15.28:0/2754244923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa4141470 con 0x7effa4070dd0 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9200640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 0x7effa41619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:13.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effaa600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 0x7effa4140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa9c00640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff980097a0 tx=0x7eff980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.770+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98037040 con 0x7effa4070dd0 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.771+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff98004030 con 0x7effa4070dd0 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.771+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98004420 con 0x7effa4070dd0 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.771+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa4141750 con 0x7effa4070dd0 2024-09-10T10:44:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.771+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa4144890 con 0x7effa4070dd0 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.772+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa40d04a0 con 0x7effa4070dd0 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.773+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff98003610 con 0x7effa4070dd0 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.773+0000 7effa1600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 0x7eff8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.773+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7eff980ac110 con 0x7effa4070dd0 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.774+0000 7effa9200640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 0x7eff8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:13.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.775+0000 7effa9200640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 0x7eff8406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff90009630 tx=0x7eff90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:13.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.778+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff9807a800 con 0x7effa4070dd0 2024-09-10T10:44:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.998+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7effa4070400 con 0x7effa4070dd0 2024-09-10T10:44:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:13.999+0000 7effa1600640 1 -- 172.21.15.28:0/2754244923 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7eff9807a100 con 0x7effa4070dd0 2024-09-10T10:44:14.000 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:14.000 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 msgr2=0x7eff8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 0x7eff8406e2f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7eff90009630 tx=0x7eff90008040 comp rx=0 tx=0).stop 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 msgr2=0x7effa413fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff980097a0 tx=0x7eff980046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 shutdown_connections 2024-09-10T10:44:14.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff8406be30 0x7eff8406e2f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effa4147360 0x7effa4140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa40d09e0 0x7effa41619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 --2- 172.21.15.28:0/2754244923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effa4070dd0 0x7effa413fd90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 >> 172.21.15.28:0/2754244923 conn(0x7effa40b2f40 msgr2=0x7effa406f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 shutdown_connections 2024-09-10T10:44:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.001+0000 7effac600640 1 -- 172.21.15.28:0/2754244923 wait complete. 2024-09-10T10:44:14.017 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T10:44:14.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:14.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.172+0000 7f8237400640 1 Processor -- start 2024-09-10T10:44:14.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.172+0000 7f8237400640 1 -- start start 2024-09-10T10:44:14.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8237400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8230161c80 0x7f823006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8237400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 0x7f82300b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8237400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f823006e870 con 0x7f82300b64b0 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8237400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f823006d9f0 con 0x7f8230161c80 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8237400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f823006dcd0 con 0x7f8230162060 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.173+0000 7f8234a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8230161c80 0x7f823006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8230161c80 0x7f823006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39576/0 (socket says 172.21.15.28:39576) 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 -- 172.21.15.28:0/3640225743 learned_addr learned my addr 172.21.15.28:0/3640225743 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f822fe00640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 0x7f82300b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 -- 172.21.15.28:0/3640225743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 msgr2=0x7f82300b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 0x7f82300b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 -- 172.21.15.28:0/3640225743 >> v1:172.21.15.28:6789/0 conn(0x7f82300b64b0 legacy=0x7f8230162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f8234a00640 1 -- 172.21.15.28:0/3640225743 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823006dfb0 con 0x7f8230161c80 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.174+0000 7f822fe00640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 0x7f82300b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.175+0000 7f8234a00640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230161c80 0x7f823006e330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8224009870 tx=0x7f822402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df9963822061f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.175+0000 7f822f400640 1 -- 172.21.15.28:0/3640225743 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224037070 con 0x7f8230161c80 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.175+0000 7f822f400640 1 -- 172.21.15.28:0/3640225743 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8224032440 con 0x7f8230161c80 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.175+0000 7f822f400640 1 -- 172.21.15.28:0/3640225743 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224032810 con 0x7f8230161c80 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 -- 172.21.15.28:0/3640225743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230161c80 msgr2=0x7f823006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230161c80 0x7f823006e330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8224009870 tx=0x7f822402a250 comp rx=0 tx=0).stop 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 -- 172.21.15.28:0/3640225743 shutdown_connections 2024-09-10T10:44:14.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230162060 0x7f82300b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 --2- 172.21.15.28:0/3640225743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230161c80 0x7f823006e330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 -- 172.21.15.28:0/3640225743 >> 172.21.15.28:0/3640225743 conn(0x7f82300d44f0 msgr2=0x7f82300d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 -- 172.21.15.28:0/3640225743 shutdown_connections 2024-09-10T10:44:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.176+0000 7f8237400640 1 -- 172.21.15.28:0/3640225743 wait complete. 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.178+0000 7f8237400640 1 Processor -- start 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.178+0000 7f8237400640 1 -- start start 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.178+0000 7f8237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82300b64b0 0x7f8230155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.178+0000 7f8237400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230161c80 0x7f82300b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8237400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8237400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f82300d2a10 con 0x7f8230162060 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8237400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82300d2890 con 0x7f82300b64b0 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8237400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82300b6960 con 0x7f8230161c80 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39586/0 (socket says 172.21.15.28:39586) 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 -- 172.21.15.28:0/1705813880 learned_addr learned my addr 172.21.15.28:0/1705813880 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8234a00640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82300b64b0 0x7f8230155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f822fe00640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230161c80 0x7f82300b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 -- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230161c80 msgr2=0x7f82300b8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230161c80 0x7f82300b8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 -- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82300b64b0 msgr2=0x7f8230155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82300b64b0 0x7f8230155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 -- 172.21.15.28:0/1705813880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82300c87e0 con 0x7f8230162060 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.179+0000 7f8235400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f822800c7d0 tx=0x7f822800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.180+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228003b70 con 0x7f8230162060 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.180+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228003d30 con 0x7f8230162060 2024-09-10T10:44:14.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.180+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822800aa90 con 0x7f8230162060 2024-09-10T10:44:14.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.180+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82300c8a80 con 0x7f8230162060 2024-09-10T10:44:14.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.180+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82300c9040 con 0x7f8230162060 2024-09-10T10:44:14.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.181+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823006e330 con 0x7f8230162060 2024-09-10T10:44:14.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.182+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f822800ac30 con 0x7f8230162060 2024-09-10T10:44:14.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.183+0000 7f822cc00640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 0x7f821006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.183+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f822808b160 con 0x7f8230162060 2024-09-10T10:44:14.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.183+0000 7f8234a00640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 0x7f821006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.184+0000 7f8234a00640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 0x7f821006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.187+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f823006e870 con 0x7f8230162060 2024-09-10T10:44:14.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.407+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82300ba470 con 0x7f8230162060 2024-09-10T10:44:14.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.408+0000 7f822cc00640 1 -- 172.21.15.28:0/1705813880 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f82280597c0 con 0x7f8230162060 2024-09-10T10:44:14.409 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:14.409 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:14.409 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:14.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 msgr2=0x7f821006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 0x7f821006e2f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8224034000 tx=0x7f822402e000 comp rx=0 tx=0).stop 2024-09-10T10:44:14.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 msgr2=0x7f823015e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f822800c7d0 tx=0x7f822800cc90 comp rx=0 tx=0).stop 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 shutdown_connections 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f821006be30 0x7f821006e2f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8230162060 0x7f823015e940 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8230161c80 0x7f82300b8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 --2- 172.21.15.28:0/1705813880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82300b64b0 0x7f8230155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 >> 172.21.15.28:0/1705813880 conn(0x7f82300d44f0 msgr2=0x7f823006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 shutdown_connections 2024-09-10T10:44:14.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.410+0000 7f8237400640 1 -- 172.21.15.28:0/1705813880 wait complete. 2024-09-10T10:44:14.427 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:14.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.584+0000 7fb79b000640 1 Processor -- start 2024-09-10T10:44:14.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.584+0000 7fb79b000640 1 -- start start 2024-09-10T10:44:14.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb794161c80 0x7fb79406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb794162060 0x7fb7940b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb7940b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb79406e870 con 0x7fb794162060 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb79406d9f0 con 0x7fb794161c80 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb79b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb79406dcd0 con 0x7fb7940b64b0 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb78b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb794162060 0x7fb7940b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb78b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb794162060 0x7fb7940b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39594/0 (socket says 172.21.15.28:39594) 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb78b400640 1 -- 172.21.15.28:0/3926233972 learned_addr learned my addr 172.21.15.28:0/3926233972 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb793e00640 1 --2- 172.21.15.28:0/3926233972 >> v2:172.21.15.138:3300/0 conn(0x7fb794161c80 0x7fb79406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb799000640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb7940b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.585+0000 7fb78b400640 1 -- 172.21.15.28:0/3926233972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 msgr2=0x7fb7940b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.586+0000 7fb78b400640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb7940b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.586+0000 7fb78b400640 1 -- 172.21.15.28:0/3926233972 >> v2:172.21.15.138:3300/0 conn(0x7fb794161c80 msgr2=0x7fb79406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.586+0000 7fb78b400640 1 --2- 172.21.15.28:0/3926233972 >> v2:172.21.15.138:3300/0 conn(0x7fb794161c80 0x7fb79406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.586+0000 7fb78b400640 1 -- 172.21.15.28:0/3926233972 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79406dfb0 con 0x7fb794162060 2024-09-10T10:44:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.586+0000 7fb799000640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb7940b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb78b400640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb7940b5f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb79408d7a0 tx=0x7fb77c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300d7154c4e1f629 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb793400640 1 -- 172.21.15.28:0/3926233972 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c037070 con 0x7fb794162060 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb793400640 1 -- 172.21.15.28:0/3926233972 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c032440 con 0x7fb794162060 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb793400640 1 -- 172.21.15.28:0/3926233972 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c032810 con 0x7fb794162060 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 -- 172.21.15.28:0/3926233972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 msgr2=0x7fb7940b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb7940b5f70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb79408d7a0 tx=0x7fb77c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 -- 172.21.15.28:0/3926233972 shutdown_connections 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb7940b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 --2- 172.21.15.28:0/3926233972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb7940b5f70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 --2- 172.21.15.28:0/3926233972 >> v2:172.21.15.138:3300/0 conn(0x7fb794161c80 0x7fb79406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.587+0000 7fb79b000640 1 -- 172.21.15.28:0/3926233972 >> 172.21.15.28:0/3926233972 conn(0x7fb7940d44f0 msgr2=0x7fb7940d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.588+0000 7fb79b000640 1 -- 172.21.15.28:0/3926233972 shutdown_connections 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.588+0000 7fb79b000640 1 -- 172.21.15.28:0/3926233972 wait complete. 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.589+0000 7fb79b000640 1 Processor -- start 2024-09-10T10:44:14.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.589+0000 7fb79b000640 1 -- start start 2024-09-10T10:44:14.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb794155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb794161c80 0x7fb7940b9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb794162500 con 0x7fb794162060 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7940d2aa0 con 0x7fb794161c80 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb79b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb794162680 con 0x7fb7940b64b0 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb799000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb799000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39598/0 (socket says 172.21.15.28:39598) 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.590+0000 7fb799000640 1 -- 172.21.15.28:0/210338408 learned_addr learned my addr 172.21.15.28:0/210338408 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb793e00640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb794155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb78b400640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb794161c80 0x7fb7940b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 -- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 msgr2=0x7fb794155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb794155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 -- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb794161c80 msgr2=0x7fb7940b9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb794161c80 0x7fb7940b9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 -- 172.21.15.28:0/210338408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7940cb5c0 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb799000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb78c007e10 tx=0x7fb78c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c01a070 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb78c003b90 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.591+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7940b9e30 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.592+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c004030 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.592+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7940c8180 con 0x7fb794162060 2024-09-10T10:44:14.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.593+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79406e330 con 0x7fb794162060 2024-09-10T10:44:14.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.593+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb78c01d070 con 0x7fb794162060 2024-09-10T10:44:14.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.594+0000 7fb790c00640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 0x7fb77006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.594+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb78c08b3e0 con 0x7fb794162060 2024-09-10T10:44:14.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.595+0000 7fb793e00640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 0x7fb77006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.596+0000 7fb793e00640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 0x7fb77006e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb784009b40 tx=0x7fb784009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:14.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.598+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb78c010070 con 0x7fb794162060 2024-09-10T10:44:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.819+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7940ba470 con 0x7fb794162060 2024-09-10T10:44:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.819+0000 7fb790c00640 1 -- 172.21.15.28:0/210338408 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fb78c0593f0 con 0x7fb794162060 2024-09-10T10:44:14.821 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:14.821 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 msgr2=0x7fb77006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 0x7fb77006e2f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb784009b40 tx=0x7fb784009190 comp rx=0 tx=0).stop 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 msgr2=0x7fb79415ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb78c007e10 tx=0x7fb78c00d4a0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 shutdown_connections 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb77006be30 0x7fb77006e2f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb794162060 0x7fb79415ee80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb794161c80 0x7fb7940b9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 --2- 172.21.15.28:0/210338408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7940b64b0 0x7fb794155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 >> 172.21.15.28:0/210338408 conn(0x7fb7940d44f0 msgr2=0x7fb79406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 shutdown_connections 2024-09-10T10:44:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.822+0000 7fb79b000640 1 -- 172.21.15.28:0/210338408 wait complete. 2024-09-10T10:44:14.839 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.994+0000 7fd523e00640 1 Processor -- start 2024-09-10T10:44:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.995+0000 7fd523e00640 1 -- start start 2024-09-10T10:44:14.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5240d0dd0 0x7fd5240d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd52415e660 0x7fd52406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:14.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5240cf000 con 0x7fd5240d1f90 2024-09-10T10:44:14.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd5240b37d0 con 0x7fd52415e660 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd5240cf330 con 0x7fd5240d0dd0 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd52415e660 0x7fd52406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd5240d1f90 0x7fd5240cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41658/0 (socket says 172.21.15.28:41658) 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd52415e660 0x7fd52406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39606/0 (socket says 172.21.15.28:39606) 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd523400640 1 -- 172.21.15.28:0/176773936 learned_addr learned my addr 172.21.15.28:0/176773936 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd522a00640 1 --2- 172.21.15.28:0/176773936 >> v2:172.21.15.138:3300/0 conn(0x7fd5240d0dd0 0x7fd5240d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 -- 172.21.15.28:0/176773936 >> v2:172.21.15.138:3300/0 conn(0x7fd5240d0dd0 msgr2=0x7fd5240d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 --2- 172.21.15.28:0/176773936 >> v2:172.21.15.138:3300/0 conn(0x7fd5240d0dd0 0x7fd5240d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 -- 172.21.15.28:0/176773936 >> v1:172.21.15.28:6789/0 conn(0x7fd5240d1f90 legacy=0x7fd5240cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:14.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.996+0000 7fd51be00640 1 -- 172.21.15.28:0/176773936 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5240d16f0 con 0x7fd52415e660 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.997+0000 7fd51be00640 1 --2- 172.21.15.28:0/176773936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd52406ccd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b6fb0 tx=0x7fd50c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40f6101ab1938f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd522000640 1 -- 172.21.15.28:0/176773936 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c037070 con 0x7fd52415e660 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd522000640 1 -- 172.21.15.28:0/176773936 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd50c032440 con 0x7fd52415e660 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd522000640 1 -- 172.21.15.28:0/176773936 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50c032810 con 0x7fd52415e660 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 -- 172.21.15.28:0/176773936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 msgr2=0x7fd52406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 --2- 172.21.15.28:0/176773936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd52406ccd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b6fb0 tx=0x7fd50c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 -- 172.21.15.28:0/176773936 shutdown_connections 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 --2- 172.21.15.28:0/176773936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd52406ccd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 --2- 172.21.15.28:0/176773936 >> v2:172.21.15.138:3300/0 conn(0x7fd5240d0dd0 0x7fd5240d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.998+0000 7fd523e00640 1 -- 172.21.15.28:0/176773936 >> 172.21.15.28:0/176773936 conn(0x7fd52408bfc0 msgr2=0x7fd5240d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.999+0000 7fd523e00640 1 -- 172.21.15.28:0/176773936 shutdown_connections 2024-09-10T10:44:15.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:14.999+0000 7fd523e00640 1 -- 172.21.15.28:0/176773936 wait complete. 2024-09-10T10:44:15.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.000+0000 7fd523e00640 1 Processor -- start 2024-09-10T10:44:15.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.000+0000 7fd523e00640 1 -- start start 2024-09-10T10:44:15.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5240d0dd0 0x7fd524156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5240d1f90 0x7fd5241e8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd5240d2f40 con 0x7fd52415e660 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5240d2c60 con 0x7fd5240d1f90 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd5240d3220 con 0x7fd5240d0dd0 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39616/0 (socket says 172.21.15.28:39616) 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.001+0000 7fd523400640 1 -- 172.21.15.28:0/715825562 learned_addr learned my addr 172.21.15.28:0/715825562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd51be00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5240d1f90 0x7fd5241e8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd522a00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5240d0dd0 0x7fd524156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 -- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5240d0dd0 msgr2=0x7fd524156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5240d0dd0 0x7fd524156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 -- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5240d1f90 msgr2=0x7fd5241e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5240d1f90 0x7fd5241e8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 -- 172.21.15.28:0/715825562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52415ea40 con 0x7fd52415e660 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd523400640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00cbc0 tx=0x7fd51c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c017070 con 0x7fd52415e660 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c00ad40 con 0x7fd52415e660 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.002+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c012600 con 0x7fd52415e660 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.003+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52415ed20 con 0x7fd52415e660 2024-09-10T10:44:15.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.003+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52415f2e0 con 0x7fd52415e660 2024-09-10T10:44:15.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.004+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5240d2420 con 0x7fd52415e660 2024-09-10T10:44:15.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.005+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd51c01a070 con 0x7fd52415e660 2024-09-10T10:44:15.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.005+0000 7fd51b400640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 0x7fd50006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.006+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd51c08ade0 con 0x7fd52415e660 2024-09-10T10:44:15.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.006+0000 7fd522a00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 0x7fd50006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.007+0000 7fd522a00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 0x7fd50006e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd5140079a0 tx=0x7fd514008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.010+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd51c058e90 con 0x7fd52415e660 2024-09-10T10:44:15.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.199+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5240b1420 con 0x7fd52415e660 2024-09-10T10:44:15.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.201+0000 7fd51b400640 1 -- 172.21.15.28:0/715825562 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18626 (secure 0 0 0) 0x7fd51c05d190 con 0x7fd52415e660 2024-09-10T10:44:15.202 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:15.202 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:43:51.987916+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:15.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 msgr2=0x7fd50006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 0x7fd50006e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd5140079a0 tx=0x7fd514008040 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 msgr2=0x7fd5240cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00cbc0 tx=0x7fd51c00a430 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 shutdown_connections 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd50006be30 0x7fd50006e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd52415e660 0x7fd5240cd9e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5240d1f90 0x7fd5241e8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 --2- 172.21.15.28:0/715825562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5240d0dd0 0x7fd524156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 >> 172.21.15.28:0/715825562 conn(0x7fd52408bfc0 msgr2=0x7fd524157b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 shutdown_connections 2024-09-10T10:44:15.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.203+0000 7fd523e00640 1 -- 172.21.15.28:0/715825562 wait complete. 2024-09-10T10:44:15.220 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.375+0000 7f333dc00640 1 Processor -- start 2024-09-10T10:44:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.375+0000 7f333dc00640 1 -- start start 2024-09-10T10:44:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33380d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f333815e660 0x7f333806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 0x7f33380cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f33380b37d0 con 0x7f333815e660 2024-09-10T10:44:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f33380cf330 con 0x7f33380d1f90 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f333dc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f33380cf000 con 0x7f33380d0dd0 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f3337400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f333815e660 0x7f333806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.376+0000 7f3337e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33380d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f333815e660 0x7f333806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39632/0 (socket says 172.21.15.28:39632) 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 -- 172.21.15.28:0/2505249925 learned_addr learned my addr 172.21.15.28:0/2505249925 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f333d200640 1 --2- 172.21.15.28:0/2505249925 >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 0x7f33380cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 -- 172.21.15.28:0/2505249925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 msgr2=0x7f33380d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 --2- 172.21.15.28:0/2505249925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33380d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 -- 172.21.15.28:0/2505249925 >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 msgr2=0x7f33380cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 --2- 172.21.15.28:0/2505249925 >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 0x7f33380cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f3337400640 1 -- 172.21.15.28:0/2505249925 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33380d16f0 con 0x7f333815e660 2024-09-10T10:44:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.377+0000 7f333d200640 1 --2- 172.21.15.28:0/2505249925 >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 0x7f33380cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.378+0000 7f3337400640 1 --2- 172.21.15.28:0/2505249925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f333806ccd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33380cb420 tx=0x7f332402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dddc0f87be2ffe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.378+0000 7f3336a00640 1 -- 172.21.15.28:0/2505249925 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324037070 con 0x7f333815e660 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.378+0000 7f3336a00640 1 -- 172.21.15.28:0/2505249925 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3324032440 con 0x7f333815e660 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.378+0000 7f3336a00640 1 -- 172.21.15.28:0/2505249925 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324032810 con 0x7f333815e660 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 -- 172.21.15.28:0/2505249925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 msgr2=0x7f333806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 --2- 172.21.15.28:0/2505249925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f333806ccd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f33380cb420 tx=0x7f332402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 -- 172.21.15.28:0/2505249925 shutdown_connections 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 --2- 172.21.15.28:0/2505249925 >> v2:172.21.15.138:3300/0 conn(0x7f33380d1f90 0x7f33380cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 --2- 172.21.15.28:0/2505249925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f333806ccd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 --2- 172.21.15.28:0/2505249925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33380d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 -- 172.21.15.28:0/2505249925 >> 172.21.15.28:0/2505249925 conn(0x7f333808bfc0 msgr2=0x7f33380d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 -- 172.21.15.28:0/2505249925 shutdown_connections 2024-09-10T10:44:15.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.379+0000 7f333dc00640 1 -- 172.21.15.28:0/2505249925 wait complete. 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.381+0000 7f333dc00640 1 Processor -- start 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.381+0000 7f333dc00640 1 -- start start 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33381e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33380d1f90 0x7f33381e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f33380d2f40 con 0x7f333815e660 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33380d2c60 con 0x7f33380d1f90 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333dc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f33380d3220 con 0x7f33380d0dd0 2024-09-10T10:44:15.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39646/0 (socket says 172.21.15.28:39646) 2024-09-10T10:44:15.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 -- 172.21.15.28:0/3107514684 learned_addr learned my addr 172.21.15.28:0/3107514684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 -- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 msgr2=0x7f33381e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33381e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 -- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33380d1f90 msgr2=0x7f33381e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33380d1f90 0x7f33381e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.382+0000 7f333d200640 1 -- 172.21.15.28:0/3107514684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381e3ec0 con 0x7f333815e660 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f333d200640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ef0 tx=0x7f333000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333001a070 con 0x7f333815e660 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3330003b90 con 0x7f333815e660 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004030 con 0x7f333815e660 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381e41a0 con 0x7f333815e660 2024-09-10T10:44:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.383+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381e4760 con 0x7f333815e660 2024-09-10T10:44:15.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.384+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f333815dea0 con 0x7f333815e660 2024-09-10T10:44:15.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.385+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f333001d070 con 0x7f333815e660 2024-09-10T10:44:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.386+0000 7f332fe00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 0x7f331806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.386+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f333008b450 con 0x7f333815e660 2024-09-10T10:44:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.386+0000 7f3337e00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 0x7f331806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:15.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.387+0000 7f3337e00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 0x7f331806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3328009a40 tx=0x7f3328009500 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:15.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.390+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f333815dea0 con 0x7f333815e660 2024-09-10T10:44:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.616+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33380d2440 con 0x7f333815e660 2024-09-10T10:44:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.617+0000 7f332fe00640 1 -- 172.21.15.28:0/3107514684 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f3330010070 con 0x7f333815e660 2024-09-10T10:44:15.618 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:15.618 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 msgr2=0x7f331806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 0x7f331806e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3328009a40 tx=0x7f3328009500 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 msgr2=0x7f33380b9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3330007ef0 tx=0x7f333000d4a0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 shutdown_connections 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331806be30 0x7f331806e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f333815e660 0x7f33380b9690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33380d1f90 0x7f33381e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 --2- 172.21.15.28:0/3107514684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33380d0dd0 0x7f33381e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 >> 172.21.15.28:0/3107514684 conn(0x7f333808bfc0 msgr2=0x7f33380b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 shutdown_connections 2024-09-10T10:44:15.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:15.619+0000 7f333dc00640 1 -- 172.21.15.28:0/3107514684 wait complete. 2024-09-10T10:44:16.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:43:46.939715+0000', 'modified': '2024-09-10T10:44:08.023962+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4284}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4284': {'gid': 4284, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6835/112106318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 112106318}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 112106318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4284, 'qdb_cluster': [4284]} 2024-09-10T10:44:16.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:44:16.039 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:44:16.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.199+0000 7f713d000640 1 Processor -- start 2024-09-10T10:44:16.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.199+0000 7f713d000640 1 -- start start 2024-09-10T10:44:16.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.200+0000 7f713d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f71380b69d0 0x7f7138162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.200+0000 7f713d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f7138154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.200+0000 7f713d000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713815eb60 con 0x7f71380af280 2024-09-10T10:44:16.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.200+0000 7f713d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f71380b6db0 con 0x7f71380b69d0 2024-09-10T10:44:16.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.200+0000 7f713d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f713815ee40 con 0x7f71380b0180 2024-09-10T10:44:16.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f71380af280 0x7f71380afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41664/0 (socket says 172.21.15.28:41664) 2024-09-10T10:44:16.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135400640 1 -- 172.21.15.28:0/1102819541 learned_addr learned my addr 172.21.15.28:0/1102819541 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135e00640 1 --2- 172.21.15.28:0/1102819541 >> v2:172.21.15.138:3300/0 conn(0x7f71380b69d0 0x7f7138162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7136800640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f7138154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7134a00640 1 -- 172.21.15.28:0/1102819541 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1636117522 0 0) 0x7f713815eb60 con 0x7f71380af280 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7134a00640 1 -- 172.21.15.28:0/1102819541 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f711c003520 con 0x7f71380af280 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135e00640 1 -- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 msgr2=0x7f7138154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135e00640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f7138154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135e00640 1 -- 172.21.15.28:0/1102819541 >> v1:172.21.15.28:6789/0 conn(0x7f71380af280 legacy=0x7f71380afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.201+0000 7f7135e00640 1 -- 172.21.15.28:0/1102819541 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713815f120 con 0x7f71380b69d0 2024-09-10T10:44:16.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.202+0000 7f7135e00640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f7138162a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c3914365dd819db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f7134a00640 1 -- 172.21.15.28:0/1102819541 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128037070 con 0x7f71380b69d0 2024-09-10T10:44:16.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f7134a00640 1 -- 172.21.15.28:0/1102819541 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712802adb0 con 0x7f71380b69d0 2024-09-10T10:44:16.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f7134a00640 1 -- 172.21.15.28:0/1102819541 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128032640 con 0x7f71380b69d0 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 -- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 msgr2=0x7f7138162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f7138162a50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71280097a0 tx=0x7f712802a060 comp rx=0 tx=0).stop 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 -- 172.21.15.28:0/1102819541 shutdown_connections 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f7138154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 --2- 172.21.15.28:0/1102819541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f7138162a50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 -- 172.21.15.28:0/1102819541 >> 172.21.15.28:0/1102819541 conn(0x7f713806bc10 msgr2=0x7f713806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 -- 172.21.15.28:0/1102819541 shutdown_connections 2024-09-10T10:44:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.203+0000 7f713d000640 1 -- 172.21.15.28:0/1102819541 wait complete. 2024-09-10T10:44:16.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.205+0000 7f713d000640 1 Processor -- start 2024-09-10T10:44:16.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.205+0000 7f713d000640 1 -- start start 2024-09-10T10:44:16.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f713815dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f713815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7138155240 con 0x7f71380af280 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71380af710 con 0x7f71380b69d0 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f713d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71381553c0 con 0x7f71380b0180 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7135e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7135e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39648/0 (socket says 172.21.15.28:39648) 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7135e00640 1 -- 172.21.15.28:0/191282842 learned_addr learned my addr 172.21.15.28:0/191282842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7135400640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f713815dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7136800640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f713815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.206+0000 7f7135e00640 1 -- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 msgr2=0x7f713815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f7135e00640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f713815dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f7135e00640 1 -- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 msgr2=0x7f713815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f7135e00640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f713815e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f7135e00640 1 -- 172.21.15.28:0/191282842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71381e7f70 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f7135e00640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f71280098c0 tx=0x7f71280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f712e000640 1 -- 172.21.15.28:0/191282842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128040030 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f712e000640 1 -- 172.21.15.28:0/191282842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7128038040 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71381e8250 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f712e000640 1 -- 172.21.15.28:0/191282842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128037070 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.207+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7138153b80 con 0x7f71380af280 2024-09-10T10:44:16.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.209+0000 7f712e000640 1 -- 172.21.15.28:0/191282842 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7128043070 con 0x7f71380af280 2024-09-10T10:44:16.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.210+0000 7f712e000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 0x7f711806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.210+0000 7f712e000640 1 -- 172.21.15.28:0/191282842 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f71280ab070 con 0x7f71380af280 2024-09-10T10:44:16.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.210+0000 7f7135400640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 0x7f711806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.211+0000 7f713feb7740 1 Processor -- start 2024-09-10T10:44:16.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.211+0000 7f713feb7740 1 -- start start 2024-09-10T10:44:16.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.211+0000 7f7135400640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 0x7f711806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71381f1c20 tx=0x7f7124009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b976cf0 0x56181b9bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b971570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56181b8d53b0 con 0x56181b9744b0 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56181b8da410 con 0x56181b978480 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56181b882620 con 0x56181b976cf0 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39664/0 (socket says 172.21.15.28:39664) 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 -- 172.21.15.28:0/3985376980 learned_addr learned my addr 172.21.15.28:0/3985376980 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135e00640 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b976cf0 0x56181b9bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7136800640 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b971570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 -- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b976cf0 msgr2=0x56181b9bc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b976cf0 0x56181b9bc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 -- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 msgr2=0x56181b971570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.212+0000 7f7135400640 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b971570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.213+0000 7f7135400640 1 -- 172.21.15.28:0/3985376980 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181b5ed2d0 con 0x56181b9744b0 2024-09-10T10:44:16.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.213+0000 7f7135400640 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f712400fb40 tx=0x7f7124000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0cc915da9a1b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.213+0000 7f7122a00640 1 -- 172.21.15.28:0/3985376980 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124025070 con 0x56181b9744b0 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.213+0000 7f7122a00640 1 -- 172.21.15.28:0/3985376980 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124015cb0 con 0x56181b9744b0 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.213+0000 7f7122a00640 1 -- 172.21.15.28:0/3985376980 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124015e60 con 0x56181b9744b0 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 -- 172.21.15.28:0/3985376980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 msgr2=0x56181b9770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f712400fb40 tx=0x7f7124000ed0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 -- 172.21.15.28:0/3985376980 shutdown_connections 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b971570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9744b0 0x56181b9770d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 --2- 172.21.15.28:0/3985376980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b976cf0 0x56181b9bc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 -- 172.21.15.28:0/3985376980 >> 172.21.15.28:0/3985376980 conn(0x56181b973740 msgr2=0x56181b9bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 -- 172.21.15.28:0/3985376980 shutdown_connections 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.214+0000 7f713feb7740 1 -- 172.21.15.28:0/3985376980 wait complete. 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.215+0000 7f713feb7740 1 -- start start 2024-09-10T10:44:16.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.215+0000 7f713feb7740 1 Processor -- start 2024-09-10T10:44:16.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b99a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b9b92a0 0x56181b99a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56181b895780 con 0x56181b976cf0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56181b8d63d0 con 0x56181b978480 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56181b972be0 con 0x56181b9b92a0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f713feb7740 10 client.0 _ll_register_callbacks cb 0x56181b8d35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39676/0 (socket says 172.21.15.28:39676) 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 -- 172.21.15.28:0/3908880605 learned_addr learned my addr 172.21.15.28:0/3908880605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7136800640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b9b92a0 0x56181b99a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135400640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b99a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 -- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b9b92a0 msgr2=0x56181b99a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b9b92a0 0x56181b99a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 -- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 msgr2=0x56181b99a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b99a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135e00640 1 -- 172.21.15.28:0/3908880605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181b6b76b0 con 0x56181b976cf0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.216+0000 7f7135400640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b99a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7135e00640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f712802a0a0 tx=0x7f7128004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dcaecd01412919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128081c40 con 0x56181b976cf0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f71280817b0 con 0x56181b976cf0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7124015cb0 con 0x56181b976cf0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f71280819b0 con 0x56181b976cf0 2024-09-10T10:44:16.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 10 client.4626 fetch_fsmap learned FSMap version 9 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56181b99f1c0 con 0x56181b976cf0 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7124015cb0 con 0x56181b976cf0 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f712804d060 con 0x56181b976cf0 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 10 client.4626 fetch_fsmap finished waiting for FSMap version 9 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 10 client.4626 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 20 client.4626 populate_metadata read hostname 'smithi028' 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181b9cbd40 0x56181b9cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 4 client.4626 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.217+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 --> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56181b8e7b40 con 0x56181b9cbd40 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.218+0000 7f7135400640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181b9cbd40 0x56181b9cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.218+0000 7f7135400640 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181b9cbd40 0x56181b9cf950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39ded739c2c7980 server_cookie=8d5fa7d7d08c68ff in_seq=0 out_seq=0 2024-09-10T10:44:16.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.218+0000 7f7117400640 10 client.4626 ms_handle_connect on v2:172.21.15.28:6834/112106318 2024-09-10T10:44:16.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.219+0000 7f7117400640 1 -- 172.21.15.28:0/3908880605 <== mds.0 v2:172.21.15.28:6834/112106318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56181b8e7b40 con 0x56181b9cbd40 2024-09-10T10:44:16.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.219+0000 7f7117400640 10 client.4626 handle_command_reply: tid=0 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.219+0000 7f713feb7740 1 client.4626 shutdown 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.219+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 msgr2=0x56181b998d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.219+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f712802a0a0 tx=0x7f7128004580 comp rx=0 tx=0).stop 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 shutdown_connections 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181b9cbd40 0x56181b9cf950 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b9b92a0 0x56181b99a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b978480 0x56181b99a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 --2- 172.21.15.28:0/3908880605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b976cf0 0x56181b998d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f7117400640 0 client.4626 ms_handle_reset on v2:172.21.15.28:6834/112106318 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 >> 172.21.15.28:0/3908880605 conn(0x56181b9744b0 msgr2=0x56181b9baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 shutdown_connections 2024-09-10T10:44:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 1 -- 172.21.15.28:0/3908880605 wait complete. 2024-09-10T10:44:16.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.220+0000 7f713feb7740 20 client.4626 trim_cache size 0 max 16384 2024-09-10T10:44:16.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 Processor -- start 2024-09-10T10:44:16.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 -- start start 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181b9b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b998850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7138200e40 con 0x56181b9b92a0 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71382001a0 con 0x56181b976cf0 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.239+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71382007f0 con 0x56181b978480 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39682/0 (socket says 172.21.15.28:39682) 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 -- 172.21.15.28:0/3594252312 learned_addr learned my addr 172.21.15.28:0/3594252312 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7135e00640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181b9b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7135400640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b998850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 -- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 msgr2=0x56181b998850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b998850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 -- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 msgr2=0x56181b9b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181b9b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7136800640 1 -- 172.21.15.28:0/3594252312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181b9734e0 con 0x56181b9b92a0 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.240+0000 7f7135400640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b998850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:16.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f7136800640 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7130009fd0 tx=0x7f7130009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04da80ff3762ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f7122000640 1 -- 172.21.15.28:0/3594252312 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130017ea0 con 0x56181b9b92a0 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f7122000640 1 -- 172.21.15.28:0/3594252312 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713001c070 con 0x56181b9b92a0 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f7122000640 1 -- 172.21.15.28:0/3594252312 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130015860 con 0x56181b9b92a0 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 -- 172.21.15.28:0/3594252312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 msgr2=0x56181b998d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7130009fd0 tx=0x7f7130009750 comp rx=0 tx=0).stop 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 -- 172.21.15.28:0/3594252312 shutdown_connections 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b998d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b998850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 --2- 172.21.15.28:0/3594252312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181b9b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 -- 172.21.15.28:0/3594252312 >> 172.21.15.28:0/3594252312 conn(0x56181b9744b0 msgr2=0x56181b971cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.241+0000 7f713feb7740 1 -- 172.21.15.28:0/3594252312 shutdown_connections 2024-09-10T10:44:16.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.242+0000 7f713feb7740 1 -- 172.21.15.28:0/3594252312 wait complete. 2024-09-10T10:44:16.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.242+0000 7f713feb7740 1 -- start start 2024-09-10T10:44:16.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.242+0000 7f713feb7740 1 Processor -- start 2024-09-10T10:44:16.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181ba39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b99d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56181b895780 con 0x56181b9b92a0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56181b8d63d0 con 0x56181b976cf0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56181b99fdf0 con 0x56181b978480 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.243+0000 7f713feb7740 10 client.0 _ll_register_callbacks cb 0x7f7128003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39698/0 (socket says 172.21.15.28:39698) 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 -- 172.21.15.28:0/2559240501 learned_addr learned my addr 172.21.15.28:0/2559240501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7135e00640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181ba39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7135400640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b99d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 -- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 msgr2=0x56181b99d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b99d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 -- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 msgr2=0x56181ba39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181ba39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 -- 172.21.15.28:0/2559240501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56181b99e290 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7135e00640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181ba39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7136800640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7130000f80 tx=0x7f713001a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2769f1e29de3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.244+0000 7f7117400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130017ea0 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f713002ede0 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f713001c070 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f7130017420 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 10 client.4632 fetch_fsmap learned FSMap version 9 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x56181ba3e400 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713001c070 con 0x56181b9b92a0 2024-09-10T10:44:16.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7f7130035050 con 0x56181b9b92a0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 10 client.4632 fetch_fsmap finished waiting for FSMap version 9 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 10 client.4632 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 20 client.4632 populate_metadata read hostname 'smithi028' 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181ba3dcb0 0x56181b970530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 4 client.4632 mds_command: new command op to 4284 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.245+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 --> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] -- command(tid 0: {"prefix": "status"}) -- 0x56181b8e1e90 con 0x56181ba3dcb0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f7135e00640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181ba3dcb0 0x56181b970530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f7135e00640 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181ba3dcb0 0x56181b970530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefeef048a03257e server_cookie=9397de9a60d1cb19 in_seq=0 out_seq=0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f7117400640 10 client.4632 ms_handle_connect on v2:172.21.15.28:6834/112106318 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f7117400640 1 -- 172.21.15.28:0/2559240501 <== mds.0 v2:172.21.15.28:6834/112106318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x56181b8e1e90 con 0x56181ba3dcb0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f7117400640 10 client.4632 handle_command_reply: tid=0 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f713feb7740 1 client.4632 shutdown 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 msgr2=0x56181b99db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.246+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7130000f80 tx=0x7f713001a070 comp rx=0 tx=0).stop 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 shutdown_connections 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56181b9b92a0 0x56181b99db10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.28:6834/112106318,v1:172.21.15.28:6835/112106318] conn(0x56181ba3dcb0 0x56181b970530 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56181b978480 0x56181b99d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 --2- 172.21.15.28:0/2559240501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56181b976cf0 0x56181ba39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f7117400640 0 client.4632 ms_handle_reset on v2:172.21.15.28:6834/112106318 2024-09-10T10:44:16.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 >> 172.21.15.28:0/2559240501 conn(0x56181b9744b0 msgr2=0x56181b9bbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 shutdown_connections 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 1 -- 172.21.15.28:0/2559240501 wait complete. 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713feb7740 20 client.4632 trim_cache size 0 max 16384 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:44:16.252 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4284, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 28.715821547000001, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 9, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 27, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 27, 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 34.525351854999997 2024-09-10T10:44:16.253 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 msgr2=0x7f711806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 0x7f711806e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f71381f1c20 tx=0x7f7124009040 comp rx=0 tx=0).stop 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 msgr2=0x7f7138153640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f71280098c0 tx=0x7f71280046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 shutdown_connections 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711806be30 0x7f711806e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71380b69d0 0x7f713815e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71380b0180 0x7f713815dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.247+0000 7f713d000640 1 --2- 172.21.15.28:0/191282842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71380af280 0x7f7138153640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:16.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.248+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 >> 172.21.15.28:0/191282842 conn(0x7f713806bc10 msgr2=0x7f71380b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:16.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.248+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 shutdown_connections 2024-09-10T10:44:16.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:16.248+0000 7f713d000640 1 -- 172.21.15.28:0/191282842 wait complete. 2024-09-10T10:44:16.264 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T10:44:16.274 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T10:44:16.274 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T10:44:16.274 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T10:44:16.274 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T10:44:16.296 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T10:44:16.296 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T10:44:16.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.297 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:44:16.331 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T10:44:16.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.333 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T10:44:16.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-10T10:44:16.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.726 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:44:16.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.762 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-10T10:44:16.863 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-09-10T10:44:16.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:44:16.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.866 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:44:16.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:16.906 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-09-10T10:44:17.009 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-09-10T10:44:17.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:44:17.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:17.012 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:44:17.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:17.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-09-10T10:44:17.146 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-09-10T10:44:17.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:44:17.149 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-10T10:44:17.149 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:44:17.150 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:44:17.150 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:44:17.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.307+0000 7f3605800640 1 Processor -- start 2024-09-10T10:44:17.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.307+0000 7f3605800640 1 -- start start 2024-09-10T10:44:17.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.308+0000 7f3605800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36000d0dd0 0x7f36000d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.308+0000 7f3605800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36000cf000 con 0x7f36000d1f90 2024-09-10T10:44:17.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.308+0000 7f3605800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36000d16f0 con 0x7f360015e660 2024-09-10T10:44:17.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.308+0000 7f3605800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f360006d3e0 con 0x7f36000d0dd0 2024-09-10T10:44:17.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.308+0000 7f35ffe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36000d0dd0 0x7f36000d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f35ffe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36000d0dd0 0x7f36000d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39702/0 (socket says 172.21.15.28:39702) 2024-09-10T10:44:17.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f35ffe00640 1 -- 172.21.15.28:0/2491886180 learned_addr learned my addr 172.21.15.28:0/2491886180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:17.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f3604e00640 1 --1- 172.21.15.28:0/2491886180 >> v1:172.21.15.28:6789/0 conn(0x7f36000d1f90 0x7f36000cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41666/0 (socket says 172.21.15.28:41666) 2024-09-10T10:44:17.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f35ffe00640 1 -- 172.21.15.28:0/2491886180 >> v1:172.21.15.138:6789/0 conn(0x7f360015e660 legacy=0x7f360006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f35ffe00640 1 -- 172.21.15.28:0/2491886180 >> v1:172.21.15.28:6789/0 conn(0x7f36000d1f90 legacy=0x7f36000cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.309+0000 7f35ffe00640 1 -- 172.21.15.28:0/2491886180 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36000d29a0 con 0x7f36000d0dd0 2024-09-10T10:44:17.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.310+0000 7f35ffe00640 1 --2- 172.21.15.28:0/2491886180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36000d11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009e50 tx=0x7f35f002a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe44d2fa381b464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.310+0000 7f35ff400640 1 -- 172.21.15.28:0/2491886180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0036070 con 0x7f36000d0dd0 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.310+0000 7f35ff400640 1 -- 172.21.15.28:0/2491886180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0031600 con 0x7f36000d0dd0 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.310+0000 7f35ff400640 1 -- 172.21.15.28:0/2491886180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00319f0 con 0x7f36000d0dd0 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 -- 172.21.15.28:0/2491886180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 msgr2=0x7f36000d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 --2- 172.21.15.28:0/2491886180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36000d11b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009e50 tx=0x7f35f002a7c0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 -- 172.21.15.28:0/2491886180 shutdown_connections 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 --2- 172.21.15.28:0/2491886180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36000d11b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 -- 172.21.15.28:0/2491886180 >> 172.21.15.28:0/2491886180 conn(0x7f360008bfc0 msgr2=0x7f36000d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 -- 172.21.15.28:0/2491886180 shutdown_connections 2024-09-10T10:44:17.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.311+0000 7f3605800640 1 -- 172.21.15.28:0/2491886180 wait complete. 2024-09-10T10:44:17.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.313+0000 7f3605800640 1 Processor -- start 2024-09-10T10:44:17.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.313+0000 7f3605800640 1 -- start start 2024-09-10T10:44:17.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36000d1f90 0x7f36001e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360015e660 0x7f36001e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f36000cf330 con 0x7f36000d0dd0 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36000d2f40 con 0x7f360015e660 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3605800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f36000d3220 con 0x7f36000d1f90 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39706/0 (socket says 172.21.15.28:39706) 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 -- 172.21.15.28:0/1241612199 learned_addr learned my addr 172.21.15.28:0/1241612199 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f3604e00640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360015e660 0x7f36001e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35f7400640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36000d1f90 0x7f36001e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 -- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36000d1f90 msgr2=0x7f36001e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36000d1f90 0x7f36001e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 -- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360015e660 msgr2=0x7f36001e6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360015e660 0x7f36001e6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.314+0000 7f35ffe00640 1 -- 172.21.15.28:0/1241612199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36000c85e0 con 0x7f36000d0dd0 2024-09-10T10:44:17.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f35ffe00640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009f70 tx=0x7f35f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:17.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0036050 con 0x7f36000d0dd0 2024-09-10T10:44:17.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0037040 con 0x7f36000d0dd0 2024-09-10T10:44:17.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0009510 con 0x7f36000d0dd0 2024-09-10T10:44:17.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36000ca4d0 con 0x7f36000d0dd0 2024-09-10T10:44:17.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.315+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36000cc610 con 0x7f36000d0dd0 2024-09-10T10:44:17.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.316+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36000d2420 con 0x7f36000d0dd0 2024-09-10T10:44:17.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.317+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35f0044020 con 0x7f36000d0dd0 2024-09-10T10:44:17.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.318+0000 7f35fcc00640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 0x7f35dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:17.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.318+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f35f00ab590 con 0x7f36000d0dd0 2024-09-10T10:44:17.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.318+0000 7f35f7400640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 0x7f35dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:17.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.319+0000 7f35f7400640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 0x7f35dc06e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36001e6c70 tx=0x7f35e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:17.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.322+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35f004b020 con 0x7f36000d0dd0 2024-09-10T10:44:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.541+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36000b3380 con 0x7f36000d0dd0 2024-09-10T10:44:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.542+0000 7f35fcc00640 1 -- 172.21.15.28:0/1241612199 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f35f00795a0 con 0x7f36000d0dd0 2024-09-10T10:44:17.543 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 msgr2=0x7f35dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 0x7f35dc06e2f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36001e6c70 tx=0x7f35e8009040 comp rx=0 tx=0).stop 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 msgr2=0x7f36001548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009f70 tx=0x7f35f0004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 shutdown_connections 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35dc06be30 0x7f35dc06e2f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360015e660 0x7f36001e6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36000d1f90 0x7f36001e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 --2- 172.21.15.28:0/1241612199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36000d0dd0 0x7f36001548f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:17.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 >> 172.21.15.28:0/1241612199 conn(0x7f360008bfc0 msgr2=0x7f360006e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:17.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 shutdown_connections 2024-09-10T10:44:17.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17.544+0000 7f3605800640 1 -- 172.21.15.28:0/1241612199 wait complete. 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:44:17.561 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:44:17.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:17.562 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:44:17.597 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:17.597 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:17.597 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:17.597 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:17.597 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42647sec preferred_lft 42647sec 2024-09-10T10:44:17.598 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:44:17.599 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:17.599 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:17.599 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:17.599 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:44:17.599 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:44:17.599 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:17.599 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:17.599 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:44:17.599 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2024-09-10T10:44:17.599 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2024-09-10T10:44:17.600 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:44:17.600 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:17.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:17.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:17.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:17.822 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:44:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:44:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:17.879 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:44:17.915 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:44:17.915 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:44:17.915 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:44:17.915 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:44:17.916 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:44:17.916 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:18.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:18.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:18.113 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:44:18.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:18.149 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-09-10T10:44:18.224 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:18.224 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:18.224 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:44:18.224 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T10:44:18.224 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:18.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:18.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:18.400 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T10:44:18.400 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:18.400 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:18.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T10:44:18.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:44:18.401 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T10:44:18.401 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T10:44:18.401 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T10:44:18.401 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:18.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:18.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:18.623 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:18.623 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:18.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:44:18.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:44:18.623 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:18.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:18.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:18.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:18.790 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:44:18.825 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:44:18.825 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:18.825 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:44:18.899 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:44:18.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:18.899 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:44:19.027 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:44:19.027 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:44:19.027 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:19.608 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:44:19.609 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:44:19.609 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:19.609 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:19.609 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:19.634 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:44:19.634 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:44:19.634 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:19.634 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:19.635 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:19.635 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:44:19.635 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:44:19.635 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:44:19.636 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:44:19.636 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:44:19.636 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:44:19.637 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:44:19.637 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:44:19.640 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:44:19.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:19.640 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:44:19.697 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:19.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:19.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:19.790 INFO:teuthology.orchestra.run.smithi028.stdout:client4505 2024-09-10T10:44:19.790 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:19.791 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:19.828 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:19.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:19.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:19.958 INFO:teuthology.orchestra.run.smithi028.stdout:client4505 2024-09-10T10:44:19.958 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:20.047 INFO:teuthology.orchestra.run.smithi028.stdout:client4505 2024-09-10T10:44:20.047 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f658987a-dd14-4573-bcd3-ee2a383c9fe0.client4505/status') 2024-09-10T10:44:20.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:20.146 INFO:teuthology.orchestra.run.smithi028.stdout:instance: client.4505 (3)172.21.15.28:0/3274337198 2024-09-10T10:44:20.146 INFO:teuthology.orchestra.run.smithi028.stdout:blocklisted: false 2024-09-10T10:44:20.146 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records in 2024-09-10T10:44:20.147 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records out 2024-09-10T10:44:20.147 INFO:teuthology.orchestra.run.smithi028.stderr:70 bytes copied, 5.4759e-05 s, 1.3 MB/s 2024-09-10T10:44:20.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:20.157 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-10T10:44:20.157 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:44:20.158 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T10:44:20.158 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:44:20.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.313+0000 7fa766400640 1 Processor -- start 2024-09-10T10:44:20.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.313+0000 7fa766400640 1 -- start start 2024-09-10T10:44:20.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa766400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7600d0dd0 0x7fa7600d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa766400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7600cf000 con 0x7fa7600d1f90 2024-09-10T10:44:20.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa766400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7600d16f0 con 0x7fa76015e660 2024-09-10T10:44:20.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa766400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa76006d3e0 con 0x7fa7600d0dd0 2024-09-10T10:44:20.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa765000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7600d0dd0 0x7fa7600d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa765a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa7600d1f90 0x7fa7600cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41682/0 (socket says 172.21.15.28:41682) 2024-09-10T10:44:20.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.314+0000 7fa765a00640 1 -- 172.21.15.28:0/2366798276 learned_addr learned my addr 172.21.15.28:0/2366798276 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:20.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.315+0000 7fa765000640 1 -- 172.21.15.28:0/2366798276 >> v1:172.21.15.138:6789/0 conn(0x7fa76015e660 legacy=0x7fa76006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.315+0000 7fa765000640 1 -- 172.21.15.28:0/2366798276 >> v1:172.21.15.28:6789/0 conn(0x7fa7600d1f90 legacy=0x7fa7600cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.315+0000 7fa75fe00640 1 -- 172.21.15.28:0/2366798276 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3433050979 0 0) 0x7fa7600cf000 con 0x7fa7600d1f90 2024-09-10T10:44:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.315+0000 7fa765000640 1 -- 172.21.15.28:0/2366798276 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7600d29a0 con 0x7fa7600d0dd0 2024-09-10T10:44:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.316+0000 7fa765000640 1 --2- 172.21.15.28:0/2366798276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600d11b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa75002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17302112551991 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.316+0000 7fa75fe00640 1 -- 172.21.15.28:0/2366798276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037070 con 0x7fa7600d0dd0 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.316+0000 7fa75fe00640 1 -- 172.21.15.28:0/2366798276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750032440 con 0x7fa7600d0dd0 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.316+0000 7fa75fe00640 1 -- 172.21.15.28:0/2366798276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750032810 con 0x7fa7600d0dd0 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 -- 172.21.15.28:0/2366798276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 msgr2=0x7fa7600d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 --2- 172.21.15.28:0/2366798276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600d11b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa75002a230 comp rx=0 tx=0).stop 2024-09-10T10:44:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 -- 172.21.15.28:0/2366798276 shutdown_connections 2024-09-10T10:44:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 --2- 172.21.15.28:0/2366798276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600d11b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 -- 172.21.15.28:0/2366798276 >> 172.21.15.28:0/2366798276 conn(0x7fa76008bfc0 msgr2=0x7fa7600d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 -- 172.21.15.28:0/2366798276 shutdown_connections 2024-09-10T10:44:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.317+0000 7fa766400640 1 -- 172.21.15.28:0/2366798276 wait complete. 2024-09-10T10:44:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.319+0000 7fa766400640 1 Processor -- start 2024-09-10T10:44:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.319+0000 7fa766400640 1 -- start start 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7600d1f90 0x7fa76015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76015e660 0x7fa76015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600cf330 con 0x7fa7600d0dd0 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600d2f40 con 0x7fa76015e660 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa766400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7600d3220 con 0x7fa7600d1f90 2024-09-10T10:44:20.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39728/0 (socket says 172.21.15.28:39728) 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 -- 172.21.15.28:0/851302324 learned_addr learned my addr 172.21.15.28:0/851302324 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 -- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7600d1f90 msgr2=0x7fa76015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7600d1f90 0x7fa76015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 -- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76015e660 msgr2=0x7fa76015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76015e660 0x7fa76015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.320+0000 7fa765000640 1 -- 172.21.15.28:0/851302324 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7601f5630 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa765000640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750037050 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa750038040 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750004470 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7601e3840 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.321+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7601e3e00 con 0x7fa7600d0dd0 2024-09-10T10:44:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.322+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7600d2420 con 0x7fa7600d0dd0 2024-09-10T10:44:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.323+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa750030900 con 0x7fa7600d0dd0 2024-09-10T10:44:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.324+0000 7fa75d600640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 0x7fa73c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.324+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa7500abea0 con 0x7fa7600d0dd0 2024-09-10T10:44:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.324+0000 7fa757e00640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 0x7fa73c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.325+0000 7fa757e00640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 0x7fa73c06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa748000f60 tx=0x7fa7480079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.328+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa75007a590 con 0x7fa7600d0dd0 2024-09-10T10:44:20.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.553+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa7601e65b0 con 0x7fa7600d0dd0 2024-09-10T10:44:20.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.553+0000 7fa75d600640 1 -- 172.21.15.28:0/851302324 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fa750079e90 con 0x7fa7600d0dd0 2024-09-10T10:44:20.555 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:44:20.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 msgr2=0x7fa73c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 0x7fa73c06e320 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa748000f60 tx=0x7fa7480079b0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 msgr2=0x7fa7600c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:20.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa7500097a0 tx=0x7fa7500046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 shutdown_connections 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73c06be60 0x7fa73c06e320 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa76015e660 0x7fa76015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7600d1f90 0x7fa76015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 --2- 172.21.15.28:0/851302324 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7600d0dd0 0x7fa7600c9250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 >> 172.21.15.28:0/851302324 conn(0x7fa76008bfc0 msgr2=0x7fa7600cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 shutdown_connections 2024-09-10T10:44:20.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20.556+0000 7fa766400640 1 -- 172.21.15.28:0/851302324 wait complete. 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T10:44:20.574 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:44:20.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:44:20.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:44:20.575 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T10:44:20.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:20.575 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:44:20.610 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:20.610 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:44:20.611 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42644sec preferred_lft 42644sec 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:20.612 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:20.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:20.613 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:44:20.705 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:44:20.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:20.709 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:44:20.745 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:44:20.745 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:44:20.745 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:44:20.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:44:20.745 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:44:20.746 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:20.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:20.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:20.944 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:44:20.979 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:44:20.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:20.980 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-09-10T10:44:21.054 INFO:teuthology.orchestra.run.smithi028.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T10:44:21.055 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:21.055 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:21.055 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:44:21.055 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T10:44:21.055 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:21.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:21.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:21.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:21.231 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ca:dd:2b:d5:86:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T10:44:21.287 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T10:44:21.288 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.288 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::c8dd:2bff:fed5:8671/64 scope link 2024-09-10T10:44:21.288 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:21.290 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T10:44:21.345 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:21.345 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ca:dd:2b:d5:86:71 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::c8dd:2bff:fed5:8671/64 scope link 2024-09-10T10:44:21.346 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:21.348 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T10:44:21.348 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T10:44:21.349 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:21.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:21.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:21.567 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:21.567 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:44:21.567 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.1 up 2024-09-10T10:44:21.567 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T10:44:21.567 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:44:21.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:21.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:21.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:21.737 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T10:44:21.772 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T10:44:21.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:21.772 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T10:44:21.847 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:44:21.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:21.847 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T10:44:21.975 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T10:44:21.976 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T10:44:21.976 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:21.991 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T10:44:21.991 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T10:44:21.991 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:21.991 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:21.991 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:22.013 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-10T10:44:22.013 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-10T10:44:22.013 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:44:22.013 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:22.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:22.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:44:22.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:44:22.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:44:22.015 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-10T10:44:22.015 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:44:22.015 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:44:22.016 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:44:22.016 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:44:22.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:44:22.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:22.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T10:44:22.074 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:22.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:22.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:22.163 INFO:teuthology.orchestra.run.smithi028.stdout:client4508 2024-09-10T10:44:22.164 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:22.164 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:22.200 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:22.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:22.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:22.329 INFO:teuthology.orchestra.run.smithi028.stdout:client4508 2024-09-10T10:44:22.329 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:22.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:22.419 INFO:teuthology.orchestra.run.smithi028.stdout:client4508 2024-09-10T10:44:22.419 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f658987a-dd14-4573-bcd3-ee2a383c9fe0.client4508/status') 2024-09-10T10:44:22.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:22.519 INFO:teuthology.orchestra.run.smithi028.stdout:instance: client.4508 (3)172.21.15.28:0/1029805312 2024-09-10T10:44:22.519 INFO:teuthology.orchestra.run.smithi028.stdout:blocklisted: false 2024-09-10T10:44:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records in 2024-09-10T10:44:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:0+1 records out 2024-09-10T10:44:22.519 INFO:teuthology.orchestra.run.smithi028.stderr:70 bytes copied, 5.7898e-05 s, 1.2 MB/s 2024-09-10T10:44:22.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:22.530 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-10T10:44:22.530 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:44:22.530 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:44:22.530 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:44:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.686+0000 7f81ad800640 1 Processor -- start 2024-09-10T10:44:22.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.686+0000 7f81ad800640 1 -- start start 2024-09-10T10:44:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.687+0000 7f81ad800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81a8072d30 0x7f81a806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.687+0000 7f81ad800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 0x7f81a815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.687+0000 7f81ad800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a80cdbe0 con 0x7f81a815b300 2024-09-10T10:44:22.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.687+0000 7f81ad800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f81a80728a0 con 0x7f81a8072d30 2024-09-10T10:44:22.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.687+0000 7f81ad800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81a806fb30 con 0x7f81a806f700 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a7400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 0x7f81a815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a7400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 0x7f81a815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:60510/0 (socket says 172.21.15.28:60510) 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f81a8072d30 0x7f81a806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f81a815b300 0x7f81a80cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:49834/0 (socket says 172.21.15.28:49834) 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a7400640 1 -- 172.21.15.28:0/492787489 learned_addr learned my addr 172.21.15.28:0/492787489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a5600640 1 -- 172.21.15.28:0/492787489 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760493033 0 0) 0x7f81a80cdbe0 con 0x7f81a815b300 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6000640 1 -- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 msgr2=0x7f81a815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6000640 1 --2- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 0x7f81a815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6000640 1 -- 172.21.15.28:0/492787489 >> v1:172.21.15.138:6789/0 conn(0x7f81a815b300 legacy=0x7f81a80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.688+0000 7f81a6000640 1 -- 172.21.15.28:0/492787489 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a80cd900 con 0x7f81a8072d30 2024-09-10T10:44:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.689+0000 7f81a6000640 1 --2- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a8072d30 0x7f81a806f1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81940097a0 tx=0x7f819402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61aec56eab8e49e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81a5600640 1 -- 172.21.15.28:0/492787489 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194037070 con 0x7f81a8072d30 2024-09-10T10:44:22.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81a5600640 1 -- 172.21.15.28:0/492787489 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819402ae50 con 0x7f81a8072d30 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81a5600640 1 -- 172.21.15.28:0/492787489 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194032660 con 0x7f81a8072d30 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 -- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a8072d30 msgr2=0x7f81a806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 --2- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a8072d30 0x7f81a806f1c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81940097a0 tx=0x7f819402a100 comp rx=0 tx=0).stop 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 -- 172.21.15.28:0/492787489 shutdown_connections 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 --2- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a806f700 0x7f81a815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 --2- 172.21.15.28:0/492787489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a8072d30 0x7f81a806f1c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 -- 172.21.15.28:0/492787489 >> 172.21.15.28:0/492787489 conn(0x7f81a8067d70 msgr2=0x7f81a806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 -- 172.21.15.28:0/492787489 shutdown_connections 2024-09-10T10:44:22.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.690+0000 7f81ad800640 1 -- 172.21.15.28:0/492787489 wait complete. 2024-09-10T10:44:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.692+0000 7f81ad800640 1 Processor -- start 2024-09-10T10:44:22.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.692+0000 7f81ad800640 1 -- start start 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a806f700 0x7f81a8148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a8072d30 0x7f81a8159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f81a80cd5c0 con 0x7f81a815b300 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8072530 con 0x7f81a806f700 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81ad800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f81a80cffe0 con 0x7f81a8072d30 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81a7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81a7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39744/0 (socket says 172.21.15.28:39744) 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.693+0000 7f81a7400640 1 -- 172.21.15.28:0/1286511276 learned_addr learned my addr 172.21.15.28:0/1286511276 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a6a00640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a806f700 0x7f81a8148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a6000640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a8072d30 0x7f81a8159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 -- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a8072d30 msgr2=0x7f81a8159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a8072d30 0x7f81a8159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 -- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a806f700 msgr2=0x7f81a8148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a806f700 0x7f81a8148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 -- 172.21.15.28:0/1286511276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a81f4cc0 con 0x7f81a815b300 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.694+0000 7f81a7400640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81a000efc0 tx=0x7f81a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.695+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a000cd20 con 0x7f81a815b300 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.695+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a0003990 con 0x7f81a815b300 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.695+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0003d80 con 0x7f81a815b300 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.695+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a81f4fa0 con 0x7f81a815b300 2024-09-10T10:44:22.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.695+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a8149460 con 0x7f81a815b300 2024-09-10T10:44:22.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.696+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8158680 con 0x7f81a815b300 2024-09-10T10:44:22.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.697+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81a0012050 con 0x7f81a815b300 2024-09-10T10:44:22.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.698+0000 7f819ea00640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 0x7f818806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:22.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.698+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f81a008b630 con 0x7f81a815b300 2024-09-10T10:44:22.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.698+0000 7f81a6a00640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 0x7f818806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:22.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.699+0000 7f81a6a00640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 0x7f818806e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8198000f60 tx=0x7f8198007e60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:22.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.702+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f81a0010040 con 0x7f81a815b300 2024-09-10T10:44:22.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.923+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f81a815ccb0 con 0x7f81a815b300 2024-09-10T10:44:22.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.924+0000 7f819ea00640 1 -- 172.21.15.28:0/1286511276 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f81a00595b0 con 0x7f81a815b300 2024-09-10T10:44:22.925 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 msgr2=0x7f818806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 0x7f818806e2f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8198000f60 tx=0x7f8198007e60 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 msgr2=0x7f81a81f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f81a000efc0 tx=0x7f81a000c490 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 shutdown_connections 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f818806be30 0x7f818806e2f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81a815b300 0x7f81a81f4190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f81a8072d30 0x7f81a8159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 --2- 172.21.15.28:0/1286511276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81a806f700 0x7f81a8148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:22.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 >> 172.21.15.28:0/1286511276 conn(0x7f81a8067d70 msgr2=0x7f81a80b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 shutdown_connections 2024-09-10T10:44:22.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:22.926+0000 7f81ad800640 1 -- 172.21.15.28:0/1286511276 wait complete. 2024-09-10T10:44:22.943 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:44:22.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:44:22.944 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:44:22.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:22.944 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-09-10T10:44:22.984 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42651sec preferred_lft 42651sec 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link noprefixroute 2024-09-10T10:44:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:22.986 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T10:44:22.986 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:22.986 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:22.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-09-10T10:44:22.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-09-10T10:44:22.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-09-10T10:44:22.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T10:44:22.987 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:23.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:23.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:23.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:23.401 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:44:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-09-10T10:44:23.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:23.459 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-09-10T10:44:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-09-10T10:44:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:44:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:44:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:44:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:44:23.527 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:23.527 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:23.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:44:23.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:44:23.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:44:23.528 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:23.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:24.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:24.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:24.108 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:44:24.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:24.143 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-09-10T10:44:24.218 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:24.218 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:24.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:44:24.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T10:44:24.218 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:24.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:24.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:24.395 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T10:44:24.395 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:24.395 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:24.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T10:44:24.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T10:44:24.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T10:44:24.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T10:44:24.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T10:44:24.396 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:24.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:24.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:24.633 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:24.633 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:24.634 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-09-10T10:44:24.634 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T10:44:24.634 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:24.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:24.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:24.801 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T10:44:24.836 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T10:44:24.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:24.837 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:44:24.911 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:44:24.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:24.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T10:44:25.049 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T10:44:25.049 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T10:44:25.049 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:25.669 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T10:44:25.669 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T10:44:25.670 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:25.670 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:25.670 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:44:25.696 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:44:25.699 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-10T10:44:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:44:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:44:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:44:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:44:25.702 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:44:25.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:25.702 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T10:44:25.759 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:25.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:25.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:25.851 INFO:teuthology.orchestra.run.smithi138.stdout:client4615 2024-09-10T10:44:25.851 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:25.851 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:25.887 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:26.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:26.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:26.015 INFO:teuthology.orchestra.run.smithi138.stdout:client4615 2024-09-10T10:44:26.015 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:26.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:26.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:26.102 INFO:teuthology.orchestra.run.smithi138.stdout:client4615 2024-09-10T10:44:26.103 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f658987a-dd14-4573-bcd3-ee2a383c9fe0.client4615/status') 2024-09-10T10:44:26.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:26.201 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4615 (3)192.168.0.1:0/1396324416 2024-09-10T10:44:26.201 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-09-10T10:44:26.201 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-09-10T10:44:26.201 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-09-10T10:44:26.201 INFO:teuthology.orchestra.run.smithi138.stderr:69 bytes copied, 8.1503e-05 s, 847 kB/s 2024-09-10T10:44:26.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:26.212 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-10T10:44:26.212 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T10:44:26.212 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T10:44:26.212 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:44:26.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.379+0000 7f92a5a00640 1 Processor -- start 2024-09-10T10:44:26.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.379+0000 7f92a5a00640 1 -- start start 2024-09-10T10:44:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f92a5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92a006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f92a5a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a00cf000 con 0x7f92a00d0dd0 2024-09-10T10:44:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f92a5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a00d29a0 con 0x7f92a00d1f90 2024-09-10T10:44:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f92a5a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00b37d0 con 0x7f92a015e660 2024-09-10T10:44:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f929fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f92a00d0dd0 0x7f92a00d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52176/0 (socket says 172.21.15.28:52176) 2024-09-10T10:44:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.380+0000 7f929fe00640 1 -- 172.21.15.28:0/1750057166 learned_addr learned my addr 172.21.15.28:0/1750057166 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 147055073 0 0) 0x7f92a00cf000 con 0x7f92a00d0dd0 2024-09-10T10:44:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9284003520 con 0x7f92a00d0dd0 2024-09-10T10:44:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929f400640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92a006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 386628282 0 0) 0x7f9284003520 con 0x7f92a00d0dd0 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 msgr2=0x7f92a006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92a006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 >> v1:172.21.15.138:6789/0 conn(0x7f92a00d1f90 legacy=0x7f92a00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a00d2c60 con 0x7f92a00d0dd0 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f9290001670 con 0x7f92a00d0dd0 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 >> v1:172.21.15.28:6789/0 conn(0x7f92a00d0dd0 legacy=0x7f92a00d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92840039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9284003fb0 0x7f9284004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9284004d40 0x7f92840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.381+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f928400bad0 con 0x7f9284004d40 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f928400bc80 con 0x7f9284003fb0 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f928400be80 con 0x7f92a015e660 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92840039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929f400640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9284003fb0 0x7f9284004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f92a5000640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9284004d40 0x7f92840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 -- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9284003fb0 msgr2=0x7f9284004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9284003fb0 0x7f9284004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 -- 172.21.15.28:0/1750057166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9284004d40 msgr2=0x7f92840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9284004d40 0x7f92840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929fe00640 1 -- 172.21.15.28:0/1750057166 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9284003520 con 0x7f92a015e660 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f929f400640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9284003fb0 0x7f9284004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.382+0000 7f92a5000640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9284004d40 0x7f92840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:26.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.384+0000 7f929fe00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92840039e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f929000c190 tx=0x7f9284005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=730d056deb8c9a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.384+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290007c60 con 0x7f92a015e660 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.384+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9290005af0 con 0x7f92a015e660 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.384+0000 7f929ea00640 1 -- 172.21.15.28:0/1750057166 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290007070 con 0x7f92a015e660 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 -- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 msgr2=0x7f92840039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 --2- 172.21.15.28:0/1750057166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a015e660 0x7f92840039e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f929000c190 tx=0x7f9284005830 comp rx=0 tx=0).stop 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5000640 1 -- 172.21.15.28:0/1750057166 reap_dead start 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 -- 172.21.15.28:0/1750057166 shutdown_connections 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 -- 172.21.15.28:0/1750057166 >> 172.21.15.28:0/1750057166 conn(0x7f92a008bfc0 msgr2=0x7f92a00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 -- 172.21.15.28:0/1750057166 shutdown_connections 2024-09-10T10:44:26.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.385+0000 7f92a5a00640 1 -- 172.21.15.28:0/1750057166 wait complete. 2024-09-10T10:44:26.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.386+0000 7f92a5a00640 1 Processor -- start 2024-09-10T10:44:26.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.387+0000 7f92a5a00640 1 -- start start 2024-09-10T10:44:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a01e47f0 0x7f92a00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a01e4c70 0x7f92a00b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f92a00d3220 con 0x7f92a015e660 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92a00cf330 con 0x7f92a01e4c70 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00d33a0 con 0x7f92a01e47f0 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34458/0 (socket says 172.21.15.28:34458) 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 -- 172.21.15.28:0/470521255 learned_addr learned my addr 172.21.15.28:0/470521255 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929f400640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a01e47f0 0x7f92a00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f92a5000640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a01e4c70 0x7f92a00b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 -- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a01e47f0 msgr2=0x7f92a00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a01e47f0 0x7f92a00b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 -- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a01e4c70 msgr2=0x7f92a00b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a01e4c70 0x7f92a00b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.388+0000 7f929fe00640 1 -- 172.21.15.28:0/470521255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01e5240 con 0x7f92a015e660 2024-09-10T10:44:26.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f929fe00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f929000c190 tx=0x7f9290001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290037040 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9290001630 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290004a00 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01e5520 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.389+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a01e5ae0 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.390+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a00d24d0 con 0x7f92a015e660 2024-09-10T10:44:26.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.391+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9290004ba0 con 0x7f92a015e660 2024-09-10T10:44:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.392+0000 7f9297e00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 0x7f928006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.392+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f92900ab210 con 0x7f92a015e660 2024-09-10T10:44:26.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.393+0000 7f929f400640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 0x7f928006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:26.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.393+0000 7f929f400640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 0x7f928006e320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92a014e8f0 tx=0x7f928c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:26.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.397+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92900799b0 con 0x7f92a015e660 2024-09-10T10:44:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.625+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f92a00d1b20 con 0x7f92a015e660 2024-09-10T10:44:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.625+0000 7f9297e00640 1 -- 172.21.15.28:0/470521255 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f92900792b0 con 0x7f92a015e660 2024-09-10T10:44:26.626 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T10:44:26.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 msgr2=0x7f928006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 0x7f928006e320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f92a014e8f0 tx=0x7f928c009190 comp rx=0 tx=0).stop 2024-09-10T10:44:26.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 msgr2=0x7f92a01f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:26.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f929000c190 tx=0x7f9290001970 comp rx=0 tx=0).stop 2024-09-10T10:44:26.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 shutdown_connections 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f928006be60 0x7f928006e320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92a01e4c70 0x7f92a00b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92a01e47f0 0x7f92a00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 --2- 172.21.15.28:0/470521255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a015e660 0x7f92a01f3fb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.627+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 >> 172.21.15.28:0/470521255 conn(0x7f92a008bfc0 msgr2=0x7f92a00b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.628+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 shutdown_connections 2024-09-10T10:44:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:26.628+0000 7f92a5a00640 1 -- 172.21.15.28:0/470521255 wait complete. 2024-09-10T10:44:26.644 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T10:44:26.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:44:26.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-09-10T10:44:26.644 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T10:44:26.644 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T10:44:26.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T10:44:26.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T10:44:26.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:44:26.645 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T10:44:26.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:26.645 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-09-10T10:44:26.680 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:44:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42647sec preferred_lft 42647sec 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link noprefixroute 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether c2:7d:5c:bc:44:42 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:44:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::88a5:65ff:fe1d:72/64 scope link 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether c2:7d:5c:bc:44:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c07d:5cff:febc:4442/64 scope link 2024-09-10T10:44:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:26.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:26.684 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:44:26.778 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-09-10T10:44:26.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:26.781 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-09-10T10:44:26.817 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-09-10T10:44:26.817 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:44:26.817 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T10:44:26.818 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T10:44:26.818 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T10:44:26.818 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:26.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:27.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:27.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:27.014 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:44:27.050 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:44:27.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:27.051 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-09-10T10:44:27.125 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T10:44:27.126 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:27.126 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:27.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:44:27.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T10:44:27.126 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:27.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:27.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:27.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:27.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T10:44:27.356 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:27.356 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:27.356 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:27.356 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 6a:6c:50:0f:2b:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::686c:50ff:fe0f:2bd2/64 scope link 2024-09-10T10:44:27.357 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:27.359 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T10:44:27.414 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:44:27.414 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 6a:6c:50:0f:2b:d2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::686c:50ff:fe0f:2bd2/64 scope link 2024-09-10T10:44:27.415 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:44:27.417 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T10:44:27.417 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:27.417 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T10:44:27.418 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:27.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:27.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:27.639 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:44:27.639 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-09-10T10:44:27.639 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-09-10T10:44:27.639 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T10:44:27.639 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-09-10T10:44:27.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:27.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:27.804 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:27.804 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T10:44:27.839 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T10:44:27.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:44:27.840 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T10:44:27.914 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:44:27.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:27.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T10:44:28.042 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T10:44:28.043 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T10:44:28.043 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:28.058 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T10:44:28.058 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T10:44:28.058 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:28.058 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:28.058 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:28.081 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:44:28.082 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:44:28.084 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-10T10:44:28.084 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:44:28.084 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:44:28.084 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:44:28.084 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:44:28.087 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:44:28.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:28.087 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T10:44:28.142 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:28.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:28.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:28.232 INFO:teuthology.orchestra.run.smithi138.stdout:client4624 2024-09-10T10:44:28.232 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:28.232 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:28.268 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:28.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:28.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:28.400 INFO:teuthology.orchestra.run.smithi138.stdout:client4624 2024-09-10T10:44:28.401 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:44:28.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:44:28.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:28.490 INFO:teuthology.orchestra.run.smithi138.stdout:client4624 2024-09-10T10:44:28.490 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f658987a-dd14-4573-bcd3-ee2a383c9fe0.client4624/status') 2024-09-10T10:44:28.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:44:28.592 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4624 (3)192.168.0.2:0/2885858424 2024-09-10T10:44:28.593 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-09-10T10:44:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-09-10T10:44:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-09-10T10:44:28.593 INFO:teuthology.orchestra.run.smithi138.stderr:69 bytes copied, 8.8854e-05 s, 777 kB/s 2024-09-10T10:44:28.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-10T10:44:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:44:28.603 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T10:44:28.655 INFO:tasks.cephfs_test_runner:Starting test: test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:44:28.656 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:28.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.822+0000 7fc965200640 1 Processor -- start 2024-09-10T10:44:28.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.822+0000 7fc965200640 1 -- start start 2024-09-10T10:44:28.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.823+0000 7fc965200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc960072320 0x7fc9600cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.823+0000 7fc965200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc96015e1f0 0x7fc96006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.823+0000 7fc965200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc960146140 con 0x7fc960072de0 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.823+0000 7fc965200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc9600cd740 con 0x7fc96015e1f0 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.823+0000 7fc965200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc9600cda20 con 0x7fc960072320 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc96015e1f0 0x7fc96006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc96015e1f0 0x7fc96006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34464/0 (socket says 172.21.15.28:34464) 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 -- 172.21.15.28:0/1882638715 learned_addr learned my addr 172.21.15.28:0/1882638715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95d400640 1 --1- 172.21.15.28:0/1882638715 >> v1:172.21.15.28:6789/0 conn(0x7fc960072de0 0x7fc96006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52192/0 (socket says 172.21.15.28:52192) 2024-09-10T10:44:28.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95de00640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc960072320 0x7fc9600cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 -- 172.21.15.28:0/1882638715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc960072320 msgr2=0x7fc9600cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc960072320 0x7fc9600cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 -- 172.21.15.28:0/1882638715 >> v1:172.21.15.28:6789/0 conn(0x7fc960072de0 legacy=0x7fc96006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.824+0000 7fc95e800640 1 -- 172.21.15.28:0/1882638715 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9600cdd00 con 0x7fc96015e1f0 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.825+0000 7fc95e800640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc96015e1f0 0x7fc96006e7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc958009e50 tx=0x7fc95802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774cfec462a1f4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:28.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.825+0000 7fc95ca00640 1 -- 172.21.15.28:0/1882638715 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958036070 con 0x7fc96015e1f0 2024-09-10T10:44:28.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.825+0000 7fc95ca00640 1 -- 172.21.15.28:0/1882638715 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc958031600 con 0x7fc96015e1f0 2024-09-10T10:44:28.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.825+0000 7fc95ca00640 1 -- 172.21.15.28:0/1882638715 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580319d0 con 0x7fc96015e1f0 2024-09-10T10:44:28.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 -- 172.21.15.28:0/1882638715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc96015e1f0 msgr2=0x7fc96006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc96015e1f0 0x7fc96006e7c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc958009e50 tx=0x7fc95802a680 comp rx=0 tx=0).stop 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 -- 172.21.15.28:0/1882638715 shutdown_connections 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc96015e1f0 0x7fc96006e7c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 --2- 172.21.15.28:0/1882638715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc960072320 0x7fc9600cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 -- 172.21.15.28:0/1882638715 >> 172.21.15.28:0/1882638715 conn(0x7fc960067d70 msgr2=0x7fc96006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 -- 172.21.15.28:0/1882638715 shutdown_connections 2024-09-10T10:44:28.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.826+0000 7fc965200640 1 -- 172.21.15.28:0/1882638715 wait complete. 2024-09-10T10:44:28.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.828+0000 7fc965200640 1 Processor -- start 2024-09-10T10:44:28.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.828+0000 7fc965200640 1 -- start start 2024-09-10T10:44:28.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 0x7fc9600aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 0x7fc960138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600cd520 con 0x7fc960072de0 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc9600cd3a0 con 0x7fc960072320 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc965200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc96006ed00 con 0x7fc96015e1f0 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34478/0 (socket says 172.21.15.28:34478) 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95d400640 1 -- 172.21.15.28:0/624459043 learned_addr learned my addr 172.21.15.28:0/624459043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95e800640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 0x7fc960138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95de00640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 0x7fc9600aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.829+0000 7fc95d400640 1 -- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 msgr2=0x7fc960138160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95d400640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 0x7fc960138160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95d400640 1 -- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 msgr2=0x7fc9600aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95d400640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 0x7fc9600aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95d400640 1 -- 172.21.15.28:0/624459043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96013c3c0 con 0x7fc960072de0 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95e800640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 0x7fc960138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95de00640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 0x7fc9600aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:28.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc95d400640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c440 tx=0x7fc94c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:28.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0074e0 con 0x7fc960072de0 2024-09-10T10:44:28.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c007680 con 0x7fc960072de0 2024-09-10T10:44:28.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c004a20 con 0x7fc960072de0 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc960138cd0 con 0x7fc960072de0 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.830+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9600aa4e0 con 0x7fc960072de0 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.831+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc960072700 con 0x7fc960072de0 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.832+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc94c01a020 con 0x7fc960072de0 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.833+0000 7fc956000640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:28.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.834+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fc94c08aa80 con 0x7fc960072de0 2024-09-10T10:44:28.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.834+0000 7fc95de00640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:28.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.835+0000 7fc95de00640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 0x7fc94006e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc950009af0 tx=0x7fc950008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:28.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:28.838+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc94c08ce20 con 0x7fc960072de0 2024-09-10T10:44:29.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.025+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9600cebc0 con 0x7fc960072de0 2024-09-10T10:44:29.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.026+0000 7fc956000640 1 -- 172.21.15.28:0/624459043 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18721 (secure 0 0 0) 0x7fc94c010300 con 0x7fc960072de0 2024-09-10T10:44:29.028 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:29.028 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":29,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:27.552353+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3f","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 msgr2=0x7fc94006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 0x7fc94006e2f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc950009af0 tx=0x7fc950008040 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 msgr2=0x7fc9600cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c440 tx=0x7fc94c001e70 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 shutdown_connections 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc94006be30 0x7fc94006e2f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.028+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc96015e1f0 0x7fc960138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.029+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc960072de0 0x7fc9600cbfe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.029+0000 7fc965200640 1 --2- 172.21.15.28:0/624459043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc960072320 0x7fc9600aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.029+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 >> 172.21.15.28:0/624459043 conn(0x7fc960067d70 msgr2=0x7fc9600b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.029+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 shutdown_connections 2024-09-10T10:44:29.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.029+0000 7fc965200640 1 -- 172.21.15.28:0/624459043 wait complete. 2024-09-10T10:44:29.046 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T10:43:45.223771+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T10:43:45.928275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T10:43:49.581202+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T10:44:29.046 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:44:29.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.206+0000 7fa997400640 1 Processor -- start 2024-09-10T10:44:29.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.206+0000 7fa997400640 1 -- start start 2024-09-10T10:44:29.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa997400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 0x7fa99006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa997400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa99015ff70 con 0x7fa9900cf280 2024-09-10T10:44:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa997400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9900d4d10 con 0x7fa990162340 2024-09-10T10:44:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa997400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa99006de90 con 0x7fa990162e30 2024-09-10T10:44:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa995400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa9900cf280 0x7fa9900d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52204/0 (socket says 172.21.15.28:52204) 2024-09-10T10:44:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.207+0000 7fa995400640 1 -- 172.21.15.28:0/456462440 learned_addr learned my addr 172.21.15.28:0/456462440 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:29.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98fe00640 1 --2- 172.21.15.28:0/456462440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 0x7fa99006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081139053 0 0) 0x7fa99015ff70 con 0x7fa9900cf280 2024-09-10T10:44:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa978003520 con 0x7fa9900cf280 2024-09-10T10:44:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3005999579 0 0) 0x7fa9900d4d10 con 0x7fa990162340 2024-09-10T10:44:29.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa99015ff70 con 0x7fa990162340 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98fe00640 1 -- 172.21.15.28:0/456462440 >> v1:172.21.15.138:6789/0 conn(0x7fa990162340 legacy=0x7fa990162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98fe00640 1 -- 172.21.15.28:0/456462440 >> v1:172.21.15.28:6789/0 conn(0x7fa9900cf280 legacy=0x7fa9900d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.208+0000 7fa98fe00640 1 -- 172.21.15.28:0/456462440 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9900d4ff0 con 0x7fa990162e30 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.209+0000 7fa98fe00640 1 --2- 172.21.15.28:0/456462440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 0x7fa99006d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c02a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3062defa94af58d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.209+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c037070 con 0x7fa990162e30 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.209+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa97c032440 con 0x7fa990162e30 2024-09-10T10:44:29.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.209+0000 7fa98f400640 1 -- 172.21.15.28:0/456462440 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0327f0 con 0x7fa990162e30 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 -- 172.21.15.28:0/456462440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 msgr2=0x7fa99006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 --2- 172.21.15.28:0/456462440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 0x7fa99006d740 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0097c0 tx=0x7fa97c02a160 comp rx=0 tx=0).stop 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 -- 172.21.15.28:0/456462440 shutdown_connections 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 --2- 172.21.15.28:0/456462440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162e30 0x7fa99006d740 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 -- 172.21.15.28:0/456462440 >> 172.21.15.28:0/456462440 conn(0x7fa99008bfc0 msgr2=0x7fa9900d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 -- 172.21.15.28:0/456462440 shutdown_connections 2024-09-10T10:44:29.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.210+0000 7fa997400640 1 -- 172.21.15.28:0/456462440 wait complete. 2024-09-10T10:44:29.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.212+0000 7fa997400640 1 Processor -- start 2024-09-10T10:44:29.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.212+0000 7fa997400640 1 -- start start 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 0x7fa990152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162340 0x7fa9900cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 0x7fa99014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9900cf710 con 0x7fa990162e30 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa990163270 con 0x7fa9900cf280 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa997400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9900d52d0 con 0x7fa990162340 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa995400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 0x7fa99014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa994a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 0x7fa990152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa98fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162340 0x7fa9900cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa995400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 0x7fa99014b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34486/0 (socket says 172.21.15.28:34486) 2024-09-10T10:44:29.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.213+0000 7fa995400640 1 -- 172.21.15.28:0/2310012889 learned_addr learned my addr 172.21.15.28:0/2310012889 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 -- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162340 msgr2=0x7fa9900cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162340 0x7fa9900cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 -- 172.21.15.28:0/2310012889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 msgr2=0x7fa99014b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 0x7fa99014b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 -- 172.21.15.28:0/2310012889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9900cbae0 con 0x7fa9900cf280 2024-09-10T10:44:29.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa994a00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 0x7fa990152b90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c3c0 tx=0x7fa98400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984017070 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9840052f0 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa990153220 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984003830 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.214+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901537e0 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.215+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9900cfd70 con 0x7fa9900cf280 2024-09-10T10:44:29.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.216+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa98401a070 con 0x7fa9900cf280 2024-09-10T10:44:29.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.217+0000 7fa98cc00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.217+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fa98408a560 con 0x7fa9900cf280 2024-09-10T10:44:29.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.218+0000 7fa98fe00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.218+0000 7fa98fe00640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 0x7fa97006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034000 tx=0x7fa97c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.223+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9900cfd70 con 0x7fa9900cf280 2024-09-10T10:44:29.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.405+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa99006d740 con 0x7fa9900cf280 2024-09-10T10:44:29.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.406+0000 7fa98cc00640 1 -- 172.21.15.28:0/2310012889 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fa9840584b0 con 0x7fa9900cf280 2024-09-10T10:44:29.407 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:44:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 msgr2=0x7fa97006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 0x7fa97006e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa97c034000 tx=0x7fa97c02e000 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 msgr2=0x7fa990152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 0x7fa990152b90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c3c0 tx=0x7fa98400c880 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 shutdown_connections 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa97006be30 0x7fa97006e2f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa990162e30 0x7fa99014b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa990162340 0x7fa9900cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 --2- 172.21.15.28:0/2310012889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9900cf280 0x7fa990152b90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.408+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 >> 172.21.15.28:0/2310012889 conn(0x7fa99008bfc0 msgr2=0x7fa990162720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.409+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 shutdown_connections 2024-09-10T10:44:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.409+0000 7fa997400640 1 -- 172.21.15.28:0/2310012889 wait complete. 2024-09-10T10:44:29.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:44:29.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.585+0000 7f52bd200640 1 Processor -- start 2024-09-10T10:44:29.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.585+0000 7f52bd200640 1 -- start start 2024-09-10T10:44:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52bd200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f52b8162340 0x7f52b806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52bd200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b815ff70 con 0x7f52b8162e30 2024-09-10T10:44:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52bd200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b80d4d10 con 0x7f52b80cf280 2024-09-10T10:44:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52bd200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f52b81632a0 con 0x7f52b8162340 2024-09-10T10:44:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52b5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f52b8162340 0x7f52b806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52b6800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f52b8162e30 0x7f52b80d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52208/0 (socket says 172.21.15.28:52208) 2024-09-10T10:44:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.586+0000 7f52b6800640 1 -- 172.21.15.28:0/2656609388 learned_addr learned my addr 172.21.15.28:0/2656609388 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.587+0000 7f52b5e00640 1 -- 172.21.15.28:0/2656609388 >> v1:172.21.15.138:6789/0 conn(0x7f52b80cf280 legacy=0x7f52b8162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.587+0000 7f52b5e00640 1 -- 172.21.15.28:0/2656609388 >> v1:172.21.15.28:6789/0 conn(0x7f52b8162e30 legacy=0x7f52b80d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.587+0000 7f52b5e00640 1 -- 172.21.15.28:0/2656609388 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b80d4ff0 con 0x7f52b8162340 2024-09-10T10:44:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.587+0000 7f52b4a00640 1 -- 172.21.15.28:0/2656609388 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577381754 0 0) 0x7f52b815ff70 con 0x7f52b8162e30 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.588+0000 7f52b5e00640 1 --2- 172.21.15.28:0/2656609388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162340 0x7f52b806d740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52a80097a0 tx=0x7f52a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a072de32a8eac3de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.588+0000 7f52b4a00640 1 -- 172.21.15.28:0/2656609388 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8037070 con 0x7f52b8162340 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.588+0000 7f52b4a00640 1 -- 172.21.15.28:0/2656609388 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8032440 con 0x7f52b8162340 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.588+0000 7f52b4a00640 1 -- 172.21.15.28:0/2656609388 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8032810 con 0x7f52b8162340 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 -- 172.21.15.28:0/2656609388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162340 msgr2=0x7f52b806d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 --2- 172.21.15.28:0/2656609388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162340 0x7f52b806d740 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52a80097a0 tx=0x7f52a802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 -- 172.21.15.28:0/2656609388 shutdown_connections 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 --2- 172.21.15.28:0/2656609388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162340 0x7f52b806d740 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 -- 172.21.15.28:0/2656609388 >> 172.21.15.28:0/2656609388 conn(0x7f52b808bfc0 msgr2=0x7f52b80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 -- 172.21.15.28:0/2656609388 shutdown_connections 2024-09-10T10:44:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.589+0000 7f52bd200640 1 -- 172.21.15.28:0/2656609388 wait complete. 2024-09-10T10:44:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.590+0000 7f52bd200640 1 Processor -- start 2024-09-10T10:44:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.591+0000 7f52bd200640 1 -- start start 2024-09-10T10:44:29.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.591+0000 7f52bd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b80cf280 0x7f52b8153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.591+0000 7f52bd200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b8162340 0x7f52b8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.591+0000 7f52bd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52bd200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d52d0 con 0x7f52b8162e30 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52bd200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52b806de90 con 0x7f52b80cf280 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52bd200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80cf6c0 con 0x7f52b8162340 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34502/0 (socket says 172.21.15.28:34502) 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 -- 172.21.15.28:0/2859138596 learned_addr learned my addr 172.21.15.28:0/2859138596 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b5e00640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b80cf280 0x7f52b8153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b5400640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b8162340 0x7f52b8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 -- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b8162340 msgr2=0x7f52b8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b8162340 0x7f52b8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 -- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b80cf280 msgr2=0x7f52b8153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b80cf280 0x7f52b8153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.592+0000 7f52b6800640 1 -- 172.21.15.28:0/2859138596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81e84f0 con 0x7f52b8162e30 2024-09-10T10:44:29.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52b6800640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c700 tx=0x7f52b000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0018070 con 0x7f52b8162e30 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b000ce10 con 0x7f52b8162e30 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b000a860 con 0x7f52b8162e30 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81e87d0 con 0x7f52b8162e30 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.593+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b8153c60 con 0x7f52b8162e30 2024-09-10T10:44:29.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.594+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b806d740 con 0x7f52b8162e30 2024-09-10T10:44:29.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.595+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52b001b070 con 0x7f52b8162e30 2024-09-10T10:44:29.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.596+0000 7f52ae000640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 0x7f529806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.596+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f52b008a6c0 con 0x7f52b8162e30 2024-09-10T10:44:29.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.596+0000 7f52b5e00640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 0x7f529806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.597+0000 7f52b5e00640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 0x7f529806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52a8034000 tx=0x7f52a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.600+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52b00586b0 con 0x7f52b8162e30 2024-09-10T10:44:29.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.781+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f52b81611d0 con 0x7f52b8162e30 2024-09-10T10:44:29.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.782+0000 7f52ae000640 1 -- 172.21.15.28:0/2859138596 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f52b005c9b0 con 0x7f52b8162e30 2024-09-10T10:44:29.783 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 msgr2=0x7f529806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 0x7f529806e2f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f52a8034000 tx=0x7f52a802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 msgr2=0x7f52b815d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f52b000c700 tx=0x7f52b000cbc0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 shutdown_connections 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529806be30 0x7f529806e2f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b8162e30 0x7f52b815d600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b8162340 0x7f52b8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 --2- 172.21.15.28:0/2859138596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b80cf280 0x7f52b8153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 >> 172.21.15.28:0/2859138596 conn(0x7f52b808bfc0 msgr2=0x7f52b80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 shutdown_connections 2024-09-10T10:44:29.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.784+0000 7f52bd200640 1 -- 172.21.15.28:0/2859138596 wait complete. 2024-09-10T10:44:29.801 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:44:29.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.958+0000 7f6f49200640 1 Processor -- start 2024-09-10T10:44:29.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.958+0000 7f6f49200640 1 -- start start 2024-09-10T10:44:29.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.959+0000 7f6f49200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f440b4be0 0x7f6f4406f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f49200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6f4415ab90 0x7f6f440d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f49200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f440b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f49200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6f440d0b50 con 0x7f6f440b4be0 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f49200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4406f4e0 con 0x7f6f4415ab90 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f49200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f440ce9c0 con 0x7f6f440d06e0 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f440b4be0 0x7f6f4406f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f440b4be0 0x7f6f4406f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34516/0 (socket says 172.21.15.28:34516) 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 -- 172.21.15.28:0/1058942223 learned_addr learned my addr 172.21.15.28:0/1058942223 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f42800640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f440b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f3be00640 1 --2- 172.21.15.28:0/1058942223 >> v2:172.21.15.138:3300/0 conn(0x7f6f4415ab90 0x7f6f440d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 -- 172.21.15.28:0/1058942223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 msgr2=0x7f6f440b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f440b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 -- 172.21.15.28:0/1058942223 >> v2:172.21.15.138:3300/0 conn(0x7f6f4415ab90 msgr2=0x7f6f440d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:29.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 --2- 172.21.15.28:0/1058942223 >> v2:172.21.15.138:3300/0 conn(0x7f6f4415ab90 0x7f6f440d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.960+0000 7f6f41e00640 1 -- 172.21.15.28:0/1058942223 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4415b130 con 0x7f6f440b4be0 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.961+0000 7f6f41e00640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f440b4be0 0x7f6f4406f880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6f4406ebb0 tx=0x7f6f3402c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a7af8f2f4feb65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.961+0000 7f6f41400640 1 -- 172.21.15.28:0/1058942223 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3403a070 con 0x7f6f440b4be0 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f41400640 1 -- 172.21.15.28:0/1058942223 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f34032480 con 0x7f6f440b4be0 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f41400640 1 -- 172.21.15.28:0/1058942223 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34032850 con 0x7f6f440b4be0 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 -- 172.21.15.28:0/1058942223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f440b4be0 msgr2=0x7f6f4406f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f440b4be0 0x7f6f4406f880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6f4406ebb0 tx=0x7f6f3402c690 comp rx=0 tx=0).stop 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 -- 172.21.15.28:0/1058942223 shutdown_connections 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f440b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 --2- 172.21.15.28:0/1058942223 >> v2:172.21.15.138:3300/0 conn(0x7f6f4415ab90 0x7f6f440d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 --2- 172.21.15.28:0/1058942223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f440b4be0 0x7f6f4406f880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 -- 172.21.15.28:0/1058942223 >> 172.21.15.28:0/1058942223 conn(0x7f6f4406bdf0 msgr2=0x7f6f4406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 -- 172.21.15.28:0/1058942223 shutdown_connections 2024-09-10T10:44:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.962+0000 7f6f49200640 1 -- 172.21.15.28:0/1058942223 wait complete. 2024-09-10T10:44:29.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.964+0000 7f6f49200640 1 Processor -- start 2024-09-10T10:44:29.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.964+0000 7f6f49200640 1 -- start start 2024-09-10T10:44:29.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 0x7f6f4409a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f4409be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4415b9d0 con 0x7f6f4415ab90 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4415b6f0 con 0x7f6f440b4be0 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f49200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f440b4fc0 con 0x7f6f440d06e0 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f42800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f42800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34520/0 (socket says 172.21.15.28:34520) 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.965+0000 7f6f42800640 1 -- 172.21.15.28:0/3903723939 learned_addr learned my addr 172.21.15.28:0/3903723939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f3be00640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f4409be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f41e00640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 0x7f6f4409a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 -- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 msgr2=0x7f6f4409be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f4409be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 -- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 msgr2=0x7f6f4409a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 0x7f6f4409a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 -- 172.21.15.28:0/3903723939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f44151d70 con 0x7f6f4415ab90 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f41e00640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 0x7f6f4409a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f42800640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c007e60 tx=0x7f6f3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c014620 con 0x7f6f4415ab90 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f3c019070 con 0x7f6f4415ab90 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c014a20 con 0x7f6f4415ab90 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4409c6f0 con 0x7f6f4415ab90 2024-09-10T10:44:29.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.966+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4409ccb0 con 0x7f6f4415ab90 2024-09-10T10:44:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.967+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f44072600 con 0x7f6f4415ab90 2024-09-10T10:44:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.968+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f3c014bc0 con 0x7f6f4415ab90 2024-09-10T10:44:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.969+0000 7f6f3a000640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 0x7f6f2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.969+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6f3c08bae0 con 0x7f6f4415ab90 2024-09-10T10:44:29.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.970+0000 7f6f41e00640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 0x7f6f2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:29.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.970+0000 7f6f41e00640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 0x7f6f2006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f34037000 tx=0x7f6f34030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:29.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:29.973+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f3c05a080 con 0x7f6f4415ab90 2024-09-10T10:44:30.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.155+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f440cfc80 con 0x7f6f4415ab90 2024-09-10T10:44:30.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.155+0000 7f6f3a000640 1 -- 172.21.15.28:0/3903723939 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f6f3c059980 con 0x7f6f4415ab90 2024-09-10T10:44:30.156 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 msgr2=0x7f6f2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 0x7f6f2006e2f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6f34037000 tx=0x7f6f34030000 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 msgr2=0x7f6f4414f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c007e60 tx=0x7f6f3c00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 shutdown_connections 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f2006be30 0x7f6f2006e2f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f4415ab90 0x7f6f4414f290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f440d06e0 0x7f6f4409be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 --2- 172.21.15.28:0/3903723939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f440b4be0 0x7f6f4409a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 >> 172.21.15.28:0/3903723939 conn(0x7f6f4406bdf0 msgr2=0x7f6f440b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.157+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 shutdown_connections 2024-09-10T10:44:30.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.158+0000 7f6f49200640 1 -- 172.21.15.28:0/3903723939 wait complete. 2024-09-10T10:44:30.174 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T10:44:30.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.331+0000 7f9174e00640 1 Processor -- start 2024-09-10T10:44:30.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.331+0000 7f9174e00640 1 -- start start 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9174e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f91700b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9174e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91700af210 0x7f9170070a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9174e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f917006cdc0 con 0x7f91700b7250 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9174e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f917006d0a0 con 0x7f91700af210 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9174e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f917006d380 con 0x7f917015e7f0 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f916e800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f91700b7250 0x7f917009aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52224/0 (socket says 172.21.15.28:52224) 2024-09-10T10:44:30.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f916e800640 1 -- 172.21.15.28:0/3014699733 learned_addr learned my addr 172.21.15.28:0/3014699733 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f9167e00640 1 --2- 172.21.15.28:0/3014699733 >> v2:172.21.15.138:3300/0 conn(0x7f91700af210 0x7f9170070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f916de00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f91700b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756359347 0 0) 0x7f917006cdc0 con 0x7f91700b7250 2024-09-10T10:44:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.333+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9154003520 con 0x7f91700b7250 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504663849 0 0) 0x7f9154003520 con 0x7f91700b7250 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 msgr2=0x7f91700b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f91700b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 >> v2:172.21.15.138:3300/0 conn(0x7f91700af210 msgr2=0x7f9170070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 --2- 172.21.15.28:0/3014699733 >> v2:172.21.15.138:3300/0 conn(0x7f91700af210 0x7f9170070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917006d660 con 0x7f91700b7250 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f91680019f0 con 0x7f91700b7250 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f9167e00640 1 --2- 172.21.15.28:0/3014699733 >> v2:172.21.15.138:3300/0 conn(0x7f91700af210 0x7f9170070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 >> v1:172.21.15.28:6789/0 conn(0x7f91700b7250 legacy=0x7f917009aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91540039b0 0x7f9154003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f9154004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f917006d0a0 con 0x7f91540039b0 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f917006d380 con 0x7f9154004d40 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f915400b9c0 con 0x7f917015e7f0 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916de00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91540039b0 0x7f9154003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916e800640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f9167e00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f9154004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916de00640 1 -- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 msgr2=0x7f9154004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916de00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f917015e7f0 0x7f9154004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.334+0000 7f916de00640 1 -- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154004d40 msgr2=0x7f91540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.335+0000 7f916de00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.335+0000 7f916de00640 1 -- 172.21.15.28:0/3014699733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9154003520 con 0x7f91540039b0 2024-09-10T10:44:30.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.335+0000 7f916e800640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9154004d40 0x7f91540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:30.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.335+0000 7f916de00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91540039b0 0x7f9154003dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9160009ac0 tx=0x7f9154005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff99a77583532dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160036070 con 0x7f91540039b0 2024-09-10T10:44:30.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9160031400 con 0x7f91540039b0 2024-09-10T10:44:30.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f916d400640 1 -- 172.21.15.28:0/3014699733 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160009690 con 0x7f91540039b0 2024-09-10T10:44:30.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 -- 172.21.15.28:0/3014699733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91540039b0 msgr2=0x7f9154003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 --2- 172.21.15.28:0/3014699733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91540039b0 0x7f9154003dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9160009ac0 tx=0x7f9154005830 comp rx=0 tx=0).stop 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f916e800640 1 -- 172.21.15.28:0/3014699733 reap_dead start 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 -- 172.21.15.28:0/3014699733 shutdown_connections 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 -- 172.21.15.28:0/3014699733 >> 172.21.15.28:0/3014699733 conn(0x7f91700b3f30 msgr2=0x7f91700b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 -- 172.21.15.28:0/3014699733 shutdown_connections 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.336+0000 7f9174e00640 1 -- 172.21.15.28:0/3014699733 wait complete. 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.338+0000 7f9174e00640 1 Processor -- start 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.338+0000 7f9174e00640 1 -- start start 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91540039b0 0x7f917009da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 0x7f917015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91700b6d10 con 0x7f91700cb6d0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91700b6150 con 0x7f917014baf0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9174e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91700b6e90 con 0x7f91540039b0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9167e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9167e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34528/0 (socket says 172.21.15.28:34528) 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f9167e00640 1 -- 172.21.15.28:0/2211700007 learned_addr learned my addr 172.21.15.28:0/2211700007 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f916de00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91540039b0 0x7f917009da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.339+0000 7f916e800640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 0x7f917015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 -- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91540039b0 msgr2=0x7f917009da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91540039b0 0x7f917009da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 -- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 msgr2=0x7f917015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 0x7f917015be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 -- 172.21.15.28:0/2211700007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917014c220 con 0x7f91700cb6d0 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f916e800640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 0x7f917015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9167e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9158011900 tx=0x7f9158011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9158019070 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9158004440 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915800f980 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917014c500 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.340+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917014cac0 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.341+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9170070a70 con 0x7f91700cb6d0 2024-09-10T10:44:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.342+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f915801c070 con 0x7f91700cb6d0 2024-09-10T10:44:30.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.343+0000 7f9166000640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.343+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f915808b640 con 0x7f91700cb6d0 2024-09-10T10:44:30.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.344+0000 7f916de00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.344+0000 7f916de00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 0x7f914c06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9160033000 tx=0x7f9160008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.348+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9158059620 con 0x7f91700cb6d0 2024-09-10T10:44:30.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.530+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f91700d20a0 con 0x7f91700cb6d0 2024-09-10T10:44:30.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.530+0000 7f9166000640 1 -- 172.21.15.28:0/2211700007 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f915805d920 con 0x7f91700cb6d0 2024-09-10T10:44:30.531 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-09-10T10:44:30.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 msgr2=0x7f914c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 0x7f914c06e2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9160033000 tx=0x7f9160008030 comp rx=0 tx=0).stop 2024-09-10T10:44:30.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 msgr2=0x7f91700cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9158011900 tx=0x7f9158011dc0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 shutdown_connections 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914c06be30 0x7f914c06e2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f917014baf0 0x7f917015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91700cb6d0 0x7f91700cbb30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 --2- 172.21.15.28:0/2211700007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91540039b0 0x7f917009da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 >> 172.21.15.28:0/2211700007 conn(0x7f91700b3f30 msgr2=0x7f91700af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 shutdown_connections 2024-09-10T10:44:30.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.532+0000 7f9174e00640 1 -- 172.21.15.28:0/2211700007 wait complete. 2024-09-10T10:44:30.549 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-10T10:44:30.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.705+0000 7f0d53200640 1 Processor -- start 2024-09-10T10:44:30.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.705+0000 7f0d53200640 1 -- start start 2024-09-10T10:44:30.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d53200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 0x7f0d4c162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d53200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0d4c0b0180 0x7f0d4c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d53200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4c15eb60 con 0x7f0d4c0af280 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d53200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4c0b6db0 con 0x7f0d4c0b0180 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d53200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c15ee40 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0d4c0b0180 0x7f0d4c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0d4c0b0180 0x7f0d4c154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34542/0 (socket says 172.21.15.28:34542) 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 -- 172.21.15.28:0/3398257688 learned_addr learned my addr 172.21.15.28:0/3398257688 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d4be00640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 0x7f0d4c162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 -- 172.21.15.28:0/3398257688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 msgr2=0x7f0d4c162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 0x7f0d4c162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 -- 172.21.15.28:0/3398257688 >> v1:172.21.15.138:6789/0 conn(0x7f0d4c0af280 legacy=0x7f0d4c0afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:30.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.707+0000 7f0d51200640 1 -- 172.21.15.28:0/3398257688 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c15f120 con 0x7f0d4c0b0180 2024-09-10T10:44:30.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.708+0000 7f0d4be00640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 0x7f0d4c162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.708+0000 7f0d51200640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c154d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009e80 tx=0x7f0d4402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb8f88d88aaf167 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d4b400640 1 -- 172.21.15.28:0/3398257688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44036070 con 0x7f0d4c0b0180 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d4b400640 1 -- 172.21.15.28:0/3398257688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d44031600 con 0x7f0d4c0b0180 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d4b400640 1 -- 172.21.15.28:0/3398257688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440319d0 con 0x7f0d4c0b0180 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 -- 172.21.15.28:0/3398257688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b0180 msgr2=0x7f0d4c154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c154d00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009e80 tx=0x7f0d4402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 -- 172.21.15.28:0/3398257688 shutdown_connections 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c154d00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 --2- 172.21.15.28:0/3398257688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0b69d0 0x7f0d4c162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 -- 172.21.15.28:0/3398257688 >> 172.21.15.28:0/3398257688 conn(0x7f0d4c06bc10 msgr2=0x7f0d4c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:30.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 -- 172.21.15.28:0/3398257688 shutdown_connections 2024-09-10T10:44:30.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.709+0000 7f0d53200640 1 -- 172.21.15.28:0/3398257688 wait complete. 2024-09-10T10:44:30.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.711+0000 7f0d53200640 1 Processor -- start 2024-09-10T10:44:30.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.711+0000 7f0d53200640 1 -- start start 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0af280 0x7f0d4c1e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c1f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c155240 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c0af6e0 con 0x7f0d4c0b0180 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d53200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c1553c0 con 0x7f0d4c0af280 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.712+0000 7f0d51200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34548/0 (socket says 172.21.15.28:34548) 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 -- 172.21.15.28:0/4099350578 learned_addr learned my addr 172.21.15.28:0/4099350578 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d43e00640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c1f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d4be00640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0af280 0x7f0d4c1e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 -- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0af280 msgr2=0x7f0d4c1e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0af280 0x7f0d4c1e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 -- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4c0b0180 msgr2=0x7f0d4c1f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c1f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:30.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 -- 172.21.15.28:0/4099350578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c0cece0 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.713+0000 7f0d51200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009fa0 tx=0x7f0d44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.714+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44036050 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.714+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d44042070 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.714+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44007bd0 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.714+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c0cefc0 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.714+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c0cf580 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.715+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c156950 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.716+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d44047460 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.717+0000 7f0d48c00640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 0x7f0d2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:30.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.717+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0d440ac060 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.717+0000 7f0d4be00640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 0x7f0d2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:30.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.718+0000 7f0d4be00640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 0x7f0d2806e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00aac0 tx=0x7f0d3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:30.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.721+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d4407a040 con 0x7f0d4c0b69d0 2024-09-10T10:44:30.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:30.976+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f0d4c0cfa20 con 0x7f0d4c0b69d0 2024-09-10T10:44:31.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.573+0000 7f0d48c00640 1 -- 172.21.15.28:0/4099350578 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v60) ==== 137+0+0 (secure 0 0 0) 0x7f0d4407e340 con 0x7f0d4c0b69d0 2024-09-10T10:44:31.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 msgr2=0x7f0d2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 0x7f0d2806e2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00aac0 tx=0x7f0d3c008040 comp rx=0 tx=0).stop 2024-09-10T10:44:31.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 msgr2=0x7f0d4c0cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009fa0 tx=0x7f0d44004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 shutdown_connections 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2806be30 0x7f0d2806e2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d4c0b69d0 0x7f0d4c0cfd80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4c0b0180 0x7f0d4c1f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 --2- 172.21.15.28:0/4099350578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4c0af280 0x7f0d4c1e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 >> 172.21.15.28:0/4099350578 conn(0x7f0d4c06bc10 msgr2=0x7f0d4c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 shutdown_connections 2024-09-10T10:44:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.577+0000 7f0d53200640 1 -- 172.21.15.28:0/4099350578 wait complete. 2024-09-10T10:44:31.597 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:44:31.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.760+0000 7f6529000640 1 Processor -- start 2024-09-10T10:44:31.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.760+0000 7f6529000640 1 -- start start 2024-09-10T10:44:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.761+0000 7f6529000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 0x7f652406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.761+0000 7f6529000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f65240cf280 0x7f65240d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.761+0000 7f6529000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f652415ff70 con 0x7f6524162340 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.761+0000 7f6529000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f652406de90 con 0x7f65240cf280 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.761+0000 7f6529000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6524163270 con 0x7f6524162e30 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6521e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6524162340 0x7f6524162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52232/0 (socket says 172.21.15.28:52232) 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6521e00640 1 -- 172.21.15.28:0/490341520 learned_addr learned my addr 172.21.15.28:0/490341520 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6522800640 1 --2- 172.21.15.28:0/490341520 >> v2:172.21.15.28:3300/0 conn(0x7f65240cf280 0x7f65240d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f651be00640 1 --2- 172.21.15.28:0/490341520 >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 0x7f652406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6521400640 1 -- 172.21.15.28:0/490341520 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469908069 0 0) 0x7f652415ff70 con 0x7f6524162340 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6522800640 1 -- 172.21.15.28:0/490341520 >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 msgr2=0x7f652406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6522800640 1 --2- 172.21.15.28:0/490341520 >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 0x7f652406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6522800640 1 -- 172.21.15.28:0/490341520 >> v1:172.21.15.28:6789/0 conn(0x7f6524162340 legacy=0x7f6524162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f6522800640 1 -- 172.21.15.28:0/490341520 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65240d4d10 con 0x7f65240cf280 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.762+0000 7f651be00640 1 --2- 172.21.15.28:0/490341520 >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 0x7f652406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:31.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.763+0000 7f6522800640 1 --2- 172.21.15.28:0/490341520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65240cf280 0x7f65240d2990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65240a23d0 tx=0x7f651c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb4880f43ea7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:31.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.763+0000 7f6521400640 1 -- 172.21.15.28:0/490341520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c007740 con 0x7f65240cf280 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.763+0000 7f6521400640 1 -- 172.21.15.28:0/490341520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c0078e0 con 0x7f65240cf280 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.763+0000 7f6521400640 1 -- 172.21.15.28:0/490341520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c007cb0 con 0x7f65240cf280 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 -- 172.21.15.28:0/490341520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65240cf280 msgr2=0x7f65240d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 --2- 172.21.15.28:0/490341520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65240cf280 0x7f65240d2990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f65240a23d0 tx=0x7f651c02a5f0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 -- 172.21.15.28:0/490341520 shutdown_connections 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 --2- 172.21.15.28:0/490341520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65240cf280 0x7f65240d2990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 --2- 172.21.15.28:0/490341520 >> v2:172.21.15.138:3300/0 conn(0x7f6524162e30 0x7f652406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 -- 172.21.15.28:0/490341520 >> 172.21.15.28:0/490341520 conn(0x7f652408bfc0 msgr2=0x7f65240d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 -- 172.21.15.28:0/490341520 shutdown_connections 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.764+0000 7f6529000640 1 -- 172.21.15.28:0/490341520 wait complete. 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.766+0000 7f6529000640 1 Processor -- start 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.766+0000 7f6529000640 1 -- start start 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65240cf280 0x7f65240cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6524162e30 0x7f652415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f65240cf6f0 con 0x7f6524162340 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65240d2470 con 0x7f65240cf280 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6529000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f65240d52d0 con 0x7f6524162e30 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34572/0 (socket says 172.21.15.28:34572) 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 -- 172.21.15.28:0/3064836519 learned_addr learned my addr 172.21.15.28:0/3064836519 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6521e00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65240cf280 0x7f65240cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f6522800640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6524162e30 0x7f652415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 -- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6524162e30 msgr2=0x7f652415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6524162e30 0x7f652415ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 -- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65240cf280 msgr2=0x7f65240cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65240cf280 0x7f65240cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.767+0000 7f651be00640 1 -- 172.21.15.28:0/3064836519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65241f56e0 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f651be00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d800 tx=0x7f650c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c018020 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f650c004030 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652415d560 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650c004420 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.768+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652415dae0 con 0x7f6524162340 2024-09-10T10:44:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.769+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652406d740 con 0x7f6524162340 2024-09-10T10:44:31.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.770+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f650c005990 con 0x7f6524162340 2024-09-10T10:44:31.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.771+0000 7f651a000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 0x7f650006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:31.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.772+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f650c08b160 con 0x7f6524162340 2024-09-10T10:44:31.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.772+0000 7f6521e00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 0x7f650006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:31.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.773+0000 7f6521e00640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 0x7f650006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6514000f60 tx=0x7f6514007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.776+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f650c022020 con 0x7f6524162340 2024-09-10T10:44:31.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.959+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f652408e640 con 0x7f6524162340 2024-09-10T10:44:31.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.963+0000 7f651a000640 1 -- 172.21.15.28:0/3064836519 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f65241f56e0 con 0x7f6524162340 2024-09-10T10:44:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 msgr2=0x7f650006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 0x7f650006e2f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6514000f60 tx=0x7f6514007d50 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 msgr2=0x7f65241f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f650c00d800 tx=0x7f650c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 shutdown_connections 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f650006be30 0x7f650006e2f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6524162e30 0x7f652415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6524162340 0x7f65241f51a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 --2- 172.21.15.28:0/3064836519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65240cf280 0x7f65240cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 >> 172.21.15.28:0/3064836519 conn(0x7f652408bfc0 msgr2=0x7f65240d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 shutdown_connections 2024-09-10T10:44:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:31.965+0000 7f6529000640 1 -- 172.21.15.28:0/3064836519 wait complete. 2024-09-10T10:44:31.982 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:44:31.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:44:32.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:44:32.166 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:44:32.167 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:32.167 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:32.203 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:32.203 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:32.278 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:32.278 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:32.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:44:32.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.549+0000 7f32c3c00640 1 Processor -- start 2024-09-10T10:44:32.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.550+0000 7f32c3c00640 1 -- start start 2024-09-10T10:44:32.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c3c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32bc072320 0x7f32bc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c3c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c3c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32bc146140 con 0x7f32bc072de0 2024-09-10T10:44:32.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c3c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f32bc0cd740 con 0x7f32bc072320 2024-09-10T10:44:32.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c3c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32bc0cda20 con 0x7f32bc15e1f0 2024-09-10T10:44:32.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32bc072320 0x7f32bc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32bc072320 0x7f32bc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34586/0 (socket says 172.21.15.28:34586) 2024-09-10T10:44:32.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 -- 172.21.15.28:0/2984206913 learned_addr learned my addr 172.21.15.28:0/2984206913 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:32.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1c00640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 -- 172.21.15.28:0/2984206913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 msgr2=0x7f32bc06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc06e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.551+0000 7f32c1200640 1 -- 172.21.15.28:0/2984206913 >> v1:172.21.15.138:6789/0 conn(0x7f32bc072de0 legacy=0x7f32bc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.552+0000 7f32c1200640 1 -- 172.21.15.28:0/2984206913 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc0cdd00 con 0x7f32bc072320 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.552+0000 7f32c1200640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072320 0x7f32bc0cce60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009870 tx=0x7f32b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f59b62538249b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32bb400640 1 -- 172.21.15.28:0/2984206913 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0037070 con 0x7f32bc072320 2024-09-10T10:44:32.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32bb400640 1 -- 172.21.15.28:0/2984206913 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b0032440 con 0x7f32bc072320 2024-09-10T10:44:32.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32bb400640 1 -- 172.21.15.28:0/2984206913 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032810 con 0x7f32bc072320 2024-09-10T10:44:32.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 -- 172.21.15.28:0/2984206913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072320 msgr2=0x7f32bc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072320 0x7f32bc0cce60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009870 tx=0x7f32b002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 -- 172.21.15.28:0/2984206913 shutdown_connections 2024-09-10T10:44:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 --2- 172.21.15.28:0/2984206913 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072320 0x7f32bc0cce60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 -- 172.21.15.28:0/2984206913 >> 172.21.15.28:0/2984206913 conn(0x7f32bc067d70 msgr2=0x7f32bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 -- 172.21.15.28:0/2984206913 shutdown_connections 2024-09-10T10:44:32.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.553+0000 7f32c3c00640 1 -- 172.21.15.28:0/2984206913 wait complete. 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.555+0000 7f32c3c00640 1 Processor -- start 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.555+0000 7f32c3c00640 1 -- start start 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 0x7f32bc0ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc1e4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc0cd520 con 0x7f32bc072de0 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32bc0cd3a0 con 0x7f32bc072320 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32c3c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32bc06ed00 con 0x7f32bc15e1f0 2024-09-10T10:44:32.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34590/0 (socket says 172.21.15.28:34590) 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.556+0000 7f32bbe00640 1 -- 172.21.15.28:0/3709122004 learned_addr learned my addr 172.21.15.28:0/3709122004 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c1200640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 0x7f32bc0ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c1c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc1e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 -- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 msgr2=0x7f32bc1e4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc1e4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 -- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 msgr2=0x7f32bc0ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 0x7f32bc0ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 -- 172.21.15.28:0/3709122004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32bc0ada20 con 0x7f32bc072de0 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c1c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc1e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c1200640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 0x7f32bc0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:32.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32bbe00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c400 tx=0x7f32a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80052d0 con 0x7f32bc072de0 2024-09-10T10:44:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32a8005490 con 0x7f32bc072de0 2024-09-10T10:44:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32bc1f7e30 con 0x7f32bc072de0 2024-09-10T10:44:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8007260 con 0x7f32bc072de0 2024-09-10T10:44:32.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.557+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32bc1f83f0 con 0x7f32bc072de0 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.558+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32bc072700 con 0x7f32bc072de0 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.559+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f32a8007400 con 0x7f32bc072de0 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.560+0000 7f32b8c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 0x7f329c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.560+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f32a808a700 con 0x7f32bc072de0 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.561+0000 7f32c1200640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 0x7f329c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:32.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.561+0000 7f32c1200640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 0x7f329c06e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034000 tx=0x7f32b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:32.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.564+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f32a8058d30 con 0x7f32bc072de0 2024-09-10T10:44:32.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.761+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32bc0cd160 con 0x7f32bc072de0 2024-09-10T10:44:32.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.761+0000 7f32b8c00640 1 -- 172.21.15.28:0/3709122004 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f32a8058630 con 0x7f32bc072de0 2024-09-10T10:44:32.762 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:44:32.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 msgr2=0x7f329c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 0x7f329c06e2f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f32b0034000 tx=0x7f32b002e000 comp rx=0 tx=0).stop 2024-09-10T10:44:32.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 msgr2=0x7f32bc15bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c400 tx=0x7f32a800c8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 shutdown_connections 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f329c06be30 0x7f329c06e2f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32bc15e1f0 0x7f32bc1e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32bc072de0 0x7f32bc15bcc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 --2- 172.21.15.28:0/3709122004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32bc072320 0x7f32bc0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 >> 172.21.15.28:0/3709122004 conn(0x7f32bc067d70 msgr2=0x7f32bc0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 shutdown_connections 2024-09-10T10:44:32.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:32.763+0000 7f32c3c00640 1 -- 172.21.15.28:0/3709122004 wait complete. 2024-09-10T10:44:32.780 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:44:32.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:32.781 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:44:32.841 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:44:32.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:32.842 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:44:32.878 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:32.878 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:32.917 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:32.918 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:32.996 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:32.996 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:33.073 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:44:33.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.229+0000 7fbb98200640 1 Processor -- start 2024-09-10T10:44:33.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.230+0000 7fbb98200640 1 -- start start 2024-09-10T10:44:33.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb98200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb9006f870 0x7fbb90162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb98200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015eb60 con 0x7fbb900b69d0 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb98200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9015ee40 con 0x7fbb900b0180 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb98200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbb900b6db0 con 0x7fbb9006f870 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb96200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb9006f870 0x7fbb90162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb96200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb9006f870 0x7fbb90162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34596/0 (socket says 172.21.15.28:34596) 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb95800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbb900b69d0 0x7fbb900afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52246/0 (socket says 172.21.15.28:52246) 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.231+0000 7fbb96200640 1 -- 172.21.15.28:0/2874615361 learned_addr learned my addr 172.21.15.28:0/2874615361 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.232+0000 7fbb96200640 1 -- 172.21.15.28:0/2874615361 >> v1:172.21.15.138:6789/0 conn(0x7fbb900b0180 legacy=0x7fbb90154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.232+0000 7fbb96200640 1 -- 172.21.15.28:0/2874615361 >> v1:172.21.15.28:6789/0 conn(0x7fbb900b69d0 legacy=0x7fbb900afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.232+0000 7fbb96200640 1 -- 172.21.15.28:0/2874615361 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9015f120 con 0x7fbb9006f870 2024-09-10T10:44:33.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb96200640 1 --2- 172.21.15.28:0/2874615361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb90162a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009e80 tx=0x7fbb8802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab792ec0941960d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb8fe00640 1 -- 172.21.15.28:0/2874615361 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88036070 con 0x7fbb9006f870 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb8fe00640 1 -- 172.21.15.28:0/2874615361 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88031600 con 0x7fbb9006f870 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb8fe00640 1 -- 172.21.15.28:0/2874615361 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8802ae60 con 0x7fbb9006f870 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb98200640 1 -- 172.21.15.28:0/2874615361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 msgr2=0x7fbb90162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb98200640 1 --2- 172.21.15.28:0/2874615361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb90162a50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009e80 tx=0x7fbb8802a680 comp rx=0 tx=0).stop 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb98200640 1 -- 172.21.15.28:0/2874615361 shutdown_connections 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.233+0000 7fbb98200640 1 --2- 172.21.15.28:0/2874615361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb90162a50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.234+0000 7fbb98200640 1 -- 172.21.15.28:0/2874615361 >> 172.21.15.28:0/2874615361 conn(0x7fbb9006bc10 msgr2=0x7fbb9006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.234+0000 7fbb98200640 1 -- 172.21.15.28:0/2874615361 shutdown_connections 2024-09-10T10:44:33.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.234+0000 7fbb98200640 1 -- 172.21.15.28:0/2874615361 wait complete. 2024-09-10T10:44:33.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.235+0000 7fbb98200640 1 Processor -- start 2024-09-10T10:44:33.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.235+0000 7fbb98200640 1 -- start start 2024-09-10T10:44:33.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb900b0180 0x7fbb900cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 0x7fbb900cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900af400 con 0x7fbb9006f870 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb900af280 con 0x7fbb900b69d0 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb98200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb900af580 con 0x7fbb900b0180 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.236+0000 7fbb95800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34610/0 (socket says 172.21.15.28:34610) 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 -- 172.21.15.28:0/1631599535 learned_addr learned my addr 172.21.15.28:0/1631599535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb96200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 0x7fbb900cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb94e00640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb900b0180 0x7fbb900cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 -- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb900b0180 msgr2=0x7fbb900cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb900b0180 0x7fbb900cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 -- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 msgr2=0x7fbb900cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 0x7fbb900cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 -- 172.21.15.28:0/1631599535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb900cf830 con 0x7fbb9006f870 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb96200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 0x7fbb900cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb95800640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c3c0 tx=0x7fbb8400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb840074e0 con 0x7fbb9006f870 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb84007680 con 0x7fbb9006f870 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.237+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84004a00 con 0x7fbb9006f870 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.238+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb900cfb10 con 0x7fbb9006f870 2024-09-10T10:44:33.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.238+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb900d00d0 con 0x7fbb9006f870 2024-09-10T10:44:33.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.239+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb90162a50 con 0x7fbb9006f870 2024-09-10T10:44:33.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.240+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb8401a020 con 0x7fbb9006f870 2024-09-10T10:44:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.241+0000 7fbb8d600640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 0x7fbb7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.241+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fbb8408a950 con 0x7fbb9006f870 2024-09-10T10:44:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.242+0000 7fbb94e00640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 0x7fbb7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.242+0000 7fbb94e00640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 0x7fbb7006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb901eb1d0 tx=0x7fbb7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.246+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb84058f90 con 0x7fbb9006f870 2024-09-10T10:44:33.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.441+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb9015c370 con 0x7fbb9006f870 2024-09-10T10:44:33.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.442+0000 7fbb8d600640 1 -- 172.21.15.28:0/1631599535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fbb84058890 con 0x7fbb9006f870 2024-09-10T10:44:33.443 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:44:33.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 msgr2=0x7fbb7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 0x7fbb7006e2f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb901eb1d0 tx=0x7fbb7c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 msgr2=0x7fbb900b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400c3c0 tx=0x7fbb8400c880 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 shutdown_connections 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb7006be30 0x7fbb7006e2f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb900b69d0 0x7fbb900cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb900b0180 0x7fbb900cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 --2- 172.21.15.28:0/1631599535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb9006f870 0x7fbb900b1d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 >> 172.21.15.28:0/1631599535 conn(0x7fbb9006bc10 msgr2=0x7fbb900afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 shutdown_connections 2024-09-10T10:44:33.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.444+0000 7fbb98200640 1 -- 172.21.15.28:0/1631599535 wait complete. 2024-09-10T10:44:33.461 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T10:44:33.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:33.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T10:44:33.528 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:44:33.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:33.528 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T10:44:33.573 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:33.573 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:33.609 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:33.609 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:33.685 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:44:33.685 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:33.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:44:33.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.956+0000 7f6c4c800640 1 Processor -- start 2024-09-10T10:44:33.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.956+0000 7f6c4c800640 1 -- start start 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4c800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c44072320 0x7f6c440cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4c800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c4406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c44146140 con 0x7f6c44072de0 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4c800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c440cd740 con 0x7f6c44072320 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4c800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c440cda20 con 0x7f6c4415e1f0 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c49e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c44072320 0x7f6c440cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c4406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c49400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6c44072de0 0x7f6c4406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:49440/0 (socket says 172.21.15.28:49440) 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.958+0000 7f6c49400640 1 -- 172.21.15.28:0/3780835116 learned_addr learned my addr 172.21.15.28:0/3780835116 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.959+0000 7f6c49e00640 1 -- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 msgr2=0x7f6c4406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.959+0000 7f6c49e00640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c4406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.959+0000 7f6c49e00640 1 -- 172.21.15.28:0/3780835116 >> v1:172.21.15.138:6789/0 conn(0x7f6c44072de0 legacy=0x7f6c4406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.959+0000 7f6c49e00640 1 -- 172.21.15.28:0/3780835116 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c440cdd00 con 0x7f6c44072320 2024-09-10T10:44:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.959+0000 7f6c4a800640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c4406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.960+0000 7f6c49e00640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c440cce60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c380097a0 tx=0x7f6c3802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e630a7cb73d4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.960+0000 7f6c48a00640 1 -- 172.21.15.28:0/3780835116 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38037070 con 0x7f6c44072320 2024-09-10T10:44:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.960+0000 7f6c48a00640 1 -- 172.21.15.28:0/3780835116 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3802ae50 con 0x7f6c44072320 2024-09-10T10:44:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.960+0000 7f6c48a00640 1 -- 172.21.15.28:0/3780835116 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38032660 con 0x7f6c44072320 2024-09-10T10:44:33.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 -- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 msgr2=0x7f6c440cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c440cce60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6c380097a0 tx=0x7f6c3802a100 comp rx=0 tx=0).stop 2024-09-10T10:44:33.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 -- 172.21.15.28:0/3780835116 shutdown_connections 2024-09-10T10:44:33.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c4406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3780835116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c440cce60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 -- 172.21.15.28:0/3780835116 >> 172.21.15.28:0/3780835116 conn(0x7f6c44067d70 msgr2=0x7f6c4406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 -- 172.21.15.28:0/3780835116 shutdown_connections 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.961+0000 7f6c4c800640 1 -- 172.21.15.28:0/3780835116 wait complete. 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.963+0000 7f6c4c800640 1 Processor -- start 2024-09-10T10:44:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.963+0000 7f6c4c800640 1 -- start start 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c44143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c441e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c440cd520 con 0x7f6c44072de0 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c440cd3a0 con 0x7f6c44072320 2024-09-10T10:44:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4c800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4406ed00 con 0x7f6c4415e1f0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39374/0 (socket says 172.21.15.28:39374) 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 -- 172.21.15.28:0/3655529498 learned_addr learned my addr 172.21.15.28:0/3655529498 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49e00640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c44143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c4a800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c441e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 -- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 msgr2=0x7f6c441e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c441e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 -- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 msgr2=0x7f6c44143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c44143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.964+0000 7f6c49400640 1 -- 172.21.15.28:0/3655529498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c440c8850 con 0x7f6c44072de0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c49400640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c400 tx=0x7f6c3400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34017070 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c34005ce0 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c440c7fb0 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340072f0 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.965+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c440c8570 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.966+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44072700 con 0x7f6c44072de0 2024-09-10T10:44:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.967+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c3401a070 con 0x7f6c44072de0 2024-09-10T10:44:33.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.968+0000 7f6c3e000640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 0x7f6c2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:33.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.968+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6c3408a460 con 0x7f6c44072de0 2024-09-10T10:44:33.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.969+0000 7f6c49e00640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 0x7f6c2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:33.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.969+0000 7f6c49e00640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 0x7f6c2806e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c38034000 tx=0x7f6c3802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:33.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:33.972+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c34058440 con 0x7f6c44072de0 2024-09-10T10:44:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.167+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6c44145810 con 0x7f6c44072de0 2024-09-10T10:44:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.168+0000 7f6c3e000640 1 -- 172.21.15.28:0/3655529498 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f6c3405c740 con 0x7f6c44072de0 2024-09-10T10:44:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 msgr2=0x7f6c2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 0x7f6c2806e2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6c38034000 tx=0x7f6c3802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 msgr2=0x7f6c44140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c400 tx=0x7f6c3400c800 comp rx=0 tx=0).stop 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 shutdown_connections 2024-09-10T10:44:34.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c2806be30 0x7f6c2806e2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c4415e1f0 0x7f6c441e7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c44072de0 0x7f6c44140df0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 --2- 172.21.15.28:0/3655529498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c44072320 0x7f6c44143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 >> 172.21.15.28:0/3655529498 conn(0x7f6c44067d70 msgr2=0x7f6c4406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 shutdown_connections 2024-09-10T10:44:34.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.170+0000 7f6c4c800640 1 -- 172.21.15.28:0/3655529498 wait complete. 2024-09-10T10:44:34.187 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T10:44:34.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:34.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T10:44:34.247 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:44:34.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:34.247 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T10:44:34.283 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:34.283 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:34.322 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:34.322 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:34.399 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:44:34.399 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:44:34.475 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:44:34.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.634+0000 7fb469c00640 1 Processor -- start 2024-09-10T10:44:34.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.635+0000 7fb469c00640 1 -- start start 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb469c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 0x7fb464165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb469c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4640cdbe0 con 0x7fb46415b300 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb469c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4640cd900 con 0x7fb464072d30 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb469c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4640728a0 con 0x7fb46406f8d0 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb462a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb46415b300 0x7fb4640cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50326/0 (socket says 172.21.15.28:50326) 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb462a00640 1 -- 172.21.15.28:0/3656343612 learned_addr learned my addr 172.21.15.28:0/3656343612 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:34.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb463400640 1 --2- 172.21.15.28:0/3656343612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 0x7fb464165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:34.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.636+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421559049 0 0) 0x7fb4640cdbe0 con 0x7fb46415b300 2024-09-10T10:44:34.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb448003520 con 0x7fb46415b300 2024-09-10T10:44:34.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260971301 0 0) 0x7fb4640cd900 con 0x7fb464072d30 2024-09-10T10:44:34.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4640cdbe0 con 0x7fb464072d30 2024-09-10T10:44:34.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb463400640 1 -- 172.21.15.28:0/3656343612 >> v1:172.21.15.138:6789/0 conn(0x7fb464072d30 legacy=0x7fb46406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb463400640 1 -- 172.21.15.28:0/3656343612 >> v1:172.21.15.28:6789/0 conn(0x7fb46415b300 legacy=0x7fb4640cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 233605057 0 0) 0x7fb448003520 con 0x7fb46415b300 2024-09-10T10:44:34.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.637+0000 7fb463400640 1 -- 172.21.15.28:0/3656343612 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464072250 con 0x7fb46406f8d0 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.638+0000 7fb463400640 1 --2- 172.21.15.28:0/3656343612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 0x7fb464165de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dc0 tx=0x7fb45c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58b5afc49da49e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.638+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454001670 con 0x7fb46406f8d0 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.638+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45c007540 con 0x7fb46406f8d0 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.638+0000 7fb462000640 1 -- 172.21.15.28:0/3656343612 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c007910 con 0x7fb46406f8d0 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 -- 172.21.15.28:0/3656343612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 msgr2=0x7fb464165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 --2- 172.21.15.28:0/3656343612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 0x7fb464165de0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009dc0 tx=0x7fb45c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 -- 172.21.15.28:0/3656343612 shutdown_connections 2024-09-10T10:44:34.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 --2- 172.21.15.28:0/3656343612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb46406f8d0 0x7fb464165de0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 -- 172.21.15.28:0/3656343612 >> 172.21.15.28:0/3656343612 conn(0x7fb464067d70 msgr2=0x7fb46406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:34.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 -- 172.21.15.28:0/3656343612 shutdown_connections 2024-09-10T10:44:34.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.639+0000 7fb469c00640 1 -- 172.21.15.28:0/3656343612 wait complete. 2024-09-10T10:44:34.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.641+0000 7fb469c00640 1 Processor -- start 2024-09-10T10:44:34.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.641+0000 7fb469c00640 1 -- start start 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 0x7fb464156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 0x7fb464154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640cffe0 con 0x7fb46406f8d0 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4640cd5c0 con 0x7fb46415b300 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb469c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4640d0160 con 0x7fb464072d30 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39380/0 (socket says 172.21.15.28:39380) 2024-09-10T10:44:34.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 -- 172.21.15.28:0/955540757 learned_addr learned my addr 172.21.15.28:0/955540757 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb45be00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 0x7fb464156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 -- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 msgr2=0x7fb464156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 0x7fb464156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 -- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 msgr2=0x7fb464154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb463400640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 0x7fb464154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 0x7fb464154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb462a00640 1 -- 172.21.15.28:0/955540757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb464157180 con 0x7fb46406f8d0 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb45be00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 0x7fb464156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.642+0000 7fb463400640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 0x7fb464154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb462a00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c5a0 tx=0x7fb45400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454017070 con 0x7fb46406f8d0 2024-09-10T10:44:34.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb454005d00 con 0x7fb46406f8d0 2024-09-10T10:44:34.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4540072b0 con 0x7fb46406f8d0 2024-09-10T10:44:34.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb464158a30 con 0x7fb46406f8d0 2024-09-10T10:44:34.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.643+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb464158ff0 con 0x7fb46406f8d0 2024-09-10T10:44:34.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.644+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46406f1c0 con 0x7fb46406f8d0 2024-09-10T10:44:34.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.645+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb45401a070 con 0x7fb46406f8d0 2024-09-10T10:44:34.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.646+0000 7fb45aa00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 0x7fb44006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:34.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.646+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fb45408a710 con 0x7fb46406f8d0 2024-09-10T10:44:34.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.646+0000 7fb45be00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 0x7fb44006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:34.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.647+0000 7fb45be00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 0x7fb44006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb464155cb0 tx=0x7fb44c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:34.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.650+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb454058cb0 con 0x7fb46406f8d0 2024-09-10T10:44:34.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.848+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb464159f80 con 0x7fb46406f8d0 2024-09-10T10:44:34.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.848+0000 7fb45aa00640 1 -- 172.21.15.28:0/955540757 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fb4540585b0 con 0x7fb46406f8d0 2024-09-10T10:44:34.851 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:44:34.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 msgr2=0x7fb44006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 0x7fb44006e2f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb464155cb0 tx=0x7fb44c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:34.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 msgr2=0x7fb464158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:34.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb45400c5a0 tx=0x7fb45400ca60 comp rx=0 tx=0).stop 2024-09-10T10:44:34.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 shutdown_connections 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb44006be30 0x7fb44006e2f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb46415b300 0x7fb464154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb464072d30 0x7fb464156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 --2- 172.21.15.28:0/955540757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb46406f8d0 0x7fb464158450 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 >> 172.21.15.28:0/955540757 conn(0x7fb464067d70 msgr2=0x7fb4640cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 shutdown_connections 2024-09-10T10:44:34.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:34.851+0000 7fb469c00640 1 -- 172.21.15.28:0/955540757 wait complete. 2024-09-10T10:44:34.869 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T10:44:34.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:34.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T10:44:34.932 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:44:34.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:44:34.932 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T10:44:34.968 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:44:34.968 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:35.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.122+0000 7f175fc00640 1 Processor -- start 2024-09-10T10:44:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.122+0000 7f175fc00640 1 -- start start 2024-09-10T10:44:35.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f175fc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f175806f870 0x7f1758162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f175fc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f175815eb60 con 0x7f17580b0180 2024-09-10T10:44:35.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f175fc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f175815ee40 con 0x7f17580b69d0 2024-09-10T10:44:35.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f175fc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f17580b6db0 con 0x7f175806f870 2024-09-10T10:44:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f1757e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f17580b0180 0x7f1758154d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50334/0 (socket says 172.21.15.28:50334) 2024-09-10T10:44:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f1757e00640 1 -- 172.21.15.28:0/2447833549 learned_addr learned my addr 172.21.15.28:0/2447833549 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f175dc00640 1 --2- 172.21.15.28:0/2447833549 >> v2:172.21.15.138:3300/0 conn(0x7f175806f870 0x7f1758162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682281541 0 0) 0x7f175815eb60 con 0x7f17580b0180 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.124+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1740003520 con 0x7f17580b0180 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779802383 0 0) 0x7f175815ee40 con 0x7f17580b69d0 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f175815eb60 con 0x7f17580b69d0 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1577861438 0 0) 0x7f1740003520 con 0x7f17580b0180 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f175dc00640 1 -- 172.21.15.28:0/2447833549 >> v1:172.21.15.138:6789/0 conn(0x7f17580b69d0 legacy=0x7f17580afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f175dc00640 1 -- 172.21.15.28:0/2447833549 >> v1:172.21.15.28:6789/0 conn(0x7f17580b0180 legacy=0x7f1758154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.125+0000 7f175dc00640 1 -- 172.21.15.28:0/2447833549 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175815f120 con 0x7f175806f870 2024-09-10T10:44:35.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.126+0000 7f175dc00640 1 --2- 172.21.15.28:0/2447833549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f175806f870 0x7f1758162a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1750009dc0 tx=0x7f175002a590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87c356e933f2b127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.126+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1744001670 con 0x7f175806f870 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.126+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750007540 con 0x7f175806f870 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.126+0000 7f1757400640 1 -- 172.21.15.28:0/2447833549 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750007910 con 0x7f175806f870 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 -- 172.21.15.28:0/2447833549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f175806f870 msgr2=0x7f1758162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 --2- 172.21.15.28:0/2447833549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f175806f870 0x7f1758162a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1750009dc0 tx=0x7f175002a590 comp rx=0 tx=0).stop 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 -- 172.21.15.28:0/2447833549 shutdown_connections 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 --2- 172.21.15.28:0/2447833549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f175806f870 0x7f1758162a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 -- 172.21.15.28:0/2447833549 >> 172.21.15.28:0/2447833549 conn(0x7f175806bc10 msgr2=0x7f175806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 -- 172.21.15.28:0/2447833549 shutdown_connections 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.127+0000 7f175fc00640 1 -- 172.21.15.28:0/2447833549 wait complete. 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.128+0000 7f175fc00640 1 Processor -- start 2024-09-10T10:44:35.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.129+0000 7f175fc00640 1 -- start start 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.129+0000 7f175fc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f175806f870 0x7f17580cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.129+0000 7f175fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 0x7f1758161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.129+0000 7f175fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175fc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f17580af400 con 0x7f17580b69d0 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17580af280 con 0x7f17580b0180 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175fc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f17580af580 con 0x7f175806f870 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39382/0 (socket says 172.21.15.28:39382) 2024-09-10T10:44:35.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 -- 172.21.15.28:0/2491721228 learned_addr learned my addr 172.21.15.28:0/2491721228 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175d200640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f175806f870 0x7f17580cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f1757e00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 0x7f1758161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 -- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f175806f870 msgr2=0x7f17580cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f175806f870 0x7f17580cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 -- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 msgr2=0x7f1758161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 0x7f1758161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f175dc00640 1 -- 172.21.15.28:0/2491721228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17580d04e0 con 0x7f17580b69d0 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.130+0000 7f1757e00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 0x7f1758161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f175dc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1750009ee0 tx=0x7f1750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750040070 con 0x7f17580b69d0 2024-09-10T10:44:35.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1750031bc0 con 0x7f17580b69d0 2024-09-10T10:44:35.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750009450 con 0x7f17580b69d0 2024-09-10T10:44:35.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17580cd3d0 con 0x7f17580b69d0 2024-09-10T10:44:35.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.131+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581f4540 con 0x7f17580b69d0 2024-09-10T10:44:35.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.132+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1758162a50 con 0x7f17580b69d0 2024-09-10T10:44:35.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.133+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1750039040 con 0x7f17580b69d0 2024-09-10T10:44:35.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.134+0000 7f1754c00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 0x7f173806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.134+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f17500ac920 con 0x7f17580b69d0 2024-09-10T10:44:35.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.134+0000 7f175d200640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 0x7f173806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.135+0000 7f175d200640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 0x7f173806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f174c0011e0 tx=0x7f174c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.138+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f17580d04e0 con 0x7f17580b69d0 2024-09-10T10:44:35.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.360+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f175815e340 con 0x7f17580b69d0 2024-09-10T10:44:35.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.361+0000 7f1754c00640 1 -- 172.21.15.28:0/2491721228 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f175007af50 con 0x7f17580b69d0 2024-09-10T10:44:35.363 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:35.364 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:35.364 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 msgr2=0x7f173806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 0x7f173806e2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f174c0011e0 tx=0x7f174c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 msgr2=0x7f175806c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1750009ee0 tx=0x7f1750004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 shutdown_connections 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f173806be30 0x7f173806e2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17580b69d0 0x7f175806c230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17580b0180 0x7f1758161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 --2- 172.21.15.28:0/2491721228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f175806f870 0x7f17580cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 >> 172.21.15.28:0/2491721228 conn(0x7f175806bc10 msgr2=0x7f175806cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 shutdown_connections 2024-09-10T10:44:35.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.363+0000 7f175fc00640 1 -- 172.21.15.28:0/2491721228 wait complete. 2024-09-10T10:44:35.381 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:35.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.535+0000 7f18ace00640 1 Processor -- start 2024-09-10T10:44:35.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.536+0000 7f18ace00640 1 -- start start 2024-09-10T10:44:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18ace00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18a8147360 0x7f18a8070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18ace00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a806cb30 con 0x7f18a80d0bb0 2024-09-10T10:44:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18ace00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a806bfb0 con 0x7f18a8070dd0 2024-09-10T10:44:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18ace00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f18a806c290 con 0x7f18a8147360 2024-09-10T10:44:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18a5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18a8147360 0x7f18a8070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18a5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18a8147360 0x7f18a8070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39396/0 (socket says 172.21.15.28:39396) 2024-09-10T10:44:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.537+0000 7f18a5e00640 1 -- 172.21.15.28:0/2659545876 learned_addr learned my addr 172.21.15.28:0/2659545876 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.538+0000 7f18a5e00640 1 -- 172.21.15.28:0/2659545876 >> v1:172.21.15.138:6789/0 conn(0x7f18a8070dd0 legacy=0x7f18a80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.538+0000 7f18a5e00640 1 -- 172.21.15.28:0/2659545876 >> v1:172.21.15.28:6789/0 conn(0x7f18a80d0bb0 legacy=0x7f18a80ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.538+0000 7f18a5e00640 1 -- 172.21.15.28:0/2659545876 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a806c570 con 0x7f18a8147360 2024-09-10T10:44:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.539+0000 7f18a5e00640 1 --2- 172.21.15.28:0/2659545876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a8070890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1898009870 tx=0x7f189802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5cb555019d730 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.539+0000 7f18a4a00640 1 -- 172.21.15.28:0/2659545876 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898037070 con 0x7f18a8147360 2024-09-10T10:44:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.539+0000 7f18a4a00640 1 -- 172.21.15.28:0/2659545876 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898032440 con 0x7f18a8147360 2024-09-10T10:44:35.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.539+0000 7f18a4a00640 1 -- 172.21.15.28:0/2659545876 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898032810 con 0x7f18a8147360 2024-09-10T10:44:35.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 -- 172.21.15.28:0/2659545876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 msgr2=0x7f18a8070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 --2- 172.21.15.28:0/2659545876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a8070890 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1898009870 tx=0x7f189802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 -- 172.21.15.28:0/2659545876 shutdown_connections 2024-09-10T10:44:35.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 --2- 172.21.15.28:0/2659545876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a8070890 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 -- 172.21.15.28:0/2659545876 >> 172.21.15.28:0/2659545876 conn(0x7f18a80b2f40 msgr2=0x7f18a80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 -- 172.21.15.28:0/2659545876 shutdown_connections 2024-09-10T10:44:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.540+0000 7f18ace00640 1 -- 172.21.15.28:0/2659545876 wait complete. 2024-09-10T10:44:35.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.542+0000 7f18ace00640 1 Processor -- start 2024-09-10T10:44:35.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.542+0000 7f18ace00640 1 -- start start 2024-09-10T10:44:35.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 0x7f18a813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a80d0bb0 0x7f18a81617a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18a806bd90 con 0x7f18a8147360 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18a806bc10 con 0x7f18a80d0bb0 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18ace00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18a80aec60 con 0x7f18a8070dd0 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 0x7f18a813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a80d0bb0 0x7f18a81617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39400/0 (socket says 172.21.15.28:39400) 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a5e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 0x7f18a813e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50934/0 (socket says 172.21.15.28:50934) 2024-09-10T10:44:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.543+0000 7f18a6800640 1 -- 172.21.15.28:0/854863648 learned_addr learned my addr 172.21.15.28:0/854863648 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:35.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 -- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 msgr2=0x7f18a813e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 0x7f18a813e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 -- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a80d0bb0 msgr2=0x7f18a81617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a80d0bb0 0x7f18a81617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 -- 172.21.15.28:0/854863648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a80b6b90 con 0x7f18a8147360 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f18a6800640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c7a0 tx=0x7f18a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0003b70 con 0x7f18a8147360 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.544+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a0003d30 con 0x7f18a8147360 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.545+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a80b5dd0 con 0x7f18a8147360 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.545+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a000aa90 con 0x7f18a8147360 2024-09-10T10:44:35.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.545+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a80b6390 con 0x7f18a8147360 2024-09-10T10:44:35.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.546+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a8070890 con 0x7f18a8147360 2024-09-10T10:44:35.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.547+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18a000ac30 con 0x7f18a8147360 2024-09-10T10:44:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.548+0000 7f189e000640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 0x7f188806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.548+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f18a008aff0 con 0x7f18a8147360 2024-09-10T10:44:35.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.548+0000 7f18a5e00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 0x7f188806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.549+0000 7f18a5e00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 0x7f188806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.552+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18a0058fd0 con 0x7f18a8147360 2024-09-10T10:44:35.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.776+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18a806dfa0 con 0x7f18a8147360 2024-09-10T10:44:35.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.777+0000 7f189e000640 1 -- 172.21.15.28:0/854863648 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f18a005d2d0 con 0x7f18a8147360 2024-09-10T10:44:35.829 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:35.829 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:35.830 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 msgr2=0x7f188806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 0x7f188806e2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1898034000 tx=0x7f189802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 msgr2=0x7f18a81f75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18a000c7a0 tx=0x7f18a000cc60 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 shutdown_connections 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f188806be30 0x7f188806e2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18a8147360 0x7f18a81f75e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a80d0bb0 0x7f18a81617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 --2- 172.21.15.28:0/854863648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18a8070dd0 0x7f18a813e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.779+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 >> 172.21.15.28:0/854863648 conn(0x7f18a80b2f40 msgr2=0x7f18a806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.780+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 shutdown_connections 2024-09-10T10:44:35.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.780+0000 7f18ace00640 1 -- 172.21.15.28:0/854863648 wait complete. 2024-09-10T10:44:35.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:35.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.986+0000 7f8f7be00640 1 Processor -- start 2024-09-10T10:44:35.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.986+0000 7f8f7be00640 1 -- start start 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8f7c06e160 0x7f8f7c0d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7c0cf800 con 0x7f8f7c0cf280 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c0cfdc0 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7c0cfae0 con 0x7f8f7c06e160 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39402/0 (socket says 172.21.15.28:39402) 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.987+0000 7f8f7aa00640 1 -- 172.21.15.28:0/420893199 learned_addr learned my addr 172.21.15.28:0/420893199 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f73e00640 1 --2- 172.21.15.28:0/420893199 >> v2:172.21.15.138:3300/0 conn(0x7f8f7c06e160 0x7f8f7c0d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f7aa00640 1 -- 172.21.15.28:0/420893199 >> v2:172.21.15.138:3300/0 conn(0x7f8f7c06e160 msgr2=0x7f8f7c0d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f7aa00640 1 --2- 172.21.15.28:0/420893199 >> v2:172.21.15.138:3300/0 conn(0x7f8f7c06e160 0x7f8f7c0d1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f7aa00640 1 -- 172.21.15.28:0/420893199 >> v1:172.21.15.28:6789/0 conn(0x7f8f7c0cf280 legacy=0x7f8f7c06d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f7aa00640 1 -- 172.21.15.28:0/420893199 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c0d20c0 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.988+0000 7f8f7a000640 1 -- 172.21.15.28:0/420893199 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1223637934 0 0) 0x7f8f7c0cf800 con 0x7f8f7c0cf280 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.989+0000 7f8f7aa00640 1 --2- 172.21.15.28:0/420893199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c073fe0 tx=0x7f8f6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298fc1101359ac8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.989+0000 7f8f7a000640 1 -- 172.21.15.28:0/420893199 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c037070 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.989+0000 7f8f7a000640 1 -- 172.21.15.28:0/420893199 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c032440 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.989+0000 7f8f7a000640 1 -- 172.21.15.28:0/420893199 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c032810 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 -- 172.21.15.28:0/420893199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c15e4b0 msgr2=0x7f8f7c0d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 --2- 172.21.15.28:0/420893199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c073fe0 tx=0x7f8f6c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 -- 172.21.15.28:0/420893199 shutdown_connections 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 --2- 172.21.15.28:0/420893199 >> v2:172.21.15.138:3300/0 conn(0x7f8f7c06e160 0x7f8f7c0d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 --2- 172.21.15.28:0/420893199 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0d0b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 -- 172.21.15.28:0/420893199 >> 172.21.15.28:0/420893199 conn(0x7f8f7c08bfc0 msgr2=0x7f8f7c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:35.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 -- 172.21.15.28:0/420893199 shutdown_connections 2024-09-10T10:44:35.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.990+0000 7f8f7be00640 1 -- 172.21.15.28:0/420893199 wait complete. 2024-09-10T10:44:35.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.991+0000 7f8f7be00640 1 Processor -- start 2024-09-10T10:44:35.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.991+0000 7f8f7be00640 1 -- start start 2024-09-10T10:44:35.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 0x7f8f7c0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d38d0 con 0x7f8f7c06e160 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c15e890 con 0x7f8f7c15e4b0 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.992+0000 7f8f7be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7c0d3a50 con 0x7f8f7c0cf280 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39410/0 (socket says 172.21.15.28:39410) 2024-09-10T10:44:35.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 -- 172.21.15.28:0/1742180290 learned_addr learned my addr 172.21.15.28:0/1742180290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:35.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7b400640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f73e00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 0x7f8f7c0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:35.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 -- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 msgr2=0x7f8f7c0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 0x7f8f7c0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 -- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 msgr2=0x7f8f7c0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 -- 172.21.15.28:0/1742180290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c15f350 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f73e00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 0x7f8f7c0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7b400640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.993+0000 7f8f7aa00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000c00 tx=0x7f8f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.994+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c037040 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.994+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c004230 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.994+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c003610 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.994+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c0b9a60 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.994+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c15f8d0 con 0x7f8f7c06e160 2024-09-10T10:44:35.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.995+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c0d4ce0 con 0x7f8f7c06e160 2024-09-10T10:44:35.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.996+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f6c0037b0 con 0x7f8f7c06e160 2024-09-10T10:44:35.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.996+0000 7f8f72a00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 0x7f8f5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:35.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.997+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f8f6c0ac070 con 0x7f8f7c06e160 2024-09-10T10:44:35.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.997+0000 7f8f73e00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 0x7f8f5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:35.998+0000 7f8f73e00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 0x7f8f5806e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0c9c50 tx=0x7f8f64008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.001+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f6c07a610 con 0x7f8f7c06e160 2024-09-10T10:44:36.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.188+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f7c06bc10 con 0x7f8f7c06e160 2024-09-10T10:44:36.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.189+0000 7f8f72a00640 1 -- 172.21.15.28:0/1742180290 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18721 (secure 0 0 0) 0x7f8f6c079f10 con 0x7f8f7c06e160 2024-09-10T10:44:36.191 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:36.191 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":29,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:27.552353+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3f","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:36.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 msgr2=0x7f8f5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 0x7f8f5806e2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0c9c50 tx=0x7f8f64008040 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 msgr2=0x7f8f7c0b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c000c00 tx=0x7f8f6c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 shutdown_connections 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f5806be30 0x7f8f5806e2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f7c15e4b0 0x7f8f7c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f7c0cf280 0x7f8f7c0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 --2- 172.21.15.28:0/1742180290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f7c06e160 0x7f8f7c0b9130 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 >> 172.21.15.28:0/1742180290 conn(0x7f8f7c08bfc0 msgr2=0x7f8f7c0b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 shutdown_connections 2024-09-10T10:44:36.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.191+0000 7f8f7be00640 1 -- 172.21.15.28:0/1742180290 wait complete. 2024-09-10T10:44:36.209 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:44:36.209 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:36.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.364+0000 7fe245800640 1 Processor -- start 2024-09-10T10:44:36.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.364+0000 7fe245800640 1 -- start start 2024-09-10T10:44:36.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe245800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 0x7fe2400d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe245800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2400d1f90 0x7fe2400cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe245800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400cf000 con 0x7fe24015e660 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe245800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe24006d3e0 con 0x7fe2400d1f90 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe245800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400b37d0 con 0x7fe2400d0dd0 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe244e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2400d1f90 0x7fe2400cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe244e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2400d1f90 0x7fe2400cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39422/0 (socket says 172.21.15.28:39422) 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.365+0000 7fe244e00640 1 -- 172.21.15.28:0/371484631 learned_addr learned my addr 172.21.15.28:0/371484631 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe23fe00640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 0x7fe2400d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe244e00640 1 -- 172.21.15.28:0/371484631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 msgr2=0x7fe2400d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe244e00640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 0x7fe2400d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe244e00640 1 -- 172.21.15.28:0/371484631 >> v1:172.21.15.138:6789/0 conn(0x7fe24015e660 legacy=0x7fe24006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe244e00640 1 -- 172.21.15.28:0/371484631 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400d16f0 con 0x7fe2400d1f90 2024-09-10T10:44:36.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.366+0000 7fe23fe00640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 0x7fe2400d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe244e00640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2400cf7c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e50 tx=0x7fe23802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a951d1bbd3ece1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe23ea00640 1 -- 172.21.15.28:0/371484631 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe238036070 con 0x7fe2400d1f90 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe23ea00640 1 -- 172.21.15.28:0/371484631 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238031600 con 0x7fe2400d1f90 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe23ea00640 1 -- 172.21.15.28:0/371484631 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2380319d0 con 0x7fe2400d1f90 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe245800640 1 -- 172.21.15.28:0/371484631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 msgr2=0x7fe2400cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.367+0000 7fe245800640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2400cf7c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe238009e50 tx=0x7fe23802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 -- 172.21.15.28:0/371484631 shutdown_connections 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2400cf7c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 --2- 172.21.15.28:0/371484631 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2400d0dd0 0x7fe2400d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 -- 172.21.15.28:0/371484631 >> 172.21.15.28:0/371484631 conn(0x7fe24008bfc0 msgr2=0x7fe2400d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:36.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 -- 172.21.15.28:0/371484631 shutdown_connections 2024-09-10T10:44:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.368+0000 7fe245800640 1 -- 172.21.15.28:0/371484631 wait complete. 2024-09-10T10:44:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.369+0000 7fe245800640 1 Processor -- start 2024-09-10T10:44:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.369+0000 7fe245800640 1 -- start start 2024-09-10T10:44:36.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2400d0dd0 0x7fe24015d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe24015e660 0x7fe2401e7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d2c60 con 0x7fe2400d1f90 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400cf330 con 0x7fe2400d0dd0 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe245800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400d2f40 con 0x7fe24015e660 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe23f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe23f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39426/0 (socket says 172.21.15.28:39426) 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.370+0000 7fe23f400640 1 -- 172.21.15.28:0/3874154869 learned_addr learned my addr 172.21.15.28:0/3874154869 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23fe00640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2400d0dd0 0x7fe24015d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe244e00640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe24015e660 0x7fe2401e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 -- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe24015e660 msgr2=0x7fe2401e7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe24015e660 0x7fe2401e7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 -- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2400d0dd0 msgr2=0x7fe24015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2400d0dd0 0x7fe24015d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 -- 172.21.15.28:0/3874154869 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400cb630 con 0x7fe2400d1f90 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe23f400640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b410 tx=0x7fe22c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c015020 con 0x7fe2400d1f90 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c011830 con 0x7fe2400d1f90 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.371+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c011c20 con 0x7fe2400d1f90 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.372+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2400cb8d0 con 0x7fe2400d1f90 2024-09-10T10:44:36.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.372+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2400cbe90 con 0x7fe2400d1f90 2024-09-10T10:44:36.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.373+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2401e86a0 con 0x7fe2400d1f90 2024-09-10T10:44:36.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.374+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe22c018020 con 0x7fe2400d1f90 2024-09-10T10:44:36.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.375+0000 7fe237e00640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.375+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fe22c08aff0 con 0x7fe2400d1f90 2024-09-10T10:44:36.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.375+0000 7fe23fe00640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.376+0000 7fe23fe00640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 0x7fe22006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2401e99f0 tx=0x7fe230008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.379+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe22c01f020 con 0x7fe2400d1f90 2024-09-10T10:44:36.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.602+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2400d1b20 con 0x7fe2400d1f90 2024-09-10T10:44:36.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.603+0000 7fe237e00640 1 -- 172.21.15.28:0/3874154869 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7fe22c058fa0 con 0x7fe2400d1f90 2024-09-10T10:44:36.605 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:36.605 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:36.605 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 msgr2=0x7fe22006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 0x7fe22006e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2401e99f0 tx=0x7fe230008040 comp rx=0 tx=0).stop 2024-09-10T10:44:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 msgr2=0x7fe2401e6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe22c00b410 tx=0x7fe22c00b8d0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 shutdown_connections 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe22006be30 0x7fe22006e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe24015e660 0x7fe2401e7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2400d1f90 0x7fe2401e6c30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 --2- 172.21.15.28:0/3874154869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2400d0dd0 0x7fe24015d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 >> 172.21.15.28:0/3874154869 conn(0x7fe24008bfc0 msgr2=0x7fe2400d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 shutdown_connections 2024-09-10T10:44:36.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.605+0000 7fe245800640 1 -- 172.21.15.28:0/3874154869 wait complete. 2024-09-10T10:44:36.624 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:36.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.777+0000 7f3b5fe00640 1 Processor -- start 2024-09-10T10:44:36.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.778+0000 7f3b5fe00640 1 -- start start 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b5815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b58072700 0x7f3b5815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b58146140 con 0x7f3b58072320 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3b580cd740 con 0x7f3b58072700 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b580cda20 con 0x7f3b58072de0 2024-09-10T10:44:36.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3b58072320 0x7f3b580cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50356/0 (socket says 172.21.15.28:50356) 2024-09-10T10:44:36.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5d400640 1 -- 172.21.15.28:0/4065681213 learned_addr learned my addr 172.21.15.28:0/4065681213 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:36.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.779+0000 7f3b5de00640 1 --2- 172.21.15.28:0/4065681213 >> v2:172.21.15.28:3300/0 conn(0x7f3b58072700 0x7f3b5815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b5ca00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b5815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b57e00640 1 -- 172.21.15.28:0/4065681213 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637798722 0 0) 0x7f3b58146140 con 0x7f3b58072320 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b57e00640 1 -- 172.21.15.28:0/4065681213 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b40003520 con 0x7f3b58072320 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b5de00640 1 -- 172.21.15.28:0/4065681213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 msgr2=0x7f3b5815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b5de00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b5815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b5de00640 1 -- 172.21.15.28:0/4065681213 >> v1:172.21.15.28:6789/0 conn(0x7f3b58072320 legacy=0x7f3b580cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.780+0000 7f3b5de00640 1 -- 172.21.15.28:0/4065681213 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580cdd00 con 0x7f3b58072700 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.781+0000 7f3b5de00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b5815d720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009d90 tx=0x7f3b5002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a92e11637a5ac0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.781+0000 7f3b57e00640 1 -- 172.21.15.28:0/4065681213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031600 con 0x7f3b58072700 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.781+0000 7f3b57e00640 1 -- 172.21.15.28:0/4065681213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b500317a0 con 0x7f3b58072700 2024-09-10T10:44:36.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.781+0000 7f3b57e00640 1 -- 172.21.15.28:0/4065681213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b50031b50 con 0x7f3b58072700 2024-09-10T10:44:36.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.781+0000 7f3b5fe00640 1 -- 172.21.15.28:0/4065681213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 msgr2=0x7f3b5815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b5815d720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009d90 tx=0x7f3b5002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 -- 172.21.15.28:0/4065681213 shutdown_connections 2024-09-10T10:44:36.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b5815d720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/4065681213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b5815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 -- 172.21.15.28:0/4065681213 >> 172.21.15.28:0/4065681213 conn(0x7f3b58067d70 msgr2=0x7f3b5806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 -- 172.21.15.28:0/4065681213 shutdown_connections 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.782+0000 7f3b5fe00640 1 -- 172.21.15.28:0/4065681213 wait complete. 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.783+0000 7f3b5fe00640 1 Processor -- start 2024-09-10T10:44:36.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.783+0000 7f3b5fe00640 1 -- start start 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 0x7f3b5813ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b581e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5815dde0 con 0x7f3b58072700 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5815dc60 con 0x7f3b58072320 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.784+0000 7f3b5fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5815df60 con 0x7f3b58072de0 2024-09-10T10:44:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39458/0 (socket says 172.21.15.28:39458) 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 -- 172.21.15.28:0/3440099709 learned_addr learned my addr 172.21.15.28:0/3440099709 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5d400640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 0x7f3b5813ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5de00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b581e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 -- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 msgr2=0x7f3b581e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b581e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 -- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 msgr2=0x7f3b5813ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 0x7f3b5813ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 -- 172.21.15.28:0/3440099709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b580b4470 con 0x7f3b58072700 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5d400640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 0x7f3b5813ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:36.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.785+0000 7f3b5ca00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d620 tx=0x7f3b4400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.786+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400b620 con 0x7f3b58072700 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.786+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b44019070 con 0x7f3b58072700 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.786+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400ba00 con 0x7f3b58072700 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.786+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b581f4150 con 0x7f3b58072700 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.786+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b581f4710 con 0x7f3b58072700 2024-09-10T10:44:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.787+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5815e390 con 0x7f3b58072700 2024-09-10T10:44:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.788+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b4400bba0 con 0x7f3b58072700 2024-09-10T10:44:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.789+0000 7f3b55600640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 0x7f3b3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.789+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f3b4408b720 con 0x7f3b58072700 2024-09-10T10:44:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.789+0000 7f3b5d400640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 0x7f3b3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:36.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.790+0000 7f3b5d400640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 0x7f3b3806e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c007fd0 tx=0x7f3b4c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:36.794+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b44059c40 con 0x7f3b58072700 2024-09-10T10:44:37.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.011+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b580cc250 con 0x7f3b58072700 2024-09-10T10:44:37.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.012+0000 7f3b55600640 1 -- 172.21.15.28:0/3440099709 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5222 (secure 0 0 0) 0x7f3b44059540 con 0x7f3b58072700 2024-09-10T10:44:37.014 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:37.014 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"btime":"2024-09-10T10:44:08:607708+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1444566321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1444566321},{"type":"v1","addr":"172.21.15.138:6836","nonce":1444566321}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3357776494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3357776494},{"type":"v1","addr":"172.21.15.138:6837","nonce":3357776494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1944650958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1944650958},{"type":"v1","addr":"172.21.15.28:6833","nonce":1944650958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:43:46.939715+0000","modified":"2024-09-10T10:44:08.023962+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4284},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4284":{"gid":4284,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/112106318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":112106318},{"type":"v1","addr":"172.21.15.28:6835","nonce":112106318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4284,"qdb_cluster":[4284]},"id":1}]} 2024-09-10T10:44:37.015 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2024-09-10T10:44:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 msgr2=0x7f3b3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 0x7f3b3806e2f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c007fd0 tx=0x7f3b4c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 msgr2=0x7f3b581e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d620 tx=0x7f3b4400dae0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 shutdown_connections 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b3806be30 0x7f3b3806e2f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b58072de0 0x7f3b581e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b58072700 0x7f3b581e76c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 --2- 172.21.15.28:0/3440099709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b58072320 0x7f3b5813ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 >> 172.21.15.28:0/3440099709 conn(0x7f3b58067d70 msgr2=0x7f3b5806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 shutdown_connections 2024-09-10T10:44:37.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.014+0000 7f3b5fe00640 1 -- 172.21.15.28:0/3440099709 wait complete. 2024-09-10T10:44:37.032 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:37.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.185+0000 7fa06f800640 1 Processor -- start 2024-09-10T10:44:37.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.185+0000 7fa06f800640 1 -- start start 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa068162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06f800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa0680af280 0x7fa068154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06f800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa06815eb60 con 0x7fa0680b69d0 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06f800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa0680b6db0 con 0x7fa0680af280 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa06815ee40 con 0x7fa0680b0180 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06ce00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa0680b69d0 0x7fa0680afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50360/0 (socket says 172.21.15.28:50360) 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06ce00640 1 -- 172.21.15.28:0/4266250145 learned_addr learned my addr 172.21.15.28:0/4266250145 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:37.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06d800640 1 --2- 172.21.15.28:0/4266250145 >> v2:172.21.15.28:3300/0 conn(0x7fa0680af280 0x7fa068154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa05fe00640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa068162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06d800640 1 -- 172.21.15.28:0/4266250145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 msgr2=0x7fa068162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa067e00640 1 -- 172.21.15.28:0/4266250145 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341336776 0 0) 0x7fa06815eb60 con 0x7fa0680b69d0 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06d800640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa068162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06d800640 1 -- 172.21.15.28:0/4266250145 >> v1:172.21.15.28:6789/0 conn(0x7fa0680b69d0 legacy=0x7fa0680afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.187+0000 7fa06d800640 1 -- 172.21.15.28:0/4266250145 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06815f120 con 0x7fa0680af280 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.188+0000 7fa06d800640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680af280 0x7fa068154d00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa060009d90 tx=0x7fa06002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887f63f6a537dec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa067e00640 1 -- 172.21.15.28:0/4266250145 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060007740 con 0x7fa0680af280 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa067e00640 1 -- 172.21.15.28:0/4266250145 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0600078e0 con 0x7fa0680af280 2024-09-10T10:44:37.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa067e00640 1 -- 172.21.15.28:0/4266250145 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060007cb0 con 0x7fa0680af280 2024-09-10T10:44:37.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 -- 172.21.15.28:0/4266250145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680af280 msgr2=0x7fa068154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680af280 0x7fa068154d00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa060009d90 tx=0x7fa06002a530 comp rx=0 tx=0).stop 2024-09-10T10:44:37.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 -- 172.21.15.28:0/4266250145 shutdown_connections 2024-09-10T10:44:37.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680af280 0x7fa068154d00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 --2- 172.21.15.28:0/4266250145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa068162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.189+0000 7fa06f800640 1 -- 172.21.15.28:0/4266250145 >> 172.21.15.28:0/4266250145 conn(0x7fa06806bc10 msgr2=0x7fa06806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.190+0000 7fa06f800640 1 -- 172.21.15.28:0/4266250145 shutdown_connections 2024-09-10T10:44:37.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.190+0000 7fa06f800640 1 -- 172.21.15.28:0/4266250145 wait complete. 2024-09-10T10:44:37.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.191+0000 7fa06f800640 1 Processor -- start 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.191+0000 7fa06f800640 1 -- start start 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0680af280 0x7fa068156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa0680b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa068155240 con 0x7fa0680b69d0 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0680af6f0 con 0x7fa0680af280 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa0681553c0 con 0x7fa0680b0180 2024-09-10T10:44:37.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39478/0 (socket says 172.21.15.28:39478) 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06d800640 1 -- 172.21.15.28:0/2288516811 learned_addr learned my addr 172.21.15.28:0/2288516811 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.192+0000 7fa06ce00640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0680af280 0x7fa068156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa05fe00640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa0680b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 -- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 msgr2=0x7fa0680b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa0680b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 -- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0680af280 msgr2=0x7fa068156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0680af280 0x7fa068156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 -- 172.21.15.28:0/2288516811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0680d0700 con 0x7fa0680b69d0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa06d800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa060009eb0 tx=0x7fa060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.193+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060040070 con 0x7fa0680b69d0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.194+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060031c40 con 0x7fa0680b69d0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.194+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0680ce5e0 con 0x7fa0680b69d0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.194+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060009450 con 0x7fa0680b69d0 2024-09-10T10:44:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.194+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa068156fd0 con 0x7fa0680b69d0 2024-09-10T10:44:37.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.195+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0680ceee0 con 0x7fa0680b69d0 2024-09-10T10:44:37.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.196+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa060039040 con 0x7fa0680b69d0 2024-09-10T10:44:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.197+0000 7fa065600640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 0x7fa04406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.197+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fa0600abeb0 con 0x7fa0680b69d0 2024-09-10T10:44:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.197+0000 7fa06ce00640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 0x7fa04406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.198+0000 7fa06ce00640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 0x7fa04406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa058007fd0 tx=0x7fa058007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.201+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa060079f30 con 0x7fa0680b69d0 2024-09-10T10:44:37.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.388+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa06815dec0 con 0x7fa0680b69d0 2024-09-10T10:44:37.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.390+0000 7fa065600640 1 -- 172.21.15.28:0/2288516811 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18721 (secure 0 0 0) 0x7fa06007e230 con 0x7fa0680b69d0 2024-09-10T10:44:37.392 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:37.392 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":29,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:27.552353+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:43:45.223771+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T10:43:45.928275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T10:43:49.581202+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3f","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 msgr2=0x7fa04406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 0x7fa04406e2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa058007fd0 tx=0x7fa058007de0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 msgr2=0x7fa068152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa060009eb0 tx=0x7fa060004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 shutdown_connections 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa04406be30 0x7fa04406e2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0680b69d0 0x7fa068152120 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0680b0180 0x7fa0680b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 --2- 172.21.15.28:0/2288516811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0680af280 0x7fa068156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 >> 172.21.15.28:0/2288516811 conn(0x7fa06806bc10 msgr2=0x7fa0680b08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.392+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 shutdown_connections 2024-09-10T10:44:37.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.393+0000 7fa06f800640 1 -- 172.21.15.28:0/2288516811 wait complete. 2024-09-10T10:44:37.410 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:44:37.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.566+0000 7f8e6b600640 1 Processor -- start 2024-09-10T10:44:37.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.566+0000 7f8e6b600640 1 -- start start 2024-09-10T10:44:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e6b600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e64162060 0x7f8e640b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e6b600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e6406d9f0 con 0x7f8e64161c80 2024-09-10T10:44:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e6b600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e6406dcd0 con 0x7f8e640b64b0 2024-09-10T10:44:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e6b600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8e6406dfb0 con 0x7f8e64162060 2024-09-10T10:44:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e63e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e64162060 0x7f8e640b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e68c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8e64161c80 0x7f8e6406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50376/0 (socket says 172.21.15.28:50376) 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e63e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e64162060 0x7f8e640b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39482/0 (socket says 172.21.15.28:39482) 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.567+0000 7f8e68c00640 1 -- 172.21.15.28:0/1306421473 learned_addr learned my addr 172.21.15.28:0/1306421473 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.568+0000 7f8e63e00640 1 -- 172.21.15.28:0/1306421473 >> v1:172.21.15.138:6789/0 conn(0x7f8e640b64b0 legacy=0x7f8e640cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.568+0000 7f8e63e00640 1 -- 172.21.15.28:0/1306421473 >> v1:172.21.15.28:6789/0 conn(0x7f8e64161c80 legacy=0x7f8e6406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.568+0000 7f8e63e00640 1 -- 172.21.15.28:0/1306421473 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e640d2890 con 0x7f8e64162060 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e63e00640 1 --2- 172.21.15.28:0/1306421473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64162060 0x7f8e640b5f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e500097a0 tx=0x7f8e5002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d927f5b83e800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e63400640 1 -- 172.21.15.28:0/1306421473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50037070 con 0x7f8e64162060 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e63400640 1 -- 172.21.15.28:0/1306421473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e50032440 con 0x7f8e64162060 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e63400640 1 -- 172.21.15.28:0/1306421473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50032810 con 0x7f8e64162060 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e6b600640 1 -- 172.21.15.28:0/1306421473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64162060 msgr2=0x7f8e640b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.569+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1306421473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64162060 0x7f8e640b5f70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e500097a0 tx=0x7f8e5002a230 comp rx=0 tx=0).stop 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.570+0000 7f8e6b600640 1 -- 172.21.15.28:0/1306421473 shutdown_connections 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.570+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1306421473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64162060 0x7f8e640b5f70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.570+0000 7f8e6b600640 1 -- 172.21.15.28:0/1306421473 >> 172.21.15.28:0/1306421473 conn(0x7f8e640d44f0 msgr2=0x7f8e640d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.570+0000 7f8e6b600640 1 -- 172.21.15.28:0/1306421473 shutdown_connections 2024-09-10T10:44:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.570+0000 7f8e6b600640 1 -- 172.21.15.28:0/1306421473 wait complete. 2024-09-10T10:44:37.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.571+0000 7f8e6b600640 1 Processor -- start 2024-09-10T10:44:37.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.571+0000 7f8e6b600640 1 -- start start 2024-09-10T10:44:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e640b64b0 0x7f8e6414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 0x7f8e640cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640b6d30 con 0x7f8e64161c80 2024-09-10T10:44:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e640b6bb0 con 0x7f8e640b64b0 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.572+0000 7f8e6b600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e640cf8f0 con 0x7f8e64162060 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39492/0 (socket says 172.21.15.28:39492) 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 -- 172.21.15.28:0/1111246044 learned_addr learned my addr 172.21.15.28:0/1111246044 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e68c00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e640b64b0 0x7f8e6414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e69600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 0x7f8e640cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 -- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 msgr2=0x7f8e640cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 0x7f8e640cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 -- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e640b64b0 msgr2=0x7f8e6414ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e640b64b0 0x7f8e6414ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 -- 172.21.15.28:0/1111246044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e6414f710 con 0x7f8e64161c80 2024-09-10T10:44:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e69600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 0x7f8e640cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:37.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.573+0000 7f8e63e00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e50035040 tx=0x7f8e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.574+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50037040 con 0x7f8e64161c80 2024-09-10T10:44:37.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.574+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e50030770 con 0x7f8e64161c80 2024-09-10T10:44:37.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.574+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50030b40 con 0x7f8e64161c80 2024-09-10T10:44:37.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.574+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e640cc110 con 0x7f8e64161c80 2024-09-10T10:44:37.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.574+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e640b74c0 con 0x7f8e64161c80 2024-09-10T10:44:37.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.575+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e640b5f70 con 0x7f8e64161c80 2024-09-10T10:44:37.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.576+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e50044020 con 0x7f8e64161c80 2024-09-10T10:44:37.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.577+0000 7f8e60c00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 0x7f8e4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.577+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f8e500ab6d0 con 0x7f8e64161c80 2024-09-10T10:44:37.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.577+0000 7f8e68c00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 0x7f8e4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.578+0000 7f8e68c00640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 0x7f8e4406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005c00 tx=0x7f8e58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.581+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e50032e20 con 0x7f8e64161c80 2024-09-10T10:44:37.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.804+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8e640ba470 con 0x7f8e64161c80 2024-09-10T10:44:37.806 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:44:37.804+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:44:37.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.806+0000 7f8e60c00640 1 -- 172.21.15.28:0/1111246044 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f8e50079d00 con 0x7f8e64161c80 2024-09-10T10:44:37.809 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:44:37.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.809+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 msgr2=0x7f8e4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.809+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 0x7f8e4406e2f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8e58005c00 tx=0x7f8e58009040 comp rx=0 tx=0).stop 2024-09-10T10:44:37.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.809+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 msgr2=0x7f8e640cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.809+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e50035040 tx=0x7f8e500046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 shutdown_connections 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e4406be30 0x7f8e4406e2f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e64162060 0x7f8e640cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e64161c80 0x7f8e640cb5c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 --2- 172.21.15.28:0/1111246044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e640b64b0 0x7f8e6414ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 >> 172.21.15.28:0/1111246044 conn(0x7f8e640d44f0 msgr2=0x7f8e640d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 shutdown_connections 2024-09-10T10:44:37.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.810+0000 7f8e6b600640 1 -- 172.21.15.28:0/1111246044 wait complete. 2024-09-10T10:44:37.820 INFO:tasks.ceph.mds.c.smithi028.stderr: -4557> 2024-09-10T10:43:52.731+0000 7fb88b400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:44:37.830 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:44:37.872 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:44:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.988+0000 7f04e1800640 1 Processor -- start 2024-09-10T10:44:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.989+0000 7f04e1800640 1 -- start start 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04e1800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f04dc0b4800 0x7f04dc0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04e1800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 0x7f04dc0d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04e1800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04dc0af7a0 con 0x7f04dc15c310 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04e1800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f04dc0afa80 con 0x7f04dc0b4800 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04e1800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc160130 con 0x7f04dc0d0370 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04d3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f04dc0b4800 0x7f04dc0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04d3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f04dc0b4800 0x7f04dc0aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39500/0 (socket says 172.21.15.28:39500) 2024-09-10T10:44:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04daa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f04dc15c310 0x7f04dc160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50388/0 (socket says 172.21.15.28:50388) 2024-09-10T10:44:37.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04d3e00640 1 -- 172.21.15.28:0/3872461937 learned_addr learned my addr 172.21.15.28:0/3872461937 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04db400640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 0x7f04dc0d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04d3e00640 1 -- 172.21.15.28:0/3872461937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 msgr2=0x7f04dc0d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.990+0000 7f04d3e00640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 0x7f04dc0d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.991+0000 7f04d3e00640 1 -- 172.21.15.28:0/3872461937 >> v1:172.21.15.28:6789/0 conn(0x7f04dc15c310 legacy=0x7f04dc160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.991+0000 7f04d3e00640 1 -- 172.21.15.28:0/3872461937 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc0af470 con 0x7f04dc0b4800 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.991+0000 7f04db400640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 0x7f04dc0d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:37.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04d3e00640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc0b4800 0x7f04dc0aef30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009870 tx=0x7f04c402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48e8b96c2a5b71c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:37.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04da000640 1 -- 172.21.15.28:0/3872461937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4037070 con 0x7f04dc0b4800 2024-09-10T10:44:37.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04da000640 1 -- 172.21.15.28:0/3872461937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c4032440 con 0x7f04dc0b4800 2024-09-10T10:44:37.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04da000640 1 -- 172.21.15.28:0/3872461937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4032810 con 0x7f04dc0b4800 2024-09-10T10:44:37.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 -- 172.21.15.28:0/3872461937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc0b4800 msgr2=0x7f04dc0aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc0b4800 0x7f04dc0aef30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009870 tx=0x7f04c402a250 comp rx=0 tx=0).stop 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 -- 172.21.15.28:0/3872461937 shutdown_connections 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0d0370 0x7f04dc0d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 --2- 172.21.15.28:0/3872461937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc0b4800 0x7f04dc0aef30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.992+0000 7f04e1800640 1 -- 172.21.15.28:0/3872461937 >> 172.21.15.28:0/3872461937 conn(0x7f04dc06bdf0 msgr2=0x7f04dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.993+0000 7f04e1800640 1 -- 172.21.15.28:0/3872461937 shutdown_connections 2024-09-10T10:44:37.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.993+0000 7f04e1800640 1 -- 172.21.15.28:0/3872461937 wait complete. 2024-09-10T10:44:37.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.994+0000 7f04e1800640 1 Processor -- start 2024-09-10T10:44:37.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.994+0000 7f04e1800640 1 -- start start 2024-09-10T10:44:37.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0b4800 0x7f04dc14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04dc0d0370 0x7f04dc09b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06e090 con 0x7f04dc15c310 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06df10 con 0x7f04dc0d0370 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04e1800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc06e210 con 0x7f04dc0b4800 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04db400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04db400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39502/0 (socket says 172.21.15.28:39502) 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.995+0000 7f04db400640 1 -- 172.21.15.28:0/641728420 learned_addr learned my addr 172.21.15.28:0/641728420 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04daa00640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0b4800 0x7f04dc14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04d3e00640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04dc0d0370 0x7f04dc09b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 -- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0b4800 msgr2=0x7f04dc14a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0b4800 0x7f04dc14a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 -- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04dc0d0370 msgr2=0x7f04dc09b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04dc0d0370 0x7f04dc09b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 -- 172.21.15.28:0/641728420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc09bfd0 con 0x7f04dc15c310 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04db400640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d400cef0 tx=0x7f04d400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d401a070 con 0x7f04dc15c310 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d400d860 con 0x7f04dc15c310 2024-09-10T10:44:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.996+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc09c2b0 con 0x7f04dc15c310 2024-09-10T10:44:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.997+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d400dc50 con 0x7f04dc15c310 2024-09-10T10:44:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.997+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc09cb50 con 0x7f04dc15c310 2024-09-10T10:44:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.998+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc09c6a0 con 0x7f04dc15c310 2024-09-10T10:44:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.998+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04d401d070 con 0x7f04dc15c310 2024-09-10T10:44:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.999+0000 7f04d2a00640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 0x7f04b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:37.999+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6676+0+0 (secure 0 0 0) 0x7f04d408bb00 con 0x7f04dc15c310 2024-09-10T10:44:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.000+0000 7f04daa00640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 0x7f04b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.001+0000 7f04daa00640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 0x7f04b806e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04dc1524d0 tx=0x7f04cc007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.003+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f04d4059ab0 con 0x7f04dc15c310 2024-09-10T10:44:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.231+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04dc0aef30 con 0x7f04dc15c310 2024-09-10T10:44:38.574 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:44:38.579 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:44:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.811+0000 7f04d2a00640 1 -- 172.21.15.28:0/641728420 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f04d405ddb0 con 0x7f04dc15c310 2024-09-10T10:44:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.813+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 msgr2=0x7f04b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.813+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 0x7f04b806e2f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04dc1524d0 tx=0x7f04cc007e80 comp rx=0 tx=0).stop 2024-09-10T10:44:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 msgr2=0x7f04dc1f79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f04d400cef0 tx=0x7f04d400a710 comp rx=0 tx=0).stop 2024-09-10T10:44:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 shutdown_connections 2024-09-10T10:44:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b806be30 0x7f04b806e2f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f04dc15c310 0x7f04dc1f79f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04dc0d0370 0x7f04dc09b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 --2- 172.21.15.28:0/641728420 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f04dc0b4800 0x7f04dc14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 >> 172.21.15.28:0/641728420 conn(0x7f04dc06bdf0 msgr2=0x7f04dc0738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 shutdown_connections 2024-09-10T10:44:38.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.814+0000 7f04e1800640 1 -- 172.21.15.28:0/641728420 wait complete. 2024-09-10T10:44:38.833 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:44:38.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.993+0000 7f203fc00640 1 Processor -- start 2024-09-10T10:44:38.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.993+0000 7f203fc00640 1 -- start start 2024-09-10T10:44:38.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.994+0000 7f203fc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203fc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f20380cfcb0 0x7f2038168600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:38.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203fc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20380d2790 con 0x7f20380cf0f0 2024-09-10T10:44:38.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203fc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f20380b3e40 con 0x7f20380cfcb0 2024-09-10T10:44:38.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203fc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f20380d2a30 con 0x7f203815e650 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f20380cfcb0 0x7f2038168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:51080/0 (socket says 172.21.15.28:51080) 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.995+0000 7f203d200640 1 -- 172.21.15.28:0/1085158541 learned_addr learned my addr 172.21.15.28:0/1085158541 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.996+0000 7f203d200640 1 -- 172.21.15.28:0/1085158541 >> v2:172.21.15.138:3300/0 conn(0x7f20380cfcb0 msgr2=0x7f2038168600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.996+0000 7f203d200640 1 --2- 172.21.15.28:0/1085158541 >> v2:172.21.15.138:3300/0 conn(0x7f20380cfcb0 0x7f2038168600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.996+0000 7f203d200640 1 -- 172.21.15.28:0/1085158541 >> v1:172.21.15.138:6789/0 conn(0x7f20380cf0f0 legacy=0x7f20380cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:38.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.996+0000 7f203d200640 1 -- 172.21.15.28:0/1085158541 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380d2d10 con 0x7f203815e650 2024-09-10T10:44:38.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.997+0000 7f203d200640 1 --2- 172.21.15.28:0/1085158541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f202c009870 tx=0x7f202c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=569ebaeec38bb589 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:38.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.997+0000 7f2037400640 1 -- 172.21.15.28:0/1085158541 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c037070 con 0x7f203815e650 2024-09-10T10:44:38.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.997+0000 7f2037400640 1 -- 172.21.15.28:0/1085158541 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c032440 con 0x7f203815e650 2024-09-10T10:44:38.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.997+0000 7f2037400640 1 -- 172.21.15.28:0/1085158541 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c032810 con 0x7f203815e650 2024-09-10T10:44:39.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 -- 172.21.15.28:0/1085158541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 msgr2=0x7f20380cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 --2- 172.21.15.28:0/1085158541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f202c009870 tx=0x7f202c02a250 comp rx=0 tx=0).stop 2024-09-10T10:44:39.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 -- 172.21.15.28:0/1085158541 shutdown_connections 2024-09-10T10:44:39.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 --2- 172.21.15.28:0/1085158541 >> v2:172.21.15.138:3300/0 conn(0x7f20380cfcb0 0x7f2038168600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 --2- 172.21.15.28:0/1085158541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f20380cebb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 -- 172.21.15.28:0/1085158541 >> 172.21.15.28:0/1085158541 conn(0x7f203808bfc0 msgr2=0x7f20380d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 -- 172.21.15.28:0/1085158541 shutdown_connections 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:38.998+0000 7f203fc00640 1 -- 172.21.15.28:0/1085158541 wait complete. 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.000+0000 7f203fc00640 1 Processor -- start 2024-09-10T10:44:39.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.000+0000 7f203fc00640 1 -- start start 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20380cf0f0 0x7f20380b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f203815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f20380d5040 con 0x7f20380cfcb0 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20380d32d0 con 0x7f20380cf0f0 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203fc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f20380d51c0 con 0x7f203815e650 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39512/0 (socket says 172.21.15.28:39512) 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 -- 172.21.15.28:0/391116886 learned_addr learned my addr 172.21.15.28:0/391116886 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203d200640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20380cf0f0 0x7f20380b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203dc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f203815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 -- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 msgr2=0x7f203815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f203815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 -- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20380cf0f0 msgr2=0x7f20380b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20380cf0f0 0x7f20380b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f2037e00640 1 -- 172.21.15.28:0/391116886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20380b7d00 con 0x7f20380cfcb0 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.001+0000 7f203dc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f203815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f2037e00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f202400c440 tx=0x7f2024001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20240052d0 con 0x7f20380cfcb0 2024-09-10T10:44:39.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2024005470 con 0x7f20380cfcb0 2024-09-10T10:44:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2024007260 con 0x7f20380cfcb0 2024-09-10T10:44:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20380b7fe0 con 0x7f20380cfcb0 2024-09-10T10:44:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.002+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20380b82c0 con 0x7f20380cfcb0 2024-09-10T10:44:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.003+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203815ea30 con 0x7f20380cfcb0 2024-09-10T10:44:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.004+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2024007400 con 0x7f20380cfcb0 2024-09-10T10:44:39.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.005+0000 7f2034c00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:39.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.005+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6676+0+0 (secure 0 0 0) 0x7f202408a6b0 con 0x7f20380cfcb0 2024-09-10T10:44:39.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.005+0000 7f203d200640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:39.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.006+0000 7f203d200640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 0x7f201806e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f202c034000 tx=0x7f202c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:39.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.013+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2024058630 con 0x7f20380cfcb0 2024-09-10T10:44:39.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.197+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20380b6c50 con 0x7f20380cfcb0 2024-09-10T10:44:39.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.813+0000 7f2034c00640 1 -- 172.21.15.28:0/391116886 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f202405c930 con 0x7f20380cfcb0 2024-09-10T10:44:39.815 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:44:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.816+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 msgr2=0x7f201806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.816+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 0x7f201806e2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f202c034000 tx=0x7f202c02e000 comp rx=0 tx=0).stop 2024-09-10T10:44:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.816+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 msgr2=0x7f20380b7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.816+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f202400c440 tx=0x7f2024001e70 comp rx=0 tx=0).stop 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 shutdown_connections 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f201806be30 0x7f201806e2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f203815e650 0x7f203815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20380cfcb0 0x7f20380b7390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 --2- 172.21.15.28:0/391116886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20380cf0f0 0x7f20380b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:39.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 >> 172.21.15.28:0/391116886 conn(0x7f203808bfc0 msgr2=0x7f20380b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:39.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.817+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 shutdown_connections 2024-09-10T10:44:39.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.818+0000 7f203fc00640 1 -- 172.21.15.28:0/391116886 wait complete. 2024-09-10T10:44:39.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:44:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.997+0000 7fc19de00640 1 Processor -- start 2024-09-10T10:44:40.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.998+0000 7fc19de00640 1 -- start start 2024-09-10T10:44:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc19de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc1980d0510 0x7fc19815f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc19de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc19815f850 0x7fc19806cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc19de00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1980d0bd0 con 0x7fc19809aa30 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc19de00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc1980d17e0 con 0x7fc1980d0510 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc19de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc1980d0980 con 0x7fc19815f850 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc18fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc1980d0510 0x7fc19815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc18fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc1980d0510 0x7fc19815f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39522/0 (socket says 172.21.15.28:39522) 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc18fe00640 1 -- 172.21.15.28:0/2882772151 learned_addr learned my addr 172.21.15.28:0/2882772151 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:39.999+0000 7fc197400640 1 --2- 172.21.15.28:0/2882772151 >> v2:172.21.15.138:3300/0 conn(0x7fc19815f850 0x7fc19806cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.000+0000 7fc18fe00640 1 -- 172.21.15.28:0/2882772151 >> v2:172.21.15.138:3300/0 conn(0x7fc19815f850 msgr2=0x7fc19806cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.000+0000 7fc18fe00640 1 --2- 172.21.15.28:0/2882772151 >> v2:172.21.15.138:3300/0 conn(0x7fc19815f850 0x7fc19806cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.000+0000 7fc18fe00640 1 -- 172.21.15.28:0/2882772151 >> v1:172.21.15.28:6789/0 conn(0x7fc19809aa30 legacy=0x7fc19809ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.000+0000 7fc18fe00640 1 -- 172.21.15.28:0/2882772151 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980d0eb0 con 0x7fc1980d0510 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc18fe00640 1 --2- 172.21.15.28:0/2882772151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d0510 0x7fc19815f310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc19806ea50 tx=0x7fc18002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8065fdaf4e79042 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc196000640 1 -- 172.21.15.28:0/2882772151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180037070 con 0x7fc1980d0510 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc196000640 1 -- 172.21.15.28:0/2882772151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc18002ae50 con 0x7fc1980d0510 2024-09-10T10:44:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc196000640 1 -- 172.21.15.28:0/2882772151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180032860 con 0x7fc1980d0510 2024-09-10T10:44:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc19de00640 1 -- 172.21.15.28:0/2882772151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d0510 msgr2=0x7fc19815f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.001+0000 7fc19de00640 1 --2- 172.21.15.28:0/2882772151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d0510 0x7fc19815f310 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc19806ea50 tx=0x7fc18002a300 comp rx=0 tx=0).stop 2024-09-10T10:44:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 -- 172.21.15.28:0/2882772151 shutdown_connections 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 --2- 172.21.15.28:0/2882772151 >> v2:172.21.15.138:3300/0 conn(0x7fc19815f850 0x7fc19806cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 --2- 172.21.15.28:0/2882772151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d0510 0x7fc19815f310 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 -- 172.21.15.28:0/2882772151 >> 172.21.15.28:0/2882772151 conn(0x7fc1980d2380 msgr2=0x7fc1980d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 -- 172.21.15.28:0/2882772151 shutdown_connections 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.002+0000 7fc19de00640 1 -- 172.21.15.28:0/2882772151 wait complete. 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.003+0000 7fc19de00640 1 Processor -- start 2024-09-10T10:44:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.003+0000 7fc19de00640 1 -- start start 2024-09-10T10:44:40.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc19809aa30 0x7fc1980c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 0x7fc198162180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc19806d460 con 0x7fc19815f850 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980d1470 con 0x7fc19809aa30 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.004+0000 7fc19de00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc19806d5e0 con 0x7fc1980d0510 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39538/0 (socket says 172.21.15.28:39538) 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 0x7fc198162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 -- 172.21.15.28:0/1464555344 learned_addr learned my addr 172.21.15.28:0/1464555344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc196a00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc19809aa30 0x7fc1980c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 -- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 msgr2=0x7fc198162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 0x7fc198162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 -- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc19809aa30 msgr2=0x7fc1980c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc19809aa30 0x7fc1980c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 -- 172.21.15.28:0/1464555344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1981f5150 con 0x7fc19815f850 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc18fe00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 0x7fc198162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.005+0000 7fc197400640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d30 tx=0x7fc19000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.006+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190018020 con 0x7fc19815f850 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.006+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190014480 con 0x7fc19815f850 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.006+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190014870 con 0x7fc19815f850 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.006+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1980c90f0 con 0x7fc19815f850 2024-09-10T10:44:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.006+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1980c8330 con 0x7fc19815f850 2024-09-10T10:44:40.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.007+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19815f310 con 0x7fc19815f850 2024-09-10T10:44:40.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.008+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc190005d10 con 0x7fc19815f850 2024-09-10T10:44:40.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.009+0000 7fc18f400640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 0x7fc17406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:40.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.009+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6238+0+0 (secure 0 0 0) 0x7fc19008b990 con 0x7fc19815f850 2024-09-10T10:44:40.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.010+0000 7fc196a00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 0x7fc17406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.010+0000 7fc196a00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 0x7fc17406e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc188001780 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:40.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.014+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc190022070 con 0x7fc19815f850 2024-09-10T10:44:40.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.231+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1980d3290 con 0x7fc19815f850 2024-09-10T10:44:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.818+0000 7fc18f400640 1 -- 172.21.15.28:0/1464555344 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7fc190010040 con 0x7fc19815f850 2024-09-10T10:44:40.820 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.822+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 msgr2=0x7fc17406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.822+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 0x7fc17406e320 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc188001780 tx=0x7fc188009040 comp rx=0 tx=0).stop 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.822+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 msgr2=0x7fc1981f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.822+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d30 tx=0x7fc19000c490 comp rx=0 tx=0).stop 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 shutdown_connections 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be60 0x7fc17406e320 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc19815f850 0x7fc1981f4c10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc1980d0510 0x7fc198162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 --2- 172.21.15.28:0/1464555344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc19809aa30 0x7fc1980c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:40.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 >> 172.21.15.28:0/1464555344 conn(0x7fc1980d2380 msgr2=0x7fc19809ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:40.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 shutdown_connections 2024-09-10T10:44:40.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:40.823+0000 7fc19de00640 1 -- 172.21.15.28:0/1464555344 wait complete. 2024-09-10T10:44:40.844 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T10:44:41.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.008+0000 7f443f000640 1 Processor -- start 2024-09-10T10:44:41.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.008+0000 7f443f000640 1 -- start start 2024-09-10T10:44:41.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f443f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f44380af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f443f000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443806ffb0 con 0x7f4438070230 2024-09-10T10:44:41.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f443f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4438162eb0 con 0x7f44380b5a20 2024-09-10T10:44:41.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f443f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f443815ed50 con 0x7f4438162a20 2024-09-10T10:44:41.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f4437e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4438070230 0x7f4438070610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50402/0 (socket says 172.21.15.28:50402) 2024-09-10T10:44:41.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f4437e00640 1 -- 172.21.15.28:0/3185004916 learned_addr learned my addr 172.21.15.28:0/3185004916 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:41.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f443d000640 1 --2- 172.21.15.28:0/3185004916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f44380af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.010+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194426025 0 0) 0x7f443806ffb0 con 0x7f4438070230 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f441c003520 con 0x7f4438070230 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423959988 0 0) 0x7f4438162eb0 con 0x7f44380b5a20 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f443806ffb0 con 0x7f44380b5a20 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f443d000640 1 -- 172.21.15.28:0/3185004916 >> v1:172.21.15.138:6789/0 conn(0x7f44380b5a20 legacy=0x7f44380ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f443d000640 1 -- 172.21.15.28:0/3185004916 >> v1:172.21.15.28:6789/0 conn(0x7f4438070230 legacy=0x7f4438070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.011+0000 7f443d000640 1 -- 172.21.15.28:0/3185004916 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443815f030 con 0x7f4438162a20 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f443d000640 1 --2- 172.21.15.28:0/3185004916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f44380af0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4430009dc0 tx=0x7f443002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45abc66e8a333581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430007740 con 0x7f4438162a20 2024-09-10T10:44:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44300078e0 con 0x7f4438162a20 2024-09-10T10:44:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f4437400640 1 -- 172.21.15.28:0/3185004916 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430007cb0 con 0x7f4438162a20 2024-09-10T10:44:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f443f000640 1 -- 172.21.15.28:0/3185004916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 msgr2=0x7f44380af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.012+0000 7f443f000640 1 --2- 172.21.15.28:0/3185004916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f44380af0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4430009dc0 tx=0x7f443002a6d0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.013+0000 7f443f000640 1 -- 172.21.15.28:0/3185004916 shutdown_connections 2024-09-10T10:44:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.013+0000 7f443f000640 1 --2- 172.21.15.28:0/3185004916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f44380af0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.013+0000 7f443f000640 1 -- 172.21.15.28:0/3185004916 >> 172.21.15.28:0/3185004916 conn(0x7f4438073480 msgr2=0x7f443806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.013+0000 7f443f000640 1 -- 172.21.15.28:0/3185004916 shutdown_connections 2024-09-10T10:44:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.013+0000 7f443f000640 1 -- 172.21.15.28:0/3185004916 wait complete. 2024-09-10T10:44:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.014+0000 7f443f000640 1 Processor -- start 2024-09-10T10:44:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.014+0000 7f443f000640 1 -- start start 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b5a20 0x7f4438147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f4438149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f443815ea20 con 0x7f4438070230 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f443815f5f0 con 0x7f44380b5a20 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f443f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f443815c290 con 0x7f4438162a20 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.015+0000 7f4437e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39552/0 (socket says 172.21.15.28:39552) 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 -- 172.21.15.28:0/361787067 learned_addr learned my addr 172.21.15.28:0/361787067 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f442fe00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b5a20 0x7f4438147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f443d000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f4438149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 -- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 msgr2=0x7f4438149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f4438149710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 -- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b5a20 msgr2=0x7f4438147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b5a20 0x7f4438147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 -- 172.21.15.28:0/361787067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4438148670 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.016+0000 7f4437e00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f442800c500 tx=0x7f442800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.017+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428017070 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.017+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4428005ce0 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.017+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280072f0 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.017+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4438148950 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.017+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4438148f10 con 0x7f4438070230 2024-09-10T10:44:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.018+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4438070610 con 0x7f4438070230 2024-09-10T10:44:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.019+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f442801a070 con 0x7f4438070230 2024-09-10T10:44:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.019+0000 7f4434c00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 0x7f441406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.020+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7f442808a1f0 con 0x7f4438070230 2024-09-10T10:44:41.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.020+0000 7f442fe00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 0x7f441406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:41.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.021+0000 7f442fe00640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 0x7f441406e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4420007fd0 tx=0x7f4420009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.024+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4428058620 con 0x7f4438070230 2024-09-10T10:44:41.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.208+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f443806d1d0 con 0x7f4438070230 2024-09-10T10:44:41.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.824+0000 7f4434c00640 1 -- 172.21.15.28:0/361787067 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f442805c920 con 0x7f4438070230 2024-09-10T10:44:41.826 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data_ec' removed 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 msgr2=0x7f441406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 0x7f441406e340 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4420007fd0 tx=0x7f4420009040 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 msgr2=0x7f4438147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f442800c500 tx=0x7f442800c9c0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 shutdown_connections 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441406be80 0x7f441406e340 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4438162a20 0x7f4438149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44380b5a20 0x7f4438147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 --2- 172.21.15.28:0/361787067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4438070230 0x7f4438147fe0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.828+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 >> 172.21.15.28:0/361787067 conn(0x7f4438073480 msgr2=0x7f44380b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.833+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 shutdown_connections 2024-09-10T10:44:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:41.833+0000 7f443f000640 1 -- 172.21.15.28:0/361787067 wait complete. 2024-09-10T10:44:41.860 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:44:41.860 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:44:41.860 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:44:41.860 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:44:41.860 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:44:41.860 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:44:41.861 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:44:41.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:44:41.861 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:44:41.861 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:44:41.861 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:44:41.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:44:41.861 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:44:41.862 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:44:41.862 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:44:41.862 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:44:41.877 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:44:41.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:44:41.878 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:44:41.878 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:44:41.879 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:44:41.880 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:44:41.922 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:44:41.922 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:41.924 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:44:41.924 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:41.924 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:44:41.925 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:42.056 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:44:42.061 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:44:42.061 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:44:45.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-10T10:44:47.880 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:44:47.880 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:44:47.885 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:44:47.886 DEBUG:teuthology.parallel:result is None 2024-09-10T10:44:47.886 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:44:48.025 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:44:48.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.053+0000 7fc19d000640 1 Processor -- start 2024-09-10T10:44:48.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.053+0000 7fc19d000640 1 -- start start 2024-09-10T10:44:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc19d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc1980d1f90 0x7fc1980cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc19d000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1980cf000 con 0x7fc19815e660 2024-09-10T10:44:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc19d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1980d29a0 con 0x7fc1980d0dd0 2024-09-10T10:44:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc19d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc19806d3e0 con 0x7fc1980d1f90 2024-09-10T10:44:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc18fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc19815e660 0x7fc19806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33086/0 (socket says 172.21.15.28:33086) 2024-09-10T10:44:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc18fe00640 1 -- 172.21.15.28:0/4001782211 learned_addr learned my addr 172.21.15.28:0/4001782211 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.054+0000 7fc197e00640 1 --2- 172.21.15.28:0/4001782211 >> v2:172.21.15.138:3300/0 conn(0x7fc1980d1f90 0x7fc1980cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.055+0000 7fc196a00640 1 -- 172.21.15.28:0/4001782211 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3344771092 0 0) 0x7fc1980cf000 con 0x7fc19815e660 2024-09-10T10:44:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.055+0000 7fc196a00640 1 -- 172.21.15.28:0/4001782211 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc17c003520 con 0x7fc19815e660 2024-09-10T10:44:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.055+0000 7fc197e00640 1 -- 172.21.15.28:0/4001782211 >> v1:172.21.15.138:6789/0 conn(0x7fc1980d0dd0 legacy=0x7fc1980d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.055+0000 7fc197e00640 1 -- 172.21.15.28:0/4001782211 >> v1:172.21.15.28:6789/0 conn(0x7fc19815e660 legacy=0x7fc19806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.055+0000 7fc197e00640 1 -- 172.21.15.28:0/4001782211 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980d2c60 con 0x7fc1980d1f90 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.056+0000 7fc197e00640 1 --2- 172.21.15.28:0/4001782211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d1f90 0x7fc1980cf7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7ab05723b535aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.056+0000 7fc196a00640 1 -- 172.21.15.28:0/4001782211 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190007740 con 0x7fc1980d1f90 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.056+0000 7fc196a00640 1 -- 172.21.15.28:0/4001782211 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1900078e0 con 0x7fc1980d1f90 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.056+0000 7fc196a00640 1 -- 172.21.15.28:0/4001782211 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190007cb0 con 0x7fc1980d1f90 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 -- 172.21.15.28:0/4001782211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d1f90 msgr2=0x7fc1980cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 --2- 172.21.15.28:0/4001782211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d1f90 0x7fc1980cf7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc190009d90 tx=0x7fc19002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 -- 172.21.15.28:0/4001782211 shutdown_connections 2024-09-10T10:44:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 --2- 172.21.15.28:0/4001782211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d1f90 0x7fc1980cf7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 -- 172.21.15.28:0/4001782211 >> 172.21.15.28:0/4001782211 conn(0x7fc19808bfc0 msgr2=0x7fc1980d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 -- 172.21.15.28:0/4001782211 shutdown_connections 2024-09-10T10:44:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.057+0000 7fc19d000640 1 -- 172.21.15.28:0/4001782211 wait complete. 2024-09-10T10:44:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.059+0000 7fc19d000640 1 Processor -- start 2024-09-10T10:44:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.059+0000 7fc19d000640 1 -- start start 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d0dd0 0x7fc1980b8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc19815e660 0x7fc1981503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980cf330 con 0x7fc1980d1f90 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980d3220 con 0x7fc1980d0dd0 2024-09-10T10:44:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc19d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980d33a0 con 0x7fc19815e660 2024-09-10T10:44:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc18fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51486/0 (socket says 172.21.15.28:51486) 2024-09-10T10:44:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc18fe00640 1 -- 172.21.15.28:0/1305176541 learned_addr learned my addr 172.21.15.28:0/1305176541 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc197400640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d0dd0 0x7fc1980b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.060+0000 7fc197e00640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc19815e660 0x7fc1981503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 -- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc19815e660 msgr2=0x7fc1981503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc19815e660 0x7fc1981503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 -- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d0dd0 msgr2=0x7fc1980b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d0dd0 0x7fc1980b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 -- 172.21.15.28:0/1305176541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980b7390 con 0x7fc1980d1f90 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18fe00640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c410 tx=0x7fc18000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180017070 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc180005ce0 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800072d0 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1980b7670 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.061+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1980b7c30 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.062+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1980d24d0 con 0x7fc1980d1f90 2024-09-10T10:44:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.063+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc18001a070 con 0x7fc1980d1f90 2024-09-10T10:44:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.064+0000 7fc18f400640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 0x7fc17406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.064+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5335+0+0 (secure 0 0 0) 0x7fc18008a0b0 con 0x7fc1980d1f90 2024-09-10T10:44:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.064+0000 7fc197400640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 0x7fc17406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.065+0000 7fc197400640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 0x7fc17406e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc188001780 tx=0x7fc188009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.068+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc180058450 con 0x7fc1980d1f90 2024-09-10T10:44:48.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:48.264+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc1980d1b20 con 0x7fc1980d1f90 2024-09-10T10:44:49.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.555+0000 7fc18f400640 1 -- 172.21.15.28:0/1305176541 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7fc18005c750 con 0x7fc1980d1f90 2024-09-10T10:44:49.556 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:44:49.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 msgr2=0x7fc17406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 0x7fc17406e2f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc188001780 tx=0x7fc188009040 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 msgr2=0x7fc1980b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c410 tx=0x7fc18000c8d0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 shutdown_connections 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc17406be30 0x7fc17406e2f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc19815e660 0x7fc1981503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1980d1f90 0x7fc1980b4270 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 --2- 172.21.15.28:0/1305176541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1980d0dd0 0x7fc1980b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 >> 172.21.15.28:0/1305176541 conn(0x7fc19808bfc0 msgr2=0x7fc19806ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 shutdown_connections 2024-09-10T10:44:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.558+0000 7fc19d000640 1 -- 172.21.15.28:0/1305176541 wait complete. 2024-09-10T10:44:49.579 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:44:49.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.738+0000 7f5fff600640 1 Processor -- start 2024-09-10T10:44:49.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.738+0000 7f5fff600640 1 -- start start 2024-09-10T10:44:49.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.739+0000 7f5fff600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80b69d0 0x7f5ff8162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.739+0000 7f5fff600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ff80b0180 0x7f5ff8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.739+0000 7f5fff600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff815eb60 con 0x7f5ff80af280 2024-09-10T10:44:49.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.739+0000 7f5fff600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5ff80b6db0 con 0x7f5ff80b0180 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.739+0000 7f5fff600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff815ee40 con 0x7f5ff80b69d0 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ff80b0180 0x7f5ff8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ff80b0180 0x7f5ff8154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51500/0 (socket says 172.21.15.28:51500) 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 -- 172.21.15.28:0/357453806 learned_addr learned my addr 172.21.15.28:0/357453806 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffcc00640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80b69d0 0x7f5ff8162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 -- 172.21.15.28:0/357453806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80b69d0 msgr2=0x7f5ff8162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80b69d0 0x7f5ff8162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 -- 172.21.15.28:0/357453806 >> v1:172.21.15.138:6789/0 conn(0x7f5ff80af280 legacy=0x7f5ff80afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.740+0000 7f5ffd600640 1 -- 172.21.15.28:0/357453806 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff815f120 con 0x7f5ff80b0180 2024-09-10T10:44:49.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.741+0000 7f5ffd600640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff8154d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009e80 tx=0x7f5ff002a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a026171dca7929 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.741+0000 7f5ff7400640 1 -- 172.21.15.28:0/357453806 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0036070 con 0x7f5ff80b0180 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.741+0000 7f5ff7400640 1 -- 172.21.15.28:0/357453806 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff0031400 con 0x7f5ff80b0180 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5ff7400640 1 -- 172.21.15.28:0/357453806 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00317d0 con 0x7f5ff80b0180 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 -- 172.21.15.28:0/357453806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 msgr2=0x7f5ff8154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff8154d00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009e80 tx=0x7f5ff002a620 comp rx=0 tx=0).stop 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 -- 172.21.15.28:0/357453806 shutdown_connections 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff8154d00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 --2- 172.21.15.28:0/357453806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80b69d0 0x7f5ff8162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 -- 172.21.15.28:0/357453806 >> 172.21.15.28:0/357453806 conn(0x7f5ff806bc10 msgr2=0x7f5ff806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 -- 172.21.15.28:0/357453806 shutdown_connections 2024-09-10T10:44:49.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.742+0000 7f5fff600640 1 -- 172.21.15.28:0/357453806 wait complete. 2024-09-10T10:44:49.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.744+0000 7f5fff600640 1 Processor -- start 2024-09-10T10:44:49.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.744+0000 7f5fff600640 1 -- start start 2024-09-10T10:44:49.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80af280 0x7f5ff81545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ff80b69d0 0x7f5ff81577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8155240 con 0x7f5ff80b0180 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff80af6e0 con 0x7f5ff80b69d0 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5fff600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff81553c0 con 0x7f5ff80af280 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5ff7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5ff7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51512/0 (socket says 172.21.15.28:51512) 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5ff7e00640 1 -- 172.21.15.28:0/3118195658 learned_addr learned my addr 172.21.15.28:0/3118195658 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5ffcc00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80af280 0x7f5ff81545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.745+0000 7f5ffd600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ff80b69d0 0x7f5ff81577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 -- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80af280 msgr2=0x7f5ff81545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80af280 0x7f5ff81545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 -- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ff80b69d0 msgr2=0x7f5ff81577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:49.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ff80b69d0 0x7f5ff81577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 -- 172.21.15.28:0/3118195658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff81e9250 con 0x7f5ff80b0180 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff7e00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c440 tx=0x7f5fe4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40052d0 con 0x7f5ff80b0180 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe4005470 con 0x7f5ff80b0180 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff81e9530 con 0x7f5ff80b0180 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007260 con 0x7f5ff80b0180 2024-09-10T10:44:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.746+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81e9af0 con 0x7f5ff80b0180 2024-09-10T10:44:49.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.747+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff80ad8a0 con 0x7f5ff80b0180 2024-09-10T10:44:49.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.748+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5fe4007400 con 0x7f5ff80b0180 2024-09-10T10:44:49.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.749+0000 7f5ff4c00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:49.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.749+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f5fe408a200 con 0x7f5ff80b0180 2024-09-10T10:44:49.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.750+0000 7f5ffcc00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:49.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.751+0000 7f5ffcc00640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 0x7f5fd806e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5fec006fd0 tx=0x7f5fec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:49.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:49.754+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ff80ad8a0 con 0x7f5ff80b0180 2024-09-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.004+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5ff806d2c0 con 0x7f5ff80b0180 2024-09-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.005+0000 7f5ff4c00640 1 -- 172.21.15.28:0/3118195658 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f5fe4058d60 con 0x7f5ff80b0180 2024-09-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:50.006 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.007 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.008 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:44:50.009 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:44:50.010 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:44:50.011 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.012 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.013 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:44:50.014 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.015 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.016 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:44:50.017 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.018 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.019 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:44:50.020 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mds", 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBmQ0nJJBAA5sPKO+mm5/vmcyNaRAXY/A==", 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.021 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBm81TJJBAAdH2njEqNgTxrhB96NlX0kA==", 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-osd", 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBm413JJBAAY1dhYF5C/DnV1XgJJYBjVA==", 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T10:44:50.022 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBm8GbJJBAAjZ3hySVqJfWsHhR0VVafUQ==", 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBmOG/JJBAA4+NQzODcwMhEcJg9IG+cMw==", 2024-09-10T10:44:50.023 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDJIuBm1HvJJBAAm7ZZ7vdeTI9+1/jkSE+jlg==", 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:44:50.024 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:44:50.025 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:44:50.026 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.026 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:50.026 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:44:50.026 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:44:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 msgr2=0x7f5fd806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 0x7f5fd806e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5fec006fd0 tx=0x7f5fec008040 comp rx=0 tx=0).stop 2024-09-10T10:44:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 msgr2=0x7f5ff81e8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c440 tx=0x7f5fe4001e70 comp rx=0 tx=0).stop 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 shutdown_connections 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5fd806be30 0x7f5fd806e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ff80b69d0 0x7f5ff81577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff80b0180 0x7f5ff81e8870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.007+0000 7f5fff600640 1 --2- 172.21.15.28:0/3118195658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ff80af280 0x7f5ff81545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.008+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 >> 172.21.15.28:0/3118195658 conn(0x7f5ff806bc10 msgr2=0x7f5ff80b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.008+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 shutdown_connections 2024-09-10T10:44:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.008+0000 7f5fff600640 1 -- 172.21.15.28:0/3118195658 wait complete. 2024-09-10T10:44:50.029 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T10:44:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.187+0000 7fd663000640 1 Processor -- start 2024-09-10T10:44:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.187+0000 7fd663000640 1 -- start start 2024-09-10T10:44:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd663000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd65c162060 0x7fd65c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd663000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 0x7fd65c0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd663000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65c06d9f0 con 0x7fd65c161c80 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd663000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd65c06dcd0 con 0x7fd65c162060 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd663000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c06dfb0 con 0x7fd65c0b64b0 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.188+0000 7fd65b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd65c162060 0x7fd65c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd65c162060 0x7fd65c0b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51514/0 (socket says 172.21.15.28:51514) 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 -- 172.21.15.28:0/1576304261 learned_addr learned my addr 172.21.15.28:0/1576304261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd661000640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 0x7fd65c0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 -- 172.21.15.28:0/1576304261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 msgr2=0x7fd65c0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 0x7fd65c0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 -- 172.21.15.28:0/1576304261 >> v1:172.21.15.138:6789/0 conn(0x7fd65c161c80 legacy=0x7fd65c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd65b400640 1 -- 172.21.15.28:0/1576304261 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c0d01f0 con 0x7fd65c162060 2024-09-10T10:44:50.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.189+0000 7fd661000640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 0x7fd65c0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:50.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.190+0000 7fd65b400640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c162060 0x7fd65c0b5f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd648009870 tx=0x7fd64802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145c1fcd86e7b070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.190+0000 7fd65aa00640 1 -- 172.21.15.28:0/1576304261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648037070 con 0x7fd65c162060 2024-09-10T10:44:50.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.190+0000 7fd65aa00640 1 -- 172.21.15.28:0/1576304261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd648032440 con 0x7fd65c162060 2024-09-10T10:44:50.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.190+0000 7fd65aa00640 1 -- 172.21.15.28:0/1576304261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648032810 con 0x7fd65c162060 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 -- 172.21.15.28:0/1576304261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c162060 msgr2=0x7fd65c0b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c162060 0x7fd65c0b5f70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd648009870 tx=0x7fd64802a270 comp rx=0 tx=0).stop 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 -- 172.21.15.28:0/1576304261 shutdown_connections 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c0b64b0 0x7fd65c0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 --2- 172.21.15.28:0/1576304261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c162060 0x7fd65c0b5f70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 -- 172.21.15.28:0/1576304261 >> 172.21.15.28:0/1576304261 conn(0x7fd65c0d44f0 msgr2=0x7fd65c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 -- 172.21.15.28:0/1576304261 shutdown_connections 2024-09-10T10:44:50.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.191+0000 7fd663000640 1 -- 172.21.15.28:0/1576304261 wait complete. 2024-09-10T10:44:50.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.193+0000 7fd663000640 1 Processor -- start 2024-09-10T10:44:50.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.193+0000 7fd663000640 1 -- start start 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 0x7fd65c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 0x7fd65c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf360 con 0x7fd65c0b64b0 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf1e0 con 0x7fd65c162060 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd663000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0cf4e0 con 0x7fd65c161c80 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd65be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd65be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51522/0 (socket says 172.21.15.28:51522) 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.194+0000 7fd65be00640 1 -- 172.21.15.28:0/1556131712 learned_addr learned my addr 172.21.15.28:0/1556131712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:50.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd661000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 0x7fd65c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65b400640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 0x7fd65c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 -- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 msgr2=0x7fd65c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 0x7fd65c0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 -- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 msgr2=0x7fd65c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 0x7fd65c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 -- 172.21.15.28:0/1556131712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c1f79c0 con 0x7fd65c0b64b0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65b400640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 0x7fd65c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd661000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 0x7fd65c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd65be00640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c400 tx=0x7fd64c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.195+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c0052d0 con 0x7fd65c0b64b0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.196+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd64c005490 con 0x7fd65c0b64b0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.196+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c1f4820 con 0x7fd65c0b64b0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.196+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c007260 con 0x7fd65c0b64b0 2024-09-10T10:44:50.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.196+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c1f4de0 con 0x7fd65c0b64b0 2024-09-10T10:44:50.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.197+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c0ba470 con 0x7fd65c0b64b0 2024-09-10T10:44:50.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.198+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd64c007400 con 0x7fd65c0b64b0 2024-09-10T10:44:50.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.198+0000 7fd653e00640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 0x7fd63c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.199+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7fd64c08a280 con 0x7fd65c0b64b0 2024-09-10T10:44:50.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.199+0000 7fd65b400640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 0x7fd63c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.200+0000 7fd65b400640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 0x7fd63c06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd648034000 tx=0x7fd64802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.203+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd64c058d50 con 0x7fd65c0b64b0 2024-09-10T10:44:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.456+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd65c0d2c80 con 0x7fd65c0b64b0 2024-09-10T10:44:50.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.458+0000 7fd653e00640 1 -- 172.21.15.28:0/1556131712 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fd64c058650 con 0x7fd65c0b64b0 2024-09-10T10:44:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 msgr2=0x7fd63c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 0x7fd63c06e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd648034000 tx=0x7fd64802e000 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 msgr2=0x7fd65c1f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00c400 tx=0x7fd64c00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 shutdown_connections 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd63c06be30 0x7fd63c06e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd65c162060 0x7fd65c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd65c161c80 0x7fd65c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 --2- 172.21.15.28:0/1556131712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd65c0b64b0 0x7fd65c1f7480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 >> 172.21.15.28:0/1556131712 conn(0x7fd65c0d44f0 msgr2=0x7fd65c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.460+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 shutdown_connections 2024-09-10T10:44:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.461+0000 7fd663000640 1 -- 172.21.15.28:0/1556131712 wait complete. 2024-09-10T10:44:50.477 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T10:44:50.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.634+0000 7fe704400640 1 Processor -- start 2024-09-10T10:44:50.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.634+0000 7fe704400640 1 -- start start 2024-09-10T10:44:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe704400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 0x7fe6fc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe704400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe6fc06f8d0 0x7fe6fc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe704400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6fc0cdbe0 con 0x7fe6fc0cd3f0 2024-09-10T10:44:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe704400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe6fc0728a0 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe704400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc0cd900 con 0x7fe6fc15b300 2024-09-10T10:44:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe702400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe6fc06f8d0 0x7fe6fc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe701000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe6fc0cd3f0 0x7fe6fc06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33094/0 (socket says 172.21.15.28:33094) 2024-09-10T10:44:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.635+0000 7fe701000640 1 -- 172.21.15.28:0/4034526348 learned_addr learned my addr 172.21.15.28:0/4034526348 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe701a00640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 0x7fe6fc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe702400640 1 -- 172.21.15.28:0/4034526348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 msgr2=0x7fe6fc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe702400640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 0x7fe6fc0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe702400640 1 -- 172.21.15.28:0/4034526348 >> v1:172.21.15.28:6789/0 conn(0x7fe6fc0cd3f0 legacy=0x7fe6fc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe702400640 1 -- 172.21.15.28:0/4034526348 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc072250 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe6fbe00640 1 -- 172.21.15.28:0/4034526348 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893898836 0 0) 0x7fe6fc0cdbe0 con 0x7fe6fc0cd3f0 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.636+0000 7fe701a00640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 0x7fe6fc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.637+0000 7fe702400640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc15b6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009d90 tx=0x7fe6f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f45b04b935df6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.637+0000 7fe6fbe00640 1 -- 172.21.15.28:0/4034526348 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007740 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.637+0000 7fe6fbe00640 1 -- 172.21.15.28:0/4034526348 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f40078e0 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.637+0000 7fe6fbe00640 1 -- 172.21.15.28:0/4034526348 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007cb0 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 -- 172.21.15.28:0/4034526348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc06f8d0 msgr2=0x7fe6fc15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc15b6e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009d90 tx=0x7fe6f402a660 comp rx=0 tx=0).stop 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 -- 172.21.15.28:0/4034526348 shutdown_connections 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc15b6e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 --2- 172.21.15.28:0/4034526348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc15b300 0x7fe6fc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 -- 172.21.15.28:0/4034526348 >> 172.21.15.28:0/4034526348 conn(0x7fe6fc067d70 msgr2=0x7fe6fc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 -- 172.21.15.28:0/4034526348 shutdown_connections 2024-09-10T10:44:50.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.638+0000 7fe704400640 1 -- 172.21.15.28:0/4034526348 wait complete. 2024-09-10T10:44:50.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.640+0000 7fe704400640 1 Processor -- start 2024-09-10T10:44:50.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.640+0000 7fe704400640 1 -- start start 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc0cd3f0 0x7fe6fc14a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0d0160 con 0x7fe6fc15b300 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc0cffe0 con 0x7fe6fc06f8d0 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe704400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6fc072d10 con 0x7fe6fc0cd3f0 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51550/0 (socket says 172.21.15.28:51550) 2024-09-10T10:44:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 -- 172.21.15.28:0/3856136530 learned_addr learned my addr 172.21.15.28:0/3856136530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:50.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe701a00640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe701000640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc0cd3f0 0x7fe6fc14a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 -- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc0cd3f0 msgr2=0x7fe6fc14a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc0cd3f0 0x7fe6fc14a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 -- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6fc06f8d0 msgr2=0x7fe6fc14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc14e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.641+0000 7fe702400640 1 -- 172.21.15.28:0/3856136530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc158250 con 0x7fe6fc15b300 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe702400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009eb0 tx=0x7fe6f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4040070 con 0x7fe6fc15b300 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4031c40 con 0x7fe6fc15b300 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4003990 con 0x7fe6fc15b300 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc0b2fb0 con 0x7fe6fc15b300 2024-09-10T10:44:50.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.642+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc0b2cd0 con 0x7fe6fc15b300 2024-09-10T10:44:50.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.643+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc152150 con 0x7fe6fc15b300 2024-09-10T10:44:50.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.644+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6f4039040 con 0x7fe6fc15b300 2024-09-10T10:44:50.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.645+0000 7fe6f9600640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 0x7fe6dc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:50.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.645+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7fe6f40abf20 con 0x7fe6fc15b300 2024-09-10T10:44:50.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.645+0000 7fe701a00640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 0x7fe6dc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:50.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.646+0000 7fe701a00640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 0x7fe6dc06e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000aac0 tx=0x7fe6f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:50.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.649+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe6f407a430 con 0x7fe6fc15b300 2024-09-10T10:44:50.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.900+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fe6fc1549f0 con 0x7fe6fc15b300 2024-09-10T10:44:50.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.902+0000 7fe6f9600640 1 -- 172.21.15.28:0/3856136530 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fe6f407e730 con 0x7fe6fc15b300 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 msgr2=0x7fe6dc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 0x7fe6dc06e320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000aac0 tx=0x7fe6f0008040 comp rx=0 tx=0).stop 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 msgr2=0x7fe6fc155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009eb0 tx=0x7fe6f4004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 shutdown_connections 2024-09-10T10:44:50.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6dc06be60 0x7fe6dc06e320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6fc15b300 0x7fe6fc155e70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6fc0cd3f0 0x7fe6fc14a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 --2- 172.21.15.28:0/3856136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6fc06f8d0 0x7fe6fc14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 >> 172.21.15.28:0/3856136530 conn(0x7fe6fc067d70 msgr2=0x7fe6fc154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 shutdown_connections 2024-09-10T10:44:50.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:50.905+0000 7fe704400640 1 -- 172.21.15.28:0/3856136530 wait complete. 2024-09-10T10:44:50.922 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T10:44:51.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.079+0000 7f8d91200640 1 Processor -- start 2024-09-10T10:44:51.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.079+0000 7f8d91200640 1 -- start start 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.080+0000 7f8d91200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d8c161c80 0x7f8d8c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.080+0000 7f8d91200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 0x7f8d8c0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.080+0000 7f8d91200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d8c06e870 con 0x7f8d8c162060 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.080+0000 7f8d91200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8d8c06d9f0 con 0x7f8d8c161c80 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.080+0000 7f8d91200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8c06dcd0 con 0x7f8d8c0b6680 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8d8c162060 0x7f8d8c0b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33096/0 (socket says 172.21.15.28:33096) 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89400640 1 -- 172.21.15.28:0/2420474095 learned_addr learned my addr 172.21.15.28:0/2420474095 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89e00640 1 --2- 172.21.15.28:0/2420474095 >> v2:172.21.15.138:3300/0 conn(0x7f8d8c161c80 0x7f8d8c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d8a800640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 0x7f8d8c0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d88a00640 1 -- 172.21.15.28:0/2420474095 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906824833 0 0) 0x7f8d78001110 con 0x7f8d8c162060 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d88a00640 1 -- 172.21.15.28:0/2420474095 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d70003520 con 0x7f8d8c162060 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89e00640 1 -- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 msgr2=0x7f8d8c0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89e00640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 0x7f8d8c0cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89e00640 1 -- 172.21.15.28:0/2420474095 >> v1:172.21.15.28:6789/0 conn(0x7f8d8c162060 legacy=0x7f8d8c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d89e00640 1 -- 172.21.15.28:0/2420474095 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c06dfb0 con 0x7f8d8c161c80 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.081+0000 7f8d8a800640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 0x7f8d8c0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.082+0000 7f8d89e00640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c161c80 0x7f8d8c06e330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0097a0 tx=0x7f8d7c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b038e1b00c203f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d88a00640 1 -- 172.21.15.28:0/2420474095 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c037070 con 0x7f8d8c161c80 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d88a00640 1 -- 172.21.15.28:0/2420474095 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d7c032440 con 0x7f8d8c161c80 2024-09-10T10:44:51.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d88a00640 1 -- 172.21.15.28:0/2420474095 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c032810 con 0x7f8d8c161c80 2024-09-10T10:44:51.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 -- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c161c80 msgr2=0x7f8d8c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c161c80 0x7f8d8c06e330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0097a0 tx=0x7f8d7c02a180 comp rx=0 tx=0).stop 2024-09-10T10:44:51.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 -- 172.21.15.28:0/2420474095 shutdown_connections 2024-09-10T10:44:51.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c0b6680 0x7f8d8c0cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 --2- 172.21.15.28:0/2420474095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c161c80 0x7f8d8c06e330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 -- 172.21.15.28:0/2420474095 >> 172.21.15.28:0/2420474095 conn(0x7f8d8c0d44f0 msgr2=0x7f8d8c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 -- 172.21.15.28:0/2420474095 shutdown_connections 2024-09-10T10:44:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.083+0000 7f8d91200640 1 -- 172.21.15.28:0/2420474095 wait complete. 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.085+0000 7f8d91200640 1 Processor -- start 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.085+0000 7f8d91200640 1 -- start start 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 0x7f8d8c1f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c162060 0x7f8d8c158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0b6cb0 con 0x7f8d8c161c80 2024-09-10T10:44:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c0b6b30 con 0x7f8d8c0b6680 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d91200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8c0cf720 con 0x7f8d8c162060 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d89400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d89e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 0x7f8d8c1f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d89400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51552/0 (socket says 172.21.15.28:51552) 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d89400640 1 -- 172.21.15.28:0/322961143 learned_addr learned my addr 172.21.15.28:0/322961143 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.086+0000 7f8d8a800640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c162060 0x7f8d8c158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 -- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c162060 msgr2=0x7f8d8c158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c162060 0x7f8d8c158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 -- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 msgr2=0x7f8d8c1f86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 0x7f8d8c1f86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 -- 172.21.15.28:0/322961143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c1f8c00 con 0x7f8d8c161c80 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89e00640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 0x7f8d8c1f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d89400640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c3c0 tx=0x7f8d7800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78017070 con 0x7f8d8c161c80 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d78005ce0 con 0x7f8d8c161c80 2024-09-10T10:44:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d780072f0 con 0x7f8d8c161c80 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8c1f4bb0 con 0x7f8d8c161c80 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.087+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8c1f5170 con 0x7f8d8c161c80 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.088+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c0cb680 con 0x7f8d8c161c80 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.089+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d7801a070 con 0x7f8d8c161c80 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.090+0000 7f8d82000640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.090+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f8d78089fd0 con 0x7f8d8c161c80 2024-09-10T10:44:51.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.091+0000 7f8d89e00640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.091+0000 7f8d89e00640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c034000 tx=0x7f8d7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.094+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d8c0cb680 con 0x7f8d8c161c80 2024-09-10T10:44:51.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.348+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f8d8c0d2f20 con 0x7f8d8c161c80 2024-09-10T10:44:51.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.350+0000 7f8d82000640 1 -- 172.21.15.28:0/322961143 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f8d8c06e870 con 0x7f8d8c161c80 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.352+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 msgr2=0x7f8d6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c034000 tx=0x7f8d7c02e000 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 msgr2=0x7f8d8c0b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d7800c3c0 tx=0x7f8d7800c7c0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 shutdown_connections 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d6c06be30 0x7f8d6c06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8c162060 0x7f8d8c158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d8c161c80 0x7f8d8c0b34d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 --2- 172.21.15.28:0/322961143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8c0b6680 0x7f8d8c1f86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 >> 172.21.15.28:0/322961143 conn(0x7f8d8c0d44f0 msgr2=0x7f8d8c0b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:51.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 shutdown_connections 2024-09-10T10:44:51.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.353+0000 7f8d91200640 1 -- 172.21.15.28:0/322961143 wait complete. 2024-09-10T10:44:51.370 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T10:44:51.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.526+0000 7f6ca6200640 1 Processor -- start 2024-09-10T10:44:51.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.526+0000 7f6ca6200640 1 -- start start 2024-09-10T10:44:51.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6ca6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 0x7f6ca006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6ca6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 0x7f6ca00d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6ca6200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca00d2700 con 0x7f6ca015e760 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6ca6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca00cedb0 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6ca6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca00cf090 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6c9f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6ca015e760 0x7f6ca00cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33108/0 (socket says 172.21.15.28:33108) 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.527+0000 7f6c9f400640 1 -- 172.21.15.28:0/327490495 learned_addr learned my addr 172.21.15.28:0/327490495 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/327490495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 0x7f6ca00d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9ea00640 1 --2- 172.21.15.28:0/327490495 >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 0x7f6ca006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9e000640 1 -- 172.21.15.28:0/327490495 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1696785674 0 0) 0x7f6c90001110 con 0x7f6ca015e760 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9e000640 1 -- 172.21.15.28:0/327490495 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c84003520 con 0x7f6ca015e760 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9fe00640 1 -- 172.21.15.28:0/327490495 >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 msgr2=0x7f6ca006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/327490495 >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 0x7f6ca006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9fe00640 1 -- 172.21.15.28:0/327490495 >> v1:172.21.15.28:6789/0 conn(0x7f6ca015e760 legacy=0x7f6ca00cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9fe00640 1 -- 172.21.15.28:0/327490495 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca00b3c80 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.528+0000 7f6c9ea00640 1 --2- 172.21.15.28:0/327490495 >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 0x7f6ca006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:51.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.529+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/327490495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 0x7f6ca00d2ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009d90 tx=0x7f6c9802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a47812bafc53754 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6c9e000640 1 -- 172.21.15.28:0/327490495 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98007740 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6c9e000640 1 -- 172.21.15.28:0/327490495 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c980078e0 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6c9e000640 1 -- 172.21.15.28:0/327490495 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98007cb0 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 -- 172.21.15.28:0/327490495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 msgr2=0x7f6ca00d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 --2- 172.21.15.28:0/327490495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 0x7f6ca00d2ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009d90 tx=0x7f6c9802a660 comp rx=0 tx=0).stop 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 -- 172.21.15.28:0/327490495 shutdown_connections 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 --2- 172.21.15.28:0/327490495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca00d4fd0 0x7f6ca00d2ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 --2- 172.21.15.28:0/327490495 >> v2:172.21.15.138:3300/0 conn(0x7f6ca00cfcc0 0x7f6ca006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 -- 172.21.15.28:0/327490495 >> 172.21.15.28:0/327490495 conn(0x7f6ca008bfc0 msgr2=0x7f6ca00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 -- 172.21.15.28:0/327490495 shutdown_connections 2024-09-10T10:44:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.530+0000 7f6ca6200640 1 -- 172.21.15.28:0/327490495 wait complete. 2024-09-10T10:44:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.532+0000 7f6ca6200640 1 Processor -- start 2024-09-10T10:44:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.532+0000 7f6ca6200640 1 -- start start 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca00d4fd0 0x7f6ca0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca015e760 0x7f6ca00b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00d33e0 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00b38e0 con 0x7f6ca00d4fd0 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6ca6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca00d3560 con 0x7f6ca015e760 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6c9f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6c9f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51568/0 (socket says 172.21.15.28:51568) 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.533+0000 7f6c9f400640 1 -- 172.21.15.28:0/1028806325 learned_addr learned my addr 172.21.15.28:0/1028806325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9ea00640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca00d4fd0 0x7f6ca0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca015e760 0x7f6ca00b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 -- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca015e760 msgr2=0x7f6ca00b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca015e760 0x7f6ca00b7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 -- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca00d4fd0 msgr2=0x7f6ca0155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca00d4fd0 0x7f6ca0155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 -- 172.21.15.28:0/1028806325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca015ae50 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c9f400640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c3d0 tx=0x7f6c9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90017070 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c90005ce0 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90007310 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.534+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca015b130 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.535+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca00b8370 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.536+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca00d3930 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.536+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c9001a070 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.537+0000 7f6c97400640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 0x7f6c8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.537+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f6c90089ff0 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.538+0000 7f6c9ea00640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 0x7f6c8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.538+0000 7f6c9ea00640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 0x7f6c8006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009500 tx=0x7f6c8c009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.541+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ca00d2700 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.794+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f6ca00b6d20 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.797+0000 7f6c97400640 1 -- 172.21.15.28:0/1028806325 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f6c90058540 con 0x7f6ca00cfcc0 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 msgr2=0x7f6c8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 0x7f6c8006e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009500 tx=0x7f6c8c009110 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 msgr2=0x7f6ca01f8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c3d0 tx=0x7f6c9000c890 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 shutdown_connections 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c8006be60 0x7f6c8006e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ca015e760 0x7f6ca00b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ca00d4fd0 0x7f6ca0155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 --2- 172.21.15.28:0/1028806325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00cfcc0 0x7f6ca01f8250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 >> 172.21.15.28:0/1028806325 conn(0x7f6ca008bfc0 msgr2=0x7f6ca0156700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:51.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 shutdown_connections 2024-09-10T10:44:51.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.799+0000 7f6ca6200640 1 -- 172.21.15.28:0/1028806325 wait complete. 2024-09-10T10:44:51.817 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T10:44:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.973+0000 7fa88ec00640 1 Processor -- start 2024-09-10T10:44:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.973+0000 7fa88ec00640 1 -- start start 2024-09-10T10:44:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.974+0000 7fa88ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8880d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.974+0000 7fa88ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa88815e660 0x7fa88806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.974+0000 7fa88ec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8880cf000 con 0x7fa8880d1f90 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.974+0000 7fa88ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa8880b37d0 con 0x7fa88815e660 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.974+0000 7fa88ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8880cf330 con 0x7fa8880d0dd0 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88e200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa8880d1f90 0x7fa8880cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33110/0 (socket says 172.21.15.28:33110) 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88e200640 1 -- 172.21.15.28:0/4286658057 learned_addr learned my addr 172.21.15.28:0/4286658057 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88d800640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8880d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88ce00640 1 --2- 172.21.15.28:0/4286658057 >> v2:172.21.15.138:3300/0 conn(0x7fa88815e660 0x7fa88806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa887e00640 1 -- 172.21.15.28:0/4286658057 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599303612 0 0) 0x7fa8880cf000 con 0x7fa8880d1f90 2024-09-10T10:44:51.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa887e00640 1 -- 172.21.15.28:0/4286658057 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa870003520 con 0x7fa8880d1f90 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88ce00640 1 -- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 msgr2=0x7fa8880d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88ce00640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8880d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88ce00640 1 -- 172.21.15.28:0/4286658057 >> v1:172.21.15.28:6789/0 conn(0x7fa8880d1f90 legacy=0x7fa8880cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88ce00640 1 -- 172.21.15.28:0/4286658057 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8880d16f0 con 0x7fa88815e660 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.975+0000 7fa88d800640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8880d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.976+0000 7fa88ce00640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa88806ccd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8740097a0 tx=0x7fa87402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405424eba7387728 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa887e00640 1 -- 172.21.15.28:0/4286658057 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874037070 con 0x7fa88815e660 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa887e00640 1 -- 172.21.15.28:0/4286658057 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa874032440 con 0x7fa88815e660 2024-09-10T10:44:51.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa887e00640 1 -- 172.21.15.28:0/4286658057 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874032810 con 0x7fa88815e660 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 -- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 msgr2=0x7fa88806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa88806ccd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa8740097a0 tx=0x7fa87402a180 comp rx=0 tx=0).stop 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 -- 172.21.15.28:0/4286658057 shutdown_connections 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa88806ccd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 --2- 172.21.15.28:0/4286658057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8880d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 -- 172.21.15.28:0/4286658057 >> 172.21.15.28:0/4286658057 conn(0x7fa88808bfc0 msgr2=0x7fa8880d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 -- 172.21.15.28:0/4286658057 shutdown_connections 2024-09-10T10:44:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.977+0000 7fa88ec00640 1 -- 172.21.15.28:0/4286658057 wait complete. 2024-09-10T10:44:51.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.979+0000 7fa88ec00640 1 Processor -- start 2024-09-10T10:44:51.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.979+0000 7fa88ec00640 1 -- start start 2024-09-10T10:44:51.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8881f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa888159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880d2f40 con 0x7fa8880d1f90 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880d2c60 con 0x7fa88815e660 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8880d3220 con 0x7fa8880d0dd0 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51574/0 (socket says 172.21.15.28:51574) 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88ce00640 1 -- 172.21.15.28:0/546701099 learned_addr learned my addr 172.21.15.28:0/546701099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88d800640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8881f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.980+0000 7fa88e200640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa888159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 -- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 msgr2=0x7fa8881f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8881f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 -- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 msgr2=0x7fa888159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa888159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 -- 172.21.15.28:0/546701099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa888159f80 con 0x7fa8880d1f90 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88e200640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa888159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:51.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ce00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa874035040 tx=0x7fa8740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874042020 con 0x7fa8880d1f90 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87403d750 con 0x7fa8880d1f90 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874037040 con 0x7fa8880d1f90 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88815a260 con 0x7fa8880d1f90 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.981+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88815a820 con 0x7fa8880d1f90 2024-09-10T10:44:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.982+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8880b27e0 con 0x7fa8880d1f90 2024-09-10T10:44:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.983+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa874045020 con 0x7fa8880d1f90 2024-09-10T10:44:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.984+0000 7fa885600640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 0x7fa86806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.984+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7fa8740ab770 con 0x7fa8880d1f90 2024-09-10T10:44:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.985+0000 7fa88d800640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 0x7fa86806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.985+0000 7fa88d800640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 0x7fa86806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa8880a2360 tx=0x7fa87c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:51.989+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa8880b27e0 con 0x7fa8880d1f90 2024-09-10T10:44:52.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-10T10:44:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.242+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fa8880d2440 con 0x7fa8880d1f90 2024-09-10T10:44:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.245+0000 7fa885600640 1 -- 172.21.15.28:0/546701099 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fa87404c030 con 0x7fa8880d1f90 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.247+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 msgr2=0x7fa86806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.247+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 0x7fa86806e320 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa8880a2360 tx=0x7fa87c009190 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.247+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 msgr2=0x7fa8881f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.247+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa874035040 tx=0x7fa8740046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.247+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 shutdown_connections 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa86806be60 0x7fa86806e320 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa88815e660 0x7fa888159710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8880d1f90 0x7fa8881f4ad0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 --2- 172.21.15.28:0/546701099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8880d0dd0 0x7fa8881f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 >> 172.21.15.28:0/546701099 conn(0x7fa88808bfc0 msgr2=0x7fa8880b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 shutdown_connections 2024-09-10T10:44:52.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.248+0000 7fa88ec00640 1 -- 172.21.15.28:0/546701099 wait complete. 2024-09-10T10:44:52.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T10:44:52.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.422+0000 7f87c7000640 1 Processor -- start 2024-09-10T10:44:52.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.422+0000 7f87c7000640 1 -- start start 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87c00d0dd0 0x7f87c00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87c00d1f90 0x7f87c00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00b37d0 con 0x7f87c00d1f90 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f87c00cf330 con 0x7f87c00d0dd0 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f87c00cf000 con 0x7f87c015e660 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c6600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87c00d1f90 0x7f87c00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c6600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87c00d1f90 0x7f87c00cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51578/0 (socket says 172.21.15.28:51578) 2024-09-10T10:44:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.423+0000 7f87c6600640 1 -- 172.21.15.28:0/2450387944 learned_addr learned my addr 172.21.15.28:0/2450387944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:52.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c5200640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c5c00640 1 --2- 172.21.15.28:0/2450387944 >> v2:172.21.15.138:3300/0 conn(0x7f87c00d0dd0 0x7f87c00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c6600640 1 -- 172.21.15.28:0/2450387944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 msgr2=0x7f87c006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:52.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c6600640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c6600640 1 -- 172.21.15.28:0/2450387944 >> v2:172.21.15.138:3300/0 conn(0x7f87c00d0dd0 msgr2=0x7f87c00d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:52.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c6600640 1 --2- 172.21.15.28:0/2450387944 >> v2:172.21.15.138:3300/0 conn(0x7f87c00d0dd0 0x7f87c00d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.424+0000 7f87c6600640 1 -- 172.21.15.28:0/2450387944 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c00d16f0 con 0x7f87c00d1f90 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.425+0000 7f87c6600640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d1f90 0x7f87c00cf7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b6fb0 tx=0x7f87b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b876bdb4169d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.425+0000 7f87bfe00640 1 -- 172.21.15.28:0/2450387944 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8036070 con 0x7f87c00d1f90 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.425+0000 7f87bfe00640 1 -- 172.21.15.28:0/2450387944 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b8031600 con 0x7f87c00d1f90 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.425+0000 7f87bfe00640 1 -- 172.21.15.28:0/2450387944 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80319d0 con 0x7f87c00d1f90 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 -- 172.21.15.28:0/2450387944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d1f90 msgr2=0x7f87c00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d1f90 0x7f87c00cf7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87c00b6fb0 tx=0x7f87b802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 -- 172.21.15.28:0/2450387944 shutdown_connections 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d1f90 0x7f87c00cf7c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 --2- 172.21.15.28:0/2450387944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 --2- 172.21.15.28:0/2450387944 >> v2:172.21.15.138:3300/0 conn(0x7f87c00d0dd0 0x7f87c00d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 -- 172.21.15.28:0/2450387944 >> 172.21.15.28:0/2450387944 conn(0x7f87c008bfc0 msgr2=0x7f87c00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 -- 172.21.15.28:0/2450387944 shutdown_connections 2024-09-10T10:44:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.426+0000 7f87c7000640 1 -- 172.21.15.28:0/2450387944 wait complete. 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.427+0000 7f87c7000640 1 Processor -- start 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 -- start start 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87c00d1f90 0x7f87c01e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c01e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f87c00d2f40 con 0x7f87c00d0dd0 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87c00d2c60 con 0x7f87c00d1f90 2024-09-10T10:44:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.428+0000 7f87c7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f87c00d3220 con 0x7f87c015e660 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51594/0 (socket says 172.21.15.28:51594) 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 -- 172.21.15.28:0/257179146 learned_addr learned my addr 172.21.15.28:0/257179146 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5200640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87c00d1f90 0x7f87c01e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c6600640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c01e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 -- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 msgr2=0x7f87c01e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c01e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 -- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87c00d1f90 msgr2=0x7f87c01e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87c00d1f90 0x7f87c01e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 -- 172.21.15.28:0/257179146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c01e3a60 con 0x7f87c00d0dd0 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.429+0000 7f87c5c00640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d800 tx=0x7f87b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.430+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4018020 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.430+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87b40096a0 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.430+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4009a70 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.430+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c01e62b0 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.430+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c01483f0 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.431+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c01481e0 con 0x7f87c00d0dd0 2024-09-10T10:44:52.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.432+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87b401b020 con 0x7f87c00d0dd0 2024-09-10T10:44:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.433+0000 7f87bd600640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 0x7f87a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.433+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f87b408acf0 con 0x7f87c00d0dd0 2024-09-10T10:44:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.433+0000 7f87c5200640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 0x7f87a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.434+0000 7f87c5200640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 0x7f87a006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87c00d54e0 tx=0x7f87ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.437+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87b408f070 con 0x7f87c00d0dd0 2024-09-10T10:44:52.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.687+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f87c00d2440 con 0x7f87c00d0dd0 2024-09-10T10:44:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.690+0000 7f87bd600640 1 -- 172.21.15.28:0/257179146 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f87b4022030 con 0x7f87c00d0dd0 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 msgr2=0x7f87a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 0x7f87a006e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f87c00d54e0 tx=0x7f87ac008040 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 msgr2=0x7f87c014a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f87b400d800 tx=0x7f87b400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 shutdown_connections 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87a006be30 0x7f87a006e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87c015e660 0x7f87c01e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87c00d1f90 0x7f87c01e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 --2- 172.21.15.28:0/257179146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87c00d0dd0 0x7f87c014a680 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 >> 172.21.15.28:0/257179146 conn(0x7f87c008bfc0 msgr2=0x7f87c0157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 shutdown_connections 2024-09-10T10:44:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.692+0000 7f87c7000640 1 -- 172.21.15.28:0/257179146 wait complete. 2024-09-10T10:44:52.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:44:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.864+0000 7f3846c00640 1 Processor -- start 2024-09-10T10:44:52.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.864+0000 7f3846c00640 1 -- start start 2024-09-10T10:44:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3846c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38400727c0 0x7f3840072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3846c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38400b44a0 con 0x7f384015a9b0 2024-09-10T10:44:52.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3846c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38400b3ee0 con 0x7f384015ad90 2024-09-10T10:44:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3846c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f38400b41c0 con 0x7f38400727c0 2024-09-10T10:44:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3844c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38400727c0 0x7f3840072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f383fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f384015a9b0 0x7f38400b34d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33126/0 (socket says 172.21.15.28:33126) 2024-09-10T10:44:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f383fe00640 1 -- 172.21.15.28:0/2461331018 learned_addr learned my addr 172.21.15.28:0/2461331018 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:52.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.865+0000 7f3844c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38400727c0 0x7f3840072bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51600/0 (socket says 172.21.15.28:51600) 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.866+0000 7f3844c00640 1 -- 172.21.15.28:0/2461331018 >> v1:172.21.15.138:6789/0 conn(0x7f384015ad90 legacy=0x7f38400720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.866+0000 7f3844c00640 1 -- 172.21.15.28:0/2461331018 >> v1:172.21.15.28:6789/0 conn(0x7f384015a9b0 legacy=0x7f38400b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.866+0000 7f3844c00640 1 -- 172.21.15.28:0/2461331018 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38400cea50 con 0x7f38400727c0 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.867+0000 7f3844c00640 1 --2- 172.21.15.28:0/2461331018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38400727c0 0x7f3840072bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f383802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520760197c527d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.867+0000 7f383ea00640 1 -- 172.21.15.28:0/2461331018 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838007740 con 0x7f38400727c0 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.867+0000 7f383ea00640 1 -- 172.21.15.28:0/2461331018 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38380078e0 con 0x7f38400727c0 2024-09-10T10:44:52.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.867+0000 7f383ea00640 1 -- 172.21.15.28:0/2461331018 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838007cb0 con 0x7f38400727c0 2024-09-10T10:44:52.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 -- 172.21.15.28:0/2461331018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38400727c0 msgr2=0x7f3840072bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 --2- 172.21.15.28:0/2461331018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38400727c0 0x7f3840072bf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d90 tx=0x7f383802a660 comp rx=0 tx=0).stop 2024-09-10T10:44:52.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 -- 172.21.15.28:0/2461331018 shutdown_connections 2024-09-10T10:44:52.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 --2- 172.21.15.28:0/2461331018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38400727c0 0x7f3840072bf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 -- 172.21.15.28:0/2461331018 >> 172.21.15.28:0/2461331018 conn(0x7f3840067d70 msgr2=0x7f384006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:52.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 -- 172.21.15.28:0/2461331018 shutdown_connections 2024-09-10T10:44:52.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.868+0000 7f3846c00640 1 -- 172.21.15.28:0/2461331018 wait complete. 2024-09-10T10:44:52.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.869+0000 7f3846c00640 1 Processor -- start 2024-09-10T10:44:52.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 -- start start 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38400727c0 0x7f38401f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 0x7f38401f1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3840073130 con 0x7f384015ad90 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38400b3be0 con 0x7f38400727c0 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.870+0000 7f3846c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38400732b0 con 0x7f384015a9b0 2024-09-10T10:44:52.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51608/0 (socket says 172.21.15.28:51608) 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 -- 172.21.15.28:0/1772175651 learned_addr learned my addr 172.21.15.28:0/1772175651 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f383fe00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38400727c0 0x7f38401f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f383f400640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 0x7f38401f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 -- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 msgr2=0x7f38401f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 0x7f38401f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 -- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38400727c0 msgr2=0x7f38401f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38400727c0 0x7f38401f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 -- 172.21.15.28:0/1772175651 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38401e5560 con 0x7f384015ad90 2024-09-10T10:44:52.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f383f400640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 0x7f38401f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:52.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.871+0000 7f3844c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3838009eb0 tx=0x7f3838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.872+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838040070 con 0x7f384015ad90 2024-09-10T10:44:52.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.872+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3838031c90 con 0x7f384015ad90 2024-09-10T10:44:52.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.872+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838045680 con 0x7f384015ad90 2024-09-10T10:44:52.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.872+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38401e5840 con 0x7f384015ad90 2024-09-10T10:44:52.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.872+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401e5e00 con 0x7f384015ad90 2024-09-10T10:44:52.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.873+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3840070290 con 0x7f384015ad90 2024-09-10T10:44:52.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.874+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3838039040 con 0x7f384015ad90 2024-09-10T10:44:52.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.875+0000 7f3837e00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 0x7f382006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:52.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.875+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f38380ac680 con 0x7f384015ad90 2024-09-10T10:44:52.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.875+0000 7f383fe00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 0x7f382006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:52.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.876+0000 7f383fe00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 0x7f382006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3830005880 tx=0x7f3830009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:52.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:52.879+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f383807b210 con 0x7f384015ad90 2024-09-10T10:44:53.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.064+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38401e6040 con 0x7f384015ad90 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.065+0000 7f3837e00640 1 -- 172.21.15.28:0/1772175651 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) ==== 84+0+65 (secure 0 0 0) 0x7f383807ab10 con 0x7f384015ad90 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:44:53.066 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:44:53.067 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:44:53.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 msgr2=0x7f382006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 0x7f382006e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3830005880 tx=0x7f3830009040 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 msgr2=0x7f384014f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3838009eb0 tx=0x7f3838004a40 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 shutdown_connections 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f382006be30 0x7f382006e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f384015ad90 0x7f384014f910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f384015a9b0 0x7f38401f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 --2- 172.21.15.28:0/1772175651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38400727c0 0x7f38401f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 >> 172.21.15.28:0/1772175651 conn(0x7f3840067d70 msgr2=0x7f38400b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 shutdown_connections 2024-09-10T10:44:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.067+0000 7f3846c00640 1 -- 172.21.15.28:0/1772175651 wait complete. 2024-09-10T10:44:53.084 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:44:53.084 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:44:53.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.243+0000 7ff374800640 1 Processor -- start 2024-09-10T10:44:53.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.243+0000 7ff374800640 1 -- start start 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.244+0000 7ff374800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff36c140790 0x7ff36c1655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.244+0000 7ff374800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36c140e70 con 0x7ff36c06fc10 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.244+0000 7ff374800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36c0cf570 con 0x7ff36c0d0610 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.244+0000 7ff374800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff36c0cf850 con 0x7ff36c140790 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff371e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff36c06fc10 0x7ff36c15b8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33138/0 (socket says 172.21.15.28:33138) 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff371e00640 1 -- 172.21.15.28:0/3818395331 learned_addr learned my addr 172.21.15.28:0/3818395331 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:53.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff372800640 1 --2- 172.21.15.28:0/3818395331 >> v2:172.21.15.28:3300/0 conn(0x7ff36c140790 0x7ff36c1655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff372800640 1 -- 172.21.15.28:0/3818395331 >> v1:172.21.15.138:6789/0 conn(0x7ff36c0d0610 legacy=0x7ff36c140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff372800640 1 -- 172.21.15.28:0/3818395331 >> v1:172.21.15.28:6789/0 conn(0x7ff36c06fc10 legacy=0x7ff36c15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff372800640 1 -- 172.21.15.28:0/3818395331 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c0cfb30 con 0x7ff36c140790 2024-09-10T10:44:53.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.245+0000 7ff370a00640 1 -- 172.21.15.28:0/3818395331 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239846630 0 0) 0x7ff36c140e70 con 0x7ff36c06fc10 2024-09-10T10:44:53.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.246+0000 7ff372800640 1 --2- 172.21.15.28:0/3818395331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c140790 0x7ff36c1655b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff36800aea0 tx=0x7ff36802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bac13f70e21e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.248+0000 7ff370a00640 1 -- 172.21.15.28:0/3818395331 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368038070 con 0x7ff36c140790 2024-09-10T10:44:53.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.248+0000 7ff370a00640 1 -- 172.21.15.28:0/3818395331 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff368008640 con 0x7ff36c140790 2024-09-10T10:44:53.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.248+0000 7ff370a00640 1 -- 172.21.15.28:0/3818395331 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368008a10 con 0x7ff36c140790 2024-09-10T10:44:53.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.248+0000 7ff374800640 1 -- 172.21.15.28:0/3818395331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c140790 msgr2=0x7ff36c1655b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.248+0000 7ff374800640 1 --2- 172.21.15.28:0/3818395331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c140790 0x7ff36c1655b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff36800aea0 tx=0x7ff36802b840 comp rx=0 tx=0).stop 2024-09-10T10:44:53.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.249+0000 7ff374800640 1 -- 172.21.15.28:0/3818395331 shutdown_connections 2024-09-10T10:44:53.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.249+0000 7ff374800640 1 --2- 172.21.15.28:0/3818395331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c140790 0x7ff36c1655b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.249+0000 7ff374800640 1 -- 172.21.15.28:0/3818395331 >> 172.21.15.28:0/3818395331 conn(0x7ff36c073590 msgr2=0x7ff36c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:53.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.249+0000 7ff374800640 1 -- 172.21.15.28:0/3818395331 shutdown_connections 2024-09-10T10:44:53.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.249+0000 7ff374800640 1 -- 172.21.15.28:0/3818395331 wait complete. 2024-09-10T10:44:53.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.250+0000 7ff374800640 1 Processor -- start 2024-09-10T10:44:53.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.250+0000 7ff374800640 1 -- start start 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff36c0d0610 0x7ff36c15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 0x7ff36c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c06fff0 con 0x7ff36c06fc10 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c0d00f0 con 0x7ff36c140790 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff374800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff36c070170 con 0x7ff36c0d0610 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.251+0000 7ff371e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51634/0 (socket says 172.21.15.28:51634) 2024-09-10T10:44:53.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 -- 172.21.15.28:0/555297172 learned_addr learned my addr 172.21.15.28:0/555297172 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff372800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 0x7ff36c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371400640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff36c0d0610 0x7ff36c15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 -- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff36c0d0610 msgr2=0x7ff36c15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff36c0d0610 0x7ff36c15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 -- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 msgr2=0x7ff36c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 0x7ff36c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 -- 172.21.15.28:0/555297172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c13f3e0 con 0x7ff36c06fc10 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff372800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 0x7ff36c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff371e00640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c4d0 tx=0x7ff36000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.252+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360018070 con 0x7ff36c06fc10 2024-09-10T10:44:53.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.253+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3600074e0 con 0x7ff36c06fc10 2024-09-10T10:44:53.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36c1e4870 con 0x7ff36c06fc10 2024-09-10T10:44:53.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.253+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360004a20 con 0x7ff36c06fc10 2024-09-10T10:44:53.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36c1e4e30 con 0x7ff36c06fc10 2024-09-10T10:44:53.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.254+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36c0cf0b0 con 0x7ff36c06fc10 2024-09-10T10:44:53.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.255+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff36001b070 con 0x7ff36c06fc10 2024-09-10T10:44:53.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.255+0000 7ff366000640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 0x7ff35006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:53.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.256+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7ff36008a0b0 con 0x7ff36c06fc10 2024-09-10T10:44:53.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.256+0000 7ff371400640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 0x7ff35006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:53.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.257+0000 7ff371400640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 0x7ff35006e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff36c13d8d0 tx=0x7ff35c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.260+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff36c13f3e0 con 0x7ff36c06fc10 2024-09-10T10:44:53.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:53.444+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff36c1e54e0 con 0x7ff36c06fc10 2024-09-10T10:44:54.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.249+0000 7ff366000640 1 -- 172.21.15.28:0/555297172 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) ==== 138+0+0 (secure 0 0 0) 0x7ff360058af0 con 0x7ff36c06fc10 2024-09-10T10:44:54.251 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:44:54.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 msgr2=0x7ff35006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 0x7ff35006e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff36c13d8d0 tx=0x7ff35c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:54.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 msgr2=0x7ff36c1e4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff36000c4d0 tx=0x7ff36000c990 comp rx=0 tx=0).stop 2024-09-10T10:44:54.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 shutdown_connections 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff35006be30 0x7ff35006e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff36c140790 0x7ff36c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff36c0d0610 0x7ff36c15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 --2- 172.21.15.28:0/555297172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff36c06fc10 0x7ff36c1e4290 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.253+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 >> 172.21.15.28:0/555297172 conn(0x7ff36c073590 msgr2=0x7ff36c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.254+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 shutdown_connections 2024-09-10T10:44:54.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.254+0000 7ff374800640 1 -- 172.21.15.28:0/555297172 wait complete. 2024-09-10T10:44:54.279 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:44:54.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.444+0000 7f4232000640 1 Processor -- start 2024-09-10T10:44:54.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.445+0000 7f4232000640 1 -- start start 2024-09-10T10:44:54.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4232000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f422c0cf790 0x7f422c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4232000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f422c0ce060 0x7f422c0ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4232000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f422c0b30e0 con 0x7f422c156740 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4232000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f422c0d0140 con 0x7f422c0cf790 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4232000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f422c06edc0 con 0x7f422c0ce060 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f422c0cf790 0x7f422c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f422c0cf790 0x7f422c0cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33524/0 (socket says 172.21.15.28:33524) 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 -- 172.21.15.28:0/4094141404 learned_addr learned my addr 172.21.15.28:0/4094141404 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f422be00640 1 --2- 172.21.15.28:0/4094141404 >> v2:172.21.15.138:3300/0 conn(0x7f422c0ce060 0x7f422c0ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 -- 172.21.15.28:0/4094141404 >> v2:172.21.15.138:3300/0 conn(0x7f422c0ce060 msgr2=0x7f422c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 --2- 172.21.15.28:0/4094141404 >> v2:172.21.15.138:3300/0 conn(0x7f422c0ce060 0x7f422c0ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f4223e00640 1 -- 172.21.15.28:0/4094141404 >> v1:172.21.15.28:6789/0 conn(0x7f422c156740 legacy=0x7f422c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.447+0000 7f4223e00640 1 -- 172.21.15.28:0/4094141404 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c0b33c0 con 0x7f422c0cf790 2024-09-10T10:44:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.446+0000 7f422aa00640 1 -- 172.21.15.28:0/4094141404 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2031189314 0 0) 0x7f422c0b30e0 con 0x7f422c156740 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.447+0000 7f4223e00640 1 --2- 172.21.15.28:0/4094141404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c0cfc00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b40f0 tx=0x7f421402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7a8f8ffce6ff2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f422aa00640 1 -- 172.21.15.28:0/4094141404 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214037070 con 0x7f422c0cf790 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f422aa00640 1 -- 172.21.15.28:0/4094141404 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4214032440 con 0x7f422c0cf790 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f422aa00640 1 -- 172.21.15.28:0/4094141404 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214032810 con 0x7f422c0cf790 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 -- 172.21.15.28:0/4094141404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 msgr2=0x7f422c0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 --2- 172.21.15.28:0/4094141404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c0cfc00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f422c0b40f0 tx=0x7f421402a270 comp rx=0 tx=0).stop 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 -- 172.21.15.28:0/4094141404 shutdown_connections 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 --2- 172.21.15.28:0/4094141404 >> v2:172.21.15.138:3300/0 conn(0x7f422c0ce060 0x7f422c0ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 --2- 172.21.15.28:0/4094141404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c0cfc00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 -- 172.21.15.28:0/4094141404 >> 172.21.15.28:0/4094141404 conn(0x7f422c06bdf0 msgr2=0x7f422c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.448+0000 7f4232000640 1 -- 172.21.15.28:0/4094141404 shutdown_connections 2024-09-10T10:44:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.449+0000 7f4232000640 1 -- 172.21.15.28:0/4094141404 wait complete. 2024-09-10T10:44:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.450+0000 7f4232000640 1 Processor -- start 2024-09-10T10:44:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.450+0000 7f4232000640 1 -- start start 2024-09-10T10:44:54.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 0x7f422c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422c156740 0x7f422c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f422c0b3c60 con 0x7f422c0cf790 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f422c0b3980 con 0x7f422c0ce060 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4232000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f422c0b4cc0 con 0x7f422c156740 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4223e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4223e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33528/0 (socket says 172.21.15.28:33528) 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.451+0000 7f4223e00640 1 -- 172.21.15.28:0/745962519 learned_addr learned my addr 172.21.15.28:0/745962519 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f422b400640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 0x7f422c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 -- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422c156740 msgr2=0x7f422c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422c156740 0x7f422c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 -- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 msgr2=0x7f422c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 0x7f422c1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 -- 172.21.15.28:0/745962519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c1f1f40 con 0x7f422c0cf790 2024-09-10T10:44:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f422b400640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 0x7f422c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223e00640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42140090b0 tx=0x7f42140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214037040 con 0x7f422c0cf790 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4214004030 con 0x7f422c0cf790 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214004400 con 0x7f422c0cf790 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.452+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c1589c0 con 0x7f422c0cf790 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.453+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c0afa40 con 0x7f422c0cf790 2024-09-10T10:44:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.454+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422c0b23a0 con 0x7f422c0cf790 2024-09-10T10:44:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.455+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f421403a070 con 0x7f422c0cf790 2024-09-10T10:44:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.456+0000 7f4223400640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 0x7f420806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.456+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5619+0+0 (secure 0 0 0) 0x7f42140ab4b0 con 0x7f422c0cf790 2024-09-10T10:44:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.456+0000 7f422b400640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 0x7f420806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.457+0000 7f422b400640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 0x7f420806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.461+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f422c0b23a0 con 0x7f422c0cf790 2024-09-10T10:44:54.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:54.646+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f422c0cff60 con 0x7f422c0cf790 2024-09-10T10:44:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.256+0000 7f4223400640 1 -- 172.21.15.28:0/745962519 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) ==== 170+0+0 (secure 0 0 0) 0x7f422c0cff60 con 0x7f422c0cf790 2024-09-10T10:44:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:44:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.260+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 msgr2=0x7f420806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.260+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 0x7f420806e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f421c000f60 tx=0x7f421c009040 comp rx=0 tx=0).stop 2024-09-10T10:44:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.260+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 msgr2=0x7f422c151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.260+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f42140090b0 tx=0x7f42140046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 shutdown_connections 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f420806be30 0x7f420806e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f422c156740 0x7f422c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f422c0cf790 0x7f422c151800 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 --2- 172.21.15.28:0/745962519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f422c0ce060 0x7f422c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.261+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 >> 172.21.15.28:0/745962519 conn(0x7f422c06bdf0 msgr2=0x7f422c0ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:55.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.262+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 shutdown_connections 2024-09-10T10:44:55.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.263+0000 7f4232000640 1 -- 172.21.15.28:0/745962519 wait complete. 2024-09-10T10:44:55.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:44:55.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.459+0000 7f4c8ec00640 1 Processor -- start 2024-09-10T10:44:55.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.459+0000 7f4c8ec00640 1 -- start start 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.460+0000 7f4c8ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c88156740 0x7f4c880b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.460+0000 7f4c8ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880b4af0 0x7f4c880cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.460+0000 7f4c8ec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c880b30e0 con 0x7f4c880cfcd0 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.460+0000 7f4c8ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4c8806edc0 con 0x7f4c88156740 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.460+0000 7f4c8ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c880b33c0 con 0x7f4c880b4af0 2024-09-10T10:44:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c88156740 0x7f4c880b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c88156740 0x7f4c880b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33540/0 (socket says 172.21.15.28:33540) 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 -- 172.21.15.28:0/1668016550 learned_addr learned my addr 172.21.15.28:0/1668016550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87400640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880b4af0 0x7f4c880cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 -- 172.21.15.28:0/1668016550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880b4af0 msgr2=0x7f4c880cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880b4af0 0x7f4c880cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 -- 172.21.15.28:0/1668016550 >> v1:172.21.15.138:6789/0 conn(0x7f4c880cfcd0 legacy=0x7f4c8815fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.461+0000 7f4c87e00640 1 -- 172.21.15.28:0/1668016550 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c880b36a0 con 0x7f4c88156740 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.462+0000 7f4c87e00640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c88156740 0x7f4c880b45b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009870 tx=0x7f4c7802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529f5486ea50df5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.462+0000 7f4c86a00640 1 -- 172.21.15.28:0/1668016550 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78037070 con 0x7f4c88156740 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c86a00640 1 -- 172.21.15.28:0/1668016550 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c78032440 con 0x7f4c88156740 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c86a00640 1 -- 172.21.15.28:0/1668016550 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78032810 con 0x7f4c88156740 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 -- 172.21.15.28:0/1668016550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c88156740 msgr2=0x7f4c880b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c88156740 0x7f4c880b45b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009870 tx=0x7f4c7802a250 comp rx=0 tx=0).stop 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 -- 172.21.15.28:0/1668016550 shutdown_connections 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880b4af0 0x7f4c880cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/1668016550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c88156740 0x7f4c880b45b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 -- 172.21.15.28:0/1668016550 >> 172.21.15.28:0/1668016550 conn(0x7f4c8806bdf0 msgr2=0x7f4c8806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 -- 172.21.15.28:0/1668016550 shutdown_connections 2024-09-10T10:44:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.463+0000 7f4c8ec00640 1 -- 172.21.15.28:0/1668016550 wait complete. 2024-09-10T10:44:55.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.465+0000 7f4c8ec00640 1 Processor -- start 2024-09-10T10:44:55.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.465+0000 7f4c8ec00640 1 -- start start 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880cfcd0 0x7f4c881f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c88156740 0x7f4c88072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880d0890 con 0x7f4c880b4af0 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880b3c60 con 0x7f4c880cfcd0 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c880d0a10 con 0x7f4c88156740 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33542/0 (socket says 172.21.15.28:33542) 2024-09-10T10:44:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c87e00640 1 -- 172.21.15.28:0/238205854 learned_addr learned my addr 172.21.15.28:0/238205854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c87400640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880cfcd0 0x7f4c881f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.466+0000 7f4c8cc00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c88156740 0x7f4c88072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 -- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c88156740 msgr2=0x7f4c88072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c88156740 0x7f4c88072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 -- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880cfcd0 msgr2=0x7f4c881f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880cfcd0 0x7f4c881f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 -- 172.21.15.28:0/238205854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c881f8320 con 0x7f4c880b4af0 2024-09-10T10:44:55.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c87e00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c7802a290 tx=0x7f4c780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78037040 con 0x7f4c880b4af0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c78042070 con 0x7f4c880b4af0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c88071f70 con 0x7f4c880b4af0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.467+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78030880 con 0x7f4c880b4af0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.468+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c88072530 con 0x7f4c880b4af0 2024-09-10T10:44:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.469+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c880b15d0 con 0x7f4c880b4af0 2024-09-10T10:44:55.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.470+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c78030a20 con 0x7f4c880b4af0 2024-09-10T10:44:55.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.471+0000 7f4c7fe00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:55.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.471+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6001+0+0 (secure 0 0 0) 0x7f4c780ab620 con 0x7f4c880b4af0 2024-09-10T10:44:55.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.471+0000 7f4c87400640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:55.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.472+0000 7f4c87400640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c88153120 tx=0x7f4c74009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:55.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.476+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c880b15d0 con 0x7f4c880b4af0 2024-09-10T10:44:55.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:55.702+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4c88156b20 con 0x7f4c880b4af0 2024-09-10T10:44:56.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.259+0000 7f4c7fe00640 1 -- 172.21.15.28:0/238205854 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v15) ==== 172+0+0 (secure 0 0 0) 0x7f4c780798e0 con 0x7f4c880b4af0 2024-09-10T10:44:56.260 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 5 and data pool 6 2024-09-10T10:44:56.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 msgr2=0x7f4c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4c88153120 tx=0x7f4c74009190 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 msgr2=0x7f4c8813e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c7802a290 tx=0x7f4c780046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 shutdown_connections 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c88156740 0x7f4c88072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880cfcd0 0x7f4c881f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 --2- 172.21.15.28:0/238205854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b4af0 0x7f4c8813e5c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.264+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 >> 172.21.15.28:0/238205854 conn(0x7f4c8806bdf0 msgr2=0x7f4c8815b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.266+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 shutdown_connections 2024-09-10T10:44:56.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.266+0000 7f4c8ec00640 1 -- 172.21.15.28:0/238205854 wait complete. 2024-09-10T10:44:56.312 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:56.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.469+0000 7fbc72800640 1 Processor -- start 2024-09-10T10:44:56.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.469+0000 7fbc72800640 1 -- start start 2024-09-10T10:44:56.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc72800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc72800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbc6c15e660 0x7fbc6c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc72800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc6c0cf000 con 0x7fbc6c0d1f90 2024-09-10T10:44:56.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc72800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6c0b37d0 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc72800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbc6c0cf330 con 0x7fbc6c15e660 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc71400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc71400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33556/0 (socket says 172.21.15.28:33556) 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.470+0000 7fbc71400640 1 -- 172.21.15.28:0/4003441926 learned_addr learned my addr 172.21.15.28:0/4003441926 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc70a00640 1 --2- 172.21.15.28:0/4003441926 >> v2:172.21.15.138:3300/0 conn(0x7fbc6c15e660 0x7fbc6c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc71400640 1 -- 172.21.15.28:0/4003441926 >> v2:172.21.15.138:3300/0 conn(0x7fbc6c15e660 msgr2=0x7fbc6c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc71400640 1 --2- 172.21.15.28:0/4003441926 >> v2:172.21.15.138:3300/0 conn(0x7fbc6c15e660 0x7fbc6c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc71400640 1 -- 172.21.15.28:0/4003441926 >> v1:172.21.15.28:6789/0 conn(0x7fbc6c0d1f90 legacy=0x7fbc6c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc71400640 1 -- 172.21.15.28:0/4003441926 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c0d16f0 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.471+0000 7fbc6be00640 1 -- 172.21.15.28:0/4003441926 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2141694675 0 0) 0x7fbc6c0cf000 con 0x7fbc6c0d1f90 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.472+0000 7fbc71400640 1 --2- 172.21.15.28:0/4003441926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b3980 tx=0x7fbc6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566bc9ae235db58a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.472+0000 7fbc6be00640 1 -- 172.21.15.28:0/4003441926 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60037070 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.472+0000 7fbc6be00640 1 -- 172.21.15.28:0/4003441926 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc60032440 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.472+0000 7fbc6be00640 1 -- 172.21.15.28:0/4003441926 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc60032810 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 -- 172.21.15.28:0/4003441926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c0d0dd0 msgr2=0x7fbc6c0d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 --2- 172.21.15.28:0/4003441926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b3980 tx=0x7fbc6002a270 comp rx=0 tx=0).stop 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 -- 172.21.15.28:0/4003441926 shutdown_connections 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 --2- 172.21.15.28:0/4003441926 >> v2:172.21.15.138:3300/0 conn(0x7fbc6c15e660 0x7fbc6c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 --2- 172.21.15.28:0/4003441926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0d11b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 -- 172.21.15.28:0/4003441926 >> 172.21.15.28:0/4003441926 conn(0x7fbc6c08bfc0 msgr2=0x7fbc6c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 -- 172.21.15.28:0/4003441926 shutdown_connections 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.473+0000 7fbc72800640 1 -- 172.21.15.28:0/4003441926 wait complete. 2024-09-10T10:44:56.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.474+0000 7fbc72800640 1 Processor -- start 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 -- start start 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 0x7fbc6c0ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0d2f40 con 0x7fbc6c15e660 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c0d2c60 con 0x7fbc6c0d1f90 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.475+0000 7fbc72800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c0d3220 con 0x7fbc6c0d0dd0 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33572/0 (socket says 172.21.15.28:33572) 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 -- 172.21.15.28:0/632754382 learned_addr learned my addr 172.21.15.28:0/632754382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc70a00640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 0x7fbc6c0ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71400640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 -- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 msgr2=0x7fbc6c0b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0b8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 -- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 msgr2=0x7fbc6c0ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 0x7fbc6c0ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 -- 172.21.15.28:0/632754382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c0b7c00 con 0x7fbc6c15e660 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc70a00640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 0x7fbc6c0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71400640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.476+0000 7fbc71e00640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c760 tx=0x7fbc6400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.477+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64003da0 con 0x7fbc6c15e660 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.477+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6400ce10 con 0x7fbc6c15e660 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.477+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6c0b7ee0 con 0x7fbc6c15e660 2024-09-10T10:44:56.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.477+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640052b0 con 0x7fbc6c15e660 2024-09-10T10:44:56.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.477+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc6c0b84a0 con 0x7fbc6c15e660 2024-09-10T10:44:56.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.478+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc6c0d2420 con 0x7fbc6c15e660 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.479+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbc64005450 con 0x7fbc6c15e660 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.480+0000 7fbc69600640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 0x7fbc4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.480+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fbc6408ae90 con 0x7fbc6c15e660 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.481+0000 7fbc71400640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 0x7fbc4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.481+0000 7fbc71400640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 0x7fbc4c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc60034000 tx=0x7fbc6002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.485+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc640596b0 con 0x7fbc6c15e660 2024-09-10T10:44:56.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.670+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc6c0b1cc0 con 0x7fbc6c15e660 2024-09-10T10:44:56.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.671+0000 7fbc69600640 1 -- 172.21.15.28:0/632754382 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17039 (secure 0 0 0) 0x7fbc64058fb0 con 0x7fbc6c15e660 2024-09-10T10:44:56.673 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:56.673 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":41,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:55.703971+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:53.486429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:54.688726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.673+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 msgr2=0x7fbc4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.673+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 0x7fbc4c06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc60034000 tx=0x7fbc6002e000 comp rx=0 tx=0).stop 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.673+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 msgr2=0x7fbc6c159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.673+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c760 tx=0x7fbc6400cc20 comp rx=0 tx=0).stop 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 shutdown_connections 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbc4c06be30 0x7fbc4c06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc6c15e660 0x7fbc6c159800 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc6c0d1f90 0x7fbc6c0ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 --2- 172.21.15.28:0/632754382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc6c0d0dd0 0x7fbc6c0b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 >> 172.21.15.28:0/632754382 conn(0x7fbc6c08bfc0 msgr2=0x7fbc6c06db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 shutdown_connections 2024-09-10T10:44:56.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.674+0000 7fbc72800640 1 -- 172.21.15.28:0/632754382 wait complete. 2024-09-10T10:44:56.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.846+0000 7f495fe00640 1 Processor -- start 2024-09-10T10:44:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.846+0000 7f495fe00640 1 -- start start 2024-09-10T10:44:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.847+0000 7f495fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 0x7f49580d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.847+0000 7f495fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f495809b690 0x7f495806cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.847+0000 7f495fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49580d0780 con 0x7f495806d4c0 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.847+0000 7f495fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f495815eb20 con 0x7f495809b690 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.847+0000 7f495fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f495806d260 con 0x7f495815e740 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f495809b690 0x7f495806cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f495809b690 0x7f495806cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33588/0 (socket says 172.21.15.28:33588) 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 -- 172.21.15.28:0/3021502780 learned_addr learned my addr 172.21.15.28:0/3021502780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495d400640 1 --2- 172.21.15.28:0/3021502780 >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 0x7f49580d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 -- 172.21.15.28:0/3021502780 >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 msgr2=0x7f49580d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 --2- 172.21.15.28:0/3021502780 >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 0x7f49580d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 -- 172.21.15.28:0/3021502780 >> v1:172.21.15.28:6789/0 conn(0x7f495806d4c0 legacy=0x7f495806d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495ca00640 1 -- 172.21.15.28:0/3021502780 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49580d0a60 con 0x7f495809b690 2024-09-10T10:44:56.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.848+0000 7f495d400640 1 --2- 172.21.15.28:0/3021502780 >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 0x7f49580d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.849+0000 7f495ca00640 1 --2- 172.21.15.28:0/3021502780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f495806cd20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4958074070 tx=0x7f494402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7248cc484d2547 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.849+0000 7f4957e00640 1 -- 172.21.15.28:0/3021502780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944037070 con 0x7f495809b690 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f4957e00640 1 -- 172.21.15.28:0/3021502780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494402ae50 con 0x7f495809b690 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f4957e00640 1 -- 172.21.15.28:0/3021502780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944032860 con 0x7f495809b690 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 -- 172.21.15.28:0/3021502780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 msgr2=0x7f495806cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 --2- 172.21.15.28:0/3021502780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f495806cd20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4958074070 tx=0x7f494402a300 comp rx=0 tx=0).stop 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 -- 172.21.15.28:0/3021502780 shutdown_connections 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 --2- 172.21.15.28:0/3021502780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f495806cd20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 --2- 172.21.15.28:0/3021502780 >> v2:172.21.15.138:3300/0 conn(0x7f495815e740 0x7f49580d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 -- 172.21.15.28:0/3021502780 >> 172.21.15.28:0/3021502780 conn(0x7f49580d23a0 msgr2=0x7f49580d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 -- 172.21.15.28:0/3021502780 shutdown_connections 2024-09-10T10:44:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.850+0000 7f495fe00640 1 -- 172.21.15.28:0/3021502780 wait complete. 2024-09-10T10:44:56.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.852+0000 7f495fe00640 1 Processor -- start 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.852+0000 7f495fe00640 1 -- start start 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f495806d4c0 0x7f49580cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f495815e740 0x7f49580cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f495809bac0 con 0x7f495809b690 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49580d1020 con 0x7f495806d4c0 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f495806dfe0 con 0x7f495815e740 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33590/0 (socket says 172.21.15.28:33590) 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495ca00640 1 -- 172.21.15.28:0/35678486 learned_addr learned my addr 172.21.15.28:0/35678486 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495d400640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f495806d4c0 0x7f49580cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.853+0000 7f495de00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f495815e740 0x7f49580cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 -- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f495815e740 msgr2=0x7f49580cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f495815e740 0x7f49580cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 -- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f495806d4c0 msgr2=0x7f49580cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f495806d4c0 0x7f49580cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 -- 172.21.15.28:0/35678486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49580cdce0 con 0x7f495809b690 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495ca00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f494402a340 tx=0x7f49440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944041070 con 0x7f495809b690 2024-09-10T10:44:56.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494403c670 con 0x7f495809b690 2024-09-10T10:44:56.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4944037040 con 0x7f495809b690 2024-09-10T10:44:56.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49580cdfc0 con 0x7f495809b690 2024-09-10T10:44:56.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.854+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49580b5440 con 0x7f495809b690 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.856+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495806cd20 con 0x7f495809b690 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.856+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4944044070 con 0x7f495809b690 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.857+0000 7f4955600640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 0x7f493806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.857+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f49440ab990 con 0x7f495809b690 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.858+0000 7f495d400640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 0x7f493806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.859+0000 7f495d400640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 0x7f493806e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f494c006e80 tx=0x7f494c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:56.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:56.862+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4944079b60 con 0x7f495809b690 2024-09-10T10:44:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.046+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49580b5610 con 0x7f495809b690 2024-09-10T10:44:57.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.048+0000 7f4955600640 1 -- 172.21.15.28:0/35678486 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17039 (secure 0 0 0) 0x7f494407de60 con 0x7f495809b690 2024-09-10T10:44:57.049 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:57.049 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":41,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:55.703971+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:53.486429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:54.688726+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 msgr2=0x7f493806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 0x7f493806e320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f494c006e80 tx=0x7f494c008040 comp rx=0 tx=0).stop 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 msgr2=0x7f49580cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f494402a340 tx=0x7f49440046c0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 shutdown_connections 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f493806be60 0x7f493806e320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f495815e740 0x7f49580cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f495809b690 0x7f49580cabc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 --2- 172.21.15.28:0/35678486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f495806d4c0 0x7f49580cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 >> 172.21.15.28:0/35678486 conn(0x7f49580d23a0 msgr2=0x7f49580af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:57.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 shutdown_connections 2024-09-10T10:44:57.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.050+0000 7f495fe00640 1 -- 172.21.15.28:0/35678486 wait complete. 2024-09-10T10:44:57.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:44:57.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.228+0000 7fe017800640 1 Processor -- start 2024-09-10T10:44:57.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.228+0000 7fe017800640 1 -- start start 2024-09-10T10:44:57.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe017800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe01015e900 0x7fe0100b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe017800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe010072050 0x7fe010072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe017800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0100b4620 con 0x7fe0100b4a20 2024-09-10T10:44:57.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe017800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100729c0 con 0x7fe010072050 2024-09-10T10:44:57.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe017800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe0100ce100 con 0x7fe01015e900 2024-09-10T10:44:57.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe015800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe010072050 0x7fe010072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe00fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe0100b4a20 0x7fe0100ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56924/0 (socket says 172.21.15.28:56924) 2024-09-10T10:44:57.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe00fe00640 1 -- 172.21.15.28:0/3195038611 learned_addr learned my addr 172.21.15.28:0/3195038611 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:57.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.229+0000 7fe014e00640 1 --2- 172.21.15.28:0/3195038611 >> v2:172.21.15.138:3300/0 conn(0x7fe01015e900 0x7fe0100b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.230+0000 7fe015800640 1 -- 172.21.15.28:0/3195038611 >> v2:172.21.15.138:3300/0 conn(0x7fe01015e900 msgr2=0x7fe0100b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.230+0000 7fe015800640 1 --2- 172.21.15.28:0/3195038611 >> v2:172.21.15.138:3300/0 conn(0x7fe01015e900 0x7fe0100b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.230+0000 7fe015800640 1 -- 172.21.15.28:0/3195038611 >> v1:172.21.15.28:6789/0 conn(0x7fe0100b4a20 legacy=0x7fe0100ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.230+0000 7fe015800640 1 -- 172.21.15.28:0/3195038611 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0100ce3a0 con 0x7fe010072050 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.230+0000 7fe00f400640 1 -- 172.21.15.28:0/3195038611 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013530344 0 0) 0x7fe0100b4620 con 0x7fe0100b4a20 2024-09-10T10:44:57.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.231+0000 7fe015800640 1 --2- 172.21.15.28:0/3195038611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe010072480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0100d0d60 tx=0x7fe00802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ab7c08cc0df5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:57.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.231+0000 7fe00f400640 1 -- 172.21.15.28:0/3195038611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008031600 con 0x7fe010072050 2024-09-10T10:44:57.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.231+0000 7fe00f400640 1 -- 172.21.15.28:0/3195038611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0080317a0 con 0x7fe010072050 2024-09-10T10:44:57.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.231+0000 7fe00f400640 1 -- 172.21.15.28:0/3195038611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008031b70 con 0x7fe010072050 2024-09-10T10:44:57.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 -- 172.21.15.28:0/3195038611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 msgr2=0x7fe010072480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 --2- 172.21.15.28:0/3195038611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe010072480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0100d0d60 tx=0x7fe00802a610 comp rx=0 tx=0).stop 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 -- 172.21.15.28:0/3195038611 shutdown_connections 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 --2- 172.21.15.28:0/3195038611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe010072480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 --2- 172.21.15.28:0/3195038611 >> v2:172.21.15.138:3300/0 conn(0x7fe01015e900 0x7fe0100b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 -- 172.21.15.28:0/3195038611 >> 172.21.15.28:0/3195038611 conn(0x7fe01006bdf0 msgr2=0x7fe01006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 -- 172.21.15.28:0/3195038611 shutdown_connections 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.232+0000 7fe017800640 1 -- 172.21.15.28:0/3195038611 wait complete. 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.233+0000 7fe017800640 1 Processor -- start 2024-09-10T10:44:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 -- start start 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0100b4a20 0x7fe0101557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01015e900 0x7fe010155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100b4e50 con 0x7fe010072050 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0100ce960 con 0x7fe01015e900 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.234+0000 7fe017800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0100cee80 con 0x7fe0100b4a20 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33602/0 (socket says 172.21.15.28:33602) 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 -- 172.21.15.28:0/1749077188 learned_addr learned my addr 172.21.15.28:0/1749077188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 -- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0100b4a20 msgr2=0x7fe0101557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0100b4a20 0x7fe0101557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 -- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01015e900 msgr2=0x7fe010155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01015e900 0x7fe010155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 -- 172.21.15.28:0/1749077188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01015d210 con 0x7fe010072050 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.235+0000 7fe014e00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe00400e870 tx=0x7fe00400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.236+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004017070 con 0x7fe010072050 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.236+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00400c940 con 0x7fe010072050 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.236+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01015d4f0 con 0x7fe010072050 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.236+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00400cd30 con 0x7fe010072050 2024-09-10T10:44:57.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.236+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01015dd90 con 0x7fe010072050 2024-09-10T10:44:57.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.237+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01015d6c0 con 0x7fe010072050 2024-09-10T10:44:57.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.238+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe00401a070 con 0x7fe010072050 2024-09-10T10:44:57.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.238+0000 7fe00cc00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 0x7fdff006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:57.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.238+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fe00408b270 con 0x7fe010072050 2024-09-10T10:44:57.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.239+0000 7fe00fe00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 0x7fdff006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:57.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.240+0000 7fe00fe00640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 0x7fdff006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007fd0 tx=0x7fdffc007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:57.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.243+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe004059a80 con 0x7fe010072050 2024-09-10T10:44:57.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:57.466+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe010072480 con 0x7fe010072050 2024-09-10T10:44:58.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.279+0000 7fe00cc00640 1 -- 172.21.15.28:0/1749077188 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fe004059380 con 0x7fe010072050 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 msgr2=0x7fdff006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 0x7fdff006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007fd0 tx=0x7fdffc007920 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 msgr2=0x7fe0101521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe00400e870 tx=0x7fe00400ed30 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 shutdown_connections 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdff006be30 0x7fdff006e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe01015e900 0x7fe010155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0100b4a20 0x7fe0101557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 --2- 172.21.15.28:0/1749077188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe010072050 0x7fe0101521d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 >> 172.21.15.28:0/1749077188 conn(0x7fe01006bdf0 msgr2=0x7fe01006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:58.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 shutdown_connections 2024-09-10T10:44:58.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.282+0000 7fe017800640 1 -- 172.21.15.28:0/1749077188 wait complete. 2024-09-10T10:44:58.302 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:44:58.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:44:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.465+0000 7f1717a00640 1 Processor -- start 2024-09-10T10:44:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.465+0000 7f1717a00640 1 -- start start 2024-09-10T10:44:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.466+0000 7f1717a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f17100b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.466+0000 7f1717a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f17100b4af0 0x7f17100cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.466+0000 7f1717a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17100b30e0 con 0x7f17100cfcd0 2024-09-10T10:44:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.466+0000 7f1717a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f171006edc0 con 0x7f17100b4af0 2024-09-10T10:44:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.466+0000 7f1717a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f17100b33c0 con 0x7f1710156740 2024-09-10T10:44:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f1715a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f17100cfcd0 0x7f171015fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56936/0 (socket says 172.21.15.28:56936) 2024-09-10T10:44:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f1715a00640 1 -- 172.21.15.28:0/3899728265 learned_addr learned my addr 172.21.15.28:0/3899728265 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170fe00640 1 --2- 172.21.15.28:0/3899728265 >> v2:172.21.15.138:3300/0 conn(0x7f17100b4af0 0x7f17100cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f1715000640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f17100b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751316484 0 0) 0x7f17100b30e0 con 0x7f17100cfcd0 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16f8003520 con 0x7f17100cfcd0 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 916709919 0 0) 0x7f16f8003520 con 0x7f17100cfcd0 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 msgr2=0x7f17100b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f17100b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 >> v2:172.21.15.138:3300/0 conn(0x7f17100b4af0 msgr2=0x7f17100cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 --2- 172.21.15.28:0/3899728265 >> v2:172.21.15.138:3300/0 conn(0x7f17100b4af0 0x7f17100cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17100b36a0 con 0x7f17100cfcd0 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f17080019f0 con 0x7f17100cfcd0 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 >> v1:172.21.15.28:6789/0 conn(0x7f17100cfcd0 legacy=0x7f171015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.467+0000 7f170f400640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16f80039b0 0x7f16f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170f400640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f8004800 0x7f16f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170f400640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f16f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f171006edc0 con 0x7f16f80039b0 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17100b33c0 con 0x7f16f8004800 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f16f800b9c0 con 0x7f1710156740 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16f80039b0 0x7f16f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f170fe00640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f8004800 0x7f16f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715a00640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f16f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 -- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 msgr2=0x7f16f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1710156740 0x7f16f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 -- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f8004800 msgr2=0x7f16f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f8004800 0x7f16f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.468+0000 7f1715000640 1 -- 172.21.15.28:0/3899728265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f8003520 con 0x7f16f80039b0 2024-09-10T10:44:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.469+0000 7f1715000640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16f80039b0 0x7f16f8003dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f16f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f631570b69d1da7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704036070 con 0x7f16f80039b0 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1704031400 con 0x7f16f80039b0 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f170f400640 1 -- 172.21.15.28:0/3899728265 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1704009690 con 0x7f16f80039b0 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 -- 172.21.15.28:0/3899728265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16f80039b0 msgr2=0x7f16f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 --2- 172.21.15.28:0/3899728265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16f80039b0 0x7f16f8003dd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1704009ac0 tx=0x7f16f8005830 comp rx=0 tx=0).stop 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1715a00640 1 -- 172.21.15.28:0/3899728265 reap_dead start 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 -- 172.21.15.28:0/3899728265 shutdown_connections 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 -- 172.21.15.28:0/3899728265 >> 172.21.15.28:0/3899728265 conn(0x7f171006bdf0 msgr2=0x7f171006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 -- 172.21.15.28:0/3899728265 shutdown_connections 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.470+0000 7f1717a00640 1 -- 172.21.15.28:0/3899728265 wait complete. 2024-09-10T10:44:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.472+0000 7f1717a00640 1 Processor -- start 2024-09-10T10:44:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.472+0000 7f1717a00640 1 -- start start 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f80039b0 0x7f17101f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 0x7f17101f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f17100d0890 con 0x7f17101537a0 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17100b3c60 con 0x7f16f80039b0 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1717a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f17100d0a10 con 0x7f171014e340 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1715a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1715a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33608/0 (socket says 172.21.15.28:33608) 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.473+0000 7f1715a00640 1 -- 172.21.15.28:0/126095451 learned_addr learned my addr 172.21.15.28:0/126095451 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715000640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f80039b0 0x7f17101f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f170fe00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 0x7f17101f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 -- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 msgr2=0x7f17101f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 0x7f17101f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 -- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f80039b0 msgr2=0x7f17101f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f80039b0 0x7f17101f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 -- 172.21.15.28:0/126095451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17101f2a30 con 0x7f17101537a0 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f170fe00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 0x7f17101f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f1715a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17100b40f0 tx=0x7f1708007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.474+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f170800abb0 con 0x7f17101537a0 2024-09-10T10:44:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.475+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1708018070 con 0x7f17101537a0 2024-09-10T10:44:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.475+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17080133f0 con 0x7f17101537a0 2024-09-10T10:44:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.475+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1710154180 con 0x7f17101537a0 2024-09-10T10:44:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.475+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17101587e0 con 0x7f17101537a0 2024-09-10T10:44:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.476+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171015bf30 con 0x7f17101537a0 2024-09-10T10:44:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.477+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1708013590 con 0x7f17101537a0 2024-09-10T10:44:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.477+0000 7f170cc00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 0x7f16f006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.478+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f170808b560 con 0x7f17101537a0 2024-09-10T10:44:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.478+0000 7f1715000640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 0x7f16f006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.479+0000 7f1715000640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 0x7f16f006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1704033000 tx=0x7f1704008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.482+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1708059ce0 con 0x7f17101537a0 2024-09-10T10:44:58.567 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:44:58.566+0000 7f4309800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:44:58.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:58.704+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f171015bc50 con 0x7f17101537a0 2024-09-10T10:44:59.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.285+0000 7f170cc00640 1 -- 172.21.15.28:0/126095451 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f17080595e0 con 0x7f17101537a0 2024-09-10T10:44:59.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 msgr2=0x7f16f006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 0x7f16f006e320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1704033000 tx=0x7f1704008030 comp rx=0 tx=0).stop 2024-09-10T10:44:59.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 msgr2=0x7f17100afb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17100b40f0 tx=0x7f1708007a90 comp rx=0 tx=0).stop 2024-09-10T10:44:59.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 shutdown_connections 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f16f006be60 0x7f16f006e320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17101537a0 0x7f17100afb50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f171014e340 0x7f17101f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 --2- 172.21.15.28:0/126095451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16f80039b0 0x7f17101f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 >> 172.21.15.28:0/126095451 conn(0x7f171006bdf0 msgr2=0x7f17100b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 shutdown_connections 2024-09-10T10:44:59.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.288+0000 7f1717a00640 1 -- 172.21.15.28:0/126095451 wait complete. 2024-09-10T10:44:59.308 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:44:59.308 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:44:59.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.468+0000 7fd554200640 1 Processor -- start 2024-09-10T10:44:59.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.468+0000 7fd554200640 1 -- start start 2024-09-10T10:44:59.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd554200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd54c15b300 0x7fd54c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd554200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c06f8d0 0x7fd54c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd554200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd54c0cdbe0 con 0x7fd54c0cd3f0 2024-09-10T10:44:59.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd554200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd54c0728a0 con 0x7fd54c15b300 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd554200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c0cd900 con 0x7fd54c06f8d0 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.469+0000 7fd551800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd54c15b300 0x7fd54c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd54c15b300 0x7fd54c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33610/0 (socket says 172.21.15.28:33610) 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 -- 172.21.15.28:0/2900454979 learned_addr learned my addr 172.21.15.28:0/2900454979 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 -- 172.21.15.28:0/2900454979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c06f8d0 msgr2=0x7fd54c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 --2- 172.21.15.28:0/2900454979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c06f8d0 0x7fd54c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 -- 172.21.15.28:0/2900454979 >> v1:172.21.15.28:6789/0 conn(0x7fd54c0cd3f0 legacy=0x7fd54c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.470+0000 7fd551800640 1 -- 172.21.15.28:0/2900454979 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c072250 con 0x7fd54c15b300 2024-09-10T10:44:59.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.471+0000 7fd551800640 1 --2- 172.21.15.28:0/2900454979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c0cceb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd540009870 tx=0x7fd54002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d857ef1c88ca2224 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.471+0000 7fd54be00640 1 -- 172.21.15.28:0/2900454979 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540037070 con 0x7fd54c15b300 2024-09-10T10:44:59.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.471+0000 7fd54be00640 1 -- 172.21.15.28:0/2900454979 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd540032440 con 0x7fd54c15b300 2024-09-10T10:44:59.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.471+0000 7fd54be00640 1 -- 172.21.15.28:0/2900454979 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540032810 con 0x7fd54c15b300 2024-09-10T10:44:59.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 -- 172.21.15.28:0/2900454979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 msgr2=0x7fd54c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 --2- 172.21.15.28:0/2900454979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c0cceb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd540009870 tx=0x7fd54002a250 comp rx=0 tx=0).stop 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 -- 172.21.15.28:0/2900454979 shutdown_connections 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 --2- 172.21.15.28:0/2900454979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c06f8d0 0x7fd54c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 --2- 172.21.15.28:0/2900454979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c0cceb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 -- 172.21.15.28:0/2900454979 >> 172.21.15.28:0/2900454979 conn(0x7fd54c067d70 msgr2=0x7fd54c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 -- 172.21.15.28:0/2900454979 shutdown_connections 2024-09-10T10:44:59.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.472+0000 7fd554200640 1 -- 172.21.15.28:0/2900454979 wait complete. 2024-09-10T10:44:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.474+0000 7fd554200640 1 Processor -- start 2024-09-10T10:44:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.474+0000 7fd554200640 1 -- start start 2024-09-10T10:44:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd54c06f8d0 0x7fd54c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c0cd3f0 0x7fd54c1e5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0d0160 con 0x7fd54c15b300 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0cffe0 con 0x7fd54c06f8d0 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd554200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c072d30 con 0x7fd54c0cd3f0 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33614/0 (socket says 172.21.15.28:33614) 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 -- 172.21.15.28:0/3500926804 learned_addr learned my addr 172.21.15.28:0/3500926804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:59.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd550e00640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c0cd3f0 0x7fd54c1e5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd551800640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd54c06f8d0 0x7fd54c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 -- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c0cd3f0 msgr2=0x7fd54c1e5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c0cd3f0 0x7fd54c1e5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 -- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd54c06f8d0 msgr2=0x7fd54c15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd54c06f8d0 0x7fd54c15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.475+0000 7fd552200640 1 -- 172.21.15.28:0/3500926804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c1e6190 con 0x7fd54c15b300 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd552200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd54400b9e0 tx=0x7fd54400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544019070 con 0x7fd54c15b300 2024-09-10T10:44:59.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54400cb20 con 0x7fd54c15b300 2024-09-10T10:44:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544014440 con 0x7fd54c15b300 2024-09-10T10:44:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c1e58f0 con 0x7fd54c15b300 2024-09-10T10:44:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.476+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c1e5eb0 con 0x7fd54c15b300 2024-09-10T10:44:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.477+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c159000 con 0x7fd54c15b300 2024-09-10T10:44:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.478+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd54401c070 con 0x7fd54c15b300 2024-09-10T10:44:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.479+0000 7fd549600640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 0x7fd52c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.479+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fd54408ad10 con 0x7fd54c15b300 2024-09-10T10:44:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.479+0000 7fd551800640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 0x7fd52c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.480+0000 7fd551800640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 0x7fd52c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd540034000 tx=0x7fd54002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.483+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd544090020 con 0x7fd54c15b300 2024-09-10T10:44:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.703+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd54c158d20 con 0x7fd54c15b300 2024-09-10T10:44:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.704+0000 7fd549600640 1 -- 172.21.15.28:0/3500926804 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fd544058e40 con 0x7fd54c15b300 2024-09-10T10:44:59.705 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:44:59.705 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":18,"btime":"2024-09-10T10:44:59:280567+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:44:58.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4845":{"gid":4845,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3203683106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3203683106},{"type":"v1","addr":"172.21.15.28:6835","nonce":3203683106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4845,"qdb_cluster":[4845]},"id":2}]} 2024-09-10T10:44:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 18 2024-09-10T10:44:59.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 msgr2=0x7fd52c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 0x7fd52c06e2f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd540034000 tx=0x7fd54002e000 comp rx=0 tx=0).stop 2024-09-10T10:44:59.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 msgr2=0x7fd54c154320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd54400b9e0 tx=0x7fd54400bea0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 shutdown_connections 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd52c06be30 0x7fd52c06e2f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd54c15b300 0x7fd54c154320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd54c0cd3f0 0x7fd54c1e5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 --2- 172.21.15.28:0/3500926804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd54c06f8d0 0x7fd54c15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 >> 172.21.15.28:0/3500926804 conn(0x7fd54c067d70 msgr2=0x7fd54c0af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 shutdown_connections 2024-09-10T10:44:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.706+0000 7fd554200640 1 -- 172.21.15.28:0/3500926804 wait complete. 2024-09-10T10:44:59.722 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:44:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.877+0000 7fa93fe00640 1 Processor -- start 2024-09-10T10:44:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.877+0000 7fa93fe00640 1 -- start start 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa938072d30 0x7fa93806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa93815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9380cdbe0 con 0x7fa93815b300 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa9380728a0 con 0x7fa938072d30 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa93806fb30 con 0x7fa93806f700 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa93815b300 0x7fa9380cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56958/0 (socket says 172.21.15.28:56958) 2024-09-10T10:44:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93d400640 1 -- 172.21.15.28:0/559227623 learned_addr learned my addr 172.21.15.28:0/559227623 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:44:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93ca00640 1 --2- 172.21.15.28:0/559227623 >> v2:172.21.15.138:3300/0 conn(0x7fa938072d30 0x7fa93806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986597799 0 0) 0x7fa9380cdbe0 con 0x7fa93815b300 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa93de00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa93815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa920003520 con 0x7fa93815b300 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.879+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4073246041 0 0) 0x7fa920003520 con 0x7fa93815b300 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 msgr2=0x7fa93815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa93815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 >> v2:172.21.15.138:3300/0 conn(0x7fa938072d30 msgr2=0x7fa93806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 --2- 172.21.15.28:0/559227623 >> v2:172.21.15.138:3300/0 conn(0x7fa938072d30 0x7fa93806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380cd900 con 0x7fa93815b300 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa92c001690 con 0x7fa93815b300 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 >> v1:172.21.15.28:6789/0 conn(0x7fa93815b300 legacy=0x7fa9380cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa9200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa920003f80 0x7fa920004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380728a0 con 0x7fa920004d40 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa93806fb30 con 0x7fa920003f80 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa92000ba70 con 0x7fa93806f700 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93d400640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa9200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93ca00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa920003f80 0x7fa920004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 -- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 msgr2=0x7fa9200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93806f700 0x7fa9200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 -- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa920003f80 msgr2=0x7fa920004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa920003f80 0x7fa920004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.880+0000 7fa93de00640 1 -- 172.21.15.28:0/559227623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa920003520 con 0x7fa920004d40 2024-09-10T10:44:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.881+0000 7fa93de00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9200051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa930009fc0 tx=0x7fa920005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7529227deabd2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930009c10 con 0x7fa920004d40 2024-09-10T10:44:59.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa930036070 con 0x7fa920004d40 2024-09-10T10:44:59.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa937e00640 1 -- 172.21.15.28:0/559227623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa930031880 con 0x7fa920004d40 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 -- 172.21.15.28:0/559227623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 msgr2=0x7fa9200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 --2- 172.21.15.28:0/559227623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9200051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa930009fc0 tx=0x7fa920005830 comp rx=0 tx=0).stop 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93de00640 1 -- 172.21.15.28:0/559227623 reap_dead start 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 -- 172.21.15.28:0/559227623 shutdown_connections 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 -- 172.21.15.28:0/559227623 >> 172.21.15.28:0/559227623 conn(0x7fa938067d70 msgr2=0x7fa93806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 -- 172.21.15.28:0/559227623 shutdown_connections 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.882+0000 7fa93fe00640 1 -- 172.21.15.28:0/559227623 wait complete. 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.884+0000 7fa93fe00640 1 Processor -- start 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.884+0000 7fa93fe00640 1 -- start start 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93814e220 0x7fa93814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa93814b6e0 0x7fa9380aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380cd5c0 con 0x7fa920004d40 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa938072530 con 0x7fa93814b6e0 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380cffe0 con 0x7fa93814e220 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33632/0 (socket says 172.21.15.28:33632) 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.885+0000 7fa93d400640 1 -- 172.21.15.28:0/3224816615 learned_addr learned my addr 172.21.15.28:0/3224816615 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93de00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa93814b6e0 0x7fa9380aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93ca00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93814e220 0x7fa93814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 -- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93814e220 msgr2=0x7fa93814ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93814e220 0x7fa93814ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 -- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa93814b6e0 msgr2=0x7fa9380aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:44:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa93814b6e0 0x7fa9380aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:44:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 -- 172.21.15.28:0/3224816615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93814e6a0 con 0x7fa920004d40 2024-09-10T10:44:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93d400640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa92c007310 tx=0x7fa92c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00f960 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92c018070 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00fd00 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9380add10 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.886+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380ae2d0 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.887+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93806f1c0 con 0x7fa920004d40 2024-09-10T10:44:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.888+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa92c01d460 con 0x7fa920004d40 2024-09-10T10:44:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.890+0000 7fa935600640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 0x7fa91806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:44:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.890+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fa92c08b100 con 0x7fa920004d40 2024-09-10T10:44:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.890+0000 7fa93ca00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 0x7fa91806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:44:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.891+0000 7fa93ca00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 0x7fa91806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b3e60 tx=0x7fa924008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:44:59.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:44:59.894+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa92c0599b0 con 0x7fa920004d40 2024-09-10T10:45:00.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.081+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa93806ed50 con 0x7fa920004d40 2024-09-10T10:45:00.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.082+0000 7fa935600640 1 -- 172.21.15.28:0/3224816615 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+17026 (secure 0 0 0) 0x7fa92c0592b0 con 0x7fa920004d40 2024-09-10T10:45:00.083 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:00.083 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":42,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:44:57.269058+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:53.486429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:54.688726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 msgr2=0x7fa91806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 0x7fa91806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa9380b3e60 tx=0x7fa924008040 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 msgr2=0x7fa9380c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa92c007310 tx=0x7fa92c0077d0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 shutdown_connections 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa91806be30 0x7fa91806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa93814b6e0 0x7fa9380aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa93814e220 0x7fa93814ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 --2- 172.21.15.28:0/3224816615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa920004d40 0x7fa9380c5440 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 >> 172.21.15.28:0/3224816615 conn(0x7fa938067d70 msgr2=0x7fa9380b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 shutdown_connections 2024-09-10T10:45:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.084+0000 7fa93fe00640 1 -- 172.21.15.28:0/3224816615 wait complete. 2024-09-10T10:45:00.101 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:45:00.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:00.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.256+0000 7fd2d3c00640 1 Processor -- start 2024-09-10T10:45:00.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.256+0000 7fd2d3c00640 1 -- start start 2024-09-10T10:45:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d3c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2cc147230 0x7fd2cc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d3c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 0x7fd2cc0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d3c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2cc06f480 con 0x7fd2cc0725f0 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d3c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc0b34d0 con 0x7fd2cc147230 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d3c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc0ae9e0 con 0x7fd2cc147e90 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2cc147230 0x7fd2cc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd2cc0725f0 0x7fd2cc161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56970/0 (socket says 172.21.15.28:56970) 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1c00640 1 -- 172.21.15.28:0/350473425 learned_addr learned my addr 172.21.15.28:0/350473425 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2cbe00640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 0x7fd2cc0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1200640 1 -- 172.21.15.28:0/350473425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 msgr2=0x7fd2cc0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1200640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 0x7fd2cc0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1200640 1 -- 172.21.15.28:0/350473425 >> v1:172.21.15.28:6789/0 conn(0x7fd2cc0725f0 legacy=0x7fd2cc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2d1200640 1 -- 172.21.15.28:0/350473425 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc06e900 con 0x7fd2cc147230 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2cbe00640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 0x7fd2cc0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.258+0000 7fd2cb400640 1 -- 172.21.15.28:0/350473425 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152868587 0 0) 0x7fd2cc06f480 con 0x7fd2cc0725f0 2024-09-10T10:45:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.259+0000 7fd2d1200640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147230 0x7fd2cc06e0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097a0 tx=0x7fd2c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35be3c77e1f3344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2cb400640 1 -- 172.21.15.28:0/350473425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0037070 con 0x7fd2cc147230 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2cb400640 1 -- 172.21.15.28:0/350473425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2cc147230 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2cb400640 1 -- 172.21.15.28:0/350473425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032810 con 0x7fd2cc147230 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 -- 172.21.15.28:0/350473425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147230 msgr2=0x7fd2cc06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147230 0x7fd2cc06e0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00097a0 tx=0x7fd2c002a180 comp rx=0 tx=0).stop 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 -- 172.21.15.28:0/350473425 shutdown_connections 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147e90 0x7fd2cc0720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/350473425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147230 0x7fd2cc06e0b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 -- 172.21.15.28:0/350473425 >> 172.21.15.28:0/350473425 conn(0x7fd2cc067d70 msgr2=0x7fd2cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 -- 172.21.15.28:0/350473425 shutdown_connections 2024-09-10T10:45:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.260+0000 7fd2d3c00640 1 -- 172.21.15.28:0/350473425 wait complete. 2024-09-10T10:45:00.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.262+0000 7fd2d3c00640 1 Processor -- start 2024-09-10T10:45:00.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.262+0000 7fd2d3c00640 1 -- start start 2024-09-10T10:45:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 0x7fd2cc1e5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147230 0x7fd2cc1e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc06eec0 con 0x7fd2cc147e90 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc072a40 con 0x7fd2cc0725f0 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d3c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc06f1a0 con 0x7fd2cc147230 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33638/0 (socket says 172.21.15.28:33638) 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d1c00640 1 -- 172.21.15.28:0/1517650691 learned_addr learned my addr 172.21.15.28:0/1517650691 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2d1200640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 0x7fd2cc1e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.263+0000 7fd2cbe00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147230 0x7fd2cc1e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 -- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147230 msgr2=0x7fd2cc1e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147230 0x7fd2cc1e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 -- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 msgr2=0x7fd2cc1e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 0x7fd2cc1e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 -- 172.21.15.28:0/1517650691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc1e6390 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1200640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 0x7fd2cc1e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d1c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c700 tx=0x7fd2c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4018070 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c400ce10 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400a860 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc1e6670 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.264+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1e6c30 con 0x7fd2cc147e90 2024-09-10T10:45:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.265+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc1e7120 con 0x7fd2cc147e90 2024-09-10T10:45:00.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.266+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c401b070 con 0x7fd2cc147e90 2024-09-10T10:45:00.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.268+0000 7fd2c8c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 0x7fd2ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.268+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fd2c408a4f0 con 0x7fd2cc147e90 2024-09-10T10:45:00.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.268+0000 7fd2d1200640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 0x7fd2ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.269+0000 7fd2d1200640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 0x7fd2ac06e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.272+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd2c40586b0 con 0x7fd2cc147e90 2024-09-10T10:45:00.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.455+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2cc147610 con 0x7fd2ac06be30 2024-09-10T10:45:00.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.462+0000 7fd2c8c00640 1 -- 172.21.15.28:0/1517650691 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245866 (secure 0 0 0) 0x7fd2cc147610 con 0x7fd2ac06be30 2024-09-10T10:45:00.469 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:00.470 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 msgr2=0x7fd2ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 0x7fd2ac06e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).stop 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 msgr2=0x7fd2cc1416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400c700 tx=0x7fd2c400cbc0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 shutdown_connections 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2ac06be30 0x7fd2ac06e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2cc147e90 0x7fd2cc1416b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2cc147230 0x7fd2cc1e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 --2- 172.21.15.28:0/1517650691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc0725f0 0x7fd2cc1e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 >> 172.21.15.28:0/1517650691 conn(0x7fd2cc067d70 msgr2=0x7fd2cc0aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 shutdown_connections 2024-09-10T10:45:00.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.466+0000 7fd2d3c00640 1 -- 172.21.15.28:0/1517650691 wait complete. 2024-09-10T10:45:00.483 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-10T10:44:58.499126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225076,"kb_used_data":4676,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749644,"statfs":{"total":773094113280,"available":772863635456,"internally_reserved":0,"allocated":4788224,"data_stored":2698577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.248267"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263853+0000","last_change":"2024-09-10T10:44:55.263853+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263853+0000","last_undegraded":"2024-09-10T10:44:55.263853+0000","last_fullsized":"2024-09-10T10:44:55.263853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271034+0000","last_change":"2024-09-10T10:44:55.271034+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271034+0000","last_undegraded":"2024-09-10T10:44:55.271034+0000","last_fullsized":"2024-09-10T10:44:55.271034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263908+0000","last_change":"2024-09-10T10:44:55.263908+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263908+0000","last_undegraded":"2024-09-10T10:44:55.263908+0000","last_fullsized":"2024-09-10T10:44:55.263908+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263405+0000","last_change":"2024-09-10T10:44:55.263405+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263405+0000","last_undegraded":"2024-09-10T10:44:55.263405+0000","last_fullsized":"2024-09-10T10:44:55.263405+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269251+0000","last_change":"2024-09-10T10:44:55.269251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269251+0000","last_undegraded":"2024-09-10T10:44:55.269251+0000","last_fullsized":"2024-09-10T10:44:55.269251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268012+0000","last_change":"2024-09-10T10:44:55.268012+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268012+0000","last_undegraded":"2024-09-10T10:44:55.268012+0000","last_fullsized":"2024-09-10T10:44:55.268012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264033+0000","last_change":"2024-09-10T10:44:55.264033+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264033+0000","last_undegraded":"2024-09-10T10:44:55.264033+0000","last_fullsized":"2024-09-10T10:44:55.264033+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265069+0000","last_change":"2024-09-10T10:44:55.265069+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265069+0000","last_undegraded":"2024-09-10T10:44:55.265069+0000","last_fullsized":"2024-09-10T10:44:55.265069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271830+0000","last_change":"2024-09-10T10:44:55.271830+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271830+0000","last_undegraded":"2024-09-10T10:44:55.271830+0000","last_fullsized":"2024-09-10T10:44:55.271830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266035+0000","last_change":"2024-09-10T10:44:55.266035+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266035+0000","last_undegraded":"2024-09-10T10:44:55.266035+0000","last_fullsized":"2024-09-10T10:44:55.266035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272854+0000","last_change":"2024-09-10T10:44:55.272854+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272854+0000","last_undegraded":"2024-09-10T10:44:55.272854+0000","last_fullsized":"2024-09-10T10:44:55.272854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270668+0000","last_change":"2024-09-10T10:44:55.270668+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270668+0000","last_undegraded":"2024-09-10T10:44:55.270668+0000","last_fullsized":"2024-09-10T10:44:55.270668+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263816+0000","last_change":"2024-09-10T10:44:55.263816+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263816+0000","last_undegraded":"2024-09-10T10:44:55.263816+0000","last_fullsized":"2024-09-10T10:44:55.263816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269525+0000","last_change":"2024-09-10T10:44:55.269525+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269525+0000","last_undegraded":"2024-09-10T10:44:55.269525+0000","last_fullsized":"2024-09-10T10:44:55.269525+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268281+0000","last_change":"2024-09-10T10:44:55.268281+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268281+0000","last_undegraded":"2024-09-10T10:44:55.268281+0000","last_fullsized":"2024-09-10T10:44:55.268281+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263005+0000","last_change":"2024-09-10T10:44:55.263005+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263005+0000","last_undegraded":"2024-09-10T10:44:55.263005+0000","last_fullsized":"2024-09-10T10:44:55.263005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267826+0000","last_change":"2024-09-10T10:44:55.267826+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267826+0000","last_undegraded":"2024-09-10T10:44:55.267826+0000","last_fullsized":"2024-09-10T10:44:55.267826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266794+0000","last_change":"2024-09-10T10:44:55.266794+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266794+0000","last_undegraded":"2024-09-10T10:44:55.266794+0000","last_fullsized":"2024-09-10T10:44:55.266794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:44:55.250795+0000","last_change":"2024-09-10T10:44:55.250795+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.250795+0000","last_undegraded":"2024-09-10T10:44:55.250795+0000","last_fullsized":"2024-09-10T10:44:55.250795+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269860+0000","last_change":"2024-09-10T10:44:55.269860+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269860+0000","last_undegraded":"2024-09-10T10:44:55.269860+0000","last_fullsized":"2024-09-10T10:44:55.269860+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267879+0000","last_change":"2024-09-10T10:44:55.267879+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267879+0000","last_undegraded":"2024-09-10T10:44:55.267879+0000","last_fullsized":"2024-09-10T10:44:55.267879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.273400+0000","last_change":"2024-09-10T10:44:55.273400+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.273400+0000","last_undegraded":"2024-09-10T10:44:55.273400+0000","last_fullsized":"2024-09-10T10:44:55.273400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266722+0000","last_change":"2024-09-10T10:44:55.266722+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266722+0000","last_undegraded":"2024-09-10T10:44:55.266722+0000","last_fullsized":"2024-09-10T10:44:55.266722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264529+0000","last_change":"2024-09-10T10:44:55.264529+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264529+0000","last_undegraded":"2024-09-10T10:44:55.264529+0000","last_fullsized":"2024-09-10T10:44:55.264529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270779+0000","last_change":"2024-09-10T10:44:55.270779+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270779+0000","last_undegraded":"2024-09-10T10:44:55.270779+0000","last_fullsized":"2024-09-10T10:44:55.270779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265957+0000","last_change":"2024-09-10T10:44:55.265957+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265957+0000","last_undegraded":"2024-09-10T10:44:55.265957+0000","last_fullsized":"2024-09-10T10:44:55.265957+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272379+0000","last_change":"2024-09-10T10:44:55.272379+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272379+0000","last_undegraded":"2024-09-10T10:44:55.272379+0000","last_fullsized":"2024-09-10T10:44:55.272379+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263201+0000","last_change":"2024-09-10T10:44:55.263201+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263201+0000","last_undegraded":"2024-09-10T10:44:55.263201+0000","last_fullsized":"2024-09-10T10:44:55.263201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267251+0000","last_change":"2024-09-10T10:44:55.267251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267251+0000","last_undegraded":"2024-09-10T10:44:55.267251+0000","last_fullsized":"2024-09-10T10:44:55.267251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267256+0000","last_change":"2024-09-10T10:44:55.267256+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267256+0000","last_undegraded":"2024-09-10T10:44:55.267256+0000","last_fullsized":"2024-09-10T10:44:55.267256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263151+0000","last_change":"2024-09-10T10:44:55.263151+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263151+0000","last_undegraded":"2024-09-10T10:44:55.263151+0000","last_fullsized":"2024-09-10T10:44:55.263151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264106+0000","last_change":"2024-09-10T10:44:55.264106+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264106+0000","last_undegraded":"2024-09-10T10:44:55.264106+0000","last_fullsized":"2024-09-10T10:44:55.264106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266238+0000","last_change":"2024-09-10T10:44:55.266238+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266238+0000","last_undegraded":"2024-09-10T10:44:55.266238+0000","last_fullsized":"2024-09-10T10:44:55.266238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282941+0000","last_change":"2024-09-10T10:44:55.277458+0000","last_active":"2024-09-10T10:44:55.282941+0000","last_peered":"2024-09-10T10:44:55.282941+0000","last_clean":"2024-09-10T10:44:55.282941+0000","last_became_active":"2024-09-10T10:44:55.277216+0000","last_became_peered":"2024-09-10T10:44:55.277216+0000","last_unstale":"2024-09-10T10:44:55.282941+0000","last_undegraded":"2024-09-10T10:44:55.282941+0000","last_fullsized":"2024-09-10T10:44:55.282941+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268161+0000","last_change":"2024-09-10T10:44:55.268161+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268161+0000","last_undegraded":"2024-09-10T10:44:55.268161+0000","last_fullsized":"2024-09-10T10:44:55.268161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284405+0000","last_change":"2024-09-10T10:44:55.281076+0000","last_active":"2024-09-10T10:44:55.284405+0000","last_peered":"2024-09-10T10:44:55.284405+0000","last_clean":"2024-09-10T10:44:55.284405+0000","last_became_active":"2024-09-10T10:44:55.280788+0000","last_became_peered":"2024-09-10T10:44:55.280788+0000","last_unstale":"2024-09-10T10:44:55.284405+0000","last_undegraded":"2024-09-10T10:44:55.284405+0000","last_fullsized":"2024-09-10T10:44:55.284405+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266360+0000","last_change":"2024-09-10T10:44:55.266360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266360+0000","last_undegraded":"2024-09-10T10:44:55.266360+0000","last_fullsized":"2024-09-10T10:44:55.266360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281259+0000","last_change":"2024-09-10T10:44:55.273775+0000","last_active":"2024-09-10T10:44:55.281259+0000","last_peered":"2024-09-10T10:44:55.281259+0000","last_clean":"2024-09-10T10:44:55.281259+0000","last_became_active":"2024-09-10T10:44:55.273439+0000","last_became_peered":"2024-09-10T10:44:55.273439+0000","last_unstale":"2024-09-10T10:44:55.281259+0000","last_undegraded":"2024-09-10T10:44:55.281259+0000","last_fullsized":"2024-09-10T10:44:55.281259+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269513+0000","last_change":"2024-09-10T10:44:55.269513+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269513+0000","last_undegraded":"2024-09-10T10:44:55.269513+0000","last_fullsized":"2024-09-10T10:44:55.269513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284887+0000","last_change":"2024-09-10T10:44:55.282020+0000","last_active":"2024-09-10T10:44:55.284887+0000","last_peered":"2024-09-10T10:44:55.284887+0000","last_clean":"2024-09-10T10:44:55.284887+0000","last_became_active":"2024-09-10T10:44:55.281745+0000","last_became_peered":"2024-09-10T10:44:55.281745+0000","last_unstale":"2024-09-10T10:44:55.284887+0000","last_undegraded":"2024-09-10T10:44:55.284887+0000","last_fullsized":"2024-09-10T10:44:55.284887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263701+0000","last_change":"2024-09-10T10:44:55.263701+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263701+0000","last_undegraded":"2024-09-10T10:44:55.263701+0000","last_fullsized":"2024-09-10T10:44:55.263701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283525+0000","last_change":"2024-09-10T10:44:55.280125+0000","last_active":"2024-09-10T10:44:55.283525+0000","last_peered":"2024-09-10T10:44:55.283525+0000","last_clean":"2024-09-10T10:44:55.283525+0000","last_became_active":"2024-09-10T10:44:55.279856+0000","last_became_peered":"2024-09-10T10:44:55.279856+0000","last_unstale":"2024-09-10T10:44:55.283525+0000","last_undegraded":"2024-09-10T10:44:55.283525+0000","last_fullsized":"2024-09-10T10:44:55.283525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269283+0000","last_change":"2024-09-10T10:44:55.269283+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269283+0000","last_undegraded":"2024-09-10T10:44:55.269283+0000","last_fullsized":"2024-09-10T10:44:55.269283+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282972+0000","last_change":"2024-09-10T10:44:55.277393+0000","last_active":"2024-09-10T10:44:55.282972+0000","last_peered":"2024-09-10T10:44:55.282972+0000","last_clean":"2024-09-10T10:44:55.282972+0000","last_became_active":"2024-09-10T10:44:55.277116+0000","last_became_peered":"2024-09-10T10:44:55.277116+0000","last_unstale":"2024-09-10T10:44:55.282972+0000","last_undegraded":"2024-09-10T10:44:55.282972+0000","last_fullsized":"2024-09-10T10:44:55.282972+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:55.889822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266735+0000","last_change":"2024-09-10T10:44:55.266735+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266735+0000","last_undegraded":"2024-09-10T10:44:55.266735+0000","last_fullsized":"2024-09-10T10:44:55.266735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.277553+0000","last_change":"2024-09-10T10:44:55.274146+0000","last_active":"2024-09-10T10:44:55.277553+0000","last_peered":"2024-09-10T10:44:55.277553+0000","last_clean":"2024-09-10T10:44:55.277553+0000","last_became_active":"2024-09-10T10:44:55.273668+0000","last_became_peered":"2024-09-10T10:44:55.273668+0000","last_unstale":"2024-09-10T10:44:55.277553+0000","last_undegraded":"2024-09-10T10:44:55.277553+0000","last_fullsized":"2024-09-10T10:44:55.277553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270346+0000","last_change":"2024-09-10T10:44:55.270346+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270346+0000","last_undegraded":"2024-09-10T10:44:55.270346+0000","last_fullsized":"2024-09-10T10:44:55.270346+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287896+0000","last_change":"2024-09-10T10:44:55.285818+0000","last_active":"2024-09-10T10:44:55.287896+0000","last_peered":"2024-09-10T10:44:55.287896+0000","last_clean":"2024-09-10T10:44:55.287896+0000","last_became_active":"2024-09-10T10:44:55.285486+0000","last_became_peered":"2024-09-10T10:44:55.285486+0000","last_unstale":"2024-09-10T10:44:55.287896+0000","last_undegraded":"2024-09-10T10:44:55.287896+0000","last_fullsized":"2024-09-10T10:44:55.287896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:12.231875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262876+0000","last_change":"2024-09-10T10:44:55.262876+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262876+0000","last_undegraded":"2024-09-10T10:44:55.262876+0000","last_fullsized":"2024-09-10T10:44:55.262876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289989+0000","last_change":"2024-09-10T10:44:55.282228+0000","last_active":"2024-09-10T10:44:55.289989+0000","last_peered":"2024-09-10T10:44:55.289989+0000","last_clean":"2024-09-10T10:44:55.289989+0000","last_became_active":"2024-09-10T10:44:55.281403+0000","last_became_peered":"2024-09-10T10:44:55.281403+0000","last_unstale":"2024-09-10T10:44:55.289989+0000","last_undegraded":"2024-09-10T10:44:55.289989+0000","last_fullsized":"2024-09-10T10:44:55.289989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269496+0000","last_change":"2024-09-10T10:44:55.269496+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269496+0000","last_undegraded":"2024-09-10T10:44:55.269496+0000","last_fullsized":"2024-09-10T10:44:55.269496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283277+0000","last_change":"2024-09-10T10:44:55.280162+0000","last_active":"2024-09-10T10:44:55.283277+0000","last_peered":"2024-09-10T10:44:55.283277+0000","last_clean":"2024-09-10T10:44:55.283277+0000","last_became_active":"2024-09-10T10:44:55.279842+0000","last_became_peered":"2024-09-10T10:44:55.279842+0000","last_unstale":"2024-09-10T10:44:55.283277+0000","last_undegraded":"2024-09-10T10:44:55.283277+0000","last_fullsized":"2024-09-10T10:44:55.283277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:32.358738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265570+0000","last_change":"2024-09-10T10:44:55.265570+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265570+0000","last_undegraded":"2024-09-10T10:44:55.265570+0000","last_fullsized":"2024-09-10T10:44:55.265570+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.279575+0000","last_change":"2024-09-10T10:44:55.274128+0000","last_active":"2024-09-10T10:44:55.279575+0000","last_peered":"2024-09-10T10:44:55.279575+0000","last_clean":"2024-09-10T10:44:55.279575+0000","last_became_active":"2024-09-10T10:44:55.273849+0000","last_became_peered":"2024-09-10T10:44:55.273849+0000","last_unstale":"2024-09-10T10:44:55.279575+0000","last_undegraded":"2024-09-10T10:44:55.279575+0000","last_fullsized":"2024-09-10T10:44:55.279575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:40.204312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265381+0000","last_change":"2024-09-10T10:44:55.265381+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265381+0000","last_undegraded":"2024-09-10T10:44:55.265381+0000","last_fullsized":"2024-09-10T10:44:55.265381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286851+0000","last_change":"2024-09-10T10:44:55.280404+0000","last_active":"2024-09-10T10:44:55.286851+0000","last_peered":"2024-09-10T10:44:55.286851+0000","last_clean":"2024-09-10T10:44:55.286851+0000","last_became_active":"2024-09-10T10:44:55.277105+0000","last_became_peered":"2024-09-10T10:44:55.277105+0000","last_unstale":"2024-09-10T10:44:55.286851+0000","last_undegraded":"2024-09-10T10:44:55.286851+0000","last_fullsized":"2024-09-10T10:44:55.286851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262621+0000","last_change":"2024-09-10T10:44:55.262621+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262621+0000","last_undegraded":"2024-09-10T10:44:55.262621+0000","last_fullsized":"2024-09-10T10:44:55.262621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290065+0000","last_change":"2024-09-10T10:44:55.283497+0000","last_active":"2024-09-10T10:44:55.290065+0000","last_peered":"2024-09-10T10:44:55.290065+0000","last_clean":"2024-09-10T10:44:55.290065+0000","last_became_active":"2024-09-10T10:44:55.280421+0000","last_became_peered":"2024-09-10T10:44:55.280421+0000","last_unstale":"2024-09-10T10:44:55.290065+0000","last_undegraded":"2024-09-10T10:44:55.290065+0000","last_fullsized":"2024-09-10T10:44:55.290065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.465795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267728+0000","last_change":"2024-09-10T10:44:55.267728+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267728+0000","last_undegraded":"2024-09-10T10:44:55.267728+0000","last_fullsized":"2024-09-10T10:44:55.267728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282353+0000","last_change":"2024-09-10T10:44:55.279940+0000","last_active":"2024-09-10T10:44:55.282353+0000","last_peered":"2024-09-10T10:44:55.282353+0000","last_clean":"2024-09-10T10:44:55.282353+0000","last_became_active":"2024-09-10T10:44:55.279402+0000","last_became_peered":"2024-09-10T10:44:55.279402+0000","last_unstale":"2024-09-10T10:44:55.282353+0000","last_undegraded":"2024-09-10T10:44:55.282353+0000","last_fullsized":"2024-09-10T10:44:55.282353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270467+0000","last_change":"2024-09-10T10:44:55.270467+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270467+0000","last_undegraded":"2024-09-10T10:44:55.270467+0000","last_fullsized":"2024-09-10T10:44:55.270467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280174+0000","last_change":"2024-09-10T10:44:55.274369+0000","last_active":"2024-09-10T10:44:55.280174+0000","last_peered":"2024-09-10T10:44:55.280174+0000","last_clean":"2024-09-10T10:44:55.280174+0000","last_became_active":"2024-09-10T10:44:55.273965+0000","last_became_peered":"2024-09-10T10:44:55.273965+0000","last_unstale":"2024-09-10T10:44:55.280174+0000","last_undegraded":"2024-09-10T10:44:55.280174+0000","last_fullsized":"2024-09-10T10:44:55.280174+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:50.577611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267713+0000","last_change":"2024-09-10T10:44:55.267713+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267713+0000","last_undegraded":"2024-09-10T10:44:55.267713+0000","last_fullsized":"2024-09-10T10:44:55.267713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281549+0000","last_change":"2024-09-10T10:44:55.277981+0000","last_active":"2024-09-10T10:44:55.281549+0000","last_peered":"2024-09-10T10:44:55.281549+0000","last_clean":"2024-09-10T10:44:55.281549+0000","last_became_active":"2024-09-10T10:44:55.276409+0000","last_became_peered":"2024-09-10T10:44:55.276409+0000","last_unstale":"2024-09-10T10:44:55.281549+0000","last_undegraded":"2024-09-10T10:44:55.281549+0000","last_fullsized":"2024-09-10T10:44:55.281549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267163+0000","last_change":"2024-09-10T10:44:55.267163+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267163+0000","last_undegraded":"2024-09-10T10:44:55.267163+0000","last_fullsized":"2024-09-10T10:44:55.267163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288028+0000","last_change":"2024-09-10T10:44:55.286607+0000","last_active":"2024-09-10T10:44:55.288028+0000","last_peered":"2024-09-10T10:44:55.288028+0000","last_clean":"2024-09-10T10:44:55.288028+0000","last_became_active":"2024-09-10T10:44:55.284761+0000","last_became_peered":"2024-09-10T10:44:55.284761+0000","last_unstale":"2024-09-10T10:44:55.288028+0000","last_undegraded":"2024-09-10T10:44:55.288028+0000","last_fullsized":"2024-09-10T10:44:55.288028+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:57.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264579+0000","last_change":"2024-09-10T10:44:55.264579+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264579+0000","last_undegraded":"2024-09-10T10:44:55.264579+0000","last_fullsized":"2024-09-10T10:44:55.264579+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280029+0000","last_change":"2024-09-10T10:44:55.274312+0000","last_active":"2024-09-10T10:44:55.280029+0000","last_peered":"2024-09-10T10:44:55.280029+0000","last_clean":"2024-09-10T10:44:55.280029+0000","last_became_active":"2024-09-10T10:44:55.273980+0000","last_became_peered":"2024-09-10T10:44:55.273980+0000","last_unstale":"2024-09-10T10:44:55.280029+0000","last_undegraded":"2024-09-10T10:44:55.280029+0000","last_fullsized":"2024-09-10T10:44:55.280029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:42.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263789+0000","last_change":"2024-09-10T10:44:55.263789+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263789+0000","last_undegraded":"2024-09-10T10:44:55.263789+0000","last_fullsized":"2024-09-10T10:44:55.263789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288013+0000","last_change":"2024-09-10T10:44:55.286650+0000","last_active":"2024-09-10T10:44:55.288013+0000","last_peered":"2024-09-10T10:44:55.288013+0000","last_clean":"2024-09-10T10:44:55.288013+0000","last_became_active":"2024-09-10T10:44:55.286360+0000","last_became_peered":"2024-09-10T10:44:55.286360+0000","last_unstale":"2024-09-10T10:44:55.288013+0000","last_undegraded":"2024-09-10T10:44:55.288013+0000","last_fullsized":"2024-09-10T10:44:55.288013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:11.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:44:55.250795+0000","last_change":"2024-09-10T10:44:55.250795+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.250795+0000","last_undegraded":"2024-09-10T10:44:55.250795+0000","last_fullsized":"2024-09-10T10:44:55.250795+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:44:54.247366+0000","last_change":"2024-09-10T10:44:54.247366+0000","last_active":"2024-09-10T10:44:54.247366+0000","last_peered":"2024-09-10T10:44:54.247366+0000","last_clean":"2024-09-10T10:44:54.247366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:54.247366+0000","last_undegraded":"2024-09-10T10:44:54.247366+0000","last_fullsized":"2024-09-10T10:44:54.247366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265739+0000","last_change":"2024-09-10T10:44:55.265739+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265739+0000","last_undegraded":"2024-09-10T10:44:55.265739+0000","last_fullsized":"2024-09-10T10:44:55.265739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:44:54.247366+0000","last_change":"2024-09-10T10:44:54.247366+0000","last_active":"2024-09-10T10:44:54.247366+0000","last_peered":"2024-09-10T10:44:54.247366+0000","last_clean":"2024-09-10T10:44:54.247366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:54.247366+0000","last_undegraded":"2024-09-10T10:44:54.247366+0000","last_fullsized":"2024-09-10T10:44:54.247366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262657+0000","last_change":"2024-09-10T10:44:55.262657+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262657+0000","last_undegraded":"2024-09-10T10:44:55.262657+0000","last_fullsized":"2024-09-10T10:44:55.262657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287717+0000","last_change":"2024-09-10T10:44:55.283286+0000","last_active":"2024-09-10T10:44:55.287717+0000","last_peered":"2024-09-10T10:44:55.287717+0000","last_clean":"2024-09-10T10:44:55.287717+0000","last_became_active":"2024-09-10T10:44:55.283045+0000","last_became_peered":"2024-09-10T10:44:55.283045+0000","last_unstale":"2024-09-10T10:44:55.287717+0000","last_undegraded":"2024-09-10T10:44:55.287717+0000","last_fullsized":"2024-09-10T10:44:55.287717+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270979+0000","last_change":"2024-09-10T10:44:55.270979+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270979+0000","last_undegraded":"2024-09-10T10:44:55.270979+0000","last_fullsized":"2024-09-10T10:44:55.270979+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287041+0000","last_change":"2024-09-10T10:44:55.285243+0000","last_active":"2024-09-10T10:44:55.287041+0000","last_peered":"2024-09-10T10:44:55.287041+0000","last_clean":"2024-09-10T10:44:55.287041+0000","last_became_active":"2024-09-10T10:44:55.284621+0000","last_became_peered":"2024-09-10T10:44:55.284621+0000","last_unstale":"2024-09-10T10:44:55.287041+0000","last_undegraded":"2024-09-10T10:44:55.287041+0000","last_fullsized":"2024-09-10T10:44:55.287041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:03.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263025+0000","last_change":"2024-09-10T10:44:55.263025+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263025+0000","last_undegraded":"2024-09-10T10:44:55.263025+0000","last_fullsized":"2024-09-10T10:44:55.263025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282476+0000","last_change":"2024-09-10T10:44:55.279589+0000","last_active":"2024-09-10T10:44:55.282476+0000","last_peered":"2024-09-10T10:44:55.282476+0000","last_clean":"2024-09-10T10:44:55.282476+0000","last_became_active":"2024-09-10T10:44:55.279308+0000","last_became_peered":"2024-09-10T10:44:55.279308+0000","last_unstale":"2024-09-10T10:44:55.282476+0000","last_undegraded":"2024-09-10T10:44:55.282476+0000","last_fullsized":"2024-09-10T10:44:55.282476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270224+0000","last_change":"2024-09-10T10:44:55.270224+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270224+0000","last_undegraded":"2024-09-10T10:44:55.270224+0000","last_fullsized":"2024-09-10T10:44:55.270224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286054+0000","last_change":"2024-09-10T10:44:55.280439+0000","last_active":"2024-09-10T10:44:55.286054+0000","last_peered":"2024-09-10T10:44:55.286054+0000","last_clean":"2024-09-10T10:44:55.286054+0000","last_became_active":"2024-09-10T10:44:55.280054+0000","last_became_peered":"2024-09-10T10:44:55.280054+0000","last_unstale":"2024-09-10T10:44:55.286054+0000","last_undegraded":"2024-09-10T10:44:55.286054+0000","last_fullsized":"2024-09-10T10:44:55.286054+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:11.846566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271856+0000","last_change":"2024-09-10T10:44:55.271856+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271856+0000","last_undegraded":"2024-09-10T10:44:55.271856+0000","last_fullsized":"2024-09-10T10:44:55.271856+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289979+0000","last_change":"2024-09-10T10:44:55.287203+0000","last_active":"2024-09-10T10:44:55.289979+0000","last_peered":"2024-09-10T10:44:55.289979+0000","last_clean":"2024-09-10T10:44:55.289979+0000","last_became_active":"2024-09-10T10:44:55.286932+0000","last_became_peered":"2024-09-10T10:44:55.286932+0000","last_unstale":"2024-09-10T10:44:55.289979+0000","last_undegraded":"2024-09-10T10:44:55.289979+0000","last_fullsized":"2024-09-10T10:44:55.289979+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.346471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:44:55.250795+0000","last_change":"2024-09-10T10:44:55.250795+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.250795+0000","last_undegraded":"2024-09-10T10:44:55.250795+0000","last_fullsized":"2024-09-10T10:44:55.250795+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290007+0000","last_change":"2024-09-10T10:44:55.287602+0000","last_active":"2024-09-10T10:44:55.290007+0000","last_peered":"2024-09-10T10:44:55.290007+0000","last_clean":"2024-09-10T10:44:55.290007+0000","last_became_active":"2024-09-10T10:44:55.287354+0000","last_became_peered":"2024-09-10T10:44:55.287354+0000","last_unstale":"2024-09-10T10:44:55.290007+0000","last_undegraded":"2024-09-10T10:44:55.290007+0000","last_fullsized":"2024-09-10T10:44:55.290007+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:11.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268246+0000","last_change":"2024-09-10T10:44:55.268246+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268246+0000","last_undegraded":"2024-09-10T10:44:55.268246+0000","last_fullsized":"2024-09-10T10:44:55.268246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290050+0000","last_change":"2024-09-10T10:44:55.283449+0000","last_active":"2024-09-10T10:44:55.290050+0000","last_peered":"2024-09-10T10:44:55.290050+0000","last_clean":"2024-09-10T10:44:55.290050+0000","last_became_active":"2024-09-10T10:44:55.281725+0000","last_became_peered":"2024-09-10T10:44:55.281725+0000","last_unstale":"2024-09-10T10:44:55.290050+0000","last_undegraded":"2024-09-10T10:44:55.290050+0000","last_fullsized":"2024-09-10T10:44:55.290050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:25.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267128+0000","last_change":"2024-09-10T10:44:55.267128+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267128+0000","last_undegraded":"2024-09-10T10:44:55.267128+0000","last_fullsized":"2024-09-10T10:44:55.267128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286868+0000","last_change":"2024-09-10T10:44:55.281075+0000","last_active":"2024-09-10T10:44:55.286868+0000","last_peered":"2024-09-10T10:44:55.286868+0000","last_clean":"2024-09-10T10:44:55.286868+0000","last_became_active":"2024-09-10T10:44:55.280803+0000","last_became_peered":"2024-09-10T10:44:55.280803+0000","last_unstale":"2024-09-10T10:44:55.286868+0000","last_undegraded":"2024-09-10T10:44:55.286868+0000","last_fullsized":"2024-09-10T10:44:55.286868+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264966+0000","last_change":"2024-09-10T10:44:55.264966+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264966+0000","last_undegraded":"2024-09-10T10:44:55.264966+0000","last_fullsized":"2024-09-10T10:44:55.264966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.285925+0000","last_change":"2024-09-10T10:44:55.280515+0000","last_active":"2024-09-10T10:44:55.285925+0000","last_peered":"2024-09-10T10:44:55.285925+0000","last_clean":"2024-09-10T10:44:55.285925+0000","last_became_active":"2024-09-10T10:44:55.280197+0000","last_became_peered":"2024-09-10T10:44:55.280197+0000","last_unstale":"2024-09-10T10:44:55.285925+0000","last_undegraded":"2024-09-10T10:44:55.285925+0000","last_fullsized":"2024-09-10T10:44:55.285925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:51.824788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267360+0000","last_change":"2024-09-10T10:44:55.267360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267360+0000","last_undegraded":"2024-09-10T10:44:55.267360+0000","last_fullsized":"2024-09-10T10:44:55.267360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280708+0000","last_change":"2024-09-10T10:44:55.276223+0000","last_active":"2024-09-10T10:44:55.280708+0000","last_peered":"2024-09-10T10:44:55.280708+0000","last_clean":"2024-09-10T10:44:55.280708+0000","last_became_active":"2024-09-10T10:44:55.275926+0000","last_became_peered":"2024-09-10T10:44:55.275926+0000","last_unstale":"2024-09-10T10:44:55.280708+0000","last_undegraded":"2024-09-10T10:44:55.280708+0000","last_fullsized":"2024-09-10T10:44:55.280708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265809+0000","last_change":"2024-09-10T10:44:55.265809+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265809+0000","last_undegraded":"2024-09-10T10:44:55.265809+0000","last_fullsized":"2024-09-10T10:44:55.265809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287061+0000","last_change":"2024-09-10T10:44:55.284148+0000","last_active":"2024-09-10T10:44:55.287061+0000","last_peered":"2024-09-10T10:44:55.287061+0000","last_clean":"2024-09-10T10:44:55.287061+0000","last_became_active":"2024-09-10T10:44:55.283869+0000","last_became_peered":"2024-09-10T10:44:55.283869+0000","last_unstale":"2024-09-10T10:44:55.287061+0000","last_undegraded":"2024-09-10T10:44:55.287061+0000","last_fullsized":"2024-09-10T10:44:55.287061+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.263348+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:44:55.263348+0000","last_peered":"2024-09-10T10:44:55.263348+0000","last_clean":"2024-09-10T10:44:55.263348+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:44:55.263348+0000","last_undegraded":"2024-09-10T10:44:55.263348+0000","last_fullsized":"2024-09-10T10:44:55.263348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":122,"acting":122,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":782541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1024000,"data_stored":769665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:00.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:00.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.678+0000 7f3314600640 1 Processor -- start 2024-09-10T10:45:00.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.679+0000 7f3314600640 1 -- start start 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3314600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f330c161c80 0x7f330c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3314600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f330c0b6680 0x7f330c0cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3314600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c06e870 con 0x7f330c162060 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3314600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f330c06d9f0 con 0x7f330c0b6680 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3314600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f330c06dcd0 con 0x7f330c161c80 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3312600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f330c0b6680 0x7f330c0cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3311200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f330c162060 0x7f330c0b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56978/0 (socket says 172.21.15.28:56978) 2024-09-10T10:45:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3311200640 1 -- 172.21.15.28:0/3276047859 learned_addr learned my addr 172.21.15.28:0/3276047859 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3311c00640 1 --2- 172.21.15.28:0/3276047859 >> v2:172.21.15.138:3300/0 conn(0x7f330c161c80 0x7f330c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3312600640 1 -- 172.21.15.28:0/3276047859 >> v2:172.21.15.138:3300/0 conn(0x7f330c161c80 msgr2=0x7f330c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3312600640 1 --2- 172.21.15.28:0/3276047859 >> v2:172.21.15.138:3300/0 conn(0x7f330c161c80 0x7f330c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3312600640 1 -- 172.21.15.28:0/3276047859 >> v1:172.21.15.28:6789/0 conn(0x7f330c162060 legacy=0x7f330c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.680+0000 7f3312600640 1 -- 172.21.15.28:0/3276047859 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c06dfb0 con 0x7f330c0b6680 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.681+0000 7f3312600640 1 --2- 172.21.15.28:0/3276047859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c0b6680 0x7f330c0cf1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f330c08d7a0 tx=0x7f330402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef16fd6de05aa21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f330be00640 1 -- 172.21.15.28:0/3276047859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304007740 con 0x7f330c0b6680 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f330be00640 1 -- 172.21.15.28:0/3276047859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33040078e0 con 0x7f330c0b6680 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f330be00640 1 -- 172.21.15.28:0/3276047859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304007cb0 con 0x7f330c0b6680 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 -- 172.21.15.28:0/3276047859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c0b6680 msgr2=0x7f330c0cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 --2- 172.21.15.28:0/3276047859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c0b6680 0x7f330c0cf1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f330c08d7a0 tx=0x7f330402a680 comp rx=0 tx=0).stop 2024-09-10T10:45:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 -- 172.21.15.28:0/3276047859 shutdown_connections 2024-09-10T10:45:00.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 --2- 172.21.15.28:0/3276047859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c0b6680 0x7f330c0cf1e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 --2- 172.21.15.28:0/3276047859 >> v2:172.21.15.138:3300/0 conn(0x7f330c161c80 0x7f330c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 -- 172.21.15.28:0/3276047859 >> 172.21.15.28:0/3276047859 conn(0x7f330c0d44f0 msgr2=0x7f330c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 -- 172.21.15.28:0/3276047859 shutdown_connections 2024-09-10T10:45:00.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.682+0000 7f3314600640 1 -- 172.21.15.28:0/3276047859 wait complete. 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.684+0000 7f3314600640 1 Processor -- start 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.684+0000 7f3314600640 1 -- start start 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f330c0b6680 0x7f330c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f330c161c80 0x7f330c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0b6b30 con 0x7f330c162060 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f330c0d2aa0 con 0x7f330c0b6680 2024-09-10T10:45:00.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3314600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0b6cb0 con 0x7f330c161c80 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3312600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3312600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33656/0 (socket says 172.21.15.28:33656) 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3312600640 1 -- 172.21.15.28:0/1870844731 learned_addr learned my addr 172.21.15.28:0/1870844731 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3311c00640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f330c0b6680 0x7f330c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.685+0000 7f3311200640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f330c161c80 0x7f330c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 -- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f330c161c80 msgr2=0x7f330c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f330c161c80 0x7f330c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 -- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f330c0b6680 msgr2=0x7f330c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f330c0b6680 0x7f330c159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 -- 172.21.15.28:0/1870844731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c159fc0 con 0x7f330c162060 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3312600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3304009e50 tx=0x7f3304004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330403f070 con 0x7f330c162060 2024-09-10T10:45:00.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3304031bf0 con 0x7f330c162060 2024-09-10T10:45:00.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304044460 con 0x7f330c162060 2024-09-10T10:45:00.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c0ca010 con 0x7f330c162060 2024-09-10T10:45:00.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.686+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c0ca5d0 con 0x7f330c162060 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.688+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c06e330 con 0x7f330c162060 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.688+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3304007790 con 0x7f330c162060 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.689+0000 7f3309600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 0x7f32ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.689+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f33040ac720 con 0x7f330c162060 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.690+0000 7f3311c00640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 0x7f32ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:00.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.691+0000 7f3311c00640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 0x7f32ec06e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:00.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.694+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f330407a850 con 0x7f330c162060 2024-09-10T10:45:00.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.875+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f330c0ba470 con 0x7f32ec06be30 2024-09-10T10:45:00.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.882+0000 7f3309600640 1 -- 172.21.15.28:0/1870844731 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246896 (secure 0 0 0) 0x7f330c0ba470 con 0x7f32ec06be30 2024-09-10T10:45:00.885 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:00.886 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 msgr2=0x7f32ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 0x7f32ec06e2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300008040 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 msgr2=0x7f330c0c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3304009e50 tx=0x7f3304004a40 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 shutdown_connections 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f32ec06be30 0x7f32ec06e2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f330c162060 0x7f330c0c99e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f330c161c80 0x7f330c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 --2- 172.21.15.28:0/1870844731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f330c0b6680 0x7f330c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:00.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.885+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 >> 172.21.15.28:0/1870844731 conn(0x7f330c0d44f0 msgr2=0x7f330c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:00.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.886+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 shutdown_connections 2024-09-10T10:45:00.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:00.886+0000 7f3314600640 1 -- 172.21.15.28:0/1870844731 wait complete. 2024-09-10T10:45:00.903 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T10:45:00.499522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225128,"kb_used_data":4728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749592,"statfs":{"total":773094113280,"available":772863582208,"internally_reserved":0,"allocated":4841472,"data_stored":2725894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001215"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263853+0000","last_change":"2024-09-10T10:44:55.263853+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263853+0000","last_undegraded":"2024-09-10T10:44:55.263853+0000","last_fullsized":"2024-09-10T10:44:55.263853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271034+0000","last_change":"2024-09-10T10:44:55.271034+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271034+0000","last_undegraded":"2024-09-10T10:44:55.271034+0000","last_fullsized":"2024-09-10T10:44:55.271034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263908+0000","last_change":"2024-09-10T10:44:55.263908+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263908+0000","last_undegraded":"2024-09-10T10:44:55.263908+0000","last_fullsized":"2024-09-10T10:44:55.263908+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263405+0000","last_change":"2024-09-10T10:44:55.263405+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263405+0000","last_undegraded":"2024-09-10T10:44:55.263405+0000","last_fullsized":"2024-09-10T10:44:55.263405+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269251+0000","last_change":"2024-09-10T10:44:55.269251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269251+0000","last_undegraded":"2024-09-10T10:44:55.269251+0000","last_fullsized":"2024-09-10T10:44:55.269251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268012+0000","last_change":"2024-09-10T10:44:55.268012+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268012+0000","last_undegraded":"2024-09-10T10:44:55.268012+0000","last_fullsized":"2024-09-10T10:44:55.268012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264033+0000","last_change":"2024-09-10T10:44:55.264033+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264033+0000","last_undegraded":"2024-09-10T10:44:55.264033+0000","last_fullsized":"2024-09-10T10:44:55.264033+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265069+0000","last_change":"2024-09-10T10:44:55.265069+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265069+0000","last_undegraded":"2024-09-10T10:44:55.265069+0000","last_fullsized":"2024-09-10T10:44:55.265069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271830+0000","last_change":"2024-09-10T10:44:55.271830+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271830+0000","last_undegraded":"2024-09-10T10:44:55.271830+0000","last_fullsized":"2024-09-10T10:44:55.271830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266035+0000","last_change":"2024-09-10T10:44:55.266035+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266035+0000","last_undegraded":"2024-09-10T10:44:55.266035+0000","last_fullsized":"2024-09-10T10:44:55.266035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272854+0000","last_change":"2024-09-10T10:44:55.272854+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272854+0000","last_undegraded":"2024-09-10T10:44:55.272854+0000","last_fullsized":"2024-09-10T10:44:55.272854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270668+0000","last_change":"2024-09-10T10:44:55.270668+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270668+0000","last_undegraded":"2024-09-10T10:44:55.270668+0000","last_fullsized":"2024-09-10T10:44:55.270668+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263816+0000","last_change":"2024-09-10T10:44:55.263816+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263816+0000","last_undegraded":"2024-09-10T10:44:55.263816+0000","last_fullsized":"2024-09-10T10:44:55.263816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269525+0000","last_change":"2024-09-10T10:44:55.269525+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269525+0000","last_undegraded":"2024-09-10T10:44:55.269525+0000","last_fullsized":"2024-09-10T10:44:55.269525+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268281+0000","last_change":"2024-09-10T10:44:55.268281+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268281+0000","last_undegraded":"2024-09-10T10:44:55.268281+0000","last_fullsized":"2024-09-10T10:44:55.268281+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263005+0000","last_change":"2024-09-10T10:44:55.263005+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263005+0000","last_undegraded":"2024-09-10T10:44:55.263005+0000","last_fullsized":"2024-09-10T10:44:55.263005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267826+0000","last_change":"2024-09-10T10:44:55.267826+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267826+0000","last_undegraded":"2024-09-10T10:44:55.267826+0000","last_fullsized":"2024-09-10T10:44:55.267826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266794+0000","last_change":"2024-09-10T10:44:55.266794+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266794+0000","last_undegraded":"2024-09-10T10:44:55.266794+0000","last_fullsized":"2024-09-10T10:44:55.266794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280488+0000","last_change":"2024-09-10T10:44:56.293837+0000","last_active":"2024-09-10T10:44:57.280488+0000","last_peered":"2024-09-10T10:44:57.280488+0000","last_clean":"2024-09-10T10:44:57.280488+0000","last_became_active":"2024-09-10T10:44:56.293570+0000","last_became_peered":"2024-09-10T10:44:56.293570+0000","last_unstale":"2024-09-10T10:44:57.280488+0000","last_undegraded":"2024-09-10T10:44:57.280488+0000","last_fullsized":"2024-09-10T10:44:57.280488+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.685917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269860+0000","last_change":"2024-09-10T10:44:55.269860+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269860+0000","last_undegraded":"2024-09-10T10:44:55.269860+0000","last_fullsized":"2024-09-10T10:44:55.269860+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267879+0000","last_change":"2024-09-10T10:44:55.267879+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267879+0000","last_undegraded":"2024-09-10T10:44:55.267879+0000","last_fullsized":"2024-09-10T10:44:55.267879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.273400+0000","last_change":"2024-09-10T10:44:55.273400+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.273400+0000","last_undegraded":"2024-09-10T10:44:55.273400+0000","last_fullsized":"2024-09-10T10:44:55.273400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266722+0000","last_change":"2024-09-10T10:44:55.266722+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266722+0000","last_undegraded":"2024-09-10T10:44:55.266722+0000","last_fullsized":"2024-09-10T10:44:55.266722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264529+0000","last_change":"2024-09-10T10:44:55.264529+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264529+0000","last_undegraded":"2024-09-10T10:44:55.264529+0000","last_fullsized":"2024-09-10T10:44:55.264529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270779+0000","last_change":"2024-09-10T10:44:55.270779+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270779+0000","last_undegraded":"2024-09-10T10:44:55.270779+0000","last_fullsized":"2024-09-10T10:44:55.270779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265957+0000","last_change":"2024-09-10T10:44:55.265957+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265957+0000","last_undegraded":"2024-09-10T10:44:55.265957+0000","last_fullsized":"2024-09-10T10:44:55.265957+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272379+0000","last_change":"2024-09-10T10:44:55.272379+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272379+0000","last_undegraded":"2024-09-10T10:44:55.272379+0000","last_fullsized":"2024-09-10T10:44:55.272379+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263201+0000","last_change":"2024-09-10T10:44:55.263201+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263201+0000","last_undegraded":"2024-09-10T10:44:55.263201+0000","last_fullsized":"2024-09-10T10:44:55.263201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267251+0000","last_change":"2024-09-10T10:44:55.267251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267251+0000","last_undegraded":"2024-09-10T10:44:55.267251+0000","last_fullsized":"2024-09-10T10:44:55.267251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267256+0000","last_change":"2024-09-10T10:44:55.267256+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267256+0000","last_undegraded":"2024-09-10T10:44:55.267256+0000","last_fullsized":"2024-09-10T10:44:55.267256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263151+0000","last_change":"2024-09-10T10:44:55.263151+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263151+0000","last_undegraded":"2024-09-10T10:44:55.263151+0000","last_fullsized":"2024-09-10T10:44:55.263151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264106+0000","last_change":"2024-09-10T10:44:55.264106+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264106+0000","last_undegraded":"2024-09-10T10:44:55.264106+0000","last_fullsized":"2024-09-10T10:44:55.264106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266238+0000","last_change":"2024-09-10T10:44:55.266238+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266238+0000","last_undegraded":"2024-09-10T10:44:55.266238+0000","last_fullsized":"2024-09-10T10:44:55.266238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282941+0000","last_change":"2024-09-10T10:44:55.277458+0000","last_active":"2024-09-10T10:44:55.282941+0000","last_peered":"2024-09-10T10:44:55.282941+0000","last_clean":"2024-09-10T10:44:55.282941+0000","last_became_active":"2024-09-10T10:44:55.277216+0000","last_became_peered":"2024-09-10T10:44:55.277216+0000","last_unstale":"2024-09-10T10:44:55.282941+0000","last_undegraded":"2024-09-10T10:44:55.282941+0000","last_fullsized":"2024-09-10T10:44:55.282941+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268161+0000","last_change":"2024-09-10T10:44:55.268161+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268161+0000","last_undegraded":"2024-09-10T10:44:55.268161+0000","last_fullsized":"2024-09-10T10:44:55.268161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284405+0000","last_change":"2024-09-10T10:44:55.281076+0000","last_active":"2024-09-10T10:44:55.284405+0000","last_peered":"2024-09-10T10:44:55.284405+0000","last_clean":"2024-09-10T10:44:55.284405+0000","last_became_active":"2024-09-10T10:44:55.280788+0000","last_became_peered":"2024-09-10T10:44:55.280788+0000","last_unstale":"2024-09-10T10:44:55.284405+0000","last_undegraded":"2024-09-10T10:44:55.284405+0000","last_fullsized":"2024-09-10T10:44:55.284405+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266360+0000","last_change":"2024-09-10T10:44:55.266360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266360+0000","last_undegraded":"2024-09-10T10:44:55.266360+0000","last_fullsized":"2024-09-10T10:44:55.266360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281259+0000","last_change":"2024-09-10T10:44:55.273775+0000","last_active":"2024-09-10T10:44:55.281259+0000","last_peered":"2024-09-10T10:44:55.281259+0000","last_clean":"2024-09-10T10:44:55.281259+0000","last_became_active":"2024-09-10T10:44:55.273439+0000","last_became_peered":"2024-09-10T10:44:55.273439+0000","last_unstale":"2024-09-10T10:44:55.281259+0000","last_undegraded":"2024-09-10T10:44:55.281259+0000","last_fullsized":"2024-09-10T10:44:55.281259+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269513+0000","last_change":"2024-09-10T10:44:55.269513+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269513+0000","last_undegraded":"2024-09-10T10:44:55.269513+0000","last_fullsized":"2024-09-10T10:44:55.269513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284887+0000","last_change":"2024-09-10T10:44:55.282020+0000","last_active":"2024-09-10T10:44:55.284887+0000","last_peered":"2024-09-10T10:44:55.284887+0000","last_clean":"2024-09-10T10:44:55.284887+0000","last_became_active":"2024-09-10T10:44:55.281745+0000","last_became_peered":"2024-09-10T10:44:55.281745+0000","last_unstale":"2024-09-10T10:44:55.284887+0000","last_undegraded":"2024-09-10T10:44:55.284887+0000","last_fullsized":"2024-09-10T10:44:55.284887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263701+0000","last_change":"2024-09-10T10:44:55.263701+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263701+0000","last_undegraded":"2024-09-10T10:44:55.263701+0000","last_fullsized":"2024-09-10T10:44:55.263701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283525+0000","last_change":"2024-09-10T10:44:55.280125+0000","last_active":"2024-09-10T10:44:55.283525+0000","last_peered":"2024-09-10T10:44:55.283525+0000","last_clean":"2024-09-10T10:44:55.283525+0000","last_became_active":"2024-09-10T10:44:55.279856+0000","last_became_peered":"2024-09-10T10:44:55.279856+0000","last_unstale":"2024-09-10T10:44:55.283525+0000","last_undegraded":"2024-09-10T10:44:55.283525+0000","last_fullsized":"2024-09-10T10:44:55.283525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269283+0000","last_change":"2024-09-10T10:44:55.269283+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269283+0000","last_undegraded":"2024-09-10T10:44:55.269283+0000","last_fullsized":"2024-09-10T10:44:55.269283+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282972+0000","last_change":"2024-09-10T10:44:55.277393+0000","last_active":"2024-09-10T10:44:55.282972+0000","last_peered":"2024-09-10T10:44:55.282972+0000","last_clean":"2024-09-10T10:44:55.282972+0000","last_became_active":"2024-09-10T10:44:55.277116+0000","last_became_peered":"2024-09-10T10:44:55.277116+0000","last_unstale":"2024-09-10T10:44:55.282972+0000","last_undegraded":"2024-09-10T10:44:55.282972+0000","last_fullsized":"2024-09-10T10:44:55.282972+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:55.889822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266735+0000","last_change":"2024-09-10T10:44:55.266735+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266735+0000","last_undegraded":"2024-09-10T10:44:55.266735+0000","last_fullsized":"2024-09-10T10:44:55.266735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.277553+0000","last_change":"2024-09-10T10:44:55.274146+0000","last_active":"2024-09-10T10:44:55.277553+0000","last_peered":"2024-09-10T10:44:55.277553+0000","last_clean":"2024-09-10T10:44:55.277553+0000","last_became_active":"2024-09-10T10:44:55.273668+0000","last_became_peered":"2024-09-10T10:44:55.273668+0000","last_unstale":"2024-09-10T10:44:55.277553+0000","last_undegraded":"2024-09-10T10:44:55.277553+0000","last_fullsized":"2024-09-10T10:44:55.277553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270346+0000","last_change":"2024-09-10T10:44:55.270346+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270346+0000","last_undegraded":"2024-09-10T10:44:55.270346+0000","last_fullsized":"2024-09-10T10:44:55.270346+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287896+0000","last_change":"2024-09-10T10:44:55.285818+0000","last_active":"2024-09-10T10:44:55.287896+0000","last_peered":"2024-09-10T10:44:55.287896+0000","last_clean":"2024-09-10T10:44:55.287896+0000","last_became_active":"2024-09-10T10:44:55.285486+0000","last_became_peered":"2024-09-10T10:44:55.285486+0000","last_unstale":"2024-09-10T10:44:55.287896+0000","last_undegraded":"2024-09-10T10:44:55.287896+0000","last_fullsized":"2024-09-10T10:44:55.287896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:12.231875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262876+0000","last_change":"2024-09-10T10:44:55.262876+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262876+0000","last_undegraded":"2024-09-10T10:44:55.262876+0000","last_fullsized":"2024-09-10T10:44:55.262876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289989+0000","last_change":"2024-09-10T10:44:55.282228+0000","last_active":"2024-09-10T10:44:55.289989+0000","last_peered":"2024-09-10T10:44:55.289989+0000","last_clean":"2024-09-10T10:44:55.289989+0000","last_became_active":"2024-09-10T10:44:55.281403+0000","last_became_peered":"2024-09-10T10:44:55.281403+0000","last_unstale":"2024-09-10T10:44:55.289989+0000","last_undegraded":"2024-09-10T10:44:55.289989+0000","last_fullsized":"2024-09-10T10:44:55.289989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269496+0000","last_change":"2024-09-10T10:44:55.269496+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269496+0000","last_undegraded":"2024-09-10T10:44:55.269496+0000","last_fullsized":"2024-09-10T10:44:55.269496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283277+0000","last_change":"2024-09-10T10:44:55.280162+0000","last_active":"2024-09-10T10:44:55.283277+0000","last_peered":"2024-09-10T10:44:55.283277+0000","last_clean":"2024-09-10T10:44:55.283277+0000","last_became_active":"2024-09-10T10:44:55.279842+0000","last_became_peered":"2024-09-10T10:44:55.279842+0000","last_unstale":"2024-09-10T10:44:55.283277+0000","last_undegraded":"2024-09-10T10:44:55.283277+0000","last_fullsized":"2024-09-10T10:44:55.283277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:32.358738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265570+0000","last_change":"2024-09-10T10:44:55.265570+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265570+0000","last_undegraded":"2024-09-10T10:44:55.265570+0000","last_fullsized":"2024-09-10T10:44:55.265570+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.279575+0000","last_change":"2024-09-10T10:44:55.274128+0000","last_active":"2024-09-10T10:44:55.279575+0000","last_peered":"2024-09-10T10:44:55.279575+0000","last_clean":"2024-09-10T10:44:55.279575+0000","last_became_active":"2024-09-10T10:44:55.273849+0000","last_became_peered":"2024-09-10T10:44:55.273849+0000","last_unstale":"2024-09-10T10:44:55.279575+0000","last_undegraded":"2024-09-10T10:44:55.279575+0000","last_fullsized":"2024-09-10T10:44:55.279575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:40.204312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265381+0000","last_change":"2024-09-10T10:44:55.265381+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265381+0000","last_undegraded":"2024-09-10T10:44:55.265381+0000","last_fullsized":"2024-09-10T10:44:55.265381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286851+0000","last_change":"2024-09-10T10:44:55.280404+0000","last_active":"2024-09-10T10:44:55.286851+0000","last_peered":"2024-09-10T10:44:55.286851+0000","last_clean":"2024-09-10T10:44:55.286851+0000","last_became_active":"2024-09-10T10:44:55.277105+0000","last_became_peered":"2024-09-10T10:44:55.277105+0000","last_unstale":"2024-09-10T10:44:55.286851+0000","last_undegraded":"2024-09-10T10:44:55.286851+0000","last_fullsized":"2024-09-10T10:44:55.286851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262621+0000","last_change":"2024-09-10T10:44:55.262621+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262621+0000","last_undegraded":"2024-09-10T10:44:55.262621+0000","last_fullsized":"2024-09-10T10:44:55.262621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290065+0000","last_change":"2024-09-10T10:44:55.283497+0000","last_active":"2024-09-10T10:44:55.290065+0000","last_peered":"2024-09-10T10:44:55.290065+0000","last_clean":"2024-09-10T10:44:55.290065+0000","last_became_active":"2024-09-10T10:44:55.280421+0000","last_became_peered":"2024-09-10T10:44:55.280421+0000","last_unstale":"2024-09-10T10:44:55.290065+0000","last_undegraded":"2024-09-10T10:44:55.290065+0000","last_fullsized":"2024-09-10T10:44:55.290065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.465795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267728+0000","last_change":"2024-09-10T10:44:55.267728+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267728+0000","last_undegraded":"2024-09-10T10:44:55.267728+0000","last_fullsized":"2024-09-10T10:44:55.267728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282353+0000","last_change":"2024-09-10T10:44:55.279940+0000","last_active":"2024-09-10T10:44:55.282353+0000","last_peered":"2024-09-10T10:44:55.282353+0000","last_clean":"2024-09-10T10:44:55.282353+0000","last_became_active":"2024-09-10T10:44:55.279402+0000","last_became_peered":"2024-09-10T10:44:55.279402+0000","last_unstale":"2024-09-10T10:44:55.282353+0000","last_undegraded":"2024-09-10T10:44:55.282353+0000","last_fullsized":"2024-09-10T10:44:55.282353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270467+0000","last_change":"2024-09-10T10:44:55.270467+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270467+0000","last_undegraded":"2024-09-10T10:44:55.270467+0000","last_fullsized":"2024-09-10T10:44:55.270467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280174+0000","last_change":"2024-09-10T10:44:55.274369+0000","last_active":"2024-09-10T10:44:55.280174+0000","last_peered":"2024-09-10T10:44:55.280174+0000","last_clean":"2024-09-10T10:44:55.280174+0000","last_became_active":"2024-09-10T10:44:55.273965+0000","last_became_peered":"2024-09-10T10:44:55.273965+0000","last_unstale":"2024-09-10T10:44:55.280174+0000","last_undegraded":"2024-09-10T10:44:55.280174+0000","last_fullsized":"2024-09-10T10:44:55.280174+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:50.577611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267713+0000","last_change":"2024-09-10T10:44:55.267713+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267713+0000","last_undegraded":"2024-09-10T10:44:55.267713+0000","last_fullsized":"2024-09-10T10:44:55.267713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281549+0000","last_change":"2024-09-10T10:44:55.277981+0000","last_active":"2024-09-10T10:44:55.281549+0000","last_peered":"2024-09-10T10:44:55.281549+0000","last_clean":"2024-09-10T10:44:55.281549+0000","last_became_active":"2024-09-10T10:44:55.276409+0000","last_became_peered":"2024-09-10T10:44:55.276409+0000","last_unstale":"2024-09-10T10:44:55.281549+0000","last_undegraded":"2024-09-10T10:44:55.281549+0000","last_fullsized":"2024-09-10T10:44:55.281549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267163+0000","last_change":"2024-09-10T10:44:55.267163+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267163+0000","last_undegraded":"2024-09-10T10:44:55.267163+0000","last_fullsized":"2024-09-10T10:44:55.267163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288028+0000","last_change":"2024-09-10T10:44:55.286607+0000","last_active":"2024-09-10T10:44:55.288028+0000","last_peered":"2024-09-10T10:44:55.288028+0000","last_clean":"2024-09-10T10:44:55.288028+0000","last_became_active":"2024-09-10T10:44:55.284761+0000","last_became_peered":"2024-09-10T10:44:55.284761+0000","last_unstale":"2024-09-10T10:44:55.288028+0000","last_undegraded":"2024-09-10T10:44:55.288028+0000","last_fullsized":"2024-09-10T10:44:55.288028+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:57.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264579+0000","last_change":"2024-09-10T10:44:55.264579+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264579+0000","last_undegraded":"2024-09-10T10:44:55.264579+0000","last_fullsized":"2024-09-10T10:44:55.264579+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280029+0000","last_change":"2024-09-10T10:44:55.274312+0000","last_active":"2024-09-10T10:44:55.280029+0000","last_peered":"2024-09-10T10:44:55.280029+0000","last_clean":"2024-09-10T10:44:55.280029+0000","last_became_active":"2024-09-10T10:44:55.273980+0000","last_became_peered":"2024-09-10T10:44:55.273980+0000","last_unstale":"2024-09-10T10:44:55.280029+0000","last_undegraded":"2024-09-10T10:44:55.280029+0000","last_fullsized":"2024-09-10T10:44:55.280029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:42.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263789+0000","last_change":"2024-09-10T10:44:55.263789+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263789+0000","last_undegraded":"2024-09-10T10:44:55.263789+0000","last_fullsized":"2024-09-10T10:44:55.263789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288013+0000","last_change":"2024-09-10T10:44:55.286650+0000","last_active":"2024-09-10T10:44:55.288013+0000","last_peered":"2024-09-10T10:44:55.288013+0000","last_clean":"2024-09-10T10:44:55.288013+0000","last_became_active":"2024-09-10T10:44:55.286360+0000","last_became_peered":"2024-09-10T10:44:55.286360+0000","last_unstale":"2024-09-10T10:44:55.288013+0000","last_undegraded":"2024-09-10T10:44:55.288013+0000","last_fullsized":"2024-09-10T10:44:55.288013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:11.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280741+0000","last_change":"2024-09-10T10:44:56.288750+0000","last_active":"2024-09-10T10:44:57.280741+0000","last_peered":"2024-09-10T10:44:57.280741+0000","last_clean":"2024-09-10T10:44:57.280741+0000","last_became_active":"2024-09-10T10:44:56.288435+0000","last_became_peered":"2024-09-10T10:44:56.288435+0000","last_unstale":"2024-09-10T10:44:57.280741+0000","last_undegraded":"2024-09-10T10:44:57.280741+0000","last_fullsized":"2024-09-10T10:44:57.280741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:49.960568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280412+0000","last_change":"2024-09-10T10:44:55.278871+0000","last_active":"2024-09-10T10:44:57.280412+0000","last_peered":"2024-09-10T10:44:57.280412+0000","last_clean":"2024-09-10T10:44:57.280412+0000","last_became_active":"2024-09-10T10:44:55.278608+0000","last_became_peered":"2024-09-10T10:44:55.278608+0000","last_unstale":"2024-09-10T10:44:57.280412+0000","last_undegraded":"2024-09-10T10:44:57.280412+0000","last_fullsized":"2024-09-10T10:44:57.280412+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:47.361070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265739+0000","last_change":"2024-09-10T10:44:55.265739+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265739+0000","last_undegraded":"2024-09-10T10:44:55.265739+0000","last_fullsized":"2024-09-10T10:44:55.265739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280206+0000","last_change":"2024-09-10T10:44:55.277794+0000","last_active":"2024-09-10T10:44:57.280206+0000","last_peered":"2024-09-10T10:44:57.280206+0000","last_clean":"2024-09-10T10:44:57.280206+0000","last_became_active":"2024-09-10T10:44:55.277054+0000","last_became_peered":"2024-09-10T10:44:55.277054+0000","last_unstale":"2024-09-10T10:44:57.280206+0000","last_undegraded":"2024-09-10T10:44:57.280206+0000","last_fullsized":"2024-09-10T10:44:57.280206+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:29.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262657+0000","last_change":"2024-09-10T10:44:55.262657+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262657+0000","last_undegraded":"2024-09-10T10:44:55.262657+0000","last_fullsized":"2024-09-10T10:44:55.262657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287717+0000","last_change":"2024-09-10T10:44:55.283286+0000","last_active":"2024-09-10T10:44:55.287717+0000","last_peered":"2024-09-10T10:44:55.287717+0000","last_clean":"2024-09-10T10:44:55.287717+0000","last_became_active":"2024-09-10T10:44:55.283045+0000","last_became_peered":"2024-09-10T10:44:55.283045+0000","last_unstale":"2024-09-10T10:44:55.287717+0000","last_undegraded":"2024-09-10T10:44:55.287717+0000","last_fullsized":"2024-09-10T10:44:55.287717+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270979+0000","last_change":"2024-09-10T10:44:55.270979+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270979+0000","last_undegraded":"2024-09-10T10:44:55.270979+0000","last_fullsized":"2024-09-10T10:44:55.270979+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287041+0000","last_change":"2024-09-10T10:44:55.285243+0000","last_active":"2024-09-10T10:44:55.287041+0000","last_peered":"2024-09-10T10:44:55.287041+0000","last_clean":"2024-09-10T10:44:55.287041+0000","last_became_active":"2024-09-10T10:44:55.284621+0000","last_became_peered":"2024-09-10T10:44:55.284621+0000","last_unstale":"2024-09-10T10:44:55.287041+0000","last_undegraded":"2024-09-10T10:44:55.287041+0000","last_fullsized":"2024-09-10T10:44:55.287041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:03.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263025+0000","last_change":"2024-09-10T10:44:55.263025+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263025+0000","last_undegraded":"2024-09-10T10:44:55.263025+0000","last_fullsized":"2024-09-10T10:44:55.263025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282476+0000","last_change":"2024-09-10T10:44:55.279589+0000","last_active":"2024-09-10T10:44:55.282476+0000","last_peered":"2024-09-10T10:44:55.282476+0000","last_clean":"2024-09-10T10:44:55.282476+0000","last_became_active":"2024-09-10T10:44:55.279308+0000","last_became_peered":"2024-09-10T10:44:55.279308+0000","last_unstale":"2024-09-10T10:44:55.282476+0000","last_undegraded":"2024-09-10T10:44:55.282476+0000","last_fullsized":"2024-09-10T10:44:55.282476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270224+0000","last_change":"2024-09-10T10:44:55.270224+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270224+0000","last_undegraded":"2024-09-10T10:44:55.270224+0000","last_fullsized":"2024-09-10T10:44:55.270224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286054+0000","last_change":"2024-09-10T10:44:55.280439+0000","last_active":"2024-09-10T10:44:55.286054+0000","last_peered":"2024-09-10T10:44:55.286054+0000","last_clean":"2024-09-10T10:44:55.286054+0000","last_became_active":"2024-09-10T10:44:55.280054+0000","last_became_peered":"2024-09-10T10:44:55.280054+0000","last_unstale":"2024-09-10T10:44:55.286054+0000","last_undegraded":"2024-09-10T10:44:55.286054+0000","last_fullsized":"2024-09-10T10:44:55.286054+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:11.846566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271856+0000","last_change":"2024-09-10T10:44:55.271856+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271856+0000","last_undegraded":"2024-09-10T10:44:55.271856+0000","last_fullsized":"2024-09-10T10:44:55.271856+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289979+0000","last_change":"2024-09-10T10:44:55.287203+0000","last_active":"2024-09-10T10:44:55.289979+0000","last_peered":"2024-09-10T10:44:55.289979+0000","last_clean":"2024-09-10T10:44:55.289979+0000","last_became_active":"2024-09-10T10:44:55.286932+0000","last_became_peered":"2024-09-10T10:44:55.286932+0000","last_unstale":"2024-09-10T10:44:55.289979+0000","last_undegraded":"2024-09-10T10:44:55.289979+0000","last_fullsized":"2024-09-10T10:44:55.289979+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.346471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280685+0000","last_change":"2024-09-10T10:44:56.275769+0000","last_active":"2024-09-10T10:44:57.280685+0000","last_peered":"2024-09-10T10:44:57.280685+0000","last_clean":"2024-09-10T10:44:57.280685+0000","last_became_active":"2024-09-10T10:44:56.275397+0000","last_became_peered":"2024-09-10T10:44:56.275397+0000","last_unstale":"2024-09-10T10:44:57.280685+0000","last_undegraded":"2024-09-10T10:44:57.280685+0000","last_fullsized":"2024-09-10T10:44:57.280685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:36.768449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290007+0000","last_change":"2024-09-10T10:44:55.287602+0000","last_active":"2024-09-10T10:44:55.290007+0000","last_peered":"2024-09-10T10:44:55.290007+0000","last_clean":"2024-09-10T10:44:55.290007+0000","last_became_active":"2024-09-10T10:44:55.287354+0000","last_became_peered":"2024-09-10T10:44:55.287354+0000","last_unstale":"2024-09-10T10:44:55.290007+0000","last_undegraded":"2024-09-10T10:44:55.290007+0000","last_fullsized":"2024-09-10T10:44:55.290007+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:11.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268246+0000","last_change":"2024-09-10T10:44:55.268246+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268246+0000","last_undegraded":"2024-09-10T10:44:55.268246+0000","last_fullsized":"2024-09-10T10:44:55.268246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290050+0000","last_change":"2024-09-10T10:44:55.283449+0000","last_active":"2024-09-10T10:44:55.290050+0000","last_peered":"2024-09-10T10:44:55.290050+0000","last_clean":"2024-09-10T10:44:55.290050+0000","last_became_active":"2024-09-10T10:44:55.281725+0000","last_became_peered":"2024-09-10T10:44:55.281725+0000","last_unstale":"2024-09-10T10:44:55.290050+0000","last_undegraded":"2024-09-10T10:44:55.290050+0000","last_fullsized":"2024-09-10T10:44:55.290050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:25.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267128+0000","last_change":"2024-09-10T10:44:55.267128+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267128+0000","last_undegraded":"2024-09-10T10:44:55.267128+0000","last_fullsized":"2024-09-10T10:44:55.267128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286868+0000","last_change":"2024-09-10T10:44:55.281075+0000","last_active":"2024-09-10T10:44:55.286868+0000","last_peered":"2024-09-10T10:44:55.286868+0000","last_clean":"2024-09-10T10:44:55.286868+0000","last_became_active":"2024-09-10T10:44:55.280803+0000","last_became_peered":"2024-09-10T10:44:55.280803+0000","last_unstale":"2024-09-10T10:44:55.286868+0000","last_undegraded":"2024-09-10T10:44:55.286868+0000","last_fullsized":"2024-09-10T10:44:55.286868+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264966+0000","last_change":"2024-09-10T10:44:55.264966+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264966+0000","last_undegraded":"2024-09-10T10:44:55.264966+0000","last_fullsized":"2024-09-10T10:44:55.264966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.285925+0000","last_change":"2024-09-10T10:44:55.280515+0000","last_active":"2024-09-10T10:44:55.285925+0000","last_peered":"2024-09-10T10:44:55.285925+0000","last_clean":"2024-09-10T10:44:55.285925+0000","last_became_active":"2024-09-10T10:44:55.280197+0000","last_became_peered":"2024-09-10T10:44:55.280197+0000","last_unstale":"2024-09-10T10:44:55.285925+0000","last_undegraded":"2024-09-10T10:44:55.285925+0000","last_fullsized":"2024-09-10T10:44:55.285925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:51.824788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267360+0000","last_change":"2024-09-10T10:44:55.267360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267360+0000","last_undegraded":"2024-09-10T10:44:55.267360+0000","last_fullsized":"2024-09-10T10:44:55.267360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280708+0000","last_change":"2024-09-10T10:44:55.276223+0000","last_active":"2024-09-10T10:44:55.280708+0000","last_peered":"2024-09-10T10:44:55.280708+0000","last_clean":"2024-09-10T10:44:55.280708+0000","last_became_active":"2024-09-10T10:44:55.275926+0000","last_became_peered":"2024-09-10T10:44:55.275926+0000","last_unstale":"2024-09-10T10:44:55.280708+0000","last_undegraded":"2024-09-10T10:44:55.280708+0000","last_fullsized":"2024-09-10T10:44:55.280708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265809+0000","last_change":"2024-09-10T10:44:55.265809+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265809+0000","last_undegraded":"2024-09-10T10:44:55.265809+0000","last_fullsized":"2024-09-10T10:44:55.265809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287061+0000","last_change":"2024-09-10T10:44:55.284148+0000","last_active":"2024-09-10T10:44:55.287061+0000","last_peered":"2024-09-10T10:44:55.287061+0000","last_clean":"2024-09-10T10:44:55.287061+0000","last_became_active":"2024-09-10T10:44:55.283869+0000","last_became_peered":"2024-09-10T10:44:55.283869+0000","last_unstale":"2024-09-10T10:44:55.287061+0000","last_undegraded":"2024-09-10T10:44:55.287061+0000","last_fullsized":"2024-09-10T10:44:55.287061+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.263348+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:44:55.263348+0000","last_peered":"2024-09-10T10:44:55.263348+0000","last_clean":"2024-09-10T10:44:55.263348+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:44:55.263348+0000","last_undegraded":"2024-09-10T10:44:55.263348+0000","last_fullsized":"2024-09-10T10:44:55.263348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":782541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1077248,"data_stored":796982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:00.907 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:01.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.097+0000 7fd2d0000640 1 Processor -- start 2024-09-10T10:45:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.097+0000 7fd2d0000640 1 -- start start 2024-09-10T10:45:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2d0000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2c80ce940 0x7fd2c80cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2d0000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c815ce80 0x7fd2c8072b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2d0000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2c80b4f20 con 0x7fd2c80726b0 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2d0000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd2c8073050 con 0x7fd2c80ce940 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2d0000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c80cd630 con 0x7fd2c815ce80 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2c80ce940 0x7fd2c80cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2c80ce940 0x7fd2c80cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33668/0 (socket says 172.21.15.28:33668) 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 -- 172.21.15.28:0/3413173165 learned_addr learned my addr 172.21.15.28:0/3413173165 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2ce000640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c815ce80 0x7fd2c8072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 -- 172.21.15.28:0/3413173165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c815ce80 msgr2=0x7fd2c8072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c815ce80 0x7fd2c8072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 -- 172.21.15.28:0/3413173165 >> v1:172.21.15.138:6789/0 conn(0x7fd2c80726b0 legacy=0x7fd2c815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.099+0000 7fd2cd600640 1 -- 172.21.15.28:0/3413173165 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c815d350 con 0x7fd2c80ce940 2024-09-10T10:45:01.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.100+0000 7fd2cd600640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c80ce940 0x7fd2c80cdac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8071c70 tx=0x7fd2bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1213987b44cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2c7e00640 1 -- 172.21.15.28:0/3413173165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc037070 con 0x7fd2c80ce940 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2c7e00640 1 -- 172.21.15.28:0/3413173165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc032440 con 0x7fd2c80ce940 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2c7e00640 1 -- 172.21.15.28:0/3413173165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc032810 con 0x7fd2c80ce940 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 -- 172.21.15.28:0/3413173165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c80ce940 msgr2=0x7fd2c80cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c80ce940 0x7fd2c80cdac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8071c70 tx=0x7fd2bc02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 -- 172.21.15.28:0/3413173165 shutdown_connections 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c815ce80 0x7fd2c8072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 --2- 172.21.15.28:0/3413173165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c80ce940 0x7fd2c80cdac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 -- 172.21.15.28:0/3413173165 >> 172.21.15.28:0/3413173165 conn(0x7fd2c806c030 msgr2=0x7fd2c806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 -- 172.21.15.28:0/3413173165 shutdown_connections 2024-09-10T10:45:01.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.101+0000 7fd2d0000640 1 -- 172.21.15.28:0/3413173165 wait complete. 2024-09-10T10:45:01.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.103+0000 7fd2d0000640 1 Processor -- start 2024-09-10T10:45:01.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.103+0000 7fd2d0000640 1 -- start start 2024-09-10T10:45:01.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c80726b0 0x7fd2c8158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c80ce940 0x7fd2c8150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c80cd2c0 con 0x7fd2c815ce80 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c80ccfe0 con 0x7fd2c80ce940 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2d0000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c80ced20 con 0x7fd2c80726b0 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2ce000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2ce000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33680/0 (socket says 172.21.15.28:33680) 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2ce000640 1 -- 172.21.15.28:0/1151170780 learned_addr learned my addr 172.21.15.28:0/1151170780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2cd600640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c80726b0 0x7fd2c8158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.104+0000 7fd2ccc00640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c80ce940 0x7fd2c8150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 -- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c80726b0 msgr2=0x7fd2c8158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c80726b0 0x7fd2c8158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 -- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c80ce940 msgr2=0x7fd2c8150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c80ce940 0x7fd2c8150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 -- 172.21.15.28:0/1151170780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c8159340 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2ce000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009fd0 tx=0x7fd2c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0014420 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0019070 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0014800 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c8159620 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.105+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c8159000 con 0x7fd2c815ce80 2024-09-10T10:45:01.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.107+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c80b58d0 con 0x7fd2c815ce80 2024-09-10T10:45:01.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.107+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c00149a0 con 0x7fd2c815ce80 2024-09-10T10:45:01.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.108+0000 7fd2c5600640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 0x7fd2a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.108+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fd2c008b7e0 con 0x7fd2c815ce80 2024-09-10T10:45:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.109+0000 7fd2cd600640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 0x7fd2a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.110+0000 7fd2cd600640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 0x7fd2a806e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc034000 tx=0x7fd2bc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.113+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd2c80b58d0 con 0x7fd2c815ce80 2024-09-10T10:45:01.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.294+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2c806e2f0 con 0x7fd2a806be30 2024-09-10T10:45:01.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.301+0000 7fd2c5600640 1 -- 172.21.15.28:0/1151170780 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246896 (secure 0 0 0) 0x7fd2c806e2f0 con 0x7fd2a806be30 2024-09-10T10:45:01.305 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:01.306 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 msgr2=0x7fd2a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 0x7fd2a806e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc034000 tx=0x7fd2bc02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 msgr2=0x7fd2c81e6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009fd0 tx=0x7fd2c000e4b0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 shutdown_connections 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2a806be30 0x7fd2a806e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2c815ce80 0x7fd2c81e6bf0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2c80ce940 0x7fd2c8150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 --2- 172.21.15.28:0/1151170780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2c80726b0 0x7fd2c8158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 >> 172.21.15.28:0/1151170780 conn(0x7fd2c806c030 msgr2=0x7fd2c80b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 shutdown_connections 2024-09-10T10:45:01.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.305+0000 7fd2d0000640 1 -- 172.21.15.28:0/1151170780 wait complete. 2024-09-10T10:45:01.323 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T10:45:00.499522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225128,"kb_used_data":4728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749592,"statfs":{"total":773094113280,"available":772863582208,"internally_reserved":0,"allocated":4841472,"data_stored":2725894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001215"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263853+0000","last_change":"2024-09-10T10:44:55.263853+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263853+0000","last_undegraded":"2024-09-10T10:44:55.263853+0000","last_fullsized":"2024-09-10T10:44:55.263853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271034+0000","last_change":"2024-09-10T10:44:55.271034+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271034+0000","last_undegraded":"2024-09-10T10:44:55.271034+0000","last_fullsized":"2024-09-10T10:44:55.271034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263908+0000","last_change":"2024-09-10T10:44:55.263908+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263908+0000","last_undegraded":"2024-09-10T10:44:55.263908+0000","last_fullsized":"2024-09-10T10:44:55.263908+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263405+0000","last_change":"2024-09-10T10:44:55.263405+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263405+0000","last_undegraded":"2024-09-10T10:44:55.263405+0000","last_fullsized":"2024-09-10T10:44:55.263405+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269251+0000","last_change":"2024-09-10T10:44:55.269251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269251+0000","last_undegraded":"2024-09-10T10:44:55.269251+0000","last_fullsized":"2024-09-10T10:44:55.269251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268012+0000","last_change":"2024-09-10T10:44:55.268012+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268012+0000","last_undegraded":"2024-09-10T10:44:55.268012+0000","last_fullsized":"2024-09-10T10:44:55.268012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264033+0000","last_change":"2024-09-10T10:44:55.264033+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264033+0000","last_undegraded":"2024-09-10T10:44:55.264033+0000","last_fullsized":"2024-09-10T10:44:55.264033+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265069+0000","last_change":"2024-09-10T10:44:55.265069+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265069+0000","last_undegraded":"2024-09-10T10:44:55.265069+0000","last_fullsized":"2024-09-10T10:44:55.265069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271830+0000","last_change":"2024-09-10T10:44:55.271830+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271830+0000","last_undegraded":"2024-09-10T10:44:55.271830+0000","last_fullsized":"2024-09-10T10:44:55.271830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266035+0000","last_change":"2024-09-10T10:44:55.266035+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266035+0000","last_undegraded":"2024-09-10T10:44:55.266035+0000","last_fullsized":"2024-09-10T10:44:55.266035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272854+0000","last_change":"2024-09-10T10:44:55.272854+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272854+0000","last_undegraded":"2024-09-10T10:44:55.272854+0000","last_fullsized":"2024-09-10T10:44:55.272854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270668+0000","last_change":"2024-09-10T10:44:55.270668+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270668+0000","last_undegraded":"2024-09-10T10:44:55.270668+0000","last_fullsized":"2024-09-10T10:44:55.270668+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263816+0000","last_change":"2024-09-10T10:44:55.263816+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263816+0000","last_undegraded":"2024-09-10T10:44:55.263816+0000","last_fullsized":"2024-09-10T10:44:55.263816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269525+0000","last_change":"2024-09-10T10:44:55.269525+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269525+0000","last_undegraded":"2024-09-10T10:44:55.269525+0000","last_fullsized":"2024-09-10T10:44:55.269525+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268281+0000","last_change":"2024-09-10T10:44:55.268281+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268281+0000","last_undegraded":"2024-09-10T10:44:55.268281+0000","last_fullsized":"2024-09-10T10:44:55.268281+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263005+0000","last_change":"2024-09-10T10:44:55.263005+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263005+0000","last_undegraded":"2024-09-10T10:44:55.263005+0000","last_fullsized":"2024-09-10T10:44:55.263005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267826+0000","last_change":"2024-09-10T10:44:55.267826+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267826+0000","last_undegraded":"2024-09-10T10:44:55.267826+0000","last_fullsized":"2024-09-10T10:44:55.267826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266794+0000","last_change":"2024-09-10T10:44:55.266794+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266794+0000","last_undegraded":"2024-09-10T10:44:55.266794+0000","last_fullsized":"2024-09-10T10:44:55.266794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280488+0000","last_change":"2024-09-10T10:44:56.293837+0000","last_active":"2024-09-10T10:44:57.280488+0000","last_peered":"2024-09-10T10:44:57.280488+0000","last_clean":"2024-09-10T10:44:57.280488+0000","last_became_active":"2024-09-10T10:44:56.293570+0000","last_became_peered":"2024-09-10T10:44:56.293570+0000","last_unstale":"2024-09-10T10:44:57.280488+0000","last_undegraded":"2024-09-10T10:44:57.280488+0000","last_fullsized":"2024-09-10T10:44:57.280488+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.685917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269860+0000","last_change":"2024-09-10T10:44:55.269860+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269860+0000","last_undegraded":"2024-09-10T10:44:55.269860+0000","last_fullsized":"2024-09-10T10:44:55.269860+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267879+0000","last_change":"2024-09-10T10:44:55.267879+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267879+0000","last_undegraded":"2024-09-10T10:44:55.267879+0000","last_fullsized":"2024-09-10T10:44:55.267879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.273400+0000","last_change":"2024-09-10T10:44:55.273400+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.273400+0000","last_undegraded":"2024-09-10T10:44:55.273400+0000","last_fullsized":"2024-09-10T10:44:55.273400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266722+0000","last_change":"2024-09-10T10:44:55.266722+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266722+0000","last_undegraded":"2024-09-10T10:44:55.266722+0000","last_fullsized":"2024-09-10T10:44:55.266722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264529+0000","last_change":"2024-09-10T10:44:55.264529+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264529+0000","last_undegraded":"2024-09-10T10:44:55.264529+0000","last_fullsized":"2024-09-10T10:44:55.264529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270779+0000","last_change":"2024-09-10T10:44:55.270779+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270779+0000","last_undegraded":"2024-09-10T10:44:55.270779+0000","last_fullsized":"2024-09-10T10:44:55.270779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265957+0000","last_change":"2024-09-10T10:44:55.265957+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265957+0000","last_undegraded":"2024-09-10T10:44:55.265957+0000","last_fullsized":"2024-09-10T10:44:55.265957+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272379+0000","last_change":"2024-09-10T10:44:55.272379+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272379+0000","last_undegraded":"2024-09-10T10:44:55.272379+0000","last_fullsized":"2024-09-10T10:44:55.272379+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263201+0000","last_change":"2024-09-10T10:44:55.263201+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263201+0000","last_undegraded":"2024-09-10T10:44:55.263201+0000","last_fullsized":"2024-09-10T10:44:55.263201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267251+0000","last_change":"2024-09-10T10:44:55.267251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267251+0000","last_undegraded":"2024-09-10T10:44:55.267251+0000","last_fullsized":"2024-09-10T10:44:55.267251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267256+0000","last_change":"2024-09-10T10:44:55.267256+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267256+0000","last_undegraded":"2024-09-10T10:44:55.267256+0000","last_fullsized":"2024-09-10T10:44:55.267256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263151+0000","last_change":"2024-09-10T10:44:55.263151+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263151+0000","last_undegraded":"2024-09-10T10:44:55.263151+0000","last_fullsized":"2024-09-10T10:44:55.263151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264106+0000","last_change":"2024-09-10T10:44:55.264106+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264106+0000","last_undegraded":"2024-09-10T10:44:55.264106+0000","last_fullsized":"2024-09-10T10:44:55.264106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266238+0000","last_change":"2024-09-10T10:44:55.266238+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266238+0000","last_undegraded":"2024-09-10T10:44:55.266238+0000","last_fullsized":"2024-09-10T10:44:55.266238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282941+0000","last_change":"2024-09-10T10:44:55.277458+0000","last_active":"2024-09-10T10:44:55.282941+0000","last_peered":"2024-09-10T10:44:55.282941+0000","last_clean":"2024-09-10T10:44:55.282941+0000","last_became_active":"2024-09-10T10:44:55.277216+0000","last_became_peered":"2024-09-10T10:44:55.277216+0000","last_unstale":"2024-09-10T10:44:55.282941+0000","last_undegraded":"2024-09-10T10:44:55.282941+0000","last_fullsized":"2024-09-10T10:44:55.282941+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268161+0000","last_change":"2024-09-10T10:44:55.268161+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268161+0000","last_undegraded":"2024-09-10T10:44:55.268161+0000","last_fullsized":"2024-09-10T10:44:55.268161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284405+0000","last_change":"2024-09-10T10:44:55.281076+0000","last_active":"2024-09-10T10:44:55.284405+0000","last_peered":"2024-09-10T10:44:55.284405+0000","last_clean":"2024-09-10T10:44:55.284405+0000","last_became_active":"2024-09-10T10:44:55.280788+0000","last_became_peered":"2024-09-10T10:44:55.280788+0000","last_unstale":"2024-09-10T10:44:55.284405+0000","last_undegraded":"2024-09-10T10:44:55.284405+0000","last_fullsized":"2024-09-10T10:44:55.284405+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266360+0000","last_change":"2024-09-10T10:44:55.266360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266360+0000","last_undegraded":"2024-09-10T10:44:55.266360+0000","last_fullsized":"2024-09-10T10:44:55.266360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281259+0000","last_change":"2024-09-10T10:44:55.273775+0000","last_active":"2024-09-10T10:44:55.281259+0000","last_peered":"2024-09-10T10:44:55.281259+0000","last_clean":"2024-09-10T10:44:55.281259+0000","last_became_active":"2024-09-10T10:44:55.273439+0000","last_became_peered":"2024-09-10T10:44:55.273439+0000","last_unstale":"2024-09-10T10:44:55.281259+0000","last_undegraded":"2024-09-10T10:44:55.281259+0000","last_fullsized":"2024-09-10T10:44:55.281259+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269513+0000","last_change":"2024-09-10T10:44:55.269513+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269513+0000","last_undegraded":"2024-09-10T10:44:55.269513+0000","last_fullsized":"2024-09-10T10:44:55.269513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284887+0000","last_change":"2024-09-10T10:44:55.282020+0000","last_active":"2024-09-10T10:44:55.284887+0000","last_peered":"2024-09-10T10:44:55.284887+0000","last_clean":"2024-09-10T10:44:55.284887+0000","last_became_active":"2024-09-10T10:44:55.281745+0000","last_became_peered":"2024-09-10T10:44:55.281745+0000","last_unstale":"2024-09-10T10:44:55.284887+0000","last_undegraded":"2024-09-10T10:44:55.284887+0000","last_fullsized":"2024-09-10T10:44:55.284887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263701+0000","last_change":"2024-09-10T10:44:55.263701+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263701+0000","last_undegraded":"2024-09-10T10:44:55.263701+0000","last_fullsized":"2024-09-10T10:44:55.263701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283525+0000","last_change":"2024-09-10T10:44:55.280125+0000","last_active":"2024-09-10T10:44:55.283525+0000","last_peered":"2024-09-10T10:44:55.283525+0000","last_clean":"2024-09-10T10:44:55.283525+0000","last_became_active":"2024-09-10T10:44:55.279856+0000","last_became_peered":"2024-09-10T10:44:55.279856+0000","last_unstale":"2024-09-10T10:44:55.283525+0000","last_undegraded":"2024-09-10T10:44:55.283525+0000","last_fullsized":"2024-09-10T10:44:55.283525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269283+0000","last_change":"2024-09-10T10:44:55.269283+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269283+0000","last_undegraded":"2024-09-10T10:44:55.269283+0000","last_fullsized":"2024-09-10T10:44:55.269283+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282972+0000","last_change":"2024-09-10T10:44:55.277393+0000","last_active":"2024-09-10T10:44:55.282972+0000","last_peered":"2024-09-10T10:44:55.282972+0000","last_clean":"2024-09-10T10:44:55.282972+0000","last_became_active":"2024-09-10T10:44:55.277116+0000","last_became_peered":"2024-09-10T10:44:55.277116+0000","last_unstale":"2024-09-10T10:44:55.282972+0000","last_undegraded":"2024-09-10T10:44:55.282972+0000","last_fullsized":"2024-09-10T10:44:55.282972+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:55.889822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266735+0000","last_change":"2024-09-10T10:44:55.266735+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266735+0000","last_undegraded":"2024-09-10T10:44:55.266735+0000","last_fullsized":"2024-09-10T10:44:55.266735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.277553+0000","last_change":"2024-09-10T10:44:55.274146+0000","last_active":"2024-09-10T10:44:55.277553+0000","last_peered":"2024-09-10T10:44:55.277553+0000","last_clean":"2024-09-10T10:44:55.277553+0000","last_became_active":"2024-09-10T10:44:55.273668+0000","last_became_peered":"2024-09-10T10:44:55.273668+0000","last_unstale":"2024-09-10T10:44:55.277553+0000","last_undegraded":"2024-09-10T10:44:55.277553+0000","last_fullsized":"2024-09-10T10:44:55.277553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270346+0000","last_change":"2024-09-10T10:44:55.270346+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270346+0000","last_undegraded":"2024-09-10T10:44:55.270346+0000","last_fullsized":"2024-09-10T10:44:55.270346+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287896+0000","last_change":"2024-09-10T10:44:55.285818+0000","last_active":"2024-09-10T10:44:55.287896+0000","last_peered":"2024-09-10T10:44:55.287896+0000","last_clean":"2024-09-10T10:44:55.287896+0000","last_became_active":"2024-09-10T10:44:55.285486+0000","last_became_peered":"2024-09-10T10:44:55.285486+0000","last_unstale":"2024-09-10T10:44:55.287896+0000","last_undegraded":"2024-09-10T10:44:55.287896+0000","last_fullsized":"2024-09-10T10:44:55.287896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:12.231875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262876+0000","last_change":"2024-09-10T10:44:55.262876+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262876+0000","last_undegraded":"2024-09-10T10:44:55.262876+0000","last_fullsized":"2024-09-10T10:44:55.262876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289989+0000","last_change":"2024-09-10T10:44:55.282228+0000","last_active":"2024-09-10T10:44:55.289989+0000","last_peered":"2024-09-10T10:44:55.289989+0000","last_clean":"2024-09-10T10:44:55.289989+0000","last_became_active":"2024-09-10T10:44:55.281403+0000","last_became_peered":"2024-09-10T10:44:55.281403+0000","last_unstale":"2024-09-10T10:44:55.289989+0000","last_undegraded":"2024-09-10T10:44:55.289989+0000","last_fullsized":"2024-09-10T10:44:55.289989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269496+0000","last_change":"2024-09-10T10:44:55.269496+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269496+0000","last_undegraded":"2024-09-10T10:44:55.269496+0000","last_fullsized":"2024-09-10T10:44:55.269496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283277+0000","last_change":"2024-09-10T10:44:55.280162+0000","last_active":"2024-09-10T10:44:55.283277+0000","last_peered":"2024-09-10T10:44:55.283277+0000","last_clean":"2024-09-10T10:44:55.283277+0000","last_became_active":"2024-09-10T10:44:55.279842+0000","last_became_peered":"2024-09-10T10:44:55.279842+0000","last_unstale":"2024-09-10T10:44:55.283277+0000","last_undegraded":"2024-09-10T10:44:55.283277+0000","last_fullsized":"2024-09-10T10:44:55.283277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:32.358738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265570+0000","last_change":"2024-09-10T10:44:55.265570+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265570+0000","last_undegraded":"2024-09-10T10:44:55.265570+0000","last_fullsized":"2024-09-10T10:44:55.265570+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.279575+0000","last_change":"2024-09-10T10:44:55.274128+0000","last_active":"2024-09-10T10:44:55.279575+0000","last_peered":"2024-09-10T10:44:55.279575+0000","last_clean":"2024-09-10T10:44:55.279575+0000","last_became_active":"2024-09-10T10:44:55.273849+0000","last_became_peered":"2024-09-10T10:44:55.273849+0000","last_unstale":"2024-09-10T10:44:55.279575+0000","last_undegraded":"2024-09-10T10:44:55.279575+0000","last_fullsized":"2024-09-10T10:44:55.279575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:40.204312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265381+0000","last_change":"2024-09-10T10:44:55.265381+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265381+0000","last_undegraded":"2024-09-10T10:44:55.265381+0000","last_fullsized":"2024-09-10T10:44:55.265381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286851+0000","last_change":"2024-09-10T10:44:55.280404+0000","last_active":"2024-09-10T10:44:55.286851+0000","last_peered":"2024-09-10T10:44:55.286851+0000","last_clean":"2024-09-10T10:44:55.286851+0000","last_became_active":"2024-09-10T10:44:55.277105+0000","last_became_peered":"2024-09-10T10:44:55.277105+0000","last_unstale":"2024-09-10T10:44:55.286851+0000","last_undegraded":"2024-09-10T10:44:55.286851+0000","last_fullsized":"2024-09-10T10:44:55.286851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262621+0000","last_change":"2024-09-10T10:44:55.262621+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262621+0000","last_undegraded":"2024-09-10T10:44:55.262621+0000","last_fullsized":"2024-09-10T10:44:55.262621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290065+0000","last_change":"2024-09-10T10:44:55.283497+0000","last_active":"2024-09-10T10:44:55.290065+0000","last_peered":"2024-09-10T10:44:55.290065+0000","last_clean":"2024-09-10T10:44:55.290065+0000","last_became_active":"2024-09-10T10:44:55.280421+0000","last_became_peered":"2024-09-10T10:44:55.280421+0000","last_unstale":"2024-09-10T10:44:55.290065+0000","last_undegraded":"2024-09-10T10:44:55.290065+0000","last_fullsized":"2024-09-10T10:44:55.290065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.465795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267728+0000","last_change":"2024-09-10T10:44:55.267728+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267728+0000","last_undegraded":"2024-09-10T10:44:55.267728+0000","last_fullsized":"2024-09-10T10:44:55.267728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282353+0000","last_change":"2024-09-10T10:44:55.279940+0000","last_active":"2024-09-10T10:44:55.282353+0000","last_peered":"2024-09-10T10:44:55.282353+0000","last_clean":"2024-09-10T10:44:55.282353+0000","last_became_active":"2024-09-10T10:44:55.279402+0000","last_became_peered":"2024-09-10T10:44:55.279402+0000","last_unstale":"2024-09-10T10:44:55.282353+0000","last_undegraded":"2024-09-10T10:44:55.282353+0000","last_fullsized":"2024-09-10T10:44:55.282353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270467+0000","last_change":"2024-09-10T10:44:55.270467+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270467+0000","last_undegraded":"2024-09-10T10:44:55.270467+0000","last_fullsized":"2024-09-10T10:44:55.270467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280174+0000","last_change":"2024-09-10T10:44:55.274369+0000","last_active":"2024-09-10T10:44:55.280174+0000","last_peered":"2024-09-10T10:44:55.280174+0000","last_clean":"2024-09-10T10:44:55.280174+0000","last_became_active":"2024-09-10T10:44:55.273965+0000","last_became_peered":"2024-09-10T10:44:55.273965+0000","last_unstale":"2024-09-10T10:44:55.280174+0000","last_undegraded":"2024-09-10T10:44:55.280174+0000","last_fullsized":"2024-09-10T10:44:55.280174+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:50.577611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267713+0000","last_change":"2024-09-10T10:44:55.267713+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267713+0000","last_undegraded":"2024-09-10T10:44:55.267713+0000","last_fullsized":"2024-09-10T10:44:55.267713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281549+0000","last_change":"2024-09-10T10:44:55.277981+0000","last_active":"2024-09-10T10:44:55.281549+0000","last_peered":"2024-09-10T10:44:55.281549+0000","last_clean":"2024-09-10T10:44:55.281549+0000","last_became_active":"2024-09-10T10:44:55.276409+0000","last_became_peered":"2024-09-10T10:44:55.276409+0000","last_unstale":"2024-09-10T10:44:55.281549+0000","last_undegraded":"2024-09-10T10:44:55.281549+0000","last_fullsized":"2024-09-10T10:44:55.281549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267163+0000","last_change":"2024-09-10T10:44:55.267163+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267163+0000","last_undegraded":"2024-09-10T10:44:55.267163+0000","last_fullsized":"2024-09-10T10:44:55.267163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288028+0000","last_change":"2024-09-10T10:44:55.286607+0000","last_active":"2024-09-10T10:44:55.288028+0000","last_peered":"2024-09-10T10:44:55.288028+0000","last_clean":"2024-09-10T10:44:55.288028+0000","last_became_active":"2024-09-10T10:44:55.284761+0000","last_became_peered":"2024-09-10T10:44:55.284761+0000","last_unstale":"2024-09-10T10:44:55.288028+0000","last_undegraded":"2024-09-10T10:44:55.288028+0000","last_fullsized":"2024-09-10T10:44:55.288028+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:57.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264579+0000","last_change":"2024-09-10T10:44:55.264579+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264579+0000","last_undegraded":"2024-09-10T10:44:55.264579+0000","last_fullsized":"2024-09-10T10:44:55.264579+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280029+0000","last_change":"2024-09-10T10:44:55.274312+0000","last_active":"2024-09-10T10:44:55.280029+0000","last_peered":"2024-09-10T10:44:55.280029+0000","last_clean":"2024-09-10T10:44:55.280029+0000","last_became_active":"2024-09-10T10:44:55.273980+0000","last_became_peered":"2024-09-10T10:44:55.273980+0000","last_unstale":"2024-09-10T10:44:55.280029+0000","last_undegraded":"2024-09-10T10:44:55.280029+0000","last_fullsized":"2024-09-10T10:44:55.280029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:42.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263789+0000","last_change":"2024-09-10T10:44:55.263789+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263789+0000","last_undegraded":"2024-09-10T10:44:55.263789+0000","last_fullsized":"2024-09-10T10:44:55.263789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288013+0000","last_change":"2024-09-10T10:44:55.286650+0000","last_active":"2024-09-10T10:44:55.288013+0000","last_peered":"2024-09-10T10:44:55.288013+0000","last_clean":"2024-09-10T10:44:55.288013+0000","last_became_active":"2024-09-10T10:44:55.286360+0000","last_became_peered":"2024-09-10T10:44:55.286360+0000","last_unstale":"2024-09-10T10:44:55.288013+0000","last_undegraded":"2024-09-10T10:44:55.288013+0000","last_fullsized":"2024-09-10T10:44:55.288013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:11.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280741+0000","last_change":"2024-09-10T10:44:56.288750+0000","last_active":"2024-09-10T10:44:57.280741+0000","last_peered":"2024-09-10T10:44:57.280741+0000","last_clean":"2024-09-10T10:44:57.280741+0000","last_became_active":"2024-09-10T10:44:56.288435+0000","last_became_peered":"2024-09-10T10:44:56.288435+0000","last_unstale":"2024-09-10T10:44:57.280741+0000","last_undegraded":"2024-09-10T10:44:57.280741+0000","last_fullsized":"2024-09-10T10:44:57.280741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:49.960568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280412+0000","last_change":"2024-09-10T10:44:55.278871+0000","last_active":"2024-09-10T10:44:57.280412+0000","last_peered":"2024-09-10T10:44:57.280412+0000","last_clean":"2024-09-10T10:44:57.280412+0000","last_became_active":"2024-09-10T10:44:55.278608+0000","last_became_peered":"2024-09-10T10:44:55.278608+0000","last_unstale":"2024-09-10T10:44:57.280412+0000","last_undegraded":"2024-09-10T10:44:57.280412+0000","last_fullsized":"2024-09-10T10:44:57.280412+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:47.361070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265739+0000","last_change":"2024-09-10T10:44:55.265739+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265739+0000","last_undegraded":"2024-09-10T10:44:55.265739+0000","last_fullsized":"2024-09-10T10:44:55.265739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280206+0000","last_change":"2024-09-10T10:44:55.277794+0000","last_active":"2024-09-10T10:44:57.280206+0000","last_peered":"2024-09-10T10:44:57.280206+0000","last_clean":"2024-09-10T10:44:57.280206+0000","last_became_active":"2024-09-10T10:44:55.277054+0000","last_became_peered":"2024-09-10T10:44:55.277054+0000","last_unstale":"2024-09-10T10:44:57.280206+0000","last_undegraded":"2024-09-10T10:44:57.280206+0000","last_fullsized":"2024-09-10T10:44:57.280206+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:29.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262657+0000","last_change":"2024-09-10T10:44:55.262657+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262657+0000","last_undegraded":"2024-09-10T10:44:55.262657+0000","last_fullsized":"2024-09-10T10:44:55.262657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287717+0000","last_change":"2024-09-10T10:44:55.283286+0000","last_active":"2024-09-10T10:44:55.287717+0000","last_peered":"2024-09-10T10:44:55.287717+0000","last_clean":"2024-09-10T10:44:55.287717+0000","last_became_active":"2024-09-10T10:44:55.283045+0000","last_became_peered":"2024-09-10T10:44:55.283045+0000","last_unstale":"2024-09-10T10:44:55.287717+0000","last_undegraded":"2024-09-10T10:44:55.287717+0000","last_fullsized":"2024-09-10T10:44:55.287717+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270979+0000","last_change":"2024-09-10T10:44:55.270979+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270979+0000","last_undegraded":"2024-09-10T10:44:55.270979+0000","last_fullsized":"2024-09-10T10:44:55.270979+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287041+0000","last_change":"2024-09-10T10:44:55.285243+0000","last_active":"2024-09-10T10:44:55.287041+0000","last_peered":"2024-09-10T10:44:55.287041+0000","last_clean":"2024-09-10T10:44:55.287041+0000","last_became_active":"2024-09-10T10:44:55.284621+0000","last_became_peered":"2024-09-10T10:44:55.284621+0000","last_unstale":"2024-09-10T10:44:55.287041+0000","last_undegraded":"2024-09-10T10:44:55.287041+0000","last_fullsized":"2024-09-10T10:44:55.287041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:03.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263025+0000","last_change":"2024-09-10T10:44:55.263025+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263025+0000","last_undegraded":"2024-09-10T10:44:55.263025+0000","last_fullsized":"2024-09-10T10:44:55.263025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282476+0000","last_change":"2024-09-10T10:44:55.279589+0000","last_active":"2024-09-10T10:44:55.282476+0000","last_peered":"2024-09-10T10:44:55.282476+0000","last_clean":"2024-09-10T10:44:55.282476+0000","last_became_active":"2024-09-10T10:44:55.279308+0000","last_became_peered":"2024-09-10T10:44:55.279308+0000","last_unstale":"2024-09-10T10:44:55.282476+0000","last_undegraded":"2024-09-10T10:44:55.282476+0000","last_fullsized":"2024-09-10T10:44:55.282476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270224+0000","last_change":"2024-09-10T10:44:55.270224+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270224+0000","last_undegraded":"2024-09-10T10:44:55.270224+0000","last_fullsized":"2024-09-10T10:44:55.270224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286054+0000","last_change":"2024-09-10T10:44:55.280439+0000","last_active":"2024-09-10T10:44:55.286054+0000","last_peered":"2024-09-10T10:44:55.286054+0000","last_clean":"2024-09-10T10:44:55.286054+0000","last_became_active":"2024-09-10T10:44:55.280054+0000","last_became_peered":"2024-09-10T10:44:55.280054+0000","last_unstale":"2024-09-10T10:44:55.286054+0000","last_undegraded":"2024-09-10T10:44:55.286054+0000","last_fullsized":"2024-09-10T10:44:55.286054+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:11.846566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271856+0000","last_change":"2024-09-10T10:44:55.271856+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271856+0000","last_undegraded":"2024-09-10T10:44:55.271856+0000","last_fullsized":"2024-09-10T10:44:55.271856+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289979+0000","last_change":"2024-09-10T10:44:55.287203+0000","last_active":"2024-09-10T10:44:55.289979+0000","last_peered":"2024-09-10T10:44:55.289979+0000","last_clean":"2024-09-10T10:44:55.289979+0000","last_became_active":"2024-09-10T10:44:55.286932+0000","last_became_peered":"2024-09-10T10:44:55.286932+0000","last_unstale":"2024-09-10T10:44:55.289979+0000","last_undegraded":"2024-09-10T10:44:55.289979+0000","last_fullsized":"2024-09-10T10:44:55.289979+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.346471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280685+0000","last_change":"2024-09-10T10:44:56.275769+0000","last_active":"2024-09-10T10:44:57.280685+0000","last_peered":"2024-09-10T10:44:57.280685+0000","last_clean":"2024-09-10T10:44:57.280685+0000","last_became_active":"2024-09-10T10:44:56.275397+0000","last_became_peered":"2024-09-10T10:44:56.275397+0000","last_unstale":"2024-09-10T10:44:57.280685+0000","last_undegraded":"2024-09-10T10:44:57.280685+0000","last_fullsized":"2024-09-10T10:44:57.280685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:36.768449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290007+0000","last_change":"2024-09-10T10:44:55.287602+0000","last_active":"2024-09-10T10:44:55.290007+0000","last_peered":"2024-09-10T10:44:55.290007+0000","last_clean":"2024-09-10T10:44:55.290007+0000","last_became_active":"2024-09-10T10:44:55.287354+0000","last_became_peered":"2024-09-10T10:44:55.287354+0000","last_unstale":"2024-09-10T10:44:55.290007+0000","last_undegraded":"2024-09-10T10:44:55.290007+0000","last_fullsized":"2024-09-10T10:44:55.290007+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:11.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268246+0000","last_change":"2024-09-10T10:44:55.268246+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268246+0000","last_undegraded":"2024-09-10T10:44:55.268246+0000","last_fullsized":"2024-09-10T10:44:55.268246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290050+0000","last_change":"2024-09-10T10:44:55.283449+0000","last_active":"2024-09-10T10:44:55.290050+0000","last_peered":"2024-09-10T10:44:55.290050+0000","last_clean":"2024-09-10T10:44:55.290050+0000","last_became_active":"2024-09-10T10:44:55.281725+0000","last_became_peered":"2024-09-10T10:44:55.281725+0000","last_unstale":"2024-09-10T10:44:55.290050+0000","last_undegraded":"2024-09-10T10:44:55.290050+0000","last_fullsized":"2024-09-10T10:44:55.290050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:25.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267128+0000","last_change":"2024-09-10T10:44:55.267128+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267128+0000","last_undegraded":"2024-09-10T10:44:55.267128+0000","last_fullsized":"2024-09-10T10:44:55.267128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286868+0000","last_change":"2024-09-10T10:44:55.281075+0000","last_active":"2024-09-10T10:44:55.286868+0000","last_peered":"2024-09-10T10:44:55.286868+0000","last_clean":"2024-09-10T10:44:55.286868+0000","last_became_active":"2024-09-10T10:44:55.280803+0000","last_became_peered":"2024-09-10T10:44:55.280803+0000","last_unstale":"2024-09-10T10:44:55.286868+0000","last_undegraded":"2024-09-10T10:44:55.286868+0000","last_fullsized":"2024-09-10T10:44:55.286868+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264966+0000","last_change":"2024-09-10T10:44:55.264966+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264966+0000","last_undegraded":"2024-09-10T10:44:55.264966+0000","last_fullsized":"2024-09-10T10:44:55.264966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.285925+0000","last_change":"2024-09-10T10:44:55.280515+0000","last_active":"2024-09-10T10:44:55.285925+0000","last_peered":"2024-09-10T10:44:55.285925+0000","last_clean":"2024-09-10T10:44:55.285925+0000","last_became_active":"2024-09-10T10:44:55.280197+0000","last_became_peered":"2024-09-10T10:44:55.280197+0000","last_unstale":"2024-09-10T10:44:55.285925+0000","last_undegraded":"2024-09-10T10:44:55.285925+0000","last_fullsized":"2024-09-10T10:44:55.285925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:51.824788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267360+0000","last_change":"2024-09-10T10:44:55.267360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267360+0000","last_undegraded":"2024-09-10T10:44:55.267360+0000","last_fullsized":"2024-09-10T10:44:55.267360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280708+0000","last_change":"2024-09-10T10:44:55.276223+0000","last_active":"2024-09-10T10:44:55.280708+0000","last_peered":"2024-09-10T10:44:55.280708+0000","last_clean":"2024-09-10T10:44:55.280708+0000","last_became_active":"2024-09-10T10:44:55.275926+0000","last_became_peered":"2024-09-10T10:44:55.275926+0000","last_unstale":"2024-09-10T10:44:55.280708+0000","last_undegraded":"2024-09-10T10:44:55.280708+0000","last_fullsized":"2024-09-10T10:44:55.280708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265809+0000","last_change":"2024-09-10T10:44:55.265809+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265809+0000","last_undegraded":"2024-09-10T10:44:55.265809+0000","last_fullsized":"2024-09-10T10:44:55.265809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287061+0000","last_change":"2024-09-10T10:44:55.284148+0000","last_active":"2024-09-10T10:44:55.287061+0000","last_peered":"2024-09-10T10:44:55.287061+0000","last_clean":"2024-09-10T10:44:55.287061+0000","last_became_active":"2024-09-10T10:44:55.283869+0000","last_became_peered":"2024-09-10T10:44:55.283869+0000","last_unstale":"2024-09-10T10:44:55.287061+0000","last_undegraded":"2024-09-10T10:44:55.287061+0000","last_fullsized":"2024-09-10T10:44:55.287061+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.263348+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:44:55.263348+0000","last_peered":"2024-09-10T10:44:55.263348+0000","last_clean":"2024-09-10T10:44:55.263348+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:44:55.263348+0000","last_undegraded":"2024-09-10T10:44:55.263348+0000","last_fullsized":"2024-09-10T10:44:55.263348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":782541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1077248,"data_stored":796982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:01.327 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-09-10T10:45:01.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263853+0000', 'last_change': '2024-09-10T10:44:55.263853+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263853+0000', 'last_undegraded': '2024-09-10T10:44:55.263853+0000', 'last_fullsized': '2024-09-10T10:44:55.263853+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.327 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-09-10T10:45:01.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.271034+0000', 'last_change': '2024-09-10T10:44:55.271034+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.271034+0000', 'last_undegraded': '2024-09-10T10:44:55.271034+0000', 'last_fullsized': '2024-09-10T10:44:55.271034+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.327 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263908+0000', 'last_change': '2024-09-10T10:44:55.263908+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263908+0000', 'last_undegraded': '2024-09-10T10:44:55.263908+0000', 'last_fullsized': '2024-09-10T10:44:55.263908+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263405+0000', 'last_change': '2024-09-10T10:44:55.263405+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263405+0000', 'last_undegraded': '2024-09-10T10:44:55.263405+0000', 'last_fullsized': '2024-09-10T10:44:55.263405+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269251+0000', 'last_change': '2024-09-10T10:44:55.269251+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269251+0000', 'last_undegraded': '2024-09-10T10:44:55.269251+0000', 'last_fullsized': '2024-09-10T10:44:55.269251+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.268012+0000', 'last_change': '2024-09-10T10:44:55.268012+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.268012+0000', 'last_undegraded': '2024-09-10T10:44:55.268012+0000', 'last_fullsized': '2024-09-10T10:44:55.268012+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-10T10:45:01.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.264033+0000', 'last_change': '2024-09-10T10:44:55.264033+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.264033+0000', 'last_undegraded': '2024-09-10T10:44:55.264033+0000', 'last_fullsized': '2024-09-10T10:44:55.264033+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265069+0000', 'last_change': '2024-09-10T10:44:55.265069+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265069+0000', 'last_undegraded': '2024-09-10T10:44:55.265069+0000', 'last_fullsized': '2024-09-10T10:44:55.265069+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.271830+0000', 'last_change': '2024-09-10T10:44:55.271830+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.271830+0000', 'last_undegraded': '2024-09-10T10:44:55.271830+0000', 'last_fullsized': '2024-09-10T10:44:55.271830+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266035+0000', 'last_change': '2024-09-10T10:44:55.266035+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266035+0000', 'last_undegraded': '2024-09-10T10:44:55.266035+0000', 'last_fullsized': '2024-09-10T10:44:55.266035+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.272854+0000', 'last_change': '2024-09-10T10:44:55.272854+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.272854+0000', 'last_undegraded': '2024-09-10T10:44:55.272854+0000', 'last_fullsized': '2024-09-10T10:44:55.272854+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-10T10:45:01.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270668+0000', 'last_change': '2024-09-10T10:44:55.270668+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270668+0000', 'last_undegraded': '2024-09-10T10:44:55.270668+0000', 'last_fullsized': '2024-09-10T10:44:55.270668+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263816+0000', 'last_change': '2024-09-10T10:44:55.263816+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263816+0000', 'last_undegraded': '2024-09-10T10:44:55.263816+0000', 'last_fullsized': '2024-09-10T10:44:55.263816+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269525+0000', 'last_change': '2024-09-10T10:44:55.269525+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269525+0000', 'last_undegraded': '2024-09-10T10:44:55.269525+0000', 'last_fullsized': '2024-09-10T10:44:55.269525+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.268281+0000', 'last_change': '2024-09-10T10:44:55.268281+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.268281+0000', 'last_undegraded': '2024-09-10T10:44:55.268281+0000', 'last_fullsized': '2024-09-10T10:44:55.268281+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263005+0000', 'last_change': '2024-09-10T10:44:55.263005+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263005+0000', 'last_undegraded': '2024-09-10T10:44:55.263005+0000', 'last_fullsized': '2024-09-10T10:44:55.263005+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267826+0000', 'last_change': '2024-09-10T10:44:55.267826+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267826+0000', 'last_undegraded': '2024-09-10T10:44:55.267826+0000', 'last_fullsized': '2024-09-10T10:44:55.267826+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.330 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266794+0000', 'last_change': '2024-09-10T10:44:55.266794+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266794+0000', 'last_undegraded': '2024-09-10T10:44:55.266794+0000', 'last_fullsized': '2024-09-10T10:44:55.266794+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269860+0000', 'last_change': '2024-09-10T10:44:55.269860+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269860+0000', 'last_undegraded': '2024-09-10T10:44:55.269860+0000', 'last_fullsized': '2024-09-10T10:44:55.269860+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267879+0000', 'last_change': '2024-09-10T10:44:55.267879+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267879+0000', 'last_undegraded': '2024-09-10T10:44:55.267879+0000', 'last_fullsized': '2024-09-10T10:44:55.267879+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.273400+0000', 'last_change': '2024-09-10T10:44:55.273400+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.273400+0000', 'last_undegraded': '2024-09-10T10:44:55.273400+0000', 'last_fullsized': '2024-09-10T10:44:55.273400+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266722+0000', 'last_change': '2024-09-10T10:44:55.266722+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266722+0000', 'last_undegraded': '2024-09-10T10:44:55.266722+0000', 'last_fullsized': '2024-09-10T10:44:55.266722+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-10T10:45:01.331 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.264529+0000', 'last_change': '2024-09-10T10:44:55.264529+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.264529+0000', 'last_undegraded': '2024-09-10T10:44:55.264529+0000', 'last_fullsized': '2024-09-10T10:44:55.264529+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270779+0000', 'last_change': '2024-09-10T10:44:55.270779+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270779+0000', 'last_undegraded': '2024-09-10T10:44:55.270779+0000', 'last_fullsized': '2024-09-10T10:44:55.270779+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265957+0000', 'last_change': '2024-09-10T10:44:55.265957+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265957+0000', 'last_undegraded': '2024-09-10T10:44:55.265957+0000', 'last_fullsized': '2024-09-10T10:44:55.265957+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.272379+0000', 'last_change': '2024-09-10T10:44:55.272379+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.272379+0000', 'last_undegraded': '2024-09-10T10:44:55.272379+0000', 'last_fullsized': '2024-09-10T10:44:55.272379+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263201+0000', 'last_change': '2024-09-10T10:44:55.263201+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263201+0000', 'last_undegraded': '2024-09-10T10:44:55.263201+0000', 'last_fullsized': '2024-09-10T10:44:55.263201+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-10T10:45:01.332 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267251+0000', 'last_change': '2024-09-10T10:44:55.267251+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267251+0000', 'last_undegraded': '2024-09-10T10:44:55.267251+0000', 'last_fullsized': '2024-09-10T10:44:55.267251+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267256+0000', 'last_change': '2024-09-10T10:44:55.267256+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267256+0000', 'last_undegraded': '2024-09-10T10:44:55.267256+0000', 'last_fullsized': '2024-09-10T10:44:55.267256+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263151+0000', 'last_change': '2024-09-10T10:44:55.263151+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263151+0000', 'last_undegraded': '2024-09-10T10:44:55.263151+0000', 'last_fullsized': '2024-09-10T10:44:55.263151+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.264106+0000', 'last_change': '2024-09-10T10:44:55.264106+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.264106+0000', 'last_undegraded': '2024-09-10T10:44:55.264106+0000', 'last_fullsized': '2024-09-10T10:44:55.264106+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266238+0000', 'last_change': '2024-09-10T10:44:55.266238+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266238+0000', 'last_undegraded': '2024-09-10T10:44:55.266238+0000', 'last_fullsized': '2024-09-10T10:44:55.266238+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.268161+0000', 'last_change': '2024-09-10T10:44:55.268161+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.268161+0000', 'last_undegraded': '2024-09-10T10:44:55.268161+0000', 'last_fullsized': '2024-09-10T10:44:55.268161+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.333 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266360+0000', 'last_change': '2024-09-10T10:44:55.266360+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266360+0000', 'last_undegraded': '2024-09-10T10:44:55.266360+0000', 'last_fullsized': '2024-09-10T10:44:55.266360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269513+0000', 'last_change': '2024-09-10T10:44:55.269513+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269513+0000', 'last_undegraded': '2024-09-10T10:44:55.269513+0000', 'last_fullsized': '2024-09-10T10:44:55.269513+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263701+0000', 'last_change': '2024-09-10T10:44:55.263701+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263701+0000', 'last_undegraded': '2024-09-10T10:44:55.263701+0000', 'last_fullsized': '2024-09-10T10:44:55.263701+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269283+0000', 'last_change': '2024-09-10T10:44:55.269283+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269283+0000', 'last_undegraded': '2024-09-10T10:44:55.269283+0000', 'last_fullsized': '2024-09-10T10:44:55.269283+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.266735+0000', 'last_change': '2024-09-10T10:44:55.266735+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.266735+0000', 'last_undegraded': '2024-09-10T10:44:55.266735+0000', 'last_fullsized': '2024-09-10T10:44:55.266735+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-10T10:45:01.334 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270346+0000', 'last_change': '2024-09-10T10:44:55.270346+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270346+0000', 'last_undegraded': '2024-09-10T10:44:55.270346+0000', 'last_fullsized': '2024-09-10T10:44:55.270346+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.262876+0000', 'last_change': '2024-09-10T10:44:55.262876+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.262876+0000', 'last_undegraded': '2024-09-10T10:44:55.262876+0000', 'last_fullsized': '2024-09-10T10:44:55.262876+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.269496+0000', 'last_change': '2024-09-10T10:44:55.269496+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.269496+0000', 'last_undegraded': '2024-09-10T10:44:55.269496+0000', 'last_fullsized': '2024-09-10T10:44:55.269496+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265570+0000', 'last_change': '2024-09-10T10:44:55.265570+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265570+0000', 'last_undegraded': '2024-09-10T10:44:55.265570+0000', 'last_fullsized': '2024-09-10T10:44:55.265570+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265381+0000', 'last_change': '2024-09-10T10:44:55.265381+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265381+0000', 'last_undegraded': '2024-09-10T10:44:55.265381+0000', 'last_fullsized': '2024-09-10T10:44:55.265381+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-10T10:45:01.335 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.262621+0000', 'last_change': '2024-09-10T10:44:55.262621+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.262621+0000', 'last_undegraded': '2024-09-10T10:44:55.262621+0000', 'last_fullsized': '2024-09-10T10:44:55.262621+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267728+0000', 'last_change': '2024-09-10T10:44:55.267728+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267728+0000', 'last_undegraded': '2024-09-10T10:44:55.267728+0000', 'last_fullsized': '2024-09-10T10:44:55.267728+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270467+0000', 'last_change': '2024-09-10T10:44:55.270467+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270467+0000', 'last_undegraded': '2024-09-10T10:44:55.270467+0000', 'last_fullsized': '2024-09-10T10:44:55.270467+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267713+0000', 'last_change': '2024-09-10T10:44:55.267713+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267713+0000', 'last_undegraded': '2024-09-10T10:44:55.267713+0000', 'last_fullsized': '2024-09-10T10:44:55.267713+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267163+0000', 'last_change': '2024-09-10T10:44:55.267163+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267163+0000', 'last_undegraded': '2024-09-10T10:44:55.267163+0000', 'last_fullsized': '2024-09-10T10:44:55.267163+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-10T10:45:01.336 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.264579+0000', 'last_change': '2024-09-10T10:44:55.264579+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.264579+0000', 'last_undegraded': '2024-09-10T10:44:55.264579+0000', 'last_fullsized': '2024-09-10T10:44:55.264579+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263789+0000', 'last_change': '2024-09-10T10:44:55.263789+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263789+0000', 'last_undegraded': '2024-09-10T10:44:55.263789+0000', 'last_fullsized': '2024-09-10T10:44:55.263789+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265739+0000', 'last_change': '2024-09-10T10:44:55.265739+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265739+0000', 'last_undegraded': '2024-09-10T10:44:55.265739+0000', 'last_fullsized': '2024-09-10T10:44:55.265739+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.262657+0000', 'last_change': '2024-09-10T10:44:55.262657+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.262657+0000', 'last_undegraded': '2024-09-10T10:44:55.262657+0000', 'last_fullsized': '2024-09-10T10:44:55.262657+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270979+0000', 'last_change': '2024-09-10T10:44:55.270979+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270979+0000', 'last_undegraded': '2024-09-10T10:44:55.270979+0000', 'last_fullsized': '2024-09-10T10:44:55.270979+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-10T10:45:01.337 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.263025+0000', 'last_change': '2024-09-10T10:44:55.263025+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.263025+0000', 'last_undegraded': '2024-09-10T10:44:55.263025+0000', 'last_fullsized': '2024-09-10T10:44:55.263025+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.270224+0000', 'last_change': '2024-09-10T10:44:55.270224+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.270224+0000', 'last_undegraded': '2024-09-10T10:44:55.270224+0000', 'last_fullsized': '2024-09-10T10:44:55.270224+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.271856+0000', 'last_change': '2024-09-10T10:44:55.271856+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.271856+0000', 'last_undegraded': '2024-09-10T10:44:55.271856+0000', 'last_fullsized': '2024-09-10T10:44:55.271856+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.268246+0000', 'last_change': '2024-09-10T10:44:55.268246+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.268246+0000', 'last_undegraded': '2024-09-10T10:44:55.268246+0000', 'last_fullsized': '2024-09-10T10:44:55.268246+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267128+0000', 'last_change': '2024-09-10T10:44:55.267128+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267128+0000', 'last_undegraded': '2024-09-10T10:44:55.267128+0000', 'last_fullsized': '2024-09-10T10:44:55.267128+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-10T10:45:01.338 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.264966+0000', 'last_change': '2024-09-10T10:44:55.264966+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.264966+0000', 'last_undegraded': '2024-09-10T10:44:55.264966+0000', 'last_fullsized': '2024-09-10T10:44:55.264966+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.339 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-10T10:45:01.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.267360+0000', 'last_change': '2024-09-10T10:44:55.267360+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.267360+0000', 'last_undegraded': '2024-09-10T10:44:55.267360+0000', 'last_fullsized': '2024-09-10T10:44:55.267360+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T10:45:01.339 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-10T10:45:01.339 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 40, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:44:55.265809+0000', 'last_change': '2024-09-10T10:44:55.265809+0000', 'last_active': '2024-09-10T10:44:55.250795+0000', 'last_peered': '2024-09-10T10:44:55.250795+0000', 'last_clean': '2024-09-10T10:44:55.250795+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:44:55.265809+0000', 'last_undegraded': '2024-09-10T10:44:55.265809+0000', 'last_fullsized': '2024-09-10T10:44:55.265809+0000', 'mapping_epoch': 40, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 40, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'last_clean_scrub_stamp': '2024-09-10T10:44:55.250795+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-10T10:45:01.339 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T10:45:01.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.518+0000 7f867b200640 1 Processor -- start 2024-09-10T10:45:01.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.518+0000 7f867b200640 1 -- start start 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f867b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f867c15e4b0 0x7f867c06ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f867b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 0x7f867c0d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f867b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867c0cee30 con 0x7f867c0d4250 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f867b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f867c0d29e0 con 0x7f867c15e4b0 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f867b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f867c0d2cc0 con 0x7f867c0d1f90 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f8679e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f867c15e4b0 0x7f867c06ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f8679e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f867c15e4b0 0x7f867c06ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33686/0 (socket says 172.21.15.28:33686) 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.519+0000 7f8679e00640 1 -- 172.21.15.28:0/1976855980 learned_addr learned my addr 172.21.15.28:0/1976855980 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679400640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 0x7f867c0d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679e00640 1 -- 172.21.15.28:0/1976855980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 msgr2=0x7f867c0d23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679e00640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 0x7f867c0d23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679e00640 1 -- 172.21.15.28:0/1976855980 >> v1:172.21.15.138:6789/0 conn(0x7f867c0d4250 legacy=0x7f867c0d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:01.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679e00640 1 -- 172.21.15.28:0/1976855980 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c0d2fa0 con 0x7f867c15e4b0 2024-09-10T10:45:01.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.520+0000 7f8679400640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 0x7f867c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.521+0000 7f8679e00640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c06ceb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f866c009870 tx=0x7f866c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e4063141183bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.521+0000 7f8678a00640 1 -- 172.21.15.28:0/1976855980 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c037070 con 0x7f867c15e4b0 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.521+0000 7f8678a00640 1 -- 172.21.15.28:0/1976855980 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c032440 con 0x7f867c15e4b0 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.521+0000 7f8678a00640 1 -- 172.21.15.28:0/1976855980 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c032810 con 0x7f867c15e4b0 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 -- 172.21.15.28:0/1976855980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 msgr2=0x7f867c06ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c06ceb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f866c009870 tx=0x7f866c02a210 comp rx=0 tx=0).stop 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 -- 172.21.15.28:0/1976855980 shutdown_connections 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d1f90 0x7f867c0d23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 --2- 172.21.15.28:0/1976855980 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c06ceb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 -- 172.21.15.28:0/1976855980 >> 172.21.15.28:0/1976855980 conn(0x7f867c09d720 msgr2=0x7f867c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 -- 172.21.15.28:0/1976855980 shutdown_connections 2024-09-10T10:45:01.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.522+0000 7f867b200640 1 -- 172.21.15.28:0/1976855980 wait complete. 2024-09-10T10:45:01.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.523+0000 7f867b200640 1 Processor -- start 2024-09-10T10:45:01.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.523+0000 7f867b200640 1 -- start start 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 0x7f867c0ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d4250 0x7f867c15d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0d0f30 con 0x7f867c15e4b0 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f867c0d0db0 con 0x7f867c0d1f90 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.524+0000 7f867b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f867c15e890 con 0x7f867c0d4250 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33694/0 (socket says 172.21.15.28:33694) 2024-09-10T10:45:01.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 -- 172.21.15.28:0/3080054611 learned_addr learned my addr 172.21.15.28:0/3080054611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f8679400640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d4250 0x7f867c15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f8679e00640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 0x7f867c0ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 -- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d4250 msgr2=0x7f867c15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d4250 0x7f867c15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 -- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 msgr2=0x7f867c0ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 0x7f867c0ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 -- 172.21.15.28:0/3080054611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c161a20 con 0x7f867c15e4b0 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f8679e00640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 0x7f867c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:01.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.525+0000 7f867a800640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867400c780 tx=0x7f867400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.526+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674004b60 con 0x7f867c15e4b0 2024-09-10T10:45:01.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.526+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8674004d00 con 0x7f867c15e4b0 2024-09-10T10:45:01.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.526+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867400aa90 con 0x7f867c15e4b0 2024-09-10T10:45:01.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.526+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c0cb630 con 0x7f867c15e4b0 2024-09-10T10:45:01.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.526+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c0cbbf0 con 0x7f867c15e4b0 2024-09-10T10:45:01.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.527+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f867c0cbdf0 con 0x7f867c15e4b0 2024-09-10T10:45:01.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.528+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f867400ac30 con 0x7f867c15e4b0 2024-09-10T10:45:01.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.529+0000 7f8672000640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 0x7f865c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.529+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f867408b250 con 0x7f867c15e4b0 2024-09-10T10:45:01.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.529+0000 7f8679e00640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 0x7f865c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.530+0000 7f8679e00640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 0x7f865c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f866c034000 tx=0x7f866c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.533+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f867c0cbdf0 con 0x7f867c15e4b0 2024-09-10T10:45:01.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.784+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f867c0d23d0 con 0x7f867c15e4b0 2024-09-10T10:45:01.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.784+0000 7f8672000640 1 -- 172.21.15.28:0/3080054611 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1556 (secure 0 0 0) 0x7f8674020070 con 0x7f867c15e4b0 2024-09-10T10:45:01.786 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:01.786 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":100,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":42,"num_osds":8,"num_up_osds":8,"osd_up_since":1725965015,"num_in_osds":8,"osd_in_since":1725965005,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":61},{"state_name":"active+clean","count":36}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":230531072,"bytes_avail":772863582208,"bytes_total":773094113280,"inactive_pgs_ratio":0.62886595726013184,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":18,"btime":"2024-09-10T10:44:59:280567+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4845}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-10T10:44:42.496361+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T10:45:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 msgr2=0x7f865c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 0x7f865c06e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f866c034000 tx=0x7f866c02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 msgr2=0x7f867c155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f867400c780 tx=0x7f867400cc40 comp rx=0 tx=0).stop 2024-09-10T10:45:01.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 shutdown_connections 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f865c06be30 0x7f865c06e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f867c15e4b0 0x7f867c155440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f867c0d4250 0x7f867c15d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 --2- 172.21.15.28:0/3080054611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f867c0d1f90 0x7f867c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 >> 172.21.15.28:0/3080054611 conn(0x7f867c09d720 msgr2=0x7f867c0cfc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 shutdown_connections 2024-09-10T10:45:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.787+0000 7f867b200640 1 -- 172.21.15.28:0/3080054611 wait complete. 2024-09-10T10:45:01.805 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T10:45:01.805 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:01.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.960+0000 7f8b46000640 1 Processor -- start 2024-09-10T10:45:01.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.960+0000 7f8b46000640 1 -- start start 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.961+0000 7f8b46000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147230 0x7f8b4006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b46000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b40147e90 0x7f8b400720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b46000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4006f480 con 0x7f8b400725f0 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b46000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8b400b34d0 con 0x7f8b40147e90 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b46000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b400ae9e0 con 0x7f8b40147230 2024-09-10T10:45:01.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b40147e90 0x7f8b400720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b40147e90 0x7f8b400720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33704/0 (socket says 172.21.15.28:33704) 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b3fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8b400725f0 0x7f8b40161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56992/0 (socket says 172.21.15.28:56992) 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 -- 172.21.15.28:0/2939711000 learned_addr learned my addr 172.21.15.28:0/2939711000 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b3f400640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147230 0x7f8b4006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 -- 172.21.15.28:0/2939711000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147230 msgr2=0x7f8b4006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147230 0x7f8b4006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 -- 172.21.15.28:0/2939711000 >> v1:172.21.15.28:6789/0 conn(0x7f8b400725f0 legacy=0x7f8b40161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.962+0000 7f8b37e00640 1 -- 172.21.15.28:0/2939711000 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4006e900 con 0x7f8b40147e90 2024-09-10T10:45:01.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.963+0000 7f8b37e00640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147e90 0x7f8b400720b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009870 tx=0x7f8b2802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b500f2c7d26a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b3ea00640 1 -- 172.21.15.28:0/2939711000 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28037070 con 0x7f8b40147e90 2024-09-10T10:45:01.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b3ea00640 1 -- 172.21.15.28:0/2939711000 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28032440 con 0x7f8b40147e90 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b3ea00640 1 -- 172.21.15.28:0/2939711000 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28032810 con 0x7f8b40147e90 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 -- 172.21.15.28:0/2939711000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147e90 msgr2=0x7f8b400720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147e90 0x7f8b400720b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009870 tx=0x7f8b2802a250 comp rx=0 tx=0).stop 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 -- 172.21.15.28:0/2939711000 shutdown_connections 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147e90 0x7f8b400720b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 --2- 172.21.15.28:0/2939711000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147230 0x7f8b4006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 -- 172.21.15.28:0/2939711000 >> 172.21.15.28:0/2939711000 conn(0x7f8b40067d70 msgr2=0x7f8b4006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 -- 172.21.15.28:0/2939711000 shutdown_connections 2024-09-10T10:45:01.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.964+0000 7f8b46000640 1 -- 172.21.15.28:0/2939711000 wait complete. 2024-09-10T10:45:01.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.966+0000 7f8b46000640 1 Processor -- start 2024-09-10T10:45:01.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.966+0000 7f8b46000640 1 -- start start 2024-09-10T10:45:01.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b400725f0 0x7f8b4015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147e90 0x7f8b400c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4006eec0 con 0x7f8b40147230 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b40072a40 con 0x7f8b400725f0 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b46000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4006f1a0 con 0x7f8b40147e90 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33714/0 (socket says 172.21.15.28:33714) 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.967+0000 7f8b37e00640 1 -- 172.21.15.28:0/1630434230 learned_addr learned my addr 172.21.15.28:0/1630434230 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b3fe00640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147e90 0x7f8b400c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b3f400640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b400725f0 0x7f8b4015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 -- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147e90 msgr2=0x7f8b400c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147e90 0x7f8b400c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 -- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b400725f0 msgr2=0x7f8b4015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b400725f0 0x7f8b4015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 -- 172.21.15.28:0/1630434230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b400c9200 con 0x7f8b40147230 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37e00640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b2802a290 tx=0x7f8b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28041030 con 0x7f8b40147230 2024-09-10T10:45:01.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28030440 con 0x7f8b40147230 2024-09-10T10:45:01.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b400c94e0 con 0x7f8b40147230 2024-09-10T10:45:01.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28037040 con 0x7f8b40147230 2024-09-10T10:45:01.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.968+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b400c9aa0 con 0x7f8b40147230 2024-09-10T10:45:01.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.970+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b40147610 con 0x7f8b40147230 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.970+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b28044070 con 0x7f8b40147230 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.972+0000 7f8b37400640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.972+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f8b280ab360 con 0x7f8b40147230 2024-09-10T10:45:01.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.972+0000 7f8b3f400640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:01.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.973+0000 7f8b3f400640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009ad0 tx=0x7f8b30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:01.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:01.977+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b2804b070 con 0x7f8b40147230 2024-09-10T10:45:02.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.158+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b400afa20 con 0x7f8b1c06be30 2024-09-10T10:45:02.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.164+0000 7f8b37400640 1 -- 172.21.15.28:0/1630434230 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246896 (secure 0 0 0) 0x7f8b400afa20 con 0x7f8b1c06be30 2024-09-10T10:45:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:02.169 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 msgr2=0x7f8b1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b30009ad0 tx=0x7f8b30008040 comp rx=0 tx=0).stop 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 msgr2=0x7f8b400ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b2802a290 tx=0x7f8b280046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 shutdown_connections 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b1c06be30 0x7f8b1c06e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b40147e90 0x7f8b400c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:02.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b40147230 0x7f8b400ca2a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 --2- 172.21.15.28:0/1630434230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b400725f0 0x7f8b4015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 >> 172.21.15.28:0/1630434230 conn(0x7f8b40067d70 msgr2=0x7f8b400b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 shutdown_connections 2024-09-10T10:45:02.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:02.168+0000 7f8b46000640 1 -- 172.21.15.28:0/1630434230 wait complete. 2024-09-10T10:45:02.185 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-10T10:45:00.499522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225128,"kb_used_data":4728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749592,"statfs":{"total":773094113280,"available":772863582208,"internally_reserved":0,"allocated":4841472,"data_stored":2725894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001215"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263853+0000","last_change":"2024-09-10T10:44:55.263853+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263853+0000","last_undegraded":"2024-09-10T10:44:55.263853+0000","last_fullsized":"2024-09-10T10:44:55.263853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271034+0000","last_change":"2024-09-10T10:44:55.271034+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271034+0000","last_undegraded":"2024-09-10T10:44:55.271034+0000","last_fullsized":"2024-09-10T10:44:55.271034+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263908+0000","last_change":"2024-09-10T10:44:55.263908+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263908+0000","last_undegraded":"2024-09-10T10:44:55.263908+0000","last_fullsized":"2024-09-10T10:44:55.263908+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263405+0000","last_change":"2024-09-10T10:44:55.263405+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263405+0000","last_undegraded":"2024-09-10T10:44:55.263405+0000","last_fullsized":"2024-09-10T10:44:55.263405+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269251+0000","last_change":"2024-09-10T10:44:55.269251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269251+0000","last_undegraded":"2024-09-10T10:44:55.269251+0000","last_fullsized":"2024-09-10T10:44:55.269251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268012+0000","last_change":"2024-09-10T10:44:55.268012+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268012+0000","last_undegraded":"2024-09-10T10:44:55.268012+0000","last_fullsized":"2024-09-10T10:44:55.268012+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264033+0000","last_change":"2024-09-10T10:44:55.264033+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264033+0000","last_undegraded":"2024-09-10T10:44:55.264033+0000","last_fullsized":"2024-09-10T10:44:55.264033+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265069+0000","last_change":"2024-09-10T10:44:55.265069+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265069+0000","last_undegraded":"2024-09-10T10:44:55.265069+0000","last_fullsized":"2024-09-10T10:44:55.265069+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271830+0000","last_change":"2024-09-10T10:44:55.271830+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271830+0000","last_undegraded":"2024-09-10T10:44:55.271830+0000","last_fullsized":"2024-09-10T10:44:55.271830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266035+0000","last_change":"2024-09-10T10:44:55.266035+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266035+0000","last_undegraded":"2024-09-10T10:44:55.266035+0000","last_fullsized":"2024-09-10T10:44:55.266035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272854+0000","last_change":"2024-09-10T10:44:55.272854+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272854+0000","last_undegraded":"2024-09-10T10:44:55.272854+0000","last_fullsized":"2024-09-10T10:44:55.272854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270668+0000","last_change":"2024-09-10T10:44:55.270668+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270668+0000","last_undegraded":"2024-09-10T10:44:55.270668+0000","last_fullsized":"2024-09-10T10:44:55.270668+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263816+0000","last_change":"2024-09-10T10:44:55.263816+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263816+0000","last_undegraded":"2024-09-10T10:44:55.263816+0000","last_fullsized":"2024-09-10T10:44:55.263816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269525+0000","last_change":"2024-09-10T10:44:55.269525+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269525+0000","last_undegraded":"2024-09-10T10:44:55.269525+0000","last_fullsized":"2024-09-10T10:44:55.269525+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268281+0000","last_change":"2024-09-10T10:44:55.268281+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268281+0000","last_undegraded":"2024-09-10T10:44:55.268281+0000","last_fullsized":"2024-09-10T10:44:55.268281+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263005+0000","last_change":"2024-09-10T10:44:55.263005+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263005+0000","last_undegraded":"2024-09-10T10:44:55.263005+0000","last_fullsized":"2024-09-10T10:44:55.263005+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267826+0000","last_change":"2024-09-10T10:44:55.267826+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267826+0000","last_undegraded":"2024-09-10T10:44:55.267826+0000","last_fullsized":"2024-09-10T10:44:55.267826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266794+0000","last_change":"2024-09-10T10:44:55.266794+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266794+0000","last_undegraded":"2024-09-10T10:44:55.266794+0000","last_fullsized":"2024-09-10T10:44:55.266794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280488+0000","last_change":"2024-09-10T10:44:56.293837+0000","last_active":"2024-09-10T10:44:57.280488+0000","last_peered":"2024-09-10T10:44:57.280488+0000","last_clean":"2024-09-10T10:44:57.280488+0000","last_became_active":"2024-09-10T10:44:56.293570+0000","last_became_peered":"2024-09-10T10:44:56.293570+0000","last_unstale":"2024-09-10T10:44:57.280488+0000","last_undegraded":"2024-09-10T10:44:57.280488+0000","last_fullsized":"2024-09-10T10:44:57.280488+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.685917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269860+0000","last_change":"2024-09-10T10:44:55.269860+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269860+0000","last_undegraded":"2024-09-10T10:44:55.269860+0000","last_fullsized":"2024-09-10T10:44:55.269860+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267879+0000","last_change":"2024-09-10T10:44:55.267879+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267879+0000","last_undegraded":"2024-09-10T10:44:55.267879+0000","last_fullsized":"2024-09-10T10:44:55.267879+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.273400+0000","last_change":"2024-09-10T10:44:55.273400+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.273400+0000","last_undegraded":"2024-09-10T10:44:55.273400+0000","last_fullsized":"2024-09-10T10:44:55.273400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266722+0000","last_change":"2024-09-10T10:44:55.266722+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266722+0000","last_undegraded":"2024-09-10T10:44:55.266722+0000","last_fullsized":"2024-09-10T10:44:55.266722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264529+0000","last_change":"2024-09-10T10:44:55.264529+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264529+0000","last_undegraded":"2024-09-10T10:44:55.264529+0000","last_fullsized":"2024-09-10T10:44:55.264529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270779+0000","last_change":"2024-09-10T10:44:55.270779+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270779+0000","last_undegraded":"2024-09-10T10:44:55.270779+0000","last_fullsized":"2024-09-10T10:44:55.270779+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265957+0000","last_change":"2024-09-10T10:44:55.265957+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265957+0000","last_undegraded":"2024-09-10T10:44:55.265957+0000","last_fullsized":"2024-09-10T10:44:55.265957+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.272379+0000","last_change":"2024-09-10T10:44:55.272379+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.272379+0000","last_undegraded":"2024-09-10T10:44:55.272379+0000","last_fullsized":"2024-09-10T10:44:55.272379+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263201+0000","last_change":"2024-09-10T10:44:55.263201+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263201+0000","last_undegraded":"2024-09-10T10:44:55.263201+0000","last_fullsized":"2024-09-10T10:44:55.263201+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267251+0000","last_change":"2024-09-10T10:44:55.267251+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267251+0000","last_undegraded":"2024-09-10T10:44:55.267251+0000","last_fullsized":"2024-09-10T10:44:55.267251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267256+0000","last_change":"2024-09-10T10:44:55.267256+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267256+0000","last_undegraded":"2024-09-10T10:44:55.267256+0000","last_fullsized":"2024-09-10T10:44:55.267256+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263151+0000","last_change":"2024-09-10T10:44:55.263151+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263151+0000","last_undegraded":"2024-09-10T10:44:55.263151+0000","last_fullsized":"2024-09-10T10:44:55.263151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264106+0000","last_change":"2024-09-10T10:44:55.264106+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264106+0000","last_undegraded":"2024-09-10T10:44:55.264106+0000","last_fullsized":"2024-09-10T10:44:55.264106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266238+0000","last_change":"2024-09-10T10:44:55.266238+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266238+0000","last_undegraded":"2024-09-10T10:44:55.266238+0000","last_fullsized":"2024-09-10T10:44:55.266238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282941+0000","last_change":"2024-09-10T10:44:55.277458+0000","last_active":"2024-09-10T10:44:55.282941+0000","last_peered":"2024-09-10T10:44:55.282941+0000","last_clean":"2024-09-10T10:44:55.282941+0000","last_became_active":"2024-09-10T10:44:55.277216+0000","last_became_peered":"2024-09-10T10:44:55.277216+0000","last_unstale":"2024-09-10T10:44:55.282941+0000","last_undegraded":"2024-09-10T10:44:55.282941+0000","last_fullsized":"2024-09-10T10:44:55.282941+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268161+0000","last_change":"2024-09-10T10:44:55.268161+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268161+0000","last_undegraded":"2024-09-10T10:44:55.268161+0000","last_fullsized":"2024-09-10T10:44:55.268161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284405+0000","last_change":"2024-09-10T10:44:55.281076+0000","last_active":"2024-09-10T10:44:55.284405+0000","last_peered":"2024-09-10T10:44:55.284405+0000","last_clean":"2024-09-10T10:44:55.284405+0000","last_became_active":"2024-09-10T10:44:55.280788+0000","last_became_peered":"2024-09-10T10:44:55.280788+0000","last_unstale":"2024-09-10T10:44:55.284405+0000","last_undegraded":"2024-09-10T10:44:55.284405+0000","last_fullsized":"2024-09-10T10:44:55.284405+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266360+0000","last_change":"2024-09-10T10:44:55.266360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266360+0000","last_undegraded":"2024-09-10T10:44:55.266360+0000","last_fullsized":"2024-09-10T10:44:55.266360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281259+0000","last_change":"2024-09-10T10:44:55.273775+0000","last_active":"2024-09-10T10:44:55.281259+0000","last_peered":"2024-09-10T10:44:55.281259+0000","last_clean":"2024-09-10T10:44:55.281259+0000","last_became_active":"2024-09-10T10:44:55.273439+0000","last_became_peered":"2024-09-10T10:44:55.273439+0000","last_unstale":"2024-09-10T10:44:55.281259+0000","last_undegraded":"2024-09-10T10:44:55.281259+0000","last_fullsized":"2024-09-10T10:44:55.281259+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269513+0000","last_change":"2024-09-10T10:44:55.269513+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269513+0000","last_undegraded":"2024-09-10T10:44:55.269513+0000","last_fullsized":"2024-09-10T10:44:55.269513+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.284887+0000","last_change":"2024-09-10T10:44:55.282020+0000","last_active":"2024-09-10T10:44:55.284887+0000","last_peered":"2024-09-10T10:44:55.284887+0000","last_clean":"2024-09-10T10:44:55.284887+0000","last_became_active":"2024-09-10T10:44:55.281745+0000","last_became_peered":"2024-09-10T10:44:55.281745+0000","last_unstale":"2024-09-10T10:44:55.284887+0000","last_undegraded":"2024-09-10T10:44:55.284887+0000","last_fullsized":"2024-09-10T10:44:55.284887+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263701+0000","last_change":"2024-09-10T10:44:55.263701+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263701+0000","last_undegraded":"2024-09-10T10:44:55.263701+0000","last_fullsized":"2024-09-10T10:44:55.263701+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283525+0000","last_change":"2024-09-10T10:44:55.280125+0000","last_active":"2024-09-10T10:44:55.283525+0000","last_peered":"2024-09-10T10:44:55.283525+0000","last_clean":"2024-09-10T10:44:55.283525+0000","last_became_active":"2024-09-10T10:44:55.279856+0000","last_became_peered":"2024-09-10T10:44:55.279856+0000","last_unstale":"2024-09-10T10:44:55.283525+0000","last_undegraded":"2024-09-10T10:44:55.283525+0000","last_fullsized":"2024-09-10T10:44:55.283525+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:29.010892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269283+0000","last_change":"2024-09-10T10:44:55.269283+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269283+0000","last_undegraded":"2024-09-10T10:44:55.269283+0000","last_fullsized":"2024-09-10T10:44:55.269283+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282972+0000","last_change":"2024-09-10T10:44:55.277393+0000","last_active":"2024-09-10T10:44:55.282972+0000","last_peered":"2024-09-10T10:44:55.282972+0000","last_clean":"2024-09-10T10:44:55.282972+0000","last_became_active":"2024-09-10T10:44:55.277116+0000","last_became_peered":"2024-09-10T10:44:55.277116+0000","last_unstale":"2024-09-10T10:44:55.282972+0000","last_undegraded":"2024-09-10T10:44:55.282972+0000","last_fullsized":"2024-09-10T10:44:55.282972+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:55.889822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.266735+0000","last_change":"2024-09-10T10:44:55.266735+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.266735+0000","last_undegraded":"2024-09-10T10:44:55.266735+0000","last_fullsized":"2024-09-10T10:44:55.266735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.277553+0000","last_change":"2024-09-10T10:44:55.274146+0000","last_active":"2024-09-10T10:44:55.277553+0000","last_peered":"2024-09-10T10:44:55.277553+0000","last_clean":"2024-09-10T10:44:55.277553+0000","last_became_active":"2024-09-10T10:44:55.273668+0000","last_became_peered":"2024-09-10T10:44:55.273668+0000","last_unstale":"2024-09-10T10:44:55.277553+0000","last_undegraded":"2024-09-10T10:44:55.277553+0000","last_fullsized":"2024-09-10T10:44:55.277553+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270346+0000","last_change":"2024-09-10T10:44:55.270346+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270346+0000","last_undegraded":"2024-09-10T10:44:55.270346+0000","last_fullsized":"2024-09-10T10:44:55.270346+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287896+0000","last_change":"2024-09-10T10:44:55.285818+0000","last_active":"2024-09-10T10:44:55.287896+0000","last_peered":"2024-09-10T10:44:55.287896+0000","last_clean":"2024-09-10T10:44:55.287896+0000","last_became_active":"2024-09-10T10:44:55.285486+0000","last_became_peered":"2024-09-10T10:44:55.285486+0000","last_unstale":"2024-09-10T10:44:55.287896+0000","last_undegraded":"2024-09-10T10:44:55.287896+0000","last_fullsized":"2024-09-10T10:44:55.287896+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:12.231875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262876+0000","last_change":"2024-09-10T10:44:55.262876+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262876+0000","last_undegraded":"2024-09-10T10:44:55.262876+0000","last_fullsized":"2024-09-10T10:44:55.262876+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289989+0000","last_change":"2024-09-10T10:44:55.282228+0000","last_active":"2024-09-10T10:44:55.289989+0000","last_peered":"2024-09-10T10:44:55.289989+0000","last_clean":"2024-09-10T10:44:55.289989+0000","last_became_active":"2024-09-10T10:44:55.281403+0000","last_became_peered":"2024-09-10T10:44:55.281403+0000","last_unstale":"2024-09-10T10:44:55.289989+0000","last_undegraded":"2024-09-10T10:44:55.289989+0000","last_fullsized":"2024-09-10T10:44:55.289989+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.269496+0000","last_change":"2024-09-10T10:44:55.269496+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.269496+0000","last_undegraded":"2024-09-10T10:44:55.269496+0000","last_fullsized":"2024-09-10T10:44:55.269496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.283277+0000","last_change":"2024-09-10T10:44:55.280162+0000","last_active":"2024-09-10T10:44:55.283277+0000","last_peered":"2024-09-10T10:44:55.283277+0000","last_clean":"2024-09-10T10:44:55.283277+0000","last_became_active":"2024-09-10T10:44:55.279842+0000","last_became_peered":"2024-09-10T10:44:55.279842+0000","last_unstale":"2024-09-10T10:44:55.283277+0000","last_undegraded":"2024-09-10T10:44:55.283277+0000","last_fullsized":"2024-09-10T10:44:55.283277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:32.358738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265570+0000","last_change":"2024-09-10T10:44:55.265570+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265570+0000","last_undegraded":"2024-09-10T10:44:55.265570+0000","last_fullsized":"2024-09-10T10:44:55.265570+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.279575+0000","last_change":"2024-09-10T10:44:55.274128+0000","last_active":"2024-09-10T10:44:55.279575+0000","last_peered":"2024-09-10T10:44:55.279575+0000","last_clean":"2024-09-10T10:44:55.279575+0000","last_became_active":"2024-09-10T10:44:55.273849+0000","last_became_peered":"2024-09-10T10:44:55.273849+0000","last_unstale":"2024-09-10T10:44:55.279575+0000","last_undegraded":"2024-09-10T10:44:55.279575+0000","last_fullsized":"2024-09-10T10:44:55.279575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:40.204312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265381+0000","last_change":"2024-09-10T10:44:55.265381+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265381+0000","last_undegraded":"2024-09-10T10:44:55.265381+0000","last_fullsized":"2024-09-10T10:44:55.265381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286851+0000","last_change":"2024-09-10T10:44:55.280404+0000","last_active":"2024-09-10T10:44:55.286851+0000","last_peered":"2024-09-10T10:44:55.286851+0000","last_clean":"2024-09-10T10:44:55.286851+0000","last_became_active":"2024-09-10T10:44:55.277105+0000","last_became_peered":"2024-09-10T10:44:55.277105+0000","last_unstale":"2024-09-10T10:44:55.286851+0000","last_undegraded":"2024-09-10T10:44:55.286851+0000","last_fullsized":"2024-09-10T10:44:55.286851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262621+0000","last_change":"2024-09-10T10:44:55.262621+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262621+0000","last_undegraded":"2024-09-10T10:44:55.262621+0000","last_fullsized":"2024-09-10T10:44:55.262621+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290065+0000","last_change":"2024-09-10T10:44:55.283497+0000","last_active":"2024-09-10T10:44:55.290065+0000","last_peered":"2024-09-10T10:44:55.290065+0000","last_clean":"2024-09-10T10:44:55.290065+0000","last_became_active":"2024-09-10T10:44:55.280421+0000","last_became_peered":"2024-09-10T10:44:55.280421+0000","last_unstale":"2024-09-10T10:44:55.290065+0000","last_undegraded":"2024-09-10T10:44:55.290065+0000","last_fullsized":"2024-09-10T10:44:55.290065+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.465795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267728+0000","last_change":"2024-09-10T10:44:55.267728+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267728+0000","last_undegraded":"2024-09-10T10:44:55.267728+0000","last_fullsized":"2024-09-10T10:44:55.267728+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282353+0000","last_change":"2024-09-10T10:44:55.279940+0000","last_active":"2024-09-10T10:44:55.282353+0000","last_peered":"2024-09-10T10:44:55.282353+0000","last_clean":"2024-09-10T10:44:55.282353+0000","last_became_active":"2024-09-10T10:44:55.279402+0000","last_became_peered":"2024-09-10T10:44:55.279402+0000","last_unstale":"2024-09-10T10:44:55.282353+0000","last_undegraded":"2024-09-10T10:44:55.282353+0000","last_fullsized":"2024-09-10T10:44:55.282353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270467+0000","last_change":"2024-09-10T10:44:55.270467+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270467+0000","last_undegraded":"2024-09-10T10:44:55.270467+0000","last_fullsized":"2024-09-10T10:44:55.270467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280174+0000","last_change":"2024-09-10T10:44:55.274369+0000","last_active":"2024-09-10T10:44:55.280174+0000","last_peered":"2024-09-10T10:44:55.280174+0000","last_clean":"2024-09-10T10:44:55.280174+0000","last_became_active":"2024-09-10T10:44:55.273965+0000","last_became_peered":"2024-09-10T10:44:55.273965+0000","last_unstale":"2024-09-10T10:44:55.280174+0000","last_undegraded":"2024-09-10T10:44:55.280174+0000","last_fullsized":"2024-09-10T10:44:55.280174+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:50.577611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267713+0000","last_change":"2024-09-10T10:44:55.267713+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267713+0000","last_undegraded":"2024-09-10T10:44:55.267713+0000","last_fullsized":"2024-09-10T10:44:55.267713+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.281549+0000","last_change":"2024-09-10T10:44:55.277981+0000","last_active":"2024-09-10T10:44:55.281549+0000","last_peered":"2024-09-10T10:44:55.281549+0000","last_clean":"2024-09-10T10:44:55.281549+0000","last_became_active":"2024-09-10T10:44:55.276409+0000","last_became_peered":"2024-09-10T10:44:55.276409+0000","last_unstale":"2024-09-10T10:44:55.281549+0000","last_undegraded":"2024-09-10T10:44:55.281549+0000","last_fullsized":"2024-09-10T10:44:55.281549+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:36.683459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267163+0000","last_change":"2024-09-10T10:44:55.267163+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267163+0000","last_undegraded":"2024-09-10T10:44:55.267163+0000","last_fullsized":"2024-09-10T10:44:55.267163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288028+0000","last_change":"2024-09-10T10:44:55.286607+0000","last_active":"2024-09-10T10:44:55.288028+0000","last_peered":"2024-09-10T10:44:55.288028+0000","last_clean":"2024-09-10T10:44:55.288028+0000","last_became_active":"2024-09-10T10:44:55.284761+0000","last_became_peered":"2024-09-10T10:44:55.284761+0000","last_unstale":"2024-09-10T10:44:55.288028+0000","last_undegraded":"2024-09-10T10:44:55.288028+0000","last_fullsized":"2024-09-10T10:44:55.288028+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:27:57.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264579+0000","last_change":"2024-09-10T10:44:55.264579+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264579+0000","last_undegraded":"2024-09-10T10:44:55.264579+0000","last_fullsized":"2024-09-10T10:44:55.264579+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280029+0000","last_change":"2024-09-10T10:44:55.274312+0000","last_active":"2024-09-10T10:44:55.280029+0000","last_peered":"2024-09-10T10:44:55.280029+0000","last_clean":"2024-09-10T10:44:55.280029+0000","last_became_active":"2024-09-10T10:44:55.273980+0000","last_became_peered":"2024-09-10T10:44:55.273980+0000","last_unstale":"2024-09-10T10:44:55.280029+0000","last_undegraded":"2024-09-10T10:44:55.280029+0000","last_fullsized":"2024-09-10T10:44:55.280029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:42.261388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263789+0000","last_change":"2024-09-10T10:44:55.263789+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263789+0000","last_undegraded":"2024-09-10T10:44:55.263789+0000","last_fullsized":"2024-09-10T10:44:55.263789+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.288013+0000","last_change":"2024-09-10T10:44:55.286650+0000","last_active":"2024-09-10T10:44:55.288013+0000","last_peered":"2024-09-10T10:44:55.288013+0000","last_clean":"2024-09-10T10:44:55.288013+0000","last_became_active":"2024-09-10T10:44:55.286360+0000","last_became_peered":"2024-09-10T10:44:55.286360+0000","last_unstale":"2024-09-10T10:44:55.288013+0000","last_undegraded":"2024-09-10T10:44:55.288013+0000","last_fullsized":"2024-09-10T10:44:55.288013+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:11.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280741+0000","last_change":"2024-09-10T10:44:56.288750+0000","last_active":"2024-09-10T10:44:57.280741+0000","last_peered":"2024-09-10T10:44:57.280741+0000","last_clean":"2024-09-10T10:44:57.280741+0000","last_became_active":"2024-09-10T10:44:56.288435+0000","last_became_peered":"2024-09-10T10:44:56.288435+0000","last_unstale":"2024-09-10T10:44:57.280741+0000","last_undegraded":"2024-09-10T10:44:57.280741+0000","last_fullsized":"2024-09-10T10:44:57.280741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:49.960568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280412+0000","last_change":"2024-09-10T10:44:55.278871+0000","last_active":"2024-09-10T10:44:57.280412+0000","last_peered":"2024-09-10T10:44:57.280412+0000","last_clean":"2024-09-10T10:44:57.280412+0000","last_became_active":"2024-09-10T10:44:55.278608+0000","last_became_peered":"2024-09-10T10:44:55.278608+0000","last_unstale":"2024-09-10T10:44:57.280412+0000","last_undegraded":"2024-09-10T10:44:57.280412+0000","last_fullsized":"2024-09-10T10:44:57.280412+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:47.361070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265739+0000","last_change":"2024-09-10T10:44:55.265739+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265739+0000","last_undegraded":"2024-09-10T10:44:55.265739+0000","last_fullsized":"2024-09-10T10:44:55.265739+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280206+0000","last_change":"2024-09-10T10:44:55.277794+0000","last_active":"2024-09-10T10:44:57.280206+0000","last_peered":"2024-09-10T10:44:57.280206+0000","last_clean":"2024-09-10T10:44:57.280206+0000","last_became_active":"2024-09-10T10:44:55.277054+0000","last_became_peered":"2024-09-10T10:44:55.277054+0000","last_unstale":"2024-09-10T10:44:57.280206+0000","last_undegraded":"2024-09-10T10:44:57.280206+0000","last_fullsized":"2024-09-10T10:44:57.280206+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:29.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.262657+0000","last_change":"2024-09-10T10:44:55.262657+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.262657+0000","last_undegraded":"2024-09-10T10:44:55.262657+0000","last_fullsized":"2024-09-10T10:44:55.262657+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287717+0000","last_change":"2024-09-10T10:44:55.283286+0000","last_active":"2024-09-10T10:44:55.287717+0000","last_peered":"2024-09-10T10:44:55.287717+0000","last_clean":"2024-09-10T10:44:55.287717+0000","last_became_active":"2024-09-10T10:44:55.283045+0000","last_became_peered":"2024-09-10T10:44:55.283045+0000","last_unstale":"2024-09-10T10:44:55.287717+0000","last_undegraded":"2024-09-10T10:44:55.287717+0000","last_fullsized":"2024-09-10T10:44:55.287717+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:17.289145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270979+0000","last_change":"2024-09-10T10:44:55.270979+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270979+0000","last_undegraded":"2024-09-10T10:44:55.270979+0000","last_fullsized":"2024-09-10T10:44:55.270979+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287041+0000","last_change":"2024-09-10T10:44:55.285243+0000","last_active":"2024-09-10T10:44:55.287041+0000","last_peered":"2024-09-10T10:44:55.287041+0000","last_clean":"2024-09-10T10:44:55.287041+0000","last_became_active":"2024-09-10T10:44:55.284621+0000","last_became_peered":"2024-09-10T10:44:55.284621+0000","last_unstale":"2024-09-10T10:44:55.287041+0000","last_undegraded":"2024-09-10T10:44:55.287041+0000","last_fullsized":"2024-09-10T10:44:55.287041+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:11:03.474854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.263025+0000","last_change":"2024-09-10T10:44:55.263025+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.263025+0000","last_undegraded":"2024-09-10T10:44:55.263025+0000","last_fullsized":"2024-09-10T10:44:55.263025+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.282476+0000","last_change":"2024-09-10T10:44:55.279589+0000","last_active":"2024-09-10T10:44:55.282476+0000","last_peered":"2024-09-10T10:44:55.282476+0000","last_clean":"2024-09-10T10:44:55.282476+0000","last_became_active":"2024-09-10T10:44:55.279308+0000","last_became_peered":"2024-09-10T10:44:55.279308+0000","last_unstale":"2024-09-10T10:44:55.282476+0000","last_undegraded":"2024-09-10T10:44:55.282476+0000","last_fullsized":"2024-09-10T10:44:55.282476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:54.645028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.270224+0000","last_change":"2024-09-10T10:44:55.270224+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.270224+0000","last_undegraded":"2024-09-10T10:44:55.270224+0000","last_fullsized":"2024-09-10T10:44:55.270224+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286054+0000","last_change":"2024-09-10T10:44:55.280439+0000","last_active":"2024-09-10T10:44:55.286054+0000","last_peered":"2024-09-10T10:44:55.286054+0000","last_clean":"2024-09-10T10:44:55.286054+0000","last_became_active":"2024-09-10T10:44:55.280054+0000","last_became_peered":"2024-09-10T10:44:55.280054+0000","last_unstale":"2024-09-10T10:44:55.286054+0000","last_undegraded":"2024-09-10T10:44:55.286054+0000","last_fullsized":"2024-09-10T10:44:55.286054+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:49:11.846566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.271856+0000","last_change":"2024-09-10T10:44:55.271856+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.271856+0000","last_undegraded":"2024-09-10T10:44:55.271856+0000","last_fullsized":"2024-09-10T10:44:55.271856+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.289979+0000","last_change":"2024-09-10T10:44:55.287203+0000","last_active":"2024-09-10T10:44:55.289979+0000","last_peered":"2024-09-10T10:44:55.289979+0000","last_clean":"2024-09-10T10:44:55.289979+0000","last_became_active":"2024-09-10T10:44:55.286932+0000","last_became_peered":"2024-09-10T10:44:55.286932+0000","last_unstale":"2024-09-10T10:44:55.289979+0000","last_undegraded":"2024-09-10T10:44:55.289979+0000","last_fullsized":"2024-09-10T10:44:55.289979+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:39:17.346471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280685+0000","last_change":"2024-09-10T10:44:56.275769+0000","last_active":"2024-09-10T10:44:57.280685+0000","last_peered":"2024-09-10T10:44:57.280685+0000","last_clean":"2024-09-10T10:44:57.280685+0000","last_became_active":"2024-09-10T10:44:56.275397+0000","last_became_peered":"2024-09-10T10:44:56.275397+0000","last_unstale":"2024-09-10T10:44:57.280685+0000","last_undegraded":"2024-09-10T10:44:57.280685+0000","last_fullsized":"2024-09-10T10:44:57.280685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:36.768449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290007+0000","last_change":"2024-09-10T10:44:55.287602+0000","last_active":"2024-09-10T10:44:55.290007+0000","last_peered":"2024-09-10T10:44:55.290007+0000","last_clean":"2024-09-10T10:44:55.290007+0000","last_became_active":"2024-09-10T10:44:55.287354+0000","last_became_peered":"2024-09-10T10:44:55.287354+0000","last_unstale":"2024-09-10T10:44:55.290007+0000","last_undegraded":"2024-09-10T10:44:55.290007+0000","last_fullsized":"2024-09-10T10:44:55.290007+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:11.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.268246+0000","last_change":"2024-09-10T10:44:55.268246+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.268246+0000","last_undegraded":"2024-09-10T10:44:55.268246+0000","last_fullsized":"2024-09-10T10:44:55.268246+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.290050+0000","last_change":"2024-09-10T10:44:55.283449+0000","last_active":"2024-09-10T10:44:55.290050+0000","last_peered":"2024-09-10T10:44:55.290050+0000","last_clean":"2024-09-10T10:44:55.290050+0000","last_became_active":"2024-09-10T10:44:55.281725+0000","last_became_peered":"2024-09-10T10:44:55.281725+0000","last_unstale":"2024-09-10T10:44:55.290050+0000","last_undegraded":"2024-09-10T10:44:55.290050+0000","last_fullsized":"2024-09-10T10:44:55.290050+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:25.223193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267128+0000","last_change":"2024-09-10T10:44:55.267128+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267128+0000","last_undegraded":"2024-09-10T10:44:55.267128+0000","last_fullsized":"2024-09-10T10:44:55.267128+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.286868+0000","last_change":"2024-09-10T10:44:55.281075+0000","last_active":"2024-09-10T10:44:55.286868+0000","last_peered":"2024-09-10T10:44:55.286868+0000","last_clean":"2024-09-10T10:44:55.286868+0000","last_became_active":"2024-09-10T10:44:55.280803+0000","last_became_peered":"2024-09-10T10:44:55.280803+0000","last_unstale":"2024-09-10T10:44:55.286868+0000","last_undegraded":"2024-09-10T10:44:55.286868+0000","last_fullsized":"2024-09-10T10:44:55.286868+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.264966+0000","last_change":"2024-09-10T10:44:55.264966+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.264966+0000","last_undegraded":"2024-09-10T10:44:55.264966+0000","last_fullsized":"2024-09-10T10:44:55.264966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.285925+0000","last_change":"2024-09-10T10:44:55.280515+0000","last_active":"2024-09-10T10:44:55.285925+0000","last_peered":"2024-09-10T10:44:55.285925+0000","last_clean":"2024-09-10T10:44:55.285925+0000","last_became_active":"2024-09-10T10:44:55.280197+0000","last_became_peered":"2024-09-10T10:44:55.280197+0000","last_unstale":"2024-09-10T10:44:55.285925+0000","last_undegraded":"2024-09-10T10:44:55.285925+0000","last_fullsized":"2024-09-10T10:44:55.285925+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:51.824788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.267360+0000","last_change":"2024-09-10T10:44:55.267360+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.267360+0000","last_undegraded":"2024-09-10T10:44:55.267360+0000","last_fullsized":"2024-09-10T10:44:55.267360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.280708+0000","last_change":"2024-09-10T10:44:55.276223+0000","last_active":"2024-09-10T10:44:55.280708+0000","last_peered":"2024-09-10T10:44:55.280708+0000","last_clean":"2024-09-10T10:44:55.280708+0000","last_became_active":"2024-09-10T10:44:55.275926+0000","last_became_peered":"2024-09-10T10:44:55.275926+0000","last_unstale":"2024-09-10T10:44:55.280708+0000","last_undegraded":"2024-09-10T10:44:55.280708+0000","last_fullsized":"2024-09-10T10:44:55.280708+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-09-10T10:44:55.265809+0000","last_change":"2024-09-10T10:44:55.265809+0000","last_active":"2024-09-10T10:44:55.250795+0000","last_peered":"2024-09-10T10:44:55.250795+0000","last_clean":"2024-09-10T10:44:55.250795+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:44:55.265809+0000","last_undegraded":"2024-09-10T10:44:55.265809+0000","last_fullsized":"2024-09-10T10:44:55.265809+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.287061+0000","last_change":"2024-09-10T10:44:55.284148+0000","last_active":"2024-09-10T10:44:55.287061+0000","last_peered":"2024-09-10T10:44:55.287061+0000","last_clean":"2024-09-10T10:44:55.287061+0000","last_became_active":"2024-09-10T10:44:55.283869+0000","last_became_peered":"2024-09-10T10:44:55.283869+0000","last_unstale":"2024-09-10T10:44:55.287061+0000","last_undegraded":"2024-09-10T10:44:55.287061+0000","last_fullsized":"2024-09-10T10:44:55.287061+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T10:44:55.263348+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:44:55.263348+0000","last_peered":"2024-09-10T10:44:55.263348+0000","last_clean":"2024-09-10T10:44:55.263348+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:44:55.263348+0000","last_undegraded":"2024-09-10T10:44:55.263348+0000","last_fullsized":"2024-09-10T10:44:55.263348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":185506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":782541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":454656,"data_stored":192173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1077248,"data_stored":796982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:05.189 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:45:05.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.355+0000 7fc3f0e00640 1 Processor -- start 2024-09-10T10:45:05.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.355+0000 7fc3f0e00640 1 -- start start 2024-09-10T10:45:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3f0e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3ec15b300 0x7fc3ec0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3f0e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3ec0cdbe0 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3f0e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3ec0728a0 con 0x7fc3ec15b300 2024-09-10T10:45:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3f0e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec0cd900 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3ea800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3ea800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:51220/0 (socket says 172.21.15.28:51220) 2024-09-10T10:45:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3ea800640 1 -- 172.21.15.28:0/2183465613 learned_addr learned my addr 172.21.15.28:0/2183465613 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.357+0000 7fc3e9e00640 1 --2- 172.21.15.28:0/2183465613 >> v2:172.21.15.138:3300/0 conn(0x7fc3ec15b300 0x7fc3ec0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.358+0000 7fc3ea800640 1 -- 172.21.15.28:0/2183465613 >> v2:172.21.15.138:3300/0 conn(0x7fc3ec15b300 msgr2=0x7fc3ec0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.358+0000 7fc3ea800640 1 --2- 172.21.15.28:0/2183465613 >> v2:172.21.15.138:3300/0 conn(0x7fc3ec15b300 0x7fc3ec0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.358+0000 7fc3e9400640 1 -- 172.21.15.28:0/2183465613 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041546670 0 0) 0x7fc3ec0cdbe0 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.358+0000 7fc3ea800640 1 -- 172.21.15.28:0/2183465613 >> v1:172.21.15.138:6789/0 conn(0x7fc3ec0cd3f0 legacy=0x7fc3ec06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.358+0000 7fc3ea800640 1 -- 172.21.15.28:0/2183465613 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec072250 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.359+0000 7fc3ea800640 1 --2- 172.21.15.28:0/2183465613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009e50 tx=0x7fc3e402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbceaf48ad742600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.359+0000 7fc3e9400640 1 -- 172.21.15.28:0/2183465613 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4036070 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.359+0000 7fc3e9400640 1 -- 172.21.15.28:0/2183465613 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e4031600 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.359+0000 7fc3e9400640 1 -- 172.21.15.28:0/2183465613 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40319d0 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 -- 172.21.15.28:0/2183465613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 msgr2=0x7fc3ec15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/2183465613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009e50 tx=0x7fc3e402a710 comp rx=0 tx=0).stop 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 -- 172.21.15.28:0/2183465613 shutdown_connections 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/2183465613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec15b6e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/2183465613 >> v2:172.21.15.138:3300/0 conn(0x7fc3ec15b300 0x7fc3ec0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 -- 172.21.15.28:0/2183465613 >> 172.21.15.28:0/2183465613 conn(0x7fc3ec067d70 msgr2=0x7fc3ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 -- 172.21.15.28:0/2183465613 shutdown_connections 2024-09-10T10:45:05.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.360+0000 7fc3f0e00640 1 -- 172.21.15.28:0/2183465613 wait complete. 2024-09-10T10:45:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.362+0000 7fc3f0e00640 1 Processor -- start 2024-09-10T10:45:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.362+0000 7fc3f0e00640 1 -- start start 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec1e3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec15b300 0x7fc3ec141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0d0160 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec0cffe0 con 0x7fc3ec15b300 2024-09-10T10:45:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3f0e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec072d30 con 0x7fc3ec06f8d0 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36010/0 (socket says 172.21.15.28:36010) 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 -- 172.21.15.28:0/1076812917 learned_addr learned my addr 172.21.15.28:0/1076812917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e9e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec1e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3ea800640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec15b300 0x7fc3ec141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 -- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 msgr2=0x7fc3ec1e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec1e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 -- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec15b300 msgr2=0x7fc3ec141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec15b300 0x7fc3ec141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.363+0000 7fc3e3e00640 1 -- 172.21.15.28:0/1076812917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec1e7c50 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3e3e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c4b0 tx=0x7fc3d400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d4005480 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4003830 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec1e7f30 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.364+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec1e84f0 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.365+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec1589a0 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.366+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3d401a070 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.367+0000 7fc3e2000640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 0x7fc3c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.367+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fc3d408a430 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.367+0000 7fc3e9e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 0x7fc3c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.368+0000 7fc3e9e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 0x7fc3c806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009a30 tx=0x7fc3dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.371+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3d4058bb0 con 0x7fc3ec0cd3f0 2024-09-10T10:45:05.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.550+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3ec06d090 con 0x7fc3c806be60 2024-09-10T10:45:05.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.558+0000 7fc3e2000640 1 -- 172.21.15.28:0/1076812917 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7fc3ec06d090 con 0x7fc3c806be60 2024-09-10T10:45:05.562 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:05.563 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 msgr2=0x7fc3c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 0x7fc3c806e320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009a30 tx=0x7fc3dc008040 comp rx=0 tx=0).stop 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 msgr2=0x7fc3ec1e7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c4b0 tx=0x7fc3d400c970 comp rx=0 tx=0).stop 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 shutdown_connections 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3c806be60 0x7fc3c806e320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec15b300 0x7fc3ec141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3ec0cd3f0 0x7fc3ec1e7190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 --2- 172.21.15.28:0/1076812917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec06f8d0 0x7fc3ec1e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 >> 172.21.15.28:0/1076812917 conn(0x7fc3ec067d70 msgr2=0x7fc3ec0b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:05.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 shutdown_connections 2024-09-10T10:45:05.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.561+0000 7fc3f0e00640 1 -- 172.21.15.28:0/1076812917 wait complete. 2024-09-10T10:45:05.578 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-10T10:45:04.500530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229544,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745176,"statfs":{"total":773094113280,"available":772859060224,"internally_reserved":0,"allocated":5087232,"data_stored":2833812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.239667"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279295+0000","last_change":"2024-09-10T10:44:56.282163+0000","last_active":"2024-09-10T10:44:57.279295+0000","last_peered":"2024-09-10T10:44:57.279295+0000","last_clean":"2024-09-10T10:44:57.279295+0000","last_became_active":"2024-09-10T10:44:56.281855+0000","last_became_peered":"2024-09-10T10:44:56.281855+0000","last_unstale":"2024-09-10T10:44:57.279295+0000","last_undegraded":"2024-09-10T10:44:57.279295+0000","last_fullsized":"2024-09-10T10:44:57.279295+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:18.018928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777599+0000","last_change":"2024-09-10T10:44:56.278746+0000","last_active":"2024-09-10T10:44:57.777599+0000","last_peered":"2024-09-10T10:44:57.777599+0000","last_clean":"2024-09-10T10:44:57.777599+0000","last_became_active":"2024-09-10T10:44:56.278455+0000","last_became_peered":"2024-09-10T10:44:56.278455+0000","last_unstale":"2024-09-10T10:44:57.777599+0000","last_undegraded":"2024-09-10T10:44:57.777599+0000","last_fullsized":"2024-09-10T10:44:57.777599+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:02.155492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279419+0000","last_change":"2024-09-10T10:44:56.295633+0000","last_active":"2024-09-10T10:44:57.279419+0000","last_peered":"2024-09-10T10:44:57.279419+0000","last_clean":"2024-09-10T10:44:57.279419+0000","last_became_active":"2024-09-10T10:44:56.295379+0000","last_became_peered":"2024-09-10T10:44:56.295379+0000","last_unstale":"2024-09-10T10:44:57.279419+0000","last_undegraded":"2024-09-10T10:44:57.279419+0000","last_fullsized":"2024-09-10T10:44:57.279419+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:40.091514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279219+0000","last_change":"2024-09-10T10:44:56.291537+0000","last_active":"2024-09-10T10:44:57.279219+0000","last_peered":"2024-09-10T10:44:57.279219+0000","last_clean":"2024-09-10T10:44:57.279219+0000","last_became_active":"2024-09-10T10:44:56.290851+0000","last_became_peered":"2024-09-10T10:44:56.290851+0000","last_unstale":"2024-09-10T10:44:57.279219+0000","last_undegraded":"2024-09-10T10:44:57.279219+0000","last_fullsized":"2024-09-10T10:44:57.279219+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:45:20.873085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777028+0000","last_change":"2024-09-10T10:44:56.280177+0000","last_active":"2024-09-10T10:44:57.777028+0000","last_peered":"2024-09-10T10:44:57.777028+0000","last_clean":"2024-09-10T10:44:57.777028+0000","last_became_active":"2024-09-10T10:44:56.279912+0000","last_became_peered":"2024-09-10T10:44:56.279912+0000","last_unstale":"2024-09-10T10:44:57.777028+0000","last_undegraded":"2024-09-10T10:44:57.777028+0000","last_fullsized":"2024-09-10T10:44:57.777028+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:09.567407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777019+0000","last_change":"2024-09-10T10:44:56.288010+0000","last_active":"2024-09-10T10:44:57.777019+0000","last_peered":"2024-09-10T10:44:57.777019+0000","last_clean":"2024-09-10T10:44:57.777019+0000","last_became_active":"2024-09-10T10:44:56.287736+0000","last_became_peered":"2024-09-10T10:44:56.287736+0000","last_unstale":"2024-09-10T10:44:57.777019+0000","last_undegraded":"2024-09-10T10:44:57.777019+0000","last_fullsized":"2024-09-10T10:44:57.777019+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:34.364686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776323+0000","last_change":"2024-09-10T10:44:56.297078+0000","last_active":"2024-09-10T10:44:57.776323+0000","last_peered":"2024-09-10T10:44:57.776323+0000","last_clean":"2024-09-10T10:44:57.776323+0000","last_became_active":"2024-09-10T10:44:56.296729+0000","last_became_peered":"2024-09-10T10:44:56.296729+0000","last_unstale":"2024-09-10T10:44:57.776323+0000","last_undegraded":"2024-09-10T10:44:57.776323+0000","last_fullsized":"2024-09-10T10:44:57.776323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:18.292575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278585+0000","last_change":"2024-09-10T10:44:56.303701+0000","last_active":"2024-09-10T10:44:57.278585+0000","last_peered":"2024-09-10T10:44:57.278585+0000","last_clean":"2024-09-10T10:44:57.278585+0000","last_became_active":"2024-09-10T10:44:56.303386+0000","last_became_peered":"2024-09-10T10:44:56.303386+0000","last_unstale":"2024-09-10T10:44:57.278585+0000","last_undegraded":"2024-09-10T10:44:57.278585+0000","last_fullsized":"2024-09-10T10:44:57.278585+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:51.614193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777712+0000","last_change":"2024-09-10T10:44:56.278814+0000","last_active":"2024-09-10T10:44:57.777712+0000","last_peered":"2024-09-10T10:44:57.777712+0000","last_clean":"2024-09-10T10:44:57.777712+0000","last_became_active":"2024-09-10T10:44:56.278545+0000","last_became_peered":"2024-09-10T10:44:56.278545+0000","last_unstale":"2024-09-10T10:44:57.777712+0000","last_undegraded":"2024-09-10T10:44:57.777712+0000","last_fullsized":"2024-09-10T10:44:57.777712+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:14.469225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777895+0000","last_change":"2024-09-10T10:44:56.293278+0000","last_active":"2024-09-10T10:44:57.777895+0000","last_peered":"2024-09-10T10:44:57.777895+0000","last_clean":"2024-09-10T10:44:57.777895+0000","last_became_active":"2024-09-10T10:44:56.293025+0000","last_became_peered":"2024-09-10T10:44:56.293025+0000","last_unstale":"2024-09-10T10:44:57.777895+0000","last_undegraded":"2024-09-10T10:44:57.777895+0000","last_fullsized":"2024-09-10T10:44:57.777895+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:14.469225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778496+0000","last_change":"2024-09-10T10:44:56.304650+0000","last_active":"2024-09-10T10:44:57.778496+0000","last_peered":"2024-09-10T10:44:57.778496+0000","last_clean":"2024-09-10T10:44:57.778496+0000","last_became_active":"2024-09-10T10:44:56.304184+0000","last_became_peered":"2024-09-10T10:44:56.304184+0000","last_unstale":"2024-09-10T10:44:57.778496+0000","last_undegraded":"2024-09-10T10:44:57.778496+0000","last_fullsized":"2024-09-10T10:44:57.778496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:21:47.676483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776825+0000","last_change":"2024-09-10T10:44:56.278974+0000","last_active":"2024-09-10T10:44:57.776825+0000","last_peered":"2024-09-10T10:44:57.776825+0000","last_clean":"2024-09-10T10:44:57.776825+0000","last_became_active":"2024-09-10T10:44:56.278794+0000","last_became_peered":"2024-09-10T10:44:56.278794+0000","last_unstale":"2024-09-10T10:44:57.776825+0000","last_undegraded":"2024-09-10T10:44:57.776825+0000","last_fullsized":"2024-09-10T10:44:57.776825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:19.078906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777080+0000","last_change":"2024-09-10T10:44:56.291913+0000","last_active":"2024-09-10T10:44:57.777080+0000","last_peered":"2024-09-10T10:44:57.777080+0000","last_clean":"2024-09-10T10:44:57.777080+0000","last_became_active":"2024-09-10T10:44:56.291638+0000","last_became_peered":"2024-09-10T10:44:56.291638+0000","last_unstale":"2024-09-10T10:44:57.777080+0000","last_undegraded":"2024-09-10T10:44:57.777080+0000","last_fullsized":"2024-09-10T10:44:57.777080+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:05.007976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276015+0000","last_change":"2024-09-10T10:44:56.282734+0000","last_active":"2024-09-10T10:44:57.276015+0000","last_peered":"2024-09-10T10:44:57.276015+0000","last_clean":"2024-09-10T10:44:57.276015+0000","last_became_active":"2024-09-10T10:44:56.282410+0000","last_became_peered":"2024-09-10T10:44:56.282410+0000","last_unstale":"2024-09-10T10:44:57.276015+0000","last_undegraded":"2024-09-10T10:44:57.276015+0000","last_fullsized":"2024-09-10T10:44:57.276015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:35.955689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776193+0000","last_change":"2024-09-10T10:44:56.277127+0000","last_active":"2024-09-10T10:44:57.776193+0000","last_peered":"2024-09-10T10:44:57.776193+0000","last_clean":"2024-09-10T10:44:57.776193+0000","last_became_active":"2024-09-10T10:44:56.276849+0000","last_became_peered":"2024-09-10T10:44:56.276849+0000","last_unstale":"2024-09-10T10:44:57.776193+0000","last_undegraded":"2024-09-10T10:44:57.776193+0000","last_fullsized":"2024-09-10T10:44:57.776193+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:29.756274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776193+0000","last_change":"2024-09-10T10:44:56.300242+0000","last_active":"2024-09-10T10:44:57.776193+0000","last_peered":"2024-09-10T10:44:57.776193+0000","last_clean":"2024-09-10T10:44:57.776193+0000","last_became_active":"2024-09-10T10:44:56.299964+0000","last_became_peered":"2024-09-10T10:44:56.299964+0000","last_unstale":"2024-09-10T10:44:57.776193+0000","last_undegraded":"2024-09-10T10:44:57.776193+0000","last_fullsized":"2024-09-10T10:44:57.776193+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:38.014875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778350+0000","last_change":"2024-09-10T10:44:56.300113+0000","last_active":"2024-09-10T10:44:57.778350+0000","last_peered":"2024-09-10T10:44:57.778350+0000","last_clean":"2024-09-10T10:44:57.778350+0000","last_became_active":"2024-09-10T10:44:56.299848+0000","last_became_peered":"2024-09-10T10:44:56.299848+0000","last_unstale":"2024-09-10T10:44:57.778350+0000","last_undegraded":"2024-09-10T10:44:57.778350+0000","last_fullsized":"2024-09-10T10:44:57.778350+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:05.912347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777242+0000","last_change":"2024-09-10T10:44:56.288125+0000","last_active":"2024-09-10T10:44:57.777242+0000","last_peered":"2024-09-10T10:44:57.777242+0000","last_clean":"2024-09-10T10:44:57.777242+0000","last_became_active":"2024-09-10T10:44:56.286865+0000","last_became_peered":"2024-09-10T10:44:56.286865+0000","last_unstale":"2024-09-10T10:44:57.777242+0000","last_undegraded":"2024-09-10T10:44:57.777242+0000","last_fullsized":"2024-09-10T10:44:57.777242+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:57.638288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280488+0000","last_change":"2024-09-10T10:44:56.293837+0000","last_active":"2024-09-10T10:44:57.280488+0000","last_peered":"2024-09-10T10:44:57.280488+0000","last_clean":"2024-09-10T10:44:57.280488+0000","last_became_active":"2024-09-10T10:44:56.293570+0000","last_became_peered":"2024-09-10T10:44:56.293570+0000","last_unstale":"2024-09-10T10:44:57.280488+0000","last_undegraded":"2024-09-10T10:44:57.280488+0000","last_fullsized":"2024-09-10T10:44:57.280488+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:24:43.685917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777227+0000","last_change":"2024-09-10T10:44:56.292989+0000","last_active":"2024-09-10T10:44:57.777227+0000","last_peered":"2024-09-10T10:44:57.777227+0000","last_clean":"2024-09-10T10:44:57.777227+0000","last_became_active":"2024-09-10T10:44:56.292531+0000","last_became_peered":"2024-09-10T10:44:56.292531+0000","last_unstale":"2024-09-10T10:44:57.777227+0000","last_undegraded":"2024-09-10T10:44:57.777227+0000","last_fullsized":"2024-09-10T10:44:57.777227+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:25.254597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777994+0000","last_change":"2024-09-10T10:44:56.290270+0000","last_active":"2024-09-10T10:44:57.777994+0000","last_peered":"2024-09-10T10:44:57.777994+0000","last_clean":"2024-09-10T10:44:57.777994+0000","last_became_active":"2024-09-10T10:44:56.289702+0000","last_became_peered":"2024-09-10T10:44:56.289702+0000","last_unstale":"2024-09-10T10:44:57.777994+0000","last_undegraded":"2024-09-10T10:44:57.777994+0000","last_fullsized":"2024-09-10T10:44:57.777994+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:48:51.300028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776973+0000","last_change":"2024-09-10T10:44:56.291038+0000","last_active":"2024-09-10T10:44:57.776973+0000","last_peered":"2024-09-10T10:44:57.776973+0000","last_clean":"2024-09-10T10:44:57.776973+0000","last_became_active":"2024-09-10T10:44:56.290370+0000","last_became_peered":"2024-09-10T10:44:56.290370+0000","last_unstale":"2024-09-10T10:44:57.776973+0000","last_undegraded":"2024-09-10T10:44:57.776973+0000","last_fullsized":"2024-09-10T10:44:57.776973+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:52.721674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776347+0000","last_change":"2024-09-10T10:44:56.269592+0000","last_active":"2024-09-10T10:44:57.776347+0000","last_peered":"2024-09-10T10:44:57.776347+0000","last_clean":"2024-09-10T10:44:57.776347+0000","last_became_active":"2024-09-10T10:44:56.269311+0000","last_became_peered":"2024-09-10T10:44:56.269311+0000","last_unstale":"2024-09-10T10:44:57.776347+0000","last_undegraded":"2024-09-10T10:44:57.776347+0000","last_fullsized":"2024-09-10T10:44:57.776347+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:03.032117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278618+0000","last_change":"2024-09-10T10:44:56.289759+0000","last_active":"2024-09-10T10:44:57.278618+0000","last_peered":"2024-09-10T10:44:57.278618+0000","last_clean":"2024-09-10T10:44:57.278618+0000","last_became_active":"2024-09-10T10:44:56.289511+0000","last_became_peered":"2024-09-10T10:44:56.289511+0000","last_unstale":"2024-09-10T10:44:57.278618+0000","last_undegraded":"2024-09-10T10:44:57.278618+0000","last_fullsized":"2024-09-10T10:44:57.278618+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:11.115904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276304+0000","last_change":"2024-09-10T10:44:56.287854+0000","last_active":"2024-09-10T10:44:57.276304+0000","last_peered":"2024-09-10T10:44:57.276304+0000","last_clean":"2024-09-10T10:44:57.276304+0000","last_became_active":"2024-09-10T10:44:56.287520+0000","last_became_peered":"2024-09-10T10:44:56.287520+0000","last_unstale":"2024-09-10T10:44:57.276304+0000","last_undegraded":"2024-09-10T10:44:57.276304+0000","last_fullsized":"2024-09-10T10:44:57.276304+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:57.109900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279802+0000","last_change":"2024-09-10T10:44:56.283774+0000","last_active":"2024-09-10T10:44:57.279802+0000","last_peered":"2024-09-10T10:44:57.279802+0000","last_clean":"2024-09-10T10:44:57.279802+0000","last_became_active":"2024-09-10T10:44:56.283431+0000","last_became_peered":"2024-09-10T10:44:56.283431+0000","last_unstale":"2024-09-10T10:44:57.279802+0000","last_undegraded":"2024-09-10T10:44:57.279802+0000","last_fullsized":"2024-09-10T10:44:57.279802+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:27.123025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778571+0000","last_change":"2024-09-10T10:44:56.292049+0000","last_active":"2024-09-10T10:44:57.778571+0000","last_peered":"2024-09-10T10:44:57.778571+0000","last_clean":"2024-09-10T10:44:57.778571+0000","last_became_active":"2024-09-10T10:44:56.289808+0000","last_became_peered":"2024-09-10T10:44:56.289808+0000","last_unstale":"2024-09-10T10:44:57.778571+0000","last_undegraded":"2024-09-10T10:44:57.778571+0000","last_fullsized":"2024-09-10T10:44:57.778571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:16.850667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777589+0000","last_change":"2024-09-10T10:44:56.300560+0000","last_active":"2024-09-10T10:44:57.777589+0000","last_peered":"2024-09-10T10:44:57.777589+0000","last_clean":"2024-09-10T10:44:57.777589+0000","last_became_active":"2024-09-10T10:44:56.299383+0000","last_became_peered":"2024-09-10T10:44:56.299383+0000","last_unstale":"2024-09-10T10:44:57.777589+0000","last_undegraded":"2024-09-10T10:44:57.777589+0000","last_fullsized":"2024-09-10T10:44:57.777589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:26.226623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776672+0000","last_change":"2024-09-10T10:44:56.272402+0000","last_active":"2024-09-10T10:44:57.776672+0000","last_peered":"2024-09-10T10:44:57.776672+0000","last_clean":"2024-09-10T10:44:57.776672+0000","last_became_active":"2024-09-10T10:44:56.271291+0000","last_became_peered":"2024-09-10T10:44:56.271291+0000","last_unstale":"2024-09-10T10:44:57.776672+0000","last_undegraded":"2024-09-10T10:44:57.776672+0000","last_fullsized":"2024-09-10T10:44:57.776672+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:11:16.696176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776219+0000","last_change":"2024-09-10T10:44:56.277167+0000","last_active":"2024-09-10T10:44:57.776219+0000","last_peered":"2024-09-10T10:44:57.776219+0000","last_clean":"2024-09-10T10:44:57.776219+0000","last_became_active":"2024-09-10T10:44:56.274984+0000","last_became_peered":"2024-09-10T10:44:56.274984+0000","last_unstale":"2024-09-10T10:44:57.776219+0000","last_undegraded":"2024-09-10T10:44:57.776219+0000","last_fullsized":"2024-09-10T10:44:57.776219+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:33:53.586700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776638+0000","last_change":"2024-09-10T10:44:56.297626+0000","last_active":"2024-09-10T10:44:57.776638+0000","last_peered":"2024-09-10T10:44:57.776638+0000","last_clean":"2024-09-10T10:44:57.776638+0000","last_became_active":"2024-09-10T10:44:56.297358+0000","last_became_peered":"2024-09-10T10:44:56.297358+0000","last_unstale":"2024-09-10T10:44:57.776638+0000","last_undegraded":"2024-09-10T10:44:57.776638+0000","last_fullsized":"2024-09-10T10:44:57.776638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:19.078906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278867+0000","last_change":"2024-09-10T10:44:56.303620+0000","last_active":"2024-09-10T10:44:57.278867+0000","last_peered":"2024-09-10T10:44:57.278867+0000","last_clean":"2024-09-10T10:44:57.278867+0000","last_became_active":"2024-09-10T10:44:56.303287+0000","last_became_peered":"2024-09-10T10:44:56.303287+0000","last_unstale":"2024-09-10T10:44:57.278867+0000","last_undegraded":"2024-09-10T10:44:57.278867+0000","last_fullsized":"2024-09-10T10:44:57.278867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:15.442283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776928+0000","last_change":"2024-09-10T10:44:56.282806+0000","last_active":"2024-09-10T10:44:57.776928+0000","last_peered":"2024-09-10T10:44:57.776928+0000","last_clean":"2024-09-10T10:44:57.776928+0000","last_became_active":"2024-09-10T10:44:56.282525+0000","last_became_peered":"2024-09-10T10:44:56.282525+0000","last_unstale":"2024-09-10T10:44:57.776928+0000","last_undegraded":"2024-09-10T10:44:57.776928+0000","last_fullsized":"2024-09-10T10:44:57.776928+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:04.710198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"41'4","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276390+0000","last_change":"2024-09-10T10:44:55.277458+0000","last_active":"2024-09-10T10:44:57.276390+0000","last_peered":"2024-09-10T10:44:57.276390+0000","last_clean":"2024-09-10T10:44:57.276390+0000","last_became_active":"2024-09-10T10:44:55.277216+0000","last_became_peered":"2024-09-10T10:44:55.277216+0000","last_unstale":"2024-09-10T10:44:57.276390+0000","last_undegraded":"2024-09-10T10:44:57.276390+0000","last_fullsized":"2024-09-10T10:44:57.276390+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777595+0000","last_change":"2024-09-10T10:44:56.293669+0000","last_active":"2024-09-10T10:44:57.777595+0000","last_peered":"2024-09-10T10:44:57.777595+0000","last_clean":"2024-09-10T10:44:57.777595+0000","last_became_active":"2024-09-10T10:44:56.293037+0000","last_became_peered":"2024-09-10T10:44:56.293037+0000","last_unstale":"2024-09-10T10:44:57.777595+0000","last_undegraded":"2024-09-10T10:44:57.777595+0000","last_fullsized":"2024-09-10T10:44:57.777595+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:52.721674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777666+0000","last_change":"2024-09-10T10:44:55.281076+0000","last_active":"2024-09-10T10:44:57.777666+0000","last_peered":"2024-09-10T10:44:57.777666+0000","last_clean":"2024-09-10T10:44:57.777666+0000","last_became_active":"2024-09-10T10:44:55.280788+0000","last_became_peered":"2024-09-10T10:44:55.280788+0000","last_unstale":"2024-09-10T10:44:57.777666+0000","last_undegraded":"2024-09-10T10:44:57.777666+0000","last_fullsized":"2024-09-10T10:44:57.777666+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777472+0000","last_change":"2024-09-10T10:44:56.286152+0000","last_active":"2024-09-10T10:44:57.777472+0000","last_peered":"2024-09-10T10:44:57.777472+0000","last_clean":"2024-09-10T10:44:57.777472+0000","last_became_active":"2024-09-10T10:44:56.278527+0000","last_became_peered":"2024-09-10T10:44:56.278527+0000","last_unstale":"2024-09-10T10:44:57.777472+0000","last_undegraded":"2024-09-10T10:44:57.777472+0000","last_fullsized":"2024-09-10T10:44:57.777472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:22.309398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279616+0000","last_change":"2024-09-10T10:44:55.273775+0000","last_active":"2024-09-10T10:44:57.279616+0000","last_peered":"2024-09-10T10:44:57.279616+0000","last_clean":"2024-09-10T10:44:57.279616+0000","last_became_active":"2024-09-10T10:44:55.273439+0000","last_became_peered":"2024-09-10T10:44:55.273439+0000","last_unstale":"2024-09-10T10:44:57.279616+0000","last_undegraded":"2024-09-10T10:44:57.279616+0000","last_fullsized":"2024-09-10T10:44:57.279616+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:03.004088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777735+0000","last_change":"2024-09-10T10:44:56.292665+0000","last_active":"2024-09-10T10:44:57.777735+0000","last_peered":"2024-09-10T10:44:57.777735+0000","last_clean":"2024-09-10T10:44:57.777735+0000","last_became_active":"2024-09-10T10:44:56.292386+0000","last_became_peered":"2024-09-10T10:44:56.292386+0000","last_unstale":"2024-09-10T10:44:57.777735+0000","last_undegraded":"2024-09-10T10:44:57.777735+0000","last_fullsized":"2024-09-10T10:44:57.777735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:16.125069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276296+0000","last_change":"2024-09-10T10:44:55.282020+0000","last_active":"2024-09-10T10:44:57.276296+0000","last_peered":"2024-09-10T10:44:57.276296+0000","last_clean":"2024-09-10T10:44:57.276296+0000","last_became_active":"2024-09-10T10:44:55.281745+0000","last_became_peered":"2024-09-10T10:44:55.281745+0000","last_unstale":"2024-09-10T10:44:57.276296+0000","last_undegraded":"2024-09-10T10:44:57.276296+0000","last_fullsized":"2024-09-10T10:44:57.276296+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278810+0000","last_change":"2024-09-10T10:44:56.282361+0000","last_active":"2024-09-10T10:44:57.278810+0000","last_peered":"2024-09-10T10:44:57.278810+0000","last_clean":"2024-09-10T10:44:57.278810+0000","last_became_active":"2024-09-10T10:44:56.282090+0000","last_became_peered":"2024-09-10T10:44:56.282090+0000","last_unstale":"2024-09-10T10:44:57.278810+0000","last_undegraded":"2024-09-10T10:44:57.278810+0000","last_fullsized":"2024-09-10T10:44:57.278810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:03.702317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276362+0000","last_change":"2024-09-10T10:44:55.280125+0000","last_active":"2024-09-10T10:44:57.276362+0000","last_peered":"2024-09-10T10:44:57.276362+0000","last_clean":"2024-09-10T10:44:57.276362+0000","last_became_active":"2024-09-10T10:44:55.279856+0000","last_became_peered":"2024-09-10T10:44:55.279856+0000","last_unstale":"2024-09-10T10:44:57.276362+0000","last_undegraded":"2024-09-10T10:44:57.276362+0000","last_fullsized":"2024-09-10T10:44:57.276362+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:08.563977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776310+0000","last_change":"2024-09-10T10:44:56.290318+0000","last_active":"2024-09-10T10:44:57.776310+0000","last_peered":"2024-09-10T10:44:57.776310+0000","last_clean":"2024-09-10T10:44:57.776310+0000","last_became_active":"2024-09-10T10:44:56.289777+0000","last_became_peered":"2024-09-10T10:44:56.289777+0000","last_unstale":"2024-09-10T10:44:57.776310+0000","last_undegraded":"2024-09-10T10:44:57.776310+0000","last_fullsized":"2024-09-10T10:44:57.776310+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:28:02.155492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777270+0000","last_change":"2024-09-10T10:44:55.277393+0000","last_active":"2024-09-10T10:44:57.777270+0000","last_peered":"2024-09-10T10:44:57.777270+0000","last_clean":"2024-09-10T10:44:57.777270+0000","last_became_active":"2024-09-10T10:44:55.277116+0000","last_became_peered":"2024-09-10T10:44:55.277116+0000","last_unstale":"2024-09-10T10:44:57.777270+0000","last_undegraded":"2024-09-10T10:44:57.777270+0000","last_fullsized":"2024-09-10T10:44:57.777270+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:26:11.989394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776522+0000","last_change":"2024-09-10T10:44:56.285917+0000","last_active":"2024-09-10T10:44:57.776522+0000","last_peered":"2024-09-10T10:44:57.776522+0000","last_clean":"2024-09-10T10:44:57.776522+0000","last_became_active":"2024-09-10T10:44:56.281023+0000","last_became_peered":"2024-09-10T10:44:56.281023+0000","last_unstale":"2024-09-10T10:44:57.776522+0000","last_undegraded":"2024-09-10T10:44:57.776522+0000","last_fullsized":"2024-09-10T10:44:57.776522+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:55:33.569990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278585+0000","last_change":"2024-09-10T10:44:55.274146+0000","last_active":"2024-09-10T10:44:57.278585+0000","last_peered":"2024-09-10T10:44:57.278585+0000","last_clean":"2024-09-10T10:44:57.278585+0000","last_became_active":"2024-09-10T10:44:55.273668+0000","last_became_peered":"2024-09-10T10:44:55.273668+0000","last_unstale":"2024-09-10T10:44:57.278585+0000","last_undegraded":"2024-09-10T10:44:57.278585+0000","last_fullsized":"2024-09-10T10:44:57.278585+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:58.896941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777356+0000","last_change":"2024-09-10T10:44:56.281756+0000","last_active":"2024-09-10T10:44:57.777356+0000","last_peered":"2024-09-10T10:44:57.777356+0000","last_clean":"2024-09-10T10:44:57.777356+0000","last_became_active":"2024-09-10T10:44:56.281357+0000","last_became_peered":"2024-09-10T10:44:56.281357+0000","last_unstale":"2024-09-10T10:44:57.777356+0000","last_undegraded":"2024-09-10T10:44:57.777356+0000","last_fullsized":"2024-09-10T10:44:57.777356+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:01.655224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776144+0000","last_change":"2024-09-10T10:44:55.285818+0000","last_active":"2024-09-10T10:44:57.776144+0000","last_peered":"2024-09-10T10:44:57.776144+0000","last_clean":"2024-09-10T10:44:57.776144+0000","last_became_active":"2024-09-10T10:44:55.285486+0000","last_became_peered":"2024-09-10T10:44:55.285486+0000","last_unstale":"2024-09-10T10:44:57.776144+0000","last_undegraded":"2024-09-10T10:44:57.776144+0000","last_fullsized":"2024-09-10T10:44:57.776144+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:11.890917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279439+0000","last_change":"2024-09-10T10:44:56.291787+0000","last_active":"2024-09-10T10:44:57.279439+0000","last_peered":"2024-09-10T10:44:57.279439+0000","last_clean":"2024-09-10T10:44:57.279439+0000","last_became_active":"2024-09-10T10:44:56.291292+0000","last_became_peered":"2024-09-10T10:44:56.291292+0000","last_unstale":"2024-09-10T10:44:57.279439+0000","last_undegraded":"2024-09-10T10:44:57.279439+0000","last_fullsized":"2024-09-10T10:44:57.279439+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:35:57.916807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279407+0000","last_change":"2024-09-10T10:44:55.282228+0000","last_active":"2024-09-10T10:44:57.279407+0000","last_peered":"2024-09-10T10:44:57.279407+0000","last_clean":"2024-09-10T10:44:57.279407+0000","last_became_active":"2024-09-10T10:44:55.281403+0000","last_became_peered":"2024-09-10T10:44:55.281403+0000","last_unstale":"2024-09-10T10:44:57.279407+0000","last_undegraded":"2024-09-10T10:44:57.279407+0000","last_fullsized":"2024-09-10T10:44:57.279407+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:18:53.776600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.276259+0000","last_change":"2024-09-10T10:44:56.291281+0000","last_active":"2024-09-10T10:44:57.276259+0000","last_peered":"2024-09-10T10:44:57.276259+0000","last_clean":"2024-09-10T10:44:57.276259+0000","last_became_active":"2024-09-10T10:44:56.291025+0000","last_became_peered":"2024-09-10T10:44:56.291025+0000","last_unstale":"2024-09-10T10:44:57.276259+0000","last_undegraded":"2024-09-10T10:44:57.276259+0000","last_fullsized":"2024-09-10T10:44:57.276259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:59.900371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777331+0000","last_change":"2024-09-10T10:44:55.280162+0000","last_active":"2024-09-10T10:44:57.777331+0000","last_peered":"2024-09-10T10:44:57.777331+0000","last_clean":"2024-09-10T10:44:57.777331+0000","last_became_active":"2024-09-10T10:44:55.279842+0000","last_became_peered":"2024-09-10T10:44:55.279842+0000","last_unstale":"2024-09-10T10:44:57.777331+0000","last_undegraded":"2024-09-10T10:44:57.777331+0000","last_fullsized":"2024-09-10T10:44:57.777331+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:59:29.307550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279949+0000","last_change":"2024-09-10T10:44:56.299695+0000","last_active":"2024-09-10T10:44:57.279949+0000","last_peered":"2024-09-10T10:44:57.279949+0000","last_clean":"2024-09-10T10:44:57.279949+0000","last_became_active":"2024-09-10T10:44:56.299383+0000","last_became_peered":"2024-09-10T10:44:56.299383+0000","last_unstale":"2024-09-10T10:44:57.279949+0000","last_undegraded":"2024-09-10T10:44:57.279949+0000","last_fullsized":"2024-09-10T10:44:57.279949+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:44.741069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"41'3","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777157+0000","last_change":"2024-09-10T10:44:55.274128+0000","last_active":"2024-09-10T10:44:57.777157+0000","last_peered":"2024-09-10T10:44:57.777157+0000","last_clean":"2024-09-10T10:44:57.777157+0000","last_became_active":"2024-09-10T10:44:55.273849+0000","last_became_peered":"2024-09-10T10:44:55.273849+0000","last_unstale":"2024-09-10T10:44:57.777157+0000","last_undegraded":"2024-09-10T10:44:57.777157+0000","last_fullsized":"2024-09-10T10:44:57.777157+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:00.373846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778210+0000","last_change":"2024-09-10T10:44:56.291223+0000","last_active":"2024-09-10T10:44:57.778210+0000","last_peered":"2024-09-10T10:44:57.778210+0000","last_clean":"2024-09-10T10:44:57.778210+0000","last_became_active":"2024-09-10T10:44:56.290922+0000","last_became_peered":"2024-09-10T10:44:56.290922+0000","last_unstale":"2024-09-10T10:44:57.778210+0000","last_undegraded":"2024-09-10T10:44:57.778210+0000","last_fullsized":"2024-09-10T10:44:57.778210+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:35.955689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"41'6","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279476+0000","last_change":"2024-09-10T10:44:55.280404+0000","last_active":"2024-09-10T10:44:57.279476+0000","last_peered":"2024-09-10T10:44:57.279476+0000","last_clean":"2024-09-10T10:44:57.279476+0000","last_became_active":"2024-09-10T10:44:55.277105+0000","last_became_peered":"2024-09-10T10:44:55.277105+0000","last_unstale":"2024-09-10T10:44:57.279476+0000","last_undegraded":"2024-09-10T10:44:57.279476+0000","last_fullsized":"2024-09-10T10:44:57.279476+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:57.748927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279301+0000","last_change":"2024-09-10T10:44:56.279847+0000","last_active":"2024-09-10T10:44:57.279301+0000","last_peered":"2024-09-10T10:44:57.279301+0000","last_clean":"2024-09-10T10:44:57.279301+0000","last_became_active":"2024-09-10T10:44:56.279225+0000","last_became_peered":"2024-09-10T10:44:56.279225+0000","last_unstale":"2024-09-10T10:44:57.279301+0000","last_undegraded":"2024-09-10T10:44:57.279301+0000","last_fullsized":"2024-09-10T10:44:57.279301+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:33:46.418131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278690+0000","last_change":"2024-09-10T10:44:55.283497+0000","last_active":"2024-09-10T10:44:57.278690+0000","last_peered":"2024-09-10T10:44:57.278690+0000","last_clean":"2024-09-10T10:44:57.278690+0000","last_became_active":"2024-09-10T10:44:55.280421+0000","last_became_peered":"2024-09-10T10:44:55.280421+0000","last_unstale":"2024-09-10T10:44:57.278690+0000","last_undegraded":"2024-09-10T10:44:57.278690+0000","last_fullsized":"2024-09-10T10:44:57.278690+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:56.106470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776912+0000","last_change":"2024-09-10T10:44:56.302134+0000","last_active":"2024-09-10T10:44:57.776912+0000","last_peered":"2024-09-10T10:44:57.776912+0000","last_clean":"2024-09-10T10:44:57.776912+0000","last_became_active":"2024-09-10T10:44:56.301808+0000","last_became_peered":"2024-09-10T10:44:56.301808+0000","last_unstale":"2024-09-10T10:44:57.776912+0000","last_undegraded":"2024-09-10T10:44:57.776912+0000","last_fullsized":"2024-09-10T10:44:57.776912+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:58.752357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777153+0000","last_change":"2024-09-10T10:44:55.279940+0000","last_active":"2024-09-10T10:44:57.777153+0000","last_peered":"2024-09-10T10:44:57.777153+0000","last_clean":"2024-09-10T10:44:57.777153+0000","last_became_active":"2024-09-10T10:44:55.279402+0000","last_became_peered":"2024-09-10T10:44:55.279402+0000","last_unstale":"2024-09-10T10:44:57.777153+0000","last_undegraded":"2024-09-10T10:44:57.777153+0000","last_fullsized":"2024-09-10T10:44:57.777153+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:40:18.075476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777390+0000","last_change":"2024-09-10T10:44:56.291206+0000","last_active":"2024-09-10T10:44:57.777390+0000","last_peered":"2024-09-10T10:44:57.777390+0000","last_clean":"2024-09-10T10:44:57.777390+0000","last_became_active":"2024-09-10T10:44:56.290933+0000","last_became_peered":"2024-09-10T10:44:56.290933+0000","last_unstale":"2024-09-10T10:44:57.777390+0000","last_undegraded":"2024-09-10T10:44:57.777390+0000","last_fullsized":"2024-09-10T10:44:57.777390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:24:10.365205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776424+0000","last_change":"2024-09-10T10:44:55.274369+0000","last_active":"2024-09-10T10:44:57.776424+0000","last_peered":"2024-09-10T10:44:57.776424+0000","last_clean":"2024-09-10T10:44:57.776424+0000","last_became_active":"2024-09-10T10:44:55.273965+0000","last_became_peered":"2024-09-10T10:44:55.273965+0000","last_unstale":"2024-09-10T10:44:57.776424+0000","last_undegraded":"2024-09-10T10:44:57.776424+0000","last_fullsized":"2024-09-10T10:44:57.776424+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:23:09.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777187+0000","last_change":"2024-09-10T10:44:56.275460+0000","last_active":"2024-09-10T10:44:57.777187+0000","last_peered":"2024-09-10T10:44:57.777187+0000","last_clean":"2024-09-10T10:44:57.777187+0000","last_became_active":"2024-09-10T10:44:56.275117+0000","last_became_peered":"2024-09-10T10:44:56.275117+0000","last_unstale":"2024-09-10T10:44:57.777187+0000","last_undegraded":"2024-09-10T10:44:57.777187+0000","last_fullsized":"2024-09-10T10:44:57.777187+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:26.226623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"41'8","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.275956+0000","last_change":"2024-09-10T10:44:55.277981+0000","last_active":"2024-09-10T10:44:57.275956+0000","last_peered":"2024-09-10T10:44:57.275956+0000","last_clean":"2024-09-10T10:44:57.275956+0000","last_became_active":"2024-09-10T10:44:55.276409+0000","last_became_peered":"2024-09-10T10:44:55.276409+0000","last_unstale":"2024-09-10T10:44:57.275956+0000","last_undegraded":"2024-09-10T10:44:57.275956+0000","last_fullsized":"2024-09-10T10:44:57.275956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777529+0000","last_change":"2024-09-10T10:44:56.302025+0000","last_active":"2024-09-10T10:44:57.777529+0000","last_peered":"2024-09-10T10:44:57.777529+0000","last_clean":"2024-09-10T10:44:57.777529+0000","last_became_active":"2024-09-10T10:44:56.301725+0000","last_became_peered":"2024-09-10T10:44:56.301725+0000","last_unstale":"2024-09-10T10:44:57.777529+0000","last_undegraded":"2024-09-10T10:44:57.777529+0000","last_fullsized":"2024-09-10T10:44:57.777529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:58.383061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778189+0000","last_change":"2024-09-10T10:44:55.286607+0000","last_active":"2024-09-10T10:44:57.778189+0000","last_peered":"2024-09-10T10:44:57.778189+0000","last_clean":"2024-09-10T10:44:57.778189+0000","last_became_active":"2024-09-10T10:44:55.284761+0000","last_became_peered":"2024-09-10T10:44:55.284761+0000","last_unstale":"2024-09-10T10:44:57.778189+0000","last_undegraded":"2024-09-10T10:44:57.778189+0000","last_fullsized":"2024-09-10T10:44:57.778189+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:31.558644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777366+0000","last_change":"2024-09-10T10:44:56.299016+0000","last_active":"2024-09-10T10:44:57.777366+0000","last_peered":"2024-09-10T10:44:57.777366+0000","last_clean":"2024-09-10T10:44:57.777366+0000","last_became_active":"2024-09-10T10:44:56.298756+0000","last_became_peered":"2024-09-10T10:44:56.298756+0000","last_unstale":"2024-09-10T10:44:57.777366+0000","last_undegraded":"2024-09-10T10:44:57.777366+0000","last_fullsized":"2024-09-10T10:44:57.777366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:17:09.567407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777560+0000","last_change":"2024-09-10T10:44:55.274312+0000","last_active":"2024-09-10T10:44:57.777560+0000","last_peered":"2024-09-10T10:44:57.777560+0000","last_clean":"2024-09-10T10:44:57.777560+0000","last_became_active":"2024-09-10T10:44:55.273980+0000","last_became_peered":"2024-09-10T10:44:55.273980+0000","last_unstale":"2024-09-10T10:44:57.777560+0000","last_undegraded":"2024-09-10T10:44:57.777560+0000","last_fullsized":"2024-09-10T10:44:57.777560+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:14.947355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778040+0000","last_change":"2024-09-10T10:44:56.300433+0000","last_active":"2024-09-10T10:44:57.778040+0000","last_peered":"2024-09-10T10:44:57.778040+0000","last_clean":"2024-09-10T10:44:57.778040+0000","last_became_active":"2024-09-10T10:44:56.300157+0000","last_became_peered":"2024-09-10T10:44:56.300157+0000","last_unstale":"2024-09-10T10:44:57.778040+0000","last_undegraded":"2024-09-10T10:44:57.778040+0000","last_fullsized":"2024-09-10T10:44:57.778040+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:35.148654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777199+0000","last_change":"2024-09-10T10:44:55.286650+0000","last_active":"2024-09-10T10:44:57.777199+0000","last_peered":"2024-09-10T10:44:57.777199+0000","last_clean":"2024-09-10T10:44:57.777199+0000","last_became_active":"2024-09-10T10:44:55.286360+0000","last_became_peered":"2024-09-10T10:44:55.286360+0000","last_unstale":"2024-09-10T10:44:57.777199+0000","last_undegraded":"2024-09-10T10:44:57.777199+0000","last_fullsized":"2024-09-10T10:44:57.777199+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:11:20.292817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280741+0000","last_change":"2024-09-10T10:44:56.288750+0000","last_active":"2024-09-10T10:44:57.280741+0000","last_peered":"2024-09-10T10:44:57.280741+0000","last_clean":"2024-09-10T10:44:57.280741+0000","last_became_active":"2024-09-10T10:44:56.288435+0000","last_became_peered":"2024-09-10T10:44:56.288435+0000","last_unstale":"2024-09-10T10:44:57.280741+0000","last_undegraded":"2024-09-10T10:44:57.280741+0000","last_fullsized":"2024-09-10T10:44:57.280741+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:49.960568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280412+0000","last_change":"2024-09-10T10:44:55.278871+0000","last_active":"2024-09-10T10:44:57.280412+0000","last_peered":"2024-09-10T10:44:57.280412+0000","last_clean":"2024-09-10T10:44:57.280412+0000","last_became_active":"2024-09-10T10:44:55.278608+0000","last_became_peered":"2024-09-10T10:44:55.278608+0000","last_unstale":"2024-09-10T10:44:57.280412+0000","last_undegraded":"2024-09-10T10:44:57.280412+0000","last_fullsized":"2024-09-10T10:44:57.280412+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:47.361070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776732+0000","last_change":"2024-09-10T10:44:56.276552+0000","last_active":"2024-09-10T10:44:57.776732+0000","last_peered":"2024-09-10T10:44:57.776732+0000","last_clean":"2024-09-10T10:44:57.776732+0000","last_became_active":"2024-09-10T10:44:56.274971+0000","last_became_peered":"2024-09-10T10:44:56.274971+0000","last_unstale":"2024-09-10T10:44:57.776732+0000","last_undegraded":"2024-09-10T10:44:57.776732+0000","last_fullsized":"2024-09-10T10:44:57.776732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:19.526916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280206+0000","last_change":"2024-09-10T10:44:55.277794+0000","last_active":"2024-09-10T10:44:57.280206+0000","last_peered":"2024-09-10T10:44:57.280206+0000","last_clean":"2024-09-10T10:44:57.280206+0000","last_became_active":"2024-09-10T10:44:55.277054+0000","last_became_peered":"2024-09-10T10:44:55.277054+0000","last_unstale":"2024-09-10T10:44:57.280206+0000","last_undegraded":"2024-09-10T10:44:57.280206+0000","last_fullsized":"2024-09-10T10:44:57.280206+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:45:29.282915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278532+0000","last_change":"2024-09-10T10:44:56.299668+0000","last_active":"2024-09-10T10:44:57.278532+0000","last_peered":"2024-09-10T10:44:57.278532+0000","last_clean":"2024-09-10T10:44:57.278532+0000","last_became_active":"2024-09-10T10:44:56.299361+0000","last_became_peered":"2024-09-10T10:44:56.299361+0000","last_unstale":"2024-09-10T10:44:57.278532+0000","last_undegraded":"2024-09-10T10:44:57.278532+0000","last_fullsized":"2024-09-10T10:44:57.278532+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:48:48.074145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777153+0000","last_change":"2024-09-10T10:44:55.283286+0000","last_active":"2024-09-10T10:44:57.777153+0000","last_peered":"2024-09-10T10:44:57.777153+0000","last_clean":"2024-09-10T10:44:57.777153+0000","last_became_active":"2024-09-10T10:44:55.283045+0000","last_became_peered":"2024-09-10T10:44:55.283045+0000","last_unstale":"2024-09-10T10:44:57.777153+0000","last_undegraded":"2024-09-10T10:44:57.777153+0000","last_fullsized":"2024-09-10T10:44:57.777153+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:04.004546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777195+0000","last_change":"2024-09-10T10:44:56.290702+0000","last_active":"2024-09-10T10:44:57.777195+0000","last_peered":"2024-09-10T10:44:57.777195+0000","last_clean":"2024-09-10T10:44:57.777195+0000","last_became_active":"2024-09-10T10:44:56.289224+0000","last_became_peered":"2024-09-10T10:44:56.289224+0000","last_unstale":"2024-09-10T10:44:57.777195+0000","last_undegraded":"2024-09-10T10:44:57.777195+0000","last_fullsized":"2024-09-10T10:44:57.777195+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:16.125069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776775+0000","last_change":"2024-09-10T10:44:55.285243+0000","last_active":"2024-09-10T10:44:57.776775+0000","last_peered":"2024-09-10T10:44:57.776775+0000","last_clean":"2024-09-10T10:44:57.776775+0000","last_became_active":"2024-09-10T10:44:55.284621+0000","last_became_peered":"2024-09-10T10:44:55.284621+0000","last_unstale":"2024-09-10T10:44:57.776775+0000","last_undegraded":"2024-09-10T10:44:57.776775+0000","last_fullsized":"2024-09-10T10:44:57.776775+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:04.830592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.278548+0000","last_change":"2024-09-10T10:44:56.299727+0000","last_active":"2024-09-10T10:44:57.278548+0000","last_peered":"2024-09-10T10:44:57.278548+0000","last_clean":"2024-09-10T10:44:57.278548+0000","last_became_active":"2024-09-10T10:44:56.299425+0000","last_became_peered":"2024-09-10T10:44:56.299425+0000","last_unstale":"2024-09-10T10:44:57.278548+0000","last_undegraded":"2024-09-10T10:44:57.278548+0000","last_fullsized":"2024-09-10T10:44:57.278548+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:20.938258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776582+0000","last_change":"2024-09-10T10:44:55.279589+0000","last_active":"2024-09-10T10:44:57.776582+0000","last_peered":"2024-09-10T10:44:57.776582+0000","last_clean":"2024-09-10T10:44:57.776582+0000","last_became_active":"2024-09-10T10:44:55.279308+0000","last_became_peered":"2024-09-10T10:44:55.279308+0000","last_unstale":"2024-09-10T10:44:57.776582+0000","last_undegraded":"2024-09-10T10:44:57.776582+0000","last_fullsized":"2024-09-10T10:44:57.776582+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:37.011446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777016+0000","last_change":"2024-09-10T10:44:56.301858+0000","last_active":"2024-09-10T10:44:57.777016+0000","last_peered":"2024-09-10T10:44:57.777016+0000","last_clean":"2024-09-10T10:44:57.777016+0000","last_became_active":"2024-09-10T10:44:56.301566+0000","last_became_peered":"2024-09-10T10:44:56.301566+0000","last_unstale":"2024-09-10T10:44:57.777016+0000","last_undegraded":"2024-09-10T10:44:57.777016+0000","last_fullsized":"2024-09-10T10:44:57.777016+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:05.007976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778142+0000","last_change":"2024-09-10T10:44:55.280439+0000","last_active":"2024-09-10T10:44:57.778142+0000","last_peered":"2024-09-10T10:44:57.778142+0000","last_clean":"2024-09-10T10:44:57.778142+0000","last_became_active":"2024-09-10T10:44:55.280054+0000","last_became_peered":"2024-09-10T10:44:55.280054+0000","last_unstale":"2024-09-10T10:44:57.778142+0000","last_undegraded":"2024-09-10T10:44:57.778142+0000","last_fullsized":"2024-09-10T10:44:57.778142+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:36.853344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778578+0000","last_change":"2024-09-10T10:44:56.304438+0000","last_active":"2024-09-10T10:44:57.778578+0000","last_peered":"2024-09-10T10:44:57.778578+0000","last_clean":"2024-09-10T10:44:57.778578+0000","last_became_active":"2024-09-10T10:44:56.303822+0000","last_became_peered":"2024-09-10T10:44:56.303822+0000","last_unstale":"2024-09-10T10:44:57.778578+0000","last_undegraded":"2024-09-10T10:44:57.778578+0000","last_fullsized":"2024-09-10T10:44:57.778578+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:38.502185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777415+0000","last_change":"2024-09-10T10:44:55.287203+0000","last_active":"2024-09-10T10:44:57.777415+0000","last_peered":"2024-09-10T10:44:57.777415+0000","last_clean":"2024-09-10T10:44:57.777415+0000","last_became_active":"2024-09-10T10:44:55.286932+0000","last_became_peered":"2024-09-10T10:44:55.286932+0000","last_unstale":"2024-09-10T10:44:57.777415+0000","last_undegraded":"2024-09-10T10:44:57.777415+0000","last_fullsized":"2024-09-10T10:44:57.777415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:04.830592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.280685+0000","last_change":"2024-09-10T10:44:56.275769+0000","last_active":"2024-09-10T10:44:57.280685+0000","last_peered":"2024-09-10T10:44:57.280685+0000","last_clean":"2024-09-10T10:44:57.280685+0000","last_became_active":"2024-09-10T10:44:56.275397+0000","last_became_peered":"2024-09-10T10:44:56.275397+0000","last_unstale":"2024-09-10T10:44:57.280685+0000","last_undegraded":"2024-09-10T10:44:57.280685+0000","last_fullsized":"2024-09-10T10:44:57.280685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:00:36.768449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777303+0000","last_change":"2024-09-10T10:44:55.287602+0000","last_active":"2024-09-10T10:44:57.777303+0000","last_peered":"2024-09-10T10:44:57.777303+0000","last_clean":"2024-09-10T10:44:57.777303+0000","last_became_active":"2024-09-10T10:44:55.287354+0000","last_became_peered":"2024-09-10T10:44:55.287354+0000","last_unstale":"2024-09-10T10:44:57.777303+0000","last_undegraded":"2024-09-10T10:44:57.777303+0000","last_fullsized":"2024-09-10T10:44:57.777303+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:42:31.558644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776608+0000","last_change":"2024-09-10T10:44:56.287094+0000","last_active":"2024-09-10T10:44:57.776608+0000","last_peered":"2024-09-10T10:44:57.776608+0000","last_clean":"2024-09-10T10:44:57.776608+0000","last_became_active":"2024-09-10T10:44:56.286818+0000","last_became_peered":"2024-09-10T10:44:56.286818+0000","last_unstale":"2024-09-10T10:44:57.776608+0000","last_undegraded":"2024-09-10T10:44:57.776608+0000","last_fullsized":"2024-09-10T10:44:57.776608+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:38.014875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279184+0000","last_change":"2024-09-10T10:44:55.283449+0000","last_active":"2024-09-10T10:44:57.279184+0000","last_peered":"2024-09-10T10:44:57.279184+0000","last_clean":"2024-09-10T10:44:57.279184+0000","last_became_active":"2024-09-10T10:44:55.281725+0000","last_became_peered":"2024-09-10T10:44:55.281725+0000","last_unstale":"2024-09-10T10:44:57.279184+0000","last_undegraded":"2024-09-10T10:44:57.279184+0000","last_fullsized":"2024-09-10T10:44:57.279184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:15.376747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776991+0000","last_change":"2024-09-10T10:44:56.282368+0000","last_active":"2024-09-10T10:44:57.776991+0000","last_peered":"2024-09-10T10:44:57.776991+0000","last_clean":"2024-09-10T10:44:57.776991+0000","last_became_active":"2024-09-10T10:44:56.282157+0000","last_became_peered":"2024-09-10T10:44:56.282157+0000","last_unstale":"2024-09-10T10:44:57.776991+0000","last_undegraded":"2024-09-10T10:44:57.776991+0000","last_fullsized":"2024-09-10T10:44:57.776991+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:45:04.007517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777287+0000","last_change":"2024-09-10T10:44:55.281075+0000","last_active":"2024-09-10T10:44:57.777287+0000","last_peered":"2024-09-10T10:44:57.777287+0000","last_clean":"2024-09-10T10:44:57.777287+0000","last_became_active":"2024-09-10T10:44:55.280803+0000","last_became_peered":"2024-09-10T10:44:55.280803+0000","last_unstale":"2024-09-10T10:44:57.777287+0000","last_undegraded":"2024-09-10T10:44:57.777287+0000","last_fullsized":"2024-09-10T10:44:57.777287+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:58.896941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776698+0000","last_change":"2024-09-10T10:44:56.292530+0000","last_active":"2024-09-10T10:44:57.776698+0000","last_peered":"2024-09-10T10:44:57.776698+0000","last_clean":"2024-09-10T10:44:57.776698+0000","last_became_active":"2024-09-10T10:44:56.292288+0000","last_became_peered":"2024-09-10T10:44:56.292288+0000","last_unstale":"2024-09-10T10:44:57.776698+0000","last_undegraded":"2024-09-10T10:44:57.776698+0000","last_fullsized":"2024-09-10T10:44:57.776698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:12.004680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776445+0000","last_change":"2024-09-10T10:44:55.280515+0000","last_active":"2024-09-10T10:44:57.776445+0000","last_peered":"2024-09-10T10:44:57.776445+0000","last_clean":"2024-09-10T10:44:57.776445+0000","last_became_active":"2024-09-10T10:44:55.280197+0000","last_became_peered":"2024-09-10T10:44:55.280197+0000","last_unstale":"2024-09-10T10:44:57.776445+0000","last_undegraded":"2024-09-10T10:44:57.776445+0000","last_fullsized":"2024-09-10T10:44:57.776445+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:58.581279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777378+0000","last_change":"2024-09-10T10:44:56.286065+0000","last_active":"2024-09-10T10:44:57.777378+0000","last_peered":"2024-09-10T10:44:57.777378+0000","last_clean":"2024-09-10T10:44:57.777378+0000","last_became_active":"2024-09-10T10:44:56.284500+0000","last_became_peered":"2024-09-10T10:44:56.284500+0000","last_unstale":"2024-09-10T10:44:57.777378+0000","last_undegraded":"2024-09-10T10:44:57.777378+0000","last_fullsized":"2024-09-10T10:44:57.777378+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:43:26.708315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.279082+0000","last_change":"2024-09-10T10:44:55.276223+0000","last_active":"2024-09-10T10:44:57.279082+0000","last_peered":"2024-09-10T10:44:57.279082+0000","last_clean":"2024-09-10T10:44:57.279082+0000","last_became_active":"2024-09-10T10:44:55.275926+0000","last_became_peered":"2024-09-10T10:44:55.275926+0000","last_unstale":"2024-09-10T10:44:57.279082+0000","last_undegraded":"2024-09-10T10:44:57.279082+0000","last_fullsized":"2024-09-10T10:44:57.279082+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:11:34.952260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.777085+0000","last_change":"2024-09-10T10:44:56.274638+0000","last_active":"2024-09-10T10:44:57.777085+0000","last_peered":"2024-09-10T10:44:57.777085+0000","last_clean":"2024-09-10T10:44:57.777085+0000","last_became_active":"2024-09-10T10:44:56.274183+0000","last_became_peered":"2024-09-10T10:44:56.274183+0000","last_unstale":"2024-09-10T10:44:57.777085+0000","last_undegraded":"2024-09-10T10:44:57.777085+0000","last_fullsized":"2024-09-10T10:44:57.777085+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:55.250795+0000","last_clean_scrub_stamp":"2024-09-10T10:44:55.250795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:43:08.334905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.778210+0000","last_change":"2024-09-10T10:44:55.284148+0000","last_active":"2024-09-10T10:44:57.778210+0000","last_peered":"2024-09-10T10:44:57.778210+0000","last_clean":"2024-09-10T10:44:57.778210+0000","last_became_active":"2024-09-10T10:44:55.283869+0000","last_became_peered":"2024-09-10T10:44:55.283869+0000","last_unstale":"2024-09-10T10:44:57.778210+0000","last_undegraded":"2024-09-10T10:44:57.778210+0000","last_fullsized":"2024-09-10T10:44:57.778210+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:44:54.247366+0000","last_clean_scrub_stamp":"2024-09-10T10:44:54.247366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:13.465795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":205,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-10T10:44:57.776257+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:44:57.776257+0000","last_peered":"2024-09-10T10:44:57.776257+0000","last_clean":"2024-09-10T10:44:57.776257+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:44:57.776257+0000","last_undegraded":"2024-09-10T10:44:57.776257+0000","last_fullsized":"2024-09-10T10:44:57.776257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":483328,"data_stored":206102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":206066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":499712,"data_stored":207768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":483328,"data_stored":206122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":495616,"data_stored":208226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1077248,"data_stored":796422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31664,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340176,"statfs":{"total":96636764160,"available":96604340224,"internally_reserved":0,"allocated":487424,"data_stored":206124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1077248,"data_stored":796982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:45:05.582 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:45:05.583 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:45:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.781+0000 7fa5e0a00640 1 Processor -- start 2024-09-10T10:45:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.782+0000 7fa5e0a00640 1 -- start start 2024-09-10T10:45:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5e0a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5e0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5e0a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5dc0b30e0 con 0x7fa5dc0b4af0 2024-09-10T10:45:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5e0a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa5dc06edc0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5e0a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b33c0 con 0x7fa5dc156740 2024-09-10T10:45:05.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5da800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5da800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36012/0 (socket says 172.21.15.28:36012) 2024-09-10T10:45:05.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5da800640 1 -- 172.21.15.28:0/1631533235 learned_addr learned my addr 172.21.15.28:0/1631533235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:05.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.783+0000 7fa5d9e00640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.784+0000 7fa5da800640 1 -- 172.21.15.28:0/1631533235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 msgr2=0x7fa5dc0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.784+0000 7fa5da800640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.784+0000 7fa5da800640 1 -- 172.21.15.28:0/1631533235 >> v1:172.21.15.138:6789/0 conn(0x7fa5dc0b4af0 legacy=0x7fa5dc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.784+0000 7fa5da800640 1 -- 172.21.15.28:0/1631533235 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc0b36a0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.785+0000 7fa5da800640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009e50 tx=0x7fa5d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c31f8ba18f41b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.785+0000 7fa5d9400640 1 -- 172.21.15.28:0/1631533235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4036070 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.785+0000 7fa5d9400640 1 -- 172.21.15.28:0/1631533235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d4031600 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.785+0000 7fa5d9400640 1 -- 172.21.15.28:0/1631533235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40319d0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 -- 172.21.15.28:0/1631533235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 msgr2=0x7fa5dc0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009e50 tx=0x7fa5d402a710 comp rx=0 tx=0).stop 2024-09-10T10:45:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 -- 172.21.15.28:0/1631533235 shutdown_connections 2024-09-10T10:45:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ce060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/1631533235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 -- 172.21.15.28:0/1631533235 >> 172.21.15.28:0/1631533235 conn(0x7fa5dc06bdf0 msgr2=0x7fa5dc06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 -- 172.21.15.28:0/1631533235 shutdown_connections 2024-09-10T10:45:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.786+0000 7fa5e0a00640 1 -- 172.21.15.28:0/1631533235 wait complete. 2024-09-10T10:45:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.787+0000 7fa5e0a00640 1 Processor -- start 2024-09-10T10:45:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.788+0000 7fa5e0a00640 1 -- start start 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.788+0000 7fa5e0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 0x7fa5dc15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5e0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5e0a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5e0a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0d0890 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5e0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc0b3c60 con 0x7fa5dc0b4af0 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5e0a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5dc0d0a10 con 0x7fa5dc156740 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36018/0 (socket says 172.21.15.28:36018) 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 -- 172.21.15.28:0/890867723 learned_addr learned my addr 172.21.15.28:0/890867723 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d9e00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 0x7fa5dc15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5da800640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 -- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 msgr2=0x7fa5dc0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 -- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 msgr2=0x7fa5dc15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 0x7fa5dc15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d1400640 1 -- 172.21.15.28:0/890867723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc15eff0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5da800640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.789+0000 7fa5d9e00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 0x7fa5dc15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5d1400640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c430 tx=0x7fa5c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40052d0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c4005490 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4007260 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc15f2d0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.790+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1f5cd0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.791+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5dc068f50 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.792+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5c4007400 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.793+0000 7fa5d2a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 0x7fa5b806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:05.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.793+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fa5c408a4d0 con 0x7fa5dc0cfea0 2024-09-10T10:45:05.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.793+0000 7fa5d9e00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 0x7fa5b806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:05.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.794+0000 7fa5d9e00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 0x7fa5b806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009af0 tx=0x7fa5cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:05.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:05.797+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5c4058ce0 con 0x7fa5dc0cfea0 2024-09-10T10:45:06.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.048+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa5dc156c40 con 0x7fa5dc0cfea0 2024-09-10T10:45:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.051+0000 7fa5d2a00640 1 -- 172.21.15.28:0/890867723 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fa5c40585e0 con 0x7fa5dc0cfea0 2024-09-10T10:45:06.052 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:45:06.052 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:45:06.052 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:45:06.053 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:45:06.053 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:45:06.053 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:45:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 msgr2=0x7fa5b806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 0x7fa5b806e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009af0 tx=0x7fa5cc008040 comp rx=0 tx=0).stop 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 msgr2=0x7fa5dc0ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c430 tx=0x7fa5c400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 shutdown_connections 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b806be60 0x7fa5b806e320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5dc156740 0x7fa5dc0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5dc0cfea0 0x7fa5dc0ae360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 --2- 172.21.15.28:0/890867723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5dc0b4af0 0x7fa5dc15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 >> 172.21.15.28:0/890867723 conn(0x7fa5dc06bdf0 msgr2=0x7fa5dc0b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 shutdown_connections 2024-09-10T10:45:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.054+0000 7fa5e0a00640 1 -- 172.21.15.28:0/890867723 wait complete. 2024-09-10T10:45:06.070 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.225+0000 7fb5fde00640 1 Processor -- start 2024-09-10T10:45:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.225+0000 7fb5fde00640 1 -- start start 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.226+0000 7fb5fde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5f80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5fde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5f80cd3f0 0x7fb5f806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5fde00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5f80cdbe0 con 0x7fb5f8072d30 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5fde00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb5f80728a0 con 0x7fb5f80cd3f0 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5fde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80cd900 con 0x7fb5f815b300 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5f7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb5f8072d30 0x7fb5f80b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40196/0 (socket says 172.21.15.28:40196) 2024-09-10T10:45:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5f7400640 1 -- 172.21.15.28:0/2986500580 learned_addr learned my addr 172.21.15.28:0/2986500580 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5f6a00640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5f80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5efe00640 1 --2- 172.21.15.28:0/2986500580 >> v2:172.21.15.138:3300/0 conn(0x7fb5f80cd3f0 0x7fb5f806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519674765 0 0) 0x7fb5f80cdbe0 con 0x7fb5f8072d30 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.227+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5dc003520 con 0x7fb5f8072d30 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3793628040 0 0) 0x7fb5dc003520 con 0x7fb5f8072d30 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 msgr2=0x7fb5f80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5f80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 >> v2:172.21.15.138:3300/0 conn(0x7fb5f80cd3f0 msgr2=0x7fb5f806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 --2- 172.21.15.28:0/2986500580 >> v2:172.21.15.138:3300/0 conn(0x7fb5f80cd3f0 0x7fb5f806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f8072250 con 0x7fb5f8072d30 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fb5f00019f0 con 0x7fb5f8072d30 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 >> v1:172.21.15.28:6789/0 conn(0x7fb5f8072d30 legacy=0x7fb5f80b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5dc003f80 0x7fb5dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80728a0 con 0x7fb5dc004d40 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80cd900 con 0x7fb5dc003f80 2024-09-10T10:45:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5dc005990 con 0x7fb5f815b300 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f7400640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5f6a00640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.228+0000 7fb5efe00640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5dc003f80 0x7fb5dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.229+0000 7fb5f7400640 1 -- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 msgr2=0x7fb5dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.229+0000 7fb5f7400640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b300 0x7fb5dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.229+0000 7fb5f7400640 1 -- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5dc003f80 msgr2=0x7fb5dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.229+0000 7fb5f7400640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5dc003f80 0x7fb5dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.229+0000 7fb5f7400640 1 -- 172.21.15.28:0/2986500580 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5dc003520 con 0x7fb5dc004d40 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5f7400640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5dc0051c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000a800 tx=0x7fb5dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c40bb6c8af1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000a3f0 con 0x7fb5dc004d40 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f002f070 con 0x7fb5dc004d40 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5f6000640 1 -- 172.21.15.28:0/2986500580 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f002bc90 con 0x7fb5dc004d40 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5fde00640 1 -- 172.21.15.28:0/2986500580 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 msgr2=0x7fb5dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5fde00640 1 --2- 172.21.15.28:0/2986500580 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5dc0051c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000a800 tx=0x7fb5dc005830 comp rx=0 tx=0).stop 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.230+0000 7fb5f7400640 1 -- 172.21.15.28:0/2986500580 reap_dead start 2024-09-10T10:45:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.231+0000 7fb5fde00640 1 -- 172.21.15.28:0/2986500580 shutdown_connections 2024-09-10T10:45:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.231+0000 7fb5fde00640 1 -- 172.21.15.28:0/2986500580 >> 172.21.15.28:0/2986500580 conn(0x7fb5f8067d70 msgr2=0x7fb5f806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.231+0000 7fb5fde00640 1 -- 172.21.15.28:0/2986500580 shutdown_connections 2024-09-10T10:45:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.231+0000 7fb5fde00640 1 -- 172.21.15.28:0/2986500580 wait complete. 2024-09-10T10:45:06.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.232+0000 7fb5fde00640 1 Processor -- start 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.232+0000 7fb5fde00640 1 -- start start 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5f81521e0 0x7fb5f81f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b6e0 0x7fb5f815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80d0160 con 0x7fb5dc004d40 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80cffe0 con 0x7fb5f81521e0 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5fde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80cceb0 con 0x7fb5f815b6e0 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5f6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36026/0 (socket says 172.21.15.28:36026) 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.233+0000 7fb5f6a00640 1 -- 172.21.15.28:0/3071976185 learned_addr learned my addr 172.21.15.28:0/3071976185 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 -- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b6e0 msgr2=0x7fb5f815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b6e0 0x7fb5f815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 -- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5f81521e0 msgr2=0x7fb5f81f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5f81521e0 0x7fb5f81f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 -- 172.21.15.28:0/3071976185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f8151140 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5f6a00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80119d0 tx=0x7fb5e8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800fad0 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e800fc70 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8014440 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.234+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f8151420 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.235+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f81519e0 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.236+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f806fb60 con 0x7fb5dc004d40 2024-09-10T10:45:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.236+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5e8012040 con 0x7fb5dc004d40 2024-09-10T10:45:06.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.238+0000 7fb5ef400640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:06.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.238+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fb5e808b690 con 0x7fb5dc004d40 2024-09-10T10:45:06.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.238+0000 7fb5efe00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:06.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.239+0000 7fb5efe00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 0x7fb5d406e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006e80 tx=0x7fb5e0006df0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.242+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5f806fb60 con 0x7fb5dc004d40 2024-09-10T10:45:06.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.462+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5f806ee80 con 0x7fb5dc004d40 2024-09-10T10:45:06.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.462+0000 7fb5ef400640 1 -- 172.21.15.28:0/3071976185 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fb5e8019030 con 0x7fb5dc004d40 2024-09-10T10:45:06.463 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:06.464 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":18,"btime":"2024-09-10T10:44:59:280567+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:44:58.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4845":{"gid":4845,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3203683106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3203683106},{"type":"v1","addr":"172.21.15.28:6835","nonce":3203683106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4845,"qdb_cluster":[4845]},"id":2}]} 2024-09-10T10:45:06.464 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 18 2024-09-10T10:45:06.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 msgr2=0x7fb5d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 0x7fb5d406e2f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0006e80 tx=0x7fb5e0006df0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 msgr2=0x7fb5f81f0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:06.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80119d0 tx=0x7fb5e8011e90 comp rx=0 tx=0).stop 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 shutdown_connections 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5d406be30 0x7fb5d406e2f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5f815b6e0 0x7fb5f815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5f81521e0 0x7fb5f81f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 --2- 172.21.15.28:0/3071976185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5dc004d40 0x7fb5f81f0360 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 >> 172.21.15.28:0/3071976185 conn(0x7fb5f8067d70 msgr2=0x7fb5f806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 shutdown_connections 2024-09-10T10:45:06.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:06.465+0000 7fb5fde00640 1 -- 172.21.15.28:0/3071976185 wait complete. 2024-09-10T10:45:06.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:44:55.704429+0000', 'modified': '2024-09-10T10:44:58.705324+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4845}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4845': {'gid': 4845, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6835/3203683106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 3203683106}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 3203683106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4845, 'qdb_cluster': [4845]} 2024-09-10T10:45:06.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:45:06.886 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:45:07.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.044+0000 7f1af6800640 1 Processor -- start 2024-09-10T10:45:07.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.044+0000 7f1af6800640 1 -- start start 2024-09-10T10:45:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1af00d1f90 0x7f1af00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af6800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af00cf000 con 0x7f1af015e660 2024-09-10T10:45:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af6800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1af006d3e0 con 0x7f1af00d1f90 2024-09-10T10:45:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af6800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00b37d0 con 0x7f1af00d0dd0 2024-09-10T10:45:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1af00d1f90 0x7f1af00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af4a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1af015e660 0x7f1af006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40210/0 (socket says 172.21.15.28:40210) 2024-09-10T10:45:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1af00d1f90 0x7f1af00cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36034/0 (socket says 172.21.15.28:36034) 2024-09-10T10:45:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.045+0000 7f1af4a00640 1 -- 172.21.15.28:0/3394941939 learned_addr learned my addr 172.21.15.28:0/3394941939 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5400640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5e00640 1 -- 172.21.15.28:0/3394941939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 msgr2=0x7f1af00d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5e00640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af00d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5e00640 1 -- 172.21.15.28:0/3394941939 >> v1:172.21.15.28:6789/0 conn(0x7f1af015e660 legacy=0x7f1af006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5e00640 1 -- 172.21.15.28:0/3394941939 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00d16f0 con 0x7f1af00d1f90 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1aefe00640 1 -- 172.21.15.28:0/3394941939 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561997398 0 0) 0x7f1af00cf000 con 0x7f1af015e660 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.046+0000 7f1af5400640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af00d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.047+0000 7f1af5e00640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00cf7c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009d90 tx=0x7f1ae802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843e4f209c82d416 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.047+0000 7f1aefe00640 1 -- 172.21.15.28:0/3394941939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8007740 con 0x7f1af00d1f90 2024-09-10T10:45:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.047+0000 7f1aefe00640 1 -- 172.21.15.28:0/3394941939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae80078e0 con 0x7f1af00d1f90 2024-09-10T10:45:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.047+0000 7f1aefe00640 1 -- 172.21.15.28:0/3394941939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8007cb0 con 0x7f1af00d1f90 2024-09-10T10:45:07.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 -- 172.21.15.28:0/3394941939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 msgr2=0x7f1af00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00cf7c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009d90 tx=0x7f1ae802a680 comp rx=0 tx=0).stop 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 -- 172.21.15.28:0/3394941939 shutdown_connections 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00cf7c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 --2- 172.21.15.28:0/3394941939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af00d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 -- 172.21.15.28:0/3394941939 >> 172.21.15.28:0/3394941939 conn(0x7f1af008bfc0 msgr2=0x7f1af00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 -- 172.21.15.28:0/3394941939 shutdown_connections 2024-09-10T10:45:07.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.048+0000 7f1af6800640 1 -- 172.21.15.28:0/3394941939 wait complete. 2024-09-10T10:45:07.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.049+0000 7f1af6800640 1 Processor -- start 2024-09-10T10:45:07.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 -- start start 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af0151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af015e660 0x7f1af00b2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00d2c60 con 0x7f1af00d1f90 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00cf330 con 0x7f1af015e660 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.050+0000 7f1af6800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00d2f40 con 0x7f1af00d0dd0 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36040/0 (socket says 172.21.15.28:36040) 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 -- 172.21.15.28:0/1900628537 learned_addr learned my addr 172.21.15.28:0/1900628537 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 -- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 msgr2=0x7f1af0151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af0151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 -- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af015e660 msgr2=0x7f1af00b2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af015e660 0x7f1af00b2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 -- 172.21.15.28:0/1900628537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00cc0d0 con 0x7f1af00d1f90 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c410 tx=0x7f1adc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.051+0000 7f1aed600640 1 -- 172.21.15.28:0/1900628537 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc0074e0 con 0x7f1af00d1f90 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.052+0000 7f1aed600640 1 -- 172.21.15.28:0/1900628537 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc0047e0 con 0x7f1af00d1f90 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.052+0000 7f1aed600640 1 -- 172.21.15.28:0/1900628537 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc004bd0 con 0x7f1af00d1f90 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.052+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0151c40 con 0x7f1af00d1f90 2024-09-10T10:45:07.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.052+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af00b7820 con 0x7f1af00d1f90 2024-09-10T10:45:07.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.054+0000 7f1aed600640 1 -- 172.21.15.28:0/1900628537 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1adc011070 con 0x7f1af00d1f90 2024-09-10T10:45:07.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.055+0000 7f1aed600640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.055+0000 7f1aed600640 1 -- 172.21.15.28:0/1900628537 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f1adc08ac90 con 0x7f1af00d1f90 2024-09-10T10:45:07.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.055+0000 7f1af5400640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.056+0000 7f1afae7d740 1 Processor -- start 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.056+0000 7f1afae7d740 1 -- start start 2024-09-10T10:45:07.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.056+0000 7f1af5400640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 0x7f1ad006e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4006fd0 tx=0x7f1ae4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e03640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3dbe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3d1c3b0 con 0x55b0f3dbf630 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3d21410 con 0x55b0f3dbdea0 2024-09-10T10:45:07.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b0f3cc9620 con 0x55b0f3dbb660 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1af5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1af5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36044/0 (socket says 172.21.15.28:36044) 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1af5e00640 1 -- 172.21.15.28:0/4111173141 learned_addr learned my addr 172.21.15.28:0/4111173141 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1af4a00640 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3dbe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.057+0000 7f1af5400640 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e03640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 -- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 msgr2=0x55b0f3dbe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3dbe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 -- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 msgr2=0x55b0f3e03640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e03640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 -- 172.21.15.28:0/4111173141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f3a342d0 con 0x55b0f3dbf630 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1af5e00640 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e80 tx=0x7f1ae802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7df325dfd4e06ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1ae1c00640 1 -- 172.21.15.28:0/4111173141 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8040070 con 0x55b0f3dbf630 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1ae1c00640 1 -- 172.21.15.28:0/4111173141 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae8007740 con 0x55b0f3dbf630 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.058+0000 7f1ae1c00640 1 -- 172.21.15.28:0/4111173141 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8007b10 con 0x55b0f3dbf630 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 -- 172.21.15.28:0/4111173141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 msgr2=0x55b0f3db8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009e80 tx=0x7f1ae802c000 comp rx=0 tx=0).stop 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 -- 172.21.15.28:0/4111173141 shutdown_connections 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3db8720 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3dbe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 --2- 172.21.15.28:0/4111173141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e03640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 -- 172.21.15.28:0/4111173141 >> 172.21.15.28:0/4111173141 conn(0x55b0f3dba8f0 msgr2=0x55b0f3e02b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 -- 172.21.15.28:0/4111173141 shutdown_connections 2024-09-10T10:45:07.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.059+0000 7f1afae7d740 1 -- 172.21.15.28:0/4111173141 wait complete. 2024-09-10T10:45:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.060+0000 7f1afae7d740 1 -- start start 2024-09-10T10:45:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.060+0000 7f1afae7d740 1 Processor -- start 2024-09-10T10:45:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3db9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 0x55b0f3de4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3cdc780 con 0x55b0f3dbf630 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3d1d3d0 con 0x55b0f3dbdea0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b0f3db9d90 con 0x55b0f3de3ed0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1afae7d740 10 client.0 _ll_register_callbacks cb 0x55b0f3d1a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36060/0 (socket says 172.21.15.28:36060) 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 -- 172.21.15.28:0/1107316693 learned_addr learned my addr 172.21.15.28:0/1107316693 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af5e00640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 0x55b0f3de4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af5400640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3db9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 -- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 msgr2=0x55b0f3de4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 0x55b0f3de4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 -- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 msgr2=0x55b0f3db9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3db9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af4a00640 1 -- 172.21.15.28:0/1107316693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f3afe6b0 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af5400640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3db9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.061+0000 7f1af5e00640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 0x55b0f3de4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1af4a00640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058cb0 tx=0x7f1adc05d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e932ab7659c185 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc058170 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f1adc058310 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae8007740 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1adc054a70 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 10 client.5046 fetch_fsmap learned FSMap version 18 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b0f3e16880 con 0x55b0f3dbf630 2024-09-10T10:45:07.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ae8007740 con 0x55b0f3dbf630 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f1adc054c10 con 0x55b0f3dbf630 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 10 client.5046 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 10 client.5046 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.062+0000 7f1afae7d740 20 client.5046 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3ddda80 0x55b0f3ddde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1afae7d740 4 client.5046 mds_command: new command op to 4845 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 --> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f3d2eb40 con 0x55b0f3ddda80 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1af5400640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3ddda80 0x55b0f3ddde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1af5400640 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3ddda80 0x55b0f3ddde60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6c8e1da41c889c server_cookie=6d8476a9f10503b9 in_seq=0 out_seq=0 2024-09-10T10:45:07.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.063+0000 7f1ad6000640 10 client.5046 ms_handle_connect on v2:172.21.15.28:6834/3203683106 2024-09-10T10:45:07.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.064+0000 7f1ad6000640 1 -- 172.21.15.28:0/1107316693 <== mds.0 v2:172.21.15.28:6834/3203683106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0f3d2eb40 con 0x55b0f3ddda80 2024-09-10T10:45:07.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.064+0000 7f1ad6000640 10 client.5046 handle_command_reply: tid=0 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.064+0000 7f1afae7d740 1 client.5046 shutdown 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.064+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 msgr2=0x55b0f3e00220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.064+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058cb0 tx=0x7f1adc05d1e0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 shutdown_connections 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3de3ed0 0x55b0f3de4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3ddda80 0x55b0f3ddde60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbf630 0x55b0f3e00220 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 --2- 172.21.15.28:0/1107316693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbdea0 0x55b0f3db9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1ad6000640 0 client.5046 ms_handle_reset on v2:172.21.15.28:6834/3203683106 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 >> 172.21.15.28:0/1107316693 conn(0x55b0f3dbb660 msgr2=0x55b0f3e020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 shutdown_connections 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 1 -- 172.21.15.28:0/1107316693 wait complete. 2024-09-10T10:45:07.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.065+0000 7f1afae7d740 20 client.5046 trim_cache size 0 max 16384 2024-09-10T10:45:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.083+0000 7f1afae7d740 1 Processor -- start 2024-09-10T10:45:07.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 -- start start 2024-09-10T10:45:07.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 0x55b0f3dde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 0x55b0f3e00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01ffc40 con 0x55b0f3dbdea0 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00b90e0 con 0x55b0f3dbb660 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.084+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00b9730 con 0x55b0f3dbf630 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36062/0 (socket says 172.21.15.28:36062) 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 -- 172.21.15.28:0/2861711335 learned_addr learned my addr 172.21.15.28:0/2861711335 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af5e00640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 0x55b0f3e00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af5400640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 0x55b0f3dde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 -- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 msgr2=0x55b0f3e00b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 0x55b0f3e00b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 -- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 msgr2=0x55b0f3dde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 0x55b0f3dde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 -- 172.21.15.28:0/2861711335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f3ddf610 con 0x55b0f3dbdea0 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af5e00640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 0x55b0f3e00b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af5400640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 0x55b0f3dde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1af4a00640 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058a20 tx=0x7f1adc006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6267bdef8b990f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1ae1200640 1 -- 172.21.15.28:0/2861711335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc058170 con 0x55b0f3dbdea0 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1ae1200640 1 -- 172.21.15.28:0/2861711335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc088070 con 0x55b0f3dbdea0 2024-09-10T10:45:07.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.085+0000 7f1ae1200640 1 -- 172.21.15.28:0/2861711335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc057be0 con 0x55b0f3dbdea0 2024-09-10T10:45:07.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 -- 172.21.15.28:0/2861711335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 msgr2=0x55b0f3e27130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058a20 tx=0x7f1adc006000 comp rx=0 tx=0).stop 2024-09-10T10:45:07.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 -- 172.21.15.28:0/2861711335 shutdown_connections 2024-09-10T10:45:07.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbf630 0x55b0f3e00b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e27130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 --2- 172.21.15.28:0/2861711335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbb660 0x55b0f3dde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 -- 172.21.15.28:0/2861711335 >> 172.21.15.28:0/2861711335 conn(0x55b0f3e02450 msgr2=0x55b0f3dba100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 -- 172.21.15.28:0/2861711335 shutdown_connections 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.086+0000 7f1afae7d740 1 -- 172.21.15.28:0/2861711335 wait complete. 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.087+0000 7f1afae7d740 1 -- start start 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.087+0000 7f1afae7d740 1 Processor -- start 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3e7b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbf630 0x55b0f3e010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3db8350 con 0x55b0f3dbdea0 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0f3e16b60 con 0x55b0f3dbf630 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b0f3db7c50 con 0x55b0f3dbb660 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1afae7d740 10 client.0 _ll_register_callbacks cb 0x7f1adc007760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1af4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36070/0 (socket says 172.21.15.28:36070) 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1af4a00640 1 -- 172.21.15.28:0/477103252 learned_addr learned my addr 172.21.15.28:0/477103252 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1af5400640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3e7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.088+0000 7f1af5e00640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbf630 0x55b0f3e010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 -- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 msgr2=0x55b0f3e7b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3e7b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 -- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbf630 msgr2=0x55b0f3e010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbf630 0x55b0f3e010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 -- 172.21.15.28:0/477103252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f3dddf00 con 0x55b0f3dbdea0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1af4a00640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058a20 tx=0x7f1adc088f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd77a42d3d9c3150 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1ad6000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc066060 con 0x55b0f3dbdea0 2024-09-10T10:45:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f1adc061be0 con 0x55b0f3dbdea0 2024-09-10T10:45:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1adc088070 con 0x55b0f3dbdea0 2024-09-10T10:45:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f1adc06c050 con 0x55b0f3dbdea0 2024-09-10T10:45:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1afae7d740 10 client.5052 fetch_fsmap learned FSMap version 18 2024-09-10T10:45:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.089+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x55b0f3dddf00 con 0x55b0f3dbdea0 2024-09-10T10:45:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1adc088070 con 0x55b0f3dbdea0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f1adc05e260 con 0x55b0f3dbdea0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 10 client.5052 fetch_fsmap finished waiting for FSMap version 18 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 10 client.5052 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 20 client.5052 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3de3ed0 0x55b0f3ddf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 4 client.5052 mds_command: new command op to 4845 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 --> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] -- command(tid 0: {"prefix": "status"}) -- 0x55b0f3d28e90 con 0x55b0f3de3ed0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1af5400640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3de3ed0 0x55b0f3ddf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1af5400640 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3de3ed0 0x55b0f3ddf7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8d6267a43a6c80 server_cookie=ed147e7884a7fda7 in_seq=0 out_seq=0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.090+0000 7f1ad6000640 10 client.5052 ms_handle_connect on v2:172.21.15.28:6834/3203683106 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1ad6000640 1 -- 172.21.15.28:0/477103252 <== mds.0 v2:172.21.15.28:6834/3203683106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b0f3d28e90 con 0x55b0f3de3ed0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1ad6000640 10 client.5052 handle_command_reply: tid=0 2024-09-10T10:45:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 client.5052 shutdown 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 msgr2=0x55b0f3e00b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f1adc058a20 tx=0x7f1adc088f00 comp rx=0 tx=0).stop 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 shutdown_connections 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0f3dbf630 0x55b0f3e010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:6834/3203683106,v1:172.21.15.28:6835/3203683106] conn(0x55b0f3de3ed0 0x55b0f3ddf7e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b0f3dbdea0 0x55b0f3e00b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 --2- 172.21.15.28:0/477103252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b0f3dbb660 0x55b0f3e7b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1ad6000640 0 client.5052 ms_handle_reset on v2:172.21.15.28:6834/3203683106 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 >> 172.21.15.28:0/477103252 conn(0x55b0f3e02450 msgr2=0x55b0f3de4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 shutdown_connections 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 1 -- 172.21.15.28:0/477103252 wait complete. 2024-09-10T10:45:07.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.091+0000 7f1afae7d740 20 client.5052 trim_cache size 0 max 16384 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4845, 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 10.830949463, 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 18, 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 42, 2024-09-10T10:45:07.097 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 42, 2024-09-10T10:45:07.098 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 19.062942726999999 2024-09-10T10:45:07.098 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 msgr2=0x7f1ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 0x7f1ad006e2f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4006fd0 tx=0x7f1ae4008040 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 msgr2=0x7f1af00b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c410 tx=0x7f1adc001e70 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 shutdown_connections 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ad006be30 0x7f1ad006e2f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af015e660 0x7f1af00b2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1af00d1f90 0x7f1af00b1b20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 --2- 172.21.15.28:0/1900628537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1af00d0dd0 0x7f1af0151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 >> 172.21.15.28:0/1900628537 conn(0x7f1af008bfc0 msgr2=0x7f1af00b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 shutdown_connections 2024-09-10T10:45:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.092+0000 7f1af6800640 1 -- 172.21.15.28:0/1900628537 wait complete. 2024-09-10T10:45:07.109 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:45:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.263+0000 7f5906600640 1 Processor -- start 2024-09-10T10:45:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.264+0000 7f5906600640 1 -- start start 2024-09-10T10:45:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f5906600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5900072de0 0x7f590015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f5906600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5900072700 0x7f590015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f5906600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5900146140 con 0x7f5900072320 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f5906600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f59000cd740 con 0x7f5900072700 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f5906600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59000cda20 con 0x7f5900072de0 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f58ffe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5900072700 0x7f590015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f58ffe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5900072700 0x7f590015d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36086/0 (socket says 172.21.15.28:36086) 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.265+0000 7f58ffe00640 1 -- 172.21.15.28:0/1359651987 learned_addr learned my addr 172.21.15.28:0/1359651987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.266+0000 7f58f7e00640 1 --2- 172.21.15.28:0/1359651987 >> v2:172.21.15.138:3300/0 conn(0x7f5900072de0 0x7f590015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.266+0000 7f58ffe00640 1 -- 172.21.15.28:0/1359651987 >> v2:172.21.15.138:3300/0 conn(0x7f5900072de0 msgr2=0x7f590015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.266+0000 7f58ffe00640 1 --2- 172.21.15.28:0/1359651987 >> v2:172.21.15.138:3300/0 conn(0x7f5900072de0 0x7f590015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.266+0000 7f58ffe00640 1 -- 172.21.15.28:0/1359651987 >> v1:172.21.15.138:6789/0 conn(0x7f5900072320 legacy=0x7f59000cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.266+0000 7f58ffe00640 1 -- 172.21.15.28:0/1359651987 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000cdd00 con 0x7f5900072700 2024-09-10T10:45:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f58ffe00640 1 --2- 172.21.15.28:0/1359651987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072700 0x7f590015d720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f590006f8c0 tx=0x7f58f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c375e497d7c378 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f58fea00640 1 -- 172.21.15.28:0/1359651987 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8036070 con 0x7f5900072700 2024-09-10T10:45:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f58fea00640 1 -- 172.21.15.28:0/1359651987 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f8031600 con 0x7f5900072700 2024-09-10T10:45:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f58fea00640 1 -- 172.21.15.28:0/1359651987 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f80319d0 con 0x7f5900072700 2024-09-10T10:45:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f5906600640 1 -- 172.21.15.28:0/1359651987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072700 msgr2=0x7f590015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.267+0000 7f5906600640 1 --2- 172.21.15.28:0/1359651987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072700 0x7f590015d720 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f590006f8c0 tx=0x7f58f802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 -- 172.21.15.28:0/1359651987 shutdown_connections 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 --2- 172.21.15.28:0/1359651987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072700 0x7f590015d720 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 --2- 172.21.15.28:0/1359651987 >> v2:172.21.15.138:3300/0 conn(0x7f5900072de0 0x7f590015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 -- 172.21.15.28:0/1359651987 >> 172.21.15.28:0/1359651987 conn(0x7f5900067d70 msgr2=0x7f590006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 -- 172.21.15.28:0/1359651987 shutdown_connections 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.268+0000 7f5906600640 1 -- 172.21.15.28:0/1359651987 wait complete. 2024-09-10T10:45:07.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.269+0000 7f5906600640 1 Processor -- start 2024-09-10T10:45:07.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.269+0000 7f5906600640 1 -- start start 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 0x7f59001e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900072de0 0x7f59001e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f590015dde0 con 0x7f5900072320 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f590015dc60 con 0x7f5900072de0 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f5906600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f590015df60 con 0x7f5900072700 2024-09-10T10:45:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.270+0000 7f58ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36092/0 (socket says 172.21.15.28:36092) 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 -- 172.21.15.28:0/3099755558 learned_addr learned my addr 172.21.15.28:0/3099755558 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ffe00640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900072de0 0x7f59001e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58f7e00640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 0x7f59001e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 -- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 msgr2=0x7f59001e75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 0x7f59001e75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 -- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900072de0 msgr2=0x7f59001e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900072de0 0x7f59001e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 -- 172.21.15.28:0/3099755558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590015c370 con 0x7f5900072320 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58f7e00640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 0x7f59001e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:07.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58ff400640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c400 tx=0x7f58f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.271+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f00052d0 con 0x7f5900072320 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.272+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f0005490 con 0x7f5900072320 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.272+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007260 con 0x7f5900072320 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.272+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59000c8d20 con 0x7f5900072320 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.272+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59000c92e0 con 0x7f5900072320 2024-09-10T10:45:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.273+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590015e320 con 0x7f5900072320 2024-09-10T10:45:07.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.274+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58f0007400 con 0x7f5900072320 2024-09-10T10:45:07.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.275+0000 7f58f7400640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:07.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.275+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f58f008a480 con 0x7f5900072320 2024-09-10T10:45:07.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.275+0000 7f58f7e00640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:07.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.276+0000 7f58f7e00640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 0x7f58dc06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58e8006fd0 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:07.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.280+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58f0058650 con 0x7f5900072320 2024-09-10T10:45:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.500+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f590006e370 con 0x7f5900072320 2024-09-10T10:45:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.500+0000 7f58f7400640 1 -- 172.21.15.28:0/3099755558 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f58f005c950 con 0x7f5900072320 2024-09-10T10:45:07.502 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 msgr2=0x7f58dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 0x7f58dc06e2f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f58e8006fd0 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 msgr2=0x7f590015be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c400 tx=0x7f58f000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 shutdown_connections 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f58dc06be30 0x7f58dc06e2f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5900072de0 0x7f59001e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5900072700 0x7f59001e75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 --2- 172.21.15.28:0/3099755558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5900072320 0x7f590015be30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 >> 172.21.15.28:0/3099755558 conn(0x7f5900067d70 msgr2=0x7f5900144b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 shutdown_connections 2024-09-10T10:45:07.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07.503+0000 7f5906600640 1 -- 172.21.15.28:0/3099755558 wait complete. 2024-09-10T10:45:07.519 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:45:07.520 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:07.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:07.521 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:45:07.556 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:45:07.556 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:45:07.557 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42597sec preferred_lft 42597sec 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:45:07.558 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:45:07.559 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:07.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:07.560 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:45:07.651 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:45:07.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:07.655 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:45:07.691 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:45:07.691 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:45:07.692 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:45:07.692 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:45:07.692 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:45:07.692 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:07.692 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:45:07.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:45:07.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:45:07.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:45:07.693 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:45:07.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:07.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:07.890 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:45:07.925 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:45:07.925 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:45:07.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:07.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:07.925 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:45:08.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:08.021 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:08.056 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:08.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:08.057 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:08.132 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:45:08.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:08.132 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:08.261 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:08.262 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:08.262 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:08.277 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:08.277 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:08.277 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:08.277 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:08.278 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:45:08.301 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:45:08.302 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:45:08.302 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:45:08.303 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:45:08.303 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:45:08.303 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:45:08.303 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:45:08.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:45:08.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:08.307 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:08.364 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:08.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:08.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:08.460 INFO:teuthology.orchestra.run.smithi028.stdout:client4754 2024-09-10T10:45:08.460 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:08.461 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:08.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:08.497 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-10T10:45:08.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:08.580 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-10T10:45:08.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:08.659 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-10T10:45:08.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:45:08.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-10T10:45:08.737 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:45:08.877 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627776 2024-09-10T10:45:08.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:08.888 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:08.888 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:08.924 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:08.924 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:09.000 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:45:09.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.197+0000 7f8e8f800640 1 Processor -- start 2024-09-10T10:45:09.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.197+0000 7f8e8f800640 1 -- start start 2024-09-10T10:45:09.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e88072d30 0x7f8e8806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8f800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e8806f700 0x7f8e8815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8f800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e880cdbe0 con 0x7f8e8815b300 2024-09-10T10:45:09.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8f800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8e8806fb50 con 0x7f8e8806f700 2024-09-10T10:45:09.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e880728a0 con 0x7f8e88072d30 2024-09-10T10:45:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e8806f700 0x7f8e8815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e8806f700 0x7f8e8815b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36100/0 (socket says 172.21.15.28:36100) 2024-09-10T10:45:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e8d800640 1 -- 172.21.15.28:0/2639784936 learned_addr learned my addr 172.21.15.28:0/2639784936 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:09.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.199+0000 7f8e87e00640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e88072d30 0x7f8e8806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.200+0000 7f8e8d800640 1 -- 172.21.15.28:0/2639784936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e88072d30 msgr2=0x7f8e8806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.200+0000 7f8e8d800640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e88072d30 0x7f8e8806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.200+0000 7f8e8d800640 1 -- 172.21.15.28:0/2639784936 >> v1:172.21.15.138:6789/0 conn(0x7f8e8815b300 legacy=0x7f8e880cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.200+0000 7f8e8d800640 1 -- 172.21.15.28:0/2639784936 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e880cd900 con 0x7f8e8806f700 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8d800640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e8806f700 0x7f8e8815b6e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009e50 tx=0x7f8e8002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab603ff9caaee1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e87400640 1 -- 172.21.15.28:0/2639784936 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80036070 con 0x7f8e8806f700 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e87400640 1 -- 172.21.15.28:0/2639784936 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80031600 con 0x7f8e8806f700 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e87400640 1 -- 172.21.15.28:0/2639784936 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e800319d0 con 0x7f8e8806f700 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 -- 172.21.15.28:0/2639784936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e8806f700 msgr2=0x7f8e8815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e8806f700 0x7f8e8815b6e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009e50 tx=0x7f8e8002a710 comp rx=0 tx=0).stop 2024-09-10T10:45:09.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 -- 172.21.15.28:0/2639784936 shutdown_connections 2024-09-10T10:45:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e8806f700 0x7f8e8815b6e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 --2- 172.21.15.28:0/2639784936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e88072d30 0x7f8e8806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.201+0000 7f8e8f800640 1 -- 172.21.15.28:0/2639784936 >> 172.21.15.28:0/2639784936 conn(0x7f8e88067d70 msgr2=0x7f8e8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.202+0000 7f8e8f800640 1 -- 172.21.15.28:0/2639784936 shutdown_connections 2024-09-10T10:45:09.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.202+0000 7f8e8f800640 1 -- 172.21.15.28:0/2639784936 wait complete. 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.203+0000 7f8e8f800640 1 Processor -- start 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.203+0000 7f8e8f800640 1 -- start start 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e8806f700 0x7f8e8815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8815b300 0x7f8e88148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8e880cd5c0 con 0x7f8e88072d30 2024-09-10T10:45:09.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e88072530 con 0x7f8e8815b300 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e880cffe0 con 0x7f8e8806f700 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36110/0 (socket says 172.21.15.28:36110) 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e87e00640 1 -- 172.21.15.28:0/52852372 learned_addr learned my addr 172.21.15.28:0/52852372 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8d800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8815b300 0x7f8e88148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.204+0000 7f8e8ce00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e8806f700 0x7f8e8815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 -- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e8806f700 msgr2=0x7f8e8815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e8806f700 0x7f8e8815cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 -- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8815b300 msgr2=0x7f8e88148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8815b300 0x7f8e88148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 -- 172.21.15.28:0/52852372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e8814f500 con 0x7f8e88072d30 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e87e00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400d800 tx=0x7f8e7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400b6c0 con 0x7f8e88072d30 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e74019070 con 0x7f8e88072d30 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400bac0 con 0x7f8e88072d30 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.205+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e8814f7e0 con 0x7f8e88072d30 2024-09-10T10:45:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.206+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8814fda0 con 0x7f8e88072d30 2024-09-10T10:45:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.207+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e880b2550 con 0x7f8e88072d30 2024-09-10T10:45:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.208+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e7400bc60 con 0x7f8e88072d30 2024-09-10T10:45:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.208+0000 7f8e84c00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 0x7f8e6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.209+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f8e7408b380 con 0x7f8e88072d30 2024-09-10T10:45:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.209+0000 7f8e8ce00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 0x7f8e6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.210+0000 7f8e8ce00640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 0x7f8e6806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e8815d940 tx=0x7f8e7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.213+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e74059540 con 0x7f8e88072d30 2024-09-10T10:45:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.412+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e880b0370 con 0x7f8e88072d30 2024-09-10T10:45:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.412+0000 7f8e84c00640 1 -- 172.21.15.28:0/52852372 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f8e7405d840 con 0x7f8e88072d30 2024-09-10T10:45:09.413 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 msgr2=0x7f8e6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 0x7f8e6806e2f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e8815d940 tx=0x7f8e7c009040 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 msgr2=0x7f8e88148860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400d800 tx=0x7f8e7400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 shutdown_connections 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e6806be30 0x7f8e6806e2f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e8815b300 0x7f8e88148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e88072d30 0x7f8e88148860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 --2- 172.21.15.28:0/52852372 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e8806f700 0x7f8e8815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.414+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 >> 172.21.15.28:0/52852372 conn(0x7f8e88067d70 msgr2=0x7f8e8815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.415+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 shutdown_connections 2024-09-10T10:45:09.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.415+0000 7f8e8f800640 1 -- 172.21.15.28:0/52852372 wait complete. 2024-09-10T10:45:09.432 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:45:09.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:09.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:09.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:45:09.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:09.528 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:45:09.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:09.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.774+0000 7fd853e00640 1 Processor -- start 2024-09-10T10:45:09.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.774+0000 7fd853e00640 1 -- start start 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd853e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd85406e160 0x7fd8540d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd853e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd8540d1260 0x7fd8540d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd853e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8540cf800 con 0x7fd85415e4b0 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd853e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd8540cfdc0 con 0x7fd85406e160 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd853e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd8540cfae0 con 0x7fd8540d1260 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd852000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd85406e160 0x7fd8540d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd852000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd85406e160 0x7fd8540d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36126/0 (socket says 172.21.15.28:36126) 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.775+0000 7fd852000640 1 -- 172.21.15.28:0/484584877 learned_addr learned my addr 172.21.15.28:0/484584877 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.776+0000 7fd853400640 1 --2- 172.21.15.28:0/484584877 >> v2:172.21.15.138:3300/0 conn(0x7fd8540d1260 0x7fd8540d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.776+0000 7fd852000640 1 -- 172.21.15.28:0/484584877 >> v2:172.21.15.138:3300/0 conn(0x7fd8540d1260 msgr2=0x7fd8540d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:09.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.776+0000 7fd852000640 1 --2- 172.21.15.28:0/484584877 >> v2:172.21.15.138:3300/0 conn(0x7fd8540d1260 0x7fd8540d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.776+0000 7fd852000640 1 -- 172.21.15.28:0/484584877 >> v1:172.21.15.138:6789/0 conn(0x7fd85415e4b0 legacy=0x7fd8540d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.776+0000 7fd852000640 1 -- 172.21.15.28:0/484584877 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8540d20c0 con 0x7fd85406e160 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.777+0000 7fd852000640 1 --2- 172.21.15.28:0/484584877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd8540d28c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8540d5390 tx=0x7fd84002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33709e8a7faf6b99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.777+0000 7fd851600640 1 -- 172.21.15.28:0/484584877 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840037070 con 0x7fd85406e160 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.777+0000 7fd851600640 1 -- 172.21.15.28:0/484584877 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd840032440 con 0x7fd85406e160 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.777+0000 7fd851600640 1 -- 172.21.15.28:0/484584877 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840032810 con 0x7fd85406e160 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 -- 172.21.15.28:0/484584877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 msgr2=0x7fd8540d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 --2- 172.21.15.28:0/484584877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd8540d28c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8540d5390 tx=0x7fd84002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 -- 172.21.15.28:0/484584877 shutdown_connections 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 --2- 172.21.15.28:0/484584877 >> v2:172.21.15.138:3300/0 conn(0x7fd8540d1260 0x7fd8540d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 --2- 172.21.15.28:0/484584877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd8540d28c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 -- 172.21.15.28:0/484584877 >> 172.21.15.28:0/484584877 conn(0x7fd85408bfc0 msgr2=0x7fd8540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:09.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 -- 172.21.15.28:0/484584877 shutdown_connections 2024-09-10T10:45:09.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.778+0000 7fd853e00640 1 -- 172.21.15.28:0/484584877 wait complete. 2024-09-10T10:45:09.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.779+0000 7fd853e00640 1 Processor -- start 2024-09-10T10:45:09.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.779+0000 7fd853e00640 1 -- start start 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8540d1260 0x7fd85415f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 0x7fd85415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8540cf280 con 0x7fd85406e160 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd85415e890 con 0x7fd85415e4b0 2024-09-10T10:45:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.780+0000 7fd853e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8540cf400 con 0x7fd8540d1260 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36142/0 (socket says 172.21.15.28:36142) 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 -- 172.21.15.28:0/3302724701 learned_addr learned my addr 172.21.15.28:0/3302724701 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd853400640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 0x7fd85415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852000640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8540d1260 0x7fd85415f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 -- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8540d1260 msgr2=0x7fd85415f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8540d1260 0x7fd85415f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 -- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 msgr2=0x7fd85415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 0x7fd85415f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 -- 172.21.15.28:0/3302724701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd854160f80 con 0x7fd85406e160 2024-09-10T10:45:09.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd853400640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 0x7fd85415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.781+0000 7fd852a00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c3f0 tx=0x7fd84400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.782+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8440052d0 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.782+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844005490 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.782+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844007260 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.782+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8540cc500 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.782+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd854161200 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.783+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8540ce780 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.784+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd844007400 con 0x7fd85406e160 2024-09-10T10:45:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.784+0000 7fd84aa00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 0x7fd83406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.785+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fd84408a560 con 0x7fd85406e160 2024-09-10T10:45:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.785+0000 7fd852000640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 0x7fd83406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.786+0000 7fd852000640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 0x7fd83406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd840034000 tx=0x7fd84002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:09.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:09.789+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd854160f80 con 0x7fd85406e160 2024-09-10T10:45:10.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.011+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8540b4c70 con 0x7fd85406e160 2024-09-10T10:45:10.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.012+0000 7fd84aa00640 1 -- 172.21.15.28:0/3302724701 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fd844058ce0 con 0x7fd85406e160 2024-09-10T10:45:10.013 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:10.013 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":18,"btime":"2024-09-10T10:44:59:280567+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:44:58.705324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4845},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4845":{"gid":4845,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3203683106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3203683106},{"type":"v1","addr":"172.21.15.28:6835","nonce":3203683106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4845,"qdb_cluster":[4845]},"id":2}]} 2024-09-10T10:45:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 18 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.014+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 msgr2=0x7fd83406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.014+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 0x7fd83406e2f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd840034000 tx=0x7fd84002e000 comp rx=0 tx=0).stop 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.014+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 msgr2=0x7fd854157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.014+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd84400c3f0 tx=0x7fd84400c8b0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 shutdown_connections 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83406be30 0x7fd83406e2f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd85415e4b0 0x7fd85415f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8540d1260 0x7fd85415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 --2- 172.21.15.28:0/3302724701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd85406e160 0x7fd854157440 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 >> 172.21.15.28:0/3302724701 conn(0x7fd85408bfc0 msgr2=0x7fd8540b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 shutdown_connections 2024-09-10T10:45:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.015+0000 7fd853e00640 1 -- 172.21.15.28:0/3302724701 wait complete. 2024-09-10T10:45:10.031 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T10:45:10.217 INFO:teuthology.orchestra.run.smithi028.stdout:{"message":"","return_code":0} 2024-09-10T10:45:10.218 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T10:45:10.218 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:45:10.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.374+0000 7fd8c1e00640 1 Processor -- start 2024-09-10T10:45:10.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.374+0000 7fd8c1e00640 1 -- start start 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8c1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd8bc15b300 0x7fd8bc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8c1e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8bc0cdbe0 con 0x7fd8bc06f8d0 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8c1e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8bc0cd900 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8c1e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd8bc0728a0 con 0x7fd8bc15b300 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8bb400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd8bc06f8d0 0x7fd8bc15b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40216/0 (socket says 172.21.15.28:40216) 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8bb400640 1 -- 172.21.15.28:0/2939085277 learned_addr learned my addr 172.21.15.28:0/2939085277 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.376+0000 7fd8baa00640 1 --2- 172.21.15.28:0/2939085277 >> v2:172.21.15.28:3300/0 conn(0x7fd8bc15b300 0x7fd8bc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.377+0000 7fd8baa00640 1 -- 172.21.15.28:0/2939085277 >> v1:172.21.15.138:6789/0 conn(0x7fd8bc0cd3f0 legacy=0x7fd8bc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.377+0000 7fd8baa00640 1 -- 172.21.15.28:0/2939085277 >> v1:172.21.15.28:6789/0 conn(0x7fd8bc06f8d0 legacy=0x7fd8bc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.377+0000 7fd8ba000640 1 -- 172.21.15.28:0/2939085277 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755739989 0 0) 0x7fd8bc0cd900 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.377+0000 7fd8baa00640 1 -- 172.21.15.28:0/2939085277 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc072250 con 0x7fd8bc15b300 2024-09-10T10:45:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8baa00640 1 --2- 172.21.15.28:0/2939085277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc15b300 0x7fd8bc0cceb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0097a0 tx=0x7fd8ac02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369d66e35526e395 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8ba000640 1 -- 172.21.15.28:0/2939085277 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac037070 con 0x7fd8bc15b300 2024-09-10T10:45:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8ba000640 1 -- 172.21.15.28:0/2939085277 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8ac02adc0 con 0x7fd8bc15b300 2024-09-10T10:45:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8ba000640 1 -- 172.21.15.28:0/2939085277 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac032660 con 0x7fd8bc15b300 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8c1e00640 1 -- 172.21.15.28:0/2939085277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc15b300 msgr2=0x7fd8bc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.378+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/2939085277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc15b300 0x7fd8bc0cceb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0097a0 tx=0x7fd8ac02a130 comp rx=0 tx=0).stop 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.379+0000 7fd8c1e00640 1 -- 172.21.15.28:0/2939085277 shutdown_connections 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.379+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/2939085277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc15b300 0x7fd8bc0cceb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.379+0000 7fd8c1e00640 1 -- 172.21.15.28:0/2939085277 >> 172.21.15.28:0/2939085277 conn(0x7fd8bc067d70 msgr2=0x7fd8bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.379+0000 7fd8c1e00640 1 -- 172.21.15.28:0/2939085277 shutdown_connections 2024-09-10T10:45:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.379+0000 7fd8c1e00640 1 -- 172.21.15.28:0/2939085277 wait complete. 2024-09-10T10:45:10.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.380+0000 7fd8c1e00640 1 Processor -- start 2024-09-10T10:45:10.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.380+0000 7fd8c1e00640 1 -- start start 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc06f8d0 0x7fd8bc137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8bc15b300 0x7fd8bc1e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc0d0160 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8bc0cffe0 con 0x7fd8bc06f8d0 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8bc072d30 con 0x7fd8bc15b300 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36162/0 (socket says 172.21.15.28:36162) 2024-09-10T10:45:10.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.381+0000 7fd8b3e00640 1 -- 172.21.15.28:0/1281142208 learned_addr learned my addr 172.21.15.28:0/1281142208 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8baa00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc06f8d0 0x7fd8bc137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8bb400640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8bc15b300 0x7fd8bc1e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 -- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8bc15b300 msgr2=0x7fd8bc1e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8bc15b300 0x7fd8bc1e51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 -- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc06f8d0 msgr2=0x7fd8bc137c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc06f8d0 0x7fd8bc137c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 -- 172.21.15.28:0/1281142208 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8bc1e4120 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c490 tx=0x7fd8a400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.382+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4005480 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.383+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a40035f0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.383+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40039c0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.383+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8bc1e4400 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.383+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8bc1381c0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.384+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8bc15a640 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.385+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8a4003b60 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.385+0000 7fd8b3400640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 0x7fd89806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.386+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fd8a408abe0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.386+0000 7fd8baa00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 0x7fd89806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.387+0000 7fd8baa00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 0x7fd89806e490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac034000 tx=0x7fd8ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.390+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd8a40593f0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.613+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd8bc152c80 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.615 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:45:10.614+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:45:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.616+0000 7fd8b3400640 1 -- 172.21.15.28:0/1281142208 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fd8a4058cf0 con 0x7fd8bc0cd3f0 2024-09-10T10:45:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:45:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.618+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 msgr2=0x7fd89806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.618+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 0x7fd89806e490 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac034000 tx=0x7fd8ac02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.618+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 msgr2=0x7fd8bc1e36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.618+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c490 tx=0x7fd8a400c950 comp rx=0 tx=0).stop 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 shutdown_connections 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd89806bfd0 0x7fd89806e490 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8bc15b300 0x7fd8bc1e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8bc0cd3f0 0x7fd8bc1e36f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 --2- 172.21.15.28:0/1281142208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8bc06f8d0 0x7fd8bc137c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 >> 172.21.15.28:0/1281142208 conn(0x7fd8bc067d70 msgr2=0x7fd8bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 shutdown_connections 2024-09-10T10:45:10.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.620+0000 7fd8c1e00640 1 -- 172.21.15.28:0/1281142208 wait complete. 2024-09-10T10:45:10.628 INFO:tasks.ceph.mds.a.smithi028.stderr: -4751> 2024-09-10T10:44:58.566+0000 7f4309800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:45:10.638 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:45:10.678 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:45:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.716+0000 7f6c863f8f40 1 Processor -- start 2024-09-10T10:45:10.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 -- start start 2024-09-10T10:45:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753cc2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccc3e0 0x556753d5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556753b9eb30 con 0x556753ccf9c0 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556753ba2b70 con 0x556753bf6470 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.717+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556753cceee0 con 0x556753ccc3e0 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36188/0 (socket says 172.21.15.28:36188) 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 -- 172.21.15.28:0/878170034 learned_addr learned my addr 172.21.15.28:0/878170034 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 -- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccc3e0 msgr2=0x556753d5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccc3e0 0x556753d5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 -- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 msgr2=0x556753cc2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c83600640 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753cc2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753cc2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 -- 172.21.15.28:0/878170034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556753cce990 con 0x556753ccf9c0 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c83600640 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753cc2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.718+0000 7f6c84000640 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0015c0 tx=0x7f6c7c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c731ecd2216bde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c81800640 1 -- 172.21.15.28:0/878170034 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c017070 con 0x556753ccf9c0 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c81800640 1 -- 172.21.15.28:0/878170034 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c7c0026b0 con 0x556753ccf9c0 2024-09-10T10:45:10.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c81800640 1 -- 172.21.15.28:0/878170034 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c002aa0 con 0x556753ccf9c0 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 -- 172.21.15.28:0/878170034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 msgr2=0x556753c429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0015c0 tx=0x7f6c7c001a80 comp rx=0 tx=0).stop 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 -- 172.21.15.28:0/878170034 shutdown_connections 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccf9c0 0x556753c429c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccc3e0 0x556753d5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/878170034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753cc2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 -- 172.21.15.28:0/878170034 >> 172.21.15.28:0/878170034 conn(0x556753c450a0 msgr2=0x556753be1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 -- 172.21.15.28:0/878170034 shutdown_connections 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.719+0000 7f6c863f8f40 1 -- 172.21.15.28:0/878170034 wait complete. 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.720+0000 7f6c863f8f40 1 Processor -- start 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.720+0000 7f6c863f8f40 1 -- start start 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753d67b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 0x556753c3e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccf9c0 0x556753cd2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556753c26770 con 0x556753ccc3e0 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556753c455a0 con 0x556753bf6470 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c863f8f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556753c457f0 con 0x556753ccf9c0 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 0x556753c3e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 0x556753c3e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36200/0 (socket says 172.21.15.28:36200) 2024-09-10T10:45:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c82c00640 1 -- 172.21.15.28:0/415228922 learned_addr learned my addr 172.21.15.28:0/415228922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c83600640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753d67b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.721+0000 7f6c84000640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccf9c0 0x556753cd2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 -- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccf9c0 msgr2=0x556753cd2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556753ccf9c0 0x556753cd2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 -- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 msgr2=0x556753d67b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556753bf6470 0x556753d67b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 -- 172.21.15.28:0/415228922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556753c39e10 con 0x556753ccc3e0 2024-09-10T10:45:10.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c82c00640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 0x556753c3e080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c70025ec0 tx=0x7f6c700057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c7be00640 1 -- 172.21.15.28:0/415228922 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7004f9d0 con 0x556753ccc3e0 2024-09-10T10:45:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c7be00640 1 -- 172.21.15.28:0/415228922 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c70067070 con 0x556753ccc3e0 2024-09-10T10:45:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c7be00640 1 -- 172.21.15.28:0/415228922 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7004fdb0 con 0x556753ccc3e0 2024-09-10T10:45:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.722+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556753c3eb60 con 0x556753ccc3e0 2024-09-10T10:45:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.723+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556753c3a3b0 con 0x556753ccc3e0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.724+0000 7f6c7be00640 1 -- 172.21.15.28:0/415228922 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c7006b400 con 0x556753ccc3e0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.725+0000 7f6c7be00640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c6406a840 0x7f6c6406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.725+0000 7f6c7be00640 1 -- 172.21.15.28:0/415228922 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f6c700cd9f0 con 0x556753ccc3e0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556753cc10b0 0x556753cc1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x556753d68090 con 0x556753cc10b0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c83600640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c6406a840 0x7f6c6406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c84000640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556753cc10b0 0x556753cc1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c83600640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c6406a840 0x7f6c6406cd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c74001880 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.726+0000 7f6c84000640 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556753cc10b0 0x556753cc1540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c84000640 1 -- 172.21.15.28:0/415228922 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v43'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6c7c00f070 con 0x556753cc10b0 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556753cc10b0 msgr2=0x556753cc1540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556753cc10b0 0x556753cc1540 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c6406a840 msgr2=0x7f6c6406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c6406a840 0x7f6c6406cd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c74000c00 tx=0x7f6c74001880 comp rx=0 tx=0).stop 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 msgr2=0x556753c3e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 --2- 172.21.15.28:0/415228922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556753ccc3e0 0x556753c3e080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c70025ec0 tx=0x7f6c700057c0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c84000640 1 -- 172.21.15.28:0/415228922 reap_dead start 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 shutdown_connections 2024-09-10T10:45:10.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 >> 172.21.15.28:0/415228922 conn(0x556753c450a0 msgr2=0x556753c25530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 shutdown_connections 2024-09-10T10:45:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.728+0000 7f6c863f8f40 1 -- 172.21.15.28:0/415228922 wait complete. 2024-09-10T10:45:10.733 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:45:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.932+0000 7f9fece00640 1 Processor -- start 2024-09-10T10:45:10.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.932+0000 7f9fece00640 1 -- start start 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fece00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fe80cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fece00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fe80ce060 0x7f9fe80ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fece00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fe80b30e0 con 0x7f9fe8156740 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fece00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9fe80d0110 con 0x7f9fe80ce060 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fece00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe806edc0 con 0x7f9fe80cf790 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fe5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9fe8156740 0x7f9fe80b45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40236/0 (socket says 172.21.15.28:40236) 2024-09-10T10:45:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fe5e00640 1 -- 172.21.15.28:0/886344786 learned_addr learned my addr 172.21.15.28:0/886344786 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:10.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fe6800640 1 --2- 172.21.15.28:0/886344786 >> v2:172.21.15.138:3300/0 conn(0x7f9fe80ce060 0x7f9fe80ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1473749347 0 0) 0x7f9fe80b30e0 con 0x7f9fe8156740 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fdd400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fe80cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.934+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fcc003520 con 0x7f9fe8156740 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2620307522 0 0) 0x7f9fcc003520 con 0x7f9fe8156740 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 msgr2=0x7f9fe80cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fe80cfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 >> v2:172.21.15.138:3300/0 conn(0x7f9fe80ce060 msgr2=0x7f9fe80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 --2- 172.21.15.28:0/886344786 >> v2:172.21.15.138:3300/0 conn(0x7f9fe80ce060 0x7f9fe80ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe80b33c0 con 0x7f9fe8156740 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f9fd8001670 con 0x7f9fe8156740 2024-09-10T10:45:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 >> v1:172.21.15.28:6789/0 conn(0x7f9fe8156740 legacy=0x7f9fe80b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fcc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fcc004800 0x7f9fcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80d0110 con 0x7f9fcc0039b0 2024-09-10T10:45:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe806edc0 con 0x7f9fcc004800 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc00b9c0 con 0x7f9fe80cf790 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fcc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fdd400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fcc004800 0x7f9fcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe6800640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 -- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 msgr2=0x7f9fcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80cf790 0x7f9fcc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 -- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fcc004800 msgr2=0x7f9fcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fcc004800 0x7f9fcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fe5e00640 1 -- 172.21.15.28:0/886344786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc003520 con 0x7f9fcc0039b0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.935+0000 7f9fdd400640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fcc004800 0x7f9fcc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.936+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fcc003df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b197e19c0f0508 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007ca0 con 0x7f9fcc0039b0 2024-09-10T10:45:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd8005ad0 con 0x7f9fcc0039b0 2024-09-10T10:45:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fe5400640 1 -- 172.21.15.28:0/886344786 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007070 con 0x7f9fcc0039b0 2024-09-10T10:45:10.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 -- 172.21.15.28:0/886344786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 msgr2=0x7f9fcc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 --2- 172.21.15.28:0/886344786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fcc003df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fcc005830 comp rx=0 tx=0).stop 2024-09-10T10:45:10.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fe6800640 1 -- 172.21.15.28:0/886344786 reap_dead start 2024-09-10T10:45:10.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 -- 172.21.15.28:0/886344786 shutdown_connections 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 -- 172.21.15.28:0/886344786 >> 172.21.15.28:0/886344786 conn(0x7f9fe806bdf0 msgr2=0x7f9fe806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 -- 172.21.15.28:0/886344786 shutdown_connections 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.937+0000 7f9fece00640 1 -- 172.21.15.28:0/886344786 wait complete. 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.939+0000 7f9fece00640 1 Processor -- start 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.939+0000 7f9fece00640 1 -- start start 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8159070 0x7f9fe80ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80ad560 0x7f9fe8157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80b3c60 con 0x7f9fcc0039b0 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe80b3980 con 0x7f9fe8159070 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fece00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe80b4cc0 con 0x7f9fe80ad560 2024-09-10T10:45:10.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fe5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fe5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36210/0 (socket says 172.21.15.28:36210) 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.940+0000 7f9fe5e00640 1 -- 172.21.15.28:0/931145929 learned_addr learned my addr 172.21.15.28:0/931145929 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe6800640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80ad560 0x7f9fe8157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fdd400640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8159070 0x7f9fe80ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 -- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80ad560 msgr2=0x7f9fe8157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80ad560 0x7f9fe8157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 -- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8159070 msgr2=0x7f9fe80ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8159070 0x7f9fe80ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 -- 172.21.15.28:0/931145929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe814a120 con 0x7f9fcc0039b0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fe5e00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000fb0 tx=0x7f9fe815c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8001630 con 0x7f9fcc0039b0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd8042070 con 0x7f9fcc0039b0 2024-09-10T10:45:10.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe814a400 con 0x7f9fcc0039b0 2024-09-10T10:45:10.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.941+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8007270 con 0x7f9fcc0039b0 2024-09-10T10:45:10.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.942+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe814a9c0 con 0x7f9fcc0039b0 2024-09-10T10:45:10.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.943+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe80d0920 con 0x7f9fcc0039b0 2024-09-10T10:45:10.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.943+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9fd8007410 con 0x7f9fcc0039b0 2024-09-10T10:45:10.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.944+0000 7f9fdea00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 0x7f9fc406e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:10.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.944+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f9fd80ab580 con 0x7f9fcc0039b0 2024-09-10T10:45:10.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.945+0000 7f9fdd400640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 0x7f9fc406e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:10.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.946+0000 7f9fdd400640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 0x7f9fc406e2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fe815b370 tx=0x7f9fd0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:10.949+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fd8079de0 con 0x7f9fcc0039b0 2024-09-10T10:45:11.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.174+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9fe8160cc0 con 0x7f9fcc0039b0 2024-09-10T10:45:11.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.602+0000 7f9fdea00640 1 -- 172.21.15.28:0/931145929 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v20) ==== 162+0+0 (secure 0 0 0) 0x7f9fd8005aa0 con 0x7f9fcc0039b0 2024-09-10T10:45:11.605 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:45:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 msgr2=0x7f9fc406e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 0x7f9fc406e2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fe815b370 tx=0x7f9fd0008040 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 msgr2=0x7f9fe813d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000fb0 tx=0x7f9fe815c830 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 shutdown_connections 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9fc406be00 0x7f9fc406e2c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fe80ad560 0x7f9fe8157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fe8159070 0x7f9fe80ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 --2- 172.21.15.28:0/931145929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9fcc0039b0 0x7f9fe813d440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 >> 172.21.15.28:0/931145929 conn(0x7f9fe806bdf0 msgr2=0x7f9fe815bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 shutdown_connections 2024-09-10T10:45:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.605+0000 7f9fece00640 1 -- 172.21.15.28:0/931145929 wait complete. 2024-09-10T10:45:11.627 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.928+0000 7f88a7200640 1 Processor -- start 2024-09-10T10:45:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.928+0000 7f88a7200640 1 -- start start 2024-09-10T10:45:11.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.929+0000 7f88a7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 0x7f88a806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.929+0000 7f88a7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 0x7f88a809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.929+0000 7f88a7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a806ddd0 con 0x7f88a80b47b0 2024-09-10T10:45:11.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.929+0000 7f88a7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f88a809b3c0 con 0x7f88a809aa30 2024-09-10T10:45:11.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.929+0000 7f88a7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88a806e0b0 con 0x7f88a81629e0 2024-09-10T10:45:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a5e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 0x7f88a806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 0x7f88a809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f889d400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f88a80b47b0 0x7f88a8070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:54150/0 (socket says 172.21.15.28:54150) 2024-09-10T10:45:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f889d400640 1 -- 172.21.15.28:0/2267636010 learned_addr learned my addr 172.21.15.28:0/2267636010 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a5e00640 1 -- 172.21.15.28:0/2267636010 >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 msgr2=0x7f88a809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a5e00640 1 --2- 172.21.15.28:0/2267636010 >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 0x7f88a809ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a5e00640 1 -- 172.21.15.28:0/2267636010 >> v1:172.21.15.138:6789/0 conn(0x7f88a80b47b0 legacy=0x7f88a8070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a5e00640 1 -- 172.21.15.28:0/2267636010 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a80d1b60 con 0x7f88a81629e0 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.930+0000 7f88a6800640 1 --2- 172.21.15.28:0/2267636010 >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 0x7f88a809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.931+0000 7f88a5e00640 1 --2- 172.21.15.28:0/2267636010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 0x7f88a806e9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88980097a0 tx=0x7f889802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67ef6b1d8f37fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a5400640 1 -- 172.21.15.28:0/2267636010 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898037070 con 0x7f88a81629e0 2024-09-10T10:45:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a5400640 1 -- 172.21.15.28:0/2267636010 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f889802ae40 con 0x7f88a81629e0 2024-09-10T10:45:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a5400640 1 -- 172.21.15.28:0/2267636010 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898032660 con 0x7f88a81629e0 2024-09-10T10:45:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 -- 172.21.15.28:0/2267636010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 msgr2=0x7f88a806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 --2- 172.21.15.28:0/2267636010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 0x7f88a806e9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f88980097a0 tx=0x7f889802a0f0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 -- 172.21.15.28:0/2267636010 shutdown_connections 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 --2- 172.21.15.28:0/2267636010 >> v2:172.21.15.138:3300/0 conn(0x7f88a809aa30 0x7f88a809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 --2- 172.21.15.28:0/2267636010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a81629e0 0x7f88a806e9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 -- 172.21.15.28:0/2267636010 >> 172.21.15.28:0/2267636010 conn(0x7f88a80b3c90 msgr2=0x7f88a80b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 -- 172.21.15.28:0/2267636010 shutdown_connections 2024-09-10T10:45:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.932+0000 7f88a7200640 1 -- 172.21.15.28:0/2267636010 wait complete. 2024-09-10T10:45:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.934+0000 7f88a7200640 1 Processor -- start 2024-09-10T10:45:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.934+0000 7f88a7200640 1 -- start start 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a809aa30 0x7f88a809c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a81629e0 0x7f88a815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8162f40 con 0x7f88a80b47b0 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8162dc0 con 0x7f88a81629e0 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f88a7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88a80d0170 con 0x7f88a809aa30 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f889d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f889d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36212/0 (socket says 172.21.15.28:36212) 2024-09-10T10:45:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.935+0000 7f889d400640 1 -- 172.21.15.28:0/1382151674 learned_addr learned my addr 172.21.15.28:0/1382151674 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f88a5e00640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a809aa30 0x7f88a809c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f88a6800640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a81629e0 0x7f88a815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 -- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a809aa30 msgr2=0x7f88a809c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a809aa30 0x7f88a809c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 -- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a81629e0 msgr2=0x7f88a815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a81629e0 0x7f88a815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 -- 172.21.15.28:0/1382151674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a80b5d50 con 0x7f88a80b47b0 2024-09-10T10:45:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889d400640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f889000c440 tx=0x7f8890001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890017070 con 0x7f88a80b47b0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8890005ce0 con 0x7f88a80b47b0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a80b6030 con 0x7f88a80b47b0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88900072f0 con 0x7f88a80b47b0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.936+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a80b6b10 con 0x7f88a80b47b0 2024-09-10T10:45:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.938+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a809ae80 con 0x7f88a80b47b0 2024-09-10T10:45:11.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.939+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f889001a070 con 0x7f88a80b47b0 2024-09-10T10:45:11.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.939+0000 7f889ea00640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 0x7f888406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:11.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.939+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f889008a340 con 0x7f88a80b47b0 2024-09-10T10:45:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.940+0000 7f88a5e00640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 0x7f888406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.941+0000 7f88a5e00640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 0x7f888406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8898034000 tx=0x7f889802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:11.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:11.944+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8890058ba0 con 0x7f88a80b47b0 2024-09-10T10:45:12.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.166+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88a80b8120 con 0x7f88a80b47b0 2024-09-10T10:45:12.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.167+0000 7f889ea00640 1 -- 172.21.15.28:0/1382151674 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5223 (secure 0 0 0) 0x7f88900584a0 con 0x7f88a80b47b0 2024-09-10T10:45:12.169 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:12.169 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":20,"btime":"2024-09-10T10:45:11:602532+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:11.602508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T10:45:12.169 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 20 2024-09-10T10:45:12.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 msgr2=0x7f888406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 0x7f888406e2f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8898034000 tx=0x7f889802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 msgr2=0x7f88a80b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f889000c440 tx=0x7f8890001e70 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 shutdown_connections 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f888406be30 0x7f888406e2f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a81629e0 0x7f88a815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88a80b47b0 0x7f88a80b7850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 --2- 172.21.15.28:0/1382151674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88a809aa30 0x7f88a809c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 >> 172.21.15.28:0/1382151674 conn(0x7f88a80b3c90 msgr2=0x7f88a80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.169+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 shutdown_connections 2024-09-10T10:45:12.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:12.170+0000 7f88a7200640 1 -- 172.21.15.28:0/1382151674 wait complete. 2024-09-10T10:45:12.590 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:44:55.704429+0000', 'modified': '2024-09-10T10:45:11.602508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5088}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/1058548427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 1058548427}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 1058548427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:45:12.590 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5088:up:replay 2024-09-10T10:45:13.591 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.757+0000 7f5811e00640 1 Processor -- start 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.757+0000 7f5811e00640 1 -- start start 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f5811e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f580c161c80 0x7f580c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f5811e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 0x7f580c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f5811e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f580c06e870 con 0x7f580c0b64b0 2024-09-10T10:45:13.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f5811e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f580c06d9f0 con 0x7f580c161c80 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f5811e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f580c06dcd0 con 0x7f580c162060 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f580aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f580c161c80 0x7f580c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f580b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f580c0b64b0 0x7f580c162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40250/0 (socket says 172.21.15.28:40250) 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.758+0000 7f580b400640 1 -- 172.21.15.28:0/919141619 learned_addr learned my addr 172.21.15.28:0/919141619 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580a000640 1 --2- 172.21.15.28:0/919141619 >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 0x7f580c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580aa00640 1 -- 172.21.15.28:0/919141619 >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 msgr2=0x7f580c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580aa00640 1 --2- 172.21.15.28:0/919141619 >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 0x7f580c0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580aa00640 1 -- 172.21.15.28:0/919141619 >> v1:172.21.15.28:6789/0 conn(0x7f580c0b64b0 legacy=0x7f580c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580aa00640 1 -- 172.21.15.28:0/919141619 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c06dfb0 con 0x7f580c161c80 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f580a000640 1 --2- 172.21.15.28:0/919141619 >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 0x7f580c0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:13.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.759+0000 7f5809600640 1 -- 172.21.15.28:0/919141619 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447957177 0 0) 0x7f5804001490 con 0x7f580c0b64b0 2024-09-10T10:45:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.760+0000 7f580aa00640 1 --2- 172.21.15.28:0/919141619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c161c80 0x7f580c06e330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f580c08e830 tx=0x7f57fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06ee41f1ecc7fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.760+0000 7f5809600640 1 -- 172.21.15.28:0/919141619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc037070 con 0x7f580c161c80 2024-09-10T10:45:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.760+0000 7f5809600640 1 -- 172.21.15.28:0/919141619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57fc032440 con 0x7f580c161c80 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.760+0000 7f5809600640 1 -- 172.21.15.28:0/919141619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc032810 con 0x7f580c161c80 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 -- 172.21.15.28:0/919141619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c161c80 msgr2=0x7f580c06e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 --2- 172.21.15.28:0/919141619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c161c80 0x7f580c06e330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f580c08e830 tx=0x7f57fc02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 -- 172.21.15.28:0/919141619 shutdown_connections 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 --2- 172.21.15.28:0/919141619 >> v2:172.21.15.138:3300/0 conn(0x7f580c162060 0x7f580c0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 --2- 172.21.15.28:0/919141619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c161c80 0x7f580c06e330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 -- 172.21.15.28:0/919141619 >> 172.21.15.28:0/919141619 conn(0x7f580c0d44f0 msgr2=0x7f580c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 -- 172.21.15.28:0/919141619 shutdown_connections 2024-09-10T10:45:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.761+0000 7f5811e00640 1 -- 172.21.15.28:0/919141619 wait complete. 2024-09-10T10:45:13.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.763+0000 7f5811e00640 1 Processor -- start 2024-09-10T10:45:13.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.763+0000 7f5811e00640 1 -- start start 2024-09-10T10:45:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.763+0000 7f5811e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580c0b64b0 0x7f580c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f5811e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f580c161c80 0x7f580c0b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f5811e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f5811e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f580c0d2a10 con 0x7f580c162060 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f5811e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f580c0d2890 con 0x7f580c0b64b0 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f5811e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f580c0b6960 con 0x7f580c161c80 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36228/0 (socket says 172.21.15.28:36228) 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 -- 172.21.15.28:0/3799509117 learned_addr learned my addr 172.21.15.28:0/3799509117 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580a000640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f580c161c80 0x7f580c0b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580aa00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580c0b64b0 0x7f580c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 -- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f580c161c80 msgr2=0x7f580c0b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f580c161c80 0x7f580c0b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 -- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580c0b64b0 msgr2=0x7f580c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580c0b64b0 0x7f580c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.764+0000 7f580b400640 1 -- 172.21.15.28:0/3799509117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c0b74e0 con 0x7f580c162060 2024-09-10T10:45:13.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f580b400640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f580400c820 tx=0x7f580400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:13.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804018070 con 0x7f580c162060 2024-09-10T10:45:13.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f580400a5e0 con 0x7f580c162060 2024-09-10T10:45:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580400a9d0 con 0x7f580c162060 2024-09-10T10:45:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c0b77c0 con 0x7f580c162060 2024-09-10T10:45:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.765+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c0b7d80 con 0x7f580c162060 2024-09-10T10:45:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.766+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580c0ba470 con 0x7f580c162060 2024-09-10T10:45:13.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.767+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f580400ab70 con 0x7f580c162060 2024-09-10T10:45:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.768+0000 7f5803400640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 0x7f57ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.768+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f580408a6f0 con 0x7f580c162060 2024-09-10T10:45:13.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.768+0000 7f580aa00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 0x7f57ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:13.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.769+0000 7f580aa00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 0x7f57ec06e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57fc034000 tx=0x7f57fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:13.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.772+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f580c06e870 con 0x7f580c162060 2024-09-10T10:45:13.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.991+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f580c160950 con 0x7f580c162060 2024-09-10T10:45:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.992+0000 7f5803400640 1 -- 172.21.15.28:0/3799509117 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5223 (secure 0 0 0) 0x7f5804058740 con 0x7f580c162060 2024-09-10T10:45:13.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:13.994 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":22,"btime":"2024-09-10T10:45:13:611719+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:12.613017+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-09-10T10:45:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 22 2024-09-10T10:45:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 msgr2=0x7f57ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 0x7f57ec06e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57fc034000 tx=0x7f57fc02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 msgr2=0x7f580c155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f580400c820 tx=0x7f580400cce0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 shutdown_connections 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57ec06be30 0x7f57ec06e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f580c162060 0x7f580c155bc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f580c161c80 0x7f580c0b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 --2- 172.21.15.28:0/3799509117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580c0b64b0 0x7f580c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 >> 172.21.15.28:0/3799509117 conn(0x7f580c0d44f0 msgr2=0x7f580c0d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 shutdown_connections 2024-09-10T10:45:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:13.994+0000 7f5811e00640 1 -- 172.21.15.28:0/3799509117 wait complete. 2024-09-10T10:45:14.415 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:44:55.704429+0000', 'modified': '2024-09-10T10:45:12.613017+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5088}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.28:6835/1058548427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 1058548427}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 1058548427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:45:14.415 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5088:up:rejoin 2024-09-10T10:45:15.416 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:15.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.580+0000 7ff334400640 1 Processor -- start 2024-09-10T10:45:15.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.580+0000 7ff334400640 1 -- start start 2024-09-10T10:45:15.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff334400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff334400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff32c072de0 0x7ff32c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff334400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c146140 con 0x7ff32c072700 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff334400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c0cd740 con 0x7ff32c072de0 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff334400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c0cda20 con 0x7ff32c072320 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff332400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff32c072700 0x7ff32c06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34848/0 (socket says 172.21.15.28:34848) 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff332400640 1 -- 172.21.15.28:0/3832994656 learned_addr learned my addr 172.21.15.28:0/3832994656 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff331a00640 1 --2- 172.21.15.28:0/3832994656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.582+0000 7ff331000640 1 --2- 172.21.15.28:0/3832994656 >> v2:172.21.15.138:3300/0 conn(0x7ff32c072de0 0x7ff32c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff32be00640 1 -- 172.21.15.28:0/3832994656 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539789706 0 0) 0x7ff32c146140 con 0x7ff32c072700 2024-09-10T10:45:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff32be00640 1 -- 172.21.15.28:0/3832994656 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff314003520 con 0x7ff32c072700 2024-09-10T10:45:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff331a00640 1 -- 172.21.15.28:0/3832994656 >> v2:172.21.15.138:3300/0 conn(0x7ff32c072de0 msgr2=0x7ff32c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff331a00640 1 --2- 172.21.15.28:0/3832994656 >> v2:172.21.15.138:3300/0 conn(0x7ff32c072de0 0x7ff32c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff331a00640 1 -- 172.21.15.28:0/3832994656 >> v1:172.21.15.28:6789/0 conn(0x7ff32c072700 legacy=0x7ff32c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.583+0000 7ff331a00640 1 -- 172.21.15.28:0/3832994656 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c0cdd00 con 0x7ff32c072320 2024-09-10T10:45:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.584+0000 7ff331a00640 1 --2- 172.21.15.28:0/3832994656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0cce60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3200097a0 tx=0x7ff32002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f358e372d434d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.584+0000 7ff32be00640 1 -- 172.21.15.28:0/3832994656 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320037070 con 0x7ff32c072320 2024-09-10T10:45:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.584+0000 7ff32be00640 1 -- 172.21.15.28:0/3832994656 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32002ae50 con 0x7ff32c072320 2024-09-10T10:45:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.584+0000 7ff32be00640 1 -- 172.21.15.28:0/3832994656 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff320032660 con 0x7ff32c072320 2024-09-10T10:45:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 -- 172.21.15.28:0/3832994656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 msgr2=0x7ff32c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 --2- 172.21.15.28:0/3832994656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0cce60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3200097a0 tx=0x7ff32002a100 comp rx=0 tx=0).stop 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 -- 172.21.15.28:0/3832994656 shutdown_connections 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 --2- 172.21.15.28:0/3832994656 >> v2:172.21.15.138:3300/0 conn(0x7ff32c072de0 0x7ff32c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 --2- 172.21.15.28:0/3832994656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0cce60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 -- 172.21.15.28:0/3832994656 >> 172.21.15.28:0/3832994656 conn(0x7ff32c067d70 msgr2=0x7ff32c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 -- 172.21.15.28:0/3832994656 shutdown_connections 2024-09-10T10:45:15.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.585+0000 7ff334400640 1 -- 172.21.15.28:0/3832994656 wait complete. 2024-09-10T10:45:15.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.587+0000 7ff334400640 1 Processor -- start 2024-09-10T10:45:15.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.587+0000 7ff334400640 1 -- start start 2024-09-10T10:45:15.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 0x7ff32c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0cd520 con 0x7ff32c072700 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c0cd3a0 con 0x7ff32c072de0 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff334400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c06ed00 con 0x7ff32c072320 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff331000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff331a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff331000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55246/0 (socket says 172.21.15.28:55246) 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff331a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0ab660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50606/0 (socket says 172.21.15.28:50606) 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff331000640 1 -- 172.21.15.28:0/4190721331 learned_addr learned my addr 172.21.15.28:0/4190721331 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.588+0000 7ff332400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 0x7ff32c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 -- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 msgr2=0x7ff32c0ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 -- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 msgr2=0x7ff32c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 0x7ff32c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 -- 172.21.15.28:0/4190721331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c13dc50 con 0x7ff32c072700 2024-09-10T10:45:15.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff332400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 0x7ff32c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff331000640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e870 tx=0x7ff31800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318017070 con 0x7ff32c072700 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31800c940 con 0x7ff32c072700 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31800cd10 con 0x7ff32c072700 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c0abe90 con 0x7ff32c072700 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.589+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c0ac450 con 0x7ff32c072700 2024-09-10T10:45:15.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.590+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32c0cce60 con 0x7ff32c072700 2024-09-10T10:45:15.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.591+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff31801a070 con 0x7ff32c072700 2024-09-10T10:45:15.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.593+0000 7ff329600640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:15.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.593+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7ff31808aeb0 con 0x7ff32c072700 2024-09-10T10:45:15.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.593+0000 7ff331a00640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:15.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.594+0000 7ff331a00640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff320034000 tx=0x7ff32002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:15.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.597+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff318058f90 con 0x7ff32c072700 2024-09-10T10:45:15.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.820+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff32c0c6580 con 0x7ff32c072700 2024-09-10T10:45:15.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.821+0000 7ff329600640 1 -- 172.21.15.28:0/4190721331 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7ff31805d290 con 0x7ff32c072700 2024-09-10T10:45:15.823 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:15.823 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:15.824 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:15.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 msgr2=0x7ff30c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff320034000 tx=0x7ff32002e000 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 msgr2=0x7ff32c15b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e870 tx=0x7ff31800ed30 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 shutdown_connections 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c072de0 0x7ff32c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c072700 0x7ff32c15b620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 --2- 172.21.15.28:0/4190721331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c072320 0x7ff32c0ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 >> 172.21.15.28:0/4190721331 conn(0x7ff32c067d70 msgr2=0x7ff32c0c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 shutdown_connections 2024-09-10T10:45:15.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:15.823+0000 7ff334400640 1 -- 172.21.15.28:0/4190721331 wait complete. 2024-09-10T10:45:16.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:44:55.704429+0000', 'modified': '2024-09-10T10:45:14.620095+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5088}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5088': {'gid': 5088, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6835/1058548427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 1058548427}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 1058548427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5088, 'qdb_cluster': [5088]} 2024-09-10T10:45:16.244 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:45:16.244 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:45:16.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.402+0000 7f4a53800640 1 Processor -- start 2024-09-10T10:45:16.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.402+0000 7f4a53800640 1 -- start start 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.403+0000 7f4a53800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a4c072320 0x7f4a4c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.403+0000 7f4a53800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a4c072de0 0x7f4a4c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a53800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c146140 con 0x7f4a4c072700 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a53800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c0cd740 con 0x7f4a4c072de0 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a53800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c0cda20 con 0x7f4a4c072320 2024-09-10T10:45:16.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a4c072de0 0x7f4a4c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a4c072de0 0x7f4a4c06e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55250/0 (socket says 172.21.15.28:55250) 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 -- 172.21.15.28:0/4178659789 learned_addr learned my addr 172.21.15.28:0/4178659789 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a50e00640 1 --2- 172.21.15.28:0/4178659789 >> v2:172.21.15.138:3300/0 conn(0x7f4a4c072320 0x7f4a4c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 -- 172.21.15.28:0/4178659789 >> v2:172.21.15.138:3300/0 conn(0x7f4a4c072320 msgr2=0x7f4a4c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 --2- 172.21.15.28:0/4178659789 >> v2:172.21.15.138:3300/0 conn(0x7f4a4c072320 0x7f4a4c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 -- 172.21.15.28:0/4178659789 >> v1:172.21.15.138:6789/0 conn(0x7f4a4c072700 legacy=0x7f4a4c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.404+0000 7f4a4be00640 1 -- 172.21.15.28:0/4178659789 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c0cdd00 con 0x7f4a4c072de0 2024-09-10T10:45:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.405+0000 7f4a4be00640 1 --2- 172.21.15.28:0/4178659789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072de0 0x7f4a4c06e0b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0b31e0 tx=0x7f4a3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88860197898f53e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a4b400640 1 -- 172.21.15.28:0/4178659789 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38037070 con 0x7f4a4c072de0 2024-09-10T10:45:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a4b400640 1 -- 172.21.15.28:0/4178659789 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38032440 con 0x7f4a4c072de0 2024-09-10T10:45:16.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a4b400640 1 -- 172.21.15.28:0/4178659789 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38032810 con 0x7f4a4c072de0 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 -- 172.21.15.28:0/4178659789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072de0 msgr2=0x7f4a4c06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 --2- 172.21.15.28:0/4178659789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072de0 0x7f4a4c06e0b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0b31e0 tx=0x7f4a3802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 -- 172.21.15.28:0/4178659789 shutdown_connections 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 --2- 172.21.15.28:0/4178659789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072de0 0x7f4a4c06e0b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 --2- 172.21.15.28:0/4178659789 >> v2:172.21.15.138:3300/0 conn(0x7f4a4c072320 0x7f4a4c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 -- 172.21.15.28:0/4178659789 >> 172.21.15.28:0/4178659789 conn(0x7f4a4c067d70 msgr2=0x7f4a4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 -- 172.21.15.28:0/4178659789 shutdown_connections 2024-09-10T10:45:16.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.406+0000 7f4a53800640 1 -- 172.21.15.28:0/4178659789 wait complete. 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.408+0000 7f4a53800640 1 Processor -- start 2024-09-10T10:45:16.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.408+0000 7f4a53800640 1 -- start start 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 0x7f4a4c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 0x7f4a4c0ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c0cd520 con 0x7f4a4c072320 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c0cd3a0 con 0x7f4a4c072de0 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a53800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c06ed00 con 0x7f4a4c072700 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55252/0 (socket says 172.21.15.28:55252) 2024-09-10T10:45:16.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.409+0000 7f4a50e00640 1 -- 172.21.15.28:0/404485579 learned_addr learned my addr 172.21.15.28:0/404485579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a4be00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 0x7f4a4c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a51800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 0x7f4a4c0ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 -- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 msgr2=0x7f4a4c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 0x7f4a4c0b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 -- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 msgr2=0x7f4a4c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 0x7f4a4c0ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 -- 172.21.15.28:0/404485579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c0aec00 con 0x7f4a4c072320 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a4be00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 0x7f4a4c0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a51800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 0x7f4a4c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a50e00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000d830 tx=0x7f4a4000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a48c00640 1 -- 172.21.15.28:0/404485579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000b6c0 con 0x7f4a4c072320 2024-09-10T10:45:16.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a48c00640 1 -- 172.21.15.28:0/404485579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a40019070 con 0x7f4a4c072320 2024-09-10T10:45:16.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c0b44a0 con 0x7f4a4c072320 2024-09-10T10:45:16.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a48c00640 1 -- 172.21.15.28:0/404485579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4000ba80 con 0x7f4a4c072320 2024-09-10T10:45:16.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.410+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c0b4a60 con 0x7f4a4c072320 2024-09-10T10:45:16.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.413+0000 7f4a48c00640 1 -- 172.21.15.28:0/404485579 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a4001e460 con 0x7f4a4c072320 2024-09-10T10:45:16.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.414+0000 7f4a48c00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.414+0000 7f4a48c00640 1 -- 172.21.15.28:0/404485579 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f4a4008b7c0 con 0x7f4a4c072320 2024-09-10T10:45:16.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.415+0000 7f4a4be00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.415+0000 7f4a567ea740 1 Processor -- start 2024-09-10T10:45:16.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.415+0000 7f4a567ea740 1 -- start start 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.415+0000 7f4a4be00640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034000 tx=0x7f4a3802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 0x55a3c11d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a3c11313b0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a3c1136410 con 0x55a3c11d4480 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a3c10de620 con 0x55a3c11d04b0 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55266/0 (socket says 172.21.15.28:55266) 2024-09-10T10:45:16.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 -- 172.21.15.28:0/3326367456 learned_addr learned my addr 172.21.15.28:0/3326367456 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a51800640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 0x55a3c11d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 -- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 msgr2=0x55a3c11d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 0x55a3c11d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 -- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 msgr2=0x55a3c11cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a50e00640 1 -- 172.21.15.28:0/3326367456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c0e492d0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 0x55a3c11d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.416+0000 7f4a51800640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a400899b0 tx=0x7f4a4000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77a902261326cee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a3d600640 1 -- 172.21.15.28:0/3326367456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4005dba0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a3d600640 1 -- 172.21.15.28:0/3326367456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400623d0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a3d600640 1 -- 172.21.15.28:0/3326367456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40055e60 con 0x55a3c11d2cf0 2024-09-10T10:45:16.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a567ea740 1 -- 172.21.15.28:0/3326367456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 msgr2=0x55a3c1218490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.417+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4a400899b0 tx=0x7f4a4000bf70 comp rx=0 tx=0).stop 2024-09-10T10:45:16.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 -- 172.21.15.28:0/3326367456 shutdown_connections 2024-09-10T10:45:16.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d04b0 0x55a3c11d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3326367456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c1218490 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 -- 172.21.15.28:0/3326367456 >> 172.21.15.28:0/3326367456 conn(0x55a3c11cf740 msgr2=0x55a3c1217960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 -- 172.21.15.28:0/3326367456 shutdown_connections 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 -- 172.21.15.28:0/3326367456 wait complete. 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.418+0000 7f4a567ea740 1 -- start start 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 Processor -- start 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 0x55a3c11fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a3c10f1780 con 0x55a3c11d2cf0 2024-09-10T10:45:16.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a3c11323d0 con 0x55a3c11d4480 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.419+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a3c11cebe0 con 0x55a3c11fce70 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a567ea740 10 client.0 _ll_register_callbacks cb 0x55a3c112f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55276/0 (socket says 172.21.15.28:55276) 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 -- 172.21.15.28:0/3809420801 learned_addr learned my addr 172.21.15.28:0/3809420801 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 -- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 msgr2=0x55a3c11fd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a51800640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 0x55a3c11fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 0x55a3c11fd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 -- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 msgr2=0x55a3c11f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 -- 172.21.15.28:0/3809420801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c0f136b0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a51800640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 0x55a3c11fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a50e00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a40055c50 tx=0x7f4a4008b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71952b382a40b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.420+0000 7f4a32000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4005dba0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f4a40061f40 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a400623d0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4a40062140 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 10 client.5136 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a3c11f69e0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a400623d0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f4a40013070 con 0x55a3c11d2cf0 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 10 client.5136 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 10 client.5136 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 20 client.5136 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c1233a20 0x55a3c1235e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 4 client.5136 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.421+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c1143b40 con 0x55a3c1233a20 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.422+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c1233a20 0x55a3c1235e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.422+0000 7f4a32000640 10 client.5136 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:16.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.422+0000 7f4a4be00640 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c1233a20 0x55a3c1235e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e299d590b8c467d7 server_cookie=7fcb70fb8ba1682d in_seq=0 out_seq=0 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.423+0000 7f4a32000640 1 -- 172.21.15.28:0/3809420801 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3c1143b40 con 0x55a3c1233a20 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.423+0000 7f4a32000640 10 client.5136 handle_command_reply: tid=0 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.423+0000 7f4a567ea740 1 client.5136 shutdown 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.423+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 msgr2=0x55a3c11f3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.423+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4a40055c50 tx=0x7f4a4008b040 comp rx=0 tx=0).stop 2024-09-10T10:45:16.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 shutdown_connections 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c1233a20 0x55a3c1235e10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11fce70 0x55a3c11fd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 --2- 172.21.15.28:0/3809420801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11d2cf0 0x55a3c11f3050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 >> 172.21.15.28:0/3809420801 conn(0x55a3c11d04b0 msgr2=0x55a3c1216ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a32000640 0 client.5136 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 shutdown_connections 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 1 -- 172.21.15.28:0/3809420801 wait complete. 2024-09-10T10:45:16.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.424+0000 7f4a567ea740 20 client.5136 trim_cache size 0 max 16384 2024-09-10T10:45:16.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 Processor -- start 2024-09-10T10:45:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 -- start start 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d2cf0 0x55a3c121f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c2002a0 con 0x55a3c11fce70 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c0cba10 con 0x55a3c11d4480 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.443+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c1ffc80 con 0x55a3c11d2cf0 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55282/0 (socket says 172.21.15.28:55282) 2024-09-10T10:45:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 -- 172.21.15.28:0/868459600 learned_addr learned my addr 172.21.15.28:0/868459600 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a50e00640 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d2cf0 0x55a3c121f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a4be00640 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 -- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d2cf0 msgr2=0x55a3c121f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d2cf0 0x55a3c121f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 -- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 msgr2=0x55a3c11f5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 -- 172.21.15.28:0/868459600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c11c8f70 con 0x55a3c11fce70 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.444+0000 7f4a51800640 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a44001d00 tx=0x7f4a440021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4b8f96185559ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a3cc00640 1 -- 172.21.15.28:0/868459600 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400fdd0 con 0x55a3c11fce70 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a3cc00640 1 -- 172.21.15.28:0/868459600 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44018070 con 0x55a3c11fce70 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a3cc00640 1 -- 172.21.15.28:0/868459600 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44003bb0 con 0x55a3c11fce70 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 -- 172.21.15.28:0/868459600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 msgr2=0x55a3c1215bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4a44001d00 tx=0x7f4a440021c0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 -- 172.21.15.28:0/868459600 shutdown_connections 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c1215bd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d4480 0x55a3c11f5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 --2- 172.21.15.28:0/868459600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d2cf0 0x55a3c121f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 -- 172.21.15.28:0/868459600 >> 172.21.15.28:0/868459600 conn(0x55a3c11d04b0 msgr2=0x55a3c11f2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.445+0000 7f4a567ea740 1 -- 172.21.15.28:0/868459600 shutdown_connections 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.446+0000 7f4a567ea740 1 -- 172.21.15.28:0/868459600 wait complete. 2024-09-10T10:45:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.446+0000 7f4a567ea740 1 -- start start 2024-09-10T10:45:16.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.446+0000 7f4a567ea740 1 Processor -- start 2024-09-10T10:45:16.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d2cf0 0x55a3c12a1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d4480 0x55a3c12826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a3c10f1780 con 0x55a3c11fce70 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a3c11323d0 con 0x55a3c11d2cf0 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a3c11f4db0 con 0x55a3c11d4480 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.447+0000 7f4a567ea740 10 client.0 _ll_register_callbacks cb 0x7f4a40070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55288/0 (socket says 172.21.15.28:55288) 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 -- 172.21.15.28:0/1201970510 learned_addr learned my addr 172.21.15.28:0/1201970510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a4be00640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d4480 0x55a3c12826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a50e00640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d2cf0 0x55a3c12a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 -- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d4480 msgr2=0x55a3c12826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d4480 0x55a3c12826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 -- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d2cf0 msgr2=0x55a3c12a1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d2cf0 0x55a3c12a1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 -- 172.21.15.28:0/1201970510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c12a2100 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a51800640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009010 tx=0x7f4a44001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5839140d35c4157f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.448+0000 7f4a32000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440050d0 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f4a44028cc0 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a44018070 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4a44006b50 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 10 client.5142 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55a3c1282c10 con 0x55a3c11fce70 2024-09-10T10:45:16.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44018070 con 0x55a3c11fce70 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f4a44005270 con 0x55a3c11fce70 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 10 client.5142 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 10 client.5142 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 20 client.5142 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c121b9b0 0x55a3c11f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 4 client.5142 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.449+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "status"}) -- 0x55a3c113de90 con 0x55a3c121b9b0 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a50e00640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c121b9b0 0x55a3c11f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a50e00640 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c121b9b0 0x55a3c11f7fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfbd003c63efd39 server_cookie=59db3be4a906e1f7 in_seq=0 out_seq=0 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a32000640 10 client.5142 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a32000640 1 -- 172.21.15.28:0/1201970510 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a3c113de90 con 0x55a3c121b9b0 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a32000640 10 client.5142 handle_command_reply: tid=0 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 client.5142 shutdown 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 msgr2=0x55a3c122b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009010 tx=0x7f4a44001250 comp rx=0 tx=0).stop 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 shutdown_connections 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55a3c121b9b0 0x55a3c11f7fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a3c11fce70 0x55a3c122b580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a3c11d4480 0x55a3c12826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 --2- 172.21.15.28:0/1201970510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a3c11d2cf0 0x55a3c12a1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a32000640 0 client.5142 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.450+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 >> 172.21.15.28:0/1201970510 conn(0x55a3c11d04b0 msgr2=0x55a3c12925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 shutdown_connections 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a567ea740 1 -- 172.21.15.28:0/1201970510 wait complete. 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a567ea740 20 client.5142 trim_cache size 0 max 16384 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:45:16.458 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5088, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 4.8469709070000002, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 23, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 43, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 43, 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 5.7705156869999996 2024-09-10T10:45:16.459 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:45:16.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 msgr2=0x7f4a2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034000 tx=0x7f4a3802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:16.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 msgr2=0x7f4a4c140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000d830 tx=0x7f4a4000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 shutdown_connections 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4a2c06be30 0x7f4a2c06e2f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a4c072de0 0x7f4a4c0ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a4c072700 0x7f4a4c0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 --2- 172.21.15.28:0/404485579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a4c072320 0x7f4a4c140fc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.451+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 >> 172.21.15.28:0/404485579 conn(0x7f4a4c067d70 msgr2=0x7f4a4c0c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.452+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 shutdown_connections 2024-09-10T10:45:16.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.452+0000 7f4a53800640 1 -- 172.21.15.28:0/404485579 wait complete. 2024-09-10T10:45:16.469 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:45:16.614 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:45:16.611+0000 7f8f76200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:45:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.624+0000 7f7a9aa00640 1 Processor -- start 2024-09-10T10:45:16.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.624+0000 7f7a9aa00640 1 -- start start 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.625+0000 7f7a9aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a9aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a9aa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a940cf000 con 0x7f7a940d0dd0 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a9aa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a940b37d0 con 0x7f7a940d1f90 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a9aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a940cf330 con 0x7f7a9415e660 2024-09-10T10:45:16.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50652/0 (socket says 172.21.15.28:50652) 2024-09-10T10:45:16.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 -- 172.21.15.28:0/1411155040 learned_addr learned my addr 172.21.15.28:0/1411155040 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a9a000640 1 --2- 172.21.15.28:0/1411155040 >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 -- 172.21.15.28:0/1411155040 >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 msgr2=0x7f7a940cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:16.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 --2- 172.21.15.28:0/1411155040 >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.626+0000 7f7a98c00640 1 -- 172.21.15.28:0/1411155040 >> v1:172.21.15.138:6789/0 conn(0x7f7a940d0dd0 legacy=0x7f7a940d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.627+0000 7f7a98c00640 1 -- 172.21.15.28:0/1411155040 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a940d29a0 con 0x7f7a9415e660 2024-09-10T10:45:16.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.628+0000 7f7a98c00640 1 --2- 172.21.15.28:0/1411155040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009870 tx=0x7f7a8002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec07eae04785b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a93e00640 1 -- 172.21.15.28:0/1411155040 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037070 con 0x7f7a9415e660 2024-09-10T10:45:16.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a93e00640 1 -- 172.21.15.28:0/1411155040 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a80032440 con 0x7f7a9415e660 2024-09-10T10:45:16.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a93e00640 1 -- 172.21.15.28:0/1411155040 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80032810 con 0x7f7a9415e660 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 -- 172.21.15.28:0/1411155040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 msgr2=0x7f7a9406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/1411155040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009870 tx=0x7f7a8002a250 comp rx=0 tx=0).stop 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 -- 172.21.15.28:0/1411155040 shutdown_connections 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/1411155040 >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/1411155040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 -- 172.21.15.28:0/1411155040 >> 172.21.15.28:0/1411155040 conn(0x7f7a9408bfc0 msgr2=0x7f7a940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 -- 172.21.15.28:0/1411155040 shutdown_connections 2024-09-10T10:45:16.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.632+0000 7f7a9aa00640 1 -- 172.21.15.28:0/1411155040 wait complete. 2024-09-10T10:45:16.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.634+0000 7f7a9aa00640 1 Processor -- start 2024-09-10T10:45:16.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.634+0000 7f7a9aa00640 1 -- start start 2024-09-10T10:45:16.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a940b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 0x7f7a94157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7a940d3220 con 0x7f7a940d1f90 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a940d2f40 con 0x7f7a9415e660 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a940d33a0 con 0x7f7a940d0dd0 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a98c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a98c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55290/0 (socket says 172.21.15.28:55290) 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a98c00640 1 -- 172.21.15.28:0/4066017149 learned_addr learned my addr 172.21.15.28:0/4066017149 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a99600640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a940b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.635+0000 7f7a9a000640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 0x7f7a94157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 -- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 msgr2=0x7f7a940b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a940b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 -- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 msgr2=0x7f7a94157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 0x7f7a94157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 -- 172.21.15.28:0/4066017149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a941f4500 con 0x7f7a940d1f90 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a9a000640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 0x7f7a94157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a98c00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a800090b0 tx=0x7f7a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80041070 con 0x7f7a940d1f90 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8003c8b0 con 0x7f7a940d1f90 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037040 con 0x7f7a940d1f90 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.636+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a940b8100 con 0x7f7a940d1f90 2024-09-10T10:45:16.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.637+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a940b86c0 con 0x7f7a940d1f90 2024-09-10T10:45:16.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.638+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a940d2440 con 0x7f7a940d1f90 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.638+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a80044070 con 0x7f7a940d1f90 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.640+0000 7f7a91600640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.640+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f7a800aba30 con 0x7f7a940d1f90 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.640+0000 7f7a99600640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.641+0000 7f7a99600640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a88000f60 tx=0x7f7a88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:16.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.645+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a941f4500 con 0x7f7a940d1f90 2024-09-10T10:45:16.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.863+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7a940b10b0 con 0x7f7a940d1f90 2024-09-10T10:45:16.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.864+0000 7f7a91600640 1 -- 172.21.15.28:0/4066017149 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f7a80079c30 con 0x7f7a940d1f90 2024-09-10T10:45:16.866 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 msgr2=0x7f7a7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7a88000f60 tx=0x7f7a88009040 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 msgr2=0x7f7a940c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a800090b0 tx=0x7f7a800046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 shutdown_connections 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a9415e660 0x7f7a94157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a940d1f90 0x7f7a940c9a70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 --2- 172.21.15.28:0/4066017149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a940b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 >> 172.21.15.28:0/4066017149 conn(0x7f7a9408bfc0 msgr2=0x7f7a940caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 shutdown_connections 2024-09-10T10:45:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:16.866+0000 7f7a9aa00640 1 -- 172.21.15.28:0/4066017149 wait complete. 2024-09-10T10:45:16.884 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:45:16.885 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:16.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.886 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:45:16.926 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:45:16.926 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:45:16.927 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:16.928 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:16.929 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:16.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:16.930 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:45:17.025 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:45:17.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.028 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:45:17.067 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:45:17.067 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:45:17.068 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:45:17.068 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:45:17.068 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:45:17.068 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:45:17.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:17.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:17.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.267 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:45:17.302 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:45:17.303 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:45:17.303 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:17.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:45:17.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:17.400 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:17.435 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:17.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:17.436 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:17.510 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:45:17.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.511 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:17.639 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:17.640 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:17.640 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:17.655 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:17.655 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:17.655 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:17.656 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:17.656 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:45:17.678 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:45:17.679 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:45:17.679 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:45:17.679 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:45:17.680 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:45:17.680 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:45:17.682 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:45:17.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.683 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:17.739 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:17.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:17.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:17.829 INFO:teuthology.orchestra.run.smithi028.stdout:client4993 2024-09-10T10:45:17.830 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:17.830 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:17.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:17.865 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-10T10:45:17.943 INFO:teuthology.orchestra.run.smithi028.stdout:file_undamaged 2024-09-10T10:45:17.944 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> import json 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> except OSError as e: 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(e.errno) 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:17.945 DEBUG:teuthology.orchestra.run.smithi028:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T10:45:17.946 DEBUG:teuthology.orchestra.run.smithi028:> print(json.dumps( 2024-09-10T10:45:17.946 DEBUG:teuthology.orchestra.run.smithi028:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T10:45:17.946 DEBUG:teuthology.orchestra.run.smithi028:> indent=2)) 2024-09-10T10:45:17.946 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:45:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-10T10:45:18.099 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T10:45:18.099 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:18.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.294+0000 7fba69c00640 1 Processor -- start 2024-09-10T10:45:18.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.294+0000 7fba69c00640 1 -- start start 2024-09-10T10:45:18.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba69c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba6415b300 0x7fba640cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba69c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba640cd3f0 0x7fba6406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba69c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba640cdbe0 con 0x7fba64072d30 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba69c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fba640728a0 con 0x7fba6415b300 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba69c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba640cd900 con 0x7fba640cd3f0 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.295+0000 7fba62a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba6415b300 0x7fba640cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba6415b300 0x7fba640cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55302/0 (socket says 172.21.15.28:55302) 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 -- 172.21.15.28:0/2692617229 learned_addr learned my addr 172.21.15.28:0/2692617229 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62000640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba640cd3f0 0x7fba6406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 -- 172.21.15.28:0/2692617229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba640cd3f0 msgr2=0x7fba6406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba640cd3f0 0x7fba6406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 -- 172.21.15.28:0/2692617229 >> v1:172.21.15.138:6789/0 conn(0x7fba64072d30 legacy=0x7fba640b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.296+0000 7fba62a00640 1 -- 172.21.15.28:0/2692617229 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64072250 con 0x7fba6415b300 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.297+0000 7fba62a00640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640cceb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba5400bb10 tx=0x7fba5402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132161611c7b1e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.297+0000 7fba61600640 1 -- 172.21.15.28:0/2692617229 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54036020 con 0x7fba6415b300 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.297+0000 7fba61600640 1 -- 172.21.15.28:0/2692617229 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba54032820 con 0x7fba6415b300 2024-09-10T10:45:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.297+0000 7fba61600640 1 -- 172.21.15.28:0/2692617229 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba54032bf0 con 0x7fba6415b300 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 -- 172.21.15.28:0/2692617229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 msgr2=0x7fba640cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640cceb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fba5400bb10 tx=0x7fba5402c920 comp rx=0 tx=0).stop 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 -- 172.21.15.28:0/2692617229 shutdown_connections 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba640cd3f0 0x7fba6406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 --2- 172.21.15.28:0/2692617229 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640cceb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 -- 172.21.15.28:0/2692617229 >> 172.21.15.28:0/2692617229 conn(0x7fba64067d70 msgr2=0x7fba6406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 -- 172.21.15.28:0/2692617229 shutdown_connections 2024-09-10T10:45:18.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.298+0000 7fba69c00640 1 -- 172.21.15.28:0/2692617229 wait complete. 2024-09-10T10:45:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.300+0000 7fba69c00640 1 Processor -- start 2024-09-10T10:45:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.300+0000 7fba69c00640 1 -- start start 2024-09-10T10:45:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba64072d30 0x7fba64148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640cd3f0 0x7fba640cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fba640d0160 con 0x7fba6415b300 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba640cffe0 con 0x7fba640cd3f0 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba69c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba6406f700 con 0x7fba64072d30 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55318/0 (socket says 172.21.15.28:55318) 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 -- 172.21.15.28:0/3290723921 learned_addr learned my addr 172.21.15.28:0/3290723921 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba62000640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640cd3f0 0x7fba640cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba62a00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba64072d30 0x7fba64148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 -- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba64072d30 msgr2=0x7fba64148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba64072d30 0x7fba64148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 -- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640cd3f0 msgr2=0x7fba640cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640cd3f0 0x7fba640cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.301+0000 7fba63400640 1 -- 172.21.15.28:0/3290723921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba641492d0 con 0x7fba6415b300 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba63400640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00c7a0 tx=0x7fba5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c003b70 con 0x7fba6415b300 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c003d30 con 0x7fba6415b300 2024-09-10T10:45:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00aa90 con 0x7fba6415b300 2024-09-10T10:45:18.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba640ab160 con 0x7fba6415b300 2024-09-10T10:45:18.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.302+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba640ab720 con 0x7fba6415b300 2024-09-10T10:45:18.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.303+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6406fac0 con 0x7fba6415b300 2024-09-10T10:45:18.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.304+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba5c00ac30 con 0x7fba6415b300 2024-09-10T10:45:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.305+0000 7fba5aa00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 0x7fba4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.305+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fba5c08aef0 con 0x7fba6415b300 2024-09-10T10:45:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.306+0000 7fba62a00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 0x7fba4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.306+0000 7fba62a00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 0x7fba4406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba54037000 tx=0x7fba54032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.309+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fba641492d0 con 0x7fba6415b300 2024-09-10T10:45:18.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.527+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba641e61b0 con 0x7fba6415b300 2024-09-10T10:45:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.528+0000 7fba5aa00640 1 -- 172.21.15.28:0/3290723921 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7fba5c058fd0 con 0x7fba6415b300 2024-09-10T10:45:18.530 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:18.530 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 msgr2=0x7fba4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 0x7fba4406e2f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fba54037000 tx=0x7fba54032000 comp rx=0 tx=0).stop 2024-09-10T10:45:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 msgr2=0x7fba640ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00c7a0 tx=0x7fba5c00cc60 comp rx=0 tx=0).stop 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 shutdown_connections 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba4406be30 0x7fba4406e2f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba6415b300 0x7fba640ac200 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba640cd3f0 0x7fba640cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 --2- 172.21.15.28:0/3290723921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba64072d30 0x7fba64148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 >> 172.21.15.28:0/3290723921 conn(0x7fba64067d70 msgr2=0x7fba64159990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 shutdown_connections 2024-09-10T10:45:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.530+0000 7fba69c00640 1 -- 172.21.15.28:0/3290723921 wait complete. 2024-09-10T10:45:18.548 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T10:45:18.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.706+0000 7f21c1600640 1 Processor -- start 2024-09-10T10:45:18.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.706+0000 7f21c1600640 1 -- start start 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21c1600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f21bc161d00 0x7f21bc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21c1600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21bc0b62b0 0x7f21bc0d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21c1600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21bc0d4300 con 0x7f21bc161920 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21c1600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f21bc0b49a0 con 0x7f21bc0b62b0 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21c1600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f21bc0b4c40 con 0x7f21bc161d00 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21baa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21bc161920 0x7f21bc0b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34862/0 (socket says 172.21.15.28:34862) 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21baa00640 1 -- 172.21.15.28:0/3650472518 learned_addr learned my addr 172.21.15.28:0/3650472518 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:18.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.707+0000 7f21bb400640 1 --2- 172.21.15.28:0/3650472518 >> v2:172.21.15.28:3300/0 conn(0x7f21bc0b62b0 0x7f21bc0d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21ba000640 1 --2- 172.21.15.28:0/3650472518 >> v2:172.21.15.138:3300/0 conn(0x7f21bc161d00 0x7f21bc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21b9600640 1 -- 172.21.15.28:0/3650472518 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3647019225 0 0) 0x7f21bc0d4300 con 0x7f21bc161920 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21b9600640 1 -- 172.21.15.28:0/3650472518 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21a0003520 con 0x7f21bc161920 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21bb400640 1 -- 172.21.15.28:0/3650472518 >> v2:172.21.15.138:3300/0 conn(0x7f21bc161d00 msgr2=0x7f21bc0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21bb400640 1 --2- 172.21.15.28:0/3650472518 >> v2:172.21.15.138:3300/0 conn(0x7f21bc161d00 0x7f21bc0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21bb400640 1 -- 172.21.15.28:0/3650472518 >> v1:172.21.15.28:6789/0 conn(0x7f21bc161920 legacy=0x7f21bc0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.708+0000 7f21bb400640 1 -- 172.21.15.28:0/3650472518 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc0d4db0 con 0x7f21bc0b62b0 2024-09-10T10:45:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.709+0000 7f21bb400640 1 --2- 172.21.15.28:0/3650472518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc0b62b0 0x7f21bc0d2600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f21bc08d9a0 tx=0x7f21b402a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78294a485cf455f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.709+0000 7f21b9600640 1 -- 172.21.15.28:0/3650472518 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4007740 con 0x7f21bc0b62b0 2024-09-10T10:45:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.709+0000 7f21b9600640 1 -- 172.21.15.28:0/3650472518 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b40078e0 con 0x7f21bc0b62b0 2024-09-10T10:45:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.709+0000 7f21b9600640 1 -- 172.21.15.28:0/3650472518 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4007c90 con 0x7f21bc0b62b0 2024-09-10T10:45:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 -- 172.21.15.28:0/3650472518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc0b62b0 msgr2=0x7f21bc0d2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 --2- 172.21.15.28:0/3650472518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc0b62b0 0x7f21bc0d2600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f21bc08d9a0 tx=0x7f21b402a560 comp rx=0 tx=0).stop 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 -- 172.21.15.28:0/3650472518 shutdown_connections 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 --2- 172.21.15.28:0/3650472518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc0b62b0 0x7f21bc0d2600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 --2- 172.21.15.28:0/3650472518 >> v2:172.21.15.138:3300/0 conn(0x7f21bc161d00 0x7f21bc0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 -- 172.21.15.28:0/3650472518 >> 172.21.15.28:0/3650472518 conn(0x7f21bc08bfc0 msgr2=0x7f21bc0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 -- 172.21.15.28:0/3650472518 shutdown_connections 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.710+0000 7f21c1600640 1 -- 172.21.15.28:0/3650472518 wait complete. 2024-09-10T10:45:18.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.712+0000 7f21c1600640 1 Processor -- start 2024-09-10T10:45:18.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.712+0000 7f21c1600640 1 -- start start 2024-09-10T10:45:18.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.712+0000 7f21c1600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21bc0b62b0 0x7f21bc155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.712+0000 7f21c1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21c1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 0x7f21bc0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21c1600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0d2b40 con 0x7f21bc161920 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21c1600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0b68d0 con 0x7f21bc161d00 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21c1600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc0d2cc0 con 0x7f21bc0b62b0 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55344/0 (socket says 172.21.15.28:55344) 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 -- 172.21.15.28:0/1848313985 learned_addr learned my addr 172.21.15.28:0/1848313985 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21baa00640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21bc0b62b0 0x7f21bc155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21bb400640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 0x7f21bc0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 -- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21bc0b62b0 msgr2=0x7f21bc155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21bc0b62b0 0x7f21bc155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 -- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 msgr2=0x7f21bc0cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 0x7f21bc0cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21ba000640 1 -- 172.21.15.28:0/1848313985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc0c9b80 con 0x7f21bc161920 2024-09-10T10:45:18.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.713+0000 7f21bb400640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 0x7f21bc0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:18.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21ba000640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f21a800d5f0 tx=0x7f21a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21b2a00640 1 -- 172.21.15.28:0/1848313985 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80125f0 con 0x7f21bc161920 2024-09-10T10:45:18.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21b2a00640 1 -- 172.21.15.28:0/1848313985 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a8019070 con 0x7f21bc161920 2024-09-10T10:45:18.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21b2a00640 1 -- 172.21.15.28:0/1848313985 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a80129d0 con 0x7f21bc161920 2024-09-10T10:45:18.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc0c9e60 con 0x7f21bc161920 2024-09-10T10:45:18.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.714+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc0ca420 con 0x7f21bc161920 2024-09-10T10:45:18.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.717+0000 7f21b2a00640 1 -- 172.21.15.28:0/1848313985 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21a8012b70 con 0x7f21bc161920 2024-09-10T10:45:18.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.718+0000 7f21b2a00640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 0x7f219c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.718+0000 7f21b2a00640 1 -- 172.21.15.28:0/1848313985 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f21a808b490 con 0x7f21bc161920 2024-09-10T10:45:18.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.718+0000 7f21baa00640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 0x7f219c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.718+0000 7f21c4409740 1 Processor -- start 2024-09-10T10:45:18.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.718+0000 7f21c4409740 1 -- start start 2024-09-10T10:45:18.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21baa00640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 0x7f219c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21ac005880 tx=0x7f21ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745546030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647454fe050 0x564745500c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564745464760 con 0x564745502020 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56474540c6c0 con 0x5647454fe050 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.719+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564745460720 con 0x564745500890 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55346/0 (socket says 172.21.15.28:55346) 2024-09-10T10:45:18.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 -- 172.21.15.28:0/2638079735 learned_addr learned my addr 172.21.15.28:0/2638079735 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21ba000640 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647454fe050 0x564745500c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21baa00640 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745546030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 -- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 msgr2=0x564745546030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745546030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 -- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647454fe050 msgr2=0x564745500c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647454fe050 0x564745500c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.720+0000 7f21bb400640 1 -- 172.21.15.28:0/2638079735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647452437c0 con 0x564745502020 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21bb400640 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009e50 tx=0x7f21b402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d597602333f76058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21a7000640 1 -- 172.21.15.28:0/2638079735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40078c0 con 0x564745502020 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21a7000640 1 -- 172.21.15.28:0/2638079735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4041070 con 0x564745502020 2024-09-10T10:45:18.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21a7000640 1 -- 172.21.15.28:0/2638079735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4007c60 con 0x564745502020 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 -- 172.21.15.28:0/2638079735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 msgr2=0x5647454fae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21b4009e50 tx=0x7f21b402e000 comp rx=0 tx=0).stop 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 -- 172.21.15.28:0/2638079735 shutdown_connections 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x5647454fae80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647454fe050 0x564745500c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 --2- 172.21.15.28:0/2638079735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745546030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.721+0000 7f21c4409740 1 -- 172.21.15.28:0/2638079735 >> 172.21.15.28:0/2638079735 conn(0x5647454fd050 msgr2=0x564745545500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.722+0000 7f21c4409740 1 -- 172.21.15.28:0/2638079735 shutdown_connections 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.722+0000 7f21c4409740 1 -- 172.21.15.28:0/2638079735 wait complete. 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.722+0000 7f21c4409740 1 -- start start 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.723+0000 7f21c4409740 1 Processor -- start 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 0x564745559420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x564745523900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5647454fc4f0 con 0x564745502020 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56474541f8e0 con 0x564745500890 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564745546860 con 0x564745523420 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21c4409740 10 client.0 _ll_register_callbacks cb 0x5647454113c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55350/0 (socket says 172.21.15.28:55350) 2024-09-10T10:45:18.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 -- 172.21.15.28:0/1828852105 learned_addr learned my addr 172.21.15.28:0/1828852105 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21baa00640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 0x564745559420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 -- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 msgr2=0x564745523900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x564745523900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 -- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 msgr2=0x564745559420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 0x564745559420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 -- 172.21.15.28:0/1828852105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647452a3f50 con 0x564745502020 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21baa00640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 0x564745559420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f21ba000640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21a80896e0 tx=0x7f21a80595d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea852c8bf08a074 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.724+0000 7f219b400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:18.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8071020 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f21a8055b30 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b4041070 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f21a8076050 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 10 client.5166 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5647451370b0 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4041070 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f21a8013070 con 0x564745502020 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 10 client.5166 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 10 client.5166 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 20 client.5166 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:18.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x56474552a770 0x56474552ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 4 client.5166 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.725+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564745471e90 con 0x56474552a770 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.726+0000 7f21baa00640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x56474552a770 0x56474552ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.726+0000 7f21baa00640 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x56474552a770 0x56474552ab50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466b6c79539094ce server_cookie=382611b53611bbdc in_seq=0 out_seq=0 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.726+0000 7f219b400640 10 client.5166 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f219b400640 1 -- 172.21.15.28:0/1828852105 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564745471e90 con 0x56474552a770 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f219b400640 10 client.5166 handle_command_reply: tid=0 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 client.5166 shutdown 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 msgr2=0x564745521260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f21a80896e0 tx=0x7f21a80595d0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 shutdown_connections 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x56474552a770 0x56474552ab50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x564745523900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745502020 0x564745521260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f219b400640 0 client.5166 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 --2- 172.21.15.28:0/1828852105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745500890 0x564745559420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 >> 172.21.15.28:0/1828852105 conn(0x5647454fe050 msgr2=0x5647454fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.727+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 shutdown_connections 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.728+0000 7f21c4409740 1 -- 172.21.15.28:0/1828852105 wait complete. 2024-09-10T10:45:18.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.728+0000 7f21c4409740 20 client.5166 trim_cache size 0 max 16384 2024-09-10T10:45:18.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 Processor -- start 2024-09-10T10:45:18.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 -- start start 2024-09-10T10:45:18.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745529b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x56474552ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc200060 con 0x564745523420 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc09eff0 con 0x564745502020 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.742+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc1ffa40 con 0x564745500890 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55352/0 (socket says 172.21.15.28:55352) 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 -- 172.21.15.28:0/1527955897 learned_addr learned my addr 172.21.15.28:0/1527955897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21ba000640 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x56474552ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21baa00640 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745529b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 -- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 msgr2=0x564745529b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745529b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 -- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 msgr2=0x56474552ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x56474552ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 -- 172.21.15.28:0/1527955897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647454feea0 con 0x564745523420 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.743+0000 7f21bb400640 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21b4042000 tx=0x7f21b402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dd29d80a584886 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21a6600640 1 -- 172.21.15.28:0/1527955897 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4003b90 con 0x564745523420 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21a6600640 1 -- 172.21.15.28:0/1527955897 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4041040 con 0x564745523420 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21a6600640 1 -- 172.21.15.28:0/1527955897 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b40076c0 con 0x564745523420 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 -- 172.21.15.28:0/1527955897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 msgr2=0x564745553b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21b4042000 tx=0x7f21b402c000 comp rx=0 tx=0).stop 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 -- 172.21.15.28:0/1527955897 shutdown_connections 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745523420 0x564745553b90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x56474552ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 --2- 172.21.15.28:0/1527955897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745500890 0x564745529b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 -- 172.21.15.28:0/1527955897 >> 172.21.15.28:0/1527955897 conn(0x5647454fe050 msgr2=0x5647454f62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 -- 172.21.15.28:0/1527955897 shutdown_connections 2024-09-10T10:45:18.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.744+0000 7f21c4409740 1 -- 172.21.15.28:0/1527955897 wait complete. 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.745+0000 7f21c4409740 1 -- start start 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.745+0000 7f21c4409740 1 Processor -- start 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x564745554c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x5647455c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5647455217a0 con 0x564745500890 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564745522f00 con 0x564745502020 2024-09-10T10:45:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564745528270 con 0x564745523420 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21c4409740 10 client.0 _ll_register_callbacks cb 0x7f21a8059c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.746+0000 7f21baa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55358/0 (socket says 172.21.15.28:55358) 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 -- 172.21.15.28:0/3308901775 learned_addr learned my addr 172.21.15.28:0/3308901775 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21bb400640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x5647455c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21ba000640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x564745554c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 -- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 msgr2=0x5647455c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x5647455c34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 -- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 msgr2=0x564745554c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x564745554c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 -- 172.21.15.28:0/3308901775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564745568050 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21bb400640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x5647455c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f21baa00640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0013a0 tx=0x7f21ac0010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9ac33456a2a672 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f219b400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac028040 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.747+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f21ac010040 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21b4041040 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f21ac014e10 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 10 client.5172 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x5647455551a0 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b4041040 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f21ac035050 con 0x564745500890 2024-09-10T10:45:18.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 10 client.5172 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 10 client.5172 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 20 client.5172 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x564745569890 0x564745548a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 4 client.5172 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56474546c1e0 con 0x564745569890 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.748+0000 7f21ba000640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x564745569890 0x564745548a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21ba000640 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x564745569890 0x564745548a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965a19aa9130108d server_cookie=b496e525eae3756a in_seq=0 out_seq=0 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f219b400640 10 client.5172 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f219b400640 1 -- 172.21.15.28:0/3308901775 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+236 (crc 0 0 0) 0x56474546c1e0 con 0x564745569890 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f219b400640 10 client.5172 handle_command_reply: tid=0 2024-09-10T10:45:18.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 client.5172 shutdown 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 msgr2=0x5647455205b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21ac0013a0 tx=0x7f21ac0010a0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 shutdown_connections 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x564745569890 0x564745548a10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564745523420 0x5647455c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564745502020 0x564745554c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 --2- 172.21.15.28:0/3308901775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564745500890 0x5647455205b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f219b400640 0 client.5172 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.749+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 >> 172.21.15.28:0/3308901775 conn(0x5647454fe050 msgr2=0x56474552ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 shutdown_connections 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c4409740 1 -- 172.21.15.28:0/3308901775 wait complete. 2024-09-10T10:45:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c4409740 20 client.5172 trim_cache size 0 max 16384 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "dentry", 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 415178632, 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627776, 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "frag": "*", 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "file_to_be_damaged", 2024-09-10T10:45:18.758 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_id": "head", 2024-09-10T10:45:18.759 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-10T10:45:18.759 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:45:18.759 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:45:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 msgr2=0x7f219c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 0x7f219c06e2f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21ac005880 tx=0x7f21ac009040 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 msgr2=0x7f21bc0a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f21a800d5f0 tx=0x7f21a800dab0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 shutdown_connections 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f219c06be30 0x7f219c06e2f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21bc161d00 0x7f21bc0cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21bc161920 0x7f21bc0a1b80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 --2- 172.21.15.28:0/1848313985 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21bc0b62b0 0x7f21bc155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.750+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 >> 172.21.15.28:0/1848313985 conn(0x7f21bc08bfc0 msgr2=0x7f21bc0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.751+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 shutdown_connections 2024-09-10T10:45:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18.751+0000 7f21c1600640 1 -- 172.21.15.28:0/1848313985 wait complete. 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> import errno 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> f.close() 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> except IOError as e: 2024-09-10T10:45:18.770 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(errno.EIO) 2024-09-10T10:45:18.771 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:45:18.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:18 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T10:45:18.883 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> import json 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> except OSError as e: 2024-09-10T10:45:18.884 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(e.errno) 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> print(json.dumps( 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> indent=2)) 2024-09-10T10:45:18.885 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:45:19.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T10:45:19.038 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T10:45:19.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:19.039 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T10:45:19.115 INFO:teuthology.orchestra.run.smithi028.stdout:2 2024-09-10T10:45:19.116 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:19.116 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:19.191 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:19.192 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:19.268 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:45:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.463+0000 7fbefb800640 1 Processor -- start 2024-09-10T10:45:19.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.463+0000 7fbefb800640 1 -- start start 2024-09-10T10:45:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.464+0000 7fbefb800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef415b300 0x7fbef40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.464+0000 7fbefb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.464+0000 7fbefb800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef40cdbe0 con 0x7fbef4072d30 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.464+0000 7fbefb800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbef40728a0 con 0x7fbef415b300 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.464+0000 7fbefb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbef40cd900 con 0x7fbef40cd3f0 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef415b300 0x7fbef40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef415b300 0x7fbef40cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55374/0 (socket says 172.21.15.28:55374) 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 -- 172.21.15.28:0/3444276077 learned_addr learned my addr 172.21.15.28:0/3444276077 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 -- 172.21.15.28:0/3444276077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 msgr2=0x7fbef406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef3e00640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 -- 172.21.15.28:0/3444276077 >> v1:172.21.15.138:6789/0 conn(0x7fbef4072d30 legacy=0x7fbef40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef8e00640 1 -- 172.21.15.28:0/3444276077 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4072250 con 0x7fbef415b300 2024-09-10T10:45:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.465+0000 7fbef3e00640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.466+0000 7fbef8e00640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40cceb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a9d680d3df740c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.466+0000 7fbef3400640 1 -- 172.21.15.28:0/3444276077 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbef415b300 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.466+0000 7fbef3400640 1 -- 172.21.15.28:0/3444276077 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbef415b300 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.466+0000 7fbef3400640 1 -- 172.21.15.28:0/3444276077 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8032810 con 0x7fbef415b300 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 -- 172.21.15.28:0/3444276077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 msgr2=0x7fbef40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40cceb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee802a200 comp rx=0 tx=0).stop 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 -- 172.21.15.28:0/3444276077 shutdown_connections 2024-09-10T10:45:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 --2- 172.21.15.28:0/3444276077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40cceb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 -- 172.21.15.28:0/3444276077 >> 172.21.15.28:0/3444276077 conn(0x7fbef4067d70 msgr2=0x7fbef406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 -- 172.21.15.28:0/3444276077 shutdown_connections 2024-09-10T10:45:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.467+0000 7fbefb800640 1 -- 172.21.15.28:0/3444276077 wait complete. 2024-09-10T10:45:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.469+0000 7fbefb800640 1 Processor -- start 2024-09-10T10:45:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.469+0000 7fbefb800640 1 -- start start 2024-09-10T10:45:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4072d30 0x7fbef4151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef41564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40d0160 con 0x7fbef415b300 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40cffe0 con 0x7fbef4072d30 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbefb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbef406f700 con 0x7fbef40cd3f0 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55384/0 (socket says 172.21.15.28:55384) 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef9800640 1 -- 172.21.15.28:0/1290622253 learned_addr learned my addr 172.21.15.28:0/1290622253 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef8e00640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4072d30 0x7fbef4151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef3e00640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef41564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef9800640 1 -- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 msgr2=0x7fbef41564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.470+0000 7fbef9800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef41564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef9800640 1 -- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4072d30 msgr2=0x7fbef4151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef9800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4072d30 0x7fbef4151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef9800640 1 -- 172.21.15.28:0/1290622253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef41523a0 con 0x7fbef415b300 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef9800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c7b0 tx=0x7fbeec00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004b60 con 0x7fbef415b300 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec004d00 con 0x7fbef415b300 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00aa90 con 0x7fbef415b300 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef4152680 con 0x7fbef415b300 2024-09-10T10:45:19.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.471+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef41f0740 con 0x7fbef415b300 2024-09-10T10:45:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.472+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef41f0940 con 0x7fbef415b300 2024-09-10T10:45:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.473+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbeec00ac30 con 0x7fbef415b300 2024-09-10T10:45:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.474+0000 7fbef0c00640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 0x7fbed406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.474+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fbeec08b130 con 0x7fbef415b300 2024-09-10T10:45:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.475+0000 7fbef8e00640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 0x7fbed406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.476+0000 7fbef8e00640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 0x7fbed406e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:19.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.479+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbef41f0940 con 0x7fbef415b300 2024-09-10T10:45:19.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.676+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbef41f0500 con 0x7fbef415b300 2024-09-10T10:45:19.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.676+0000 7fbef0c00640 1 -- 172.21.15.28:0/1290622253 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v43) ==== 80+0+122 (secure 0 0 0) 0x7fbeec020070 con 0x7fbef415b300 2024-09-10T10:45:19.678 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3203683106 2024-09-11T10:45:10.614652+0000 2024-09-10T10:45:19.679 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3203683106 2024-09-11T10:45:10.614652+0000 2024-09-10T10:45:19.679 INFO:teuthology.orchestra.run.smithi028.stderr:listed 2 entries 2024-09-10T10:45:19.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 msgr2=0x7fbed406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 0x7fbed406e340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:19.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 msgr2=0x7fbef40b2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c7b0 tx=0x7fbeec00cc70 comp rx=0 tx=0).stop 2024-09-10T10:45:19.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 shutdown_connections 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be80 0x7fbed406e340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef415b300 0x7fbef40b2ac0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef40cd3f0 0x7fbef41564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 --2- 172.21.15.28:0/1290622253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef4072d30 0x7fbef4151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 >> 172.21.15.28:0/1290622253 conn(0x7fbef4067d70 msgr2=0x7fbef40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 shutdown_connections 2024-09-10T10:45:19.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.679+0000 7fbefb800640 1 -- 172.21.15.28:0/1290622253 wait complete. 2024-09-10T10:45:19.697 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:45:19.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:19.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:19.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:45:19.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:19.759 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:45:19.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:19.796 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start /subdir repair 2024-09-10T10:45:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.993+0000 7fde60800640 1 Processor -- start 2024-09-10T10:45:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.994+0000 7fde60800640 1 -- start start 2024-09-10T10:45:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde60800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fde58156740 0x7fde580b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde60800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fde580b4af0 0x7fde580cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde60800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde580b30e0 con 0x7fde580cfcd0 2024-09-10T10:45:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde60800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fde580d0140 con 0x7fde580b4af0 2024-09-10T10:45:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde60800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fde5806edc0 con 0x7fde58156740 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde5d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fde580b4af0 0x7fde580cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde5e800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fde580cfcd0 0x7fde5815fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34872/0 (socket says 172.21.15.28:34872) 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde5e800640 1 -- 172.21.15.28:0/254784500 learned_addr learned my addr 172.21.15.28:0/254784500 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.995+0000 7fde5de00640 1 --2- 172.21.15.28:0/254784500 >> v2:172.21.15.138:3300/0 conn(0x7fde58156740 0x7fde580b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.996+0000 7fde5d400640 1 -- 172.21.15.28:0/254784500 >> v2:172.21.15.138:3300/0 conn(0x7fde58156740 msgr2=0x7fde580b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.996+0000 7fde5d400640 1 --2- 172.21.15.28:0/254784500 >> v2:172.21.15.138:3300/0 conn(0x7fde58156740 0x7fde580b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.996+0000 7fde5d400640 1 -- 172.21.15.28:0/254784500 >> v1:172.21.15.28:6789/0 conn(0x7fde580cfcd0 legacy=0x7fde5815fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.996+0000 7fde5d400640 1 -- 172.21.15.28:0/254784500 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde580b33c0 con 0x7fde580b4af0 2024-09-10T10:45:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde5d400640 1 --2- 172.21.15.28:0/254784500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde580cf790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fde580b4150 tx=0x7fde4802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651c593f3bdeb92b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde5ca00640 1 -- 172.21.15.28:0/254784500 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48037070 con 0x7fde580b4af0 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde5ca00640 1 -- 172.21.15.28:0/254784500 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde48032440 con 0x7fde580b4af0 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde5ca00640 1 -- 172.21.15.28:0/254784500 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48032810 con 0x7fde580b4af0 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 -- 172.21.15.28:0/254784500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 msgr2=0x7fde580cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 --2- 172.21.15.28:0/254784500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde580cf790 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fde580b4150 tx=0x7fde4802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 -- 172.21.15.28:0/254784500 shutdown_connections 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 --2- 172.21.15.28:0/254784500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde580cf790 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 --2- 172.21.15.28:0/254784500 >> v2:172.21.15.138:3300/0 conn(0x7fde58156740 0x7fde580b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.997+0000 7fde60800640 1 -- 172.21.15.28:0/254784500 >> 172.21.15.28:0/254784500 conn(0x7fde5806bdf0 msgr2=0x7fde5806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.998+0000 7fde60800640 1 -- 172.21.15.28:0/254784500 shutdown_connections 2024-09-10T10:45:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.998+0000 7fde60800640 1 -- 172.21.15.28:0/254784500 wait complete. 2024-09-10T10:45:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:19.999+0000 7fde60800640 1 Processor -- start 2024-09-10T10:45:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.000+0000 7fde60800640 1 -- start start 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.000+0000 7fde60800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde60800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde580cfcd0 0x7fde58154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde60800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde58156740 0x7fde580b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde60800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fde580b3c60 con 0x7fde580b4af0 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde60800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde580b3980 con 0x7fde58156740 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde60800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fde580d0890 con 0x7fde580cfcd0 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55402/0 (socket says 172.21.15.28:55402) 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 -- 172.21.15.28:0/1506652634 learned_addr learned my addr 172.21.15.28:0/1506652634 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5d400640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde580cfcd0 0x7fde58154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5e800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde58156740 0x7fde580b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 -- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde580cfcd0 msgr2=0x7fde58154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde580cfcd0 0x7fde58154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 -- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde58156740 msgr2=0x7fde580b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde58156740 0x7fde580b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.001+0000 7fde5de00640 1 -- 172.21.15.28:0/1506652634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde580af860 con 0x7fde580b4af0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde5de00640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d5f0 tx=0x7fde4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde52000640 1 -- 172.21.15.28:0/1506652634 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0125f0 con 0x7fde580b4af0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde52000640 1 -- 172.21.15.28:0/1506652634 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c019070 con 0x7fde580b4af0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde52000640 1 -- 172.21.15.28:0/1506652634 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0129d0 con 0x7fde580b4af0 2024-09-10T10:45:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde580afb40 con 0x7fde580b4af0 2024-09-10T10:45:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.002+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde58154c80 con 0x7fde580b4af0 2024-09-10T10:45:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.004+0000 7fde52000640 1 -- 172.21.15.28:0/1506652634 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde4c012b70 con 0x7fde580b4af0 2024-09-10T10:45:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.005+0000 7fde52000640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 0x7fde3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.005+0000 7fde52000640 1 -- 172.21.15.28:0/1506652634 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fde4c08b390 con 0x7fde580b4af0 2024-09-10T10:45:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.005+0000 7fde636bb740 1 Processor -- start 2024-09-10T10:45:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.005+0000 7fde5d400640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 0x7fde3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.005+0000 7fde636bb740 1 -- start start 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde5d400640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 0x7fde3c06e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde4802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f2ba0 0x562ad19fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 0x562ad19fa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562ad1910970 con 0x562ad19d6c40 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ad18b88d0 con 0x562ad19f1460 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.006+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562ad190c930 con 0x562ad19f2ba0 2024-09-10T10:45:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55418/0 (socket says 172.21.15.28:55418) 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 -- 172.21.15.28:0/1597100419 learned_addr learned my addr 172.21.15.28:0/1597100419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5d400640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f2ba0 0x562ad19fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5e800640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 0x562ad19fa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 -- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f2ba0 msgr2=0x562ad19fa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f2ba0 0x562ad19fa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 -- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 msgr2=0x562ad19fa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 0x562ad19fa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 -- 172.21.15.28:0/1597100419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad16efa20 con 0x562ad19d6c40 2024-09-10T10:45:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5e800640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 0x562ad19fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde5de00640 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde4c061b10 tx=0x7fde4c059c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2072ed9e31080f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde46600640 1 -- 172.21.15.28:0/1597100419 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c062040 con 0x562ad19d6c40 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.007+0000 7fde46600640 1 -- 172.21.15.28:0/1597100419 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c057760 con 0x562ad19d6c40 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde46600640 1 -- 172.21.15.28:0/1597100419 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c055ad0 con 0x562ad19d6c40 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 -- 172.21.15.28:0/1597100419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 msgr2=0x562ad19d7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde4c061b10 tx=0x7fde4c059c20 comp rx=0 tx=0).stop 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 -- 172.21.15.28:0/1597100419 shutdown_connections 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f1460 0x562ad19fa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f2ba0 0x562ad19fa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 --2- 172.21.15.28:0/1597100419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19d6c40 0x562ad19d7020 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 -- 172.21.15.28:0/1597100419 >> 172.21.15.28:0/1597100419 conn(0x562ad19a9500 msgr2=0x562ad19f1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 -- 172.21.15.28:0/1597100419 shutdown_connections 2024-09-10T10:45:20.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.008+0000 7fde636bb740 1 -- 172.21.15.28:0/1597100419 wait complete. 2024-09-10T10:45:20.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.009+0000 7fde636bb740 1 -- start start 2024-09-10T10:45:20.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.009+0000 7fde636bb740 1 Processor -- start 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19d6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562ad19acbf0 con 0x562ad19f1460 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562ad18cbaf0 con 0x562ad19f2ba0 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562ad19aced0 con 0x562ad19f62e0 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde636bb740 10 client.0 _ll_register_callbacks cb 0x562ad18bd5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55428/0 (socket says 172.21.15.28:55428) 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.010+0000 7fde5de00640 1 -- 172.21.15.28:0/354564430 learned_addr learned my addr 172.21.15.28:0/354564430 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5e800640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5d400640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19d6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 -- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 msgr2=0x562ad19f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 -- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 msgr2=0x562ad19d6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19d6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 -- 172.21.15.28:0/354564430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ad15e3310 con 0x562ad19f1460 2024-09-10T10:45:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5d400640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde5de00640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde4c089730 tx=0x7fde4c020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debd55c4c5a20ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c062040 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fde4c08b0c0 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c057760 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fde4c055ad0 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde636bb740 10 client.5193 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.011+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x562ad17501b0 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde4c057760 con 0x562ad19f1460 2024-09-10T10:45:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fde4c013070 con 0x562ad19f1460 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 10 client.5193 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 10 client.5193 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 20 client.5193 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad19f6a90 0x562ad19f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 4 client.5193 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ad18e5100 con 0x562ad19f6a90 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.012+0000 7fde5d400640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad19f6a90 0x562ad19f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.013+0000 7fde5d400640 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad19f6a90 0x562ad19f6ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9aa07bc8d1f73ce server_cookie=6d1b58800832cd29 in_seq=0 out_seq=0 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.013+0000 7fde3aa00640 10 client.5193 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.013+0000 7fde3aa00640 1 -- 172.21.15.28:0/354564430 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ad18e5100 con 0x562ad19f6a90 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.013+0000 7fde3aa00640 10 client.5193 handle_command_reply: tid=0 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.013+0000 7fde636bb740 1 client.5193 shutdown 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 msgr2=0x562ad19d35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fde4c089730 tx=0x7fde4c020070 comp rx=0 tx=0).stop 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 shutdown_connections 2024-09-10T10:45:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad19f6a90 0x562ad19f6ea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19d6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde3aa00640 0 client.5193 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 --2- 172.21.15.28:0/354564430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad19d35f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 >> 172.21.15.28:0/354564430 conn(0x562ad19d6c40 msgr2=0x562ad19f4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 shutdown_connections 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 1 -- 172.21.15.28:0/354564430 wait complete. 2024-09-10T10:45:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.014+0000 7fde636bb740 20 client.5193 trim_cache size 0 max 16384 2024-09-10T10:45:20.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.032+0000 7fde636bb740 1 Processor -- start 2024-09-10T10:45:20.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.032+0000 7fde636bb740 1 -- start start 2024-09-10T10:45:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f1460 0x562ad19ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fde58205560 con 0x562ad19f2ba0 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde582045e0 con 0x562ad19f62e0 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fde58072a60 con 0x562ad19f1460 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde5d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde5d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55432/0 (socket says 172.21.15.28:55432) 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.033+0000 7fde5d400640 1 -- 172.21.15.28:0/852506194 learned_addr learned my addr 172.21.15.28:0/852506194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 -- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f1460 msgr2=0x562ad19ec6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f1460 0x562ad19ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 -- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f62e0 msgr2=0x562ad19f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 -- 172.21.15.28:0/852506194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ad15bc3d0 con 0x562ad19f2ba0 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde5d400640 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde48035040 tx=0x7fde4802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f2da446be4dacf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde45c00640 1 -- 172.21.15.28:0/852506194 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48037040 con 0x562ad19f2ba0 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde45c00640 1 -- 172.21.15.28:0/852506194 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde480328a0 con 0x562ad19f2ba0 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.034+0000 7fde45c00640 1 -- 172.21.15.28:0/852506194 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48032a50 con 0x562ad19f2ba0 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- 172.21.15.28:0/852506194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 msgr2=0x562ad19f0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fde48035040 tx=0x7fde4802c000 comp rx=0 tx=0).stop 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- 172.21.15.28:0/852506194 shutdown_connections 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 --2- 172.21.15.28:0/852506194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f1460 0x562ad19ec6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- 172.21.15.28:0/852506194 >> 172.21.15.28:0/852506194 conn(0x562ad19d6c40 msgr2=0x562ad19c3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- 172.21.15.28:0/852506194 shutdown_connections 2024-09-10T10:45:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- 172.21.15.28:0/852506194 wait complete. 2024-09-10T10:45:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.035+0000 7fde636bb740 1 -- start start 2024-09-10T10:45:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.036+0000 7fde636bb740 1 Processor -- start 2024-09-10T10:45:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.036+0000 7fde636bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.036+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.036+0000 7fde636bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde636bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562ad19f1a10 con 0x562ad19f1460 2024-09-10T10:45:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fde58203f90 con 0x562ad19f2ba0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde636bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562ad19d4550 con 0x562ad19f62e0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde636bb740 10 client.0 _ll_register_callbacks cb 0x7fde4c0595a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55442/0 (socket says 172.21.15.28:55442) 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 -- 172.21.15.28:0/1888541925 learned_addr learned my addr 172.21.15.28:0/1888541925 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5d400640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5e800640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 -- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 msgr2=0x562ad19f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 -- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 msgr2=0x562ad19f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 -- 172.21.15.28:0/1888541925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ad19d3580 con 0x562ad19f1460 2024-09-10T10:45:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5d400640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5e800640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde5de00640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde4c061b10 tx=0x7fde4c0895b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c6fe28fd97e1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.037+0000 7fde3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c062040 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fde4c0711d0 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde480328a0 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fde4c0713f0 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 10 client.5199 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x562ad19d1b50 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde480328a0 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fde4c064070 con 0x562ad19f1460 2024-09-10T10:45:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 10 client.5199 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 10 client.5199 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 20 client.5199 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad1a6f680 0x562ad19f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 4 client.5199 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}] 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.038+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair"]}) -- 0x562ad191e0a0 con 0x562ad1a6f680 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.039+0000 7fde5d400640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad1a6f680 0x562ad19f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.039+0000 7fde5d400640 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad1a6f680 0x562ad19f0e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52c97e410a1511e server_cookie=a3604769fe62976a in_seq=0 out_seq=0 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.039+0000 7fde3aa00640 10 client.5199 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde3aa00640 1 -- 172.21.15.28:0/1888541925 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ad191e0a0 con 0x562ad1a6f680 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde3aa00640 10 client.5199 handle_command_reply: tid=0 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 client.5199 shutdown 2024-09-10T10:45:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 msgr2=0x562ad1a6f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fde4c061b10 tx=0x7fde4c0895b0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 shutdown_connections 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x562ad1a6f680 0x562ad19f0e80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562ad19f62e0 0x562ad19f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562ad19f2ba0 0x562ad19f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 --2- 172.21.15.28:0/1888541925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562ad19f1460 0x562ad1a6f140 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 >> 172.21.15.28:0/1888541925 conn(0x562ad19d6c40 msgr2=0x562ad19ebaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde3aa00640 0 client.5199 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 shutdown_connections 2024-09-10T10:45:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 1 -- 172.21.15.28:0/1888541925 wait complete. 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.040+0000 7fde636bb740 20 client.5199 trim_cache size 0 max 16384 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "2e966dde-1c6e-4e65-9473-4fc807ab8b7a", 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2024-09-10T10:45:20.047 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:45:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 msgr2=0x7fde3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 0x7fde3c06e320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fde48034000 tx=0x7fde4802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 msgr2=0x7fde581f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00d5f0 tx=0x7fde4c00dab0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 shutdown_connections 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde3c06be60 0x7fde3c06e320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde58156740 0x7fde580b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fde580cfcd0 0x7fde58154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 --2- 172.21.15.28:0/1506652634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde580b4af0 0x7fde581f7fd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 >> 172.21.15.28:0/1506652634 conn(0x7fde5806bdf0 msgr2=0x7fde5815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 shutdown_connections 2024-09-10T10:45:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.041+0000 7fde60800640 1 -- 172.21.15.28:0/1506652634 wait complete. 2024-09-10T10:45:20.059 INFO:tasks.cephfs.test_damage:{ "return_code": 0, "scrub_tag": "2e966dde-1c6e-4e65-9473-4fc807ab8b7a", "mode": "asynchronous" } 2024-09-10T10:45:20.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:20.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T10:45:20.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.214+0000 7f7c70e00640 1 Processor -- start 2024-09-10T10:45:20.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.214+0000 7f7c70e00640 1 -- start start 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c70e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7c6c0b59e0 0x7f7c6c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c70e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c6c0b5df0 con 0x7f7c6c06f910 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c70e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7c6c0b4ce0 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c70e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c0b3c40 con 0x7f7c6c15f500 2024-09-10T10:45:20.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50788/0 (socket says 172.21.15.28:50788) 2024-09-10T10:45:20.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c63e00640 1 -- 172.21.15.28:0/33221537 learned_addr learned my addr 172.21.15.28:0/33221537 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.216+0000 7f7c6a800640 1 --2- 172.21.15.28:0/33221537 >> v2:172.21.15.138:3300/0 conn(0x7f7c6c0b59e0 0x7f7c6c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.217+0000 7f7c63e00640 1 -- 172.21.15.28:0/33221537 >> v2:172.21.15.138:3300/0 conn(0x7f7c6c0b59e0 msgr2=0x7f7c6c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.217+0000 7f7c63e00640 1 --2- 172.21.15.28:0/33221537 >> v2:172.21.15.138:3300/0 conn(0x7f7c6c0b59e0 0x7f7c6c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.217+0000 7f7c63e00640 1 -- 172.21.15.28:0/33221537 >> v1:172.21.15.138:6789/0 conn(0x7f7c6c06f910 legacy=0x7f7c6c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.217+0000 7f7c63e00640 1 -- 172.21.15.28:0/33221537 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c0b3f20 con 0x7f7c6c15f500 2024-09-10T10:45:20.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.217+0000 7f7c69400640 1 -- 172.21.15.28:0/33221537 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480594089 0 0) 0x7f7c5c001110 con 0x7f7c6c06f910 2024-09-10T10:45:20.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.218+0000 7f7c63e00640 1 --2- 172.21.15.28:0/33221537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5402a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b5166ae8d1a028 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.218+0000 7f7c69400640 1 -- 172.21.15.28:0/33221537 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037070 con 0x7f7c6c15f500 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.218+0000 7f7c69400640 1 -- 172.21.15.28:0/33221537 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54032440 con 0x7f7c6c15f500 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.218+0000 7f7c69400640 1 -- 172.21.15.28:0/33221537 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54032810 con 0x7f7c6c15f500 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 -- 172.21.15.28:0/33221537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 msgr2=0x7f7c6c0b54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 --2- 172.21.15.28:0/33221537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5402a130 comp rx=0 tx=0).stop 2024-09-10T10:45:20.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 -- 172.21.15.28:0/33221537 shutdown_connections 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 --2- 172.21.15.28:0/33221537 >> v2:172.21.15.138:3300/0 conn(0x7f7c6c0b59e0 0x7f7c6c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 --2- 172.21.15.28:0/33221537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c15f500 0x7f7c6c0b54a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 -- 172.21.15.28:0/33221537 >> 172.21.15.28:0/33221537 conn(0x7f7c6c073530 msgr2=0x7f7c6c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 -- 172.21.15.28:0/33221537 shutdown_connections 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.219+0000 7f7c70e00640 1 -- 172.21.15.28:0/33221537 wait complete. 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.220+0000 7f7c70e00640 1 Processor -- start 2024-09-10T10:45:20.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 -- start start 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c06f910 0x7f7c6c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 0x7f7c6c0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c0b47c0 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c0b44e0 con 0x7f7c6c15f500 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.221+0000 7f7c70e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c06fcf0 con 0x7f7c6c06f910 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55454/0 (socket says 172.21.15.28:55454) 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 -- 172.21.15.28:0/3640352068 learned_addr learned my addr 172.21.15.28:0/3640352068 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c6a800640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 0x7f7c6c0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c69e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c06f910 0x7f7c6c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 -- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c06f910 msgr2=0x7f7c6c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c06f910 0x7f7c6c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 -- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 msgr2=0x7f7c6c0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 0x7f7c6c0b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c63e00640 1 -- 172.21.15.28:0/3640352068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c15be30 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.222+0000 7f7c6a800640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 0x7f7c6c0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c63e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c62000640 1 -- 172.21.15.28:0/3640352068 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54041020 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c62000640 1 -- 172.21.15.28:0/3640352068 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54030640 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c62000640 1 -- 172.21.15.28:0/3640352068 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54037040 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c15c110 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.223+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c15c6d0 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.226+0000 7f7c62000640 1 -- 172.21.15.28:0/3640352068 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c54044020 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.227+0000 7f7c62000640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 0x7f7c4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.227+0000 7f7c62000640 1 -- 172.21.15.28:0/3640352068 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f7c540ab830 con 0x7f7c6c0b59e0 2024-09-10T10:45:20.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.227+0000 7f7c69e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 0x7f7c4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.227+0000 7f7c73e2b740 1 Processor -- start 2024-09-10T10:45:20.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.227+0000 7f7c73e2b740 1 -- start start 2024-09-10T10:45:20.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c69e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 0x7f7c4806e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346762c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346718480 0x55b346763740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b346680660 con 0x55b346732800 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b346628680 con 0x55b346718480 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b34667c620 con 0x55b346732420 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55458/0 (socket says 172.21.15.28:55458) 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c63e00640 1 -- 172.21.15.28:0/2412533027 learned_addr learned my addr 172.21.15.28:0/2412533027 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c6a800640 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346718480 0x55b346763740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.228+0000 7f7c69e00640 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346762c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 -- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 msgr2=0x55b346762c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346762c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 -- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346718480 msgr2=0x55b346763740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346718480 0x55b346763740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 -- 172.21.15.28:0/2412533027 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34645d7d0 con 0x55b346732800 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c63e00640 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c54079570 tx=0x7f7c540791d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67f69b133edce52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c5aa00640 1 -- 172.21.15.28:0/2412533027 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5407dc20 con 0x55b346732800 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c5aa00640 1 -- 172.21.15.28:0/2412533027 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54082450 con 0x55b346732800 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.229+0000 7f7c5aa00640 1 -- 172.21.15.28:0/2412533027 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54077b70 con 0x55b346732800 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 -- 172.21.15.28:0/2412533027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 msgr2=0x55b346763200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c54079570 tx=0x7f7c540791d0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 -- 172.21.15.28:0/2412533027 shutdown_connections 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346718480 0x55b346763740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/2412533027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346762c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 -- 172.21.15.28:0/2412533027 >> 172.21.15.28:0/2412533027 conn(0x55b346719220 msgr2=0x55b346761840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 -- 172.21.15.28:0/2412533027 shutdown_connections 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.230+0000 7f7c73e2b740 1 -- 172.21.15.28:0/2412533027 wait complete. 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.231+0000 7f7c73e2b740 1 -- start start 2024-09-10T10:45:20.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.231+0000 7f7c73e2b740 1 Processor -- start 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732420 0x55b34675b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346742c40 0x55b3467430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b346680660 con 0x55b346732800 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b3466094c0 con 0x55b346732420 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b3467173e0 con 0x55b346742c40 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c73e2b740 10 client.0 _ll_register_callbacks cb 0x55b346678e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55474/0 (socket says 172.21.15.28:55474) 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 -- 172.21.15.28:0/1275646711 learned_addr learned my addr 172.21.15.28:0/1275646711 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732420 0x55b34675b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346742c40 0x55b3467430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 -- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346742c40 msgr2=0x55b3467430c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346742c40 0x55b3467430c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 -- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732420 msgr2=0x55b34675b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732420 0x55b34675b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.232+0000 7f7c63e00640 1 -- 172.21.15.28:0/1275646711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3463933f0 con 0x55b346732800 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c54035040 tx=0x7f7c54009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ce5b45c85eff6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54097070 con 0x55b346732800 2024-09-10T10:45:20.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f7c54097210 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c54082450 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7c5407dc20 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c73e2b740 10 client.5208 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b3463530f0 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c54082450 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f7c5403a030 con 0x55b346732800 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.233+0000 7f7c73e2b740 10 client.5208 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c73e2b740 10 client.5208 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c73e2b740 20 client.5208 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b34675c820 0x55b34675cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c73e2b740 4 client.5208 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b34668dd90 con 0x55b34675c820 2024-09-10T10:45:20.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b34675c820 0x55b34675cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b34675c820 0x55b34675cc00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db99c7fcabcdb7e1 server_cookie=37af1a1432f6c3d8 in_seq=0 out_seq=0 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.234+0000 7f7c4f400640 10 client.5208 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c4f400640 1 -- 172.21.15.28:0/1275646711 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b34668dd90 con 0x55b34675c820 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c4f400640 10 client.5208 handle_command_reply: tid=0 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 client.5208 shutdown 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 msgr2=0x55b34675e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7c54035040 tx=0x7f7c54009220 comp rx=0 tx=0).stop 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 shutdown_connections 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b34675c820 0x55b34675cc00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346742c40 0x55b3467430c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b34675e810 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c4f400640 0 client.5208 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1275646711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732420 0x55b34675b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.235+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 >> 172.21.15.28:0/1275646711 conn(0x55b346718480 msgr2=0x55b346763ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.236+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 shutdown_connections 2024-09-10T10:45:20.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.236+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1275646711 wait complete. 2024-09-10T10:45:20.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.236+0000 7f7c73e2b740 20 client.5208 trim_cache size 0 max 16384 2024-09-10T10:45:20.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.254+0000 7f7c73e2b740 1 Processor -- start 2024-09-10T10:45:20.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.254+0000 7f7c73e2b740 1 -- start start 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346785950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 0x55b346764030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c201200 con 0x55b346732800 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c2002a0 con 0x55b346742c40 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7c6c15a3d0 con 0x55b346732420 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55478/0 (socket says 172.21.15.28:55478) 2024-09-10T10:45:20.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 -- 172.21.15.28:0/1108394096 learned_addr learned my addr 172.21.15.28:0/1108394096 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 -- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 msgr2=0x55b346785950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 0x55b346764030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346785950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346785950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 -- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 msgr2=0x55b346764030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 0x55b346764030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c63e00640 1 -- 172.21.15.28:0/1108394096 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34671d770 con 0x55b346732800 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 0x55b346764030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.255+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346785950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c54075cd0 tx=0x7f7c54093070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d652d9ddd9730081 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c5a000640 1 -- 172.21.15.28:0/1108394096 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54097040 con 0x55b346732800 2024-09-10T10:45:20.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c5a000640 1 -- 172.21.15.28:0/1108394096 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c540a8070 con 0x55b346732800 2024-09-10T10:45:20.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c5a000640 1 -- 172.21.15.28:0/1108394096 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54082450 con 0x55b346732800 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1108394096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 msgr2=0x55b346763af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7c54075cd0 tx=0x7f7c54093070 comp rx=0 tx=0).stop 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1108394096 shutdown_connections 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346742c40 0x55b346764030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346732800 0x55b346763af0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1108394096 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b346785950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.256+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1108394096 >> 172.21.15.28:0/1108394096 conn(0x55b346718480 msgr2=0x55b34675b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.257+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1108394096 shutdown_connections 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.257+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1108394096 wait complete. 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.257+0000 7f7c73e2b740 1 -- start start 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.257+0000 7f7c73e2b740 1 Processor -- start 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b3467e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732800 0x55b346732be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b346771490 con 0x55b346742c40 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c1ffc50 con 0x55b346732800 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b34671d310 con 0x55b346732420 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.258+0000 7f7c73e2b740 10 client.0 _ll_register_callbacks cb 0x7f7c64009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55482/0 (socket says 172.21.15.28:55482) 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 -- 172.21.15.28:0/1488362422 learned_addr learned my addr 172.21.15.28:0/1488362422 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:20.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b3467e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c63e00640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732800 0x55b346732be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 -- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 msgr2=0x55b3467e70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b3467e70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 -- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732800 msgr2=0x55b346732be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732800 0x55b346732be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 -- 172.21.15.28:0/1488362422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b346765370 con 0x55b346742c40 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c6a800640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c64018e40 tx=0x7f7c64016710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27395ad5b0422f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.259+0000 7f7c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64016e60 con 0x55b346742c40 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f7c64025d10 con 0x55b346742c40 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c540a8070 con 0x55b346742c40 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7c64019640 con 0x55b346742c40 2024-09-10T10:45:20.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 10 client.5214 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:20.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55b346774720 con 0x55b346742c40 2024-09-10T10:45:20.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c540a8070 con 0x55b346742c40 2024-09-10T10:45:20.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f7c640197e0 con 0x55b346742c40 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 10 client.5214 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 10 client.5214 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 20 client.5214 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b346773a50 0x55b346773e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 4 client.5214 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.260+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3466880e0 con 0x55b346773a50 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b346773a50 0x55b346773e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c69e00640 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b346773a50 0x55b346773e60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757ac89f4a295d5b server_cookie=3865495121cca335 in_seq=0 out_seq=0 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c4f400640 10 client.5214 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c4f400640 1 -- 172.21.15.28:0/1488362422 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x55b3466880e0 con 0x55b346773a50 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c4f400640 10 client.5214 handle_command_reply: tid=0 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.261+0000 7f7c73e2b740 1 client.5214 shutdown 2024-09-10T10:45:20.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 msgr2=0x55b346733120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7c64018e40 tx=0x7f7c64016710 comp rx=0 tx=0).stop 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 shutdown_connections 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55b346773a50 0x55b346773e60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b346742c40 0x55b346733120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b346732800 0x55b346732be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 --2- 172.21.15.28:0/1488362422 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b346732420 0x55b3467e70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c4f400640 0 client.5214 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 >> 172.21.15.28:0/1488362422 conn(0x55b346718480 msgr2=0x55b346760c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 shutdown_connections 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 1 -- 172.21.15.28:0/1488362422 wait complete. 2024-09-10T10:45:20.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.262+0000 7f7c73e2b740 20 client.5214 trim_cache size 0 max 16384 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "2e966dde-1c6e-4e65-9473-4fc807ab8b7a": { 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/subdir", 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "2e966dde-1c6e-4e65-9473-4fc807ab8b7a", 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "repair" 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:45:20.269 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 msgr2=0x7f7c4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 0x7f7c4806e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c009040 comp rx=0 tx=0).stop 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 msgr2=0x7f7c6c15d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c54000c00 tx=0x7f7c540046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 shutdown_connections 2024-09-10T10:45:20.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c4806be30 0x7f7c4806e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c6c15f500 0x7f7c6c0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c6c0b59e0 0x7f7c6c15d8c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 --2- 172.21.15.28:0/3640352068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7c6c06f910 0x7f7c6c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 >> 172.21.15.28:0/3640352068 conn(0x7f7c6c073530 msgr2=0x7f7c6c0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 shutdown_connections 2024-09-10T10:45:20.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:20.263+0000 7f7c70e00640 1 -- 172.21.15.28:0/3640352068 wait complete. 2024-09-10T10:45:20.280 INFO:tasks.cephfs.filesystem:scrub status for tag:2e966dde-1c6e-4e65-9473-4fc807ab8b7a - {'path': '/subdir', 'tag': '2e966dde-1c6e-4e65-9473-4fc807ab8b7a', 'options': 'repair'} 2024-09-10T10:45:38.816 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:45:38.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:45:50.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-10T10:45:50.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.448+0000 7f026da00640 1 Processor -- start 2024-09-10T10:45:50.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.449+0000 7f026da00640 1 -- start start 2024-09-10T10:45:50.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f026da00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0268155930 0x7f02680addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f026da00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026815c1e0 con 0x7f02680d0a80 2024-09-10T10:45:50.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f026da00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026815c4c0 con 0x7f026815b720 2024-09-10T10:45:50.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f026da00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02680b3340 con 0x7f0268155930 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f0267400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f02680d0a80 0x7f02680b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:55020/0 (socket says 172.21.15.28:55020) 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f0267400640 1 -- 172.21.15.28:0/3620670352 learned_addr learned my addr 172.21.15.28:0/3620670352 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f0266a00640 1 --2- 172.21.15.28:0/3620670352 >> v2:172.21.15.138:3300/0 conn(0x7f0268155930 0x7f02680addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.450+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95643512 0 0) 0x7f026815c1e0 con 0x7f02680d0a80 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f024c003520 con 0x7f02680d0a80 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523426861 0 0) 0x7f026815c4c0 con 0x7f026815b720 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f026815c1e0 con 0x7f026815b720 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2584460540 0 0) 0x7f024c003520 con 0x7f02680d0a80 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266a00640 1 -- 172.21.15.28:0/3620670352 >> v1:172.21.15.138:6789/0 conn(0x7f026815b720 legacy=0x7f02680d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266a00640 1 -- 172.21.15.28:0/3620670352 >> v1:172.21.15.28:6789/0 conn(0x7f02680d0a80 legacy=0x7f02680b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.451+0000 7f0266a00640 1 -- 172.21.15.28:0/3620670352 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02680b3620 con 0x7f0268155930 2024-09-10T10:45:50.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.452+0000 7f0266a00640 1 --2- 172.21.15.28:0/3620670352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0268155930 0x7f02680addd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02580097a0 tx=0x7f025802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa0ed2a0be1691b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.452+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258037070 con 0x7f0268155930 2024-09-10T10:45:50.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.452+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025802ac50 con 0x7f0268155930 2024-09-10T10:45:50.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.452+0000 7f0266000640 1 -- 172.21.15.28:0/3620670352 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600019f0 con 0x7f0268155930 2024-09-10T10:45:50.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 -- 172.21.15.28:0/3620670352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0268155930 msgr2=0x7f02680addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 --2- 172.21.15.28:0/3620670352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0268155930 0x7f02680addd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f02580097a0 tx=0x7f025802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 -- 172.21.15.28:0/3620670352 shutdown_connections 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 --2- 172.21.15.28:0/3620670352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0268155930 0x7f02680addd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 -- 172.21.15.28:0/3620670352 >> 172.21.15.28:0/3620670352 conn(0x7f026806bdf0 msgr2=0x7f026806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 -- 172.21.15.28:0/3620670352 shutdown_connections 2024-09-10T10:45:50.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.453+0000 7f026da00640 1 -- 172.21.15.28:0/3620670352 wait complete. 2024-09-10T10:45:50.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.454+0000 7f026da00640 1 Processor -- start 2024-09-10T10:45:50.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.455+0000 7f026da00640 1 -- start start 2024-09-10T10:45:50.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.455+0000 7f026da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02680d0a80 0x7f02680ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.455+0000 7f026da00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0268155930 0x7f0268149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.455+0000 7f026da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f026da00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02680ae490 con 0x7f026815b720 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f026da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02680ae310 con 0x7f02680d0a80 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f026da00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f026815bbb0 con 0x7f0268155930 2024-09-10T10:45:50.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58626/0 (socket says 172.21.15.28:58626) 2024-09-10T10:45:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 -- 172.21.15.28:0/1852297163 learned_addr learned my addr 172.21.15.28:0/1852297163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0266a00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02680d0a80 0x7f02680ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f025fe00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0268155930 0x7f0268149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 -- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0268155930 msgr2=0x7f0268149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0268155930 0x7f0268149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 -- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02680d0a80 msgr2=0x7f02680ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02680d0a80 0x7f02680ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.456+0000 7f0267400640 1 -- 172.21.15.28:0/1852297163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02680cec70 con 0x7f026815b720 2024-09-10T10:45:50.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f0267400640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f026000ccd0 tx=0x7f026000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f025ea00640 1 -- 172.21.15.28:0/1852297163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260017070 con 0x7f026815b720 2024-09-10T10:45:50.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f025ea00640 1 -- 172.21.15.28:0/1852297163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0260012400 con 0x7f026815b720 2024-09-10T10:45:50.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f025ea00640 1 -- 172.21.15.28:0/1852297163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600127f0 con 0x7f026815b720 2024-09-10T10:45:50.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02680cd2a0 con 0x7f026815b720 2024-09-10T10:45:50.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.457+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02680cd860 con 0x7f026815b720 2024-09-10T10:45:50.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.459+0000 7f025ea00640 1 -- 172.21.15.28:0/1852297163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f026001a070 con 0x7f026815b720 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.460+0000 7f025ea00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 0x7f024406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.461+0000 7f025ea00640 1 -- 172.21.15.28:0/1852297163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f026008afe0 con 0x7f026815b720 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.461+0000 7f0266a00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 0x7f024406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.461+0000 7f0270873740 1 Processor -- start 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.461+0000 7f0270873740 1 -- start start 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0266a00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 0x7f024406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0258034000 tx=0x7f025802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f2bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f2c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c501e49660 con 0x55c501ee1480 2024-09-10T10:45:50.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c501df1680 con 0x55c501efb800 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.462+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c501e45620 con 0x55c501efb420 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58642/0 (socket says 172.21.15.28:58642) 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 -- 172.21.15.28:0/4255697061 learned_addr learned my addr 172.21.15.28:0/4255697061 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f025fe00640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f2c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0266a00640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f2bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 -- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 msgr2=0x55c501f2bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f2bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 -- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 msgr2=0x55c501f2c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f2c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 -- 172.21.15.28:0/4255697061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c501c267d0 con 0x55c501ee1480 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f025fe00640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f2c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0267400640 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0260058dc0 tx=0x7f0260059900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cc625d596c0794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0257400640 1 -- 172.21.15.28:0/4255697061 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600573c0 con 0x55c501ee1480 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0257400640 1 -- 172.21.15.28:0/4255697061 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026005d170 con 0x55c501ee1480 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.463+0000 7f0257400640 1 -- 172.21.15.28:0/4255697061 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260061810 con 0x55c501ee1480 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 -- 172.21.15.28:0/4255697061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 msgr2=0x55c501f2c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0260058dc0 tx=0x7f0260059900 comp rx=0 tx=0).stop 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 -- 172.21.15.28:0/4255697061 shutdown_connections 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501ee1480 0x55c501f2c740 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f2c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 --2- 172.21.15.28:0/4255697061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f2bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 -- 172.21.15.28:0/4255697061 >> 172.21.15.28:0/4255697061 conn(0x55c501ee2220 msgr2=0x55c501f2a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 -- 172.21.15.28:0/4255697061 shutdown_connections 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.464+0000 7f0270873740 1 -- 172.21.15.28:0/4255697061 wait complete. 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.465+0000 7f0270873740 1 -- start start 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.465+0000 7f0270873740 1 Processor -- start 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501f23a00 0x55c501f23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c501ee03e0 con 0x55c501efb420 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c501dd24c0 con 0x55c501efb800 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c501ee06c0 con 0x55c501f23a00 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0270873740 10 client.0 _ll_register_callbacks cb 0x55c501e41e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0266a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0266a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58652/0 (socket says 172.21.15.28:58652) 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.466+0000 7f0266a00640 1 -- 172.21.15.28:0/407849881 learned_addr learned my addr 172.21.15.28:0/407849881 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0267400640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501f23a00 0x55c501f23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f025fe00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 -- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501f23a00 msgr2=0x55c501f23e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501f23a00 0x55c501f23e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 -- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 msgr2=0x55c501f220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 -- 172.21.15.28:0/407849881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c501b5c3f0 con 0x55c501efb420 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0266a00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0258035040 tx=0x7f0258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941a13fba7c5dca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f024be00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258037040 con 0x55c501efb420 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f0258032650 con 0x55c501efb420 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.467+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026005d170 con 0x55c501efb420 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0258056050 con 0x55c501efb420 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 10 client.5226 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:50.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55c501b1c0f0 con 0x55c501efb420 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026005d170 con 0x55c501efb420 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f0258059020 con 0x55c501efb420 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 10 client.5226 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 10 client.5226 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 20 client.5226 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3bc80 0x55c501f3e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 4 client.5226 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c501e56d90 con 0x55c501f3bc80 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.468+0000 7f025fe00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3bc80 0x55c501f3e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.469+0000 7f025fe00640 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3bc80 0x55c501f3e570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a37f03af41b67aa server_cookie=1fb4f8e245ac3b26 in_seq=0 out_seq=0 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.469+0000 7f024be00640 10 client.5226 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.469+0000 7f024be00640 1 -- 172.21.15.28:0/407849881 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c501e56d90 con 0x55c501f3bc80 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f024be00640 10 client.5226 handle_command_reply: tid=0 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 client.5226 shutdown 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 msgr2=0x55c501f3d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0258035040 tx=0x7f0258004730 comp rx=0 tx=0).stop 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 shutdown_connections 2024-09-10T10:45:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3bc80 0x55c501f3e570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501f23a00 0x55c501f23e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb800 0x55c501f220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 --2- 172.21.15.28:0/407849881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb420 0x55c501f3d970 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f024be00640 0 client.5226 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 >> 172.21.15.28:0/407849881 conn(0x55c501ee1480 msgr2=0x55c501f28ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 shutdown_connections 2024-09-10T10:45:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 1 -- 172.21.15.28:0/407849881 wait complete. 2024-09-10T10:45:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.470+0000 7f0270873740 20 client.5226 trim_cache size 0 max 16384 2024-09-10T10:45:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.488+0000 7f0270873740 1 Processor -- start 2024-09-10T10:45:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.488+0000 7f0270873740 1 -- start start 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb420 0x55c501f3d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb800 0x55c501f3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02682011b0 con 0x55c501f23a00 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02682001b0 con 0x55c501efb420 2024-09-10T10:45:50.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f026814e990 con 0x55c501efb800 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0267400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58658/0 (socket says 172.21.15.28:58658) 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.489+0000 7f0267400640 1 -- 172.21.15.28:0/2066275170 learned_addr learned my addr 172.21.15.28:0/2066275170 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 -- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb800 msgr2=0x55c501f3d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb800 0x55c501f3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 -- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb420 msgr2=0x55c501f3d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb420 0x55c501f3d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 -- 172.21.15.28:0/2066275170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c501f27140 con 0x55c501f23a00 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0267400640 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02600012e0 tx=0x7f026001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1694c1a0a640046 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0256a00640 1 -- 172.21.15.28:0/2066275170 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260055360 con 0x55c501f23a00 2024-09-10T10:45:50.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0256a00640 1 -- 172.21.15.28:0/2066275170 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026005d170 con 0x55c501f23a00 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.490+0000 7f0256a00640 1 -- 172.21.15.28:0/2066275170 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260061810 con 0x55c501f23a00 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- 172.21.15.28:0/2066275170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 msgr2=0x55c501f20750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02600012e0 tx=0x7f026001e000 comp rx=0 tx=0).stop 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- 172.21.15.28:0/2066275170 shutdown_connections 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501f23a00 0x55c501f20750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb800 0x55c501f3d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 --2- 172.21.15.28:0/2066275170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501efb420 0x55c501f3d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- 172.21.15.28:0/2066275170 >> 172.21.15.28:0/2066275170 conn(0x55c501ee1480 msgr2=0x55c501f109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- 172.21.15.28:0/2066275170 shutdown_connections 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- 172.21.15.28:0/2066275170 wait complete. 2024-09-10T10:45:50.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.491+0000 7f0270873740 1 -- start start 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.492+0000 7f0270873740 1 Processor -- start 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501f23a00 0x55c501f3d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02681ffb60 con 0x55c501efb800 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c501f102e0 con 0x55c501f23a00 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c501ef9ca0 con 0x55c501efb420 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0270873740 10 client.0 _ll_register_callbacks cb 0x7f0258035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58662/0 (socket says 172.21.15.28:58662) 2024-09-10T10:45:50.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 -- 172.21.15.28:0/3426350952 learned_addr learned my addr 172.21.15.28:0/3426350952 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0266a00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f0267400640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501f23a00 0x55c501f3d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 -- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 msgr2=0x55c501f448c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f448c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 -- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501f23a00 msgr2=0x55c501f3d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501f23a00 0x55c501f3d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.493+0000 7f025fe00640 1 -- 172.21.15.28:0/3426350952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c501f8f300 con 0x55c501efb800 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f025fe00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f02680b1a60 tx=0x7f0250004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dee4d8a1891b07c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02500189c0 con 0x55c501efb800 2024-09-10T10:45:50.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026005d170 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f0250018b60 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0250018d60 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 10 client.5232 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x55c501ef98d0 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f026005d170 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7f025002f080 con 0x55c501efb800 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 10 client.5232 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 10 client.5232 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 20 client.5232 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3d690 0x55c501f217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 4 client.5232 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.494+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c501e510e0 con 0x55c501f3d690 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f0266a00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3d690 0x55c501f217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f0266a00640 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3d690 0x55c501f217b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3955a6f3688e1c server_cookie=99d9b69d6163cb62 in_seq=0 out_seq=0 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f024be00640 10 client.5232 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f024be00640 1 -- 172.21.15.28:0/3426350952 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c501e510e0 con 0x55c501f3d690 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f024be00640 10 client.5232 handle_command_reply: tid=0 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.495+0000 7f0270873740 1 client.5232 shutdown 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 msgr2=0x55c501f8edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f02680b1a60 tx=0x7f0250004d50 comp rx=0 tx=0).stop 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 shutdown_connections 2024-09-10T10:45:50.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x55c501f3d690 0x55c501f217b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c501f23a00 0x55c501f3d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c501efb800 0x55c501f8edc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 --2- 172.21.15.28:0/3426350952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c501efb420 0x55c501f448c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f024be00640 0 client.5232 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 >> 172.21.15.28:0/3426350952 conn(0x55c501ee1480 msgr2=0x55c501f20750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 shutdown_connections 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 1 -- 172.21.15.28:0/3426350952 wait complete. 2024-09-10T10:45:50.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f0270873740 20 client.5232 trim_cache size 0 max 16384 2024-09-10T10:45:50.507 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:45:50.507 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2024-09-10T10:45:50.507 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2024-09-10T10:45:50.507 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 msgr2=0x7f024406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 0x7f024406e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0258034000 tx=0x7f025802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 msgr2=0x7f02681e7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.496+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f026000ccd0 tx=0x7f026000a430 comp rx=0 tx=0).stop 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 shutdown_connections 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f024406be30 0x7f024406e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f026815b720 0x7f02681e7dc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0268155930 0x7f0268149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 --2- 172.21.15.28:0/1852297163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02680d0a80 0x7f02680ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 >> 172.21.15.28:0/1852297163 conn(0x7f026806bdf0 msgr2=0x7f02680af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 shutdown_connections 2024-09-10T10:45:50.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.497+0000 7f026da00640 1 -- 172.21.15.28:0/1852297163 wait complete. 2024-09-10T10:45:50.515 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:45:50.515 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:45:50.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.670+0000 7f125d400640 1 Processor -- start 2024-09-10T10:45:50.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.670+0000 7f125d400640 1 -- start start 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.671+0000 7f125d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1258147e90 0x7f12580720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.671+0000 7f125d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12580725f0 0x7f12580aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.671+0000 7f125d400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1258072a00 con 0x7f1258147230 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.671+0000 7f125d400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f12580b34d0 con 0x7f12580725f0 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.671+0000 7f125d400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f12580ae9e0 con 0x7f1258147e90 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12580725f0 0x7f12580aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12580725f0 0x7f12580aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58666/0 (socket says 172.21.15.28:58666) 2024-09-10T10:45:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 -- 172.21.15.28:0/1258606003 learned_addr learned my addr 172.21.15.28:0/1258606003 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1256800640 1 --1- 172.21.15.28:0/1258606003 >> v1:172.21.15.28:6789/0 conn(0x7f1258147230 0x7f125806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:55022/0 (socket says 172.21.15.28:55022) 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1255e00640 1 --2- 172.21.15.28:0/1258606003 >> v2:172.21.15.138:3300/0 conn(0x7f1258147e90 0x7f12580720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 -- 172.21.15.28:0/1258606003 >> v2:172.21.15.138:3300/0 conn(0x7f1258147e90 msgr2=0x7f12580720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 --2- 172.21.15.28:0/1258606003 >> v2:172.21.15.138:3300/0 conn(0x7f1258147e90 0x7f12580720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 -- 172.21.15.28:0/1258606003 >> v1:172.21.15.28:6789/0 conn(0x7f1258147230 legacy=0x7f125806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.672+0000 7f1257200640 1 -- 172.21.15.28:0/1258606003 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125806f480 con 0x7f12580725f0 2024-09-10T10:45:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.673+0000 7f1257200640 1 --2- 172.21.15.28:0/1258606003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12580725f0 0x7f12580aee10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f125806ccb0 tx=0x7f125002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d213076e7a148b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.673+0000 7f1255400640 1 -- 172.21.15.28:0/1258606003 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250036070 con 0x7f12580725f0 2024-09-10T10:45:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.673+0000 7f1255400640 1 -- 172.21.15.28:0/1258606003 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1250031600 con 0x7f12580725f0 2024-09-10T10:45:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.673+0000 7f1255400640 1 -- 172.21.15.28:0/1258606003 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12500319d0 con 0x7f12580725f0 2024-09-10T10:45:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 -- 172.21.15.28:0/1258606003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12580725f0 msgr2=0x7f12580aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 --2- 172.21.15.28:0/1258606003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12580725f0 0x7f12580aee10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f125806ccb0 tx=0x7f125002a730 comp rx=0 tx=0).stop 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 -- 172.21.15.28:0/1258606003 shutdown_connections 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 --2- 172.21.15.28:0/1258606003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12580725f0 0x7f12580aee10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 --2- 172.21.15.28:0/1258606003 >> v2:172.21.15.138:3300/0 conn(0x7f1258147e90 0x7f12580720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 -- 172.21.15.28:0/1258606003 >> 172.21.15.28:0/1258606003 conn(0x7f1258067d70 msgr2=0x7f125806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 -- 172.21.15.28:0/1258606003 shutdown_connections 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.674+0000 7f125d400640 1 -- 172.21.15.28:0/1258606003 wait complete. 2024-09-10T10:45:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.676+0000 7f125d400640 1 Processor -- start 2024-09-10T10:45:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.676+0000 7f125d400640 1 -- start start 2024-09-10T10:45:50.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 0x7f125813f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1258147e90 0x7f12580b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f125806eec0 con 0x7f1258147230 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f125806ebe0 con 0x7f1258147e90 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f125d400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f125806f1a0 con 0x7f12580725f0 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58678/0 (socket says 172.21.15.28:58678) 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 -- 172.21.15.28:0/3415677484 learned_addr learned my addr 172.21.15.28:0/3415677484 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1257200640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1258147e90 0x7f12580b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1256800640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 0x7f125813f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 -- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 msgr2=0x7f125813f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 0x7f125813f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 -- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1258147e90 msgr2=0x7f12580b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1258147e90 0x7f12580b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1255e00640 1 -- 172.21.15.28:0/3415677484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1258160770 con 0x7f1258147230 2024-09-10T10:45:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.677+0000 7f1256800640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 0x7f125813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f1255e00640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f124400d5f0 tx=0x7f124400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440125f0 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1244019070 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440129d0 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1258160d30 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.678+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12581612f0 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.679+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1258147610 con 0x7f1258147230 2024-09-10T10:45:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.680+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1244012b70 con 0x7f1258147230 2024-09-10T10:45:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.681+0000 7f124ea00640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 0x7f123806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.681+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f124408b560 con 0x7f1258147230 2024-09-10T10:45:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.681+0000 7f1256800640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 0x7f123806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:50.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.682+0000 7f1256800640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 0x7f123806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1248007fd0 tx=0x7f1248007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:50.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.686+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1244059bc0 con 0x7f1258147230 2024-09-10T10:45:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.908+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12580b4d30 con 0x7f1258147230 2024-09-10T10:45:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.908+0000 7f124ea00640 1 -- 172.21.15.28:0/3415677484 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f12440594c0 con 0x7f1258147230 2024-09-10T10:45:50.910 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:45:50.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 msgr2=0x7f123806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 0x7f123806e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1248007fd0 tx=0x7f1248007d60 comp rx=0 tx=0).stop 2024-09-10T10:45:50.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 msgr2=0x7f1258161810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f124400d5f0 tx=0x7f124400dab0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 shutdown_connections 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f123806be30 0x7f123806e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1258147e90 0x7f12580b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1258147230 0x7f1258161810 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 --2- 172.21.15.28:0/3415677484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12580725f0 0x7f125813f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 >> 172.21.15.28:0/3415677484 conn(0x7f1258067d70 msgr2=0x7f125806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 shutdown_connections 2024-09-10T10:45:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:50.910+0000 7f125d400640 1 -- 172.21.15.28:0/3415677484 wait complete. 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:45:50.929 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:50.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.930 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:45:50.965 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:45:50.966 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42554sec preferred_lft 42554sec 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.967 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:45:50.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:50.969 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:45:51.062 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:45:51.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.065 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:45:51.101 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:45:51.102 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:45:51.102 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:45:51.102 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:45:51.102 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:45:51.102 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:45:51.102 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:45:51.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:45:51.102 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:45:51.103 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:45:51.103 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:45:51.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:45:51.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.300 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:45:51.336 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:45:51.336 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:45:51.336 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:51.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.337 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:45:51.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:51.433 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:51.468 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:45:51.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:45:51.468 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:51.543 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:45:51.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.543 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:51.671 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:51.671 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:51.671 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:51.686 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:51.686 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:51.686 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:51.687 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:51.687 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:51.709 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:45:51.709 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:45:51.709 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:45:51.709 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:45:51.710 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:45:51.710 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:45:51.710 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:45:51.710 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:45:51.711 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:45:51.711 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:45:51.711 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:45:51.711 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:45:51.711 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:45:51.714 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:45:51.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.714 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:51.770 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:45:51.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:45:51.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:51.859 INFO:teuthology.orchestra.run.smithi028.stdout:client4868 2024-09-10T10:45:51.859 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:51.859 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:51.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:51.894 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T10:45:51.971 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:45:51.972 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:45:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.089+0000 7f669f68cf40 1 Processor -- start 2024-09-10T10:45:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.089+0000 7f669f68cf40 1 -- start start 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83b28570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b83b71800 0x555b83b28ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555b83acdb30 con 0x7f6684002460 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b83ad1b70 con 0x555b83bf8f80 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669f68cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555b83b56790 con 0x555b83b71800 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58698/0 (socket says 172.21.15.28:58698) 2024-09-10T10:45:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669c800640 1 -- 172.21.15.28:0/3501543139 learned_addr learned my addr 172.21.15.28:0/3501543139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669d200640 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b83b71800 0x555b83b28ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.090+0000 7f669be00640 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83b28570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 -- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b83b71800 msgr2=0x555b83b28ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b83b71800 0x555b83b28ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 -- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 msgr2=0x555b83b28570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83b28570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 -- 172.21.15.28:0/3501543139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b83b6de10 con 0x7f6684002460 2024-09-10T10:45:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669c800640 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6690000bc0 tx=0x7f669004f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6e364c7da73334 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669aa00640 1 -- 172.21.15.28:0/3501543139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690001670 con 0x7f6684002460 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669aa00640 1 -- 172.21.15.28:0/3501543139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690066070 con 0x7f6684002460 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.091+0000 7f669aa00640 1 -- 172.21.15.28:0/3501543139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690061670 con 0x7f6684002460 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 -- 172.21.15.28:0/3501543139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 msgr2=0x555b83bf4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6690000bc0 tx=0x7f669004f000 comp rx=0 tx=0).stop 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 -- 172.21.15.28:0/3501543139 shutdown_connections 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b83b71800 0x555b83b28ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83b28570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 --2- 172.21.15.28:0/3501543139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6684002460 0x555b83bf4e50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 -- 172.21.15.28:0/3501543139 >> 172.21.15.28:0/3501543139 conn(0x555b83b53b90 msgr2=0x555b83b52b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 -- 172.21.15.28:0/3501543139 shutdown_connections 2024-09-10T10:45:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.092+0000 7f669f68cf40 1 -- 172.21.15.28:0/3501543139 wait complete. 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.093+0000 7f669f68cf40 1 Processor -- start 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.093+0000 7f669f68cf40 1 -- start start 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 0x555b83be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83bf8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6684002460 0x555b83bf6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555b83b534f0 con 0x555b83b71800 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b83b6b950 con 0x555b83bf8f80 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669f68cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555b83b537d0 con 0x7f6684002460 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 0x555b83be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 0x555b83be05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58708/0 (socket says 172.21.15.28:58708) 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 -- 172.21.15.28:0/528846929 learned_addr learned my addr 172.21.15.28:0/528846929 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669d200640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6684002460 0x555b83bf6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669be00640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83bf8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 -- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6684002460 msgr2=0x555b83bf6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6684002460 0x555b83bf6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 -- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 msgr2=0x555b83bf8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83bf8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669c800640 1 -- 172.21.15.28:0/528846929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b83bf5a00 con 0x555b83b71800 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669d200640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6684002460 0x555b83bf6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.094+0000 7f669be00640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b83bf8f80 0x555b83bf8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:52.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f669c800640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 0x555b83be05c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6690046ac0 tx=0x7f66900019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f6699600640 1 -- 172.21.15.28:0/528846929 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690061790 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f6699600640 1 -- 172.21.15.28:0/528846929 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690071070 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f6699600640 1 -- 172.21.15.28:0/528846929 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690061b70 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b83bf5ca0 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.095+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b83be0b00 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.097+0000 7f6699600640 1 -- 172.21.15.28:0/528846929 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f669005e050 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f6699600640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f667c06a840 0x7f667c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f6699600640 1 -- 172.21.15.28:0/528846929 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f66900d0b80 con 0x555b83b71800 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f669f68cf40 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555b83bf6cc0 0x555b83bf2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45) -- 0x555b83be0ed0 con 0x555b83bf6cc0 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f669be00640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f667c06a840 0x7f667c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.098+0000 7f669d200640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555b83bf6cc0 0x555b83bf2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.099+0000 7f669d200640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555b83bf6cc0 0x555b83bf2820 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.099+0000 7f669be00640 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f667c06a840 0x7f667c06cd00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6688000f60 tx=0x7f6688000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.100+0000 7f669d200640 1 -- 172.21.15.28:0/528846929 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v45'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6694011040 con 0x555b83bf6cc0 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.100+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555b83bf6cc0 msgr2=0x555b83bf2820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.100+0000 7f669f68cf40 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555b83bf6cc0 0x555b83bf2820 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f667c06a840 msgr2=0x7f667c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f667c06a840 0x7f667c06cd00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6688000f60 tx=0x7f6688000ed0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 msgr2=0x555b83be05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 --2- 172.21.15.28:0/528846929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b83b71800 0x555b83be05c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6690046ac0 tx=0x7f66900019c0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669d200640 1 -- 172.21.15.28:0/528846929 reap_dead start 2024-09-10T10:45:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 shutdown_connections 2024-09-10T10:45:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 >> 172.21.15.28:0/528846929 conn(0x555b83b53b90 msgr2=0x555b83bfa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 shutdown_connections 2024-09-10T10:45:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.101+0000 7f669f68cf40 1 -- 172.21.15.28:0/528846929 wait complete. 2024-09-10T10:45:52.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:52.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.261+0000 7f90e9200640 1 Processor -- start 2024-09-10T10:45:52.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.261+0000 7f90e9200640 1 -- start start 2024-09-10T10:45:52.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90e415b300 0x7f90e40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f90e40cd3f0 0x7f90e406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e9200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90e40cdbe0 con 0x7f90e4072d30 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e9200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f90e40728a0 con 0x7f90e40cd3f0 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e9200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90e40cd900 con 0x7f90e415b300 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90dbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f90e40cd3f0 0x7f90e406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90dbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f90e40cd3f0 0x7f90e406f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58712/0 (socket says 172.21.15.28:58712) 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90dbe00640 1 -- 172.21.15.28:0/2070066104 learned_addr learned my addr 172.21.15.28:0/2070066104 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90e1e00640 1 --2- 172.21.15.28:0/2070066104 >> v2:172.21.15.138:3300/0 conn(0x7f90e415b300 0x7f90e40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90dbe00640 1 -- 172.21.15.28:0/2070066104 >> v2:172.21.15.138:3300/0 conn(0x7f90e415b300 msgr2=0x7f90e40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.263+0000 7f90dbe00640 1 --2- 172.21.15.28:0/2070066104 >> v2:172.21.15.138:3300/0 conn(0x7f90e415b300 0x7f90e40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.264+0000 7f90dbe00640 1 -- 172.21.15.28:0/2070066104 >> v1:172.21.15.138:6789/0 conn(0x7f90e4072d30 legacy=0x7f90e40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.264+0000 7f90dbe00640 1 -- 172.21.15.28:0/2070066104 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e4072250 con 0x7f90e40cd3f0 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90dbe00640 1 --2- 172.21.15.28:0/2070066104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e406f1c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f90e4070250 tx=0x7f90cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7318378aa9a699d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e1400640 1 -- 172.21.15.28:0/2070066104 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc037070 con 0x7f90e40cd3f0 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e1400640 1 -- 172.21.15.28:0/2070066104 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc032440 con 0x7f90e40cd3f0 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e1400640 1 -- 172.21.15.28:0/2070066104 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc032810 con 0x7f90e40cd3f0 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 -- 172.21.15.28:0/2070066104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 msgr2=0x7f90e406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 --2- 172.21.15.28:0/2070066104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e406f1c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f90e4070250 tx=0x7f90cc02a270 comp rx=0 tx=0).stop 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 -- 172.21.15.28:0/2070066104 shutdown_connections 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 --2- 172.21.15.28:0/2070066104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e406f1c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 --2- 172.21.15.28:0/2070066104 >> v2:172.21.15.138:3300/0 conn(0x7f90e415b300 0x7f90e40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 -- 172.21.15.28:0/2070066104 >> 172.21.15.28:0/2070066104 conn(0x7f90e4067d70 msgr2=0x7f90e406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.265+0000 7f90e9200640 1 -- 172.21.15.28:0/2070066104 shutdown_connections 2024-09-10T10:45:52.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.266+0000 7f90e9200640 1 -- 172.21.15.28:0/2070066104 wait complete. 2024-09-10T10:45:52.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.267+0000 7f90e9200640 1 Processor -- start 2024-09-10T10:45:52.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.267+0000 7f90e9200640 1 -- start start 2024-09-10T10:45:52.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 0x7f90e4151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e415b300 0x7f90e415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f90e40d0160 con 0x7f90e40cd3f0 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90e40cffe0 con 0x7f90e415b300 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90e9200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f90e406f700 con 0x7f90e4072d30 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90dbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58720/0 (socket says 172.21.15.28:58720) 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.268+0000 7f90dbe00640 1 -- 172.21.15.28:0/4175363693 learned_addr learned my addr 172.21.15.28:0/4175363693 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90e1e00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 0x7f90e4151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90e2800640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e415b300 0x7f90e415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 -- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 msgr2=0x7f90e4151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 0x7f90e4151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 -- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e415b300 msgr2=0x7f90e415c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e415b300 0x7f90e415c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 -- 172.21.15.28:0/4175363693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e41f8d40 con 0x7f90e40cd3f0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90e1e00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 0x7f90e4151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.269+0000 7f90dbe00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f90cc02a2b0 tx=0x7f90cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.270+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc041020 con 0x7f90e40cd3f0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.270+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc0306c0 con 0x7f90e40cd3f0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.270+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e41f8780 con 0x7f90e40cd3f0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.270+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc037040 con 0x7f90e40cd3f0 2024-09-10T10:45:52.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.270+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e415b810 con 0x7f90e40cd3f0 2024-09-10T10:45:52.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.271+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e406fac0 con 0x7f90e40cd3f0 2024-09-10T10:45:52.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.272+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90cc044020 con 0x7f90e40cd3f0 2024-09-10T10:45:52.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.272+0000 7f90da000640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 0x7f90c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.273+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f90cc0ab8d0 con 0x7f90e40cd3f0 2024-09-10T10:45:52.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.273+0000 7f90e1e00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 0x7f90c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.274+0000 7f90e1e00640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 0x7f90c006e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006fd0 tx=0x7f90d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.277+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90cc04b030 con 0x7f90e40cd3f0 2024-09-10T10:45:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.499+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90e415a740 con 0x7f90e40cd3f0 2024-09-10T10:45:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.500+0000 7f90da000640 1 -- 172.21.15.28:0/4175363693 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f90cc07a050 con 0x7f90e40cd3f0 2024-09-10T10:45:52.502 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:52.502 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:52.503 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 msgr2=0x7f90c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 0x7f90c006e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006fd0 tx=0x7f90d4008040 comp rx=0 tx=0).stop 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 msgr2=0x7f90e41f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f90cc02a2b0 tx=0x7f90cc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 shutdown_connections 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90c006be30 0x7f90c006e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90e415b300 0x7f90e415c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90e40cd3f0 0x7f90e41f7e80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 --2- 172.21.15.28:0/4175363693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90e4072d30 0x7f90e4151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 >> 172.21.15.28:0/4175363693 conn(0x7f90e4067d70 msgr2=0x7f90e406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 shutdown_connections 2024-09-10T10:45:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.502+0000 7f90e9200640 1 -- 172.21.15.28:0/4175363693 wait complete. 2024-09-10T10:45:52.521 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 415178632 2024-09-10T10:45:52.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.679+0000 7ffa3ee00640 1 Processor -- start 2024-09-10T10:45:52.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.679+0000 7ffa3ee00640 1 -- start start 2024-09-10T10:45:52.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa380d0450 0x7ffa3806d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffa380b6640 0x7ffa38167de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa380d1f70 con 0x7ffa38147d80 2024-09-10T10:45:52.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa380b51a0 con 0x7ffa380d0450 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffa3806d8c0 con 0x7ffa380b6640 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3da00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffa38147d80 0x7ffa380cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:55032/0 (socket says 172.21.15.28:55032) 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3da00640 1 -- 172.21.15.28:0/3772572735 learned_addr learned my addr 172.21.15.28:0/3772572735 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3772572735 >> v2:172.21.15.28:3300/0 conn(0x7ffa380d0450 0x7ffa3806d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3772572735 >> v2:172.21.15.138:3300/0 conn(0x7ffa380b6640 0x7ffa38167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3d000640 1 -- 172.21.15.28:0/3772572735 >> v2:172.21.15.138:3300/0 conn(0x7ffa380b6640 msgr2=0x7ffa38167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3772572735 >> v2:172.21.15.138:3300/0 conn(0x7ffa380b6640 0x7ffa38167de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa37e00640 1 -- 172.21.15.28:0/3772572735 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1712707626 0 0) 0x7ffa380d1f70 con 0x7ffa38147d80 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3d000640 1 -- 172.21.15.28:0/3772572735 >> v1:172.21.15.28:6789/0 conn(0x7ffa38147d80 legacy=0x7ffa380cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.681+0000 7ffa3d000640 1 -- 172.21.15.28:0/3772572735 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3806dba0 con 0x7ffa380d0450 2024-09-10T10:45:52.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.682+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3772572735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa380d0450 0x7ffa3806d230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa38072750 tx=0x7ffa2402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fc2555f8012768 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa37e00640 1 -- 172.21.15.28:0/3772572735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24037070 con 0x7ffa380d0450 2024-09-10T10:45:52.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa37e00640 1 -- 172.21.15.28:0/3772572735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24032440 con 0x7ffa380d0450 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa37e00640 1 -- 172.21.15.28:0/3772572735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2402ae60 con 0x7ffa380d0450 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3772572735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa380d0450 msgr2=0x7ffa3806d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3772572735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa380d0450 0x7ffa3806d230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ffa38072750 tx=0x7ffa2402a110 comp rx=0 tx=0).stop 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3772572735 shutdown_connections 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3772572735 >> v2:172.21.15.138:3300/0 conn(0x7ffa380b6640 0x7ffa38167de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3772572735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa380d0450 0x7ffa3806d230 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3772572735 >> 172.21.15.28:0/3772572735 conn(0x7ffa3806fee0 msgr2=0x7ffa380702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3772572735 shutdown_connections 2024-09-10T10:45:52.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.683+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3772572735 wait complete. 2024-09-10T10:45:52.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.685+0000 7ffa3ee00640 1 Processor -- start 2024-09-10T10:45:52.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.685+0000 7ffa3ee00640 1 -- start start 2024-09-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 0x7ffa3815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa380d0450 0x7ffa381e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa380716d0 con 0x7ffa38147d80 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3806e160 con 0x7ffa380d0450 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa38071850 con 0x7ffa380b6640 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58732/0 (socket says 172.21.15.28:58732) 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3e400640 1 -- 172.21.15.28:0/3579977092 learned_addr learned my addr 172.21.15.28:0/3579977092 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa380d0450 0x7ffa381e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.686+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 0x7ffa3815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 -- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 msgr2=0x7ffa3815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 0x7ffa3815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 -- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa380d0450 msgr2=0x7ffa381e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa380d0450 0x7ffa381e6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 -- 172.21.15.28:0/3579977092 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa381e4430 con 0x7ffa38147d80 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 0x7ffa3815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007e70 tx=0x7ffa3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa35600640 1 -- 172.21.15.28:0/3579977092 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30014620 con 0x7ffa38147d80 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa35600640 1 -- 172.21.15.28:0/3579977092 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa30019070 con 0x7ffa38147d80 2024-09-10T10:45:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa35600640 1 -- 172.21.15.28:0/3579977092 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30014a20 con 0x7ffa38147d80 2024-09-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa381e4710 con 0x7ffa38147d80 2024-09-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.687+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381e4cd0 con 0x7ffa38147d80 2024-09-10T10:45:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.689+0000 7ffa35600640 1 -- 172.21.15.28:0/3579977092 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa30014bc0 con 0x7ffa38147d80 2024-09-10T10:45:52.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.690+0000 7ffa35600640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.690+0000 7ffa35600640 1 -- 172.21.15.28:0/3579977092 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7ffa3008b930 con 0x7ffa38147d80 2024-09-10T10:45:52.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.691+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.691+0000 7ffa432b2740 1 Processor -- start 2024-09-10T10:45:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.691+0000 7ffa432b2740 1 -- start start 2024-09-10T10:45:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.691+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000f60 tx=0x7ffa2c007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c4ca0 0x5617895c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5617894fd8b0 con 0x5617895e0c00 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5617894a5810 con 0x5617895df4c0 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5617894f9870 con 0x5617895c4ca0 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58744/0 (socket says 172.21.15.28:58744) 2024-09-10T10:45:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa3d000640 1 -- 172.21.15.28:0/2826909518 learned_addr learned my addr 172.21.15.28:0/2826909518 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa3e400640 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.692+0000 7ffa3da00640 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c4ca0 0x5617895c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 -- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c4ca0 msgr2=0x5617895c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c4ca0 0x5617895c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 -- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 msgr2=0x5617895e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 -- 172.21.15.28:0/2826909518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617892dc900 con 0x5617895e0c00 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa3d000640 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402a150 tx=0x7ffa2402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660a2452af866842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa2a000640 1 -- 172.21.15.28:0/2826909518 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24041030 con 0x5617895e0c00 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa2a000640 1 -- 172.21.15.28:0/2826909518 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24038040 con 0x5617895e0c00 2024-09-10T10:45:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.693+0000 7ffa2a000640 1 -- 172.21.15.28:0/2826909518 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24037070 con 0x5617895e0c00 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 -- 172.21.15.28:0/2826909518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 msgr2=0x5617895e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402a150 tx=0x7ffa2402c000 comp rx=0 tx=0).stop 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 -- 172.21.15.28:0/2826909518 shutdown_connections 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895e84b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 --2- 172.21.15.28:0/2826909518 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c4ca0 0x5617895c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 -- 172.21.15.28:0/2826909518 >> 172.21.15.28:0/2826909518 conn(0x5617895977f0 msgr2=0x5617895dfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 -- 172.21.15.28:0/2826909518 shutdown_connections 2024-09-10T10:45:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.694+0000 7ffa432b2740 1 -- 172.21.15.28:0/2826909518 wait complete. 2024-09-10T10:45:52.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.695+0000 7ffa432b2740 1 -- start start 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.695+0000 7ffa432b2740 1 Processor -- start 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56178959ac50 con 0x5617895e0c00 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5617894b8a30 con 0x5617895df4c0 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56178959af30 con 0x5617895c6670 2024-09-10T10:45:52.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa432b2740 10 client.0 _ll_register_callbacks cb 0x5617894aa510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58760/0 (socket says 172.21.15.28:58760) 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa3d000640 1 -- 172.21.15.28:0/3502468850 learned_addr learned my addr 172.21.15.28:0/3502468850 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.696+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 -- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 msgr2=0x5617895c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 -- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 msgr2=0x5617895db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 -- 172.21.15.28:0/3502468850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617891d01f0 con 0x5617895e0c00 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402a700 tx=0x7ffa24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5732733a858d9e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2404d030 con 0x5617895e0c00 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7ffa24003820 con 0x5617895e0c00 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa24038040 con 0x5617895e0c00 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa24056050 con 0x5617895e0c00 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa432b2740 10 client.5271 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:52.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.697+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56178933d090 con 0x5617895e0c00 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24038040 con 0x5617895e0c00 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7ffa24003a20 con 0x5617895e0c00 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 10 client.5271 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 10 client.5271 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 20 client.5271 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895c30b0 0x5617895c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 4 client.5271 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56178950afe0 con 0x5617895c30b0 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.698+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895c30b0 0x5617895c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.699+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895c30b0 0x5617895c3490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347fb8b7887204b1 server_cookie=77b1b3ba8858266e in_seq=0 out_seq=0 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.699+0000 7ffa1ea00640 10 client.5271 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.699+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3502468850 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56178950afe0 con 0x5617895c30b0 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.699+0000 7ffa1ea00640 10 client.5271 handle_command_reply: tid=0 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.699+0000 7ffa432b2740 1 client.5271 shutdown 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 msgr2=0x5617895b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffa2402a700 tx=0x7ffa24004730 comp rx=0 tx=0).stop 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 shutdown_connections 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895c30b0 0x5617895c3490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895e0c00 0x5617895b35d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3502468850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895df4c0 0x5617895db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa1ea00640 0 client.5271 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 >> 172.21.15.28:0/3502468850 conn(0x5617895c4ca0 msgr2=0x5617895dde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 shutdown_connections 2024-09-10T10:45:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 1 -- 172.21.15.28:0/3502468850 wait complete. 2024-09-10T10:45:52.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.700+0000 7ffa432b2740 20 client.5271 trim_cache size 0 max 16384 2024-09-10T10:45:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.714+0000 7ffa432b2740 1 Processor -- start 2024-09-10T10:45:52.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.714+0000 7ffa432b2740 1 -- start start 2024-09-10T10:45:52.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.714+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.714+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa432b2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381ffd60 con 0x5617895df4c0 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa381ff0c0 con 0x5617895e0c00 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa381ff710 con 0x5617895c6670 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58770/0 (socket says 172.21.15.28:58770) 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 -- 172.21.15.28:0/1225480793 learned_addr learned my addr 172.21.15.28:0/1225480793 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 -- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 msgr2=0x5617895dfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 -- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 msgr2=0x561789603400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 -- 172.21.15.28:0/1225480793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617891a92b0 con 0x5617895df4c0 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.715+0000 7ffa3d000640 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa24004570 tx=0x7ffa24034070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e40d99351de00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa29600640 1 -- 172.21.15.28:0/1225480793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24041040 con 0x5617895df4c0 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa29600640 1 -- 172.21.15.28:0/1225480793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24003ae0 con 0x5617895df4c0 2024-09-10T10:45:52.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa29600640 1 -- 172.21.15.28:0/1225480793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24037040 con 0x5617895df4c0 2024-09-10T10:45:52.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 -- 172.21.15.28:0/1225480793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 msgr2=0x5617895eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa24004570 tx=0x7ffa24034070 comp rx=0 tx=0).stop 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 -- 172.21.15.28:0/1225480793 shutdown_connections 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895df4c0 0x5617895eefe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 --2- 172.21.15.28:0/1225480793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895c6670 0x5617895dfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 -- 172.21.15.28:0/1225480793 >> 172.21.15.28:0/1225480793 conn(0x5617895c4ca0 msgr2=0x5617895db820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 -- 172.21.15.28:0/1225480793 shutdown_connections 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.716+0000 7ffa432b2740 1 -- 172.21.15.28:0/1225480793 wait complete. 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.717+0000 7ffa432b2740 1 -- start start 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.717+0000 7ffa432b2740 1 Processor -- start 2024-09-10T10:45:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895df4c0 0x5617896036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5617895c4130 con 0x5617895c6670 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5617895dcb10 con 0x5617895e0c00 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5617895b3b10 con 0x5617895df4c0 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa432b2740 10 client.0 _ll_register_callbacks cb 0x7ffa240040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa3da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa3da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58774/0 (socket says 172.21.15.28:58774) 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa3da00640 1 -- 172.21.15.28:0/3509935011 learned_addr learned my addr 172.21.15.28:0/3509935011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895df4c0 0x5617896036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.718+0000 7ffa3e400640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 -- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895df4c0 msgr2=0x5617896036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895df4c0 0x5617896036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 -- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 msgr2=0x561789603c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 -- 172.21.15.28:0/3509935011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617895dba40 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa3da00640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c005e90 tx=0x7ffa2c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e72d3dd6fd38cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c028040 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7ffa2c010040 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa24003ae0 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa2c035050 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa432b2740 10 client.5277 fetch_fsmap learned FSMap version 23 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.719+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x56178965d6a0 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa24003ae0 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7ffa2c038020 con 0x5617895c6670 2024-09-10T10:45:52.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 10 client.5277 fetch_fsmap finished waiting for FSMap version 23 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 10 client.5277 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 20 client.5277 populate_metadata read hostname 'smithi028' 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895d9120 0x5617895c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 4 client.5277 mds_command: new command op to 5088 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 415178632}] 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 --> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] -- command(tid 0: {"prefix": "damage rm", "damage_id": 415178632}) -- 0x561789505330 con 0x5617895d9120 2024-09-10T10:45:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.720+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895d9120 0x5617895c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa3d000640 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895d9120 0x5617895c5080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e5b167862d58fb7 server_cookie=50f05fb16674d76e in_seq=0 out_seq=0 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa1ea00640 10 client.5277 ms_handle_connect on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa1ea00640 1 -- 172.21.15.28:0/3509935011 <== mds.0 v2:172.21.15.28:6834/1058548427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+0 (crc 0 0 0) 0x561789505330 con 0x5617895d9120 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa1ea00640 10 client.5277 handle_command_reply: tid=0 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 client.5277 shutdown 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 msgr2=0x56178965d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c005e90 tx=0x7ffa2c001080 comp rx=0 tx=0).stop 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 shutdown_connections 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:6834/1058548427,v1:172.21.15.28:6835/1058548427] conn(0x5617895d9120 0x5617895c5080 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617895e0c00 0x561789603c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617895df4c0 0x5617896036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 --2- 172.21.15.28:0/3509935011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617895c6670 0x56178965d160 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa1ea00640 0 client.5277 ms_handle_reset on v2:172.21.15.28:6834/1058548427 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 >> 172.21.15.28:0/3509935011 conn(0x5617895c4ca0 msgr2=0x5617895eefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 shutdown_connections 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 1 -- 172.21.15.28:0/3509935011 wait complete. 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.721+0000 7ffa432b2740 20 client.5277 trim_cache size 0 max 16384 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 msgr2=0x7ffa1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 0x7ffa1806e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000f60 tx=0x7ffa2c007ca0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 msgr2=0x7ffa381e71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffa30007e70 tx=0x7ffa3000e4b0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 shutdown_connections 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa1806be30 0x7ffa1806e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa38147d80 0x7ffa381e71a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa380d0450 0x7ffa381e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 --2- 172.21.15.28:0/3579977092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa380b6640 0x7ffa3815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 >> 172.21.15.28:0/3579977092 conn(0x7ffa3806fee0 msgr2=0x7ffa38146e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 shutdown_connections 2024-09-10T10:45:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52.722+0000 7ffa3ee00640 1 -- 172.21.15.28:0/3579977092 wait complete. 2024-09-10T10:45:52.739 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:45:52.739 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> import errno 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> f = open("/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged", "w") 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> f.close() 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> except IOError as e: 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(errno.EIO) 2024-09-10T10:45:52.740 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:45:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:45:52.856 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:45:52.857 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:45:52.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:45:53.082 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:45:53.083 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:45:53.083 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:53.083 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.120 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:53.120 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:45:53.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.390+0000 7eff92800640 1 Processor -- start 2024-09-10T10:45:53.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.390+0000 7eff92800640 1 -- start start 2024-09-10T10:45:53.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff92800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7eff8c072700 0x7eff8c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff92800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8c146140 con 0x7eff8c072320 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff92800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff8c0cd740 con 0x7eff8c072de0 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff92800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7eff8c0cda20 con 0x7eff8c072700 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7eff8c072320 0x7eff8c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:55036/0 (socket says 172.21.15.28:55036) 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8b400640 1 -- 172.21.15.28:0/246000332 learned_addr learned my addr 172.21.15.28:0/246000332 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8be00640 1 --2- 172.21.15.28:0/246000332 >> v2:172.21.15.28:3300/0 conn(0x7eff8c072700 0x7eff8c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8be00640 1 -- 172.21.15.28:0/246000332 >> v1:172.21.15.138:6789/0 conn(0x7eff8c072de0 legacy=0x7eff8c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:53.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8be00640 1 -- 172.21.15.28:0/246000332 >> v1:172.21.15.28:6789/0 conn(0x7eff8c072320 legacy=0x7eff8c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.392+0000 7eff8be00640 1 -- 172.21.15.28:0/246000332 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c0cdd00 con 0x7eff8c072700 2024-09-10T10:45:53.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.393+0000 7eff8be00640 1 --2- 172.21.15.28:0/246000332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072700 0x7eff8c15d8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b66cd0806a4c742 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff8aa00640 1 -- 172.21.15.28:0/246000332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84031600 con 0x7eff8c072700 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff8aa00640 1 -- 172.21.15.28:0/246000332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff840317a0 con 0x7eff8c072700 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff8aa00640 1 -- 172.21.15.28:0/246000332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84031b70 con 0x7eff8c072700 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 -- 172.21.15.28:0/246000332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072700 msgr2=0x7eff8c15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 --2- 172.21.15.28:0/246000332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072700 0x7eff8c15d8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8402a610 comp rx=0 tx=0).stop 2024-09-10T10:45:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 -- 172.21.15.28:0/246000332 shutdown_connections 2024-09-10T10:45:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 --2- 172.21.15.28:0/246000332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072700 0x7eff8c15d8f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 -- 172.21.15.28:0/246000332 >> 172.21.15.28:0/246000332 conn(0x7eff8c067d70 msgr2=0x7eff8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 -- 172.21.15.28:0/246000332 shutdown_connections 2024-09-10T10:45:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.394+0000 7eff92800640 1 -- 172.21.15.28:0/246000332 wait complete. 2024-09-10T10:45:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.396+0000 7eff92800640 1 Processor -- start 2024-09-10T10:45:53.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.396+0000 7eff92800640 1 -- start start 2024-09-10T10:45:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7eff8c072700 0x7eff8c0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c072de0 0x7eff8c0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c15dfb0 con 0x7eff8c072320 2024-09-10T10:45:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c15de30 con 0x7eff8c072de0 2024-09-10T10:45:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff92800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c15e130 con 0x7eff8c072700 2024-09-10T10:45:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff8b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff8b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58792/0 (socket says 172.21.15.28:58792) 2024-09-10T10:45:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.397+0000 7eff8b400640 1 -- 172.21.15.28:0/2883895588 learned_addr learned my addr 172.21.15.28:0/2883895588 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8be00640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c072de0 0x7eff8c0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff83e00640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7eff8c072700 0x7eff8c0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 -- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7eff8c072700 msgr2=0x7eff8c0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7eff8c072700 0x7eff8c0b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 -- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c072de0 msgr2=0x7eff8c0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c072de0 0x7eff8c0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 -- 172.21.15.28:0/2883895588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c0ad8a0 con 0x7eff8c072320 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff8b400640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c400 tx=0x7eff7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:53.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c017070 con 0x7eff8c072320 2024-09-10T10:45:53.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff7c005ce0 con 0x7eff8c072320 2024-09-10T10:45:53.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c0072f0 con 0x7eff8c072320 2024-09-10T10:45:53.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.398+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c0cae70 con 0x7eff8c072320 2024-09-10T10:45:53.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.399+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c0cb710 con 0x7eff8c072320 2024-09-10T10:45:53.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.400+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c0cce60 con 0x7eff8c072320 2024-09-10T10:45:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.400+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff7c01a070 con 0x7eff8c072320 2024-09-10T10:45:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.401+0000 7eff83400640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 0x7eff6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.401+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7eff7c08a340 con 0x7eff8c072320 2024-09-10T10:45:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.402+0000 7eff83e00640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 0x7eff6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.403+0000 7eff83e00640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 0x7eff6806e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff74006fd0 tx=0x7eff74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:53.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.406+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7eff7c058510 con 0x7eff8c072320 2024-09-10T10:45:53.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.600+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7eff8c0ce500 con 0x7eff8c072320 2024-09-10T10:45:53.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.601+0000 7eff83400640 1 -- 172.21.15.28:0/2883895588 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v45) ==== 80+0+122 (secure 0 0 0) 0x7eff7c05c810 con 0x7eff8c072320 2024-09-10T10:45:53.603 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3203683106 2024-09-11T10:45:10.614652+0000 2024-09-10T10:45:53.603 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3203683106 2024-09-11T10:45:10.614652+0000 2024-09-10T10:45:53.604 INFO:teuthology.orchestra.run.smithi028.stderr:listed 2 entries 2024-09-10T10:45:53.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 msgr2=0x7eff6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 0x7eff6806e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff74006fd0 tx=0x7eff74008040 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 msgr2=0x7eff8c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7eff7c00c400 tx=0x7eff7c00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 shutdown_connections 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff6806be30 0x7eff6806e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c072de0 0x7eff8c0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7eff8c072700 0x7eff8c0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 --2- 172.21.15.28:0/2883895588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff8c072320 0x7eff8c15bee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 >> 172.21.15.28:0/2883895588 conn(0x7eff8c067d70 msgr2=0x7eff8c0746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 shutdown_connections 2024-09-10T10:45:53.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.603+0000 7eff92800640 1 -- 172.21.15.28:0/2883895588 wait complete. 2024-09-10T10:45:53.622 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:45:53.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:53.622 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:45:53.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:45:53.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:45:53.685 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:45:53.721 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:45:53.721 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:45:53.721 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:45:53.762 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:45:53.762 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:45:53.762 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:45:53.797 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:45:53.797 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:45:53.797 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:45:53.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:45:53.839 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:53.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.993+0000 7f992a800640 1 Processor -- start 2024-09-10T10:45:53.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.994+0000 7f992a800640 1 -- start start 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f992a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f992415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f992a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9924072700 0x7f992415d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f992a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9924146140 con 0x7f9924072320 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f992a800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f99240cd740 con 0x7f9924072700 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f992a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99240cda20 con 0x7f9924072de0 2024-09-10T10:45:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f9923e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9924072700 0x7f992415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f9923e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9924072700 0x7f992415d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34220/0 (socket says 172.21.15.28:34220) 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.995+0000 7f9923e00640 1 -- 172.21.15.28:0/1058848694 learned_addr learned my addr 172.21.15.28:0/1058848694 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.996+0000 7f9923e00640 1 -- 172.21.15.28:0/1058848694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 msgr2=0x7f992415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.996+0000 7f9923e00640 1 --2- 172.21.15.28:0/1058848694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f992415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.996+0000 7f9923e00640 1 -- 172.21.15.28:0/1058848694 >> v1:172.21.15.138:6789/0 conn(0x7f9924072320 legacy=0x7f99240cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:53.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.996+0000 7f9923e00640 1 -- 172.21.15.28:0/1058848694 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99240cdd00 con 0x7f9924072700 2024-09-10T10:45:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f9923e00640 1 --2- 172.21.15.28:0/1058848694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072700 0x7f992415d720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f991c009e50 tx=0x7f991c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ab03edae73704 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f9922a00640 1 -- 172.21.15.28:0/1058848694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c036070 con 0x7f9924072700 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f9922a00640 1 -- 172.21.15.28:0/1058848694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991c031600 con 0x7f9924072700 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f9922a00640 1 -- 172.21.15.28:0/1058848694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c0319d0 con 0x7f9924072700 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f992a800640 1 -- 172.21.15.28:0/1058848694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072700 msgr2=0x7f992415d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f992a800640 1 --2- 172.21.15.28:0/1058848694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072700 0x7f992415d720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f991c009e50 tx=0x7f991c02a630 comp rx=0 tx=0).stop 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.997+0000 7f992a800640 1 -- 172.21.15.28:0/1058848694 shutdown_connections 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.998+0000 7f992a800640 1 --2- 172.21.15.28:0/1058848694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072700 0x7f992415d720 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.998+0000 7f992a800640 1 --2- 172.21.15.28:0/1058848694 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f992415d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.998+0000 7f992a800640 1 -- 172.21.15.28:0/1058848694 >> 172.21.15.28:0/1058848694 conn(0x7f9924067d70 msgr2=0x7f992406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.998+0000 7f992a800640 1 -- 172.21.15.28:0/1058848694 shutdown_connections 2024-09-10T10:45:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.998+0000 7f992a800640 1 -- 172.21.15.28:0/1058848694 wait complete. 2024-09-10T10:45:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.999+0000 7f992a800640 1 Processor -- start 2024-09-10T10:45:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:53.999+0000 7f992a800640 1 -- start start 2024-09-10T10:45:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924072700 0x7f99240cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f99240aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f992415dde0 con 0x7f9924072320 2024-09-10T10:45:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f992415dc60 con 0x7f9924072700 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f992a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f992415df60 con 0x7f9924072de0 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f9923400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f9923400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34224/0 (socket says 172.21.15.28:34224) 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 -- 172.21.15.28:0/159879098 learned_addr learned my addr 172.21.15.28:0/159879098 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.000+0000 7f991be00640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924072700 0x7f99240cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923e00640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f99240aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 -- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 msgr2=0x7f99240aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f99240aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 -- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924072700 msgr2=0x7f99240cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924072700 0x7f99240cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 -- 172.21.15.28:0/159879098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99240ad8a0 con 0x7f9924072320 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f9923400640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f991400c430 tx=0x7f991400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914011790 con 0x7f9924072320 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9914017070 con 0x7f9924072320 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.001+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914011bb0 con 0x7f9924072320 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.002+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99241f49a0 con 0x7f9924072320 2024-09-10T10:45:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.002+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241f4f60 con 0x7f9924072320 2024-09-10T10:45:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.003+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99240ce460 con 0x7f9924072320 2024-09-10T10:45:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.004+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f991401c460 con 0x7f9924072320 2024-09-10T10:45:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.004+0000 7f991b400640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.005+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f991408aea0 con 0x7f9924072320 2024-09-10T10:45:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.005+0000 7f991be00640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.006+0000 7f991be00640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 0x7f990006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99240ce0f0 tx=0x7f990c009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.009+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f99240ce460 con 0x7f9924072320 2024-09-10T10:45:54.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.229+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f992415e390 con 0x7f9924072320 2024-09-10T10:45:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.230+0000 7f991b400640 1 -- 172.21.15.28:0/159879098 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f9914058f40 con 0x7f9924072320 2024-09-10T10:45:54.232 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:54.232 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 msgr2=0x7f990006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 0x7f990006e2f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99240ce0f0 tx=0x7f990c009060 comp rx=0 tx=0).stop 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 msgr2=0x7f99240b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f991400c430 tx=0x7f991400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 shutdown_connections 2024-09-10T10:45:54.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f990006be30 0x7f990006e2f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9924072de0 0x7f99240aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9924072700 0x7f99240cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 --2- 172.21.15.28:0/159879098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9924072320 0x7f99240b35d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 >> 172.21.15.28:0/159879098 conn(0x7f9924067d70 msgr2=0x7f99240c59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 shutdown_connections 2024-09-10T10:45:54.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.232+0000 7f992a800640 1 -- 172.21.15.28:0/159879098 wait complete. 2024-09-10T10:45:54.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:54.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.405+0000 7f28c6a00640 1 Processor -- start 2024-09-10T10:45:54.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.405+0000 7f28c6a00640 1 -- start start 2024-09-10T10:45:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.406+0000 7f28c6a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 0x7f28c00b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.406+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 0x7f28c00b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.406+0000 7f28c6a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c006d9f0 con 0x7f28c0161c80 2024-09-10T10:45:54.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.406+0000 7f28c6a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f28c006dcd0 con 0x7f28c0162060 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.406+0000 7f28c6a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28c006dfb0 con 0x7f28c00b64b0 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 0x7f28c00b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bfe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f28c0161c80 0x7f28c006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:46758/0 (socket says 172.21.15.28:46758) 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bfe00640 1 -- 172.21.15.28:0/2842346787 learned_addr learned my addr 172.21.15.28:0/2842346787 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bf400640 1 --2- 172.21.15.28:0/2842346787 >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 0x7f28c00b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bea00640 1 -- 172.21.15.28:0/2842346787 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637979328 0 0) 0x7f28c006d9f0 con 0x7f28c0161c80 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bea00640 1 -- 172.21.15.28:0/2842346787 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28a4003520 con 0x7f28c0161c80 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28c4a00640 1 -- 172.21.15.28:0/2842346787 >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 msgr2=0x7f28c00b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28c4a00640 1 --2- 172.21.15.28:0/2842346787 >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 0x7f28c00b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28c4a00640 1 -- 172.21.15.28:0/2842346787 >> v1:172.21.15.138:6789/0 conn(0x7f28c0161c80 legacy=0x7f28c006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28c4a00640 1 -- 172.21.15.28:0/2842346787 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00d01f0 con 0x7f28c00b64b0 2024-09-10T10:45:54.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.407+0000 7f28bf400640 1 --2- 172.21.15.28:0/2842346787 >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 0x7f28c00b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.408+0000 7f28c4a00640 1 --2- 172.21.15.28:0/2842346787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 0x7f28c00b6960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a52978fdf93881 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.408+0000 7f28bea00640 1 -- 172.21.15.28:0/2842346787 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8007740 con 0x7f28c00b64b0 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28bea00640 1 -- 172.21.15.28:0/2842346787 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b80078e0 con 0x7f28c00b64b0 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28bea00640 1 -- 172.21.15.28:0/2842346787 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8007c90 con 0x7f28c00b64b0 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 -- 172.21.15.28:0/2842346787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 msgr2=0x7f28c00b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 --2- 172.21.15.28:0/2842346787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 0x7f28c00b6960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b802a660 comp rx=0 tx=0).stop 2024-09-10T10:45:54.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 -- 172.21.15.28:0/2842346787 shutdown_connections 2024-09-10T10:45:54.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 --2- 172.21.15.28:0/2842346787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00b64b0 0x7f28c00b6960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 --2- 172.21.15.28:0/2842346787 >> v2:172.21.15.138:3300/0 conn(0x7f28c0162060 0x7f28c00b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 -- 172.21.15.28:0/2842346787 >> 172.21.15.28:0/2842346787 conn(0x7f28c00d44f0 msgr2=0x7f28c00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 -- 172.21.15.28:0/2842346787 shutdown_connections 2024-09-10T10:45:54.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.409+0000 7f28c6a00640 1 -- 172.21.15.28:0/2842346787 wait complete. 2024-09-10T10:45:54.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.411+0000 7f28c6a00640 1 Processor -- start 2024-09-10T10:45:54.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.411+0000 7f28c6a00640 1 -- start start 2024-09-10T10:45:54.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c00b64b0 0x7f28c00b7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0161c80 0x7f28c01f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28c00cf360 con 0x7f28c0162060 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28c00cf1e0 con 0x7f28c00b64b0 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c6a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28c00cf4e0 con 0x7f28c0161c80 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34230/0 (socket says 172.21.15.28:34230) 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28c4a00640 1 -- 172.21.15.28:0/3203863563 learned_addr learned my addr 172.21.15.28:0/3203863563 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28bfe00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c00b64b0 0x7f28c00b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.412+0000 7f28bf400640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0161c80 0x7f28c01f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 -- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0161c80 msgr2=0x7f28c01f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0161c80 0x7f28c01f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 -- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c00b64b0 msgr2=0x7f28c00b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c00b64b0 0x7f28c00b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 -- 172.21.15.28:0/3203863563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00b7f60 con 0x7f28c0162060 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8009450 con 0x7f28c0162060 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8004240 con 0x7f28c0162060 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c00b8240 con 0x7f28c0162060 2024-09-10T10:45:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8004630 con 0x7f28c0162060 2024-09-10T10:45:54.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.413+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c00b8800 con 0x7f28c0162060 2024-09-10T10:45:54.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.414+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c00b5f70 con 0x7f28c0162060 2024-09-10T10:45:54.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.415+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28b8039040 con 0x7f28c0162060 2024-09-10T10:45:54.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.416+0000 7f28b7e00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 0x7f28a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.416+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f28b80ac070 con 0x7f28c0162060 2024-09-10T10:45:54.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.417+0000 7f28bfe00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 0x7f28a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.417+0000 7f28bfe00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 0x7f28a006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005880 tx=0x7f28b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.420+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28b8040030 con 0x7f28c0162060 2024-09-10T10:45:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.640+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c00d3890 con 0x7f28c0162060 2024-09-10T10:45:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.640+0000 7f28b7e00640 1 -- 172.21.15.28:0/3203863563 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7f28b807a1d0 con 0x7f28c0162060 2024-09-10T10:45:54.643 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:54.643 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:54.643 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 msgr2=0x7f28a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 0x7f28a006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f28b0005880 tx=0x7f28b0009040 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 msgr2=0x7f28c00b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009d90 tx=0x7f28b8004a40 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 shutdown_connections 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f28a006be30 0x7f28a006e2f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0162060 0x7f28c00b21e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0161c80 0x7f28c01f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 --2- 172.21.15.28:0/3203863563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c00b64b0 0x7f28c00b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 >> 172.21.15.28:0/3203863563 conn(0x7f28c00d44f0 msgr2=0x7f28c00b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 shutdown_connections 2024-09-10T10:45:54.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.642+0000 7f28c6a00640 1 -- 172.21.15.28:0/3203863563 wait complete. 2024-09-10T10:45:54.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:45:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.816+0000 7ff1cf800640 1 Processor -- start 2024-09-10T10:45:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.817+0000 7ff1cf800640 1 -- start start 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cf800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1c815c490 0x7ff1c809b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cf800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff1c809b980 0x7ff1c815c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cf800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c80d01c0 con 0x7ff1c815cdb0 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cf800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c80cff70 con 0x7ff1c815c490 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cf800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c80d0dd0 con 0x7ff1c809b980 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1c815c490 0x7ff1c809b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1c815c490 0x7ff1c809b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34242/0 (socket says 172.21.15.28:34242) 2024-09-10T10:45:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 -- 172.21.15.28:0/1540710859 learned_addr learned my addr 172.21.15.28:0/1540710859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 -- 172.21.15.28:0/1540710859 >> v2:172.21.15.138:3300/0 conn(0x7ff1c809b980 msgr2=0x7ff1c815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 --2- 172.21.15.28:0/1540710859 >> v2:172.21.15.138:3300/0 conn(0x7ff1c809b980 0x7ff1c815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 -- 172.21.15.28:0/1540710859 >> v1:172.21.15.138:6789/0 conn(0x7ff1c815cdb0 legacy=0x7ff1c815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.818+0000 7ff1cce00640 1 -- 172.21.15.28:0/1540710859 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80d04a0 con 0x7ff1c815c490 2024-09-10T10:45:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.819+0000 7ff1cce00640 1 --2- 172.21.15.28:0/1540710859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815c490 0x7ff1c809b440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8074070 tx=0x7ff1bc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d165ec51af83e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1c7400640 1 -- 172.21.15.28:0/1540710859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc037070 con 0x7ff1c815c490 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1c7400640 1 -- 172.21.15.28:0/1540710859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc032440 con 0x7ff1c815c490 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1c7400640 1 -- 172.21.15.28:0/1540710859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc032810 con 0x7ff1c815c490 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 -- 172.21.15.28:0/1540710859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815c490 msgr2=0x7ff1c809b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 --2- 172.21.15.28:0/1540710859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815c490 0x7ff1c809b440 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8074070 tx=0x7ff1bc02a2c0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 -- 172.21.15.28:0/1540710859 shutdown_connections 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 --2- 172.21.15.28:0/1540710859 >> v2:172.21.15.138:3300/0 conn(0x7ff1c809b980 0x7ff1c815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 --2- 172.21.15.28:0/1540710859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815c490 0x7ff1c809b440 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 -- 172.21.15.28:0/1540710859 >> 172.21.15.28:0/1540710859 conn(0x7ff1c80d23a0 msgr2=0x7ff1c80d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.820+0000 7ff1cf800640 1 -- 172.21.15.28:0/1540710859 shutdown_connections 2024-09-10T10:45:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.821+0000 7ff1cf800640 1 -- 172.21.15.28:0/1540710859 wait complete. 2024-09-10T10:45:54.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.822+0000 7ff1cf800640 1 Processor -- start 2024-09-10T10:45:54.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.822+0000 7ff1cf800640 1 -- start start 2024-09-10T10:45:54.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1c809b980 0x7ff1c815f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 0x7ff1c815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80cdfa0 con 0x7ff1c815cdb0 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c80d0a60 con 0x7ff1c815c490 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.823+0000 7ff1cf800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c80d1260 con 0x7ff1c809b980 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34256/0 (socket says 172.21.15.28:34256) 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 -- 172.21.15.28:0/956933530 learned_addr learned my addr 172.21.15.28:0/956933530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:54.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cce00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1c809b980 0x7ff1c815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1c7e00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 0x7ff1c815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 -- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1c809b980 msgr2=0x7ff1c815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1c809b980 0x7ff1c815f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 -- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 msgr2=0x7ff1c815b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 0x7ff1c815b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 -- 172.21.15.28:0/956933530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c80cb3f0 con 0x7ff1c815cdb0 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1c7e00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 0x7ff1c815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.824+0000 7ff1cd800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b9c0 tx=0x7ff1c000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.825+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0019070 con 0x7ff1c815cdb0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.825+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c000cba0 con 0x7ff1c815cdb0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.825+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0014440 con 0x7ff1c815cdb0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.825+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c815eb80 con 0x7ff1c815cdb0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.825+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c815f140 con 0x7ff1c815cdb0 2024-09-10T10:45:54.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.826+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c80d3190 con 0x7ff1c815cdb0 2024-09-10T10:45:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.827+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1c001c070 con 0x7ff1c815cdb0 2024-09-10T10:45:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.828+0000 7ff1c4c00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.828+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7ff1c008ae90 con 0x7ff1c815cdb0 2024-09-10T10:45:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.828+0000 7ff1cce00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.829+0000 7ff1cce00640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 0x7ff1a806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc034000 tx=0x7ff1bc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:54.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:54.832+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff1c0058ea0 con 0x7ff1c815cdb0 2024-09-10T10:45:55.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.019+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c80b8320 con 0x7ff1c815cdb0 2024-09-10T10:45:55.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.021+0000 7ff1c4c00640 1 -- 172.21.15.28:0/956933530 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+17251 (secure 0 0 0) 0x7ff1c008c020 con 0x7ff1c815cdb0 2024-09-10T10:45:55.023 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:55.023 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":45,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:45:27.660574+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:53.486429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:54.688726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2d","mappings":[{"from":4,"to":2}]},{"pgid":"6.30","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/3203683106":"2024-09-11T10:45:10.614652+0000","172.21.15.28:6834/3203683106":"2024-09-11T10:45:10.614652+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:55.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 msgr2=0x7ff1a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 0x7ff1a806e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc034000 tx=0x7ff1bc02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 msgr2=0x7ff1c80cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff1c000b9c0 tx=0x7ff1c000be80 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 shutdown_connections 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1a806be30 0x7ff1a806e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1c815cdb0 0x7ff1c80cb910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c815c490 0x7ff1c815b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 --2- 172.21.15.28:0/956933530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1c809b980 0x7ff1c815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 >> 172.21.15.28:0/956933530 conn(0x7ff1c80d23a0 msgr2=0x7ff1c80b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 shutdown_connections 2024-09-10T10:45:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.023+0000 7ff1cf800640 1 -- 172.21.15.28:0/956933530 wait complete. 2024-09-10T10:45:55.042 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:45:55.042 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:55.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.195+0000 7feb73200640 1 Processor -- start 2024-09-10T10:45:55.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.195+0000 7feb73200640 1 -- start start 2024-09-10T10:45:55.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb73200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feb6c0b5f80 0x7feb6c0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb73200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb73200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c06fdb0 con 0x7feb6c0afd70 2024-09-10T10:45:55.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb73200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c0b6910 con 0x7feb6c0b5f80 2024-09-10T10:45:55.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb73200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c070090 con 0x7feb6c0cd650 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb71200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb71200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:48572/0 (socket says 172.21.15.28:48572) 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb71200640 1 -- 172.21.15.28:0/493122472 learned_addr learned my addr 172.21.15.28:0/493122472 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.197+0000 7feb63400640 1 --2- 172.21.15.28:0/493122472 >> v2:172.21.15.138:3300/0 conn(0x7feb6c0b5f80 0x7feb6c0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.198+0000 7feb71200640 1 -- 172.21.15.28:0/493122472 >> v2:172.21.15.138:3300/0 conn(0x7feb6c0b5f80 msgr2=0x7feb6c0b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.198+0000 7feb71200640 1 --2- 172.21.15.28:0/493122472 >> v2:172.21.15.138:3300/0 conn(0x7feb6c0b5f80 0x7feb6c0b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.198+0000 7feb71200640 1 -- 172.21.15.28:0/493122472 >> v1:172.21.15.138:6789/0 conn(0x7feb6c0afd70 legacy=0x7feb6c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.198+0000 7feb71200640 1 -- 172.21.15.28:0/493122472 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c070370 con 0x7feb6c0cd650 2024-09-10T10:45:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.198+0000 7feb6b400640 1 -- 172.21.15.28:0/493122472 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838857171 0 0) 0x7feb6c06fdb0 con 0x7feb6c0afd70 2024-09-10T10:45:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.199+0000 7feb71200640 1 --2- 172.21.15.28:0/493122472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb6402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5012a98cb26169e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.199+0000 7feb6b400640 1 -- 172.21.15.28:0/493122472 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64036070 con 0x7feb6c0cd650 2024-09-10T10:45:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.199+0000 7feb6b400640 1 -- 172.21.15.28:0/493122472 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb64031600 con 0x7feb6c0cd650 2024-09-10T10:45:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.199+0000 7feb6b400640 1 -- 172.21.15.28:0/493122472 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640319d0 con 0x7feb6c0cd650 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 -- 172.21.15.28:0/493122472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 msgr2=0x7feb6c0cdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 --2- 172.21.15.28:0/493122472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb6402a710 comp rx=0 tx=0).stop 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 -- 172.21.15.28:0/493122472 shutdown_connections 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 --2- 172.21.15.28:0/493122472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0cdab0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 --2- 172.21.15.28:0/493122472 >> v2:172.21.15.138:3300/0 conn(0x7feb6c0b5f80 0x7feb6c0b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 -- 172.21.15.28:0/493122472 >> 172.21.15.28:0/493122472 conn(0x7feb6c06bc10 msgr2=0x7feb6c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 -- 172.21.15.28:0/493122472 shutdown_connections 2024-09-10T10:45:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.200+0000 7feb73200640 1 -- 172.21.15.28:0/493122472 wait complete. 2024-09-10T10:45:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.201+0000 7feb73200640 1 Processor -- start 2024-09-10T10:45:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 -- start start 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c0b5f80 0x7feb6c142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c0b0150 con 0x7feb6c0afd70 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c06fa80 con 0x7feb6c0b5f80 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.202+0000 7feb73200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c160060 con 0x7feb6c0cd650 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34268/0 (socket says 172.21.15.28:34268) 2024-09-10T10:45:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 -- 172.21.15.28:0/3921220553 learned_addr learned my addr 172.21.15.28:0/3921220553 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:55.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb71200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb63400640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c0b5f80 0x7feb6c142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 -- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 msgr2=0x7feb6c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 -- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c0b5f80 msgr2=0x7feb6c142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c0b5f80 0x7feb6c142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.203+0000 7feb6be00640 1 -- 172.21.15.28:0/3921220553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0b0c80 con 0x7feb6c0afd70 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb6be00640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c4b0 tx=0x7feb5c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c017070 con 0x7feb6c0afd70 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5c005480 con 0x7feb6c0afd70 2024-09-10T10:45:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c0b0f60 con 0x7feb6c0afd70 2024-09-10T10:45:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c003830 con 0x7feb6c0afd70 2024-09-10T10:45:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.204+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c0b1520 con 0x7feb6c0afd70 2024-09-10T10:45:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.205+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c0b63d0 con 0x7feb6c0afd70 2024-09-10T10:45:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.206+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feb5c01a070 con 0x7feb6c0afd70 2024-09-10T10:45:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.207+0000 7feb68c00640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 0x7feb4806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.207+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7feb5c08a420 con 0x7feb6c0afd70 2024-09-10T10:45:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.207+0000 7feb63400640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 0x7feb4806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.208+0000 7feb63400640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 0x7feb4806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b2280 tx=0x7feb54009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.211+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb5c058550 con 0x7feb6c0afd70 2024-09-10T10:45:55.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.434+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb6c067fc0 con 0x7feb6c0afd70 2024-09-10T10:45:55.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.434+0000 7feb68c00640 1 -- 172.21.15.28:0/3921220553 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7feb5c05c850 con 0x7feb6c0afd70 2024-09-10T10:45:55.436 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:55.437 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:55.437 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.436+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 msgr2=0x7feb4806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.436+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 0x7feb4806e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b2280 tx=0x7feb54009190 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.436+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 msgr2=0x7feb6c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.436+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c4b0 tx=0x7feb5c00c970 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 shutdown_connections 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feb4806be30 0x7feb4806e2f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feb6c0cd650 0x7feb6c0b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb6c0b5f80 0x7feb6c142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 --2- 172.21.15.28:0/3921220553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb6c0afd70 0x7feb6c145480 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 >> 172.21.15.28:0/3921220553 conn(0x7feb6c06bc10 msgr2=0x7feb6c152890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 shutdown_connections 2024-09-10T10:45:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.437+0000 7feb73200640 1 -- 172.21.15.28:0/3921220553 wait complete. 2024-09-10T10:45:55.454 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.608+0000 7fd052000640 1 Processor -- start 2024-09-10T10:45:55.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.608+0000 7fd052000640 1 -- start start 2024-09-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd04c0d4180 0x7fd04c0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c15e6a0 0x7fd04c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd04c06dd60 0x7fd04c0d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c0cefb0 con 0x7fd04c06dd60 2024-09-10T10:45:55.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c0d4aa0 con 0x7fd04c0d4180 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd052000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c0d2c20 con 0x7fd04c15e6a0 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd051600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd04c06dd60 0x7fd04c0d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd051600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd04c06dd60 0x7fd04c0d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34276/0 (socket says 172.21.15.28:34276) 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd051600640 1 -- 172.21.15.28:0/3864062087 learned_addr learned my addr 172.21.15.28:0/3864062087 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd050c00640 1 --2- 172.21.15.28:0/3864062087 >> v2:172.21.15.138:3300/0 conn(0x7fd04c0d4180 0x7fd04c0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.609+0000 7fd04be00640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c15e6a0 0x7fd04c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.610+0000 7fd051600640 1 -- 172.21.15.28:0/3864062087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c15e6a0 msgr2=0x7fd04c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.610+0000 7fd051600640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c15e6a0 0x7fd04c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.610+0000 7fd051600640 1 -- 172.21.15.28:0/3864062087 >> v2:172.21.15.138:3300/0 conn(0x7fd04c0d4180 msgr2=0x7fd04c0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.610+0000 7fd051600640 1 --2- 172.21.15.28:0/3864062087 >> v2:172.21.15.138:3300/0 conn(0x7fd04c0d4180 0x7fd04c0d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.610+0000 7fd051600640 1 -- 172.21.15.28:0/3864062087 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c06e390 con 0x7fd04c06dd60 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.611+0000 7fd051600640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c06dd60 0x7fd04c0d0d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0b6ea0 tx=0x7fd04402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a90e301520bbfd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.611+0000 7fd04b400640 1 -- 172.21.15.28:0/3864062087 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044036070 con 0x7fd04c06dd60 2024-09-10T10:45:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.611+0000 7fd04b400640 1 -- 172.21.15.28:0/3864062087 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044031600 con 0x7fd04c06dd60 2024-09-10T10:45:55.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.611+0000 7fd04b400640 1 -- 172.21.15.28:0/3864062087 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440319d0 con 0x7fd04c06dd60 2024-09-10T10:45:55.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 -- 172.21.15.28:0/3864062087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c06dd60 msgr2=0x7fd04c0d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c06dd60 0x7fd04c0d0d40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0b6ea0 tx=0x7fd04402a610 comp rx=0 tx=0).stop 2024-09-10T10:45:55.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 -- 172.21.15.28:0/3864062087 shutdown_connections 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c06dd60 0x7fd04c0d0d40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 --2- 172.21.15.28:0/3864062087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c15e6a0 0x7fd04c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 --2- 172.21.15.28:0/3864062087 >> v2:172.21.15.138:3300/0 conn(0x7fd04c0d4180 0x7fd04c0d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 -- 172.21.15.28:0/3864062087 >> 172.21.15.28:0/3864062087 conn(0x7fd04c09d6f0 msgr2=0x7fd04c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 -- 172.21.15.28:0/3864062087 shutdown_connections 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.612+0000 7fd052000640 1 -- 172.21.15.28:0/3864062087 wait complete. 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.613+0000 7fd052000640 1 Processor -- start 2024-09-10T10:45:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 -- start start 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd04c06dd60 0x7fd04c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c0d4180 0x7fd04c15b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c0cebd0 con 0x7fd04c15e6a0 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06e950 con 0x7fd04c06dd60 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.614+0000 7fd052000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c0d1280 con 0x7fd04c0d4180 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34284/0 (socket says 172.21.15.28:34284) 2024-09-10T10:45:55.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 -- 172.21.15.28:0/299707882 learned_addr learned my addr 172.21.15.28:0/299707882 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:55.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd050c00640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd04c06dd60 0x7fd04c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd04be00640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c0d4180 0x7fd04c15b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 -- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c0d4180 msgr2=0x7fd04c15b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c0d4180 0x7fd04c15b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 -- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd04c06dd60 msgr2=0x7fd04c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd04c06dd60 0x7fd04c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.615+0000 7fd051600640 1 -- 172.21.15.28:0/299707882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c15bd00 con 0x7fd04c15e6a0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd051600640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd044034040 tx=0x7fd044004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044036050 con 0x7fd04c15e6a0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd044042070 con 0x7fd04c15e6a0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044009510 con 0x7fd04c15e6a0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c0cabd0 con 0x7fd04c15e6a0 2024-09-10T10:45:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.616+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c15bfa0 con 0x7fd04c15e6a0 2024-09-10T10:45:55.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.617+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c0d4560 con 0x7fd04c15e6a0 2024-09-10T10:45:55.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.618+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd044007bf0 con 0x7fd04c15e6a0 2024-09-10T10:45:55.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.619+0000 7fd048c00640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 0x7fd02c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:55.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.619+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7fd0440abc90 con 0x7fd04c15e6a0 2024-09-10T10:45:55.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.619+0000 7fd050c00640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 0x7fd02c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:55.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.620+0000 7fd050c00640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 0x7fd02c06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd040009ac0 tx=0x7fd040009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:55.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.623+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd04c0cabd0 con 0x7fd04c15e6a0 2024-09-10T10:45:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.846+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04c15c170 con 0x7fd04c15e6a0 2024-09-10T10:45:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.847+0000 7fd048c00640 1 -- 172.21.15.28:0/299707882 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5230 (secure 0 0 0) 0x7fd044079e60 con 0x7fd04c15e6a0 2024-09-10T10:45:55.849 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:55.849 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"btime":"2024-09-10T10:45:14:620103+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:44:55.704429+0000","modified":"2024-09-10T10:45:14.620095+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5088},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5088":{"gid":5088,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/1058548427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1058548427},{"type":"v1","addr":"172.21.15.28:6835","nonce":1058548427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5088,"qdb_cluster":[5088]},"id":2}]} 2024-09-10T10:45:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 23 2024-09-10T10:45:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 msgr2=0x7fd02c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 0x7fd02c06e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd040009ac0 tx=0x7fd040009190 comp rx=0 tx=0).stop 2024-09-10T10:45:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 msgr2=0x7fd04c0ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd044034040 tx=0x7fd044004a40 comp rx=0 tx=0).stop 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 shutdown_connections 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd02c06be30 0x7fd02c06e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd04c15e6a0 0x7fd04c0ca120 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd04c0d4180 0x7fd04c15b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 --2- 172.21.15.28:0/299707882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd04c06dd60 0x7fd04c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 >> 172.21.15.28:0/299707882 conn(0x7fd04c09d6f0 msgr2=0x7fd04c0b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 shutdown_connections 2024-09-10T10:45:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:55.849+0000 7fd052000640 1 -- 172.21.15.28:0/299707882 wait complete. 2024-09-10T10:45:55.867 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:45:56.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.021+0000 7f6943000640 1 Processor -- start 2024-09-10T10:45:56.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.021+0000 7f6943000640 1 -- start start 2024-09-10T10:45:56.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.022+0000 7f6943000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f693c15b300 0x7f693c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.022+0000 7f6943000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0cdbe0 con 0x7f693c06f8d0 2024-09-10T10:45:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.022+0000 7f6943000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f693c0cd900 con 0x7f693c0cd3f0 2024-09-10T10:45:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.022+0000 7f6943000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f693c0728a0 con 0x7f693c15b300 2024-09-10T10:45:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f6941000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f693c06f8d0 0x7f693c15b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51980/0 (socket says 172.21.15.28:51980) 2024-09-10T10:45:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f6941000640 1 -- 172.21.15.28:0/919179740 learned_addr learned my addr 172.21.15.28:0/919179740 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693be00640 1 --2- 172.21.15.28:0/919179740 >> v2:172.21.15.138:3300/0 conn(0x7f693c15b300 0x7f693c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 349065189 0 0) 0x7f693c0cdbe0 con 0x7f693c06f8d0 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6920003520 con 0x7f693c06f8d0 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693be00640 1 -- 172.21.15.28:0/919179740 >> v1:172.21.15.138:6789/0 conn(0x7f693c0cd3f0 legacy=0x7f693c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933740928 0 0) 0x7f693c0cd900 con 0x7f693c0cd3f0 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.023+0000 7f693be00640 1 -- 172.21.15.28:0/919179740 >> v1:172.21.15.28:6789/0 conn(0x7f693c06f8d0 legacy=0x7f693c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.024+0000 7f693be00640 1 -- 172.21.15.28:0/919179740 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c072250 con 0x7f693c15b300 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f693be00640 1 --2- 172.21.15.28:0/919179740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c15b300 0x7f693c0cceb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692c0097a0 tx=0x7f692c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ec456f5dd79d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c037070 con 0x7f693c15b300 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c02ac30 con 0x7f693c15b300 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f693b400640 1 -- 172.21.15.28:0/919179740 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69340019f0 con 0x7f693c15b300 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f6943000640 1 -- 172.21.15.28:0/919179740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c15b300 msgr2=0x7f693c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.025+0000 7f6943000640 1 --2- 172.21.15.28:0/919179740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c15b300 0x7f693c0cceb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f692c0097a0 tx=0x7f692c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.026+0000 7f6943000640 1 -- 172.21.15.28:0/919179740 shutdown_connections 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.026+0000 7f6943000640 1 --2- 172.21.15.28:0/919179740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c15b300 0x7f693c0cceb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.026+0000 7f6943000640 1 -- 172.21.15.28:0/919179740 >> 172.21.15.28:0/919179740 conn(0x7f693c067d70 msgr2=0x7f693c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.026+0000 7f6943000640 1 -- 172.21.15.28:0/919179740 shutdown_connections 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.026+0000 7f6943000640 1 -- 172.21.15.28:0/919179740 wait complete. 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.027+0000 7f6943000640 1 Processor -- start 2024-09-10T10:45:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.027+0000 7f6943000640 1 -- start start 2024-09-10T10:45:56.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c06f8d0 0x7f693c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 0x7f693c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0d0160 con 0x7f693c0cd3f0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f693c0cffe0 con 0x7f693c06f8d0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6943000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f693c072d30 con 0x7f693c15b300 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.028+0000 7f6933e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34286/0 (socket says 172.21.15.28:34286) 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 -- 172.21.15.28:0/3949355346 learned_addr learned my addr 172.21.15.28:0/3949355346 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f693be00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c06f8d0 0x7f693c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6941000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 0x7f693c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 -- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 msgr2=0x7f693c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 0x7f693c157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 -- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c06f8d0 msgr2=0x7f693c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c06f8d0 0x7f693c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 -- 172.21.15.28:0/3949355346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693c1e7160 con 0x7f693c0cd3f0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6941000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 0x7f693c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6933e00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f692400c4b0 tx=0x7f692400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924017070 con 0x7f693c0cd3f0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924005480 con 0x7f693c0cd3f0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693c1e7440 con 0x7f693c0cd3f0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.029+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924003830 con 0x7f693c0cd3f0 2024-09-10T10:45:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.030+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693c155a40 con 0x7f693c0cd3f0 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.031+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693c0aef40 con 0x7f693c0cd3f0 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.032+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f692401a070 con 0x7f693c0cd3f0 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.032+0000 7f6938c00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 0x7f691806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.032+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7f692408a510 con 0x7f693c0cd3f0 2024-09-10T10:45:56.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.033+0000 7f693be00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 0x7f691806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.034+0000 7f693be00640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 0x7f691806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f692c034000 tx=0x7f692c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.037+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6924058c00 con 0x7f693c0cd3f0 2024-09-10T10:45:56.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.225+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f693c15c390 con 0x7f693c0cd3f0 2024-09-10T10:45:56.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.226+0000 7f6938c00640 1 -- 172.21.15.28:0/3949355346 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+17251 (secure 0 0 0) 0x7f6924058500 con 0x7f693c0cd3f0 2024-09-10T10:45:56.228 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:45:56.229 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":45,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:45:27.660574+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:44:53.486429+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-10T10:44:54.688726+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2d","mappings":[{"from":4,"to":2}]},{"pgid":"6.30","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/3203683106":"2024-09-11T10:45:10.614652+0000","172.21.15.28:6834/3203683106":"2024-09-11T10:45:10.614652+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:45:56.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 msgr2=0x7f691806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 0x7f691806e2f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f692c034000 tx=0x7f692c02e000 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 msgr2=0x7f693c157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f692400c4b0 tx=0x7f692400c970 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 shutdown_connections 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f691806be30 0x7f691806e2f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f693c15b300 0x7f693c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f693c0cd3f0 0x7f693c157240 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 --2- 172.21.15.28:0/3949355346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f693c06f8d0 0x7f693c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 >> 172.21.15.28:0/3949355346 conn(0x7f693c067d70 msgr2=0x7f693c1e7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 shutdown_connections 2024-09-10T10:45:56.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.228+0000 7f6943000640 1 -- 172.21.15.28:0/3949355346 wait complete. 2024-09-10T10:45:56.246 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:45:56.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.400+0000 7fdceaa00640 1 Processor -- start 2024-09-10T10:45:56.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.400+0000 7fdceaa00640 1 -- start start 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdceaa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdce4072320 0x7fdce40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdceaa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 0x7fdce406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdceaa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdce4146140 con 0x7fdce4072de0 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdceaa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdce40cd740 con 0x7fdce4072320 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdceaa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdce40cda20 con 0x7fdce415e1f0 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdce4072320 0x7fdce40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdce4072de0 0x7fdce406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51994/0 (socket says 172.21.15.28:51994) 2024-09-10T10:45:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdce4072320 0x7fdce40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34290/0 (socket says 172.21.15.28:34290) 2024-09-10T10:45:56.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3400640 1 -- 172.21.15.28:0/1137546778 learned_addr learned my addr 172.21.15.28:0/1137546778 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 -- 172.21.15.28:0/1137546778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 msgr2=0x7fdce406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce8a00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 0x7fdce406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 0x7fdce406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 -- 172.21.15.28:0/1137546778 >> v1:172.21.15.28:6789/0 conn(0x7fdce4072de0 legacy=0x7fdce406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce2a00640 1 -- 172.21.15.28:0/1137546778 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892980650 0 0) 0x7fdce4146140 con 0x7fdce4072de0 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.402+0000 7fdce3e00640 1 -- 172.21.15.28:0/1137546778 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce40cdd00 con 0x7fdce4072320 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.403+0000 7fdce8a00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 0x7fdce406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.403+0000 7fdce3e00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce4072320 0x7fdce40cce60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40097a0 tx=0x7fdcd402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3943d464434b9097 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdce2a00640 1 -- 172.21.15.28:0/1137546778 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4037070 con 0x7fdce4072320 2024-09-10T10:45:56.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdce2a00640 1 -- 172.21.15.28:0/1137546778 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd402ae50 con 0x7fdce4072320 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdce2a00640 1 -- 172.21.15.28:0/1137546778 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4032660 con 0x7fdce4072320 2024-09-10T10:45:56.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 -- 172.21.15.28:0/1137546778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce4072320 msgr2=0x7fdce40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce4072320 0x7fdce40cce60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40097a0 tx=0x7fdcd402a100 comp rx=0 tx=0).stop 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 -- 172.21.15.28:0/1137546778 shutdown_connections 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce415e1f0 0x7fdce406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 --2- 172.21.15.28:0/1137546778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce4072320 0x7fdce40cce60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 -- 172.21.15.28:0/1137546778 >> 172.21.15.28:0/1137546778 conn(0x7fdce4067d70 msgr2=0x7fdce406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 -- 172.21.15.28:0/1137546778 shutdown_connections 2024-09-10T10:45:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.404+0000 7fdceaa00640 1 -- 172.21.15.28:0/1137546778 wait complete. 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.406+0000 7fdceaa00640 1 Processor -- start 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.406+0000 7fdceaa00640 1 -- start start 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdce4072320 0x7fdce4142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce4072de0 0x7fdce40cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdce40cd520 con 0x7fdce415e1f0 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdce40cd3a0 con 0x7fdce4072320 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdceaa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdce406ed00 con 0x7fdce4072de0 2024-09-10T10:45:56.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdce8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdce8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34302/0 (socket says 172.21.15.28:34302) 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.407+0000 7fdce8a00640 1 -- 172.21.15.28:0/411031143 learned_addr learned my addr 172.21.15.28:0/411031143 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce3e00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdce4072320 0x7fdce4142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce3400640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce4072de0 0x7fdce40cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 -- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce4072de0 msgr2=0x7fdce40cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce4072de0 0x7fdce40cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 -- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdce4072320 msgr2=0x7fdce4142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdce4072320 0x7fdce4142d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 -- 172.21.15.28:0/411031143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce40b41f0 con 0x7fdce415e1f0 2024-09-10T10:45:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdce8a00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00cd80 tx=0x7fdcdc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00f680 con 0x7fdce415e1f0 2024-09-10T10:45:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc00f840 con 0x7fdce415e1f0 2024-09-10T10:45:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00fc30 con 0x7fdce415e1f0 2024-09-10T10:45:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.408+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce40b44d0 con 0x7fdce415e1f0 2024-09-10T10:45:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.409+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce40b4a90 con 0x7fdce415e1f0 2024-09-10T10:45:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.410+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce40b4c90 con 0x7fdce415e1f0 2024-09-10T10:45:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.410+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcdc0213f0 con 0x7fdce415e1f0 2024-09-10T10:45:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.411+0000 7fdcdbe00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 0x7fdcc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.411+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6245+0+0 (secure 0 0 0) 0x7fdcdc08be00 con 0x7fdce415e1f0 2024-09-10T10:45:56.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.412+0000 7fdce3e00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 0x7fdcc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.413+0000 7fdce3e00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 0x7fdcc406e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4034000 tx=0x7fdcd402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.416+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdce40b4c90 con 0x7fdce415e1f0 2024-09-10T10:45:56.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.634+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdce4072700 con 0x7fdce415e1f0 2024-09-10T10:45:56.637 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:45:56.635+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:45:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.638+0000 7fdcdbe00640 1 -- 172.21.15.28:0/411031143 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fdcdc014030 con 0x7fdce415e1f0 2024-09-10T10:45:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:45:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 msgr2=0x7fdcc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 0x7fdcc406e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4034000 tx=0x7fdcd402e000 comp rx=0 tx=0).stop 2024-09-10T10:45:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 msgr2=0x7fdce40cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00cd80 tx=0x7fdcdc00a650 comp rx=0 tx=0).stop 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 shutdown_connections 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcc406be30 0x7fdcc406e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdce415e1f0 0x7fdce40cc700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdce4072de0 0x7fdce40cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 --2- 172.21.15.28:0/411031143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdce4072320 0x7fdce4142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 >> 172.21.15.28:0/411031143 conn(0x7fdce4067d70 msgr2=0x7fdce40b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 shutdown_connections 2024-09-10T10:45:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.641+0000 7fdceaa00640 1 -- 172.21.15.28:0/411031143 wait complete. 2024-09-10T10:45:56.650 INFO:tasks.ceph.mds.a.smithi028.stderr: -3442> 2024-09-10T10:45:16.611+0000 7f8f76200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:45:56.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:45:56.700 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:45:56.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.818+0000 7f834cc00640 1 Processor -- start 2024-09-10T10:45:56.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.819+0000 7f834cc00640 1 -- start start 2024-09-10T10:45:56.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f834cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f834815e6c0 0x7f83480cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f834cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 0x7f83480b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f834cc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83480d34f0 con 0x7f83480d5090 2024-09-10T10:45:56.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f834cc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f83480cf400 con 0x7f834815e6c0 2024-09-10T10:45:56.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f834cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83480b10d0 con 0x7f83480cfcb0 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f834815e6c0 0x7f83480cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f833d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83480d5090 0x7f83480cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52004/0 (socket says 172.21.15.28:52004) 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f833d400640 1 -- 172.21.15.28:0/221872236 learned_addr learned my addr 172.21.15.28:0/221872236 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8346800640 1 --2- 172.21.15.28:0/221872236 >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 0x7f83480b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345e00640 1 -- 172.21.15.28:0/221872236 >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 msgr2=0x7f83480b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345e00640 1 --2- 172.21.15.28:0/221872236 >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 0x7f83480b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345e00640 1 -- 172.21.15.28:0/221872236 >> v1:172.21.15.28:6789/0 conn(0x7f83480d5090 legacy=0x7f83480cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345e00640 1 -- 172.21.15.28:0/221872236 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83480d3790 con 0x7f834815e6c0 2024-09-10T10:45:56.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8345400640 1 -- 172.21.15.28:0/221872236 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182654484 0 0) 0x7f83480d34f0 con 0x7f83480d5090 2024-09-10T10:45:56.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.820+0000 7f8346800640 1 --2- 172.21.15.28:0/221872236 >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 0x7f83480b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.821+0000 7f8345e00640 1 --2- 172.21.15.28:0/221872236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f834815e6c0 0x7f83480cebb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83480d4500 tx=0x7f833802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f596048cc1c9881 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f8345400640 1 -- 172.21.15.28:0/221872236 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338037070 con 0x7f834815e6c0 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f8345400640 1 -- 172.21.15.28:0/221872236 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8338032440 con 0x7f834815e6c0 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f8345400640 1 -- 172.21.15.28:0/221872236 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833802ae60 con 0x7f834815e6c0 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 -- 172.21.15.28:0/221872236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f834815e6c0 msgr2=0x7f83480cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 --2- 172.21.15.28:0/221872236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f834815e6c0 0x7f83480cebb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83480d4500 tx=0x7f833802a110 comp rx=0 tx=0).stop 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 -- 172.21.15.28:0/221872236 shutdown_connections 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 --2- 172.21.15.28:0/221872236 >> v2:172.21.15.138:3300/0 conn(0x7f83480cfcb0 0x7f83480b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 --2- 172.21.15.28:0/221872236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f834815e6c0 0x7f83480cebb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 -- 172.21.15.28:0/221872236 >> 172.21.15.28:0/221872236 conn(0x7f834808bfc0 msgr2=0x7f83480d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 -- 172.21.15.28:0/221872236 shutdown_connections 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.822+0000 7f834cc00640 1 -- 172.21.15.28:0/221872236 wait complete. 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.824+0000 7f834cc00640 1 Processor -- start 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.824+0000 7f834cc00640 1 -- start start 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 0x7f83481ea960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834815e6c0 0x7f834806de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f834815eaa0 con 0x7f83480d5090 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83480cf0f0 con 0x7f83480cfcb0 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f834cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f834815ec20 con 0x7f834815e6c0 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.825+0000 7f833d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34334/0 (socket says 172.21.15.28:34334) 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 -- 172.21.15.28:0/3320102633 learned_addr learned my addr 172.21.15.28:0/3320102633 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f8345e00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 0x7f83481ea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f8346800640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834815e6c0 0x7f834806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 -- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834815e6c0 msgr2=0x7f834806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834815e6c0 0x7f834806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 -- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 msgr2=0x7f83481ea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 0x7f83481ea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:56.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 -- 172.21.15.28:0/3320102633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481eaea0 con 0x7f83480d5090 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f8345e00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 0x7f83481ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833d400640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f833000c400 tx=0x7f833000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.826+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330017070 con 0x7f83480d5090 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.827+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8330005ce0 con 0x7f83480d5090 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.827+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330007310 con 0x7f83480d5090 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.827+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83480b75e0 con 0x7f83480d5090 2024-09-10T10:45:56.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.827+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83480b7ba0 con 0x7f83480d5090 2024-09-10T10:45:56.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.828+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83480b6d30 con 0x7f83480d5090 2024-09-10T10:45:56.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.829+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f833001a070 con 0x7f83480d5090 2024-09-10T10:45:56.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.830+0000 7f833ea00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 0x7f832406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:56.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.830+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6331+0+0 (secure 0 0 0) 0x7f833008a490 con 0x7f83480d5090 2024-09-10T10:45:56.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.830+0000 7f8345e00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 0x7f832406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:56.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.831+0000 7f8345e00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 0x7f832406e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8338034000 tx=0x7f833802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:56.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:56.835+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8330058570 con 0x7f83480d5090 2024-09-10T10:45:57.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.059+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f83480d22f0 con 0x7f83480d5090 2024-09-10T10:45:57.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.640+0000 7f833ea00640 1 -- 172.21.15.28:0/3320102633 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f833005c870 con 0x7f83480d5090 2024-09-10T10:45:57.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.643+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 msgr2=0x7f832406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.643+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 0x7f832406e320 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8338034000 tx=0x7f833802e000 comp rx=0 tx=0).stop 2024-09-10T10:45:57.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.643+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 msgr2=0x7f8348153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.643+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f833000c400 tx=0x7f833000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 shutdown_connections 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832406be60 0x7f832406e320 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f834815e6c0 0x7f834806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83480d5090 0x7f8348153c10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 --2- 172.21.15.28:0/3320102633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83480cfcb0 0x7f83481ea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 >> 172.21.15.28:0/3320102633 conn(0x7f834808bfc0 msgr2=0x7f83480b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 shutdown_connections 2024-09-10T10:45:57.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.644+0000 7f834cc00640 1 -- 172.21.15.28:0/3320102633 wait complete. 2024-09-10T10:45:57.663 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:45:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.866+0000 7f3558a00640 1 Processor -- start 2024-09-10T10:45:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.866+0000 7f3558a00640 1 -- start start 2024-09-10T10:45:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3558a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35540d2950 0x7f35540d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3558a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35540b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3558a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35540b1030 con 0x7f35540b56a0 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3558a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f35540b1c00 con 0x7f35540d2950 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3558a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35540b12d0 con 0x7f35540d2d30 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3552a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f35540b56a0 0x7f35540b3c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52006/0 (socket says 172.21.15.28:52006) 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3552a00640 1 -- 172.21.15.28:0/2476813661 learned_addr learned my addr 172.21.15.28:0/2476813661 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.867+0000 7f3553400640 1 --2- 172.21.15.28:0/2476813661 >> v2:172.21.15.138:3300/0 conn(0x7f35540d2950 0x7f35540d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3553e00640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35540b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3552000640 1 -- 172.21.15.28:0/2476813661 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271894005 0 0) 0x7f35540b1030 con 0x7f35540b56a0 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3552000640 1 -- 172.21.15.28:0/2476813661 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3538003520 con 0x7f35540b56a0 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3553400640 1 -- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 msgr2=0x7f35540b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3553400640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35540b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3553400640 1 -- 172.21.15.28:0/2476813661 >> v1:172.21.15.28:6789/0 conn(0x7f35540b56a0 legacy=0x7f35540b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.868+0000 7f3553400640 1 -- 172.21.15.28:0/2476813661 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35540b15b0 con 0x7f35540d2950 2024-09-10T10:45:57.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.869+0000 7f3553400640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540d4c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35440097a0 tx=0x7f354402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4eeeb7e6fd002e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.869+0000 7f3552000640 1 -- 172.21.15.28:0/2476813661 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544037070 con 0x7f35540d2950 2024-09-10T10:45:57.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3552000640 1 -- 172.21.15.28:0/2476813661 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354402ae50 con 0x7f35540d2950 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3552000640 1 -- 172.21.15.28:0/2476813661 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544032660 con 0x7f35540d2950 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 -- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 msgr2=0x7f35540d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540d4c40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f35440097a0 tx=0x7f354402a100 comp rx=0 tx=0).stop 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 -- 172.21.15.28:0/2476813661 shutdown_connections 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35540b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 --2- 172.21.15.28:0/2476813661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540d4c40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 -- 172.21.15.28:0/2476813661 >> 172.21.15.28:0/2476813661 conn(0x7f355408bfc0 msgr2=0x7f35540d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 -- 172.21.15.28:0/2476813661 shutdown_connections 2024-09-10T10:45:57.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.870+0000 7f3558a00640 1 -- 172.21.15.28:0/2476813661 wait complete. 2024-09-10T10:45:57.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.872+0000 7f3558a00640 1 Processor -- start 2024-09-10T10:45:57.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.872+0000 7f3558a00640 1 -- start start 2024-09-10T10:45:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540b9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35541f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35540b48a0 con 0x7f35540b56a0 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35540d5180 con 0x7f35540d2950 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3558a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35540b4a20 con 0x7f35540d2d30 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34346/0 (socket says 172.21.15.28:34346) 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 -- 172.21.15.28:0/2249977782 learned_addr learned my addr 172.21.15.28:0/2249977782 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3552a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540b9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553e00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35541f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 -- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 msgr2=0x7f35541f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35541f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 -- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 msgr2=0x7f35540b9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540b9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553400640 1 -- 172.21.15.28:0/2249977782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35541f57c0 con 0x7f35540b56a0 2024-09-10T10:45:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.873+0000 7f3553e00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35541f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:45:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f3553400640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3544035040 tx=0x7f35440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544041070 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354403c460 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3544037040 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35541f4f00 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.874+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35540b76a0 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.875+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35540d4c40 con 0x7f35540b56a0 2024-09-10T10:45:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.876+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f354403a070 con 0x7f35540b56a0 2024-09-10T10:45:57.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.877+0000 7f354b400640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 0x7f353406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:57.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.877+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6331+0+0 (secure 0 0 0) 0x7f35440ab360 con 0x7f35540b56a0 2024-09-10T10:45:57.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.877+0000 7f3552a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 0x7f353406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:57.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.878+0000 7f3552a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 0x7f353406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f355406d8a0 tx=0x7f3540009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:57.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:57.881+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3544079a00 con 0x7f35540b56a0 2024-09-10T10:45:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.064+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35540ce830 con 0x7f35540b56a0 2024-09-10T10:45:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.644+0000 7f354b400640 1 -- 172.21.15.28:0/2249977782 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7f35440ab120 con 0x7f35540b56a0 2024-09-10T10:45:58.647 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 msgr2=0x7f353406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 0x7f353406e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f355406d8a0 tx=0x7f3540009040 comp rx=0 tx=0).stop 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 msgr2=0x7f35541f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3544035040 tx=0x7f35440046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 shutdown_connections 2024-09-10T10:45:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353406be60 0x7f353406e320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35540d2d30 0x7f35541f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35540d2950 0x7f35540b9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 --2- 172.21.15.28:0/2249977782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35540b56a0 0x7f35541f7100 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3544035040 tx=0x7f35440046c0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 >> 172.21.15.28:0/2249977782 conn(0x7f355408bfc0 msgr2=0x7f35540b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 shutdown_connections 2024-09-10T10:45:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.649+0000 7f3558a00640 1 -- 172.21.15.28:0/2249977782 wait complete. 2024-09-10T10:45:58.670 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:45:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.834+0000 7f2af3200640 1 Processor -- start 2024-09-10T10:45:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.834+0000 7f2af3200640 1 -- start start 2024-09-10T10:45:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec15e900 0x7f2aec0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2aec0b4a20 0x7f2aec0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0b4620 con 0x7f2aec072050 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec0ce100 con 0x7f2aec0b4a20 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0ce3a0 con 0x7f2aec15e900 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2aeb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2aec0b4a20 0x7f2aec0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2aeb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2aec0b4a20 0x7f2aec0ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34354/0 (socket says 172.21.15.28:34354) 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2aeb400640 1 -- 172.21.15.28:0/3622587342 learned_addr learned my addr 172.21.15.28:0/3622587342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.835+0000 7f2af1200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2aec072050 0x7f2aec0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52014/0 (socket says 172.21.15.28:52014) 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.836+0000 7f2aebe00640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec15e900 0x7f2aec0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.836+0000 7f2aeb400640 1 -- 172.21.15.28:0/3622587342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec15e900 msgr2=0x7f2aec0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.836+0000 7f2aeb400640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec15e900 0x7f2aec0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.836+0000 7f2aeb400640 1 -- 172.21.15.28:0/3622587342 >> v1:172.21.15.28:6789/0 conn(0x7f2aec072050 legacy=0x7f2aec0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.836+0000 7f2aeb400640 1 -- 172.21.15.28:0/3622587342 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0ce680 con 0x7f2aec0b4a20 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2aeb400640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec0b4a20 0x7f2aec0ad020 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009870 tx=0x7f2ad802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14788f381c2364d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2aeaa00640 1 -- 172.21.15.28:0/3622587342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8037070 con 0x7f2aec0b4a20 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2aeaa00640 1 -- 172.21.15.28:0/3622587342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ad8032440 con 0x7f2aec0b4a20 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2aeaa00640 1 -- 172.21.15.28:0/3622587342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8032810 con 0x7f2aec0b4a20 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2af3200640 1 -- 172.21.15.28:0/3622587342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec0b4a20 msgr2=0x7f2aec0ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.837+0000 7f2af3200640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec0b4a20 0x7f2aec0ad020 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009870 tx=0x7f2ad802a1d0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 -- 172.21.15.28:0/3622587342 shutdown_connections 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec0b4a20 0x7f2aec0ad020 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 --2- 172.21.15.28:0/3622587342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec15e900 0x7f2aec0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 -- 172.21.15.28:0/3622587342 >> 172.21.15.28:0/3622587342 conn(0x7f2aec06bdf0 msgr2=0x7f2aec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 -- 172.21.15.28:0/3622587342 shutdown_connections 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.838+0000 7f2af3200640 1 -- 172.21.15.28:0/3622587342 wait complete. 2024-09-10T10:45:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.839+0000 7f2af3200640 1 Processor -- start 2024-09-10T10:45:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.839+0000 7f2af3200640 1 -- start start 2024-09-10T10:45:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 0x7f2aec150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec0b4a20 0x7f2aec0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0ad560 con 0x7f2aec15e900 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0b4e20 con 0x7f2aec072050 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0cf590 con 0x7f2aec0b4a20 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.840+0000 7f2af1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34370/0 (socket says 172.21.15.28:34370) 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 -- 172.21.15.28:0/3762954842 learned_addr learned my addr 172.21.15.28:0/3762954842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2aebe00640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 0x7f2aec150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2aeb400640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec0b4a20 0x7f2aec0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 -- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec0b4a20 msgr2=0x7f2aec0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec0b4a20 0x7f2aec0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 -- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 msgr2=0x7f2aec150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 0x7f2aec150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 -- 172.21.15.28:0/3762954842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0c9cc0 con 0x7f2aec15e900 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2aebe00640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 0x7f2aec150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2af1200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c780 tx=0x7f2ae400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.841+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4017070 con 0x7f2aec15e900 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.842+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae400a640 con 0x7f2aec15e900 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.842+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400aa30 con 0x7f2aec15e900 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.842+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec0cb6a0 con 0x7f2aec15e900 2024-09-10T10:45:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.842+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1553b0 con 0x7f2aec15e900 2024-09-10T10:45:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.843+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec0ad020 con 0x7f2aec15e900 2024-09-10T10:45:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.844+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ae401a070 con 0x7f2aec15e900 2024-09-10T10:45:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.844+0000 7f2ae3e00640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 0x7f2acc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.845+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5893+0+0 (secure 0 0 0) 0x7f2ae408a440 con 0x7f2aec15e900 2024-09-10T10:45:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.845+0000 7f2aebe00640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 0x7f2acc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:58.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.846+0000 7f2aebe00640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 0x7f2acc06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006fd0 tx=0x7f2adc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:58.849+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ae40586e0 con 0x7f2aec15e900 2024-09-10T10:45:59.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.030+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2aec06ed90 con 0x7f2aec15e900 2024-09-10T10:45:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.650+0000 7f2ae3e00640 1 -- 172.21.15.28:0/3762954842 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7f2ae405c9e0 con 0x7f2aec15e900 2024-09-10T10:45:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:45:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 msgr2=0x7f2acc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 0x7f2acc06e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2adc006fd0 tx=0x7f2adc008040 comp rx=0 tx=0).stop 2024-09-10T10:45:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 msgr2=0x7f2aec06f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c780 tx=0x7f2ae400cc40 comp rx=0 tx=0).stop 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 shutdown_connections 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2acc06be30 0x7f2acc06e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2aec15e900 0x7f2aec06f590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2aec0b4a20 0x7f2aec0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 --2- 172.21.15.28:0/3762954842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2aec072050 0x7f2aec150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.653+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 >> 172.21.15.28:0/3762954842 conn(0x7f2aec06bdf0 msgr2=0x7f2aec15ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.656+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 shutdown_connections 2024-09-10T10:45:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.656+0000 7f2af3200640 1 -- 172.21.15.28:0/3762954842 wait complete. 2024-09-10T10:45:59.680 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:45:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.837+0000 7f105ec00640 1 Processor -- start 2024-09-10T10:45:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.838+0000 7f105ec00640 1 -- start start 2024-09-10T10:45:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f10580b3560 0x7f10580d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10580d08b0 0x7f105806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105ec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10580b39a0 con 0x7f105815e5e0 2024-09-10T10:45:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f105815eda0 con 0x7f10580d08b0 2024-09-10T10:45:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f105815f080 con 0x7f10580b3560 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10580d08b0 0x7f105806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10580d08b0 0x7f105806e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34378/0 (socket says 172.21.15.28:34378) 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 -- 172.21.15.28:0/3242606726 learned_addr learned my addr 172.21.15.28:0/3242606726 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 -- 172.21.15.28:0/3242606726 >> v2:172.21.15.138:3300/0 conn(0x7f10580b3560 msgr2=0x7f10580d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 --2- 172.21.15.28:0/3242606726 >> v2:172.21.15.138:3300/0 conn(0x7f10580b3560 0x7f10580d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 -- 172.21.15.28:0/3242606726 >> v1:172.21.15.138:6789/0 conn(0x7f105815e5e0 legacy=0x7f105806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:45:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.839+0000 7f105cc00640 1 -- 172.21.15.28:0/3242606726 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105815f360 con 0x7f10580d08b0 2024-09-10T10:45:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.840+0000 7f105cc00640 1 --2- 172.21.15.28:0/3242606726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f105806e1f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1058071790 tx=0x7f105002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25440a0716cd248c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f1057400640 1 -- 172.21.15.28:0/3242606726 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050036070 con 0x7f10580d08b0 2024-09-10T10:45:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f1057400640 1 -- 172.21.15.28:0/3242606726 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1050031600 con 0x7f10580d08b0 2024-09-10T10:45:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f1057400640 1 -- 172.21.15.28:0/3242606726 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10500319d0 con 0x7f10580d08b0 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 -- 172.21.15.28:0/3242606726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 msgr2=0x7f105806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 --2- 172.21.15.28:0/3242606726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f105806e1f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1058071790 tx=0x7f105002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 -- 172.21.15.28:0/3242606726 shutdown_connections 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 --2- 172.21.15.28:0/3242606726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f105806e1f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 --2- 172.21.15.28:0/3242606726 >> v2:172.21.15.138:3300/0 conn(0x7f10580b3560 0x7f10580d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 -- 172.21.15.28:0/3242606726 >> 172.21.15.28:0/3242606726 conn(0x7f105806bdf0 msgr2=0x7f105806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 -- 172.21.15.28:0/3242606726 shutdown_connections 2024-09-10T10:45:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.841+0000 7f105ec00640 1 -- 172.21.15.28:0/3242606726 wait complete. 2024-09-10T10:45:59.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.843+0000 7f105ec00640 1 Processor -- start 2024-09-10T10:45:59.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.843+0000 7f105ec00640 1 -- start start 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10580b3560 0x7f10580cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f105815e5e0 0x7f10580cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f105806fd00 con 0x7f10580d08b0 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f105815f920 con 0x7f105815e5e0 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f105815e9c0 con 0x7f10580b3560 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f104fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f104fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34394/0 (socket says 172.21.15.28:34394) 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f104fe00640 1 -- 172.21.15.28:0/1373000727 learned_addr learned my addr 172.21.15.28:0/1373000727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f1057e00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10580b3560 0x7f10580cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.844+0000 7f105cc00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f105815e5e0 0x7f10580cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 -- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10580b3560 msgr2=0x7f10580cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10580b3560 0x7f10580cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 -- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f105815e5e0 msgr2=0x7f10580cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f105815e5e0 0x7f10580cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 -- 172.21.15.28:0/1373000727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10580b1080 con 0x7f10580d08b0 2024-09-10T10:45:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f104fe00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f104000b440 tx=0x7f104000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040015020 con 0x7f10580d08b0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1040011890 con 0x7f10580d08b0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040011c80 con 0x7f10580d08b0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10580ccc10 con 0x7f10580d08b0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.845+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10580b0cc0 con 0x7f10580d08b0 2024-09-10T10:45:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.846+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1058161b30 con 0x7f10580d08b0 2024-09-10T10:45:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.847+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1040018020 con 0x7f10580d08b0 2024-09-10T10:45:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.848+0000 7f1054c00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 0x7f103406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:45:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.848+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f104008ade0 con 0x7f10580d08b0 2024-09-10T10:45:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.848+0000 7f1057e00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 0x7f103406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:45:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.849+0000 7f1057e00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 0x7f103406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10480017f0 tx=0x7f1048009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:45:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:45:59.853+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1058161b30 con 0x7f10580d08b0 2024-09-10T10:46:00.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.035+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f105806c690 con 0x7f10580d08b0 2024-09-10T10:46:00.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.039+0000 7f1054c00640 1 -- 172.21.15.28:0/1373000727 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f105806c690 con 0x7f10580d08b0 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 msgr2=0x7f103406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 0x7f103406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f10480017f0 tx=0x7f1048009040 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 msgr2=0x7f10580b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f104000b440 tx=0x7f104000b900 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 shutdown_connections 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f103406be30 0x7f103406e2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f105815e5e0 0x7f10580cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10580d08b0 0x7f10580b0780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 --2- 172.21.15.28:0/1373000727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f10580b3560 0x7f10580cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 >> 172.21.15.28:0/1373000727 conn(0x7f105806bdf0 msgr2=0x7f105815ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 shutdown_connections 2024-09-10T10:46:00.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.041+0000 7f105ec00640 1 -- 172.21.15.28:0/1373000727 wait complete. 2024-09-10T10:46:00.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry' 2024-09-10T10:46:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.213+0000 7f5acd000640 1 Processor -- start 2024-09-10T10:46:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.214+0000 7f5acd000640 1 -- start start 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 0x7f5ac80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac806f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80728a0 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80cdbe0 con 0x7f5ac815b300 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5acd000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac80cd900 con 0x7f5ac8072d30 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5abfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5abfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34400/0 (socket says 172.21.15.28:34400) 2024-09-10T10:46:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5abfe00640 1 -- 172.21.15.28:0/4292807774 learned_addr learned my addr 172.21.15.28:0/4292807774 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.215+0000 7f5ac5e00640 1 --2- 172.21.15.28:0/4292807774 >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 0x7f5ac80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5ac6800640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5abfe00640 1 -- 172.21.15.28:0/4292807774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 msgr2=0x7f5ac806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac806f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5abfe00640 1 -- 172.21.15.28:0/4292807774 >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 msgr2=0x7f5ac80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4292807774 >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 0x7f5ac80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5abfe00640 1 -- 172.21.15.28:0/4292807774 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac8072250 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.216+0000 7f5ac5e00640 1 --2- 172.21.15.28:0/4292807774 >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 0x7f5ac80cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:00.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.217+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8070250 tx=0x7f5ab002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e188b67626ffb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.217+0000 7f5ac5400640 1 -- 172.21.15.28:0/4292807774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0038070 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.217+0000 7f5ac5400640 1 -- 172.21.15.28:0/4292807774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0031710 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.217+0000 7f5ac5400640 1 -- 172.21.15.28:0/4292807774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0031ae0 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 -- 172.21.15.28:0/4292807774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 msgr2=0x7f5ac806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8070250 tx=0x7f5ab002c3f0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 -- 172.21.15.28:0/4292807774 shutdown_connections 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac806f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 --2- 172.21.15.28:0/4292807774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac806f1c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 --2- 172.21.15.28:0/4292807774 >> v2:172.21.15.138:3300/0 conn(0x7f5ac815b300 0x7f5ac80cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 -- 172.21.15.28:0/4292807774 >> 172.21.15.28:0/4292807774 conn(0x7f5ac8067d70 msgr2=0x7f5ac806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 -- 172.21.15.28:0/4292807774 shutdown_connections 2024-09-10T10:46:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.218+0000 7f5acd000640 1 -- 172.21.15.28:0/4292807774 wait complete. 2024-09-10T10:46:00.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.219+0000 7f5acd000640 1 Processor -- start 2024-09-10T10:46:00.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 -- start start 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac81f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac815b300 0x7f5ac80b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80d0160 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80cffe0 con 0x7f5ac815b300 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.220+0000 7f5acd000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac80b3c40 con 0x7f5ac8072d30 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34404/0 (socket says 172.21.15.28:34404) 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 -- 172.21.15.28:0/4139913435 learned_addr learned my addr 172.21.15.28:0/4139913435 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5ac5e00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac81f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5ac6800640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac815b300 0x7f5ac80b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 -- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 msgr2=0x7f5ac81f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac81f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 -- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac815b300 msgr2=0x7f5ac80b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac815b300 0x7f5ac80b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 -- 172.21.15.28:0/4139913435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80cbe30 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.221+0000 7f5abfe00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ab002c430 tx=0x7f5ab00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.222+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0038040 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.222+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab0033830 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.222+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0033c40 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.222+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac80b1190 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.222+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80cc3b0 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.223+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac80b2de0 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.224+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ab003e430 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.225+0000 7f5abe000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 0x7f5aa406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.225+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f5ab00ab750 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.225+0000 7f5ac5e00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 0x7f5aa406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.226+0000 7f5ac5e00640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 0x7f5aa406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8006fd0 tx=0x7f5ab8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.229+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ab0079c60 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.485+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]} v 0) -- 0x7f5ac8159730 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.653+0000 7f5abe000640 1 -- 172.21.15.28:0/4139913435 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_damaged_dentry"]}]=0 v131) ==== 134+0+0 (secure 0 0 0) 0x7f5ab007df60 con 0x7f5ac80cd3f0 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 msgr2=0x7f5aa406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 0x7f5aa406e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8006fd0 tx=0x7f5ab8008040 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 msgr2=0x7f5ac80cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5ab002c430 tx=0x7f5ab00046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 shutdown_connections 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5aa406be30 0x7f5aa406e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac815b300 0x7f5ac80b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ac80cd3f0 0x7f5ac80cb8f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 --2- 172.21.15.28:0/4139913435 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5ac8072d30 0x7f5ac81f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.654+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 >> 172.21.15.28:0/4139913435 conn(0x7f5ac8067d70 msgr2=0x7f5ac80b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.655+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 shutdown_connections 2024-09-10T10:46:00.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.655+0000 7f5acd000640 1 -- 172.21.15.28:0/4139913435 wait complete. 2024-09-10T10:46:00.673 INFO:tasks.cephfs_test_runner:test_damaged_dentry (tasks.cephfs.test_damage.TestDamage) ... ok 2024-09-10T10:46:00.673 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) That the MDS won't abort when the dentry is already known to be damaged. 2024-09-10T10:46:00.673 INFO:tasks.cephfs_test_runner:test_dentry_first_existing (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:46:00.674 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:00.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.828+0000 7f2edf000640 1 Processor -- start 2024-09-10T10:46:00.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.828+0000 7f2edf000640 1 -- start start 2024-09-10T10:46:00.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2edf000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed8169830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2edf000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed806d9f0 con 0x7f2ed8161c80 2024-09-10T10:46:00.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2edf000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed806dcd0 con 0x7f2ed8162060 2024-09-10T10:46:00.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2edf000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed806dfb0 con 0x7f2ed80b6680 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2ed7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2ed8161c80 0x7f2ed806e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52020/0 (socket says 172.21.15.28:52020) 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2ed7e00640 1 -- 172.21.15.28:0/3675285743 learned_addr learned my addr 172.21.15.28:0/3675285743 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2edd000640 1 --2- 172.21.15.28:0/3675285743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed8169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136949359 0 0) 0x7f2ed806d9f0 con 0x7f2ed8161c80 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.830+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ebc003520 con 0x7f2ed8161c80 2024-09-10T10:46:00.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.831+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967047230 0 0) 0x7f2ed806dcd0 con 0x7f2ed8162060 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.831+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed806d9f0 con 0x7f2ed8162060 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.831+0000 7f2edd000640 1 -- 172.21.15.28:0/3675285743 >> v1:172.21.15.138:6789/0 conn(0x7f2ed8162060 legacy=0x7f2ed80b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.831+0000 7f2edd000640 1 -- 172.21.15.28:0/3675285743 >> v1:172.21.15.28:6789/0 conn(0x7f2ed8161c80 legacy=0x7f2ed806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.831+0000 7f2edd000640 1 -- 172.21.15.28:0/3675285743 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80d01f0 con 0x7f2ed80b6680 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2edd000640 1 --2- 172.21.15.28:0/3675285743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed8169830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009dc0 tx=0x7f2ed002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58cb5f95bcb4c50a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0007740 con 0x7f2ed80b6680 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed00078e0 con 0x7f2ed80b6680 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2ed7400640 1 -- 172.21.15.28:0/3675285743 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0007cb0 con 0x7f2ed80b6680 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2edf000640 1 -- 172.21.15.28:0/3675285743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 msgr2=0x7f2ed8169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.832+0000 7f2edf000640 1 --2- 172.21.15.28:0/3675285743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed8169830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009dc0 tx=0x7f2ed002a6d0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.833+0000 7f2edf000640 1 -- 172.21.15.28:0/3675285743 shutdown_connections 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.833+0000 7f2edf000640 1 --2- 172.21.15.28:0/3675285743 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed8169830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.833+0000 7f2edf000640 1 -- 172.21.15.28:0/3675285743 >> 172.21.15.28:0/3675285743 conn(0x7f2ed80d44f0 msgr2=0x7f2ed80d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:00.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.833+0000 7f2edf000640 1 -- 172.21.15.28:0/3675285743 shutdown_connections 2024-09-10T10:46:00.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.833+0000 7f2edf000640 1 -- 172.21.15.28:0/3675285743 wait complete. 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.834+0000 7f2edf000640 1 Processor -- start 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.834+0000 7f2edf000640 1 -- start start 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed80b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8162060 0x7f2ed8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80b6cb0 con 0x7f2ed8161c80 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed80b6b30 con 0x7f2ed8162060 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2edf000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed8169d70 con 0x7f2ed80b6680 2024-09-10T10:46:00.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.835+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34406/0 (socket says 172.21.15.28:34406) 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ed7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed80b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 -- 172.21.15.28:0/3718391822 learned_addr learned my addr 172.21.15.28:0/3718391822 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2edd000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8162060 0x7f2ed8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 -- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 msgr2=0x7f2ed80b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed80b88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 -- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8162060 msgr2=0x7f2ed8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8162060 0x7f2ed8158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 -- 172.21.15.28:0/3718391822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed80b9cd0 con 0x7f2ed8161c80 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ecfe00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000c400 tx=0x7f2ec000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.836+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0017070 con 0x7f2ed8161c80 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.837+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec00052d0 con 0x7f2ed8161c80 2024-09-10T10:46:00.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.837+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec0003810 con 0x7f2ed8161c80 2024-09-10T10:46:00.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.837+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed80b9fb0 con 0x7f2ed8161c80 2024-09-10T10:46:00.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.837+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed815dfc0 con 0x7f2ed8161c80 2024-09-10T10:46:00.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.838+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed80b5f70 con 0x7f2ed8161c80 2024-09-10T10:46:00.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.840+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ec001a070 con 0x7f2ed8161c80 2024-09-10T10:46:00.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.840+0000 7f2ed4c00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 0x7f2eb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:00.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.840+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f2ec008a120 con 0x7f2ed8161c80 2024-09-10T10:46:00.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.840+0000 7f2ed7e00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 0x7f2eb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:00.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.841+0000 7f2ed7e00640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 0x7f2eb406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000f60 tx=0x7f2ec8007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:00.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:00.845+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ec0058590 con 0x7f2ed8161c80 2024-09-10T10:46:01.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.030+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ed815e190 con 0x7f2ed8161c80 2024-09-10T10:46:01.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.031+0000 7f2ed4c00640 1 -- 172.21.15.28:0/3718391822 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13459 (secure 0 0 0) 0x7f2ec005c890 con 0x7f2ed8161c80 2024-09-10T10:46:01.033 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:01.033 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":48,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:45:59.646780+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6834/1058548427":"2024-09-11T10:45:56.636763+0000","172.21.15.28:6835/1058548427":"2024-09-11T10:45:56.636763+0000","172.21.15.28:6835/3203683106":"2024-09-11T10:45:10.614652+0000","172.21.15.28:6834/3203683106":"2024-09-11T10:45:10.614652+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:01.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 msgr2=0x7f2eb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 0x7f2eb406e2f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8000f60 tx=0x7f2ec8007d30 comp rx=0 tx=0).stop 2024-09-10T10:46:01.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 msgr2=0x7f2ed814fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000c400 tx=0x7f2ec000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 shutdown_connections 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb406be30 0x7f2eb406e2f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed8162060 0x7f2ed8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed8161c80 0x7f2ed814fc60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 --2- 172.21.15.28:0/3718391822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed80b6680 0x7f2ed80b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 >> 172.21.15.28:0/3718391822 conn(0x7f2ed80d44f0 msgr2=0x7f2ed80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 shutdown_connections 2024-09-10T10:46:01.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.033+0000 7f2edf000640 1 -- 172.21.15.28:0/3718391822 wait complete. 2024-09-10T10:46:01.053 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:46:01.053 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:46:01.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.211+0000 7f4166200640 1 Processor -- start 2024-09-10T10:46:01.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.211+0000 7f4166200640 1 -- start start 2024-09-10T10:46:01.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f4166200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f416015c490 0x7f416009aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f4166200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f416009af70 0x7f41600b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f4166200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4160072180 con 0x7f41600b0a00 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f4166200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f416009b3b0 con 0x7f416015c490 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f4166200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f416015c870 con 0x7f416009af70 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.212+0000 7f415f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f416015c490 0x7f416009aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f416015c490 0x7f416009aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34420/0 (socket says 172.21.15.28:34420) 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 -- 172.21.15.28:0/3245494180 learned_addr learned my addr 172.21.15.28:0/3245494180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:01.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 -- 172.21.15.28:0/3245494180 >> v2:172.21.15.138:3300/0 conn(0x7f416009af70 msgr2=0x7f41600b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:01.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 --2- 172.21.15.28:0/3245494180 >> v2:172.21.15.138:3300/0 conn(0x7f416009af70 0x7f41600b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 -- 172.21.15.28:0/3245494180 >> v1:172.21.15.138:6789/0 conn(0x7f41600b0a00 legacy=0x7f41600b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:01.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.213+0000 7f415f400640 1 -- 172.21.15.28:0/3245494180 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4160071bc0 con 0x7f416015c490 2024-09-10T10:46:01.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.214+0000 7f415f400640 1 --2- 172.21.15.28:0/3245494180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416015c490 0x7f416009aa30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41600b71b0 tx=0x7f415002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144f99fc1acecd54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.214+0000 7f415e000640 1 -- 172.21.15.28:0/3245494180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150037070 con 0x7f416015c490 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.214+0000 7f415e000640 1 -- 172.21.15.28:0/3245494180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4150032440 con 0x7f416015c490 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.214+0000 7f415e000640 1 -- 172.21.15.28:0/3245494180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4150032810 con 0x7f416015c490 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 -- 172.21.15.28:0/3245494180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416015c490 msgr2=0x7f416009aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 --2- 172.21.15.28:0/3245494180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416015c490 0x7f416009aa30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f41600b71b0 tx=0x7f415002a230 comp rx=0 tx=0).stop 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 -- 172.21.15.28:0/3245494180 shutdown_connections 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 --2- 172.21.15.28:0/3245494180 >> v2:172.21.15.138:3300/0 conn(0x7f416009af70 0x7f41600b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 --2- 172.21.15.28:0/3245494180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416015c490 0x7f416009aa30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 -- 172.21.15.28:0/3245494180 >> 172.21.15.28:0/3245494180 conn(0x7f41600b3f30 msgr2=0x7f41600b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 -- 172.21.15.28:0/3245494180 shutdown_connections 2024-09-10T10:46:01.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.215+0000 7f4166200640 1 -- 172.21.15.28:0/3245494180 wait complete. 2024-09-10T10:46:01.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.216+0000 7f4166200640 1 Processor -- start 2024-09-10T10:46:01.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.217+0000 7f4166200640 1 -- start start 2024-09-10T10:46:01.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.217+0000 7f4166200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 0x7f4160148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.217+0000 7f4166200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f4166200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416015c490 0x7f416015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f4166200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41600affa0 con 0x7f41600b0a00 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f4166200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41600afcc0 con 0x7f416009af70 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f4166200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f416009b8f0 con 0x7f416015c490 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34426/0 (socket says 172.21.15.28:34426) 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 -- 172.21.15.28:0/115396262 learned_addr learned my addr 172.21.15.28:0/115396262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415f400640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 0x7f4160148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415fe00640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416015c490 0x7f416015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 -- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416015c490 msgr2=0x7f416015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416015c490 0x7f416015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 -- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 msgr2=0x7f4160148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 0x7f4160148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415ea00640 1 -- 172.21.15.28:0/115396262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416015efc0 con 0x7f41600b0a00 2024-09-10T10:46:01.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.218+0000 7f415f400640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 0x7f4160148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:01.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f415ea00640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f414c00b410 tx=0x7f414c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c015020 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f414c011850 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c011c40 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416006dc80 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.219+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416006d6c0 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.220+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41600b04c0 con 0x7f41600b0a00 2024-09-10T10:46:01.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.221+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f414c018020 con 0x7f41600b0a00 2024-09-10T10:46:01.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.222+0000 7f4157400640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.222+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f414c08ac90 con 0x7f41600b0a00 2024-09-10T10:46:01.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.222+0000 7f415f400640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.223+0000 7f415f400640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 0x7f414006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4150034000 tx=0x7f415002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.226+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f414c01f020 con 0x7f41600b0a00 2024-09-10T10:46:01.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.412+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41600b5880 con 0x7f41600b0a00 2024-09-10T10:46:01.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.412+0000 7f4157400640 1 -- 172.21.15.28:0/115396262 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f414c0172c0 con 0x7f41600b0a00 2024-09-10T10:46:01.413 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:46:01.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 msgr2=0x7f414006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 0x7f414006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4150034000 tx=0x7f415002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 msgr2=0x7f416006cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f414c00b410 tx=0x7f414c00b8d0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 shutdown_connections 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f414006be30 0x7f414006e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f416015c490 0x7f416015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41600b0a00 0x7f416006cd20 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 --2- 172.21.15.28:0/115396262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f416009af70 0x7f4160148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 >> 172.21.15.28:0/115396262 conn(0x7f41600b3f30 msgr2=0x7f41600b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 shutdown_connections 2024-09-10T10:46:01.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.414+0000 7f4166200640 1 -- 172.21.15.28:0/115396262 wait complete. 2024-09-10T10:46:01.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-10T10:46:01.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.587+0000 7f3674a00640 1 Processor -- start 2024-09-10T10:46:01.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.587+0000 7f3674a00640 1 -- start start 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.588+0000 7f3674a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f36700cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.588+0000 7f3674a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3670072de0 0x7f367006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.588+0000 7f3674a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3670146140 con 0x7f3670072700 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.588+0000 7f3674a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f36700cd740 con 0x7f3670072de0 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.588+0000 7f3674a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f36700cda20 con 0x7f3670072320 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3670072de0 0x7f367006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3670072de0 0x7f367006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34428/0 (socket says 172.21.15.28:34428) 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 -- 172.21.15.28:0/1985750159 learned_addr learned my addr 172.21.15.28:0/1985750159 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366de00640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f36700cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 -- 172.21.15.28:0/1985750159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 msgr2=0x7f36700cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:01.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f36700cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 -- 172.21.15.28:0/1985750159 >> v1:172.21.15.138:6789/0 conn(0x7f3670072700 legacy=0x7f367006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:01.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.589+0000 7f366d400640 1 -- 172.21.15.28:0/1985750159 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36700cdd00 con 0x7f3670072de0 2024-09-10T10:46:01.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.590+0000 7f366d400640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f367006e0b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f365c009870 tx=0x7f365c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba82292a84b37249 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.590+0000 7f366ca00640 1 -- 172.21.15.28:0/1985750159 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c037070 con 0x7f3670072de0 2024-09-10T10:46:01.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.590+0000 7f366ca00640 1 -- 172.21.15.28:0/1985750159 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365c032440 con 0x7f3670072de0 2024-09-10T10:46:01.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.590+0000 7f366ca00640 1 -- 172.21.15.28:0/1985750159 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c032810 con 0x7f3670072de0 2024-09-10T10:46:01.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 -- 172.21.15.28:0/1985750159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 msgr2=0x7f367006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f367006e0b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f365c009870 tx=0x7f365c02a180 comp rx=0 tx=0).stop 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 -- 172.21.15.28:0/1985750159 shutdown_connections 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f367006e0b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 --2- 172.21.15.28:0/1985750159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f36700cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 -- 172.21.15.28:0/1985750159 >> 172.21.15.28:0/1985750159 conn(0x7f3670067d70 msgr2=0x7f367006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 -- 172.21.15.28:0/1985750159 shutdown_connections 2024-09-10T10:46:01.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.591+0000 7f3674a00640 1 -- 172.21.15.28:0/1985750159 wait complete. 2024-09-10T10:46:01.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.593+0000 7f3674a00640 1 Processor -- start 2024-09-10T10:46:01.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.593+0000 7f3674a00640 1 -- start start 2024-09-10T10:46:01.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f367013f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3670072700 0x7f36701e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f36700cd520 con 0x7f3670072de0 2024-09-10T10:46:01.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36700cd3a0 con 0x7f3670072700 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f3674a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f367006ed00 con 0x7f3670072320 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f366e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f366e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34434/0 (socket says 172.21.15.28:34434) 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f366e800640 1 -- 172.21.15.28:0/2960037606 learned_addr learned my addr 172.21.15.28:0/2960037606 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f366de00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f367013f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.594+0000 7f366d400640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3670072700 0x7f36701e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 -- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 msgr2=0x7f367013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f367013f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 -- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3670072700 msgr2=0x7f36701e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3670072700 0x7f36701e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 -- 172.21.15.28:0/2960037606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36701428d0 con 0x7f3670072de0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f366e800640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f366800c780 tx=0x7f366800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668004b60 con 0x7f3670072de0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3668004d00 con 0x7f3670072de0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366800aa90 con 0x7f3670072de0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36701f75f0 con 0x7f3670072de0 2024-09-10T10:46:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.595+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701f7bb0 con 0x7f3670072de0 2024-09-10T10:46:01.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.596+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36700cce60 con 0x7f3670072de0 2024-09-10T10:46:01.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.597+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f366800ac30 con 0x7f3670072de0 2024-09-10T10:46:01.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.599+0000 7f3666000640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 0x7f365006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:01.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.599+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f366808ae60 con 0x7f3670072de0 2024-09-10T10:46:01.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.599+0000 7f366de00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 0x7f365006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:01.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.600+0000 7f366de00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 0x7f365006e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3660006fd0 tx=0x7f3660008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:01.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.603+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3668020070 con 0x7f3670072de0 2024-09-10T10:46:01.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:01.855+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7f367006bf90 con 0x7f3670072de0 2024-09-10T10:46:02.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.663+0000 7f3666000640 1 -- 172.21.15.28:0/2960037606 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v133) ==== 144+0+0 (secure 0 0 0) 0x7f36680592d0 con 0x7f3670072de0 2024-09-10T10:46:02.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 msgr2=0x7f365006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 0x7f365006e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3660006fd0 tx=0x7f3660008040 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 msgr2=0x7f36701e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f366800c780 tx=0x7f366800cc40 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 shutdown_connections 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f365006be30 0x7f365006e2f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3670072de0 0x7f36701e9810 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3670072700 0x7f36701e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 --2- 172.21.15.28:0/2960037606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3670072320 0x7f367013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 >> 172.21.15.28:0/2960037606 conn(0x7f3670067d70 msgr2=0x7f36700cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 shutdown_connections 2024-09-10T10:46:02.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.666+0000 7f3674a00640 1 -- 172.21.15.28:0/2960037606 wait complete. 2024-09-10T10:46:02.688 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:46:02.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.851+0000 7f427e200640 1 Processor -- start 2024-09-10T10:46:02.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.851+0000 7f427e200640 1 -- start start 2024-09-10T10:46:02.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.852+0000 7f427e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42780cf790 0x7f42780cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.852+0000 7f427e200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42780b30e0 con 0x7f42780ce060 2024-09-10T10:46:02.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.852+0000 7f427e200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42780b33c0 con 0x7f4278156740 2024-09-10T10:46:02.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.852+0000 7f427e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f42780d0140 con 0x7f42780cf790 2024-09-10T10:46:02.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4277e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f42780ce060 0x7f42780ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52036/0 (socket says 172.21.15.28:52036) 2024-09-10T10:46:02.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4277e00640 1 -- 172.21.15.28:0/3282747926 learned_addr learned my addr 172.21.15.28:0/3282747926 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:02.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f426fe00640 1 --2- 172.21.15.28:0/3282747926 >> v2:172.21.15.138:3300/0 conn(0x7f42780cf790 0x7f42780cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270858234 0 0) 0x7f42780b30e0 con 0x7f42780ce060 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f425c003520 con 0x7f42780ce060 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2822496395 0 0) 0x7f42780b33c0 con 0x7f4278156740 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42780b30e0 con 0x7f4278156740 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 147986107 0 0) 0x7f425c003520 con 0x7f42780ce060 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 >> v2:172.21.15.138:3300/0 conn(0x7f42780cf790 msgr2=0x7f42780cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 --2- 172.21.15.28:0/3282747926 >> v2:172.21.15.138:3300/0 conn(0x7f42780cf790 0x7f42780cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 >> v1:172.21.15.138:6789/0 conn(0x7f4278156740 legacy=0x7f42780b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42780b36a0 con 0x7f42780ce060 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f42700019f0 con 0x7f42780ce060 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.853+0000 7f426fe00640 1 --2- 172.21.15.28:0/3282747926 >> v2:172.21.15.138:3300/0 conn(0x7f42780cf790 0x7f42780cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 >> v1:172.21.15.28:6789/0 conn(0x7f42780ce060 legacy=0x7f42780ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c003b90 0x7f425c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f425c004800 0x7f425c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 0x7f425c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42780d0140 con 0x7f425c0051c0 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f425c00c090 con 0x7f425c004800 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f425c00c290 con 0x7f425c003b90 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 0x7f425c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277400640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c003b90 0x7f425c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f426fe00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f425c004800 0x7f425c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 -- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c003b90 msgr2=0x7f425c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c003b90 0x7f425c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 -- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f425c004800 msgr2=0x7f425c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f425c004800 0x7f425c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.854+0000 7f4277e00640 1 -- 172.21.15.28:0/3282747926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c003520 con 0x7f425c0051c0 2024-09-10T10:46:02.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.855+0000 7f4277e00640 1 -- 172.21.15.28:0/3282747926 reap_dead start 2024-09-10T10:46:02.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.855+0000 7f4277e00640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 0x7f425c00b9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f427000a710 tx=0x7f425c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6d78ab0867c28f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:02.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427000a330 con 0x7f425c0051c0 2024-09-10T10:46:02.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427002f070 con 0x7f425c0051c0 2024-09-10T10:46:02.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f4276a00640 1 -- 172.21.15.28:0/3282747926 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427002bb60 con 0x7f425c0051c0 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 -- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 msgr2=0x7f425c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 0x7f425c00b9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f427000a710 tx=0x7f425c005830 comp rx=0 tx=0).stop 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 -- 172.21.15.28:0/3282747926 shutdown_connections 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 --2- 172.21.15.28:0/3282747926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f425c0051c0 0x7f425c00b9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 -- 172.21.15.28:0/3282747926 >> 172.21.15.28:0/3282747926 conn(0x7f427806bdf0 msgr2=0x7f427806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 -- 172.21.15.28:0/3282747926 shutdown_connections 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.856+0000 7f427e200640 1 -- 172.21.15.28:0/3282747926 wait complete. 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.858+0000 7f427e200640 1 Processor -- start 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.858+0000 7f427e200640 1 -- start start 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c0051c0 0x7f42781f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42780ad680 0x7f4278153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b3c60 con 0x7f4278154f80 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f427806edc0 con 0x7f42780ad680 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f427e200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f42780d0890 con 0x7f425c0051c0 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f426fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f426fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34440/0 (socket says 172.21.15.28:34440) 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f426fe00640 1 -- 172.21.15.28:0/1619747862 learned_addr learned my addr 172.21.15.28:0/1619747862 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.859+0000 7f4277400640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c0051c0 0x7f42781f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f4277e00640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42780ad680 0x7f4278153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 -- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c0051c0 msgr2=0x7f42781f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c0051c0 0x7f42781f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 -- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42780ad680 msgr2=0x7f4278153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:02.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42780ad680 0x7f4278153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 -- 172.21.15.28:0/1619747862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781f49f0 con 0x7f4278154f80 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426fe00640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4260011a90 tx=0x7f4260011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000fc10 con 0x7f4278154f80 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426000fdb0 con 0x7f4278154f80 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260004260 con 0x7f4278154f80 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.860+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781f4cd0 con 0x7f4278154f80 2024-09-10T10:46:02.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.861+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781f5290 con 0x7f4278154f80 2024-09-10T10:46:02.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.862+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4278156b20 con 0x7f4278154f80 2024-09-10T10:46:02.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.862+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4260012040 con 0x7f4278154f80 2024-09-10T10:46:02.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.864+0000 7f426f400640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 0x7f425406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:02.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.864+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f426008b150 con 0x7f4278154f80 2024-09-10T10:46:02.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.864+0000 7f4277400640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 0x7f425406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:02.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.865+0000 7f4277400640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 0x7f425406e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f427815c5a0 tx=0x7f4268009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:02.869+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4260019030 con 0x7f4278154f80 2024-09-10T10:46:03.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.053+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f427806c4b0 con 0x7f4278154f80 2024-09-10T10:46:03.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.056+0000 7f426f400640 1 -- 172.21.15.28:0/1619747862 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f42600595c0 con 0x7f4278154f80 2024-09-10T10:46:03.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 msgr2=0x7f425406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 0x7f425406e320 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f427815c5a0 tx=0x7f4268009040 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 msgr2=0x7f42781527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4260011a90 tx=0x7f4260011f50 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 shutdown_connections 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f425406be60 0x7f425406e320 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42780ad680 0x7f4278153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4278154f80 0x7f42781527d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 --2- 172.21.15.28:0/1619747862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f425c0051c0 0x7f42781f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 >> 172.21.15.28:0/1619747862 conn(0x7f427806bdf0 msgr2=0x7f427815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 shutdown_connections 2024-09-10T10:46:03.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.059+0000 7f427e200640 1 -- 172.21.15.28:0/1619747862 wait complete. 2024-09-10T10:46:03.076 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:46:03.076 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:03.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:03.260 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:46:03.261 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:03.261 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:03.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:03.297 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:03.297 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:03.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:03.334 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:03.334 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:03.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:03.372 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:03.372 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:03.410 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:03.410 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:46:03.410 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:03.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.566+0000 7f6605000640 1 Processor -- start 2024-09-10T10:46:03.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.566+0000 7f6605000640 1 -- start start 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.567+0000 7f6605000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f660006f8d0 0x7f6600165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.567+0000 7f6605000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66000cdbe0 con 0x7f6600072d30 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.567+0000 7f6605000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66000cd900 con 0x7f660015b300 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.567+0000 7f6605000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f66000728a0 con 0x7f660006f8d0 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fe800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f660006f8d0 0x7f6600165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fd400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6600072d30 0x7f660006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52046/0 (socket says 172.21.15.28:52046) 2024-09-10T10:46:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fd400640 1 -- 172.21.15.28:0/1075211951 learned_addr learned my addr 172.21.15.28:0/1075211951 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fe800640 1 -- 172.21.15.28:0/1075211951 >> v1:172.21.15.138:6789/0 conn(0x7f660015b300 legacy=0x7f66000cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fe800640 1 -- 172.21.15.28:0/1075211951 >> v1:172.21.15.28:6789/0 conn(0x7f6600072d30 legacy=0x7f660006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fe800640 1 -- 172.21.15.28:0/1075211951 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6600072250 con 0x7f660006f8d0 2024-09-10T10:46:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.568+0000 7f65fca00640 1 -- 172.21.15.28:0/1075211951 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300848094 0 0) 0x7f66000cdbe0 con 0x7f6600072d30 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.569+0000 7f65fe800640 1 --2- 172.21.15.28:0/1075211951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f660006f8d0 0x7f6600165de0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f65f800bfa0 tx=0x7f65f802c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7cf112017b9bb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.569+0000 7f65fca00640 1 -- 172.21.15.28:0/1075211951 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8039070 con 0x7f660006f8d0 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f65fca00640 1 -- 172.21.15.28:0/1075211951 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f8009540 con 0x7f660006f8d0 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f65fca00640 1 -- 172.21.15.28:0/1075211951 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f8009910 con 0x7f660006f8d0 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 -- 172.21.15.28:0/1075211951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f660006f8d0 msgr2=0x7f6600165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 --2- 172.21.15.28:0/1075211951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f660006f8d0 0x7f6600165de0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f65f800bfa0 tx=0x7f65f802c870 comp rx=0 tx=0).stop 2024-09-10T10:46:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 -- 172.21.15.28:0/1075211951 shutdown_connections 2024-09-10T10:46:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 --2- 172.21.15.28:0/1075211951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f660006f8d0 0x7f6600165de0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 -- 172.21.15.28:0/1075211951 >> 172.21.15.28:0/1075211951 conn(0x7f6600067d70 msgr2=0x7f660006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 -- 172.21.15.28:0/1075211951 shutdown_connections 2024-09-10T10:46:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.570+0000 7f6605000640 1 -- 172.21.15.28:0/1075211951 wait complete. 2024-09-10T10:46:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.572+0000 7f6605000640 1 Processor -- start 2024-09-10T10:46:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.572+0000 7f6605000640 1 -- start start 2024-09-10T10:46:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 0x7f66001545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660015b300 0x7f66000b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66000cffe0 con 0x7f6600072d30 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66000cd5c0 con 0x7f660015b300 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f6605000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66000d0160 con 0x7f660006f8d0 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f65fd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f65fd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34460/0 (socket says 172.21.15.28:34460) 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f65fd400640 1 -- 172.21.15.28:0/851454328 learned_addr learned my addr 172.21.15.28:0/851454328 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f65fe800640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660015b300 0x7f66000b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.573+0000 7f65fde00640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 0x7f66001545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 -- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 msgr2=0x7f66001545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 0x7f66001545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 -- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660015b300 msgr2=0x7f66000b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660015b300 0x7f66000b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 -- 172.21.15.28:0/851454328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66001f4910 con 0x7f6600072d30 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fde00640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 0x7f66001545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65fd400640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c4a0 tx=0x7f65ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec018070 con 0x7f6600072d30 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec0074e0 con 0x7f6600072d30 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66001f4bf0 con 0x7f6600072d30 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec004a20 con 0x7f6600072d30 2024-09-10T10:46:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.574+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66001f51b0 con 0x7f6600072d30 2024-09-10T10:46:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.575+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6600158df0 con 0x7f6600072d30 2024-09-10T10:46:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.576+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65ec01b070 con 0x7f6600072d30 2024-09-10T10:46:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.577+0000 7f65f6000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 0x7f65e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.577+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7f65ec089fe0 con 0x7f6600072d30 2024-09-10T10:46:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.578+0000 7f65fde00640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 0x7f65e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.579+0000 7f65fde00640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 0x7f65e006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007fd0 tx=0x7f65f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:03.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.581+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f65ec058470 con 0x7f6600072d30 2024-09-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.802+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6600158b10 con 0x7f6600072d30 2024-09-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.802+0000 7f65f6000640 1 -- 172.21.15.28:0/851454328 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+3914 (secure 0 0 0) 0x7f65ec05c770 con 0x7f6600072d30 2024-09-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:03.803 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"btime":"2024-09-10T10:45:57:639874+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3141863866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3141863866},{"type":"v1","addr":"172.21.15.138:6836","nonce":3141863866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4833,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1329457758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1329457758},{"type":"v1","addr":"172.21.15.28:6833","nonce":1329457758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4836,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2638073579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2638073579},{"type":"v1","addr":"172.21.15.138:6837","nonce":2638073579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/407923568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":407923568},{"type":"v1","addr":"172.21.15.28:6835","nonce":407923568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[]} 2024-09-10T10:46:03.804 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 msgr2=0x7f65e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 0x7f65e006e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f65f0007fd0 tx=0x7f65f0009040 comp rx=0 tx=0).stop 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 msgr2=0x7f6600157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00c4a0 tx=0x7f65ec00c960 comp rx=0 tx=0).stop 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 shutdown_connections 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f65e006be30 0x7f65e006e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f660015b300 0x7f66000b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6600072d30 0x7f6600157c90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 --2- 172.21.15.28:0/851454328 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f660006f8d0 0x7f66001545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:03.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.804+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 >> 172.21.15.28:0/851454328 conn(0x7f6600067d70 msgr2=0x7f660006ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:03.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.805+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 shutdown_connections 2024-09-10T10:46:03.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:03.805+0000 7f6605000640 1 -- 172.21.15.28:0/851454328 wait complete. 2024-09-10T10:46:03.821 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:46:03.821 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:46:03.822 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:46:03.822 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:03.822 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:46:03.822 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:46:03.822 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:46:03.822 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:03.822 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:46:03.822 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:46:03.823 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:46:03.823 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:03.823 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:46:03.823 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:46:03.823 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:46:03.823 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:03.837 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:46:03.837 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:46:03.838 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:46:03.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:46:03.842 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:46:03.842 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:46:03.843 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:46:03.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:46:03.881 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:46:03.881 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:03.883 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:46:03.883 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:03.886 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:46:03.886 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:03.888 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:46:03.888 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:03.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:46:04.013 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:46:04.016 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:46:04.017 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:46:04.022 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:46:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.055+0000 7fd7f4a00640 1 Processor -- start 2024-09-10T10:46:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.055+0000 7fd7f4a00640 1 -- start start 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 0x7fd7f00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 0x7fd7f006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f00b37d0 con 0x7fd7f00d1f90 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f00cf330 con 0x7fd7f015e660 2024-09-10T10:46:04.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7f4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f00cf000 con 0x7fd7f00d0dd0 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7efe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7eea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 0x7fd7f006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7ef400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 0x7fd7f00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7efe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41170/0 (socket says 172.21.15.28:41170) 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.056+0000 7fd7efe00640 1 -- 172.21.15.28:0/4026594994 learned_addr learned my addr 172.21.15.28:0/4026594994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7efe00640 1 -- 172.21.15.28:0/4026594994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 msgr2=0x7fd7f00d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7efe00640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 0x7fd7f00d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7efe00640 1 -- 172.21.15.28:0/4026594994 >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 msgr2=0x7fd7f006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7efe00640 1 --2- 172.21.15.28:0/4026594994 >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 0x7fd7f006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7efe00640 1 -- 172.21.15.28:0/4026594994 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00d16f0 con 0x7fd7f00d1f90 2024-09-10T10:46:04.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7eea00640 1 --2- 172.21.15.28:0/4026594994 >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 0x7fd7f006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:04.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.057+0000 7fd7ef400640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 0x7fd7f00d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:04.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.058+0000 7fd7efe00640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b3dd0 tx=0x7fd7e802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1864bae118bf82b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.058+0000 7fd7ee000640 1 -- 172.21.15.28:0/4026594994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8036070 con 0x7fd7f00d1f90 2024-09-10T10:46:04.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.058+0000 7fd7ee000640 1 -- 172.21.15.28:0/4026594994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e8031600 con 0x7fd7f00d1f90 2024-09-10T10:46:04.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.058+0000 7fd7ee000640 1 -- 172.21.15.28:0/4026594994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80319d0 con 0x7fd7f00d1f90 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 -- 172.21.15.28:0/4026594994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f00d1f90 msgr2=0x7fd7f00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b3dd0 tx=0x7fd7e802a690 comp rx=0 tx=0).stop 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 -- 172.21.15.28:0/4026594994 shutdown_connections 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f00d1f90 0x7fd7f00cf7c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/4026594994 >> v2:172.21.15.138:3300/0 conn(0x7fd7f015e660 0x7fd7f006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/4026594994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d0dd0 0x7fd7f00d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 -- 172.21.15.28:0/4026594994 >> 172.21.15.28:0/4026594994 conn(0x7fd7f008bfc0 msgr2=0x7fd7f00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 -- 172.21.15.28:0/4026594994 shutdown_connections 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.059+0000 7fd7f4a00640 1 -- 172.21.15.28:0/4026594994 wait complete. 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.060+0000 7fd7f4a00640 1 Processor -- start 2024-09-10T10:46:04.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.060+0000 7fd7f4a00640 1 -- start start 2024-09-10T10:46:04.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f00d0dd0 0x7fd7f01e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d1f90 0x7fd7f00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00d2f40 con 0x7fd7f015e660 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f00d2c60 con 0x7fd7f00d0dd0 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.061+0000 7fd7f4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f00d3220 con 0x7fd7f00d1f90 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41174/0 (socket says 172.21.15.28:41174) 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 -- 172.21.15.28:0/1600810133 learned_addr learned my addr 172.21.15.28:0/1600810133 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7ef400640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f00d0dd0 0x7fd7f01e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7eea00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d1f90 0x7fd7f00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 -- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d1f90 msgr2=0x7fd7f00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d1f90 0x7fd7f00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 -- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f00d0dd0 msgr2=0x7fd7f01e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f00d0dd0 0x7fd7f01e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 -- 172.21.15.28:0/1600810133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f00cc6f0 con 0x7fd7f015e660 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.062+0000 7fd7efe00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8034040 tx=0x7fd7e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.063+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8041020 con 0x7fd7f015e660 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.063+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e80077a0 con 0x7fd7f015e660 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.063+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8036040 con 0x7fd7f015e660 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.063+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f00cc9d0 con 0x7fd7f015e660 2024-09-10T10:46:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.063+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f00ccf90 con 0x7fd7f015e660 2024-09-10T10:46:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.064+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f004ffb0 con 0x7fd7f015e660 2024-09-10T10:46:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.065+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7e8044020 con 0x7fd7f015e660 2024-09-10T10:46:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.066+0000 7fd7e7400640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 0x7fd7d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.066+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5421+0+0 (secure 0 0 0) 0x7fd7e80ab730 con 0x7fd7f015e660 2024-09-10T10:46:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.066+0000 7fd7ef400640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 0x7fd7d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.067+0000 7fd7ef400640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 0x7fd7d006e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009560 tx=0x7fd7e0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.070+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7f004ffb0 con 0x7fd7f015e660 2024-09-10T10:46:04.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.266+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd7f00d1b20 con 0x7fd7f015e660 2024-09-10T10:46:04.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.677+0000 7fd7e7400640 1 -- 172.21.15.28:0/1600810133 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) ==== 97+0+0 (secure 0 0 0) 0x7fd7e804b030 con 0x7fd7f015e660 2024-09-10T10:46:04.679 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:46:04.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 msgr2=0x7fd7d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 0x7fd7d006e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0009560 tx=0x7fd7e0009190 comp rx=0 tx=0).stop 2024-09-10T10:46:04.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 msgr2=0x7fd7f01e9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8034040 tx=0x7fd7e8004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 shutdown_connections 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7d006be60 0x7fd7d006e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7f015e660 0x7fd7f01e9420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7f00d1f90 0x7fd7f00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 --2- 172.21.15.28:0/1600810133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7f00d0dd0 0x7fd7f01e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 >> 172.21.15.28:0/1600810133 conn(0x7fd7f008bfc0 msgr2=0x7fd7f00b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 shutdown_connections 2024-09-10T10:46:04.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.681+0000 7fd7f4a00640 1 -- 172.21.15.28:0/1600810133 wait complete. 2024-09-10T10:46:04.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:46:04.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.856+0000 7f9c83200640 1 Processor -- start 2024-09-10T10:46:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.856+0000 7f9c83200640 1 -- start start 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.857+0000 7f9c83200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c072320 0x7f9c7c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.857+0000 7f9c83200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9c7c15e1f0 0x7f9c7c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.857+0000 7f9c83200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7c146140 con 0x7f9c7c072de0 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.857+0000 7f9c83200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c0cd740 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.857+0000 7f9c83200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c0cda20 con 0x7f9c7c072320 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c73400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9c7c072de0 0x7f9c7c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50716/0 (socket says 172.21.15.28:50716) 2024-09-10T10:46:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c73400640 1 -- 172.21.15.28:0/4009792393 learned_addr learned my addr 172.21.15.28:0/4009792393 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c81200640 1 --2- 172.21.15.28:0/4009792393 >> v2:172.21.15.28:3300/0 conn(0x7f9c7c15e1f0 0x7f9c7c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c7be00640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c072320 0x7f9c7c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c7b400640 1 -- 172.21.15.28:0/4009792393 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804606173 0 0) 0x7f9c7c146140 con 0x7f9c7c072de0 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c7b400640 1 -- 172.21.15.28:0/4009792393 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c60003520 con 0x7f9c7c072de0 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c81200640 1 -- 172.21.15.28:0/4009792393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c072320 msgr2=0x7f9c7c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c81200640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c072320 0x7f9c7c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c81200640 1 -- 172.21.15.28:0/4009792393 >> v1:172.21.15.28:6789/0 conn(0x7f9c7c072de0 legacy=0x7f9c7c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.858+0000 7f9c81200640 1 -- 172.21.15.28:0/4009792393 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c0cdd00 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.859+0000 7f9c81200640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c15e1f0 0x7f9c7c06e7c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009d90 tx=0x7f9c7402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45a0bf9362bef67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.859+0000 7f9c7b400640 1 -- 172.21.15.28:0/4009792393 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74031600 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c7b400640 1 -- 172.21.15.28:0/4009792393 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c740317a0 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c7b400640 1 -- 172.21.15.28:0/4009792393 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74031b50 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 -- 172.21.15.28:0/4009792393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c15e1f0 msgr2=0x7f9c7c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c15e1f0 0x7f9c7c06e7c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009d90 tx=0x7f9c7402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 -- 172.21.15.28:0/4009792393 shutdown_connections 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c15e1f0 0x7f9c7c06e7c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 --2- 172.21.15.28:0/4009792393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c072320 0x7f9c7c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 -- 172.21.15.28:0/4009792393 >> 172.21.15.28:0/4009792393 conn(0x7f9c7c067d70 msgr2=0x7f9c7c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 -- 172.21.15.28:0/4009792393 shutdown_connections 2024-09-10T10:46:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.860+0000 7f9c83200640 1 -- 172.21.15.28:0/4009792393 wait complete. 2024-09-10T10:46:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.862+0000 7f9c83200640 1 Processor -- start 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.862+0000 7f9c83200640 1 -- start start 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c072de0 0x7f9c7c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 0x7f9c7c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c0cd520 con 0x7f9c7c072320 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c0cd3a0 con 0x7f9c7c072de0 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c83200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c06ed00 con 0x7f9c7c15e1f0 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41182/0 (socket says 172.21.15.28:41182) 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.863+0000 7f9c7be00640 1 -- 172.21.15.28:0/6690965 learned_addr learned my addr 172.21.15.28:0/6690965 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c81200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 0x7f9c7c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 -- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 msgr2=0x7f9c7c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 0x7f9c7c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 -- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c072de0 msgr2=0x7f9c7c0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c072de0 0x7f9c7c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 -- 172.21.15.28:0/6690965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c140640 con 0x7f9c7c072320 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c81200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 0x7f9c7c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c7be00640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d5f0 tx=0x7f9c6c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c0125f0 con 0x7f9c7c072320 2024-09-10T10:46:04.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6c019070 con 0x7f9c7c072320 2024-09-10T10:46:04.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c0129d0 con 0x7f9c7c072320 2024-09-10T10:46:04.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.864+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c140920 con 0x7f9c7c072320 2024-09-10T10:46:04.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.865+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c140ee0 con 0x7f9c7c072320 2024-09-10T10:46:04.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.866+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c072700 con 0x7f9c7c072320 2024-09-10T10:46:04.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.866+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c6c012b70 con 0x7f9c7c072320 2024-09-10T10:46:04.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.867+0000 7f9c78c00640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:04.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.867+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5249+0+0 (secure 0 0 0) 0x7f9c6c08b150 con 0x7f9c7c072320 2024-09-10T10:46:04.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.868+0000 7f9c73400640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:04.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.868+0000 7f9c73400640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 0x7f9c5806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c64000f60 tx=0x7f9c640078f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:04.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:04.871+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c6c059c20 con 0x7f9c7c072320 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.121+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9c7c142d70 con 0x7f9c7c072320 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.122+0000 7f9c78c00640 1 -- 172.21.15.28:0/6690965 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f9c6c059520 con 0x7f9c7c072320 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.123 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:46:05.124 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.125 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.126 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:05.127 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.128 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:46:05.129 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.130 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.131 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.132 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.133 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.134 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:05.135 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:46:05.136 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:05.137 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:05.138 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:46:05.139 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:05.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 msgr2=0x7f9c5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 0x7f9c5806e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c64000f60 tx=0x7f9c640078f0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 msgr2=0x7f9c7c0cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c00d5f0 tx=0x7f9c6c00dab0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 shutdown_connections 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5806be30 0x7f9c5806e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c7c15e1f0 0x7f9c7c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c7c072de0 0x7f9c7c0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 --2- 172.21.15.28:0/6690965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c7c072320 0x7f9c7c0cad00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 >> 172.21.15.28:0/6690965 conn(0x7f9c7c067d70 msgr2=0x7f9c7c0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 shutdown_connections 2024-09-10T10:46:05.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.124+0000 7f9c83200640 1 -- 172.21.15.28:0/6690965 wait complete. 2024-09-10T10:46:05.142 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:46:05.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.299+0000 7fe785000640 1 Processor -- start 2024-09-10T10:46:05.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.299+0000 7fe785000640 1 -- start start 2024-09-10T10:46:05.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe780162240 0x7fe7800b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe780162620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 0x7fe7800ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe7800cde30 con 0x7fe780162240 2024-09-10T10:46:05.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe780073840 con 0x7fe78006fb00 2024-09-10T10:46:05.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe785000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7800cdb00 con 0x7fe780073420 2024-09-10T10:46:05.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe77de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe780162240 0x7fe7800b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe77de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe780162240 0x7fe7800b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41186/0 (socket says 172.21.15.28:41186) 2024-09-10T10:46:05.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.300+0000 7fe77de00640 1 -- 172.21.15.28:0/52554727 learned_addr learned my addr 172.21.15.28:0/52554727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe777e00640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe780162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77de00640 1 -- 172.21.15.28:0/52554727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 msgr2=0x7fe780162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77de00640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe780162620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77e800640 1 --2- 172.21.15.28:0/52554727 >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 0x7fe7800ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77de00640 1 -- 172.21.15.28:0/52554727 >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 msgr2=0x7fe7800ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77de00640 1 --2- 172.21.15.28:0/52554727 >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 0x7fe7800ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77de00640 1 -- 172.21.15.28:0/52554727 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe780162b60 con 0x7fe780162240 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe777e00640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe780162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.301+0000 7fe77e800640 1 --2- 172.21.15.28:0/52554727 >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 0x7fe7800ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:05.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.302+0000 7fe77de00640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe780162240 0x7fe7800b3810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe780071930 tx=0x7fe77002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcd7ce758540779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.302+0000 7fe77d400640 1 -- 172.21.15.28:0/52554727 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770037070 con 0x7fe780162240 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.302+0000 7fe77d400640 1 -- 172.21.15.28:0/52554727 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe770032440 con 0x7fe780162240 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.302+0000 7fe77d400640 1 -- 172.21.15.28:0/52554727 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770032810 con 0x7fe780162240 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 -- 172.21.15.28:0/52554727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe780162240 msgr2=0x7fe7800b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe780162240 0x7fe7800b3810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe780071930 tx=0x7fe77002a1d0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 -- 172.21.15.28:0/52554727 shutdown_connections 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 --2- 172.21.15.28:0/52554727 >> v2:172.21.15.138:3300/0 conn(0x7fe78006fb00 0x7fe7800ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe780162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 --2- 172.21.15.28:0/52554727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe780162240 0x7fe7800b3810 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 -- 172.21.15.28:0/52554727 >> 172.21.15.28:0/52554727 conn(0x7fe7800b68a0 msgr2=0x7fe7800b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 -- 172.21.15.28:0/52554727 shutdown_connections 2024-09-10T10:46:05.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.303+0000 7fe785000640 1 -- 172.21.15.28:0/52554727 wait complete. 2024-09-10T10:46:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.304+0000 7fe785000640 1 Processor -- start 2024-09-10T10:46:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.304+0000 7fe785000640 1 -- start start 2024-09-10T10:46:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe7800b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 0x7fe780160f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe78006caa0 con 0x7fe78006fb00 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7800b0940 con 0x7fe780162240 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.305+0000 7fe785000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe78006ff70 con 0x7fe780073420 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41196/0 (socket says 172.21.15.28:41196) 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 -- 172.21.15.28:0/3542661692 learned_addr learned my addr 172.21.15.28:0/3542661692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe777e00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe7800b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77e800640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 0x7fe780160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 -- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 msgr2=0x7fe7800b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe7800b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 -- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 msgr2=0x7fe780160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 0x7fe780160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 -- 172.21.15.28:0/3542661692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe780158070 con 0x7fe78006fb00 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77e800640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 0x7fe780160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.306+0000 7fe77de00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a210 tx=0x7fe7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.307+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770037040 con 0x7fe78006fb00 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.307+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe770042070 con 0x7fe78006fb00 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.307+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770030bb0 con 0x7fe78006fb00 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.307+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7801594d0 con 0x7fe78006fb00 2024-09-10T10:46:05.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.307+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe780157ab0 con 0x7fe78006fb00 2024-09-10T10:46:05.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.308+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe780162620 con 0x7fe78006fb00 2024-09-10T10:46:05.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.309+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe770047460 con 0x7fe78006fb00 2024-09-10T10:46:05.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.310+0000 7fe776000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 0x7fe75c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.310+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5249+0+0 (secure 0 0 0) 0x7fe7700ab8b0 con 0x7fe78006fb00 2024-09-10T10:46:05.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.310+0000 7fe777e00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 0x7fe75c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.311+0000 7fe777e00640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 0x7fe75c06e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7801f7900 tx=0x7fe768008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.314+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe780158070 con 0x7fe78006fb00 2024-09-10T10:46:05.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.500+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe780073af0 con 0x7fe78006fb00 2024-09-10T10:46:05.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.501+0000 7fe776000640 1 -- 172.21.15.28:0/3542661692 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) ==== 84+0+65 (secure 0 0 0) 0x7fe77007a380 con 0x7fe78006fb00 2024-09-10T10:46:05.502 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:05.502 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:46:05.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:05.503 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:46:05.503 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:46:05.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:05.503 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:46:05.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 msgr2=0x7fe75c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 0x7fe75c06e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7801f7900 tx=0x7fe768008040 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 msgr2=0x7fe7801495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a210 tx=0x7fe7700046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 shutdown_connections 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75c06be30 0x7fe75c06e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe780162240 0x7fe780160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe780073420 0x7fe7800b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 --2- 172.21.15.28:0/3542661692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78006fb00 0x7fe7801495b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 >> 172.21.15.28:0/3542661692 conn(0x7fe7800b68a0 msgr2=0x7fe7800ae140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 shutdown_connections 2024-09-10T10:46:05.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.503+0000 7fe785000640 1 -- 172.21.15.28:0/3542661692 wait complete. 2024-09-10T10:46:05.520 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:46:05.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:46:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.682+0000 7f4b46000640 1 Processor -- start 2024-09-10T10:46:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.683+0000 7f4b46000640 1 -- start start 2024-09-10T10:46:05.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.684+0000 7f4b46000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 0x7f4b40162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.684+0000 7f4b46000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400d2b50 con 0x7f4b4006dc80 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.684+0000 7f4b46000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b400d2e30 con 0x7f4b4006e6b0 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.684+0000 7f4b46000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b400b3af0 con 0x7f4b400d4360 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b45600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4b4006dc80 0x7f4b400ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50728/0 (socket says 172.21.15.28:50728) 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b45600640 1 -- 172.21.15.28:0/1239583337 learned_addr learned my addr 172.21.15.28:0/1239583337 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:05.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b44c00640 1 --2- 172.21.15.28:0/1239583337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 0x7f4b40162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470297471 0 0) 0x7f4b400d2b50 con 0x7f4b4006dc80 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b28003520 con 0x7f4b4006dc80 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470394699 0 0) 0x7f4b400d2e30 con 0x7f4b4006e6b0 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b400d2b50 con 0x7f4b4006e6b0 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b44c00640 1 -- 172.21.15.28:0/1239583337 >> v1:172.21.15.138:6789/0 conn(0x7f4b4006e6b0 legacy=0x7f4b400d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b44c00640 1 -- 172.21.15.28:0/1239583337 >> v1:172.21.15.28:6789/0 conn(0x7f4b4006dc80 legacy=0x7f4b400ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.685+0000 7f4b44c00640 1 -- 172.21.15.28:0/1239583337 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4006d1a0 con 0x7f4b400d4360 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.686+0000 7f4b44c00640 1 --2- 172.21.15.28:0/1239583337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 0x7f4b40162990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b340097d0 tx=0x7f4b3402a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bfe656ea5a2eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34037070 con 0x7f4b400d4360 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b34032440 con 0x7f4b400d4360 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b3f400640 1 -- 172.21.15.28:0/1239583337 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b34032810 con 0x7f4b400d4360 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 -- 172.21.15.28:0/1239583337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 msgr2=0x7f4b40162990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 --2- 172.21.15.28:0/1239583337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 0x7f4b40162990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b340097d0 tx=0x7f4b3402a2a0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 -- 172.21.15.28:0/1239583337 shutdown_connections 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 --2- 172.21.15.28:0/1239583337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b400d4360 0x7f4b40162990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 -- 172.21.15.28:0/1239583337 >> 172.21.15.28:0/1239583337 conn(0x7f4b4009d740 msgr2=0x7f4b4008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 -- 172.21.15.28:0/1239583337 shutdown_connections 2024-09-10T10:46:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.687+0000 7f4b46000640 1 -- 172.21.15.28:0/1239583337 wait complete. 2024-09-10T10:46:05.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.689+0000 7f4b46000640 1 Processor -- start 2024-09-10T10:46:05.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.689+0000 7f4b46000640 1 -- start start 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b4006dc80 0x7f4b400cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b400d4360 0x7f4b40159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4006d760 con 0x7f4b4006e6b0 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40162ed0 con 0x7f4b400d4360 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b46000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4015e6d0 con 0x7f4b4006dc80 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41210/0 (socket says 172.21.15.28:41210) 2024-09-10T10:46:05.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.690+0000 7f4b3fe00640 1 -- 172.21.15.28:0/2593832807 learned_addr learned my addr 172.21.15.28:0/2593832807 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b44c00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b4006dc80 0x7f4b400cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b45600640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b400d4360 0x7f4b40159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 -- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b4006dc80 msgr2=0x7f4b400cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b4006dc80 0x7f4b400cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 -- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b400d4360 msgr2=0x7f4b40159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b400d4360 0x7f4b40159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 -- 172.21.15.28:0/2593832807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b400cd920 con 0x7f4b4006e6b0 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3fe00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c430 tx=0x7f4b2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c017070 con 0x7f4b4006e6b0 2024-09-10T10:46:05.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0052d0 con 0x7f4b4006e6b0 2024-09-10T10:46:05.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003810 con 0x7f4b4006e6b0 2024-09-10T10:46:05.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.691+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b400cc880 con 0x7f4b4006e6b0 2024-09-10T10:46:05.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.692+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b400cce40 con 0x7f4b4006e6b0 2024-09-10T10:46:05.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.693+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b40162990 con 0x7f4b4006e6b0 2024-09-10T10:46:05.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.693+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b2c01a070 con 0x7f4b4006e6b0 2024-09-10T10:46:05.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.694+0000 7f4b3cc00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 0x7f4b2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:05.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.694+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5249+0+0 (secure 0 0 0) 0x7f4b2c08a180 con 0x7f4b4006e6b0 2024-09-10T10:46:05.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.695+0000 7f4b44c00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 0x7f4b2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:05.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.695+0000 7f4b44c00640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 0x7f4b2006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b34034000 tx=0x7f4b3402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:05.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.698+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b2c058c50 con 0x7f4b4006e6b0 2024-09-10T10:46:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:05.886+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4b400d51a0 con 0x7f4b4006e6b0 2024-09-10T10:46:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.685+0000 7f4b3cc00640 1 -- 172.21.15.28:0/2593832807 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v53) ==== 138+0+0 (secure 0 0 0) 0x7f4b2c058550 con 0x7f4b4006e6b0 2024-09-10T10:46:06.687 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 msgr2=0x7f4b2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 0x7f4b2006e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b34034000 tx=0x7f4b3402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 msgr2=0x7f4b400b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c430 tx=0x7f4b2c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 shutdown_connections 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b2006be30 0x7f4b2006e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b400d4360 0x7f4b40159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b4006e6b0 0x7f4b400b2360 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 --2- 172.21.15.28:0/2593832807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b4006dc80 0x7f4b400cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.688+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 >> 172.21.15.28:0/2593832807 conn(0x7f4b4009d740 msgr2=0x7f4b400d4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.689+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 shutdown_connections 2024-09-10T10:46:06.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.689+0000 7f4b46000640 1 -- 172.21.15.28:0/2593832807 wait complete. 2024-09-10T10:46:06.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:46:06.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.879+0000 7ff8aea00640 1 Processor -- start 2024-09-10T10:46:06.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.879+0000 7ff8aea00640 1 -- start start 2024-09-10T10:46:06.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.880+0000 7ff8aea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 0x7ff8a80b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.880+0000 7ff8aea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff8a8072050 0x7ff8a8072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a80b4620 con 0x7ff8a815e900 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aea00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a80b4f60 con 0x7ff8a8072050 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a80729c0 con 0x7ff8a80b4020 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8a7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff8a815e900 0x7ff8a80ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50734/0 (socket says 172.21.15.28:50734) 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8a7e00640 1 -- 172.21.15.28:0/1789169415 learned_addr learned my addr 172.21.15.28:0/1789169415 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aca00640 1 --2- 172.21.15.28:0/1789169415 >> v2:172.21.15.28:3300/0 conn(0x7ff8a8072050 0x7ff8a8072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff89fe00640 1 --2- 172.21.15.28:0/1789169415 >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 0x7ff8a80b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8a7400640 1 -- 172.21.15.28:0/1789169415 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701086791 0 0) 0x7ff8a80b4620 con 0x7ff8a815e900 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8a7400640 1 -- 172.21.15.28:0/1789169415 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff88c003520 con 0x7ff8a815e900 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aca00640 1 -- 172.21.15.28:0/1789169415 >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 msgr2=0x7ff8a80b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aca00640 1 --2- 172.21.15.28:0/1789169415 >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 0x7ff8a80b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aca00640 1 -- 172.21.15.28:0/1789169415 >> v1:172.21.15.28:6789/0 conn(0x7ff8a815e900 legacy=0x7ff8a80ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff8aca00640 1 -- 172.21.15.28:0/1789169415 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a80ce100 con 0x7ff8a8072050 2024-09-10T10:46:06.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.881+0000 7ff89fe00640 1 --2- 172.21.15.28:0/1789169415 >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 0x7ff8a80b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.882+0000 7ff8aca00640 1 --2- 172.21.15.28:0/1789169415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a8072050 0x7ff8a8072480 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8071c90 tx=0x7ff8a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683fa496b3a04a2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8a7400640 1 -- 172.21.15.28:0/1789169415 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0031600 con 0x7ff8a8072050 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8a7400640 1 -- 172.21.15.28:0/1789169415 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a00317a0 con 0x7ff8a8072050 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8a7400640 1 -- 172.21.15.28:0/1789169415 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0031b50 con 0x7ff8a8072050 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 -- 172.21.15.28:0/1789169415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a8072050 msgr2=0x7ff8a8072480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 --2- 172.21.15.28:0/1789169415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a8072050 0x7ff8a8072480 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8071c90 tx=0x7ff8a002a610 comp rx=0 tx=0).stop 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 -- 172.21.15.28:0/1789169415 shutdown_connections 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 --2- 172.21.15.28:0/1789169415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a8072050 0x7ff8a8072480 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 --2- 172.21.15.28:0/1789169415 >> v2:172.21.15.138:3300/0 conn(0x7ff8a80b4020 0x7ff8a80b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 -- 172.21.15.28:0/1789169415 >> 172.21.15.28:0/1789169415 conn(0x7ff8a806bdf0 msgr2=0x7ff8a806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 -- 172.21.15.28:0/1789169415 shutdown_connections 2024-09-10T10:46:06.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.883+0000 7ff8aea00640 1 -- 172.21.15.28:0/1789169415 wait complete. 2024-09-10T10:46:06.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.885+0000 7ff8aea00640 1 Processor -- start 2024-09-10T10:46:06.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.885+0000 7ff8aea00640 1 -- start start 2024-09-10T10:46:06.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a8072050 0x7ff8a814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a80b4020 0x7ff8a815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a80ce960 con 0x7ff8a815e900 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a80ce680 con 0x7ff8a8072050 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a80cee80 con 0x7ff8a80b4020 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41228/0 (socket says 172.21.15.28:41228) 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8aca00640 1 -- 172.21.15.28:0/4280658313 learned_addr learned my addr 172.21.15.28:0/4280658313 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff89fe00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a80b4020 0x7ff8a815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.886+0000 7ff8a7e00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a8072050 0x7ff8a814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 -- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a80b4020 msgr2=0x7ff8a815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a80b4020 0x7ff8a815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 -- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a8072050 msgr2=0x7ff8a814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:06.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a8072050 0x7ff8a814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 -- 172.21.15.28:0/4280658313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a814aa30 con 0x7ff8a815e900 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8aca00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff8a002ac00 tx=0x7ff8a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0040070 con 0x7ff8a815e900 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8a0030c00 con 0x7ff8a815e900 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.887+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0009450 con 0x7ff8a815e900 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.888+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a815bed0 con 0x7ff8a815e900 2024-09-10T10:46:06.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.888+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a815c490 con 0x7ff8a815e900 2024-09-10T10:46:06.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.889+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a80707e0 con 0x7ff8a815e900 2024-09-10T10:46:06.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.890+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff8a0039040 con 0x7ff8a815e900 2024-09-10T10:46:06.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.891+0000 7ff8a4c00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 0x7ff88406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:06.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.891+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5619+0+0 (secure 0 0 0) 0x7ff8a00abf40 con 0x7ff8a815e900 2024-09-10T10:46:06.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.891+0000 7ff8a7e00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 0x7ff88406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:06.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.892+0000 7ff8a7e00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 0x7ff88406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff898005880 tx=0x7ff898009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:06.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:06.896+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff8a80707e0 con 0x7ff8a815e900 2024-09-10T10:46:07.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.081+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff8a80d05d0 con 0x7ff8a815e900 2024-09-10T10:46:07.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.688+0000 7ff8a4c00640 1 -- 172.21.15.28:0/4280658313 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v54) ==== 170+0+0 (secure 0 0 0) 0x7ff8a007a2e0 con 0x7ff8a815e900 2024-09-10T10:46:07.689 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 msgr2=0x7ff88406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 0x7ff88406e2f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff898005880 tx=0x7ff898009040 comp rx=0 tx=0).stop 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 msgr2=0x7ff8a815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff8a002ac00 tx=0x7ff8a0004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 shutdown_connections 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff88406be30 0x7ff88406e2f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8a815e900 0x7ff8a815b470 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8a80b4020 0x7ff8a815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 --2- 172.21.15.28:0/4280658313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8a8072050 0x7ff8a814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.691+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 >> 172.21.15.28:0/4280658313 conn(0x7ff8a806bdf0 msgr2=0x7ff8a815ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:07.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.692+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 shutdown_connections 2024-09-10T10:46:07.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.692+0000 7ff8aea00640 1 -- 172.21.15.28:0/4280658313 wait complete. 2024-09-10T10:46:07.723 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:46:07.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.882+0000 7f5d83e00640 1 Processor -- start 2024-09-10T10:46:07.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.882+0000 7f5d83e00640 1 -- start start 2024-09-10T10:46:07.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d83e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d840b3640 0x7f5d8406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:07.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d83e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d840b1170 con 0x7f5d840d2ac0 2024-09-10T10:46:07.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d83e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d840b1450 con 0x7f5d84161ac0 2024-09-10T10:46:07.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d83e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5d840ceba0 con 0x7f5d840b3640 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d83400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d840b3640 0x7f5d8406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.883+0000 7f5d82000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5d840d2ac0 0x7f5d840b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50736/0 (socket says 172.21.15.28:50736) 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.884+0000 7f5d82000640 1 -- 172.21.15.28:0/4284238803 learned_addr learned my addr 172.21.15.28:0/4284238803 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.884+0000 7f5d83400640 1 -- 172.21.15.28:0/4284238803 >> v1:172.21.15.138:6789/0 conn(0x7f5d84161ac0 legacy=0x7f5d840d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.884+0000 7f5d83400640 1 -- 172.21.15.28:0/4284238803 >> v1:172.21.15.28:6789/0 conn(0x7f5d840d2ac0 legacy=0x7f5d840b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.884+0000 7f5d83400640 1 -- 172.21.15.28:0/4284238803 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d840b1730 con 0x7f5d840b3640 2024-09-10T10:46:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.884+0000 7f5d81600640 1 -- 172.21.15.28:0/4284238803 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905190216 0 0) 0x7f5d840b1170 con 0x7f5d840d2ac0 2024-09-10T10:46:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.885+0000 7f5d83400640 1 --2- 172.21.15.28:0/4284238803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d8406e6c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009d90 tx=0x7f5d7c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce266b87405f05d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.885+0000 7f5d81600640 1 -- 172.21.15.28:0/4284238803 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007740 con 0x7f5d840b3640 2024-09-10T10:46:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.885+0000 7f5d81600640 1 -- 172.21.15.28:0/4284238803 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0078e0 con 0x7f5d840b3640 2024-09-10T10:46:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.885+0000 7f5d81600640 1 -- 172.21.15.28:0/4284238803 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c007cb0 con 0x7f5d840b3640 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 -- 172.21.15.28:0/4284238803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 msgr2=0x7f5d8406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 --2- 172.21.15.28:0/4284238803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d8406e6c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009d90 tx=0x7f5d7c02a730 comp rx=0 tx=0).stop 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 -- 172.21.15.28:0/4284238803 shutdown_connections 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 --2- 172.21.15.28:0/4284238803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d8406e6c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 -- 172.21.15.28:0/4284238803 >> 172.21.15.28:0/4284238803 conn(0x7f5d8408bfc0 msgr2=0x7f5d840d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 -- 172.21.15.28:0/4284238803 shutdown_connections 2024-09-10T10:46:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.886+0000 7f5d83e00640 1 -- 172.21.15.28:0/4284238803 wait complete. 2024-09-10T10:46:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.888+0000 7f5d83e00640 1 Processor -- start 2024-09-10T10:46:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.888+0000 7f5d83e00640 1 -- start start 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 0x7f5d840cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 0x7f5d840d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84161ea0 con 0x7f5d840b3640 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8406dfb0 con 0x7f5d84161ac0 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d84162020 con 0x7f5d840d2ac0 2024-09-10T10:46:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41254/0 (socket says 172.21.15.28:41254) 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82a00640 1 -- 172.21.15.28:0/1791994624 learned_addr learned my addr 172.21.15.28:0/1791994624 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d83400640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 0x7f5d840d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82000640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 0x7f5d840cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82a00640 1 -- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 msgr2=0x7f5d840cfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.889+0000 7f5d82a00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 0x7f5d840cfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d82a00640 1 -- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 msgr2=0x7f5d840d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d82a00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 0x7f5d840d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d82a00640 1 -- 172.21.15.28:0/1791994624 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d840cc670 con 0x7f5d840b3640 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d82000640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 0x7f5d840cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d83400640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 0x7f5d840d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d82a00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b3f0 tx=0x7f5d7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74015020 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d74011830 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74011c20 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d840cc950 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.890+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d840cd430 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.891+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8406e6c0 con 0x7f5d840b3640 2024-09-10T10:46:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.892+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d74018020 con 0x7f5d840b3640 2024-09-10T10:46:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.893+0000 7f5d7aa00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 0x7f5d6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.893+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6001+0+0 (secure 0 0 0) 0x7f5d7408ac20 con 0x7f5d840b3640 2024-09-10T10:46:07.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.894+0000 7f5d82000640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 0x7f5d6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:07.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.894+0000 7f5d82000640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 0x7f5d6406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d84158040 tx=0x7f5d70007c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:07.898+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d7401f030 con 0x7f5d840b3640 2024-09-10T10:46:08.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.119+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5d8415ede0 con 0x7f5d840b3640 2024-09-10T10:46:08.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.714+0000 7f5d7aa00640 1 -- 172.21.15.28:0/1791994624 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v28) ==== 172+0+0 (secure 0 0 0) 0x7f5d740172c0 con 0x7f5d840b3640 2024-09-10T10:46:08.718 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 7 and data pool 8 2024-09-10T10:46:08.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 msgr2=0x7f5d6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 0x7f5d6406e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d84158040 tx=0x7f5d70007c10 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 msgr2=0x7f5d840b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b3f0 tx=0x7f5d7400b8b0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 shutdown_connections 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d6406be30 0x7f5d6406e2f0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5d84158040 tx=0x7f5d70007c10 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d84161ac0 0x7f5d840d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d840d2ac0 0x7f5d840cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 --2- 172.21.15.28:0/1791994624 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d840b3640 0x7f5d840b6850 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b3f0 tx=0x7f5d7400b8b0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.727+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 >> 172.21.15.28:0/1791994624 conn(0x7f5d8408bfc0 msgr2=0x7f5d840b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.735+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 shutdown_connections 2024-09-10T10:46:08.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.736+0000 7f5d83e00640 1 -- 172.21.15.28:0/1791994624 wait complete. 2024-09-10T10:46:08.763 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:08.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.918+0000 7f1805000640 1 Processor -- start 2024-09-10T10:46:08.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.918+0000 7f1805000640 1 -- start start 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f1805000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1800072de0 0x7f180015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f1805000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1800146140 con 0x7f1800072700 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f1805000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18000cd740 con 0x7f1800072320 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f1805000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18000cda20 con 0x7f1800072de0 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f17fe800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1800072700 0x7f180015d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50744/0 (socket says 172.21.15.28:50744) 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f17fe800640 1 -- 172.21.15.28:0/1656926042 learned_addr learned my addr 172.21.15.28:0/1656926042 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f17f7e00640 1 --2- 172.21.15.28:0/1656926042 >> v2:172.21.15.138:3300/0 conn(0x7f1800072de0 0x7f180015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787498939 0 0) 0x7f1800146140 con 0x7f1800072700 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.920+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17e4003520 con 0x7f1800072700 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.921+0000 7f17f7e00640 1 -- 172.21.15.28:0/1656926042 >> v1:172.21.15.138:6789/0 conn(0x7f1800072320 legacy=0x7f18000cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.921+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3232626789 0 0) 0x7f18000cd740 con 0x7f1800072320 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.921+0000 7f17f7e00640 1 -- 172.21.15.28:0/1656926042 >> v1:172.21.15.28:6789/0 conn(0x7f1800072700 legacy=0x7f180015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.921+0000 7f17f7e00640 1 -- 172.21.15.28:0/1656926042 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18000cdd00 con 0x7f1800072de0 2024-09-10T10:46:08.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.922+0000 7f17f7e00640 1 --2- 172.21.15.28:0/1656926042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072de0 0x7f180015d1e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17e80097a0 tx=0x7f17e802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e80800492a939b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.922+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8037070 con 0x7f1800072de0 2024-09-10T10:46:08.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.922+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e8032440 con 0x7f1800072de0 2024-09-10T10:46:08.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.922+0000 7f17fd400640 1 -- 172.21.15.28:0/1656926042 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80327f0 con 0x7f1800072de0 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 -- 172.21.15.28:0/1656926042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072de0 msgr2=0x7f180015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 --2- 172.21.15.28:0/1656926042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072de0 0x7f180015d1e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17e80097a0 tx=0x7f17e802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 -- 172.21.15.28:0/1656926042 shutdown_connections 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 --2- 172.21.15.28:0/1656926042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072de0 0x7f180015d1e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 -- 172.21.15.28:0/1656926042 >> 172.21.15.28:0/1656926042 conn(0x7f1800067d70 msgr2=0x7f180006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 -- 172.21.15.28:0/1656926042 shutdown_connections 2024-09-10T10:46:08.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.923+0000 7f1805000640 1 -- 172.21.15.28:0/1656926042 wait complete. 2024-09-10T10:46:08.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.925+0000 7f1805000640 1 Processor -- start 2024-09-10T10:46:08.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.925+0000 7f1805000640 1 -- start start 2024-09-10T10:46:08.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 0x7f18001f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 0x7f18000c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f180015dfb0 con 0x7f1800072de0 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f180015de30 con 0x7f1800072320 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f1805000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f180015e130 con 0x7f1800072700 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41256/0 (socket says 172.21.15.28:41256) 2024-09-10T10:46:08.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 -- 172.21.15.28:0/3293979099 learned_addr learned my addr 172.21.15.28:0/3293979099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fde00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 0x7f18001f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17f7e00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 0x7f18000c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 -- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 msgr2=0x7f18000c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 0x7f18000c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 -- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 msgr2=0x7f18001f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 0x7f18001f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fe800640 1 -- 172.21.15.28:0/3293979099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18000ca8b0 con 0x7f1800072de0 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17f7e00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 0x7f18000c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.926+0000 7f17fde00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 0x7f18001f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:08.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f17fe800640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17f800cbb0 tx=0x7f17f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8017070 con 0x7f1800072de0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f800ad40 con 0x7f1800072de0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8012600 con 0x7f1800072de0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18000ca2f0 con 0x7f1800072de0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.927+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1800135740 con 0x7f1800072de0 2024-09-10T10:46:08.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.928+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18000cce60 con 0x7f1800072de0 2024-09-10T10:46:08.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.929+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f17f801a070 con 0x7f1800072de0 2024-09-10T10:46:08.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.930+0000 7f17f6000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 0x7f17dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:08.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.930+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f17f808ae20 con 0x7f1800072de0 2024-09-10T10:46:08.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.930+0000 7f17fde00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 0x7f17dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:08.932 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:46:08.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.931+0000 7f17fde00640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 0x7f17dc06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f17f0000f60 tx=0x7f17f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:08.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:46:08.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:08.935+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f17f8058ec0 con 0x7f1800072de0 2024-09-10T10:46:09.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.122+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18001439f0 con 0x7f1800072de0 2024-09-10T10:46:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.123+0000 7f17f6000640 1 -- 172.21.15.28:0/3293979099 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16979 (secure 0 0 0) 0x7f17f805d1c0 con 0x7f1800072de0 2024-09-10T10:46:09.124 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:09.125 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:08.120453+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:05.928754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:07.123356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.125+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 msgr2=0x7f17dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.125+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 0x7f17dc06e2f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f17f0000f60 tx=0x7f17f0009040 comp rx=0 tx=0).stop 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.125+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 msgr2=0x7f18000c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.125+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f17f800cbb0 tx=0x7f17f800a430 comp rx=0 tx=0).stop 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 shutdown_connections 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17dc06be30 0x7f17dc06e2f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1800072de0 0x7f18000c54d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1800072700 0x7f18000c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 --2- 172.21.15.28:0/3293979099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1800072320 0x7f18001f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 >> 172.21.15.28:0/3293979099 conn(0x7f1800067d70 msgr2=0x7f18000b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:09.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 shutdown_connections 2024-09-10T10:46:09.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.126+0000 7f1805000640 1 -- 172.21.15.28:0/3293979099 wait complete. 2024-09-10T10:46:09.143 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.299+0000 7fd38d200640 1 Processor -- start 2024-09-10T10:46:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.299+0000 7fd38d200640 1 -- start start 2024-09-10T10:46:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.300+0000 7fd38d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880d1f90 0x7fd3880cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.300+0000 7fd38d200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3880cf000 con 0x7fd38815e660 2024-09-10T10:46:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.300+0000 7fd38d200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3880d29a0 con 0x7fd3880d0dd0 2024-09-10T10:46:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.300+0000 7fd38d200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd38806d3e0 con 0x7fd3880d1f90 2024-09-10T10:46:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.300+0000 7fd386a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd38815e660 0x7fd38806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50760/0 (socket says 172.21.15.28:50760) 2024-09-10T10:46:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386a00640 1 -- 172.21.15.28:0/593638182 learned_addr learned my addr 172.21.15.28:0/593638182 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd387e00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880d1f90 0x7fd3880cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645278981 0 0) 0x7fd3880cf000 con 0x7fd38815e660 2024-09-10T10:46:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd36c003520 con 0x7fd38815e660 2024-09-10T10:46:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690447966 0 0) 0x7fd3880d29a0 con 0x7fd3880d0dd0 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3880cf000 con 0x7fd3880d0dd0 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1476594003 0 0) 0x7fd36c003520 con 0x7fd38815e660 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880d1f90 msgr2=0x7fd3880cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880d1f90 0x7fd3880cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 >> v1:172.21.15.138:6789/0 conn(0x7fd3880d0dd0 legacy=0x7fd3880d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3880d2c60 con 0x7fd38815e660 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fd374001670 con 0x7fd38815e660 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 >> v1:172.21.15.28:6789/0 conn(0x7fd38815e660 legacy=0x7fd38806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c003b70 0x7fd36c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0051c0 0x7fd36c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd38806d3e0 con 0x7fd36c004800 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c00bfe0 con 0x7fd36c003b70 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.301+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd36c00c1e0 con 0x7fd36c0051c0 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd387e00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880d1f90 0x7fd3880cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd387400640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c003b70 0x7fd36c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd387e00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0051c0 0x7fd36c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 -- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0051c0 msgr2=0x7fd36c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd36c0051c0 0x7fd36c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 -- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c003b70 msgr2=0x7fd36c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c003b70 0x7fd36c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.302+0000 7fd386a00640 1 -- 172.21.15.28:0/593638182 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c003520 con 0x7fd36c004800 2024-09-10T10:46:09.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.303+0000 7fd386a00640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd38815ea60 tx=0x7fd36c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba3e2775a45083c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.303+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374007c00 con 0x7fd36c004800 2024-09-10T10:46:09.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.303+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd374005af0 con 0x7fd36c004800 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.303+0000 7fd386000640 1 -- 172.21.15.28:0/593638182 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374007070 con 0x7fd36c004800 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 -- 172.21.15.28:0/593638182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd36c004800 msgr2=0x7fd36c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 --2- 172.21.15.28:0/593638182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd36c004800 0x7fd36c004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd38815ea60 tx=0x7fd36c005830 comp rx=0 tx=0).stop 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd387e00640 1 -- 172.21.15.28:0/593638182 reap_dead start 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 -- 172.21.15.28:0/593638182 shutdown_connections 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 -- 172.21.15.28:0/593638182 >> 172.21.15.28:0/593638182 conn(0x7fd38808bfc0 msgr2=0x7fd3880d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 -- 172.21.15.28:0/593638182 shutdown_connections 2024-09-10T10:46:09.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.304+0000 7fd38d200640 1 -- 172.21.15.28:0/593638182 wait complete. 2024-09-10T10:46:09.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.306+0000 7fd38d200640 1 Processor -- start 2024-09-10T10:46:09.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.306+0000 7fd38d200640 1 -- start start 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd3881f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 0x7fd38815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880cf330 con 0x7fd3880b9f60 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3880d3220 con 0x7fd36c004800 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd38d200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd3880d33a0 con 0x7fd3880b9b00 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd387e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd387e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41280/0 (socket says 172.21.15.28:41280) 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd387e00640 1 -- 172.21.15.28:0/4120699326 learned_addr learned my addr 172.21.15.28:0/4120699326 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd386a00640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 0x7fd38815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.307+0000 7fd387400640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd3881f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 -- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 msgr2=0x7fd38815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 0x7fd38815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 -- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 msgr2=0x7fd3881f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd3881f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 -- 172.21.15.28:0/4120699326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881e3750 con 0x7fd3880b9f60 2024-09-10T10:46:09.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd386a00640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 0x7fd38815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:09.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd387e00640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3880d0850 tx=0x7fd38000f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380015430 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38001d070 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800157f0 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881e39f0 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.308+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881e3fb0 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.309+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd388155af0 con 0x7fd3880b9f60 2024-09-10T10:46:09.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.310+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd380015990 con 0x7fd3880b9f60 2024-09-10T10:46:09.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.311+0000 7fd37f400640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 0x7fd36806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.311+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7fd38008baf0 con 0x7fd3880b9f60 2024-09-10T10:46:09.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.312+0000 7fd387400640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 0x7fd36806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.312+0000 7fd387400640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 0x7fd36806e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd378001850 tx=0x7fd378009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.315+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd380010040 con 0x7fd3880b9f60 2024-09-10T10:46:09.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.501+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3880b5bf0 con 0x7fd3880b9f60 2024-09-10T10:46:09.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.503+0000 7fd37f400640 1 -- 172.21.15.28:0/4120699326 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+16979 (secure 0 0 0) 0x7fd3880b5bf0 con 0x7fd3880b9f60 2024-09-10T10:46:09.504 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:09.504 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:08.120453+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:05.928754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:07.123356+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:09.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 msgr2=0x7fd36806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 0x7fd36806e320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd378001850 tx=0x7fd378009040 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 msgr2=0x7fd38815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd3880d0850 tx=0x7fd38000f7b0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 shutdown_connections 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd36806be60 0x7fd36806e320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3880b9f60 0x7fd38815c280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd3880b9b00 0x7fd38815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 --2- 172.21.15.28:0/4120699326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd36c004800 0x7fd3881f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 >> 172.21.15.28:0/4120699326 conn(0x7fd38808bfc0 msgr2=0x7fd3880b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 shutdown_connections 2024-09-10T10:46:09.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.505+0000 7fd38d200640 1 -- 172.21.15.28:0/4120699326 wait complete. 2024-09-10T10:46:09.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:46:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.681+0000 7f3fede00640 1 Processor -- start 2024-09-10T10:46:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.681+0000 7f3fede00640 1 -- start start 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.682+0000 7f3fede00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 0x7f3fe815f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.682+0000 7f3fede00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 0x7f3fe80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.682+0000 7f3fede00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe8067c30 con 0x7f3fe81587a0 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fede00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe815f760 con 0x7f3fe8070560 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fede00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe806ecb0 con 0x7f3fe806e7d0 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3feca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3fe81587a0 0x7f3fe806fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:55444/0 (socket says 172.21.15.28:55444) 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3feca00640 1 -- 172.21.15.28:0/3893488231 learned_addr learned my addr 172.21.15.28:0/3893488231 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:09.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fed400640 1 --2- 172.21.15.28:0/3893488231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 0x7f3fe80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/3893488231 >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 0x7f3fe815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fe7400640 1 -- 172.21.15.28:0/3893488231 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2270362561 0 0) 0x7f3fe8067c30 con 0x7f3fe81587a0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.683+0000 7f3fe7400640 1 -- 172.21.15.28:0/3893488231 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3fd0003520 con 0x7f3fe81587a0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.684+0000 7f3fed400640 1 -- 172.21.15.28:0/3893488231 >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 msgr2=0x7f3fe815f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.684+0000 7f3fed400640 1 --2- 172.21.15.28:0/3893488231 >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 0x7f3fe815f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.684+0000 7f3fed400640 1 -- 172.21.15.28:0/3893488231 >> v1:172.21.15.138:6789/0 conn(0x7f3fe81587a0 legacy=0x7f3fe806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.684+0000 7f3fed400640 1 -- 172.21.15.28:0/3893488231 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe806ef90 con 0x7f3fe806e7d0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.684+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/3893488231 >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 0x7f3fe815f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fed400640 1 --2- 172.21.15.28:0/3893488231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 0x7f3fe80ae740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009d90 tx=0x7f3fe002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8b6901750b94b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fe7400640 1 -- 172.21.15.28:0/3893488231 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007740 con 0x7f3fe806e7d0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fe7400640 1 -- 172.21.15.28:0/3893488231 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe00078e0 con 0x7f3fe806e7d0 2024-09-10T10:46:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fe7400640 1 -- 172.21.15.28:0/3893488231 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007cb0 con 0x7f3fe806e7d0 2024-09-10T10:46:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 -- 172.21.15.28:0/3893488231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 msgr2=0x7f3fe80ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 --2- 172.21.15.28:0/3893488231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 0x7f3fe80ae740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009d90 tx=0x7f3fe002a660 comp rx=0 tx=0).stop 2024-09-10T10:46:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 -- 172.21.15.28:0/3893488231 shutdown_connections 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 --2- 172.21.15.28:0/3893488231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe806e7d0 0x7f3fe80ae740 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 --2- 172.21.15.28:0/3893488231 >> v2:172.21.15.138:3300/0 conn(0x7f3fe8070560 0x7f3fe815f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.685+0000 7f3fede00640 1 -- 172.21.15.28:0/3893488231 >> 172.21.15.28:0/3893488231 conn(0x7f3fe8073380 msgr2=0x7f3fe806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.686+0000 7f3fede00640 1 -- 172.21.15.28:0/3893488231 shutdown_connections 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.686+0000 7f3fede00640 1 -- 172.21.15.28:0/3893488231 wait complete. 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.687+0000 7f3fede00640 1 Processor -- start 2024-09-10T10:46:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.687+0000 7f3fede00640 1 -- start start 2024-09-10T10:46:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 0x7f3fe8151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 0x7f3fe80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe806f830 con 0x7f3fe806e7d0 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe806f550 con 0x7f3fe81587a0 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3fede00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe8158b80 con 0x7f3fe8070560 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3feca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.688+0000 7f3feca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41286/0 (socket says 172.21.15.28:41286) 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 -- 172.21.15.28:0/2582311247 learned_addr learned my addr 172.21.15.28:0/2582311247 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fed400640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 0x7f3fe80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 0x7f3fe8151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 -- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 msgr2=0x7f3fe8151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 0x7f3fe8151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 -- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 msgr2=0x7f3fe80b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 0x7f3fe80b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 -- 172.21.15.28:0/2582311247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe815d970 con 0x7f3fe806e7d0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fed400640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 0x7f3fe80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 0x7f3fe8151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3feca00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c390 tx=0x7f3fdc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc017070 con 0x7f3fe806e7d0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc005ce0 con 0x7f3fe806e7d0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc0072f0 con 0x7f3fe806e7d0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.689+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe815dc50 con 0x7f3fe806e7d0 2024-09-10T10:46:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.690+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe8155070 con 0x7f3fe806e7d0 2024-09-10T10:46:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.690+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe815f220 con 0x7f3fe806e7d0 2024-09-10T10:46:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.691+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fdc01a070 con 0x7f3fe806e7d0 2024-09-10T10:46:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.692+0000 7f3fe4c00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.692+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6101+0+0 (secure 0 0 0) 0x7f3fdc08a440 con 0x7f3fe806e7d0 2024-09-10T10:46:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.693+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.694+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fe81605d0 tx=0x7f3fd4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.696+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3fdc058c50 con 0x7f3fe806e7d0 2024-09-10T10:46:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:09.923+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3fe815e0f0 con 0x7f3fe806e7d0 2024-09-10T10:46:10.686 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:46:10.685+0000 7fd19e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:46:10.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.725+0000 7f3fe4c00640 1 -- 172.21.15.28:0/2582311247 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f3fdc058550 con 0x7f3fe806e7d0 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.729+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 msgr2=0x7f3fc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.729+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fe81605d0 tx=0x7f3fd4008040 comp rx=0 tx=0).stop 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.729+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 msgr2=0x7f3fe814df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.729+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c390 tx=0x7f3fdc00c850 comp rx=0 tx=0).stop 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 shutdown_connections 2024-09-10T10:46:10.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe81587a0 0x7f3fe80b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe8070560 0x7f3fe8151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 --2- 172.21.15.28:0/2582311247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe806e7d0 0x7f3fe814df00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 >> 172.21.15.28:0/2582311247 conn(0x7f3fe8073380 msgr2=0x7f3fe80b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 shutdown_connections 2024-09-10T10:46:10.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.730+0000 7f3fede00640 1 -- 172.21.15.28:0/2582311247 wait complete. 2024-09-10T10:46:10.750 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:46:10.750 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:46:10.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.912+0000 7f569e400640 1 Processor -- start 2024-09-10T10:46:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.912+0000 7f569e400640 1 -- start start 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.913+0000 7f569e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 0x7f56980b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.913+0000 7f569e400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 0x7f56980cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.913+0000 7f569e400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f56980cfcd0 0x7f569815fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f569e400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f569806edc0 con 0x7f56980cfcd0 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f569e400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f56980b30e0 con 0x7f56980b4af0 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f569e400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56980b33c0 con 0x7f5698156740 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f56980cfcd0 0x7f569815fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f56980cfcd0 0x7f569815fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41288/0 (socket says 172.21.15.28:41288) 2024-09-10T10:46:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 -- 172.21.15.28:0/3135845232 learned_addr learned my addr 172.21.15.28:0/3135845232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f568fe00640 1 --2- 172.21.15.28:0/3135845232 >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 0x7f56980cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 -- 172.21.15.28:0/3135845232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 msgr2=0x7f56980b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697400640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 0x7f56980b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 0x7f56980b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 -- 172.21.15.28:0/3135845232 >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 msgr2=0x7f56980cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 --2- 172.21.15.28:0/3135845232 >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 0x7f56980cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697e00640 1 -- 172.21.15.28:0/3135845232 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56980b36a0 con 0x7f56980cfcd0 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f5697400640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 0x7f56980b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.914+0000 7f568fe00640 1 --2- 172.21.15.28:0/3135845232 >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 0x7f56980cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.915+0000 7f5697e00640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56980cfcd0 0x7f569815fe20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f56980cd9a0 tx=0x7f569002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26065465bb050024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f5696a00640 1 -- 172.21.15.28:0/3135845232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690036070 con 0x7f56980cfcd0 2024-09-10T10:46:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f5696a00640 1 -- 172.21.15.28:0/3135845232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690031600 con 0x7f56980cfcd0 2024-09-10T10:46:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f5696a00640 1 -- 172.21.15.28:0/3135845232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56900319d0 con 0x7f56980cfcd0 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 -- 172.21.15.28:0/3135845232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56980cfcd0 msgr2=0x7f569815fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56980cfcd0 0x7f569815fe20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f56980cd9a0 tx=0x7f569002a6a0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 -- 172.21.15.28:0/3135845232 shutdown_connections 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56980cfcd0 0x7f569815fe20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 --2- 172.21.15.28:0/3135845232 >> v2:172.21.15.138:3300/0 conn(0x7f56980b4af0 0x7f56980cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 --2- 172.21.15.28:0/3135845232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5698156740 0x7f56980b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 -- 172.21.15.28:0/3135845232 >> 172.21.15.28:0/3135845232 conn(0x7f569806bdf0 msgr2=0x7f569806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 -- 172.21.15.28:0/3135845232 shutdown_connections 2024-09-10T10:46:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.916+0000 7f569e400640 1 -- 172.21.15.28:0/3135845232 wait complete. 2024-09-10T10:46:10.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.918+0000 7f569e400640 1 Processor -- start 2024-09-10T10:46:10.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.918+0000 7f569e400640 1 -- start start 2024-09-10T10:46:10.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56980b4af0 0x7f569815b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56980cfcd0 0x7f56981f4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56980d0890 con 0x7f5698156740 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56980b3c60 con 0x7f56980cfcd0 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f569e400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56980d0a10 con 0x7f56980b4af0 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f5697e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f5697e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41302/0 (socket says 172.21.15.28:41302) 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f5697e00640 1 -- 172.21.15.28:0/3462393807 learned_addr learned my addr 172.21.15.28:0/3462393807 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f5697400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56980b4af0 0x7f569815b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.919+0000 7f568fe00640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56980cfcd0 0x7f56981f4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 -- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56980b4af0 msgr2=0x7f569815b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56980b4af0 0x7f569815b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 -- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56980cfcd0 msgr2=0x7f56981f4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56980cfcd0 0x7f56981f4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 -- 172.21.15.28:0/3462393807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56981f40a0 con 0x7f5698156740 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f5697e00640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f40 tx=0x7f5690004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690036050 con 0x7f5698156740 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5690042070 con 0x7f5698156740 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690009510 con 0x7f5698156740 2024-09-10T10:46:10.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56981f4380 con 0x7f5698156740 2024-09-10T10:46:10.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.920+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56981f3d00 con 0x7f5698156740 2024-09-10T10:46:10.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.921+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5698156b20 con 0x7f5698156740 2024-09-10T10:46:10.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.922+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5690007bd0 con 0x7f5698156740 2024-09-10T10:46:10.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.923+0000 7f568f400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 0x7f567406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:10.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.923+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f56900abfc0 con 0x7f5698156740 2024-09-10T10:46:10.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.924+0000 7f5697400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 0x7f567406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:10.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.925+0000 7f5697400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 0x7f567406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f568800a8d0 tx=0x7f5688008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:10.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:10.928+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5690045360 con 0x7f5698156740 2024-09-10T10:46:11.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.147+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f569806ce20 con 0x7f5698156740 2024-09-10T10:46:11.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.727+0000 7f568f400640 1 -- 172.21.15.28:0/3462393807 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f569007a7d0 con 0x7f5698156740 2024-09-10T10:46:11.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 msgr2=0x7f567406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 0x7f567406e2f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f568800a8d0 tx=0x7f5688008040 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 msgr2=0x7f56981531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f40 tx=0x7f5690004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 shutdown_connections 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f567406be30 0x7f567406e2f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5698156740 0x7f56981531e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56980cfcd0 0x7f56981f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 --2- 172.21.15.28:0/3462393807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56980b4af0 0x7f569815b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 >> 172.21.15.28:0/3462393807 conn(0x7f569806bdf0 msgr2=0x7f56980b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 shutdown_connections 2024-09-10T10:46:11.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.730+0000 7f569e400640 1 -- 172.21.15.28:0/3462393807 wait complete. 2024-09-10T10:46:11.750 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:46:11.751 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:11.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.913+0000 7f4da5400640 1 Processor -- start 2024-09-10T10:46:11.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.913+0000 7f4da5400640 1 -- start start 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4da5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4da00d0450 0x7f4da0169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4da5400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da00d2c20 con 0x7f4da00d4180 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4da5400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da006e390 con 0x7f4da015e6a0 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4da5400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4da00cefb0 con 0x7f4da00d0450 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4d9fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4da00d4180 0x7f4da00d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50762/0 (socket says 172.21.15.28:50762) 2024-09-10T10:46:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4d9fe00640 1 -- 172.21.15.28:0/4039489225 learned_addr learned my addr 172.21.15.28:0/4039489225 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:11.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4da4a00640 1 --2- 172.21.15.28:0/4039489225 >> v2:172.21.15.138:3300/0 conn(0x7f4da00d0450 0x7f4da0169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.914+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3983581875 0 0) 0x7f4da00d2c20 con 0x7f4da00d4180 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d84003520 con 0x7f4da00d4180 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 101880688 0 0) 0x7f4da006e390 con 0x7f4da015e6a0 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4da00d2c20 con 0x7f4da015e6a0 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4da4a00640 1 -- 172.21.15.28:0/4039489225 >> v1:172.21.15.138:6789/0 conn(0x7f4da015e6a0 legacy=0x7f4da006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4da4a00640 1 -- 172.21.15.28:0/4039489225 >> v1:172.21.15.28:6789/0 conn(0x7f4da00d4180 legacy=0x7f4da00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.915+0000 7f4da4a00640 1 -- 172.21.15.28:0/4039489225 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da006e670 con 0x7f4da00d0450 2024-09-10T10:46:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.916+0000 7f4da4a00640 1 --2- 172.21.15.28:0/4039489225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da0169000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009d90 tx=0x7f4d9802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a79f1a08de763e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.916+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007740 con 0x7f4da00d0450 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.916+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d980078e0 con 0x7f4da00d0450 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.916+0000 7f4d9f400640 1 -- 172.21.15.28:0/4039489225 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98007c90 con 0x7f4da00d0450 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 -- 172.21.15.28:0/4039489225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 msgr2=0x7f4da0169000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 --2- 172.21.15.28:0/4039489225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da0169000 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009d90 tx=0x7f4d9802a660 comp rx=0 tx=0).stop 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 -- 172.21.15.28:0/4039489225 shutdown_connections 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 --2- 172.21.15.28:0/4039489225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da0169000 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 -- 172.21.15.28:0/4039489225 >> 172.21.15.28:0/4039489225 conn(0x7f4da009d6f0 msgr2=0x7f4da008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 -- 172.21.15.28:0/4039489225 shutdown_connections 2024-09-10T10:46:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.917+0000 7f4da5400640 1 -- 172.21.15.28:0/4039489225 wait complete. 2024-09-10T10:46:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.919+0000 7f4da5400640 1 Processor -- start 2024-09-10T10:46:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.919+0000 7f4da5400640 1 -- start start 2024-09-10T10:46:11.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da00d4180 0x7f4da00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4da006df30 con 0x7f4da015e6a0 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00cebd0 con 0x7f4da00d0450 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da5400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4da006e0b0 con 0x7f4da00d4180 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4d9fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41310/0 (socket says 172.21.15.28:41310) 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4da4a00640 1 -- 172.21.15.28:0/199016134 learned_addr learned my addr 172.21.15.28:0/199016134 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.920+0000 7f4d97e00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da00d4180 0x7f4da00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 -- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da00d4180 msgr2=0x7f4da00c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da00d4180 0x7f4da00c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 -- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 msgr2=0x7f4da014fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da014fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 -- 172.21.15.28:0/199016134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da01500f0 con 0x7f4da015e6a0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4d9fe00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da4a00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009eb0 tx=0x7f4d98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9803f070 con 0x7f4da015e6a0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d98031c60 con 0x7f4da015e6a0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98009450 con 0x7f4da015e6a0 2024-09-10T10:46:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.921+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00c83c0 con 0x7f4da015e6a0 2024-09-10T10:46:11.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.922+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00c8980 con 0x7f4da015e6a0 2024-09-10T10:46:11.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.922+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00d4560 con 0x7f4da015e6a0 2024-09-10T10:46:11.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.923+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4d9803b410 con 0x7f4da015e6a0 2024-09-10T10:46:11.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.924+0000 7f4d9cc00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 0x7f4d7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.924+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f4d980ac230 con 0x7f4da015e6a0 2024-09-10T10:46:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.925+0000 7f4d9fe00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 0x7f4d7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.925+0000 7f4d9fe00640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 0x7f4d7c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:11.929+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4da01500f0 con 0x7f4da015e6a0 2024-09-10T10:46:12.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.147+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4da00ce7d0 con 0x7f4da015e6a0 2024-09-10T10:46:12.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.148+0000 7f4d9cc00640 1 -- 172.21.15.28:0/199016134 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7f4d9807a360 con 0x7f4da015e6a0 2024-09-10T10:46:12.149 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:12.149 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":32,"btime":"2024-09-10T10:46:11:726443+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:11.149029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/20691839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":20691839},{"type":"v1","addr":"172.21.15.28:6835","nonce":20691839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":3}]} 2024-09-10T10:46:12.150 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 32 2024-09-10T10:46:12.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 msgr2=0x7f4d7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 0x7f4d7c06e2f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90007d70 comp rx=0 tx=0).stop 2024-09-10T10:46:12.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 msgr2=0x7f4da00c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009eb0 tx=0x7f4d98004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 shutdown_connections 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4d7c06be30 0x7f4d7c06e2f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4da015e6a0 0x7f4da00c9c40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4da00d4180 0x7f4da00c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 --2- 172.21.15.28:0/199016134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4da00d0450 0x7f4da014fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 >> 172.21.15.28:0/199016134 conn(0x7f4da009d6f0 msgr2=0x7f4da00b9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 shutdown_connections 2024-09-10T10:46:12.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.150+0000 7f4da5400640 1 -- 172.21.15.28:0/199016134 wait complete. 2024-09-10T10:46:12.167 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:12.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.321+0000 7f59a7800640 1 Processor -- start 2024-09-10T10:46:12.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.321+0000 7f59a7800640 1 -- start start 2024-09-10T10:46:12.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.322+0000 7f59a7800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a7800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f59a00727c0 0x7f59a00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a7800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a006f480 con 0x7f59a0147e90 2024-09-10T10:46:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a7800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f59a00b34d0 con 0x7f59a00727c0 2024-09-10T10:46:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a7800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00ae9e0 con 0x7f59a0147230 2024-09-10T10:46:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f599fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f59a0147e90 0x7f59a00720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50774/0 (socket says 172.21.15.28:50774) 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f599fe00640 1 -- 172.21.15.28:0/2272903549 learned_addr learned my addr 172.21.15.28:0/2272903549 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a5800640 1 --2- 172.21.15.28:0/2272903549 >> v2:172.21.15.28:3300/0 conn(0x7f59a00727c0 0x7f59a00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f59a4e00640 1 --2- 172.21.15.28:0/2272903549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.324+0000 7f59a4e00640 1 -- 172.21.15.28:0/2272903549 >> v2:172.21.15.28:3300/0 conn(0x7f59a00727c0 msgr2=0x7f59a00aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.323+0000 7f599f400640 1 -- 172.21.15.28:0/2272903549 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827721343 0 0) 0x7f59a006f480 con 0x7f59a0147e90 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.324+0000 7f59a4e00640 1 --2- 172.21.15.28:0/2272903549 >> v2:172.21.15.28:3300/0 conn(0x7f59a00727c0 0x7f59a00aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.324+0000 7f59a4e00640 1 -- 172.21.15.28:0/2272903549 >> v1:172.21.15.28:6789/0 conn(0x7f59a0147e90 legacy=0x7f59a00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.324+0000 7f59a4e00640 1 -- 172.21.15.28:0/2272903549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a006e900 con 0x7f59a0147230 2024-09-10T10:46:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f59a4e00640 1 --2- 172.21.15.28:0/2272903549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a006e0b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59940097a0 tx=0x7f599402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e3aefbee5f5319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f599f400640 1 -- 172.21.15.28:0/2272903549 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994037070 con 0x7f59a0147230 2024-09-10T10:46:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f599f400640 1 -- 172.21.15.28:0/2272903549 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5994032440 con 0x7f59a0147230 2024-09-10T10:46:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f599f400640 1 -- 172.21.15.28:0/2272903549 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994032810 con 0x7f59a0147230 2024-09-10T10:46:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f59a7800640 1 -- 172.21.15.28:0/2272903549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 msgr2=0x7f59a006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.325+0000 7f59a7800640 1 --2- 172.21.15.28:0/2272903549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a006e0b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59940097a0 tx=0x7f599402a180 comp rx=0 tx=0).stop 2024-09-10T10:46:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 -- 172.21.15.28:0/2272903549 shutdown_connections 2024-09-10T10:46:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 --2- 172.21.15.28:0/2272903549 >> v2:172.21.15.28:3300/0 conn(0x7f59a00727c0 0x7f59a00aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 --2- 172.21.15.28:0/2272903549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a006e0b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 -- 172.21.15.28:0/2272903549 >> 172.21.15.28:0/2272903549 conn(0x7f59a0067d70 msgr2=0x7f59a006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 -- 172.21.15.28:0/2272903549 shutdown_connections 2024-09-10T10:46:12.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.326+0000 7f59a7800640 1 -- 172.21.15.28:0/2272903549 wait complete. 2024-09-10T10:46:12.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.327+0000 7f59a7800640 1 Processor -- start 2024-09-10T10:46:12.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.327+0000 7f59a7800640 1 -- start start 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a01f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59a0147e90 0x7f59a0140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59a006f1a0 con 0x7f59a00727c0 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59a006eec0 con 0x7f59a0147e90 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a7800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59a006e5f0 con 0x7f59a0147230 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.328+0000 7f59a4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41326/0 (socket says 172.21.15.28:41326) 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 -- 172.21.15.28:0/1990175234 learned_addr learned my addr 172.21.15.28:0/1990175234 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f599fe00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a01f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a5800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59a0147e90 0x7f59a0140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 -- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 msgr2=0x7f59a01f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a01f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 -- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59a0147e90 msgr2=0x7f59a0140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59a0147e90 0x7f59a0140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 -- 172.21.15.28:0/1990175234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a01f4a10 con 0x7f59a00727c0 2024-09-10T10:46:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f599fe00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a01f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.329+0000 7f59a4e00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5994000c00 tx=0x7f59940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.330+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994037040 con 0x7f59a00727c0 2024-09-10T10:46:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.330+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5994042070 con 0x7f59a00727c0 2024-09-10T10:46:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.330+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994030940 con 0x7f59a00727c0 2024-09-10T10:46:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.330+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a0141340 con 0x7f59a00727c0 2024-09-10T10:46:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.330+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a0140040 con 0x7f59a00727c0 2024-09-10T10:46:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.331+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a0147610 con 0x7f59a00727c0 2024-09-10T10:46:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.332+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5994030ae0 con 0x7f59a00727c0 2024-09-10T10:46:12.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.333+0000 7f599cc00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 0x7f598006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.333+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f59940ab920 con 0x7f59a00727c0 2024-09-10T10:46:12.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.333+0000 7f599fe00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 0x7f598006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.334+0000 7f599fe00640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 0x7f598006e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59a00b2ce0 tx=0x7f598c007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.337+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59a01f4a10 con 0x7f59a00727c0 2024-09-10T10:46:12.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.522+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59a00008d0 con 0x7f59a00727c0 2024-09-10T10:46:12.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.523+0000 7f599cc00640 1 -- 172.21.15.28:0/1990175234 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16966 (secure 0 0 0) 0x7f599407a130 con 0x7f59a00727c0 2024-09-10T10:46:12.524 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:12.524 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":56,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:09.712358+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:05.928754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:07.123356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:12.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 msgr2=0x7f598006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 0x7f598006e2f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59a00b2ce0 tx=0x7f598c007ca0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 msgr2=0x7f59a00cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5994000c00 tx=0x7f59940046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 shutdown_connections 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f598006be30 0x7f598006e2f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59a0147e90 0x7f59a0140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59a0147230 0x7f59a01f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 --2- 172.21.15.28:0/1990175234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59a00727c0 0x7f59a00cf360 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 >> 172.21.15.28:0/1990175234 conn(0x7f59a0067d70 msgr2=0x7f59a00b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.525+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 shutdown_connections 2024-09-10T10:46:12.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.526+0000 7f59a7800640 1 -- 172.21.15.28:0/1990175234 wait complete. 2024-09-10T10:46:12.542 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:46:12.543 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:12.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.698+0000 7f98c9a00640 1 Processor -- start 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.698+0000 7f98c9a00640 1 -- start start 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c9a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f98c4072320 0x7f98c40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c9a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c4146140 con 0x7f98c4072700 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c9a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f98c40cd740 con 0x7f98c4072320 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98c40cda20 con 0x7f98c4072de0 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f98c4072320 0x7f98c40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c3400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f98c4072700 0x7f98c406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50784/0 (socket says 172.21.15.28:50784) 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.699+0000 7f98c3400640 1 -- 172.21.15.28:0/3990244026 learned_addr learned my addr 172.21.15.28:0/3990244026 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c2000640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c2a00640 1 -- 172.21.15.28:0/3990244026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 msgr2=0x7f98c406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:12.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c2a00640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c2a00640 1 -- 172.21.15.28:0/3990244026 >> v1:172.21.15.28:6789/0 conn(0x7f98c4072700 legacy=0x7f98c406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c1600640 1 -- 172.21.15.28:0/3990244026 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908495802 0 0) 0x7f98c4146140 con 0x7f98c4072700 2024-09-10T10:46:12.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.700+0000 7f98c2a00640 1 -- 172.21.15.28:0/3990244026 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c40cdd00 con 0x7f98c4072320 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.701+0000 7f98c2a00640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072320 0x7f98c40cce60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097a0 tx=0x7f98b402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6796aff08d989f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.701+0000 7f98c1600640 1 -- 172.21.15.28:0/3990244026 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4037070 con 0x7f98c4072320 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.701+0000 7f98c1600640 1 -- 172.21.15.28:0/3990244026 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b402ae50 con 0x7f98c4072320 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.701+0000 7f98c1600640 1 -- 172.21.15.28:0/3990244026 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4032660 con 0x7f98c4072320 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 -- 172.21.15.28:0/3990244026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072320 msgr2=0x7f98c40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072320 0x7f98c40cce60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097a0 tx=0x7f98b402a100 comp rx=0 tx=0).stop 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 -- 172.21.15.28:0/3990244026 shutdown_connections 2024-09-10T10:46:12.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 --2- 172.21.15.28:0/3990244026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072320 0x7f98c40cce60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 -- 172.21.15.28:0/3990244026 >> 172.21.15.28:0/3990244026 conn(0x7f98c4067d70 msgr2=0x7f98c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 -- 172.21.15.28:0/3990244026 shutdown_connections 2024-09-10T10:46:12.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.702+0000 7f98c9a00640 1 -- 172.21.15.28:0/3990244026 wait complete. 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.703+0000 7f98c9a00640 1 Processor -- start 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.704+0000 7f98c9a00640 1 -- start start 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.704+0000 7f98c9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c4072320 0x7f98c41e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.704+0000 7f98c9a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.704+0000 7f98c9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c9a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40cd520 con 0x7f98c4072700 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98c40cd3a0 con 0x7f98c4072320 2024-09-10T10:46:12.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98c406ed00 con 0x7f98c4072de0 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41344/0 (socket says 172.21.15.28:41344) 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 -- 172.21.15.28:0/1365199545 learned_addr learned my addr 172.21.15.28:0/1365199545 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c4072320 0x7f98c41e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c3400640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 -- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 msgr2=0x7f98c415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c415b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 -- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c4072320 msgr2=0x7f98c41e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c4072320 0x7f98c41e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c2000640 1 -- 172.21.15.28:0/1365199545 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41e27d0 con 0x7f98c4072700 2024-09-10T10:46:12.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.705+0000 7f98c3400640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98c2000640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d800 tx=0x7f98b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b000b6c0 con 0x7f98c4072700 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b0019070 con 0x7f98c4072700 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b000ba80 con 0x7f98c4072700 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c41e2b10 con 0x7f98c4072700 2024-09-10T10:46:12.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.706+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c40a9770 con 0x7f98c4072700 2024-09-10T10:46:12.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.707+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c40cce60 con 0x7f98c4072700 2024-09-10T10:46:12.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.708+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98b000bc20 con 0x7f98c4072700 2024-09-10T10:46:12.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.709+0000 7f98baa00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:12.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.709+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f98b008b430 con 0x7f98c4072700 2024-09-10T10:46:12.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.709+0000 7f98c2a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:12.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.710+0000 7f98c2a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 0x7f98a406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98b4034000 tx=0x7f98b402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:12.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.713+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98c41e2b10 con 0x7f98c4072700 2024-09-10T10:46:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.895+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98c4075790 con 0x7f98a406be30 2024-09-10T10:46:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.903+0000 7f98baa00640 1 -- 172.21.15.28:0/1365199545 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7f98c4075790 con 0x7f98a406be30 2024-09-10T10:46:12.907 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.906+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 msgr2=0x7f98a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.906+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 0x7f98a406e2f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f98b4034000 tx=0x7f98b402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.906+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 msgr2=0x7f98c40ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.906+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d800 tx=0x7f98b000dcc0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.906+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 shutdown_connections 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98a406be30 0x7f98a406e2f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98c4072de0 0x7f98c415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98c4072700 0x7f98c40ca070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 --2- 172.21.15.28:0/1365199545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98c4072320 0x7f98c41e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 >> 172.21.15.28:0/1365199545 conn(0x7f98c4067d70 msgr2=0x7f98c40c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 shutdown_connections 2024-09-10T10:46:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:12.907+0000 7f98c9a00640 1 -- 172.21.15.28:0/1365199545 wait complete. 2024-09-10T10:46:12.924 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T10:46:12.517753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239224,"kb_used_data":6324,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735496,"statfs":{"total":773094113280,"available":772849147904,"internally_reserved":0,"allocated":6475776,"data_stored":3511663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.831169"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725248+0000","last_change":"2024-09-10T10:46:08.731491+0000","last_active":"2024-09-10T10:46:09.725248+0000","last_peered":"2024-09-10T10:46:09.725248+0000","last_clean":"2024-09-10T10:46:09.725248+0000","last_became_active":"2024-09-10T10:46:08.731001+0000","last_became_peered":"2024-09-10T10:46:08.731001+0000","last_unstale":"2024-09-10T10:46:09.725248+0000","last_undegraded":"2024-09-10T10:46:09.725248+0000","last_fullsized":"2024-09-10T10:46:09.725248+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:59.699199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814157+0000","last_change":"2024-09-10T10:46:08.736172+0000","last_active":"2024-09-10T10:46:09.814157+0000","last_peered":"2024-09-10T10:46:09.814157+0000","last_clean":"2024-09-10T10:46:09.814157+0000","last_became_active":"2024-09-10T10:46:08.735806+0000","last_became_peered":"2024-09-10T10:46:08.735806+0000","last_unstale":"2024-09-10T10:46:09.814157+0000","last_undegraded":"2024-09-10T10:46:09.814157+0000","last_fullsized":"2024-09-10T10:46:09.814157+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:19.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723807+0000","last_change":"2024-09-10T10:46:08.735286+0000","last_active":"2024-09-10T10:46:09.723807+0000","last_peered":"2024-09-10T10:46:09.723807+0000","last_clean":"2024-09-10T10:46:09.723807+0000","last_became_active":"2024-09-10T10:46:08.735028+0000","last_became_peered":"2024-09-10T10:46:08.735028+0000","last_unstale":"2024-09-10T10:46:09.723807+0000","last_undegraded":"2024-09-10T10:46:09.723807+0000","last_fullsized":"2024-09-10T10:46:09.723807+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:25.081964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814391+0000","last_change":"2024-09-10T10:46:08.722002+0000","last_active":"2024-09-10T10:46:09.814391+0000","last_peered":"2024-09-10T10:46:09.814391+0000","last_clean":"2024-09-10T10:46:09.814391+0000","last_became_active":"2024-09-10T10:46:08.721729+0000","last_became_peered":"2024-09-10T10:46:08.721729+0000","last_unstale":"2024-09-10T10:46:09.814391+0000","last_undegraded":"2024-09-10T10:46:09.814391+0000","last_fullsized":"2024-09-10T10:46:09.814391+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813826+0000","last_change":"2024-09-10T10:46:08.728209+0000","last_active":"2024-09-10T10:46:09.813826+0000","last_peered":"2024-09-10T10:46:09.813826+0000","last_clean":"2024-09-10T10:46:09.813826+0000","last_became_active":"2024-09-10T10:46:08.727995+0000","last_became_peered":"2024-09-10T10:46:08.727995+0000","last_unstale":"2024-09-10T10:46:09.813826+0000","last_undegraded":"2024-09-10T10:46:09.813826+0000","last_fullsized":"2024-09-10T10:46:09.813826+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813299+0000","last_change":"2024-09-10T10:46:08.722366+0000","last_active":"2024-09-10T10:46:09.813299+0000","last_peered":"2024-09-10T10:46:09.813299+0000","last_clean":"2024-09-10T10:46:09.813299+0000","last_became_active":"2024-09-10T10:46:08.721216+0000","last_became_peered":"2024-09-10T10:46:08.721216+0000","last_unstale":"2024-09-10T10:46:09.813299+0000","last_undegraded":"2024-09-10T10:46:09.813299+0000","last_fullsized":"2024-09-10T10:46:09.813299+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:19.587816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.732487+0000","last_change":"2024-09-10T10:46:08.727859+0000","last_active":"2024-09-10T10:46:08.732487+0000","last_peered":"2024-09-10T10:46:08.732487+0000","last_clean":"2024-09-10T10:46:08.732487+0000","last_became_active":"2024-09-10T10:46:08.727591+0000","last_became_peered":"2024-09-10T10:46:08.727591+0000","last_unstale":"2024-09-10T10:46:08.732487+0000","last_undegraded":"2024-09-10T10:46:08.732487+0000","last_fullsized":"2024-09-10T10:46:08.732487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:14.148707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813223+0000","last_change":"2024-09-10T10:46:08.727596+0000","last_active":"2024-09-10T10:46:09.813223+0000","last_peered":"2024-09-10T10:46:09.813223+0000","last_clean":"2024-09-10T10:46:09.813223+0000","last_became_active":"2024-09-10T10:46:08.727279+0000","last_became_peered":"2024-09-10T10:46:08.727279+0000","last_unstale":"2024-09-10T10:46:09.813223+0000","last_undegraded":"2024-09-10T10:46:09.813223+0000","last_fullsized":"2024-09-10T10:46:09.813223+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:50.976281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725740+0000","last_change":"2024-09-10T10:46:08.732159+0000","last_active":"2024-09-10T10:46:09.725740+0000","last_peered":"2024-09-10T10:46:09.725740+0000","last_clean":"2024-09-10T10:46:09.725740+0000","last_became_active":"2024-09-10T10:46:08.731852+0000","last_became_peered":"2024-09-10T10:46:08.731852+0000","last_unstale":"2024-09-10T10:46:09.725740+0000","last_undegraded":"2024-09-10T10:46:09.725740+0000","last_fullsized":"2024-09-10T10:46:09.725740+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.885943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.745802+0000","last_change":"2024-09-10T10:46:08.740693+0000","last_active":"2024-09-10T10:46:08.745802+0000","last_peered":"2024-09-10T10:46:08.745802+0000","last_clean":"2024-09-10T10:46:08.745802+0000","last_became_active":"2024-09-10T10:46:08.740409+0000","last_became_peered":"2024-09-10T10:46:08.740409+0000","last_unstale":"2024-09-10T10:46:08.745802+0000","last_undegraded":"2024-09-10T10:46:08.745802+0000","last_fullsized":"2024-09-10T10:46:08.745802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:17.881635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724643+0000","last_change":"2024-09-10T10:46:08.737434+0000","last_active":"2024-09-10T10:46:09.724643+0000","last_peered":"2024-09-10T10:46:09.724643+0000","last_clean":"2024-09-10T10:46:09.724643+0000","last_became_active":"2024-09-10T10:46:08.737147+0000","last_became_peered":"2024-09-10T10:46:08.737147+0000","last_unstale":"2024-09-10T10:46:09.724643+0000","last_undegraded":"2024-09-10T10:46:09.724643+0000","last_fullsized":"2024-09-10T10:46:09.724643+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:08.531864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813300+0000","last_change":"2024-09-10T10:46:08.723796+0000","last_active":"2024-09-10T10:46:09.813300+0000","last_peered":"2024-09-10T10:46:09.813300+0000","last_clean":"2024-09-10T10:46:09.813300+0000","last_became_active":"2024-09-10T10:46:08.723499+0000","last_became_peered":"2024-09-10T10:46:08.723499+0000","last_unstale":"2024-09-10T10:46:09.813300+0000","last_undegraded":"2024-09-10T10:46:09.813300+0000","last_fullsized":"2024-09-10T10:46:09.813300+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:04.324094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813383+0000","last_change":"2024-09-10T10:46:08.718578+0000","last_active":"2024-09-10T10:46:09.813383+0000","last_peered":"2024-09-10T10:46:09.813383+0000","last_clean":"2024-09-10T10:46:09.813383+0000","last_became_active":"2024-09-10T10:46:08.718226+0000","last_became_peered":"2024-09-10T10:46:08.718226+0000","last_unstale":"2024-09-10T10:46:09.813383+0000","last_undegraded":"2024-09-10T10:46:09.813383+0000","last_fullsized":"2024-09-10T10:46:09.813383+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723965+0000","last_change":"2024-09-10T10:46:08.734994+0000","last_active":"2024-09-10T10:46:09.723965+0000","last_peered":"2024-09-10T10:46:09.723965+0000","last_clean":"2024-09-10T10:46:09.723965+0000","last_became_active":"2024-09-10T10:46:08.734622+0000","last_became_peered":"2024-09-10T10:46:08.734622+0000","last_unstale":"2024-09-10T10:46:09.723965+0000","last_undegraded":"2024-09-10T10:46:09.723965+0000","last_fullsized":"2024-09-10T10:46:09.723965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.340396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813373+0000","last_change":"2024-09-10T10:46:08.742258+0000","last_active":"2024-09-10T10:46:09.813373+0000","last_peered":"2024-09-10T10:46:09.813373+0000","last_clean":"2024-09-10T10:46:09.813373+0000","last_became_active":"2024-09-10T10:46:08.741356+0000","last_became_peered":"2024-09-10T10:46:08.741356+0000","last_unstale":"2024-09-10T10:46:09.813373+0000","last_undegraded":"2024-09-10T10:46:09.813373+0000","last_fullsized":"2024-09-10T10:46:09.813373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:04.324094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813354+0000","last_change":"2024-09-10T10:46:08.714995+0000","last_active":"2024-09-10T10:46:09.813354+0000","last_peered":"2024-09-10T10:46:09.813354+0000","last_clean":"2024-09-10T10:46:09.813354+0000","last_became_active":"2024-09-10T10:46:08.714707+0000","last_became_peered":"2024-09-10T10:46:08.714707+0000","last_unstale":"2024-09-10T10:46:09.813354+0000","last_undegraded":"2024-09-10T10:46:09.813354+0000","last_fullsized":"2024-09-10T10:46:09.813354+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725490+0000","last_change":"2024-09-10T10:46:08.731261+0000","last_active":"2024-09-10T10:46:09.725490+0000","last_peered":"2024-09-10T10:46:09.725490+0000","last_clean":"2024-09-10T10:46:09.725490+0000","last_became_active":"2024-09-10T10:46:08.730617+0000","last_became_peered":"2024-09-10T10:46:08.730617+0000","last_unstale":"2024-09-10T10:46:09.725490+0000","last_undegraded":"2024-09-10T10:46:09.725490+0000","last_fullsized":"2024-09-10T10:46:09.725490+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:26.132612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814459+0000","last_change":"2024-09-10T10:46:08.720781+0000","last_active":"2024-09-10T10:46:09.814459+0000","last_peered":"2024-09-10T10:46:09.814459+0000","last_clean":"2024-09-10T10:46:09.814459+0000","last_became_active":"2024-09-10T10:46:08.720467+0000","last_became_peered":"2024-09-10T10:46:08.720467+0000","last_unstale":"2024-09-10T10:46:09.814459+0000","last_undegraded":"2024-09-10T10:46:09.814459+0000","last_fullsized":"2024-09-10T10:46:09.814459+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724000+0000","last_change":"2024-09-10T10:46:08.728441+0000","last_active":"2024-09-10T10:46:09.724000+0000","last_peered":"2024-09-10T10:46:09.724000+0000","last_clean":"2024-09-10T10:46:09.724000+0000","last_became_active":"2024-09-10T10:46:08.728187+0000","last_became_peered":"2024-09-10T10:46:08.728187+0000","last_unstale":"2024-09-10T10:46:09.724000+0000","last_undegraded":"2024-09-10T10:46:09.724000+0000","last_fullsized":"2024-09-10T10:46:09.724000+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:06.385445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813911+0000","last_change":"2024-09-10T10:46:08.731118+0000","last_active":"2024-09-10T10:46:09.813911+0000","last_peered":"2024-09-10T10:46:09.813911+0000","last_clean":"2024-09-10T10:46:09.813911+0000","last_became_active":"2024-09-10T10:46:08.730817+0000","last_became_peered":"2024-09-10T10:46:08.730817+0000","last_unstale":"2024-09-10T10:46:09.813911+0000","last_undegraded":"2024-09-10T10:46:09.813911+0000","last_fullsized":"2024-09-10T10:46:09.813911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:27.127913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813969+0000","last_change":"2024-09-10T10:46:08.740265+0000","last_active":"2024-09-10T10:46:09.813969+0000","last_peered":"2024-09-10T10:46:09.813969+0000","last_clean":"2024-09-10T10:46:09.813969+0000","last_became_active":"2024-09-10T10:46:08.740022+0000","last_became_peered":"2024-09-10T10:46:08.740022+0000","last_unstale":"2024-09-10T10:46:09.813969+0000","last_undegraded":"2024-09-10T10:46:09.813969+0000","last_fullsized":"2024-09-10T10:46:09.813969+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724604+0000","last_change":"2024-09-10T10:46:08.737717+0000","last_active":"2024-09-10T10:46:09.724604+0000","last_peered":"2024-09-10T10:46:09.724604+0000","last_clean":"2024-09-10T10:46:09.724604+0000","last_became_active":"2024-09-10T10:46:08.735752+0000","last_became_peered":"2024-09-10T10:46:08.735752+0000","last_unstale":"2024-09-10T10:46:09.724604+0000","last_undegraded":"2024-09-10T10:46:09.724604+0000","last_fullsized":"2024-09-10T10:46:09.724604+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:47.039950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724086+0000","last_change":"2024-09-10T10:46:08.740244+0000","last_active":"2024-09-10T10:46:09.724086+0000","last_peered":"2024-09-10T10:46:09.724086+0000","last_clean":"2024-09-10T10:46:09.724086+0000","last_became_active":"2024-09-10T10:46:08.739402+0000","last_became_peered":"2024-09-10T10:46:08.739402+0000","last_unstale":"2024-09-10T10:46:09.724086+0000","last_undegraded":"2024-09-10T10:46:09.724086+0000","last_fullsized":"2024-09-10T10:46:09.724086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:03.741143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725517+0000","last_change":"2024-09-10T10:46:08.728789+0000","last_active":"2024-09-10T10:46:09.725517+0000","last_peered":"2024-09-10T10:46:09.725517+0000","last_clean":"2024-09-10T10:46:09.725517+0000","last_became_active":"2024-09-10T10:46:08.726155+0000","last_became_peered":"2024-09-10T10:46:08.726155+0000","last_unstale":"2024-09-10T10:46:09.725517+0000","last_undegraded":"2024-09-10T10:46:09.725517+0000","last_fullsized":"2024-09-10T10:46:09.725517+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:02.655655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725381+0000","last_change":"2024-09-10T10:46:08.737127+0000","last_active":"2024-09-10T10:46:09.725381+0000","last_peered":"2024-09-10T10:46:09.725381+0000","last_clean":"2024-09-10T10:46:09.725381+0000","last_became_active":"2024-09-10T10:46:08.735205+0000","last_became_peered":"2024-09-10T10:46:08.735205+0000","last_unstale":"2024-09-10T10:46:09.725381+0000","last_undegraded":"2024-09-10T10:46:09.725381+0000","last_fullsized":"2024-09-10T10:46:09.725381+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:46.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725784+0000","last_change":"2024-09-10T10:46:08.731743+0000","last_active":"2024-09-10T10:46:09.725784+0000","last_peered":"2024-09-10T10:46:09.725784+0000","last_clean":"2024-09-10T10:46:09.725784+0000","last_became_active":"2024-09-10T10:46:08.727605+0000","last_became_peered":"2024-09-10T10:46:08.727605+0000","last_unstale":"2024-09-10T10:46:09.725784+0000","last_undegraded":"2024-09-10T10:46:09.725784+0000","last_fullsized":"2024-09-10T10:46:09.725784+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:25.624393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813518+0000","last_change":"2024-09-10T10:46:08.727247+0000","last_active":"2024-09-10T10:46:09.813518+0000","last_peered":"2024-09-10T10:46:09.813518+0000","last_clean":"2024-09-10T10:46:09.813518+0000","last_became_active":"2024-09-10T10:46:08.726948+0000","last_became_peered":"2024-09-10T10:46:08.726948+0000","last_unstale":"2024-09-10T10:46:09.813518+0000","last_undegraded":"2024-09-10T10:46:09.813518+0000","last_fullsized":"2024-09-10T10:46:09.813518+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814087+0000","last_change":"2024-09-10T10:46:08.739472+0000","last_active":"2024-09-10T10:46:09.814087+0000","last_peered":"2024-09-10T10:46:09.814087+0000","last_clean":"2024-09-10T10:46:09.814087+0000","last_became_active":"2024-09-10T10:46:08.738694+0000","last_became_peered":"2024-09-10T10:46:08.738694+0000","last_unstale":"2024-09-10T10:46:09.814087+0000","last_undegraded":"2024-09-10T10:46:09.814087+0000","last_fullsized":"2024-09-10T10:46:09.814087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:26.559484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724388+0000","last_change":"2024-09-10T10:46:08.726859+0000","last_active":"2024-09-10T10:46:09.724388+0000","last_peered":"2024-09-10T10:46:09.724388+0000","last_clean":"2024-09-10T10:46:09.724388+0000","last_became_active":"2024-09-10T10:46:08.726603+0000","last_became_peered":"2024-09-10T10:46:08.726603+0000","last_unstale":"2024-09-10T10:46:09.724388+0000","last_undegraded":"2024-09-10T10:46:09.724388+0000","last_fullsized":"2024-09-10T10:46:09.724388+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:00.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813942+0000","last_change":"2024-09-10T10:46:08.721678+0000","last_active":"2024-09-10T10:46:09.813942+0000","last_peered":"2024-09-10T10:46:09.813942+0000","last_clean":"2024-09-10T10:46:09.813942+0000","last_became_active":"2024-09-10T10:46:08.721394+0000","last_became_peered":"2024-09-10T10:46:08.721394+0000","last_unstale":"2024-09-10T10:46:09.813942+0000","last_undegraded":"2024-09-10T10:46:09.813942+0000","last_fullsized":"2024-09-10T10:46:09.813942+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:38.677474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724282+0000","last_change":"2024-09-10T10:46:08.729061+0000","last_active":"2024-09-10T10:46:09.724282+0000","last_peered":"2024-09-10T10:46:09.724282+0000","last_clean":"2024-09-10T10:46:09.724282+0000","last_became_active":"2024-09-10T10:46:08.728778+0000","last_became_peered":"2024-09-10T10:46:08.728778+0000","last_unstale":"2024-09-10T10:46:09.724282+0000","last_undegraded":"2024-09-10T10:46:09.724282+0000","last_fullsized":"2024-09-10T10:46:09.724282+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:09.643039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813099+0000","last_change":"2024-09-10T10:46:08.733398+0000","last_active":"2024-09-10T10:46:09.813099+0000","last_peered":"2024-09-10T10:46:09.813099+0000","last_clean":"2024-09-10T10:46:09.813099+0000","last_became_active":"2024-09-10T10:46:08.733109+0000","last_became_peered":"2024-09-10T10:46:08.733109+0000","last_unstale":"2024-09-10T10:46:09.813099+0000","last_undegraded":"2024-09-10T10:46:09.813099+0000","last_fullsized":"2024-09-10T10:46:09.813099+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813628+0000","last_change":"2024-09-10T10:46:08.724201+0000","last_active":"2024-09-10T10:46:09.813628+0000","last_peered":"2024-09-10T10:46:09.813628+0000","last_clean":"2024-09-10T10:46:09.813628+0000","last_became_active":"2024-09-10T10:46:08.723940+0000","last_became_peered":"2024-09-10T10:46:08.723940+0000","last_unstale":"2024-09-10T10:46:09.813628+0000","last_undegraded":"2024-09-10T10:46:09.813628+0000","last_fullsized":"2024-09-10T10:46:09.813628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:35.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722703+0000","last_change":"2024-09-10T10:46:07.716998+0000","last_active":"2024-09-10T10:46:09.722703+0000","last_peered":"2024-09-10T10:46:09.722703+0000","last_clean":"2024-09-10T10:46:09.722703+0000","last_became_active":"2024-09-10T10:46:07.716359+0000","last_became_peered":"2024-09-10T10:46:07.716359+0000","last_unstale":"2024-09-10T10:46:09.722703+0000","last_undegraded":"2024-09-10T10:46:09.722703+0000","last_fullsized":"2024-09-10T10:46:09.722703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:29.370839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813155+0000","last_change":"2024-09-10T10:46:08.728403+0000","last_active":"2024-09-10T10:46:09.813155+0000","last_peered":"2024-09-10T10:46:09.813155+0000","last_clean":"2024-09-10T10:46:09.813155+0000","last_became_active":"2024-09-10T10:46:08.728096+0000","last_became_peered":"2024-09-10T10:46:08.728096+0000","last_unstale":"2024-09-10T10:46:09.813155+0000","last_undegraded":"2024-09-10T10:46:09.813155+0000","last_fullsized":"2024-09-10T10:46:09.813155+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:00.688666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813565+0000","last_change":"2024-09-10T10:46:07.722043+0000","last_active":"2024-09-10T10:46:09.813565+0000","last_peered":"2024-09-10T10:46:09.813565+0000","last_clean":"2024-09-10T10:46:09.813565+0000","last_became_active":"2024-09-10T10:46:07.721747+0000","last_became_peered":"2024-09-10T10:46:07.721747+0000","last_unstale":"2024-09-10T10:46:09.813565+0000","last_undegraded":"2024-09-10T10:46:09.813565+0000","last_fullsized":"2024-09-10T10:46:09.813565+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:29.447715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813802+0000","last_change":"2024-09-10T10:46:08.724342+0000","last_active":"2024-09-10T10:46:09.813802+0000","last_peered":"2024-09-10T10:46:09.813802+0000","last_clean":"2024-09-10T10:46:09.813802+0000","last_became_active":"2024-09-10T10:46:08.724021+0000","last_became_peered":"2024-09-10T10:46:08.724021+0000","last_unstale":"2024-09-10T10:46:09.813802+0000","last_undegraded":"2024-09-10T10:46:09.813802+0000","last_fullsized":"2024-09-10T10:46:09.813802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:45.383097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"55'2","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.753341+0000","last_change":"2024-09-10T10:46:07.713863+0000","last_active":"2024-09-10T10:46:08.753341+0000","last_peered":"2024-09-10T10:46:08.753341+0000","last_clean":"2024-09-10T10:46:08.753341+0000","last_became_active":"2024-09-10T10:46:07.710971+0000","last_became_peered":"2024-09-10T10:46:07.710971+0000","last_unstale":"2024-09-10T10:46:08.753341+0000","last_undegraded":"2024-09-10T10:46:08.753341+0000","last_fullsized":"2024-09-10T10:46:08.753341+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:11.338846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.736082+0000","last_change":"2024-09-10T10:46:08.730931+0000","last_active":"2024-09-10T10:46:08.736082+0000","last_peered":"2024-09-10T10:46:08.736082+0000","last_clean":"2024-09-10T10:46:08.736082+0000","last_became_active":"2024-09-10T10:46:08.729480+0000","last_became_peered":"2024-09-10T10:46:08.729480+0000","last_unstale":"2024-09-10T10:46:08.736082+0000","last_undegraded":"2024-09-10T10:46:08.736082+0000","last_fullsized":"2024-09-10T10:46:08.736082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:28.671820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814186+0000","last_change":"2024-09-10T10:46:07.712592+0000","last_active":"2024-09-10T10:46:09.814186+0000","last_peered":"2024-09-10T10:46:09.814186+0000","last_clean":"2024-09-10T10:46:09.814186+0000","last_became_active":"2024-09-10T10:46:07.711623+0000","last_became_peered":"2024-09-10T10:46:07.711623+0000","last_unstale":"2024-09-10T10:46:09.814186+0000","last_undegraded":"2024-09-10T10:46:09.814186+0000","last_fullsized":"2024-09-10T10:46:09.814186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:23:56.413128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722671+0000","last_change":"2024-09-10T10:46:08.742399+0000","last_active":"2024-09-10T10:46:09.722671+0000","last_peered":"2024-09-10T10:46:09.722671+0000","last_clean":"2024-09-10T10:46:09.722671+0000","last_became_active":"2024-09-10T10:46:08.741955+0000","last_became_peered":"2024-09-10T10:46:08.741955+0000","last_unstale":"2024-09-10T10:46:09.722671+0000","last_undegraded":"2024-09-10T10:46:09.722671+0000","last_fullsized":"2024-09-10T10:46:09.722671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:35.816372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725073+0000","last_change":"2024-09-10T10:46:07.716629+0000","last_active":"2024-09-10T10:46:09.725073+0000","last_peered":"2024-09-10T10:46:09.725073+0000","last_clean":"2024-09-10T10:46:09.725073+0000","last_became_active":"2024-09-10T10:46:07.716365+0000","last_became_peered":"2024-09-10T10:46:07.716365+0000","last_unstale":"2024-09-10T10:46:09.725073+0000","last_undegraded":"2024-09-10T10:46:09.725073+0000","last_fullsized":"2024-09-10T10:46:09.725073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:45.382980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814332+0000","last_change":"2024-09-10T10:46:08.729147+0000","last_active":"2024-09-10T10:46:09.814332+0000","last_peered":"2024-09-10T10:46:09.814332+0000","last_clean":"2024-09-10T10:46:09.814332+0000","last_became_active":"2024-09-10T10:46:08.728709+0000","last_became_peered":"2024-09-10T10:46:08.728709+0000","last_unstale":"2024-09-10T10:46:09.814332+0000","last_undegraded":"2024-09-10T10:46:09.814332+0000","last_fullsized":"2024-09-10T10:46:09.814332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:34.551797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814035+0000","last_change":"2024-09-10T10:46:07.708895+0000","last_active":"2024-09-10T10:46:09.814035+0000","last_peered":"2024-09-10T10:46:09.814035+0000","last_clean":"2024-09-10T10:46:09.814035+0000","last_became_active":"2024-09-10T10:46:07.708616+0000","last_became_peered":"2024-09-10T10:46:07.708616+0000","last_unstale":"2024-09-10T10:46:09.814035+0000","last_undegraded":"2024-09-10T10:46:09.814035+0000","last_fullsized":"2024-09-10T10:46:09.814035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:50.958753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.738484+0000","last_change":"2024-09-10T10:46:08.733837+0000","last_active":"2024-09-10T10:46:08.738484+0000","last_peered":"2024-09-10T10:46:08.738484+0000","last_clean":"2024-09-10T10:46:08.738484+0000","last_became_active":"2024-09-10T10:46:08.733303+0000","last_became_peered":"2024-09-10T10:46:08.733303+0000","last_unstale":"2024-09-10T10:46:08.738484+0000","last_undegraded":"2024-09-10T10:46:08.738484+0000","last_fullsized":"2024-09-10T10:46:08.738484+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:33.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813593+0000","last_change":"2024-09-10T10:46:07.715697+0000","last_active":"2024-09-10T10:46:09.813593+0000","last_peered":"2024-09-10T10:46:09.813593+0000","last_clean":"2024-09-10T10:46:09.813593+0000","last_became_active":"2024-09-10T10:46:07.715368+0000","last_became_peered":"2024-09-10T10:46:07.715368+0000","last_unstale":"2024-09-10T10:46:09.813593+0000","last_undegraded":"2024-09-10T10:46:09.813593+0000","last_fullsized":"2024-09-10T10:46:09.813593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:11.287273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814909+0000","last_change":"2024-09-10T10:46:08.723901+0000","last_active":"2024-09-10T10:46:09.814909+0000","last_peered":"2024-09-10T10:46:09.814909+0000","last_clean":"2024-09-10T10:46:09.814909+0000","last_became_active":"2024-09-10T10:46:08.723584+0000","last_became_peered":"2024-09-10T10:46:08.723584+0000","last_unstale":"2024-09-10T10:46:09.814909+0000","last_undegraded":"2024-09-10T10:46:09.814909+0000","last_fullsized":"2024-09-10T10:46:09.814909+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725660+0000","last_change":"2024-09-10T10:46:07.715434+0000","last_active":"2024-09-10T10:46:09.725660+0000","last_peered":"2024-09-10T10:46:09.725660+0000","last_clean":"2024-09-10T10:46:09.725660+0000","last_became_active":"2024-09-10T10:46:07.715169+0000","last_became_peered":"2024-09-10T10:46:07.715169+0000","last_unstale":"2024-09-10T10:46:09.725660+0000","last_undegraded":"2024-09-10T10:46:09.725660+0000","last_fullsized":"2024-09-10T10:46:09.725660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:01.966583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722350+0000","last_change":"2024-09-10T10:46:08.733184+0000","last_active":"2024-09-10T10:46:09.722350+0000","last_peered":"2024-09-10T10:46:09.722350+0000","last_clean":"2024-09-10T10:46:09.722350+0000","last_became_active":"2024-09-10T10:46:08.729876+0000","last_became_peered":"2024-09-10T10:46:08.729876+0000","last_unstale":"2024-09-10T10:46:09.722350+0000","last_undegraded":"2024-09-10T10:46:09.722350+0000","last_fullsized":"2024-09-10T10:46:09.722350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:52.246946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814954+0000","last_change":"2024-09-10T10:46:07.714757+0000","last_active":"2024-09-10T10:46:09.814954+0000","last_peered":"2024-09-10T10:46:09.814954+0000","last_clean":"2024-09-10T10:46:09.814954+0000","last_became_active":"2024-09-10T10:46:07.714453+0000","last_became_peered":"2024-09-10T10:46:07.714453+0000","last_unstale":"2024-09-10T10:46:09.814954+0000","last_undegraded":"2024-09-10T10:46:09.814954+0000","last_fullsized":"2024-09-10T10:46:09.814954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:38.551812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813377+0000","last_change":"2024-09-10T10:46:08.722159+0000","last_active":"2024-09-10T10:46:09.813377+0000","last_peered":"2024-09-10T10:46:09.813377+0000","last_clean":"2024-09-10T10:46:09.813377+0000","last_became_active":"2024-09-10T10:46:08.721864+0000","last_became_peered":"2024-09-10T10:46:08.721864+0000","last_unstale":"2024-09-10T10:46:09.813377+0000","last_undegraded":"2024-09-10T10:46:09.813377+0000","last_fullsized":"2024-09-10T10:46:09.813377+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:19.587816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814778+0000","last_change":"2024-09-10T10:46:07.714691+0000","last_active":"2024-09-10T10:46:09.814778+0000","last_peered":"2024-09-10T10:46:09.814778+0000","last_clean":"2024-09-10T10:46:09.814778+0000","last_became_active":"2024-09-10T10:46:07.714126+0000","last_became_peered":"2024-09-10T10:46:07.714126+0000","last_unstale":"2024-09-10T10:46:09.814778+0000","last_undegraded":"2024-09-10T10:46:09.814778+0000","last_fullsized":"2024-09-10T10:46:09.814778+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:09.345594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724130+0000","last_change":"2024-09-10T10:46:08.739881+0000","last_active":"2024-09-10T10:46:09.724130+0000","last_peered":"2024-09-10T10:46:09.724130+0000","last_clean":"2024-09-10T10:46:09.724130+0000","last_became_active":"2024-09-10T10:46:08.739565+0000","last_became_peered":"2024-09-10T10:46:08.739565+0000","last_unstale":"2024-09-10T10:46:09.724130+0000","last_undegraded":"2024-09-10T10:46:09.724130+0000","last_fullsized":"2024-09-10T10:46:09.724130+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:35.816372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814538+0000","last_change":"2024-09-10T10:46:07.718457+0000","last_active":"2024-09-10T10:46:09.814538+0000","last_peered":"2024-09-10T10:46:09.814538+0000","last_clean":"2024-09-10T10:46:09.814538+0000","last_became_active":"2024-09-10T10:46:07.718172+0000","last_became_peered":"2024-09-10T10:46:07.718172+0000","last_unstale":"2024-09-10T10:46:09.814538+0000","last_undegraded":"2024-09-10T10:46:09.814538+0000","last_fullsized":"2024-09-10T10:46:09.814538+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:47.945241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813825+0000","last_change":"2024-09-10T10:46:08.734452+0000","last_active":"2024-09-10T10:46:09.813825+0000","last_peered":"2024-09-10T10:46:09.813825+0000","last_clean":"2024-09-10T10:46:09.813825+0000","last_became_active":"2024-09-10T10:46:08.734154+0000","last_became_peered":"2024-09-10T10:46:08.734154+0000","last_unstale":"2024-09-10T10:46:09.813825+0000","last_undegraded":"2024-09-10T10:46:09.813825+0000","last_fullsized":"2024-09-10T10:46:09.813825+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"55'6","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.758940+0000","last_change":"2024-09-10T10:46:07.714594+0000","last_active":"2024-09-10T10:46:08.758940+0000","last_peered":"2024-09-10T10:46:08.758940+0000","last_clean":"2024-09-10T10:46:08.758940+0000","last_became_active":"2024-09-10T10:46:07.712264+0000","last_became_peered":"2024-09-10T10:46:07.712264+0000","last_unstale":"2024-09-10T10:46:08.758940+0000","last_undegraded":"2024-09-10T10:46:08.758940+0000","last_fullsized":"2024-09-10T10:46:08.758940+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:46.816918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814811+0000","last_change":"2024-09-10T10:46:08.729539+0000","last_active":"2024-09-10T10:46:09.814811+0000","last_peered":"2024-09-10T10:46:09.814811+0000","last_clean":"2024-09-10T10:46:09.814811+0000","last_became_active":"2024-09-10T10:46:08.729136+0000","last_became_peered":"2024-09-10T10:46:08.729136+0000","last_unstale":"2024-09-10T10:46:09.814811+0000","last_undegraded":"2024-09-10T10:46:09.814811+0000","last_fullsized":"2024-09-10T10:46:09.814811+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:00.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813488+0000","last_change":"2024-09-10T10:46:07.720375+0000","last_active":"2024-09-10T10:46:09.813488+0000","last_peered":"2024-09-10T10:46:09.813488+0000","last_clean":"2024-09-10T10:46:09.813488+0000","last_became_active":"2024-09-10T10:46:07.719754+0000","last_became_peered":"2024-09-10T10:46:07.719754+0000","last_unstale":"2024-09-10T10:46:09.813488+0000","last_undegraded":"2024-09-10T10:46:09.813488+0000","last_fullsized":"2024-09-10T10:46:09.813488+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:15.131104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724818+0000","last_change":"2024-09-10T10:46:08.730562+0000","last_active":"2024-09-10T10:46:09.724818+0000","last_peered":"2024-09-10T10:46:09.724818+0000","last_clean":"2024-09-10T10:46:09.724818+0000","last_became_active":"2024-09-10T10:46:08.730236+0000","last_became_peered":"2024-09-10T10:46:08.730236+0000","last_unstale":"2024-09-10T10:46:09.724818+0000","last_undegraded":"2024-09-10T10:46:09.724818+0000","last_fullsized":"2024-09-10T10:46:09.724818+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:01.438641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813394+0000","last_change":"2024-09-10T10:46:07.705803+0000","last_active":"2024-09-10T10:46:09.813394+0000","last_peered":"2024-09-10T10:46:09.813394+0000","last_clean":"2024-09-10T10:46:09.813394+0000","last_became_active":"2024-09-10T10:46:07.705541+0000","last_became_peered":"2024-09-10T10:46:07.705541+0000","last_unstale":"2024-09-10T10:46:09.813394+0000","last_undegraded":"2024-09-10T10:46:09.813394+0000","last_fullsized":"2024-09-10T10:46:09.813394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:57.846918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813679+0000","last_change":"2024-09-10T10:46:08.723566+0000","last_active":"2024-09-10T10:46:09.813679+0000","last_peered":"2024-09-10T10:46:09.813679+0000","last_clean":"2024-09-10T10:46:09.813679+0000","last_became_active":"2024-09-10T10:46:08.722919+0000","last_became_peered":"2024-09-10T10:46:08.722919+0000","last_unstale":"2024-09-10T10:46:09.813679+0000","last_undegraded":"2024-09-10T10:46:09.813679+0000","last_fullsized":"2024-09-10T10:46:09.813679+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:27.127913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813313+0000","last_change":"2024-09-10T10:46:07.711985+0000","last_active":"2024-09-10T10:46:09.813313+0000","last_peered":"2024-09-10T10:46:09.813313+0000","last_clean":"2024-09-10T10:46:09.813313+0000","last_became_active":"2024-09-10T10:46:07.710241+0000","last_became_peered":"2024-09-10T10:46:07.710241+0000","last_unstale":"2024-09-10T10:46:09.813313+0000","last_undegraded":"2024-09-10T10:46:09.813313+0000","last_fullsized":"2024-09-10T10:46:09.813313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:14.706240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.726199+0000","last_change":"2024-09-10T10:46:08.729307+0000","last_active":"2024-09-10T10:46:09.726199+0000","last_peered":"2024-09-10T10:46:09.726199+0000","last_clean":"2024-09-10T10:46:09.726199+0000","last_became_active":"2024-09-10T10:46:08.729028+0000","last_became_peered":"2024-09-10T10:46:08.729028+0000","last_unstale":"2024-09-10T10:46:09.726199+0000","last_undegraded":"2024-09-10T10:46:09.726199+0000","last_fullsized":"2024-09-10T10:46:09.726199+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:18.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813402+0000","last_change":"2024-09-10T10:46:07.721960+0000","last_active":"2024-09-10T10:46:09.813402+0000","last_peered":"2024-09-10T10:46:09.813402+0000","last_clean":"2024-09-10T10:46:09.813402+0000","last_became_active":"2024-09-10T10:46:07.721652+0000","last_became_peered":"2024-09-10T10:46:07.721652+0000","last_unstale":"2024-09-10T10:46:09.813402+0000","last_undegraded":"2024-09-10T10:46:09.813402+0000","last_fullsized":"2024-09-10T10:46:09.813402+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:22.544691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.732387+0000","last_change":"2024-09-10T10:46:08.724451+0000","last_active":"2024-09-10T10:46:08.732387+0000","last_peered":"2024-09-10T10:46:08.732387+0000","last_clean":"2024-09-10T10:46:08.732387+0000","last_became_active":"2024-09-10T10:46:08.724164+0000","last_became_peered":"2024-09-10T10:46:08.724164+0000","last_unstale":"2024-09-10T10:46:08.732387+0000","last_undegraded":"2024-09-10T10:46:08.732387+0000","last_fullsized":"2024-09-10T10:46:08.732387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:51.510103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813690+0000","last_change":"2024-09-10T10:46:07.722761+0000","last_active":"2024-09-10T10:46:09.813690+0000","last_peered":"2024-09-10T10:46:09.813690+0000","last_clean":"2024-09-10T10:46:09.813690+0000","last_became_active":"2024-09-10T10:46:07.722475+0000","last_became_peered":"2024-09-10T10:46:07.722475+0000","last_unstale":"2024-09-10T10:46:09.813690+0000","last_undegraded":"2024-09-10T10:46:09.813690+0000","last_fullsized":"2024-09-10T10:46:09.813690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:32.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723671+0000","last_change":"2024-09-10T10:46:08.731134+0000","last_active":"2024-09-10T10:46:09.723671+0000","last_peered":"2024-09-10T10:46:09.723671+0000","last_clean":"2024-09-10T10:46:09.723671+0000","last_became_active":"2024-09-10T10:46:08.729420+0000","last_became_peered":"2024-09-10T10:46:08.729420+0000","last_unstale":"2024-09-10T10:46:09.723671+0000","last_undegraded":"2024-09-10T10:46:09.723671+0000","last_fullsized":"2024-09-10T10:46:09.723671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:52.246946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813730+0000","last_change":"2024-09-10T10:46:07.720940+0000","last_active":"2024-09-10T10:46:09.813730+0000","last_peered":"2024-09-10T10:46:09.813730+0000","last_clean":"2024-09-10T10:46:09.813730+0000","last_became_active":"2024-09-10T10:46:07.720668+0000","last_became_peered":"2024-09-10T10:46:07.720668+0000","last_unstale":"2024-09-10T10:46:09.813730+0000","last_undegraded":"2024-09-10T10:46:09.813730+0000","last_fullsized":"2024-09-10T10:46:09.813730+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:26.871070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813203+0000","last_change":"2024-09-10T10:46:08.744432+0000","last_active":"2024-09-10T10:46:09.813203+0000","last_peered":"2024-09-10T10:46:09.813203+0000","last_clean":"2024-09-10T10:46:09.813203+0000","last_became_active":"2024-09-10T10:46:08.740365+0000","last_became_peered":"2024-09-10T10:46:08.740365+0000","last_unstale":"2024-09-10T10:46:09.813203+0000","last_undegraded":"2024-09-10T10:46:09.813203+0000","last_fullsized":"2024-09-10T10:46:09.813203+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814615+0000","last_change":"2024-09-10T10:46:07.714468+0000","last_active":"2024-09-10T10:46:09.814615+0000","last_peered":"2024-09-10T10:46:09.814615+0000","last_clean":"2024-09-10T10:46:09.814615+0000","last_became_active":"2024-09-10T10:46:07.714129+0000","last_became_peered":"2024-09-10T10:46:07.714129+0000","last_unstale":"2024-09-10T10:46:09.814615+0000","last_undegraded":"2024-09-10T10:46:09.814615+0000","last_fullsized":"2024-09-10T10:46:09.814615+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:51.520301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813417+0000","last_change":"2024-09-10T10:46:08.731968+0000","last_active":"2024-09-10T10:46:09.813417+0000","last_peered":"2024-09-10T10:46:09.813417+0000","last_clean":"2024-09-10T10:46:09.813417+0000","last_became_active":"2024-09-10T10:46:08.729878+0000","last_became_peered":"2024-09-10T10:46:08.729878+0000","last_unstale":"2024-09-10T10:46:09.813417+0000","last_undegraded":"2024-09-10T10:46:09.813417+0000","last_fullsized":"2024-09-10T10:46:09.813417+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814223+0000","last_change":"2024-09-10T10:46:07.707015+0000","last_active":"2024-09-10T10:46:09.814223+0000","last_peered":"2024-09-10T10:46:09.814223+0000","last_clean":"2024-09-10T10:46:09.814223+0000","last_became_active":"2024-09-10T10:46:07.706746+0000","last_became_peered":"2024-09-10T10:46:07.706746+0000","last_unstale":"2024-09-10T10:46:09.814223+0000","last_undegraded":"2024-09-10T10:46:09.814223+0000","last_fullsized":"2024-09-10T10:46:09.814223+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:21.110648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814125+0000","last_change":"2024-09-10T10:46:08.727633+0000","last_active":"2024-09-10T10:46:09.814125+0000","last_peered":"2024-09-10T10:46:09.814125+0000","last_clean":"2024-09-10T10:46:09.814125+0000","last_became_active":"2024-09-10T10:46:08.727270+0000","last_became_peered":"2024-09-10T10:46:08.727270+0000","last_unstale":"2024-09-10T10:46:09.814125+0000","last_undegraded":"2024-09-10T10:46:09.814125+0000","last_fullsized":"2024-09-10T10:46:09.814125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:51.976999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724415+0000","last_change":"2024-09-10T10:46:07.710387+0000","last_active":"2024-09-10T10:46:09.724415+0000","last_peered":"2024-09-10T10:46:09.724415+0000","last_clean":"2024-09-10T10:46:09.724415+0000","last_became_active":"2024-09-10T10:46:07.708742+0000","last_became_peered":"2024-09-10T10:46:07.708742+0000","last_unstale":"2024-09-10T10:46:09.724415+0000","last_undegraded":"2024-09-10T10:46:09.724415+0000","last_fullsized":"2024-09-10T10:46:09.724415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:49.969992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814635+0000","last_change":"2024-09-10T10:46:08.741798+0000","last_active":"2024-09-10T10:46:09.814635+0000","last_peered":"2024-09-10T10:46:09.814635+0000","last_clean":"2024-09-10T10:46:09.814635+0000","last_became_active":"2024-09-10T10:46:08.741504+0000","last_became_peered":"2024-09-10T10:46:08.741504+0000","last_unstale":"2024-09-10T10:46:09.814635+0000","last_undegraded":"2024-09-10T10:46:09.814635+0000","last_fullsized":"2024-09-10T10:46:09.814635+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724596+0000","last_change":"2024-09-10T10:46:07.717198+0000","last_active":"2024-09-10T10:46:09.724596+0000","last_peered":"2024-09-10T10:46:09.724596+0000","last_clean":"2024-09-10T10:46:09.724596+0000","last_became_active":"2024-09-10T10:46:07.716874+0000","last_became_peered":"2024-09-10T10:46:07.716874+0000","last_unstale":"2024-09-10T10:46:09.724596+0000","last_undegraded":"2024-09-10T10:46:09.724596+0000","last_fullsized":"2024-09-10T10:46:09.724596+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:37.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724876+0000","last_change":"2024-09-10T10:46:08.739788+0000","last_active":"2024-09-10T10:46:09.724876+0000","last_peered":"2024-09-10T10:46:09.724876+0000","last_clean":"2024-09-10T10:46:09.724876+0000","last_became_active":"2024-09-10T10:46:08.739502+0000","last_became_peered":"2024-09-10T10:46:08.739502+0000","last_unstale":"2024-09-10T10:46:09.724876+0000","last_undegraded":"2024-09-10T10:46:09.724876+0000","last_fullsized":"2024-09-10T10:46:09.724876+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:23.466620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.715555+0000","last_change":"2024-09-10T10:46:07.712656+0000","last_active":"2024-09-10T10:46:08.715555+0000","last_peered":"2024-09-10T10:46:08.715555+0000","last_clean":"2024-09-10T10:46:08.715555+0000","last_became_active":"2024-09-10T10:46:07.712362+0000","last_became_peered":"2024-09-10T10:46:07.712362+0000","last_unstale":"2024-09-10T10:46:08.715555+0000","last_undegraded":"2024-09-10T10:46:08.715555+0000","last_fullsized":"2024-09-10T10:46:08.715555+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:10.504934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813746+0000","last_change":"2024-09-10T10:46:08.727931+0000","last_active":"2024-09-10T10:46:09.813746+0000","last_peered":"2024-09-10T10:46:09.813746+0000","last_clean":"2024-09-10T10:46:09.813746+0000","last_became_active":"2024-09-10T10:46:08.727650+0000","last_became_peered":"2024-09-10T10:46:08.727650+0000","last_unstale":"2024-09-10T10:46:09.813746+0000","last_undegraded":"2024-09-10T10:46:09.813746+0000","last_fullsized":"2024-09-10T10:46:09.813746+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:26.559484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717316+0000","last_change":"2024-09-10T10:46:07.703518+0000","last_active":"2024-09-10T10:46:08.717316+0000","last_peered":"2024-09-10T10:46:08.717316+0000","last_clean":"2024-09-10T10:46:08.717316+0000","last_became_active":"2024-09-10T10:46:07.703228+0000","last_became_peered":"2024-09-10T10:46:07.703228+0000","last_unstale":"2024-09-10T10:46:08.717316+0000","last_undegraded":"2024-09-10T10:46:08.717316+0000","last_fullsized":"2024-09-10T10:46:08.717316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:07.526127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.812984+0000","last_change":"2024-09-10T10:46:08.729574+0000","last_active":"2024-09-10T10:46:09.812984+0000","last_peered":"2024-09-10T10:46:09.812984+0000","last_clean":"2024-09-10T10:46:09.812984+0000","last_became_active":"2024-09-10T10:46:08.729259+0000","last_became_peered":"2024-09-10T10:46:08.729259+0000","last_unstale":"2024-09-10T10:46:09.812984+0000","last_undegraded":"2024-09-10T10:46:09.812984+0000","last_fullsized":"2024-09-10T10:46:09.812984+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:50.976281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814462+0000","last_change":"2024-09-10T10:46:07.716713+0000","last_active":"2024-09-10T10:46:09.814462+0000","last_peered":"2024-09-10T10:46:09.814462+0000","last_clean":"2024-09-10T10:46:09.814462+0000","last_became_active":"2024-09-10T10:46:07.716445+0000","last_became_peered":"2024-09-10T10:46:07.716445+0000","last_unstale":"2024-09-10T10:46:09.814462+0000","last_undegraded":"2024-09-10T10:46:09.814462+0000","last_fullsized":"2024-09-10T10:46:09.814462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:33.003887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.734302+0000","last_change":"2024-09-10T10:46:08.729158+0000","last_active":"2024-09-10T10:46:08.734302+0000","last_peered":"2024-09-10T10:46:08.734302+0000","last_clean":"2024-09-10T10:46:08.734302+0000","last_became_active":"2024-09-10T10:46:08.728872+0000","last_became_peered":"2024-09-10T10:46:08.728872+0000","last_unstale":"2024-09-10T10:46:08.734302+0000","last_undegraded":"2024-09-10T10:46:08.734302+0000","last_fullsized":"2024-09-10T10:46:08.734302+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:16.179598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814448+0000","last_change":"2024-09-10T10:46:07.718458+0000","last_active":"2024-09-10T10:46:09.814448+0000","last_peered":"2024-09-10T10:46:09.814448+0000","last_clean":"2024-09-10T10:46:09.814448+0000","last_became_active":"2024-09-10T10:46:07.718229+0000","last_became_peered":"2024-09-10T10:46:07.718229+0000","last_unstale":"2024-09-10T10:46:09.814448+0000","last_undegraded":"2024-09-10T10:46:09.814448+0000","last_fullsized":"2024-09-10T10:46:09.814448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:21.948804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725139+0000","last_change":"2024-09-10T10:46:08.732363+0000","last_active":"2024-09-10T10:46:09.725139+0000","last_peered":"2024-09-10T10:46:09.725139+0000","last_clean":"2024-09-10T10:46:09.725139+0000","last_became_active":"2024-09-10T10:46:08.732091+0000","last_became_peered":"2024-09-10T10:46:08.732091+0000","last_unstale":"2024-09-10T10:46:09.725139+0000","last_undegraded":"2024-09-10T10:46:09.725139+0000","last_fullsized":"2024-09-10T10:46:09.725139+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:31.871643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725371+0000","last_change":"2024-09-10T10:46:07.708779+0000","last_active":"2024-09-10T10:46:09.725371+0000","last_peered":"2024-09-10T10:46:09.725371+0000","last_clean":"2024-09-10T10:46:09.725371+0000","last_became_active":"2024-09-10T10:46:07.708506+0000","last_became_peered":"2024-09-10T10:46:07.708506+0000","last_unstale":"2024-09-10T10:46:09.725371+0000","last_undegraded":"2024-09-10T10:46:09.725371+0000","last_fullsized":"2024-09-10T10:46:09.725371+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:35.116526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.735676+0000","last_change":"2024-09-10T10:46:08.731256+0000","last_active":"2024-09-10T10:46:08.735676+0000","last_peered":"2024-09-10T10:46:08.735676+0000","last_clean":"2024-09-10T10:46:08.735676+0000","last_became_active":"2024-09-10T10:46:08.728615+0000","last_became_peered":"2024-09-10T10:46:08.728615+0000","last_unstale":"2024-09-10T10:46:08.735676+0000","last_undegraded":"2024-09-10T10:46:08.735676+0000","last_fullsized":"2024-09-10T10:46:08.735676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:23.031322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717697+0000","last_change":"2024-09-10T10:46:07.709486+0000","last_active":"2024-09-10T10:46:08.717697+0000","last_peered":"2024-09-10T10:46:08.717697+0000","last_clean":"2024-09-10T10:46:08.717697+0000","last_became_active":"2024-09-10T10:46:07.707686+0000","last_became_peered":"2024-09-10T10:46:07.707686+0000","last_unstale":"2024-09-10T10:46:08.717697+0000","last_undegraded":"2024-09-10T10:46:08.717697+0000","last_fullsized":"2024-09-10T10:46:08.717697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:48.823340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813890+0000","last_change":"2024-09-10T10:46:08.743902+0000","last_active":"2024-09-10T10:46:09.813890+0000","last_peered":"2024-09-10T10:46:09.813890+0000","last_clean":"2024-09-10T10:46:09.813890+0000","last_became_active":"2024-09-10T10:46:08.743541+0000","last_became_peered":"2024-09-10T10:46:08.743541+0000","last_unstale":"2024-09-10T10:46:09.813890+0000","last_undegraded":"2024-09-10T10:46:09.813890+0000","last_fullsized":"2024-09-10T10:46:09.813890+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725447+0000","last_change":"2024-09-10T10:46:07.718465+0000","last_active":"2024-09-10T10:46:09.725447+0000","last_peered":"2024-09-10T10:46:09.725447+0000","last_clean":"2024-09-10T10:46:09.725447+0000","last_became_active":"2024-09-10T10:46:07.718183+0000","last_became_peered":"2024-09-10T10:46:07.718183+0000","last_unstale":"2024-09-10T10:46:09.725447+0000","last_undegraded":"2024-09-10T10:46:09.725447+0000","last_fullsized":"2024-09-10T10:46:09.725447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:42.124747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.738462+0000","last_change":"2024-09-10T10:46:08.733569+0000","last_active":"2024-09-10T10:46:08.738462+0000","last_peered":"2024-09-10T10:46:08.738462+0000","last_clean":"2024-09-10T10:46:08.738462+0000","last_became_active":"2024-09-10T10:46:08.732933+0000","last_became_peered":"2024-09-10T10:46:08.732933+0000","last_unstale":"2024-09-10T10:46:08.738462+0000","last_undegraded":"2024-09-10T10:46:08.738462+0000","last_fullsized":"2024-09-10T10:46:08.738462+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:34.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813401+0000","last_change":"2024-09-10T10:46:07.705416+0000","last_active":"2024-09-10T10:46:09.813401+0000","last_peered":"2024-09-10T10:46:09.813401+0000","last_clean":"2024-09-10T10:46:09.813401+0000","last_became_active":"2024-09-10T10:46:07.704115+0000","last_became_peered":"2024-09-10T10:46:07.704115+0000","last_unstale":"2024-09-10T10:46:09.813401+0000","last_undegraded":"2024-09-10T10:46:09.813401+0000","last_fullsized":"2024-09-10T10:46:09.813401+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:46.837747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813564+0000","last_change":"2024-09-10T10:46:08.725273+0000","last_active":"2024-09-10T10:46:09.813564+0000","last_peered":"2024-09-10T10:46:09.813564+0000","last_clean":"2024-09-10T10:46:09.813564+0000","last_became_active":"2024-09-10T10:46:08.724954+0000","last_became_peered":"2024-09-10T10:46:08.724954+0000","last_unstale":"2024-09-10T10:46:09.813564+0000","last_undegraded":"2024-09-10T10:46:09.813564+0000","last_fullsized":"2024-09-10T10:46:09.813564+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813282+0000","last_change":"2024-09-10T10:46:07.713210+0000","last_active":"2024-09-10T10:46:09.813282+0000","last_peered":"2024-09-10T10:46:09.813282+0000","last_clean":"2024-09-10T10:46:09.813282+0000","last_became_active":"2024-09-10T10:46:07.710889+0000","last_became_peered":"2024-09-10T10:46:07.710889+0000","last_unstale":"2024-09-10T10:46:09.813282+0000","last_undegraded":"2024-09-10T10:46:09.813282+0000","last_fullsized":"2024-09-10T10:46:09.813282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:29.447715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722347+0000","last_change":"2024-09-10T10:46:08.744802+0000","last_active":"2024-09-10T10:46:09.722347+0000","last_peered":"2024-09-10T10:46:09.722347+0000","last_clean":"2024-09-10T10:46:09.722347+0000","last_became_active":"2024-09-10T10:46:08.744502+0000","last_became_peered":"2024-09-10T10:46:08.744502+0000","last_unstale":"2024-09-10T10:46:09.722347+0000","last_undegraded":"2024-09-10T10:46:09.722347+0000","last_fullsized":"2024-09-10T10:46:09.722347+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.340396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717734+0000","last_change":"2024-09-10T10:46:07.713246+0000","last_active":"2024-09-10T10:46:08.717734+0000","last_peered":"2024-09-10T10:46:08.717734+0000","last_clean":"2024-09-10T10:46:08.717734+0000","last_became_active":"2024-09-10T10:46:07.712946+0000","last_became_peered":"2024-09-10T10:46:07.712946+0000","last_unstale":"2024-09-10T10:46:08.717734+0000","last_undegraded":"2024-09-10T10:46:08.717734+0000","last_fullsized":"2024-09-10T10:46:08.717734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:01.541318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":232,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813340+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:09.813340+0000","last_peered":"2024-09-10T10:46:09.813340+0000","last_clean":"2024-09-10T10:46:09.813340+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:09.813340+0000","last_undegraded":"2024-09-10T10:46:09.813340+0000","last_fullsized":"2024-09-10T10:46:09.813340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":667648,"data_stored":293601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":663552,"data_stored":294127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":295249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":663552,"data_stored":293635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":663552,"data_stored":293579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":877291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":294127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1228800,"data_stored":870054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:12.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:13.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.119+0000 7fef86e00640 1 Processor -- start 2024-09-10T10:46:13.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.119+0000 7fef86e00640 1 -- start start 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef86e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef8015b300 0x7fef800cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef86e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800cdbe0 con 0x7fef8006f8d0 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef86e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef800cd900 con 0x7fef800cd3f0 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef86e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fef800728a0 con 0x7fef8015b300 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef7fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef8015b300 0x7fef800cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef84e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fef8006f8d0 0x7fef8015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50786/0 (socket says 172.21.15.28:50786) 2024-09-10T10:46:13.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef84e00640 1 -- 172.21.15.28:0/1816175669 learned_addr learned my addr 172.21.15.28:0/1816175669 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:13.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef7f400640 1 -- 172.21.15.28:0/1816175669 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2931247736 0 0) 0x7fef800cdbe0 con 0x7fef8006f8d0 2024-09-10T10:46:13.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef7fe00640 1 -- 172.21.15.28:0/1816175669 >> v1:172.21.15.138:6789/0 conn(0x7fef800cd3f0 legacy=0x7fef8006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef7fe00640 1 -- 172.21.15.28:0/1816175669 >> v1:172.21.15.28:6789/0 conn(0x7fef8006f8d0 legacy=0x7fef8015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.121+0000 7fef7fe00640 1 -- 172.21.15.28:0/1816175669 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef80072250 con 0x7fef8015b300 2024-09-10T10:46:13.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.122+0000 7fef7fe00640 1 --2- 172.21.15.28:0/1816175669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef8015b300 0x7fef800cceb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef700097a0 tx=0x7fef7002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7229274c0f03c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef7f400640 1 -- 172.21.15.28:0/1816175669 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70037070 con 0x7fef8015b300 2024-09-10T10:46:13.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef7f400640 1 -- 172.21.15.28:0/1816175669 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef70032440 con 0x7fef8015b300 2024-09-10T10:46:13.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef7f400640 1 -- 172.21.15.28:0/1816175669 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70032810 con 0x7fef8015b300 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 -- 172.21.15.28:0/1816175669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef8015b300 msgr2=0x7fef800cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 --2- 172.21.15.28:0/1816175669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef8015b300 0x7fef800cceb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef700097a0 tx=0x7fef7002a230 comp rx=0 tx=0).stop 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 -- 172.21.15.28:0/1816175669 shutdown_connections 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 --2- 172.21.15.28:0/1816175669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef8015b300 0x7fef800cceb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 -- 172.21.15.28:0/1816175669 >> 172.21.15.28:0/1816175669 conn(0x7fef80067d70 msgr2=0x7fef8006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 -- 172.21.15.28:0/1816175669 shutdown_connections 2024-09-10T10:46:13.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.123+0000 7fef86e00640 1 -- 172.21.15.28:0/1816175669 wait complete. 2024-09-10T10:46:13.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.125+0000 7fef86e00640 1 Processor -- start 2024-09-10T10:46:13.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.125+0000 7fef86e00640 1 -- start start 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 0x7fef80159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef8015b300 0x7fef800aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fef800d0160 con 0x7fef800cd3f0 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef800cffe0 con 0x7fef8006f8d0 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef86e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef80072d30 con 0x7fef8015b300 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41370/0 (socket says 172.21.15.28:41370) 2024-09-10T10:46:13.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef77e00640 1 -- 172.21.15.28:0/2680677567 learned_addr learned my addr 172.21.15.28:0/2680677567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef7fe00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 0x7fef80159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.126+0000 7fef84e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef8015b300 0x7fef800aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 -- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef8015b300 msgr2=0x7fef800aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef8015b300 0x7fef800aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 -- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 msgr2=0x7fef80159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 0x7fef80159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 -- 172.21.15.28:0/2680677567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef800aaa20 con 0x7fef800cd3f0 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef7fe00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 0x7fef80159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:13.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef77e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c450 tx=0x7fef6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef680052d0 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68005470 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68012870 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.127+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801f3610 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.128+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801f3bd0 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.129+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef8015da90 con 0x7fef800cd3f0 2024-09-10T10:46:13.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.130+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef68012a10 con 0x7fef800cd3f0 2024-09-10T10:46:13.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.130+0000 7fef7cc00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.130+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fef6808aaa0 con 0x7fef800cd3f0 2024-09-10T10:46:13.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.131+0000 7fef7fe00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.132+0000 7fef7fe00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 0x7fef5c06e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef70034000 tx=0x7fef7002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.135+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef68058c70 con 0x7fef800cd3f0 2024-09-10T10:46:13.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.316+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef8006d090 con 0x7fef5c06be30 2024-09-10T10:46:13.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.323+0000 7fef7cc00640 1 -- 172.21.15.28:0/2680677567 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7fef8006d090 con 0x7fef5c06be30 2024-09-10T10:46:13.327 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:13.328 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:13.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 msgr2=0x7fef5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 0x7fef5c06e2f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef70034000 tx=0x7fef7002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 msgr2=0x7fef80159670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fef6800c450 tx=0x7fef6800c910 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 shutdown_connections 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef5c06be30 0x7fef5c06e2f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef8015b300 0x7fef800aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef800cd3f0 0x7fef80159670 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 --2- 172.21.15.28:0/2680677567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef8006f8d0 0x7fef80159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 >> 172.21.15.28:0/2680677567 conn(0x7fef80067d70 msgr2=0x7fef8015c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 shutdown_connections 2024-09-10T10:46:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.327+0000 7fef86e00640 1 -- 172.21.15.28:0/2680677567 wait complete. 2024-09-10T10:46:13.345 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-10T10:46:12.517753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239224,"kb_used_data":6324,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735496,"statfs":{"total":773094113280,"available":772849147904,"internally_reserved":0,"allocated":6475776,"data_stored":3511663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.831169"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725248+0000","last_change":"2024-09-10T10:46:08.731491+0000","last_active":"2024-09-10T10:46:09.725248+0000","last_peered":"2024-09-10T10:46:09.725248+0000","last_clean":"2024-09-10T10:46:09.725248+0000","last_became_active":"2024-09-10T10:46:08.731001+0000","last_became_peered":"2024-09-10T10:46:08.731001+0000","last_unstale":"2024-09-10T10:46:09.725248+0000","last_undegraded":"2024-09-10T10:46:09.725248+0000","last_fullsized":"2024-09-10T10:46:09.725248+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:59.699199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814157+0000","last_change":"2024-09-10T10:46:08.736172+0000","last_active":"2024-09-10T10:46:09.814157+0000","last_peered":"2024-09-10T10:46:09.814157+0000","last_clean":"2024-09-10T10:46:09.814157+0000","last_became_active":"2024-09-10T10:46:08.735806+0000","last_became_peered":"2024-09-10T10:46:08.735806+0000","last_unstale":"2024-09-10T10:46:09.814157+0000","last_undegraded":"2024-09-10T10:46:09.814157+0000","last_fullsized":"2024-09-10T10:46:09.814157+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:19.951263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723807+0000","last_change":"2024-09-10T10:46:08.735286+0000","last_active":"2024-09-10T10:46:09.723807+0000","last_peered":"2024-09-10T10:46:09.723807+0000","last_clean":"2024-09-10T10:46:09.723807+0000","last_became_active":"2024-09-10T10:46:08.735028+0000","last_became_peered":"2024-09-10T10:46:08.735028+0000","last_unstale":"2024-09-10T10:46:09.723807+0000","last_undegraded":"2024-09-10T10:46:09.723807+0000","last_fullsized":"2024-09-10T10:46:09.723807+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:57:25.081964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814391+0000","last_change":"2024-09-10T10:46:08.722002+0000","last_active":"2024-09-10T10:46:09.814391+0000","last_peered":"2024-09-10T10:46:09.814391+0000","last_clean":"2024-09-10T10:46:09.814391+0000","last_became_active":"2024-09-10T10:46:08.721729+0000","last_became_peered":"2024-09-10T10:46:08.721729+0000","last_unstale":"2024-09-10T10:46:09.814391+0000","last_undegraded":"2024-09-10T10:46:09.814391+0000","last_fullsized":"2024-09-10T10:46:09.814391+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813826+0000","last_change":"2024-09-10T10:46:08.728209+0000","last_active":"2024-09-10T10:46:09.813826+0000","last_peered":"2024-09-10T10:46:09.813826+0000","last_clean":"2024-09-10T10:46:09.813826+0000","last_became_active":"2024-09-10T10:46:08.727995+0000","last_became_peered":"2024-09-10T10:46:08.727995+0000","last_unstale":"2024-09-10T10:46:09.813826+0000","last_undegraded":"2024-09-10T10:46:09.813826+0000","last_fullsized":"2024-09-10T10:46:09.813826+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813299+0000","last_change":"2024-09-10T10:46:08.722366+0000","last_active":"2024-09-10T10:46:09.813299+0000","last_peered":"2024-09-10T10:46:09.813299+0000","last_clean":"2024-09-10T10:46:09.813299+0000","last_became_active":"2024-09-10T10:46:08.721216+0000","last_became_peered":"2024-09-10T10:46:08.721216+0000","last_unstale":"2024-09-10T10:46:09.813299+0000","last_undegraded":"2024-09-10T10:46:09.813299+0000","last_fullsized":"2024-09-10T10:46:09.813299+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:19.587816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.732487+0000","last_change":"2024-09-10T10:46:08.727859+0000","last_active":"2024-09-10T10:46:08.732487+0000","last_peered":"2024-09-10T10:46:08.732487+0000","last_clean":"2024-09-10T10:46:08.732487+0000","last_became_active":"2024-09-10T10:46:08.727591+0000","last_became_peered":"2024-09-10T10:46:08.727591+0000","last_unstale":"2024-09-10T10:46:08.732487+0000","last_undegraded":"2024-09-10T10:46:08.732487+0000","last_fullsized":"2024-09-10T10:46:08.732487+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:33:14.148707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813223+0000","last_change":"2024-09-10T10:46:08.727596+0000","last_active":"2024-09-10T10:46:09.813223+0000","last_peered":"2024-09-10T10:46:09.813223+0000","last_clean":"2024-09-10T10:46:09.813223+0000","last_became_active":"2024-09-10T10:46:08.727279+0000","last_became_peered":"2024-09-10T10:46:08.727279+0000","last_unstale":"2024-09-10T10:46:09.813223+0000","last_undegraded":"2024-09-10T10:46:09.813223+0000","last_fullsized":"2024-09-10T10:46:09.813223+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:50.976281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725740+0000","last_change":"2024-09-10T10:46:08.732159+0000","last_active":"2024-09-10T10:46:09.725740+0000","last_peered":"2024-09-10T10:46:09.725740+0000","last_clean":"2024-09-10T10:46:09.725740+0000","last_became_active":"2024-09-10T10:46:08.731852+0000","last_became_peered":"2024-09-10T10:46:08.731852+0000","last_unstale":"2024-09-10T10:46:09.725740+0000","last_undegraded":"2024-09-10T10:46:09.725740+0000","last_fullsized":"2024-09-10T10:46:09.725740+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:47.885943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.745802+0000","last_change":"2024-09-10T10:46:08.740693+0000","last_active":"2024-09-10T10:46:08.745802+0000","last_peered":"2024-09-10T10:46:08.745802+0000","last_clean":"2024-09-10T10:46:08.745802+0000","last_became_active":"2024-09-10T10:46:08.740409+0000","last_became_peered":"2024-09-10T10:46:08.740409+0000","last_unstale":"2024-09-10T10:46:08.745802+0000","last_undegraded":"2024-09-10T10:46:08.745802+0000","last_fullsized":"2024-09-10T10:46:08.745802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:17.881635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724643+0000","last_change":"2024-09-10T10:46:08.737434+0000","last_active":"2024-09-10T10:46:09.724643+0000","last_peered":"2024-09-10T10:46:09.724643+0000","last_clean":"2024-09-10T10:46:09.724643+0000","last_became_active":"2024-09-10T10:46:08.737147+0000","last_became_peered":"2024-09-10T10:46:08.737147+0000","last_unstale":"2024-09-10T10:46:09.724643+0000","last_undegraded":"2024-09-10T10:46:09.724643+0000","last_fullsized":"2024-09-10T10:46:09.724643+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:08.531864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813300+0000","last_change":"2024-09-10T10:46:08.723796+0000","last_active":"2024-09-10T10:46:09.813300+0000","last_peered":"2024-09-10T10:46:09.813300+0000","last_clean":"2024-09-10T10:46:09.813300+0000","last_became_active":"2024-09-10T10:46:08.723499+0000","last_became_peered":"2024-09-10T10:46:08.723499+0000","last_unstale":"2024-09-10T10:46:09.813300+0000","last_undegraded":"2024-09-10T10:46:09.813300+0000","last_fullsized":"2024-09-10T10:46:09.813300+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:04.324094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813383+0000","last_change":"2024-09-10T10:46:08.718578+0000","last_active":"2024-09-10T10:46:09.813383+0000","last_peered":"2024-09-10T10:46:09.813383+0000","last_clean":"2024-09-10T10:46:09.813383+0000","last_became_active":"2024-09-10T10:46:08.718226+0000","last_became_peered":"2024-09-10T10:46:08.718226+0000","last_unstale":"2024-09-10T10:46:09.813383+0000","last_undegraded":"2024-09-10T10:46:09.813383+0000","last_fullsized":"2024-09-10T10:46:09.813383+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723965+0000","last_change":"2024-09-10T10:46:08.734994+0000","last_active":"2024-09-10T10:46:09.723965+0000","last_peered":"2024-09-10T10:46:09.723965+0000","last_clean":"2024-09-10T10:46:09.723965+0000","last_became_active":"2024-09-10T10:46:08.734622+0000","last_became_peered":"2024-09-10T10:46:08.734622+0000","last_unstale":"2024-09-10T10:46:09.723965+0000","last_undegraded":"2024-09-10T10:46:09.723965+0000","last_fullsized":"2024-09-10T10:46:09.723965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.340396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813373+0000","last_change":"2024-09-10T10:46:08.742258+0000","last_active":"2024-09-10T10:46:09.813373+0000","last_peered":"2024-09-10T10:46:09.813373+0000","last_clean":"2024-09-10T10:46:09.813373+0000","last_became_active":"2024-09-10T10:46:08.741356+0000","last_became_peered":"2024-09-10T10:46:08.741356+0000","last_unstale":"2024-09-10T10:46:09.813373+0000","last_undegraded":"2024-09-10T10:46:09.813373+0000","last_fullsized":"2024-09-10T10:46:09.813373+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:04.324094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813354+0000","last_change":"2024-09-10T10:46:08.714995+0000","last_active":"2024-09-10T10:46:09.813354+0000","last_peered":"2024-09-10T10:46:09.813354+0000","last_clean":"2024-09-10T10:46:09.813354+0000","last_became_active":"2024-09-10T10:46:08.714707+0000","last_became_peered":"2024-09-10T10:46:08.714707+0000","last_unstale":"2024-09-10T10:46:09.813354+0000","last_undegraded":"2024-09-10T10:46:09.813354+0000","last_fullsized":"2024-09-10T10:46:09.813354+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725490+0000","last_change":"2024-09-10T10:46:08.731261+0000","last_active":"2024-09-10T10:46:09.725490+0000","last_peered":"2024-09-10T10:46:09.725490+0000","last_clean":"2024-09-10T10:46:09.725490+0000","last_became_active":"2024-09-10T10:46:08.730617+0000","last_became_peered":"2024-09-10T10:46:08.730617+0000","last_unstale":"2024-09-10T10:46:09.725490+0000","last_undegraded":"2024-09-10T10:46:09.725490+0000","last_fullsized":"2024-09-10T10:46:09.725490+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:26.132612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814459+0000","last_change":"2024-09-10T10:46:08.720781+0000","last_active":"2024-09-10T10:46:09.814459+0000","last_peered":"2024-09-10T10:46:09.814459+0000","last_clean":"2024-09-10T10:46:09.814459+0000","last_became_active":"2024-09-10T10:46:08.720467+0000","last_became_peered":"2024-09-10T10:46:08.720467+0000","last_unstale":"2024-09-10T10:46:09.814459+0000","last_undegraded":"2024-09-10T10:46:09.814459+0000","last_fullsized":"2024-09-10T10:46:09.814459+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724000+0000","last_change":"2024-09-10T10:46:08.728441+0000","last_active":"2024-09-10T10:46:09.724000+0000","last_peered":"2024-09-10T10:46:09.724000+0000","last_clean":"2024-09-10T10:46:09.724000+0000","last_became_active":"2024-09-10T10:46:08.728187+0000","last_became_peered":"2024-09-10T10:46:08.728187+0000","last_unstale":"2024-09-10T10:46:09.724000+0000","last_undegraded":"2024-09-10T10:46:09.724000+0000","last_fullsized":"2024-09-10T10:46:09.724000+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:06.385445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813911+0000","last_change":"2024-09-10T10:46:08.731118+0000","last_active":"2024-09-10T10:46:09.813911+0000","last_peered":"2024-09-10T10:46:09.813911+0000","last_clean":"2024-09-10T10:46:09.813911+0000","last_became_active":"2024-09-10T10:46:08.730817+0000","last_became_peered":"2024-09-10T10:46:08.730817+0000","last_unstale":"2024-09-10T10:46:09.813911+0000","last_undegraded":"2024-09-10T10:46:09.813911+0000","last_fullsized":"2024-09-10T10:46:09.813911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:27.127913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813969+0000","last_change":"2024-09-10T10:46:08.740265+0000","last_active":"2024-09-10T10:46:09.813969+0000","last_peered":"2024-09-10T10:46:09.813969+0000","last_clean":"2024-09-10T10:46:09.813969+0000","last_became_active":"2024-09-10T10:46:08.740022+0000","last_became_peered":"2024-09-10T10:46:08.740022+0000","last_unstale":"2024-09-10T10:46:09.813969+0000","last_undegraded":"2024-09-10T10:46:09.813969+0000","last_fullsized":"2024-09-10T10:46:09.813969+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724604+0000","last_change":"2024-09-10T10:46:08.737717+0000","last_active":"2024-09-10T10:46:09.724604+0000","last_peered":"2024-09-10T10:46:09.724604+0000","last_clean":"2024-09-10T10:46:09.724604+0000","last_became_active":"2024-09-10T10:46:08.735752+0000","last_became_peered":"2024-09-10T10:46:08.735752+0000","last_unstale":"2024-09-10T10:46:09.724604+0000","last_undegraded":"2024-09-10T10:46:09.724604+0000","last_fullsized":"2024-09-10T10:46:09.724604+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:47.039950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724086+0000","last_change":"2024-09-10T10:46:08.740244+0000","last_active":"2024-09-10T10:46:09.724086+0000","last_peered":"2024-09-10T10:46:09.724086+0000","last_clean":"2024-09-10T10:46:09.724086+0000","last_became_active":"2024-09-10T10:46:08.739402+0000","last_became_peered":"2024-09-10T10:46:08.739402+0000","last_unstale":"2024-09-10T10:46:09.724086+0000","last_undegraded":"2024-09-10T10:46:09.724086+0000","last_fullsized":"2024-09-10T10:46:09.724086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:03.741143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725517+0000","last_change":"2024-09-10T10:46:08.728789+0000","last_active":"2024-09-10T10:46:09.725517+0000","last_peered":"2024-09-10T10:46:09.725517+0000","last_clean":"2024-09-10T10:46:09.725517+0000","last_became_active":"2024-09-10T10:46:08.726155+0000","last_became_peered":"2024-09-10T10:46:08.726155+0000","last_unstale":"2024-09-10T10:46:09.725517+0000","last_undegraded":"2024-09-10T10:46:09.725517+0000","last_fullsized":"2024-09-10T10:46:09.725517+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:02.655655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725381+0000","last_change":"2024-09-10T10:46:08.737127+0000","last_active":"2024-09-10T10:46:09.725381+0000","last_peered":"2024-09-10T10:46:09.725381+0000","last_clean":"2024-09-10T10:46:09.725381+0000","last_became_active":"2024-09-10T10:46:08.735205+0000","last_became_peered":"2024-09-10T10:46:08.735205+0000","last_unstale":"2024-09-10T10:46:09.725381+0000","last_undegraded":"2024-09-10T10:46:09.725381+0000","last_fullsized":"2024-09-10T10:46:09.725381+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:46.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725784+0000","last_change":"2024-09-10T10:46:08.731743+0000","last_active":"2024-09-10T10:46:09.725784+0000","last_peered":"2024-09-10T10:46:09.725784+0000","last_clean":"2024-09-10T10:46:09.725784+0000","last_became_active":"2024-09-10T10:46:08.727605+0000","last_became_peered":"2024-09-10T10:46:08.727605+0000","last_unstale":"2024-09-10T10:46:09.725784+0000","last_undegraded":"2024-09-10T10:46:09.725784+0000","last_fullsized":"2024-09-10T10:46:09.725784+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:25.624393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813518+0000","last_change":"2024-09-10T10:46:08.727247+0000","last_active":"2024-09-10T10:46:09.813518+0000","last_peered":"2024-09-10T10:46:09.813518+0000","last_clean":"2024-09-10T10:46:09.813518+0000","last_became_active":"2024-09-10T10:46:08.726948+0000","last_became_peered":"2024-09-10T10:46:08.726948+0000","last_unstale":"2024-09-10T10:46:09.813518+0000","last_undegraded":"2024-09-10T10:46:09.813518+0000","last_fullsized":"2024-09-10T10:46:09.813518+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814087+0000","last_change":"2024-09-10T10:46:08.739472+0000","last_active":"2024-09-10T10:46:09.814087+0000","last_peered":"2024-09-10T10:46:09.814087+0000","last_clean":"2024-09-10T10:46:09.814087+0000","last_became_active":"2024-09-10T10:46:08.738694+0000","last_became_peered":"2024-09-10T10:46:08.738694+0000","last_unstale":"2024-09-10T10:46:09.814087+0000","last_undegraded":"2024-09-10T10:46:09.814087+0000","last_fullsized":"2024-09-10T10:46:09.814087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:26.559484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724388+0000","last_change":"2024-09-10T10:46:08.726859+0000","last_active":"2024-09-10T10:46:09.724388+0000","last_peered":"2024-09-10T10:46:09.724388+0000","last_clean":"2024-09-10T10:46:09.724388+0000","last_became_active":"2024-09-10T10:46:08.726603+0000","last_became_peered":"2024-09-10T10:46:08.726603+0000","last_unstale":"2024-09-10T10:46:09.724388+0000","last_undegraded":"2024-09-10T10:46:09.724388+0000","last_fullsized":"2024-09-10T10:46:09.724388+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:00.698467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813942+0000","last_change":"2024-09-10T10:46:08.721678+0000","last_active":"2024-09-10T10:46:09.813942+0000","last_peered":"2024-09-10T10:46:09.813942+0000","last_clean":"2024-09-10T10:46:09.813942+0000","last_became_active":"2024-09-10T10:46:08.721394+0000","last_became_peered":"2024-09-10T10:46:08.721394+0000","last_unstale":"2024-09-10T10:46:09.813942+0000","last_undegraded":"2024-09-10T10:46:09.813942+0000","last_fullsized":"2024-09-10T10:46:09.813942+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:56:38.677474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724282+0000","last_change":"2024-09-10T10:46:08.729061+0000","last_active":"2024-09-10T10:46:09.724282+0000","last_peered":"2024-09-10T10:46:09.724282+0000","last_clean":"2024-09-10T10:46:09.724282+0000","last_became_active":"2024-09-10T10:46:08.728778+0000","last_became_peered":"2024-09-10T10:46:08.728778+0000","last_unstale":"2024-09-10T10:46:09.724282+0000","last_undegraded":"2024-09-10T10:46:09.724282+0000","last_fullsized":"2024-09-10T10:46:09.724282+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:09.643039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813099+0000","last_change":"2024-09-10T10:46:08.733398+0000","last_active":"2024-09-10T10:46:09.813099+0000","last_peered":"2024-09-10T10:46:09.813099+0000","last_clean":"2024-09-10T10:46:09.813099+0000","last_became_active":"2024-09-10T10:46:08.733109+0000","last_became_peered":"2024-09-10T10:46:08.733109+0000","last_unstale":"2024-09-10T10:46:09.813099+0000","last_undegraded":"2024-09-10T10:46:09.813099+0000","last_fullsized":"2024-09-10T10:46:09.813099+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:11.850575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813628+0000","last_change":"2024-09-10T10:46:08.724201+0000","last_active":"2024-09-10T10:46:09.813628+0000","last_peered":"2024-09-10T10:46:09.813628+0000","last_clean":"2024-09-10T10:46:09.813628+0000","last_became_active":"2024-09-10T10:46:08.723940+0000","last_became_peered":"2024-09-10T10:46:08.723940+0000","last_unstale":"2024-09-10T10:46:09.813628+0000","last_undegraded":"2024-09-10T10:46:09.813628+0000","last_fullsized":"2024-09-10T10:46:09.813628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:01:35.548818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"55'4","reported_seq":18,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722703+0000","last_change":"2024-09-10T10:46:07.716998+0000","last_active":"2024-09-10T10:46:09.722703+0000","last_peered":"2024-09-10T10:46:09.722703+0000","last_clean":"2024-09-10T10:46:09.722703+0000","last_became_active":"2024-09-10T10:46:07.716359+0000","last_became_peered":"2024-09-10T10:46:07.716359+0000","last_unstale":"2024-09-10T10:46:09.722703+0000","last_undegraded":"2024-09-10T10:46:09.722703+0000","last_fullsized":"2024-09-10T10:46:09.722703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:29.370839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813155+0000","last_change":"2024-09-10T10:46:08.728403+0000","last_active":"2024-09-10T10:46:09.813155+0000","last_peered":"2024-09-10T10:46:09.813155+0000","last_clean":"2024-09-10T10:46:09.813155+0000","last_became_active":"2024-09-10T10:46:08.728096+0000","last_became_peered":"2024-09-10T10:46:08.728096+0000","last_unstale":"2024-09-10T10:46:09.813155+0000","last_undegraded":"2024-09-10T10:46:09.813155+0000","last_fullsized":"2024-09-10T10:46:09.813155+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:00.688666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813565+0000","last_change":"2024-09-10T10:46:07.722043+0000","last_active":"2024-09-10T10:46:09.813565+0000","last_peered":"2024-09-10T10:46:09.813565+0000","last_clean":"2024-09-10T10:46:09.813565+0000","last_became_active":"2024-09-10T10:46:07.721747+0000","last_became_peered":"2024-09-10T10:46:07.721747+0000","last_unstale":"2024-09-10T10:46:09.813565+0000","last_undegraded":"2024-09-10T10:46:09.813565+0000","last_fullsized":"2024-09-10T10:46:09.813565+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:29.447715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813802+0000","last_change":"2024-09-10T10:46:08.724342+0000","last_active":"2024-09-10T10:46:09.813802+0000","last_peered":"2024-09-10T10:46:09.813802+0000","last_clean":"2024-09-10T10:46:09.813802+0000","last_became_active":"2024-09-10T10:46:08.724021+0000","last_became_peered":"2024-09-10T10:46:08.724021+0000","last_unstale":"2024-09-10T10:46:09.813802+0000","last_undegraded":"2024-09-10T10:46:09.813802+0000","last_fullsized":"2024-09-10T10:46:09.813802+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:45.383097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"55'2","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.753341+0000","last_change":"2024-09-10T10:46:07.713863+0000","last_active":"2024-09-10T10:46:08.753341+0000","last_peered":"2024-09-10T10:46:08.753341+0000","last_clean":"2024-09-10T10:46:08.753341+0000","last_became_active":"2024-09-10T10:46:07.710971+0000","last_became_peered":"2024-09-10T10:46:07.710971+0000","last_unstale":"2024-09-10T10:46:08.753341+0000","last_undegraded":"2024-09-10T10:46:08.753341+0000","last_fullsized":"2024-09-10T10:46:08.753341+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:11.338846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.736082+0000","last_change":"2024-09-10T10:46:08.730931+0000","last_active":"2024-09-10T10:46:08.736082+0000","last_peered":"2024-09-10T10:46:08.736082+0000","last_clean":"2024-09-10T10:46:08.736082+0000","last_became_active":"2024-09-10T10:46:08.729480+0000","last_became_peered":"2024-09-10T10:46:08.729480+0000","last_unstale":"2024-09-10T10:46:08.736082+0000","last_undegraded":"2024-09-10T10:46:08.736082+0000","last_fullsized":"2024-09-10T10:46:08.736082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:28.671820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814186+0000","last_change":"2024-09-10T10:46:07.712592+0000","last_active":"2024-09-10T10:46:09.814186+0000","last_peered":"2024-09-10T10:46:09.814186+0000","last_clean":"2024-09-10T10:46:09.814186+0000","last_became_active":"2024-09-10T10:46:07.711623+0000","last_became_peered":"2024-09-10T10:46:07.711623+0000","last_unstale":"2024-09-10T10:46:09.814186+0000","last_undegraded":"2024-09-10T10:46:09.814186+0000","last_fullsized":"2024-09-10T10:46:09.814186+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:23:56.413128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722671+0000","last_change":"2024-09-10T10:46:08.742399+0000","last_active":"2024-09-10T10:46:09.722671+0000","last_peered":"2024-09-10T10:46:09.722671+0000","last_clean":"2024-09-10T10:46:09.722671+0000","last_became_active":"2024-09-10T10:46:08.741955+0000","last_became_peered":"2024-09-10T10:46:08.741955+0000","last_unstale":"2024-09-10T10:46:09.722671+0000","last_undegraded":"2024-09-10T10:46:09.722671+0000","last_fullsized":"2024-09-10T10:46:09.722671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:35.816372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725073+0000","last_change":"2024-09-10T10:46:07.716629+0000","last_active":"2024-09-10T10:46:09.725073+0000","last_peered":"2024-09-10T10:46:09.725073+0000","last_clean":"2024-09-10T10:46:09.725073+0000","last_became_active":"2024-09-10T10:46:07.716365+0000","last_became_peered":"2024-09-10T10:46:07.716365+0000","last_unstale":"2024-09-10T10:46:09.725073+0000","last_undegraded":"2024-09-10T10:46:09.725073+0000","last_fullsized":"2024-09-10T10:46:09.725073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:29:45.382980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814332+0000","last_change":"2024-09-10T10:46:08.729147+0000","last_active":"2024-09-10T10:46:09.814332+0000","last_peered":"2024-09-10T10:46:09.814332+0000","last_clean":"2024-09-10T10:46:09.814332+0000","last_became_active":"2024-09-10T10:46:08.728709+0000","last_became_peered":"2024-09-10T10:46:08.728709+0000","last_unstale":"2024-09-10T10:46:09.814332+0000","last_undegraded":"2024-09-10T10:46:09.814332+0000","last_fullsized":"2024-09-10T10:46:09.814332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:02:34.551797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814035+0000","last_change":"2024-09-10T10:46:07.708895+0000","last_active":"2024-09-10T10:46:09.814035+0000","last_peered":"2024-09-10T10:46:09.814035+0000","last_clean":"2024-09-10T10:46:09.814035+0000","last_became_active":"2024-09-10T10:46:07.708616+0000","last_became_peered":"2024-09-10T10:46:07.708616+0000","last_unstale":"2024-09-10T10:46:09.814035+0000","last_undegraded":"2024-09-10T10:46:09.814035+0000","last_fullsized":"2024-09-10T10:46:09.814035+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:22:50.958753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.738484+0000","last_change":"2024-09-10T10:46:08.733837+0000","last_active":"2024-09-10T10:46:08.738484+0000","last_peered":"2024-09-10T10:46:08.738484+0000","last_clean":"2024-09-10T10:46:08.738484+0000","last_became_active":"2024-09-10T10:46:08.733303+0000","last_became_peered":"2024-09-10T10:46:08.733303+0000","last_unstale":"2024-09-10T10:46:08.738484+0000","last_undegraded":"2024-09-10T10:46:08.738484+0000","last_fullsized":"2024-09-10T10:46:08.738484+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:33.178535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813593+0000","last_change":"2024-09-10T10:46:07.715697+0000","last_active":"2024-09-10T10:46:09.813593+0000","last_peered":"2024-09-10T10:46:09.813593+0000","last_clean":"2024-09-10T10:46:09.813593+0000","last_became_active":"2024-09-10T10:46:07.715368+0000","last_became_peered":"2024-09-10T10:46:07.715368+0000","last_unstale":"2024-09-10T10:46:09.813593+0000","last_undegraded":"2024-09-10T10:46:09.813593+0000","last_fullsized":"2024-09-10T10:46:09.813593+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:11.287273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814909+0000","last_change":"2024-09-10T10:46:08.723901+0000","last_active":"2024-09-10T10:46:09.814909+0000","last_peered":"2024-09-10T10:46:09.814909+0000","last_clean":"2024-09-10T10:46:09.814909+0000","last_became_active":"2024-09-10T10:46:08.723584+0000","last_became_peered":"2024-09-10T10:46:08.723584+0000","last_unstale":"2024-09-10T10:46:09.814909+0000","last_undegraded":"2024-09-10T10:46:09.814909+0000","last_fullsized":"2024-09-10T10:46:09.814909+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725660+0000","last_change":"2024-09-10T10:46:07.715434+0000","last_active":"2024-09-10T10:46:09.725660+0000","last_peered":"2024-09-10T10:46:09.725660+0000","last_clean":"2024-09-10T10:46:09.725660+0000","last_became_active":"2024-09-10T10:46:07.715169+0000","last_became_peered":"2024-09-10T10:46:07.715169+0000","last_unstale":"2024-09-10T10:46:09.725660+0000","last_undegraded":"2024-09-10T10:46:09.725660+0000","last_fullsized":"2024-09-10T10:46:09.725660+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:01.966583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722350+0000","last_change":"2024-09-10T10:46:08.733184+0000","last_active":"2024-09-10T10:46:09.722350+0000","last_peered":"2024-09-10T10:46:09.722350+0000","last_clean":"2024-09-10T10:46:09.722350+0000","last_became_active":"2024-09-10T10:46:08.729876+0000","last_became_peered":"2024-09-10T10:46:08.729876+0000","last_unstale":"2024-09-10T10:46:09.722350+0000","last_undegraded":"2024-09-10T10:46:09.722350+0000","last_fullsized":"2024-09-10T10:46:09.722350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:52.246946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814954+0000","last_change":"2024-09-10T10:46:07.714757+0000","last_active":"2024-09-10T10:46:09.814954+0000","last_peered":"2024-09-10T10:46:09.814954+0000","last_clean":"2024-09-10T10:46:09.814954+0000","last_became_active":"2024-09-10T10:46:07.714453+0000","last_became_peered":"2024-09-10T10:46:07.714453+0000","last_unstale":"2024-09-10T10:46:09.814954+0000","last_undegraded":"2024-09-10T10:46:09.814954+0000","last_fullsized":"2024-09-10T10:46:09.814954+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:01:38.551812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813377+0000","last_change":"2024-09-10T10:46:08.722159+0000","last_active":"2024-09-10T10:46:09.813377+0000","last_peered":"2024-09-10T10:46:09.813377+0000","last_clean":"2024-09-10T10:46:09.813377+0000","last_became_active":"2024-09-10T10:46:08.721864+0000","last_became_peered":"2024-09-10T10:46:08.721864+0000","last_unstale":"2024-09-10T10:46:09.813377+0000","last_undegraded":"2024-09-10T10:46:09.813377+0000","last_fullsized":"2024-09-10T10:46:09.813377+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:26:19.587816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814778+0000","last_change":"2024-09-10T10:46:07.714691+0000","last_active":"2024-09-10T10:46:09.814778+0000","last_peered":"2024-09-10T10:46:09.814778+0000","last_clean":"2024-09-10T10:46:09.814778+0000","last_became_active":"2024-09-10T10:46:07.714126+0000","last_became_peered":"2024-09-10T10:46:07.714126+0000","last_unstale":"2024-09-10T10:46:09.814778+0000","last_undegraded":"2024-09-10T10:46:09.814778+0000","last_fullsized":"2024-09-10T10:46:09.814778+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:09.345594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724130+0000","last_change":"2024-09-10T10:46:08.739881+0000","last_active":"2024-09-10T10:46:09.724130+0000","last_peered":"2024-09-10T10:46:09.724130+0000","last_clean":"2024-09-10T10:46:09.724130+0000","last_became_active":"2024-09-10T10:46:08.739565+0000","last_became_peered":"2024-09-10T10:46:08.739565+0000","last_unstale":"2024-09-10T10:46:09.724130+0000","last_undegraded":"2024-09-10T10:46:09.724130+0000","last_fullsized":"2024-09-10T10:46:09.724130+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:35.816372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"55'3","reported_seq":17,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814538+0000","last_change":"2024-09-10T10:46:07.718457+0000","last_active":"2024-09-10T10:46:09.814538+0000","last_peered":"2024-09-10T10:46:09.814538+0000","last_clean":"2024-09-10T10:46:09.814538+0000","last_became_active":"2024-09-10T10:46:07.718172+0000","last_became_peered":"2024-09-10T10:46:07.718172+0000","last_unstale":"2024-09-10T10:46:09.814538+0000","last_undegraded":"2024-09-10T10:46:09.814538+0000","last_fullsized":"2024-09-10T10:46:09.814538+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:47.945241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813825+0000","last_change":"2024-09-10T10:46:08.734452+0000","last_active":"2024-09-10T10:46:09.813825+0000","last_peered":"2024-09-10T10:46:09.813825+0000","last_clean":"2024-09-10T10:46:09.813825+0000","last_became_active":"2024-09-10T10:46:08.734154+0000","last_became_peered":"2024-09-10T10:46:08.734154+0000","last_unstale":"2024-09-10T10:46:09.813825+0000","last_undegraded":"2024-09-10T10:46:09.813825+0000","last_fullsized":"2024-09-10T10:46:09.813825+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"55'6","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.758940+0000","last_change":"2024-09-10T10:46:07.714594+0000","last_active":"2024-09-10T10:46:08.758940+0000","last_peered":"2024-09-10T10:46:08.758940+0000","last_clean":"2024-09-10T10:46:08.758940+0000","last_became_active":"2024-09-10T10:46:07.712264+0000","last_became_peered":"2024-09-10T10:46:07.712264+0000","last_unstale":"2024-09-10T10:46:08.758940+0000","last_undegraded":"2024-09-10T10:46:08.758940+0000","last_fullsized":"2024-09-10T10:46:08.758940+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:46.816918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814811+0000","last_change":"2024-09-10T10:46:08.729539+0000","last_active":"2024-09-10T10:46:09.814811+0000","last_peered":"2024-09-10T10:46:09.814811+0000","last_clean":"2024-09-10T10:46:09.814811+0000","last_became_active":"2024-09-10T10:46:08.729136+0000","last_became_peered":"2024-09-10T10:46:08.729136+0000","last_unstale":"2024-09-10T10:46:09.814811+0000","last_undegraded":"2024-09-10T10:46:09.814811+0000","last_fullsized":"2024-09-10T10:46:09.814811+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:37:00.786494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813488+0000","last_change":"2024-09-10T10:46:07.720375+0000","last_active":"2024-09-10T10:46:09.813488+0000","last_peered":"2024-09-10T10:46:09.813488+0000","last_clean":"2024-09-10T10:46:09.813488+0000","last_became_active":"2024-09-10T10:46:07.719754+0000","last_became_peered":"2024-09-10T10:46:07.719754+0000","last_unstale":"2024-09-10T10:46:09.813488+0000","last_undegraded":"2024-09-10T10:46:09.813488+0000","last_fullsized":"2024-09-10T10:46:09.813488+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:15.131104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724818+0000","last_change":"2024-09-10T10:46:08.730562+0000","last_active":"2024-09-10T10:46:09.724818+0000","last_peered":"2024-09-10T10:46:09.724818+0000","last_clean":"2024-09-10T10:46:09.724818+0000","last_became_active":"2024-09-10T10:46:08.730236+0000","last_became_peered":"2024-09-10T10:46:08.730236+0000","last_unstale":"2024-09-10T10:46:09.724818+0000","last_undegraded":"2024-09-10T10:46:09.724818+0000","last_fullsized":"2024-09-10T10:46:09.724818+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:01.438641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813394+0000","last_change":"2024-09-10T10:46:07.705803+0000","last_active":"2024-09-10T10:46:09.813394+0000","last_peered":"2024-09-10T10:46:09.813394+0000","last_clean":"2024-09-10T10:46:09.813394+0000","last_became_active":"2024-09-10T10:46:07.705541+0000","last_became_peered":"2024-09-10T10:46:07.705541+0000","last_unstale":"2024-09-10T10:46:09.813394+0000","last_undegraded":"2024-09-10T10:46:09.813394+0000","last_fullsized":"2024-09-10T10:46:09.813394+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:34:57.846918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813679+0000","last_change":"2024-09-10T10:46:08.723566+0000","last_active":"2024-09-10T10:46:09.813679+0000","last_peered":"2024-09-10T10:46:09.813679+0000","last_clean":"2024-09-10T10:46:09.813679+0000","last_became_active":"2024-09-10T10:46:08.722919+0000","last_became_peered":"2024-09-10T10:46:08.722919+0000","last_unstale":"2024-09-10T10:46:09.813679+0000","last_undegraded":"2024-09-10T10:46:09.813679+0000","last_fullsized":"2024-09-10T10:46:09.813679+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:27.127913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813313+0000","last_change":"2024-09-10T10:46:07.711985+0000","last_active":"2024-09-10T10:46:09.813313+0000","last_peered":"2024-09-10T10:46:09.813313+0000","last_clean":"2024-09-10T10:46:09.813313+0000","last_became_active":"2024-09-10T10:46:07.710241+0000","last_became_peered":"2024-09-10T10:46:07.710241+0000","last_unstale":"2024-09-10T10:46:09.813313+0000","last_undegraded":"2024-09-10T10:46:09.813313+0000","last_fullsized":"2024-09-10T10:46:09.813313+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:14.706240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.726199+0000","last_change":"2024-09-10T10:46:08.729307+0000","last_active":"2024-09-10T10:46:09.726199+0000","last_peered":"2024-09-10T10:46:09.726199+0000","last_clean":"2024-09-10T10:46:09.726199+0000","last_became_active":"2024-09-10T10:46:08.729028+0000","last_became_peered":"2024-09-10T10:46:08.729028+0000","last_unstale":"2024-09-10T10:46:09.726199+0000","last_undegraded":"2024-09-10T10:46:09.726199+0000","last_fullsized":"2024-09-10T10:46:09.726199+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:18.436383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"55'8","reported_seq":22,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813402+0000","last_change":"2024-09-10T10:46:07.721960+0000","last_active":"2024-09-10T10:46:09.813402+0000","last_peered":"2024-09-10T10:46:09.813402+0000","last_clean":"2024-09-10T10:46:09.813402+0000","last_became_active":"2024-09-10T10:46:07.721652+0000","last_became_peered":"2024-09-10T10:46:07.721652+0000","last_unstale":"2024-09-10T10:46:09.813402+0000","last_undegraded":"2024-09-10T10:46:09.813402+0000","last_fullsized":"2024-09-10T10:46:09.813402+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:22.544691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.732387+0000","last_change":"2024-09-10T10:46:08.724451+0000","last_active":"2024-09-10T10:46:08.732387+0000","last_peered":"2024-09-10T10:46:08.732387+0000","last_clean":"2024-09-10T10:46:08.732387+0000","last_became_active":"2024-09-10T10:46:08.724164+0000","last_became_peered":"2024-09-10T10:46:08.724164+0000","last_unstale":"2024-09-10T10:46:08.732387+0000","last_undegraded":"2024-09-10T10:46:08.732387+0000","last_fullsized":"2024-09-10T10:46:08.732387+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:51.510103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813690+0000","last_change":"2024-09-10T10:46:07.722761+0000","last_active":"2024-09-10T10:46:09.813690+0000","last_peered":"2024-09-10T10:46:09.813690+0000","last_clean":"2024-09-10T10:46:09.813690+0000","last_became_active":"2024-09-10T10:46:07.722475+0000","last_became_peered":"2024-09-10T10:46:07.722475+0000","last_unstale":"2024-09-10T10:46:09.813690+0000","last_undegraded":"2024-09-10T10:46:09.813690+0000","last_fullsized":"2024-09-10T10:46:09.813690+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:46:32.301872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.723671+0000","last_change":"2024-09-10T10:46:08.731134+0000","last_active":"2024-09-10T10:46:09.723671+0000","last_peered":"2024-09-10T10:46:09.723671+0000","last_clean":"2024-09-10T10:46:09.723671+0000","last_became_active":"2024-09-10T10:46:08.729420+0000","last_became_peered":"2024-09-10T10:46:08.729420+0000","last_unstale":"2024-09-10T10:46:09.723671+0000","last_undegraded":"2024-09-10T10:46:09.723671+0000","last_fullsized":"2024-09-10T10:46:09.723671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:27:52.246946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813730+0000","last_change":"2024-09-10T10:46:07.720940+0000","last_active":"2024-09-10T10:46:09.813730+0000","last_peered":"2024-09-10T10:46:09.813730+0000","last_clean":"2024-09-10T10:46:09.813730+0000","last_became_active":"2024-09-10T10:46:07.720668+0000","last_became_peered":"2024-09-10T10:46:07.720668+0000","last_unstale":"2024-09-10T10:46:09.813730+0000","last_undegraded":"2024-09-10T10:46:09.813730+0000","last_fullsized":"2024-09-10T10:46:09.813730+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:26.871070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813203+0000","last_change":"2024-09-10T10:46:08.744432+0000","last_active":"2024-09-10T10:46:09.813203+0000","last_peered":"2024-09-10T10:46:09.813203+0000","last_clean":"2024-09-10T10:46:09.813203+0000","last_became_active":"2024-09-10T10:46:08.740365+0000","last_became_peered":"2024-09-10T10:46:08.740365+0000","last_unstale":"2024-09-10T10:46:09.813203+0000","last_undegraded":"2024-09-10T10:46:09.813203+0000","last_fullsized":"2024-09-10T10:46:09.813203+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:44:34.327875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"55'1","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814615+0000","last_change":"2024-09-10T10:46:07.714468+0000","last_active":"2024-09-10T10:46:09.814615+0000","last_peered":"2024-09-10T10:46:09.814615+0000","last_clean":"2024-09-10T10:46:09.814615+0000","last_became_active":"2024-09-10T10:46:07.714129+0000","last_became_peered":"2024-09-10T10:46:07.714129+0000","last_unstale":"2024-09-10T10:46:09.814615+0000","last_undegraded":"2024-09-10T10:46:09.814615+0000","last_fullsized":"2024-09-10T10:46:09.814615+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:51.520301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813417+0000","last_change":"2024-09-10T10:46:08.731968+0000","last_active":"2024-09-10T10:46:09.813417+0000","last_peered":"2024-09-10T10:46:09.813417+0000","last_clean":"2024-09-10T10:46:09.813417+0000","last_became_active":"2024-09-10T10:46:08.729878+0000","last_became_peered":"2024-09-10T10:46:08.729878+0000","last_unstale":"2024-09-10T10:46:09.813417+0000","last_undegraded":"2024-09-10T10:46:09.813417+0000","last_fullsized":"2024-09-10T10:46:09.813417+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.702245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814223+0000","last_change":"2024-09-10T10:46:07.707015+0000","last_active":"2024-09-10T10:46:09.814223+0000","last_peered":"2024-09-10T10:46:09.814223+0000","last_clean":"2024-09-10T10:46:09.814223+0000","last_became_active":"2024-09-10T10:46:07.706746+0000","last_became_peered":"2024-09-10T10:46:07.706746+0000","last_unstale":"2024-09-10T10:46:09.814223+0000","last_undegraded":"2024-09-10T10:46:09.814223+0000","last_fullsized":"2024-09-10T10:46:09.814223+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:21.110648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814125+0000","last_change":"2024-09-10T10:46:08.727633+0000","last_active":"2024-09-10T10:46:09.814125+0000","last_peered":"2024-09-10T10:46:09.814125+0000","last_clean":"2024-09-10T10:46:09.814125+0000","last_became_active":"2024-09-10T10:46:08.727270+0000","last_became_peered":"2024-09-10T10:46:08.727270+0000","last_unstale":"2024-09-10T10:46:09.814125+0000","last_undegraded":"2024-09-10T10:46:09.814125+0000","last_fullsized":"2024-09-10T10:46:09.814125+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:51.976999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724415+0000","last_change":"2024-09-10T10:46:07.710387+0000","last_active":"2024-09-10T10:46:09.724415+0000","last_peered":"2024-09-10T10:46:09.724415+0000","last_clean":"2024-09-10T10:46:09.724415+0000","last_became_active":"2024-09-10T10:46:07.708742+0000","last_became_peered":"2024-09-10T10:46:07.708742+0000","last_unstale":"2024-09-10T10:46:09.724415+0000","last_undegraded":"2024-09-10T10:46:09.724415+0000","last_fullsized":"2024-09-10T10:46:09.724415+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:49.969992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814635+0000","last_change":"2024-09-10T10:46:08.741798+0000","last_active":"2024-09-10T10:46:09.814635+0000","last_peered":"2024-09-10T10:46:09.814635+0000","last_clean":"2024-09-10T10:46:09.814635+0000","last_became_active":"2024-09-10T10:46:08.741504+0000","last_became_peered":"2024-09-10T10:46:08.741504+0000","last_unstale":"2024-09-10T10:46:09.814635+0000","last_undegraded":"2024-09-10T10:46:09.814635+0000","last_fullsized":"2024-09-10T10:46:09.814635+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:49:43.308094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724596+0000","last_change":"2024-09-10T10:46:07.717198+0000","last_active":"2024-09-10T10:46:09.724596+0000","last_peered":"2024-09-10T10:46:09.724596+0000","last_clean":"2024-09-10T10:46:09.724596+0000","last_became_active":"2024-09-10T10:46:07.716874+0000","last_became_peered":"2024-09-10T10:46:07.716874+0000","last_unstale":"2024-09-10T10:46:09.724596+0000","last_undegraded":"2024-09-10T10:46:09.724596+0000","last_fullsized":"2024-09-10T10:46:09.724596+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:36:37.678077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.724876+0000","last_change":"2024-09-10T10:46:08.739788+0000","last_active":"2024-09-10T10:46:09.724876+0000","last_peered":"2024-09-10T10:46:09.724876+0000","last_clean":"2024-09-10T10:46:09.724876+0000","last_became_active":"2024-09-10T10:46:08.739502+0000","last_became_peered":"2024-09-10T10:46:08.739502+0000","last_unstale":"2024-09-10T10:46:09.724876+0000","last_undegraded":"2024-09-10T10:46:09.724876+0000","last_fullsized":"2024-09-10T10:46:09.724876+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:23.466620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.715555+0000","last_change":"2024-09-10T10:46:07.712656+0000","last_active":"2024-09-10T10:46:08.715555+0000","last_peered":"2024-09-10T10:46:08.715555+0000","last_clean":"2024-09-10T10:46:08.715555+0000","last_became_active":"2024-09-10T10:46:07.712362+0000","last_became_peered":"2024-09-10T10:46:07.712362+0000","last_unstale":"2024-09-10T10:46:08.715555+0000","last_undegraded":"2024-09-10T10:46:08.715555+0000","last_fullsized":"2024-09-10T10:46:08.715555+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:28:10.504934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813746+0000","last_change":"2024-09-10T10:46:08.727931+0000","last_active":"2024-09-10T10:46:09.813746+0000","last_peered":"2024-09-10T10:46:09.813746+0000","last_clean":"2024-09-10T10:46:09.813746+0000","last_became_active":"2024-09-10T10:46:08.727650+0000","last_became_peered":"2024-09-10T10:46:08.727650+0000","last_unstale":"2024-09-10T10:46:09.813746+0000","last_undegraded":"2024-09-10T10:46:09.813746+0000","last_fullsized":"2024-09-10T10:46:09.813746+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:26.559484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717316+0000","last_change":"2024-09-10T10:46:07.703518+0000","last_active":"2024-09-10T10:46:08.717316+0000","last_peered":"2024-09-10T10:46:08.717316+0000","last_clean":"2024-09-10T10:46:08.717316+0000","last_became_active":"2024-09-10T10:46:07.703228+0000","last_became_peered":"2024-09-10T10:46:07.703228+0000","last_unstale":"2024-09-10T10:46:08.717316+0000","last_undegraded":"2024-09-10T10:46:08.717316+0000","last_fullsized":"2024-09-10T10:46:08.717316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:07.526127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.812984+0000","last_change":"2024-09-10T10:46:08.729574+0000","last_active":"2024-09-10T10:46:09.812984+0000","last_peered":"2024-09-10T10:46:09.812984+0000","last_clean":"2024-09-10T10:46:09.812984+0000","last_became_active":"2024-09-10T10:46:08.729259+0000","last_became_peered":"2024-09-10T10:46:08.729259+0000","last_unstale":"2024-09-10T10:46:09.812984+0000","last_undegraded":"2024-09-10T10:46:09.812984+0000","last_fullsized":"2024-09-10T10:46:09.812984+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:52:50.976281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"55'2","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814462+0000","last_change":"2024-09-10T10:46:07.716713+0000","last_active":"2024-09-10T10:46:09.814462+0000","last_peered":"2024-09-10T10:46:09.814462+0000","last_clean":"2024-09-10T10:46:09.814462+0000","last_became_active":"2024-09-10T10:46:07.716445+0000","last_became_peered":"2024-09-10T10:46:07.716445+0000","last_unstale":"2024-09-10T10:46:09.814462+0000","last_undegraded":"2024-09-10T10:46:09.814462+0000","last_fullsized":"2024-09-10T10:46:09.814462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:33.003887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.734302+0000","last_change":"2024-09-10T10:46:08.729158+0000","last_active":"2024-09-10T10:46:08.734302+0000","last_peered":"2024-09-10T10:46:08.734302+0000","last_clean":"2024-09-10T10:46:08.734302+0000","last_became_active":"2024-09-10T10:46:08.728872+0000","last_became_peered":"2024-09-10T10:46:08.728872+0000","last_unstale":"2024-09-10T10:46:08.734302+0000","last_undegraded":"2024-09-10T10:46:08.734302+0000","last_fullsized":"2024-09-10T10:46:08.734302+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:26:16.179598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.814448+0000","last_change":"2024-09-10T10:46:07.718458+0000","last_active":"2024-09-10T10:46:09.814448+0000","last_peered":"2024-09-10T10:46:09.814448+0000","last_clean":"2024-09-10T10:46:09.814448+0000","last_became_active":"2024-09-10T10:46:07.718229+0000","last_became_peered":"2024-09-10T10:46:07.718229+0000","last_unstale":"2024-09-10T10:46:09.814448+0000","last_undegraded":"2024-09-10T10:46:09.814448+0000","last_fullsized":"2024-09-10T10:46:09.814448+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:57:21.948804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725139+0000","last_change":"2024-09-10T10:46:08.732363+0000","last_active":"2024-09-10T10:46:09.725139+0000","last_peered":"2024-09-10T10:46:09.725139+0000","last_clean":"2024-09-10T10:46:09.725139+0000","last_became_active":"2024-09-10T10:46:08.732091+0000","last_became_peered":"2024-09-10T10:46:08.732091+0000","last_unstale":"2024-09-10T10:46:09.725139+0000","last_undegraded":"2024-09-10T10:46:09.725139+0000","last_fullsized":"2024-09-10T10:46:09.725139+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:31.871643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725371+0000","last_change":"2024-09-10T10:46:07.708779+0000","last_active":"2024-09-10T10:46:09.725371+0000","last_peered":"2024-09-10T10:46:09.725371+0000","last_clean":"2024-09-10T10:46:09.725371+0000","last_became_active":"2024-09-10T10:46:07.708506+0000","last_became_peered":"2024-09-10T10:46:07.708506+0000","last_unstale":"2024-09-10T10:46:09.725371+0000","last_undegraded":"2024-09-10T10:46:09.725371+0000","last_fullsized":"2024-09-10T10:46:09.725371+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:35.116526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.735676+0000","last_change":"2024-09-10T10:46:08.731256+0000","last_active":"2024-09-10T10:46:08.735676+0000","last_peered":"2024-09-10T10:46:08.735676+0000","last_clean":"2024-09-10T10:46:08.735676+0000","last_became_active":"2024-09-10T10:46:08.728615+0000","last_became_peered":"2024-09-10T10:46:08.728615+0000","last_unstale":"2024-09-10T10:46:08.735676+0000","last_undegraded":"2024-09-10T10:46:08.735676+0000","last_fullsized":"2024-09-10T10:46:08.735676+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:23.031322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717697+0000","last_change":"2024-09-10T10:46:07.709486+0000","last_active":"2024-09-10T10:46:08.717697+0000","last_peered":"2024-09-10T10:46:08.717697+0000","last_clean":"2024-09-10T10:46:08.717697+0000","last_became_active":"2024-09-10T10:46:07.707686+0000","last_became_peered":"2024-09-10T10:46:07.707686+0000","last_unstale":"2024-09-10T10:46:08.717697+0000","last_undegraded":"2024-09-10T10:46:08.717697+0000","last_fullsized":"2024-09-10T10:46:08.717697+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:48.823340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813890+0000","last_change":"2024-09-10T10:46:08.743902+0000","last_active":"2024-09-10T10:46:09.813890+0000","last_peered":"2024-09-10T10:46:09.813890+0000","last_clean":"2024-09-10T10:46:09.813890+0000","last_became_active":"2024-09-10T10:46:08.743541+0000","last_became_peered":"2024-09-10T10:46:08.743541+0000","last_unstale":"2024-09-10T10:46:09.813890+0000","last_undegraded":"2024-09-10T10:46:09.813890+0000","last_fullsized":"2024-09-10T10:46:09.813890+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.725447+0000","last_change":"2024-09-10T10:46:07.718465+0000","last_active":"2024-09-10T10:46:09.725447+0000","last_peered":"2024-09-10T10:46:09.725447+0000","last_clean":"2024-09-10T10:46:09.725447+0000","last_became_active":"2024-09-10T10:46:07.718183+0000","last_became_peered":"2024-09-10T10:46:07.718183+0000","last_unstale":"2024-09-10T10:46:09.725447+0000","last_undegraded":"2024-09-10T10:46:09.725447+0000","last_fullsized":"2024-09-10T10:46:09.725447+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:42.124747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.738462+0000","last_change":"2024-09-10T10:46:08.733569+0000","last_active":"2024-09-10T10:46:08.738462+0000","last_peered":"2024-09-10T10:46:08.738462+0000","last_clean":"2024-09-10T10:46:08.738462+0000","last_became_active":"2024-09-10T10:46:08.732933+0000","last_became_peered":"2024-09-10T10:46:08.732933+0000","last_unstale":"2024-09-10T10:46:08.738462+0000","last_undegraded":"2024-09-10T10:46:08.738462+0000","last_fullsized":"2024-09-10T10:46:08.738462+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:34.479584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813401+0000","last_change":"2024-09-10T10:46:07.705416+0000","last_active":"2024-09-10T10:46:09.813401+0000","last_peered":"2024-09-10T10:46:09.813401+0000","last_clean":"2024-09-10T10:46:09.813401+0000","last_became_active":"2024-09-10T10:46:07.704115+0000","last_became_peered":"2024-09-10T10:46:07.704115+0000","last_unstale":"2024-09-10T10:46:09.813401+0000","last_undegraded":"2024-09-10T10:46:09.813401+0000","last_fullsized":"2024-09-10T10:46:09.813401+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:46.837747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813564+0000","last_change":"2024-09-10T10:46:08.725273+0000","last_active":"2024-09-10T10:46:09.813564+0000","last_peered":"2024-09-10T10:46:09.813564+0000","last_clean":"2024-09-10T10:46:09.813564+0000","last_became_active":"2024-09-10T10:46:08.724954+0000","last_became_peered":"2024-09-10T10:46:08.724954+0000","last_unstale":"2024-09-10T10:46:09.813564+0000","last_undegraded":"2024-09-10T10:46:09.813564+0000","last_fullsized":"2024-09-10T10:46:09.813564+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:48:30.377128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813282+0000","last_change":"2024-09-10T10:46:07.713210+0000","last_active":"2024-09-10T10:46:09.813282+0000","last_peered":"2024-09-10T10:46:09.813282+0000","last_clean":"2024-09-10T10:46:09.813282+0000","last_became_active":"2024-09-10T10:46:07.710889+0000","last_became_peered":"2024-09-10T10:46:07.710889+0000","last_unstale":"2024-09-10T10:46:09.813282+0000","last_undegraded":"2024-09-10T10:46:09.813282+0000","last_fullsized":"2024-09-10T10:46:09.813282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:29.447715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.722347+0000","last_change":"2024-09-10T10:46:08.744802+0000","last_active":"2024-09-10T10:46:09.722347+0000","last_peered":"2024-09-10T10:46:09.722347+0000","last_clean":"2024-09-10T10:46:09.722347+0000","last_became_active":"2024-09-10T10:46:08.744502+0000","last_became_peered":"2024-09-10T10:46:08.744502+0000","last_unstale":"2024-09-10T10:46:09.722347+0000","last_undegraded":"2024-09-10T10:46:09.722347+0000","last_fullsized":"2024-09-10T10:46:09.722347+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:07.685871+0000","last_clean_scrub_stamp":"2024-09-10T10:46:07.685871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:48.340396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-10T10:46:08.717734+0000","last_change":"2024-09-10T10:46:07.713246+0000","last_active":"2024-09-10T10:46:08.717734+0000","last_peered":"2024-09-10T10:46:08.717734+0000","last_clean":"2024-09-10T10:46:08.717734+0000","last_became_active":"2024-09-10T10:46:07.712946+0000","last_became_peered":"2024-09-10T10:46:07.712946+0000","last_unstale":"2024-09-10T10:46:08.717734+0000","last_undegraded":"2024-09-10T10:46:08.717734+0000","last_fullsized":"2024-09-10T10:46:08.717734+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:06.679582+0000","last_clean_scrub_stamp":"2024-09-10T10:46:06.679582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:01.541318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":232,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-10T10:46:09.813340+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:09.813340+0000","last_peered":"2024-09-10T10:46:09.813340+0000","last_clean":"2024-09-10T10:46:09.813340+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:09.813340+0000","last_undegraded":"2024-09-10T10:46:09.813340+0000","last_fullsized":"2024-09-10T10:46:09.813340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":667648,"data_stored":293601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":663552,"data_stored":294127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":679936,"data_stored":295249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509474,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":663552,"data_stored":293635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":663552,"data_stored":293579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":877291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31836,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":663552,"data_stored":294127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1228800,"data_stored":870054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:13.348 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:46:13.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:46:13.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.549+0000 7f7f4ac00640 1 Processor -- start 2024-09-10T10:46:13.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.550+0000 7f7f4ac00640 1 -- start start 2024-09-10T10:46:13.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f4ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f44156740 0x7f7f440b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f4ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7f440b4af0 0x7f7f440cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f4ac00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f440b30e0 con 0x7f7f440cfcd0 2024-09-10T10:46:13.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f4ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4406edc0 con 0x7f7f440b4af0 2024-09-10T10:46:13.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f4ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440b33c0 con 0x7f7f44156740 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f3be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7f440b4af0 0x7f7f440cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f48c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7f440cfcd0 0x7f7f4415fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50796/0 (socket says 172.21.15.28:50796) 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f48c00640 1 -- 172.21.15.28:0/2042800351 learned_addr learned my addr 172.21.15.28:0/2042800351 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f43e00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f44156740 0x7f7f440b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f3be00640 1 -- 172.21.15.28:0/2042800351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f44156740 msgr2=0x7f7f440b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f3be00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f44156740 0x7f7f440b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f3be00640 1 -- 172.21.15.28:0/2042800351 >> v1:172.21.15.28:6789/0 conn(0x7f7f440cfcd0 legacy=0x7f7f4415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.552+0000 7f7f3be00640 1 -- 172.21.15.28:0/2042800351 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f440b36a0 con 0x7f7f440b4af0 2024-09-10T10:46:13.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.551+0000 7f7f43400640 1 -- 172.21.15.28:0/2042800351 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111582925 0 0) 0x7f7f440b30e0 con 0x7f7f440cfcd0 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.552+0000 7f7f3be00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440b4af0 0x7f7f440cf790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097a0 tx=0x7f7f2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b71b7231fa8ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f43400640 1 -- 172.21.15.28:0/2042800351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c037070 con 0x7f7f440b4af0 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f43400640 1 -- 172.21.15.28:0/2042800351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2c032440 con 0x7f7f440b4af0 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f43400640 1 -- 172.21.15.28:0/2042800351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c032810 con 0x7f7f440b4af0 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2042800351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440b4af0 msgr2=0x7f7f440cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440b4af0 0x7f7f440cf790 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0097a0 tx=0x7f7f2c02a180 comp rx=0 tx=0).stop 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2042800351 shutdown_connections 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440b4af0 0x7f7f440cf790 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2042800351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f44156740 0x7f7f440b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2042800351 >> 172.21.15.28:0/2042800351 conn(0x7f7f4406bdf0 msgr2=0x7f7f4406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:13.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.553+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2042800351 shutdown_connections 2024-09-10T10:46:13.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.554+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2042800351 wait complete. 2024-09-10T10:46:13.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.555+0000 7f7f4ac00640 1 Processor -- start 2024-09-10T10:46:13.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.555+0000 7f7f4ac00640 1 -- start start 2024-09-10T10:46:13.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 0x7f7f44149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 0x7f7f440b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440d0890 con 0x7f7f440cfcd0 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f440b3c60 con 0x7f7f44156740 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.556+0000 7f7f4ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f440d0a10 con 0x7f7f440b4af0 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41396/0 (socket says 172.21.15.28:41396) 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 -- 172.21.15.28:0/2379937405 learned_addr learned my addr 172.21.15.28:0/2379937405 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f43e00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 0x7f7f44149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f48c00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 0x7f7f440b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 -- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 msgr2=0x7f7f44149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 0x7f7f44149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 -- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 msgr2=0x7f7f440b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 0x7f7f440b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 -- 172.21.15.28:0/2379937405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4414b860 con 0x7f7f440cfcd0 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f43e00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 0x7f7f44149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f48c00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 0x7f7f440b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:13.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.557+0000 7f7f3be00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c035040 tx=0x7f7f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.558+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c037050 con 0x7f7f440cfcd0 2024-09-10T10:46:13.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.558+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2c038040 con 0x7f7f440cfcd0 2024-09-10T10:46:13.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.558+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4414ab50 con 0x7f7f440cfcd0 2024-09-10T10:46:13.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.558+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c004490 con 0x7f7f440cfcd0 2024-09-10T10:46:13.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.558+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4414ae30 con 0x7f7f440cfcd0 2024-09-10T10:46:13.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.559+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f44149a70 con 0x7f7f440cfcd0 2024-09-10T10:46:13.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.560+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f2c03c6d0 con 0x7f7f440cfcd0 2024-09-10T10:46:13.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.561+0000 7f7f40c00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 0x7f7f2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.561+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f7f2c0aba30 con 0x7f7f440cfcd0 2024-09-10T10:46:13.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.561+0000 7f7f43e00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 0x7f7f2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.562+0000 7f7f43e00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 0x7f7f2006e320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009a20 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:13.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.565+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f2c07a1b0 con 0x7f7f440cfcd0 2024-09-10T10:46:13.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.815+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7f4415d230 con 0x7f7f440cfcd0 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.818+0000 7f7f40c00640 1 -- 172.21.15.28:0/2379937405 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f7f2c079ab0 con 0x7f7f440cfcd0 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:46:13.819 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:46:13.820 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:46:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 msgr2=0x7f7f2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 0x7f7f2006e320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009a20 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2024-09-10T10:46:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 msgr2=0x7f7f4414c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c035040 tx=0x7f7f2c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 shutdown_connections 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f2006be60 0x7f7f2006e320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f44156740 0x7f7f440b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f440cfcd0 0x7f7f4414c300 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 --2- 172.21.15.28:0/2379937405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f440b4af0 0x7f7f44149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 >> 172.21.15.28:0/2379937405 conn(0x7f7f4406bdf0 msgr2=0x7f7f440ad470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 shutdown_connections 2024-09-10T10:46:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.820+0000 7f7f4ac00640 1 -- 172.21.15.28:0/2379937405 wait complete. 2024-09-10T10:46:13.837 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.993+0000 7fa126000640 1 Processor -- start 2024-09-10T10:46:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.993+0000 7fa126000640 1 -- start start 2024-09-10T10:46:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa126000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1200d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa126000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1200cf000 con 0x7fa1200d1f90 2024-09-10T10:46:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa126000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1200d16f0 con 0x7fa12015e660 2024-09-10T10:46:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa126000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa12006d3e0 con 0x7fa1200d0dd0 2024-09-10T10:46:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa125600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa1200d1f90 0x7fa1200cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34764/0 (socket says 172.21.15.28:34764) 2024-09-10T10:46:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.994+0000 7fa125600640 1 -- 172.21.15.28:0/3060056790 learned_addr learned my addr 172.21.15.28:0/3060056790 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa124c00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1200d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114608886 0 0) 0x7fa1200cf000 con 0x7fa1200d1f90 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa108003520 con 0x7fa1200d1f90 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2616195801 0 0) 0x7fa108003520 con 0x7fa1200d1f90 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 msgr2=0x7fa1200d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1200d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 >> v1:172.21.15.138:6789/0 conn(0x7fa12015e660 legacy=0x7fa12006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1200d29a0 con 0x7fa1200d1f90 2024-09-10T10:46:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa1180019f0 con 0x7fa1200d1f90 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 >> v1:172.21.15.28:6789/0 conn(0x7fa1200d1f90 legacy=0x7fa1200cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa108003fb0 0x7fa108004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108004d40 0x7fa1080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa12006d3e0 con 0x7fa108003fb0 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa10800bb00 con 0x7fa108004d40 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.995+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa10800bd00 con 0x7fa1200d0dd0 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa108003fb0 0x7fa108004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa124c00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa125600640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108004d40 0x7fa1080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 -- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 msgr2=0x7fa1080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200d0dd0 0x7fa1080039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 -- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108004d40 msgr2=0x7fa1080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108004d40 0x7fa1080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa11fe00640 1 -- 172.21.15.28:0/3060056790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa108003520 con 0x7fa108003fb0 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.996+0000 7fa125600640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108004d40 0x7fa1080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.997+0000 7fa11fe00640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa108003fb0 0x7fa108004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1200b3980 tx=0x7fa108005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0524effb67d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.997+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c001370 con 0x7fa108003fb0 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.997+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c0052d0 con 0x7fa108003fb0 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.997+0000 7fa11f400640 1 -- 172.21.15.28:0/3060056790 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c02aa90 con 0x7fa108003fb0 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 -- 172.21.15.28:0/3060056790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa108003fb0 msgr2=0x7fa108004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 --2- 172.21.15.28:0/3060056790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa108003fb0 0x7fa108004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa1200b3980 tx=0x7fa108005830 comp rx=0 tx=0).stop 2024-09-10T10:46:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa125600640 1 -- 172.21.15.28:0/3060056790 reap_dead start 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 -- 172.21.15.28:0/3060056790 shutdown_connections 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 -- 172.21.15.28:0/3060056790 >> 172.21.15.28:0/3060056790 conn(0x7fa12008bfc0 msgr2=0x7fa1200d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 -- 172.21.15.28:0/3060056790 shutdown_connections 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:13.998+0000 7fa126000640 1 -- 172.21.15.28:0/3060056790 wait complete. 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.000+0000 7fa126000640 1 Processor -- start 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.000+0000 7fa126000640 1 -- start start 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108003fb0 0x7fa120155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200cb9d0 0x7fa1200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200cf330 con 0x7fa1200b4200 2024-09-10T10:46:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200d2f40 con 0x7fa108003fb0 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa126000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1200d3220 con 0x7fa1200cb9d0 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36086/0 (socket says 172.21.15.28:36086) 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 -- 172.21.15.28:0/974553588 learned_addr learned my addr 172.21.15.28:0/974553588 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa124c00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108003fb0 0x7fa120155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa125600640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200cb9d0 0x7fa1200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 -- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200cb9d0 msgr2=0x7fa1200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200cb9d0 0x7fa1200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 -- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108003fb0 msgr2=0x7fa120155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108003fb0 0x7fa120155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.001+0000 7fa11fe00640 1 -- 172.21.15.28:0/974553588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1200cbf50 con 0x7fa1200b4200 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa11fe00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa10c000c00 tx=0x7fa10c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c042070 con 0x7fa1200b4200 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c005480 con 0x7fa1200b4200 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c037040 con 0x7fa1200b4200 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1200cc230 con 0x7fa1200b4200 2024-09-10T10:46:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.002+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1200cc7f0 con 0x7fa1200b4200 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.003+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1200d2440 con 0x7fa1200b4200 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.004+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa10c045070 con 0x7fa1200b4200 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.005+0000 7fa11cc00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 0x7fa10006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.005+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fa10c0aa6c0 con 0x7fa1200b4200 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.005+0000 7fa124c00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 0x7fa10006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.006+0000 7fa124c00640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 0x7fa10006e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa114009bc0 tx=0x7fa114009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.009+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa10c078880 con 0x7fa1200b4200 2024-09-10T10:46:14.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.230+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1200d1b20 con 0x7fa1200b4200 2024-09-10T10:46:14.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.231+0000 7fa11cc00640 1 -- 172.21.15.28:0/974553588 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7fa10c07cb80 con 0x7fa1200b4200 2024-09-10T10:46:14.232 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:14.232 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":32,"btime":"2024-09-10T10:46:11:726443+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:11.149029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/20691839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":20691839},{"type":"v1","addr":"172.21.15.28:6835","nonce":20691839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":3}]} 2024-09-10T10:46:14.232 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 32 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 msgr2=0x7fa10006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 0x7fa10006e320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa114009bc0 tx=0x7fa114009190 comp rx=0 tx=0).stop 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 msgr2=0x7fa1200cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa10c000c00 tx=0x7fa10c001120 comp rx=0 tx=0).stop 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 shutdown_connections 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa10006be60 0x7fa10006e320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1200cb9d0 0x7fa1200ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1200b4200 0x7fa1200cb490 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 --2- 172.21.15.28:0/974553588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa108003fb0 0x7fa120155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 >> 172.21.15.28:0/974553588 conn(0x7fa12008bfc0 msgr2=0x7fa1200b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 shutdown_connections 2024-09-10T10:46:14.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.233+0000 7fa126000640 1 -- 172.21.15.28:0/974553588 wait complete. 2024-09-10T10:46:14.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:08.120814+0000', 'modified': '2024-09-10T10:46:11.149029+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5418}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5418': {'gid': 5418, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6835/20691839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6834', 'nonce': 20691839}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 20691839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5418, 'qdb_cluster': [5418]} 2024-09-10T10:46:14.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:46:14.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:46:14.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.810+0000 7f4fb7a00640 1 Processor -- start 2024-09-10T10:46:14.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.810+0000 7f4fb7a00640 1 -- start start 2024-09-10T10:46:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.811+0000 7f4fb7a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fb0072320 0x7f4fb00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.811+0000 7f4fb7a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4fb0072de0 0x7f4fb006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb7a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fb0146140 con 0x7f4fb0072700 2024-09-10T10:46:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb7a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00cd740 con 0x7f4fb0072320 2024-09-10T10:46:14.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb7a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4fb00cda20 con 0x7f4fb0072de0 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fb0072320 0x7f4fb00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fb0072320 0x7f4fb00cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36090/0 (socket says 172.21.15.28:36090) 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 -- 172.21.15.28:0/4283883723 learned_addr learned my addr 172.21.15.28:0/4283883723 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fafe00640 1 --2- 172.21.15.28:0/4283883723 >> v2:172.21.15.138:3300/0 conn(0x7f4fb0072de0 0x7f4fb006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 -- 172.21.15.28:0/4283883723 >> v2:172.21.15.138:3300/0 conn(0x7f4fb0072de0 msgr2=0x7f4fb006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 --2- 172.21.15.28:0/4283883723 >> v2:172.21.15.138:3300/0 conn(0x7f4fb0072de0 0x7f4fb006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 -- 172.21.15.28:0/4283883723 >> v1:172.21.15.138:6789/0 conn(0x7f4fb0072700 legacy=0x7f4fb006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.812+0000 7f4fb5000640 1 -- 172.21.15.28:0/4283883723 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb00cdd00 con 0x7f4fb0072320 2024-09-10T10:46:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.813+0000 7f4fb5000640 1 --2- 172.21.15.28:0/4283883723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072320 0x7f4fb00cce60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4fb006f8c0 tx=0x7f4fa402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4701799047a3230e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.813+0000 7f4faf400640 1 -- 172.21.15.28:0/4283883723 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4037070 con 0x7f4fb0072320 2024-09-10T10:46:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.813+0000 7f4faf400640 1 -- 172.21.15.28:0/4283883723 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa4032440 con 0x7f4fb0072320 2024-09-10T10:46:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.813+0000 7f4faf400640 1 -- 172.21.15.28:0/4283883723 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4032810 con 0x7f4fb0072320 2024-09-10T10:46:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 -- 172.21.15.28:0/4283883723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072320 msgr2=0x7f4fb00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/4283883723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072320 0x7f4fb00cce60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4fb006f8c0 tx=0x7f4fa402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 -- 172.21.15.28:0/4283883723 shutdown_connections 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/4283883723 >> v2:172.21.15.138:3300/0 conn(0x7f4fb0072de0 0x7f4fb006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/4283883723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072320 0x7f4fb00cce60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 -- 172.21.15.28:0/4283883723 >> 172.21.15.28:0/4283883723 conn(0x7f4fb0067d70 msgr2=0x7f4fb006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 -- 172.21.15.28:0/4283883723 shutdown_connections 2024-09-10T10:46:14.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.814+0000 7f4fb7a00640 1 -- 172.21.15.28:0/4283883723 wait complete. 2024-09-10T10:46:14.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.816+0000 7f4fb7a00640 1 Processor -- start 2024-09-10T10:46:14.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.816+0000 7f4fb7a00640 1 -- start start 2024-09-10T10:46:14.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fb0072320 0x7f4fb00a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fb0072de0 0x7f4fb01e6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00cd520 con 0x7f4fb0072700 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00cd3a0 con 0x7f4fb0072de0 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb006ed00 con 0x7f4fb0072320 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36098/0 (socket says 172.21.15.28:36098) 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 -- 172.21.15.28:0/3594433046 learned_addr learned my addr 172.21.15.28:0/3594433046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fb0072320 0x7f4fb00a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fb0072de0 0x7f4fb01e6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 -- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fb0072320 msgr2=0x7f4fb00a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fb0072320 0x7f4fb00a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 -- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fb0072de0 msgr2=0x7f4fb01e6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fb0072de0 0x7f4fb01e6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.817+0000 7f4fafe00640 1 -- 172.21.15.28:0/3594433046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01e7770 con 0x7f4fb0072700 2024-09-10T10:46:14.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4fafe00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d620 tx=0x7f4f9c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4facc00640 1 -- 172.21.15.28:0/3594433046 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0125f0 con 0x7f4fb0072700 2024-09-10T10:46:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4facc00640 1 -- 172.21.15.28:0/3594433046 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c019070 con 0x7f4fb0072700 2024-09-10T10:46:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4facc00640 1 -- 172.21.15.28:0/3594433046 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0129d0 con 0x7f4fb0072700 2024-09-10T10:46:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01e7a50 con 0x7f4fb0072700 2024-09-10T10:46:14.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.818+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01e8010 con 0x7f4fb0072700 2024-09-10T10:46:14.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.821+0000 7f4facc00640 1 -- 172.21.15.28:0/3594433046 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f9c012b70 con 0x7f4fb0072700 2024-09-10T10:46:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.822+0000 7f4facc00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 0x7f4f9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.822+0000 7f4facc00640 1 -- 172.21.15.28:0/3594433046 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f4f9c08b320 con 0x7f4fb0072700 2024-09-10T10:46:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.823+0000 7f4fba840740 1 Processor -- start 2024-09-10T10:46:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.823+0000 7f4fba840740 1 -- start start 2024-09-10T10:46:14.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.823+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 0x7f4f9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 0x7f4f9006e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4034000 tx=0x7f4fa402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d644b0 0x55c664d670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664d61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c664cc53b0 con 0x55c664d66cf0 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c664cca410 con 0x55c664d644b0 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c664c72620 con 0x55c664d68480 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fb5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fb5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36106/0 (socket says 172.21.15.28:36106) 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fb5000640 1 -- 172.21.15.28:0/3499823632 learned_addr learned my addr 172.21.15.28:0/3499823632 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fafe00640 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d644b0 0x55c664d670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.824+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664d61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 -- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 msgr2=0x55c664d61570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664d61570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 -- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d644b0 msgr2=0x55c664d670d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d644b0 0x55c664d670d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 -- 172.21.15.28:0/3499823632 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6649dd2d0 con 0x55c664d66cf0 2024-09-10T10:46:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009960 tx=0x7f4fa402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d5f81c8bd2a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fa1600640 1 -- 172.21.15.28:0/3499823632 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4037040 con 0x55c664d66cf0 2024-09-10T10:46:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fa1600640 1 -- 172.21.15.28:0/3499823632 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa404d070 con 0x55c664d66cf0 2024-09-10T10:46:14.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.825+0000 7f4fa1600640 1 -- 172.21.15.28:0/3499823632 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa403c440 con 0x55c664d66cf0 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 -- 172.21.15.28:0/3499823632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 msgr2=0x55c664dac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4009960 tx=0x7f4fa402c000 comp rx=0 tx=0).stop 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 -- 172.21.15.28:0/3499823632 shutdown_connections 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664d61570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d644b0 0x55c664d670d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 --2- 172.21.15.28:0/3499823632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dac490 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 -- 172.21.15.28:0/3499823632 >> 172.21.15.28:0/3499823632 conn(0x55c664d63740 msgr2=0x55c664dab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 -- 172.21.15.28:0/3499823632 shutdown_connections 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.826+0000 7f4fba840740 1 -- 172.21.15.28:0/3499823632 wait complete. 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.827+0000 7f4fba840740 1 -- start start 2024-09-10T10:46:14.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.827+0000 7f4fba840740 1 Processor -- start 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664dc0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664dbf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c664c85780 con 0x55c664d8faa0 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c664cc63d0 con 0x55c664d66cf0 2024-09-10T10:46:14.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c664d62be0 con 0x55c664d68480 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.828+0000 7f4fba840740 10 client.0 _ll_register_callbacks cb 0x55c664cc35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36114/0 (socket says 172.21.15.28:36114) 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 -- 172.21.15.28:0/3279418383 learned_addr learned my addr 172.21.15.28:0/3279418383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664dc0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fafe00640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664dbf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 -- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 msgr2=0x55c664dbf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664dbf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 -- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 msgr2=0x55c664dc0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664dc0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 -- 172.21.15.28:0/3279418383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c664aa76b0 con 0x55c664d8faa0 2024-09-10T10:46:14.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664dc0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8011690 tx=0x7f4fa8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee0f8a6ec47b360 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.829+0000 7f4f96000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa801b020 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fa404d070 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f4fa8017690 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fa8025050 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 10 client.5544 fetch_fsmap learned FSMap version 32 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55c664d8f2a0 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fa404d070 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7f4fa8028020 con 0x55c664d8faa0 2024-09-10T10:46:14.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 10 client.5544 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 10 client.5544 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 20 client.5544 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664dcc160 0x55c664dce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 4 client.5544 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.830+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 --> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c664cd7b40 con 0x55c664dcc160 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.831+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664dcc160 0x55c664dce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.831+0000 7f4fb5000640 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664dcc160 0x55c664dce550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b89ebf2099b67da server_cookie=8e3d719c8ee84a2f in_seq=0 out_seq=0 2024-09-10T10:46:14.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.831+0000 7f4f96000640 10 client.5544 ms_handle_connect on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.832+0000 7f4f96000640 1 -- 172.21.15.28:0/3279418383 <== mds.0 v2:172.21.15.28:6834/20691839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c664cd7b40 con 0x55c664dcc160 2024-09-10T10:46:14.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.832+0000 7f4f96000640 10 client.5544 handle_command_reply: tid=0 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.832+0000 7f4fba840740 1 client.5544 shutdown 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 msgr2=0x55c664d8ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8011690 tx=0x7f4fa8011a90 comp rx=0 tx=0).stop 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 shutdown_connections 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d8faa0 0x55c664d8ff20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664dcc160 0x55c664dce550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664dbf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 --2- 172.21.15.28:0/3279418383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664dc0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4f96000640 0 client.5544 ms_handle_reset on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 >> 172.21.15.28:0/3279418383 conn(0x55c664d644b0 msgr2=0x55c664daaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 shutdown_connections 2024-09-10T10:46:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 1 -- 172.21.15.28:0/3279418383 wait complete. 2024-09-10T10:46:14.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.833+0000 7f4fba840740 20 client.5544 trim_cache size 0 max 16384 2024-09-10T10:46:14.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.852+0000 7f4fba840740 1 Processor -- start 2024-09-10T10:46:14.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.852+0000 7f4fba840740 1 -- start start 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664da8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d8faa0 0x55c664d8dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb01fbd60 con 0x55c664d68480 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb00b50f0 con 0x55c664d66cf0 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4fb01fb8d0 con 0x55c664d8faa0 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36122/0 (socket says 172.21.15.28:36122) 2024-09-10T10:46:14.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 -- 172.21.15.28:0/2658766286 learned_addr learned my addr 172.21.15.28:0/2658766286 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 -- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d8faa0 msgr2=0x55c664d8dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d8faa0 0x55c664d8dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 -- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 msgr2=0x55c664da8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664da8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.853+0000 7f4fafe00640 1 -- 172.21.15.28:0/2658766286 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c664d8aa00 con 0x55c664d68480 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fafe00640 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c089650 tx=0x7f4f9c08b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30aee3559120767 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fa0c00640 1 -- 172.21.15.28:0/2658766286 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c089730 con 0x55c664d68480 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fa0c00640 1 -- 172.21.15.28:0/2658766286 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c0577e0 con 0x55c664d68480 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fa0c00640 1 -- 172.21.15.28:0/2658766286 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c0711d0 con 0x55c664d68480 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 -- 172.21.15.28:0/2658766286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 msgr2=0x55c664d8d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c089650 tx=0x7f4f9c08b040 comp rx=0 tx=0).stop 2024-09-10T10:46:14.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 -- 172.21.15.28:0/2658766286 shutdown_connections 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d8faa0 0x55c664d8dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d68480 0x55c664d8d730 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 --2- 172.21.15.28:0/2658766286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d66cf0 0x55c664da8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.854+0000 7f4fba840740 1 -- 172.21.15.28:0/2658766286 >> 172.21.15.28:0/2658766286 conn(0x55c664d644b0 msgr2=0x55c664daaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.855+0000 7f4fba840740 1 -- 172.21.15.28:0/2658766286 shutdown_connections 2024-09-10T10:46:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.855+0000 7f4fba840740 1 -- 172.21.15.28:0/2658766286 wait complete. 2024-09-10T10:46:14.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.855+0000 7f4fba840740 1 -- start start 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.855+0000 7f4fba840740 1 Processor -- start 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664e2e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d8faa0 0x55c664d8d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c664d8a4d0 con 0x55c664d66cf0 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c664dbfe70 con 0x55c664d8faa0 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c664d91530 con 0x55c664d68480 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fba840740 10 client.0 _ll_register_callbacks cb 0x7f4fa8005150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.856+0000 7f4fb5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36130/0 (socket says 172.21.15.28:36130) 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 -- 172.21.15.28:0/4048188368 learned_addr learned my addr 172.21.15.28:0/4048188368 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fafe00640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664e2e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5a00640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d8faa0 0x55c664d8d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 -- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 msgr2=0x55c664e2e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664e2e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 -- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d8faa0 msgr2=0x55c664d8d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d8faa0 0x55c664d8d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 -- 172.21.15.28:0/4048188368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c664d61e60 con 0x55c664d66cf0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fafe00640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664e2e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fb5000640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40090b0 tx=0x7f4fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c21287f9953adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4f96000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa403c8d0 con 0x55c664d66cf0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f4fa403ca70 con 0x55c664d66cf0 2024-09-10T10:46:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.857+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c0577e0 con 0x55c664d66cf0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4fa403cc70 con 0x55c664d66cf0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 10 client.5550 fetch_fsmap learned FSMap version 32 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x55c664d8ed90 con 0x55c664d66cf0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f9c0577e0 con 0x55c664d66cf0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7f4fa405a300 con 0x55c664d66cf0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 10 client.5550 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 10 client.5550 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 20 client.5550 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664d88ce0 0x55c664d890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 4 client.5550 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 --> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] -- command(tid 0: {"prefix": "status"}) -- 0x55c664cd1e90 con 0x55c664d88ce0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.858+0000 7f4fafe00640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664d88ce0 0x55c664d890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fafe00640 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664d88ce0 0x55c664d890c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a05c21d18217b7 server_cookie=bf4701bf206e5140 in_seq=0 out_seq=0 2024-09-10T10:46:14.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4f96000640 10 client.5550 ms_handle_connect on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4f96000640 1 -- 172.21.15.28:0/4048188368 <== mds.0 v2:172.21.15.28:6834/20691839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55c664cd1e90 con 0x55c664d88ce0 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4f96000640 10 client.5550 handle_command_reply: tid=0 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 client.5550 shutdown 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 msgr2=0x55c664dc5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40090b0 tx=0x7f4fa40046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 shutdown_connections 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c664d8faa0 0x55c664d8d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x55c664d88ce0 0x55c664d890c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c664d68480 0x55c664e2e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 --2- 172.21.15.28:0/4048188368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c664d66cf0 0x55c664dc5370 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 >> 172.21.15.28:0/4048188368 conn(0x55c664d644b0 msgr2=0x55c664daae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4f96000640 0 client.5550 ms_handle_reset on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 shutdown_connections 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.859+0000 7f4fba840740 1 -- 172.21.15.28:0/4048188368 wait complete. 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fba840740 20 client.5550 trim_cache size 0 max 16384 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5418, 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:46:14.865 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.1462631280000002, 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 32, 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 56, 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 56, 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 10.834078464999999 2024-09-10T10:46:14.866 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 msgr2=0x7f4f9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 0x7f4f9006e2f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4fa4034000 tx=0x7f4fa402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 msgr2=0x7f4fb01e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d620 tx=0x7f4f9c00dae0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 shutdown_connections 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f9006be30 0x7f4f9006e2f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fb0072de0 0x7f4fb01e6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fb0072700 0x7f4fb01e69e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 --2- 172.21.15.28:0/3594433046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fb0072320 0x7f4fb00a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:14.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 >> 172.21.15.28:0/3594433046 conn(0x7f4fb0067d70 msgr2=0x7f4fb00ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:14.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 shutdown_connections 2024-09-10T10:46:14.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:14.860+0000 7f4fb7a00640 1 -- 172.21.15.28:0/3594433046 wait complete. 2024-09-10T10:46:14.876 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:46:15.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.032+0000 7fe090000640 1 Processor -- start 2024-09-10T10:46:15.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.032+0000 7fe090000640 1 -- start start 2024-09-10T10:46:15.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe090000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe08815e760 0x7fe08806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe090000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe0880cfcc0 0x7fe0880d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe090000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0880d2700 con 0x7fe0880d4fd0 2024-09-10T10:46:15.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe090000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe0880cedb0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe090000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe0880cf090 con 0x7fe08815e760 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe08e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe0880cfcc0 0x7fe0880d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe08e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe0880cfcc0 0x7fe0880d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36146/0 (socket says 172.21.15.28:36146) 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.033+0000 7fe08e000640 1 -- 172.21.15.28:0/2091155040 learned_addr learned my addr 172.21.15.28:0/2091155040 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.034+0000 7fe08d600640 1 --2- 172.21.15.28:0/2091155040 >> v2:172.21.15.138:3300/0 conn(0x7fe08815e760 0x7fe08806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.034+0000 7fe08e000640 1 -- 172.21.15.28:0/2091155040 >> v2:172.21.15.138:3300/0 conn(0x7fe08815e760 msgr2=0x7fe08806cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:15.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.034+0000 7fe08e000640 1 --2- 172.21.15.28:0/2091155040 >> v2:172.21.15.138:3300/0 conn(0x7fe08815e760 0x7fe08806cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.034+0000 7fe08e000640 1 -- 172.21.15.28:0/2091155040 >> v1:172.21.15.138:6789/0 conn(0x7fe0880d4fd0 legacy=0x7fe0880cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.034+0000 7fe08e000640 1 -- 172.21.15.28:0/2091155040 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0880b3c80 con 0x7fe0880cfcc0 2024-09-10T10:46:15.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.035+0000 7fe08e000640 1 --2- 172.21.15.28:0/2091155040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe0880d2ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe08808ca50 tx=0x7fe08002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0b25aa2380ec52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:15.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.035+0000 7fe087e00640 1 -- 172.21.15.28:0/2091155040 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe080036070 con 0x7fe0880cfcc0 2024-09-10T10:46:15.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.035+0000 7fe087e00640 1 -- 172.21.15.28:0/2091155040 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe080031600 con 0x7fe0880cfcc0 2024-09-10T10:46:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.035+0000 7fe087e00640 1 -- 172.21.15.28:0/2091155040 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800319d0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 -- 172.21.15.28:0/2091155040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 msgr2=0x7fe0880d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 --2- 172.21.15.28:0/2091155040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe0880d2ea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe08808ca50 tx=0x7fe08002a730 comp rx=0 tx=0).stop 2024-09-10T10:46:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 -- 172.21.15.28:0/2091155040 shutdown_connections 2024-09-10T10:46:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 --2- 172.21.15.28:0/2091155040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe0880d2ea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 --2- 172.21.15.28:0/2091155040 >> v2:172.21.15.138:3300/0 conn(0x7fe08815e760 0x7fe08806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 -- 172.21.15.28:0/2091155040 >> 172.21.15.28:0/2091155040 conn(0x7fe08808bfc0 msgr2=0x7fe0880d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 -- 172.21.15.28:0/2091155040 shutdown_connections 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.036+0000 7fe090000640 1 -- 172.21.15.28:0/2091155040 wait complete. 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.037+0000 7fe090000640 1 Processor -- start 2024-09-10T10:46:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 -- start start 2024-09-10T10:46:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 0x7fe0880b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815e760 0x7fe08815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0880d33e0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0880b38e0 con 0x7fe0880d4fd0 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.038+0000 7fe090000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0880d3560 con 0x7fe08815e760 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36160/0 (socket says 172.21.15.28:36160) 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 -- 172.21.15.28:0/2282844939 learned_addr learned my addr 172.21.15.28:0/2282844939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08cc00640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 0x7fe0880b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08e000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815e760 0x7fe08815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 -- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815e760 msgr2=0x7fe08815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815e760 0x7fe08815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 -- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 msgr2=0x7fe0880b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 0x7fe0880b9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 -- 172.21.15.28:0/2282844939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08815d110 con 0x7fe0880cfcc0 2024-09-10T10:46:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08cc00640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 0x7fe0880b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.039+0000 7fe08d600640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e870 tx=0x7fe07c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.040+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017070 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.040+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07c00c940 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.040+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00cd30 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.040+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0880ba2a0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.040+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe088161020 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.041+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0880d3890 con 0x7fe0880cfcc0 2024-09-10T10:46:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.042+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe07c01a070 con 0x7fe0880cfcc0 2024-09-10T10:46:15.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.043+0000 7fe085600640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 0x7fe06806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:15.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.043+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fe07c08ae90 con 0x7fe0880cfcc0 2024-09-10T10:46:15.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.043+0000 7fe08cc00640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 0x7fe06806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.044+0000 7fe08cc00640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 0x7fe06806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe0881513f0 tx=0x7fe074009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:15.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.047+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe07c058fc0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.267+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0881e99e0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.267+0000 7fe085600640 1 -- 172.21.15.28:0/2282844939 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7fe07c05d2c0 con 0x7fe0880cfcc0 2024-09-10T10:46:15.269 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:46:15.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 msgr2=0x7fe06806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 0x7fe06806e2f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe0881513f0 tx=0x7fe074009040 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 msgr2=0x7fe088150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e870 tx=0x7fe07c00ed30 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 shutdown_connections 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe06806be30 0x7fe06806e2f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe08815e760 0x7fe08815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0880d4fd0 0x7fe0880b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 --2- 172.21.15.28:0/2282844939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0880cfcc0 0x7fe088150890 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 >> 172.21.15.28:0/2282844939 conn(0x7fe08808bfc0 msgr2=0x7fe0880b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 shutdown_connections 2024-09-10T10:46:15.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15.269+0000 7fe090000640 1 -- 172.21.15.28:0/2282844939 wait complete. 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:15.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:46:15.288 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:46:15.288 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:46:15.288 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:15.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.288 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:46:15.323 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:46:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:46:15.323 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:15.324 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:46:15.325 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:15.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.326 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:46:15.418 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:46:15.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.421 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:46:15.457 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:46:15.457 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:46:15.457 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:46:15.457 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:46:15.457 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:46:15.457 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:46:15.458 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:46:15.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:46:15.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:46:15.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:46:15.458 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:46:15.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:46:15.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:15.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.653 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:46:15.689 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:46:15.689 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:46:15.689 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:15.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:46:15.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:15.786 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:15.822 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:15.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:15.822 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:15.897 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:46:15.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:15.897 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:16.026 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:16.026 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:16.026 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:16.042 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:16.042 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:16.042 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:46:16.064 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:46:16.065 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:46:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:46:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:46:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:46:16.066 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:46:16.069 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:46:16.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:16.069 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:16.125 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:46:16.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:16.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:16.220 INFO:teuthology.orchestra.run.smithi028.stdout:client5338 2024-09-10T10:46:16.220 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:16.220 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:16.255 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/b') 2024-09-10T10:46:16.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:16.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:16.394 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-10T10:46:16.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.550+0000 7ff333000640 1 Processor -- start 2024-09-10T10:46:16.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.550+0000 7ff333000640 1 -- start start 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.551+0000 7ff333000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c0b3560 0x7ff32c0d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.551+0000 7ff333000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff32c0d08b0 0x7ff32c06e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.551+0000 7ff333000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff32c0b39a0 con 0x7ff32c15e5e0 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff333000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff32c15eda0 con 0x7ff32c0d08b0 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff333000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c15f080 con 0x7ff32c0b3560 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff32be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff32c15e5e0 0x7ff32c06f5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34780/0 (socket says 172.21.15.28:34780) 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff32be00640 1 -- 172.21.15.28:0/2094124022 learned_addr learned my addr 172.21.15.28:0/2094124022 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff331000640 1 --2- 172.21.15.28:0/2094124022 >> v2:172.21.15.28:3300/0 conn(0x7ff32c0d08b0 0x7ff32c06e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff32b400640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c0b3560 0x7ff32c0d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff32aa00640 1 -- 172.21.15.28:0/2094124022 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2928544603 0 0) 0x7ff32c0b39a0 con 0x7ff32c15e5e0 2024-09-10T10:46:16.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff331000640 1 -- 172.21.15.28:0/2094124022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c0b3560 msgr2=0x7ff32c0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff331000640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c0b3560 0x7ff32c0d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff331000640 1 -- 172.21.15.28:0/2094124022 >> v1:172.21.15.28:6789/0 conn(0x7ff32c15e5e0 legacy=0x7ff32c06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.552+0000 7ff331000640 1 -- 172.21.15.28:0/2094124022 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c15f360 con 0x7ff32c0d08b0 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.553+0000 7ff331000640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c06e1f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff324009dc0 tx=0x7ff32402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a16566ab8af8168 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff32aa00640 1 -- 172.21.15.28:0/2094124022 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32402ae60 con 0x7ff32c0d08b0 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff32aa00640 1 -- 172.21.15.28:0/2094124022 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff324007740 con 0x7ff32c0d08b0 2024-09-10T10:46:16.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff32aa00640 1 -- 172.21.15.28:0/2094124022 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324007b10 con 0x7ff32c0d08b0 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 -- 172.21.15.28:0/2094124022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 msgr2=0x7ff32c06e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c06e1f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff324009dc0 tx=0x7ff32402a680 comp rx=0 tx=0).stop 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 -- 172.21.15.28:0/2094124022 shutdown_connections 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c06e1f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 --2- 172.21.15.28:0/2094124022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c0b3560 0x7ff32c0d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 -- 172.21.15.28:0/2094124022 >> 172.21.15.28:0/2094124022 conn(0x7ff32c06bdf0 msgr2=0x7ff32c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 -- 172.21.15.28:0/2094124022 shutdown_connections 2024-09-10T10:46:16.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.554+0000 7ff333000640 1 -- 172.21.15.28:0/2094124022 wait complete. 2024-09-10T10:46:16.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.556+0000 7ff333000640 1 Processor -- start 2024-09-10T10:46:16.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.556+0000 7ff333000640 1 -- start start 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c0b3560 0x7ff32c0c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c15e5e0 0x7ff32c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c06fd00 con 0x7ff32c0d08b0 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c15f920 con 0x7ff32c0b3560 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff333000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c15e9c0 con 0x7ff32c15e5e0 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36182/0 (socket says 172.21.15.28:36182) 2024-09-10T10:46:16.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff32b400640 1 -- 172.21.15.28:0/1773900782 learned_addr learned my addr 172.21.15.28:0/1773900782 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff32be00640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c0b3560 0x7ff32c0c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.557+0000 7ff331000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c15e5e0 0x7ff32c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 -- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c15e5e0 msgr2=0x7ff32c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c15e5e0 0x7ff32c0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 -- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c0b3560 msgr2=0x7ff32c0c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c0b3560 0x7ff32c0c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 -- 172.21.15.28:0/1773900782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32c0b2cc0 con 0x7ff32c0d08b0 2024-09-10T10:46:16.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff32b400640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e870 tx=0x7ff31800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff323e00640 1 -- 172.21.15.28:0/1773900782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318017070 con 0x7ff32c0d08b0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff323e00640 1 -- 172.21.15.28:0/1773900782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31800c940 con 0x7ff32c0d08b0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32c0c9bd0 con 0x7ff32c0d08b0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.558+0000 7ff323e00640 1 -- 172.21.15.28:0/1773900782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31800cd30 con 0x7ff32c0d08b0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.559+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff32c0ca190 con 0x7ff32c0d08b0 2024-09-10T10:46:16.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.561+0000 7ff323e00640 1 -- 172.21.15.28:0/1773900782 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff31801a070 con 0x7ff32c0d08b0 2024-09-10T10:46:16.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.561+0000 7ff323e00640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.561+0000 7ff323e00640 1 -- 172.21.15.28:0/1773900782 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7ff31808ae90 con 0x7ff32c0d08b0 2024-09-10T10:46:16.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.562+0000 7ff32be00640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.562+0000 7ff335e33740 1 Processor -- start 2024-09-10T10:46:16.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.562+0000 7ff335e33740 1 -- start start 2024-09-10T10:46:16.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff32be00640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff31c000f60 tx=0x7ff31c007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1423c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13d9480 0x5607e1424740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607e1341660 con 0x5607e13f3800 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607e12e9680 con 0x5607e13d9480 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.563+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5607e133d620 con 0x5607e13f3420 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36194/0 (socket says 172.21.15.28:36194) 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1423c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 -- 172.21.15.28:0/2325396822 learned_addr learned my addr 172.21.15.28:0/2325396822 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff331000640 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13d9480 0x5607e1424740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 -- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 msgr2=0x5607e1423c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1423c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 -- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13d9480 msgr2=0x5607e1424740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13d9480 0x5607e1424740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 -- 172.21.15.28:0/2325396822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e111e7d0 con 0x5607e13f3800 2024-09-10T10:46:16.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff32b400640 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3180615c0 tx=0x7ff318058c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6846412617a6fa30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.564+0000 7ff320c00640 1 -- 172.21.15.28:0/2325396822 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318087070 con 0x5607e13f3800 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff320c00640 1 -- 172.21.15.28:0/2325396822 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff318061af0 con 0x5607e13f3800 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff320c00640 1 -- 172.21.15.28:0/2325396822 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318021040 con 0x5607e13f3800 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 -- 172.21.15.28:0/2325396822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 msgr2=0x5607e1424200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff3180615c0 tx=0x7ff318058c30 comp rx=0 tx=0).stop 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 -- 172.21.15.28:0/2325396822 shutdown_connections 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13d9480 0x5607e1424740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3800 0x5607e1424200 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 --2- 172.21.15.28:0/2325396822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1423c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 -- 172.21.15.28:0/2325396822 >> 172.21.15.28:0/2325396822 conn(0x5607e13da220 msgr2=0x5607e1422840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 -- 172.21.15.28:0/2325396822 shutdown_connections 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.565+0000 7ff335e33740 1 -- 172.21.15.28:0/2325396822 wait complete. 2024-09-10T10:46:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.566+0000 7ff335e33740 1 -- start start 2024-09-10T10:46:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.566+0000 7ff335e33740 1 Processor -- start 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e141fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e13dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607e13d83e0 con 0x5607e13e0300 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607e12ca4c0 con 0x5607e13f3800 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5607e13d86c0 con 0x5607e13f3420 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.567+0000 7ff335e33740 10 client.0 _ll_register_callbacks cb 0x5607e1339e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36196/0 (socket says 172.21.15.28:36196) 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 -- 172.21.15.28:0/3402766223 learned_addr learned my addr 172.21.15.28:0/3402766223 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff32be00640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e141fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff32b400640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e13dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 -- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 msgr2=0x5607e141fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e141fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 -- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 msgr2=0x5607e13dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e13dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 -- 172.21.15.28:0/3402766223 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e10543f0 con 0x5607e13e0300 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff331000640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff324009eb0 tx=0x7ff32402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d206018fc89f859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.568+0000 7ff315600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324009a30 con 0x5607e13e0300 2024-09-10T10:46:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7ff324048c80 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff318061af0 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff3240045b0 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 10 client.5571 fetch_fsmap learned FSMap version 32 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5607e10140f0 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff318061af0 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7ff324007540 con 0x5607e13e0300 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 10 client.5571 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 10 client.5571 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 20 client.5571 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e1436b70 0x5607e141db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 4 client.5571 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.569+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 --> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e134ed90 con 0x5607e1436b70 2024-09-10T10:46:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.570+0000 7ff32be00640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e1436b70 0x5607e141db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.570+0000 7ff32be00640 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e1436b70 0x5607e141db40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6097f0a5ac494fc server_cookie=bdb2c3d14fe39ed5 in_seq=0 out_seq=0 2024-09-10T10:46:16.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.570+0000 7ff315600640 10 client.5571 ms_handle_connect on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff315600640 1 -- 172.21.15.28:0/3402766223 <== mds.0 v2:172.21.15.28:6834/20691839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607e134ed90 con 0x5607e1436b70 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff315600640 10 client.5571 handle_command_reply: tid=0 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 client.5571 shutdown 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 msgr2=0x5607e1436160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff324009eb0 tx=0x7ff32402c000 comp rx=0 tx=0).stop 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 shutdown_connections 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1436160 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e13dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e1436b70 0x5607e141db40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 --2- 172.21.15.28:0/3402766223 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e141fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 >> 172.21.15.28:0/3402766223 conn(0x5607e13d9480 msgr2=0x5607e141c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff315600640 0 client.5571 ms_handle_reset on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 shutdown_connections 2024-09-10T10:46:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.571+0000 7ff335e33740 1 -- 172.21.15.28:0/3402766223 wait complete. 2024-09-10T10:46:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.572+0000 7ff335e33740 20 client.5571 trim_cache size 0 max 16384 2024-09-10T10:46:16.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.587+0000 7ff335e33740 1 Processor -- start 2024-09-10T10:46:16.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.587+0000 7ff335e33740 1 -- start start 2024-09-10T10:46:16.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 0x5607e1446f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c200f80 con 0x5607e13f3420 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1fffe0 con 0x5607e13e0300 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff32c0cbc60 con 0x5607e13f3800 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36210/0 (socket says 172.21.15.28:36210) 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 -- 172.21.15.28:0/2384448882 learned_addr learned my addr 172.21.15.28:0/2384448882 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff331000640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32be00640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 0x5607e1446f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 -- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 msgr2=0x5607e1423b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 -- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 msgr2=0x5607e1446f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 0x5607e1446f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32b400640 1 -- 172.21.15.28:0/2384448882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e13e0130 con 0x5607e13f3420 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff331000640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.588+0000 7ff32be00640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 0x5607e1446f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff32b400640 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff318070040 tx=0x7ff31801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba04cbe239d1f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff317e00640 1 -- 172.21.15.28:0/2384448882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318072030 con 0x5607e13f3420 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff317e00640 1 -- 172.21.15.28:0/2384448882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff318057060 con 0x5607e13f3420 2024-09-10T10:46:16.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff317e00640 1 -- 172.21.15.28:0/2384448882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318087070 con 0x5607e13f3420 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 -- 172.21.15.28:0/2384448882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 msgr2=0x5607e149af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff318070040 tx=0x7ff31801e000 comp rx=0 tx=0).stop 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 -- 172.21.15.28:0/2384448882 shutdown_connections 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13f3420 0x5607e149af00 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 --2- 172.21.15.28:0/2384448882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13e0300 0x5607e1446f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 -- 172.21.15.28:0/2384448882 >> 172.21.15.28:0/2384448882 conn(0x5607e13d9480 msgr2=0x5607e13f1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 -- 172.21.15.28:0/2384448882 shutdown_connections 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.589+0000 7ff335e33740 1 -- 172.21.15.28:0/2384448882 wait complete. 2024-09-10T10:46:16.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.590+0000 7ff335e33740 1 -- start start 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.590+0000 7ff335e33740 1 Processor -- start 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1489cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5607e1425270 con 0x5607e13e0300 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff32c1ff990 con 0x5607e13f3800 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5607e13d7dd0 con 0x5607e13f3420 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff335e33740 10 client.0 _ll_register_callbacks cb 0x7ff324003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:16.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff32be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.591+0000 7ff32be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36212/0 (socket says 172.21.15.28:36212) 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 -- 172.21.15.28:0/3069909957 learned_addr learned my addr 172.21.15.28:0/3069909957 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32b400640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1489cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff331000640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 -- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 msgr2=0x5607e1489cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1489cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 -- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 msgr2=0x5607e1423b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 -- 172.21.15.28:0/3069909957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e1426870 con 0x5607e13e0300 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff331000640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff32be00640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0011e0 tx=0x7ff31c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c59dc884dd4081a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.592+0000 7ff315600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0223f0 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff318057060 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7ff31c010040 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff31c037050 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 10 client.5577 fetch_fsmap learned FSMap version 32 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5607e1426bb0 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff318057060 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 32) ==== 3502+0+0 (secure 0 0 0) 0x7ff31c03a020 con 0x5607e13e0300 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 10 client.5577 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 10 client.5577 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 20 client.5577 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e149a5f0 0x5607e14240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 4 client.5577 mds_command: new command op to 5418 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:16.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.593+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 --> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607e13490e0 con 0x5607e149a5f0 2024-09-10T10:46:16.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.594+0000 7ff32b400640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e149a5f0 0x5607e14240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.594+0000 7ff32b400640 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e149a5f0 0x5607e14240d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abb7a6609d942d server_cookie=f5b33752bb214528 in_seq=0 out_seq=0 2024-09-10T10:46:16.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.594+0000 7ff315600640 10 client.5577 ms_handle_connect on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff315600640 1 -- 172.21.15.28:0/3069909957 <== mds.0 v2:172.21.15.28:6834/20691839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607e13490e0 con 0x5607e149a5f0 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff315600640 10 client.5577 handle_command_reply: tid=0 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 client.5577 shutdown 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 msgr2=0x5607e1446690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff31c0011e0 tx=0x7ff31c000ed0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 shutdown_connections 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607e13f3800 0x5607e1423b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:6834/20691839,v1:172.21.15.28:6835/20691839] conn(0x5607e149a5f0 0x5607e14240d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5607e13f3420 0x5607e1489cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 --2- 172.21.15.28:0/3069909957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5607e13e0300 0x5607e1446690 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff315600640 0 client.5577 ms_handle_reset on v2:172.21.15.28:6834/20691839 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 >> 172.21.15.28:0/3069909957 conn(0x5607e13d9480 msgr2=0x5607e1446c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 shutdown_connections 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 1 -- 172.21.15.28:0/3069909957 wait complete. 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.608+0000 7ff335e33740 20 client.5577 trim_cache size 0 max 16384 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:46:16.611 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:46:16.612 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:16.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 msgr2=0x7ff30c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff31c000f60 tx=0x7ff31c007d50 comp rx=0 tx=0).stop 2024-09-10T10:46:16.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 msgr2=0x7ff32c145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff31800e870 tx=0x7ff31800ed30 comp rx=0 tx=0).stop 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 shutdown_connections 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff30c06be30 0x7ff30c06e2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff32c15e5e0 0x7ff32c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff32c0d08b0 0x7ff32c145610 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 --2- 172.21.15.28:0/1773900782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff32c0b3560 0x7ff32c0c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 >> 172.21.15.28:0/1773900782 conn(0x7ff32c06bdf0 msgr2=0x7ff32c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 shutdown_connections 2024-09-10T10:46:16.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.609+0000 7ff333000640 1 -- 172.21.15.28:0/1773900782 wait complete. 2024-09-10T10:46:16.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-10T10:46:16.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.785+0000 7fa5dc200640 1 Processor -- start 2024-09-10T10:46:16.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.786+0000 7fa5dc200640 1 -- start start 2024-09-10T10:46:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5dc200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa5d40cf790 0x7fa5d40cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5dc200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5d40ce060 0x7fa5d40ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5dc200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d40b30e0 con 0x7fa5d4156740 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5dc200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d40d0140 con 0x7fa5d40ce060 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5dc200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d406edc0 con 0x7fa5d40cf790 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5d40ce060 0x7fa5d40ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5d40ce060 0x7fa5d40ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36220/0 (socket says 172.21.15.28:36220) 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 -- 172.21.15.28:0/2293178728 learned_addr learned my addr 172.21.15.28:0/2293178728 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5d8e00640 1 --2- 172.21.15.28:0/2293178728 >> v2:172.21.15.138:3300/0 conn(0x7fa5d40cf790 0x7fa5d40cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 -- 172.21.15.28:0/2293178728 >> v2:172.21.15.138:3300/0 conn(0x7fa5d40cf790 msgr2=0x7fa5d40cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 --2- 172.21.15.28:0/2293178728 >> v2:172.21.15.138:3300/0 conn(0x7fa5d40cf790 0x7fa5d40cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 -- 172.21.15.28:0/2293178728 >> v1:172.21.15.138:6789/0 conn(0x7fa5d4156740 legacy=0x7fa5d40b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.787+0000 7fa5da200640 1 -- 172.21.15.28:0/2293178728 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d40b33c0 con 0x7fa5d40ce060 2024-09-10T10:46:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.788+0000 7fa5da200640 1 --2- 172.21.15.28:0/2293178728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40ce060 0x7fa5d40ce480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b5260 tx=0x7fa5cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735add68d794c8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5d3e00640 1 -- 172.21.15.28:0/2293178728 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc036070 con 0x7fa5d40ce060 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5d3e00640 1 -- 172.21.15.28:0/2293178728 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5cc031600 con 0x7fa5d40ce060 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5d3e00640 1 -- 172.21.15.28:0/2293178728 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0319d0 con 0x7fa5d40ce060 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 -- 172.21.15.28:0/2293178728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40ce060 msgr2=0x7fa5d40ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2293178728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40ce060 0x7fa5d40ce480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa5d40b5260 tx=0x7fa5cc02a6f0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 -- 172.21.15.28:0/2293178728 shutdown_connections 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2293178728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40ce060 0x7fa5d40ce480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2293178728 >> v2:172.21.15.138:3300/0 conn(0x7fa5d40cf790 0x7fa5d40cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 -- 172.21.15.28:0/2293178728 >> 172.21.15.28:0/2293178728 conn(0x7fa5d406bdf0 msgr2=0x7fa5d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 -- 172.21.15.28:0/2293178728 shutdown_connections 2024-09-10T10:46:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.789+0000 7fa5dc200640 1 -- 172.21.15.28:0/2293178728 wait complete. 2024-09-10T10:46:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.791+0000 7fa5dc200640 1 Processor -- start 2024-09-10T10:46:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.791+0000 7fa5dc200640 1 -- start start 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d40ce060 0x7fa5d4154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d4156740 0x7fa5d414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d40b3c60 con 0x7fa5d40cf790 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d40b3980 con 0x7fa5d4156740 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5dc200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d40b4cc0 con 0x7fa5d40ce060 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5d8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5d8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36236/0 (socket says 172.21.15.28:36236) 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.792+0000 7fa5d8e00640 1 -- 172.21.15.28:0/2908595884 learned_addr learned my addr 172.21.15.28:0/2908595884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5da200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d4156740 0x7fa5d414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d9800640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d40ce060 0x7fa5d4154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 -- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d40ce060 msgr2=0x7fa5d4154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d40ce060 0x7fa5d4154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 -- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d4156740 msgr2=0x7fa5d414c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d4156740 0x7fa5d414c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 -- 172.21.15.28:0/2908595884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d4154e40 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d8e00640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000d800 tx=0x7fa5c000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.793+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000b6c0 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.794+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5c0019070 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.794+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000ba80 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.794+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d4155120 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.794+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d414ce00 con 0x7fa5d40cf790 2024-09-10T10:46:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.795+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d4155640 con 0x7fa5d40cf790 2024-09-10T10:46:16.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.796+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5c000bc20 con 0x7fa5d40cf790 2024-09-10T10:46:16.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.796+0000 7fa5d1600640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 0x7fa5b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:16.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.796+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fa5c008b3e0 con 0x7fa5d40cf790 2024-09-10T10:46:16.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.797+0000 7fa5d9800640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 0x7fa5b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:16.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.798+0000 7fa5d9800640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 0x7fa5b406e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8006fd0 tx=0x7fa5c8009060 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:16.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.801+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5d4155640 con 0x7fa5d40cf790 2024-09-10T10:46:16.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.983+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7fa5d40d0890 con 0x7fa5d40cf790 2024-09-10T10:46:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.987+0000 7fa5d1600640 1 -- 172.21.15.28:0/2908595884 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v13)=0 v13) ==== 135+0+0 (secure 0 0 0) 0x7fa5c0059510 con 0x7fa5d40cf790 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 msgr2=0x7fa5b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 0x7fa5b406e2f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8006fd0 tx=0x7fa5c8009060 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 msgr2=0x7fa5d40b0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000d800 tx=0x7fa5c000dcc0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 shutdown_connections 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa5b406be30 0x7fa5b406e2f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5d4156740 0x7fa5d414c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5d40cf790 0x7fa5d40b0770 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 --2- 172.21.15.28:0/2908595884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5d40ce060 0x7fa5d4154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:16.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 >> 172.21.15.28:0/2908595884 conn(0x7fa5d406bdf0 msgr2=0x7fa5d40ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 shutdown_connections 2024-09-10T10:46:16.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:16.989+0000 7fa5dc200640 1 -- 172.21.15.28:0/2908595884 wait complete. 2024-09-10T10:46:17.006 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-10T10:46:17.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.163+0000 7f8031a00640 1 Processor -- start 2024-09-10T10:46:17.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.163+0000 7f8031a00640 1 -- start start 2024-09-10T10:46:17.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.164+0000 7f8031a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 0x7f802c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.164+0000 7f8031a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 0x7f802c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.164+0000 7f8031a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802c073980 con 0x7f802c0cc280 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.164+0000 7f8031a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f802c0af980 con 0x7f802c0af5a0 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.164+0000 7f8031a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f802c073c20 con 0x7f802c15d2a0 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8031000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f802c0cc280 0x7f802c0cc6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34788/0 (socket says 172.21.15.28:34788) 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8031000640 1 -- 172.21.15.28:0/488233788 learned_addr learned my addr 172.21.15.28:0/488233788 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f802be00640 1 --2- 172.21.15.28:0/488233788 >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 0x7f802c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8023e00640 1 --2- 172.21.15.28:0/488233788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 0x7f802c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f802b400640 1 -- 172.21.15.28:0/488233788 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043038285 0 0) 0x7f802c073980 con 0x7f802c0cc280 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f802b400640 1 -- 172.21.15.28:0/488233788 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8010003520 con 0x7f802c0cc280 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8023e00640 1 -- 172.21.15.28:0/488233788 >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 msgr2=0x7f802c071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8023e00640 1 --2- 172.21.15.28:0/488233788 >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 0x7f802c071cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8023e00640 1 -- 172.21.15.28:0/488233788 >> v1:172.21.15.28:6789/0 conn(0x7f802c0cc280 legacy=0x7f802c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f8023e00640 1 -- 172.21.15.28:0/488233788 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c06dda0 con 0x7f802c15d2a0 2024-09-10T10:46:17.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.165+0000 7f802be00640 1 --2- 172.21.15.28:0/488233788 >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 0x7f802c071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.167+0000 7f8023e00640 1 --2- 172.21.15.28:0/488233788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 0x7f802c06e5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80140097a0 tx=0x7f801402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=218c8eb49fa98bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.167+0000 7f802b400640 1 -- 172.21.15.28:0/488233788 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014037070 con 0x7f802c15d2a0 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.167+0000 7f802b400640 1 -- 172.21.15.28:0/488233788 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8014032440 con 0x7f802c15d2a0 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.167+0000 7f802b400640 1 -- 172.21.15.28:0/488233788 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014032810 con 0x7f802c15d2a0 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 -- 172.21.15.28:0/488233788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 msgr2=0x7f802c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 --2- 172.21.15.28:0/488233788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 0x7f802c06e5a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f80140097a0 tx=0x7f801402a180 comp rx=0 tx=0).stop 2024-09-10T10:46:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 -- 172.21.15.28:0/488233788 shutdown_connections 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 --2- 172.21.15.28:0/488233788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c15d2a0 0x7f802c06e5a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 --2- 172.21.15.28:0/488233788 >> v2:172.21.15.138:3300/0 conn(0x7f802c0af5a0 0x7f802c071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 -- 172.21.15.28:0/488233788 >> 172.21.15.28:0/488233788 conn(0x7f802c0b1bd0 msgr2=0x7f802c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 -- 172.21.15.28:0/488233788 shutdown_connections 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.168+0000 7f8031a00640 1 -- 172.21.15.28:0/488233788 wait complete. 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.169+0000 7f8031a00640 1 Processor -- start 2024-09-10T10:46:17.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.169+0000 7f8031a00640 1 -- start start 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0af5a0 0x7f802c14c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cc280 0x7f802c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f802c072230 con 0x7f802c15d2a0 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f802c073f00 con 0x7f802c0cc280 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.170+0000 7f8031a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f802c0723b0 con 0x7f802c0af5a0 2024-09-10T10:46:17.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36250/0 (socket says 172.21.15.28:36250) 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 -- 172.21.15.28:0/1954671705 learned_addr learned my addr 172.21.15.28:0/1954671705 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f802be00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0af5a0 0x7f802c14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8023e00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cc280 0x7f802c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 -- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0af5a0 msgr2=0x7f802c14c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0af5a0 0x7f802c14c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 -- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cc280 msgr2=0x7f802c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cc280 0x7f802c09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.171+0000 7f8031000640 1 -- 172.21.15.28:0/1954671705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c0cdad0 con 0x7f802c15d2a0 2024-09-10T10:46:17.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8031000640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f802400cb60 tx=0x7f802400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:17.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024017070 con 0x7f802c15d2a0 2024-09-10T10:46:17.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f802400ad40 con 0x7f802c15d2a0 2024-09-10T10:46:17.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8024012600 con 0x7f802c15d2a0 2024-09-10T10:46:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c0cddb0 con 0x7f802c15d2a0 2024-09-10T10:46:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.172+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c0ce370 con 0x7f802c15d2a0 2024-09-10T10:46:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.173+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c071cf0 con 0x7f802c15d2a0 2024-09-10T10:46:17.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.174+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f802401a070 con 0x7f802c15d2a0 2024-09-10T10:46:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.175+0000 7f8028c00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 0x7f800806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.175+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f802408adc0 con 0x7f802c15d2a0 2024-09-10T10:46:17.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.175+0000 7f802be00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 0x7f800806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:17.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.176+0000 7f802be00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 0x7f800806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f801c009560 tx=0x7f801c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.179+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8024058f00 con 0x7f802c15d2a0 2024-09-10T10:46:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.362+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f802c0b21c0 con 0x7f802c15d2a0 2024-09-10T10:46:17.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.366+0000 7f8028c00640 1 -- 172.21.15.28:0/1954671705 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v14)=0 v14) ==== 138+0+0 (secure 0 0 0) 0x7f802405d200 con 0x7f802c15d2a0 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.368+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 msgr2=0x7f800806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 0x7f800806e2f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f801c009560 tx=0x7f801c009190 comp rx=0 tx=0).stop 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 msgr2=0x7f802c0b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f802400cb60 tx=0x7f802400a430 comp rx=0 tx=0).stop 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 shutdown_connections 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f800806be30 0x7f800806e2f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f802c15d2a0 0x7f802c0b4ab0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f802c0cc280 0x7f802c09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 --2- 172.21.15.28:0/1954671705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f802c0af5a0 0x7f802c14c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:17.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 >> 172.21.15.28:0/1954671705 conn(0x7f802c0b1bd0 msgr2=0x7f802c0ada80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:17.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 shutdown_connections 2024-09-10T10:46:17.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:17.369+0000 7f8031a00640 1 -- 172.21.15.28:0/1954671705 wait complete. 2024-09-10T10:46:22.386 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mv a/b a/c; sync .') 2024-09-10T10:46:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:22.480 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:46:22.479+0000 7fd1a1200640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5338:44 nref=7 cr=0x56409f54ae00)) (dversion lock w=1 last_client=5338) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x56409f4ee280] 2024-09-10T10:46:22.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:22.491 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:22.491 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:22.529 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:46:22.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.726+0000 7fa859c00640 1 Processor -- start 2024-09-10T10:46:22.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.726+0000 7fa859c00640 1 -- start start 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.727+0000 7fa859c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa854072320 0x7fa8540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.727+0000 7fa859c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa854072de0 0x7fa85406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.727+0000 7fa859c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa854146140 con 0x7fa854072700 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.727+0000 7fa859c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540cd740 con 0x7fa854072de0 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.727+0000 7fa859c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa8540cda20 con 0x7fa854072320 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa854072de0 0x7fa85406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa854072de0 0x7fa85406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36258/0 (socket says 172.21.15.28:36258) 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 -- 172.21.15.28:0/3202401782 learned_addr learned my addr 172.21.15.28:0/3202401782 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:22.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852a00640 1 --2- 172.21.15.28:0/3202401782 >> v2:172.21.15.138:3300/0 conn(0x7fa854072320 0x7fa8540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 -- 172.21.15.28:0/3202401782 >> v2:172.21.15.138:3300/0 conn(0x7fa854072320 msgr2=0x7fa8540cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 --2- 172.21.15.28:0/3202401782 >> v2:172.21.15.138:3300/0 conn(0x7fa854072320 0x7fa8540cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 -- 172.21.15.28:0/3202401782 >> v1:172.21.15.138:6789/0 conn(0x7fa854072700 legacy=0x7fa85406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.728+0000 7fa852000640 1 -- 172.21.15.28:0/3202401782 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540cdd00 con 0x7fa854072de0 2024-09-10T10:46:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.729+0000 7fa852000640 1 --2- 172.21.15.28:0/3202401782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072de0 0x7fa85406e0b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8540b31e0 tx=0x7fa84002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa0dfb044f9c08d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa851600640 1 -- 172.21.15.28:0/3202401782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa854072de0 2024-09-10T10:46:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa851600640 1 -- 172.21.15.28:0/3202401782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840032440 con 0x7fa854072de0 2024-09-10T10:46:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa851600640 1 -- 172.21.15.28:0/3202401782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840032810 con 0x7fa854072de0 2024-09-10T10:46:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 -- 172.21.15.28:0/3202401782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072de0 msgr2=0x7fa85406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 --2- 172.21.15.28:0/3202401782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072de0 0x7fa85406e0b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa8540b31e0 tx=0x7fa84002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 -- 172.21.15.28:0/3202401782 shutdown_connections 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 --2- 172.21.15.28:0/3202401782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072de0 0x7fa85406e0b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 --2- 172.21.15.28:0/3202401782 >> v2:172.21.15.138:3300/0 conn(0x7fa854072320 0x7fa8540cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 -- 172.21.15.28:0/3202401782 >> 172.21.15.28:0/3202401782 conn(0x7fa854067d70 msgr2=0x7fa85406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 -- 172.21.15.28:0/3202401782 shutdown_connections 2024-09-10T10:46:22.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.730+0000 7fa859c00640 1 -- 172.21.15.28:0/3202401782 wait complete. 2024-09-10T10:46:22.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.732+0000 7fa859c00640 1 Processor -- start 2024-09-10T10:46:22.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.732+0000 7fa859c00640 1 -- start start 2024-09-10T10:46:22.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa854072700 0x7fa8540c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa854072de0 0x7fa85406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540cd520 con 0x7fa854072320 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540cd3a0 con 0x7fa854072700 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa859c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa85406ed00 con 0x7fa854072de0 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa852a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa852a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36266/0 (socket says 172.21.15.28:36266) 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa852a00640 1 -- 172.21.15.28:0/2140657450 learned_addr learned my addr 172.21.15.28:0/2140657450 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa852000640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa854072700 0x7fa8540c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.733+0000 7fa853400640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa854072de0 0x7fa85406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 -- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa854072de0 msgr2=0x7fa85406e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa854072de0 0x7fa85406e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 -- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa854072700 msgr2=0x7fa8540c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa854072700 0x7fa8540c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 -- 172.21.15.28:0/2140657450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8540c6d90 con 0x7fa854072320 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa852a00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e930 tx=0x7fa84400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844017070 con 0x7fa854072320 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84400ca00 con 0x7fa854072320 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa854143160 con 0x7fa854072320 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84400cdf0 con 0x7fa854072320 2024-09-10T10:46:22.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.734+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8540c6a40 con 0x7fa854072320 2024-09-10T10:46:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.735+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa854142f80 con 0x7fa854072320 2024-09-10T10:46:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.736+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa84401a070 con 0x7fa854072320 2024-09-10T10:46:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.737+0000 7fa84aa00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.737+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fa84408b3f0 con 0x7fa854072320 2024-09-10T10:46:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.738+0000 7fa852000640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:22.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.738+0000 7fa852000640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 0x7fa83406e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa840034000 tx=0x7fa84002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:22.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.742+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa854142f80 con 0x7fa854072320 2024-09-10T10:46:22.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.940+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa85413fb00 con 0x7fa854072320 2024-09-10T10:46:22.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.940+0000 7fa84aa00640 1 -- 172.21.15.28:0/2140657450 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) ==== 80+0+0 (secure 0 0 0) 0x7fa8440595e0 con 0x7fa854072320 2024-09-10T10:46:22.941 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 msgr2=0x7fa83406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 0x7fa83406e2f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa840034000 tx=0x7fa84002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 msgr2=0x7fa8540c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa84400e930 tx=0x7fa84400edf0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 shutdown_connections 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa83406be30 0x7fa83406e2f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa854072de0 0x7fa85406e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa854072700 0x7fa8540c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 --2- 172.21.15.28:0/2140657450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa854072320 0x7fa8540c8640 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:22.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.942+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 >> 172.21.15.28:0/2140657450 conn(0x7fa854067d70 msgr2=0x7fa85406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:22.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.943+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 shutdown_connections 2024-09-10T10:46:22.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:22.943+0000 7fa859c00640 1 -- 172.21.15.28:0/2140657450 wait complete. 2024-09-10T10:46:22.960 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:46:22.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:22.960 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:23.026 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:46:23.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:23.027 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:46:23.062 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.257+0000 7fb8e3a00640 1 Processor -- start 2024-09-10T10:46:23.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.257+0000 7fb8e3a00640 1 -- start start 2024-09-10T10:46:23.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.258+0000 7fb8e3a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb8dc15b300 0x7fb8dc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.258+0000 7fb8e3a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8dc0cd3f0 0x7fb8dc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.258+0000 7fb8e3a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8dc0cdbe0 con 0x7fb8dc072d30 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.258+0000 7fb8e3a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0728a0 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.258+0000 7fb8e3a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb8dc0cd900 con 0x7fb8dc15b300 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8dbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8dc0cd3f0 0x7fb8dc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8e1a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb8dc072d30 0x7fb8dc0b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34800/0 (socket says 172.21.15.28:34800) 2024-09-10T10:46:23.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8e1a00640 1 -- 172.21.15.28:0/3601379852 learned_addr learned my addr 172.21.15.28:0/3601379852 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8e1000640 1 --2- 172.21.15.28:0/3601379852 >> v2:172.21.15.138:3300/0 conn(0x7fb8dc15b300 0x7fb8dc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8db400640 1 -- 172.21.15.28:0/3601379852 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2298649897 0 0) 0x7fb8dc0cdbe0 con 0x7fb8dc072d30 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8dbe00640 1 -- 172.21.15.28:0/3601379852 >> v2:172.21.15.138:3300/0 conn(0x7fb8dc15b300 msgr2=0x7fb8dc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8dbe00640 1 --2- 172.21.15.28:0/3601379852 >> v2:172.21.15.138:3300/0 conn(0x7fb8dc15b300 0x7fb8dc0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8dbe00640 1 -- 172.21.15.28:0/3601379852 >> v1:172.21.15.28:6789/0 conn(0x7fb8dc072d30 legacy=0x7fb8dc0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.259+0000 7fb8dbe00640 1 -- 172.21.15.28:0/3601379852 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc072250 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.260+0000 7fb8dbe00640 1 --2- 172.21.15.28:0/3601379852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc06f1c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc070250 tx=0x7fb8c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5616d0f29144d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.260+0000 7fb8db400640 1 -- 172.21.15.28:0/3601379852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8037070 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.260+0000 7fb8db400640 1 -- 172.21.15.28:0/3601379852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c8032440 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8db400640 1 -- 172.21.15.28:0/3601379852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8032810 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 -- 172.21.15.28:0/3601379852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc0cd3f0 msgr2=0x7fb8dc06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/3601379852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc06f1c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc070250 tx=0x7fb8c802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 -- 172.21.15.28:0/3601379852 shutdown_connections 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/3601379852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc06f1c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/3601379852 >> v2:172.21.15.138:3300/0 conn(0x7fb8dc15b300 0x7fb8dc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 -- 172.21.15.28:0/3601379852 >> 172.21.15.28:0/3601379852 conn(0x7fb8dc067d70 msgr2=0x7fb8dc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:23.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 -- 172.21.15.28:0/3601379852 shutdown_connections 2024-09-10T10:46:23.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.261+0000 7fb8e3a00640 1 -- 172.21.15.28:0/3601379852 wait complete. 2024-09-10T10:46:23.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.263+0000 7fb8e3a00640 1 Processor -- start 2024-09-10T10:46:23.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.263+0000 7fb8e3a00640 1 -- start start 2024-09-10T10:46:23.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc0c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 0x7fb8dc0c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0d0160 con 0x7fb8dc072d30 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0cffe0 con 0x7fb8dc0cd3f0 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e3a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc06f700 con 0x7fb8dc15b300 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36288/0 (socket says 172.21.15.28:36288) 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 -- 172.21.15.28:0/322863675 learned_addr learned my addr 172.21.15.28:0/322863675 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 0x7fb8dc0c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8dbe00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc0c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 -- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 msgr2=0x7fb8dc0c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 0x7fb8dc0c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 -- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8dc0cd3f0 msgr2=0x7fb8dc0c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc0c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1000640 1 -- 172.21.15.28:0/322863675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc1577c0 con 0x7fb8dc072d30 2024-09-10T10:46:23.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.264+0000 7fb8e1a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 0x7fb8dc0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:23.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8e1000640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000d5f0 tx=0x7fb8d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00125f0 con 0x7fb8dc072d30 2024-09-10T10:46:23.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d0019070 con 0x7fb8dc072d30 2024-09-10T10:46:23.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00129d0 con 0x7fb8dc072d30 2024-09-10T10:46:23.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc157b00 con 0x7fb8dc072d30 2024-09-10T10:46:23.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.265+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc158340 con 0x7fb8dc072d30 2024-09-10T10:46:23.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.266+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8dc06fac0 con 0x7fb8dc072d30 2024-09-10T10:46:23.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.267+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb8d0012b70 con 0x7fb8dc072d30 2024-09-10T10:46:23.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.268+0000 7fb8d8c00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 0x7fb8bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.268+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7fb8d008b440 con 0x7fb8dc072d30 2024-09-10T10:46:23.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.269+0000 7fb8dbe00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 0x7fb8bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.269+0000 7fb8dbe00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 0x7fb8bc06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8034000 tx=0x7fb8c802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.273+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb8dc157b00 con 0x7fb8dc072d30 2024-09-10T10:46:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.491+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8dc06e150 con 0x7fb8dc072d30 2024-09-10T10:46:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.492+0000 7fb8d8c00640 1 -- 172.21.15.28:0/322863675 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7fb8d0059bc0 con 0x7fb8dc072d30 2024-09-10T10:46:23.493 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:23.493 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":32,"btime":"2024-09-10T10:46:11:726443+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:11.149029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/20691839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":20691839},{"type":"v1","addr":"172.21.15.28:6835","nonce":20691839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":3}]} 2024-09-10T10:46:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 32 2024-09-10T10:46:23.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 msgr2=0x7fb8bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 0x7fb8bc06e2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8034000 tx=0x7fb8c802e000 comp rx=0 tx=0).stop 2024-09-10T10:46:23.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 msgr2=0x7fb8dc140be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000d5f0 tx=0x7fb8d000dab0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 shutdown_connections 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb8bc06be30 0x7fb8bc06e2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8dc15b300 0x7fb8dc0c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8dc0cd3f0 0x7fb8dc0c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 --2- 172.21.15.28:0/322863675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8dc072d30 0x7fb8dc140be0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 >> 172.21.15.28:0/322863675 conn(0x7fb8dc067d70 msgr2=0x7fb8dc0cb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 shutdown_connections 2024-09-10T10:46:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.494+0000 7fb8e3a00640 1 -- 172.21.15.28:0/322863675 wait complete. 2024-09-10T10:46:23.512 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:23.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.668+0000 7f3bb7e00640 1 Processor -- start 2024-09-10T10:46:23.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.668+0000 7f3bb7e00640 1 -- start start 2024-09-10T10:46:23.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.669+0000 7f3bb7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3bb815bef0 0x7f3bb80b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.669+0000 7f3bb7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3bb80b25c0 0x7f3bb80aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.669+0000 7f3bb7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb80b5470 con 0x7f3bb80af1b0 2024-09-10T10:46:23.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.669+0000 7f3bb7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80b3640 con 0x7f3bb815bef0 2024-09-10T10:46:23.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.669+0000 7f3bb7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb80b5190 con 0x7f3bb80b25c0 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3bb80af1b0 0x7f3bb8071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34810/0 (socket says 172.21.15.28:34810) 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb7400640 1 -- 172.21.15.28:0/2656417059 learned_addr learned my addr 172.21.15.28:0/2656417059 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/2656417059 >> v2:172.21.15.28:3300/0 conn(0x7f3bb815bef0 0x7f3bb80b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6000640 1 --2- 172.21.15.28:0/2656417059 >> v2:172.21.15.138:3300/0 conn(0x7f3bb80b25c0 0x7f3bb80aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6a00640 1 -- 172.21.15.28:0/2656417059 >> v2:172.21.15.138:3300/0 conn(0x7f3bb80b25c0 msgr2=0x7f3bb80aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/2656417059 >> v2:172.21.15.138:3300/0 conn(0x7f3bb80b25c0 0x7f3bb80aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6a00640 1 -- 172.21.15.28:0/2656417059 >> v1:172.21.15.28:6789/0 conn(0x7f3bb80af1b0 legacy=0x7f3bb8071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.670+0000 7f3bb6a00640 1 -- 172.21.15.28:0/2656417059 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80b38e0 con 0x7f3bb815bef0 2024-09-10T10:46:23.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.671+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/2656417059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb815bef0 0x7f3bb80b40e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80afb80 tx=0x7f3ba802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416f2d874e7087bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.671+0000 7f3bb5600640 1 -- 172.21.15.28:0/2656417059 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8037070 con 0x7f3bb815bef0 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.671+0000 7f3bb5600640 1 -- 172.21.15.28:0/2656417059 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba8032440 con 0x7f3bb815bef0 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.671+0000 7f3bb5600640 1 -- 172.21.15.28:0/2656417059 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba802ae60 con 0x7f3bb815bef0 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 -- 172.21.15.28:0/2656417059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb815bef0 msgr2=0x7f3bb80b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/2656417059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb815bef0 0x7f3bb80b40e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80afb80 tx=0x7f3ba802a110 comp rx=0 tx=0).stop 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 -- 172.21.15.28:0/2656417059 shutdown_connections 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/2656417059 >> v2:172.21.15.138:3300/0 conn(0x7f3bb80b25c0 0x7f3bb80aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/2656417059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb815bef0 0x7f3bb80b40e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 -- 172.21.15.28:0/2656417059 >> 172.21.15.28:0/2656417059 conn(0x7f3bb806c240 msgr2=0x7f3bb80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:23.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 -- 172.21.15.28:0/2656417059 shutdown_connections 2024-09-10T10:46:23.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.672+0000 7f3bb7e00640 1 -- 172.21.15.28:0/2656417059 wait complete. 2024-09-10T10:46:23.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.674+0000 7f3bb7e00640 1 Processor -- start 2024-09-10T10:46:23.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.674+0000 7f3bb7e00640 1 -- start start 2024-09-10T10:46:23.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb80af1b0 0x7f3bb809d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb815bef0 0x7f3bb81599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80b4e60 con 0x7f3bb80b25c0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb80af640 con 0x7f3bb80af1b0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb80b4b10 con 0x7f3bb815bef0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36320/0 (socket says 172.21.15.28:36320) 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 -- 172.21.15.28:0/1310645695 learned_addr learned my addr 172.21.15.28:0/1310645695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb7400640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb815bef0 0x7f3bb81599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb80af1b0 0x7f3bb809d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 -- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb815bef0 msgr2=0x7f3bb81599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb815bef0 0x7f3bb81599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 -- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb80af1b0 msgr2=0x7f3bb809d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb80af1b0 0x7f3bb809d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.675+0000 7f3bb6000640 1 -- 172.21.15.28:0/1310645695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb81f40a0 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3bb6000640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9b0 tx=0x7f3ba400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4017070 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba400c9a0 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb81f4380 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400cd90 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.676+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb81f4940 con 0x7f3bb80b25c0 2024-09-10T10:46:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.677+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb80b2a30 con 0x7f3bb80b25c0 2024-09-10T10:46:23.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.678+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ba401a070 con 0x7f3bb80b25c0 2024-09-10T10:46:23.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.680+0000 7f3baea00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 0x7f3b9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:23.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.680+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f3ba408b380 con 0x7f3bb80b25c0 2024-09-10T10:46:23.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.680+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 0x7f3b9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:23.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.681+0000 7f3bb6a00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 0x7f3b9806e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8034000 tx=0x7f3ba802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:23.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.684+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3ba4059b90 con 0x7f3bb80b25c0 2024-09-10T10:46:23.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.911+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb80b17d0 con 0x7f3bb80b25c0 2024-09-10T10:46:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.912+0000 7f3baea00640 1 -- 172.21.15.28:0/1310645695 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5223 (secure 0 0 0) 0x7f3ba4059490 con 0x7f3bb80b25c0 2024-09-10T10:46:23.913 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:23.913 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":32,"btime":"2024-09-10T10:46:11:726443+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:11.149029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5418},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5418":{"gid":5418,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/20691839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":20691839},{"type":"v1","addr":"172.21.15.28:6835","nonce":20691839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5418,"qdb_cluster":[5418]},"id":3}]} 2024-09-10T10:46:23.913 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 32 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 msgr2=0x7f3b9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 0x7f3b9806e2f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8034000 tx=0x7f3ba802e000 comp rx=0 tx=0).stop 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 msgr2=0x7f3bb80ce900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9b0 tx=0x7f3ba400ee70 comp rx=0 tx=0).stop 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 shutdown_connections 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b9806be30 0x7f3b9806e2f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bb815bef0 0x7f3bb81599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bb80b25c0 0x7f3bb80ce900 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 --2- 172.21.15.28:0/1310645695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bb80af1b0 0x7f3bb809d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 >> 172.21.15.28:0/1310645695 conn(0x7f3bb806c240 msgr2=0x7f3bb80b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 shutdown_connections 2024-09-10T10:46:23.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:23.914+0000 7f3bb7e00640 1 -- 172.21.15.28:0/1310645695 wait complete. 2024-09-10T10:46:23.930 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-09-10T10:46:24.104 INFO:teuthology.orchestra.run.smithi028.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-10T10:46:22.478078+0000","btime":"2024-09-10T10:46:16.380243+0000","mode":16877,"uid":1000,"gid":1333,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-10T10:46:22.478078+0000","atime":"2024-09-10T10:46:16.380243+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-10T10:46:22.478078+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T10:46:22.478078+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T10:46:22.478078+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":0,"dirtyparent":0,"dirty":1,"authpin":0},"nref":2,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"3","committed_version":"3","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":0,"dirty":1,"authpin":0},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-10T10:46:24.105 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-10T10:46:22.478078+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-10T10:46:16.380243+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-10T10:46:16.380243+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-10T10:46:22.478078+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "3", "committing_version": "3", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 0, "lock": 0, "request": 0 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/a", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-10T10:46:22.478078+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1333, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-09-10T10:46:22.478078+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 2, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 0, "dirfrag": 1, "dirty": 1, "dirtyparent": 0, "lock": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-10T10:46:22.478078+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-09-10T10:46:24.105 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-10T10:46:22.478078+0000', 'btime': '2024-09-10T10:46:16.380243+0000', 'mode': 16877, 'uid': 1000, 'gid': 1333, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-10T10:46:22.478078+0000', 'atime': '2024-09-10T10:46:16.380243+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-10T10:46:22.478078+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T10:46:22.478078+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T10:46:22.478078+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'dirfrag': 1, 'caps': 0, 'dirtyparent': 0, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '3', 'committed_version': '3', 'is_rep': False, 'dir_auth': '', 'states': ['auth', 'dirty', 'complete'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 1, 'dirty': 1, 'authpin': 0}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'request': 0, 'lock': 0, 'inodepin': 0, 'dirty': 1, 'authpin': 0}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-10T10:46:24.105 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:46:24.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.260+0000 7f333a200640 1 Processor -- start 2024-09-10T10:46:24.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.261+0000 7f333a200640 1 -- start start 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f333a200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3334159ac0 0x7f333415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f333a200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f33340b64f0 0x7f333415da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f333a200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f333415d4c0 con 0x7f33340afc40 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f333a200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f33340cf110 con 0x7f33340b64f0 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f333a200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f33340ce590 con 0x7f3334159ac0 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f3339800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f33340b64f0 0x7f333415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f3333e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f33340afc40 0x7f33340b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46394/0 (socket says 172.21.15.28:46394) 2024-09-10T10:46:24.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f3333e00640 1 -- 172.21.15.28:0/896177600 learned_addr learned my addr 172.21.15.28:0/896177600 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.262+0000 7f3338e00640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3334159ac0 0x7f333415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.263+0000 7f3339800640 1 -- 172.21.15.28:0/896177600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3334159ac0 msgr2=0x7f333415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.263+0000 7f3339800640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3334159ac0 0x7f333415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.263+0000 7f3339800640 1 -- 172.21.15.28:0/896177600 >> v1:172.21.15.28:6789/0 conn(0x7f33340afc40 legacy=0x7f33340b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.263+0000 7f3339800640 1 -- 172.21.15.28:0/896177600 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33340ce870 con 0x7f33340b64f0 2024-09-10T10:46:24.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.263+0000 7f3333400640 1 -- 172.21.15.28:0/896177600 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3979291433 0 0) 0x7f3320001110 con 0x7f33340afc40 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f3339800640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33340b64f0 0x7f333415da60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f332c009d90 tx=0x7f332c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08bab1300eceec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f3333400640 1 -- 172.21.15.28:0/896177600 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c007740 con 0x7f33340b64f0 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f3333400640 1 -- 172.21.15.28:0/896177600 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c0078e0 con 0x7f33340b64f0 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f3333400640 1 -- 172.21.15.28:0/896177600 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c007cb0 con 0x7f33340b64f0 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f333a200640 1 -- 172.21.15.28:0/896177600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33340b64f0 msgr2=0x7f333415da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.264+0000 7f333a200640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33340b64f0 0x7f333415da60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f332c009d90 tx=0x7f332c02a660 comp rx=0 tx=0).stop 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 -- 172.21.15.28:0/896177600 shutdown_connections 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f33340b64f0 0x7f333415da60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 --2- 172.21.15.28:0/896177600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3334159ac0 0x7f333415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 -- 172.21.15.28:0/896177600 >> 172.21.15.28:0/896177600 conn(0x7f3334073470 msgr2=0x7f333406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 -- 172.21.15.28:0/896177600 shutdown_connections 2024-09-10T10:46:24.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.265+0000 7f333a200640 1 -- 172.21.15.28:0/896177600 wait complete. 2024-09-10T10:46:24.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.266+0000 7f333a200640 1 Processor -- start 2024-09-10T10:46:24.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.266+0000 7f333a200640 1 -- start start 2024-09-10T10:46:24.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33340afc40 0x7f3334142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33340b64f0 0x7f333415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3334159ea0 con 0x7f3334159ac0 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f33340cee30 con 0x7f33340b64f0 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.267+0000 7f333a200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f333415dfa0 con 0x7f33340afc40 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35834/0 (socket says 172.21.15.28:35834) 2024-09-10T10:46:24.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 -- 172.21.15.28:0/840790765 learned_addr learned my addr 172.21.15.28:0/840790765 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:24.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 -- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33340afc40 msgr2=0x7f3334142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33340afc40 0x7f3334142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 -- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33340b64f0 msgr2=0x7f333415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33340b64f0 0x7f333415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 -- 172.21.15.28:0/840790765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33340b1690 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.268+0000 7f3339800640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f332c009eb0 tx=0x7f332c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.269+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c040070 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.269+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c031c40 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.269+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c009450 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.269+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33340b1950 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.269+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33340b1f10 con 0x7f3334159ac0 2024-09-10T10:46:24.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.270+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33340cf610 con 0x7f3334159ac0 2024-09-10T10:46:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.271+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f332c039040 con 0x7f3334159ac0 2024-09-10T10:46:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.271+0000 7f3330c00640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 0x7f331406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.272+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6101+0+0 (secure 0 0 0) 0x7f332c0abf30 con 0x7f3334159ac0 2024-09-10T10:46:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.272+0000 7f3338e00640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 0x7f331406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.273+0000 7f3338e00640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 0x7f331406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3328006a90 tx=0x7f3328006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.276+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33340cf610 con 0x7f3334159ac0 2024-09-10T10:46:24.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.495+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f333415da60 con 0x7f3334159ac0 2024-09-10T10:46:24.496 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:46:24.496+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:46:24.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.498+0000 7f3330c00640 1 -- 172.21.15.28:0/840790765 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f332c07a6b0 con 0x7f3334159ac0 2024-09-10T10:46:24.499 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 msgr2=0x7f331406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 0x7f331406e2f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3328006a90 tx=0x7f3328006a20 comp rx=0 tx=0).stop 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 msgr2=0x7f333415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f332c009eb0 tx=0x7f332c004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 shutdown_connections 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f331406be30 0x7f331406e2f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3334159ac0 0x7f333415e8b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f33340b64f0 0x7f333415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 --2- 172.21.15.28:0/840790765 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f33340afc40 0x7f3334142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 >> 172.21.15.28:0/840790765 conn(0x7f3334073470 msgr2=0x7f33341573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 shutdown_connections 2024-09-10T10:46:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.501+0000 7f333a200640 1 -- 172.21.15.28:0/840790765 wait complete. 2024-09-10T10:46:24.509 INFO:tasks.ceph.mds.a.smithi028.stderr: -4202> 2024-09-10T10:46:10.685+0000 7fd19e000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:46:24.517 INFO:tasks.ceph.mds.a.smithi028.stderr: -344> 2024-09-10T10:46:22.479+0000 7fd1a1200640 -1 mds.0.cache.den(0x10000000000 c) newly corrupt dentry to be committed: [dentry #0x1/a/c [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.5338:44 nref=7 cr=0x56409f54ae00)) (dversion lock w=1 last_client=5338) pv=4 v=3 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x56409f4ee280] 2024-09-10T10:46:24.519 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T10:46:24.560 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:46:24.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.679+0000 7f7583200640 1 Processor -- start 2024-09-10T10:46:24.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.679+0000 7f7583200640 1 -- start start 2024-09-10T10:46:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.680+0000 7f7583200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 0x7f758415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.680+0000 7f7583200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758415eca0 con 0x7f7584161f40 2024-09-10T10:46:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7583200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f758415ef80 con 0x7f75840af230 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7583200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75840b0510 con 0x7f758415e190 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7584161f40 0x7f75840af6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46410/0 (socket says 172.21.15.28:46410) 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581400640 1 -- 172.21.15.28:0/3727907284 learned_addr learned my addr 172.21.15.28:0/3727907284 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581e00640 1 --2- 172.21.15.28:0/3727907284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 0x7f758415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678944955 0 0) 0x7f758415eca0 con 0x7f7584161f40 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7568003520 con 0x7f7584161f40 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1490365311 0 0) 0x7f758415ef80 con 0x7f75840af230 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581e00640 1 -- 172.21.15.28:0/3727907284 >> v1:172.21.15.138:6789/0 conn(0x7f75840af230 legacy=0x7f75840afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581e00640 1 -- 172.21.15.28:0/3727907284 >> v1:172.21.15.28:6789/0 conn(0x7f7584161f40 legacy=0x7f75840af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.681+0000 7f7581e00640 1 -- 172.21.15.28:0/3727907284 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758415f260 con 0x7f758415e190 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7581e00640 1 --2- 172.21.15.28:0/3727907284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 0x7f758415e570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75740097d0 tx=0x7f757402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138daf99a7d0a494 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574037070 con 0x7f758415e190 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7574032440 con 0x7f758415e190 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7580a00640 1 -- 172.21.15.28:0/3727907284 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574032810 con 0x7f758415e190 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7583200640 1 -- 172.21.15.28:0/3727907284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 msgr2=0x7f758415e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.683+0000 7f7583200640 1 --2- 172.21.15.28:0/3727907284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 0x7f758415e570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f75740097d0 tx=0x7f757402a180 comp rx=0 tx=0).stop 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.684+0000 7f7583200640 1 -- 172.21.15.28:0/3727907284 shutdown_connections 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.684+0000 7f7583200640 1 --2- 172.21.15.28:0/3727907284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f758415e190 0x7f758415e570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.684+0000 7f7583200640 1 -- 172.21.15.28:0/3727907284 >> 172.21.15.28:0/3727907284 conn(0x7f75840733b0 msgr2=0x7f758406e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.684+0000 7f7583200640 1 -- 172.21.15.28:0/3727907284 shutdown_connections 2024-09-10T10:46:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.684+0000 7f7583200640 1 -- 172.21.15.28:0/3727907284 wait complete. 2024-09-10T10:46:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.685+0000 7f7583200640 1 Processor -- start 2024-09-10T10:46:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.685+0000 7f7583200640 1 -- start start 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75840af230 0x7f75841573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f758415e190 0x7f75841f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f758406cda0 con 0x7f7584161f40 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f758406cc20 con 0x7f758415e190 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.686+0000 7f7583200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f758406cf20 con 0x7f75840af230 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35852/0 (socket says 172.21.15.28:35852) 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 -- 172.21.15.28:0/2519460620 learned_addr learned my addr 172.21.15.28:0/2519460620 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7581e00640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75840af230 0x7f75841573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7581400640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f758415e190 0x7f75841f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 -- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75840af230 msgr2=0x7f75841573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75840af230 0x7f75841573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 -- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f758415e190 msgr2=0x7f75841f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f758415e190 0x7f75841f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 -- 172.21.15.28:0/2519460620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7584159130 con 0x7f7584161f40 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.687+0000 7f7582800640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c760 tx=0x7f757c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.688+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c003da0 con 0x7f7584161f40 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.688+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757c005070 con 0x7f7584161f40 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.688+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75841593d0 con 0x7f7584161f40 2024-09-10T10:46:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.688+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c005460 con 0x7f7584161f40 2024-09-10T10:46:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.688+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7584159eb0 con 0x7f7584161f40 2024-09-10T10:46:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.689+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f758415e570 con 0x7f7584161f40 2024-09-10T10:46:24.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.690+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f757c005600 con 0x7f7584161f40 2024-09-10T10:46:24.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.691+0000 7f757a000640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:24.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.691+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7f757c08af00 con 0x7f7584161f40 2024-09-10T10:46:24.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.691+0000 7f7581e00640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:24.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.692+0000 7f7581e00640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:24.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.695+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f757c0596c0 con 0x7f7584161f40 2024-09-10T10:46:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.884+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f75840ae420 con 0x7f7584161f40 2024-09-10T10:46:24.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.887+0000 7f757a000640 1 -- 172.21.15.28:0/2519460620 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f757c058fc0 con 0x7f7584161f40 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 msgr2=0x7f756406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 0x7f756406e2f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7574034000 tx=0x7f757402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 msgr2=0x7f7584148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f757c00c760 tx=0x7f757c00cc20 comp rx=0 tx=0).stop 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 shutdown_connections 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f756406be30 0x7f756406e2f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7584161f40 0x7f7584148380 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f758415e190 0x7f75841f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 --2- 172.21.15.28:0/2519460620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75840af230 0x7f75841573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 >> 172.21.15.28:0/2519460620 conn(0x7f75840733b0 msgr2=0x7f75840b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 shutdown_connections 2024-09-10T10:46:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:24.890+0000 7f7583200640 1 -- 172.21.15.28:0/2519460620 wait complete. 2024-09-10T10:46:24.907 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_abort_on_newly_corrupt_dentry False 2024-09-10T10:46:25.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.106+0000 7fe0d5a00640 1 Processor -- start 2024-09-10T10:46:25.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.107+0000 7fe0d5a00640 1 -- start start 2024-09-10T10:46:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0d5a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe0d0072760 0x7fe0d00b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0d5a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d00b4620 con 0x7fe0d015e900 2024-09-10T10:46:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0d5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d00ce100 con 0x7fe0d00b4020 2024-09-10T10:46:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0d5a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d00b4f60 con 0x7fe0d0072760 2024-09-10T10:46:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0cea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe0d015e900 0x7fe0d00ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46424/0 (socket says 172.21.15.28:46424) 2024-09-10T10:46:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0cea00640 1 -- 172.21.15.28:0/2113523139 learned_addr learned my addr 172.21.15.28:0/2113523139 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:25.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.108+0000 7fe0cf400640 1 --2- 172.21.15.28:0/2113523139 >> v2:172.21.15.28:3300/0 conn(0x7fe0d0072760 0x7fe0d00b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.109+0000 7fe0cd600640 1 -- 172.21.15.28:0/2113523139 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 132249776 0 0) 0x7fe0d00b4620 con 0x7fe0d015e900 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.109+0000 7fe0cd600640 1 -- 172.21.15.28:0/2113523139 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0b4005730 con 0x7fe0d015e900 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.109+0000 7fe0cf400640 1 -- 172.21.15.28:0/2113523139 >> v1:172.21.15.138:6789/0 conn(0x7fe0d00b4020 legacy=0x7fe0d0072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.109+0000 7fe0cf400640 1 -- 172.21.15.28:0/2113523139 >> v1:172.21.15.28:6789/0 conn(0x7fe0d015e900 legacy=0x7fe0d00ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.109+0000 7fe0cf400640 1 -- 172.21.15.28:0/2113523139 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d00ce3a0 con 0x7fe0d0072760 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0cf400640 1 --2- 172.21.15.28:0/2113523139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d0072760 0x7fe0d00b4a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009d90 tx=0x7fe0c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9039b0bb6b4fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0cd600640 1 -- 172.21.15.28:0/2113523139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8007740 con 0x7fe0d0072760 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0cd600640 1 -- 172.21.15.28:0/2113523139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c80078e0 con 0x7fe0d0072760 2024-09-10T10:46:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0cd600640 1 -- 172.21.15.28:0/2113523139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8007cb0 con 0x7fe0d0072760 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0d5a00640 1 -- 172.21.15.28:0/2113523139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d0072760 msgr2=0x7fe0d00b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.110+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/2113523139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d0072760 0x7fe0d00b4a20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009d90 tx=0x7fe0c802a660 comp rx=0 tx=0).stop 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.111+0000 7fe0d5a00640 1 -- 172.21.15.28:0/2113523139 shutdown_connections 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.111+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/2113523139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d0072760 0x7fe0d00b4a20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.111+0000 7fe0d5a00640 1 -- 172.21.15.28:0/2113523139 >> 172.21.15.28:0/2113523139 conn(0x7fe0d006bdf0 msgr2=0x7fe0d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.111+0000 7fe0d5a00640 1 -- 172.21.15.28:0/2113523139 shutdown_connections 2024-09-10T10:46:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.111+0000 7fe0d5a00640 1 -- 172.21.15.28:0/2113523139 wait complete. 2024-09-10T10:46:25.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.112+0000 7fe0d5a00640 1 Processor -- start 2024-09-10T10:46:25.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.112+0000 7fe0d5a00640 1 -- start start 2024-09-10T10:46:25.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d0072760 0x7fe0d0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 0x7fe0d00b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00cee80 con 0x7fe0d015e900 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00ce960 con 0x7fe0d0072760 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.113+0000 7fe0d5a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d00cf000 con 0x7fe0d00b4020 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35860/0 (socket says 172.21.15.28:35860) 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 -- 172.21.15.28:0/1883603147 learned_addr learned my addr 172.21.15.28:0/1883603147 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cea00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d0072760 0x7fe0d0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0ce000640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 0x7fe0d00b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 -- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 msgr2=0x7fe0d00b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 0x7fe0d00b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 -- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d0072760 msgr2=0x7fe0d0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d0072760 0x7fe0d0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 -- 172.21.15.28:0/1883603147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d00b0b00 con 0x7fe0d015e900 2024-09-10T10:46:25.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0ce000640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 0x7fe0d00b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:25.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.114+0000 7fe0cf400640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009eb0 tx=0x7fe0c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.115+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8040070 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.115+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c8031bc0 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.115+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d0155140 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.115+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8009450 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.115+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d0155700 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.116+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d00b4a20 con 0x7fe0d015e900 2024-09-10T10:46:25.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.117+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe0c8039040 con 0x7fe0d015e900 2024-09-10T10:46:25.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.118+0000 7fe0c6a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.118+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7fe0c80ac980 con 0x7fe0d015e900 2024-09-10T10:46:25.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.119+0000 7fe0cea00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.120+0000 7fe0cea00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 0x7fe0b006e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00011e0 tx=0x7fe0c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.123+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe0d00b0b00 con 0x7fe0d015e900 2024-09-10T10:46:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.304+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}] v 0) -- 0x7fe0d006c370 con 0x7fe0d015e900 2024-09-10T10:46:25.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.304+0000 7fe0c6a00640 1 -- 172.21.15.28:0/1883603147 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_abort_on_newly_corrupt_dentry}]=0 v15)=0 v15) ==== 135+0+0 (secure 0 0 0) 0x7fe0c807b0c0 con 0x7fe0d015e900 2024-09-10T10:46:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.306+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 msgr2=0x7fe0b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.306+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 0x7fe0b006e2f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00011e0 tx=0x7fe0c0009040 comp rx=0 tx=0).stop 2024-09-10T10:46:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.306+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 msgr2=0x7fe0d0153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.306+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8009eb0 tx=0x7fe0c8004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:25.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 shutdown_connections 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe0b006be30 0x7fe0b006e2f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0d015e900 0x7fe0d0153510 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0d00b4020 0x7fe0d00b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 --2- 172.21.15.28:0/1883603147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d0072760 0x7fe0d0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 >> 172.21.15.28:0/1883603147 conn(0x7fe0d006bdf0 msgr2=0x7fe0d0072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 shutdown_connections 2024-09-10T10:46:25.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.307+0000 7fe0d5a00640 1 -- 172.21.15.28:0/1883603147 wait complete. 2024-09-10T10:46:25.324 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:46:25.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.480+0000 7fb228800640 1 Processor -- start 2024-09-10T10:46:25.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.481+0000 7fb228800640 1 -- start start 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb228800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 0x7fb2200cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb228800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200b30e0 con 0x7fb2200ce060 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb228800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2200b33c0 con 0x7fb220156740 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb228800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200d0110 con 0x7fb2200cf790 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb226800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb2200ce060 0x7fb2200ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46440/0 (socket says 172.21.15.28:46440) 2024-09-10T10:46:25.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb226800640 1 -- 172.21.15.28:0/3430555753 learned_addr learned my addr 172.21.15.28:0/3430555753 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb225400640 1 --2- 172.21.15.28:0/3430555753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 0x7fb2200cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796015311 0 0) 0x7fb2200b30e0 con 0x7fb2200ce060 2024-09-10T10:46:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.482+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb208003520 con 0x7fb2200ce060 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2420529887 0 0) 0x7fb2200b33c0 con 0x7fb220156740 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2200b30e0 con 0x7fb220156740 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb225400640 1 -- 172.21.15.28:0/3430555753 >> v1:172.21.15.138:6789/0 conn(0x7fb220156740 legacy=0x7fb2200b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb225400640 1 -- 172.21.15.28:0/3430555753 >> v1:172.21.15.28:6789/0 conn(0x7fb2200ce060 legacy=0x7fb2200ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2641698354 0 0) 0x7fb208003520 con 0x7fb2200ce060 2024-09-10T10:46:25.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.483+0000 7fb225400640 1 -- 172.21.15.28:0/3430555753 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2200b36a0 con 0x7fb2200cf790 2024-09-10T10:46:25.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.484+0000 7fb225400640 1 --2- 172.21.15.28:0/3430555753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 0x7fb2200cfbd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097d0 tx=0x7fb21002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bf9c01b43b8d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.484+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210037070 con 0x7fb2200cf790 2024-09-10T10:46:25.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.484+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21002ac30 con 0x7fb2200cf790 2024-09-10T10:46:25.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.484+0000 7fb224a00640 1 -- 172.21.15.28:0/3430555753 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0019f0 con 0x7fb2200cf790 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 -- 172.21.15.28:0/3430555753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 msgr2=0x7fb2200cfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 --2- 172.21.15.28:0/3430555753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 0x7fb2200cfbd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb2100097d0 tx=0x7fb21002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 -- 172.21.15.28:0/3430555753 shutdown_connections 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 --2- 172.21.15.28:0/3430555753 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2200cf790 0x7fb2200cfbd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 -- 172.21.15.28:0/3430555753 >> 172.21.15.28:0/3430555753 conn(0x7fb22006bdf0 msgr2=0x7fb22006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 -- 172.21.15.28:0/3430555753 shutdown_connections 2024-09-10T10:46:25.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.485+0000 7fb228800640 1 -- 172.21.15.28:0/3430555753 wait complete. 2024-09-10T10:46:25.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.487+0000 7fb228800640 1 Processor -- start 2024-09-10T10:46:25.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.487+0000 7fb228800640 1 -- start start 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 0x7fb220072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb220156740 0x7fb2200733d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb2200b3c60 con 0x7fb2200ce060 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb22006edc0 con 0x7fb2200cf790 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb228800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2200b4cc0 con 0x7fb220156740 2024-09-10T10:46:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35866/0 (socket says 172.21.15.28:35866) 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 -- 172.21.15.28:0/3837067659 learned_addr learned my addr 172.21.15.28:0/3837067659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb226800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb220156740 0x7fb2200733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225400640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 0x7fb220072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 -- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb220156740 msgr2=0x7fb2200733d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb220156740 0x7fb2200733d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 -- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 msgr2=0x7fb220072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 0x7fb220072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225e00640 1 -- 172.21.15.28:0/3837067659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220149de0 con 0x7fb2200ce060 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.488+0000 7fb225400640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 0x7fb220072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb225e00640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c430 tx=0x7fb21400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214017070 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2140052f0 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214003830 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22014a0c0 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.489+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22014a3a0 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.490+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2200d0920 con 0x7fb2200ce060 2024-09-10T10:46:25.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.491+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb21401a070 con 0x7fb2200ce060 2024-09-10T10:46:25.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.492+0000 7fb21a000640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:25.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.492+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7fb21408a450 con 0x7fb2200ce060 2024-09-10T10:46:25.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.492+0000 7fb225400640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:25.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.493+0000 7fb225400640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 0x7fb20406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb210034000 tx=0x7fb21002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:25.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.496+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb2140585c0 con 0x7fb2200ce060 2024-09-10T10:46:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:25.720+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb2200b2bf0 con 0x7fb2200ce060 2024-09-10T10:46:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.588+0000 7fb21a000640 1 -- 172.21.15.28:0/3837067659 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v35) ==== 162+0+0 (secure 0 0 0) 0x7fb21405c8c0 con 0x7fb2200ce060 2024-09-10T10:46:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:46:26.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 msgr2=0x7fb20406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 0x7fb20406e2f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb210034000 tx=0x7fb21002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:26.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 msgr2=0x7fb2201f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb21400c430 tx=0x7fb21400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 shutdown_connections 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb20406be30 0x7fb20406e2f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb220156740 0x7fb2200733d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2200cf790 0x7fb220072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 --2- 172.21.15.28:0/3837067659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2200ce060 0x7fb2201f7160 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 >> 172.21.15.28:0/3837067659 conn(0x7fb22006bdf0 msgr2=0x7fb2200b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 shutdown_connections 2024-09-10T10:46:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.592+0000 7fb228800640 1 -- 172.21.15.28:0/3837067659 wait complete. 2024-09-10T10:46:26.615 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:26.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.775+0000 7fb766600640 1 Processor -- start 2024-09-10T10:46:26.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.775+0000 7fb766600640 1 -- start start 2024-09-10T10:46:26.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb766600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7600d4180 0x7fb7600d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb766600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb76015e6a0 0x7fb76006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb766600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600d4aa0 con 0x7fb76006dd60 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb766600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600cefb0 con 0x7fb76015e6a0 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb766600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600d2c20 con 0x7fb7600d4180 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb75fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb76015e6a0 0x7fb76006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb75fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb76015e6a0 0x7fb76006d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35876/0 (socket says 172.21.15.28:35876) 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.776+0000 7fb75fe00640 1 -- 172.21.15.28:0/188946842 learned_addr learned my addr 172.21.15.28:0/188946842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:26.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.777+0000 7fb765200640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7600d4180 0x7fb7600d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.777+0000 7fb75fe00640 1 -- 172.21.15.28:0/188946842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7600d4180 msgr2=0x7fb7600d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:26.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.777+0000 7fb75fe00640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7600d4180 0x7fb7600d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.777+0000 7fb75fe00640 1 -- 172.21.15.28:0/188946842 >> v1:172.21.15.138:6789/0 conn(0x7fb76006dd60 legacy=0x7fb7600d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.777+0000 7fb75fe00640 1 -- 172.21.15.28:0/188946842 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76006e390 con 0x7fb76015e6a0 2024-09-10T10:46:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.778+0000 7fb75fe00640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb76006d820 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009870 tx=0x7fb74c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14865b17af73f3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.778+0000 7fb75f400640 1 -- 172.21.15.28:0/188946842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c037070 con 0x7fb76015e6a0 2024-09-10T10:46:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.778+0000 7fb75f400640 1 -- 172.21.15.28:0/188946842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb74c032440 con 0x7fb76015e6a0 2024-09-10T10:46:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.778+0000 7fb75f400640 1 -- 172.21.15.28:0/188946842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74c032810 con 0x7fb76015e6a0 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 -- 172.21.15.28:0/188946842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 msgr2=0x7fb76006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb76006d820 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009870 tx=0x7fb74c02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 -- 172.21.15.28:0/188946842 shutdown_connections 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb76006d820 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 --2- 172.21.15.28:0/188946842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7600d4180 0x7fb7600d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 -- 172.21.15.28:0/188946842 >> 172.21.15.28:0/188946842 conn(0x7fb76009d6f0 msgr2=0x7fb76008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 -- 172.21.15.28:0/188946842 shutdown_connections 2024-09-10T10:46:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.779+0000 7fb766600640 1 -- 172.21.15.28:0/188946842 wait complete. 2024-09-10T10:46:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.780+0000 7fb766600640 1 Processor -- start 2024-09-10T10:46:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.781+0000 7fb766600640 1 -- start start 2024-09-10T10:46:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb76006dd60 0x7fb76014fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7600d4180 0x7fb7600c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600cebd0 con 0x7fb76015e6a0 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb76006e950 con 0x7fb7600d4180 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb766600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600d0450 con 0x7fb76006dd60 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35884/0 (socket says 172.21.15.28:35884) 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 -- 172.21.15.28:0/1918802644 learned_addr learned my addr 172.21.15.28:0/1918802644 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765200640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb76006dd60 0x7fb76014fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb75fe00640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7600d4180 0x7fb7600c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 -- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb76006dd60 msgr2=0x7fb76014fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb76006dd60 0x7fb76014fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 -- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7600d4180 msgr2=0x7fb7600c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7600d4180 0x7fb7600c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.782+0000 7fb765c00640 1 -- 172.21.15.28:0/1918802644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7600b75b0 con 0x7fb76015e6a0 2024-09-10T10:46:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb765c00640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb75800c7a0 tx=0x7fb75800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003b70 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758003d30 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800aa90 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7600b7890 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.783+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7600b7e50 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.784+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7600d4560 con 0x7fb76015e6a0 2024-09-10T10:46:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.785+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb75800ac30 con 0x7fb76015e6a0 2024-09-10T10:46:26.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.786+0000 7fb75cc00640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 0x7fb74006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:26.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.786+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7fb75808aeb0 con 0x7fb76015e6a0 2024-09-10T10:46:26.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.786+0000 7fb765200640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 0x7fb74006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.787+0000 7fb765200640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 0x7fb74006e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7600d1390 tx=0x7fb754009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:26.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:26.790+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb758058f90 con 0x7fb76015e6a0 2024-09-10T10:46:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.015+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7600ce7d0 con 0x7fb76015e6a0 2024-09-10T10:46:27.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.015+0000 7fb75cc00640 1 -- 172.21.15.28:0/1918802644 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5223 (secure 0 0 0) 0x7fb75805d290 con 0x7fb76015e6a0 2024-09-10T10:46:27.017 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:27.017 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":36,"btime":"2024-09-10T10:46:26:592488+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:26.592478+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T10:46:27.018 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 36 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 msgr2=0x7fb74006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 0x7fb74006e320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb7600d1390 tx=0x7fb754009190 comp rx=0 tx=0).stop 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 msgr2=0x7fb7600b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb75800c7a0 tx=0x7fb75800cc60 comp rx=0 tx=0).stop 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 shutdown_connections 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb74006be60 0x7fb74006e320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb76015e6a0 0x7fb7600b8650 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7600d4180 0x7fb7600c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 --2- 172.21.15.28:0/1918802644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb76006dd60 0x7fb76014fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 >> 172.21.15.28:0/1918802644 conn(0x7fb76009d6f0 msgr2=0x7fb76015d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 shutdown_connections 2024-09-10T10:46:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.018+0000 7fb766600640 1 -- 172.21.15.28:0/1918802644 wait complete. 2024-09-10T10:46:27.037 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-10T10:46:27.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.194+0000 7f37e7200640 1 Processor -- start 2024-09-10T10:46:27.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.194+0000 7f37e7200640 1 -- start start 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37e7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37e00705c0 0x7f37e00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37e7200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e00b4a50 con 0x7f37e0070110 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37e7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e0070b10 con 0x7f37e015e760 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37e7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f37e00b5ea0 con 0x7f37e00705c0 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37d7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f37e0070110 0x7f37e006bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46456/0 (socket says 172.21.15.28:46456) 2024-09-10T10:46:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.195+0000 7f37d7e00640 1 -- 172.21.15.28:0/2728455611 learned_addr learned my addr 172.21.15.28:0/2728455611 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:27.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37e5200640 1 --2- 172.21.15.28:0/2728455611 >> v2:172.21.15.138:3300/0 conn(0x7f37e00705c0 0x7f37e00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4282717473 0 0) 0x7f37e00b4a50 con 0x7f37e0070110 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37c4003520 con 0x7f37e0070110 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260648297 0 0) 0x7f37e0070b10 con 0x7f37e015e760 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37e00b4a50 con 0x7f37e015e760 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37e5200640 1 -- 172.21.15.28:0/2728455611 >> v1:172.21.15.138:6789/0 conn(0x7f37e015e760 legacy=0x7f37e006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37e5200640 1 -- 172.21.15.28:0/2728455611 >> v1:172.21.15.28:6789/0 conn(0x7f37e0070110 legacy=0x7f37e006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428973673 0 0) 0x7f37c4003520 con 0x7f37e0070110 2024-09-10T10:46:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.196+0000 7f37e5200640 1 -- 172.21.15.28:0/2728455611 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0070df0 con 0x7f37e00705c0 2024-09-10T10:46:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.197+0000 7f37e5200640 1 --2- 172.21.15.28:0/2728455611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e00b6310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009d90 tx=0x7f37d802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6c12cc2edf362b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.197+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8001670 con 0x7f37e00705c0 2024-09-10T10:46:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.197+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8007540 con 0x7f37e00705c0 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.197+0000 7f37df400640 1 -- 172.21.15.28:0/2728455611 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80078f0 con 0x7f37e00705c0 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 -- 172.21.15.28:0/2728455611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 msgr2=0x7f37e00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 --2- 172.21.15.28:0/2728455611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e00b6310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009d90 tx=0x7f37d802a680 comp rx=0 tx=0).stop 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 -- 172.21.15.28:0/2728455611 shutdown_connections 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 --2- 172.21.15.28:0/2728455611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e00b6310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 -- 172.21.15.28:0/2728455611 >> 172.21.15.28:0/2728455611 conn(0x7f37e0073490 msgr2=0x7f37e00b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 -- 172.21.15.28:0/2728455611 shutdown_connections 2024-09-10T10:46:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.198+0000 7f37e7200640 1 -- 172.21.15.28:0/2728455611 wait complete. 2024-09-10T10:46:27.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.200+0000 7f37e7200640 1 Processor -- start 2024-09-10T10:46:27.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.200+0000 7f37e7200640 1 -- start start 2024-09-10T10:46:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 0x7f37e01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00b46e0 con 0x7f37e015e760 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00710d0 con 0x7f37e00705c0 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00ae240 con 0x7f37e0070110 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35896/0 (socket says 172.21.15.28:35896) 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 -- 172.21.15.28:0/1000784801 learned_addr learned my addr 172.21.15.28:0/1000784801 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 0x7f37e01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37d7e00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 -- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 msgr2=0x7f37e01587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 0x7f37e01587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 -- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 msgr2=0x7f37e0159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e0159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37e5200640 1 -- 172.21.15.28:0/1000784801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e0159110 con 0x7f37e015e760 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 0x7f37e01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.201+0000 7f37d7e00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:27.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37e5200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009eb0 tx=0x7f37d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37dcc00640 1 -- 172.21.15.28:0/1000784801 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8040070 con 0x7f37e015e760 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37dcc00640 1 -- 172.21.15.28:0/1000784801 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d8031c40 con 0x7f37e015e760 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37dcc00640 1 -- 172.21.15.28:0/1000784801 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8009450 con 0x7f37e015e760 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e0160d20 con 0x7f37e015e760 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.202+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e01612e0 con 0x7f37e015e760 2024-09-10T10:46:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.204+0000 7f37dcc00640 1 -- 172.21.15.28:0/1000784801 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37d8039040 con 0x7f37e015e760 2024-09-10T10:46:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.206+0000 7f37dcc00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 0x7f37bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.206+0000 7f37dcc00640 1 -- 172.21.15.28:0/1000784801 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7f37d80ac4a0 con 0x7f37e015e760 2024-09-10T10:46:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.206+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 0x7f37bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.206+0000 7f37ea1f0740 1 Processor -- start 2024-09-10T10:46:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 -- start start 2024-09-10T10:46:27.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 0x7f37bc06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37d00011e0 tx=0x7f37d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d32c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d33200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e064c50660 con 0x55e064ce8480 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e064bf8680 con 0x55e064d02800 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.207+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e064c4c620 con 0x55e064d02420 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35908/0 (socket says 172.21.15.28:35908) 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 -- 172.21.15.28:0/2225307447 learned_addr learned my addr 172.21.15.28:0/2225307447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37d7e00640 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d33200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37dfe00640 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d32c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 -- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 msgr2=0x55e064d32c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d32c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 -- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 msgr2=0x55e064d33200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d33200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 -- 172.21.15.28:0/2225307447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e064a2d7d0 con 0x55e064ce8480 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.208+0000 7f37e5200640 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37d8082b80 tx=0x7f37d807a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d114e6bda221123b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.209+0000 7f37d4c00640 1 -- 172.21.15.28:0/2225307447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8049030 con 0x55e064ce8480 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.209+0000 7f37d4c00640 1 -- 172.21.15.28:0/2225307447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d807a4d0 con 0x55e064ce8480 2024-09-10T10:46:27.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.209+0000 7f37d4c00640 1 -- 172.21.15.28:0/2225307447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d807e880 con 0x55e064ce8480 2024-09-10T10:46:27.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 -- 172.21.15.28:0/2225307447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 msgr2=0x55e064d33740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f37d8082b80 tx=0x7f37d807a3d0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 -- 172.21.15.28:0/2225307447 shutdown_connections 2024-09-10T10:46:27.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064ce8480 0x55e064d33740 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d33200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/2225307447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d32c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 -- 172.21.15.28:0/2225307447 >> 172.21.15.28:0/2225307447 conn(0x55e064ce9220 msgr2=0x55e064d31840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 -- 172.21.15.28:0/2225307447 shutdown_connections 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.210+0000 7f37ea1f0740 1 -- 172.21.15.28:0/2225307447 wait complete. 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.211+0000 7f37ea1f0740 1 -- start start 2024-09-10T10:46:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.211+0000 7f37ea1f0740 1 Processor -- start 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d17290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d27510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e064ce73e0 con 0x55e064d02800 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e064bd94c0 con 0x55e064d417b0 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e064ce76c0 con 0x55e064d02420 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37ea1f0740 10 client.0 _ll_register_callbacks cb 0x55e064c48e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35912/0 (socket says 172.21.15.28:35912) 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 -- 172.21.15.28:0/347426967 learned_addr learned my addr 172.21.15.28:0/347426967 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37e5200640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d27510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37dfe00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d17290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 -- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 msgr2=0x55e064d17290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d17290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 -- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 msgr2=0x55e064d27510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d27510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37d7e00640 1 -- 172.21.15.28:0/347426967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0649633f0 con 0x55e064d02800 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.212+0000 7f37e5200640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d27510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37d7e00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f37c80011e0 tx=0x7f37c8011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f10157ed5a7ede1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37cd600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8005ae0 con 0x55e064d02800 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7f37c8005c80 con 0x55e064d02800 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d807a4d0 con 0x55e064d02800 2024-09-10T10:46:27.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f37c8022050 con 0x55e064d02800 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37ea1f0740 10 client.5664 fetch_fsmap learned FSMap version 36 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e0649230f0 con 0x55e064d02800 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.213+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d807a4d0 con 0x55e064d02800 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f37c8025020 con 0x55e064d02800 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 10 client.5664 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 10 client.5664 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 20 client.5664 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d2c340 0x55e064d2da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 4 client.5664 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 --> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e064c5dd90 con 0x55e064d2c340 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.214+0000 7f37dfe00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d2c340 0x55e064d2da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.215+0000 7f37dfe00640 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d2c340 0x55e064d2da60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d17e088a0cd07d44 server_cookie=86eb8a1d07c51952 in_seq=0 out_seq=0 2024-09-10T10:46:27.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.215+0000 7f37cd600640 10 client.5664 ms_handle_connect on v2:172.21.15.28:6834/3042558016 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.215+0000 7f37cd600640 1 -- 172.21.15.28:0/347426967 <== mds.0 v2:172.21.15.28:6834/3042558016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e064c5dd90 con 0x55e064d2c340 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37cd600640 10 client.5664 handle_command_reply: tid=0 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 client.5664 shutdown 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 msgr2=0x55e064ced200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f37c80011e0 tx=0x7f37c8011960 comp rx=0 tx=0).stop 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 shutdown_connections 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d2c340 0x55e064d2da60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d27510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02800 0x55e064ced200 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/347426967 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02420 0x55e064d17290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 >> 172.21.15.28:0/347426967 conn(0x55e064ce8480 msgr2=0x55e064d2fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37cd600640 0 client.5664 ms_handle_reset on v2:172.21.15.28:6834/3042558016 2024-09-10T10:46:27.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 shutdown_connections 2024-09-10T10:46:27.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 1 -- 172.21.15.28:0/347426967 wait complete. 2024-09-10T10:46:27.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.216+0000 7f37ea1f0740 20 client.5664 trim_cache size 0 max 16384 2024-09-10T10:46:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.231+0000 7f37ea1f0740 1 Processor -- start 2024-09-10T10:46:27.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.231+0000 7f37ea1f0740 1 -- start start 2024-09-10T10:46:27.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d34a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d417b0 0x55e064d34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0201040 con 0x55e064d02420 2024-09-10T10:46:27.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37e0200040 con 0x55e064d02800 2024-09-10T10:46:27.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37e015fbe0 con 0x55e064d417b0 2024-09-10T10:46:27.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35918/0 (socket says 172.21.15.28:35918) 2024-09-10T10:46:27.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.232+0000 7f37dfe00640 1 -- 172.21.15.28:0/1108979858 learned_addr learned my addr 172.21.15.28:0/1108979858 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 -- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d417b0 msgr2=0x55e064d34fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d417b0 0x55e064d34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 -- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 msgr2=0x55e064d34a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d34a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 -- 172.21.15.28:0/1108979858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e064ce8990 con 0x55e064d02420 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37dfe00640 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f37d00051e0 tx=0x7f37d0017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3766b76f93cc144 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37cfe00640 1 -- 172.21.15.28:0/1108979858 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00037f0 con 0x55e064d02420 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37cfe00640 1 -- 172.21.15.28:0/1108979858 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d0003990 con 0x55e064d02420 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.233+0000 7f37cfe00640 1 -- 172.21.15.28:0/1108979858 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0003b40 con 0x55e064d02420 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 -- 172.21.15.28:0/1108979858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 msgr2=0x55e064d33840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f37d00051e0 tx=0x7f37d0017040 comp rx=0 tx=0).stop 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 -- 172.21.15.28:0/1108979858 shutdown_connections 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d417b0 0x55e064d34fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d02800 0x55e064d34a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/1108979858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064d33840 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 -- 172.21.15.28:0/1108979858 >> 172.21.15.28:0/1108979858 conn(0x55e064ce8480 msgr2=0x55e064ced0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 -- 172.21.15.28:0/1108979858 shutdown_connections 2024-09-10T10:46:27.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.234+0000 7f37ea1f0740 1 -- 172.21.15.28:0/1108979858 wait complete. 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.235+0000 7f37ea1f0740 1 -- start start 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.235+0000 7f37ea1f0740 1 Processor -- start 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02800 0x55e064d2a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d2ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e064d2fcb0 con 0x55e064d02420 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37e01ffa20 con 0x55e064d417b0 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e064d303e0 con 0x55e064d02800 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37ea1f0740 10 client.0 _ll_register_callbacks cb 0x7f37c8001590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35926/0 (socket says 172.21.15.28:35926) 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 -- 172.21.15.28:0/335623509 learned_addr learned my addr 172.21.15.28:0/335623509 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37e5200640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d2ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37d7e00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02800 0x55e064d2a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 -- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02800 msgr2=0x55e064d2a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02800 0x55e064d2a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 -- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 msgr2=0x55e064d2ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.236+0000 7f37dfe00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d2ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37dfe00640 1 -- 172.21.15.28:0/335623509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e064ceb000 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37dfe00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005600 tx=0x7f37d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e445ef6fc5c10104 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37cd600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d0003e10 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7f37d0025430 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d0003990 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f37d0025630 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37ea1f0740 10 client.5670 fetch_fsmap learned FSMap version 36 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.237+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e064db1320 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d0003990 con 0x55e064d02420 2024-09-10T10:46:27.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f37d003f080 con 0x55e064d02420 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 10 client.5670 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 10 client.5670 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 20 client.5670 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d01310 0x55e064d2d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 4 client.5670 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 --> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e064c580e0 con 0x55e064d01310 2024-09-10T10:46:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37d7e00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d01310 0x55e064d2d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37d7e00640 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d01310 0x55e064d2d7e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3b8b70cd93ede2 server_cookie=f1a3ae0f1b65c755 in_seq=0 out_seq=0 2024-09-10T10:46:27.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.238+0000 7f37cd600640 10 client.5670 ms_handle_connect on v2:172.21.15.28:6834/3042558016 2024-09-10T10:46:27.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37cd600640 1 -- 172.21.15.28:0/335623509 <== mds.0 v2:172.21.15.28:6834/3042558016 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e064c580e0 con 0x55e064d01310 2024-09-10T10:46:27.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37cd600640 10 client.5670 handle_command_reply: tid=0 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 client.5670 shutdown 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 msgr2=0x55e064daf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f37d0005600 tx=0x7f37d0011000 comp rx=0 tx=0).stop 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 shutdown_connections 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:6834/3042558016,v1:172.21.15.28:6835/3042558016] conn(0x55e064d01310 0x55e064d2d7e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e064d417b0 0x55e064d2ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e064d02800 0x55e064d2a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 --2- 172.21.15.28:0/335623509 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e064d02420 0x55e064daf680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37cd600640 0 client.5670 ms_handle_reset on v2:172.21.15.28:6834/3042558016 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 >> 172.21.15.28:0/335623509 conn(0x55e064ce8480 msgr2=0x55e064d17fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 shutdown_connections 2024-09-10T10:46:27.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 1 -- 172.21.15.28:0/335623509 wait complete. 2024-09-10T10:46:27.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.239+0000 7f37ea1f0740 20 client.5670 trim_cache size 0 max 16384 2024-09-10T10:46:27.246 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:27.246 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:46:27.246 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:46:27.246 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 msgr2=0x7f37bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 0x7f37bc06e2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f37d00011e0 tx=0x7f37d0009040 comp rx=0 tx=0).stop 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 msgr2=0x7f37e015a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009eb0 tx=0x7f37d8004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 shutdown_connections 2024-09-10T10:46:27.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37bc06be30 0x7f37bc06e2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37e015e760 0x7f37e015a250 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37e00705c0 0x7f37e0159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 --2- 172.21.15.28:0/1000784801 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37e0070110 0x7f37e01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 >> 172.21.15.28:0/1000784801 conn(0x7f37e0073490 msgr2=0x7f37e00aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 shutdown_connections 2024-09-10T10:46:27.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.240+0000 7f37e7200640 1 -- 172.21.15.28:0/1000784801 wait complete. 2024-09-10T10:46:27.257 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:27.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.411+0000 7fe20e400640 1 Processor -- start 2024-09-10T10:46:27.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.412+0000 7fe20e400640 1 -- start start 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe20e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 0x7fe208169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe20e400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2080d2c20 con 0x7fe20815e6a0 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe20e400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe20806e390 con 0x7fe2080d4180 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe20e400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2080cefb0 con 0x7fe2080d0450 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe207e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe20815e6a0 0x7fe20806d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46460/0 (socket says 172.21.15.28:46460) 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe207e00640 1 -- 172.21.15.28:0/3774390022 learned_addr learned my addr 172.21.15.28:0/3774390022 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:27.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe20da00640 1 --2- 172.21.15.28:0/3774390022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 0x7fe208169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.413+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239386803 0 0) 0x7fe2080d2c20 con 0x7fe20815e6a0 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1ec003520 con 0x7fe20815e6a0 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 633652389 0 0) 0x7fe20806e390 con 0x7fe2080d4180 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2080d2c20 con 0x7fe2080d4180 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe20da00640 1 -- 172.21.15.28:0/3774390022 >> v1:172.21.15.138:6789/0 conn(0x7fe2080d4180 legacy=0x7fe2080d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe20da00640 1 -- 172.21.15.28:0/3774390022 >> v1:172.21.15.28:6789/0 conn(0x7fe20815e6a0 legacy=0x7fe20806d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.414+0000 7fe20da00640 1 -- 172.21.15.28:0/3774390022 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20806e670 con 0x7fe2080d0450 2024-09-10T10:46:27.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.415+0000 7fe20da00640 1 --2- 172.21.15.28:0/3774390022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 0x7fe208169000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe200009dc0 tx=0x7fe20002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4672c5371abcec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.415+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20002ae60 con 0x7fe2080d0450 2024-09-10T10:46:27.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.415+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe200007740 con 0x7fe2080d0450 2024-09-10T10:46:27.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.415+0000 7fe207400640 1 -- 172.21.15.28:0/3774390022 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200007af0 con 0x7fe2080d0450 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 -- 172.21.15.28:0/3774390022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 msgr2=0x7fe208169000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 --2- 172.21.15.28:0/3774390022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 0x7fe208169000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe200009dc0 tx=0x7fe20002a640 comp rx=0 tx=0).stop 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 -- 172.21.15.28:0/3774390022 shutdown_connections 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 --2- 172.21.15.28:0/3774390022 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2080d0450 0x7fe208169000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 -- 172.21.15.28:0/3774390022 >> 172.21.15.28:0/3774390022 conn(0x7fe20809d6f0 msgr2=0x7fe20808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 -- 172.21.15.28:0/3774390022 shutdown_connections 2024-09-10T10:46:27.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.416+0000 7fe20e400640 1 -- 172.21.15.28:0/3774390022 wait complete. 2024-09-10T10:46:27.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.417+0000 7fe20e400640 1 Processor -- start 2024-09-10T10:46:27.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.418+0000 7fe20e400640 1 -- start start 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 0x7fe2080cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20815e6a0 0x7fe2080cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe20806df30 con 0x7fe2080d0450 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2080cebd0 con 0x7fe2080d4180 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20e400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe20806e0b0 con 0x7fe20815e6a0 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35934/0 (socket says 172.21.15.28:35934) 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 -- 172.21.15.28:0/4177903769 learned_addr learned my addr 172.21.15.28:0/4177903769 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20da00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20815e6a0 0x7fe2080cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe207e00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 0x7fe2080cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 -- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20815e6a0 msgr2=0x7fe2080cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20815e6a0 0x7fe2080cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 -- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 msgr2=0x7fe2080cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 0x7fe2080cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe20d000640 1 -- 172.21.15.28:0/4177903769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe208159f90 con 0x7fe2080d0450 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.419+0000 7fe207e00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 0x7fe2080cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe20d000640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c430 tx=0x7fe1f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8017070 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f80052f0 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20815a270 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8003830 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.420+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2080b7a30 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.421+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2080d4560 con 0x7fe2080d0450 2024-09-10T10:46:27.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.422+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe1f801a070 con 0x7fe2080d0450 2024-09-10T10:46:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.423+0000 7fe204c00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 0x7fe1e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.424+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6187+0+0 (secure 0 0 0) 0x7fe1f808a500 con 0x7fe2080d0450 2024-09-10T10:46:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.424+0000 7fe207e00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 0x7fe1e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.425+0000 7fe207e00640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 0x7fe1e806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe208154da0 tx=0x7fe1f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.428+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe1f8058cc0 con 0x7fe2080d0450 2024-09-10T10:46:27.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.653+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2080b1af0 con 0x7fe2080d0450 2024-09-10T10:46:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.653+0000 7fe204c00640 1 -- 172.21.15.28:0/4177903769 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fe1f80585c0 con 0x7fe2080d0450 2024-09-10T10:46:27.656 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:27.656 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-10T10:46:27:594648+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:26.606535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T10:46:27.656 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 msgr2=0x7fe1e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 0x7fe1e806e2f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe208154da0 tx=0x7fe1f4009040 comp rx=0 tx=0).stop 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 msgr2=0x7fe2080b9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800c430 tx=0x7fe1f800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 shutdown_connections 2024-09-10T10:46:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe1e806be30 0x7fe1e806e2f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe20815e6a0 0x7fe2080cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2080d4180 0x7fe2080cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 --2- 172.21.15.28:0/4177903769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2080d0450 0x7fe2080b9c30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 >> 172.21.15.28:0/4177903769 conn(0x7fe20809d6f0 msgr2=0x7fe2081ea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 shutdown_connections 2024-09-10T10:46:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.656+0000 7fe20e400640 1 -- 172.21.15.28:0/4177903769 wait complete. 2024-09-10T10:46:27.673 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.828+0000 7fa32f000640 1 Processor -- start 2024-09-10T10:46:27.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.828+0000 7fa32f000640 1 -- start start 2024-09-10T10:46:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32f000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3280d0dd0 0x7fa3280d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3280cf000 con 0x7fa3280d1f90 2024-09-10T10:46:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32f000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3280b37d0 con 0x7fa3280d0dd0 2024-09-10T10:46:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280cf330 con 0x7fa32815e660 2024-09-10T10:46:27.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.829+0000 7fa32d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3280d0dd0 0x7fa3280d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:40452/0 (socket says 172.21.15.28:40452) 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32e600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa3280d1f90 0x7fa3280cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:54250/0 (socket says 172.21.15.28:54250) 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32d200640 1 -- 172.21.15.28:0/328113912 learned_addr learned my addr 172.21.15.28:0/328113912 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa327e00640 1 -- 172.21.15.28:0/328113912 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 495304664 0 0) 0x7fa3280cf000 con 0x7fa3280d1f90 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa327e00640 1 -- 172.21.15.28:0/328113912 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa310003520 con 0x7fa3280d1f90 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32dc00640 1 -- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 msgr2=0x7fa32806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32dc00640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32dc00640 1 -- 172.21.15.28:0/328113912 >> v1:172.21.15.138:6789/0 conn(0x7fa3280d1f90 legacy=0x7fa3280cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32dc00640 1 -- 172.21.15.28:0/328113912 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3280d16f0 con 0x7fa3280d0dd0 2024-09-10T10:46:27.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.830+0000 7fa32d200640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:27.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.831+0000 7fa32dc00640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280d11b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009d90 tx=0x7fa31c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ecb728e2285d06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.831+0000 7fa327e00640 1 -- 172.21.15.28:0/328113912 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c007740 con 0x7fa3280d0dd0 2024-09-10T10:46:27.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa327e00640 1 -- 172.21.15.28:0/328113912 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c0078e0 con 0x7fa3280d0dd0 2024-09-10T10:46:27.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa327e00640 1 -- 172.21.15.28:0/328113912 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c007cb0 con 0x7fa3280d0dd0 2024-09-10T10:46:27.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 -- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 msgr2=0x7fa3280d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280d11b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009d90 tx=0x7fa31c02a660 comp rx=0 tx=0).stop 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 -- 172.21.15.28:0/328113912 shutdown_connections 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa32806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 --2- 172.21.15.28:0/328113912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280d11b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 -- 172.21.15.28:0/328113912 >> 172.21.15.28:0/328113912 conn(0x7fa32808bfc0 msgr2=0x7fa3280d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 -- 172.21.15.28:0/328113912 shutdown_connections 2024-09-10T10:46:27.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.832+0000 7fa32f000640 1 -- 172.21.15.28:0/328113912 wait complete. 2024-09-10T10:46:27.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.834+0000 7fa32f000640 1 Processor -- start 2024-09-10T10:46:27.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.834+0000 7fa32f000640 1 -- start start 2024-09-10T10:46:27.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa3281e4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280d2f40 con 0x7fa3280d1f90 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3280d2c60 con 0x7fa3280d0dd0 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3280d3220 con 0x7fa32815e660 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35946/0 (socket says 172.21.15.28:35946) 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32d200640 1 -- 172.21.15.28:0/3039561611 learned_addr learned my addr 172.21.15.28:0/3039561611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:27.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:34838/0 (socket says 172.21.15.28:34838) 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.835+0000 7fa32e600640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa3281e4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 -- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 msgr2=0x7fa3281e4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa3281e4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 -- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 msgr2=0x7fa3280cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 -- 172.21.15.28:0/3039561611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3281e5940 con 0x7fa3280d1f90 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32e600640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa3281e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32dc00640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32d200640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa314006cc0 tx=0x7fa31400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314017020 con 0x7fa3280d1f90 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa314004240 con 0x7fa3280d1f90 2024-09-10T10:46:27.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31400c730 con 0x7fa3280d1f90 2024-09-10T10:46:27.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3281e5c20 con 0x7fa3280d1f90 2024-09-10T10:46:27.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.836+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa328156420 con 0x7fa3280d1f90 2024-09-10T10:46:27.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.837+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3280d2440 con 0x7fa3280d1f90 2024-09-10T10:46:27.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.838+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa31401a020 con 0x7fa3280d1f90 2024-09-10T10:46:27.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.839+0000 7fa325600640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:27.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.839+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6216+0+0 (secure 0 0 0) 0x7fa31408aff0 con 0x7fa3280d1f90 2024-09-10T10:46:27.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.839+0000 7fa32dc00640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:27.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.840+0000 7fa32dc00640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 0x7fa30806e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa31c034000 tx=0x7fa31c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:27.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:27.843+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa314021030 con 0x7fa3280d1f90 2024-09-10T10:46:28.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.065+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3280d1b20 con 0x7fa3280d1f90 2024-09-10T10:46:28.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.065+0000 7fa325600640 1 -- 172.21.15.28:0/3039561611 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fa3140590b0 con 0x7fa3280d1f90 2024-09-10T10:46:28.068 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:28.068 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-10T10:46:27:594648+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:26.606535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T10:46:28.068 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 msgr2=0x7fa30806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 0x7fa30806e2f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa31c034000 tx=0x7fa31c02f000 comp rx=0 tx=0).stop 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 msgr2=0x7fa328155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa314006cc0 tx=0x7fa31400ed60 comp rx=0 tx=0).stop 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 shutdown_connections 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa30806be30 0x7fa30806e2f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa32815e660 0x7fa3281e4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3280d1f90 0x7fa328155ee0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 --2- 172.21.15.28:0/3039561611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3280d0dd0 0x7fa3280cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 >> 172.21.15.28:0/3039561611 conn(0x7fa32808bfc0 msgr2=0x7fa3280b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 shutdown_connections 2024-09-10T10:46:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.068+0000 7fa32f000640 1 -- 172.21.15.28:0/3039561611 wait complete. 2024-09-10T10:46:28.086 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.239+0000 7f1efa000640 1 Processor -- start 2024-09-10T10:46:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.239+0000 7f1efa000640 1 -- start start 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ef415b300 0x7f1ef40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef40cd3f0 0x7f1ef406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ef4072d30 0x7f1ef406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef40728a0 con 0x7f1ef415b300 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ef40cdbe0 con 0x7f1ef4072d30 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.240+0000 7f1efa000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef40cd900 con 0x7f1ef40cd3f0 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ef415b300 0x7f1ef40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ef415b300 0x7f1ef40cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35950/0 (socket says 172.21.15.28:35950) 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 -- 172.21.15.28:0/3121138578 learned_addr learned my addr 172.21.15.28:0/3121138578 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef2a00640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef40cd3f0 0x7f1ef406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3e00640 1 --2- 172.21.15.28:0/3121138578 >> v2:172.21.15.138:3300/0 conn(0x7f1ef4072d30 0x7f1ef406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 -- 172.21.15.28:0/3121138578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef40cd3f0 msgr2=0x7f1ef406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef40cd3f0 0x7f1ef406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 -- 172.21.15.28:0/3121138578 >> v2:172.21.15.138:3300/0 conn(0x7f1ef4072d30 msgr2=0x7f1ef406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 --2- 172.21.15.28:0/3121138578 >> v2:172.21.15.138:3300/0 conn(0x7f1ef4072d30 0x7f1ef406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.241+0000 7f1ef3400640 1 -- 172.21.15.28:0/3121138578 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef4072250 con 0x7f1ef415b300 2024-09-10T10:46:28.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.242+0000 7f1ef3400640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40cceb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4072040 tx=0x7f1ee402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efce49fee182a8d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.242+0000 7f1ef2000640 1 -- 172.21.15.28:0/3121138578 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4037070 con 0x7f1ef415b300 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.242+0000 7f1ef2000640 1 -- 172.21.15.28:0/3121138578 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ee4032440 con 0x7f1ef415b300 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.242+0000 7f1ef2000640 1 -- 172.21.15.28:0/3121138578 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4032810 con 0x7f1ef415b300 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 -- 172.21.15.28:0/3121138578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 msgr2=0x7f1ef40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40cceb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4072040 tx=0x7f1ee402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 -- 172.21.15.28:0/3121138578 shutdown_connections 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 --2- 172.21.15.28:0/3121138578 >> v2:172.21.15.138:3300/0 conn(0x7f1ef4072d30 0x7f1ef406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef40cd3f0 0x7f1ef406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 --2- 172.21.15.28:0/3121138578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40cceb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 -- 172.21.15.28:0/3121138578 >> 172.21.15.28:0/3121138578 conn(0x7f1ef4067d70 msgr2=0x7f1ef406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 -- 172.21.15.28:0/3121138578 shutdown_connections 2024-09-10T10:46:28.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.243+0000 7f1efa000640 1 -- 172.21.15.28:0/3121138578 wait complete. 2024-09-10T10:46:28.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.245+0000 7f1efa000640 1 Processor -- start 2024-09-10T10:46:28.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.245+0000 7f1efa000640 1 -- start start 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef4072d30 0x7f1ef4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 0x7f1ef40c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef40d0160 con 0x7f1ef415b300 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef40cffe0 con 0x7f1ef40cd3f0 2024-09-10T10:46:28.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1efa000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ef40b3c40 con 0x7f1ef4072d30 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1ef3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1ef3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35954/0 (socket says 172.21.15.28:35954) 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1ef3e00640 1 -- 172.21.15.28:0/1145423342 learned_addr learned my addr 172.21.15.28:0/1145423342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1ef3400640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef4072d30 0x7f1ef4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.246+0000 7f1ef2a00640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 0x7f1ef40c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 -- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef4072d30 msgr2=0x7f1ef4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef4072d30 0x7f1ef4147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 -- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 msgr2=0x7f1ef40c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 0x7f1ef40c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 -- 172.21.15.28:0/1145423342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef4148190 con 0x7f1ef415b300 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef2a00640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 0x7f1ef40c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1ef3e00640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1eec007e60 tx=0x7f1eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:28.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec014620 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec019070 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec014a20 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef40ca350 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.247+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef40ca910 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.248+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ef415a7e0 con 0x7f1ef415b300 2024-09-10T10:46:28.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.249+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1eec014bc0 con 0x7f1ef415b300 2024-09-10T10:46:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.250+0000 7f1eeb400640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 0x7f1ed406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:28.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.251+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6216+0+0 (secure 0 0 0) 0x7f1eec08b8e0 con 0x7f1ef415b300 2024-09-10T10:46:28.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.251+0000 7f1ef3400640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 0x7f1ed406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:28.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.252+0000 7f1ef3400640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 0x7f1ed406e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4034000 tx=0x7f1ee402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:28.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.255+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1eec05a080 con 0x7f1ef415b300 2024-09-10T10:46:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.474+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ef4151e00 con 0x7f1ef415b300 2024-09-10T10:46:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.474+0000 7f1eeb400640 1 -- 172.21.15.28:0/1145423342 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7f1eec059980 con 0x7f1ef415b300 2024-09-10T10:46:28.477 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:28.477 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"btime":"2024-09-10T10:46:27:594648+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:26.606535+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T10:46:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 37 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.476+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 msgr2=0x7f1ed406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.476+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 0x7f1ed406e320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4034000 tx=0x7f1ee402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 msgr2=0x7f1ef40c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1eec007e60 tx=0x7f1eec00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 shutdown_connections 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1ed406be60 0x7f1ed406e320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ef415b300 0x7f1ef40c9a10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ef40cd3f0 0x7f1ef40c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 --2- 172.21.15.28:0/1145423342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ef4072d30 0x7f1ef4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 >> 172.21.15.28:0/1145423342 conn(0x7f1ef4067d70 msgr2=0x7f1ef41597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 shutdown_connections 2024-09-10T10:46:28.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:28.477+0000 7f1efa000640 1 -- 172.21.15.28:0/1145423342 wait complete. 2024-09-10T10:46:28.494 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump tree /a 0 2024-09-10T10:46:28.670 INFO:teuthology.orchestra.run.smithi028.stdout:[{"ino":1099511627776,"rdev":0,"ctime":"2024-09-10T10:46:22.478078+0000","btime":"2024-09-10T10:46:16.380243+0000","mode":16877,"uid":1000,"gid":1333,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-09-10T10:46:22.478078+0000","atime":"2024-09-10T10:46:16.380243+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-09-10T10:46:22.478078+0000","num_files":0,"num_subdirs":1,"change_attr":3},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T10:46:22.478078+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":2,"rsnaps":0,"rctime":"2024-09-10T10:46:22.478078+0000"},"version":9,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[{"key":"security.selinux","val":"unconfined_u:object_r:unlabeled_t:s0\u0000"}],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":1,"dirfrag":1,"dirty":1},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"],"client_caps":[],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[{"path":"/a","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1},"nref":1,"dentries":[{"path":"a/c","alternate_name":"","path_ino":1,"snap_first":18446744073709551606,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627777,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"inodepin":1,"dirty":1},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"a/b","alternate_name":"","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":false,"is_remote":false,"is_null":true,"is_new":false,"inode":0,"remote_type":"","version":5,"projected_version":5,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirty":1},"nref":1,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-09-10T10:46:28.671 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-09-10T10:46:22.478078+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-09-10T10:46:16.380243+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-09-10T10:46:16.380243+0000", "change_attr": 3, "client_caps": [], "client_ranges": [], "ctime": "2024-09-10T10:46:22.478078+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "a/c", "path_ino": 1, "pins": { "dirty": 1, "inodepin": 1 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 18446744073709551606, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "alternate_name": "", "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 0, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": true, "is_primary": false, "is_remote": false, "lock": {}, "nref": 1, "path": "a/b", "path_ino": 1, "pins": { "dirty": 1 }, "projected_version": 5, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 5, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/a", "pins": { "child": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-09-10T10:46:22.478078+0000", "num_files": 0, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1333, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-09-10T10:46:22.478078+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "dirfrag": 1, "dirty": 1, "dirtyscattered": 1 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-09-10T10:46:22.478078+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 9, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [ { "key": "security.selinux", "val": "unconfined_u:object_r:unlabeled_t:s0\u0000" } ] } ] 2024-09-10T10:46:28.672 DEBUG:tasks.cephfs.test_damage:[{'ino': 1099511627776, 'rdev': 0, 'ctime': '2024-09-10T10:46:22.478078+0000', 'btime': '2024-09-10T10:46:16.380243+0000', 'mode': 16877, 'uid': 1000, 'gid': 1333, 'nlink': 1, 'dir_layout': {'dir_hash': 2, 'unused1': 0, 'unused2': 0, 'unused3': 0}, 'layout': {'stripe_unit': 0, 'stripe_count': 0, 'object_size': 0, 'pool_id': -1, 'pool_ns': ''}, 'old_pools': [], 'size': 0, 'truncate_seq': 1, 'truncate_size': 18446744073709551615, 'truncate_from': 0, 'truncate_pending': 0, 'mtime': '2024-09-10T10:46:22.478078+0000', 'atime': '2024-09-10T10:46:16.380243+0000', 'time_warp_seq': 0, 'change_attr': 3, 'export_pin': -1, 'export_ephemeral_random_pin': 0, 'export_ephemeral_distributed_pin': False, 'quiesce_block': False, 'client_ranges': [], 'dirstat': {'version': 0, 'mtime': '2024-09-10T10:46:22.478078+0000', 'num_files': 0, 'num_subdirs': 1, 'change_attr': 3}, 'rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T10:46:22.478078+0000'}, 'accounted_rstat': {'version': 0, 'rbytes': 0, 'rfiles': 0, 'rsubdirs': 2, 'rsnaps': 0, 'rctime': '2024-09-10T10:46:22.478078+0000'}, 'version': 9, 'file_data_version': 0, 'xattr_version': 1, 'backtrace_version': 2, 'stray_prior_path': '', 'max_size_ever': 0, 'quota': {'max_bytes': 0, 'max_files': 0}, 'last_scrub_stamp': '0.000000', 'last_scrub_version': 0, 'symlink': '', 'xattrs': [{'key': 'security.selinux', 'val': 'unconfined_u:object_r:unlabeled_t:s0\x00'}], 'dirfragtree': {'splits': []}, 'old_inodes': [], 'oldest_snap': 18446744073709551614, 'damage_flags': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirtyscattered': 1, 'dirfrag': 1, 'dirty': 1}, 'nref': 3, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'iversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'authlock': {}, 'linklock': {}, 'dirfragtreelock': {}, 'filelock': {}, 'xattrlock': {}, 'snaplock': {}, 'nestlock': {'gather_set': [], 'state': 'lock', 'type': 'inest', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'flocklock': {}, 'policylock': {}, 'quiescelock': {'gather_set': [], 'state': 'lock', 'type': 'iquiesce', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty'], 'client_caps': [], 'loner': -1, 'want_loner': -1, 'mds_caps_wanted': [], 'dirfrags': [{'path': '/a', 'dirfrag': '0x10000000000', 'snapid_first': 2, 'projected_version': '7', 'version': '7', 'committing_version': '0', 'committed_version': '0', 'is_rep': False, 'dir_auth': '', 'states': ['auth'], 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'auth_pins': 0, 'is_frozen': False, 'is_freezing': False, 'pins': {'child': 1}, 'nref': 1, 'dentries': [{'path': 'a/c', 'alternate_name': '', 'path_ino': 1, 'snap_first': 18446744073709551606, 'snap_last': 18446744073709551614, 'is_primary': True, 'is_remote': False, 'is_null': False, 'is_new': False, 'inode': 1099511627777, 'remote_type': '', 'version': 4, 'projected_version': 4, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'inodepin': 1, 'dirty': 1}, 'nref': 2, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}, {'path': 'a/b', 'alternate_name': '', 'path_ino': 1, 'snap_first': 2, 'snap_last': 18446744073709551614, 'is_primary': False, 'is_remote': False, 'is_null': True, 'is_new': False, 'inode': 0, 'remote_type': '', 'version': 5, 'projected_version': 5, 'auth_pins': 0, 'is_auth': True, 'auth_state': {'replicas': {}}, 'replica_state': {'authority': [0, -2], 'replica_nonce': 0}, 'is_frozen': False, 'is_freezing': False, 'pins': {'dirty': 1}, 'nref': 1, 'lock': {}, 'versionlock': {'gather_set': [], 'state': 'lock', 'type': 'dversion', 'is_leased': False, 'num_rdlocks': 0, 'num_wrlocks': 0, 'num_xlocks': 0, 'xlock_by': {}}, 'states': ['auth', 'dirty']}]}], 'is_subvolume': False}] 2024-09-10T10:46:28.672 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:46:28.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:28.761 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:28.845 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:46:28.846 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:46:28.846 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:28.846 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:28.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:28.882 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:46:28.882 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:28.882 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:28.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:28.920 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:46:28.920 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:28.920 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:28.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:28.957 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:46:28.957 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:28.958 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:28.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:28.996 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:29.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.150+0000 7f0043e00640 1 Processor -- start 2024-09-10T10:46:29.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.150+0000 7f0043e00640 1 -- start start 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.151+0000 7f0043e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 0x7f00440b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.151+0000 7f0043e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00440b25c0 0x7f00440aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0043e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00440b5470 con 0x7f00440af1b0 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0043e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f00440b3640 con 0x7f00440b25c0 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0043e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00440b5190 con 0x7f004415bef0 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0043400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f00440af1b0 0x7f0044071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46476/0 (socket says 172.21.15.28:46476) 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f003be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00440b25c0 0x7f00440aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0043400640 1 -- 172.21.15.28:0/1873698441 learned_addr learned my addr 172.21.15.28:0/1873698441 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:29.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0042a00640 1 --2- 172.21.15.28:0/1873698441 >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 0x7f00440b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f003be00640 1 -- 172.21.15.28:0/1873698441 >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 msgr2=0x7f00440b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f003be00640 1 --2- 172.21.15.28:0/1873698441 >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 0x7f00440b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f003be00640 1 -- 172.21.15.28:0/1873698441 >> v1:172.21.15.28:6789/0 conn(0x7f00440af1b0 legacy=0x7f0044071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f003be00640 1 -- 172.21.15.28:0/1873698441 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00440b38e0 con 0x7f00440b25c0 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0042000640 1 -- 172.21.15.28:0/1873698441 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016244148 0 0) 0x7f00440b5470 con 0x7f00440af1b0 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.152+0000 7f0042a00640 1 --2- 172.21.15.28:0/1873698441 >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 0x7f00440b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.153+0000 7f003be00640 1 --2- 172.21.15.28:0/1873698441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440b25c0 0x7f00440aec70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f004406cbb0 tx=0x7f002c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac4a97fdf95dc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0042000640 1 -- 172.21.15.28:0/1873698441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c037070 con 0x7f00440b25c0 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0042000640 1 -- 172.21.15.28:0/1873698441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c032440 con 0x7f00440b25c0 2024-09-10T10:46:29.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0042000640 1 -- 172.21.15.28:0/1873698441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c02ae60 con 0x7f00440b25c0 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 -- 172.21.15.28:0/1873698441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440b25c0 msgr2=0x7f00440aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 --2- 172.21.15.28:0/1873698441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440b25c0 0x7f00440aec70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f004406cbb0 tx=0x7f002c02a110 comp rx=0 tx=0).stop 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 -- 172.21.15.28:0/1873698441 shutdown_connections 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 --2- 172.21.15.28:0/1873698441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440b25c0 0x7f00440aec70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 --2- 172.21.15.28:0/1873698441 >> v2:172.21.15.138:3300/0 conn(0x7f004415bef0 0x7f00440b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 -- 172.21.15.28:0/1873698441 >> 172.21.15.28:0/1873698441 conn(0x7f004406c240 msgr2=0x7f00440b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 -- 172.21.15.28:0/1873698441 shutdown_connections 2024-09-10T10:46:29.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.154+0000 7f0043e00640 1 -- 172.21.15.28:0/1873698441 wait complete. 2024-09-10T10:46:29.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.156+0000 7f0043e00640 1 Processor -- start 2024-09-10T10:46:29.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.156+0000 7f0043e00640 1 -- start start 2024-09-10T10:46:29.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00440b25c0 0x7f004409c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 0x7f004409c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00440b4e60 con 0x7f00440af1b0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00440af640 con 0x7f004415bef0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00440b4b10 con 0x7f00440b25c0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0042a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0042a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35972/0 (socket says 172.21.15.28:35972) 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0042a00640 1 -- 172.21.15.28:0/1925171666 learned_addr learned my addr 172.21.15.28:0/1925171666 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f003be00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00440b25c0 0x7f004409c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.157+0000 7f0043400640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 0x7f004409c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 -- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00440b25c0 msgr2=0x7f004409c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00440b25c0 0x7f004409c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 -- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 msgr2=0x7f004409c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 0x7f004409c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 -- 172.21.15.28:0/1925171666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441f3f00 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0043400640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 0x7f004409c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0042a00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f003400d7d0 tx=0x7f003400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034004030 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00340041d0 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00340096a0 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441f41a0 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.158+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441f4760 con 0x7f00440af1b0 2024-09-10T10:46:29.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.159+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440b2a30 con 0x7f00440af1b0 2024-09-10T10:46:29.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.160+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0034009840 con 0x7f00440af1b0 2024-09-10T10:46:29.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.161+0000 7f003aa00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 0x7f002006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.161+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f003408b2f0 con 0x7f00440af1b0 2024-09-10T10:46:29.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.162+0000 7f003be00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 0x7f002006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.162+0000 7f003be00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 0x7f002006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f002c034000 tx=0x7f002c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.166+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0034059a00 con 0x7f00440af1b0 2024-09-10T10:46:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.387+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00440b3f30 con 0x7f00440af1b0 2024-09-10T10:46:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.387+0000 7f003aa00640 1 -- 172.21.15.28:0/1925171666 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5223 (secure 0 0 0) 0x7f0034059300 con 0x7f00440af1b0 2024-09-10T10:46:29.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:29.390 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":38,"btime":"2024-09-10T10:46:28:601751+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:27.604592+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-09-10T10:46:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 38 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.389+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 msgr2=0x7f002006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.389+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 0x7f002006e2f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f002c034000 tx=0x7f002c02e000 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 msgr2=0x7f004406f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f003400d7d0 tx=0x7f003400dc90 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 shutdown_connections 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002006be30 0x7f002006e2f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f004415bef0 0x7f004409c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00440b25c0 0x7f004409c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 --2- 172.21.15.28:0/1925171666 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00440af1b0 0x7f004406f810 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 >> 172.21.15.28:0/1925171666 conn(0x7f004406c240 msgr2=0x7f0044072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 shutdown_connections 2024-09-10T10:46:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.390+0000 7f0043e00640 1 -- 172.21.15.28:0/1925171666 wait complete. 2024-09-10T10:46:29.408 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:29.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.563+0000 7f28c4a00640 1 Processor -- start 2024-09-10T10:46:29.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.563+0000 7f28c4a00640 1 -- start start 2024-09-10T10:46:29.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0147360 0x7f28c0070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28c4a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f28c00d0bb0 0x7f28c00ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28c4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c006cb30 con 0x7f28c0070dd0 2024-09-10T10:46:29.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28c4a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f28c006bfb0 con 0x7f28c00d0bb0 2024-09-10T10:46:29.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28c4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28c006c290 con 0x7f28c0147360 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28be800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f28c00d0bb0 0x7f28c00ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28be800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f28c00d0bb0 0x7f28c00ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35988/0 (socket says 172.21.15.28:35988) 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.564+0000 7f28be800640 1 -- 172.21.15.28:0/954029408 learned_addr learned my addr 172.21.15.28:0/954029408 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.565+0000 7f28bde00640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0147360 0x7f28c0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.565+0000 7f28be800640 1 -- 172.21.15.28:0/954029408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0147360 msgr2=0x7f28c0070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.565+0000 7f28be800640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0147360 0x7f28c0070890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.565+0000 7f28be800640 1 -- 172.21.15.28:0/954029408 >> v1:172.21.15.138:6789/0 conn(0x7f28c0070dd0 legacy=0x7f28c00d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.565+0000 7f28be800640 1 -- 172.21.15.28:0/954029408 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c006c570 con 0x7f28c00d0bb0 2024-09-10T10:46:29.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28be800640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c00d0bb0 0x7f28c00ae550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009e50 tx=0x7f28b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd44e08e350efb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28bd400640 1 -- 172.21.15.28:0/954029408 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8036070 con 0x7f28c00d0bb0 2024-09-10T10:46:29.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28bd400640 1 -- 172.21.15.28:0/954029408 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8031600 con 0x7f28c00d0bb0 2024-09-10T10:46:29.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28bd400640 1 -- 172.21.15.28:0/954029408 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80319d0 con 0x7f28c00d0bb0 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28c4a00640 1 -- 172.21.15.28:0/954029408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c00d0bb0 msgr2=0x7f28c00ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.566+0000 7f28c4a00640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c00d0bb0 0x7f28c00ae550 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009e50 tx=0x7f28b802a680 comp rx=0 tx=0).stop 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 -- 172.21.15.28:0/954029408 shutdown_connections 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c00d0bb0 0x7f28c00ae550 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 --2- 172.21.15.28:0/954029408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c0147360 0x7f28c0070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 -- 172.21.15.28:0/954029408 >> 172.21.15.28:0/954029408 conn(0x7f28c00b2f40 msgr2=0x7f28c00b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 -- 172.21.15.28:0/954029408 shutdown_connections 2024-09-10T10:46:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.567+0000 7f28c4a00640 1 -- 172.21.15.28:0/954029408 wait complete. 2024-09-10T10:46:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.568+0000 7f28c4a00640 1 Processor -- start 2024-09-10T10:46:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.568+0000 7f28c4a00640 1 -- start start 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c0070dd0 0x7f28c0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00d0bb0 0x7f28c00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28c006bd90 con 0x7f28c0147360 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28c006bc10 con 0x7f28c0070dd0 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.569+0000 7f28c4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f28c00aea90 con 0x7f28c00d0bb0 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36000/0 (socket says 172.21.15.28:36000) 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 -- 172.21.15.28:0/3611519638 learned_addr learned my addr 172.21.15.28:0/3611519638 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28bde00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c0070dd0 0x7f28c0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28b7e00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00d0bb0 0x7f28c00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 -- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00d0bb0 msgr2=0x7f28c00b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00d0bb0 0x7f28c00b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 -- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c0070dd0 msgr2=0x7f28c0155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c0070dd0 0x7f28c0155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 -- 172.21.15.28:0/3611519638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c00b59c0 con 0x7f28c0147360 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.570+0000 7f28be800640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.571+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b803f020 con 0x7f28c0147360 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.571+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8037040 con 0x7f28c0147360 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.571+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8036070 con 0x7f28c0147360 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.571+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c00b5ca0 con 0x7f28c0147360 2024-09-10T10:46:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.571+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c009b6b0 con 0x7f28c0147360 2024-09-10T10:46:29.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.572+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c009b8b0 con 0x7f28c0147360 2024-09-10T10:46:29.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.573+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f28b8042020 con 0x7f28c0147360 2024-09-10T10:46:29.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.574+0000 7f28b6000640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 0x7f289c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.574+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f28b80ab4e0 con 0x7f28c0147360 2024-09-10T10:46:29.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.575+0000 7f28bde00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 0x7f289c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.575+0000 7f28bde00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 0x7f289c06e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.579+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28c009b8b0 con 0x7f28c0147360 2024-09-10T10:46:29.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.799+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28c00b0bf0 con 0x7f28c0147360 2024-09-10T10:46:29.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.799+0000 7f28b6000640 1 -- 172.21.15.28:0/3611519638 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5230 (secure 0 0 0) 0x7f28b8049030 con 0x7f28c0147360 2024-09-10T10:46:29.802 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:29.802 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":39,"btime":"2024-09-10T10:46:29:610384+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:29.610380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-09-10T10:46:29.802 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 39 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.801+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 msgr2=0x7f289c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.801+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 0x7f289c06e2f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0008040 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.801+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 msgr2=0x7f28c009d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.801+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f28b8034040 tx=0x7f28b8004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 shutdown_connections 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f289c06be30 0x7f289c06e2f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f28c0147360 0x7f28c009d810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f28c00d0bb0 0x7f28c00b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 --2- 172.21.15.28:0/3611519638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28c0070dd0 0x7f28c0155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 >> 172.21.15.28:0/3611519638 conn(0x7f28c00b2f40 msgr2=0x7f28c00af020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 shutdown_connections 2024-09-10T10:46:29.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.802+0000 7f28c4a00640 1 -- 172.21.15.28:0/3611519638 wait complete. 2024-09-10T10:46:29.819 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:29.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.972+0000 7f91f1600640 1 Processor -- start 2024-09-10T10:46:29.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.972+0000 7f91f1600640 1 -- start start 2024-09-10T10:46:29.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 0x7f91ec162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91ec0af280 0x7f91ec0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec0b6db0 con 0x7f91ec0af280 2024-09-10T10:46:29.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec15eb60 con 0x7f91ec0b69d0 2024-09-10T10:46:29.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.973+0000 7f91f1600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec15ee40 con 0x7f91ec0affb0 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91ec0af280 0x7f91ec0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91ec0af280 0x7f91ec0afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36012/0 (socket says 172.21.15.28:36012) 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 -- 172.21.15.28:0/734574644 learned_addr learned my addr 172.21.15.28:0/734574644 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91eaa00640 1 --2- 172.21.15.28:0/734574644 >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 0x7f91ec162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91eb400640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 -- 172.21.15.28:0/734574644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 msgr2=0x7f91ec154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.974+0000 7f91e3e00640 1 -- 172.21.15.28:0/734574644 >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 msgr2=0x7f91ec162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.975+0000 7f91e3e00640 1 --2- 172.21.15.28:0/734574644 >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 0x7f91ec162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.975+0000 7f91e3e00640 1 -- 172.21.15.28:0/734574644 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec15f120 con 0x7f91ec0af280 2024-09-10T10:46:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.975+0000 7f91eb400640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.975+0000 7f91eaa00640 1 --2- 172.21.15.28:0/734574644 >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 0x7f91ec162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91e3e00640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0af280 0x7f91ec0afa70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f91ec070980 tx=0x7f91d402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7125de5b15f53ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91ea000640 1 -- 172.21.15.28:0/734574644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4037070 con 0x7f91ec0af280 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91ea000640 1 -- 172.21.15.28:0/734574644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d402ae00 con 0x7f91ec0af280 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91ea000640 1 -- 172.21.15.28:0/734574644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4032660 con 0x7f91ec0af280 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 -- 172.21.15.28:0/734574644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0af280 msgr2=0x7f91ec0afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0af280 0x7f91ec0afa70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f91ec070980 tx=0x7f91d402a0b0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 -- 172.21.15.28:0/734574644 shutdown_connections 2024-09-10T10:46:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 --2- 172.21.15.28:0/734574644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0af280 0x7f91ec0afa70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 --2- 172.21.15.28:0/734574644 >> v2:172.21.15.138:3300/0 conn(0x7f91ec0b69d0 0x7f91ec162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.976+0000 7f91f1600640 1 -- 172.21.15.28:0/734574644 >> 172.21.15.28:0/734574644 conn(0x7f91ec06bc10 msgr2=0x7f91ec06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.977+0000 7f91f1600640 1 -- 172.21.15.28:0/734574644 shutdown_connections 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.977+0000 7f91f1600640 1 -- 172.21.15.28:0/734574644 wait complete. 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.978+0000 7f91f1600640 1 Processor -- start 2024-09-10T10:46:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.978+0000 7f91f1600640 1 -- start start 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0af280 0x7f91ec147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec155240 con 0x7f91ec0b69d0 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec0af710 con 0x7f91ec0af280 2024-09-10T10:46:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91f1600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec1553c0 con 0x7f91ec0affb0 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91eb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91eb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36022/0 (socket says 172.21.15.28:36022) 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.979+0000 7f91eb400640 1 -- 172.21.15.28:0/2243500193 learned_addr learned my addr 172.21.15.28:0/2243500193 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91e3e00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eaa00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0af280 0x7f91ec147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 -- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 msgr2=0x7f91ec15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 -- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0af280 msgr2=0x7f91ec147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0af280 0x7f91ec147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 -- 172.21.15.28:0/2243500193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec0cef70 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91e3e00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91eb400640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007e10 tx=0x7f91e400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.980+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e401a070 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.981+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e4003b90 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.981+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec0cf250 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.981+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4004030 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.981+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec0cf810 con 0x7f91ec0b69d0 2024-09-10T10:46:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.982+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec155820 con 0x7f91ec0b69d0 2024-09-10T10:46:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.983+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91e401d070 con 0x7f91ec0b69d0 2024-09-10T10:46:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.983+0000 7f91e2a00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 0x7f91c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.984+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f91e408b470 con 0x7f91ec0b69d0 2024-09-10T10:46:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.984+0000 7f91eaa00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 0x7f91c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.985+0000 7f91eaa00640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 0x7f91c806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:29.988+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f91e4010070 con 0x7f91ec0b69d0 2024-09-10T10:46:30.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.173+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91ec06d750 con 0x7f91ec0b69d0 2024-09-10T10:46:30.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.174+0000 7f91e2a00640 1 -- 172.21.15.28:0/2243500193 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+17140 (secure 0 0 0) 0x7f91e4059af0 con 0x7f91ec0b69d0 2024-09-10T10:46:30.177 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:30.177 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":59,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:28.596504+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:05.928754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:07.123356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/20691839":"2024-09-11T10:46:24.496417+0000","172.21.15.28:6834/20691839":"2024-09-11T10:46:24.496417+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.176+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 msgr2=0x7f91c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.176+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 0x7f91c806e2f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f91dc006fd0 tx=0x7f91dc008040 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.176+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 msgr2=0x7f91ec15de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.176+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007e10 tx=0x7f91e400d4a0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.176+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 shutdown_connections 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91c806be30 0x7f91c806e2f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91ec0b69d0 0x7f91ec15de00 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91ec0affb0 0x7f91ec15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 --2- 172.21.15.28:0/2243500193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91ec0af280 0x7f91ec147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 >> 172.21.15.28:0/2243500193 conn(0x7f91ec06bc10 msgr2=0x7f91ec0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 shutdown_connections 2024-09-10T10:46:30.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.177+0000 7f91f1600640 1 -- 172.21.15.28:0/2243500193 wait complete. 2024-09-10T10:46:30.196 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:46:30.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:30.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.349+0000 7fbcf1000640 1 Processor -- start 2024-09-10T10:46:30.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.350+0000 7fbcf1000640 1 -- start start 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcf1000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbcec06f8d0 0x7fbcec15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcf1000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcec0cdbe0 con 0x7fbcec0cd3f0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcf1000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbcec0728a0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcf1000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec0cd900 con 0x7fbcec15b300 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbce9400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbcec0cd3f0 0x7fbcec06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46480/0 (socket says 172.21.15.28:46480) 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbce9400640 1 -- 172.21.15.28:0/1573371197 learned_addr learned my addr 172.21.15.28:0/1573371197 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.351+0000 7fbcea800640 1 --2- 172.21.15.28:0/1573371197 >> v2:172.21.15.28:3300/0 conn(0x7fbcec06f8d0 0x7fbcec15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbce9e00640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbce8a00640 1 -- 172.21.15.28:0/1573371197 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123078060 0 0) 0x7fbcec0cdbe0 con 0x7fbcec0cd3f0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbce8a00640 1 -- 172.21.15.28:0/1573371197 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcd0003520 con 0x7fbcec0cd3f0 2024-09-10T10:46:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbcea800640 1 -- 172.21.15.28:0/1573371197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 msgr2=0x7fbcec0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbcea800640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbcea800640 1 -- 172.21.15.28:0/1573371197 >> v1:172.21.15.28:6789/0 conn(0x7fbcec0cd3f0 legacy=0x7fbcec06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbcea800640 1 -- 172.21.15.28:0/1573371197 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec072250 con 0x7fbcec06f8d0 2024-09-10T10:46:30.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.352+0000 7fbce9e00640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.353+0000 7fbcea800640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec15b6e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009d90 tx=0x7fbce402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a7e7e0c2339125 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.353+0000 7fbce8a00640 1 -- 172.21.15.28:0/1573371197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4007740 con 0x7fbcec06f8d0 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.353+0000 7fbce8a00640 1 -- 172.21.15.28:0/1573371197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce40078e0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.353+0000 7fbce8a00640 1 -- 172.21.15.28:0/1573371197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4007cb0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 -- 172.21.15.28:0/1573371197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 msgr2=0x7fbcec15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec15b6e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009d90 tx=0x7fbce402a660 comp rx=0 tx=0).stop 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 -- 172.21.15.28:0/1573371197 shutdown_connections 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec15b6e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 --2- 172.21.15.28:0/1573371197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 -- 172.21.15.28:0/1573371197 >> 172.21.15.28:0/1573371197 conn(0x7fbcec067d70 msgr2=0x7fbcec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 -- 172.21.15.28:0/1573371197 shutdown_connections 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.354+0000 7fbcf1000640 1 -- 172.21.15.28:0/1573371197 wait complete. 2024-09-10T10:46:30.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.355+0000 7fbcf1000640 1 Processor -- start 2024-09-10T10:46:30.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 -- start start 2024-09-10T10:46:30.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 0x7fbcec0a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0d0160 con 0x7fbcec06f8d0 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec0cffe0 con 0x7fbcec0cd3f0 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.356+0000 7fbcf1000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbcec072d30 con 0x7fbcec15b300 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36034/0 (socket says 172.21.15.28:36034) 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 -- 172.21.15.28:0/4096304163 learned_addr learned my addr 172.21.15.28:0/4096304163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbcea800640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 -- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 msgr2=0x7fbcec0ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9400640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 0x7fbcec0a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 -- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 msgr2=0x7fbcec0a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 0x7fbcec0a9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 -- 172.21.15.28:0/4096304163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec0ab480 con 0x7fbcec06f8d0 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbcea800640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9400640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 0x7fbcec0a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.357+0000 7fbce9e00640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00d7d0 tx=0x7fbcdc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.358+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc004030 con 0x7fbcec06f8d0 2024-09-10T10:46:30.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.358+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcdc0041d0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.358+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0096a0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.358+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec0ab760 con 0x7fbcec06f8d0 2024-09-10T10:46:30.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.358+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec0c8e30 con 0x7fbcec06f8d0 2024-09-10T10:46:30.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.359+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcec06efe0 con 0x7fbcec06f8d0 2024-09-10T10:46:30.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.360+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcdc009840 con 0x7fbcec06f8d0 2024-09-10T10:46:30.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.361+0000 7fbce2000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 0x7fbccc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.361+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7fbcdc08b380 con 0x7fbcec06f8d0 2024-09-10T10:46:30.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.361+0000 7fbce9400640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 0x7fbccc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.362+0000 7fbce9400640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 0x7fbccc06e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0cab40 tx=0x7fbcd8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.365+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcdc059b20 con 0x7fbcec06f8d0 2024-09-10T10:46:30.511 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:46:30.509+0000 7fdb8fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:46:30.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.584+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcec06ed00 con 0x7fbcec06f8d0 2024-09-10T10:46:30.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.584+0000 7fbce2000640 1 -- 172.21.15.28:0/4096304163 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5230 (secure 0 0 0) 0x7fbcdc059420 con 0x7fbcec06f8d0 2024-09-10T10:46:30.587 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:30.587 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":39,"btime":"2024-09-10T10:46:29:610384+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:29.610380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-09-10T10:46:30.587 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 39 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 msgr2=0x7fbccc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 0x7fbccc06e2f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0cab40 tx=0x7fbcd8009040 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 msgr2=0x7fbcec14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00d7d0 tx=0x7fbcdc00dc90 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 shutdown_connections 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06be30 0x7fbccc06e2f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcec15b300 0x7fbcec0ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec0cd3f0 0x7fbcec0a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.586+0000 7fbcf1000640 1 --2- 172.21.15.28:0/4096304163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcec06f8d0 0x7fbcec14ee60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.587+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 >> 172.21.15.28:0/4096304163 conn(0x7fbcec067d70 msgr2=0x7fbcec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.587+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 shutdown_connections 2024-09-10T10:46:30.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.587+0000 7fbcf1000640 1 -- 172.21.15.28:0/4096304163 wait complete. 2024-09-10T10:46:30.606 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.760+0000 7f0da3200640 1 Processor -- start 2024-09-10T10:46:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.760+0000 7f0da3200640 1 -- start start 2024-09-10T10:46:30.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da40d0dd0 0x7f0da40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0da415e660 0x7f0da406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da40d1f90 0x7f0da40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0da40b37d0 con 0x7f0da40d0dd0 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0da40cf330 con 0x7f0da415e660 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.761+0000 7f0da3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40cf000 con 0x7f0da40d1f90 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da40d0dd0 0x7f0da40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da40d0dd0 0x7f0da40d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36042/0 (socket says 172.21.15.28:36042) 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 -- 172.21.15.28:0/3606854700 learned_addr learned my addr 172.21.15.28:0/3606854700 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da2800640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da40d1f90 0x7f0da40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1400640 1 --2- 172.21.15.28:0/3606854700 >> v2:172.21.15.138:3300/0 conn(0x7f0da415e660 0x7f0da406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 -- 172.21.15.28:0/3606854700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da40d1f90 msgr2=0x7f0da40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da40d1f90 0x7f0da40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 -- 172.21.15.28:0/3606854700 >> v2:172.21.15.138:3300/0 conn(0x7f0da415e660 msgr2=0x7f0da406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 --2- 172.21.15.28:0/3606854700 >> v2:172.21.15.138:3300/0 conn(0x7f0da415e660 0x7f0da406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.762+0000 7f0da1e00640 1 -- 172.21.15.28:0/3606854700 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da40d16f0 con 0x7f0da40d0dd0 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.763+0000 7f0da1e00640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da40d11b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0da40b3980 tx=0x7f0d9402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68ad71330339c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da0a00640 1 -- 172.21.15.28:0/3606854700 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94037070 con 0x7f0da40d0dd0 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da0a00640 1 -- 172.21.15.28:0/3606854700 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d94032440 con 0x7f0da40d0dd0 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da0a00640 1 -- 172.21.15.28:0/3606854700 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94032810 con 0x7f0da40d0dd0 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 -- 172.21.15.28:0/3606854700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 msgr2=0x7f0da40d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da40d11b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0da40b3980 tx=0x7f0d9402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 -- 172.21.15.28:0/3606854700 shutdown_connections 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da40d1f90 0x7f0da40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 --2- 172.21.15.28:0/3606854700 >> v2:172.21.15.138:3300/0 conn(0x7f0da415e660 0x7f0da406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 --2- 172.21.15.28:0/3606854700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da40d11b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 -- 172.21.15.28:0/3606854700 >> 172.21.15.28:0/3606854700 conn(0x7f0da408bfc0 msgr2=0x7f0da40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 -- 172.21.15.28:0/3606854700 shutdown_connections 2024-09-10T10:46:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.764+0000 7f0da3200640 1 -- 172.21.15.28:0/3606854700 wait complete. 2024-09-10T10:46:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.766+0000 7f0da3200640 1 Processor -- start 2024-09-10T10:46:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.766+0000 7f0da3200640 1 -- start start 2024-09-10T10:46:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da40d1f90 0x7f0da40cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 0x7f0da40cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40d2f40 con 0x7f0da40d0dd0 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40d2c60 con 0x7f0da40d1f90 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0da40d3220 con 0x7f0da415e660 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36052/0 (socket says 172.21.15.28:36052) 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da1e00640 1 -- 172.21.15.28:0/1319595977 learned_addr learned my addr 172.21.15.28:0/1319595977 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da1400640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da40d1f90 0x7f0da40cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.767+0000 7f0da2800640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 0x7f0da40cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 -- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 msgr2=0x7f0da40cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 0x7f0da40cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 -- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da40d1f90 msgr2=0x7f0da40cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da40d1f90 0x7f0da40cc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 -- 172.21.15.28:0/1319595977 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da41e60d0 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da2800640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 0x7f0da40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da1e00640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402a220 tx=0x7f0d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94041020 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d940306c0 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94037040 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da40b9b00 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.768+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da40ba0c0 con 0x7f0da40d0dd0 2024-09-10T10:46:30.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.769+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da40d2440 con 0x7f0da40d0dd0 2024-09-10T10:46:30.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.770+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d94044020 con 0x7f0da40d0dd0 2024-09-10T10:46:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.772+0000 7f0d9a000640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 0x7f0d8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.772+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f0d940ab840 con 0x7f0da40d0dd0 2024-09-10T10:46:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.772+0000 7f0da1400640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 0x7f0d8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.773+0000 7f0da1400640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 0x7f0d8406e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0da415c650 tx=0x7f0d90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.776+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d9404b030 con 0x7f0da40d0dd0 2024-09-10T10:46:30.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.996+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0da40d1b20 con 0x7f0da40d0dd0 2024-09-10T10:46:30.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.997+0000 7f0d9a000640 1 -- 172.21.15.28:0/1319595977 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5230 (secure 0 0 0) 0x7f0d9407ee20 con 0x7f0da40d0dd0 2024-09-10T10:46:30.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:30.999 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":39,"btime":"2024-09-10T10:46:29:610384+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:08.120814+0000","modified":"2024-09-10T10:46:29.610380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/3042558016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3042558016},{"type":"v1","addr":"172.21.15.28:6835","nonce":3042558016}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5625,"qdb_cluster":[5625]},"id":3}]} 2024-09-10T10:46:30.999 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 39 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 msgr2=0x7f0d8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 0x7f0d8406e2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0da415c650 tx=0x7f0d90008040 comp rx=0 tx=0).stop 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 msgr2=0x7f0da414d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0d9402a220 tx=0x7f0d940046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 shutdown_connections 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8406be30 0x7f0d8406e2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da415e660 0x7f0da40cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da40d1f90 0x7f0da40cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 --2- 172.21.15.28:0/1319595977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da40d0dd0 0x7f0da414d5b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 >> 172.21.15.28:0/1319595977 conn(0x7f0da408bfc0 msgr2=0x7f0da4157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 shutdown_connections 2024-09-10T10:46:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:30.999+0000 7f0da3200640 1 -- 172.21.15.28:0/1319595977 wait complete. 2024-09-10T10:46:31.017 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.171+0000 7f86e6200640 1 Processor -- start 2024-09-10T10:46:31.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.171+0000 7f86e6200640 1 -- start start 2024-09-10T10:46:31.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e015b300 0x7f86e00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f86e00cd3f0 0x7f86e006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e0072d30 0x7f86e006f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f86e00728a0 con 0x7f86e015b300 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f86e00cdbe0 con 0x7f86e00cd3f0 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.172+0000 7f86e6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f86e00cd900 con 0x7f86e0072d30 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e015b300 0x7f86e00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e015b300 0x7f86e00cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36064/0 (socket says 172.21.15.28:36064) 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 -- 172.21.15.28:0/500000603 learned_addr learned my addr 172.21.15.28:0/500000603 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86dfe00640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e0072d30 0x7f86e006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86dea00640 1 --2- 172.21.15.28:0/500000603 >> v2:172.21.15.138:3300/0 conn(0x7f86e00cd3f0 0x7f86e006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 -- 172.21.15.28:0/500000603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e0072d30 msgr2=0x7f86e006f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e0072d30 0x7f86e006f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 -- 172.21.15.28:0/500000603 >> v2:172.21.15.138:3300/0 conn(0x7f86e00cd3f0 msgr2=0x7f86e006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 --2- 172.21.15.28:0/500000603 >> v2:172.21.15.138:3300/0 conn(0x7f86e00cd3f0 0x7f86e006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.173+0000 7f86df400640 1 -- 172.21.15.28:0/500000603 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e0072250 con 0x7f86e015b300 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.174+0000 7f86df400640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e015b300 0x7f86e00cceb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b57c0 tx=0x7f86d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ce3a84d7910984 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86de000640 1 -- 172.21.15.28:0/500000603 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0037070 con 0x7f86e015b300 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86de000640 1 -- 172.21.15.28:0/500000603 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d0032440 con 0x7f86e015b300 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86de000640 1 -- 172.21.15.28:0/500000603 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0032810 con 0x7f86e015b300 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 -- 172.21.15.28:0/500000603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e015b300 msgr2=0x7f86e00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e015b300 0x7f86e00cceb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b57c0 tx=0x7f86d002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 -- 172.21.15.28:0/500000603 shutdown_connections 2024-09-10T10:46:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e0072d30 0x7f86e006f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 --2- 172.21.15.28:0/500000603 >> v2:172.21.15.138:3300/0 conn(0x7f86e00cd3f0 0x7f86e006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 --2- 172.21.15.28:0/500000603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e015b300 0x7f86e00cceb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 -- 172.21.15.28:0/500000603 >> 172.21.15.28:0/500000603 conn(0x7f86e0067d70 msgr2=0x7f86e006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 -- 172.21.15.28:0/500000603 shutdown_connections 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.175+0000 7f86e6200640 1 -- 172.21.15.28:0/500000603 wait complete. 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.177+0000 7f86e6200640 1 Processor -- start 2024-09-10T10:46:31.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.177+0000 7f86e6200640 1 -- start start 2024-09-10T10:46:31.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e00cd3f0 0x7f86e0166c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e015b300 0x7f86e00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00d0160 con 0x7f86e0072d30 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e00cffe0 con 0x7f86e00cd3f0 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86e6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f86e00b3c40 con 0x7f86e015b300 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86df400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86df400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36070/0 (socket says 172.21.15.28:36070) 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86df400640 1 -- 172.21.15.28:0/4234667191 learned_addr learned my addr 172.21.15.28:0/4234667191 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86dea00640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e00cd3f0 0x7f86e0166c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.178+0000 7f86dfe00640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e015b300 0x7f86e00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 -- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e015b300 msgr2=0x7f86e00caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e015b300 0x7f86e00caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 -- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e00cd3f0 msgr2=0x7f86e0166c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e00cd3f0 0x7f86e0166c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 -- 172.21.15.28:0/4234667191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e015d470 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86df400640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86d002a220 tx=0x7f86d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0041020 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d00306c0 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0037040 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.179+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e015d750 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.180+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e015dd10 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.181+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e01569b0 con 0x7f86e0072d30 2024-09-10T10:46:31.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.181+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f86d0044020 con 0x7f86e0072d30 2024-09-10T10:46:31.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.182+0000 7f86d7400640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 0x7f86c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.182+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f86d00ab700 con 0x7f86e0072d30 2024-09-10T10:46:31.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.183+0000 7f86dea00640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 0x7f86c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.184+0000 7f86dea00640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 0x7f86c006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86e0166c10 tx=0x7f86cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.187+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86d004b030 con 0x7f86e0072d30 2024-09-10T10:46:31.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.371+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86e01566d0 con 0x7f86e0072d30 2024-09-10T10:46:31.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.372+0000 7f86d7400640 1 -- 172.21.15.28:0/4234667191 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+17140 (secure 0 0 0) 0x7f86d00798d0 con 0x7f86e0072d30 2024-09-10T10:46:31.375 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:31.375 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":59,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:28.596504+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:05.928754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:07.123356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/20691839":"2024-09-11T10:46:24.496417+0000","172.21.15.28:6834/20691839":"2024-09-11T10:46:24.496417+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.374+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 msgr2=0x7f86c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.374+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 0x7f86c006e2f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f86e0166c10 tx=0x7f86cc008040 comp rx=0 tx=0).stop 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.374+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 msgr2=0x7f86e015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.374+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86d002a220 tx=0x7f86d00046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 shutdown_connections 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f86c006be30 0x7f86c006e2f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f86e015b300 0x7f86e00caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e00cd3f0 0x7f86e0166c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 --2- 172.21.15.28:0/4234667191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e0072d30 0x7f86e015a070 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 >> 172.21.15.28:0/4234667191 conn(0x7f86e0067d70 msgr2=0x7f86e0152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 shutdown_connections 2024-09-10T10:46:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.375+0000 7f86e6200640 1 -- 172.21.15.28:0/4234667191 wait complete. 2024-09-10T10:46:31.393 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:46:31.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.548+0000 7f5372400640 1 Processor -- start 2024-09-10T10:46:31.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.548+0000 7f5372400640 1 -- start start 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f536c072320 0x7f536c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 0x7f536c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072700 0x7f536c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f536c146140 con 0x7f536c072320 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f536c0cd740 con 0x7f536c072de0 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.549+0000 7f5372400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f536c0cda20 con 0x7f536c072700 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f536c072320 0x7f536c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f536c072320 0x7f536c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36080/0 (socket says 172.21.15.28:36080) 2024-09-10T10:46:31.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 -- 172.21.15.28:0/124713447 learned_addr learned my addr 172.21.15.28:0/124713447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:31.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536aa00640 1 --2- 172.21.15.28:0/124713447 >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 0x7f536c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536be00640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072700 0x7f536c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 -- 172.21.15.28:0/124713447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072700 msgr2=0x7f536c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072700 0x7f536c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 -- 172.21.15.28:0/124713447 >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 msgr2=0x7f536c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 --2- 172.21.15.28:0/124713447 >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 0x7f536c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536b400640 1 -- 172.21.15.28:0/124713447 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c0cdd00 con 0x7f536c072320 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.550+0000 7f536aa00640 1 --2- 172.21.15.28:0/124713447 >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 0x7f536c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.551+0000 7f536b400640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c0cce60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f536c06f8c0 tx=0x7f535c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e282aa125872de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.551+0000 7f536a000640 1 -- 172.21.15.28:0/124713447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c037070 con 0x7f536c072320 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.551+0000 7f536a000640 1 -- 172.21.15.28:0/124713447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c032440 con 0x7f536c072320 2024-09-10T10:46:31.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.551+0000 7f536a000640 1 -- 172.21.15.28:0/124713447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c032810 con 0x7f536c072320 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 -- 172.21.15.28:0/124713447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 msgr2=0x7f536c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c0cce60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f536c06f8c0 tx=0x7f535c02a140 comp rx=0 tx=0).stop 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 -- 172.21.15.28:0/124713447 shutdown_connections 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072700 0x7f536c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 --2- 172.21.15.28:0/124713447 >> v2:172.21.15.138:3300/0 conn(0x7f536c072de0 0x7f536c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 --2- 172.21.15.28:0/124713447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c0cce60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 -- 172.21.15.28:0/124713447 >> 172.21.15.28:0/124713447 conn(0x7f536c067d70 msgr2=0x7f536c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 -- 172.21.15.28:0/124713447 shutdown_connections 2024-09-10T10:46:31.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.552+0000 7f5372400640 1 -- 172.21.15.28:0/124713447 wait complete. 2024-09-10T10:46:31.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.554+0000 7f5372400640 1 Processor -- start 2024-09-10T10:46:31.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.554+0000 7f5372400640 1 -- start start 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 0x7f536c133ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 0x7f536c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0cd520 con 0x7f536c072320 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0cd3a0 con 0x7f536c072700 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f5372400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f536c06eb30 con 0x7f536c072de0 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f536b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f536aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 0x7f536c133ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f536b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36082/0 (socket says 172.21.15.28:36082) 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f536b400640 1 -- 172.21.15.28:0/3798589917 learned_addr learned my addr 172.21.15.28:0/3798589917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.555+0000 7f536be00640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 0x7f536c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 -- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 msgr2=0x7f536c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 0x7f536c0cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 -- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 msgr2=0x7f536c133ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 0x7f536c133ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 -- 172.21.15.28:0/3798589917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c1340e0 con 0x7f536c072320 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536be00640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 0x7f536c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:31.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536aa00640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 0x7f536c133ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f536b400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c041020 con 0x7f536c072320 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c0306c0 con 0x7f536c072320 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c037040 con 0x7f536c072320 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.556+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c142ca0 con 0x7f536c072320 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.557+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c143260 con 0x7f536c072320 2024-09-10T10:46:31.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.558+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c0cce60 con 0x7f536c072320 2024-09-10T10:46:31.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.559+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f535c044020 con 0x7f536c072320 2024-09-10T10:46:31.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.559+0000 7f5363400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 0x7f534c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.559+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6216+0+0 (secure 0 0 0) 0x7f535c0ab870 con 0x7f536c072320 2024-09-10T10:46:31.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.560+0000 7f536aa00640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 0x7f534c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.561+0000 7f536aa00640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 0x7f534c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f536c1452d0 tx=0x7f5358008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.564+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f535c04b030 con 0x7f536c072320 2024-09-10T10:46:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.788+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f536c0ce500 con 0x7f536c072320 2024-09-10T10:46:31.790 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:46:31.789+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:46:31.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.791+0000 7f5363400640 1 -- 172.21.15.28:0/3798589917 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7f535c079f80 con 0x7f536c072320 2024-09-10T10:46:31.793 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:46:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 msgr2=0x7f534c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 0x7f534c06e2f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f536c1452d0 tx=0x7f5358008040 comp rx=0 tx=0).stop 2024-09-10T10:46:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 msgr2=0x7f536c1e46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 shutdown_connections 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534c06be30 0x7f534c06e2f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f536c072de0 0x7f536c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f536c072700 0x7f536c133ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 --2- 172.21.15.28:0/3798589917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f536c072320 0x7f536c1e46d0 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.793+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 >> 172.21.15.28:0/3798589917 conn(0x7f536c067d70 msgr2=0x7f536c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.794+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 shutdown_connections 2024-09-10T10:46:31.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.794+0000 7f5372400640 1 -- 172.21.15.28:0/3798589917 wait complete. 2024-09-10T10:46:31.800 INFO:tasks.ceph.mds.a.smithi028.stderr: -67> 2024-09-10T10:46:30.509+0000 7fdb8fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:46:31.813 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:46:31.845 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:46:31.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.973+0000 7f9903200640 1 Processor -- start 2024-09-10T10:46:31.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.973+0000 7f9903200640 1 -- start start 2024-09-10T10:46:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.974+0000 7f9903200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9904145970 0x7f990406e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9903200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f99040b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9903200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f990406eb50 con 0x7f99040ad4a0 2024-09-10T10:46:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9903200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f99040b30e0 con 0x7f9904145970 2024-09-10T10:46:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9903200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f990406ee20 con 0x7f99040af3c0 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f98fbe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f99040ad4a0 0x7f99040aecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46494/0 (socket says 172.21.15.28:46494) 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f98fbe00640 1 -- 172.21.15.28:0/4085760045 learned_addr learned my addr 172.21.15.28:0/4085760045 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9901e00640 1 --2- 172.21.15.28:0/4085760045 >> v2:172.21.15.138:3300/0 conn(0x7f9904145970 0x7f990406e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9902800640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f99040b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897348743 0 0) 0x7f990406eb50 con 0x7f99040ad4a0 2024-09-10T10:46:31.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.975+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8003520 con 0x7f99040ad4a0 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.976+0000 7f9901e00640 1 -- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 msgr2=0x7f99040b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.976+0000 7f9901e00640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f99040b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.976+0000 7f9901e00640 1 -- 172.21.15.28:0/4085760045 >> v1:172.21.15.28:6789/0 conn(0x7f99040ad4a0 legacy=0x7f99040aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.976+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1454861067 0 0) 0x7f98e8003520 con 0x7f99040ad4a0 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.976+0000 7f9901e00640 1 -- 172.21.15.28:0/4085760045 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990406f100 con 0x7f9904145970 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.977+0000 7f9901e00640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9904145970 0x7f990406e610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097a0 tx=0x7f98f402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46625afd2cff12ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.977+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4037070 con 0x7f9904145970 2024-09-10T10:46:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.977+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f402aba0 con 0x7f9904145970 2024-09-10T10:46:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.977+0000 7f9901400640 1 -- 172.21.15.28:0/4085760045 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec001690 con 0x7f9904145970 2024-09-10T10:46:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 -- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9904145970 msgr2=0x7f990406e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9904145970 0x7f990406e610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097a0 tx=0x7f98f402a110 comp rx=0 tx=0).stop 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 -- 172.21.15.28:0/4085760045 shutdown_connections 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f99040b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 --2- 172.21.15.28:0/4085760045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9904145970 0x7f990406e610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 -- 172.21.15.28:0/4085760045 >> 172.21.15.28:0/4085760045 conn(0x7f990406c240 msgr2=0x7f99040b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 -- 172.21.15.28:0/4085760045 shutdown_connections 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.978+0000 7f9903200640 1 -- 172.21.15.28:0/4085760045 wait complete. 2024-09-10T10:46:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.979+0000 7f9903200640 1 Processor -- start 2024-09-10T10:46:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 -- start start 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99040ad4a0 0x7f9904141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f990415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f99040b3ab0 con 0x7f9904145970 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9904145d50 con 0x7f99040ad4a0 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.980+0000 7f9903200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99040b3c30 con 0x7f99040af3c0 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36108/0 (socket says 172.21.15.28:36108) 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 -- 172.21.15.28:0/4149355567 learned_addr learned my addr 172.21.15.28:0/4149355567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f98fbe00640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f990415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9901e00640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99040ad4a0 0x7f9904141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 -- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 msgr2=0x7f990415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f990415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 -- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99040ad4a0 msgr2=0x7f9904141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99040ad4a0 0x7f9904141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 -- 172.21.15.28:0/4149355567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9904142050 con 0x7f9904145970 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.981+0000 7f9902800640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009dd0 tx=0x7f98fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.982+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc018020 con 0x7f9904145970 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.982+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98fc014480 con 0x7f9904145970 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.982+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc014870 con 0x7f9904145970 2024-09-10T10:46:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.982+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9904142330 con 0x7f9904145970 2024-09-10T10:46:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.982+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990415bdc0 con 0x7f9904145970 2024-09-10T10:46:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.983+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99040b3570 con 0x7f9904145970 2024-09-10T10:46:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.984+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98fc005d10 con 0x7f9904145970 2024-09-10T10:46:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.985+0000 7f98fa000640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 0x7f98e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.985+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6302+0+0 (secure 0 0 0) 0x7f98fc08bc50 con 0x7f9904145970 2024-09-10T10:46:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.985+0000 7f9901e00640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 0x7f98e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:31.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.986+0000 7f9901e00640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 0x7f98e006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f98f4034000 tx=0x7f98f402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:31.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:31.989+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98fc022070 con 0x7f9904145970 2024-09-10T10:46:32.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.210+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f990406fb70 con 0x7f9904145970 2024-09-10T10:46:32.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.796+0000 7f98fa000640 1 -- 172.21.15.28:0/4149355567 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7f98fc010040 con 0x7f9904145970 2024-09-10T10:46:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 msgr2=0x7f98e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 0x7f98e006e2f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f98f4034000 tx=0x7f98f402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 msgr2=0x7f990413ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009dd0 tx=0x7f98fc00c490 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 shutdown_connections 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98e006be30 0x7f98e006e2f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904145970 0x7f990413ed50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99040af3c0 0x7f990415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 --2- 172.21.15.28:0/4149355567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f99040ad4a0 0x7f9904141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 >> 172.21.15.28:0/4149355567 conn(0x7f990406c240 msgr2=0x7f99040b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 shutdown_connections 2024-09-10T10:46:32.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:32.799+0000 7f9903200640 1 -- 172.21.15.28:0/4149355567 wait complete. 2024-09-10T10:46:32.820 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:46:33.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.024+0000 7ff951200640 1 Processor -- start 2024-09-10T10:46:33.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.024+0000 7ff951200640 1 -- start start 2024-09-10T10:46:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 0x7ff94c0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff94c09aa30 0x7ff94c162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 0x7ff94c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c09bd20 con 0x7ff94c09aa30 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff94c15c580 con 0x7ff94c162150 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff951200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c162a70 con 0x7ff94c0b1050 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.025+0000 7ff941400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff94c09aa30 0x7ff94c162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff94c09aa30 0x7ff94c162530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36110/0 (socket says 172.21.15.28:36110) 2024-09-10T10:46:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 -- 172.21.15.28:0/1921869029 learned_addr learned my addr 172.21.15.28:0/1921869029 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff94a800640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 0x7ff94c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff949e00640 1 --2- 172.21.15.28:0/1921869029 >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 0x7ff94c0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 -- 172.21.15.28:0/1921869029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 msgr2=0x7ff94c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 0x7ff94c0b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 -- 172.21.15.28:0/1921869029 >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 msgr2=0x7ff94c0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 --2- 172.21.15.28:0/1921869029 >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 0x7ff94c0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff941400640 1 -- 172.21.15.28:0/1921869029 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c09b1a0 con 0x7ff94c09aa30 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff949e00640 1 --2- 172.21.15.28:0/1921869029 >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 0x7ff94c0d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.026+0000 7ff94a800640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 0x7ff94c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.027+0000 7ff941400640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c162530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff94c073380 tx=0x7ff93402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f60d8fac108a266 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.027+0000 7ff949400640 1 -- 172.21.15.28:0/1921869029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934037070 con 0x7ff94c09aa30 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.027+0000 7ff949400640 1 -- 172.21.15.28:0/1921869029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934032440 con 0x7ff94c09aa30 2024-09-10T10:46:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.027+0000 7ff949400640 1 -- 172.21.15.28:0/1921869029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934032810 con 0x7ff94c09aa30 2024-09-10T10:46:33.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 -- 172.21.15.28:0/1921869029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 msgr2=0x7ff94c162530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c162530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff94c073380 tx=0x7ff93402a140 comp rx=0 tx=0).stop 2024-09-10T10:46:33.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 -- 172.21.15.28:0/1921869029 shutdown_connections 2024-09-10T10:46:33.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c0b1050 0x7ff94c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 --2- 172.21.15.28:0/1921869029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c162530 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 --2- 172.21.15.28:0/1921869029 >> v2:172.21.15.138:3300/0 conn(0x7ff94c162150 0x7ff94c0d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 -- 172.21.15.28:0/1921869029 >> 172.21.15.28:0/1921869029 conn(0x7ff94c0b3dd0 msgr2=0x7ff94c0b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 -- 172.21.15.28:0/1921869029 shutdown_connections 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.028+0000 7ff951200640 1 -- 172.21.15.28:0/1921869029 wait complete. 2024-09-10T10:46:33.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.030+0000 7ff951200640 1 Processor -- start 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.030+0000 7ff951200640 1 -- start start 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff94c0b1050 0x7ff94c15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c162150 0x7ff94c0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c09ba40 con 0x7ff94c09aa30 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c09b760 con 0x7ff94c0b1050 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff951200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff94c09aec0 con 0x7ff94c162150 2024-09-10T10:46:33.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36112/0 (socket says 172.21.15.28:36112) 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 -- 172.21.15.28:0/2878849418 learned_addr learned my addr 172.21.15.28:0/2878849418 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff94a800640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c162150 0x7ff94c0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff941400640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff94c0b1050 0x7ff94c15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 -- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c162150 msgr2=0x7ff94c0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c162150 0x7ff94c0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 -- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff94c0b1050 msgr2=0x7ff94c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff94c0b1050 0x7ff94c15df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.031+0000 7ff949e00640 1 -- 172.21.15.28:0/2878849418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c0b1620 con 0x7ff94c09aa30 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff949e00640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00c7f0 tx=0x7ff93c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c015070 con 0x7ff94c09aa30 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c007820 con 0x7ff94c09aa30 2024-09-10T10:46:33.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c007bd0 con 0x7ff94c09aa30 2024-09-10T10:46:33.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c0b1900 con 0x7ff94c09aa30 2024-09-10T10:46:33.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.032+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c0b1ec0 con 0x7ff94c09aa30 2024-09-10T10:46:33.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.033+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94c0cdab0 con 0x7ff94c09aa30 2024-09-10T10:46:33.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.034+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff93c018070 con 0x7ff94c09aa30 2024-09-10T10:46:33.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.035+0000 7ff942a00640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 0x7ff92806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.035+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6302+0+0 (secure 0 0 0) 0x7ff93c08b2c0 con 0x7ff94c09aa30 2024-09-10T10:46:33.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.035+0000 7ff941400640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 0x7ff92806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.036+0000 7ff941400640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 0x7ff92806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff934034000 tx=0x7ff93402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.039+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff93c059980 con 0x7ff94c09aa30 2024-09-10T10:46:33.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.223+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff94c0b0830 con 0x7ff94c09aa30 2024-09-10T10:46:33.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.798+0000 7ff942a00640 1 -- 172.21.15.28:0/2878849418 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) ==== 181+0+0 (secure 0 0 0) 0x7ff93c059280 con 0x7ff94c09aa30 2024-09-10T10:46:33.801 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:46:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.801+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 msgr2=0x7ff92806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.801+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 0x7ff92806e2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff934034000 tx=0x7ff93402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.801+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 msgr2=0x7ff94c0b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00c7f0 tx=0x7ff93c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 shutdown_connections 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92806be30 0x7ff92806e2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff94c162150 0x7ff94c0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff94c0b1050 0x7ff94c15df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 --2- 172.21.15.28:0/2878849418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff94c09aa30 0x7ff94c0b26a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 >> 172.21.15.28:0/2878849418 conn(0x7ff94c0b3dd0 msgr2=0x7ff94c0d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 shutdown_connections 2024-09-10T10:46:33.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.802+0000 7ff951200640 1 -- 172.21.15.28:0/2878849418 wait complete. 2024-09-10T10:46:33.826 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:46:33.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.989+0000 7f59f9c00640 1 Processor -- start 2024-09-10T10:46:33.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.990+0000 7f59f9c00640 1 -- start start 2024-09-10T10:46:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f9c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f59f4162e30 0x7f59f406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f9c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f59f40cf280 0x7f59f40d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f9c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f415ff70 con 0x7f59f4162340 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f9c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f59f406de90 con 0x7f59f40cf280 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f9c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59f4163270 con 0x7f59f4162e30 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f59f4162340 0x7f59f4162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41272/0 (socket says 172.21.15.28:41272) 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f2a00640 1 -- 172.21.15.28:0/4093524587 learned_addr learned my addr 172.21.15.28:0/4093524587 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.991+0000 7f59f3400640 1 --2- 172.21.15.28:0/4093524587 >> v2:172.21.15.28:3300/0 conn(0x7f59f40cf280 0x7f59f40d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f2000640 1 --2- 172.21.15.28:0/4093524587 >> v2:172.21.15.138:3300/0 conn(0x7f59f4162e30 0x7f59f406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f1600640 1 -- 172.21.15.28:0/4093524587 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201103629 0 0) 0x7f59f415ff70 con 0x7f59f4162340 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f1600640 1 -- 172.21.15.28:0/4093524587 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59d8003520 con 0x7f59f4162340 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f3400640 1 -- 172.21.15.28:0/4093524587 >> v2:172.21.15.138:3300/0 conn(0x7f59f4162e30 msgr2=0x7f59f406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f3400640 1 --2- 172.21.15.28:0/4093524587 >> v2:172.21.15.138:3300/0 conn(0x7f59f4162e30 0x7f59f406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f3400640 1 -- 172.21.15.28:0/4093524587 >> v1:172.21.15.28:6789/0 conn(0x7f59f4162340 legacy=0x7f59f4162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.992+0000 7f59f3400640 1 -- 172.21.15.28:0/4093524587 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f40d4d10 con 0x7f59f40cf280 2024-09-10T10:46:33.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.993+0000 7f59f3400640 1 --2- 172.21.15.28:0/4093524587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f40d2990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f59f40a23d0 tx=0x7f59e402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00ac77067300061 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.993+0000 7f59f1600640 1 -- 172.21.15.28:0/4093524587 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4037070 con 0x7f59f40cf280 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.993+0000 7f59f1600640 1 -- 172.21.15.28:0/4093524587 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e4032440 con 0x7f59f40cf280 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.993+0000 7f59f1600640 1 -- 172.21.15.28:0/4093524587 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e402ae60 con 0x7f59f40cf280 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 -- 172.21.15.28:0/4093524587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 msgr2=0x7f59f40d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 --2- 172.21.15.28:0/4093524587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f40d2990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f59f40a23d0 tx=0x7f59e402a110 comp rx=0 tx=0).stop 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 -- 172.21.15.28:0/4093524587 shutdown_connections 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 --2- 172.21.15.28:0/4093524587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f40d2990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 --2- 172.21.15.28:0/4093524587 >> v2:172.21.15.138:3300/0 conn(0x7f59f4162e30 0x7f59f406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:33.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 -- 172.21.15.28:0/4093524587 >> 172.21.15.28:0/4093524587 conn(0x7f59f408bfc0 msgr2=0x7f59f40d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:33.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 -- 172.21.15.28:0/4093524587 shutdown_connections 2024-09-10T10:46:33.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.994+0000 7f59f9c00640 1 -- 172.21.15.28:0/4093524587 wait complete. 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.995+0000 7f59f9c00640 1 Processor -- start 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.995+0000 7f59f9c00640 1 -- start start 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f4162340 0x7f59f40cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59f4162e30 0x7f59f40b63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59f40cf6f0 con 0x7f59f40cf280 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59f40d2470 con 0x7f59f4162340 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.996+0000 7f59f9c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59f40d52d0 con 0x7f59f4162e30 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48766/0 (socket says 172.21.15.28:48766) 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 -- 172.21.15.28:0/2446725827 learned_addr learned my addr 172.21.15.28:0/2446725827 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2000640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f4162340 0x7f59f40cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f3400640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59f4162e30 0x7f59f40b63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 -- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59f4162e30 msgr2=0x7f59f40b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59f4162e30 0x7f59f40b63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 -- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f4162340 msgr2=0x7f59f40cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f4162340 0x7f59f40cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 -- 172.21.15.28:0/2446725827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f40cb740 con 0x7f59f40cf280 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.997+0000 7f59f2a00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00cb70 tx=0x7f59ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.998+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec017070 con 0x7f59f40cf280 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.998+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec00ad40 con 0x7f59f40cf280 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.998+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f40cba20 con 0x7f59f40cf280 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.998+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec012620 con 0x7f59f40cf280 2024-09-10T10:46:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.998+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f40cbfe0 con 0x7f59f40cf280 2024-09-10T10:46:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:33.999+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f406d740 con 0x7f59f40cf280 2024-09-10T10:46:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.000+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f59ec01a070 con 0x7f59f40cf280 2024-09-10T10:46:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.001+0000 7f59eaa00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.001+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5864+0+0 (secure 0 0 0) 0x7f59ec08ad80 con 0x7f59f40cf280 2024-09-10T10:46:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.001+0000 7f59f2000640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.002+0000 7f59f2000640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 0x7f59d406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59f41f7b80 tx=0x7f59e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:34.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.005+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f59ec059680 con 0x7f59f40cf280 2024-09-10T10:46:34.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.188+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f59f408e640 con 0x7f59f40cf280 2024-09-10T10:46:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.804+0000 7f59eaa00640 1 -- 172.21.15.28:0/2446725827 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) ==== 169+0+0 (secure 0 0 0) 0x7f59ec058f80 con 0x7f59f40cf280 2024-09-10T10:46:34.807 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:46:34.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.808+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 msgr2=0x7f59d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.808+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 0x7f59d406e2f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f59f41f7b80 tx=0x7f59e0008040 comp rx=0 tx=0).stop 2024-09-10T10:46:34.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.808+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 msgr2=0x7f59f414a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.808+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00cb70 tx=0x7f59ec00a430 comp rx=0 tx=0).stop 2024-09-10T10:46:34.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 shutdown_connections 2024-09-10T10:46:34.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f59d406be30 0x7f59d406e2f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f59f4162e30 0x7f59f40b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f4162340 0x7f59f40cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 --2- 172.21.15.28:0/2446725827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59f40cf280 0x7f59f414a9f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 >> 172.21.15.28:0/2446725827 conn(0x7f59f408bfc0 msgr2=0x7f59f40d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:34.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 shutdown_connections 2024-09-10T10:46:34.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.809+0000 7f59f9c00640 1 -- 172.21.15.28:0/2446725827 wait complete. 2024-09-10T10:46:34.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:46:34.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.988+0000 7f0374c00640 1 Processor -- start 2024-09-10T10:46:34.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.988+0000 7f0374c00640 1 -- start start 2024-09-10T10:46:34.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.989+0000 7f0374c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f037015c310 0x7f0370160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0374c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f03700b4800 0x7f03700aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0374c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03700d0370 0x7f03700d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0374c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0370160e70 con 0x7f03700b4800 2024-09-10T10:46:34.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0374c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f03700af7a0 con 0x7f037015c310 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0374c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03700afa80 con 0x7f03700d0370 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f03700b4800 0x7f03700aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f03700b4800 0x7f03700aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48776/0 (socket says 172.21.15.28:48776) 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 -- 172.21.15.28:0/766113678 learned_addr learned my addr 172.21.15.28:0/766113678 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f036de00640 1 --2- 172.21.15.28:0/766113678 >> v2:172.21.15.138:3300/0 conn(0x7f037015c310 0x7f0370160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f036e800640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03700d0370 0x7f03700d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 -- 172.21.15.28:0/766113678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03700d0370 msgr2=0x7f03700d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03700d0370 0x7f03700d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 -- 172.21.15.28:0/766113678 >> v2:172.21.15.138:3300/0 conn(0x7f037015c310 msgr2=0x7f0370160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 --2- 172.21.15.28:0/766113678 >> v2:172.21.15.138:3300/0 conn(0x7f037015c310 0x7f0370160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.990+0000 7f0365400640 1 -- 172.21.15.28:0/766113678 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0370160130 con 0x7f03700b4800 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.991+0000 7f0365400640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700b4800 0x7f03700aef30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03700b2f80 tx=0x7f035802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6df1deb0907908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f036d400640 1 -- 172.21.15.28:0/766113678 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358037070 con 0x7f03700b4800 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f036d400640 1 -- 172.21.15.28:0/766113678 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0358032440 con 0x7f03700b4800 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f036d400640 1 -- 172.21.15.28:0/766113678 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358032810 con 0x7f03700b4800 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 -- 172.21.15.28:0/766113678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700b4800 msgr2=0x7f03700aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700b4800 0x7f03700aef30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03700b2f80 tx=0x7f035802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 -- 172.21.15.28:0/766113678 shutdown_connections 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03700d0370 0x7f03700d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 --2- 172.21.15.28:0/766113678 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700b4800 0x7f03700aef30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 --2- 172.21.15.28:0/766113678 >> v2:172.21.15.138:3300/0 conn(0x7f037015c310 0x7f0370160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 -- 172.21.15.28:0/766113678 >> 172.21.15.28:0/766113678 conn(0x7f037006bdf0 msgr2=0x7f037006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:34.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 -- 172.21.15.28:0/766113678 shutdown_connections 2024-09-10T10:46:34.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.992+0000 7f0374c00640 1 -- 172.21.15.28:0/766113678 wait complete. 2024-09-10T10:46:34.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.994+0000 7f0374c00640 1 Processor -- start 2024-09-10T10:46:34.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.994+0000 7f0374c00640 1 -- start start 2024-09-10T10:46:34.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 0x7f037009da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 0x7f0370149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:34.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f037006df10 con 0x7f03700d0370 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f037015c6f0 con 0x7f03700b4800 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0374c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f037006e090 con 0x7f037015c310 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0365400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0365400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48788/0 (socket says 172.21.15.28:48788) 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f0365400640 1 -- 172.21.15.28:0/1148424565 learned_addr learned my addr 172.21.15.28:0/1148424565 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f036de00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 0x7f037009da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.995+0000 7f036e800640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 0x7f0370149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 -- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 msgr2=0x7f0370149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 0x7f0370149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 -- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 msgr2=0x7f037009da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 0x7f037009da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:34.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 -- 172.21.15.28:0/1148424565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03700737f0 con 0x7f03700d0370 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f036de00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 0x7f037009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f036e800640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 0x7f0370149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0365400640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0358000c00 tx=0x7f03580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358041020 con 0x7f03700d0370 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03580306c0 con 0x7f03700d0370 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358037040 con 0x7f03700d0370 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0370146db0 con 0x7f03700d0370 2024-09-10T10:46:35.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.996+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0370147370 con 0x7f03700d0370 2024-09-10T10:46:35.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.998+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700aef30 con 0x7f03700d0370 2024-09-10T10:46:35.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.998+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0358044020 con 0x7f03700d0370 2024-09-10T10:46:35.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.999+0000 7f0366a00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 0x7f034c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:34.999+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f03580ab600 con 0x7f03700d0370 2024-09-10T10:46:35.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.000+0000 7f036de00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 0x7f034c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.000+0000 7f036de00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 0x7f034c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.003+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f035804b030 con 0x7f03700d0370 2024-09-10T10:46:35.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.186+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f03700cfea0 con 0x7f03700d0370 2024-09-10T10:46:35.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.190+0000 7f0366a00640 1 -- 172.21.15.28:0/1148424565 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f035807a030 con 0x7f03700d0370 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 msgr2=0x7f034c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 0x7f034c06e2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0360006fd0 tx=0x7f0360008040 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 msgr2=0x7f0370148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0358000c00 tx=0x7f03580046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 shutdown_connections 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f034c06be30 0x7f034c06e2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037015c310 0x7f0370149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03700d0370 0x7f0370148e10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 --2- 172.21.15.28:0/1148424565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03700b4800 0x7f037009da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 >> 172.21.15.28:0/1148424565 conn(0x7f037006bdf0 msgr2=0x7f03700afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.192+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 shutdown_connections 2024-09-10T10:46:35.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.193+0000 7f0374c00640 1 -- 172.21.15.28:0/1148424565 wait complete. 2024-09-10T10:46:35.211 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_abort_on_newly_corrupt_dentry 2024-09-10T10:46:35.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.369+0000 7f3d40000640 1 Processor -- start 2024-09-10T10:46:35.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.369+0000 7f3d40000640 1 -- start start 2024-09-10T10:46:35.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d40000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d380cf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d40000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d3815ff70 con 0x7f3d38162340 2024-09-10T10:46:35.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d40000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d3806dbd0 con 0x7f3d38162e30 2024-09-10T10:46:35.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d40000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d3806de90 con 0x7f3d3806d740 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d3d600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3d38162340 0x7f3d38162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41278/0 (socket says 172.21.15.28:41278) 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d3d600640 1 -- 172.21.15.28:0/689073596 learned_addr learned my addr 172.21.15.28:0/689073596 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.370+0000 7f3d3e000640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d380cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203359723 0 0) 0x7f3d3815ff70 con 0x7f3d38162340 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d20003520 con 0x7f3d38162340 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655883031 0 0) 0x7f3d3806dbd0 con 0x7f3d38162e30 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d3815ff70 con 0x7f3d38162e30 2024-09-10T10:46:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2702554443 0 0) 0x7f3d20003520 con 0x7f3d38162340 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 msgr2=0x7f3d380cf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d380cf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 >> v1:172.21.15.138:6789/0 conn(0x7f3d38162e30 legacy=0x7f3d380d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380d4d10 con 0x7f3d38162340 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f3d2c001670 con 0x7f3d38162340 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 >> v1:172.21.15.28:6789/0 conn(0x7f3d38162340 legacy=0x7f3d38162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d20003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d20004d40 0x7f3d200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3806de90 con 0x7f3d20003b70 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2000bb50 con 0x7f3d20004d40 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2000bd20 con 0x7f3d3806d740 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.371+0000 7f3d3d600640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d20003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3e000640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d20004d40 0x7f3d200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3d600640 1 -- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 msgr2=0x7f3d20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3d600640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d20004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3d600640 1 -- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d20004d40 msgr2=0x7f3d200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3d600640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d20004d40 0x7f3d200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3d600640 1 -- 172.21.15.28:0/689073596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d20003520 con 0x7f3d20003b70 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d3806d740 0x7f3d20004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.372+0000 7f3d3e000640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d20004d40 0x7f3d200051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.373+0000 7f3d3d600640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d20003f70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c110 tx=0x7f3d20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b677e8389a80d230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.373+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c007c40 con 0x7f3d20003b70 2024-09-10T10:46:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.373+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c005ad0 con 0x7f3d20003b70 2024-09-10T10:46:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.373+0000 7f3d37e00640 1 -- 172.21.15.28:0/689073596 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c007070 con 0x7f3d20003b70 2024-09-10T10:46:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 -- 172.21.15.28:0/689073596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 msgr2=0x7f3d20003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 --2- 172.21.15.28:0/689073596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d20003f70 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c110 tx=0x7f3d20005830 comp rx=0 tx=0).stop 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d3e000640 1 -- 172.21.15.28:0/689073596 reap_dead start 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 -- 172.21.15.28:0/689073596 shutdown_connections 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 -- 172.21.15.28:0/689073596 >> 172.21.15.28:0/689073596 conn(0x7f3d3808bfc0 msgr2=0x7f3d380d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 -- 172.21.15.28:0/689073596 shutdown_connections 2024-09-10T10:46:35.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.374+0000 7f3d40000640 1 -- 172.21.15.28:0/689073596 wait complete. 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.375+0000 7f3d40000640 1 Processor -- start 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 -- start start 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 0x7f3d380cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d381567d0 0x7f3d3815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380d2470 con 0x7f3d20003b70 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38163270 con 0x7f3d380cd190 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.376+0000 7f3d40000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d380d52d0 con 0x7f3d381567d0 2024-09-10T10:46:35.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48806/0 (socket says 172.21.15.28:48806) 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 -- 172.21.15.28:0/967984689 learned_addr learned my addr 172.21.15.28:0/967984689 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 0x7f3d380cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3e000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d381567d0 0x7f3d3815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 -- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d381567d0 msgr2=0x7f3d3815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d381567d0 0x7f3d3815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 -- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 msgr2=0x7f3d380cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 0x7f3d380cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3d600640 1 -- 172.21.15.28:0/967984689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380cb640 con 0x7f3d20003b70 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.377+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 0x7f3d380cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d3d600640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c110 tx=0x7f3d2c001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c043070 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c038040 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c001670 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d380cb8e0 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.378+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d380cbea0 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.379+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d380d3fe0 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.380+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d2c0072b0 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.381+0000 7f3d35600640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 0x7f3d1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.381+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f3d2c0aa8b0 con 0x7f3d20003b70 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.381+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 0x7f3d1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.382+0000 7f3d3cc00640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 0x7f3d1806e320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ce5a0 tx=0x7f3d24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.385+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d380d3fe0 con 0x7f3d20003b70 2024-09-10T10:46:35.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.571+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"} v 0) -- 0x7f3d380cf280 con 0x7f3d20003b70 2024-09-10T10:46:35.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.574+0000 7f3d35600640 1 -- 172.21.15.28:0/967984689 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_abort_on_newly_corrupt_dentry"}]=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7f3d2c005aa0 con 0x7f3d20003b70 2024-09-10T10:46:35.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 msgr2=0x7f3d1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 0x7f3d1806e320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d380ce5a0 tx=0x7f3d24009040 comp rx=0 tx=0).stop 2024-09-10T10:46:35.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 msgr2=0x7f3d381593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c110 tx=0x7f3d2c001970 comp rx=0 tx=0).stop 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 shutdown_connections 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d1806be60 0x7f3d1806e320 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d381567d0 0x7f3d3815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d380cd190 0x7f3d380cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 --2- 172.21.15.28:0/967984689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d20003b70 0x7f3d381593a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 >> 172.21.15.28:0/967984689 conn(0x7f3d3808bfc0 msgr2=0x7f3d38160240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 shutdown_connections 2024-09-10T10:46:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.577+0000 7f3d40000640 1 -- 172.21.15.28:0/967984689 wait complete. 2024-09-10T10:46:35.595 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T10:46:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.752+0000 7fcc80e00640 1 Processor -- start 2024-09-10T10:46:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.752+0000 7fcc80e00640 1 -- start start 2024-09-10T10:46:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.753+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc80e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 0x7fcc7c0d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc80e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7c160e70 con 0x7fcc7c0b4800 2024-09-10T10:46:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc80e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcc7c0af7a0 con 0x7fcc7c0d0370 2024-09-10T10:46:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc80e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c0afa80 con 0x7fcc7c15c310 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc7a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 0x7fcc7c0d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc79e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc7a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 0x7fcc7c0d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:46132/0 (socket says 172.21.15.28:46132) 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc7a800640 1 -- 172.21.15.28:0/1709089917 learned_addr learned my addr 172.21.15.28:0/1709089917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.754+0000 7fcc79e00640 1 --2- 172.21.15.28:0/1709089917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:49876/0 (socket says 172.21.15.28:49876) 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.755+0000 7fcc79e00640 1 -- 172.21.15.28:0/1709089917 >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 msgr2=0x7fcc7c0d0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.755+0000 7fcc79e00640 1 --2- 172.21.15.28:0/1709089917 >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 0x7fcc7c0d0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.755+0000 7fcc79e00640 1 -- 172.21.15.28:0/1709089917 >> v1:172.21.15.138:6789/0 conn(0x7fcc7c0b4800 legacy=0x7fcc7c0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.755+0000 7fcc79e00640 1 -- 172.21.15.28:0/1709089917 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c160130 con 0x7fcc7c15c310 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.756+0000 7fcc79e00640 1 --2- 172.21.15.28:0/1709089917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0097a0 tx=0x7fcc6c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf6487766477eff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.756+0000 7fcc79400640 1 -- 172.21.15.28:0/1709089917 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c037070 con 0x7fcc7c15c310 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.756+0000 7fcc79400640 1 -- 172.21.15.28:0/1709089917 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc6c032440 con 0x7fcc7c15c310 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.756+0000 7fcc79400640 1 -- 172.21.15.28:0/1709089917 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c032810 con 0x7fcc7c15c310 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 -- 172.21.15.28:0/1709089917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 msgr2=0x7fcc7c160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 --2- 172.21.15.28:0/1709089917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0097a0 tx=0x7fcc6c02a180 comp rx=0 tx=0).stop 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 -- 172.21.15.28:0/1709089917 shutdown_connections 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 --2- 172.21.15.28:0/1709089917 >> v2:172.21.15.138:3300/0 conn(0x7fcc7c0d0370 0x7fcc7c0d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 --2- 172.21.15.28:0/1709089917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c15c310 0x7fcc7c160930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 -- 172.21.15.28:0/1709089917 >> 172.21.15.28:0/1709089917 conn(0x7fcc7c06bdf0 msgr2=0x7fcc7c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 -- 172.21.15.28:0/1709089917 shutdown_connections 2024-09-10T10:46:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.757+0000 7fcc80e00640 1 -- 172.21.15.28:0/1709089917 wait complete. 2024-09-10T10:46:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.758+0000 7fcc80e00640 1 Processor -- start 2024-09-10T10:46:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 -- start start 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 0x7fcc7c1f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 0x7fcc7c15d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c06e090 con 0x7fcc7c15c310 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7c06df10 con 0x7fcc7c0d0370 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.759+0000 7fcc80e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7c06e210 con 0x7fcc7c0b4800 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48816/0 (socket says 172.21.15.28:48816) 2024-09-10T10:46:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 -- 172.21.15.28:0/179119584 learned_addr learned my addr 172.21.15.28:0/179119584 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc73e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 0x7fcc7c15d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc79e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 0x7fcc7c1f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 -- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 msgr2=0x7fcc7c1f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 0x7fcc7c1f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 -- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 msgr2=0x7fcc7c15d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 0x7fcc7c15d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 -- 172.21.15.28:0/179119584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7c1f4b80 con 0x7fcc7c15c310 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc79e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 0x7fcc7c1f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc73e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 0x7fcc7c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.760+0000 7fcc7a800640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009d90 tx=0x7fcc7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.761+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7400cd20 con 0x7fcc7c15c310 2024-09-10T10:46:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.761+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc74003990 con 0x7fcc7c15c310 2024-09-10T10:46:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.761+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74003d60 con 0x7fcc7c15c310 2024-09-10T10:46:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.761+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7c1f4e60 con 0x7fcc7c15c310 2024-09-10T10:46:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.761+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc7c15db80 con 0x7fcc7c15c310 2024-09-10T10:46:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.762+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc7c1f5380 con 0x7fcc7c15c310 2024-09-10T10:46:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.763+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc74012050 con 0x7fcc7c15c310 2024-09-10T10:46:35.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.764+0000 7fcc72000640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 0x7fcc5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:35.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.764+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7fcc7408b320 con 0x7fcc7c15c310 2024-09-10T10:46:35.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.764+0000 7fcc79e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 0x7fcc5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.765+0000 7fcc79e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 0x7fcc5806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c034000 tx=0x7fcc6c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:35.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.768+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc7c1f5380 con 0x7fcc7c15c310 2024-09-10T10:46:35.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.952+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7fcc7c1f7a70 con 0x7fcc7c15c310 2024-09-10T10:46:35.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.952+0000 7fcc72000640 1 -- 172.21.15.28:0/179119584 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fcc74010040 con 0x7fcc7c15c310 2024-09-10T10:46:35.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 msgr2=0x7fcc5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 0x7fcc5806e2f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c034000 tx=0x7fcc6c02e000 comp rx=0 tx=0).stop 2024-09-10T10:46:35.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 msgr2=0x7fcc7c1f8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcc74009d90 tx=0x7fcc7400c490 comp rx=0 tx=0).stop 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 shutdown_connections 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc5806be30 0x7fcc5806e2f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcc7c15c310 0x7fcc7c1f8130 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c0d0370 0x7fcc7c15d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 --2- 172.21.15.28:0/179119584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcc7c0b4800 0x7fcc7c1f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 >> 172.21.15.28:0/179119584 conn(0x7fcc7c06bdf0 msgr2=0x7fcc7c0b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 shutdown_connections 2024-09-10T10:46:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:35.955+0000 7fcc80e00640 1 -- 172.21.15.28:0/179119584 wait complete. 2024-09-10T10:46:35.973 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing' 2024-09-10T10:46:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.127+0000 7fb412e00640 1 Processor -- start 2024-09-10T10:46:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.127+0000 7fb412e00640 1 -- start start 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb40c0d0dd0 0x7fb40c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb40c0d1f90 0x7fb40c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb40c0cf000 con 0x7fb40c15e660 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c06d3e0 con 0x7fb40c0d1f90 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb40c0b37d0 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb411000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb40c15e660 0x7fb40c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41280/0 (socket says 172.21.15.28:41280) 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb411000640 1 -- 172.21.15.28:0/3382200154 learned_addr learned my addr 172.21.15.28:0/3382200154 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb412400640 1 --2- 172.21.15.28:0/3382200154 >> v2:172.21.15.28:3300/0 conn(0x7fb40c0d1f90 0x7fb40c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.128+0000 7fb411a00640 1 --2- 172.21.15.28:0/3382200154 >> v2:172.21.15.138:3300/0 conn(0x7fb40c0d0dd0 0x7fb40c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb40be00640 1 -- 172.21.15.28:0/3382200154 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964180808 0 0) 0x7fb40c0cf000 con 0x7fb40c15e660 2024-09-10T10:46:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb40be00640 1 -- 172.21.15.28:0/3382200154 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3f4003520 con 0x7fb40c15e660 2024-09-10T10:46:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb412400640 1 -- 172.21.15.28:0/3382200154 >> v2:172.21.15.138:3300/0 conn(0x7fb40c0d0dd0 msgr2=0x7fb40c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb412400640 1 --2- 172.21.15.28:0/3382200154 >> v2:172.21.15.138:3300/0 conn(0x7fb40c0d0dd0 0x7fb40c0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb412400640 1 -- 172.21.15.28:0/3382200154 >> v1:172.21.15.28:6789/0 conn(0x7fb40c15e660 legacy=0x7fb40c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.129+0000 7fb412400640 1 -- 172.21.15.28:0/3382200154 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c0d16f0 con 0x7fb40c0d1f90 2024-09-10T10:46:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.130+0000 7fb412400640 1 --2- 172.21.15.28:0/3382200154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0cf7c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b6fb0 tx=0x7fb40402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48ac8f634f70123 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.130+0000 7fb40be00640 1 -- 172.21.15.28:0/3382200154 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404007740 con 0x7fb40c0d1f90 2024-09-10T10:46:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.130+0000 7fb40be00640 1 -- 172.21.15.28:0/3382200154 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4040078e0 con 0x7fb40c0d1f90 2024-09-10T10:46:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.130+0000 7fb40be00640 1 -- 172.21.15.28:0/3382200154 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404007cb0 con 0x7fb40c0d1f90 2024-09-10T10:46:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 -- 172.21.15.28:0/3382200154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d1f90 msgr2=0x7fb40c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 --2- 172.21.15.28:0/3382200154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0cf7c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b6fb0 tx=0x7fb40402a680 comp rx=0 tx=0).stop 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 -- 172.21.15.28:0/3382200154 shutdown_connections 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 --2- 172.21.15.28:0/3382200154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0cf7c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 --2- 172.21.15.28:0/3382200154 >> v2:172.21.15.138:3300/0 conn(0x7fb40c0d0dd0 0x7fb40c0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 -- 172.21.15.28:0/3382200154 >> 172.21.15.28:0/3382200154 conn(0x7fb40c08bfc0 msgr2=0x7fb40c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 -- 172.21.15.28:0/3382200154 shutdown_connections 2024-09-10T10:46:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.131+0000 7fb412e00640 1 -- 172.21.15.28:0/3382200154 wait complete. 2024-09-10T10:46:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 Processor -- start 2024-09-10T10:46:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 -- start start 2024-09-10T10:46:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c15e660 0x7fb40c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0d2c60 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c0cf330 con 0x7fb40c0d1f90 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.133+0000 7fb412e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c0d2f40 con 0x7fb40c15e660 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48824/0 (socket says 172.21.15.28:48824) 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 -- 172.21.15.28:0/1722332884 learned_addr learned my addr 172.21.15.28:0/1722332884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411000640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb412400640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c15e660 0x7fb40c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 -- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c15e660 msgr2=0x7fb40c1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c15e660 0x7fb40c1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 -- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 msgr2=0x7fb40c0ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411a00640 1 -- 172.21.15.28:0/1722332884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1f7960 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.134+0000 7fb411000640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:36.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb411a00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9b0 tx=0x7fb40000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:36.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400017070 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40000c9a0 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40000cd90 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1f7c40 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.135+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1f8200 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.136+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c0d3220 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.137+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb40001a070 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.138+0000 7fb409600640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 0x7fb3ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.138+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7fb40008afe0 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.139+0000 7fb411000640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 0x7fb3ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.139+0000 7fb411000640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 0x7fb3ec06e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007fd0 tx=0x7fb3f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:36.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.142+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb400059490 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.391+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]} v 0) -- 0x7fb40c0d4cf0 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.812+0000 7fb409600640 1 -- 172.21.15.28:0/1722332884 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_existing"]}]=0 v164) ==== 141+0+0 (secure 0 0 0) 0x7fb40005d790 con 0x7fb40c0d0dd0 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 msgr2=0x7fb3ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 0x7fb3ec06e2f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8007fd0 tx=0x7fb3f8009040 comp rx=0 tx=0).stop 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 msgr2=0x7fb40c0b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb40000e9b0 tx=0x7fb40000ee70 comp rx=0 tx=0).stop 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 shutdown_connections 2024-09-10T10:46:36.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb3ec06be30 0x7fb3ec06e2f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb40c15e660 0x7fb40c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb40c0d1f90 0x7fb40c0ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 --2- 172.21.15.28:0/1722332884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb40c0d0dd0 0x7fb40c0b75c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 >> 172.21.15.28:0/1722332884 conn(0x7fb40c08bfc0 msgr2=0x7fb40c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 shutdown_connections 2024-09-10T10:46:36.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.816+0000 7fb412e00640 1 -- 172.21.15.28:0/1722332884 wait complete. 2024-09-10T10:46:36.835 INFO:tasks.cephfs_test_runner:That the MDS won't abort when the dentry is already known to be damaged. ... ok 2024-09-10T10:46:36.835 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-10T10:46:36.835 INFO:tasks.cephfs_test_runner:test_dentry_first_precommit (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:46:36.836 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.991+0000 7f6855000640 1 Processor -- start 2024-09-10T10:46:36.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.991+0000 7f6855000640 1 -- start start 2024-09-10T10:46:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.992+0000 7f6855000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6850072320 0x7f68500cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.992+0000 7f6855000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f685015e1f0 0x7f685006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:36.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.992+0000 7f6855000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6850146140 con 0x7f6850072de0 2024-09-10T10:46:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f6855000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f68500cd740 con 0x7f6850072320 2024-09-10T10:46:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f6855000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68500cda20 con 0x7f685015e1f0 2024-09-10T10:46:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6850072320 0x7f68500cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f6847e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6850072de0 0x7f685006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41286/0 (socket says 172.21.15.28:41286) 2024-09-10T10:46:36.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f6847e00640 1 -- 172.21.15.28:0/3834954087 learned_addr learned my addr 172.21.15.28:0/3834954087 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684e800640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f685015e1f0 0x7f685006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684de00640 1 -- 172.21.15.28:0/3834954087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f685015e1f0 msgr2=0x7f685006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684de00640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f685015e1f0 0x7f685006e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684de00640 1 -- 172.21.15.28:0/3834954087 >> v1:172.21.15.28:6789/0 conn(0x7f6850072de0 legacy=0x7f685006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684de00640 1 -- 172.21.15.28:0/3834954087 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500cdd00 con 0x7f6850072320 2024-09-10T10:46:36.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.993+0000 7f684d400640 1 -- 172.21.15.28:0/3834954087 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141914501 0 0) 0x7f6850146140 con 0x7f6850072de0 2024-09-10T10:46:36.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.994+0000 7f684de00640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072320 0x7f68500cce60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f68400097a0 tx=0x7f684002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1320500c904dde62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f684d400640 1 -- 172.21.15.28:0/3834954087 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840037070 con 0x7f6850072320 2024-09-10T10:46:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f684d400640 1 -- 172.21.15.28:0/3834954087 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684002ae40 con 0x7f6850072320 2024-09-10T10:46:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f684d400640 1 -- 172.21.15.28:0/3834954087 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840032660 con 0x7f6850072320 2024-09-10T10:46:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 -- 172.21.15.28:0/3834954087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072320 msgr2=0x7f68500cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072320 0x7f68500cce60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f68400097a0 tx=0x7f684002a0f0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 -- 172.21.15.28:0/3834954087 shutdown_connections 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f685015e1f0 0x7f685006e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 --2- 172.21.15.28:0/3834954087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072320 0x7f68500cce60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 -- 172.21.15.28:0/3834954087 >> 172.21.15.28:0/3834954087 conn(0x7f6850067d70 msgr2=0x7f685006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 -- 172.21.15.28:0/3834954087 shutdown_connections 2024-09-10T10:46:37.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.995+0000 7f6855000640 1 -- 172.21.15.28:0/3834954087 wait complete. 2024-09-10T10:46:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.997+0000 7f6855000640 1 Processor -- start 2024-09-10T10:46:37.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.997+0000 7f6855000640 1 -- start start 2024-09-10T10:46:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6850072320 0x7f685015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f685015e1f0 0x7f68500aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f68500cd520 con 0x7f6850072de0 2024-09-10T10:46:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68500cd3a0 con 0x7f685015e1f0 2024-09-10T10:46:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6855000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f685006ed00 con 0x7f6850072320 2024-09-10T10:46:37.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6847e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6847e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48828/0 (socket says 172.21.15.28:48828) 2024-09-10T10:46:37.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f6847e00640 1 -- 172.21.15.28:0/904325402 learned_addr learned my addr 172.21.15.28:0/904325402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:37.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f684de00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6850072320 0x7f685015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.998+0000 7f684e800640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f685015e1f0 0x7f68500aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 -- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6850072320 msgr2=0x7f685015bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6850072320 0x7f685015bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 -- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f685015e1f0 msgr2=0x7f68500aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f685015e1f0 0x7f68500aaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 -- 172.21.15.28:0/904325402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500ccbd0 con 0x7f6850072de0 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6847e00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f683800c500 tx=0x7f683800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838018070 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6838007580 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838004a20 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:36.999+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68500cc050 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.000+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68500cc610 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.000+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6850072700 con 0x7f6850072de0 2024-09-10T10:46:37.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.002+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f683801b070 con 0x7f6850072de0 2024-09-10T10:46:37.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.002+0000 7f6846000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.003+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f6838089fa0 con 0x7f6850072de0 2024-09-10T10:46:37.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.003+0000 7f684de00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.004+0000 7f684de00640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 0x7f682c06e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6840034000 tx=0x7f684002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.007+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6838058410 con 0x7f6850072de0 2024-09-10T10:46:37.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.194+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68500c8e00 con 0x7f6850072de0 2024-09-10T10:46:37.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.194+0000 7f6846000640 1 -- 172.21.15.28:0/904325402 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+13455 (secure 0 0 0) 0x7f683805c710 con 0x7f6850072de0 2024-09-10T10:46:37.197 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:37.197 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":62,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:34.800487+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6834/3042558016":"2024-09-11T10:46:31.790545+0000","172.21.15.28:6835/20691839":"2024-09-11T10:46:24.496417+0000","172.21.15.28:6835/3042558016":"2024-09-11T10:46:31.790545+0000","172.21.15.28:6834/20691839":"2024-09-11T10:46:24.496417+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 msgr2=0x7f682c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 0x7f682c06e2f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6840034000 tx=0x7f684002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 msgr2=0x7f68500aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f683800c500 tx=0x7f683800c9c0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 shutdown_connections 2024-09-10T10:46:37.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f682c06be30 0x7f682c06e2f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f685015e1f0 0x7f68500aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6850072de0 0x7f68500aa4d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 --2- 172.21.15.28:0/904325402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6850072320 0x7f685015bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 >> 172.21.15.28:0/904325402 conn(0x7f6850067d70 msgr2=0x7f68500c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 shutdown_connections 2024-09-10T10:46:37.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.197+0000 7f6855000640 1 -- 172.21.15.28:0/904325402 wait complete. 2024-09-10T10:46:37.215 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:46:37.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:46:37.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 Processor -- start 2024-09-10T10:46:37.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 -- start start 2024-09-10T10:46:37.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 0x7f25b00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b015ff70 con 0x7f25b0162340 2024-09-10T10:46:37.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f25b006de90 con 0x7f25b00cf280 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b8400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0163270 con 0x7f25b0162e30 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b6400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 0x7f25b00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:49926/0 (socket says 172.21.15.28:49926) 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b5000640 1 -- 172.21.15.28:0/3895743017 learned_addr learned my addr 172.21.15.28:0/3895743017 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b5a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f25b0162340 0x7f25b0162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:49500/0 (socket says 172.21.15.28:49500) 2024-09-10T10:46:37.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.374+0000 7f25b6400640 1 --2- 172.21.15.28:0/3895743017 >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 0x7f25b00d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:46180/0 (socket says 172.21.15.28:46180) 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.375+0000 7f25b5000640 1 -- 172.21.15.28:0/3895743017 >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 msgr2=0x7f25b00d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.375+0000 7f25b5000640 1 --2- 172.21.15.28:0/3895743017 >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 0x7f25b00d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.375+0000 7f25b5000640 1 -- 172.21.15.28:0/3895743017 >> v1:172.21.15.138:6789/0 conn(0x7f25b0162340 legacy=0x7f25b0162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.375+0000 7f25b5000640 1 -- 172.21.15.28:0/3895743017 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00d4d10 con 0x7f25b0162e30 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.375+0000 7f25afe00640 1 -- 172.21.15.28:0/3895743017 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536040461 0 0) 0x7f25b015ff70 con 0x7f25b0162340 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.376+0000 7f25b5000640 1 --2- 172.21.15.28:0/3895743017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f259c009850 tx=0x7f259c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb3857d1d33dddd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.376+0000 7f25afe00640 1 -- 172.21.15.28:0/3895743017 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c004030 con 0x7f25b0162e30 2024-09-10T10:46:37.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.376+0000 7f25afe00640 1 -- 172.21.15.28:0/3895743017 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c0041d0 con 0x7f25b0162e30 2024-09-10T10:46:37.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.376+0000 7f25afe00640 1 -- 172.21.15.28:0/3895743017 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c003610 con 0x7f25b0162e30 2024-09-10T10:46:37.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 -- 172.21.15.28:0/3895743017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 msgr2=0x7f25b006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 --2- 172.21.15.28:0/3895743017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f259c009850 tx=0x7f259c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 -- 172.21.15.28:0/3895743017 shutdown_connections 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 --2- 172.21.15.28:0/3895743017 >> v2:172.21.15.138:3300/0 conn(0x7f25b00cf280 0x7f25b00d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 --2- 172.21.15.28:0/3895743017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b0162e30 0x7f25b006d740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 -- 172.21.15.28:0/3895743017 >> 172.21.15.28:0/3895743017 conn(0x7f25b008bfc0 msgr2=0x7f25b00d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 -- 172.21.15.28:0/3895743017 shutdown_connections 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.377+0000 7f25b8400640 1 -- 172.21.15.28:0/3895743017 wait complete. 2024-09-10T10:46:37.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.379+0000 7f25b8400640 1 Processor -- start 2024-09-10T10:46:37.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.379+0000 7f25b8400640 1 -- start start 2024-09-10T10:46:37.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00cf280 0x7f25b00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0162340 0x7f25b00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00cf710 con 0x7f25b0162e30 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b00d2470 con 0x7f25b0162340 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b8400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25b00d52d0 con 0x7f25b00cf280 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48836/0 (socket says 172.21.15.28:48836) 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 -- 172.21.15.28:0/3279023973 learned_addr learned my addr 172.21.15.28:0/3279023973 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b5a00640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00cf280 0x7f25b00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b5000640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0162340 0x7f25b00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 -- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00cf280 msgr2=0x7f25b00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00cf280 0x7f25b00cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 -- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0162340 msgr2=0x7f25b00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0162340 0x7f25b00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.380+0000 7f25b6400640 1 -- 172.21.15.28:0/3279023973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b00ce3c0 con 0x7f25b0162e30 2024-09-10T10:46:37.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25b6400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009dd0 tx=0x7f25a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8018020 con 0x7f25b0162e30 2024-09-10T10:46:37.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a8014480 con 0x7f25b0162e30 2024-09-10T10:46:37.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b00cc470 con 0x7f25b0162e30 2024-09-10T10:46:37.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8014870 con 0x7f25b0162e30 2024-09-10T10:46:37.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.381+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b00cca30 con 0x7f25b0162e30 2024-09-10T10:46:37.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.382+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b008e640 con 0x7f25b0162e30 2024-09-10T10:46:37.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.383+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f25a8005d10 con 0x7f25b0162e30 2024-09-10T10:46:37.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.384+0000 7f25ad600640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 0x7f259006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.384+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f25a808b820 con 0x7f25b0162e30 2024-09-10T10:46:37.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.385+0000 7f25b5a00640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 0x7f259006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.385+0000 7f25b5a00640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 0x7f259006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000f60 tx=0x7f25a4007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.389+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25a8022070 con 0x7f25b0162e30 2024-09-10T10:46:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.570+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25b0161e60 con 0x7f25b0162e30 2024-09-10T10:46:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.570+0000 7f25ad600640 1 -- 172.21.15.28:0/3279023973 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) ==== 93+0+10 (secure 0 0 0) 0x7f25a8010040 con 0x7f25b0162e30 2024-09-10T10:46:37.573 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 msgr2=0x7f259006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 0x7f259006e2f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f25a4000f60 tx=0x7f25a4007d50 comp rx=0 tx=0).stop 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 msgr2=0x7f25b0155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009dd0 tx=0x7f25a800c490 comp rx=0 tx=0).stop 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 shutdown_connections 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f259006be30 0x7f259006e2f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25b0162e30 0x7f25b0155a10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0162340 0x7f25b00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 --2- 172.21.15.28:0/3279023973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00cf280 0x7f25b00cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.572+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 >> 172.21.15.28:0/3279023973 conn(0x7f25b008bfc0 msgr2=0x7f25b00d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.573+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 shutdown_connections 2024-09-10T10:46:37.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.573+0000 7f25b8400640 1 -- 172.21.15.28:0/3279023973 wait complete. 2024-09-10T10:46:37.590 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-10T10:46:37.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.745+0000 7efbff200640 1 Processor -- start 2024-09-10T10:46:37.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.745+0000 7efbff200640 1 -- start start 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.745+0000 7efbff200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efbf8072320 0x7efbf80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbff200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf8146140 con 0x7efbf815e1f0 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbff200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf806eed0 con 0x7efbf8072de0 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbff200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efbf80cd740 con 0x7efbf8072320 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbfd200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efbf815e1f0 0x7efbf806e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41296/0 (socket says 172.21.15.28:41296) 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbfd200640 1 -- 172.21.15.28:0/1722691157 learned_addr learned my addr 172.21.15.28:0/1722691157 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:37.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1722691157 >> v2:172.21.15.28:3300/0 conn(0x7efbf8072320 0x7efbf80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbf7e00640 1 -- 172.21.15.28:0/1722691157 >> v1:172.21.15.138:6789/0 conn(0x7efbf8072de0 legacy=0x7efbf806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:37.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbf7e00640 1 -- 172.21.15.28:0/1722691157 >> v1:172.21.15.28:6789/0 conn(0x7efbf815e1f0 legacy=0x7efbf806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.746+0000 7efbf7e00640 1 -- 172.21.15.28:0/1722691157 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf80cda20 con 0x7efbf8072320 2024-09-10T10:46:37.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.747+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1722691157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf8072320 0x7efbf80cce60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097a0 tx=0x7efbe802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bc46058841cc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbf7400640 1 -- 172.21.15.28:0/1722691157 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8037070 con 0x7efbf8072320 2024-09-10T10:46:37.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbf7400640 1 -- 172.21.15.28:0/1722691157 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe802ae50 con 0x7efbf8072320 2024-09-10T10:46:37.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbf7400640 1 -- 172.21.15.28:0/1722691157 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8032660 con 0x7efbf8072320 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 -- 172.21.15.28:0/1722691157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf8072320 msgr2=0x7efbf80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 --2- 172.21.15.28:0/1722691157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf8072320 0x7efbf80cce60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efbe80097a0 tx=0x7efbe802a140 comp rx=0 tx=0).stop 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 -- 172.21.15.28:0/1722691157 shutdown_connections 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 --2- 172.21.15.28:0/1722691157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf8072320 0x7efbf80cce60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 -- 172.21.15.28:0/1722691157 >> 172.21.15.28:0/1722691157 conn(0x7efbf8067d70 msgr2=0x7efbf806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 -- 172.21.15.28:0/1722691157 shutdown_connections 2024-09-10T10:46:37.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.748+0000 7efbff200640 1 -- 172.21.15.28:0/1722691157 wait complete. 2024-09-10T10:46:37.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.750+0000 7efbff200640 1 Processor -- start 2024-09-10T10:46:37.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.750+0000 7efbff200640 1 -- start start 2024-09-10T10:46:37.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 0x7efbf80b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 0x7efbf8144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80cd3a0 con 0x7efbf815e1f0 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efbf815c9e0 con 0x7efbf8072320 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbff200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efbf80cd520 con 0x7efbf8072de0 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbfd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbfd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48858/0 (socket says 172.21.15.28:48858) 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbfd200640 1 -- 172.21.15.28:0/1990871261 learned_addr learned my addr 172.21.15.28:0/1990871261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 0x7efbf80b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbfd200640 1 -- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 msgr2=0x7efbf8144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.751+0000 7efbefe00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 0x7efbf8144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbfd200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 0x7efbf8144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbfd200640 1 -- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 msgr2=0x7efbf80b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbfd200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 0x7efbf80b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:37.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbfd200640 1 -- 172.21.15.28:0/1990871261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8144c20 con 0x7efbf815e1f0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 0x7efbf80b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbefe00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 0x7efbf8144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbfd200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efbf000caa0 tx=0x7efbf000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0017070 con 0x7efbf815e1f0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf000ac00 con 0x7efbf815e1f0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0012400 con 0x7efbf815e1f0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf80a9830 con 0x7efbf815e1f0 2024-09-10T10:46:37.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.752+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf80a9df0 con 0x7efbf815e1f0 2024-09-10T10:46:37.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.753+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf8145c10 con 0x7efbf815e1f0 2024-09-10T10:46:37.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.754+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efbf001a070 con 0x7efbf815e1f0 2024-09-10T10:46:37.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.755+0000 7efbf4c00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 0x7efbd406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:37.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.755+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7efbf008a950 con 0x7efbf815e1f0 2024-09-10T10:46:37.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.755+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 0x7efbd406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:37.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.757+0000 7efbf7e00640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 0x7efbd406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034000 tx=0x7efbe802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:37.760+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efbf8145c10 con 0x7efbf815e1f0 2024-09-10T10:46:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.012+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7efbf8145930 con 0x7efbf815e1f0 2024-09-10T10:46:38.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.821+0000 7efbf4c00640 1 -- 172.21.15.28:0/1990871261 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v166) ==== 145+0+0 (secure 0 0 0) 0x7efbf0058d30 con 0x7efbf815e1f0 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 msgr2=0x7efbd406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 0x7efbd406e2f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efbe8034000 tx=0x7efbe802e000 comp rx=0 tx=0).stop 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 msgr2=0x7efbf80cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efbf000caa0 tx=0x7efbf000cf60 comp rx=0 tx=0).stop 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 shutdown_connections 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbd406be30 0x7efbd406e2f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf815e1f0 0x7efbf80cc600 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf8072de0 0x7efbf8144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 --2- 172.21.15.28:0/1990871261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf8072320 0x7efbf80b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:38.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 >> 172.21.15.28:0/1990871261 conn(0x7efbf8067d70 msgr2=0x7efbf80c6780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:38.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 shutdown_connections 2024-09-10T10:46:38.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:38.824+0000 7efbff200640 1 -- 172.21.15.28:0/1990871261 wait complete. 2024-09-10T10:46:38.845 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.008+0000 7f9aa2600640 1 Processor -- start 2024-09-10T10:46:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.008+0000 7f9aa2600640 1 -- start start 2024-09-10T10:46:39.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.009+0000 7f9aa2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a9c15b570 0x7f9a9c071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.009+0000 7f9aa2600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9a9c15b950 0x7f9a9c0cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.009+0000 7f9aa2600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9c073b20 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.009+0000 7f9aa2600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9c0cd3c0 con 0x7f9a9c15b950 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.009+0000 7f9aa2600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9c0740e0 con 0x7f9a9c15b570 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a9be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9a9c0ccf50 0x7f9a9c0cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41304/0 (socket says 172.21.15.28:41304) 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a9be00640 1 -- 172.21.15.28:0/1657933289 learned_addr learned my addr 172.21.15.28:0/1657933289 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a93e00640 1 --2- 172.21.15.28:0/1657933289 >> v2:172.21.15.28:3300/0 conn(0x7f9a9c15b950 0x7f9a9c0cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a9b400640 1 --2- 172.21.15.28:0/1657933289 >> v2:172.21.15.138:3300/0 conn(0x7f9a9c15b570 0x7f9a9c071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a9aa00640 1 -- 172.21.15.28:0/1657933289 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689210486 0 0) 0x7f9a9c073b20 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a9aa00640 1 -- 172.21.15.28:0/1657933289 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a80005730 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a93e00640 1 -- 172.21.15.28:0/1657933289 >> v2:172.21.15.138:3300/0 conn(0x7f9a9c15b570 msgr2=0x7f9a9c071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a93e00640 1 --2- 172.21.15.28:0/1657933289 >> v2:172.21.15.138:3300/0 conn(0x7f9a9c15b570 0x7f9a9c071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a93e00640 1 -- 172.21.15.28:0/1657933289 >> v1:172.21.15.28:6789/0 conn(0x7f9a9c0ccf50 legacy=0x7f9a9c0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.010+0000 7f9a93e00640 1 -- 172.21.15.28:0/1657933289 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c073e00 con 0x7f9a9c15b950 2024-09-10T10:46:39.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.011+0000 7f9a93e00640 1 --2- 172.21.15.28:0/1657933289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0cca10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0cd9a0 tx=0x7f9a8402a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3609241a4c5c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.011+0000 7f9a9aa00640 1 -- 172.21.15.28:0/1657933289 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84037070 con 0x7f9a9c15b950 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.011+0000 7f9a9aa00640 1 -- 172.21.15.28:0/1657933289 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a84032440 con 0x7f9a9c15b950 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.011+0000 7f9a9aa00640 1 -- 172.21.15.28:0/1657933289 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840327f0 con 0x7f9a9c15b950 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 -- 172.21.15.28:0/1657933289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c15b950 msgr2=0x7f9a9c0cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 --2- 172.21.15.28:0/1657933289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0cca10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0cd9a0 tx=0x7f9a8402a120 comp rx=0 tx=0).stop 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 -- 172.21.15.28:0/1657933289 shutdown_connections 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 --2- 172.21.15.28:0/1657933289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0cca10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 --2- 172.21.15.28:0/1657933289 >> v2:172.21.15.138:3300/0 conn(0x7f9a9c15b570 0x7f9a9c071e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 -- 172.21.15.28:0/1657933289 >> 172.21.15.28:0/1657933289 conn(0x7f9a9c06bdf0 msgr2=0x7f9a9c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 -- 172.21.15.28:0/1657933289 shutdown_connections 2024-09-10T10:46:39.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.012+0000 7f9aa2600640 1 -- 172.21.15.28:0/1657933289 wait complete. 2024-09-10T10:46:39.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.014+0000 7f9aa2600640 1 Processor -- start 2024-09-10T10:46:39.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.014+0000 7f9aa2600640 1 -- start start 2024-09-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a9c15b570 0x7f9a9c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c072450 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9c15bd90 con 0x7f9a9c15b950 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9aa2600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9c0cf840 con 0x7f9a9c15b570 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48882/0 (socket says 172.21.15.28:48882) 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 -- 172.21.15.28:0/421199893 learned_addr learned my addr 172.21.15.28:0/421199893 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9be00640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a93e00640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a9c15b570 0x7f9a9c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 -- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a9c15b570 msgr2=0x7f9a9c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a9c15b570 0x7f9a9c0b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 -- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a9c15b950 msgr2=0x7f9a9c0b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.015+0000 7f9a9b400640 1 -- 172.21.15.28:0/421199893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a9c0af350 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9a9b400640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e870 tx=0x7f9a8c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c017070 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00c940 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a9c15cc40 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00cd30 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.016+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a9c15d200 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.017+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a9c071e40 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.018+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a8c01a070 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.019+0000 7f9a93400640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 0x7f9a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.019+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f9a8c08acc0 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.019+0000 7f9a93e00640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 0x7f9a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.020+0000 7f9a93e00640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 0x7f9a7806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a84034000 tx=0x7f9a8402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.023+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a8c091080 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.033 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:46:39.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:46:39.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.211+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9a9c0ce730 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.215+0000 7f9a93400640 1 -- 172.21.15.28:0/421199893 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f9a8c0596f0 con 0x7f9a9c0ccf50 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 msgr2=0x7f9a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 0x7f9a7806e2f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a84034000 tx=0x7f9a8402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 msgr2=0x7f9a9c147670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00e870 tx=0x7f9a8c00ed30 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 shutdown_connections 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a7806be30 0x7f9a7806e2f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a9c15b950 0x7f9a9c0b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a9c15b570 0x7f9a9c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 --2- 172.21.15.28:0/421199893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a9c0ccf50 0x7f9a9c147670 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.217+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 >> 172.21.15.28:0/421199893 conn(0x7f9a9c06bdf0 msgr2=0x7f9a9c159730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.218+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 shutdown_connections 2024-09-10T10:46:39.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.218+0000 7f9aa2600640 1 -- 172.21.15.28:0/421199893 wait complete. 2024-09-10T10:46:39.234 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:46:39.235 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:39.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:46:39.415 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:46:39.416 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:39.416 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:39.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:39.452 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:39.452 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:39.488 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:39.488 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:39.488 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:39.527 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:46:39.527 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:46:39.527 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:39.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:46:39.564 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:46:39.564 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.722+0000 7f61dfe00640 1 Processor -- start 2024-09-10T10:46:39.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.722+0000 7f61dfe00640 1 -- start start 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.723+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 0x7f61e00d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61dfe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e00d4aa0 con 0x7f61e00cf440 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61dfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61e00d2c20 con 0x7f61e015e6a0 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61dfe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61e00cefb0 con 0x7f61e00d4180 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61df400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f61e00cf440 0x7f61e00cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41316/0 (socket says 172.21.15.28:41316) 2024-09-10T10:46:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61df400640 1 -- 172.21.15.28:0/2371562079 learned_addr learned my addr 172.21.15.28:0/2371562079 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61dea00640 1 --2- 172.21.15.28:0/2371562079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 0x7f61e00d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932972968 0 0) 0x7f61e00d4aa0 con 0x7f61e00cf440 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.724+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61c4003520 con 0x7f61e00cf440 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.725+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 858071535 0 0) 0x7f61c4003520 con 0x7f61e00cf440 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.725+0000 7f61dea00640 1 -- 172.21.15.28:0/2371562079 >> v1:172.21.15.138:6789/0 conn(0x7f61e015e6a0 legacy=0x7f61e006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.725+0000 7f61dea00640 1 -- 172.21.15.28:0/2371562079 >> v1:172.21.15.28:6789/0 conn(0x7f61e00cf440 legacy=0x7f61e00cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.725+0000 7f61dea00640 1 -- 172.21.15.28:0/2371562079 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e006e390 con 0x7f61e00d4180 2024-09-10T10:46:39.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.726+0000 7f61dea00640 1 --2- 172.21.15.28:0/2371562079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 0x7f61e00d4560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d002a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4edefb76e783d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.726+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037070 con 0x7f61e00d4180 2024-09-10T10:46:39.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.726+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d002aac0 con 0x7f61e00d4180 2024-09-10T10:46:39.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.726+0000 7f61de000640 1 -- 172.21.15.28:0/2371562079 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d80019f0 con 0x7f61e00d4180 2024-09-10T10:46:39.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 -- 172.21.15.28:0/2371562079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 msgr2=0x7f61e00d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 --2- 172.21.15.28:0/2371562079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 0x7f61e00d4560 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d002a030 comp rx=0 tx=0).stop 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 -- 172.21.15.28:0/2371562079 shutdown_connections 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 --2- 172.21.15.28:0/2371562079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e00d4180 0x7f61e00d4560 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 -- 172.21.15.28:0/2371562079 >> 172.21.15.28:0/2371562079 conn(0x7f61e009d6f0 msgr2=0x7f61e008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 -- 172.21.15.28:0/2371562079 shutdown_connections 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.727+0000 7f61dfe00640 1 -- 172.21.15.28:0/2371562079 wait complete. 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.728+0000 7f61dfe00640 1 Processor -- start 2024-09-10T10:46:39.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.728+0000 7f61dfe00640 1 -- start start 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61e00d4180 0x7f61e015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e015e6a0 0x7f61e00cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61e00cebd0 con 0x7f61e00cf440 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61e006e950 con 0x7f61e00d4180 2024-09-10T10:46:39.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.729+0000 7f61dfe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61e006df30 con 0x7f61e015e6a0 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48890/0 (socket says 172.21.15.28:48890) 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 -- 172.21.15.28:0/1162866432 learned_addr learned my addr 172.21.15.28:0/1162866432 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61d7e00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61e00d4180 0x7f61e015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61df400640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e015e6a0 0x7f61e00cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 -- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e015e6a0 msgr2=0x7f61e00cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e015e6a0 0x7f61e00cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 -- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61e00d4180 msgr2=0x7f61e015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61e00d4180 0x7f61e015bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 -- 172.21.15.28:0/1162866432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e015c0e0 con 0x7f61e00cf440 2024-09-10T10:46:39.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.730+0000 7f61dea00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.731+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0037040 con 0x7f61e00cf440 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.731+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d0004030 con 0x7f61e00cf440 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.731+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00043e0 con 0x7f61e00cf440 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.731+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e01f73f0 con 0x7f61e00cf440 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.731+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e01f7970 con 0x7f61e00cf440 2024-09-10T10:46:39.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.732+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e00d4560 con 0x7f61e00cf440 2024-09-10T10:46:39.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.733+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61d003a040 con 0x7f61e00cf440 2024-09-10T10:46:39.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.734+0000 7f61d6a00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 0x7f61bc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:39.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.734+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7f61d00ab0b0 con 0x7f61e00cf440 2024-09-10T10:46:39.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.734+0000 7f61d7e00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 0x7f61bc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:39.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.735+0000 7f61d7e00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 0x7f61bc06e320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61e0156450 tx=0x7f61c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:39.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.738+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f61d0042020 con 0x7f61e00cf440 2024-09-10T10:46:39.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.959+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e00cb400 con 0x7f61e00cf440 2024-09-10T10:46:39.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.960+0000 7f61d6a00640 1 -- 172.21.15.28:0/1162866432 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3917 (secure 0 0 0) 0x7f61d0079c90 con 0x7f61e00cf440 2024-09-10T10:46:39.961 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:39.961 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":41,"btime":"2024-09-10T10:46:32:793186+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4976,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6833/1398074635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1398074635},{"type":"v1","addr":"172.21.15.28:6833","nonce":1398074635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":5227,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2656252577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2656252577},{"type":"v1","addr":"172.21.15.138:6836","nonce":2656252577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5230,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3701035236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3701035236},{"type":"v1","addr":"172.21.15.138:6837","nonce":3701035236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5742,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2975933338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2975933338},{"type":"v1","addr":"172.21.15.28:6835","nonce":2975933338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-10T10:46:39.962 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 41 2024-09-10T10:46:39.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 msgr2=0x7f61bc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 0x7f61bc06e320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61e0156450 tx=0x7f61c8009040 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 msgr2=0x7f61e0155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f61d00097d0 tx=0x7f61d00046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 shutdown_connections 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f61bc06be60 0x7f61bc06e320 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f61e015e6a0 0x7f61e00cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61e00d4180 0x7f61e015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 --2- 172.21.15.28:0/1162866432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61e00cf440 0x7f61e0155c80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 >> 172.21.15.28:0/1162866432 conn(0x7f61e009d6f0 msgr2=0x7f61e00b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 shutdown_connections 2024-09-10T10:46:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:39.962+0000 7f61dfe00640 1 -- 172.21.15.28:0/1162866432 wait complete. 2024-09-10T10:46:39.979 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:46:39.979 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:46:39.980 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:46:39.980 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:39.980 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:46:39.980 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:46:39.980 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:46:39.980 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:39.981 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:46:39.981 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:46:39.981 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:46:39.981 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:39.981 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:46:39.981 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:46:39.981 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:46:39.982 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:46:39.994 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:46:39.994 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:46:39.995 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:46:39.995 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:46:39.997 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:46:39.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:46:39.998 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:46:39.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:46:40.039 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:46:40.039 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:40.041 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:46:40.041 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:40.041 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:46:40.042 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:40.043 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:46:40.043 DEBUG:teuthology.parallel:result is None 2024-09-10T10:46:40.043 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:46:40.169 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:46:40.169 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:46:40.171 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:46:40.172 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:46:40.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.209+0000 7fbd7d000640 1 Processor -- start 2024-09-10T10:46:40.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.209+0000 7fbd7d000640 1 -- start start 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d0dd0 0x7fbd780d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd7815e660 0x7fbd7806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd780d1f90 0x7fbd780cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd780b37d0 con 0x7fbd7815e660 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd780cf330 con 0x7fbd780d1f90 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd7d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd780cf000 con 0x7fbd780d0dd0 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.210+0000 7fbd76a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd7815e660 0x7fbd7806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd7815e660 0x7fbd7806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48930/0 (socket says 172.21.15.28:48930) 2024-09-10T10:46:40.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 -- 172.21.15.28:0/2317102310 learned_addr learned my addr 172.21.15.28:0/2317102310 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd77400640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d0dd0 0x7fbd780d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd77e00640 1 --2- 172.21.15.28:0/2317102310 >> v2:172.21.15.138:3300/0 conn(0x7fbd780d1f90 0x7fbd780cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 -- 172.21.15.28:0/2317102310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d0dd0 msgr2=0x7fbd780d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d0dd0 0x7fbd780d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 -- 172.21.15.28:0/2317102310 >> v2:172.21.15.138:3300/0 conn(0x7fbd780d1f90 msgr2=0x7fbd780cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 --2- 172.21.15.28:0/2317102310 >> v2:172.21.15.138:3300/0 conn(0x7fbd780d1f90 0x7fbd780cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.211+0000 7fbd76a00640 1 -- 172.21.15.28:0/2317102310 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd780d16f0 con 0x7fbd7815e660 2024-09-10T10:46:40.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.212+0000 7fbd76a00640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd7806ccd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd780cb420 tx=0x7fbd6402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db881c2c1bc65c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.212+0000 7fbd76000640 1 -- 172.21.15.28:0/2317102310 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64037070 con 0x7fbd7815e660 2024-09-10T10:46:40.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.212+0000 7fbd76000640 1 -- 172.21.15.28:0/2317102310 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd64032440 con 0x7fbd7815e660 2024-09-10T10:46:40.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.212+0000 7fbd76000640 1 -- 172.21.15.28:0/2317102310 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64032810 con 0x7fbd7815e660 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 -- 172.21.15.28:0/2317102310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 msgr2=0x7fbd7806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd7806ccd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbd780cb420 tx=0x7fbd6402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 -- 172.21.15.28:0/2317102310 shutdown_connections 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 --2- 172.21.15.28:0/2317102310 >> v2:172.21.15.138:3300/0 conn(0x7fbd780d1f90 0x7fbd780cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd7806ccd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 --2- 172.21.15.28:0/2317102310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d0dd0 0x7fbd780d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 -- 172.21.15.28:0/2317102310 >> 172.21.15.28:0/2317102310 conn(0x7fbd7808bfc0 msgr2=0x7fbd780d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 -- 172.21.15.28:0/2317102310 shutdown_connections 2024-09-10T10:46:40.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.213+0000 7fbd7d000640 1 -- 172.21.15.28:0/2317102310 wait complete. 2024-09-10T10:46:40.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.215+0000 7fbd7d000640 1 Processor -- start 2024-09-10T10:46:40.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.215+0000 7fbd7d000640 1 -- start start 2024-09-10T10:46:40.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 0x7fbd780b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 0x7fbd780ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd780d2f40 con 0x7fbd7815e660 2024-09-10T10:46:40.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd780d2c60 con 0x7fbd780d0dd0 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd7d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd780d3220 con 0x7fbd780d1f90 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48932/0 (socket says 172.21.15.28:48932) 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 -- 172.21.15.28:0/4284648890 learned_addr learned my addr 172.21.15.28:0/4284648890 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd76a00640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 0x7fbd780ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77400640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 0x7fbd780b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 -- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 msgr2=0x7fbd780ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 0x7fbd780ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 -- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 msgr2=0x7fbd780b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 0x7fbd780b9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77e00640 1 -- 172.21.15.28:0/4284648890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd780ccce0 con 0x7fbd7815e660 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd76a00640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 0x7fbd780ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:40.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.216+0000 7fbd77400640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 0x7fbd780b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd77e00640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000efc0 tx=0x7fbd7000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70018020 con 0x7fbd7815e660 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd70014480 con 0x7fbd7815e660 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd70014870 con 0x7fbd7815e660 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd781e4260 con 0x7fbd7815e660 2024-09-10T10:46:40.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.217+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd780cd240 con 0x7fbd7815e660 2024-09-10T10:46:40.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.218+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd780d2440 con 0x7fbd7815e660 2024-09-10T10:46:40.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.219+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd70005d10 con 0x7fbd7815e660 2024-09-10T10:46:40.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.221+0000 7fbd6f400640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:40.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.221+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5421+0+0 (secure 0 0 0) 0x7fbd7008b860 con 0x7fbd7815e660 2024-09-10T10:46:40.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.221+0000 7fbd77400640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:40.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.222+0000 7fbd77400640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 0x7fbd5806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009a40 tx=0x7fbd68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:40.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.225+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd70022070 con 0x7fbd7815e660 2024-09-10T10:46:40.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.422+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbd780d1b20 con 0x7fbd7815e660 2024-09-10T10:46:40.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.835+0000 7fbd6f400640 1 -- 172.21.15.28:0/4284648890 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) ==== 97+0+0 (secure 0 0 0) 0x7fbd780ccce0 con 0x7fbd7815e660 2024-09-10T10:46:40.836 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:46:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 msgr2=0x7fbd5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 0x7fbd5806e2f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009a40 tx=0x7fbd68008040 comp rx=0 tx=0).stop 2024-09-10T10:46:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 msgr2=0x7fbd781e5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000efc0 tx=0x7fbd7000c490 comp rx=0 tx=0).stop 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 shutdown_connections 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd5806be30 0x7fbd5806e2f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7815e660 0x7fbd781e5bc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd780d1f90 0x7fbd780ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 --2- 172.21.15.28:0/4284648890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd780d0dd0 0x7fbd780b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 >> 172.21.15.28:0/4284648890 conn(0x7fbd7808bfc0 msgr2=0x7fbd780b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 shutdown_connections 2024-09-10T10:46:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:40.838+0000 7fbd7d000640 1 -- 172.21.15.28:0/4284648890 wait complete. 2024-09-10T10:46:40.857 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:46:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.013+0000 7f711bc00640 1 Processor -- start 2024-09-10T10:46:41.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.013+0000 7f711bc00640 1 -- start start 2024-09-10T10:46:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f711415b300 0x7f71140cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7114072d30 0x7f711406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f71140728a0 con 0x7f711415b300 2024-09-10T10:46:41.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f71140cdbe0 con 0x7f7114072d30 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.014+0000 7f711bc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71140cd900 con 0x7f71140cd3f0 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f711415b300 0x7f71140cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f711415b300 0x7f71140cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48946/0 (socket says 172.21.15.28:48946) 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 -- 172.21.15.28:0/3423745359 learned_addr learned my addr 172.21.15.28:0/3423745359 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119c00640 1 --2- 172.21.15.28:0/3423745359 >> v2:172.21.15.138:3300/0 conn(0x7f7114072d30 0x7f711406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7113e00640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 -- 172.21.15.28:0/3423745359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 msgr2=0x7f711406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 -- 172.21.15.28:0/3423745359 >> v2:172.21.15.138:3300/0 conn(0x7f7114072d30 msgr2=0x7f711406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 --2- 172.21.15.28:0/3423745359 >> v2:172.21.15.138:3300/0 conn(0x7f7114072d30 0x7f711406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.015+0000 7f7119200640 1 -- 172.21.15.28:0/3423745359 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7114072250 con 0x7f711415b300 2024-09-10T10:46:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.016+0000 7f7119200640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711415b300 0x7f71140cceb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7114072040 tx=0x7f710802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6ccf594f8f6e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f7113400640 1 -- 172.21.15.28:0/3423745359 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108037070 con 0x7f711415b300 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f7113400640 1 -- 172.21.15.28:0/3423745359 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7108032440 con 0x7f711415b300 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f7113400640 1 -- 172.21.15.28:0/3423745359 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108032810 con 0x7f711415b300 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 -- 172.21.15.28:0/3423745359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711415b300 msgr2=0x7f71140cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711415b300 0x7f71140cceb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7114072040 tx=0x7f710802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 -- 172.21.15.28:0/3423745359 shutdown_connections 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 --2- 172.21.15.28:0/3423745359 >> v2:172.21.15.138:3300/0 conn(0x7f7114072d30 0x7f711406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 --2- 172.21.15.28:0/3423745359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711415b300 0x7f71140cceb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 -- 172.21.15.28:0/3423745359 >> 172.21.15.28:0/3423745359 conn(0x7f7114067d70 msgr2=0x7f711406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 -- 172.21.15.28:0/3423745359 shutdown_connections 2024-09-10T10:46:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.017+0000 7f711bc00640 1 -- 172.21.15.28:0/3423745359 wait complete. 2024-09-10T10:46:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.019+0000 7f711bc00640 1 Processor -- start 2024-09-10T10:46:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.019+0000 7f711bc00640 1 -- start start 2024-09-10T10:46:41.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711414f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711415b300 0x7f7114147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71140d0160 con 0x7f7114072d30 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71140cffe0 con 0x7f711415b300 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f711bc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71140b3c40 con 0x7f71140cd3f0 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f7119200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f7119200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48960/0 (socket says 172.21.15.28:48960) 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f7119200640 1 -- 172.21.15.28:0/1218294901 learned_addr learned my addr 172.21.15.28:0/1218294901 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f7113e00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711414f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.020+0000 7f7119c00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711415b300 0x7f7114147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 -- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 msgr2=0x7f711414f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711414f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 -- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711415b300 msgr2=0x7f7114147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711415b300 0x7f7114147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 -- 172.21.15.28:0/1218294901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71141f8860 con 0x7f7114072d30 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7119200640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7108000c00 tx=0x7f71080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108041020 con 0x7f7114072d30 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7108030640 con 0x7f7114072d30 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108037040 con 0x7f7114072d30 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71141587a0 con 0x7f7114072d30 2024-09-10T10:46:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.021+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7114158d60 con 0x7f7114072d30 2024-09-10T10:46:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.023+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711406f180 con 0x7f7114072d30 2024-09-10T10:46:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.023+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7108044020 con 0x7f7114072d30 2024-09-10T10:46:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.024+0000 7f7110c00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 0x7f70f406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.024+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f71080ab400 con 0x7f7114072d30 2024-09-10T10:46:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.025+0000 7f7113e00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 0x7f70f406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.025+0000 7f7113e00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 0x7f70f406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7100006fd0 tx=0x7f7100008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.029+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f710804b030 con 0x7f7114072d30 2024-09-10T10:46:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.280+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f711406ebc0 con 0x7f7114072d30 2024-09-10T10:46:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.280+0000 7f7110c00640 1 -- 172.21.15.28:0/1218294901 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f7108079ed0 con 0x7f7114072d30 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:41.282 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.283 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:46:41.284 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.285 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.286 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.287 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.288 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.289 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.290 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.291 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.292 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:46:41.293 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:46:41.294 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:46:41.295 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 msgr2=0x7f70f406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 0x7f70f406e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7100006fd0 tx=0x7f7100008040 comp rx=0 tx=0).stop 2024-09-10T10:46:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 msgr2=0x7f7114159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7108000c00 tx=0x7f71080046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 shutdown_connections 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70f406be60 0x7f70f406e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711415b300 0x7f7114147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f71140cd3f0 0x7f711414f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 --2- 172.21.15.28:0/1218294901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7114072d30 0x7f7114159560 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 >> 172.21.15.28:0/1218294901 conn(0x7f7114067d70 msgr2=0x7f71140b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 shutdown_connections 2024-09-10T10:46:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.283+0000 7f711bc00640 1 -- 172.21.15.28:0/1218294901 wait complete. 2024-09-10T10:46:41.300 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:46:41.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.455+0000 7f02e6000640 1 Processor -- start 2024-09-10T10:46:41.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.455+0000 7f02e6000640 1 -- start start 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02e6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02e00720b0 0x7f02e0072490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02e6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e015c8d0 0x7f02e015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02e6000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e006f840 con 0x7f02e006e0b0 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02e6000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f02e015d250 con 0x7f02e00720b0 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02e6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02e00729d0 con 0x7f02e015c8d0 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02df400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02e00720b0 0x7f02e0072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02dfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f02e006e0b0 0x7f02e006e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41334/0 (socket says 172.21.15.28:41334) 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02dfe00640 1 -- 172.21.15.28:0/1492818850 learned_addr learned my addr 172.21.15.28:0/1492818850 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02d7e00640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e015c8d0 0x7f02e015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.457+0000 7f02df400640 1 -- 172.21.15.28:0/1492818850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e015c8d0 msgr2=0x7f02e015cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.458+0000 7f02df400640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e015c8d0 0x7f02e015cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.458+0000 7f02df400640 1 -- 172.21.15.28:0/1492818850 >> v1:172.21.15.28:6789/0 conn(0x7f02e006e0b0 legacy=0x7f02e006e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.458+0000 7f02df400640 1 -- 172.21.15.28:0/1492818850 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e015d4b0 con 0x7f02e00720b0 2024-09-10T10:46:41.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.458+0000 7f02dea00640 1 -- 172.21.15.28:0/1492818850 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4189316650 0 0) 0x7f02e006f840 con 0x7f02e006e0b0 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02df400640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e00720b0 0x7f02e0072490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f02d00097a0 tx=0x7f02d002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cce95ae0a71458 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02dea00640 1 -- 172.21.15.28:0/1492818850 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0037070 con 0x7f02e00720b0 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02dea00640 1 -- 172.21.15.28:0/1492818850 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d0032440 con 0x7f02e00720b0 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02dea00640 1 -- 172.21.15.28:0/1492818850 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0032810 con 0x7f02e00720b0 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02e6000640 1 -- 172.21.15.28:0/1492818850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e00720b0 msgr2=0x7f02e0072490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02e6000640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e00720b0 0x7f02e0072490 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f02d00097a0 tx=0x7f02d002a180 comp rx=0 tx=0).stop 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.459+0000 7f02e6000640 1 -- 172.21.15.28:0/1492818850 shutdown_connections 2024-09-10T10:46:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.460+0000 7f02e6000640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e015c8d0 0x7f02e015cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.460+0000 7f02e6000640 1 --2- 172.21.15.28:0/1492818850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e00720b0 0x7f02e0072490 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.460+0000 7f02e6000640 1 -- 172.21.15.28:0/1492818850 >> 172.21.15.28:0/1492818850 conn(0x7f02e0067d70 msgr2=0x7f02e006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.460+0000 7f02e6000640 1 -- 172.21.15.28:0/1492818850 shutdown_connections 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.460+0000 7f02e6000640 1 -- 172.21.15.28:0/1492818850 wait complete. 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.461+0000 7f02e6000640 1 Processor -- start 2024-09-10T10:46:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.461+0000 7f02e6000640 1 -- start start 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e006e0b0 0x7f02e00af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00720b0 0x7f02e015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02e015dd50 con 0x7f02e015c8d0 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02e015da70 con 0x7f02e00720b0 2024-09-10T10:46:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.462+0000 7f02e6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02e015e030 con 0x7f02e006e0b0 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48976/0 (socket says 172.21.15.28:48976) 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 -- 172.21.15.28:0/1335430616 learned_addr learned my addr 172.21.15.28:0/1335430616 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02d7e00640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00720b0 0x7f02e015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02df400640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e006e0b0 0x7f02e00af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 -- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e006e0b0 msgr2=0x7f02e00af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e006e0b0 0x7f02e00af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 -- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00720b0 msgr2=0x7f02e015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00720b0 0x7f02e015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 -- 172.21.15.28:0/1335430616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e00afc50 con 0x7f02e015c8d0 2024-09-10T10:46:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.463+0000 7f02dfe00640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c7a0 tx=0x7f02d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.464+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d8018070 con 0x7f02e015c8d0 2024-09-10T10:46:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.464+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d800a640 con 0x7f02e015c8d0 2024-09-10T10:46:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.464+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d800aab0 con 0x7f02e015c8d0 2024-09-10T10:46:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.464+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e00a9490 con 0x7f02e015c8d0 2024-09-10T10:46:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.464+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e00a9a10 con 0x7f02e015c8d0 2024-09-10T10:46:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.465+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e0146110 con 0x7f02e015c8d0 2024-09-10T10:46:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.466+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02d801b070 con 0x7f02e015c8d0 2024-09-10T10:46:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.467+0000 7f02d7400640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 0x7f02bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.467+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f02d808a250 con 0x7f02e015c8d0 2024-09-10T10:46:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.467+0000 7f02df400640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 0x7f02bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.468+0000 7f02df400640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 0x7f02bc06e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f02d0034000 tx=0x7f02d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.471+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02e00afc50 con 0x7f02e015c8d0 2024-09-10T10:46:41.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.656+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f02e015e810 con 0x7f02e015c8d0 2024-09-10T10:46:41.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.656+0000 7f02d7400640 1 -- 172.21.15.28:0/1335430616 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7f02d8058760 con 0x7f02e015c8d0 2024-09-10T10:46:41.657 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:41.657 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:46:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:46:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:46:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:46:41.658 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:46:41.658 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.658+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 msgr2=0x7f02bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.658+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 0x7f02bc06e2f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f02d0034000 tx=0x7f02d002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.658+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 msgr2=0x7f02e0140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.658+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f02d800c7a0 tx=0x7f02d800cc60 comp rx=0 tx=0).stop 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 shutdown_connections 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f02bc06be30 0x7f02bc06e2f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02e015c8d0 0x7f02e0140530 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02e00720b0 0x7f02e015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 --2- 172.21.15.28:0/1335430616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02e006e0b0 0x7f02e00af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 >> 172.21.15.28:0/1335430616 conn(0x7f02e0067d70 msgr2=0x7f02e00b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 shutdown_connections 2024-09-10T10:46:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.659+0000 7f02e6000640 1 -- 172.21.15.28:0/1335430616 wait complete. 2024-09-10T10:46:41.675 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:46:41.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:46:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.834+0000 7f48df200640 1 Processor -- start 2024-09-10T10:46:41.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.834+0000 7f48df200640 1 -- start start 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48df200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48e00d29c0 0x7f48e00d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48df200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48e00d1060 0x7f48e006d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48df200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48e00d52e0 con 0x7f48e015e660 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48df200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f48e00d3410 con 0x7f48e00d29c0 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48df200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f48e00d2e50 con 0x7f48e00d1060 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48dd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48e00d29c0 0x7f48e00d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48dd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f48e00d29c0 0x7f48e00d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48990/0 (socket says 172.21.15.28:48990) 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48dd400640 1 -- 172.21.15.28:0/1698455932 learned_addr learned my addr 172.21.15.28:0/1698455932 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.836+0000 7f48de800640 1 --2- 172.21.15.28:0/1698455932 >> v2:172.21.15.138:3300/0 conn(0x7f48e00d1060 0x7f48e006d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.837+0000 7f48dd400640 1 -- 172.21.15.28:0/1698455932 >> v2:172.21.15.138:3300/0 conn(0x7f48e00d1060 msgr2=0x7f48e006d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.837+0000 7f48dd400640 1 --2- 172.21.15.28:0/1698455932 >> v2:172.21.15.138:3300/0 conn(0x7f48e00d1060 0x7f48e006d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.837+0000 7f48dd400640 1 -- 172.21.15.28:0/1698455932 >> v1:172.21.15.138:6789/0 conn(0x7f48e015e660 legacy=0x7f48e00d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.837+0000 7f48dd400640 1 -- 172.21.15.28:0/1698455932 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e00d3130 con 0x7f48e00d29c0 2024-09-10T10:46:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.838+0000 7f48dd400640 1 --2- 172.21.15.28:0/1698455932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d29c0 0x7f48e00d0b20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f48e00cebf0 tx=0x7f48cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5708f1f8bca4b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.839+0000 7f48dca00640 1 -- 172.21.15.28:0/1698455932 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc037070 con 0x7f48e00d29c0 2024-09-10T10:46:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.839+0000 7f48dca00640 1 -- 172.21.15.28:0/1698455932 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48cc032440 con 0x7f48e00d29c0 2024-09-10T10:46:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48dca00640 1 -- 172.21.15.28:0/1698455932 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc032810 con 0x7f48e00d29c0 2024-09-10T10:46:41.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 -- 172.21.15.28:0/1698455932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d29c0 msgr2=0x7f48e00d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 --2- 172.21.15.28:0/1698455932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d29c0 0x7f48e00d0b20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f48e00cebf0 tx=0x7f48cc02a270 comp rx=0 tx=0).stop 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 -- 172.21.15.28:0/1698455932 shutdown_connections 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 --2- 172.21.15.28:0/1698455932 >> v2:172.21.15.138:3300/0 conn(0x7f48e00d1060 0x7f48e006d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 --2- 172.21.15.28:0/1698455932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d29c0 0x7f48e00d0b20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.840+0000 7f48df200640 1 -- 172.21.15.28:0/1698455932 >> 172.21.15.28:0/1698455932 conn(0x7f48e008bfc0 msgr2=0x7f48e00d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.841+0000 7f48df200640 1 -- 172.21.15.28:0/1698455932 shutdown_connections 2024-09-10T10:46:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.841+0000 7f48df200640 1 -- 172.21.15.28:0/1698455932 wait complete. 2024-09-10T10:46:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.842+0000 7f48df200640 1 Processor -- start 2024-09-10T10:46:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.842+0000 7f48df200640 1 -- start start 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e015e660 0x7f48e015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e014d910 0x7f48e014af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48e006dc10 con 0x7f48e00d1060 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48e006d1b0 con 0x7f48e014d910 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.843+0000 7f48df200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48e006dd90 con 0x7f48e015e660 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49004/0 (socket says 172.21.15.28:49004) 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 -- 172.21.15.28:0/476128919 learned_addr learned my addr 172.21.15.28:0/476128919 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dd400640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e015e660 0x7f48e015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48de800640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e014d910 0x7f48e014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 -- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e015e660 msgr2=0x7f48e015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e015e660 0x7f48e015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 -- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e014d910 msgr2=0x7f48e014af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e014d910 0x7f48e014af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 -- 172.21.15.28:0/476128919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e01f87f0 con 0x7f48e00d1060 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.844+0000 7f48dde00640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c3f0 tx=0x7f48d000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.845+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d00052d0 con 0x7f48e00d1060 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.845+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48d0005490 con 0x7f48e00d1060 2024-09-10T10:46:41.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.845+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0007260 con 0x7f48e00d1060 2024-09-10T10:46:41.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.845+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e014deb0 con 0x7f48e00d1060 2024-09-10T10:46:41.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.845+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e014e470 con 0x7f48e00d1060 2024-09-10T10:46:41.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.846+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e006d6d0 con 0x7f48e00d1060 2024-09-10T10:46:41.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.847+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f48d0007400 con 0x7f48e00d1060 2024-09-10T10:46:41.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.848+0000 7f48d6000640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 0x7f48c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:41.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.848+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5249+0+0 (secure 0 0 0) 0x7f48d008a180 con 0x7f48e00d1060 2024-09-10T10:46:41.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.848+0000 7f48dd400640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 0x7f48c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:41.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.849+0000 7f48dd400640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 0x7f48c006e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48cc034000 tx=0x7f48cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:41.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:41.853+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f48d0058600 con 0x7f48e00d1060 2024-09-10T10:46:42.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.036+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f48e00d04d0 con 0x7f48e00d1060 2024-09-10T10:46:42.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.844+0000 7f48d6000640 1 -- 172.21.15.28:0/476128919 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7f48d005c900 con 0x7f48e00d1060 2024-09-10T10:46:42.845 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:46:42.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 msgr2=0x7f48c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 0x7f48c006e2f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f48cc034000 tx=0x7f48cc02e000 comp rx=0 tx=0).stop 2024-09-10T10:46:42.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 msgr2=0x7f48e01f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:42.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f48d000c3f0 tx=0x7f48d000c8b0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 shutdown_connections 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f48c006be30 0x7f48c006e2f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48e014d910 0x7f48e014af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48e015e660 0x7f48e015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 --2- 172.21.15.28:0/476128919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48e00d1060 0x7f48e01f82b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 >> 172.21.15.28:0/476128919 conn(0x7f48e008bfc0 msgr2=0x7f48e00c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 shutdown_connections 2024-09-10T10:46:42.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:42.847+0000 7f48df200640 1 -- 172.21.15.28:0/476128919 wait complete. 2024-09-10T10:46:42.872 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:46:43.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.039+0000 7f89b9400640 1 Processor -- start 2024-09-10T10:46:43.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.040+0000 7f89b9400640 1 -- start start 2024-09-10T10:46:43.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f89b4156740 0x7f89b40b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40cfea0 0x7f89b40ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b9400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b40b30e0 con 0x7f89b40b4af0 2024-09-10T10:46:43.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b9400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f89b406edc0 con 0x7f89b40cfea0 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f89b40b33c0 con 0x7f89b4156740 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40cfea0 0x7f89b40ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40cfea0 0x7f89b40ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49014/0 (socket says 172.21.15.28:49014) 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 -- 172.21.15.28:0/1751692280 learned_addr learned my addr 172.21.15.28:0/1751692280 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b2800640 1 --2- 172.21.15.28:0/1751692280 >> v2:172.21.15.138:3300/0 conn(0x7f89b4156740 0x7f89b40b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 -- 172.21.15.28:0/1751692280 >> v2:172.21.15.138:3300/0 conn(0x7f89b4156740 msgr2=0x7f89b40b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 --2- 172.21.15.28:0/1751692280 >> v2:172.21.15.138:3300/0 conn(0x7f89b4156740 0x7f89b40b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 -- 172.21.15.28:0/1751692280 >> v1:172.21.15.138:6789/0 conn(0x7f89b40b4af0 legacy=0x7f89b40cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.041+0000 7f89b3200640 1 -- 172.21.15.28:0/1751692280 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40b36a0 con 0x7f89b40cfea0 2024-09-10T10:46:43.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.042+0000 7f89b3200640 1 --2- 172.21.15.28:0/1751692280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40cfea0 0x7f89b40ce060 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b4150 tx=0x7f89ac02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6e47dd3aa98c95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b1e00640 1 -- 172.21.15.28:0/1751692280 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac036070 con 0x7f89b40cfea0 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b1e00640 1 -- 172.21.15.28:0/1751692280 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac031600 con 0x7f89b40cfea0 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b1e00640 1 -- 172.21.15.28:0/1751692280 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0319d0 con 0x7f89b40cfea0 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 -- 172.21.15.28:0/1751692280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40cfea0 msgr2=0x7f89b40ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 --2- 172.21.15.28:0/1751692280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40cfea0 0x7f89b40ce060 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f89b40b4150 tx=0x7f89ac02a730 comp rx=0 tx=0).stop 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 -- 172.21.15.28:0/1751692280 shutdown_connections 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 --2- 172.21.15.28:0/1751692280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40cfea0 0x7f89b40ce060 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 --2- 172.21.15.28:0/1751692280 >> v2:172.21.15.138:3300/0 conn(0x7f89b4156740 0x7f89b40b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 -- 172.21.15.28:0/1751692280 >> 172.21.15.28:0/1751692280 conn(0x7f89b406bdf0 msgr2=0x7f89b406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 -- 172.21.15.28:0/1751692280 shutdown_connections 2024-09-10T10:46:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.043+0000 7f89b9400640 1 -- 172.21.15.28:0/1751692280 wait complete. 2024-09-10T10:46:43.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.045+0000 7f89b9400640 1 Processor -- start 2024-09-10T10:46:43.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.045+0000 7f89b9400640 1 -- start start 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89b40b4af0 0x7f89b4152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 0x7f89b4152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40d0890 con 0x7f89b4156740 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40b3c60 con 0x7f89b40cfea0 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b9400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40d0a10 con 0x7f89b40b4af0 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49020/0 (socket says 172.21.15.28:49020) 2024-09-10T10:46:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b3200640 1 -- 172.21.15.28:0/3898766138 learned_addr learned my addr 172.21.15.28:0/3898766138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89b2800640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89b40b4af0 0x7f89b4152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.046+0000 7f89abe00640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 0x7f89b4152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 -- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89b40b4af0 msgr2=0x7f89b4152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89b40b4af0 0x7f89b4152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 -- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 msgr2=0x7f89b4152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 0x7f89b4152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 -- 172.21.15.28:0/3898766138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41f4260 con 0x7f89b4156740 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89abe00640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 0x7f89b4152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b3200640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02ad20 tx=0x7f89ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac03f020 con 0x7f89b4156740 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac037040 con 0x7f89b4156740 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac036070 con 0x7f89b4156740 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.047+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b41f4540 con 0x7f89b4156740 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.048+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b4140f70 con 0x7f89b4156740 2024-09-10T10:46:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.049+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b4068f50 con 0x7f89b4156740 2024-09-10T10:46:43.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.049+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f89ac042020 con 0x7f89b4156740 2024-09-10T10:46:43.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.051+0000 7f89aaa00640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:43.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.051+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5619+0+0 (secure 0 0 0) 0x7f89ac0ab2a0 con 0x7f89b4156740 2024-09-10T10:46:43.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.051+0000 7f89b2800640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:43.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.052+0000 7f89b2800640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 0x7f899006e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006fd0 tx=0x7f89a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:43.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.056+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89ac049030 con 0x7f89b4156740 2024-09-10T10:46:43.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.242+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f89b4156c40 con 0x7f89b4156740 2024-09-10T10:46:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.852+0000 7f89aaa00640 1 -- 172.21.15.28:0/3898766138 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f89ac079c00 con 0x7f89b4156740 2024-09-10T10:46:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 msgr2=0x7f899006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 0x7f899006e320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006fd0 tx=0x7f89a4008040 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 msgr2=0x7f89b41401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02ad20 tx=0x7f89ac004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 shutdown_connections 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b4156740 0x7f89b41401b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89b40cfea0 0x7f89b4152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 --2- 172.21.15.28:0/3898766138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89b40b4af0 0x7f89b4152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 >> 172.21.15.28:0/3898766138 conn(0x7f89b406bdf0 msgr2=0x7f89b40b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.856+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 shutdown_connections 2024-09-10T10:46:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:43.857+0000 7f89b9400640 1 -- 172.21.15.28:0/3898766138 wait complete. 2024-09-10T10:46:43.891 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:46:44.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.051+0000 7f9161400640 1 Processor -- start 2024-09-10T10:46:44.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.051+0000 7f9161400640 1 -- start start 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9161400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f915c0cf790 0x7f915c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9161400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f915c0b30e0 con 0x7f915c156740 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9161400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f915c0b33c0 con 0x7f915c0ce060 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9161400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f915c0d0140 con 0x7f915c0cf790 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9153e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f915c0cf790 0x7f915c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9153e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f915c0cf790 0x7f915c0cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49952/0 (socket says 172.21.15.28:49952) 2024-09-10T10:46:44.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9153e00640 1 -- 172.21.15.28:0/4012990221 learned_addr learned my addr 172.21.15.28:0/4012990221 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:44.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9153e00640 1 -- 172.21.15.28:0/4012990221 >> v1:172.21.15.138:6789/0 conn(0x7f915c0ce060 legacy=0x7f915c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.052+0000 7f9153e00640 1 -- 172.21.15.28:0/4012990221 >> v1:172.21.15.28:6789/0 conn(0x7f915c156740 legacy=0x7f915c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.053+0000 7f9153e00640 1 -- 172.21.15.28:0/4012990221 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c0b36a0 con 0x7f915c0cf790 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9153e00640 1 --2- 172.21.15.28:0/4012990221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c0cf790 0x7f915c0cfc00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9144009870 tx=0x7f914402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1db48f29fb9f139 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9159e00640 1 -- 172.21.15.28:0/4012990221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144037070 con 0x7f915c0cf790 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9159e00640 1 -- 172.21.15.28:0/4012990221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f914402ae50 con 0x7f915c0cf790 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9159e00640 1 -- 172.21.15.28:0/4012990221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9144032860 con 0x7f915c0cf790 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9161400640 1 -- 172.21.15.28:0/4012990221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c0cf790 msgr2=0x7f915c0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9161400640 1 --2- 172.21.15.28:0/4012990221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c0cf790 0x7f915c0cfc00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9144009870 tx=0x7f914402a300 comp rx=0 tx=0).stop 2024-09-10T10:46:44.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9161400640 1 -- 172.21.15.28:0/4012990221 shutdown_connections 2024-09-10T10:46:44.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9161400640 1 --2- 172.21.15.28:0/4012990221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c0cf790 0x7f915c0cfc00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.054+0000 7f9161400640 1 -- 172.21.15.28:0/4012990221 >> 172.21.15.28:0/4012990221 conn(0x7f915c06bdf0 msgr2=0x7f915c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:44.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.055+0000 7f9161400640 1 -- 172.21.15.28:0/4012990221 shutdown_connections 2024-09-10T10:46:44.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.055+0000 7f9161400640 1 -- 172.21.15.28:0/4012990221 wait complete. 2024-09-10T10:46:44.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.056+0000 7f9161400640 1 Processor -- start 2024-09-10T10:46:44.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.056+0000 7f9161400640 1 -- start start 2024-09-10T10:46:44.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f915c0ce060 0x7f915c14b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:44.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f915c0cf790 0x7f915c073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:44.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:44.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f915c0b3c60 con 0x7f915c156740 2024-09-10T10:46:44.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f915c06edc0 con 0x7f915c0ce060 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.057+0000 7f9161400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f915c0b4cc0 con 0x7f915c0cf790 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49962/0 (socket says 172.21.15.28:49962) 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 -- 172.21.15.28:0/3420005269 learned_addr learned my addr 172.21.15.28:0/3420005269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915a800640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f915c0ce060 0x7f915c14b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f9153e00640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f915c0cf790 0x7f915c073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 -- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f915c0cf790 msgr2=0x7f915c073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f915c0cf790 0x7f915c073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 -- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f915c0ce060 msgr2=0x7f915c14b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f915c0ce060 0x7f915c14b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 -- 172.21.15.28:0/3420005269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f915c1548d0 con 0x7f915c156740 2024-09-10T10:46:44.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.058+0000 7f915b200640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f915400c6b0 tx=0x7f915400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:44.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.059+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154003b70 con 0x7f915c156740 2024-09-10T10:46:44.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.059+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9154003d30 con 0x7f915c156740 2024-09-10T10:46:44.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.059+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f915c150850 con 0x7f915c156740 2024-09-10T10:46:44.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.059+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154005310 con 0x7f915c156740 2024-09-10T10:46:44.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.059+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f915c150e10 con 0x7f915c156740 2024-09-10T10:46:44.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.060+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f915c06c1d0 con 0x7f915c156740 2024-09-10T10:46:44.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.061+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91540054b0 con 0x7f915c156740 2024-09-10T10:46:44.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.062+0000 7f9152a00640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 0x7f913806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:44.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.062+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6001+0+0 (secure 0 0 0) 0x7f915408ad40 con 0x7f915c156740 2024-09-10T10:46:44.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.062+0000 7f915a800640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 0x7f913806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:44.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.063+0000 7f915a800640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 0x7f913806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f914c000f60 tx=0x7f914c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:44.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.067+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f915c06c1d0 con 0x7f915c156740 2024-09-10T10:46:44.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.289+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f915c06ce20 con 0x7f915c156740 2024-09-10T10:46:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.866+0000 7f9152a00640 1 -- 172.21.15.28:0/3420005269 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v44) ==== 173+0+0 (secure 0 0 0) 0x7f9154058ed0 con 0x7f915c156740 2024-09-10T10:46:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 9 and data pool 10 2024-09-10T10:46:44.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.876+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 msgr2=0x7f913806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.876+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 0x7f913806e2f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f914c000f60 tx=0x7f914c009040 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.876+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 msgr2=0x7f915c15b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.876+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f915400c6b0 tx=0x7f915400cb70 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 shutdown_connections 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f913806be30 0x7f913806e2f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f915c156740 0x7f915c15b3f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f915c0cf790 0x7f915c073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 --2- 172.21.15.28:0/3420005269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f915c0ce060 0x7f915c14b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.877+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 >> 172.21.15.28:0/3420005269 conn(0x7f915c06bdf0 msgr2=0x7f915c15c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:44.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.878+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 shutdown_connections 2024-09-10T10:46:44.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:44.878+0000 7f9161400640 1 -- 172.21.15.28:0/3420005269 wait complete. 2024-09-10T10:46:44.905 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:45.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.062+0000 7f98d8a00640 1 Processor -- start 2024-09-10T10:46:45.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.062+0000 7f98d8a00640 1 -- start start 2024-09-10T10:46:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f98d415e660 0x7f98d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d8a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d40cf000 con 0x7f98d40d0dd0 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d8a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f98d40b37d0 con 0x7f98d415e660 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98d40cf330 con 0x7f98d40d1f90 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f98d415e660 0x7f98d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f98d415e660 0x7f98d406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49966/0 (socket says 172.21.15.28:49966) 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.063+0000 7f98d1e00640 1 -- 172.21.15.28:0/2841173821 learned_addr learned my addr 172.21.15.28:0/2841173821 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d3200640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d1e00640 1 -- 172.21.15.28:0/2841173821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 msgr2=0x7f98d40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d1e00640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d1e00640 1 -- 172.21.15.28:0/2841173821 >> v1:172.21.15.28:6789/0 conn(0x7f98d40d0dd0 legacy=0x7f98d40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d1e00640 1 -- 172.21.15.28:0/2841173821 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d40d29a0 con 0x7f98d415e660 2024-09-10T10:46:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.064+0000 7f98d3200640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:45.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.065+0000 7f98d1e00640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d406ccd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009870 tx=0x7f98c002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb066e799aa4d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.065+0000 7f98d1400640 1 -- 172.21.15.28:0/2841173821 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0037070 con 0x7f98d415e660 2024-09-10T10:46:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.065+0000 7f98d1400640 1 -- 172.21.15.28:0/2841173821 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c0032440 con 0x7f98d415e660 2024-09-10T10:46:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.065+0000 7f98d1400640 1 -- 172.21.15.28:0/2841173821 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0032810 con 0x7f98d415e660 2024-09-10T10:46:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 -- 172.21.15.28:0/2841173821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 msgr2=0x7f98d406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d406ccd0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009870 tx=0x7f98c002a250 comp rx=0 tx=0).stop 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 -- 172.21.15.28:0/2841173821 shutdown_connections 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d40cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 --2- 172.21.15.28:0/2841173821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d406ccd0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 -- 172.21.15.28:0/2841173821 >> 172.21.15.28:0/2841173821 conn(0x7f98d408bfc0 msgr2=0x7f98d40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 -- 172.21.15.28:0/2841173821 shutdown_connections 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.066+0000 7f98d8a00640 1 -- 172.21.15.28:0/2841173821 wait complete. 2024-09-10T10:46:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.067+0000 7f98d8a00640 1 Processor -- start 2024-09-10T10:46:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 -- start start 2024-09-10T10:46:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 0x7f98d40b9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d4160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98d40d3220 con 0x7f98d415e660 2024-09-10T10:46:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98d40d2f40 con 0x7f98d40d0dd0 2024-09-10T10:46:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.068+0000 7f98d8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98d40d33a0 con 0x7f98d40d1f90 2024-09-10T10:46:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49974/0 (socket says 172.21.15.28:49974) 2024-09-10T10:46:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 -- 172.21.15.28:0/1874206562 learned_addr learned my addr 172.21.15.28:0/1874206562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d1e00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d4160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d2800640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 0x7f98d40b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 -- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 msgr2=0x7f98d4160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d4160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 -- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 msgr2=0x7f98d40b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 0x7f98d40b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d3200640 1 -- 172.21.15.28:0/1874206562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d40b86d0 con 0x7f98d415e660 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.069+0000 7f98d2800640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 0x7f98d40b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98d3200640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98cc007f20 tx=0x7f98cc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc01a070 con 0x7f98d415e660 2024-09-10T10:46:45.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc003b90 con 0x7f98d415e660 2024-09-10T10:46:45.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d40b89b0 con 0x7f98d415e660 2024-09-10T10:46:45.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc004030 con 0x7f98d415e660 2024-09-10T10:46:45.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.070+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d40b8f70 con 0x7f98d415e660 2024-09-10T10:46:45.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.071+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d40d2440 con 0x7f98d415e660 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.072+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f98cc01d070 con 0x7f98d415e660 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.073+0000 7f98caa00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 0x7f98b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.073+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f98cc08b310 con 0x7f98d415e660 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.073+0000 7f98d2800640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 0x7f98b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.074+0000 7f98d2800640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 0x7f98b406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98c4001700 tx=0x7f98c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.077+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98cc010070 con 0x7f98d415e660 2024-09-10T10:46:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.262+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98d40d1b20 con 0x7f98d415e660 2024-09-10T10:46:45.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.263+0000 7f98caa00640 1 -- 172.21.15.28:0/1874206562 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17029 (secure 0 0 0) 0x7f98cc059440 con 0x7f98d415e660 2024-09-10T10:46:45.265 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:45.265 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":69,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:44.290970+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:42.078607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:43.284849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 msgr2=0x7f98b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 0x7f98b406e2f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98c4001700 tx=0x7f98c4009040 comp rx=0 tx=0).stop 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 msgr2=0x7f98d415ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98cc007f20 tx=0x7f98cc00d560 comp rx=0 tx=0).stop 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 shutdown_connections 2024-09-10T10:46:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f98b406be30 0x7f98b406e2f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f98d415e660 0x7f98d415ea40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98d40d1f90 0x7f98d4160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 --2- 172.21.15.28:0/1874206562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98d40d0dd0 0x7f98d40b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 >> 172.21.15.28:0/1874206562 conn(0x7f98d408bfc0 msgr2=0x7f98d40b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 shutdown_connections 2024-09-10T10:46:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.266+0000 7f98d8a00640 1 -- 172.21.15.28:0/1874206562 wait complete. 2024-09-10T10:46:45.283 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:45.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.437+0000 7f9dc2e00640 1 Processor -- start 2024-09-10T10:46:45.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.438+0000 7f9dc2e00640 1 -- start start 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9dc2e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9dbc072320 0x7f9dbc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9dc2e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc146140 con 0x7f9dbc072de0 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9dc2e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dbc06eed0 con 0x7f9dbc15e1f0 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9dc2e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9dbc0cd740 con 0x7f9dbc072320 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9db3400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9dbc072de0 0x7f9dbc06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56682/0 (socket says 172.21.15.28:56682) 2024-09-10T10:46:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9db3400640 1 -- 172.21.15.28:0/3600550669 learned_addr learned my addr 172.21.15.28:0/3600550669 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.439+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/3600550669 >> v2:172.21.15.138:3300/0 conn(0x7f9dbc072320 0x7f9dbc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142780427 0 0) 0x7f9dbc146140 con 0x7f9dbc072de0 2024-09-10T10:46:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9da0003520 con 0x7f9dbc072de0 2024-09-10T10:46:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4092872207 0 0) 0x7f9da0003520 con 0x7f9dbc072de0 2024-09-10T10:46:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 >> v2:172.21.15.138:3300/0 conn(0x7f9dbc072320 msgr2=0x7f9dbc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 --2- 172.21.15.28:0/3600550669 >> v2:172.21.15.138:3300/0 conn(0x7f9dbc072320 0x7f9dbc0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 >> v1:172.21.15.138:6789/0 conn(0x7f9dbc15e1f0 legacy=0x7f9dbc06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0cda20 con 0x7f9dbc072de0 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f9da4001670 con 0x7f9dbc072de0 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/3600550669 >> v2:172.21.15.138:3300/0 conn(0x7f9dbc072320 0x7f9dbc0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 >> v1:172.21.15.28:6789/0 conn(0x7f9dbc072de0 legacy=0x7f9dbc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9da00039b0 0x7f9da0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 0x7f9da0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da00051c0 0x7f9da000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc0cd740 con 0x7f9da0004800 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9da000c160 con 0x7f9da00051c0 2024-09-10T10:46:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9da000c360 con 0x7f9da00039b0 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9db3400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 0x7f9da0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9da00039b0 0x7f9da0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.440+0000 7f9dc0e00640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da00051c0 0x7f9da000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9db3400640 1 -- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9da00039b0 msgr2=0x7f9da0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9db3400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9da00039b0 0x7f9da0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9db3400640 1 -- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da00051c0 msgr2=0x7f9da000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9db3400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da00051c0 0x7f9da000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9db3400640 1 -- 172.21.15.28:0/3600550669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da0003520 con 0x7f9da0004800 2024-09-10T10:46:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.441+0000 7f9dc0e00640 1 -- 172.21.15.28:0/3600550669 reap_dead start 2024-09-10T10:46:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.442+0000 7f9db3400640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 0x7f9da0004c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c240 tx=0x7f9da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d33c492ae8ecd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.442+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4007d10 con 0x7f9da0004800 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.442+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da4005ad0 con 0x7f9da0004800 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.442+0000 7f9dbb400640 1 -- 172.21.15.28:0/3600550669 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4007070 con 0x7f9da0004800 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 -- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 msgr2=0x7f9da0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 0x7f9da0004c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c240 tx=0x7f9da0005830 comp rx=0 tx=0).stop 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 -- 172.21.15.28:0/3600550669 shutdown_connections 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/3600550669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9da0004800 0x7f9da0004c80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 -- 172.21.15.28:0/3600550669 >> 172.21.15.28:0/3600550669 conn(0x7f9dbc067d70 msgr2=0x7f9dbc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 -- 172.21.15.28:0/3600550669 shutdown_connections 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.443+0000 7f9dc2e00640 1 -- 172.21.15.28:0/3600550669 wait complete. 2024-09-10T10:46:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.444+0000 7f9dc2e00640 1 Processor -- start 2024-09-10T10:46:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.444+0000 7f9dc2e00640 1 -- start start 2024-09-10T10:46:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da0004800 0x7f9dbc1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 0x7f9dbc1e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc072700 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dbc15c9e0 con 0x7f9da0004800 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.445+0000 7f9dc2e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9dbc072880 con 0x7f9dbc0b35d0 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50002/0 (socket says 172.21.15.28:50002) 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 -- 172.21.15.28:0/2281640117 learned_addr learned my addr 172.21.15.28:0/2281640117 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da0004800 0x7f9dbc1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9db3400640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 0x7f9dbc1e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 -- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 msgr2=0x7f9dbc1e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 0x7f9dbc1e77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 -- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da0004800 msgr2=0x7f9dbc1f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da0004800 0x7f9dbc1f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 -- 172.21.15.28:0/2281640117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dbc0b3ad0 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9db3400640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 0x7f9dbc1e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.446+0000 7f9dc0e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc15e680 tx=0x7f9db400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.447+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db401a070 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.447+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9db400fe90 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.447+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dbc0b3db0 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.447+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4003970 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.447+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dbc0b4370 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.448+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dbc0cce60 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.449+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9db4003b10 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.450+0000 7f9db8c00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.450+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f9db408a380 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.450+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.451+0000 7f9dbbe00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 0x7f9d9806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0cf340 tx=0x7f9dac009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.454+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9db40585c0 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.641+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dbc0ce4c0 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.642+0000 7f9db8c00640 1 -- 172.21.15.28:0/2281640117 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17029 (secure 0 0 0) 0x7f9db405c8c0 con 0x7f9dbc0ab1a0 2024-09-10T10:46:45.643 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:45.643 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":69,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:44.290970+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:42.078607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:43.284849+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:45.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 msgr2=0x7f9d9806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 0x7f9d9806e320 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0cf340 tx=0x7f9dac009190 comp rx=0 tx=0).stop 2024-09-10T10:46:45.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 msgr2=0x7f9dbc1e7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc15e680 tx=0x7f9db400fbe0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 shutdown_connections 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d9806be60 0x7f9d9806e320 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dbc0ab1a0 0x7f9dbc1e7d10 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dbc0b35d0 0x7f9dbc1e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 --2- 172.21.15.28:0/2281640117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da0004800 0x7f9dbc1f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 >> 172.21.15.28:0/2281640117 conn(0x7f9dbc067d70 msgr2=0x7f9dbc06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 shutdown_connections 2024-09-10T10:46:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.644+0000 7f9dc2e00640 1 -- 172.21.15.28:0/2281640117 wait complete. 2024-09-10T10:46:45.661 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:46:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.821+0000 7f7667600640 1 Processor -- start 2024-09-10T10:46:45.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.822+0000 7f7667600640 1 -- start start 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7667600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7660156740 0x7f76600b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7667600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f76600cfea0 0x7f76600ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7667600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76600b30e0 con 0x7f76600b4af0 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7667600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f766006edc0 con 0x7f76600cfea0 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7667600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76600b33c0 con 0x7f7660156740 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f76600cfea0 0x7f76600ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f76600cfea0 0x7f76600ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50006/0 (socket says 172.21.15.28:50006) 2024-09-10T10:46:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 -- 172.21.15.28:0/4030658050 learned_addr learned my addr 172.21.15.28:0/4030658050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7664c00640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7660156740 0x7f76600b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 -- 172.21.15.28:0/4030658050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7660156740 msgr2=0x7f76600b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7660156740 0x7f76600b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 -- 172.21.15.28:0/4030658050 >> v1:172.21.15.138:6789/0 conn(0x7f76600b4af0 legacy=0x7f76600cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.823+0000 7f7665600640 1 -- 172.21.15.28:0/4030658050 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76600b36a0 con 0x7f76600cfea0 2024-09-10T10:46:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7665600640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76600cfea0 0x7f76600ce060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7658009e50 tx=0x7f765802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85186883f68c1639 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f765f400640 1 -- 172.21.15.28:0/4030658050 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658036070 con 0x7f76600cfea0 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f765f400640 1 -- 172.21.15.28:0/4030658050 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658031600 con 0x7f76600cfea0 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f765f400640 1 -- 172.21.15.28:0/4030658050 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580319d0 con 0x7f76600cfea0 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 -- 172.21.15.28:0/4030658050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76600cfea0 msgr2=0x7f76600ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76600cfea0 0x7f76600ce060 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7658009e50 tx=0x7f765802a710 comp rx=0 tx=0).stop 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 -- 172.21.15.28:0/4030658050 shutdown_connections 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76600cfea0 0x7f76600ce060 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 --2- 172.21.15.28:0/4030658050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7660156740 0x7f76600b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.825+0000 7f7667600640 1 -- 172.21.15.28:0/4030658050 >> 172.21.15.28:0/4030658050 conn(0x7f766006bdf0 msgr2=0x7f766006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.826+0000 7f7667600640 1 -- 172.21.15.28:0/4030658050 shutdown_connections 2024-09-10T10:46:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.826+0000 7f7667600640 1 -- 172.21.15.28:0/4030658050 wait complete. 2024-09-10T10:46:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.827+0000 7f7667600640 1 Processor -- start 2024-09-10T10:46:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.827+0000 7f7667600640 1 -- start start 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76600b4af0 0x7f766015e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76600cfea0 0x7f76600ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f76600d0890 con 0x7f7660156740 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76600b3c60 con 0x7f76600b4af0 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.828+0000 7f7667600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76600d0a10 con 0x7f76600cfea0 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50022/0 (socket says 172.21.15.28:50022) 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 -- 172.21.15.28:0/416986607 learned_addr learned my addr 172.21.15.28:0/416986607 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f765fe00640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76600cfea0 0x7f76600ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7664c00640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76600b4af0 0x7f766015e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 -- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76600cfea0 msgr2=0x7f76600ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76600cfea0 0x7f76600ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 -- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76600b4af0 msgr2=0x7f766015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76600b4af0 0x7f766015e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 -- 172.21.15.28:0/416986607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766015eff0 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.829+0000 7f7665600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7658009f70 tx=0x7f7658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.830+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765803f020 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.830+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7658037040 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.830+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658036070 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.830+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766015f2d0 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.830+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76601f5cd0 con 0x7f7660156740 2024-09-10T10:46:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.831+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76601f5ed0 con 0x7f7660156740 2024-09-10T10:46:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.832+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7658042020 con 0x7f7660156740 2024-09-10T10:46:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.832+0000 7f765cc00640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 0x7f764006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.833+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6101+0+0 (secure 0 0 0) 0x7f76580ab320 con 0x7f7660156740 2024-09-10T10:46:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.833+0000 7f7664c00640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 0x7f764006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.834+0000 7f7664c00640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 0x7f764006e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7654009a30 tx=0x7f7654008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:45.837+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76601f5ed0 con 0x7f7660156740 2024-09-10T10:46:46.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.060+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7660068f50 con 0x7f7660156740 2024-09-10T10:46:46.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.884+0000 7f765cc00640 1 -- 172.21.15.28:0/416986607 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) ==== 118+0+0 (secure 0 0 0) 0x7f7658049030 con 0x7f7660156740 2024-09-10T10:46:46.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 msgr2=0x7f764006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 0x7f764006e320 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7654009a30 tx=0x7f7654008040 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 msgr2=0x7f76600ae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7658009f70 tx=0x7f7658004a40 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 shutdown_connections 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f764006be60 0x7f764006e320 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7660156740 0x7f76600ae8a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76600cfea0 0x7f76600ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 --2- 172.21.15.28:0/416986607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76600b4af0 0x7f766015e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 >> 172.21.15.28:0/416986607 conn(0x7f766006bdf0 msgr2=0x7f76600b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 shutdown_connections 2024-09-10T10:46:46.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:46.887+0000 7f7667600640 1 -- 172.21.15.28:0/416986607 wait complete. 2024-09-10T10:46:46.908 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:46:46.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:46:47.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.072+0000 7fcd21c00640 1 Processor -- start 2024-09-10T10:46:47.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.072+0000 7fcd21c00640 1 -- start start 2024-09-10T10:46:47.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd21c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c15f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:47.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd21c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0d0950 con 0x7fcd1c15f850 2024-09-10T10:46:47.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd21c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0d0bd0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd21c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d17e0 con 0x7fcd1c0d0510 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd1b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcd1c15f850 0x7fcd1c06cf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56694/0 (socket says 172.21.15.28:56694) 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd1b400640 1 -- 172.21.15.28:0/242093575 learned_addr learned my addr 172.21.15.28:0/242093575 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.073+0000 7fcd12000640 1 --2- 172.21.15.28:0/242093575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081088063 0 0) 0x7fcd14001490 con 0x7fcd1c15f850 2024-09-10T10:46:47.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd00003520 con 0x7fcd1c15f850 2024-09-10T10:46:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12880772 0 0) 0x7fcd1c0d0bd0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd14001490 con 0x7fcd1c09aa30 2024-09-10T10:46:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd12000640 1 -- 172.21.15.28:0/242093575 >> v1:172.21.15.138:6789/0 conn(0x7fcd1c09aa30 legacy=0x7fcd1c09ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd12000640 1 -- 172.21.15.28:0/242093575 >> v1:172.21.15.28:6789/0 conn(0x7fcd1c15f850 legacy=0x7fcd1c06cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.074+0000 7fcd12000640 1 -- 172.21.15.28:0/242093575 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0d0eb0 con 0x7fcd1c0d0510 2024-09-10T10:46:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.075+0000 7fcd12000640 1 --2- 172.21.15.28:0/242093575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c15f310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097d0 tx=0x7fcd0402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edd8343a633cce84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd1c0d0510 2024-09-10T10:46:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd04032440 con 0x7fcd1c0d0510 2024-09-10T10:46:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd1a000640 1 -- 172.21.15.28:0/242093575 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd040327f0 con 0x7fcd1c0d0510 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 -- 172.21.15.28:0/242093575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 msgr2=0x7fcd1c15f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 --2- 172.21.15.28:0/242093575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c15f310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097d0 tx=0x7fcd0402a210 comp rx=0 tx=0).stop 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 -- 172.21.15.28:0/242093575 shutdown_connections 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 --2- 172.21.15.28:0/242093575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c15f310 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 -- 172.21.15.28:0/242093575 >> 172.21.15.28:0/242093575 conn(0x7fcd1c0d2380 msgr2=0x7fcd1c0d2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 -- 172.21.15.28:0/242093575 shutdown_connections 2024-09-10T10:46:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.076+0000 7fcd21c00640 1 -- 172.21.15.28:0/242093575 wait complete. 2024-09-10T10:46:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.078+0000 7fcd21c00640 1 Processor -- start 2024-09-10T10:46:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.078+0000 7fcd21c00640 1 -- start start 2024-09-10T10:46:47.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:47.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c161730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd1c15f850 0x7fcd1c158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c06d630 con 0x7fcd1c09aa30 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d1470 con 0x7fcd1c15f850 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd21c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c06d7b0 con 0x7fcd1c0d0510 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd1aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50034/0 (socket says 172.21.15.28:50034) 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd1aa00640 1 -- 172.21.15.28:0/4283275537 learned_addr learned my addr 172.21.15.28:0/4283275537 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd1b400640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd1c15f850 0x7fcd1c158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.079+0000 7fcd12000640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 -- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 msgr2=0x7fcd1c161730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c161730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 -- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd1c15f850 msgr2=0x7fcd1c158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd1c15f850 0x7fcd1c158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 -- 172.21.15.28:0/4283275537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0c6c90 con 0x7fcd1c09aa30 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd1aa00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c490 tx=0x7fcd0c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0052d0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c005470 con 0x7fcd1c09aa30 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c012850 con 0x7fcd1c09aa30 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.080+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c0c6f70 con 0x7fcd1c09aa30 2024-09-10T10:46:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.081+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c0c7810 con 0x7fcd1c09aa30 2024-09-10T10:46:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.081+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c0c7360 con 0x7fcd1c09aa30 2024-09-10T10:46:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.082+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd0c0129f0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.083+0000 7fcd13400640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.083+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fcd0c08ab80 con 0x7fcd1c09aa30 2024-09-10T10:46:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.084+0000 7fcd12000640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.085+0000 7fcd12000640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 0x7fccf806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034000 tx=0x7fcd0402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.088+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd0c058cb0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.307+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcd1c0d0bd0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.885+0000 7fcd13400640 1 -- 172.21.15.28:0/4283275537 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7fcd0c05cfb0 con 0x7fcd1c09aa30 2024-09-10T10:46:47.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 msgr2=0x7fccf806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 0x7fccf806e2f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd04034000 tx=0x7fcd0402e000 comp rx=0 tx=0).stop 2024-09-10T10:46:47.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 msgr2=0x7fcd1c0b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c490 tx=0x7fcd0c00c950 comp rx=0 tx=0).stop 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 shutdown_connections 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccf806be30 0x7fccf806e2f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd1c15f850 0x7fcd1c158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd1c0d0510 0x7fcd1c161730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 --2- 172.21.15.28:0/4283275537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd1c09aa30 0x7fcd1c0b7610 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 >> 172.21.15.28:0/4283275537 conn(0x7fcd1c0d2380 msgr2=0x7fcd1c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 shutdown_connections 2024-09-10T10:46:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:47.888+0000 7fcd21c00640 1 -- 172.21.15.28:0/4283275537 wait complete. 2024-09-10T10:46:47.906 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:46:47.906 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.062+0000 7fd556000640 1 Processor -- start 2024-09-10T10:46:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.062+0000 7fd556000640 1 -- start start 2024-09-10T10:46:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd556000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 0x7fd5500720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd556000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd5500725f0 0x7fd5500aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd556000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd550072a00 con 0x7fd550147230 2024-09-10T10:46:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd556000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500b34d0 con 0x7fd5500725f0 2024-09-10T10:46:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd556000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500ae9e0 con 0x7fd550147e90 2024-09-10T10:46:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd54f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd550147230 0x7fd55006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56698/0 (socket says 172.21.15.28:56698) 2024-09-10T10:46:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd54f400640 1 -- 172.21.15.28:0/889494101 learned_addr learned my addr 172.21.15.28:0/889494101 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.063+0000 7fd54fe00640 1 --2- 172.21.15.28:0/889494101 >> v2:172.21.15.28:3300/0 conn(0x7fd5500725f0 0x7fd5500aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54ea00640 1 --2- 172.21.15.28:0/889494101 >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 0x7fd5500720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54e000640 1 -- 172.21.15.28:0/889494101 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3422155878 0 0) 0x7fd540001110 con 0x7fd550147230 2024-09-10T10:46:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54e000640 1 -- 172.21.15.28:0/889494101 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd534003520 con 0x7fd550147230 2024-09-10T10:46:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54fe00640 1 -- 172.21.15.28:0/889494101 >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 msgr2=0x7fd5500720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54fe00640 1 --2- 172.21.15.28:0/889494101 >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 0x7fd5500720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54fe00640 1 -- 172.21.15.28:0/889494101 >> v1:172.21.15.28:6789/0 conn(0x7fd550147230 legacy=0x7fd55006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54fe00640 1 -- 172.21.15.28:0/889494101 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55006f480 con 0x7fd5500725f0 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.064+0000 7fd54ea00640 1 --2- 172.21.15.28:0/889494101 >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 0x7fd5500720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.065+0000 7fd54fe00640 1 --2- 172.21.15.28:0/889494101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd5500aee10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd55006ccb0 tx=0x7fd54802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec0a53ed8405897 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.065+0000 7fd54e000640 1 -- 172.21.15.28:0/889494101 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548007740 con 0x7fd5500725f0 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.065+0000 7fd54e000640 1 -- 172.21.15.28:0/889494101 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5480078e0 con 0x7fd5500725f0 2024-09-10T10:46:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.065+0000 7fd54e000640 1 -- 172.21.15.28:0/889494101 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548007c90 con 0x7fd5500725f0 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 -- 172.21.15.28:0/889494101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 msgr2=0x7fd5500aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 --2- 172.21.15.28:0/889494101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd5500aee10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd55006ccb0 tx=0x7fd54802a680 comp rx=0 tx=0).stop 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 -- 172.21.15.28:0/889494101 shutdown_connections 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 --2- 172.21.15.28:0/889494101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd5500aee10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 --2- 172.21.15.28:0/889494101 >> v2:172.21.15.138:3300/0 conn(0x7fd550147e90 0x7fd5500720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 -- 172.21.15.28:0/889494101 >> 172.21.15.28:0/889494101 conn(0x7fd550067d70 msgr2=0x7fd55006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 -- 172.21.15.28:0/889494101 shutdown_connections 2024-09-10T10:46:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.066+0000 7fd556000640 1 -- 172.21.15.28:0/889494101 wait complete. 2024-09-10T10:46:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.068+0000 7fd556000640 1 Processor -- start 2024-09-10T10:46:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.068+0000 7fd556000640 1 -- start start 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd550147230 0x7fd5500c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 0x7fd5501457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd55006eec0 con 0x7fd5500725f0 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd55006ebe0 con 0x7fd550147230 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd556000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd55006f1a0 con 0x7fd550147e90 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd54f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd54f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50056/0 (socket says 172.21.15.28:50056) 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd54f400640 1 -- 172.21.15.28:0/4292089279 learned_addr learned my addr 172.21.15.28:0/4292089279 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd54ea00640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd550147230 0x7fd5500c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.069+0000 7fd54fe00640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 0x7fd5501457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 -- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 msgr2=0x7fd5501457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 0x7fd5501457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 -- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd550147230 msgr2=0x7fd5500c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd550147230 0x7fd5500c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 -- 172.21.15.28:0/4292089279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500c7380 con 0x7fd5500725f0 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54fe00640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 0x7fd5501457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd54f400640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd54000c760 tx=0x7fd54000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540017070 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd540005ad0 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd540007e20 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5500c7660 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.070+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5500c7c20 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.071+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd550147610 con 0x7fd5500725f0 2024-09-10T10:46:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.072+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd54001a070 con 0x7fd5500725f0 2024-09-10T10:46:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.073+0000 7fd547400640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 0x7fd53006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.073+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd54008a3a0 con 0x7fd5500725f0 2024-09-10T10:46:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.074+0000 7fd54ea00640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 0x7fd53006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.074+0000 7fd54ea00640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 0x7fd53006e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5500732a0 tx=0x7fd53c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.077+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd550072a00 con 0x7fd5500725f0 2024-09-10T10:46:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.296+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5501478c0 con 0x7fd5500725f0 2024-09-10T10:46:48.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.297+0000 7fd547400640 1 -- 172.21.15.28:0/4292089279 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5230 (secure 0 0 0) 0x7fd5400584d0 con 0x7fd5500725f0 2024-09-10T10:46:48.298 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:48.298 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":48,"btime":"2024-09-10T10:46:47:884742+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:46:47.308289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3429810116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3429810116},{"type":"v1","addr":"172.21.15.138:6837","nonce":3429810116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5844,"qdb_cluster":[5844]},"id":4}]} 2024-09-10T10:46:48.299 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 48 2024-09-10T10:46:48.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 msgr2=0x7fd53006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 0x7fd53006e2f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd5500732a0 tx=0x7fd53c008040 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 msgr2=0x7fd55015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd54000c760 tx=0x7fd54000cc20 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 shutdown_connections 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd53006be30 0x7fd53006e2f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd550147e90 0x7fd5501457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd550147230 0x7fd5500c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 --2- 172.21.15.28:0/4292089279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5500725f0 0x7fd55015de70 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 >> 172.21.15.28:0/4292089279 conn(0x7fd550067d70 msgr2=0x7fd5500b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 shutdown_connections 2024-09-10T10:46:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.299+0000 7fd556000640 1 -- 172.21.15.28:0/4292089279 wait complete. 2024-09-10T10:46:48.318 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:46:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.472+0000 7f8ed5800640 1 Processor -- start 2024-09-10T10:46:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.473+0000 7f8ed5800640 1 -- start start 2024-09-10T10:46:48.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8ed015b300 0x7f8ed00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed00cd3f0 0x7f8ed006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ed0072d30 0x7f8ed006f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed00728a0 con 0x7f8ed0072d30 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed00cdbe0 con 0x7f8ed015b300 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ed5800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00cd900 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ecf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ed0072d30 0x7f8ed006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ecf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ed0072d30 0x7f8ed006f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50072/0 (socket says 172.21.15.28:50072) 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ecf400640 1 -- 172.21.15.28:0/2400681884 learned_addr learned my addr 172.21.15.28:0/2400681884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ecea00640 1 --2- 172.21.15.28:0/2400681884 >> v2:172.21.15.138:3300/0 conn(0x7f8ed015b300 0x7f8ed00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.474+0000 7f8ec7e00640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed00cd3f0 0x7f8ed006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.475+0000 7f8ecf400640 1 -- 172.21.15.28:0/2400681884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed00cd3f0 msgr2=0x7f8ed006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.475+0000 7f8ecf400640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed00cd3f0 0x7f8ed006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.475+0000 7f8ecf400640 1 -- 172.21.15.28:0/2400681884 >> v2:172.21.15.138:3300/0 conn(0x7f8ed015b300 msgr2=0x7f8ed00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.475+0000 7f8ecf400640 1 --2- 172.21.15.28:0/2400681884 >> v2:172.21.15.138:3300/0 conn(0x7f8ed015b300 0x7f8ed00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.475+0000 7f8ecf400640 1 -- 172.21.15.28:0/2400681884 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed0072250 con 0x7f8ed0072d30 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ecf400640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0072d30 0x7f8ed006f700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0070250 tx=0x7f8ec802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09d8bb9cc0da62f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ece000640 1 -- 172.21.15.28:0/2400681884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8036070 con 0x7f8ed0072d30 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ece000640 1 -- 172.21.15.28:0/2400681884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec8031600 con 0x7f8ed0072d30 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ece000640 1 -- 172.21.15.28:0/2400681884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80319d0 con 0x7f8ed0072d30 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 -- 172.21.15.28:0/2400681884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0072d30 msgr2=0x7f8ed006f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0072d30 0x7f8ed006f700 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0070250 tx=0x7f8ec802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 -- 172.21.15.28:0/2400681884 shutdown_connections 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0072d30 0x7f8ed006f700 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2400681884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed00cd3f0 0x7f8ed006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2400681884 >> v2:172.21.15.138:3300/0 conn(0x7f8ed015b300 0x7f8ed00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.476+0000 7f8ed5800640 1 -- 172.21.15.28:0/2400681884 >> 172.21.15.28:0/2400681884 conn(0x7f8ed0067d70 msgr2=0x7f8ed006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.477+0000 7f8ed5800640 1 -- 172.21.15.28:0/2400681884 shutdown_connections 2024-09-10T10:46:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.477+0000 7f8ed5800640 1 -- 172.21.15.28:0/2400681884 wait complete. 2024-09-10T10:46:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.478+0000 7f8ed5800640 1 Processor -- start 2024-09-10T10:46:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.478+0000 7f8ed5800640 1 -- start start 2024-09-10T10:46:48.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed0072d30 0x7f8ed014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 0x7f8ed01e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00d0160 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed00cffe0 con 0x7f8ed015b300 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ed5800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed00b3c40 con 0x7f8ed0072d30 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.479+0000 7f8ec7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50084/0 (socket says 172.21.15.28:50084) 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 -- 172.21.15.28:0/2487563734 learned_addr learned my addr 172.21.15.28:0/2487563734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ecf400640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 0x7f8ed01e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ecea00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed0072d30 0x7f8ed014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 -- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed0072d30 msgr2=0x7f8ed014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed0072d30 0x7f8ed014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 -- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 msgr2=0x7f8ed01e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 0x7f8ed01e4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 -- 172.21.15.28:0/2487563734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed01554c0 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ecf400640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 0x7f8ed01e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec7e00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800e860 tx=0x7f8eb800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8017070 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb800c940 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.480+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800cd10 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.481+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed0155760 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.481+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed0155d20 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.482+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed015bc10 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.483+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8eb801a070 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.483+0000 7f8ec6a00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 0x7f8eac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.483+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f8eb808ad50 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.484+0000 7f8ecea00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 0x7f8eac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.484+0000 7f8ecea00640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 0x7f8eac06e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009a30 tx=0x7f8ec0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.488+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8eb80594d0 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.673+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ed015b930 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.675+0000 7f8ec6a00640 1 -- 172.21.15.28:0/2487563734 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+17016 (secure 0 0 0) 0x7f8eb8058dd0 con 0x7f8ed00cd3f0 2024-09-10T10:46:48.676 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:48.676 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":70,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:46:45.865880+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:42.078607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:43.284849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:46:48.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 msgr2=0x7f8eac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 0x7f8eac06e2f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009a30 tx=0x7f8ec0008040 comp rx=0 tx=0).stop 2024-09-10T10:46:48.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 msgr2=0x7f8ed01e4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800e860 tx=0x7f8eb800ed20 comp rx=0 tx=0).stop 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 shutdown_connections 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8eac06be30 0x7f8eac06e2f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ed015b300 0x7f8ed01e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed00cd3f0 0x7f8ed01e4740 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 --2- 172.21.15.28:0/2487563734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ed0072d30 0x7f8ed014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 >> 172.21.15.28:0/2487563734 conn(0x7f8ed0067d70 msgr2=0x7f8ed00cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 shutdown_connections 2024-09-10T10:46:48.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.677+0000 7f8ed5800640 1 -- 172.21.15.28:0/2487563734 wait complete. 2024-09-10T10:46:48.694 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:46:48.694 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:48.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.848+0000 7fce33e00640 1 Processor -- start 2024-09-10T10:46:48.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.848+0000 7fce33e00640 1 -- start start 2024-09-10T10:46:48.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce33e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fce2c06f8d0 0x7fce2c165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce33e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c0cdbe0 con 0x7fce2c072d30 2024-09-10T10:46:48.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce33e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce2c0cd900 con 0x7fce2c15b300 2024-09-10T10:46:48.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce33e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fce2c0728a0 con 0x7fce2c06f8d0 2024-09-10T10:46:48.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fce2c06f8d0 0x7fce2c165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fce2c06f8d0 0x7fce2c165de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50090/0 (socket says 172.21.15.28:50090) 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce30a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fce2c072d30 0x7fce2c06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56706/0 (socket says 172.21.15.28:56706) 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 -- 172.21.15.28:0/1560853052 learned_addr learned my addr 172.21.15.28:0/1560853052 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 -- 172.21.15.28:0/1560853052 >> v1:172.21.15.138:6789/0 conn(0x7fce2c15b300 legacy=0x7fce2c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 -- 172.21.15.28:0/1560853052 >> v1:172.21.15.28:6789/0 conn(0x7fce2c072d30 legacy=0x7fce2c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.850+0000 7fce31e00640 1 -- 172.21.15.28:0/1560853052 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c072250 con 0x7fce2c06f8d0 2024-09-10T10:46:48.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.851+0000 7fce31e00640 1 --2- 172.21.15.28:0/1560853052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c06f8d0 0x7fce2c165de0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fce24009e50 tx=0x7fce2402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4b0b565483e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce2be00640 1 -- 172.21.15.28:0/1560853052 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24036070 con 0x7fce2c06f8d0 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce2be00640 1 -- 172.21.15.28:0/1560853052 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce24031600 con 0x7fce2c06f8d0 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce2be00640 1 -- 172.21.15.28:0/1560853052 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce240319d0 con 0x7fce2c06f8d0 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 -- 172.21.15.28:0/1560853052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c06f8d0 msgr2=0x7fce2c165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 --2- 172.21.15.28:0/1560853052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c06f8d0 0x7fce2c165de0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fce24009e50 tx=0x7fce2402a6d0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 -- 172.21.15.28:0/1560853052 shutdown_connections 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 --2- 172.21.15.28:0/1560853052 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c06f8d0 0x7fce2c165de0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 -- 172.21.15.28:0/1560853052 >> 172.21.15.28:0/1560853052 conn(0x7fce2c067d70 msgr2=0x7fce2c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 -- 172.21.15.28:0/1560853052 shutdown_connections 2024-09-10T10:46:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.852+0000 7fce33e00640 1 -- 172.21.15.28:0/1560853052 wait complete. 2024-09-10T10:46:48.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.854+0000 7fce33e00640 1 Processor -- start 2024-09-10T10:46:48.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.854+0000 7fce33e00640 1 -- start start 2024-09-10T10:46:48.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 0x7fce2c0adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c15b300 0x7fce2c1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0cffe0 con 0x7fce2c072d30 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c0cd5c0 con 0x7fce2c15b300 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce33e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce2c0d0160 con 0x7fce2c06f8d0 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce30a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce30a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50100/0 (socket says 172.21.15.28:50100) 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce30a00640 1 -- 172.21.15.28:0/1073646550 learned_addr learned my addr 172.21.15.28:0/1073646550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce31e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c15b300 0x7fce2c1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.855+0000 7fce31400640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 0x7fce2c0adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 -- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 msgr2=0x7fce2c0adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 0x7fce2c0adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 -- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c15b300 msgr2=0x7fce2c1555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c15b300 0x7fce2c1555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 -- 172.21.15.28:0/1073646550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c1469c0 con 0x7fce2c072d30 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce31400640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 0x7fce2c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce30a00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c400 tx=0x7fce1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce180074e0 con 0x7fce2c072d30 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce18007680 con 0x7fce2c072d30 2024-09-10T10:46:48.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18004a00 con 0x7fce2c072d30 2024-09-10T10:46:48.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.856+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c146ca0 con 0x7fce2c072d30 2024-09-10T10:46:48.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.857+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c147260 con 0x7fce2c072d30 2024-09-10T10:46:48.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.858+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c0ce0c0 con 0x7fce2c072d30 2024-09-10T10:46:48.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.859+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce1801a020 con 0x7fce2c072d30 2024-09-10T10:46:48.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.859+0000 7fce29600640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 0x7fce0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:48.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.860+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fce1808a810 con 0x7fce2c072d30 2024-09-10T10:46:48.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.860+0000 7fce31400640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 0x7fce0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:48.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.861+0000 7fce31400640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 0x7fce0c06e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:48.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:48.864+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce2c0ce0c0 con 0x7fce2c072d30 2024-09-10T10:46:49.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.045+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce2c06f1c0 con 0x7fce0c06be30 2024-09-10T10:46:49.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.052+0000 7fce29600640 1 -- 172.21.15.28:0/1073646550 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250382 (secure 0 0 0) 0x7fce2c06f1c0 con 0x7fce0c06be30 2024-09-10T10:46:49.058 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:49.060 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:49.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 msgr2=0x7fce0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 0x7fce0c06e2f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fce20006fd0 tx=0x7fce20008040 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 msgr2=0x7fce2c143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c400 tx=0x7fce1800c8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 shutdown_connections 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce0c06be30 0x7fce0c06e2f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce2c15b300 0x7fce2c1555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.055+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce2c072d30 0x7fce2c143fd0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.056+0000 7fce33e00640 1 --2- 172.21.15.28:0/1073646550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce2c06f8d0 0x7fce2c0adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.056+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 >> 172.21.15.28:0/1073646550 conn(0x7fce2c067d70 msgr2=0x7fce2c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.056+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 shutdown_connections 2024-09-10T10:46:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.056+0000 7fce33e00640 1 -- 172.21.15.28:0/1073646550 wait complete. 2024-09-10T10:46:49.073 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T10:46:48.526234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244664,"kb_used_data":7612,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730056,"statfs":{"total":773094113280,"available":772843577344,"internally_reserved":0,"allocated":7794688,"data_stored":4154135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.680947"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895722+0000","last_change":"2024-09-10T10:46:44.892310+0000","last_active":"2024-09-10T10:46:44.895722+0000","last_peered":"2024-09-10T10:46:44.895722+0000","last_clean":"2024-09-10T10:46:44.895722+0000","last_became_active":"2024-09-10T10:46:44.892049+0000","last_became_peered":"2024-09-10T10:46:44.892049+0000","last_unstale":"2024-09-10T10:46:44.895722+0000","last_undegraded":"2024-09-10T10:46:44.895722+0000","last_fullsized":"2024-09-10T10:46:44.895722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892585+0000","last_change":"2024-09-10T10:46:44.884268+0000","last_active":"2024-09-10T10:46:44.892585+0000","last_peered":"2024-09-10T10:46:44.892585+0000","last_clean":"2024-09-10T10:46:44.892585+0000","last_became_active":"2024-09-10T10:46:44.883101+0000","last_became_peered":"2024-09-10T10:46:44.883101+0000","last_unstale":"2024-09-10T10:46:44.892585+0000","last_undegraded":"2024-09-10T10:46:44.892585+0000","last_fullsized":"2024-09-10T10:46:44.892585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:14.573857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882869+0000","last_change":"2024-09-10T10:46:44.875622+0000","last_active":"2024-09-10T10:46:44.882869+0000","last_peered":"2024-09-10T10:46:44.882869+0000","last_clean":"2024-09-10T10:46:44.882869+0000","last_became_active":"2024-09-10T10:46:44.875330+0000","last_became_peered":"2024-09-10T10:46:44.875330+0000","last_unstale":"2024-09-10T10:46:44.882869+0000","last_undegraded":"2024-09-10T10:46:44.882869+0000","last_fullsized":"2024-09-10T10:46:44.882869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892693+0000","last_change":"2024-09-10T10:46:44.884914+0000","last_active":"2024-09-10T10:46:44.892693+0000","last_peered":"2024-09-10T10:46:44.892693+0000","last_clean":"2024-09-10T10:46:44.892693+0000","last_became_active":"2024-09-10T10:46:44.884638+0000","last_became_peered":"2024-09-10T10:46:44.884638+0000","last_unstale":"2024-09-10T10:46:44.892693+0000","last_undegraded":"2024-09-10T10:46:44.892693+0000","last_fullsized":"2024-09-10T10:46:44.892693+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:20.876408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.884835+0000","last_change":"2024-09-10T10:46:44.883459+0000","last_active":"2024-09-10T10:46:44.884835+0000","last_peered":"2024-09-10T10:46:44.884835+0000","last_clean":"2024-09-10T10:46:44.884835+0000","last_became_active":"2024-09-10T10:46:44.883173+0000","last_became_peered":"2024-09-10T10:46:44.883173+0000","last_unstale":"2024-09-10T10:46:44.884835+0000","last_undegraded":"2024-09-10T10:46:44.884835+0000","last_fullsized":"2024-09-10T10:46:44.884835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893848+0000","last_change":"2024-09-10T10:46:44.890220+0000","last_active":"2024-09-10T10:46:44.893848+0000","last_peered":"2024-09-10T10:46:44.893848+0000","last_clean":"2024-09-10T10:46:44.893848+0000","last_became_active":"2024-09-10T10:46:44.889918+0000","last_became_peered":"2024-09-10T10:46:44.889918+0000","last_unstale":"2024-09-10T10:46:44.893848+0000","last_undegraded":"2024-09-10T10:46:44.893848+0000","last_fullsized":"2024-09-10T10:46:44.893848+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:58.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887361+0000","last_change":"2024-09-10T10:46:44.877805+0000","last_active":"2024-09-10T10:46:44.887361+0000","last_peered":"2024-09-10T10:46:44.887361+0000","last_clean":"2024-09-10T10:46:44.887361+0000","last_became_active":"2024-09-10T10:46:44.875492+0000","last_became_peered":"2024-09-10T10:46:44.875492+0000","last_unstale":"2024-09-10T10:46:44.887361+0000","last_undegraded":"2024-09-10T10:46:44.887361+0000","last_fullsized":"2024-09-10T10:46:44.887361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:42.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895164+0000","last_change":"2024-09-10T10:46:44.886608+0000","last_active":"2024-09-10T10:46:44.895164+0000","last_peered":"2024-09-10T10:46:44.895164+0000","last_clean":"2024-09-10T10:46:44.895164+0000","last_became_active":"2024-09-10T10:46:44.886293+0000","last_became_peered":"2024-09-10T10:46:44.886293+0000","last_unstale":"2024-09-10T10:46:44.895164+0000","last_undegraded":"2024-09-10T10:46:44.895164+0000","last_fullsized":"2024-09-10T10:46:44.895164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887118+0000","last_change":"2024-09-10T10:46:44.878902+0000","last_active":"2024-09-10T10:46:44.887118+0000","last_peered":"2024-09-10T10:46:44.887118+0000","last_clean":"2024-09-10T10:46:44.887118+0000","last_became_active":"2024-09-10T10:46:44.878564+0000","last_became_peered":"2024-09-10T10:46:44.878564+0000","last_unstale":"2024-09-10T10:46:44.887118+0000","last_undegraded":"2024-09-10T10:46:44.887118+0000","last_fullsized":"2024-09-10T10:46:44.887118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887230+0000","last_change":"2024-09-10T10:46:44.882681+0000","last_active":"2024-09-10T10:46:44.887230+0000","last_peered":"2024-09-10T10:46:44.887230+0000","last_clean":"2024-09-10T10:46:44.887230+0000","last_became_active":"2024-09-10T10:46:44.882405+0000","last_became_peered":"2024-09-10T10:46:44.882405+0000","last_unstale":"2024-09-10T10:46:44.887230+0000","last_undegraded":"2024-09-10T10:46:44.887230+0000","last_fullsized":"2024-09-10T10:46:44.887230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894549+0000","last_change":"2024-09-10T10:46:44.883967+0000","last_active":"2024-09-10T10:46:44.894549+0000","last_peered":"2024-09-10T10:46:44.894549+0000","last_clean":"2024-09-10T10:46:44.894549+0000","last_became_active":"2024-09-10T10:46:44.877104+0000","last_became_peered":"2024-09-10T10:46:44.877104+0000","last_unstale":"2024-09-10T10:46:44.894549+0000","last_undegraded":"2024-09-10T10:46:44.894549+0000","last_fullsized":"2024-09-10T10:46:44.894549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894319+0000","last_change":"2024-09-10T10:46:44.880957+0000","last_active":"2024-09-10T10:46:44.894319+0000","last_peered":"2024-09-10T10:46:44.894319+0000","last_clean":"2024-09-10T10:46:44.894319+0000","last_became_active":"2024-09-10T10:46:44.880691+0000","last_became_peered":"2024-09-10T10:46:44.880691+0000","last_unstale":"2024-09-10T10:46:44.894319+0000","last_undegraded":"2024-09-10T10:46:44.894319+0000","last_fullsized":"2024-09-10T10:46:44.894319+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:06.745405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891012+0000","last_change":"2024-09-10T10:46:44.882838+0000","last_active":"2024-09-10T10:46:44.891012+0000","last_peered":"2024-09-10T10:46:44.891012+0000","last_clean":"2024-09-10T10:46:44.891012+0000","last_became_active":"2024-09-10T10:46:44.880913+0000","last_became_peered":"2024-09-10T10:46:44.880913+0000","last_unstale":"2024-09-10T10:46:44.891012+0000","last_undegraded":"2024-09-10T10:46:44.891012+0000","last_fullsized":"2024-09-10T10:46:44.891012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881711+0000","last_change":"2024-09-10T10:46:44.879407+0000","last_active":"2024-09-10T10:46:44.881711+0000","last_peered":"2024-09-10T10:46:44.881711+0000","last_clean":"2024-09-10T10:46:44.881711+0000","last_became_active":"2024-09-10T10:46:44.879124+0000","last_became_peered":"2024-09-10T10:46:44.879124+0000","last_unstale":"2024-09-10T10:46:44.881711+0000","last_undegraded":"2024-09-10T10:46:44.881711+0000","last_fullsized":"2024-09-10T10:46:44.881711+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879528+0000","last_change":"2024-09-10T10:46:44.874639+0000","last_active":"2024-09-10T10:46:44.879528+0000","last_peered":"2024-09-10T10:46:44.879528+0000","last_clean":"2024-09-10T10:46:44.879528+0000","last_became_active":"2024-09-10T10:46:44.874387+0000","last_became_peered":"2024-09-10T10:46:44.874387+0000","last_unstale":"2024-09-10T10:46:44.879528+0000","last_undegraded":"2024-09-10T10:46:44.879528+0000","last_fullsized":"2024-09-10T10:46:44.879528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:38:57.911434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874284+0000","last_change":"2024-09-10T10:46:44.865791+0000","last_active":"2024-09-10T10:46:44.874284+0000","last_peered":"2024-09-10T10:46:44.874284+0000","last_clean":"2024-09-10T10:46:44.874284+0000","last_became_active":"2024-09-10T10:46:44.865491+0000","last_became_peered":"2024-09-10T10:46:44.865491+0000","last_unstale":"2024-09-10T10:46:44.874284+0000","last_undegraded":"2024-09-10T10:46:44.874284+0000","last_fullsized":"2024-09-10T10:46:44.874284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887094+0000","last_change":"2024-09-10T10:46:44.878815+0000","last_active":"2024-09-10T10:46:44.887094+0000","last_peered":"2024-09-10T10:46:44.887094+0000","last_clean":"2024-09-10T10:46:44.887094+0000","last_became_active":"2024-09-10T10:46:44.878192+0000","last_became_peered":"2024-09-10T10:46:44.878192+0000","last_unstale":"2024-09-10T10:46:44.887094+0000","last_undegraded":"2024-09-10T10:46:44.887094+0000","last_fullsized":"2024-09-10T10:46:44.887094+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879723+0000","last_change":"2024-09-10T10:46:44.871127+0000","last_active":"2024-09-10T10:46:44.879723+0000","last_peered":"2024-09-10T10:46:44.879723+0000","last_clean":"2024-09-10T10:46:44.879723+0000","last_became_active":"2024-09-10T10:46:44.870842+0000","last_became_peered":"2024-09-10T10:46:44.870842+0000","last_unstale":"2024-09-10T10:46:44.879723+0000","last_undegraded":"2024-09-10T10:46:44.879723+0000","last_fullsized":"2024-09-10T10:46:44.879723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:14.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894528+0000","last_change":"2024-09-10T10:46:44.882561+0000","last_active":"2024-09-10T10:46:44.894528+0000","last_peered":"2024-09-10T10:46:44.894528+0000","last_clean":"2024-09-10T10:46:44.894528+0000","last_became_active":"2024-09-10T10:46:44.877515+0000","last_became_peered":"2024-09-10T10:46:44.877515+0000","last_unstale":"2024-09-10T10:46:44.894528+0000","last_undegraded":"2024-09-10T10:46:44.894528+0000","last_fullsized":"2024-09-10T10:46:44.894528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:35.562296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881741+0000","last_change":"2024-09-10T10:46:44.878483+0000","last_active":"2024-09-10T10:46:44.881741+0000","last_peered":"2024-09-10T10:46:44.881741+0000","last_clean":"2024-09-10T10:46:44.881741+0000","last_became_active":"2024-09-10T10:46:44.874186+0000","last_became_peered":"2024-09-10T10:46:44.874186+0000","last_unstale":"2024-09-10T10:46:44.881741+0000","last_undegraded":"2024-09-10T10:46:44.881741+0000","last_fullsized":"2024-09-10T10:46:44.881741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893869+0000","last_change":"2024-09-10T10:46:44.890973+0000","last_active":"2024-09-10T10:46:44.893869+0000","last_peered":"2024-09-10T10:46:44.893869+0000","last_clean":"2024-09-10T10:46:44.893869+0000","last_became_active":"2024-09-10T10:46:44.889471+0000","last_became_peered":"2024-09-10T10:46:44.889471+0000","last_unstale":"2024-09-10T10:46:44.893869+0000","last_undegraded":"2024-09-10T10:46:44.893869+0000","last_fullsized":"2024-09-10T10:46:44.893869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:28.051816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894600+0000","last_change":"2024-09-10T10:46:44.888201+0000","last_active":"2024-09-10T10:46:44.894600+0000","last_peered":"2024-09-10T10:46:44.894600+0000","last_clean":"2024-09-10T10:46:44.894600+0000","last_became_active":"2024-09-10T10:46:44.887970+0000","last_became_peered":"2024-09-10T10:46:44.887970+0000","last_unstale":"2024-09-10T10:46:44.894600+0000","last_undegraded":"2024-09-10T10:46:44.894600+0000","last_fullsized":"2024-09-10T10:46:44.894600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891947+0000","last_change":"2024-09-10T10:46:44.881612+0000","last_active":"2024-09-10T10:46:44.891947+0000","last_peered":"2024-09-10T10:46:44.891947+0000","last_clean":"2024-09-10T10:46:44.891947+0000","last_became_active":"2024-09-10T10:46:44.881250+0000","last_became_peered":"2024-09-10T10:46:44.881250+0000","last_unstale":"2024-09-10T10:46:44.891947+0000","last_undegraded":"2024-09-10T10:46:44.891947+0000","last_fullsized":"2024-09-10T10:46:44.891947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:44.946566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891469+0000","last_change":"2024-09-10T10:46:44.878716+0000","last_active":"2024-09-10T10:46:44.891469+0000","last_peered":"2024-09-10T10:46:44.891469+0000","last_clean":"2024-09-10T10:46:44.891469+0000","last_became_active":"2024-09-10T10:46:44.878422+0000","last_became_peered":"2024-09-10T10:46:44.878422+0000","last_unstale":"2024-09-10T10:46:44.891469+0000","last_undegraded":"2024-09-10T10:46:44.891469+0000","last_fullsized":"2024-09-10T10:46:44.891469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:41.176611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.888369+0000","last_change":"2024-09-10T10:46:44.886054+0000","last_active":"2024-09-10T10:46:44.888369+0000","last_peered":"2024-09-10T10:46:44.888369+0000","last_clean":"2024-09-10T10:46:44.888369+0000","last_became_active":"2024-09-10T10:46:44.885269+0000","last_became_peered":"2024-09-10T10:46:44.885269+0000","last_unstale":"2024-09-10T10:46:44.888369+0000","last_undegraded":"2024-09-10T10:46:44.888369+0000","last_fullsized":"2024-09-10T10:46:44.888369+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874510+0000","last_change":"2024-09-10T10:46:44.869461+0000","last_active":"2024-09-10T10:46:44.874510+0000","last_peered":"2024-09-10T10:46:44.874510+0000","last_clean":"2024-09-10T10:46:44.874510+0000","last_became_active":"2024-09-10T10:46:44.869194+0000","last_became_peered":"2024-09-10T10:46:44.869194+0000","last_unstale":"2024-09-10T10:46:44.874510+0000","last_undegraded":"2024-09-10T10:46:44.874510+0000","last_fullsized":"2024-09-10T10:46:44.874510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887172+0000","last_change":"2024-09-10T10:46:44.879808+0000","last_active":"2024-09-10T10:46:44.887172+0000","last_peered":"2024-09-10T10:46:44.887172+0000","last_clean":"2024-09-10T10:46:44.887172+0000","last_became_active":"2024-09-10T10:46:44.879312+0000","last_became_peered":"2024-09-10T10:46:44.879312+0000","last_unstale":"2024-09-10T10:46:44.887172+0000","last_undegraded":"2024-09-10T10:46:44.887172+0000","last_fullsized":"2024-09-10T10:46:44.887172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892504+0000","last_change":"2024-09-10T10:46:44.881613+0000","last_active":"2024-09-10T10:46:44.892504+0000","last_peered":"2024-09-10T10:46:44.892504+0000","last_clean":"2024-09-10T10:46:44.892504+0000","last_became_active":"2024-09-10T10:46:44.881356+0000","last_became_peered":"2024-09-10T10:46:44.881356+0000","last_unstale":"2024-09-10T10:46:44.892504+0000","last_undegraded":"2024-09-10T10:46:44.892504+0000","last_fullsized":"2024-09-10T10:46:44.892504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:04.057439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.883453+0000","last_change":"2024-09-10T10:46:44.882083+0000","last_active":"2024-09-10T10:46:44.883453+0000","last_peered":"2024-09-10T10:46:44.883453+0000","last_clean":"2024-09-10T10:46:44.883453+0000","last_became_active":"2024-09-10T10:46:44.878774+0000","last_became_peered":"2024-09-10T10:46:44.878774+0000","last_unstale":"2024-09-10T10:46:44.883453+0000","last_undegraded":"2024-09-10T10:46:44.883453+0000","last_fullsized":"2024-09-10T10:46:44.883453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890870+0000","last_change":"2024-09-10T10:46:44.884761+0000","last_active":"2024-09-10T10:46:44.890870+0000","last_peered":"2024-09-10T10:46:44.890870+0000","last_clean":"2024-09-10T10:46:44.890870+0000","last_became_active":"2024-09-10T10:46:44.884412+0000","last_became_peered":"2024-09-10T10:46:44.884412+0000","last_unstale":"2024-09-10T10:46:44.890870+0000","last_undegraded":"2024-09-10T10:46:44.890870+0000","last_fullsized":"2024-09-10T10:46:44.890870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903682+0000","last_change":"2024-09-10T10:46:43.873120+0000","last_active":"2024-09-10T10:46:44.903682+0000","last_peered":"2024-09-10T10:46:44.903682+0000","last_clean":"2024-09-10T10:46:44.903682+0000","last_became_active":"2024-09-10T10:46:43.872682+0000","last_became_peered":"2024-09-10T10:46:43.872682+0000","last_unstale":"2024-09-10T10:46:44.903682+0000","last_undegraded":"2024-09-10T10:46:44.903682+0000","last_fullsized":"2024-09-10T10:46:44.903682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.538093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892606+0000","last_change":"2024-09-10T10:46:44.885626+0000","last_active":"2024-09-10T10:46:44.892606+0000","last_peered":"2024-09-10T10:46:44.892606+0000","last_clean":"2024-09-10T10:46:44.892606+0000","last_became_active":"2024-09-10T10:46:44.883452+0000","last_became_peered":"2024-09-10T10:46:44.883452+0000","last_unstale":"2024-09-10T10:46:44.892606+0000","last_undegraded":"2024-09-10T10:46:44.892606+0000","last_fullsized":"2024-09-10T10:46:44.892606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:57.060214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897771+0000","last_change":"2024-09-10T10:46:43.875822+0000","last_active":"2024-09-10T10:46:44.897771+0000","last_peered":"2024-09-10T10:46:44.897771+0000","last_clean":"2024-09-10T10:46:44.897771+0000","last_became_active":"2024-09-10T10:46:43.875557+0000","last_became_peered":"2024-09-10T10:46:43.875557+0000","last_unstale":"2024-09-10T10:46:44.897771+0000","last_undegraded":"2024-09-10T10:46:44.897771+0000","last_fullsized":"2024-09-10T10:46:44.897771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.628419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.900588+0000","last_change":"2024-09-10T10:46:43.876519+0000","last_active":"2024-09-10T10:46:44.900588+0000","last_peered":"2024-09-10T10:46:44.900588+0000","last_clean":"2024-09-10T10:46:44.900588+0000","last_became_active":"2024-09-10T10:46:43.876247+0000","last_became_peered":"2024-09-10T10:46:43.876247+0000","last_unstale":"2024-09-10T10:46:44.900588+0000","last_undegraded":"2024-09-10T10:46:44.900588+0000","last_fullsized":"2024-09-10T10:46:44.900588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:00.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887341+0000","last_change":"2024-09-10T10:46:44.877202+0000","last_active":"2024-09-10T10:46:44.887341+0000","last_peered":"2024-09-10T10:46:44.887341+0000","last_clean":"2024-09-10T10:46:44.887341+0000","last_became_active":"2024-09-10T10:46:44.876159+0000","last_became_peered":"2024-09-10T10:46:44.876159+0000","last_unstale":"2024-09-10T10:46:44.887341+0000","last_undegraded":"2024-09-10T10:46:44.887341+0000","last_fullsized":"2024-09-10T10:46:44.887341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:01.241200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851870+0000","last_change":"2024-09-10T10:46:42.851870+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851870+0000","last_undegraded":"2024-09-10T10:46:42.851870+0000","last_fullsized":"2024-09-10T10:46:42.851870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.880924+0000","last_change":"2024-09-10T10:46:44.878110+0000","last_active":"2024-09-10T10:46:44.880924+0000","last_peered":"2024-09-10T10:46:44.880924+0000","last_clean":"2024-09-10T10:46:44.880924+0000","last_became_active":"2024-09-10T10:46:44.877841+0000","last_became_peered":"2024-09-10T10:46:44.877841+0000","last_unstale":"2024-09-10T10:46:44.880924+0000","last_undegraded":"2024-09-10T10:46:44.880924+0000","last_fullsized":"2024-09-10T10:46:44.880924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:45.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898049+0000","last_change":"2024-09-10T10:46:43.875159+0000","last_active":"2024-09-10T10:46:44.898049+0000","last_peered":"2024-09-10T10:46:44.898049+0000","last_clean":"2024-09-10T10:46:44.898049+0000","last_became_active":"2024-09-10T10:46:43.874602+0000","last_became_peered":"2024-09-10T10:46:43.874602+0000","last_unstale":"2024-09-10T10:46:44.898049+0000","last_undegraded":"2024-09-10T10:46:44.898049+0000","last_fullsized":"2024-09-10T10:46:44.898049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:48.767108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889322+0000","last_change":"2024-09-10T10:46:44.888134+0000","last_active":"2024-09-10T10:46:44.889322+0000","last_peered":"2024-09-10T10:46:44.889322+0000","last_clean":"2024-09-10T10:46:44.889322+0000","last_became_active":"2024-09-10T10:46:44.887851+0000","last_became_peered":"2024-09-10T10:46:44.887851+0000","last_unstale":"2024-09-10T10:46:44.889322+0000","last_undegraded":"2024-09-10T10:46:44.889322+0000","last_fullsized":"2024-09-10T10:46:44.889322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849833+0000","last_change":"2024-09-10T10:46:42.849833+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849833+0000","last_undegraded":"2024-09-10T10:46:42.849833+0000","last_fullsized":"2024-09-10T10:46:42.849833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.896764+0000","last_change":"2024-09-10T10:46:44.893318+0000","last_active":"2024-09-10T10:46:44.896764+0000","last_peered":"2024-09-10T10:46:44.896764+0000","last_clean":"2024-09-10T10:46:44.896764+0000","last_became_active":"2024-09-10T10:46:44.893039+0000","last_became_peered":"2024-09-10T10:46:44.893039+0000","last_unstale":"2024-09-10T10:46:44.896764+0000","last_undegraded":"2024-09-10T10:46:44.896764+0000","last_fullsized":"2024-09-10T10:46:44.896764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870255+0000","last_change":"2024-09-10T10:46:43.879425+0000","last_active":"2024-09-10T10:46:44.870255+0000","last_peered":"2024-09-10T10:46:44.870255+0000","last_clean":"2024-09-10T10:46:44.870255+0000","last_became_active":"2024-09-10T10:46:43.879154+0000","last_became_peered":"2024-09-10T10:46:43.879154+0000","last_unstale":"2024-09-10T10:46:44.870255+0000","last_undegraded":"2024-09-10T10:46:44.870255+0000","last_fullsized":"2024-09-10T10:46:44.870255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:41.361020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879467+0000","last_change":"2024-09-10T10:46:44.871663+0000","last_active":"2024-09-10T10:46:44.879467+0000","last_peered":"2024-09-10T10:46:44.879467+0000","last_clean":"2024-09-10T10:46:44.879467+0000","last_became_active":"2024-09-10T10:46:44.869465+0000","last_became_peered":"2024-09-10T10:46:44.869465+0000","last_unstale":"2024-09-10T10:46:44.879467+0000","last_undegraded":"2024-09-10T10:46:44.879467+0000","last_fullsized":"2024-09-10T10:46:44.879467+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:50.834542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.857464+0000","last_change":"2024-09-10T10:46:43.870161+0000","last_active":"2024-09-10T10:46:44.857464+0000","last_peered":"2024-09-10T10:46:44.857464+0000","last_clean":"2024-09-10T10:46:44.857464+0000","last_became_active":"2024-09-10T10:46:43.869918+0000","last_became_peered":"2024-09-10T10:46:43.869918+0000","last_unstale":"2024-09-10T10:46:44.857464+0000","last_undegraded":"2024-09-10T10:46:44.857464+0000","last_fullsized":"2024-09-10T10:46:44.857464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:11.275462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887228+0000","last_change":"2024-09-10T10:46:44.884495+0000","last_active":"2024-09-10T10:46:44.887228+0000","last_peered":"2024-09-10T10:46:44.887228+0000","last_clean":"2024-09-10T10:46:44.887228+0000","last_became_active":"2024-09-10T10:46:44.884231+0000","last_became_peered":"2024-09-10T10:46:44.884231+0000","last_unstale":"2024-09-10T10:46:44.887228+0000","last_undegraded":"2024-09-10T10:46:44.887228+0000","last_fullsized":"2024-09-10T10:46:44.887228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.858268+0000","last_change":"2024-09-10T10:46:43.867286+0000","last_active":"2024-09-10T10:46:44.858268+0000","last_peered":"2024-09-10T10:46:44.858268+0000","last_clean":"2024-09-10T10:46:44.858268+0000","last_became_active":"2024-09-10T10:46:43.867053+0000","last_became_peered":"2024-09-10T10:46:43.867053+0000","last_unstale":"2024-09-10T10:46:44.858268+0000","last_undegraded":"2024-09-10T10:46:44.858268+0000","last_fullsized":"2024-09-10T10:46:44.858268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.574114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881233+0000","last_change":"2024-09-10T10:46:44.870275+0000","last_active":"2024-09-10T10:46:44.881233+0000","last_peered":"2024-09-10T10:46:44.881233+0000","last_clean":"2024-09-10T10:46:44.881233+0000","last_became_active":"2024-09-10T10:46:44.869919+0000","last_became_peered":"2024-09-10T10:46:44.869919+0000","last_unstale":"2024-09-10T10:46:44.881233+0000","last_undegraded":"2024-09-10T10:46:44.881233+0000","last_fullsized":"2024-09-10T10:46:44.881233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:32.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867247+0000","last_change":"2024-09-10T10:46:43.881828+0000","last_active":"2024-09-10T10:46:44.867247+0000","last_peered":"2024-09-10T10:46:44.867247+0000","last_clean":"2024-09-10T10:46:44.867247+0000","last_became_active":"2024-09-10T10:46:43.881524+0000","last_became_peered":"2024-09-10T10:46:43.881524+0000","last_unstale":"2024-09-10T10:46:44.867247+0000","last_undegraded":"2024-09-10T10:46:44.867247+0000","last_fullsized":"2024-09-10T10:46:44.867247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:18.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890526+0000","last_change":"2024-09-10T10:46:44.877148+0000","last_active":"2024-09-10T10:46:44.890526+0000","last_peered":"2024-09-10T10:46:44.890526+0000","last_clean":"2024-09-10T10:46:44.890526+0000","last_became_active":"2024-09-10T10:46:44.876860+0000","last_became_peered":"2024-09-10T10:46:44.876860+0000","last_unstale":"2024-09-10T10:46:44.890526+0000","last_undegraded":"2024-09-10T10:46:44.890526+0000","last_fullsized":"2024-09-10T10:46:44.890526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.904206+0000","last_change":"2024-09-10T10:46:43.874911+0000","last_active":"2024-09-10T10:46:44.904206+0000","last_peered":"2024-09-10T10:46:44.904206+0000","last_clean":"2024-09-10T10:46:44.904206+0000","last_became_active":"2024-09-10T10:46:43.874629+0000","last_became_peered":"2024-09-10T10:46:43.874629+0000","last_unstale":"2024-09-10T10:46:44.904206+0000","last_undegraded":"2024-09-10T10:46:44.904206+0000","last_fullsized":"2024-09-10T10:46:44.904206+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:31.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893676+0000","last_change":"2024-09-10T10:46:44.889982+0000","last_active":"2024-09-10T10:46:44.893676+0000","last_peered":"2024-09-10T10:46:44.893676+0000","last_clean":"2024-09-10T10:46:44.893676+0000","last_became_active":"2024-09-10T10:46:44.889481+0000","last_became_peered":"2024-09-10T10:46:44.889481+0000","last_unstale":"2024-09-10T10:46:44.893676+0000","last_undegraded":"2024-09-10T10:46:44.893676+0000","last_fullsized":"2024-09-10T10:46:44.893676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:42.436804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849881+0000","last_change":"2024-09-10T10:46:42.849881+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849881+0000","last_undegraded":"2024-09-10T10:46:42.849881+0000","last_fullsized":"2024-09-10T10:46:42.849881+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894670+0000","last_change":"2024-09-10T10:46:44.886563+0000","last_active":"2024-09-10T10:46:44.894670+0000","last_peered":"2024-09-10T10:46:44.894670+0000","last_clean":"2024-09-10T10:46:44.894670+0000","last_became_active":"2024-09-10T10:46:44.886299+0000","last_became_peered":"2024-09-10T10:46:44.886299+0000","last_unstale":"2024-09-10T10:46:44.894670+0000","last_undegraded":"2024-09-10T10:46:44.894670+0000","last_fullsized":"2024-09-10T10:46:44.894670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897647+0000","last_change":"2024-09-10T10:46:43.868983+0000","last_active":"2024-09-10T10:46:44.897647+0000","last_peered":"2024-09-10T10:46:44.897647+0000","last_clean":"2024-09-10T10:46:44.897647+0000","last_became_active":"2024-09-10T10:46:43.868631+0000","last_became_peered":"2024-09-10T10:46:43.868631+0000","last_unstale":"2024-09-10T10:46:44.897647+0000","last_undegraded":"2024-09-10T10:46:44.897647+0000","last_fullsized":"2024-09-10T10:46:44.897647+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892284+0000","last_change":"2024-09-10T10:46:44.885879+0000","last_active":"2024-09-10T10:46:44.892284+0000","last_peered":"2024-09-10T10:46:44.892284+0000","last_clean":"2024-09-10T10:46:44.892284+0000","last_became_active":"2024-09-10T10:46:44.885500+0000","last_became_peered":"2024-09-10T10:46:44.885500+0000","last_unstale":"2024-09-10T10:46:44.892284+0000","last_undegraded":"2024-09-10T10:46:44.892284+0000","last_fullsized":"2024-09-10T10:46:44.892284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874282+0000","last_change":"2024-09-10T10:46:43.879565+0000","last_active":"2024-09-10T10:46:44.874282+0000","last_peered":"2024-09-10T10:46:44.874282+0000","last_clean":"2024-09-10T10:46:44.874282+0000","last_became_active":"2024-09-10T10:46:43.879254+0000","last_became_peered":"2024-09-10T10:46:43.879254+0000","last_unstale":"2024-09-10T10:46:44.874282+0000","last_undegraded":"2024-09-10T10:46:44.874282+0000","last_fullsized":"2024-09-10T10:46:44.874282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881138+0000","last_change":"2024-09-10T10:46:44.878593+0000","last_active":"2024-09-10T10:46:44.881138+0000","last_peered":"2024-09-10T10:46:44.881138+0000","last_clean":"2024-09-10T10:46:44.881138+0000","last_became_active":"2024-09-10T10:46:44.878283+0000","last_became_peered":"2024-09-10T10:46:44.878283+0000","last_unstale":"2024-09-10T10:46:44.881138+0000","last_undegraded":"2024-09-10T10:46:44.881138+0000","last_fullsized":"2024-09-10T10:46:44.881138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.865622+0000","last_change":"2024-09-10T10:46:43.877389+0000","last_active":"2024-09-10T10:46:44.865622+0000","last_peered":"2024-09-10T10:46:44.865622+0000","last_clean":"2024-09-10T10:46:44.865622+0000","last_became_active":"2024-09-10T10:46:43.876235+0000","last_became_peered":"2024-09-10T10:46:43.876235+0000","last_unstale":"2024-09-10T10:46:44.865622+0000","last_undegraded":"2024-09-10T10:46:44.865622+0000","last_fullsized":"2024-09-10T10:46:44.865622+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:52.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881110+0000","last_change":"2024-09-10T10:46:44.878187+0000","last_active":"2024-09-10T10:46:44.881110+0000","last_peered":"2024-09-10T10:46:44.881110+0000","last_clean":"2024-09-10T10:46:44.881110+0000","last_became_active":"2024-09-10T10:46:44.877922+0000","last_became_peered":"2024-09-10T10:46:44.877922+0000","last_unstale":"2024-09-10T10:46:44.881110+0000","last_undegraded":"2024-09-10T10:46:44.881110+0000","last_fullsized":"2024-09-10T10:46:44.881110+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.902316+0000","last_change":"2024-09-10T10:46:43.879721+0000","last_active":"2024-09-10T10:46:44.902316+0000","last_peered":"2024-09-10T10:46:44.902316+0000","last_clean":"2024-09-10T10:46:44.902316+0000","last_became_active":"2024-09-10T10:46:43.879448+0000","last_became_peered":"2024-09-10T10:46:43.879448+0000","last_unstale":"2024-09-10T10:46:44.902316+0000","last_undegraded":"2024-09-10T10:46:44.902316+0000","last_fullsized":"2024-09-10T10:46:44.902316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882849+0000","last_change":"2024-09-10T10:46:44.871707+0000","last_active":"2024-09-10T10:46:44.882849+0000","last_peered":"2024-09-10T10:46:44.882849+0000","last_clean":"2024-09-10T10:46:44.882849+0000","last_became_active":"2024-09-10T10:46:44.871400+0000","last_became_peered":"2024-09-10T10:46:44.871400+0000","last_unstale":"2024-09-10T10:46:44.882849+0000","last_undegraded":"2024-09-10T10:46:44.882849+0000","last_fullsized":"2024-09-10T10:46:44.882849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898284+0000","last_change":"2024-09-10T10:46:43.866850+0000","last_active":"2024-09-10T10:46:44.898284+0000","last_peered":"2024-09-10T10:46:44.898284+0000","last_clean":"2024-09-10T10:46:44.898284+0000","last_became_active":"2024-09-10T10:46:43.866575+0000","last_became_peered":"2024-09-10T10:46:43.866575+0000","last_unstale":"2024-09-10T10:46:44.898284+0000","last_undegraded":"2024-09-10T10:46:44.898284+0000","last_fullsized":"2024-09-10T10:46:44.898284+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:07.913685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890441+0000","last_change":"2024-09-10T10:46:44.875025+0000","last_active":"2024-09-10T10:46:44.890441+0000","last_peered":"2024-09-10T10:46:44.890441+0000","last_clean":"2024-09-10T10:46:44.890441+0000","last_became_active":"2024-09-10T10:46:44.873173+0000","last_became_peered":"2024-09-10T10:46:44.873173+0000","last_unstale":"2024-09-10T10:46:44.890441+0000","last_undegraded":"2024-09-10T10:46:44.890441+0000","last_fullsized":"2024-09-10T10:46:44.890441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903260+0000","last_change":"2024-09-10T10:46:43.871807+0000","last_active":"2024-09-10T10:46:44.903260+0000","last_peered":"2024-09-10T10:46:44.903260+0000","last_clean":"2024-09-10T10:46:44.903260+0000","last_became_active":"2024-09-10T10:46:43.871540+0000","last_became_peered":"2024-09-10T10:46:43.871540+0000","last_unstale":"2024-09-10T10:46:44.903260+0000","last_undegraded":"2024-09-10T10:46:44.903260+0000","last_fullsized":"2024-09-10T10:46:44.903260+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:34.555709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894618+0000","last_change":"2024-09-10T10:46:44.888440+0000","last_active":"2024-09-10T10:46:44.894618+0000","last_peered":"2024-09-10T10:46:44.894618+0000","last_clean":"2024-09-10T10:46:44.894618+0000","last_became_active":"2024-09-10T10:46:44.888154+0000","last_became_peered":"2024-09-10T10:46:44.888154+0000","last_unstale":"2024-09-10T10:46:44.894618+0000","last_undegraded":"2024-09-10T10:46:44.894618+0000","last_fullsized":"2024-09-10T10:46:44.894618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.905983+0000","last_change":"2024-09-10T10:46:43.876691+0000","last_active":"2024-09-10T10:46:44.905983+0000","last_peered":"2024-09-10T10:46:44.905983+0000","last_clean":"2024-09-10T10:46:44.905983+0000","last_became_active":"2024-09-10T10:46:43.876227+0000","last_became_peered":"2024-09-10T10:46:43.876227+0000","last_unstale":"2024-09-10T10:46:44.905983+0000","last_undegraded":"2024-09-10T10:46:44.905983+0000","last_fullsized":"2024-09-10T10:46:44.905983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:31.964863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892464+0000","last_change":"2024-09-10T10:46:44.882771+0000","last_active":"2024-09-10T10:46:44.892464+0000","last_peered":"2024-09-10T10:46:44.892464+0000","last_clean":"2024-09-10T10:46:44.892464+0000","last_became_active":"2024-09-10T10:46:44.881897+0000","last_became_peered":"2024-09-10T10:46:44.881897+0000","last_unstale":"2024-09-10T10:46:44.892464+0000","last_undegraded":"2024-09-10T10:46:44.892464+0000","last_fullsized":"2024-09-10T10:46:44.892464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:22.411021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867582+0000","last_change":"2024-09-10T10:46:43.873156+0000","last_active":"2024-09-10T10:46:44.867582+0000","last_peered":"2024-09-10T10:46:44.867582+0000","last_clean":"2024-09-10T10:46:44.867582+0000","last_became_active":"2024-09-10T10:46:43.872889+0000","last_became_peered":"2024-09-10T10:46:43.872889+0000","last_unstale":"2024-09-10T10:46:44.867582+0000","last_undegraded":"2024-09-10T10:46:44.867582+0000","last_fullsized":"2024-09-10T10:46:44.867582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852599+0000","last_change":"2024-09-10T10:46:42.852599+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852599+0000","last_undegraded":"2024-09-10T10:46:42.852599+0000","last_fullsized":"2024-09-10T10:46:42.852599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872927+0000","last_change":"2024-09-10T10:46:44.868747+0000","last_active":"2024-09-10T10:46:44.872927+0000","last_peered":"2024-09-10T10:46:44.872927+0000","last_clean":"2024-09-10T10:46:44.872927+0000","last_became_active":"2024-09-10T10:46:44.868455+0000","last_became_peered":"2024-09-10T10:46:44.868455+0000","last_unstale":"2024-09-10T10:46:44.872927+0000","last_undegraded":"2024-09-10T10:46:44.872927+0000","last_fullsized":"2024-09-10T10:46:44.872927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874479+0000","last_change":"2024-09-10T10:46:43.876827+0000","last_active":"2024-09-10T10:46:44.874479+0000","last_peered":"2024-09-10T10:46:44.874479+0000","last_clean":"2024-09-10T10:46:44.874479+0000","last_became_active":"2024-09-10T10:46:43.875591+0000","last_became_peered":"2024-09-10T10:46:43.875591+0000","last_unstale":"2024-09-10T10:46:44.874479+0000","last_undegraded":"2024-09-10T10:46:44.874479+0000","last_fullsized":"2024-09-10T10:46:44.874479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889806+0000","last_change":"2024-09-10T10:46:44.883361+0000","last_active":"2024-09-10T10:46:44.889806+0000","last_peered":"2024-09-10T10:46:44.889806+0000","last_clean":"2024-09-10T10:46:44.889806+0000","last_became_active":"2024-09-10T10:46:44.879641+0000","last_became_peered":"2024-09-10T10:46:44.879641+0000","last_unstale":"2024-09-10T10:46:44.889806+0000","last_undegraded":"2024-09-10T10:46:44.889806+0000","last_fullsized":"2024-09-10T10:46:44.889806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849801+0000","last_change":"2024-09-10T10:46:42.849801+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849801+0000","last_undegraded":"2024-09-10T10:46:42.849801+0000","last_fullsized":"2024-09-10T10:46:42.849801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879741+0000","last_change":"2024-09-10T10:46:44.873964+0000","last_active":"2024-09-10T10:46:44.879741+0000","last_peered":"2024-09-10T10:46:44.879741+0000","last_clean":"2024-09-10T10:46:44.879741+0000","last_became_active":"2024-09-10T10:46:44.873324+0000","last_became_peered":"2024-09-10T10:46:44.873324+0000","last_unstale":"2024-09-10T10:46:44.879741+0000","last_undegraded":"2024-09-10T10:46:44.879741+0000","last_fullsized":"2024-09-10T10:46:44.879741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:46.801922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897908+0000","last_change":"2024-09-10T10:46:43.880123+0000","last_active":"2024-09-10T10:46:44.897908+0000","last_peered":"2024-09-10T10:46:44.897908+0000","last_clean":"2024-09-10T10:46:44.897908+0000","last_became_active":"2024-09-10T10:46:43.879861+0000","last_became_peered":"2024-09-10T10:46:43.879861+0000","last_unstale":"2024-09-10T10:46:44.897908+0000","last_undegraded":"2024-09-10T10:46:44.897908+0000","last_fullsized":"2024-09-10T10:46:44.897908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:40.170023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881256+0000","last_change":"2024-09-10T10:46:44.875117+0000","last_active":"2024-09-10T10:46:44.881256+0000","last_peered":"2024-09-10T10:46:44.881256+0000","last_clean":"2024-09-10T10:46:44.881256+0000","last_became_active":"2024-09-10T10:46:44.874858+0000","last_became_peered":"2024-09-10T10:46:44.874858+0000","last_unstale":"2024-09-10T10:46:44.881256+0000","last_undegraded":"2024-09-10T10:46:44.881256+0000","last_fullsized":"2024-09-10T10:46:44.881256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:08.230635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.859204+0000","last_change":"2024-09-10T10:46:43.870732+0000","last_active":"2024-09-10T10:46:44.859204+0000","last_peered":"2024-09-10T10:46:44.859204+0000","last_clean":"2024-09-10T10:46:44.859204+0000","last_became_active":"2024-09-10T10:46:43.870484+0000","last_became_peered":"2024-09-10T10:46:43.870484+0000","last_unstale":"2024-09-10T10:46:44.859204+0000","last_undegraded":"2024-09-10T10:46:44.859204+0000","last_fullsized":"2024-09-10T10:46:44.859204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:05.738817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881485+0000","last_change":"2024-09-10T10:46:44.875783+0000","last_active":"2024-09-10T10:46:44.881485+0000","last_peered":"2024-09-10T10:46:44.881485+0000","last_clean":"2024-09-10T10:46:44.881485+0000","last_became_active":"2024-09-10T10:46:44.875511+0000","last_became_peered":"2024-09-10T10:46:44.875511+0000","last_unstale":"2024-09-10T10:46:44.881485+0000","last_undegraded":"2024-09-10T10:46:44.881485+0000","last_fullsized":"2024-09-10T10:46:44.881485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:37.571008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852053+0000","last_change":"2024-09-10T10:46:42.852053+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852053+0000","last_undegraded":"2024-09-10T10:46:42.852053+0000","last_fullsized":"2024-09-10T10:46:42.852053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892438+0000","last_change":"2024-09-10T10:46:44.882540+0000","last_active":"2024-09-10T10:46:44.892438+0000","last_peered":"2024-09-10T10:46:44.892438+0000","last_clean":"2024-09-10T10:46:44.892438+0000","last_became_active":"2024-09-10T10:46:44.881322+0000","last_became_peered":"2024-09-10T10:46:44.881322+0000","last_unstale":"2024-09-10T10:46:44.892438+0000","last_undegraded":"2024-09-10T10:46:44.892438+0000","last_fullsized":"2024-09-10T10:46:44.892438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:05.028664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867719+0000","last_change":"2024-09-10T10:46:43.880848+0000","last_active":"2024-09-10T10:46:44.867719+0000","last_peered":"2024-09-10T10:46:44.867719+0000","last_clean":"2024-09-10T10:46:44.867719+0000","last_became_active":"2024-09-10T10:46:43.880588+0000","last_became_peered":"2024-09-10T10:46:43.880588+0000","last_unstale":"2024-09-10T10:46:44.867719+0000","last_undegraded":"2024-09-10T10:46:44.867719+0000","last_fullsized":"2024-09-10T10:46:44.867719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887215+0000","last_change":"2024-09-10T10:46:44.879311+0000","last_active":"2024-09-10T10:46:44.887215+0000","last_peered":"2024-09-10T10:46:44.887215+0000","last_clean":"2024-09-10T10:46:44.887215+0000","last_became_active":"2024-09-10T10:46:44.867480+0000","last_became_peered":"2024-09-10T10:46:44.867480+0000","last_unstale":"2024-09-10T10:46:44.887215+0000","last_undegraded":"2024-09-10T10:46:44.887215+0000","last_fullsized":"2024-09-10T10:46:44.887215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.866776+0000","last_change":"2024-09-10T10:46:43.878330+0000","last_active":"2024-09-10T10:46:44.866776+0000","last_peered":"2024-09-10T10:46:44.866776+0000","last_clean":"2024-09-10T10:46:44.866776+0000","last_became_active":"2024-09-10T10:46:43.877459+0000","last_became_peered":"2024-09-10T10:46:43.877459+0000","last_unstale":"2024-09-10T10:46:44.866776+0000","last_undegraded":"2024-09-10T10:46:44.866776+0000","last_fullsized":"2024-09-10T10:46:44.866776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:13.837013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872902+0000","last_change":"2024-09-10T10:46:44.868549+0000","last_active":"2024-09-10T10:46:44.872902+0000","last_peered":"2024-09-10T10:46:44.872902+0000","last_clean":"2024-09-10T10:46:44.872902+0000","last_became_active":"2024-09-10T10:46:44.868232+0000","last_became_peered":"2024-09-10T10:46:44.868232+0000","last_unstale":"2024-09-10T10:46:44.872902+0000","last_undegraded":"2024-09-10T10:46:44.872902+0000","last_fullsized":"2024-09-10T10:46:44.872902+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870340+0000","last_change":"2024-09-10T10:46:43.879611+0000","last_active":"2024-09-10T10:46:44.870340+0000","last_peered":"2024-09-10T10:46:44.870340+0000","last_clean":"2024-09-10T10:46:44.870340+0000","last_became_active":"2024-09-10T10:46:43.879308+0000","last_became_peered":"2024-09-10T10:46:43.879308+0000","last_unstale":"2024-09-10T10:46:44.870340+0000","last_undegraded":"2024-09-10T10:46:44.870340+0000","last_fullsized":"2024-09-10T10:46:44.870340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:57.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867820+0000","last_change":"2024-09-10T10:46:43.879656+0000","last_active":"2024-09-10T10:46:44.867820+0000","last_peered":"2024-09-10T10:46:44.867820+0000","last_clean":"2024-09-10T10:46:44.867820+0000","last_became_active":"2024-09-10T10:46:43.879098+0000","last_became_peered":"2024-09-10T10:46:43.879098+0000","last_unstale":"2024-09-10T10:46:44.867820+0000","last_undegraded":"2024-09-10T10:46:44.867820+0000","last_fullsized":"2024-09-10T10:46:44.867820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879442+0000","last_change":"2024-09-10T10:46:44.869395+0000","last_active":"2024-09-10T10:46:44.879442+0000","last_peered":"2024-09-10T10:46:44.879442+0000","last_clean":"2024-09-10T10:46:44.879442+0000","last_became_active":"2024-09-10T10:46:44.869065+0000","last_became_peered":"2024-09-10T10:46:44.869065+0000","last_unstale":"2024-09-10T10:46:44.879442+0000","last_undegraded":"2024-09-10T10:46:44.879442+0000","last_fullsized":"2024-09-10T10:46:44.879442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:52.312041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851961+0000","last_change":"2024-09-10T10:46:42.851961+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851961+0000","last_undegraded":"2024-09-10T10:46:42.851961+0000","last_fullsized":"2024-09-10T10:46:42.851961+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891869+0000","last_change":"2024-09-10T10:46:44.885648+0000","last_active":"2024-09-10T10:46:44.891869+0000","last_peered":"2024-09-10T10:46:44.891869+0000","last_clean":"2024-09-10T10:46:44.891869+0000","last_became_active":"2024-09-10T10:46:44.885219+0000","last_became_peered":"2024-09-10T10:46:44.885219+0000","last_unstale":"2024-09-10T10:46:44.891869+0000","last_undegraded":"2024-09-10T10:46:44.891869+0000","last_fullsized":"2024-09-10T10:46:44.891869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:19.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.850800+0000","last_change":"2024-09-10T10:46:42.850800+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.850800+0000","last_undegraded":"2024-09-10T10:46:42.850800+0000","last_fullsized":"2024-09-10T10:46:42.850800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":257,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.871099+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:44.871099+0000","last_peered":"2024-09-10T10:46:44.871099+0000","last_clean":"2024-09-10T10:46:44.871099+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:44.871099+0000","last_undegraded":"2024-09-10T10:46:44.871099+0000","last_fullsized":"2024-09-10T10:46:44.871099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":367029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":374778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":374356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":835584,"data_stored":374252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":964598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1396736,"data_stored":950730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:49.077 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.267+0000 7fe15ba00640 1 Processor -- start 2024-09-10T10:46:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.267+0000 7fe15ba00640 1 -- start start 2024-09-10T10:46:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe15ba00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe15ba00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe15ba00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1540cf770 con 0x7fe1540b6760 2024-09-10T10:46:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe15ba00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe1541409c0 con 0x7fe1540b6b40 2024-09-10T10:46:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe15ba00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe1540cfa50 con 0x7fe154140010 2024-09-10T10:46:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe159a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe153e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe153e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:55738/0 (socket says 172.21.15.28:55738) 2024-09-10T10:46:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe159000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe1540b6760 0x7fe15415b8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:33290/0 (socket says 172.21.15.28:33290) 2024-09-10T10:46:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe153e00640 1 -- 172.21.15.28:0/930914953 learned_addr learned my addr 172.21.15.28:0/930914953 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.269+0000 7fe159a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:51698/0 (socket says 172.21.15.28:51698) 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.270+0000 7fe159a00640 1 -- 172.21.15.28:0/930914953 >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 msgr2=0x7fe1540cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.270+0000 7fe159a00640 1 --2- 172.21.15.28:0/930914953 >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.270+0000 7fe159a00640 1 -- 172.21.15.28:0/930914953 >> v1:172.21.15.138:6789/0 conn(0x7fe1540b6760 legacy=0x7fe15415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.270+0000 7fe159a00640 1 -- 172.21.15.28:0/930914953 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1540cfd30 con 0x7fe154140010 2024-09-10T10:46:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.270+0000 7fe153e00640 1 --2- 172.21.15.28:0/930914953 >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:46:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.271+0000 7fe159a00640 1 --2- 172.21.15.28:0/930914953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38704e80c8a3717b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.271+0000 7fe153400640 1 -- 172.21.15.28:0/930914953 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c007740 con 0x7fe154140010 2024-09-10T10:46:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.271+0000 7fe153400640 1 -- 172.21.15.28:0/930914953 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14c0078e0 con 0x7fe154140010 2024-09-10T10:46:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.271+0000 7fe153400640 1 -- 172.21.15.28:0/930914953 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14c007cb0 con 0x7fe154140010 2024-09-10T10:46:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 -- 172.21.15.28:0/930914953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 msgr2=0x7fe154140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 --2- 172.21.15.28:0/930914953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c02a660 comp rx=0 tx=0).stop 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 -- 172.21.15.28:0/930914953 shutdown_connections 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 --2- 172.21.15.28:0/930914953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe154140480 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 --2- 172.21.15.28:0/930914953 >> v2:172.21.15.138:3300/0 conn(0x7fe1540b6b40 0x7fe1540cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 -- 172.21.15.28:0/930914953 >> 172.21.15.28:0/930914953 conn(0x7fe15406bc10 msgr2=0x7fe15406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 -- 172.21.15.28:0/930914953 shutdown_connections 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.272+0000 7fe15ba00640 1 -- 172.21.15.28:0/930914953 wait complete. 2024-09-10T10:46:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.274+0000 7fe15ba00640 1 Processor -- start 2024-09-10T10:46:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.274+0000 7fe15ba00640 1 -- start start 2024-09-10T10:46:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1540b6760 0x7fe154138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe1541662f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540d0810 con 0x7fe1540b6b40 2024-09-10T10:46:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1540d02f0 con 0x7fe1540b6760 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe15ba00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe154067950 con 0x7fe154140010 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50112/0 (socket says 172.21.15.28:50112) 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 -- 172.21.15.28:0/900364104 learned_addr learned my addr 172.21.15.28:0/900364104 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe159a00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe1541662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe159000640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1540b6760 0x7fe154138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 -- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 msgr2=0x7fe1541662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe1541662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 -- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1540b6760 msgr2=0x7fe154138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1540b6760 0x7fe154138640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.275+0000 7fe153e00640 1 -- 172.21.15.28:0/900364104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe154142ff0 con 0x7fe1540b6b40 2024-09-10T10:46:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe153e00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e9d0 tx=0x7fe14000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe140017070 con 0x7fe1540b6b40 2024-09-10T10:46:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe14000ca60 con 0x7fe1540b6b40 2024-09-10T10:46:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14000ce10 con 0x7fe1540b6b40 2024-09-10T10:46:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe154143290 con 0x7fe1540b6b40 2024-09-10T10:46:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.276+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe154143850 con 0x7fe1540b6b40 2024-09-10T10:46:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.277+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154067cd0 con 0x7fe1540b6b40 2024-09-10T10:46:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.278+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe14001a070 con 0x7fe1540b6b40 2024-09-10T10:46:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.279+0000 7fe150c00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.279+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fe14008afe0 con 0x7fe1540b6b40 2024-09-10T10:46:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.279+0000 7fe159000640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.280+0000 7fe159000640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 0x7fe13406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f60 tx=0x7fe148007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.283+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe140059760 con 0x7fe1540b6b40 2024-09-10T10:46:49.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.464+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe15413f840 con 0x7fe13406be30 2024-09-10T10:46:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.471+0000 7fe150c00640 1 -- 172.21.15.28:0/900364104 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250382 (secure 0 0 0) 0x7fe15413f840 con 0x7fe13406be30 2024-09-10T10:46:49.474 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:49.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 msgr2=0x7fe13406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 0x7fe13406e2f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f60 tx=0x7fe148007cc0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 msgr2=0x7fe154142ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe14000e9d0 tx=0x7fe14000ee90 comp rx=0 tx=0).stop 2024-09-10T10:46:49.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 shutdown_connections 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe13406be30 0x7fe13406e2f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe154140010 0x7fe1541662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1540b6b40 0x7fe154142ab0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 --2- 172.21.15.28:0/900364104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1540b6760 0x7fe154138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 >> 172.21.15.28:0/900364104 conn(0x7fe15406bc10 msgr2=0x7fe15406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 shutdown_connections 2024-09-10T10:46:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.474+0000 7fe15ba00640 1 -- 172.21.15.28:0/900364104 wait complete. 2024-09-10T10:46:49.491 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T10:46:48.526234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244664,"kb_used_data":7612,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730056,"statfs":{"total":773094113280,"available":772843577344,"internally_reserved":0,"allocated":7794688,"data_stored":4154135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.680947"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895722+0000","last_change":"2024-09-10T10:46:44.892310+0000","last_active":"2024-09-10T10:46:44.895722+0000","last_peered":"2024-09-10T10:46:44.895722+0000","last_clean":"2024-09-10T10:46:44.895722+0000","last_became_active":"2024-09-10T10:46:44.892049+0000","last_became_peered":"2024-09-10T10:46:44.892049+0000","last_unstale":"2024-09-10T10:46:44.895722+0000","last_undegraded":"2024-09-10T10:46:44.895722+0000","last_fullsized":"2024-09-10T10:46:44.895722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892585+0000","last_change":"2024-09-10T10:46:44.884268+0000","last_active":"2024-09-10T10:46:44.892585+0000","last_peered":"2024-09-10T10:46:44.892585+0000","last_clean":"2024-09-10T10:46:44.892585+0000","last_became_active":"2024-09-10T10:46:44.883101+0000","last_became_peered":"2024-09-10T10:46:44.883101+0000","last_unstale":"2024-09-10T10:46:44.892585+0000","last_undegraded":"2024-09-10T10:46:44.892585+0000","last_fullsized":"2024-09-10T10:46:44.892585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:14.573857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882869+0000","last_change":"2024-09-10T10:46:44.875622+0000","last_active":"2024-09-10T10:46:44.882869+0000","last_peered":"2024-09-10T10:46:44.882869+0000","last_clean":"2024-09-10T10:46:44.882869+0000","last_became_active":"2024-09-10T10:46:44.875330+0000","last_became_peered":"2024-09-10T10:46:44.875330+0000","last_unstale":"2024-09-10T10:46:44.882869+0000","last_undegraded":"2024-09-10T10:46:44.882869+0000","last_fullsized":"2024-09-10T10:46:44.882869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892693+0000","last_change":"2024-09-10T10:46:44.884914+0000","last_active":"2024-09-10T10:46:44.892693+0000","last_peered":"2024-09-10T10:46:44.892693+0000","last_clean":"2024-09-10T10:46:44.892693+0000","last_became_active":"2024-09-10T10:46:44.884638+0000","last_became_peered":"2024-09-10T10:46:44.884638+0000","last_unstale":"2024-09-10T10:46:44.892693+0000","last_undegraded":"2024-09-10T10:46:44.892693+0000","last_fullsized":"2024-09-10T10:46:44.892693+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:20.876408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.884835+0000","last_change":"2024-09-10T10:46:44.883459+0000","last_active":"2024-09-10T10:46:44.884835+0000","last_peered":"2024-09-10T10:46:44.884835+0000","last_clean":"2024-09-10T10:46:44.884835+0000","last_became_active":"2024-09-10T10:46:44.883173+0000","last_became_peered":"2024-09-10T10:46:44.883173+0000","last_unstale":"2024-09-10T10:46:44.884835+0000","last_undegraded":"2024-09-10T10:46:44.884835+0000","last_fullsized":"2024-09-10T10:46:44.884835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893848+0000","last_change":"2024-09-10T10:46:44.890220+0000","last_active":"2024-09-10T10:46:44.893848+0000","last_peered":"2024-09-10T10:46:44.893848+0000","last_clean":"2024-09-10T10:46:44.893848+0000","last_became_active":"2024-09-10T10:46:44.889918+0000","last_became_peered":"2024-09-10T10:46:44.889918+0000","last_unstale":"2024-09-10T10:46:44.893848+0000","last_undegraded":"2024-09-10T10:46:44.893848+0000","last_fullsized":"2024-09-10T10:46:44.893848+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:58.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887361+0000","last_change":"2024-09-10T10:46:44.877805+0000","last_active":"2024-09-10T10:46:44.887361+0000","last_peered":"2024-09-10T10:46:44.887361+0000","last_clean":"2024-09-10T10:46:44.887361+0000","last_became_active":"2024-09-10T10:46:44.875492+0000","last_became_peered":"2024-09-10T10:46:44.875492+0000","last_unstale":"2024-09-10T10:46:44.887361+0000","last_undegraded":"2024-09-10T10:46:44.887361+0000","last_fullsized":"2024-09-10T10:46:44.887361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:42.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895164+0000","last_change":"2024-09-10T10:46:44.886608+0000","last_active":"2024-09-10T10:46:44.895164+0000","last_peered":"2024-09-10T10:46:44.895164+0000","last_clean":"2024-09-10T10:46:44.895164+0000","last_became_active":"2024-09-10T10:46:44.886293+0000","last_became_peered":"2024-09-10T10:46:44.886293+0000","last_unstale":"2024-09-10T10:46:44.895164+0000","last_undegraded":"2024-09-10T10:46:44.895164+0000","last_fullsized":"2024-09-10T10:46:44.895164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887118+0000","last_change":"2024-09-10T10:46:44.878902+0000","last_active":"2024-09-10T10:46:44.887118+0000","last_peered":"2024-09-10T10:46:44.887118+0000","last_clean":"2024-09-10T10:46:44.887118+0000","last_became_active":"2024-09-10T10:46:44.878564+0000","last_became_peered":"2024-09-10T10:46:44.878564+0000","last_unstale":"2024-09-10T10:46:44.887118+0000","last_undegraded":"2024-09-10T10:46:44.887118+0000","last_fullsized":"2024-09-10T10:46:44.887118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887230+0000","last_change":"2024-09-10T10:46:44.882681+0000","last_active":"2024-09-10T10:46:44.887230+0000","last_peered":"2024-09-10T10:46:44.887230+0000","last_clean":"2024-09-10T10:46:44.887230+0000","last_became_active":"2024-09-10T10:46:44.882405+0000","last_became_peered":"2024-09-10T10:46:44.882405+0000","last_unstale":"2024-09-10T10:46:44.887230+0000","last_undegraded":"2024-09-10T10:46:44.887230+0000","last_fullsized":"2024-09-10T10:46:44.887230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894549+0000","last_change":"2024-09-10T10:46:44.883967+0000","last_active":"2024-09-10T10:46:44.894549+0000","last_peered":"2024-09-10T10:46:44.894549+0000","last_clean":"2024-09-10T10:46:44.894549+0000","last_became_active":"2024-09-10T10:46:44.877104+0000","last_became_peered":"2024-09-10T10:46:44.877104+0000","last_unstale":"2024-09-10T10:46:44.894549+0000","last_undegraded":"2024-09-10T10:46:44.894549+0000","last_fullsized":"2024-09-10T10:46:44.894549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894319+0000","last_change":"2024-09-10T10:46:44.880957+0000","last_active":"2024-09-10T10:46:44.894319+0000","last_peered":"2024-09-10T10:46:44.894319+0000","last_clean":"2024-09-10T10:46:44.894319+0000","last_became_active":"2024-09-10T10:46:44.880691+0000","last_became_peered":"2024-09-10T10:46:44.880691+0000","last_unstale":"2024-09-10T10:46:44.894319+0000","last_undegraded":"2024-09-10T10:46:44.894319+0000","last_fullsized":"2024-09-10T10:46:44.894319+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:06.745405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891012+0000","last_change":"2024-09-10T10:46:44.882838+0000","last_active":"2024-09-10T10:46:44.891012+0000","last_peered":"2024-09-10T10:46:44.891012+0000","last_clean":"2024-09-10T10:46:44.891012+0000","last_became_active":"2024-09-10T10:46:44.880913+0000","last_became_peered":"2024-09-10T10:46:44.880913+0000","last_unstale":"2024-09-10T10:46:44.891012+0000","last_undegraded":"2024-09-10T10:46:44.891012+0000","last_fullsized":"2024-09-10T10:46:44.891012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881711+0000","last_change":"2024-09-10T10:46:44.879407+0000","last_active":"2024-09-10T10:46:44.881711+0000","last_peered":"2024-09-10T10:46:44.881711+0000","last_clean":"2024-09-10T10:46:44.881711+0000","last_became_active":"2024-09-10T10:46:44.879124+0000","last_became_peered":"2024-09-10T10:46:44.879124+0000","last_unstale":"2024-09-10T10:46:44.881711+0000","last_undegraded":"2024-09-10T10:46:44.881711+0000","last_fullsized":"2024-09-10T10:46:44.881711+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879528+0000","last_change":"2024-09-10T10:46:44.874639+0000","last_active":"2024-09-10T10:46:44.879528+0000","last_peered":"2024-09-10T10:46:44.879528+0000","last_clean":"2024-09-10T10:46:44.879528+0000","last_became_active":"2024-09-10T10:46:44.874387+0000","last_became_peered":"2024-09-10T10:46:44.874387+0000","last_unstale":"2024-09-10T10:46:44.879528+0000","last_undegraded":"2024-09-10T10:46:44.879528+0000","last_fullsized":"2024-09-10T10:46:44.879528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:38:57.911434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874284+0000","last_change":"2024-09-10T10:46:44.865791+0000","last_active":"2024-09-10T10:46:44.874284+0000","last_peered":"2024-09-10T10:46:44.874284+0000","last_clean":"2024-09-10T10:46:44.874284+0000","last_became_active":"2024-09-10T10:46:44.865491+0000","last_became_peered":"2024-09-10T10:46:44.865491+0000","last_unstale":"2024-09-10T10:46:44.874284+0000","last_undegraded":"2024-09-10T10:46:44.874284+0000","last_fullsized":"2024-09-10T10:46:44.874284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887094+0000","last_change":"2024-09-10T10:46:44.878815+0000","last_active":"2024-09-10T10:46:44.887094+0000","last_peered":"2024-09-10T10:46:44.887094+0000","last_clean":"2024-09-10T10:46:44.887094+0000","last_became_active":"2024-09-10T10:46:44.878192+0000","last_became_peered":"2024-09-10T10:46:44.878192+0000","last_unstale":"2024-09-10T10:46:44.887094+0000","last_undegraded":"2024-09-10T10:46:44.887094+0000","last_fullsized":"2024-09-10T10:46:44.887094+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879723+0000","last_change":"2024-09-10T10:46:44.871127+0000","last_active":"2024-09-10T10:46:44.879723+0000","last_peered":"2024-09-10T10:46:44.879723+0000","last_clean":"2024-09-10T10:46:44.879723+0000","last_became_active":"2024-09-10T10:46:44.870842+0000","last_became_peered":"2024-09-10T10:46:44.870842+0000","last_unstale":"2024-09-10T10:46:44.879723+0000","last_undegraded":"2024-09-10T10:46:44.879723+0000","last_fullsized":"2024-09-10T10:46:44.879723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:14.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894528+0000","last_change":"2024-09-10T10:46:44.882561+0000","last_active":"2024-09-10T10:46:44.894528+0000","last_peered":"2024-09-10T10:46:44.894528+0000","last_clean":"2024-09-10T10:46:44.894528+0000","last_became_active":"2024-09-10T10:46:44.877515+0000","last_became_peered":"2024-09-10T10:46:44.877515+0000","last_unstale":"2024-09-10T10:46:44.894528+0000","last_undegraded":"2024-09-10T10:46:44.894528+0000","last_fullsized":"2024-09-10T10:46:44.894528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:35.562296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881741+0000","last_change":"2024-09-10T10:46:44.878483+0000","last_active":"2024-09-10T10:46:44.881741+0000","last_peered":"2024-09-10T10:46:44.881741+0000","last_clean":"2024-09-10T10:46:44.881741+0000","last_became_active":"2024-09-10T10:46:44.874186+0000","last_became_peered":"2024-09-10T10:46:44.874186+0000","last_unstale":"2024-09-10T10:46:44.881741+0000","last_undegraded":"2024-09-10T10:46:44.881741+0000","last_fullsized":"2024-09-10T10:46:44.881741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893869+0000","last_change":"2024-09-10T10:46:44.890973+0000","last_active":"2024-09-10T10:46:44.893869+0000","last_peered":"2024-09-10T10:46:44.893869+0000","last_clean":"2024-09-10T10:46:44.893869+0000","last_became_active":"2024-09-10T10:46:44.889471+0000","last_became_peered":"2024-09-10T10:46:44.889471+0000","last_unstale":"2024-09-10T10:46:44.893869+0000","last_undegraded":"2024-09-10T10:46:44.893869+0000","last_fullsized":"2024-09-10T10:46:44.893869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:28.051816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894600+0000","last_change":"2024-09-10T10:46:44.888201+0000","last_active":"2024-09-10T10:46:44.894600+0000","last_peered":"2024-09-10T10:46:44.894600+0000","last_clean":"2024-09-10T10:46:44.894600+0000","last_became_active":"2024-09-10T10:46:44.887970+0000","last_became_peered":"2024-09-10T10:46:44.887970+0000","last_unstale":"2024-09-10T10:46:44.894600+0000","last_undegraded":"2024-09-10T10:46:44.894600+0000","last_fullsized":"2024-09-10T10:46:44.894600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891947+0000","last_change":"2024-09-10T10:46:44.881612+0000","last_active":"2024-09-10T10:46:44.891947+0000","last_peered":"2024-09-10T10:46:44.891947+0000","last_clean":"2024-09-10T10:46:44.891947+0000","last_became_active":"2024-09-10T10:46:44.881250+0000","last_became_peered":"2024-09-10T10:46:44.881250+0000","last_unstale":"2024-09-10T10:46:44.891947+0000","last_undegraded":"2024-09-10T10:46:44.891947+0000","last_fullsized":"2024-09-10T10:46:44.891947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:44.946566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891469+0000","last_change":"2024-09-10T10:46:44.878716+0000","last_active":"2024-09-10T10:46:44.891469+0000","last_peered":"2024-09-10T10:46:44.891469+0000","last_clean":"2024-09-10T10:46:44.891469+0000","last_became_active":"2024-09-10T10:46:44.878422+0000","last_became_peered":"2024-09-10T10:46:44.878422+0000","last_unstale":"2024-09-10T10:46:44.891469+0000","last_undegraded":"2024-09-10T10:46:44.891469+0000","last_fullsized":"2024-09-10T10:46:44.891469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:41.176611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.888369+0000","last_change":"2024-09-10T10:46:44.886054+0000","last_active":"2024-09-10T10:46:44.888369+0000","last_peered":"2024-09-10T10:46:44.888369+0000","last_clean":"2024-09-10T10:46:44.888369+0000","last_became_active":"2024-09-10T10:46:44.885269+0000","last_became_peered":"2024-09-10T10:46:44.885269+0000","last_unstale":"2024-09-10T10:46:44.888369+0000","last_undegraded":"2024-09-10T10:46:44.888369+0000","last_fullsized":"2024-09-10T10:46:44.888369+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874510+0000","last_change":"2024-09-10T10:46:44.869461+0000","last_active":"2024-09-10T10:46:44.874510+0000","last_peered":"2024-09-10T10:46:44.874510+0000","last_clean":"2024-09-10T10:46:44.874510+0000","last_became_active":"2024-09-10T10:46:44.869194+0000","last_became_peered":"2024-09-10T10:46:44.869194+0000","last_unstale":"2024-09-10T10:46:44.874510+0000","last_undegraded":"2024-09-10T10:46:44.874510+0000","last_fullsized":"2024-09-10T10:46:44.874510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887172+0000","last_change":"2024-09-10T10:46:44.879808+0000","last_active":"2024-09-10T10:46:44.887172+0000","last_peered":"2024-09-10T10:46:44.887172+0000","last_clean":"2024-09-10T10:46:44.887172+0000","last_became_active":"2024-09-10T10:46:44.879312+0000","last_became_peered":"2024-09-10T10:46:44.879312+0000","last_unstale":"2024-09-10T10:46:44.887172+0000","last_undegraded":"2024-09-10T10:46:44.887172+0000","last_fullsized":"2024-09-10T10:46:44.887172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892504+0000","last_change":"2024-09-10T10:46:44.881613+0000","last_active":"2024-09-10T10:46:44.892504+0000","last_peered":"2024-09-10T10:46:44.892504+0000","last_clean":"2024-09-10T10:46:44.892504+0000","last_became_active":"2024-09-10T10:46:44.881356+0000","last_became_peered":"2024-09-10T10:46:44.881356+0000","last_unstale":"2024-09-10T10:46:44.892504+0000","last_undegraded":"2024-09-10T10:46:44.892504+0000","last_fullsized":"2024-09-10T10:46:44.892504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:04.057439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.883453+0000","last_change":"2024-09-10T10:46:44.882083+0000","last_active":"2024-09-10T10:46:44.883453+0000","last_peered":"2024-09-10T10:46:44.883453+0000","last_clean":"2024-09-10T10:46:44.883453+0000","last_became_active":"2024-09-10T10:46:44.878774+0000","last_became_peered":"2024-09-10T10:46:44.878774+0000","last_unstale":"2024-09-10T10:46:44.883453+0000","last_undegraded":"2024-09-10T10:46:44.883453+0000","last_fullsized":"2024-09-10T10:46:44.883453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890870+0000","last_change":"2024-09-10T10:46:44.884761+0000","last_active":"2024-09-10T10:46:44.890870+0000","last_peered":"2024-09-10T10:46:44.890870+0000","last_clean":"2024-09-10T10:46:44.890870+0000","last_became_active":"2024-09-10T10:46:44.884412+0000","last_became_peered":"2024-09-10T10:46:44.884412+0000","last_unstale":"2024-09-10T10:46:44.890870+0000","last_undegraded":"2024-09-10T10:46:44.890870+0000","last_fullsized":"2024-09-10T10:46:44.890870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903682+0000","last_change":"2024-09-10T10:46:43.873120+0000","last_active":"2024-09-10T10:46:44.903682+0000","last_peered":"2024-09-10T10:46:44.903682+0000","last_clean":"2024-09-10T10:46:44.903682+0000","last_became_active":"2024-09-10T10:46:43.872682+0000","last_became_peered":"2024-09-10T10:46:43.872682+0000","last_unstale":"2024-09-10T10:46:44.903682+0000","last_undegraded":"2024-09-10T10:46:44.903682+0000","last_fullsized":"2024-09-10T10:46:44.903682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.538093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892606+0000","last_change":"2024-09-10T10:46:44.885626+0000","last_active":"2024-09-10T10:46:44.892606+0000","last_peered":"2024-09-10T10:46:44.892606+0000","last_clean":"2024-09-10T10:46:44.892606+0000","last_became_active":"2024-09-10T10:46:44.883452+0000","last_became_peered":"2024-09-10T10:46:44.883452+0000","last_unstale":"2024-09-10T10:46:44.892606+0000","last_undegraded":"2024-09-10T10:46:44.892606+0000","last_fullsized":"2024-09-10T10:46:44.892606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:57.060214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897771+0000","last_change":"2024-09-10T10:46:43.875822+0000","last_active":"2024-09-10T10:46:44.897771+0000","last_peered":"2024-09-10T10:46:44.897771+0000","last_clean":"2024-09-10T10:46:44.897771+0000","last_became_active":"2024-09-10T10:46:43.875557+0000","last_became_peered":"2024-09-10T10:46:43.875557+0000","last_unstale":"2024-09-10T10:46:44.897771+0000","last_undegraded":"2024-09-10T10:46:44.897771+0000","last_fullsized":"2024-09-10T10:46:44.897771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.628419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.900588+0000","last_change":"2024-09-10T10:46:43.876519+0000","last_active":"2024-09-10T10:46:44.900588+0000","last_peered":"2024-09-10T10:46:44.900588+0000","last_clean":"2024-09-10T10:46:44.900588+0000","last_became_active":"2024-09-10T10:46:43.876247+0000","last_became_peered":"2024-09-10T10:46:43.876247+0000","last_unstale":"2024-09-10T10:46:44.900588+0000","last_undegraded":"2024-09-10T10:46:44.900588+0000","last_fullsized":"2024-09-10T10:46:44.900588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:00.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887341+0000","last_change":"2024-09-10T10:46:44.877202+0000","last_active":"2024-09-10T10:46:44.887341+0000","last_peered":"2024-09-10T10:46:44.887341+0000","last_clean":"2024-09-10T10:46:44.887341+0000","last_became_active":"2024-09-10T10:46:44.876159+0000","last_became_peered":"2024-09-10T10:46:44.876159+0000","last_unstale":"2024-09-10T10:46:44.887341+0000","last_undegraded":"2024-09-10T10:46:44.887341+0000","last_fullsized":"2024-09-10T10:46:44.887341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:01.241200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851870+0000","last_change":"2024-09-10T10:46:42.851870+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851870+0000","last_undegraded":"2024-09-10T10:46:42.851870+0000","last_fullsized":"2024-09-10T10:46:42.851870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.880924+0000","last_change":"2024-09-10T10:46:44.878110+0000","last_active":"2024-09-10T10:46:44.880924+0000","last_peered":"2024-09-10T10:46:44.880924+0000","last_clean":"2024-09-10T10:46:44.880924+0000","last_became_active":"2024-09-10T10:46:44.877841+0000","last_became_peered":"2024-09-10T10:46:44.877841+0000","last_unstale":"2024-09-10T10:46:44.880924+0000","last_undegraded":"2024-09-10T10:46:44.880924+0000","last_fullsized":"2024-09-10T10:46:44.880924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:45.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898049+0000","last_change":"2024-09-10T10:46:43.875159+0000","last_active":"2024-09-10T10:46:44.898049+0000","last_peered":"2024-09-10T10:46:44.898049+0000","last_clean":"2024-09-10T10:46:44.898049+0000","last_became_active":"2024-09-10T10:46:43.874602+0000","last_became_peered":"2024-09-10T10:46:43.874602+0000","last_unstale":"2024-09-10T10:46:44.898049+0000","last_undegraded":"2024-09-10T10:46:44.898049+0000","last_fullsized":"2024-09-10T10:46:44.898049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:48.767108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889322+0000","last_change":"2024-09-10T10:46:44.888134+0000","last_active":"2024-09-10T10:46:44.889322+0000","last_peered":"2024-09-10T10:46:44.889322+0000","last_clean":"2024-09-10T10:46:44.889322+0000","last_became_active":"2024-09-10T10:46:44.887851+0000","last_became_peered":"2024-09-10T10:46:44.887851+0000","last_unstale":"2024-09-10T10:46:44.889322+0000","last_undegraded":"2024-09-10T10:46:44.889322+0000","last_fullsized":"2024-09-10T10:46:44.889322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849833+0000","last_change":"2024-09-10T10:46:42.849833+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849833+0000","last_undegraded":"2024-09-10T10:46:42.849833+0000","last_fullsized":"2024-09-10T10:46:42.849833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.896764+0000","last_change":"2024-09-10T10:46:44.893318+0000","last_active":"2024-09-10T10:46:44.896764+0000","last_peered":"2024-09-10T10:46:44.896764+0000","last_clean":"2024-09-10T10:46:44.896764+0000","last_became_active":"2024-09-10T10:46:44.893039+0000","last_became_peered":"2024-09-10T10:46:44.893039+0000","last_unstale":"2024-09-10T10:46:44.896764+0000","last_undegraded":"2024-09-10T10:46:44.896764+0000","last_fullsized":"2024-09-10T10:46:44.896764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870255+0000","last_change":"2024-09-10T10:46:43.879425+0000","last_active":"2024-09-10T10:46:44.870255+0000","last_peered":"2024-09-10T10:46:44.870255+0000","last_clean":"2024-09-10T10:46:44.870255+0000","last_became_active":"2024-09-10T10:46:43.879154+0000","last_became_peered":"2024-09-10T10:46:43.879154+0000","last_unstale":"2024-09-10T10:46:44.870255+0000","last_undegraded":"2024-09-10T10:46:44.870255+0000","last_fullsized":"2024-09-10T10:46:44.870255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:41.361020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879467+0000","last_change":"2024-09-10T10:46:44.871663+0000","last_active":"2024-09-10T10:46:44.879467+0000","last_peered":"2024-09-10T10:46:44.879467+0000","last_clean":"2024-09-10T10:46:44.879467+0000","last_became_active":"2024-09-10T10:46:44.869465+0000","last_became_peered":"2024-09-10T10:46:44.869465+0000","last_unstale":"2024-09-10T10:46:44.879467+0000","last_undegraded":"2024-09-10T10:46:44.879467+0000","last_fullsized":"2024-09-10T10:46:44.879467+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:50.834542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.857464+0000","last_change":"2024-09-10T10:46:43.870161+0000","last_active":"2024-09-10T10:46:44.857464+0000","last_peered":"2024-09-10T10:46:44.857464+0000","last_clean":"2024-09-10T10:46:44.857464+0000","last_became_active":"2024-09-10T10:46:43.869918+0000","last_became_peered":"2024-09-10T10:46:43.869918+0000","last_unstale":"2024-09-10T10:46:44.857464+0000","last_undegraded":"2024-09-10T10:46:44.857464+0000","last_fullsized":"2024-09-10T10:46:44.857464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:11.275462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887228+0000","last_change":"2024-09-10T10:46:44.884495+0000","last_active":"2024-09-10T10:46:44.887228+0000","last_peered":"2024-09-10T10:46:44.887228+0000","last_clean":"2024-09-10T10:46:44.887228+0000","last_became_active":"2024-09-10T10:46:44.884231+0000","last_became_peered":"2024-09-10T10:46:44.884231+0000","last_unstale":"2024-09-10T10:46:44.887228+0000","last_undegraded":"2024-09-10T10:46:44.887228+0000","last_fullsized":"2024-09-10T10:46:44.887228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.858268+0000","last_change":"2024-09-10T10:46:43.867286+0000","last_active":"2024-09-10T10:46:44.858268+0000","last_peered":"2024-09-10T10:46:44.858268+0000","last_clean":"2024-09-10T10:46:44.858268+0000","last_became_active":"2024-09-10T10:46:43.867053+0000","last_became_peered":"2024-09-10T10:46:43.867053+0000","last_unstale":"2024-09-10T10:46:44.858268+0000","last_undegraded":"2024-09-10T10:46:44.858268+0000","last_fullsized":"2024-09-10T10:46:44.858268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.574114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881233+0000","last_change":"2024-09-10T10:46:44.870275+0000","last_active":"2024-09-10T10:46:44.881233+0000","last_peered":"2024-09-10T10:46:44.881233+0000","last_clean":"2024-09-10T10:46:44.881233+0000","last_became_active":"2024-09-10T10:46:44.869919+0000","last_became_peered":"2024-09-10T10:46:44.869919+0000","last_unstale":"2024-09-10T10:46:44.881233+0000","last_undegraded":"2024-09-10T10:46:44.881233+0000","last_fullsized":"2024-09-10T10:46:44.881233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:32.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867247+0000","last_change":"2024-09-10T10:46:43.881828+0000","last_active":"2024-09-10T10:46:44.867247+0000","last_peered":"2024-09-10T10:46:44.867247+0000","last_clean":"2024-09-10T10:46:44.867247+0000","last_became_active":"2024-09-10T10:46:43.881524+0000","last_became_peered":"2024-09-10T10:46:43.881524+0000","last_unstale":"2024-09-10T10:46:44.867247+0000","last_undegraded":"2024-09-10T10:46:44.867247+0000","last_fullsized":"2024-09-10T10:46:44.867247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:18.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890526+0000","last_change":"2024-09-10T10:46:44.877148+0000","last_active":"2024-09-10T10:46:44.890526+0000","last_peered":"2024-09-10T10:46:44.890526+0000","last_clean":"2024-09-10T10:46:44.890526+0000","last_became_active":"2024-09-10T10:46:44.876860+0000","last_became_peered":"2024-09-10T10:46:44.876860+0000","last_unstale":"2024-09-10T10:46:44.890526+0000","last_undegraded":"2024-09-10T10:46:44.890526+0000","last_fullsized":"2024-09-10T10:46:44.890526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.904206+0000","last_change":"2024-09-10T10:46:43.874911+0000","last_active":"2024-09-10T10:46:44.904206+0000","last_peered":"2024-09-10T10:46:44.904206+0000","last_clean":"2024-09-10T10:46:44.904206+0000","last_became_active":"2024-09-10T10:46:43.874629+0000","last_became_peered":"2024-09-10T10:46:43.874629+0000","last_unstale":"2024-09-10T10:46:44.904206+0000","last_undegraded":"2024-09-10T10:46:44.904206+0000","last_fullsized":"2024-09-10T10:46:44.904206+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:31.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893676+0000","last_change":"2024-09-10T10:46:44.889982+0000","last_active":"2024-09-10T10:46:44.893676+0000","last_peered":"2024-09-10T10:46:44.893676+0000","last_clean":"2024-09-10T10:46:44.893676+0000","last_became_active":"2024-09-10T10:46:44.889481+0000","last_became_peered":"2024-09-10T10:46:44.889481+0000","last_unstale":"2024-09-10T10:46:44.893676+0000","last_undegraded":"2024-09-10T10:46:44.893676+0000","last_fullsized":"2024-09-10T10:46:44.893676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:42.436804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849881+0000","last_change":"2024-09-10T10:46:42.849881+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849881+0000","last_undegraded":"2024-09-10T10:46:42.849881+0000","last_fullsized":"2024-09-10T10:46:42.849881+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894670+0000","last_change":"2024-09-10T10:46:44.886563+0000","last_active":"2024-09-10T10:46:44.894670+0000","last_peered":"2024-09-10T10:46:44.894670+0000","last_clean":"2024-09-10T10:46:44.894670+0000","last_became_active":"2024-09-10T10:46:44.886299+0000","last_became_peered":"2024-09-10T10:46:44.886299+0000","last_unstale":"2024-09-10T10:46:44.894670+0000","last_undegraded":"2024-09-10T10:46:44.894670+0000","last_fullsized":"2024-09-10T10:46:44.894670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897647+0000","last_change":"2024-09-10T10:46:43.868983+0000","last_active":"2024-09-10T10:46:44.897647+0000","last_peered":"2024-09-10T10:46:44.897647+0000","last_clean":"2024-09-10T10:46:44.897647+0000","last_became_active":"2024-09-10T10:46:43.868631+0000","last_became_peered":"2024-09-10T10:46:43.868631+0000","last_unstale":"2024-09-10T10:46:44.897647+0000","last_undegraded":"2024-09-10T10:46:44.897647+0000","last_fullsized":"2024-09-10T10:46:44.897647+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892284+0000","last_change":"2024-09-10T10:46:44.885879+0000","last_active":"2024-09-10T10:46:44.892284+0000","last_peered":"2024-09-10T10:46:44.892284+0000","last_clean":"2024-09-10T10:46:44.892284+0000","last_became_active":"2024-09-10T10:46:44.885500+0000","last_became_peered":"2024-09-10T10:46:44.885500+0000","last_unstale":"2024-09-10T10:46:44.892284+0000","last_undegraded":"2024-09-10T10:46:44.892284+0000","last_fullsized":"2024-09-10T10:46:44.892284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874282+0000","last_change":"2024-09-10T10:46:43.879565+0000","last_active":"2024-09-10T10:46:44.874282+0000","last_peered":"2024-09-10T10:46:44.874282+0000","last_clean":"2024-09-10T10:46:44.874282+0000","last_became_active":"2024-09-10T10:46:43.879254+0000","last_became_peered":"2024-09-10T10:46:43.879254+0000","last_unstale":"2024-09-10T10:46:44.874282+0000","last_undegraded":"2024-09-10T10:46:44.874282+0000","last_fullsized":"2024-09-10T10:46:44.874282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881138+0000","last_change":"2024-09-10T10:46:44.878593+0000","last_active":"2024-09-10T10:46:44.881138+0000","last_peered":"2024-09-10T10:46:44.881138+0000","last_clean":"2024-09-10T10:46:44.881138+0000","last_became_active":"2024-09-10T10:46:44.878283+0000","last_became_peered":"2024-09-10T10:46:44.878283+0000","last_unstale":"2024-09-10T10:46:44.881138+0000","last_undegraded":"2024-09-10T10:46:44.881138+0000","last_fullsized":"2024-09-10T10:46:44.881138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.865622+0000","last_change":"2024-09-10T10:46:43.877389+0000","last_active":"2024-09-10T10:46:44.865622+0000","last_peered":"2024-09-10T10:46:44.865622+0000","last_clean":"2024-09-10T10:46:44.865622+0000","last_became_active":"2024-09-10T10:46:43.876235+0000","last_became_peered":"2024-09-10T10:46:43.876235+0000","last_unstale":"2024-09-10T10:46:44.865622+0000","last_undegraded":"2024-09-10T10:46:44.865622+0000","last_fullsized":"2024-09-10T10:46:44.865622+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:52.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881110+0000","last_change":"2024-09-10T10:46:44.878187+0000","last_active":"2024-09-10T10:46:44.881110+0000","last_peered":"2024-09-10T10:46:44.881110+0000","last_clean":"2024-09-10T10:46:44.881110+0000","last_became_active":"2024-09-10T10:46:44.877922+0000","last_became_peered":"2024-09-10T10:46:44.877922+0000","last_unstale":"2024-09-10T10:46:44.881110+0000","last_undegraded":"2024-09-10T10:46:44.881110+0000","last_fullsized":"2024-09-10T10:46:44.881110+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.902316+0000","last_change":"2024-09-10T10:46:43.879721+0000","last_active":"2024-09-10T10:46:44.902316+0000","last_peered":"2024-09-10T10:46:44.902316+0000","last_clean":"2024-09-10T10:46:44.902316+0000","last_became_active":"2024-09-10T10:46:43.879448+0000","last_became_peered":"2024-09-10T10:46:43.879448+0000","last_unstale":"2024-09-10T10:46:44.902316+0000","last_undegraded":"2024-09-10T10:46:44.902316+0000","last_fullsized":"2024-09-10T10:46:44.902316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882849+0000","last_change":"2024-09-10T10:46:44.871707+0000","last_active":"2024-09-10T10:46:44.882849+0000","last_peered":"2024-09-10T10:46:44.882849+0000","last_clean":"2024-09-10T10:46:44.882849+0000","last_became_active":"2024-09-10T10:46:44.871400+0000","last_became_peered":"2024-09-10T10:46:44.871400+0000","last_unstale":"2024-09-10T10:46:44.882849+0000","last_undegraded":"2024-09-10T10:46:44.882849+0000","last_fullsized":"2024-09-10T10:46:44.882849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898284+0000","last_change":"2024-09-10T10:46:43.866850+0000","last_active":"2024-09-10T10:46:44.898284+0000","last_peered":"2024-09-10T10:46:44.898284+0000","last_clean":"2024-09-10T10:46:44.898284+0000","last_became_active":"2024-09-10T10:46:43.866575+0000","last_became_peered":"2024-09-10T10:46:43.866575+0000","last_unstale":"2024-09-10T10:46:44.898284+0000","last_undegraded":"2024-09-10T10:46:44.898284+0000","last_fullsized":"2024-09-10T10:46:44.898284+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:07.913685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890441+0000","last_change":"2024-09-10T10:46:44.875025+0000","last_active":"2024-09-10T10:46:44.890441+0000","last_peered":"2024-09-10T10:46:44.890441+0000","last_clean":"2024-09-10T10:46:44.890441+0000","last_became_active":"2024-09-10T10:46:44.873173+0000","last_became_peered":"2024-09-10T10:46:44.873173+0000","last_unstale":"2024-09-10T10:46:44.890441+0000","last_undegraded":"2024-09-10T10:46:44.890441+0000","last_fullsized":"2024-09-10T10:46:44.890441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903260+0000","last_change":"2024-09-10T10:46:43.871807+0000","last_active":"2024-09-10T10:46:44.903260+0000","last_peered":"2024-09-10T10:46:44.903260+0000","last_clean":"2024-09-10T10:46:44.903260+0000","last_became_active":"2024-09-10T10:46:43.871540+0000","last_became_peered":"2024-09-10T10:46:43.871540+0000","last_unstale":"2024-09-10T10:46:44.903260+0000","last_undegraded":"2024-09-10T10:46:44.903260+0000","last_fullsized":"2024-09-10T10:46:44.903260+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:34.555709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894618+0000","last_change":"2024-09-10T10:46:44.888440+0000","last_active":"2024-09-10T10:46:44.894618+0000","last_peered":"2024-09-10T10:46:44.894618+0000","last_clean":"2024-09-10T10:46:44.894618+0000","last_became_active":"2024-09-10T10:46:44.888154+0000","last_became_peered":"2024-09-10T10:46:44.888154+0000","last_unstale":"2024-09-10T10:46:44.894618+0000","last_undegraded":"2024-09-10T10:46:44.894618+0000","last_fullsized":"2024-09-10T10:46:44.894618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.905983+0000","last_change":"2024-09-10T10:46:43.876691+0000","last_active":"2024-09-10T10:46:44.905983+0000","last_peered":"2024-09-10T10:46:44.905983+0000","last_clean":"2024-09-10T10:46:44.905983+0000","last_became_active":"2024-09-10T10:46:43.876227+0000","last_became_peered":"2024-09-10T10:46:43.876227+0000","last_unstale":"2024-09-10T10:46:44.905983+0000","last_undegraded":"2024-09-10T10:46:44.905983+0000","last_fullsized":"2024-09-10T10:46:44.905983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:31.964863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892464+0000","last_change":"2024-09-10T10:46:44.882771+0000","last_active":"2024-09-10T10:46:44.892464+0000","last_peered":"2024-09-10T10:46:44.892464+0000","last_clean":"2024-09-10T10:46:44.892464+0000","last_became_active":"2024-09-10T10:46:44.881897+0000","last_became_peered":"2024-09-10T10:46:44.881897+0000","last_unstale":"2024-09-10T10:46:44.892464+0000","last_undegraded":"2024-09-10T10:46:44.892464+0000","last_fullsized":"2024-09-10T10:46:44.892464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:22.411021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867582+0000","last_change":"2024-09-10T10:46:43.873156+0000","last_active":"2024-09-10T10:46:44.867582+0000","last_peered":"2024-09-10T10:46:44.867582+0000","last_clean":"2024-09-10T10:46:44.867582+0000","last_became_active":"2024-09-10T10:46:43.872889+0000","last_became_peered":"2024-09-10T10:46:43.872889+0000","last_unstale":"2024-09-10T10:46:44.867582+0000","last_undegraded":"2024-09-10T10:46:44.867582+0000","last_fullsized":"2024-09-10T10:46:44.867582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852599+0000","last_change":"2024-09-10T10:46:42.852599+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852599+0000","last_undegraded":"2024-09-10T10:46:42.852599+0000","last_fullsized":"2024-09-10T10:46:42.852599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872927+0000","last_change":"2024-09-10T10:46:44.868747+0000","last_active":"2024-09-10T10:46:44.872927+0000","last_peered":"2024-09-10T10:46:44.872927+0000","last_clean":"2024-09-10T10:46:44.872927+0000","last_became_active":"2024-09-10T10:46:44.868455+0000","last_became_peered":"2024-09-10T10:46:44.868455+0000","last_unstale":"2024-09-10T10:46:44.872927+0000","last_undegraded":"2024-09-10T10:46:44.872927+0000","last_fullsized":"2024-09-10T10:46:44.872927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874479+0000","last_change":"2024-09-10T10:46:43.876827+0000","last_active":"2024-09-10T10:46:44.874479+0000","last_peered":"2024-09-10T10:46:44.874479+0000","last_clean":"2024-09-10T10:46:44.874479+0000","last_became_active":"2024-09-10T10:46:43.875591+0000","last_became_peered":"2024-09-10T10:46:43.875591+0000","last_unstale":"2024-09-10T10:46:44.874479+0000","last_undegraded":"2024-09-10T10:46:44.874479+0000","last_fullsized":"2024-09-10T10:46:44.874479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889806+0000","last_change":"2024-09-10T10:46:44.883361+0000","last_active":"2024-09-10T10:46:44.889806+0000","last_peered":"2024-09-10T10:46:44.889806+0000","last_clean":"2024-09-10T10:46:44.889806+0000","last_became_active":"2024-09-10T10:46:44.879641+0000","last_became_peered":"2024-09-10T10:46:44.879641+0000","last_unstale":"2024-09-10T10:46:44.889806+0000","last_undegraded":"2024-09-10T10:46:44.889806+0000","last_fullsized":"2024-09-10T10:46:44.889806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849801+0000","last_change":"2024-09-10T10:46:42.849801+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849801+0000","last_undegraded":"2024-09-10T10:46:42.849801+0000","last_fullsized":"2024-09-10T10:46:42.849801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879741+0000","last_change":"2024-09-10T10:46:44.873964+0000","last_active":"2024-09-10T10:46:44.879741+0000","last_peered":"2024-09-10T10:46:44.879741+0000","last_clean":"2024-09-10T10:46:44.879741+0000","last_became_active":"2024-09-10T10:46:44.873324+0000","last_became_peered":"2024-09-10T10:46:44.873324+0000","last_unstale":"2024-09-10T10:46:44.879741+0000","last_undegraded":"2024-09-10T10:46:44.879741+0000","last_fullsized":"2024-09-10T10:46:44.879741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:46.801922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897908+0000","last_change":"2024-09-10T10:46:43.880123+0000","last_active":"2024-09-10T10:46:44.897908+0000","last_peered":"2024-09-10T10:46:44.897908+0000","last_clean":"2024-09-10T10:46:44.897908+0000","last_became_active":"2024-09-10T10:46:43.879861+0000","last_became_peered":"2024-09-10T10:46:43.879861+0000","last_unstale":"2024-09-10T10:46:44.897908+0000","last_undegraded":"2024-09-10T10:46:44.897908+0000","last_fullsized":"2024-09-10T10:46:44.897908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:40.170023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881256+0000","last_change":"2024-09-10T10:46:44.875117+0000","last_active":"2024-09-10T10:46:44.881256+0000","last_peered":"2024-09-10T10:46:44.881256+0000","last_clean":"2024-09-10T10:46:44.881256+0000","last_became_active":"2024-09-10T10:46:44.874858+0000","last_became_peered":"2024-09-10T10:46:44.874858+0000","last_unstale":"2024-09-10T10:46:44.881256+0000","last_undegraded":"2024-09-10T10:46:44.881256+0000","last_fullsized":"2024-09-10T10:46:44.881256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:08.230635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.859204+0000","last_change":"2024-09-10T10:46:43.870732+0000","last_active":"2024-09-10T10:46:44.859204+0000","last_peered":"2024-09-10T10:46:44.859204+0000","last_clean":"2024-09-10T10:46:44.859204+0000","last_became_active":"2024-09-10T10:46:43.870484+0000","last_became_peered":"2024-09-10T10:46:43.870484+0000","last_unstale":"2024-09-10T10:46:44.859204+0000","last_undegraded":"2024-09-10T10:46:44.859204+0000","last_fullsized":"2024-09-10T10:46:44.859204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:05.738817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881485+0000","last_change":"2024-09-10T10:46:44.875783+0000","last_active":"2024-09-10T10:46:44.881485+0000","last_peered":"2024-09-10T10:46:44.881485+0000","last_clean":"2024-09-10T10:46:44.881485+0000","last_became_active":"2024-09-10T10:46:44.875511+0000","last_became_peered":"2024-09-10T10:46:44.875511+0000","last_unstale":"2024-09-10T10:46:44.881485+0000","last_undegraded":"2024-09-10T10:46:44.881485+0000","last_fullsized":"2024-09-10T10:46:44.881485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:37.571008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852053+0000","last_change":"2024-09-10T10:46:42.852053+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852053+0000","last_undegraded":"2024-09-10T10:46:42.852053+0000","last_fullsized":"2024-09-10T10:46:42.852053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892438+0000","last_change":"2024-09-10T10:46:44.882540+0000","last_active":"2024-09-10T10:46:44.892438+0000","last_peered":"2024-09-10T10:46:44.892438+0000","last_clean":"2024-09-10T10:46:44.892438+0000","last_became_active":"2024-09-10T10:46:44.881322+0000","last_became_peered":"2024-09-10T10:46:44.881322+0000","last_unstale":"2024-09-10T10:46:44.892438+0000","last_undegraded":"2024-09-10T10:46:44.892438+0000","last_fullsized":"2024-09-10T10:46:44.892438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:05.028664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867719+0000","last_change":"2024-09-10T10:46:43.880848+0000","last_active":"2024-09-10T10:46:44.867719+0000","last_peered":"2024-09-10T10:46:44.867719+0000","last_clean":"2024-09-10T10:46:44.867719+0000","last_became_active":"2024-09-10T10:46:43.880588+0000","last_became_peered":"2024-09-10T10:46:43.880588+0000","last_unstale":"2024-09-10T10:46:44.867719+0000","last_undegraded":"2024-09-10T10:46:44.867719+0000","last_fullsized":"2024-09-10T10:46:44.867719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887215+0000","last_change":"2024-09-10T10:46:44.879311+0000","last_active":"2024-09-10T10:46:44.887215+0000","last_peered":"2024-09-10T10:46:44.887215+0000","last_clean":"2024-09-10T10:46:44.887215+0000","last_became_active":"2024-09-10T10:46:44.867480+0000","last_became_peered":"2024-09-10T10:46:44.867480+0000","last_unstale":"2024-09-10T10:46:44.887215+0000","last_undegraded":"2024-09-10T10:46:44.887215+0000","last_fullsized":"2024-09-10T10:46:44.887215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.866776+0000","last_change":"2024-09-10T10:46:43.878330+0000","last_active":"2024-09-10T10:46:44.866776+0000","last_peered":"2024-09-10T10:46:44.866776+0000","last_clean":"2024-09-10T10:46:44.866776+0000","last_became_active":"2024-09-10T10:46:43.877459+0000","last_became_peered":"2024-09-10T10:46:43.877459+0000","last_unstale":"2024-09-10T10:46:44.866776+0000","last_undegraded":"2024-09-10T10:46:44.866776+0000","last_fullsized":"2024-09-10T10:46:44.866776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:13.837013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872902+0000","last_change":"2024-09-10T10:46:44.868549+0000","last_active":"2024-09-10T10:46:44.872902+0000","last_peered":"2024-09-10T10:46:44.872902+0000","last_clean":"2024-09-10T10:46:44.872902+0000","last_became_active":"2024-09-10T10:46:44.868232+0000","last_became_peered":"2024-09-10T10:46:44.868232+0000","last_unstale":"2024-09-10T10:46:44.872902+0000","last_undegraded":"2024-09-10T10:46:44.872902+0000","last_fullsized":"2024-09-10T10:46:44.872902+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870340+0000","last_change":"2024-09-10T10:46:43.879611+0000","last_active":"2024-09-10T10:46:44.870340+0000","last_peered":"2024-09-10T10:46:44.870340+0000","last_clean":"2024-09-10T10:46:44.870340+0000","last_became_active":"2024-09-10T10:46:43.879308+0000","last_became_peered":"2024-09-10T10:46:43.879308+0000","last_unstale":"2024-09-10T10:46:44.870340+0000","last_undegraded":"2024-09-10T10:46:44.870340+0000","last_fullsized":"2024-09-10T10:46:44.870340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:57.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867820+0000","last_change":"2024-09-10T10:46:43.879656+0000","last_active":"2024-09-10T10:46:44.867820+0000","last_peered":"2024-09-10T10:46:44.867820+0000","last_clean":"2024-09-10T10:46:44.867820+0000","last_became_active":"2024-09-10T10:46:43.879098+0000","last_became_peered":"2024-09-10T10:46:43.879098+0000","last_unstale":"2024-09-10T10:46:44.867820+0000","last_undegraded":"2024-09-10T10:46:44.867820+0000","last_fullsized":"2024-09-10T10:46:44.867820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879442+0000","last_change":"2024-09-10T10:46:44.869395+0000","last_active":"2024-09-10T10:46:44.879442+0000","last_peered":"2024-09-10T10:46:44.879442+0000","last_clean":"2024-09-10T10:46:44.879442+0000","last_became_active":"2024-09-10T10:46:44.869065+0000","last_became_peered":"2024-09-10T10:46:44.869065+0000","last_unstale":"2024-09-10T10:46:44.879442+0000","last_undegraded":"2024-09-10T10:46:44.879442+0000","last_fullsized":"2024-09-10T10:46:44.879442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:52.312041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851961+0000","last_change":"2024-09-10T10:46:42.851961+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851961+0000","last_undegraded":"2024-09-10T10:46:42.851961+0000","last_fullsized":"2024-09-10T10:46:42.851961+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891869+0000","last_change":"2024-09-10T10:46:44.885648+0000","last_active":"2024-09-10T10:46:44.891869+0000","last_peered":"2024-09-10T10:46:44.891869+0000","last_clean":"2024-09-10T10:46:44.891869+0000","last_became_active":"2024-09-10T10:46:44.885219+0000","last_became_peered":"2024-09-10T10:46:44.885219+0000","last_unstale":"2024-09-10T10:46:44.891869+0000","last_undegraded":"2024-09-10T10:46:44.891869+0000","last_fullsized":"2024-09-10T10:46:44.891869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:19.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.850800+0000","last_change":"2024-09-10T10:46:42.850800+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.850800+0000","last_undegraded":"2024-09-10T10:46:42.850800+0000","last_fullsized":"2024-09-10T10:46:42.850800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":257,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.871099+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:44.871099+0000","last_peered":"2024-09-10T10:46:44.871099+0000","last_clean":"2024-09-10T10:46:44.871099+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:44.871099+0000","last_undegraded":"2024-09-10T10:46:44.871099+0000","last_fullsized":"2024-09-10T10:46:44.871099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":367029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":374778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":374356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":835584,"data_stored":374252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":964598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1396736,"data_stored":950730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:49.495 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:49.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.687+0000 7f232d000640 1 Processor -- start 2024-09-10T10:46:49.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.687+0000 7f232d000640 1 -- start start 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f232d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 0x7f23280d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f232d000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23280d0980 con 0x7f23280d1b70 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f232d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23280d0c60 con 0x7f232815e5d0 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f232d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f232815e9b0 con 0x7f232806c320 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f2326a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f23280d1b70 0x7f232806bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56722/0 (socket says 172.21.15.28:56722) 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.688+0000 7f2326a00640 1 -- 172.21.15.28:0/180588412 learned_addr learned my addr 172.21.15.28:0/180588412 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:49.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2327e00640 1 --2- 172.21.15.28:0/180588412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 0x7f23280d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4046123161 0 0) 0x7f23280d0980 con 0x7f23280d1b70 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f230c003520 con 0x7f23280d1b70 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954458459 0 0) 0x7f23280d0c60 con 0x7f232815e5d0 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23280d0980 con 0x7f232815e5d0 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1654924207 0 0) 0x7f230c003520 con 0x7f23280d1b70 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2327e00640 1 -- 172.21.15.28:0/180588412 >> v1:172.21.15.138:6789/0 conn(0x7f232815e5d0 legacy=0x7f2328073780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2327e00640 1 -- 172.21.15.28:0/180588412 >> v1:172.21.15.28:6789/0 conn(0x7f23280d1b70 legacy=0x7f232806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.689+0000 7f2327e00640 1 -- 172.21.15.28:0/180588412 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2328072f80 con 0x7f232806c320 2024-09-10T10:46:49.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.690+0000 7f2327e00640 1 --2- 172.21.15.28:0/180588412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 0x7f23280d0310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2320009dc0 tx=0x7f232002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c90c776583a24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.691+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314001670 con 0x7f232806c320 2024-09-10T10:46:49.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.691+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2320007540 con 0x7f232806c320 2024-09-10T10:46:49.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.691+0000 7f2326000640 1 -- 172.21.15.28:0/180588412 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320007910 con 0x7f232806c320 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 -- 172.21.15.28:0/180588412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 msgr2=0x7f23280d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 --2- 172.21.15.28:0/180588412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 0x7f23280d0310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2320009dc0 tx=0x7f232002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 -- 172.21.15.28:0/180588412 shutdown_connections 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 --2- 172.21.15.28:0/180588412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232806c320 0x7f23280d0310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 -- 172.21.15.28:0/180588412 >> 172.21.15.28:0/180588412 conn(0x7f23280d1080 msgr2=0x7f23280d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 -- 172.21.15.28:0/180588412 shutdown_connections 2024-09-10T10:46:49.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.692+0000 7f232d000640 1 -- 172.21.15.28:0/180588412 wait complete. 2024-09-10T10:46:49.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.693+0000 7f232d000640 1 Processor -- start 2024-09-10T10:46:49.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 -- start start 2024-09-10T10:46:49.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 0x7f232815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 0x7f23280ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f232806cb60 con 0x7f23280d1b70 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f232806c9e0 con 0x7f232806c320 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.694+0000 7f232d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2328162030 con 0x7f232815e5d0 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50122/0 (socket says 172.21.15.28:50122) 2024-09-10T10:46:49.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 -- 172.21.15.28:0/3174719749 learned_addr learned my addr 172.21.15.28:0/3174719749 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2327e00640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 0x7f23280ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2327400640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 0x7f232815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 -- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 msgr2=0x7f23280ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 0x7f23280ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 -- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 msgr2=0x7f232815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 0x7f232815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 -- 172.21.15.28:0/3174719749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232815b740 con 0x7f23280d1b70 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2327400640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 0x7f232815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2327e00640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 0x7f23280ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.695+0000 7f2326a00640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2314000fd0 tx=0x7f231400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.696+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314017070 con 0x7f23280d1b70 2024-09-10T10:46:49.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.696+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2314005ad0 con 0x7f23280d1b70 2024-09-10T10:46:49.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.696+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2314007070 con 0x7f23280d1b70 2024-09-10T10:46:49.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.696+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2328156040 con 0x7f23280d1b70 2024-09-10T10:46:49.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.696+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232815bcc0 con 0x7f23280d1b70 2024-09-10T10:46:49.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.697+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23280d0310 con 0x7f23280d1b70 2024-09-10T10:46:49.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.698+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f231401a070 con 0x7f23280d1b70 2024-09-10T10:46:49.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.699+0000 7f231f400640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 0x7f230806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:49.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.699+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f231408a550 con 0x7f23280d1b70 2024-09-10T10:46:49.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.699+0000 7f2327400640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 0x7f230806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:49.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.700+0000 7f2327400640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 0x7f230806e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23180011e0 tx=0x7f2318009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:49.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.703+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2314058d60 con 0x7f23280d1b70 2024-09-10T10:46:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.885+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2328073ec0 con 0x7f230806be30 2024-09-10T10:46:49.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.892+0000 7f231f400640 1 -- 172.21.15.28:0/3174719749 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250382 (secure 0 0 0) 0x7f2328073ec0 con 0x7f230806be30 2024-09-10T10:46:49.896 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:49.897 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 msgr2=0x7f230806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 0x7f230806e2f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23180011e0 tx=0x7f2318009040 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 msgr2=0x7f2328155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2314000fd0 tx=0x7f231400cbe0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 shutdown_connections 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f230806be30 0x7f230806e2f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f232815e5d0 0x7f23280ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23280d1b70 0x7f2328155b00 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 --2- 172.21.15.28:0/3174719749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f232806c320 0x7f232815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 >> 172.21.15.28:0/3174719749 conn(0x7f23280d1080 msgr2=0x7f23280adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:49.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 shutdown_connections 2024-09-10T10:46:49.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:49.896+0000 7f232d000640 1 -- 172.21.15.28:0/3174719749 wait complete. 2024-09-10T10:46:49.913 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T10:46:48.526234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":182,"acting":182,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244664,"kb_used_data":7612,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730056,"statfs":{"total":773094113280,"available":772843577344,"internally_reserved":0,"allocated":7794688,"data_stored":4154135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.680947"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895722+0000","last_change":"2024-09-10T10:46:44.892310+0000","last_active":"2024-09-10T10:46:44.895722+0000","last_peered":"2024-09-10T10:46:44.895722+0000","last_clean":"2024-09-10T10:46:44.895722+0000","last_became_active":"2024-09-10T10:46:44.892049+0000","last_became_peered":"2024-09-10T10:46:44.892049+0000","last_unstale":"2024-09-10T10:46:44.895722+0000","last_undegraded":"2024-09-10T10:46:44.895722+0000","last_fullsized":"2024-09-10T10:46:44.895722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892585+0000","last_change":"2024-09-10T10:46:44.884268+0000","last_active":"2024-09-10T10:46:44.892585+0000","last_peered":"2024-09-10T10:46:44.892585+0000","last_clean":"2024-09-10T10:46:44.892585+0000","last_became_active":"2024-09-10T10:46:44.883101+0000","last_became_peered":"2024-09-10T10:46:44.883101+0000","last_unstale":"2024-09-10T10:46:44.892585+0000","last_undegraded":"2024-09-10T10:46:44.892585+0000","last_fullsized":"2024-09-10T10:46:44.892585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:14.573857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882869+0000","last_change":"2024-09-10T10:46:44.875622+0000","last_active":"2024-09-10T10:46:44.882869+0000","last_peered":"2024-09-10T10:46:44.882869+0000","last_clean":"2024-09-10T10:46:44.882869+0000","last_became_active":"2024-09-10T10:46:44.875330+0000","last_became_peered":"2024-09-10T10:46:44.875330+0000","last_unstale":"2024-09-10T10:46:44.882869+0000","last_undegraded":"2024-09-10T10:46:44.882869+0000","last_fullsized":"2024-09-10T10:46:44.882869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892693+0000","last_change":"2024-09-10T10:46:44.884914+0000","last_active":"2024-09-10T10:46:44.892693+0000","last_peered":"2024-09-10T10:46:44.892693+0000","last_clean":"2024-09-10T10:46:44.892693+0000","last_became_active":"2024-09-10T10:46:44.884638+0000","last_became_peered":"2024-09-10T10:46:44.884638+0000","last_unstale":"2024-09-10T10:46:44.892693+0000","last_undegraded":"2024-09-10T10:46:44.892693+0000","last_fullsized":"2024-09-10T10:46:44.892693+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:20.876408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.884835+0000","last_change":"2024-09-10T10:46:44.883459+0000","last_active":"2024-09-10T10:46:44.884835+0000","last_peered":"2024-09-10T10:46:44.884835+0000","last_clean":"2024-09-10T10:46:44.884835+0000","last_became_active":"2024-09-10T10:46:44.883173+0000","last_became_peered":"2024-09-10T10:46:44.883173+0000","last_unstale":"2024-09-10T10:46:44.884835+0000","last_undegraded":"2024-09-10T10:46:44.884835+0000","last_fullsized":"2024-09-10T10:46:44.884835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893848+0000","last_change":"2024-09-10T10:46:44.890220+0000","last_active":"2024-09-10T10:46:44.893848+0000","last_peered":"2024-09-10T10:46:44.893848+0000","last_clean":"2024-09-10T10:46:44.893848+0000","last_became_active":"2024-09-10T10:46:44.889918+0000","last_became_peered":"2024-09-10T10:46:44.889918+0000","last_unstale":"2024-09-10T10:46:44.893848+0000","last_undegraded":"2024-09-10T10:46:44.893848+0000","last_fullsized":"2024-09-10T10:46:44.893848+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:58.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887361+0000","last_change":"2024-09-10T10:46:44.877805+0000","last_active":"2024-09-10T10:46:44.887361+0000","last_peered":"2024-09-10T10:46:44.887361+0000","last_clean":"2024-09-10T10:46:44.887361+0000","last_became_active":"2024-09-10T10:46:44.875492+0000","last_became_peered":"2024-09-10T10:46:44.875492+0000","last_unstale":"2024-09-10T10:46:44.887361+0000","last_undegraded":"2024-09-10T10:46:44.887361+0000","last_fullsized":"2024-09-10T10:46:44.887361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:42.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895164+0000","last_change":"2024-09-10T10:46:44.886608+0000","last_active":"2024-09-10T10:46:44.895164+0000","last_peered":"2024-09-10T10:46:44.895164+0000","last_clean":"2024-09-10T10:46:44.895164+0000","last_became_active":"2024-09-10T10:46:44.886293+0000","last_became_peered":"2024-09-10T10:46:44.886293+0000","last_unstale":"2024-09-10T10:46:44.895164+0000","last_undegraded":"2024-09-10T10:46:44.895164+0000","last_fullsized":"2024-09-10T10:46:44.895164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887118+0000","last_change":"2024-09-10T10:46:44.878902+0000","last_active":"2024-09-10T10:46:44.887118+0000","last_peered":"2024-09-10T10:46:44.887118+0000","last_clean":"2024-09-10T10:46:44.887118+0000","last_became_active":"2024-09-10T10:46:44.878564+0000","last_became_peered":"2024-09-10T10:46:44.878564+0000","last_unstale":"2024-09-10T10:46:44.887118+0000","last_undegraded":"2024-09-10T10:46:44.887118+0000","last_fullsized":"2024-09-10T10:46:44.887118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887230+0000","last_change":"2024-09-10T10:46:44.882681+0000","last_active":"2024-09-10T10:46:44.887230+0000","last_peered":"2024-09-10T10:46:44.887230+0000","last_clean":"2024-09-10T10:46:44.887230+0000","last_became_active":"2024-09-10T10:46:44.882405+0000","last_became_peered":"2024-09-10T10:46:44.882405+0000","last_unstale":"2024-09-10T10:46:44.887230+0000","last_undegraded":"2024-09-10T10:46:44.887230+0000","last_fullsized":"2024-09-10T10:46:44.887230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894549+0000","last_change":"2024-09-10T10:46:44.883967+0000","last_active":"2024-09-10T10:46:44.894549+0000","last_peered":"2024-09-10T10:46:44.894549+0000","last_clean":"2024-09-10T10:46:44.894549+0000","last_became_active":"2024-09-10T10:46:44.877104+0000","last_became_peered":"2024-09-10T10:46:44.877104+0000","last_unstale":"2024-09-10T10:46:44.894549+0000","last_undegraded":"2024-09-10T10:46:44.894549+0000","last_fullsized":"2024-09-10T10:46:44.894549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894319+0000","last_change":"2024-09-10T10:46:44.880957+0000","last_active":"2024-09-10T10:46:44.894319+0000","last_peered":"2024-09-10T10:46:44.894319+0000","last_clean":"2024-09-10T10:46:44.894319+0000","last_became_active":"2024-09-10T10:46:44.880691+0000","last_became_peered":"2024-09-10T10:46:44.880691+0000","last_unstale":"2024-09-10T10:46:44.894319+0000","last_undegraded":"2024-09-10T10:46:44.894319+0000","last_fullsized":"2024-09-10T10:46:44.894319+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:06.745405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891012+0000","last_change":"2024-09-10T10:46:44.882838+0000","last_active":"2024-09-10T10:46:44.891012+0000","last_peered":"2024-09-10T10:46:44.891012+0000","last_clean":"2024-09-10T10:46:44.891012+0000","last_became_active":"2024-09-10T10:46:44.880913+0000","last_became_peered":"2024-09-10T10:46:44.880913+0000","last_unstale":"2024-09-10T10:46:44.891012+0000","last_undegraded":"2024-09-10T10:46:44.891012+0000","last_fullsized":"2024-09-10T10:46:44.891012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881711+0000","last_change":"2024-09-10T10:46:44.879407+0000","last_active":"2024-09-10T10:46:44.881711+0000","last_peered":"2024-09-10T10:46:44.881711+0000","last_clean":"2024-09-10T10:46:44.881711+0000","last_became_active":"2024-09-10T10:46:44.879124+0000","last_became_peered":"2024-09-10T10:46:44.879124+0000","last_unstale":"2024-09-10T10:46:44.881711+0000","last_undegraded":"2024-09-10T10:46:44.881711+0000","last_fullsized":"2024-09-10T10:46:44.881711+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879528+0000","last_change":"2024-09-10T10:46:44.874639+0000","last_active":"2024-09-10T10:46:44.879528+0000","last_peered":"2024-09-10T10:46:44.879528+0000","last_clean":"2024-09-10T10:46:44.879528+0000","last_became_active":"2024-09-10T10:46:44.874387+0000","last_became_peered":"2024-09-10T10:46:44.874387+0000","last_unstale":"2024-09-10T10:46:44.879528+0000","last_undegraded":"2024-09-10T10:46:44.879528+0000","last_fullsized":"2024-09-10T10:46:44.879528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:38:57.911434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874284+0000","last_change":"2024-09-10T10:46:44.865791+0000","last_active":"2024-09-10T10:46:44.874284+0000","last_peered":"2024-09-10T10:46:44.874284+0000","last_clean":"2024-09-10T10:46:44.874284+0000","last_became_active":"2024-09-10T10:46:44.865491+0000","last_became_peered":"2024-09-10T10:46:44.865491+0000","last_unstale":"2024-09-10T10:46:44.874284+0000","last_undegraded":"2024-09-10T10:46:44.874284+0000","last_fullsized":"2024-09-10T10:46:44.874284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887094+0000","last_change":"2024-09-10T10:46:44.878815+0000","last_active":"2024-09-10T10:46:44.887094+0000","last_peered":"2024-09-10T10:46:44.887094+0000","last_clean":"2024-09-10T10:46:44.887094+0000","last_became_active":"2024-09-10T10:46:44.878192+0000","last_became_peered":"2024-09-10T10:46:44.878192+0000","last_unstale":"2024-09-10T10:46:44.887094+0000","last_undegraded":"2024-09-10T10:46:44.887094+0000","last_fullsized":"2024-09-10T10:46:44.887094+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879723+0000","last_change":"2024-09-10T10:46:44.871127+0000","last_active":"2024-09-10T10:46:44.879723+0000","last_peered":"2024-09-10T10:46:44.879723+0000","last_clean":"2024-09-10T10:46:44.879723+0000","last_became_active":"2024-09-10T10:46:44.870842+0000","last_became_peered":"2024-09-10T10:46:44.870842+0000","last_unstale":"2024-09-10T10:46:44.879723+0000","last_undegraded":"2024-09-10T10:46:44.879723+0000","last_fullsized":"2024-09-10T10:46:44.879723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:14.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894528+0000","last_change":"2024-09-10T10:46:44.882561+0000","last_active":"2024-09-10T10:46:44.894528+0000","last_peered":"2024-09-10T10:46:44.894528+0000","last_clean":"2024-09-10T10:46:44.894528+0000","last_became_active":"2024-09-10T10:46:44.877515+0000","last_became_peered":"2024-09-10T10:46:44.877515+0000","last_unstale":"2024-09-10T10:46:44.894528+0000","last_undegraded":"2024-09-10T10:46:44.894528+0000","last_fullsized":"2024-09-10T10:46:44.894528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:35.562296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881741+0000","last_change":"2024-09-10T10:46:44.878483+0000","last_active":"2024-09-10T10:46:44.881741+0000","last_peered":"2024-09-10T10:46:44.881741+0000","last_clean":"2024-09-10T10:46:44.881741+0000","last_became_active":"2024-09-10T10:46:44.874186+0000","last_became_peered":"2024-09-10T10:46:44.874186+0000","last_unstale":"2024-09-10T10:46:44.881741+0000","last_undegraded":"2024-09-10T10:46:44.881741+0000","last_fullsized":"2024-09-10T10:46:44.881741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893869+0000","last_change":"2024-09-10T10:46:44.890973+0000","last_active":"2024-09-10T10:46:44.893869+0000","last_peered":"2024-09-10T10:46:44.893869+0000","last_clean":"2024-09-10T10:46:44.893869+0000","last_became_active":"2024-09-10T10:46:44.889471+0000","last_became_peered":"2024-09-10T10:46:44.889471+0000","last_unstale":"2024-09-10T10:46:44.893869+0000","last_undegraded":"2024-09-10T10:46:44.893869+0000","last_fullsized":"2024-09-10T10:46:44.893869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:28.051816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894600+0000","last_change":"2024-09-10T10:46:44.888201+0000","last_active":"2024-09-10T10:46:44.894600+0000","last_peered":"2024-09-10T10:46:44.894600+0000","last_clean":"2024-09-10T10:46:44.894600+0000","last_became_active":"2024-09-10T10:46:44.887970+0000","last_became_peered":"2024-09-10T10:46:44.887970+0000","last_unstale":"2024-09-10T10:46:44.894600+0000","last_undegraded":"2024-09-10T10:46:44.894600+0000","last_fullsized":"2024-09-10T10:46:44.894600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891947+0000","last_change":"2024-09-10T10:46:44.881612+0000","last_active":"2024-09-10T10:46:44.891947+0000","last_peered":"2024-09-10T10:46:44.891947+0000","last_clean":"2024-09-10T10:46:44.891947+0000","last_became_active":"2024-09-10T10:46:44.881250+0000","last_became_peered":"2024-09-10T10:46:44.881250+0000","last_unstale":"2024-09-10T10:46:44.891947+0000","last_undegraded":"2024-09-10T10:46:44.891947+0000","last_fullsized":"2024-09-10T10:46:44.891947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:44.946566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891469+0000","last_change":"2024-09-10T10:46:44.878716+0000","last_active":"2024-09-10T10:46:44.891469+0000","last_peered":"2024-09-10T10:46:44.891469+0000","last_clean":"2024-09-10T10:46:44.891469+0000","last_became_active":"2024-09-10T10:46:44.878422+0000","last_became_peered":"2024-09-10T10:46:44.878422+0000","last_unstale":"2024-09-10T10:46:44.891469+0000","last_undegraded":"2024-09-10T10:46:44.891469+0000","last_fullsized":"2024-09-10T10:46:44.891469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:41.176611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.888369+0000","last_change":"2024-09-10T10:46:44.886054+0000","last_active":"2024-09-10T10:46:44.888369+0000","last_peered":"2024-09-10T10:46:44.888369+0000","last_clean":"2024-09-10T10:46:44.888369+0000","last_became_active":"2024-09-10T10:46:44.885269+0000","last_became_peered":"2024-09-10T10:46:44.885269+0000","last_unstale":"2024-09-10T10:46:44.888369+0000","last_undegraded":"2024-09-10T10:46:44.888369+0000","last_fullsized":"2024-09-10T10:46:44.888369+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874510+0000","last_change":"2024-09-10T10:46:44.869461+0000","last_active":"2024-09-10T10:46:44.874510+0000","last_peered":"2024-09-10T10:46:44.874510+0000","last_clean":"2024-09-10T10:46:44.874510+0000","last_became_active":"2024-09-10T10:46:44.869194+0000","last_became_peered":"2024-09-10T10:46:44.869194+0000","last_unstale":"2024-09-10T10:46:44.874510+0000","last_undegraded":"2024-09-10T10:46:44.874510+0000","last_fullsized":"2024-09-10T10:46:44.874510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887172+0000","last_change":"2024-09-10T10:46:44.879808+0000","last_active":"2024-09-10T10:46:44.887172+0000","last_peered":"2024-09-10T10:46:44.887172+0000","last_clean":"2024-09-10T10:46:44.887172+0000","last_became_active":"2024-09-10T10:46:44.879312+0000","last_became_peered":"2024-09-10T10:46:44.879312+0000","last_unstale":"2024-09-10T10:46:44.887172+0000","last_undegraded":"2024-09-10T10:46:44.887172+0000","last_fullsized":"2024-09-10T10:46:44.887172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892504+0000","last_change":"2024-09-10T10:46:44.881613+0000","last_active":"2024-09-10T10:46:44.892504+0000","last_peered":"2024-09-10T10:46:44.892504+0000","last_clean":"2024-09-10T10:46:44.892504+0000","last_became_active":"2024-09-10T10:46:44.881356+0000","last_became_peered":"2024-09-10T10:46:44.881356+0000","last_unstale":"2024-09-10T10:46:44.892504+0000","last_undegraded":"2024-09-10T10:46:44.892504+0000","last_fullsized":"2024-09-10T10:46:44.892504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:04.057439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.883453+0000","last_change":"2024-09-10T10:46:44.882083+0000","last_active":"2024-09-10T10:46:44.883453+0000","last_peered":"2024-09-10T10:46:44.883453+0000","last_clean":"2024-09-10T10:46:44.883453+0000","last_became_active":"2024-09-10T10:46:44.878774+0000","last_became_peered":"2024-09-10T10:46:44.878774+0000","last_unstale":"2024-09-10T10:46:44.883453+0000","last_undegraded":"2024-09-10T10:46:44.883453+0000","last_fullsized":"2024-09-10T10:46:44.883453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890870+0000","last_change":"2024-09-10T10:46:44.884761+0000","last_active":"2024-09-10T10:46:44.890870+0000","last_peered":"2024-09-10T10:46:44.890870+0000","last_clean":"2024-09-10T10:46:44.890870+0000","last_became_active":"2024-09-10T10:46:44.884412+0000","last_became_peered":"2024-09-10T10:46:44.884412+0000","last_unstale":"2024-09-10T10:46:44.890870+0000","last_undegraded":"2024-09-10T10:46:44.890870+0000","last_fullsized":"2024-09-10T10:46:44.890870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903682+0000","last_change":"2024-09-10T10:46:43.873120+0000","last_active":"2024-09-10T10:46:44.903682+0000","last_peered":"2024-09-10T10:46:44.903682+0000","last_clean":"2024-09-10T10:46:44.903682+0000","last_became_active":"2024-09-10T10:46:43.872682+0000","last_became_peered":"2024-09-10T10:46:43.872682+0000","last_unstale":"2024-09-10T10:46:44.903682+0000","last_undegraded":"2024-09-10T10:46:44.903682+0000","last_fullsized":"2024-09-10T10:46:44.903682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.538093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892606+0000","last_change":"2024-09-10T10:46:44.885626+0000","last_active":"2024-09-10T10:46:44.892606+0000","last_peered":"2024-09-10T10:46:44.892606+0000","last_clean":"2024-09-10T10:46:44.892606+0000","last_became_active":"2024-09-10T10:46:44.883452+0000","last_became_peered":"2024-09-10T10:46:44.883452+0000","last_unstale":"2024-09-10T10:46:44.892606+0000","last_undegraded":"2024-09-10T10:46:44.892606+0000","last_fullsized":"2024-09-10T10:46:44.892606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:57.060214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897771+0000","last_change":"2024-09-10T10:46:43.875822+0000","last_active":"2024-09-10T10:46:44.897771+0000","last_peered":"2024-09-10T10:46:44.897771+0000","last_clean":"2024-09-10T10:46:44.897771+0000","last_became_active":"2024-09-10T10:46:43.875557+0000","last_became_peered":"2024-09-10T10:46:43.875557+0000","last_unstale":"2024-09-10T10:46:44.897771+0000","last_undegraded":"2024-09-10T10:46:44.897771+0000","last_fullsized":"2024-09-10T10:46:44.897771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.628419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.900588+0000","last_change":"2024-09-10T10:46:43.876519+0000","last_active":"2024-09-10T10:46:44.900588+0000","last_peered":"2024-09-10T10:46:44.900588+0000","last_clean":"2024-09-10T10:46:44.900588+0000","last_became_active":"2024-09-10T10:46:43.876247+0000","last_became_peered":"2024-09-10T10:46:43.876247+0000","last_unstale":"2024-09-10T10:46:44.900588+0000","last_undegraded":"2024-09-10T10:46:44.900588+0000","last_fullsized":"2024-09-10T10:46:44.900588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:00.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887341+0000","last_change":"2024-09-10T10:46:44.877202+0000","last_active":"2024-09-10T10:46:44.887341+0000","last_peered":"2024-09-10T10:46:44.887341+0000","last_clean":"2024-09-10T10:46:44.887341+0000","last_became_active":"2024-09-10T10:46:44.876159+0000","last_became_peered":"2024-09-10T10:46:44.876159+0000","last_unstale":"2024-09-10T10:46:44.887341+0000","last_undegraded":"2024-09-10T10:46:44.887341+0000","last_fullsized":"2024-09-10T10:46:44.887341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:01.241200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851870+0000","last_change":"2024-09-10T10:46:42.851870+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851870+0000","last_undegraded":"2024-09-10T10:46:42.851870+0000","last_fullsized":"2024-09-10T10:46:42.851870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.880924+0000","last_change":"2024-09-10T10:46:44.878110+0000","last_active":"2024-09-10T10:46:44.880924+0000","last_peered":"2024-09-10T10:46:44.880924+0000","last_clean":"2024-09-10T10:46:44.880924+0000","last_became_active":"2024-09-10T10:46:44.877841+0000","last_became_peered":"2024-09-10T10:46:44.877841+0000","last_unstale":"2024-09-10T10:46:44.880924+0000","last_undegraded":"2024-09-10T10:46:44.880924+0000","last_fullsized":"2024-09-10T10:46:44.880924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:45.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898049+0000","last_change":"2024-09-10T10:46:43.875159+0000","last_active":"2024-09-10T10:46:44.898049+0000","last_peered":"2024-09-10T10:46:44.898049+0000","last_clean":"2024-09-10T10:46:44.898049+0000","last_became_active":"2024-09-10T10:46:43.874602+0000","last_became_peered":"2024-09-10T10:46:43.874602+0000","last_unstale":"2024-09-10T10:46:44.898049+0000","last_undegraded":"2024-09-10T10:46:44.898049+0000","last_fullsized":"2024-09-10T10:46:44.898049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:48.767108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889322+0000","last_change":"2024-09-10T10:46:44.888134+0000","last_active":"2024-09-10T10:46:44.889322+0000","last_peered":"2024-09-10T10:46:44.889322+0000","last_clean":"2024-09-10T10:46:44.889322+0000","last_became_active":"2024-09-10T10:46:44.887851+0000","last_became_peered":"2024-09-10T10:46:44.887851+0000","last_unstale":"2024-09-10T10:46:44.889322+0000","last_undegraded":"2024-09-10T10:46:44.889322+0000","last_fullsized":"2024-09-10T10:46:44.889322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849833+0000","last_change":"2024-09-10T10:46:42.849833+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849833+0000","last_undegraded":"2024-09-10T10:46:42.849833+0000","last_fullsized":"2024-09-10T10:46:42.849833+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.896764+0000","last_change":"2024-09-10T10:46:44.893318+0000","last_active":"2024-09-10T10:46:44.896764+0000","last_peered":"2024-09-10T10:46:44.896764+0000","last_clean":"2024-09-10T10:46:44.896764+0000","last_became_active":"2024-09-10T10:46:44.893039+0000","last_became_peered":"2024-09-10T10:46:44.893039+0000","last_unstale":"2024-09-10T10:46:44.896764+0000","last_undegraded":"2024-09-10T10:46:44.896764+0000","last_fullsized":"2024-09-10T10:46:44.896764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870255+0000","last_change":"2024-09-10T10:46:43.879425+0000","last_active":"2024-09-10T10:46:44.870255+0000","last_peered":"2024-09-10T10:46:44.870255+0000","last_clean":"2024-09-10T10:46:44.870255+0000","last_became_active":"2024-09-10T10:46:43.879154+0000","last_became_peered":"2024-09-10T10:46:43.879154+0000","last_unstale":"2024-09-10T10:46:44.870255+0000","last_undegraded":"2024-09-10T10:46:44.870255+0000","last_fullsized":"2024-09-10T10:46:44.870255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:41.361020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879467+0000","last_change":"2024-09-10T10:46:44.871663+0000","last_active":"2024-09-10T10:46:44.879467+0000","last_peered":"2024-09-10T10:46:44.879467+0000","last_clean":"2024-09-10T10:46:44.879467+0000","last_became_active":"2024-09-10T10:46:44.869465+0000","last_became_peered":"2024-09-10T10:46:44.869465+0000","last_unstale":"2024-09-10T10:46:44.879467+0000","last_undegraded":"2024-09-10T10:46:44.879467+0000","last_fullsized":"2024-09-10T10:46:44.879467+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:50.834542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.857464+0000","last_change":"2024-09-10T10:46:43.870161+0000","last_active":"2024-09-10T10:46:44.857464+0000","last_peered":"2024-09-10T10:46:44.857464+0000","last_clean":"2024-09-10T10:46:44.857464+0000","last_became_active":"2024-09-10T10:46:43.869918+0000","last_became_peered":"2024-09-10T10:46:43.869918+0000","last_unstale":"2024-09-10T10:46:44.857464+0000","last_undegraded":"2024-09-10T10:46:44.857464+0000","last_fullsized":"2024-09-10T10:46:44.857464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:11.275462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887228+0000","last_change":"2024-09-10T10:46:44.884495+0000","last_active":"2024-09-10T10:46:44.887228+0000","last_peered":"2024-09-10T10:46:44.887228+0000","last_clean":"2024-09-10T10:46:44.887228+0000","last_became_active":"2024-09-10T10:46:44.884231+0000","last_became_peered":"2024-09-10T10:46:44.884231+0000","last_unstale":"2024-09-10T10:46:44.887228+0000","last_undegraded":"2024-09-10T10:46:44.887228+0000","last_fullsized":"2024-09-10T10:46:44.887228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.858268+0000","last_change":"2024-09-10T10:46:43.867286+0000","last_active":"2024-09-10T10:46:44.858268+0000","last_peered":"2024-09-10T10:46:44.858268+0000","last_clean":"2024-09-10T10:46:44.858268+0000","last_became_active":"2024-09-10T10:46:43.867053+0000","last_became_peered":"2024-09-10T10:46:43.867053+0000","last_unstale":"2024-09-10T10:46:44.858268+0000","last_undegraded":"2024-09-10T10:46:44.858268+0000","last_fullsized":"2024-09-10T10:46:44.858268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.574114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881233+0000","last_change":"2024-09-10T10:46:44.870275+0000","last_active":"2024-09-10T10:46:44.881233+0000","last_peered":"2024-09-10T10:46:44.881233+0000","last_clean":"2024-09-10T10:46:44.881233+0000","last_became_active":"2024-09-10T10:46:44.869919+0000","last_became_peered":"2024-09-10T10:46:44.869919+0000","last_unstale":"2024-09-10T10:46:44.881233+0000","last_undegraded":"2024-09-10T10:46:44.881233+0000","last_fullsized":"2024-09-10T10:46:44.881233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:32.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867247+0000","last_change":"2024-09-10T10:46:43.881828+0000","last_active":"2024-09-10T10:46:44.867247+0000","last_peered":"2024-09-10T10:46:44.867247+0000","last_clean":"2024-09-10T10:46:44.867247+0000","last_became_active":"2024-09-10T10:46:43.881524+0000","last_became_peered":"2024-09-10T10:46:43.881524+0000","last_unstale":"2024-09-10T10:46:44.867247+0000","last_undegraded":"2024-09-10T10:46:44.867247+0000","last_fullsized":"2024-09-10T10:46:44.867247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:18.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890526+0000","last_change":"2024-09-10T10:46:44.877148+0000","last_active":"2024-09-10T10:46:44.890526+0000","last_peered":"2024-09-10T10:46:44.890526+0000","last_clean":"2024-09-10T10:46:44.890526+0000","last_became_active":"2024-09-10T10:46:44.876860+0000","last_became_peered":"2024-09-10T10:46:44.876860+0000","last_unstale":"2024-09-10T10:46:44.890526+0000","last_undegraded":"2024-09-10T10:46:44.890526+0000","last_fullsized":"2024-09-10T10:46:44.890526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.904206+0000","last_change":"2024-09-10T10:46:43.874911+0000","last_active":"2024-09-10T10:46:44.904206+0000","last_peered":"2024-09-10T10:46:44.904206+0000","last_clean":"2024-09-10T10:46:44.904206+0000","last_became_active":"2024-09-10T10:46:43.874629+0000","last_became_peered":"2024-09-10T10:46:43.874629+0000","last_unstale":"2024-09-10T10:46:44.904206+0000","last_undegraded":"2024-09-10T10:46:44.904206+0000","last_fullsized":"2024-09-10T10:46:44.904206+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:31.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893676+0000","last_change":"2024-09-10T10:46:44.889982+0000","last_active":"2024-09-10T10:46:44.893676+0000","last_peered":"2024-09-10T10:46:44.893676+0000","last_clean":"2024-09-10T10:46:44.893676+0000","last_became_active":"2024-09-10T10:46:44.889481+0000","last_became_peered":"2024-09-10T10:46:44.889481+0000","last_unstale":"2024-09-10T10:46:44.893676+0000","last_undegraded":"2024-09-10T10:46:44.893676+0000","last_fullsized":"2024-09-10T10:46:44.893676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:42.436804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849881+0000","last_change":"2024-09-10T10:46:42.849881+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849881+0000","last_undegraded":"2024-09-10T10:46:42.849881+0000","last_fullsized":"2024-09-10T10:46:42.849881+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894670+0000","last_change":"2024-09-10T10:46:44.886563+0000","last_active":"2024-09-10T10:46:44.894670+0000","last_peered":"2024-09-10T10:46:44.894670+0000","last_clean":"2024-09-10T10:46:44.894670+0000","last_became_active":"2024-09-10T10:46:44.886299+0000","last_became_peered":"2024-09-10T10:46:44.886299+0000","last_unstale":"2024-09-10T10:46:44.894670+0000","last_undegraded":"2024-09-10T10:46:44.894670+0000","last_fullsized":"2024-09-10T10:46:44.894670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897647+0000","last_change":"2024-09-10T10:46:43.868983+0000","last_active":"2024-09-10T10:46:44.897647+0000","last_peered":"2024-09-10T10:46:44.897647+0000","last_clean":"2024-09-10T10:46:44.897647+0000","last_became_active":"2024-09-10T10:46:43.868631+0000","last_became_peered":"2024-09-10T10:46:43.868631+0000","last_unstale":"2024-09-10T10:46:44.897647+0000","last_undegraded":"2024-09-10T10:46:44.897647+0000","last_fullsized":"2024-09-10T10:46:44.897647+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892284+0000","last_change":"2024-09-10T10:46:44.885879+0000","last_active":"2024-09-10T10:46:44.892284+0000","last_peered":"2024-09-10T10:46:44.892284+0000","last_clean":"2024-09-10T10:46:44.892284+0000","last_became_active":"2024-09-10T10:46:44.885500+0000","last_became_peered":"2024-09-10T10:46:44.885500+0000","last_unstale":"2024-09-10T10:46:44.892284+0000","last_undegraded":"2024-09-10T10:46:44.892284+0000","last_fullsized":"2024-09-10T10:46:44.892284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874282+0000","last_change":"2024-09-10T10:46:43.879565+0000","last_active":"2024-09-10T10:46:44.874282+0000","last_peered":"2024-09-10T10:46:44.874282+0000","last_clean":"2024-09-10T10:46:44.874282+0000","last_became_active":"2024-09-10T10:46:43.879254+0000","last_became_peered":"2024-09-10T10:46:43.879254+0000","last_unstale":"2024-09-10T10:46:44.874282+0000","last_undegraded":"2024-09-10T10:46:44.874282+0000","last_fullsized":"2024-09-10T10:46:44.874282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881138+0000","last_change":"2024-09-10T10:46:44.878593+0000","last_active":"2024-09-10T10:46:44.881138+0000","last_peered":"2024-09-10T10:46:44.881138+0000","last_clean":"2024-09-10T10:46:44.881138+0000","last_became_active":"2024-09-10T10:46:44.878283+0000","last_became_peered":"2024-09-10T10:46:44.878283+0000","last_unstale":"2024-09-10T10:46:44.881138+0000","last_undegraded":"2024-09-10T10:46:44.881138+0000","last_fullsized":"2024-09-10T10:46:44.881138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.865622+0000","last_change":"2024-09-10T10:46:43.877389+0000","last_active":"2024-09-10T10:46:44.865622+0000","last_peered":"2024-09-10T10:46:44.865622+0000","last_clean":"2024-09-10T10:46:44.865622+0000","last_became_active":"2024-09-10T10:46:43.876235+0000","last_became_peered":"2024-09-10T10:46:43.876235+0000","last_unstale":"2024-09-10T10:46:44.865622+0000","last_undegraded":"2024-09-10T10:46:44.865622+0000","last_fullsized":"2024-09-10T10:46:44.865622+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:52.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881110+0000","last_change":"2024-09-10T10:46:44.878187+0000","last_active":"2024-09-10T10:46:44.881110+0000","last_peered":"2024-09-10T10:46:44.881110+0000","last_clean":"2024-09-10T10:46:44.881110+0000","last_became_active":"2024-09-10T10:46:44.877922+0000","last_became_peered":"2024-09-10T10:46:44.877922+0000","last_unstale":"2024-09-10T10:46:44.881110+0000","last_undegraded":"2024-09-10T10:46:44.881110+0000","last_fullsized":"2024-09-10T10:46:44.881110+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.902316+0000","last_change":"2024-09-10T10:46:43.879721+0000","last_active":"2024-09-10T10:46:44.902316+0000","last_peered":"2024-09-10T10:46:44.902316+0000","last_clean":"2024-09-10T10:46:44.902316+0000","last_became_active":"2024-09-10T10:46:43.879448+0000","last_became_peered":"2024-09-10T10:46:43.879448+0000","last_unstale":"2024-09-10T10:46:44.902316+0000","last_undegraded":"2024-09-10T10:46:44.902316+0000","last_fullsized":"2024-09-10T10:46:44.902316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882849+0000","last_change":"2024-09-10T10:46:44.871707+0000","last_active":"2024-09-10T10:46:44.882849+0000","last_peered":"2024-09-10T10:46:44.882849+0000","last_clean":"2024-09-10T10:46:44.882849+0000","last_became_active":"2024-09-10T10:46:44.871400+0000","last_became_peered":"2024-09-10T10:46:44.871400+0000","last_unstale":"2024-09-10T10:46:44.882849+0000","last_undegraded":"2024-09-10T10:46:44.882849+0000","last_fullsized":"2024-09-10T10:46:44.882849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898284+0000","last_change":"2024-09-10T10:46:43.866850+0000","last_active":"2024-09-10T10:46:44.898284+0000","last_peered":"2024-09-10T10:46:44.898284+0000","last_clean":"2024-09-10T10:46:44.898284+0000","last_became_active":"2024-09-10T10:46:43.866575+0000","last_became_peered":"2024-09-10T10:46:43.866575+0000","last_unstale":"2024-09-10T10:46:44.898284+0000","last_undegraded":"2024-09-10T10:46:44.898284+0000","last_fullsized":"2024-09-10T10:46:44.898284+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:07.913685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890441+0000","last_change":"2024-09-10T10:46:44.875025+0000","last_active":"2024-09-10T10:46:44.890441+0000","last_peered":"2024-09-10T10:46:44.890441+0000","last_clean":"2024-09-10T10:46:44.890441+0000","last_became_active":"2024-09-10T10:46:44.873173+0000","last_became_peered":"2024-09-10T10:46:44.873173+0000","last_unstale":"2024-09-10T10:46:44.890441+0000","last_undegraded":"2024-09-10T10:46:44.890441+0000","last_fullsized":"2024-09-10T10:46:44.890441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903260+0000","last_change":"2024-09-10T10:46:43.871807+0000","last_active":"2024-09-10T10:46:44.903260+0000","last_peered":"2024-09-10T10:46:44.903260+0000","last_clean":"2024-09-10T10:46:44.903260+0000","last_became_active":"2024-09-10T10:46:43.871540+0000","last_became_peered":"2024-09-10T10:46:43.871540+0000","last_unstale":"2024-09-10T10:46:44.903260+0000","last_undegraded":"2024-09-10T10:46:44.903260+0000","last_fullsized":"2024-09-10T10:46:44.903260+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:34.555709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894618+0000","last_change":"2024-09-10T10:46:44.888440+0000","last_active":"2024-09-10T10:46:44.894618+0000","last_peered":"2024-09-10T10:46:44.894618+0000","last_clean":"2024-09-10T10:46:44.894618+0000","last_became_active":"2024-09-10T10:46:44.888154+0000","last_became_peered":"2024-09-10T10:46:44.888154+0000","last_unstale":"2024-09-10T10:46:44.894618+0000","last_undegraded":"2024-09-10T10:46:44.894618+0000","last_fullsized":"2024-09-10T10:46:44.894618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.905983+0000","last_change":"2024-09-10T10:46:43.876691+0000","last_active":"2024-09-10T10:46:44.905983+0000","last_peered":"2024-09-10T10:46:44.905983+0000","last_clean":"2024-09-10T10:46:44.905983+0000","last_became_active":"2024-09-10T10:46:43.876227+0000","last_became_peered":"2024-09-10T10:46:43.876227+0000","last_unstale":"2024-09-10T10:46:44.905983+0000","last_undegraded":"2024-09-10T10:46:44.905983+0000","last_fullsized":"2024-09-10T10:46:44.905983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:31.964863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892464+0000","last_change":"2024-09-10T10:46:44.882771+0000","last_active":"2024-09-10T10:46:44.892464+0000","last_peered":"2024-09-10T10:46:44.892464+0000","last_clean":"2024-09-10T10:46:44.892464+0000","last_became_active":"2024-09-10T10:46:44.881897+0000","last_became_peered":"2024-09-10T10:46:44.881897+0000","last_unstale":"2024-09-10T10:46:44.892464+0000","last_undegraded":"2024-09-10T10:46:44.892464+0000","last_fullsized":"2024-09-10T10:46:44.892464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:22.411021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867582+0000","last_change":"2024-09-10T10:46:43.873156+0000","last_active":"2024-09-10T10:46:44.867582+0000","last_peered":"2024-09-10T10:46:44.867582+0000","last_clean":"2024-09-10T10:46:44.867582+0000","last_became_active":"2024-09-10T10:46:43.872889+0000","last_became_peered":"2024-09-10T10:46:43.872889+0000","last_unstale":"2024-09-10T10:46:44.867582+0000","last_undegraded":"2024-09-10T10:46:44.867582+0000","last_fullsized":"2024-09-10T10:46:44.867582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852599+0000","last_change":"2024-09-10T10:46:42.852599+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852599+0000","last_undegraded":"2024-09-10T10:46:42.852599+0000","last_fullsized":"2024-09-10T10:46:42.852599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872927+0000","last_change":"2024-09-10T10:46:44.868747+0000","last_active":"2024-09-10T10:46:44.872927+0000","last_peered":"2024-09-10T10:46:44.872927+0000","last_clean":"2024-09-10T10:46:44.872927+0000","last_became_active":"2024-09-10T10:46:44.868455+0000","last_became_peered":"2024-09-10T10:46:44.868455+0000","last_unstale":"2024-09-10T10:46:44.872927+0000","last_undegraded":"2024-09-10T10:46:44.872927+0000","last_fullsized":"2024-09-10T10:46:44.872927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874479+0000","last_change":"2024-09-10T10:46:43.876827+0000","last_active":"2024-09-10T10:46:44.874479+0000","last_peered":"2024-09-10T10:46:44.874479+0000","last_clean":"2024-09-10T10:46:44.874479+0000","last_became_active":"2024-09-10T10:46:43.875591+0000","last_became_peered":"2024-09-10T10:46:43.875591+0000","last_unstale":"2024-09-10T10:46:44.874479+0000","last_undegraded":"2024-09-10T10:46:44.874479+0000","last_fullsized":"2024-09-10T10:46:44.874479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889806+0000","last_change":"2024-09-10T10:46:44.883361+0000","last_active":"2024-09-10T10:46:44.889806+0000","last_peered":"2024-09-10T10:46:44.889806+0000","last_clean":"2024-09-10T10:46:44.889806+0000","last_became_active":"2024-09-10T10:46:44.879641+0000","last_became_peered":"2024-09-10T10:46:44.879641+0000","last_unstale":"2024-09-10T10:46:44.889806+0000","last_undegraded":"2024-09-10T10:46:44.889806+0000","last_fullsized":"2024-09-10T10:46:44.889806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.849801+0000","last_change":"2024-09-10T10:46:42.849801+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.849801+0000","last_undegraded":"2024-09-10T10:46:42.849801+0000","last_fullsized":"2024-09-10T10:46:42.849801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879741+0000","last_change":"2024-09-10T10:46:44.873964+0000","last_active":"2024-09-10T10:46:44.879741+0000","last_peered":"2024-09-10T10:46:44.879741+0000","last_clean":"2024-09-10T10:46:44.879741+0000","last_became_active":"2024-09-10T10:46:44.873324+0000","last_became_peered":"2024-09-10T10:46:44.873324+0000","last_unstale":"2024-09-10T10:46:44.879741+0000","last_undegraded":"2024-09-10T10:46:44.879741+0000","last_fullsized":"2024-09-10T10:46:44.879741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:46.801922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897908+0000","last_change":"2024-09-10T10:46:43.880123+0000","last_active":"2024-09-10T10:46:44.897908+0000","last_peered":"2024-09-10T10:46:44.897908+0000","last_clean":"2024-09-10T10:46:44.897908+0000","last_became_active":"2024-09-10T10:46:43.879861+0000","last_became_peered":"2024-09-10T10:46:43.879861+0000","last_unstale":"2024-09-10T10:46:44.897908+0000","last_undegraded":"2024-09-10T10:46:44.897908+0000","last_fullsized":"2024-09-10T10:46:44.897908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:40.170023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881256+0000","last_change":"2024-09-10T10:46:44.875117+0000","last_active":"2024-09-10T10:46:44.881256+0000","last_peered":"2024-09-10T10:46:44.881256+0000","last_clean":"2024-09-10T10:46:44.881256+0000","last_became_active":"2024-09-10T10:46:44.874858+0000","last_became_peered":"2024-09-10T10:46:44.874858+0000","last_unstale":"2024-09-10T10:46:44.881256+0000","last_undegraded":"2024-09-10T10:46:44.881256+0000","last_fullsized":"2024-09-10T10:46:44.881256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:08.230635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.859204+0000","last_change":"2024-09-10T10:46:43.870732+0000","last_active":"2024-09-10T10:46:44.859204+0000","last_peered":"2024-09-10T10:46:44.859204+0000","last_clean":"2024-09-10T10:46:44.859204+0000","last_became_active":"2024-09-10T10:46:43.870484+0000","last_became_peered":"2024-09-10T10:46:43.870484+0000","last_unstale":"2024-09-10T10:46:44.859204+0000","last_undegraded":"2024-09-10T10:46:44.859204+0000","last_fullsized":"2024-09-10T10:46:44.859204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:05.738817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881485+0000","last_change":"2024-09-10T10:46:44.875783+0000","last_active":"2024-09-10T10:46:44.881485+0000","last_peered":"2024-09-10T10:46:44.881485+0000","last_clean":"2024-09-10T10:46:44.881485+0000","last_became_active":"2024-09-10T10:46:44.875511+0000","last_became_peered":"2024-09-10T10:46:44.875511+0000","last_unstale":"2024-09-10T10:46:44.881485+0000","last_undegraded":"2024-09-10T10:46:44.881485+0000","last_fullsized":"2024-09-10T10:46:44.881485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:37.571008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.852053+0000","last_change":"2024-09-10T10:46:42.852053+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.852053+0000","last_undegraded":"2024-09-10T10:46:42.852053+0000","last_fullsized":"2024-09-10T10:46:42.852053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892438+0000","last_change":"2024-09-10T10:46:44.882540+0000","last_active":"2024-09-10T10:46:44.892438+0000","last_peered":"2024-09-10T10:46:44.892438+0000","last_clean":"2024-09-10T10:46:44.892438+0000","last_became_active":"2024-09-10T10:46:44.881322+0000","last_became_peered":"2024-09-10T10:46:44.881322+0000","last_unstale":"2024-09-10T10:46:44.892438+0000","last_undegraded":"2024-09-10T10:46:44.892438+0000","last_fullsized":"2024-09-10T10:46:44.892438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:05.028664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867719+0000","last_change":"2024-09-10T10:46:43.880848+0000","last_active":"2024-09-10T10:46:44.867719+0000","last_peered":"2024-09-10T10:46:44.867719+0000","last_clean":"2024-09-10T10:46:44.867719+0000","last_became_active":"2024-09-10T10:46:43.880588+0000","last_became_peered":"2024-09-10T10:46:43.880588+0000","last_unstale":"2024-09-10T10:46:44.867719+0000","last_undegraded":"2024-09-10T10:46:44.867719+0000","last_fullsized":"2024-09-10T10:46:44.867719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887215+0000","last_change":"2024-09-10T10:46:44.879311+0000","last_active":"2024-09-10T10:46:44.887215+0000","last_peered":"2024-09-10T10:46:44.887215+0000","last_clean":"2024-09-10T10:46:44.887215+0000","last_became_active":"2024-09-10T10:46:44.867480+0000","last_became_peered":"2024-09-10T10:46:44.867480+0000","last_unstale":"2024-09-10T10:46:44.887215+0000","last_undegraded":"2024-09-10T10:46:44.887215+0000","last_fullsized":"2024-09-10T10:46:44.887215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.866776+0000","last_change":"2024-09-10T10:46:43.878330+0000","last_active":"2024-09-10T10:46:44.866776+0000","last_peered":"2024-09-10T10:46:44.866776+0000","last_clean":"2024-09-10T10:46:44.866776+0000","last_became_active":"2024-09-10T10:46:43.877459+0000","last_became_peered":"2024-09-10T10:46:43.877459+0000","last_unstale":"2024-09-10T10:46:44.866776+0000","last_undegraded":"2024-09-10T10:46:44.866776+0000","last_fullsized":"2024-09-10T10:46:44.866776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:13.837013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872902+0000","last_change":"2024-09-10T10:46:44.868549+0000","last_active":"2024-09-10T10:46:44.872902+0000","last_peered":"2024-09-10T10:46:44.872902+0000","last_clean":"2024-09-10T10:46:44.872902+0000","last_became_active":"2024-09-10T10:46:44.868232+0000","last_became_peered":"2024-09-10T10:46:44.868232+0000","last_unstale":"2024-09-10T10:46:44.872902+0000","last_undegraded":"2024-09-10T10:46:44.872902+0000","last_fullsized":"2024-09-10T10:46:44.872902+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870340+0000","last_change":"2024-09-10T10:46:43.879611+0000","last_active":"2024-09-10T10:46:44.870340+0000","last_peered":"2024-09-10T10:46:44.870340+0000","last_clean":"2024-09-10T10:46:44.870340+0000","last_became_active":"2024-09-10T10:46:43.879308+0000","last_became_peered":"2024-09-10T10:46:43.879308+0000","last_unstale":"2024-09-10T10:46:44.870340+0000","last_undegraded":"2024-09-10T10:46:44.870340+0000","last_fullsized":"2024-09-10T10:46:44.870340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:57.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:46:43.845107+0000","last_change":"2024-09-10T10:46:43.845107+0000","last_active":"2024-09-10T10:46:43.845107+0000","last_peered":"2024-09-10T10:46:43.845107+0000","last_clean":"2024-09-10T10:46:43.845107+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:43.845107+0000","last_undegraded":"2024-09-10T10:46:43.845107+0000","last_fullsized":"2024-09-10T10:46:43.845107+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867820+0000","last_change":"2024-09-10T10:46:43.879656+0000","last_active":"2024-09-10T10:46:44.867820+0000","last_peered":"2024-09-10T10:46:44.867820+0000","last_clean":"2024-09-10T10:46:44.867820+0000","last_became_active":"2024-09-10T10:46:43.879098+0000","last_became_peered":"2024-09-10T10:46:43.879098+0000","last_unstale":"2024-09-10T10:46:44.867820+0000","last_undegraded":"2024-09-10T10:46:44.867820+0000","last_fullsized":"2024-09-10T10:46:44.867820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879442+0000","last_change":"2024-09-10T10:46:44.869395+0000","last_active":"2024-09-10T10:46:44.879442+0000","last_peered":"2024-09-10T10:46:44.879442+0000","last_clean":"2024-09-10T10:46:44.879442+0000","last_became_active":"2024-09-10T10:46:44.869065+0000","last_became_peered":"2024-09-10T10:46:44.869065+0000","last_unstale":"2024-09-10T10:46:44.879442+0000","last_undegraded":"2024-09-10T10:46:44.879442+0000","last_fullsized":"2024-09-10T10:46:44.879442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:52.312041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.851961+0000","last_change":"2024-09-10T10:46:42.851961+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.851961+0000","last_undegraded":"2024-09-10T10:46:42.851961+0000","last_fullsized":"2024-09-10T10:46:42.851961+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891869+0000","last_change":"2024-09-10T10:46:44.885648+0000","last_active":"2024-09-10T10:46:44.891869+0000","last_peered":"2024-09-10T10:46:44.891869+0000","last_clean":"2024-09-10T10:46:44.891869+0000","last_became_active":"2024-09-10T10:46:44.885219+0000","last_became_peered":"2024-09-10T10:46:44.885219+0000","last_unstale":"2024-09-10T10:46:44.891869+0000","last_undegraded":"2024-09-10T10:46:44.891869+0000","last_fullsized":"2024-09-10T10:46:44.891869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:19.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":67,"state":"creating+peering","last_fresh":"2024-09-10T10:46:42.850800+0000","last_change":"2024-09-10T10:46:42.850800+0000","last_active":"2024-09-10T10:46:42.838519+0000","last_peered":"2024-09-10T10:46:42.838519+0000","last_clean":"2024-09-10T10:46:42.838519+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:46:42.850800+0000","last_undegraded":"2024-09-10T10:46:42.850800+0000","last_fullsized":"2024-09-10T10:46:42.850800+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":257,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.871099+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:44.871099+0000","last_peered":"2024-09-10T10:46:44.871099+0000","last_clean":"2024-09-10T10:46:44.871099+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:44.871099+0000","last_undegraded":"2024-09-10T10:46:44.871099+0000","last_fullsized":"2024-09-10T10:46:44.871099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":116,"acting":116,"num_store_stats":7},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":367029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":374778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":374356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":835584,"data_stored":374252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":964598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28392,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343448,"statfs":{"total":96636764160,"available":96607690752,"internally_reserved":0,"allocated":1396736,"data_stored":950730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:49.917 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-09-10T10:46:49.917 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.917 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1c is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.851870+0000', 'last_change': '2024-09-10T10:46:42.851870+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.851870+0000', 'last_undegraded': '2024-09-10T10:46:42.851870+0000', 'last_fullsized': '2024-09-10T10:46:42.851870+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:PG 9.1a is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.849833+0000', 'last_change': '2024-09-10T10:46:42.849833+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.849833+0000', 'last_undegraded': '2024-09-10T10:46:42.849833+0000', 'last_fullsized': '2024-09-10T10:46:42.849833+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:PG 9.14 is not active+clean 2024-09-10T10:46:49.918 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.849881+0000', 'last_change': '2024-09-10T10:46:42.849881+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.849881+0000', 'last_undegraded': '2024-09-10T10:46:42.849881+0000', 'last_fullsized': '2024-09-10T10:46:42.849881+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 9.b is not active+clean 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.852599+0000', 'last_change': '2024-09-10T10:46:42.852599+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.852599+0000', 'last_undegraded': '2024-09-10T10:46:42.852599+0000', 'last_fullsized': '2024-09-10T10:46:42.852599+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 9.9 is not active+clean 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.849801+0000', 'last_change': '2024-09-10T10:46:42.849801+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.849801+0000', 'last_undegraded': '2024-09-10T10:46:42.849801+0000', 'last_fullsized': '2024-09-10T10:46:42.849801+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 9.5 is not active+clean 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.852053+0000', 'last_change': '2024-09-10T10:46:42.852053+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.852053+0000', 'last_undegraded': '2024-09-10T10:46:42.852053+0000', 'last_fullsized': '2024-09-10T10:46:42.852053+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-10T10:46:43.845107+0000', 'last_change': '2024-09-10T10:46:43.845107+0000', 'last_active': '2024-09-10T10:46:43.845107+0000', 'last_peered': '2024-09-10T10:46:43.845107+0000', 'last_clean': '2024-09-10T10:46:43.845107+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:43.845107+0000', 'last_undegraded': '2024-09-10T10:46:43.845107+0000', 'last_fullsized': '2024-09-10T10:46:43.845107+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:43.845107+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-10T10:46:49.919 INFO:tasks.ceph.ceph_manager.ceph:PG 9.0 is not active+clean 2024-09-10T10:46:49.920 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.851961+0000', 'last_change': '2024-09-10T10:46:42.851961+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.851961+0000', 'last_undegraded': '2024-09-10T10:46:42.851961+0000', 'last_fullsized': '2024-09-10T10:46:42.851961+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.920 INFO:tasks.ceph.ceph_manager.ceph:PG 9.8 is not active+clean 2024-09-10T10:46:49.920 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 67, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:46:42.850800+0000', 'last_change': '2024-09-10T10:46:42.850800+0000', 'last_active': '2024-09-10T10:46:42.838519+0000', 'last_peered': '2024-09-10T10:46:42.838519+0000', 'last_clean': '2024-09-10T10:46:42.838519+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:46:42.850800+0000', 'last_undegraded': '2024-09-10T10:46:42.850800+0000', 'last_fullsized': '2024-09-10T10:46:42.850800+0000', 'mapping_epoch': 67, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 67, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'last_clean_scrub_stamp': '2024-09-10T10:46:42.838519+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:46:49.920 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T10:46:50.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.106+0000 7fb0cbc00640 1 Processor -- start 2024-09-10T10:46:50.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.106+0000 7fb0cbc00640 1 -- start start 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0cbc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0c415b300 0x7fb0c40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0cbc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0cbc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c40cdbe0 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0cbc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c40728a0 con 0x7fb0c415b300 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0cbc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c40cd900 con 0x7fb0c406f8d0 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0c9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0c415b300 0x7fb0c40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0c9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0c415b300 0x7fb0c40cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50134/0 (socket says 172.21.15.28:50134) 2024-09-10T10:46:50.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0c9200640 1 -- 172.21.15.28:0/1599677662 learned_addr learned my addr 172.21.15.28:0/1599677662 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.107+0000 7fb0c3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb0c40cd3f0 0x7fb0c406f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56732/0 (socket says 172.21.15.28:56732) 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9c00640 1 --2- 172.21.15.28:0/1599677662 >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9200640 1 -- 172.21.15.28:0/1599677662 >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 msgr2=0x7fb0c415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9200640 1 --2- 172.21.15.28:0/1599677662 >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9200640 1 -- 172.21.15.28:0/1599677662 >> v1:172.21.15.28:6789/0 conn(0x7fb0c40cd3f0 legacy=0x7fb0c406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9200640 1 -- 172.21.15.28:0/1599677662 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c4072250 con 0x7fb0c415b300 2024-09-10T10:46:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.108+0000 7fb0c9c00640 1 --2- 172.21.15.28:0/1599677662 >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:50.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.109+0000 7fb0c9200640 1 --2- 172.21.15.28:0/1599677662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c415b300 0x7fb0c40cceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b38f0 tx=0x7fb0b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9404d6ae4343bd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.109+0000 7fb0c3400640 1 -- 172.21.15.28:0/1599677662 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8037070 con 0x7fb0c415b300 2024-09-10T10:46:50.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.109+0000 7fb0c3400640 1 -- 172.21.15.28:0/1599677662 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b8032440 con 0x7fb0c415b300 2024-09-10T10:46:50.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.109+0000 7fb0c3400640 1 -- 172.21.15.28:0/1599677662 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8032810 con 0x7fb0c415b300 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 -- 172.21.15.28:0/1599677662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c415b300 msgr2=0x7fb0c40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/1599677662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c415b300 0x7fb0c40cceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b38f0 tx=0x7fb0b802a270 comp rx=0 tx=0).stop 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 -- 172.21.15.28:0/1599677662 shutdown_connections 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/1599677662 >> v2:172.21.15.138:3300/0 conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/1599677662 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c415b300 0x7fb0c40cceb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 -- 172.21.15.28:0/1599677662 >> 172.21.15.28:0/1599677662 conn(0x7fb0c4067d70 msgr2=0x7fb0c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 -- 172.21.15.28:0/1599677662 shutdown_connections 2024-09-10T10:46:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.110+0000 7fb0cbc00640 1 -- 172.21.15.28:0/1599677662 wait complete. 2024-09-10T10:46:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.111+0000 7fb0cbc00640 1 Processor -- start 2024-09-10T10:46:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 -- start start 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 0x7fb0c414d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40d0160 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c40cffe0 con 0x7fb0c406f8d0 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.112+0000 7fb0cbc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c4072d30 con 0x7fb0c415b300 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50136/0 (socket says 172.21.15.28:50136) 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 -- 172.21.15.28:0/865993320 learned_addr learned my addr 172.21.15.28:0/865993320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c9200640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c9c00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 0x7fb0c414d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 -- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 msgr2=0x7fb0c414d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 0x7fb0c414d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 -- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 msgr2=0x7fb0c415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c3e00640 1 -- 172.21.15.28:0/865993320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c415bcc0 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c9200640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.113+0000 7fb0c9c00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 0x7fb0c414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0c3e00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c400 tx=0x7fb0b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00074e0 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b0007680 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0004a00 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c415bfa0 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.114+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c415c560 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.115+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c414a680 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.116+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0b001a020 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.117+0000 7fb0c0c00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 0x7fb0a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.117+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fb0b008a8f0 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.117+0000 7fb0c9200640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 0x7fb0a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.118+0000 7fb0c9200640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 0x7fb0a406e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8034000 tx=0x7fb0b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.121+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0b0059070 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.372+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fb0c40af030 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.372+0000 7fb0c0c00640 1 -- 172.21.15.28:0/865993320 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1637 (secure 0 0 0) 0x7fb0b0058970 con 0x7fb0c40cd3f0 2024-09-10T10:46:50.374 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:50.374 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":208,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":70,"num_osds":8,"num_up_osds":8,"osd_up_since":1725965015,"num_in_osds":8,"osd_in_since":1725965005,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":8},{"state_name":"unknown","count":6}],"num_pgs":97,"num_pools":3,"num_objects":17,"data_bytes":591166,"bytes_used":250535936,"bytes_avail":772843577344,"bytes_total":773094113280,"unknown_pgs_ratio":0.061855670064687729,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":1982,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":48,"btime":"2024-09-10T10:46:47:884742+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"b","status":"up:active","gid":5844}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-10T10:46:32.522832+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.374+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 msgr2=0x7fb0a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.374+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 0x7fb0a406e2f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8034000 tx=0x7fb0b802e000 comp rx=0 tx=0).stop 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.374+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 msgr2=0x7fb0c414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.374+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c400 tx=0x7fb0b000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 shutdown_connections 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0a406be30 0x7fb0a406e2f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0c415b300 0x7fb0c414d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c40cd3f0 0x7fb0c414a140 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 --2- 172.21.15.28:0/865993320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c406f8d0 0x7fb0c415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 >> 172.21.15.28:0/865993320 conn(0x7fb0c4067d70 msgr2=0x7fb0c40af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 shutdown_connections 2024-09-10T10:46:50.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.375+0000 7fb0cbc00640 1 -- 172.21.15.28:0/865993320 wait complete. 2024-09-10T10:46:50.391 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T10:46:50.391 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.547+0000 7f5522600640 1 Processor -- start 2024-09-10T10:46:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.547+0000 7f5522600640 1 -- start start 2024-09-10T10:46:50.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5522600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f551c15e660 0x7f551c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5522600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551c0cf000 con 0x7f551c0d1f90 2024-09-10T10:46:50.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5522600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551c0d29a0 con 0x7f551c0d0dd0 2024-09-10T10:46:50.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5522600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f551c0b37d0 con 0x7f551c15e660 2024-09-10T10:46:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5521c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f551c0d1f90 0x7f551c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56744/0 (socket says 172.21.15.28:56744) 2024-09-10T10:46:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f5521c00640 1 -- 172.21.15.28:0/1899361788 learned_addr learned my addr 172.21.15.28:0/1899361788 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.548+0000 7f551be00640 1 --2- 172.21.15.28:0/1899361788 >> v2:172.21.15.138:3300/0 conn(0x7f551c15e660 0x7f551c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.549+0000 7f551b400640 1 -- 172.21.15.28:0/1899361788 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1535907017 0 0) 0x7f551c0cf000 con 0x7f551c0d1f90 2024-09-10T10:46:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.549+0000 7f551b400640 1 -- 172.21.15.28:0/1899361788 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5504003520 con 0x7f551c0d1f90 2024-09-10T10:46:50.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.549+0000 7f551be00640 1 -- 172.21.15.28:0/1899361788 >> v1:172.21.15.138:6789/0 conn(0x7f551c0d0dd0 legacy=0x7f551c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.549+0000 7f551be00640 1 -- 172.21.15.28:0/1899361788 >> v1:172.21.15.28:6789/0 conn(0x7f551c0d1f90 legacy=0x7f551c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.549+0000 7f551be00640 1 -- 172.21.15.28:0/1899361788 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c0d2c60 con 0x7f551c15e660 2024-09-10T10:46:50.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.550+0000 7f551be00640 1 --2- 172.21.15.28:0/1899361788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c15e660 0x7f551c06ccd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d2e6d7b51739070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.550+0000 7f551b400640 1 -- 172.21.15.28:0/1899361788 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508037070 con 0x7f551c15e660 2024-09-10T10:46:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.550+0000 7f551b400640 1 -- 172.21.15.28:0/1899361788 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5508032440 con 0x7f551c15e660 2024-09-10T10:46:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.550+0000 7f551b400640 1 -- 172.21.15.28:0/1899361788 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508032810 con 0x7f551c15e660 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 -- 172.21.15.28:0/1899361788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c15e660 msgr2=0x7f551c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 --2- 172.21.15.28:0/1899361788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c15e660 0x7f551c06ccd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55080097a0 tx=0x7f550802a230 comp rx=0 tx=0).stop 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 -- 172.21.15.28:0/1899361788 shutdown_connections 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 --2- 172.21.15.28:0/1899361788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c15e660 0x7f551c06ccd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 -- 172.21.15.28:0/1899361788 >> 172.21.15.28:0/1899361788 conn(0x7f551c08bfc0 msgr2=0x7f551c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 -- 172.21.15.28:0/1899361788 shutdown_connections 2024-09-10T10:46:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.551+0000 7f5522600640 1 -- 172.21.15.28:0/1899361788 wait complete. 2024-09-10T10:46:50.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.553+0000 7f5522600640 1 Processor -- start 2024-09-10T10:46:50.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.553+0000 7f5522600640 1 -- start start 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c0d0dd0 0x7f551c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f551c0d1f90 0x7f551c0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0d3220 con 0x7f551c15e660 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0cf330 con 0x7f551c0d0dd0 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5522600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f551c0d33a0 con 0x7f551c0d1f90 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5521c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5521c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50146/0 (socket says 172.21.15.28:50146) 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5521c00640 1 -- 172.21.15.28:0/1020088778 learned_addr learned my addr 172.21.15.28:0/1020088778 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f551be00640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f551c0d1f90 0x7f551c0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.554+0000 7f5521200640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c0d0dd0 0x7f551c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 -- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f551c0d1f90 msgr2=0x7f551c0b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f551c0d1f90 0x7f551c0b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 -- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c0d0dd0 msgr2=0x7f551c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c0d0dd0 0x7f551c15c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 -- 172.21.15.28:0/1020088778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c0cc310 con 0x7f551c15e660 2024-09-10T10:46:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5521c00640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f551400cb40 tx=0x7f551400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514017070 con 0x7f551c15e660 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551400aca0 con 0x7f551c15e660 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514012620 con 0x7f551c15e660 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c0cc5f0 con 0x7f551c15e660 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.555+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1e6c50 con 0x7f551c15e660 2024-09-10T10:46:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.556+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f551c06d2a0 con 0x7f551c15e660 2024-09-10T10:46:50.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.557+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f551401a070 con 0x7f551c15e660 2024-09-10T10:46:50.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.558+0000 7f5518c00640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 0x7f54fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:50.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.558+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f551408ae50 con 0x7f551c15e660 2024-09-10T10:46:50.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.559+0000 7f5521200640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 0x7f54fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:50.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.559+0000 7f5521200640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 0x7f54fc06e320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5510001780 tx=0x7f5510009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:50.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.563+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5514058f80 con 0x7f551c15e660 2024-09-10T10:46:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.743+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f551c0d4fb0 con 0x7f54fc06be60 2024-09-10T10:46:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.750+0000 7f5518c00640 1 -- 172.21.15.28:0/1020088778 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251963 (secure 0 0 0) 0x7f551c0d4fb0 con 0x7f54fc06be60 2024-09-10T10:46:50.754 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:50.755 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:50.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 msgr2=0x7f54fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 0x7f54fc06e320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5510001780 tx=0x7f5510009040 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 msgr2=0x7f551c0b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f551400cb40 tx=0x7f551400a060 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 shutdown_connections 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54fc06be60 0x7f54fc06e320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f551c15e660 0x7f551c0b9d90 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f551c0d1f90 0x7f551c0b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 --2- 172.21.15.28:0/1020088778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f551c0d0dd0 0x7f551c15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 >> 172.21.15.28:0/1020088778 conn(0x7f551c08bfc0 msgr2=0x7f551c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 shutdown_connections 2024-09-10T10:46:50.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:50.754+0000 7f5522600640 1 -- 172.21.15.28:0/1020088778 wait complete. 2024-09-10T10:46:50.770 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-10T10:46:50.526730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244724,"kb_used_data":7668,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729996,"statfs":{"total":773094113280,"available":772843515904,"internally_reserved":0,"allocated":7852032,"data_stored":4176385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.672775"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895722+0000","last_change":"2024-09-10T10:46:44.892310+0000","last_active":"2024-09-10T10:46:44.895722+0000","last_peered":"2024-09-10T10:46:44.895722+0000","last_clean":"2024-09-10T10:46:44.895722+0000","last_became_active":"2024-09-10T10:46:44.892049+0000","last_became_peered":"2024-09-10T10:46:44.892049+0000","last_unstale":"2024-09-10T10:46:44.895722+0000","last_undegraded":"2024-09-10T10:46:44.895722+0000","last_fullsized":"2024-09-10T10:46:44.895722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810797+0000","last_change":"2024-09-10T10:46:44.881811+0000","last_active":"2024-09-10T10:46:46.810797+0000","last_peered":"2024-09-10T10:46:46.810797+0000","last_clean":"2024-09-10T10:46:46.810797+0000","last_became_active":"2024-09-10T10:46:44.881403+0000","last_became_peered":"2024-09-10T10:46:44.881403+0000","last_unstale":"2024-09-10T10:46:46.810797+0000","last_undegraded":"2024-09-10T10:46:46.810797+0000","last_fullsized":"2024-09-10T10:46:46.810797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:01.468381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892585+0000","last_change":"2024-09-10T10:46:44.884268+0000","last_active":"2024-09-10T10:46:44.892585+0000","last_peered":"2024-09-10T10:46:44.892585+0000","last_clean":"2024-09-10T10:46:44.892585+0000","last_became_active":"2024-09-10T10:46:44.883101+0000","last_became_peered":"2024-09-10T10:46:44.883101+0000","last_unstale":"2024-09-10T10:46:44.892585+0000","last_undegraded":"2024-09-10T10:46:44.892585+0000","last_fullsized":"2024-09-10T10:46:44.892585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:14.573857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882869+0000","last_change":"2024-09-10T10:46:44.875622+0000","last_active":"2024-09-10T10:46:44.882869+0000","last_peered":"2024-09-10T10:46:44.882869+0000","last_clean":"2024-09-10T10:46:44.882869+0000","last_became_active":"2024-09-10T10:46:44.875330+0000","last_became_peered":"2024-09-10T10:46:44.875330+0000","last_unstale":"2024-09-10T10:46:44.882869+0000","last_undegraded":"2024-09-10T10:46:44.882869+0000","last_fullsized":"2024-09-10T10:46:44.882869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892693+0000","last_change":"2024-09-10T10:46:44.884914+0000","last_active":"2024-09-10T10:46:44.892693+0000","last_peered":"2024-09-10T10:46:44.892693+0000","last_clean":"2024-09-10T10:46:44.892693+0000","last_became_active":"2024-09-10T10:46:44.884638+0000","last_became_peered":"2024-09-10T10:46:44.884638+0000","last_unstale":"2024-09-10T10:46:44.892693+0000","last_undegraded":"2024-09-10T10:46:44.892693+0000","last_fullsized":"2024-09-10T10:46:44.892693+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:20.876408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811010+0000","last_change":"2024-09-10T10:46:44.872857+0000","last_active":"2024-09-10T10:46:46.811010+0000","last_peered":"2024-09-10T10:46:46.811010+0000","last_clean":"2024-09-10T10:46:46.811010+0000","last_became_active":"2024-09-10T10:46:44.871922+0000","last_became_peered":"2024-09-10T10:46:44.871922+0000","last_unstale":"2024-09-10T10:46:46.811010+0000","last_undegraded":"2024-09-10T10:46:46.811010+0000","last_fullsized":"2024-09-10T10:46:46.811010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:55.192636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.884835+0000","last_change":"2024-09-10T10:46:44.883459+0000","last_active":"2024-09-10T10:46:44.884835+0000","last_peered":"2024-09-10T10:46:44.884835+0000","last_clean":"2024-09-10T10:46:44.884835+0000","last_became_active":"2024-09-10T10:46:44.883173+0000","last_became_peered":"2024-09-10T10:46:44.883173+0000","last_unstale":"2024-09-10T10:46:44.884835+0000","last_undegraded":"2024-09-10T10:46:44.884835+0000","last_fullsized":"2024-09-10T10:46:44.884835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893848+0000","last_change":"2024-09-10T10:46:44.890220+0000","last_active":"2024-09-10T10:46:44.893848+0000","last_peered":"2024-09-10T10:46:44.893848+0000","last_clean":"2024-09-10T10:46:44.893848+0000","last_became_active":"2024-09-10T10:46:44.889918+0000","last_became_peered":"2024-09-10T10:46:44.889918+0000","last_unstale":"2024-09-10T10:46:44.893848+0000","last_undegraded":"2024-09-10T10:46:44.893848+0000","last_fullsized":"2024-09-10T10:46:44.893848+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:58.573654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887361+0000","last_change":"2024-09-10T10:46:44.877805+0000","last_active":"2024-09-10T10:46:44.887361+0000","last_peered":"2024-09-10T10:46:44.887361+0000","last_clean":"2024-09-10T10:46:44.887361+0000","last_became_active":"2024-09-10T10:46:44.875492+0000","last_became_peered":"2024-09-10T10:46:44.875492+0000","last_unstale":"2024-09-10T10:46:44.887361+0000","last_undegraded":"2024-09-10T10:46:44.887361+0000","last_fullsized":"2024-09-10T10:46:44.887361+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:42.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.895164+0000","last_change":"2024-09-10T10:46:44.886608+0000","last_active":"2024-09-10T10:46:44.895164+0000","last_peered":"2024-09-10T10:46:44.895164+0000","last_clean":"2024-09-10T10:46:44.895164+0000","last_became_active":"2024-09-10T10:46:44.886293+0000","last_became_peered":"2024-09-10T10:46:44.886293+0000","last_unstale":"2024-09-10T10:46:44.895164+0000","last_undegraded":"2024-09-10T10:46:44.895164+0000","last_fullsized":"2024-09-10T10:46:44.895164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887118+0000","last_change":"2024-09-10T10:46:44.878902+0000","last_active":"2024-09-10T10:46:44.887118+0000","last_peered":"2024-09-10T10:46:44.887118+0000","last_clean":"2024-09-10T10:46:44.887118+0000","last_became_active":"2024-09-10T10:46:44.878564+0000","last_became_peered":"2024-09-10T10:46:44.878564+0000","last_unstale":"2024-09-10T10:46:44.887118+0000","last_undegraded":"2024-09-10T10:46:44.887118+0000","last_fullsized":"2024-09-10T10:46:44.887118+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887230+0000","last_change":"2024-09-10T10:46:44.882681+0000","last_active":"2024-09-10T10:46:44.887230+0000","last_peered":"2024-09-10T10:46:44.887230+0000","last_clean":"2024-09-10T10:46:44.887230+0000","last_became_active":"2024-09-10T10:46:44.882405+0000","last_became_peered":"2024-09-10T10:46:44.882405+0000","last_unstale":"2024-09-10T10:46:44.887230+0000","last_undegraded":"2024-09-10T10:46:44.887230+0000","last_fullsized":"2024-09-10T10:46:44.887230+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894549+0000","last_change":"2024-09-10T10:46:44.883967+0000","last_active":"2024-09-10T10:46:44.894549+0000","last_peered":"2024-09-10T10:46:44.894549+0000","last_clean":"2024-09-10T10:46:44.894549+0000","last_became_active":"2024-09-10T10:46:44.877104+0000","last_became_peered":"2024-09-10T10:46:44.877104+0000","last_unstale":"2024-09-10T10:46:44.894549+0000","last_undegraded":"2024-09-10T10:46:44.894549+0000","last_fullsized":"2024-09-10T10:46:44.894549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894319+0000","last_change":"2024-09-10T10:46:44.880957+0000","last_active":"2024-09-10T10:46:44.894319+0000","last_peered":"2024-09-10T10:46:44.894319+0000","last_clean":"2024-09-10T10:46:44.894319+0000","last_became_active":"2024-09-10T10:46:44.880691+0000","last_became_peered":"2024-09-10T10:46:44.880691+0000","last_unstale":"2024-09-10T10:46:44.894319+0000","last_undegraded":"2024-09-10T10:46:44.894319+0000","last_fullsized":"2024-09-10T10:46:44.894319+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:06.745405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891012+0000","last_change":"2024-09-10T10:46:44.882838+0000","last_active":"2024-09-10T10:46:44.891012+0000","last_peered":"2024-09-10T10:46:44.891012+0000","last_clean":"2024-09-10T10:46:44.891012+0000","last_became_active":"2024-09-10T10:46:44.880913+0000","last_became_peered":"2024-09-10T10:46:44.880913+0000","last_unstale":"2024-09-10T10:46:44.891012+0000","last_undegraded":"2024-09-10T10:46:44.891012+0000","last_fullsized":"2024-09-10T10:46:44.891012+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881711+0000","last_change":"2024-09-10T10:46:44.879407+0000","last_active":"2024-09-10T10:46:44.881711+0000","last_peered":"2024-09-10T10:46:44.881711+0000","last_clean":"2024-09-10T10:46:44.881711+0000","last_became_active":"2024-09-10T10:46:44.879124+0000","last_became_peered":"2024-09-10T10:46:44.879124+0000","last_unstale":"2024-09-10T10:46:44.881711+0000","last_undegraded":"2024-09-10T10:46:44.881711+0000","last_fullsized":"2024-09-10T10:46:44.881711+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:14.843601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879528+0000","last_change":"2024-09-10T10:46:44.874639+0000","last_active":"2024-09-10T10:46:44.879528+0000","last_peered":"2024-09-10T10:46:44.879528+0000","last_clean":"2024-09-10T10:46:44.879528+0000","last_became_active":"2024-09-10T10:46:44.874387+0000","last_became_peered":"2024-09-10T10:46:44.874387+0000","last_unstale":"2024-09-10T10:46:44.879528+0000","last_undegraded":"2024-09-10T10:46:44.879528+0000","last_fullsized":"2024-09-10T10:46:44.879528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:38:57.911434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874284+0000","last_change":"2024-09-10T10:46:44.865791+0000","last_active":"2024-09-10T10:46:44.874284+0000","last_peered":"2024-09-10T10:46:44.874284+0000","last_clean":"2024-09-10T10:46:44.874284+0000","last_became_active":"2024-09-10T10:46:44.865491+0000","last_became_peered":"2024-09-10T10:46:44.865491+0000","last_unstale":"2024-09-10T10:46:44.874284+0000","last_undegraded":"2024-09-10T10:46:44.874284+0000","last_fullsized":"2024-09-10T10:46:44.874284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887094+0000","last_change":"2024-09-10T10:46:44.878815+0000","last_active":"2024-09-10T10:46:44.887094+0000","last_peered":"2024-09-10T10:46:44.887094+0000","last_clean":"2024-09-10T10:46:44.887094+0000","last_became_active":"2024-09-10T10:46:44.878192+0000","last_became_peered":"2024-09-10T10:46:44.878192+0000","last_unstale":"2024-09-10T10:46:44.887094+0000","last_undegraded":"2024-09-10T10:46:44.887094+0000","last_fullsized":"2024-09-10T10:46:44.887094+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879723+0000","last_change":"2024-09-10T10:46:44.871127+0000","last_active":"2024-09-10T10:46:44.879723+0000","last_peered":"2024-09-10T10:46:44.879723+0000","last_clean":"2024-09-10T10:46:44.879723+0000","last_became_active":"2024-09-10T10:46:44.870842+0000","last_became_peered":"2024-09-10T10:46:44.870842+0000","last_unstale":"2024-09-10T10:46:44.879723+0000","last_undegraded":"2024-09-10T10:46:44.879723+0000","last_fullsized":"2024-09-10T10:46:44.879723+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:36:14.121534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894528+0000","last_change":"2024-09-10T10:46:44.882561+0000","last_active":"2024-09-10T10:46:44.894528+0000","last_peered":"2024-09-10T10:46:44.894528+0000","last_clean":"2024-09-10T10:46:44.894528+0000","last_became_active":"2024-09-10T10:46:44.877515+0000","last_became_peered":"2024-09-10T10:46:44.877515+0000","last_unstale":"2024-09-10T10:46:44.894528+0000","last_undegraded":"2024-09-10T10:46:44.894528+0000","last_fullsized":"2024-09-10T10:46:44.894528+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:35.562296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811184+0000","last_change":"2024-09-10T10:46:44.869105+0000","last_active":"2024-09-10T10:46:46.811184+0000","last_peered":"2024-09-10T10:46:46.811184+0000","last_clean":"2024-09-10T10:46:46.811184+0000","last_became_active":"2024-09-10T10:46:44.868848+0000","last_became_peered":"2024-09-10T10:46:44.868848+0000","last_unstale":"2024-09-10T10:46:46.811184+0000","last_undegraded":"2024-09-10T10:46:46.811184+0000","last_fullsized":"2024-09-10T10:46:46.811184+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:39.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881741+0000","last_change":"2024-09-10T10:46:44.878483+0000","last_active":"2024-09-10T10:46:44.881741+0000","last_peered":"2024-09-10T10:46:44.881741+0000","last_clean":"2024-09-10T10:46:44.881741+0000","last_became_active":"2024-09-10T10:46:44.874186+0000","last_became_peered":"2024-09-10T10:46:44.874186+0000","last_unstale":"2024-09-10T10:46:44.881741+0000","last_undegraded":"2024-09-10T10:46:44.881741+0000","last_fullsized":"2024-09-10T10:46:44.881741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893869+0000","last_change":"2024-09-10T10:46:44.890973+0000","last_active":"2024-09-10T10:46:44.893869+0000","last_peered":"2024-09-10T10:46:44.893869+0000","last_clean":"2024-09-10T10:46:44.893869+0000","last_became_active":"2024-09-10T10:46:44.889471+0000","last_became_peered":"2024-09-10T10:46:44.889471+0000","last_unstale":"2024-09-10T10:46:44.893869+0000","last_undegraded":"2024-09-10T10:46:44.893869+0000","last_fullsized":"2024-09-10T10:46:44.893869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:28.051816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894600+0000","last_change":"2024-09-10T10:46:44.888201+0000","last_active":"2024-09-10T10:46:44.894600+0000","last_peered":"2024-09-10T10:46:44.894600+0000","last_clean":"2024-09-10T10:46:44.894600+0000","last_became_active":"2024-09-10T10:46:44.887970+0000","last_became_peered":"2024-09-10T10:46:44.887970+0000","last_unstale":"2024-09-10T10:46:44.894600+0000","last_undegraded":"2024-09-10T10:46:44.894600+0000","last_fullsized":"2024-09-10T10:46:44.894600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891947+0000","last_change":"2024-09-10T10:46:44.881612+0000","last_active":"2024-09-10T10:46:44.891947+0000","last_peered":"2024-09-10T10:46:44.891947+0000","last_clean":"2024-09-10T10:46:44.891947+0000","last_became_active":"2024-09-10T10:46:44.881250+0000","last_became_peered":"2024-09-10T10:46:44.881250+0000","last_unstale":"2024-09-10T10:46:44.891947+0000","last_undegraded":"2024-09-10T10:46:44.891947+0000","last_fullsized":"2024-09-10T10:46:44.891947+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:29:44.946566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891469+0000","last_change":"2024-09-10T10:46:44.878716+0000","last_active":"2024-09-10T10:46:44.891469+0000","last_peered":"2024-09-10T10:46:44.891469+0000","last_clean":"2024-09-10T10:46:44.891469+0000","last_became_active":"2024-09-10T10:46:44.878422+0000","last_became_peered":"2024-09-10T10:46:44.878422+0000","last_unstale":"2024-09-10T10:46:44.891469+0000","last_undegraded":"2024-09-10T10:46:44.891469+0000","last_fullsized":"2024-09-10T10:46:44.891469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:41.176611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.888369+0000","last_change":"2024-09-10T10:46:44.886054+0000","last_active":"2024-09-10T10:46:44.888369+0000","last_peered":"2024-09-10T10:46:44.888369+0000","last_clean":"2024-09-10T10:46:44.888369+0000","last_became_active":"2024-09-10T10:46:44.885269+0000","last_became_peered":"2024-09-10T10:46:44.885269+0000","last_unstale":"2024-09-10T10:46:44.888369+0000","last_undegraded":"2024-09-10T10:46:44.888369+0000","last_fullsized":"2024-09-10T10:46:44.888369+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874510+0000","last_change":"2024-09-10T10:46:44.869461+0000","last_active":"2024-09-10T10:46:44.874510+0000","last_peered":"2024-09-10T10:46:44.874510+0000","last_clean":"2024-09-10T10:46:44.874510+0000","last_became_active":"2024-09-10T10:46:44.869194+0000","last_became_peered":"2024-09-10T10:46:44.869194+0000","last_unstale":"2024-09-10T10:46:44.874510+0000","last_undegraded":"2024-09-10T10:46:44.874510+0000","last_fullsized":"2024-09-10T10:46:44.874510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887172+0000","last_change":"2024-09-10T10:46:44.879808+0000","last_active":"2024-09-10T10:46:44.887172+0000","last_peered":"2024-09-10T10:46:44.887172+0000","last_clean":"2024-09-10T10:46:44.887172+0000","last_became_active":"2024-09-10T10:46:44.879312+0000","last_became_peered":"2024-09-10T10:46:44.879312+0000","last_unstale":"2024-09-10T10:46:44.887172+0000","last_undegraded":"2024-09-10T10:46:44.887172+0000","last_fullsized":"2024-09-10T10:46:44.887172+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892504+0000","last_change":"2024-09-10T10:46:44.881613+0000","last_active":"2024-09-10T10:46:44.892504+0000","last_peered":"2024-09-10T10:46:44.892504+0000","last_clean":"2024-09-10T10:46:44.892504+0000","last_became_active":"2024-09-10T10:46:44.881356+0000","last_became_peered":"2024-09-10T10:46:44.881356+0000","last_unstale":"2024-09-10T10:46:44.892504+0000","last_undegraded":"2024-09-10T10:46:44.892504+0000","last_fullsized":"2024-09-10T10:46:44.892504+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:04.057439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.883453+0000","last_change":"2024-09-10T10:46:44.882083+0000","last_active":"2024-09-10T10:46:44.883453+0000","last_peered":"2024-09-10T10:46:44.883453+0000","last_clean":"2024-09-10T10:46:44.883453+0000","last_became_active":"2024-09-10T10:46:44.878774+0000","last_became_peered":"2024-09-10T10:46:44.878774+0000","last_unstale":"2024-09-10T10:46:44.883453+0000","last_undegraded":"2024-09-10T10:46:44.883453+0000","last_fullsized":"2024-09-10T10:46:44.883453+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890870+0000","last_change":"2024-09-10T10:46:44.884761+0000","last_active":"2024-09-10T10:46:44.890870+0000","last_peered":"2024-09-10T10:46:44.890870+0000","last_clean":"2024-09-10T10:46:44.890870+0000","last_became_active":"2024-09-10T10:46:44.884412+0000","last_became_peered":"2024-09-10T10:46:44.884412+0000","last_unstale":"2024-09-10T10:46:44.890870+0000","last_undegraded":"2024-09-10T10:46:44.890870+0000","last_fullsized":"2024-09-10T10:46:44.890870+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903682+0000","last_change":"2024-09-10T10:46:43.873120+0000","last_active":"2024-09-10T10:46:44.903682+0000","last_peered":"2024-09-10T10:46:44.903682+0000","last_clean":"2024-09-10T10:46:44.903682+0000","last_became_active":"2024-09-10T10:46:43.872682+0000","last_became_peered":"2024-09-10T10:46:43.872682+0000","last_unstale":"2024-09-10T10:46:44.903682+0000","last_undegraded":"2024-09-10T10:46:44.903682+0000","last_fullsized":"2024-09-10T10:46:44.903682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.538093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892606+0000","last_change":"2024-09-10T10:46:44.885626+0000","last_active":"2024-09-10T10:46:44.892606+0000","last_peered":"2024-09-10T10:46:44.892606+0000","last_clean":"2024-09-10T10:46:44.892606+0000","last_became_active":"2024-09-10T10:46:44.883452+0000","last_became_peered":"2024-09-10T10:46:44.883452+0000","last_unstale":"2024-09-10T10:46:44.892606+0000","last_undegraded":"2024-09-10T10:46:44.892606+0000","last_fullsized":"2024-09-10T10:46:44.892606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:57.060214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897771+0000","last_change":"2024-09-10T10:46:43.875822+0000","last_active":"2024-09-10T10:46:44.897771+0000","last_peered":"2024-09-10T10:46:44.897771+0000","last_clean":"2024-09-10T10:46:44.897771+0000","last_became_active":"2024-09-10T10:46:43.875557+0000","last_became_peered":"2024-09-10T10:46:43.875557+0000","last_unstale":"2024-09-10T10:46:44.897771+0000","last_undegraded":"2024-09-10T10:46:44.897771+0000","last_fullsized":"2024-09-10T10:46:44.897771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.628419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810719+0000","last_change":"2024-09-10T10:46:44.876687+0000","last_active":"2024-09-10T10:46:46.810719+0000","last_peered":"2024-09-10T10:46:46.810719+0000","last_clean":"2024-09-10T10:46:46.810719+0000","last_became_active":"2024-09-10T10:46:44.875941+0000","last_became_peered":"2024-09-10T10:46:44.875941+0000","last_unstale":"2024-09-10T10:46:46.810719+0000","last_undegraded":"2024-09-10T10:46:46.810719+0000","last_fullsized":"2024-09-10T10:46:46.810719+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:24.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.900588+0000","last_change":"2024-09-10T10:46:43.876519+0000","last_active":"2024-09-10T10:46:44.900588+0000","last_peered":"2024-09-10T10:46:44.900588+0000","last_clean":"2024-09-10T10:46:44.900588+0000","last_became_active":"2024-09-10T10:46:43.876247+0000","last_became_peered":"2024-09-10T10:46:43.876247+0000","last_unstale":"2024-09-10T10:46:44.900588+0000","last_undegraded":"2024-09-10T10:46:44.900588+0000","last_fullsized":"2024-09-10T10:46:44.900588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:00.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887341+0000","last_change":"2024-09-10T10:46:44.877202+0000","last_active":"2024-09-10T10:46:44.887341+0000","last_peered":"2024-09-10T10:46:44.887341+0000","last_clean":"2024-09-10T10:46:44.887341+0000","last_became_active":"2024-09-10T10:46:44.876159+0000","last_became_peered":"2024-09-10T10:46:44.876159+0000","last_unstale":"2024-09-10T10:46:44.887341+0000","last_undegraded":"2024-09-10T10:46:44.887341+0000","last_fullsized":"2024-09-10T10:46:44.887341+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:01.241200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810414+0000","last_change":"2024-09-10T10:46:43.880092+0000","last_active":"2024-09-10T10:46:46.810414+0000","last_peered":"2024-09-10T10:46:46.810414+0000","last_clean":"2024-09-10T10:46:46.810414+0000","last_became_active":"2024-09-10T10:46:43.876472+0000","last_became_peered":"2024-09-10T10:46:43.876472+0000","last_unstale":"2024-09-10T10:46:46.810414+0000","last_undegraded":"2024-09-10T10:46:46.810414+0000","last_fullsized":"2024-09-10T10:46:46.810414+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:16.961700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.880924+0000","last_change":"2024-09-10T10:46:44.878110+0000","last_active":"2024-09-10T10:46:44.880924+0000","last_peered":"2024-09-10T10:46:44.880924+0000","last_clean":"2024-09-10T10:46:44.880924+0000","last_became_active":"2024-09-10T10:46:44.877841+0000","last_became_peered":"2024-09-10T10:46:44.877841+0000","last_unstale":"2024-09-10T10:46:44.880924+0000","last_undegraded":"2024-09-10T10:46:44.880924+0000","last_fullsized":"2024-09-10T10:46:44.880924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:45.501475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898049+0000","last_change":"2024-09-10T10:46:43.875159+0000","last_active":"2024-09-10T10:46:44.898049+0000","last_peered":"2024-09-10T10:46:44.898049+0000","last_clean":"2024-09-10T10:46:44.898049+0000","last_became_active":"2024-09-10T10:46:43.874602+0000","last_became_peered":"2024-09-10T10:46:43.874602+0000","last_unstale":"2024-09-10T10:46:44.898049+0000","last_undegraded":"2024-09-10T10:46:44.898049+0000","last_fullsized":"2024-09-10T10:46:44.898049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:48.767108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889322+0000","last_change":"2024-09-10T10:46:44.888134+0000","last_active":"2024-09-10T10:46:44.889322+0000","last_peered":"2024-09-10T10:46:44.889322+0000","last_clean":"2024-09-10T10:46:44.889322+0000","last_became_active":"2024-09-10T10:46:44.887851+0000","last_became_peered":"2024-09-10T10:46:44.887851+0000","last_unstale":"2024-09-10T10:46:44.889322+0000","last_undegraded":"2024-09-10T10:46:44.889322+0000","last_fullsized":"2024-09-10T10:46:44.889322+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811213+0000","last_change":"2024-09-10T10:46:43.878707+0000","last_active":"2024-09-10T10:46:46.811213+0000","last_peered":"2024-09-10T10:46:46.811213+0000","last_clean":"2024-09-10T10:46:46.811213+0000","last_became_active":"2024-09-10T10:46:43.878296+0000","last_became_peered":"2024-09-10T10:46:43.878296+0000","last_unstale":"2024-09-10T10:46:46.811213+0000","last_undegraded":"2024-09-10T10:46:46.811213+0000","last_fullsized":"2024-09-10T10:46:46.811213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:41.855498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.896764+0000","last_change":"2024-09-10T10:46:44.893318+0000","last_active":"2024-09-10T10:46:44.896764+0000","last_peered":"2024-09-10T10:46:44.896764+0000","last_clean":"2024-09-10T10:46:44.896764+0000","last_became_active":"2024-09-10T10:46:44.893039+0000","last_became_peered":"2024-09-10T10:46:44.893039+0000","last_unstale":"2024-09-10T10:46:44.896764+0000","last_undegraded":"2024-09-10T10:46:44.896764+0000","last_fullsized":"2024-09-10T10:46:44.896764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870255+0000","last_change":"2024-09-10T10:46:43.879425+0000","last_active":"2024-09-10T10:46:44.870255+0000","last_peered":"2024-09-10T10:46:44.870255+0000","last_clean":"2024-09-10T10:46:44.870255+0000","last_became_active":"2024-09-10T10:46:43.879154+0000","last_became_peered":"2024-09-10T10:46:43.879154+0000","last_unstale":"2024-09-10T10:46:44.870255+0000","last_undegraded":"2024-09-10T10:46:44.870255+0000","last_fullsized":"2024-09-10T10:46:44.870255+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:41.361020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879467+0000","last_change":"2024-09-10T10:46:44.871663+0000","last_active":"2024-09-10T10:46:44.879467+0000","last_peered":"2024-09-10T10:46:44.879467+0000","last_clean":"2024-09-10T10:46:44.879467+0000","last_became_active":"2024-09-10T10:46:44.869465+0000","last_became_peered":"2024-09-10T10:46:44.869465+0000","last_unstale":"2024-09-10T10:46:44.879467+0000","last_undegraded":"2024-09-10T10:46:44.879467+0000","last_fullsized":"2024-09-10T10:46:44.879467+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:49:50.834542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.857464+0000","last_change":"2024-09-10T10:46:43.870161+0000","last_active":"2024-09-10T10:46:44.857464+0000","last_peered":"2024-09-10T10:46:44.857464+0000","last_clean":"2024-09-10T10:46:44.857464+0000","last_became_active":"2024-09-10T10:46:43.869918+0000","last_became_peered":"2024-09-10T10:46:43.869918+0000","last_unstale":"2024-09-10T10:46:44.857464+0000","last_undegraded":"2024-09-10T10:46:44.857464+0000","last_fullsized":"2024-09-10T10:46:44.857464+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:11.275462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887228+0000","last_change":"2024-09-10T10:46:44.884495+0000","last_active":"2024-09-10T10:46:44.887228+0000","last_peered":"2024-09-10T10:46:44.887228+0000","last_clean":"2024-09-10T10:46:44.887228+0000","last_became_active":"2024-09-10T10:46:44.884231+0000","last_became_peered":"2024-09-10T10:46:44.884231+0000","last_unstale":"2024-09-10T10:46:44.887228+0000","last_undegraded":"2024-09-10T10:46:44.887228+0000","last_fullsized":"2024-09-10T10:46:44.887228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.858268+0000","last_change":"2024-09-10T10:46:43.867286+0000","last_active":"2024-09-10T10:46:44.858268+0000","last_peered":"2024-09-10T10:46:44.858268+0000","last_clean":"2024-09-10T10:46:44.858268+0000","last_became_active":"2024-09-10T10:46:43.867053+0000","last_became_peered":"2024-09-10T10:46:43.867053+0000","last_unstale":"2024-09-10T10:46:44.858268+0000","last_undegraded":"2024-09-10T10:46:44.858268+0000","last_fullsized":"2024-09-10T10:46:44.858268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.574114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881233+0000","last_change":"2024-09-10T10:46:44.870275+0000","last_active":"2024-09-10T10:46:44.881233+0000","last_peered":"2024-09-10T10:46:44.881233+0000","last_clean":"2024-09-10T10:46:44.881233+0000","last_became_active":"2024-09-10T10:46:44.869919+0000","last_became_peered":"2024-09-10T10:46:44.869919+0000","last_unstale":"2024-09-10T10:46:44.881233+0000","last_undegraded":"2024-09-10T10:46:44.881233+0000","last_fullsized":"2024-09-10T10:46:44.881233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:32.393495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867247+0000","last_change":"2024-09-10T10:46:43.881828+0000","last_active":"2024-09-10T10:46:44.867247+0000","last_peered":"2024-09-10T10:46:44.867247+0000","last_clean":"2024-09-10T10:46:44.867247+0000","last_became_active":"2024-09-10T10:46:43.881524+0000","last_became_peered":"2024-09-10T10:46:43.881524+0000","last_unstale":"2024-09-10T10:46:44.867247+0000","last_undegraded":"2024-09-10T10:46:44.867247+0000","last_fullsized":"2024-09-10T10:46:44.867247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:18.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890526+0000","last_change":"2024-09-10T10:46:44.877148+0000","last_active":"2024-09-10T10:46:44.890526+0000","last_peered":"2024-09-10T10:46:44.890526+0000","last_clean":"2024-09-10T10:46:44.890526+0000","last_became_active":"2024-09-10T10:46:44.876860+0000","last_became_peered":"2024-09-10T10:46:44.876860+0000","last_unstale":"2024-09-10T10:46:44.890526+0000","last_undegraded":"2024-09-10T10:46:44.890526+0000","last_fullsized":"2024-09-10T10:46:44.890526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:08.920272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.904206+0000","last_change":"2024-09-10T10:46:43.874911+0000","last_active":"2024-09-10T10:46:44.904206+0000","last_peered":"2024-09-10T10:46:44.904206+0000","last_clean":"2024-09-10T10:46:44.904206+0000","last_became_active":"2024-09-10T10:46:43.874629+0000","last_became_peered":"2024-09-10T10:46:43.874629+0000","last_unstale":"2024-09-10T10:46:44.904206+0000","last_undegraded":"2024-09-10T10:46:44.904206+0000","last_fullsized":"2024-09-10T10:46:44.904206+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:31.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.893676+0000","last_change":"2024-09-10T10:46:44.889982+0000","last_active":"2024-09-10T10:46:44.893676+0000","last_peered":"2024-09-10T10:46:44.893676+0000","last_clean":"2024-09-10T10:46:44.893676+0000","last_became_active":"2024-09-10T10:46:44.889481+0000","last_became_peered":"2024-09-10T10:46:44.889481+0000","last_unstale":"2024-09-10T10:46:44.893676+0000","last_undegraded":"2024-09-10T10:46:44.893676+0000","last_fullsized":"2024-09-10T10:46:44.893676+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:22:42.436804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810449+0000","last_change":"2024-09-10T10:46:43.876633+0000","last_active":"2024-09-10T10:46:46.810449+0000","last_peered":"2024-09-10T10:46:46.810449+0000","last_clean":"2024-09-10T10:46:46.810449+0000","last_became_active":"2024-09-10T10:46:43.876340+0000","last_became_peered":"2024-09-10T10:46:43.876340+0000","last_unstale":"2024-09-10T10:46:46.810449+0000","last_undegraded":"2024-09-10T10:46:46.810449+0000","last_fullsized":"2024-09-10T10:46:46.810449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:13.563191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894670+0000","last_change":"2024-09-10T10:46:44.886563+0000","last_active":"2024-09-10T10:46:44.894670+0000","last_peered":"2024-09-10T10:46:44.894670+0000","last_clean":"2024-09-10T10:46:44.894670+0000","last_became_active":"2024-09-10T10:46:44.886299+0000","last_became_peered":"2024-09-10T10:46:44.886299+0000","last_unstale":"2024-09-10T10:46:44.894670+0000","last_undegraded":"2024-09-10T10:46:44.894670+0000","last_fullsized":"2024-09-10T10:46:44.894670+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897647+0000","last_change":"2024-09-10T10:46:43.868983+0000","last_active":"2024-09-10T10:46:44.897647+0000","last_peered":"2024-09-10T10:46:44.897647+0000","last_clean":"2024-09-10T10:46:44.897647+0000","last_became_active":"2024-09-10T10:46:43.868631+0000","last_became_peered":"2024-09-10T10:46:43.868631+0000","last_unstale":"2024-09-10T10:46:44.897647+0000","last_undegraded":"2024-09-10T10:46:44.897647+0000","last_fullsized":"2024-09-10T10:46:44.897647+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892284+0000","last_change":"2024-09-10T10:46:44.885879+0000","last_active":"2024-09-10T10:46:44.892284+0000","last_peered":"2024-09-10T10:46:44.892284+0000","last_clean":"2024-09-10T10:46:44.892284+0000","last_became_active":"2024-09-10T10:46:44.885500+0000","last_became_peered":"2024-09-10T10:46:44.885500+0000","last_unstale":"2024-09-10T10:46:44.892284+0000","last_undegraded":"2024-09-10T10:46:44.892284+0000","last_fullsized":"2024-09-10T10:46:44.892284+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874282+0000","last_change":"2024-09-10T10:46:43.879565+0000","last_active":"2024-09-10T10:46:44.874282+0000","last_peered":"2024-09-10T10:46:44.874282+0000","last_clean":"2024-09-10T10:46:44.874282+0000","last_became_active":"2024-09-10T10:46:43.879254+0000","last_became_peered":"2024-09-10T10:46:43.879254+0000","last_unstale":"2024-09-10T10:46:44.874282+0000","last_undegraded":"2024-09-10T10:46:44.874282+0000","last_fullsized":"2024-09-10T10:46:44.874282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881138+0000","last_change":"2024-09-10T10:46:44.878593+0000","last_active":"2024-09-10T10:46:44.881138+0000","last_peered":"2024-09-10T10:46:44.881138+0000","last_clean":"2024-09-10T10:46:44.881138+0000","last_became_active":"2024-09-10T10:46:44.878283+0000","last_became_peered":"2024-09-10T10:46:44.878283+0000","last_unstale":"2024-09-10T10:46:44.881138+0000","last_undegraded":"2024-09-10T10:46:44.881138+0000","last_fullsized":"2024-09-10T10:46:44.881138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:30:22.548504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.865622+0000","last_change":"2024-09-10T10:46:43.877389+0000","last_active":"2024-09-10T10:46:44.865622+0000","last_peered":"2024-09-10T10:46:44.865622+0000","last_clean":"2024-09-10T10:46:44.865622+0000","last_became_active":"2024-09-10T10:46:43.876235+0000","last_became_peered":"2024-09-10T10:46:43.876235+0000","last_unstale":"2024-09-10T10:46:44.865622+0000","last_undegraded":"2024-09-10T10:46:44.865622+0000","last_fullsized":"2024-09-10T10:46:44.865622+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:52.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881110+0000","last_change":"2024-09-10T10:46:44.878187+0000","last_active":"2024-09-10T10:46:44.881110+0000","last_peered":"2024-09-10T10:46:44.881110+0000","last_clean":"2024-09-10T10:46:44.881110+0000","last_became_active":"2024-09-10T10:46:44.877922+0000","last_became_peered":"2024-09-10T10:46:44.877922+0000","last_unstale":"2024-09-10T10:46:44.881110+0000","last_undegraded":"2024-09-10T10:46:44.881110+0000","last_fullsized":"2024-09-10T10:46:44.881110+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.902316+0000","last_change":"2024-09-10T10:46:43.879721+0000","last_active":"2024-09-10T10:46:44.902316+0000","last_peered":"2024-09-10T10:46:44.902316+0000","last_clean":"2024-09-10T10:46:44.902316+0000","last_became_active":"2024-09-10T10:46:43.879448+0000","last_became_peered":"2024-09-10T10:46:43.879448+0000","last_unstale":"2024-09-10T10:46:44.902316+0000","last_undegraded":"2024-09-10T10:46:44.902316+0000","last_fullsized":"2024-09-10T10:46:44.902316+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.882849+0000","last_change":"2024-09-10T10:46:44.871707+0000","last_active":"2024-09-10T10:46:44.882849+0000","last_peered":"2024-09-10T10:46:44.882849+0000","last_clean":"2024-09-10T10:46:44.882849+0000","last_became_active":"2024-09-10T10:46:44.871400+0000","last_became_peered":"2024-09-10T10:46:44.871400+0000","last_unstale":"2024-09-10T10:46:44.882849+0000","last_undegraded":"2024-09-10T10:46:44.882849+0000","last_fullsized":"2024-09-10T10:46:44.882849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.898284+0000","last_change":"2024-09-10T10:46:43.866850+0000","last_active":"2024-09-10T10:46:44.898284+0000","last_peered":"2024-09-10T10:46:44.898284+0000","last_clean":"2024-09-10T10:46:44.898284+0000","last_became_active":"2024-09-10T10:46:43.866575+0000","last_became_peered":"2024-09-10T10:46:43.866575+0000","last_unstale":"2024-09-10T10:46:44.898284+0000","last_undegraded":"2024-09-10T10:46:44.898284+0000","last_fullsized":"2024-09-10T10:46:44.898284+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:07.913685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.890441+0000","last_change":"2024-09-10T10:46:44.875025+0000","last_active":"2024-09-10T10:46:44.890441+0000","last_peered":"2024-09-10T10:46:44.890441+0000","last_clean":"2024-09-10T10:46:44.890441+0000","last_became_active":"2024-09-10T10:46:44.873173+0000","last_became_peered":"2024-09-10T10:46:44.873173+0000","last_unstale":"2024-09-10T10:46:44.890441+0000","last_undegraded":"2024-09-10T10:46:44.890441+0000","last_fullsized":"2024-09-10T10:46:44.890441+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:02.580701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.903260+0000","last_change":"2024-09-10T10:46:43.871807+0000","last_active":"2024-09-10T10:46:44.903260+0000","last_peered":"2024-09-10T10:46:44.903260+0000","last_clean":"2024-09-10T10:46:44.903260+0000","last_became_active":"2024-09-10T10:46:43.871540+0000","last_became_peered":"2024-09-10T10:46:43.871540+0000","last_unstale":"2024-09-10T10:46:44.903260+0000","last_undegraded":"2024-09-10T10:46:44.903260+0000","last_fullsized":"2024-09-10T10:46:44.903260+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:34.555709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.894618+0000","last_change":"2024-09-10T10:46:44.888440+0000","last_active":"2024-09-10T10:46:44.894618+0000","last_peered":"2024-09-10T10:46:44.894618+0000","last_clean":"2024-09-10T10:46:44.894618+0000","last_became_active":"2024-09-10T10:46:44.888154+0000","last_became_peered":"2024-09-10T10:46:44.888154+0000","last_unstale":"2024-09-10T10:46:44.894618+0000","last_undegraded":"2024-09-10T10:46:44.894618+0000","last_fullsized":"2024-09-10T10:46:44.894618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.905983+0000","last_change":"2024-09-10T10:46:43.876691+0000","last_active":"2024-09-10T10:46:44.905983+0000","last_peered":"2024-09-10T10:46:44.905983+0000","last_clean":"2024-09-10T10:46:44.905983+0000","last_became_active":"2024-09-10T10:46:43.876227+0000","last_became_peered":"2024-09-10T10:46:43.876227+0000","last_unstale":"2024-09-10T10:46:44.905983+0000","last_undegraded":"2024-09-10T10:46:44.905983+0000","last_fullsized":"2024-09-10T10:46:44.905983+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:31.964863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892464+0000","last_change":"2024-09-10T10:46:44.882771+0000","last_active":"2024-09-10T10:46:44.892464+0000","last_peered":"2024-09-10T10:46:44.892464+0000","last_clean":"2024-09-10T10:46:44.892464+0000","last_became_active":"2024-09-10T10:46:44.881897+0000","last_became_peered":"2024-09-10T10:46:44.881897+0000","last_unstale":"2024-09-10T10:46:44.892464+0000","last_undegraded":"2024-09-10T10:46:44.892464+0000","last_fullsized":"2024-09-10T10:46:44.892464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:27:22.411021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867582+0000","last_change":"2024-09-10T10:46:43.873156+0000","last_active":"2024-09-10T10:46:44.867582+0000","last_peered":"2024-09-10T10:46:44.867582+0000","last_clean":"2024-09-10T10:46:44.867582+0000","last_became_active":"2024-09-10T10:46:43.872889+0000","last_became_peered":"2024-09-10T10:46:43.872889+0000","last_unstale":"2024-09-10T10:46:44.867582+0000","last_undegraded":"2024-09-10T10:46:44.867582+0000","last_fullsized":"2024-09-10T10:46:44.867582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810286+0000","last_change":"2024-09-10T10:46:44.878982+0000","last_active":"2024-09-10T10:46:46.810286+0000","last_peered":"2024-09-10T10:46:46.810286+0000","last_clean":"2024-09-10T10:46:46.810286+0000","last_became_active":"2024-09-10T10:46:44.878626+0000","last_became_peered":"2024-09-10T10:46:44.878626+0000","last_unstale":"2024-09-10T10:46:46.810286+0000","last_undegraded":"2024-09-10T10:46:46.810286+0000","last_fullsized":"2024-09-10T10:46:46.810286+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:23.712304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811378+0000","last_change":"2024-09-10T10:46:43.866904+0000","last_active":"2024-09-10T10:46:46.811378+0000","last_peered":"2024-09-10T10:46:46.811378+0000","last_clean":"2024-09-10T10:46:46.811378+0000","last_became_active":"2024-09-10T10:46:43.866582+0000","last_became_peered":"2024-09-10T10:46:43.866582+0000","last_unstale":"2024-09-10T10:46:46.811378+0000","last_undegraded":"2024-09-10T10:46:46.811378+0000","last_fullsized":"2024-09-10T10:46:46.811378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:23.463417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872927+0000","last_change":"2024-09-10T10:46:44.868747+0000","last_active":"2024-09-10T10:46:44.872927+0000","last_peered":"2024-09-10T10:46:44.872927+0000","last_clean":"2024-09-10T10:46:44.872927+0000","last_became_active":"2024-09-10T10:46:44.868455+0000","last_became_peered":"2024-09-10T10:46:44.868455+0000","last_unstale":"2024-09-10T10:46:44.872927+0000","last_undegraded":"2024-09-10T10:46:44.872927+0000","last_fullsized":"2024-09-10T10:46:44.872927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.874479+0000","last_change":"2024-09-10T10:46:43.876827+0000","last_active":"2024-09-10T10:46:44.874479+0000","last_peered":"2024-09-10T10:46:44.874479+0000","last_clean":"2024-09-10T10:46:44.874479+0000","last_became_active":"2024-09-10T10:46:43.875591+0000","last_became_peered":"2024-09-10T10:46:43.875591+0000","last_unstale":"2024-09-10T10:46:44.874479+0000","last_undegraded":"2024-09-10T10:46:44.874479+0000","last_fullsized":"2024-09-10T10:46:44.874479+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.889806+0000","last_change":"2024-09-10T10:46:44.883361+0000","last_active":"2024-09-10T10:46:44.889806+0000","last_peered":"2024-09-10T10:46:44.889806+0000","last_clean":"2024-09-10T10:46:44.889806+0000","last_became_active":"2024-09-10T10:46:44.879641+0000","last_became_peered":"2024-09-10T10:46:44.879641+0000","last_unstale":"2024-09-10T10:46:44.889806+0000","last_undegraded":"2024-09-10T10:46:44.889806+0000","last_fullsized":"2024-09-10T10:46:44.889806+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810132+0000","last_change":"2024-09-10T10:46:43.875319+0000","last_active":"2024-09-10T10:46:46.810132+0000","last_peered":"2024-09-10T10:46:46.810132+0000","last_clean":"2024-09-10T10:46:46.810132+0000","last_became_active":"2024-09-10T10:46:43.873628+0000","last_became_peered":"2024-09-10T10:46:43.873628+0000","last_unstale":"2024-09-10T10:46:46.810132+0000","last_undegraded":"2024-09-10T10:46:46.810132+0000","last_fullsized":"2024-09-10T10:46:46.810132+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:11.118440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879741+0000","last_change":"2024-09-10T10:46:44.873964+0000","last_active":"2024-09-10T10:46:44.879741+0000","last_peered":"2024-09-10T10:46:44.879741+0000","last_clean":"2024-09-10T10:46:44.879741+0000","last_became_active":"2024-09-10T10:46:44.873324+0000","last_became_peered":"2024-09-10T10:46:44.873324+0000","last_unstale":"2024-09-10T10:46:44.879741+0000","last_undegraded":"2024-09-10T10:46:44.879741+0000","last_fullsized":"2024-09-10T10:46:44.879741+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:21:46.801922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.897908+0000","last_change":"2024-09-10T10:46:43.880123+0000","last_active":"2024-09-10T10:46:44.897908+0000","last_peered":"2024-09-10T10:46:44.897908+0000","last_clean":"2024-09-10T10:46:44.897908+0000","last_became_active":"2024-09-10T10:46:43.879861+0000","last_became_peered":"2024-09-10T10:46:43.879861+0000","last_unstale":"2024-09-10T10:46:44.897908+0000","last_undegraded":"2024-09-10T10:46:44.897908+0000","last_fullsized":"2024-09-10T10:46:44.897908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:40.170023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881256+0000","last_change":"2024-09-10T10:46:44.875117+0000","last_active":"2024-09-10T10:46:44.881256+0000","last_peered":"2024-09-10T10:46:44.881256+0000","last_clean":"2024-09-10T10:46:44.881256+0000","last_became_active":"2024-09-10T10:46:44.874858+0000","last_became_peered":"2024-09-10T10:46:44.874858+0000","last_unstale":"2024-09-10T10:46:44.881256+0000","last_undegraded":"2024-09-10T10:46:44.881256+0000","last_fullsized":"2024-09-10T10:46:44.881256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:01:08.230635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.859204+0000","last_change":"2024-09-10T10:46:43.870732+0000","last_active":"2024-09-10T10:46:44.859204+0000","last_peered":"2024-09-10T10:46:44.859204+0000","last_clean":"2024-09-10T10:46:44.859204+0000","last_became_active":"2024-09-10T10:46:43.870484+0000","last_became_peered":"2024-09-10T10:46:43.870484+0000","last_unstale":"2024-09-10T10:46:44.859204+0000","last_undegraded":"2024-09-10T10:46:44.859204+0000","last_fullsized":"2024-09-10T10:46:44.859204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:05.738817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.881485+0000","last_change":"2024-09-10T10:46:44.875783+0000","last_active":"2024-09-10T10:46:44.881485+0000","last_peered":"2024-09-10T10:46:44.881485+0000","last_clean":"2024-09-10T10:46:44.881485+0000","last_became_active":"2024-09-10T10:46:44.875511+0000","last_became_peered":"2024-09-10T10:46:44.875511+0000","last_unstale":"2024-09-10T10:46:44.881485+0000","last_undegraded":"2024-09-10T10:46:44.881485+0000","last_fullsized":"2024-09-10T10:46:44.881485+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:52:37.571008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810667+0000","last_change":"2024-09-10T10:46:43.874338+0000","last_active":"2024-09-10T10:46:46.810667+0000","last_peered":"2024-09-10T10:46:46.810667+0000","last_clean":"2024-09-10T10:46:46.810667+0000","last_became_active":"2024-09-10T10:46:43.874013+0000","last_became_peered":"2024-09-10T10:46:43.874013+0000","last_unstale":"2024-09-10T10:46:46.810667+0000","last_undegraded":"2024-09-10T10:46:46.810667+0000","last_fullsized":"2024-09-10T10:46:46.810667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:55.976080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.892438+0000","last_change":"2024-09-10T10:46:44.882540+0000","last_active":"2024-09-10T10:46:44.892438+0000","last_peered":"2024-09-10T10:46:44.892438+0000","last_clean":"2024-09-10T10:46:44.892438+0000","last_became_active":"2024-09-10T10:46:44.881322+0000","last_became_peered":"2024-09-10T10:46:44.881322+0000","last_unstale":"2024-09-10T10:46:44.892438+0000","last_undegraded":"2024-09-10T10:46:44.892438+0000","last_fullsized":"2024-09-10T10:46:44.892438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:12:05.028664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867719+0000","last_change":"2024-09-10T10:46:43.880848+0000","last_active":"2024-09-10T10:46:44.867719+0000","last_peered":"2024-09-10T10:46:44.867719+0000","last_clean":"2024-09-10T10:46:44.867719+0000","last_became_active":"2024-09-10T10:46:43.880588+0000","last_became_peered":"2024-09-10T10:46:43.880588+0000","last_unstale":"2024-09-10T10:46:44.867719+0000","last_undegraded":"2024-09-10T10:46:44.867719+0000","last_fullsized":"2024-09-10T10:46:44.867719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.887215+0000","last_change":"2024-09-10T10:46:44.879311+0000","last_active":"2024-09-10T10:46:44.887215+0000","last_peered":"2024-09-10T10:46:44.887215+0000","last_clean":"2024-09-10T10:46:44.887215+0000","last_became_active":"2024-09-10T10:46:44.867480+0000","last_became_peered":"2024-09-10T10:46:44.867480+0000","last_unstale":"2024-09-10T10:46:44.887215+0000","last_undegraded":"2024-09-10T10:46:44.887215+0000","last_fullsized":"2024-09-10T10:46:44.887215+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.866776+0000","last_change":"2024-09-10T10:46:43.878330+0000","last_active":"2024-09-10T10:46:44.866776+0000","last_peered":"2024-09-10T10:46:44.866776+0000","last_clean":"2024-09-10T10:46:44.866776+0000","last_became_active":"2024-09-10T10:46:43.877459+0000","last_became_peered":"2024-09-10T10:46:43.877459+0000","last_unstale":"2024-09-10T10:46:44.866776+0000","last_undegraded":"2024-09-10T10:46:44.866776+0000","last_fullsized":"2024-09-10T10:46:44.866776+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:13.837013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.872902+0000","last_change":"2024-09-10T10:46:44.868549+0000","last_active":"2024-09-10T10:46:44.872902+0000","last_peered":"2024-09-10T10:46:44.872902+0000","last_clean":"2024-09-10T10:46:44.872902+0000","last_became_active":"2024-09-10T10:46:44.868232+0000","last_became_peered":"2024-09-10T10:46:44.868232+0000","last_unstale":"2024-09-10T10:46:44.872902+0000","last_undegraded":"2024-09-10T10:46:44.872902+0000","last_fullsized":"2024-09-10T10:46:44.872902+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.870340+0000","last_change":"2024-09-10T10:46:43.879611+0000","last_active":"2024-09-10T10:46:44.870340+0000","last_peered":"2024-09-10T10:46:44.870340+0000","last_clean":"2024-09-10T10:46:44.870340+0000","last_became_active":"2024-09-10T10:46:43.879308+0000","last_became_peered":"2024-09-10T10:46:43.879308+0000","last_unstale":"2024-09-10T10:46:44.870340+0000","last_undegraded":"2024-09-10T10:46:44.870340+0000","last_fullsized":"2024-09-10T10:46:44.870340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:57.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811151+0000","last_change":"2024-09-10T10:46:44.868446+0000","last_active":"2024-09-10T10:46:46.811151+0000","last_peered":"2024-09-10T10:46:46.811151+0000","last_clean":"2024-09-10T10:46:46.811151+0000","last_became_active":"2024-09-10T10:46:44.868167+0000","last_became_peered":"2024-09-10T10:46:44.868167+0000","last_unstale":"2024-09-10T10:46:46.811151+0000","last_undegraded":"2024-09-10T10:46:46.811151+0000","last_fullsized":"2024-09-10T10:46:46.811151+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:39.838300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.867820+0000","last_change":"2024-09-10T10:46:43.879656+0000","last_active":"2024-09-10T10:46:44.867820+0000","last_peered":"2024-09-10T10:46:44.867820+0000","last_clean":"2024-09-10T10:46:44.867820+0000","last_became_active":"2024-09-10T10:46:43.879098+0000","last_became_peered":"2024-09-10T10:46:43.879098+0000","last_unstale":"2024-09-10T10:46:44.867820+0000","last_undegraded":"2024-09-10T10:46:44.867820+0000","last_fullsized":"2024-09-10T10:46:44.867820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.879442+0000","last_change":"2024-09-10T10:46:44.869395+0000","last_active":"2024-09-10T10:46:44.879442+0000","last_peered":"2024-09-10T10:46:44.879442+0000","last_clean":"2024-09-10T10:46:44.879442+0000","last_became_active":"2024-09-10T10:46:44.869065+0000","last_became_peered":"2024-09-10T10:46:44.869065+0000","last_unstale":"2024-09-10T10:46:44.879442+0000","last_undegraded":"2024-09-10T10:46:44.879442+0000","last_fullsized":"2024-09-10T10:46:44.879442+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:19:52.312041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810175+0000","last_change":"2024-09-10T10:46:43.870879+0000","last_active":"2024-09-10T10:46:46.810175+0000","last_peered":"2024-09-10T10:46:46.810175+0000","last_clean":"2024-09-10T10:46:46.810175+0000","last_became_active":"2024-09-10T10:46:43.870201+0000","last_became_peered":"2024-09-10T10:46:43.870201+0000","last_unstale":"2024-09-10T10:46:46.810175+0000","last_undegraded":"2024-09-10T10:46:46.810175+0000","last_fullsized":"2024-09-10T10:46:46.810175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:33.476365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.891869+0000","last_change":"2024-09-10T10:46:44.885648+0000","last_active":"2024-09-10T10:46:44.891869+0000","last_peered":"2024-09-10T10:46:44.891869+0000","last_clean":"2024-09-10T10:46:44.891869+0000","last_became_active":"2024-09-10T10:46:44.885219+0000","last_became_peered":"2024-09-10T10:46:44.885219+0000","last_unstale":"2024-09-10T10:46:44.891869+0000","last_undegraded":"2024-09-10T10:46:44.891869+0000","last_fullsized":"2024-09-10T10:46:44.891869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:19.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810294+0000","last_change":"2024-09-10T10:46:43.871118+0000","last_active":"2024-09-10T10:46:46.810294+0000","last_peered":"2024-09-10T10:46:46.810294+0000","last_clean":"2024-09-10T10:46:46.810294+0000","last_became_active":"2024-09-10T10:46:43.870597+0000","last_became_peered":"2024-09-10T10:46:43.870597+0000","last_unstale":"2024-09-10T10:46:46.810294+0000","last_undegraded":"2024-09-10T10:46:46.810294+0000","last_fullsized":"2024-09-10T10:46:46.810294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:04.466041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":257,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T10:46:44.871099+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:44.871099+0000","last_peered":"2024-09-10T10:46:44.871099+0000","last_clean":"2024-09-10T10:46:44.871099+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:44.871099+0000","last_undegraded":"2024-09-10T10:46:44.871099+0000","last_fullsized":"2024-09-10T10:46:44.871099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":367029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":831488,"data_stored":374778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":374356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":835584,"data_stored":374252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1425408,"data_stored":964598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":827392,"data_stored":374196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28452,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343388,"statfs":{"total":96636764160,"available":96607629312,"internally_reserved":0,"allocated":1454080,"data_stored":972980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:50.841 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:46:50.841+0000 7fe03c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:46:53.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:46:53.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.941+0000 7f9d37800640 1 Processor -- start 2024-09-10T10:46:53.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.941+0000 7f9d37800640 1 -- start start 2024-09-10T10:46:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9d30072320 0x7f9d300cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9d30072de0 0x7f9d3006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072700 0x7f9d3006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9d30146140 con 0x7f9d30072de0 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9d300cd740 con 0x7f9d30072320 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d37800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300cda20 con 0x7f9d30072700 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9d30072de0 0x7f9d3006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9d30072de0 0x7f9d3006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47952/0 (socket says 172.21.15.28:47952) 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 -- 172.21.15.28:0/910330668 learned_addr learned my addr 172.21.15.28:0/910330668 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d34e00640 1 --2- 172.21.15.28:0/910330668 >> v2:172.21.15.138:3300/0 conn(0x7f9d30072320 0x7f9d300cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d35800640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072700 0x7f9d3006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 -- 172.21.15.28:0/910330668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072700 msgr2=0x7f9d3006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072700 0x7f9d3006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 -- 172.21.15.28:0/910330668 >> v2:172.21.15.138:3300/0 conn(0x7f9d30072320 msgr2=0x7f9d300cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 --2- 172.21.15.28:0/910330668 >> v2:172.21.15.138:3300/0 conn(0x7f9d30072320 0x7f9d300cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.943+0000 7f9d27e00640 1 -- 172.21.15.28:0/910330668 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300cdd00 con 0x7f9d30072de0 2024-09-10T10:46:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.944+0000 7f9d27e00640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072de0 0x7f9d3006e0b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d300b31e0 tx=0x7f9d1802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d568641a1b69a8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d2fe00640 1 -- 172.21.15.28:0/910330668 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18037070 con 0x7f9d30072de0 2024-09-10T10:46:53.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d2fe00640 1 -- 172.21.15.28:0/910330668 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d18032440 con 0x7f9d30072de0 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d2fe00640 1 -- 172.21.15.28:0/910330668 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18032810 con 0x7f9d30072de0 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 -- 172.21.15.28:0/910330668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072de0 msgr2=0x7f9d3006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072de0 0x7f9d3006e0b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d300b31e0 tx=0x7f9d1802a140 comp rx=0 tx=0).stop 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 -- 172.21.15.28:0/910330668 shutdown_connections 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072700 0x7f9d3006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 --2- 172.21.15.28:0/910330668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072de0 0x7f9d3006e0b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 --2- 172.21.15.28:0/910330668 >> v2:172.21.15.138:3300/0 conn(0x7f9d30072320 0x7f9d300cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 -- 172.21.15.28:0/910330668 >> 172.21.15.28:0/910330668 conn(0x7f9d30067d70 msgr2=0x7f9d3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.945+0000 7f9d37800640 1 -- 172.21.15.28:0/910330668 shutdown_connections 2024-09-10T10:46:53.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.946+0000 7f9d37800640 1 -- 172.21.15.28:0/910330668 wait complete. 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.947+0000 7f9d37800640 1 Processor -- start 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.947+0000 7f9d37800640 1 -- start start 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d30072320 0x7f9d300ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072de0 0x7f9d300cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300cd520 con 0x7f9d30072700 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d300cd3a0 con 0x7f9d30072320 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d37800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9d3006eb30 con 0x7f9d30072de0 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47954/0 (socket says 172.21.15.28:47954) 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d27e00640 1 -- 172.21.15.28:0/3025638190 learned_addr learned my addr 172.21.15.28:0/3025638190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d34e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d30072320 0x7f9d300ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.948+0000 7f9d35800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072de0 0x7f9d300cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 -- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072de0 msgr2=0x7f9d300cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072de0 0x7f9d300cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 -- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d30072320 msgr2=0x7f9d300ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d30072320 0x7f9d300ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 -- 172.21.15.28:0/3025638190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30138e40 con 0x7f9d30072700 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d27e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a180 tx=0x7f9d180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18041020 con 0x7f9d30072700 2024-09-10T10:46:53.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d180306c0 con 0x7f9d30072700 2024-09-10T10:46:53.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18037040 con 0x7f9d30072700 2024-09-10T10:46:53.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.949+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d301f6c00 con 0x7f9d30072700 2024-09-10T10:46:53.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.950+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d30138360 con 0x7f9d30072700 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.951+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d300cce60 con 0x7f9d30072700 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.951+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d18044020 con 0x7f9d30072700 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.952+0000 7f9d2d600640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 0x7f9d0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.952+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f9d180ab710 con 0x7f9d30072700 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.953+0000 7f9d34e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 0x7f9d0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:53.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.954+0000 7f9d34e00640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 0x7f9d0c06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d20006fd0 tx=0x7f9d20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:53.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:53.957+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9d1804b030 con 0x7f9d30072700 2024-09-10T10:46:54.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.137+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9d300c5720 con 0x7f9d0c06be30 2024-09-10T10:46:54.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.144+0000 7f9d2d600640 1 -- 172.21.15.28:0/3025638190 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251996 (secure 0 0 0) 0x7f9d300c5720 con 0x7f9d0c06be30 2024-09-10T10:46:54.148 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:54.149 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 msgr2=0x7f9d0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 0x7f9d0c06e2f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9d20006fd0 tx=0x7f9d20008040 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 msgr2=0x7f9d300cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9d1802a180 tx=0x7f9d180046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 shutdown_connections 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0c06be30 0x7f9d0c06e2f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9d30072de0 0x7f9d300cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9d30072700 0x7f9d300cb6e0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 --2- 172.21.15.28:0/3025638190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d30072320 0x7f9d300ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 >> 172.21.15.28:0/3025638190 conn(0x7f9d30067d70 msgr2=0x7f9d300cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 shutdown_connections 2024-09-10T10:46:54.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.148+0000 7f9d37800640 1 -- 172.21.15.28:0/3025638190 wait complete. 2024-09-10T10:46:54.165 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-10T10:46:52.527226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244836,"kb_used_data":7784,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729884,"statfs":{"total":773094113280,"available":772843401216,"internally_reserved":0,"allocated":7970816,"data_stored":4232660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001939"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810784+0000","last_change":"2024-09-10T10:46:44.892310+0000","last_active":"2024-09-10T10:46:46.810784+0000","last_peered":"2024-09-10T10:46:46.810784+0000","last_clean":"2024-09-10T10:46:46.810784+0000","last_became_active":"2024-09-10T10:46:44.892049+0000","last_became_peered":"2024-09-10T10:46:44.892049+0000","last_unstale":"2024-09-10T10:46:46.810784+0000","last_undegraded":"2024-09-10T10:46:46.810784+0000","last_fullsized":"2024-09-10T10:46:46.810784+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:39.944044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810797+0000","last_change":"2024-09-10T10:46:44.881811+0000","last_active":"2024-09-10T10:46:46.810797+0000","last_peered":"2024-09-10T10:46:46.810797+0000","last_clean":"2024-09-10T10:46:46.810797+0000","last_became_active":"2024-09-10T10:46:44.881403+0000","last_became_peered":"2024-09-10T10:46:44.881403+0000","last_unstale":"2024-09-10T10:46:46.810797+0000","last_undegraded":"2024-09-10T10:46:46.810797+0000","last_fullsized":"2024-09-10T10:46:46.810797+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:01.468381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810821+0000","last_change":"2024-09-10T10:46:44.884268+0000","last_active":"2024-09-10T10:46:46.810821+0000","last_peered":"2024-09-10T10:46:46.810821+0000","last_clean":"2024-09-10T10:46:46.810821+0000","last_became_active":"2024-09-10T10:46:44.883101+0000","last_became_peered":"2024-09-10T10:46:44.883101+0000","last_unstale":"2024-09-10T10:46:46.810821+0000","last_undegraded":"2024-09-10T10:46:46.810821+0000","last_fullsized":"2024-09-10T10:46:46.810821+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:37.582746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890725+0000","last_change":"2024-09-10T10:46:44.875622+0000","last_active":"2024-09-10T10:46:45.890725+0000","last_peered":"2024-09-10T10:46:45.890725+0000","last_clean":"2024-09-10T10:46:45.890725+0000","last_became_active":"2024-09-10T10:46:44.875330+0000","last_became_peered":"2024-09-10T10:46:44.875330+0000","last_unstale":"2024-09-10T10:46:45.890725+0000","last_undegraded":"2024-09-10T10:46:45.890725+0000","last_fullsized":"2024-09-10T10:46:45.890725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:35.231024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810774+0000","last_change":"2024-09-10T10:46:44.884914+0000","last_active":"2024-09-10T10:46:46.810774+0000","last_peered":"2024-09-10T10:46:46.810774+0000","last_clean":"2024-09-10T10:46:46.810774+0000","last_became_active":"2024-09-10T10:46:44.884638+0000","last_became_peered":"2024-09-10T10:46:44.884638+0000","last_unstale":"2024-09-10T10:46:46.810774+0000","last_undegraded":"2024-09-10T10:46:46.810774+0000","last_fullsized":"2024-09-10T10:46:46.810774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:29.256965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811010+0000","last_change":"2024-09-10T10:46:44.872857+0000","last_active":"2024-09-10T10:46:46.811010+0000","last_peered":"2024-09-10T10:46:46.811010+0000","last_clean":"2024-09-10T10:46:46.811010+0000","last_became_active":"2024-09-10T10:46:44.871922+0000","last_became_peered":"2024-09-10T10:46:44.871922+0000","last_unstale":"2024-09-10T10:46:46.811010+0000","last_undegraded":"2024-09-10T10:46:46.811010+0000","last_fullsized":"2024-09-10T10:46:46.811010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:55.192636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810368+0000","last_change":"2024-09-10T10:46:44.883459+0000","last_active":"2024-09-10T10:46:46.810368+0000","last_peered":"2024-09-10T10:46:46.810368+0000","last_clean":"2024-09-10T10:46:46.810368+0000","last_became_active":"2024-09-10T10:46:44.883173+0000","last_became_peered":"2024-09-10T10:46:44.883173+0000","last_unstale":"2024-09-10T10:46:46.810368+0000","last_undegraded":"2024-09-10T10:46:46.810368+0000","last_fullsized":"2024-09-10T10:46:46.810368+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:40.377096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810559+0000","last_change":"2024-09-10T10:46:44.890220+0000","last_active":"2024-09-10T10:46:46.810559+0000","last_peered":"2024-09-10T10:46:46.810559+0000","last_clean":"2024-09-10T10:46:46.810559+0000","last_became_active":"2024-09-10T10:46:44.889918+0000","last_became_peered":"2024-09-10T10:46:44.889918+0000","last_unstale":"2024-09-10T10:46:46.810559+0000","last_undegraded":"2024-09-10T10:46:46.810559+0000","last_fullsized":"2024-09-10T10:46:46.810559+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:54:39.399281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890951+0000","last_change":"2024-09-10T10:46:44.877805+0000","last_active":"2024-09-10T10:46:45.890951+0000","last_peered":"2024-09-10T10:46:45.890951+0000","last_clean":"2024-09-10T10:46:45.890951+0000","last_became_active":"2024-09-10T10:46:44.875492+0000","last_became_peered":"2024-09-10T10:46:44.875492+0000","last_unstale":"2024-09-10T10:46:45.890951+0000","last_undegraded":"2024-09-10T10:46:45.890951+0000","last_fullsized":"2024-09-10T10:46:45.890951+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:15.601160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871871+0000","last_change":"2024-09-10T10:46:44.886608+0000","last_active":"2024-09-10T10:46:45.871871+0000","last_peered":"2024-09-10T10:46:45.871871+0000","last_clean":"2024-09-10T10:46:45.871871+0000","last_became_active":"2024-09-10T10:46:44.886293+0000","last_became_peered":"2024-09-10T10:46:44.886293+0000","last_unstale":"2024-09-10T10:46:45.871871+0000","last_undegraded":"2024-09-10T10:46:45.871871+0000","last_fullsized":"2024-09-10T10:46:45.871871+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890687+0000","last_change":"2024-09-10T10:46:44.878902+0000","last_active":"2024-09-10T10:46:45.890687+0000","last_peered":"2024-09-10T10:46:45.890687+0000","last_clean":"2024-09-10T10:46:45.890687+0000","last_became_active":"2024-09-10T10:46:44.878564+0000","last_became_peered":"2024-09-10T10:46:44.878564+0000","last_unstale":"2024-09-10T10:46:45.890687+0000","last_undegraded":"2024-09-10T10:46:45.890687+0000","last_fullsized":"2024-09-10T10:46:45.890687+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:39.944044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890724+0000","last_change":"2024-09-10T10:46:44.882681+0000","last_active":"2024-09-10T10:46:45.890724+0000","last_peered":"2024-09-10T10:46:45.890724+0000","last_clean":"2024-09-10T10:46:45.890724+0000","last_became_active":"2024-09-10T10:46:44.882405+0000","last_became_peered":"2024-09-10T10:46:44.882405+0000","last_unstale":"2024-09-10T10:46:45.890724+0000","last_undegraded":"2024-09-10T10:46:45.890724+0000","last_fullsized":"2024-09-10T10:46:45.890724+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:36.014176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870447+0000","last_change":"2024-09-10T10:46:44.883967+0000","last_active":"2024-09-10T10:46:45.870447+0000","last_peered":"2024-09-10T10:46:45.870447+0000","last_clean":"2024-09-10T10:46:45.870447+0000","last_became_active":"2024-09-10T10:46:44.877104+0000","last_became_peered":"2024-09-10T10:46:44.877104+0000","last_unstale":"2024-09-10T10:46:45.870447+0000","last_undegraded":"2024-09-10T10:46:45.870447+0000","last_fullsized":"2024-09-10T10:46:45.870447+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:49:39.900378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870399+0000","last_change":"2024-09-10T10:46:44.880957+0000","last_active":"2024-09-10T10:46:45.870399+0000","last_peered":"2024-09-10T10:46:45.870399+0000","last_clean":"2024-09-10T10:46:45.870399+0000","last_became_active":"2024-09-10T10:46:44.880691+0000","last_became_peered":"2024-09-10T10:46:44.880691+0000","last_unstale":"2024-09-10T10:46:45.870399+0000","last_undegraded":"2024-09-10T10:46:45.870399+0000","last_fullsized":"2024-09-10T10:46:45.870399+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:42.544680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871234+0000","last_change":"2024-09-10T10:46:44.882838+0000","last_active":"2024-09-10T10:46:45.871234+0000","last_peered":"2024-09-10T10:46:45.871234+0000","last_clean":"2024-09-10T10:46:45.871234+0000","last_became_active":"2024-09-10T10:46:44.880913+0000","last_became_peered":"2024-09-10T10:46:44.880913+0000","last_unstale":"2024-09-10T10:46:45.871234+0000","last_undegraded":"2024-09-10T10:46:45.871234+0000","last_fullsized":"2024-09-10T10:46:45.871234+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:35.562296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.894298+0000","last_change":"2024-09-10T10:46:44.879407+0000","last_active":"2024-09-10T10:46:45.894298+0000","last_peered":"2024-09-10T10:46:45.894298+0000","last_clean":"2024-09-10T10:46:45.894298+0000","last_became_active":"2024-09-10T10:46:44.879124+0000","last_became_peered":"2024-09-10T10:46:44.879124+0000","last_unstale":"2024-09-10T10:46:45.894298+0000","last_undegraded":"2024-09-10T10:46:45.894298+0000","last_fullsized":"2024-09-10T10:46:45.894298+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:12.282050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872649+0000","last_change":"2024-09-10T10:46:44.874639+0000","last_active":"2024-09-10T10:46:45.872649+0000","last_peered":"2024-09-10T10:46:45.872649+0000","last_clean":"2024-09-10T10:46:45.872649+0000","last_became_active":"2024-09-10T10:46:44.874387+0000","last_became_peered":"2024-09-10T10:46:44.874387+0000","last_unstale":"2024-09-10T10:46:45.872649+0000","last_undegraded":"2024-09-10T10:46:45.872649+0000","last_fullsized":"2024-09-10T10:46:45.872649+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:18:15.908078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890921+0000","last_change":"2024-09-10T10:46:44.865791+0000","last_active":"2024-09-10T10:46:45.890921+0000","last_peered":"2024-09-10T10:46:45.890921+0000","last_clean":"2024-09-10T10:46:45.890921+0000","last_became_active":"2024-09-10T10:46:44.865491+0000","last_became_peered":"2024-09-10T10:46:44.865491+0000","last_unstale":"2024-09-10T10:46:45.890921+0000","last_undegraded":"2024-09-10T10:46:45.890921+0000","last_fullsized":"2024-09-10T10:46:45.890921+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:36:07.012650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890869+0000","last_change":"2024-09-10T10:46:44.878815+0000","last_active":"2024-09-10T10:46:45.890869+0000","last_peered":"2024-09-10T10:46:45.890869+0000","last_clean":"2024-09-10T10:46:45.890869+0000","last_became_active":"2024-09-10T10:46:44.878192+0000","last_became_peered":"2024-09-10T10:46:44.878192+0000","last_unstale":"2024-09-10T10:46:45.890869+0000","last_undegraded":"2024-09-10T10:46:45.890869+0000","last_fullsized":"2024-09-10T10:46:45.890869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:47.164216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872489+0000","last_change":"2024-09-10T10:46:44.871127+0000","last_active":"2024-09-10T10:46:45.872489+0000","last_peered":"2024-09-10T10:46:45.872489+0000","last_clean":"2024-09-10T10:46:45.872489+0000","last_became_active":"2024-09-10T10:46:44.870842+0000","last_became_peered":"2024-09-10T10:46:44.870842+0000","last_unstale":"2024-09-10T10:46:45.872489+0000","last_undegraded":"2024-09-10T10:46:45.872489+0000","last_fullsized":"2024-09-10T10:46:45.872489+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:37:36.090978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870454+0000","last_change":"2024-09-10T10:46:44.882561+0000","last_active":"2024-09-10T10:46:45.870454+0000","last_peered":"2024-09-10T10:46:45.870454+0000","last_clean":"2024-09-10T10:46:45.870454+0000","last_became_active":"2024-09-10T10:46:44.877515+0000","last_became_peered":"2024-09-10T10:46:44.877515+0000","last_unstale":"2024-09-10T10:46:45.870454+0000","last_undegraded":"2024-09-10T10:46:45.870454+0000","last_fullsized":"2024-09-10T10:46:45.870454+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:39:45.802274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811184+0000","last_change":"2024-09-10T10:46:44.869105+0000","last_active":"2024-09-10T10:46:46.811184+0000","last_peered":"2024-09-10T10:46:46.811184+0000","last_clean":"2024-09-10T10:46:46.811184+0000","last_became_active":"2024-09-10T10:46:44.868848+0000","last_became_peered":"2024-09-10T10:46:44.868848+0000","last_unstale":"2024-09-10T10:46:46.811184+0000","last_undegraded":"2024-09-10T10:46:46.811184+0000","last_fullsized":"2024-09-10T10:46:46.811184+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:21:39.964842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.894355+0000","last_change":"2024-09-10T10:46:44.878483+0000","last_active":"2024-09-10T10:46:45.894355+0000","last_peered":"2024-09-10T10:46:45.894355+0000","last_clean":"2024-09-10T10:46:45.894355+0000","last_became_active":"2024-09-10T10:46:44.874186+0000","last_became_peered":"2024-09-10T10:46:44.874186+0000","last_unstale":"2024-09-10T10:46:45.894355+0000","last_undegraded":"2024-09-10T10:46:45.894355+0000","last_fullsized":"2024-09-10T10:46:45.894355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:29.274667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810572+0000","last_change":"2024-09-10T10:46:44.890973+0000","last_active":"2024-09-10T10:46:46.810572+0000","last_peered":"2024-09-10T10:46:46.810572+0000","last_clean":"2024-09-10T10:46:46.810572+0000","last_became_active":"2024-09-10T10:46:44.889471+0000","last_became_peered":"2024-09-10T10:46:44.889471+0000","last_unstale":"2024-09-10T10:46:46.810572+0000","last_undegraded":"2024-09-10T10:46:46.810572+0000","last_fullsized":"2024-09-10T10:46:46.810572+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:09.302750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870867+0000","last_change":"2024-09-10T10:46:44.888201+0000","last_active":"2024-09-10T10:46:45.870867+0000","last_peered":"2024-09-10T10:46:45.870867+0000","last_clean":"2024-09-10T10:46:45.870867+0000","last_became_active":"2024-09-10T10:46:44.887970+0000","last_became_peered":"2024-09-10T10:46:44.887970+0000","last_unstale":"2024-09-10T10:46:45.870867+0000","last_undegraded":"2024-09-10T10:46:45.870867+0000","last_fullsized":"2024-09-10T10:46:45.870867+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:24.045178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810881+0000","last_change":"2024-09-10T10:46:44.881612+0000","last_active":"2024-09-10T10:46:46.810881+0000","last_peered":"2024-09-10T10:46:46.810881+0000","last_clean":"2024-09-10T10:46:46.810881+0000","last_became_active":"2024-09-10T10:46:44.881250+0000","last_became_peered":"2024-09-10T10:46:44.881250+0000","last_unstale":"2024-09-10T10:46:46.810881+0000","last_undegraded":"2024-09-10T10:46:46.810881+0000","last_fullsized":"2024-09-10T10:46:46.810881+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:30.598106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870639+0000","last_change":"2024-09-10T10:46:44.878716+0000","last_active":"2024-09-10T10:46:45.870639+0000","last_peered":"2024-09-10T10:46:45.870639+0000","last_clean":"2024-09-10T10:46:45.870639+0000","last_became_active":"2024-09-10T10:46:44.878422+0000","last_became_peered":"2024-09-10T10:46:44.878422+0000","last_unstale":"2024-09-10T10:46:45.870639+0000","last_undegraded":"2024-09-10T10:46:45.870639+0000","last_fullsized":"2024-09-10T10:46:45.870639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:26:23.199185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811031+0000","last_change":"2024-09-10T10:46:44.886054+0000","last_active":"2024-09-10T10:46:46.811031+0000","last_peered":"2024-09-10T10:46:46.811031+0000","last_clean":"2024-09-10T10:46:46.811031+0000","last_became_active":"2024-09-10T10:46:44.885269+0000","last_became_peered":"2024-09-10T10:46:44.885269+0000","last_unstale":"2024-09-10T10:46:46.811031+0000","last_undegraded":"2024-09-10T10:46:46.811031+0000","last_fullsized":"2024-09-10T10:46:46.811031+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:08.460819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890684+0000","last_change":"2024-09-10T10:46:44.869461+0000","last_active":"2024-09-10T10:46:45.890684+0000","last_peered":"2024-09-10T10:46:45.890684+0000","last_clean":"2024-09-10T10:46:45.890684+0000","last_became_active":"2024-09-10T10:46:44.869194+0000","last_became_peered":"2024-09-10T10:46:44.869194+0000","last_unstale":"2024-09-10T10:46:45.890684+0000","last_undegraded":"2024-09-10T10:46:45.890684+0000","last_fullsized":"2024-09-10T10:46:45.890684+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:10:08.460819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890309+0000","last_change":"2024-09-10T10:46:44.879808+0000","last_active":"2024-09-10T10:46:45.890309+0000","last_peered":"2024-09-10T10:46:45.890309+0000","last_clean":"2024-09-10T10:46:45.890309+0000","last_became_active":"2024-09-10T10:46:44.879312+0000","last_became_peered":"2024-09-10T10:46:44.879312+0000","last_unstale":"2024-09-10T10:46:45.890309+0000","last_undegraded":"2024-09-10T10:46:45.890309+0000","last_fullsized":"2024-09-10T10:46:45.890309+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:40.377096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810569+0000","last_change":"2024-09-10T10:46:44.881613+0000","last_active":"2024-09-10T10:46:46.810569+0000","last_peered":"2024-09-10T10:46:46.810569+0000","last_clean":"2024-09-10T10:46:46.810569+0000","last_became_active":"2024-09-10T10:46:44.881356+0000","last_became_peered":"2024-09-10T10:46:44.881356+0000","last_unstale":"2024-09-10T10:46:46.810569+0000","last_undegraded":"2024-09-10T10:46:46.810569+0000","last_fullsized":"2024-09-10T10:46:46.810569+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:51.674256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810179+0000","last_change":"2024-09-10T10:46:44.882083+0000","last_active":"2024-09-10T10:46:46.810179+0000","last_peered":"2024-09-10T10:46:46.810179+0000","last_clean":"2024-09-10T10:46:46.810179+0000","last_became_active":"2024-09-10T10:46:44.878774+0000","last_became_peered":"2024-09-10T10:46:44.878774+0000","last_unstale":"2024-09-10T10:46:46.810179+0000","last_undegraded":"2024-09-10T10:46:46.810179+0000","last_fullsized":"2024-09-10T10:46:46.810179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:54.595619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891070+0000","last_change":"2024-09-10T10:46:44.884761+0000","last_active":"2024-09-10T10:46:45.891070+0000","last_peered":"2024-09-10T10:46:45.891070+0000","last_clean":"2024-09-10T10:46:45.891070+0000","last_became_active":"2024-09-10T10:46:44.884412+0000","last_became_peered":"2024-09-10T10:46:44.884412+0000","last_unstale":"2024-09-10T10:46:45.891070+0000","last_undegraded":"2024-09-10T10:46:45.891070+0000","last_fullsized":"2024-09-10T10:46:45.891070+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:53:29.411763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810908+0000","last_change":"2024-09-10T10:46:43.873120+0000","last_active":"2024-09-10T10:46:46.810908+0000","last_peered":"2024-09-10T10:46:46.810908+0000","last_clean":"2024-09-10T10:46:46.810908+0000","last_became_active":"2024-09-10T10:46:43.872682+0000","last_became_peered":"2024-09-10T10:46:43.872682+0000","last_unstale":"2024-09-10T10:46:46.810908+0000","last_undegraded":"2024-09-10T10:46:46.810908+0000","last_fullsized":"2024-09-10T10:46:46.810908+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:41.538093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811013+0000","last_change":"2024-09-10T10:46:44.885626+0000","last_active":"2024-09-10T10:46:46.811013+0000","last_peered":"2024-09-10T10:46:46.811013+0000","last_clean":"2024-09-10T10:46:46.811013+0000","last_became_active":"2024-09-10T10:46:44.883452+0000","last_became_peered":"2024-09-10T10:46:44.883452+0000","last_unstale":"2024-09-10T10:46:46.811013+0000","last_undegraded":"2024-09-10T10:46:46.811013+0000","last_fullsized":"2024-09-10T10:46:46.811013+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:03:57.107708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810694+0000","last_change":"2024-09-10T10:46:43.875822+0000","last_active":"2024-09-10T10:46:46.810694+0000","last_peered":"2024-09-10T10:46:46.810694+0000","last_clean":"2024-09-10T10:46:46.810694+0000","last_became_active":"2024-09-10T10:46:43.875557+0000","last_became_peered":"2024-09-10T10:46:43.875557+0000","last_unstale":"2024-09-10T10:46:46.810694+0000","last_undegraded":"2024-09-10T10:46:46.810694+0000","last_fullsized":"2024-09-10T10:46:46.810694+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:26.628419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810719+0000","last_change":"2024-09-10T10:46:44.876687+0000","last_active":"2024-09-10T10:46:46.810719+0000","last_peered":"2024-09-10T10:46:46.810719+0000","last_clean":"2024-09-10T10:46:46.810719+0000","last_became_active":"2024-09-10T10:46:44.875941+0000","last_became_peered":"2024-09-10T10:46:44.875941+0000","last_unstale":"2024-09-10T10:46:46.810719+0000","last_undegraded":"2024-09-10T10:46:46.810719+0000","last_fullsized":"2024-09-10T10:46:46.810719+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:31:24.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811133+0000","last_change":"2024-09-10T10:46:43.876519+0000","last_active":"2024-09-10T10:46:46.811133+0000","last_peered":"2024-09-10T10:46:46.811133+0000","last_clean":"2024-09-10T10:46:46.811133+0000","last_became_active":"2024-09-10T10:46:43.876247+0000","last_became_peered":"2024-09-10T10:46:43.876247+0000","last_unstale":"2024-09-10T10:46:46.811133+0000","last_undegraded":"2024-09-10T10:46:46.811133+0000","last_fullsized":"2024-09-10T10:46:46.811133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:00.234612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891058+0000","last_change":"2024-09-10T10:46:44.877202+0000","last_active":"2024-09-10T10:46:45.891058+0000","last_peered":"2024-09-10T10:46:45.891058+0000","last_clean":"2024-09-10T10:46:45.891058+0000","last_became_active":"2024-09-10T10:46:44.876159+0000","last_became_peered":"2024-09-10T10:46:44.876159+0000","last_unstale":"2024-09-10T10:46:45.891058+0000","last_undegraded":"2024-09-10T10:46:45.891058+0000","last_fullsized":"2024-09-10T10:46:45.891058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:42:52.442191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810414+0000","last_change":"2024-09-10T10:46:43.880092+0000","last_active":"2024-09-10T10:46:46.810414+0000","last_peered":"2024-09-10T10:46:46.810414+0000","last_clean":"2024-09-10T10:46:46.810414+0000","last_became_active":"2024-09-10T10:46:43.876472+0000","last_became_peered":"2024-09-10T10:46:43.876472+0000","last_unstale":"2024-09-10T10:46:46.810414+0000","last_undegraded":"2024-09-10T10:46:46.810414+0000","last_fullsized":"2024-09-10T10:46:46.810414+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:16.961700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871580+0000","last_change":"2024-09-10T10:46:44.878110+0000","last_active":"2024-09-10T10:46:45.871580+0000","last_peered":"2024-09-10T10:46:45.871580+0000","last_clean":"2024-09-10T10:46:45.871580+0000","last_became_active":"2024-09-10T10:46:44.877841+0000","last_became_peered":"2024-09-10T10:46:44.877841+0000","last_unstale":"2024-09-10T10:46:45.871580+0000","last_undegraded":"2024-09-10T10:46:45.871580+0000","last_fullsized":"2024-09-10T10:46:45.871580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:41:53.966705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872065+0000","last_change":"2024-09-10T10:46:43.875159+0000","last_active":"2024-09-10T10:46:45.872065+0000","last_peered":"2024-09-10T10:46:45.872065+0000","last_clean":"2024-09-10T10:46:45.872065+0000","last_became_active":"2024-09-10T10:46:43.874602+0000","last_became_peered":"2024-09-10T10:46:43.874602+0000","last_unstale":"2024-09-10T10:46:45.872065+0000","last_undegraded":"2024-09-10T10:46:45.872065+0000","last_fullsized":"2024-09-10T10:46:45.872065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:48.767108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810389+0000","last_change":"2024-09-10T10:46:44.888134+0000","last_active":"2024-09-10T10:46:46.810389+0000","last_peered":"2024-09-10T10:46:46.810389+0000","last_clean":"2024-09-10T10:46:46.810389+0000","last_became_active":"2024-09-10T10:46:44.887851+0000","last_became_peered":"2024-09-10T10:46:44.887851+0000","last_unstale":"2024-09-10T10:46:46.810389+0000","last_undegraded":"2024-09-10T10:46:46.810389+0000","last_fullsized":"2024-09-10T10:46:46.810389+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:13.042339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811213+0000","last_change":"2024-09-10T10:46:43.878707+0000","last_active":"2024-09-10T10:46:46.811213+0000","last_peered":"2024-09-10T10:46:46.811213+0000","last_clean":"2024-09-10T10:46:46.811213+0000","last_became_active":"2024-09-10T10:46:43.878296+0000","last_became_peered":"2024-09-10T10:46:43.878296+0000","last_unstale":"2024-09-10T10:46:46.811213+0000","last_undegraded":"2024-09-10T10:46:46.811213+0000","last_fullsized":"2024-09-10T10:46:46.811213+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:41.855498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871592+0000","last_change":"2024-09-10T10:46:44.893318+0000","last_active":"2024-09-10T10:46:45.871592+0000","last_peered":"2024-09-10T10:46:45.871592+0000","last_clean":"2024-09-10T10:46:45.871592+0000","last_became_active":"2024-09-10T10:46:44.893039+0000","last_became_peered":"2024-09-10T10:46:44.893039+0000","last_unstale":"2024-09-10T10:46:45.871592+0000","last_undegraded":"2024-09-10T10:46:45.871592+0000","last_fullsized":"2024-09-10T10:46:45.871592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:42.367607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810489+0000","last_change":"2024-09-10T10:46:43.879425+0000","last_active":"2024-09-10T10:46:46.810489+0000","last_peered":"2024-09-10T10:46:46.810489+0000","last_clean":"2024-09-10T10:46:46.810489+0000","last_became_active":"2024-09-10T10:46:43.879154+0000","last_became_peered":"2024-09-10T10:46:43.879154+0000","last_unstale":"2024-09-10T10:46:46.810489+0000","last_undegraded":"2024-09-10T10:46:46.810489+0000","last_fullsized":"2024-09-10T10:46:46.810489+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:16:41.361020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872372+0000","last_change":"2024-09-10T10:46:44.871663+0000","last_active":"2024-09-10T10:46:45.872372+0000","last_peered":"2024-09-10T10:46:45.872372+0000","last_clean":"2024-09-10T10:46:45.872372+0000","last_became_active":"2024-09-10T10:46:44.869465+0000","last_became_peered":"2024-09-10T10:46:44.869465+0000","last_unstale":"2024-09-10T10:46:45.872372+0000","last_undegraded":"2024-09-10T10:46:45.872372+0000","last_fullsized":"2024-09-10T10:46:45.872372+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:41:05.055626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890226+0000","last_change":"2024-09-10T10:46:43.870161+0000","last_active":"2024-09-10T10:46:45.890226+0000","last_peered":"2024-09-10T10:46:45.890226+0000","last_clean":"2024-09-10T10:46:45.890226+0000","last_became_active":"2024-09-10T10:46:43.869918+0000","last_became_peered":"2024-09-10T10:46:43.869918+0000","last_unstale":"2024-09-10T10:46:45.890226+0000","last_undegraded":"2024-09-10T10:46:45.890226+0000","last_fullsized":"2024-09-10T10:46:45.890226+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:11.275462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811179+0000","last_change":"2024-09-10T10:46:44.884495+0000","last_active":"2024-09-10T10:46:46.811179+0000","last_peered":"2024-09-10T10:46:46.811179+0000","last_clean":"2024-09-10T10:46:46.811179+0000","last_became_active":"2024-09-10T10:46:44.884231+0000","last_became_peered":"2024-09-10T10:46:44.884231+0000","last_unstale":"2024-09-10T10:46:46.811179+0000","last_undegraded":"2024-09-10T10:46:46.811179+0000","last_fullsized":"2024-09-10T10:46:46.811179+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:36.014176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891047+0000","last_change":"2024-09-10T10:46:43.867286+0000","last_active":"2024-09-10T10:46:45.891047+0000","last_peered":"2024-09-10T10:46:45.891047+0000","last_clean":"2024-09-10T10:46:45.891047+0000","last_became_active":"2024-09-10T10:46:43.867053+0000","last_became_peered":"2024-09-10T10:46:43.867053+0000","last_unstale":"2024-09-10T10:46:45.891047+0000","last_undegraded":"2024-09-10T10:46:45.891047+0000","last_fullsized":"2024-09-10T10:46:45.891047+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:01.574114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872271+0000","last_change":"2024-09-10T10:46:44.870275+0000","last_active":"2024-09-10T10:46:45.872271+0000","last_peered":"2024-09-10T10:46:45.872271+0000","last_clean":"2024-09-10T10:46:45.872271+0000","last_became_active":"2024-09-10T10:46:44.869919+0000","last_became_peered":"2024-09-10T10:46:44.869919+0000","last_unstale":"2024-09-10T10:46:45.872271+0000","last_undegraded":"2024-09-10T10:46:45.872271+0000","last_fullsized":"2024-09-10T10:46:45.872271+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:18.152456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870863+0000","last_change":"2024-09-10T10:46:43.881828+0000","last_active":"2024-09-10T10:46:45.870863+0000","last_peered":"2024-09-10T10:46:45.870863+0000","last_clean":"2024-09-10T10:46:45.870863+0000","last_became_active":"2024-09-10T10:46:43.881524+0000","last_became_peered":"2024-09-10T10:46:43.881524+0000","last_unstale":"2024-09-10T10:46:45.870863+0000","last_undegraded":"2024-09-10T10:46:45.870863+0000","last_fullsized":"2024-09-10T10:46:45.870863+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:18.283684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870716+0000","last_change":"2024-09-10T10:46:44.877148+0000","last_active":"2024-09-10T10:46:45.870716+0000","last_peered":"2024-09-10T10:46:45.870716+0000","last_clean":"2024-09-10T10:46:45.870716+0000","last_became_active":"2024-09-10T10:46:44.876860+0000","last_became_peered":"2024-09-10T10:46:44.876860+0000","last_unstale":"2024-09-10T10:46:45.870716+0000","last_undegraded":"2024-09-10T10:46:45.870716+0000","last_fullsized":"2024-09-10T10:46:45.870716+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:08.030878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810961+0000","last_change":"2024-09-10T10:46:43.874911+0000","last_active":"2024-09-10T10:46:46.810961+0000","last_peered":"2024-09-10T10:46:46.810961+0000","last_clean":"2024-09-10T10:46:46.810961+0000","last_became_active":"2024-09-10T10:46:43.874629+0000","last_became_peered":"2024-09-10T10:46:43.874629+0000","last_unstale":"2024-09-10T10:46:46.810961+0000","last_undegraded":"2024-09-10T10:46:46.810961+0000","last_fullsized":"2024-09-10T10:46:46.810961+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:31.281824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810860+0000","last_change":"2024-09-10T10:46:44.889982+0000","last_active":"2024-09-10T10:46:46.810860+0000","last_peered":"2024-09-10T10:46:46.810860+0000","last_clean":"2024-09-10T10:46:46.810860+0000","last_became_active":"2024-09-10T10:46:44.889481+0000","last_became_peered":"2024-09-10T10:46:44.889481+0000","last_unstale":"2024-09-10T10:46:46.810860+0000","last_undegraded":"2024-09-10T10:46:46.810860+0000","last_fullsized":"2024-09-10T10:46:46.810860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:34.177608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810449+0000","last_change":"2024-09-10T10:46:43.876633+0000","last_active":"2024-09-10T10:46:46.810449+0000","last_peered":"2024-09-10T10:46:46.810449+0000","last_clean":"2024-09-10T10:46:46.810449+0000","last_became_active":"2024-09-10T10:46:43.876340+0000","last_became_peered":"2024-09-10T10:46:43.876340+0000","last_unstale":"2024-09-10T10:46:46.810449+0000","last_undegraded":"2024-09-10T10:46:46.810449+0000","last_fullsized":"2024-09-10T10:46:46.810449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:04:13.563191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871106+0000","last_change":"2024-09-10T10:46:44.886563+0000","last_active":"2024-09-10T10:46:45.871106+0000","last_peered":"2024-09-10T10:46:45.871106+0000","last_clean":"2024-09-10T10:46:45.871106+0000","last_became_active":"2024-09-10T10:46:44.886299+0000","last_became_peered":"2024-09-10T10:46:44.886299+0000","last_unstale":"2024-09-10T10:46:45.871106+0000","last_undegraded":"2024-09-10T10:46:45.871106+0000","last_fullsized":"2024-09-10T10:46:45.871106+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:41.176611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890634+0000","last_change":"2024-09-10T10:46:43.868983+0000","last_active":"2024-09-10T10:46:45.890634+0000","last_peered":"2024-09-10T10:46:45.890634+0000","last_clean":"2024-09-10T10:46:45.890634+0000","last_became_active":"2024-09-10T10:46:43.868631+0000","last_became_peered":"2024-09-10T10:46:43.868631+0000","last_unstale":"2024-09-10T10:46:45.890634+0000","last_undegraded":"2024-09-10T10:46:45.890634+0000","last_fullsized":"2024-09-10T10:46:45.890634+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:28.268079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871375+0000","last_change":"2024-09-10T10:46:44.885879+0000","last_active":"2024-09-10T10:46:45.871375+0000","last_peered":"2024-09-10T10:46:45.871375+0000","last_clean":"2024-09-10T10:46:45.871375+0000","last_became_active":"2024-09-10T10:46:44.885500+0000","last_became_peered":"2024-09-10T10:46:44.885500+0000","last_unstale":"2024-09-10T10:46:45.871375+0000","last_undegraded":"2024-09-10T10:46:45.871375+0000","last_fullsized":"2024-09-10T10:46:45.871375+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:53:08.192965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871512+0000","last_change":"2024-09-10T10:46:43.879565+0000","last_active":"2024-09-10T10:46:45.871512+0000","last_peered":"2024-09-10T10:46:45.871512+0000","last_clean":"2024-09-10T10:46:45.871512+0000","last_became_active":"2024-09-10T10:46:43.879254+0000","last_became_peered":"2024-09-10T10:46:43.879254+0000","last_unstale":"2024-09-10T10:46:45.871512+0000","last_undegraded":"2024-09-10T10:46:45.871512+0000","last_fullsized":"2024-09-10T10:46:45.871512+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871162+0000","last_change":"2024-09-10T10:46:44.878593+0000","last_active":"2024-09-10T10:46:45.871162+0000","last_peered":"2024-09-10T10:46:45.871162+0000","last_clean":"2024-09-10T10:46:45.871162+0000","last_became_active":"2024-09-10T10:46:44.878283+0000","last_became_peered":"2024-09-10T10:46:44.878283+0000","last_unstale":"2024-09-10T10:46:45.871162+0000","last_undegraded":"2024-09-10T10:46:45.871162+0000","last_fullsized":"2024-09-10T10:46:45.871162+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:06.745405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810601+0000","last_change":"2024-09-10T10:46:43.877389+0000","last_active":"2024-09-10T10:46:46.810601+0000","last_peered":"2024-09-10T10:46:46.810601+0000","last_clean":"2024-09-10T10:46:46.810601+0000","last_became_active":"2024-09-10T10:46:43.876235+0000","last_became_peered":"2024-09-10T10:46:43.876235+0000","last_unstale":"2024-09-10T10:46:46.810601+0000","last_undegraded":"2024-09-10T10:46:46.810601+0000","last_fullsized":"2024-09-10T10:46:46.810601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:52.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810233+0000","last_change":"2024-09-10T10:46:44.878187+0000","last_active":"2024-09-10T10:46:46.810233+0000","last_peered":"2024-09-10T10:46:46.810233+0000","last_clean":"2024-09-10T10:46:46.810233+0000","last_became_active":"2024-09-10T10:46:44.877922+0000","last_became_peered":"2024-09-10T10:46:44.877922+0000","last_unstale":"2024-09-10T10:46:46.810233+0000","last_undegraded":"2024-09-10T10:46:46.810233+0000","last_fullsized":"2024-09-10T10:46:46.810233+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:16.058506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871209+0000","last_change":"2024-09-10T10:46:43.879721+0000","last_active":"2024-09-10T10:46:45.871209+0000","last_peered":"2024-09-10T10:46:45.871209+0000","last_clean":"2024-09-10T10:46:45.871209+0000","last_became_active":"2024-09-10T10:46:43.879448+0000","last_became_peered":"2024-09-10T10:46:43.879448+0000","last_unstale":"2024-09-10T10:46:45.871209+0000","last_undegraded":"2024-09-10T10:46:45.871209+0000","last_fullsized":"2024-09-10T10:46:45.871209+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890924+0000","last_change":"2024-09-10T10:46:44.871707+0000","last_active":"2024-09-10T10:46:45.890924+0000","last_peered":"2024-09-10T10:46:45.890924+0000","last_clean":"2024-09-10T10:46:45.890924+0000","last_became_active":"2024-09-10T10:46:44.871400+0000","last_became_peered":"2024-09-10T10:46:44.871400+0000","last_unstale":"2024-09-10T10:46:45.890924+0000","last_undegraded":"2024-09-10T10:46:45.890924+0000","last_fullsized":"2024-09-10T10:46:45.890924+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:32:10.676188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891163+0000","last_change":"2024-09-10T10:46:43.866850+0000","last_active":"2024-09-10T10:46:45.891163+0000","last_peered":"2024-09-10T10:46:45.891163+0000","last_clean":"2024-09-10T10:46:45.891163+0000","last_became_active":"2024-09-10T10:46:43.866575+0000","last_became_peered":"2024-09-10T10:46:43.866575+0000","last_unstale":"2024-09-10T10:46:45.891163+0000","last_undegraded":"2024-09-10T10:46:45.891163+0000","last_fullsized":"2024-09-10T10:46:45.891163+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:29:07.913685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870730+0000","last_change":"2024-09-10T10:46:44.875025+0000","last_active":"2024-09-10T10:46:45.870730+0000","last_peered":"2024-09-10T10:46:45.870730+0000","last_clean":"2024-09-10T10:46:45.870730+0000","last_became_active":"2024-09-10T10:46:44.873173+0000","last_became_peered":"2024-09-10T10:46:44.873173+0000","last_unstale":"2024-09-10T10:46:45.870730+0000","last_undegraded":"2024-09-10T10:46:45.870730+0000","last_fullsized":"2024-09-10T10:46:45.870730+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:38.814890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890848+0000","last_change":"2024-09-10T10:46:43.871807+0000","last_active":"2024-09-10T10:46:45.890848+0000","last_peered":"2024-09-10T10:46:45.890848+0000","last_clean":"2024-09-10T10:46:45.890848+0000","last_became_active":"2024-09-10T10:46:43.871540+0000","last_became_peered":"2024-09-10T10:46:43.871540+0000","last_unstale":"2024-09-10T10:46:45.890848+0000","last_undegraded":"2024-09-10T10:46:45.890848+0000","last_fullsized":"2024-09-10T10:46:45.890848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:34.555709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870868+0000","last_change":"2024-09-10T10:46:44.888440+0000","last_active":"2024-09-10T10:46:45.870868+0000","last_peered":"2024-09-10T10:46:45.870868+0000","last_clean":"2024-09-10T10:46:45.870868+0000","last_became_active":"2024-09-10T10:46:44.888154+0000","last_became_peered":"2024-09-10T10:46:44.888154+0000","last_unstale":"2024-09-10T10:46:45.870868+0000","last_undegraded":"2024-09-10T10:46:45.870868+0000","last_fullsized":"2024-09-10T10:46:45.870868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:23:01.783628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811101+0000","last_change":"2024-09-10T10:46:43.876691+0000","last_active":"2024-09-10T10:46:46.811101+0000","last_peered":"2024-09-10T10:46:46.811101+0000","last_clean":"2024-09-10T10:46:46.811101+0000","last_became_active":"2024-09-10T10:46:43.876227+0000","last_became_peered":"2024-09-10T10:46:43.876227+0000","last_unstale":"2024-09-10T10:46:46.811101+0000","last_undegraded":"2024-09-10T10:46:46.811101+0000","last_fullsized":"2024-09-10T10:46:46.811101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:31.964863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811076+0000","last_change":"2024-09-10T10:46:44.882771+0000","last_active":"2024-09-10T10:46:46.811076+0000","last_peered":"2024-09-10T10:46:46.811076+0000","last_clean":"2024-09-10T10:46:46.811076+0000","last_became_active":"2024-09-10T10:46:44.881897+0000","last_became_peered":"2024-09-10T10:46:44.881897+0000","last_unstale":"2024-09-10T10:46:46.811076+0000","last_undegraded":"2024-09-10T10:46:46.811076+0000","last_fullsized":"2024-09-10T10:46:46.811076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:22.400504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.894466+0000","last_change":"2024-09-10T10:46:43.873156+0000","last_active":"2024-09-10T10:46:45.894466+0000","last_peered":"2024-09-10T10:46:45.894466+0000","last_clean":"2024-09-10T10:46:45.894466+0000","last_became_active":"2024-09-10T10:46:43.872889+0000","last_became_peered":"2024-09-10T10:46:43.872889+0000","last_unstale":"2024-09-10T10:46:45.894466+0000","last_undegraded":"2024-09-10T10:46:45.894466+0000","last_fullsized":"2024-09-10T10:46:45.894466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:10.922261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810286+0000","last_change":"2024-09-10T10:46:44.878982+0000","last_active":"2024-09-10T10:46:46.810286+0000","last_peered":"2024-09-10T10:46:46.810286+0000","last_clean":"2024-09-10T10:46:46.810286+0000","last_became_active":"2024-09-10T10:46:44.878626+0000","last_became_peered":"2024-09-10T10:46:44.878626+0000","last_unstale":"2024-09-10T10:46:46.810286+0000","last_undegraded":"2024-09-10T10:46:46.810286+0000","last_fullsized":"2024-09-10T10:46:46.810286+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:23.712304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811378+0000","last_change":"2024-09-10T10:46:43.866904+0000","last_active":"2024-09-10T10:46:46.811378+0000","last_peered":"2024-09-10T10:46:46.811378+0000","last_clean":"2024-09-10T10:46:46.811378+0000","last_became_active":"2024-09-10T10:46:43.866582+0000","last_became_peered":"2024-09-10T10:46:43.866582+0000","last_unstale":"2024-09-10T10:46:46.811378+0000","last_undegraded":"2024-09-10T10:46:46.811378+0000","last_fullsized":"2024-09-10T10:46:46.811378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:17:23.463417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870202+0000","last_change":"2024-09-10T10:46:44.868747+0000","last_active":"2024-09-10T10:46:45.870202+0000","last_peered":"2024-09-10T10:46:45.870202+0000","last_clean":"2024-09-10T10:46:45.870202+0000","last_became_active":"2024-09-10T10:46:44.868455+0000","last_became_peered":"2024-09-10T10:46:44.868455+0000","last_unstale":"2024-09-10T10:46:45.870202+0000","last_undegraded":"2024-09-10T10:46:45.870202+0000","last_fullsized":"2024-09-10T10:46:45.870202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:49:27.635006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871624+0000","last_change":"2024-09-10T10:46:43.876827+0000","last_active":"2024-09-10T10:46:45.871624+0000","last_peered":"2024-09-10T10:46:45.871624+0000","last_clean":"2024-09-10T10:46:45.871624+0000","last_became_active":"2024-09-10T10:46:43.875591+0000","last_became_peered":"2024-09-10T10:46:43.875591+0000","last_unstale":"2024-09-10T10:46:45.871624+0000","last_undegraded":"2024-09-10T10:46:45.871624+0000","last_fullsized":"2024-09-10T10:46:45.871624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890469+0000","last_change":"2024-09-10T10:46:44.883361+0000","last_active":"2024-09-10T10:46:45.890469+0000","last_peered":"2024-09-10T10:46:45.890469+0000","last_clean":"2024-09-10T10:46:45.890469+0000","last_became_active":"2024-09-10T10:46:44.879641+0000","last_became_peered":"2024-09-10T10:46:44.879641+0000","last_unstale":"2024-09-10T10:46:45.890469+0000","last_undegraded":"2024-09-10T10:46:45.890469+0000","last_fullsized":"2024-09-10T10:46:45.890469+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:41:13.042339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810132+0000","last_change":"2024-09-10T10:46:43.875319+0000","last_active":"2024-09-10T10:46:46.810132+0000","last_peered":"2024-09-10T10:46:46.810132+0000","last_clean":"2024-09-10T10:46:46.810132+0000","last_became_active":"2024-09-10T10:46:43.873628+0000","last_became_peered":"2024-09-10T10:46:43.873628+0000","last_unstale":"2024-09-10T10:46:46.810132+0000","last_undegraded":"2024-09-10T10:46:46.810132+0000","last_fullsized":"2024-09-10T10:46:46.810132+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:11.118440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872283+0000","last_change":"2024-09-10T10:46:44.873964+0000","last_active":"2024-09-10T10:46:45.872283+0000","last_peered":"2024-09-10T10:46:45.872283+0000","last_clean":"2024-09-10T10:46:45.872283+0000","last_became_active":"2024-09-10T10:46:44.873324+0000","last_became_peered":"2024-09-10T10:46:44.873324+0000","last_unstale":"2024-09-10T10:46:45.872283+0000","last_undegraded":"2024-09-10T10:46:45.872283+0000","last_fullsized":"2024-09-10T10:46:45.872283+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:05.157586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891176+0000","last_change":"2024-09-10T10:46:43.880123+0000","last_active":"2024-09-10T10:46:45.891176+0000","last_peered":"2024-09-10T10:46:45.891176+0000","last_clean":"2024-09-10T10:46:45.891176+0000","last_became_active":"2024-09-10T10:46:43.879861+0000","last_became_peered":"2024-09-10T10:46:43.879861+0000","last_unstale":"2024-09-10T10:46:45.891176+0000","last_undegraded":"2024-09-10T10:46:45.891176+0000","last_fullsized":"2024-09-10T10:46:45.891176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:33:40.170023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.872202+0000","last_change":"2024-09-10T10:46:44.875117+0000","last_active":"2024-09-10T10:46:45.872202+0000","last_peered":"2024-09-10T10:46:45.872202+0000","last_clean":"2024-09-10T10:46:45.872202+0000","last_became_active":"2024-09-10T10:46:44.874858+0000","last_became_peered":"2024-09-10T10:46:44.874858+0000","last_unstale":"2024-09-10T10:46:45.872202+0000","last_undegraded":"2024-09-10T10:46:45.872202+0000","last_fullsized":"2024-09-10T10:46:45.872202+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:24.855534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"70'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.891051+0000","last_change":"2024-09-10T10:46:43.870732+0000","last_active":"2024-09-10T10:46:45.891051+0000","last_peered":"2024-09-10T10:46:45.891051+0000","last_clean":"2024-09-10T10:46:45.891051+0000","last_became_active":"2024-09-10T10:46:43.870484+0000","last_became_peered":"2024-09-10T10:46:43.870484+0000","last_unstale":"2024-09-10T10:46:45.891051+0000","last_undegraded":"2024-09-10T10:46:45.891051+0000","last_fullsized":"2024-09-10T10:46:45.891051+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:05.738817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810813+0000","last_change":"2024-09-10T10:46:44.875783+0000","last_active":"2024-09-10T10:46:46.810813+0000","last_peered":"2024-09-10T10:46:46.810813+0000","last_clean":"2024-09-10T10:46:46.810813+0000","last_became_active":"2024-09-10T10:46:44.875511+0000","last_became_peered":"2024-09-10T10:46:44.875511+0000","last_unstale":"2024-09-10T10:46:46.810813+0000","last_undegraded":"2024-09-10T10:46:46.810813+0000","last_fullsized":"2024-09-10T10:46:46.810813+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:27.641993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810667+0000","last_change":"2024-09-10T10:46:43.874338+0000","last_active":"2024-09-10T10:46:46.810667+0000","last_peered":"2024-09-10T10:46:46.810667+0000","last_clean":"2024-09-10T10:46:46.810667+0000","last_became_active":"2024-09-10T10:46:43.874013+0000","last_became_peered":"2024-09-10T10:46:43.874013+0000","last_unstale":"2024-09-10T10:46:46.810667+0000","last_undegraded":"2024-09-10T10:46:46.810667+0000","last_fullsized":"2024-09-10T10:46:46.810667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:06:55.976080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811081+0000","last_change":"2024-09-10T10:46:44.882540+0000","last_active":"2024-09-10T10:46:46.811081+0000","last_peered":"2024-09-10T10:46:46.811081+0000","last_clean":"2024-09-10T10:46:46.811081+0000","last_became_active":"2024-09-10T10:46:44.881322+0000","last_became_peered":"2024-09-10T10:46:44.881322+0000","last_unstale":"2024-09-10T10:46:46.811081+0000","last_undegraded":"2024-09-10T10:46:46.811081+0000","last_fullsized":"2024-09-10T10:46:46.811081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:56:08.323955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.894569+0000","last_change":"2024-09-10T10:46:43.880848+0000","last_active":"2024-09-10T10:46:45.894569+0000","last_peered":"2024-09-10T10:46:45.894569+0000","last_clean":"2024-09-10T10:46:45.894569+0000","last_became_active":"2024-09-10T10:46:43.880588+0000","last_became_peered":"2024-09-10T10:46:43.880588+0000","last_unstale":"2024-09-10T10:46:45.894569+0000","last_undegraded":"2024-09-10T10:46:45.894569+0000","last_fullsized":"2024-09-10T10:46:45.894569+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:51:05.810356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.890267+0000","last_change":"2024-09-10T10:46:44.879311+0000","last_active":"2024-09-10T10:46:45.890267+0000","last_peered":"2024-09-10T10:46:45.890267+0000","last_clean":"2024-09-10T10:46:45.890267+0000","last_became_active":"2024-09-10T10:46:44.867480+0000","last_became_peered":"2024-09-10T10:46:44.867480+0000","last_unstale":"2024-09-10T10:46:45.890267+0000","last_undegraded":"2024-09-10T10:46:45.890267+0000","last_fullsized":"2024-09-10T10:46:45.890267+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:16.058506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870559+0000","last_change":"2024-09-10T10:46:43.878330+0000","last_active":"2024-09-10T10:46:45.870559+0000","last_peered":"2024-09-10T10:46:45.870559+0000","last_clean":"2024-09-10T10:46:45.870559+0000","last_became_active":"2024-09-10T10:46:43.877459+0000","last_became_peered":"2024-09-10T10:46:43.877459+0000","last_unstale":"2024-09-10T10:46:45.870559+0000","last_undegraded":"2024-09-10T10:46:45.870559+0000","last_fullsized":"2024-09-10T10:46:45.870559+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:13.837013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.870222+0000","last_change":"2024-09-10T10:46:44.868549+0000","last_active":"2024-09-10T10:46:45.870222+0000","last_peered":"2024-09-10T10:46:45.870222+0000","last_clean":"2024-09-10T10:46:45.870222+0000","last_became_active":"2024-09-10T10:46:44.868232+0000","last_became_peered":"2024-09-10T10:46:44.868232+0000","last_unstale":"2024-09-10T10:46:45.870222+0000","last_undegraded":"2024-09-10T10:46:45.870222+0000","last_fullsized":"2024-09-10T10:46:45.870222+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:58:01.241200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810399+0000","last_change":"2024-09-10T10:46:43.879611+0000","last_active":"2024-09-10T10:46:46.810399+0000","last_peered":"2024-09-10T10:46:46.810399+0000","last_clean":"2024-09-10T10:46:46.810399+0000","last_became_active":"2024-09-10T10:46:43.879308+0000","last_became_peered":"2024-09-10T10:46:43.879308+0000","last_unstale":"2024-09-10T10:46:46.810399+0000","last_undegraded":"2024-09-10T10:46:46.810399+0000","last_fullsized":"2024-09-10T10:46:46.810399+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:57.246965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.811151+0000","last_change":"2024-09-10T10:46:44.868446+0000","last_active":"2024-09-10T10:46:46.811151+0000","last_peered":"2024-09-10T10:46:46.811151+0000","last_clean":"2024-09-10T10:46:46.811151+0000","last_became_active":"2024-09-10T10:46:44.868167+0000","last_became_peered":"2024-09-10T10:46:44.868167+0000","last_unstale":"2024-09-10T10:46:46.811151+0000","last_undegraded":"2024-09-10T10:46:46.811151+0000","last_fullsized":"2024-09-10T10:46:46.811151+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:05:39.838300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.894326+0000","last_change":"2024-09-10T10:46:43.879656+0000","last_active":"2024-09-10T10:46:45.894326+0000","last_peered":"2024-09-10T10:46:45.894326+0000","last_clean":"2024-09-10T10:46:45.894326+0000","last_became_active":"2024-09-10T10:46:43.879098+0000","last_became_peered":"2024-09-10T10:46:43.879098+0000","last_unstale":"2024-09-10T10:46:45.894326+0000","last_undegraded":"2024-09-10T10:46:45.894326+0000","last_fullsized":"2024-09-10T10:46:45.894326+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:11:38.125519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871949+0000","last_change":"2024-09-10T10:46:44.869395+0000","last_active":"2024-09-10T10:46:45.871949+0000","last_peered":"2024-09-10T10:46:45.871949+0000","last_clean":"2024-09-10T10:46:45.871949+0000","last_became_active":"2024-09-10T10:46:44.869065+0000","last_became_peered":"2024-09-10T10:46:44.869065+0000","last_unstale":"2024-09-10T10:46:45.871949+0000","last_undegraded":"2024-09-10T10:46:45.871949+0000","last_fullsized":"2024-09-10T10:46:45.871949+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:10.187823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810175+0000","last_change":"2024-09-10T10:46:43.870879+0000","last_active":"2024-09-10T10:46:46.810175+0000","last_peered":"2024-09-10T10:46:46.810175+0000","last_clean":"2024-09-10T10:46:46.810175+0000","last_became_active":"2024-09-10T10:46:43.870201+0000","last_became_peered":"2024-09-10T10:46:43.870201+0000","last_unstale":"2024-09-10T10:46:46.810175+0000","last_undegraded":"2024-09-10T10:46:46.810175+0000","last_fullsized":"2024-09-10T10:46:46.810175+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:16:33.476365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871775+0000","last_change":"2024-09-10T10:46:44.885648+0000","last_active":"2024-09-10T10:46:45.871775+0000","last_peered":"2024-09-10T10:46:45.871775+0000","last_clean":"2024-09-10T10:46:45.871775+0000","last_became_active":"2024-09-10T10:46:44.885219+0000","last_became_peered":"2024-09-10T10:46:44.885219+0000","last_unstale":"2024-09-10T10:46:45.871775+0000","last_undegraded":"2024-09-10T10:46:45.871775+0000","last_fullsized":"2024-09-10T10:46:45.871775+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:43.845107+0000","last_clean_scrub_stamp":"2024-09-10T10:46:43.845107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:58.253553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:46.810294+0000","last_change":"2024-09-10T10:46:43.871118+0000","last_active":"2024-09-10T10:46:46.810294+0000","last_peered":"2024-09-10T10:46:46.810294+0000","last_clean":"2024-09-10T10:46:46.810294+0000","last_became_active":"2024-09-10T10:46:43.870597+0000","last_became_peered":"2024-09-10T10:46:43.870597+0000","last_unstale":"2024-09-10T10:46:46.810294+0000","last_undegraded":"2024-09-10T10:46:46.810294+0000","last_fullsized":"2024-09-10T10:46:46.810294+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:46:42.838519+0000","last_clean_scrub_stamp":"2024-09-10T10:46:42.838519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:35:04.466041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":259,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T10:46:45.871313+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:46:45.871313+0000","last_peered":"2024-09-10T10:46:45.871313+0000","last_clean":"2024-09-10T10:46:45.871313+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:46:45.871313+0000","last_undegraded":"2024-09-10T10:46:45.871313+0000","last_fullsized":"2024-09-10T10:46:45.871313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":380888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":383152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":843776,"data_stored":381470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":847872,"data_stored":381048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":380944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1437696,"data_stored":971290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":839680,"data_stored":380888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28452,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343388,"statfs":{"total":96636764160,"available":96607629312,"internally_reserved":0,"allocated":1454080,"data_stored":972980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:46:54.169 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:46:54.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:46:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.367+0000 7f3d14600640 1 Processor -- start 2024-09-10T10:46:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.367+0000 7f3d14600640 1 -- start start 2024-09-10T10:46:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.368+0000 7f3d14600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d14600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d0c0adce0 0x7f3d0c0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d14600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d0c0ae690 con 0x7f3d0c067ae0 2024-09-10T10:46:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d14600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3d0c0aec30 con 0x7f3d0c0adce0 2024-09-10T10:46:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d14600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c067ff0 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d11c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d12600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d0c0adce0 0x7f3d0c0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d11c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:32892/0 (socket says 172.21.15.28:32892) 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.369+0000 7f3d11c00640 1 -- 172.21.15.28:0/2899603142 learned_addr learned my addr 172.21.15.28:0/2899603142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.370+0000 7f3d11c00640 1 -- 172.21.15.28:0/2899603142 >> v2:172.21.15.138:3300/0 conn(0x7f3d0c0adce0 msgr2=0x7f3d0c0ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.370+0000 7f3d11c00640 1 --2- 172.21.15.28:0/2899603142 >> v2:172.21.15.138:3300/0 conn(0x7f3d0c0adce0 0x7f3d0c0ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.370+0000 7f3d11c00640 1 -- 172.21.15.28:0/2899603142 >> v1:172.21.15.138:6789/0 conn(0x7f3d0c067ae0 legacy=0x7f3d0c15f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.370+0000 7f3d11c00640 1 -- 172.21.15.28:0/2899603142 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c0af390 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d11c00640 1 --2- 172.21.15.28:0/2899603142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009870 tx=0x7f3d0002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f294c26a3e859e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d0be00640 1 -- 172.21.15.28:0/2899603142 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00037070 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d0be00640 1 -- 172.21.15.28:0/2899603142 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d00032440 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d0be00640 1 -- 172.21.15.28:0/2899603142 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00032810 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 -- 172.21.15.28:0/2899603142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 msgr2=0x7f3d0c0b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 --2- 172.21.15.28:0/2899603142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009870 tx=0x7f3d0002a250 comp rx=0 tx=0).stop 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 -- 172.21.15.28:0/2899603142 shutdown_connections 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 --2- 172.21.15.28:0/2899603142 >> v2:172.21.15.138:3300/0 conn(0x7f3d0c0adce0 0x7f3d0c0ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 --2- 172.21.15.28:0/2899603142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15e2e0 0x7f3d0c0b0150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.371+0000 7f3d14600640 1 -- 172.21.15.28:0/2899603142 >> 172.21.15.28:0/2899603142 conn(0x7f3d0c073590 msgr2=0x7f3d0c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.372+0000 7f3d14600640 1 -- 172.21.15.28:0/2899603142 shutdown_connections 2024-09-10T10:46:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.372+0000 7f3d14600640 1 -- 172.21.15.28:0/2899603142 wait complete. 2024-09-10T10:46:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.373+0000 7f3d14600640 1 Processor -- start 2024-09-10T10:46:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.373+0000 7f3d14600640 1 -- start start 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c067ae0 0x7f3d0c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c0adce0 0x7f3d0c161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c0afc30 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c0af950 con 0x7f3d0c0adce0 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d14600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c15e6c0 con 0x7f3d0c067ae0 2024-09-10T10:46:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d12600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d12600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47960/0 (socket says 172.21.15.28:47960) 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.374+0000 7f3d12600640 1 -- 172.21.15.28:0/127567894 learned_addr learned my addr 172.21.15.28:0/127567894 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d11c00640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c067ae0 0x7f3d0c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d11200640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c0adce0 0x7f3d0c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 -- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c067ae0 msgr2=0x7f3d0c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c067ae0 0x7f3d0c156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 -- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c0adce0 msgr2=0x7f3d0c161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c0adce0 0x7f3d0c161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 -- 172.21.15.28:0/127567894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c159340 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d12600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009dd0 tx=0x7f3d0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.375+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04018020 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.376+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04014480 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.376+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04014870 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.376+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c159600 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.376+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c159bc0 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.377+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c0ae150 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.378+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d04005d10 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.378+0000 7f3d09600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.379+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f3d0408ba30 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.379+0000 7f3d11c00640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.380+0000 7f3d11c00640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3d00034000 tx=0x7f3d0002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.383+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d04022070 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.635+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3d0c0b6a10 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.638+0000 7f3d09600640 1 -- 172.21.15.28:0/127567894 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f3d04010040 con 0x7f3d0c15e2e0 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:46:54.639 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 msgr2=0x7f3cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3d00034000 tx=0x7f3d0002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 msgr2=0x7f3d0c155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009dd0 tx=0x7f3d0400c490 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 shutdown_connections 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15e2e0 0x7f3d0c155e60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c0adce0 0x7f3d0c161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 --2- 172.21.15.28:0/127567894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c067ae0 0x7f3d0c156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 >> 172.21.15.28:0/127567894 conn(0x7f3d0c073590 msgr2=0x7f3d0c0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 shutdown_connections 2024-09-10T10:46:54.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.640+0000 7f3d14600640 1 -- 172.21.15.28:0/127567894 wait complete. 2024-09-10T10:46:54.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:46:54.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.814+0000 7f82ffc00640 1 Processor -- start 2024-09-10T10:46:54.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.814+0000 7f82ffc00640 1 -- start start 2024-09-10T10:46:54.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82f815b300 0x7f82f80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f82f80cd3f0 0x7f82f806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f8072d30 0x7f82f806f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80728a0 con 0x7f82f815b300 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f82f80cdbe0 con 0x7f82f80cd3f0 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.815+0000 7f82ffc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80cd900 con 0x7f82f8072d30 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82f815b300 0x7f82f80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82f815b300 0x7f82f80cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47968/0 (socket says 172.21.15.28:47968) 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 -- 172.21.15.28:0/3272531320 learned_addr learned my addr 172.21.15.28:0/3272531320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fdc00640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f8072d30 0x7f82f806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82f7e00640 1 --2- 172.21.15.28:0/3272531320 >> v2:172.21.15.138:3300/0 conn(0x7f82f80cd3f0 0x7f82f806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 -- 172.21.15.28:0/3272531320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f8072d30 msgr2=0x7f82f806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f8072d30 0x7f82f806f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 -- 172.21.15.28:0/3272531320 >> v2:172.21.15.138:3300/0 conn(0x7f82f80cd3f0 msgr2=0x7f82f806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 --2- 172.21.15.28:0/3272531320 >> v2:172.21.15.138:3300/0 conn(0x7f82f80cd3f0 0x7f82f806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.816+0000 7f82fd200640 1 -- 172.21.15.28:0/3272531320 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f8072250 con 0x7f82f815b300 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.817+0000 7f82fd200640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f815b300 0x7f82f80cceb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f82f80b57c0 tx=0x7f82ec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9b0bdd79e73c61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.817+0000 7f82f7400640 1 -- 172.21.15.28:0/3272531320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037070 con 0x7f82f815b300 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82f7400640 1 -- 172.21.15.28:0/3272531320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec032440 con 0x7f82f815b300 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82f7400640 1 -- 172.21.15.28:0/3272531320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec032810 con 0x7f82f815b300 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 -- 172.21.15.28:0/3272531320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f815b300 msgr2=0x7f82f80cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f815b300 0x7f82f80cceb0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f82f80b57c0 tx=0x7f82ec02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 -- 172.21.15.28:0/3272531320 shutdown_connections 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f8072d30 0x7f82f806f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 --2- 172.21.15.28:0/3272531320 >> v2:172.21.15.138:3300/0 conn(0x7f82f80cd3f0 0x7f82f806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 --2- 172.21.15.28:0/3272531320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f815b300 0x7f82f80cceb0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 -- 172.21.15.28:0/3272531320 >> 172.21.15.28:0/3272531320 conn(0x7f82f8067d70 msgr2=0x7f82f806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 -- 172.21.15.28:0/3272531320 shutdown_connections 2024-09-10T10:46:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.818+0000 7f82ffc00640 1 -- 172.21.15.28:0/3272531320 wait complete. 2024-09-10T10:46:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.820+0000 7f82ffc00640 1 Processor -- start 2024-09-10T10:46:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.820+0000 7f82ffc00640 1 -- start start 2024-09-10T10:46:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f80cd3f0 0x7f82f80af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82f815b300 0x7f82f81e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80d0160 con 0x7f82f8072d30 2024-09-10T10:46:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82f80cffe0 con 0x7f82f815b300 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82ffc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82f80b3c40 con 0x7f82f80cd3f0 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47976/0 (socket says 172.21.15.28:47976) 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 -- 172.21.15.28:0/2427255333 learned_addr learned my addr 172.21.15.28:0/2427255333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 -- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f80cd3f0 msgr2=0x7f82f80af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f80cd3f0 0x7f82f80af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 -- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82f815b300 msgr2=0x7f82f81e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82f815b300 0x7f82f81e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.821+0000 7f82fd200640 1 -- 172.21.15.28:0/2427255333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f81e56b0 con 0x7f82f8072d30 2024-09-10T10:46:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82fd200640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f82ec02a220 tx=0x7f82ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec041020 con 0x7f82f8072d30 2024-09-10T10:46:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec0306c0 con 0x7f82f8072d30 2024-09-10T10:46:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec037040 con 0x7f82f8072d30 2024-09-10T10:46:54.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f81e5990 con 0x7f82f8072d30 2024-09-10T10:46:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.822+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f81e5c70 con 0x7f82f8072d30 2024-09-10T10:46:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.823+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f80b41f0 con 0x7f82f8072d30 2024-09-10T10:46:54.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.824+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82ec044020 con 0x7f82f8072d30 2024-09-10T10:46:54.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.825+0000 7f82f4c00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 0x7f82d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:54.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.825+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f82ec0ab7a0 con 0x7f82f8072d30 2024-09-10T10:46:54.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.825+0000 7f82f7e00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 0x7f82d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:54.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.826+0000 7f82f7e00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 0x7f82d806e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f82e40098f0 tx=0x7f82e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:54.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:54.829+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82ec04b030 con 0x7f82f8072d30 2024-09-10T10:46:55.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.052+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82f81e8700 con 0x7f82f8072d30 2024-09-10T10:46:55.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.052+0000 7f82f4c00640 1 -- 172.21.15.28:0/2427255333 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5230 (secure 0 0 0) 0x7f82ec079fb0 con 0x7f82f8072d30 2024-09-10T10:46:55.054 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:46:55.054 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":48,"btime":"2024-09-10T10:46:47:884742+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:46:47.308289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5844},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5844":{"gid":5844,"name":"b","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/3429810116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":3429810116},{"type":"v1","addr":"172.21.15.138:6837","nonce":3429810116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5844,"qdb_cluster":[5844]},"id":4}]} 2024-09-10T10:46:55.054 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 48 2024-09-10T10:46:55.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 msgr2=0x7f82d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 0x7f82d806e2f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f82e40098f0 tx=0x7f82e4008040 comp rx=0 tx=0).stop 2024-09-10T10:46:55.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 msgr2=0x7f82f8158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f82ec02a220 tx=0x7f82ec0046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 shutdown_connections 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82d806be30 0x7f82d806e2f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82f815b300 0x7f82f81e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82f80cd3f0 0x7f82f80af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 --2- 172.21.15.28:0/2427255333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82f8072d30 0x7f82f8158ff0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 >> 172.21.15.28:0/2427255333 conn(0x7f82f8067d70 msgr2=0x7f82f815ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 shutdown_connections 2024-09-10T10:46:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.055+0000 7f82ffc00640 1 -- 172.21.15.28:0/2427255333 wait complete. 2024-09-10T10:46:55.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:46:47.308289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5844}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5844': {'gid': 5844, 'name': 'b', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6837/3429810116', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 3429810116}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 3429810116}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5844, 'qdb_cluster': [5844]} 2024-09-10T10:46:55.476 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:46:55.476 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:46:55.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.635+0000 7feebee00640 1 Processor -- start 2024-09-10T10:46:55.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.635+0000 7feebee00640 1 -- start start 2024-09-10T10:46:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feebee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7feeb80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feebee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feeb80725f0 0x7feeb80aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feebee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeb806f480 con 0x7feeb8147230 2024-09-10T10:46:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feebee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feeb80b34d0 con 0x7feeb80725f0 2024-09-10T10:46:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feebee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feeb80ae9e0 con 0x7feeb8147e90 2024-09-10T10:46:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feeb7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7feeb8147230 0x7feeb806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42050/0 (socket says 172.21.15.28:42050) 2024-09-10T10:46:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.636+0000 7feeb7e00640 1 -- 172.21.15.28:0/3601216255 learned_addr learned my addr 172.21.15.28:0/3601216255 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb7400640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7feeb80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feebce00640 1 --2- 172.21.15.28:0/3601216255 >> v2:172.21.15.138:3300/0 conn(0x7feeb80725f0 0x7feeb80aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096468871 0 0) 0x7feeb806f480 con 0x7feeb8147230 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee9c003520 con 0x7feeb8147230 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197323348 0 0) 0x7fee9c003520 con 0x7feeb8147230 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 msgr2=0x7feeb80720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7feeb80720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 >> v2:172.21.15.138:3300/0 conn(0x7feeb80725f0 msgr2=0x7feeb80aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 --2- 172.21.15.28:0/3601216255 >> v2:172.21.15.138:3300/0 conn(0x7feeb80725f0 0x7feeb80aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb806e900 con 0x7feeb8147230 2024-09-10T10:46:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7feea8001670 con 0x7feeb8147230 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 >> v1:172.21.15.28:6789/0 conn(0x7feeb8147230 legacy=0x7feeb806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7fee9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c00b9c0 con 0x7fee9c004d40 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee9c00bb40 con 0x7fee9c0039b0 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.637+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee9c00bcc0 con 0x7feeb8147e90 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feeb7400640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7fee9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 -- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 msgr2=0x7fee9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb8147e90 0x7fee9c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 -- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c0039b0 msgr2=0x7fee9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c0039b0 0x7fee9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.638+0000 7feebce00640 1 -- 172.21.15.28:0/3601216255 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9c003520 con 0x7fee9c004d40 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.639+0000 7feebce00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009fc0 tx=0x7fee9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aae731a5b562ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.639+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0009c10 con 0x7fee9c004d40 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.639+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb0036070 con 0x7fee9c004d40 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.639+0000 7feeb6a00640 1 -- 172.21.15.28:0/3601216255 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb002d070 con 0x7fee9c004d40 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 -- 172.21.15.28:0/3601216255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 msgr2=0x7fee9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 --2- 172.21.15.28:0/3601216255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7fee9c0051c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009fc0 tx=0x7fee9c005830 comp rx=0 tx=0).stop 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebce00640 1 -- 172.21.15.28:0/3601216255 reap_dead start 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 -- 172.21.15.28:0/3601216255 shutdown_connections 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 -- 172.21.15.28:0/3601216255 >> 172.21.15.28:0/3601216255 conn(0x7feeb8067d70 msgr2=0x7feeb806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 -- 172.21.15.28:0/3601216255 shutdown_connections 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.640+0000 7feebee00640 1 -- 172.21.15.28:0/3601216255 wait complete. 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.642+0000 7feebee00640 1 Processor -- start 2024-09-10T10:46:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.642+0000 7feebee00640 1 -- start start 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 0x7feeb8145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 0x7feeb81f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feeb806eec0 con 0x7fee9c004d40 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8072a40 con 0x7feeb81455b0 2024-09-10T10:46:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feeb806f1a0 con 0x7feeb81f4950 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48000/0 (socket says 172.21.15.28:48000) 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 -- 172.21.15.28:0/829894996 learned_addr learned my addr 172.21.15.28:0/829894996 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feebce00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 0x7feeb81f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7400640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 0x7feeb8145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 -- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 msgr2=0x7feeb8145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 0x7feeb8145070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 -- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 msgr2=0x7feeb81f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 0x7feeb81f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.643+0000 7feeb7e00640 1 -- 172.21.15.28:0/829894996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb81f7f10 con 0x7fee9c004d40 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feeb7400640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 0x7feeb8145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feebce00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 0x7feeb81f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feeb7e00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feea8007340 tx=0x7feea8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feeafe00640 1 -- 172.21.15.28:0/829894996 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800f940 con 0x7fee9c004d40 2024-09-10T10:46:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feeafe00640 1 -- 172.21.15.28:0/829894996 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8018070 con 0x7fee9c004d40 2024-09-10T10:46:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feeafe00640 1 -- 172.21.15.28:0/829894996 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea800fd20 con 0x7fee9c004d40 2024-09-10T10:46:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb81f3e70 con 0x7fee9c004d40 2024-09-10T10:46:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.644+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81f4430 con 0x7fee9c004d40 2024-09-10T10:46:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.647+0000 7feeafe00640 1 -- 172.21.15.28:0/829894996 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feea801d460 con 0x7fee9c004d40 2024-09-10T10:46:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.648+0000 7feeafe00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 0x7fee9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.648+0000 7feeafe00640 1 -- 172.21.15.28:0/829894996 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7feea808b1d0 con 0x7fee9c004d40 2024-09-10T10:46:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.648+0000 7feec1c85740 1 Processor -- start 2024-09-10T10:46:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.648+0000 7feeb7400640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 0x7fee9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 -- start start 2024-09-10T10:46:55.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feeb7400640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 0x7fee9806e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feeb80cf620 tx=0x7feea4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d574b0 0x562191d5a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d54570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562191cb83b0 con 0x562191d59cf0 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562191cbd410 con 0x562191d574b0 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.649+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562191c65620 con 0x562191d5b480 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48008/0 (socket says 172.21.15.28:48008) 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 -- 172.21.15.28:0/2369800961 learned_addr learned my addr 172.21.15.28:0/2369800961 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7400640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d574b0 0x562191d5a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feebce00640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d54570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 -- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 msgr2=0x562191d54570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d54570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 -- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d574b0 msgr2=0x562191d5a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d574b0 0x562191d5a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 -- 172.21.15.28:0/2369800961 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621919d02d0 con 0x562191d59cf0 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feebce00640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d54570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.650+0000 7feeb7e00640 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feea8061990 tx=0x7feea8058d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73843e351010d90e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feeacc00640 1 -- 172.21.15.28:0/2369800961 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea805d690 con 0x562191d59cf0 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feeacc00640 1 -- 172.21.15.28:0/2369800961 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8061ec0 con 0x562191d59cf0 2024-09-10T10:46:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feeacc00640 1 -- 172.21.15.28:0/2369800961 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8055950 con 0x562191d59cf0 2024-09-10T10:46:55.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 -- 172.21.15.28:0/2369800961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 msgr2=0x562191d9f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feea8061990 tx=0x7feea8058d00 comp rx=0 tx=0).stop 2024-09-10T10:46:55.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 -- 172.21.15.28:0/2369800961 shutdown_connections 2024-09-10T10:46:55.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d54570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d574b0 0x562191d5a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 --2- 172.21.15.28:0/2369800961 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9f490 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 -- 172.21.15.28:0/2369800961 >> 172.21.15.28:0/2369800961 conn(0x562191d56740 msgr2=0x562191d9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 -- 172.21.15.28:0/2369800961 shutdown_connections 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.651+0000 7feec1c85740 1 -- 172.21.15.28:0/2369800961 wait complete. 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.652+0000 7feec1c85740 1 -- start start 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.652+0000 7feec1c85740 1 Processor -- start 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562191c78780 con 0x562191d59cf0 2024-09-10T10:46:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562191cb93d0 con 0x562191d7afe0 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562191d55be0 con 0x562191d5b480 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feec1c85740 10 client.0 _ll_register_callbacks cb 0x562191cb65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.653+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48014/0 (socket says 172.21.15.28:48014) 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 -- 172.21.15.28:0/3499672056 learned_addr learned my addr 172.21.15.28:0/3499672056 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feebce00640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7400640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 -- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 msgr2=0x562191d7aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 -- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 msgr2=0x562191d784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 -- 172.21.15.28:0/3499672056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562191a9a6b0 con 0x562191d59cf0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7400640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feea8061990 tx=0x7feea805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76bbf59083a533 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feea1600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8061a30 con 0x562191d59cf0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7feea8072d80 con 0x562191d59cf0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.654+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea8061ec0 con 0x562191d59cf0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feea8061bd0 con 0x562191d59cf0 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 10 client.5991 fetch_fsmap learned FSMap version 48 2024-09-10T10:46:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x562191d79930 con 0x562191d59cf0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8061ec0 con 0x562191d59cf0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7feea80781a0 con 0x562191d59cf0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 10 client.5991 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 10 client.5991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 20 client.5991 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db25b0 0x562191db2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 4 client.5991 mds_command: new command op to 5844 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.655+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 --> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562191ccab40 con 0x562191db25b0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.656+0000 7feeb7400640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db25b0 0x562191db2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.656+0000 7feeb7400640 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db25b0 0x562191db2990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61a3213eb3673f2 server_cookie=d0b225a07cc36874 in_seq=0 out_seq=0 2024-09-10T10:46:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.656+0000 7feea1600640 10 client.5991 ms_handle_connect on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:55.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.657+0000 7feea1600640 1 -- 172.21.15.28:0/3499672056 <== mds.0 v2:172.21.15.138:6835/3429810116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562191ccab40 con 0x562191db25b0 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.657+0000 7feea1600640 10 client.5991 handle_command_reply: tid=0 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.657+0000 7feec1c85740 1 client.5991 shutdown 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 msgr2=0x562191d550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feea8061990 tx=0x7feea805c040 comp rx=0 tx=0).stop 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 shutdown_connections 2024-09-10T10:46:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db25b0 0x562191db2990 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 --2- 172.21.15.28:0/3499672056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d550f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feea1600640 0 client.5991 ms_handle_reset on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 >> 172.21.15.28:0/3499672056 conn(0x562191d574b0 msgr2=0x562191d9dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 shutdown_connections 2024-09-10T10:46:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 1 -- 172.21.15.28:0/3499672056 wait complete. 2024-09-10T10:46:55.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.658+0000 7feec1c85740 20 client.5991 trim_cache size 0 max 16384 2024-09-10T10:46:55.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.677+0000 7feec1c85740 1 Processor -- start 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.677+0000 7feec1c85740 1 -- start start 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d5b480 0x562191d9bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 0x562191d9c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8200370 con 0x562191d59cf0 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ff6d0 con 0x562191d5b480 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feeb81ffd20 con 0x562191d7afe0 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48016/0 (socket says 172.21.15.28:48016) 2024-09-10T10:46:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feeb7e00640 1 -- 172.21.15.28:0/3067493734 learned_addr learned my addr 172.21.15.28:0/3067493734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feeb7400640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d5b480 0x562191d9bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.678+0000 7feebce00640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 0x562191d9c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 -- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 msgr2=0x562191d9c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 0x562191d9c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 -- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d5b480 msgr2=0x562191d9bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d5b480 0x562191d9bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 -- 172.21.15.28:0/3067493734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562191d50030 con 0x562191d59cf0 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feebce00640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 0x562191d9c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feea8059220 tx=0x7feea808a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd60c76dc40ac5d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feea3e00640 1 -- 172.21.15.28:0/3067493734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8061a30 con 0x562191d59cf0 2024-09-10T10:46:55.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feea3e00640 1 -- 172.21.15.28:0/3067493734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8086030 con 0x562191d59cf0 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.679+0000 7feea3e00640 1 -- 172.21.15.28:0/3067493734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8058e20 con 0x562191d59cf0 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- 172.21.15.28:0/3067493734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 msgr2=0x562191d9ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feea8059220 tx=0x7feea808a000 comp rx=0 tx=0).stop 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- 172.21.15.28:0/3067493734 shutdown_connections 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d7afe0 0x562191d9c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d5b480 0x562191d9bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 --2- 172.21.15.28:0/3067493734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191d9ba70 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- 172.21.15.28:0/3067493734 >> 172.21.15.28:0/3067493734 conn(0x562191d574b0 msgr2=0x562191dc2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- 172.21.15.28:0/3067493734 shutdown_connections 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- 172.21.15.28:0/3067493734 wait complete. 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.680+0000 7feec1c85740 1 -- start start 2024-09-10T10:46:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.681+0000 7feec1c85740 1 Processor -- start 2024-09-10T10:46:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562191c78780 con 0x562191d59cf0 2024-09-10T10:46:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562191d55be0 con 0x562191d7afe0 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562191cb93d0 con 0x562191d5b480 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feec1c85740 10 client.0 _ll_register_callbacks cb 0x7feea8070070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48022/0 (socket says 172.21.15.28:48022) 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 -- 172.21.15.28:0/3678529542 learned_addr learned my addr 172.21.15.28:0/3678529542 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feebce00640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7400640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 -- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 msgr2=0x562191d7a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 -- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 msgr2=0x562191d7a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d7a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 -- 172.21.15.28:0/3678529542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562191d7fcc0 con 0x562191d59cf0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feeb7e00640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feea8057590 tx=0x7feea8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05cdd02fff32df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.682+0000 7feea1600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea807c070 con 0x562191d59cf0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7feea8058930 con 0x562191d59cf0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea8086030 con 0x562191d59cf0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7feea805d690 con 0x562191d59cf0 2024-09-10T10:46:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 10 client.5997 fetch_fsmap learned FSMap version 48 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x562191d7fcc0 con 0x562191d59cf0 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea8086030 con 0x562191d59cf0 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7feea805d120 con 0x562191d59cf0 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 10 client.5997 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 10 client.5997 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 20 client.5997 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db6280 0x562191db6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 4 client.5997 mds_command: new command op to 5844 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.683+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 --> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] -- command(tid 0: {"prefix": "status"}) -- 0x562191cc4e90 con 0x562191db6280 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.684+0000 7feeb7400640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db6280 0x562191db6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.684+0000 7feeb7400640 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db6280 0x562191db6660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f5ef595e94c228 server_cookie=2f4cb0086a3fefc6 in_seq=0 out_seq=0 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.684+0000 7feea1600640 10 client.5997 ms_handle_connect on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feea1600640 1 -- 172.21.15.28:0/3678529542 <== mds.0 v2:172.21.15.138:6835/3429810116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x562191cc4e90 con 0x562191db6280 2024-09-10T10:46:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feea1600640 10 client.5997 handle_command_reply: tid=0 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 client.5997 shutdown 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 msgr2=0x562191e06240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feea8057590 tx=0x7feea8001a40 comp rx=0 tx=0).stop 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 shutdown_connections 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x562191db6280 0x562191db6660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562191d7afe0 0x562191d7a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562191d5b480 0x562191d7a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feea1600640 0 client.5997 ms_handle_reset on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 --2- 172.21.15.28:0/3678529542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562191d59cf0 0x562191e06240 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 >> 172.21.15.28:0/3678529542 conn(0x562191d574b0 msgr2=0x562191d55f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 shutdown_connections 2024-09-10T10:46:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 1 -- 172.21.15.28:0/3678529542 wait complete. 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.685+0000 7feec1c85740 20 client.5997 trim_cache size 0 max 16384 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5844, 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 10.817083923, 2024-09-10T10:46:55.693 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 48, 2024-09-10T10:46:55.694 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 70, 2024-09-10T10:46:55.694 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 70, 2024-09-10T10:46:55.694 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 15.512838358 2024-09-10T10:46:55.694 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:55.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 msgr2=0x7fee9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 0x7fee9806e2f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feeb80cf620 tx=0x7feea4008040 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 msgr2=0x7feeb81f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feea8007340 tx=0x7feea8007800 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 shutdown_connections 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee9806be30 0x7fee9806e2f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feeb81455b0 0x7feeb81f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feeb81f4950 0x7feeb8145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 --2- 172.21.15.28:0/829894996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee9c004d40 0x7feeb81f35c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 >> 172.21.15.28:0/829894996 conn(0x7feeb8067d70 msgr2=0x7feeb806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 shutdown_connections 2024-09-10T10:46:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.686+0000 7feebee00640 1 -- 172.21.15.28:0/829894996 wait complete. 2024-09-10T10:46:55.703 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:46:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.858+0000 7f754ae00640 1 Processor -- start 2024-09-10T10:46:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.858+0000 7f754ae00640 1 -- start start 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.859+0000 7f754ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f754415b300 0x7f75440cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.859+0000 7f754ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75440cdbe0 con 0x7f754406f8d0 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.859+0000 7f754ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75440cd900 con 0x7f75440cd3f0 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.859+0000 7f754ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f75440728a0 con 0x7f754415b300 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f754415b300 0x7f75440cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f754415b300 0x7f75440cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48026/0 (socket says 172.21.15.28:48026) 2024-09-10T10:46:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 -- 172.21.15.28:0/3261277849 learned_addr learned my addr 172.21.15.28:0/3261277849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7548e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f754406f8d0 0x7f754415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42064/0 (socket says 172.21.15.28:42064) 2024-09-10T10:46:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 -- 172.21.15.28:0/3261277849 >> v1:172.21.15.138:6789/0 conn(0x7f75440cd3f0 legacy=0x7f754406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 -- 172.21.15.28:0/3261277849 >> v1:172.21.15.28:6789/0 conn(0x7f754406f8d0 legacy=0x7f754415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.860+0000 7f7543e00640 1 -- 172.21.15.28:0/3261277849 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7544072250 con 0x7f754415b300 2024-09-10T10:46:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.861+0000 7f7543e00640 1 --2- 172.21.15.28:0/3261277849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754415b300 0x7f75440cceb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7534009870 tx=0x7f753402c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f7473cc4f299fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.861+0000 7f7543400640 1 -- 172.21.15.28:0/3261277849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534038070 con 0x7f754415b300 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.861+0000 7f7543400640 1 -- 172.21.15.28:0/3261277849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7534031710 con 0x7f754415b300 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.861+0000 7f7543400640 1 -- 172.21.15.28:0/3261277849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534031ae0 con 0x7f754415b300 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 -- 172.21.15.28:0/3261277849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754415b300 msgr2=0x7f75440cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 --2- 172.21.15.28:0/3261277849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754415b300 0x7f75440cceb0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7534009870 tx=0x7f753402c4b0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 -- 172.21.15.28:0/3261277849 shutdown_connections 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 --2- 172.21.15.28:0/3261277849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754415b300 0x7f75440cceb0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 -- 172.21.15.28:0/3261277849 >> 172.21.15.28:0/3261277849 conn(0x7f7544067d70 msgr2=0x7f754406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 -- 172.21.15.28:0/3261277849 shutdown_connections 2024-09-10T10:46:55.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.862+0000 7f754ae00640 1 -- 172.21.15.28:0/3261277849 wait complete. 2024-09-10T10:46:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.864+0000 7f754ae00640 1 Processor -- start 2024-09-10T10:46:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.864+0000 7f754ae00640 1 -- start start 2024-09-10T10:46:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 0x7f75441e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 0x7f75441e5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f75440d0160 con 0x7f754406f8d0 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75440cffe0 con 0x7f75440cd3f0 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f754ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7544072d30 con 0x7f754415b300 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48042/0 (socket says 172.21.15.28:48042) 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 -- 172.21.15.28:0/3900533283 learned_addr learned my addr 172.21.15.28:0/3900533283 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f753be00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 0x7f75441e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7548e00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 0x7f75441e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 -- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 msgr2=0x7f75441e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 0x7f75441e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 -- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 msgr2=0x7f75441e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 0x7f75441e53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7543e00640 1 -- 172.21.15.28:0/3900533283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754415ca20 con 0x7f754406f8d0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f7548e00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 0x7f75441e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.865+0000 7f753be00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 0x7f75441e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f7543e00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7534009870 tx=0x7f75340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534038040 con 0x7f754406f8d0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7534043070 con 0x7f754406f8d0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534033a70 con 0x7f754406f8d0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754415cd00 con 0x7f754406f8d0 2024-09-10T10:46:55.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.866+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754415d2c0 con 0x7f754406f8d0 2024-09-10T10:46:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.867+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7544159040 con 0x7f754406f8d0 2024-09-10T10:46:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.868+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7534033c10 con 0x7f754406f8d0 2024-09-10T10:46:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.869+0000 7f7540c00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 0x7f752006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.869+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f75340abe50 con 0x7f754406f8d0 2024-09-10T10:46:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.870+0000 7f753be00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 0x7f752006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.870+0000 7f753be00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 0x7f752006e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f754415e230 tx=0x7f752c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:55.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:55.873+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7534046360 con 0x7f754406f8d0 2024-09-10T10:46:56.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.094+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7544158d60 con 0x7f754406f8d0 2024-09-10T10:46:56.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.094+0000 7f7540c00640 1 -- 172.21.15.28:0/3900533283 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f753407a5d0 con 0x7f754406f8d0 2024-09-10T10:46:56.095 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 msgr2=0x7f752006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 0x7f752006e2f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f754415e230 tx=0x7f752c009040 comp rx=0 tx=0).stop 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 msgr2=0x7f75441e36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7534009870 tx=0x7f75340046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 shutdown_connections 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f752006be30 0x7f752006e2f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f754415b300 0x7f75441e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75440cd3f0 0x7f75441e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 --2- 172.21.15.28:0/3900533283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f754406f8d0 0x7f75441e36f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 >> 172.21.15.28:0/3900533283 conn(0x7f7544067d70 msgr2=0x7f754406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 shutdown_connections 2024-09-10T10:46:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56.097+0000 7f754ae00640 1 -- 172.21.15.28:0/3900533283 wait complete. 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:46:56.115 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:56.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.116 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:46:56.152 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42489sec preferred_lft 42489sec 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:46:56.153 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:46:56.154 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:56.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.155 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:46:56.246 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:46:56.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.249 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:46:56.285 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:46:56.285 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:46:56.285 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:46:56.285 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:46:56.285 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:46:56.285 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:46:56.285 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:46:56.286 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:46:56.286 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:46:56.286 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:46:56.286 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:46:56.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:46:56.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:56.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.483 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:46:56.518 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:46:56.518 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:46:56.518 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:56.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.519 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:46:56.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:56.616 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:56.651 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:56.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:56.651 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:56.725 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:46:56.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:56.855 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:56.855 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:56.855 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:56.870 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:56.870 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:56.870 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:56.870 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:56.871 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:56.893 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:56.893 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:56.893 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:56.893 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:56.893 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:56.894 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:46:56.894 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:46:56.894 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:46:56.894 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:46:56.894 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:46:56.895 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:46:56.895 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:46:56.895 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:46:56.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:46:56.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:56.898 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:56.955 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:46:57.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:57.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:57.057 INFO:teuthology.orchestra.run.smithi028.stdout:client6009 2024-09-10T10:46:57.057 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:57.057 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:57.093 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d; sync .') 2024-09-10T10:46:57.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:57.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:57.242 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:57.242 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:57.278 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:46:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.475+0000 7f2295000640 1 Processor -- start 2024-09-10T10:46:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.475+0000 7f2295000640 1 -- start start 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2295000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2295000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22900cdbe0 con 0x7f2290072d30 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2295000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22900cd900 con 0x7f229015b300 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2295000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22900728a0 con 0x7f229006f8d0 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2285400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2290072d30 0x7f229006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42072/0 (socket says 172.21.15.28:42072) 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f2285400640 1 -- 172.21.15.28:0/923410182 learned_addr learned my addr 172.21.15.28:0/923410182 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f228e800640 1 --2- 172.21.15.28:0/923410182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115584233 0 0) 0x7f22900cdbe0 con 0x7f2290072d30 2024-09-10T10:46:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.477+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2274005730 con 0x7f2290072d30 2024-09-10T10:46:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730258461 0 0) 0x7f22900cd900 con 0x7f229015b300 2024-09-10T10:46:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22900cdbe0 con 0x7f229015b300 2024-09-10T10:46:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2644470081 0 0) 0x7f2274005730 con 0x7f2290072d30 2024-09-10T10:46:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228e800640 1 -- 172.21.15.28:0/923410182 >> v1:172.21.15.138:6789/0 conn(0x7f229015b300 legacy=0x7f22900cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228e800640 1 -- 172.21.15.28:0/923410182 >> v1:172.21.15.28:6789/0 conn(0x7f2290072d30 legacy=0x7f229006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.478+0000 7f228e800640 1 -- 172.21.15.28:0/923410182 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2290072250 con 0x7f229006f8d0 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.479+0000 7f228e800640 1 --2- 172.21.15.28:0/923410182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290165de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2288009dc0 tx=0x7f228802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea56b87a1d92568 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.479+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278001670 con 0x7f229006f8d0 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.479+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2288007540 con 0x7f229006f8d0 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.479+0000 7f228d400640 1 -- 172.21.15.28:0/923410182 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22880078f0 con 0x7f229006f8d0 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 -- 172.21.15.28:0/923410182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 msgr2=0x7f2290165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 --2- 172.21.15.28:0/923410182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290165de0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2288009dc0 tx=0x7f228802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 -- 172.21.15.28:0/923410182 shutdown_connections 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 --2- 172.21.15.28:0/923410182 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290165de0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 -- 172.21.15.28:0/923410182 >> 172.21.15.28:0/923410182 conn(0x7f2290067d70 msgr2=0x7f229006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:57.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 -- 172.21.15.28:0/923410182 shutdown_connections 2024-09-10T10:46:57.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.480+0000 7f2295000640 1 -- 172.21.15.28:0/923410182 wait complete. 2024-09-10T10:46:57.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.482+0000 7f2295000640 1 Processor -- start 2024-09-10T10:46:57.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.482+0000 7f2295000640 1 -- start start 2024-09-10T10:46:57.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:57.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:57.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 0x7f22900b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f22900cffe0 con 0x7f2290072d30 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22900cd5c0 con 0x7f229015b300 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2295000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22900d0160 con 0x7f229006f8d0 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48056/0 (socket says 172.21.15.28:48056) 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 -- 172.21.15.28:0/3290441129 learned_addr learned my addr 172.21.15.28:0/3290441129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:57.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f228de00640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:57.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f228e800640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 0x7f22900b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:57.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 -- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 msgr2=0x7f2290151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 -- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 msgr2=0x7f22900b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 0x7f22900b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f2285400640 1 -- 172.21.15.28:0/3290441129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22900b1430 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f228de00640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.483+0000 7f228e800640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 0x7f22900b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2285400640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f227800c770 tx=0x7f227800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278017070 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2278005ad0 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278007070 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22900b1710 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.484+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22900b1cd0 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.485+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22900b1ed0 con 0x7f2290072d30 2024-09-10T10:46:57.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.486+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f227801a070 con 0x7f2290072d30 2024-09-10T10:46:57.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.487+0000 7f2286a00640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:57.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.487+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f227808a4a0 con 0x7f2290072d30 2024-09-10T10:46:57.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.488+0000 7f228de00640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:57.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.488+0000 7f228de00640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 0x7f226c06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2280001610 tx=0x7f2280009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:57.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.492+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f22900b1ed0 con 0x7f2290072d30 2024-09-10T10:46:57.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.689+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f229006e710 con 0x7f2290072d30 2024-09-10T10:46:57.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.689+0000 7f2286a00640 1 -- 172.21.15.28:0/3290441129 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7f2278058660 con 0x7f2290072d30 2024-09-10T10:46:57.691 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.691+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 msgr2=0x7f226c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.691+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 0x7f226c06e2f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2280001610 tx=0x7f2280009040 comp rx=0 tx=0).stop 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.691+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 msgr2=0x7f22900b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f227800c770 tx=0x7f227800cc30 comp rx=0 tx=0).stop 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 shutdown_connections 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f226c06be30 0x7f226c06e2f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229015b300 0x7f22900b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2290072d30 0x7f22900b21f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 --2- 172.21.15.28:0/3290441129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f229006f8d0 0x7f2290151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:57.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 >> 172.21.15.28:0/3290441129 conn(0x7f2290067d70 msgr2=0x7f229006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:57.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 shutdown_connections 2024-09-10T10:46:57.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:57.692+0000 7f2295000640 1 -- 172.21.15.28:0/3290441129 wait complete. 2024-09-10T10:46:57.708 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:46:57.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:57.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:57.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:46:57.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:57.774 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:46:57.810 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-10T10:46:58.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.008+0000 7fd4d7200640 1 Processor -- start 2024-09-10T10:46:58.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.008+0000 7fd4d7200640 1 -- start start 2024-09-10T10:46:58.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4d80b3400 0x7fd4d80d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d7200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d80cee50 con 0x7fd4d815e550 2024-09-10T10:46:58.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d80d4ca0 con 0x7fd4d80d2920 2024-09-10T10:46:58.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d7200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d80d4a50 con 0x7fd4d80b3400 2024-09-10T10:46:58.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d5400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4d80b3400 0x7fd4d80d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd4d815e550 0x7fd4d806cd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42080/0 (socket says 172.21.15.28:42080) 2024-09-10T10:46:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d5e00640 1 -- 172.21.15.28:0/1835825397 learned_addr learned my addr 172.21.15.28:0/1835825397 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.009+0000 7fd4d5400640 1 --2- 172.21.15.28:0/1835825397 >> v2:172.21.15.28:3300/0 conn(0x7fd4d80b3400 0x7fd4d80d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48062/0 (socket says 172.21.15.28:48062) 2024-09-10T10:46:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.010+0000 7fd4d5400640 1 -- 172.21.15.28:0/1835825397 >> v1:172.21.15.138:6789/0 conn(0x7fd4d80d2920 legacy=0x7fd4d80d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.010+0000 7fd4d5400640 1 -- 172.21.15.28:0/1835825397 >> v1:172.21.15.28:6789/0 conn(0x7fd4d815e550 legacy=0x7fd4d806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.010+0000 7fd4d5400640 1 -- 172.21.15.28:0/1835825397 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d80d4f80 con 0x7fd4d80b3400 2024-09-10T10:46:58.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d5400640 1 --2- 172.21.15.28:0/1835825397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80b3400 0x7fd4d80d1f10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097a0 tx=0x7fd4c402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466fb751071abf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d4a00640 1 -- 172.21.15.28:0/1835825397 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4037070 con 0x7fd4d80b3400 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d4a00640 1 -- 172.21.15.28:0/1835825397 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c402adc0 con 0x7fd4d80b3400 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d4a00640 1 -- 172.21.15.28:0/1835825397 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4032660 con 0x7fd4d80b3400 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d7200640 1 -- 172.21.15.28:0/1835825397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80b3400 msgr2=0x7fd4d80d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.011+0000 7fd4d7200640 1 --2- 172.21.15.28:0/1835825397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80b3400 0x7fd4d80d1f10 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40097a0 tx=0x7fd4c402a0b0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.012+0000 7fd4d7200640 1 -- 172.21.15.28:0/1835825397 shutdown_connections 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.012+0000 7fd4d7200640 1 --2- 172.21.15.28:0/1835825397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80b3400 0x7fd4d80d1f10 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.012+0000 7fd4d7200640 1 -- 172.21.15.28:0/1835825397 >> 172.21.15.28:0/1835825397 conn(0x7fd4d808bfc0 msgr2=0x7fd4d80d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.012+0000 7fd4d7200640 1 -- 172.21.15.28:0/1835825397 shutdown_connections 2024-09-10T10:46:58.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.012+0000 7fd4d7200640 1 -- 172.21.15.28:0/1835825397 wait complete. 2024-09-10T10:46:58.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.013+0000 7fd4d7200640 1 Processor -- start 2024-09-10T10:46:58.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.013+0000 7fd4d7200640 1 -- start start 2024-09-10T10:46:58.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4d80b3400 0x7fd4d8158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d815e550 0x7fd4d80cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d80cf610 con 0x7fd4d80d2920 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d80cf180 con 0x7fd4d815e550 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.014+0000 7fd4d7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d80cf790 con 0x7fd4d80b3400 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48078/0 (socket says 172.21.15.28:48078) 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 -- 172.21.15.28:0/81511106 learned_addr learned my addr 172.21.15.28:0/81511106 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4d80b3400 0x7fd4d8158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d6800640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d815e550 0x7fd4d80cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 -- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4d80b3400 msgr2=0x7fd4d8158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4d80b3400 0x7fd4d8158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 -- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d815e550 msgr2=0x7fd4d80cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d815e550 0x7fd4d80cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 -- 172.21.15.28:0/81511106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d8158b50 con 0x7fd4d80d2920 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.015+0000 7fd4d5400640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098c0 tx=0x7fd4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.016+0000 7fd4ce000640 1 -- 172.21.15.28:0/81511106 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4037040 con 0x7fd4d80d2920 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.016+0000 7fd4ce000640 1 -- 172.21.15.28:0/81511106 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c4042070 con 0x7fd4d80d2920 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.016+0000 7fd4ce000640 1 -- 172.21.15.28:0/81511106 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c403b870 con 0x7fd4d80d2920 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.016+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d81543b0 con 0x7fd4d80d2920 2024-09-10T10:46:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.016+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d81540d0 con 0x7fd4d80d2920 2024-09-10T10:46:58.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.018+0000 7fd4ce000640 1 -- 172.21.15.28:0/81511106 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4c403ba10 con 0x7fd4d80d2920 2024-09-10T10:46:58.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.019+0000 7fd4ce000640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 0x7fd4b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.019+0000 7fd4ce000640 1 -- 172.21.15.28:0/81511106 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd4c40ab600 con 0x7fd4d80d2920 2024-09-10T10:46:58.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.019+0000 7fd4dfe98740 1 Processor -- start 2024-09-10T10:46:58.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.019+0000 7fd4dfe98740 1 -- start start 2024-09-10T10:46:58.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.019+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 0x7fd4b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 0x7fd4b806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f60 tx=0x7fd4c8007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906dc9e0 0x5568906db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x5568907273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556890644660 con 0x5568906f64f0 2024-09-10T10:46:58.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568905ec680 con 0x5568906dc9e0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556890640620 con 0x556890726ea0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48092/0 (socket says 172.21.15.28:48092) 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.020+0000 7fd4d5e00640 1 -- 172.21.15.28:0/1404406168 learned_addr learned my addr 172.21.15.28:0/1404406168 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5400640 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906dc9e0 0x5568906db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d6800640 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x5568907273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 -- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 msgr2=0x5568907273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x5568907273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 -- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906dc9e0 msgr2=0x5568906db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906dc9e0 0x5568906db240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 -- 172.21.15.28:0/1404406168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568904217d0 con 0x5568906f64f0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8001770 tx=0x7fd4c8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4836bde0e300e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4c2a00640 1 -- 172.21.15.28:0/1404406168 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8025070 con 0x5568906f64f0 2024-09-10T10:46:58.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4c2a00640 1 -- 172.21.15.28:0/1404406168 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c801b050 con 0x5568906f64f0 2024-09-10T10:46:58.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.021+0000 7fd4c2a00640 1 -- 172.21.15.28:0/1404406168 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8014bd0 con 0x5568906f64f0 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 -- 172.21.15.28:0/1404406168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 msgr2=0x5568906f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8001770 tx=0x7fd4c8012000 comp rx=0 tx=0).stop 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 -- 172.21.15.28:0/1404406168 shutdown_connections 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x5568907273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906dc9e0 0x5568906db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/1404406168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568906f64f0 0x5568906f68d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 -- 172.21.15.28:0/1404406168 >> 172.21.15.28:0/1404406168 conn(0x5568906dd410 msgr2=0x556890725a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 -- 172.21.15.28:0/1404406168 shutdown_connections 2024-09-10T10:46:58.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.022+0000 7fd4dfe98740 1 -- 172.21.15.28:0/1404406168 wait complete. 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.023+0000 7fd4dfe98740 1 -- start start 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.023+0000 7fd4dfe98740 1 Processor -- start 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906f64f0 0x55689071c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x556890735ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568906db940 con 0x55689071bd50 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568905cd4c0 con 0x5568906f64f0 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568906dbc20 con 0x556890726ea0 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4dfe98740 10 client.0 _ll_register_callbacks cb 0x55689063ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48094/0 (socket says 172.21.15.28:48094) 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4d6800640 1 -- 172.21.15.28:0/3155794228 learned_addr learned my addr 172.21.15.28:0/3155794228 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906f64f0 0x55689071c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.024+0000 7fd4d5400640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x556890735ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 -- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 msgr2=0x556890735ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x556890735ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 -- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906f64f0 msgr2=0x55689071c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906f64f0 0x55689071c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 -- 172.21.15.28:0/3155794228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568903573f0 con 0x55689071bd50 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4d6800640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011690 tx=0x7fd4d0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d494c3cf93373306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4b7400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001b020 con 0x55689071bd50 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd4d000f7b0 con 0x55689071bd50 2024-09-10T10:46:58.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c801b050 con 0x55689071bd50 2024-09-10T10:46:58.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd4d0025050 con 0x55689071bd50 2024-09-10T10:46:58.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4dfe98740 10 client.6030 fetch_fsmap learned FSMap version 48 2024-09-10T10:46:58.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.025+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x5568903170f0 con 0x55689071bd50 2024-09-10T10:46:58.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c801b050 con 0x55689071bd50 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7fd4d0028020 con 0x55689071bd50 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 10 client.6030 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 10 client.6030 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 20 client.6030 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x55689071e5d0 0x55689071e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 4 client.6030 mds_command: new command op to 5844 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 --> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556890651d90 con 0x55689071e5d0 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.026+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x55689071e5d0 0x55689071e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.027+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x55689071e5d0 0x55689071e9d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1948b93505cc03e8 server_cookie=e8cf7d90793efe36 in_seq=0 out_seq=0 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.027+0000 7fd4b7400640 10 client.6030 ms_handle_connect on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4b7400640 1 -- 172.21.15.28:0/3155794228 <== mds.0 v2:172.21.15.138:6835/3429810116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556890651d90 con 0x55689071e5d0 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4b7400640 10 client.6030 handle_command_reply: tid=0 2024-09-10T10:46:58.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 client.6030 shutdown 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 msgr2=0x5568907196a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011690 tx=0x7fd4d0011b50 comp rx=0 tx=0).stop 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 shutdown_connections 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x55689071e5d0 0x55689071e9d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568907196a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556890726ea0 0x556890735ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/3155794228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568906f64f0 0x55689071c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4b7400640 0 client.6030 ms_handle_reset on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.028+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 >> 172.21.15.28:0/3155794228 conn(0x5568906dc9e0 msgr2=0x5568906da140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.029+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 shutdown_connections 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.029+0000 7fd4dfe98740 1 -- 172.21.15.28:0/3155794228 wait complete. 2024-09-10T10:46:58.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.029+0000 7fd4dfe98740 20 client.6030 trim_cache size 0 max 16384 2024-09-10T10:46:58.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.044+0000 7fd4dfe98740 1 Processor -- start 2024-09-10T10:46:58.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.044+0000 7fd4dfe98740 1 -- start start 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x556890748f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556890726ea0 0x5568906df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d8200f70 con 0x55689071bd50 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81fffe0 con 0x556890726ea0 2024-09-10T10:46:58.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d81f7fc0 con 0x5568906f64f0 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48098/0 (socket says 172.21.15.28:48098) 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4d5400640 1 -- 172.21.15.28:0/2069011637 learned_addr learned my addr 172.21.15.28:0/2069011637 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x556890748f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.045+0000 7fd4d6800640 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556890726ea0 0x5568906df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 -- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 msgr2=0x556890748f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x556890748f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 -- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556890726ea0 msgr2=0x5568906df730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556890726ea0 0x5568906df730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 -- 172.21.15.28:0/2069011637 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568906f6a70 con 0x55689071bd50 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4d5400640 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4081dd0 tx=0x7fd4c40ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d94a7d7ff07b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4c2000640 1 -- 172.21.15.28:0/2069011637 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c407dad0 con 0x55689071bd50 2024-09-10T10:46:58.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4c2000640 1 -- 172.21.15.28:0/2069011637 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c4082300 con 0x55689071bd50 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.046+0000 7fd4c2000640 1 -- 172.21.15.28:0/2069011637 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4075e40 con 0x55689071bd50 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- 172.21.15.28:0/2069011637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 msgr2=0x5568906df1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4081dd0 tx=0x7fd4c40ab040 comp rx=0 tx=0).stop 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- 172.21.15.28:0/2069011637 shutdown_connections 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556890726ea0 0x5568906df730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55689071bd50 0x5568906df1f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/2069011637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x556890748f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- 172.21.15.28:0/2069011637 >> 172.21.15.28:0/2069011637 conn(0x5568906dc9e0 msgr2=0x55689071d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- 172.21.15.28:0/2069011637 shutdown_connections 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- 172.21.15.28:0/2069011637 wait complete. 2024-09-10T10:46:58.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.047+0000 7fd4dfe98740 1 -- start start 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.048+0000 7fd4dfe98740 1 Processor -- start 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.048+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x55689078e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.048+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55689071bd50 0x556890734150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4dfe98740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568906deb70 con 0x556890726ea0 2024-09-10T10:46:58.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d81f8610 con 0x55689071bd50 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4dfe98740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556890722950 con 0x5568906f64f0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4dfe98740 10 client.0 _ll_register_callbacks cb 0x7fd4d0005150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48100/0 (socket says 172.21.15.28:48100) 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 -- 172.21.15.28:0/901552657 learned_addr learned my addr 172.21.15.28:0/901552657 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x55689078e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d5400640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55689071bd50 0x556890734150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 -- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 msgr2=0x55689078e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x55689078e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 -- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55689071bd50 msgr2=0x556890734150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55689071bd50 0x556890734150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.049+0000 7fd4d6800640 1 -- 172.21.15.28:0/901552657 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556890734a30 con 0x556890726ea0 2024-09-10T10:46:58.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4d6800640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011100 tx=0x7fd4d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900989a6843770c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d002d070 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fd4d0017690 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c4082300 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd4d001b040 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 10 client.6036 fetch_fsmap learned FSMap version 48 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) -- 0x556890722410 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c4082300 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 48) ==== 3502+0+0 (secure 0 0 0) 0x7fd4d003b080 con 0x556890726ea0 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 10 client.6036 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 10 client.6036 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 20 client.6036 populate_metadata read hostname 'smithi028' 2024-09-10T10:46:58.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x5568906ffb50 0x5568906df1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 4 client.6036 mds_command: new command op to 5844 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:46:58.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.050+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 --> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55689064c0e0 con 0x5568906ffb50 2024-09-10T10:46:58.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.051+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x5568906ffb50 0x5568906df1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.051+0000 7fd4d5e00640 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x5568906ffb50 0x5568906df1f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f10949f1700012 server_cookie=f665ed5761ee30ac in_seq=0 out_seq=0 2024-09-10T10:46:58.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.051+0000 7fd4b7400640 10 client.6036 ms_handle_connect on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4b7400640 1 -- 172.21.15.28:0/901552657 <== mds.0 v2:172.21.15.138:6835/3429810116 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55689064c0e0 con 0x5568906ffb50 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4b7400640 10 client.6036 handle_command_reply: tid=0 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 client.6036 shutdown 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 msgr2=0x556890721ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0011100 tx=0x7fd4d0014000 comp rx=0 tx=0).stop 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 shutdown_connections 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:6835/3429810116,v1:172.21.15.138:6837/3429810116] conn(0x5568906ffb50 0x5568906df1f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556890726ea0 0x556890721ed0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55689071bd50 0x556890734150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 --2- 172.21.15.28:0/901552657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568906f64f0 0x55689078e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4b7400640 0 client.6036 ms_handle_reset on v2:172.21.15.138:6835/3429810116 2024-09-10T10:46:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.064+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 >> 172.21.15.28:0/901552657 conn(0x5568906dc9e0 msgr2=0x55689071c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 shutdown_connections 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4dfe98740 1 -- 172.21.15.28:0/901552657 wait complete. 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4dfe98740 20 client.6036 trim_cache size 0 max 16384 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:46:58.067 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 msgr2=0x7fd4b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 0x7fd4b806e2f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f60 tx=0x7fd4c8007e80 comp rx=0 tx=0).stop 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 msgr2=0x7fd4d806dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40098c0 tx=0x7fd4c40046c0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 shutdown_connections 2024-09-10T10:46:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4b806be30 0x7fd4b806e2f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4d815e550 0x7fd4d80cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4d80d2920 0x7fd4d806dcd0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 --2- 172.21.15.28:0/81511106 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4d80b3400 0x7fd4d8158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.065+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 >> 172.21.15.28:0/81511106 conn(0x7fd4d808bfc0 msgr2=0x7fd4d80b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.066+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 shutdown_connections 2024-09-10T10:46:58.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.066+0000 7fd4d7200640 1 -- 172.21.15.28:0/81511106 wait complete. 2024-09-10T10:46:58.082 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:46:58.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.237+0000 7f46d4a00640 1 Processor -- start 2024-09-10T10:46:58.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.237+0000 7f46d4a00640 1 -- start start 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46d4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46d0072320 0x7f46d00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46d4a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d0146140 con 0x7f46d015e1f0 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46d4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46d006eed0 con 0x7f46d0072de0 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46d4a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f46d00cd740 con 0x7f46d0072320 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46ce800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f46d015e1f0 0x7f46d006e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42090/0 (socket says 172.21.15.28:42090) 2024-09-10T10:46:58.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.238+0000 7f46ce800640 1 -- 172.21.15.28:0/3133626239 learned_addr learned my addr 172.21.15.28:0/3133626239 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:58.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cde00640 1 --2- 172.21.15.28:0/3133626239 >> v2:172.21.15.138:3300/0 conn(0x7f46d0072320 0x7f46d00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cca00640 1 -- 172.21.15.28:0/3133626239 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544357934 0 0) 0x7f46d0146140 con 0x7f46d015e1f0 2024-09-10T10:46:58.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cca00640 1 -- 172.21.15.28:0/3133626239 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b4003520 con 0x7f46d015e1f0 2024-09-10T10:46:58.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cde00640 1 -- 172.21.15.28:0/3133626239 >> v1:172.21.15.138:6789/0 conn(0x7f46d0072de0 legacy=0x7f46d006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cde00640 1 -- 172.21.15.28:0/3133626239 >> v1:172.21.15.28:6789/0 conn(0x7f46d015e1f0 legacy=0x7f46d006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.239+0000 7f46cde00640 1 -- 172.21.15.28:0/3133626239 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d00cda20 con 0x7f46d0072320 2024-09-10T10:46:58.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.240+0000 7f46cde00640 1 --2- 172.21.15.28:0/3133626239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d00cce60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097a0 tx=0x7f46c002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c5afe01c24ade81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46cca00640 1 -- 172.21.15.28:0/3133626239 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0037070 con 0x7f46d0072320 2024-09-10T10:46:58.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46cca00640 1 -- 172.21.15.28:0/3133626239 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c0032440 con 0x7f46d0072320 2024-09-10T10:46:58.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46cca00640 1 -- 172.21.15.28:0/3133626239 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0032810 con 0x7f46d0072320 2024-09-10T10:46:58.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 -- 172.21.15.28:0/3133626239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 msgr2=0x7f46d00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 --2- 172.21.15.28:0/3133626239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d00cce60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f46c00097a0 tx=0x7f46c002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 -- 172.21.15.28:0/3133626239 shutdown_connections 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 --2- 172.21.15.28:0/3133626239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d00cce60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 -- 172.21.15.28:0/3133626239 >> 172.21.15.28:0/3133626239 conn(0x7f46d0067d70 msgr2=0x7f46d006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 -- 172.21.15.28:0/3133626239 shutdown_connections 2024-09-10T10:46:58.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.241+0000 7f46d4a00640 1 -- 172.21.15.28:0/3133626239 wait complete. 2024-09-10T10:46:58.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.243+0000 7f46d4a00640 1 Processor -- start 2024-09-10T10:46:58.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.243+0000 7f46d4a00640 1 -- start start 2024-09-10T10:46:58.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d0141110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 0x7f46d00cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f46d00cd3a0 con 0x7f46d0072de0 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46d015c9e0 con 0x7f46d0072320 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46d4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f46d00cd520 con 0x7f46d015e1f0 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46cd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46cd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48108/0 (socket says 172.21.15.28:48108) 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46cd400640 1 -- 172.21.15.28:0/4013045732 learned_addr learned my addr 172.21.15.28:0/4013045732 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46cde00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d0141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.244+0000 7f46ce800640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 0x7f46d00cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 -- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 msgr2=0x7f46d00cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 0x7f46d00cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 -- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 msgr2=0x7f46d0141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d0141110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 -- 172.21.15.28:0/4013045732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01e7390 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46ce800640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 0x7f46d00cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46cd400640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c400 tx=0x7f46bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0052d0 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46bc005490 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc007260 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d01e7670 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.245+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01e7c30 con 0x7f46d0072de0 2024-09-10T10:46:58.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.246+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d0072700 con 0x7f46d0072de0 2024-09-10T10:46:58.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.247+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46bc007400 con 0x7f46d0072de0 2024-09-10T10:46:58.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.248+0000 7f46c6000640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:58.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.248+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f46bc08a520 con 0x7f46d0072de0 2024-09-10T10:46:58.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.249+0000 7f46cde00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:58.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.249+0000 7f46cde00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 0x7f46b006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46c0034000 tx=0x7f46c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:58.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.252+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46bc058650 con 0x7f46d0072de0 2024-09-10T10:46:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.472+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46d0144fd0 con 0x7f46d0072de0 2024-09-10T10:46:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.473+0000 7f46c6000640 1 -- 172.21.15.28:0/4013045732 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) ==== 53+0+73 (secure 0 0 0) 0x7f46bc05c950 con 0x7f46d0072de0 2024-09-10T10:46:58.474 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 msgr2=0x7f46b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 0x7f46b006e2f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f46c0034000 tx=0x7f46c002e000 comp rx=0 tx=0).stop 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 msgr2=0x7f46d00ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00c400 tx=0x7f46bc00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 shutdown_connections 2024-09-10T10:46:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46b006be30 0x7f46b006e2f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46d015e1f0 0x7f46d00cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46d0072de0 0x7f46d00ca810 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 --2- 172.21.15.28:0/4013045732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46d0072320 0x7f46d0141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 >> 172.21.15.28:0/4013045732 conn(0x7f46d0067d70 msgr2=0x7f46d00ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 shutdown_connections 2024-09-10T10:46:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58.475+0000 7f46d4a00640 1 -- 172.21.15.28:0/4013045732 wait complete. 2024-09-10T10:46:58.492 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:46:58.492 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:46:58.492 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:46:58.492 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:46:58.492 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:58.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:46:58.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:46:58.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:46:58.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:58.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:58.493 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:46:58.529 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:46:58.529 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:46:58.529 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:46:58.529 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:46:58.530 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42486sec preferred_lft 42486sec 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.531 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:46:58.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:58.532 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:46:58.624 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:46:58.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:58.627 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:46:58.664 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:46:58.665 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:46:58.665 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:46:58.665 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:46:58.665 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:46:58.666 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:46:58.666 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:46:58.666 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:46:58.666 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:46:58.666 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:46:58.667 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:46:58.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:46:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:58.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:58.862 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:46:58.897 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:46:58.897 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:46:58.898 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:58.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:58.898 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:46:58.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:58.995 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:59.031 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:46:59.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:46:59.032 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:59.107 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:46:59.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:59.108 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:59.236 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:59.237 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:59.237 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:59.252 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:59.252 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:59.252 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:59.252 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:59.253 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:59.275 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:46:59.275 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:46:59.275 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:46:59.275 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:46:59.275 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:46:59.276 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:46:59.276 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:46:59.276 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:46:59.278 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:46:59.278 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:46:59.278 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:46:59.278 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:46:59.278 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:46:59.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:46:59.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:46:59.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:46:59.337 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:46:59.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:59.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:59.427 INFO:teuthology.orchestra.run.smithi028.stdout:client5387 2024-09-10T10:46:59.427 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:59.427 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:59.463 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 711 a/b/d; sync .') 2024-09-10T10:46:59.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:46:59.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:46:59.598 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:46:59.598 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:46:59.634 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:46:59.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.830+0000 7fdf7e000640 1 Processor -- start 2024-09-10T10:46:59.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.830+0000 7fdf7e000640 1 -- start start 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf7e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf780b6690 0x7fdf781691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf7e000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf780d3690 con 0x7fdf7815e5f0 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf7e000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf780d3970 con 0x7fdf780d30f0 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf7e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7806d530 con 0x7fdf780b6690 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf77400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdf7815e5f0 0x7fdf780d4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42104/0 (socket says 172.21.15.28:42104) 2024-09-10T10:46:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.831+0000 7fdf77400640 1 -- 172.21.15.28:0/1136219928 learned_addr learned my addr 172.21.15.28:0/1136219928 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:46:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf77e00640 1 --2- 172.21.15.28:0/1136219928 >> v2:172.21.15.138:3300/0 conn(0x7fdf780b6690 0x7fdf781691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639351449 0 0) 0x7fdf780d3690 con 0x7fdf7815e5f0 2024-09-10T10:46:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf5c003520 con 0x7fdf7815e5f0 2024-09-10T10:46:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431541360 0 0) 0x7fdf5c003520 con 0x7fdf7815e5f0 2024-09-10T10:46:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 >> v2:172.21.15.138:3300/0 conn(0x7fdf780b6690 msgr2=0x7fdf781691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 --2- 172.21.15.28:0/1136219928 >> v2:172.21.15.138:3300/0 conn(0x7fdf780b6690 0x7fdf781691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 >> v1:172.21.15.138:6789/0 conn(0x7fdf780d30f0 legacy=0x7fdf780b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7806cee0 con 0x7fdf7815e5f0 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fdf68001670 con 0x7fdf7815e5f0 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 >> v1:172.21.15.28:6789/0 conn(0x7fdf7815e5f0 legacy=0x7fdf780d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5c0039e0 0x7fdf5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c004800 0x7fdf5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 0x7fdf5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7806d530 con 0x7fdf5c0051c0 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c00c090 con 0x7fdf5c0039e0 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c00c290 con 0x7fdf5c004800 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.832+0000 7fdf77e00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 0x7fdf5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77400640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5c0039e0 0x7fdf5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf6fe00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c004800 0x7fdf5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77e00640 1 -- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c004800 msgr2=0x7fdf5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77e00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c004800 0x7fdf5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77e00640 1 -- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5c0039e0 msgr2=0x7fdf5c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77e00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf5c0039e0 0x7fdf5c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.833+0000 7fdf77e00640 1 -- 172.21.15.28:0/1136219928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c003520 con 0x7fdf5c0051c0 2024-09-10T10:46:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.834+0000 7fdf77e00640 1 -- 172.21.15.28:0/1136219928 reap_dead start 2024-09-10T10:46:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.834+0000 7fdf77e00640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 0x7fdf5c00b9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009f90 tx=0x7fdf5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffed29ce57103eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.834+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70035020 con 0x7fdf5c0051c0 2024-09-10T10:46:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.834+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70003b90 con 0x7fdf5c0051c0 2024-09-10T10:46:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.834+0000 7fdf76a00640 1 -- 172.21.15.28:0/1136219928 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009c10 con 0x7fdf5c0051c0 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 -- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 msgr2=0x7fdf5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 0x7fdf5c00b9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009f90 tx=0x7fdf5c005830 comp rx=0 tx=0).stop 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 -- 172.21.15.28:0/1136219928 shutdown_connections 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 --2- 172.21.15.28:0/1136219928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf5c0051c0 0x7fdf5c00b9c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 -- 172.21.15.28:0/1136219928 >> 172.21.15.28:0/1136219928 conn(0x7fdf7808bfc0 msgr2=0x7fdf780d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 -- 172.21.15.28:0/1136219928 shutdown_connections 2024-09-10T10:46:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.835+0000 7fdf7e000640 1 -- 172.21.15.28:0/1136219928 wait complete. 2024-09-10T10:46:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.837+0000 7fdf7e000640 1 Processor -- start 2024-09-10T10:46:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.837+0000 7fdf7e000640 1 -- start start 2024-09-10T10:46:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c0051c0 0x7fdf78150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf780b9520 0x7fdf780b9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf780b5f80 con 0x7fdf780c92c0 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7815e9d0 con 0x7fdf780b9520 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf7e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf780b6100 con 0x7fdf5c0051c0 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf77400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c0051c0 0x7fdf78150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48128/0 (socket says 172.21.15.28:48128) 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf77e00640 1 -- 172.21.15.28:0/4204747800 learned_addr learned my addr 172.21.15.28:0/4204747800 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf6fe00640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf780b9520 0x7fdf780b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.838+0000 7fdf77e00640 1 -- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c0051c0 msgr2=0x7fdf78150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf77e00640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c0051c0 0x7fdf78150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf77e00640 1 -- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf780b9520 msgr2=0x7fdf780b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf77e00640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf780b9520 0x7fdf780b9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:46:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf77e00640 1 -- 172.21.15.28:0/4204747800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf780c9c80 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf77e00640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000e230 tx=0x7fdf70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70035070 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70041070 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009c10 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf780c9f10 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.839+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf780ca4d0 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.840+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf780b1140 con 0x7fdf780c92c0 2024-09-10T10:46:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.841+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf70005d10 con 0x7fdf780c92c0 2024-09-10T10:46:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.843+0000 7fdf6f400640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 0x7fdf5406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:46:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.843+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7fdf700abbf0 con 0x7fdf780c92c0 2024-09-10T10:46:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.843+0000 7fdf77400640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 0x7fdf5406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:46:59.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.844+0000 7fdf77400640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 0x7fdf5406e320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001cd0 tx=0x7fdf68000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:46:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:46:59.847+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf780b1140 con 0x7fdf780c92c0 2024-09-10T10:47:00.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.051+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf780d4dd0 con 0x7fdf780c92c0 2024-09-10T10:47:00.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.051+0000 7fdf6f400640 1 -- 172.21.15.28:0/4204747800 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7fdf70079c90 con 0x7fdf780c92c0 2024-09-10T10:47:00.052 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.053+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 msgr2=0x7fdf5406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.053+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 0x7fdf5406e320 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdf68001cd0 tx=0x7fdf68000ed0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.053+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 msgr2=0x7fdf780c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.053+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000e230 tx=0x7fdf70009750 comp rx=0 tx=0).stop 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 shutdown_connections 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5406be60 0x7fdf5406e320 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf780c92c0 0x7fdf780c9740 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf780b9520 0x7fdf780b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 --2- 172.21.15.28:0/4204747800 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf5c0051c0 0x7fdf78150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 >> 172.21.15.28:0/4204747800 conn(0x7fdf7808bfc0 msgr2=0x7fdf7806e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 shutdown_connections 2024-09-10T10:47:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.054+0000 7fdf7e000640 1 -- 172.21.15.28:0/4204747800 wait complete. 2024-09-10T10:47:00.071 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:47:00.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:47:00.071 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:47:00.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:47:00.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:47:00.137 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:47:00.173 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:47:00.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.368+0000 7f2194600640 1 Processor -- start 2024-09-10T10:47:00.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.369+0000 7f2194600640 1 -- start start 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2194600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2194600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c06f480 con 0x7f218c147230 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2194600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f218c06e900 con 0x7f218c0725f0 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2194600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f218c0b34d0 con 0x7f218c147e90 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2191c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f218c147230 0x7f218c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42106/0 (socket says 172.21.15.28:42106) 2024-09-10T10:47:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.370+0000 7f2191c00640 1 -- 172.21.15.28:0/2249181416 learned_addr learned my addr 172.21.15.28:0/2249181416 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f2191200640 1 --2- 172.21.15.28:0/2249181416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1317416702 0 0) 0x7f218c06f480 con 0x7f218c147230 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2174003520 con 0x7f218c147230 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494360002 0 0) 0x7f218c06e900 con 0x7f218c0725f0 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f218c06f480 con 0x7f218c0725f0 2024-09-10T10:47:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f2191200640 1 -- 172.21.15.28:0/2249181416 >> v1:172.21.15.138:6789/0 conn(0x7f218c0725f0 legacy=0x7f218c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f2191200640 1 -- 172.21.15.28:0/2249181416 >> v1:172.21.15.28:6789/0 conn(0x7f218c147230 legacy=0x7f218c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.371+0000 7f2191200640 1 -- 172.21.15.28:0/2249181416 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c06ebe0 con 0x7f218c147e90 2024-09-10T10:47:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.372+0000 7f2191200640 1 --2- 172.21.15.28:0/2249181416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c0720b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21780097d0 tx=0x7f217802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=517098e2a8c9731b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.372+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178037070 con 0x7f218c147e90 2024-09-10T10:47:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.372+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2178032440 con 0x7f218c147e90 2024-09-10T10:47:00.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f218be00640 1 -- 172.21.15.28:0/2249181416 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21780327f0 con 0x7f218c147e90 2024-09-10T10:47:00.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 -- 172.21.15.28:0/2249181416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 msgr2=0x7f218c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 --2- 172.21.15.28:0/2249181416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c0720b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21780097d0 tx=0x7f217802a210 comp rx=0 tx=0).stop 2024-09-10T10:47:00.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 -- 172.21.15.28:0/2249181416 shutdown_connections 2024-09-10T10:47:00.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 --2- 172.21.15.28:0/2249181416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c0720b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 -- 172.21.15.28:0/2249181416 >> 172.21.15.28:0/2249181416 conn(0x7f218c067d70 msgr2=0x7f218c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 -- 172.21.15.28:0/2249181416 shutdown_connections 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.373+0000 7f2194600640 1 -- 172.21.15.28:0/2249181416 wait complete. 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.375+0000 7f2194600640 1 Processor -- start 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.375+0000 7f2194600640 1 -- start start 2024-09-10T10:47:00.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f218c0725f0 0x7f218c142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f218c072a40 con 0x7f218c147230 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f218c06f1a0 con 0x7f218c0725f0 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2194600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f218c147610 con 0x7f218c147e90 2024-09-10T10:47:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48144/0 (socket says 172.21.15.28:48144) 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 -- 172.21.15.28:0/1182019 learned_addr learned my addr 172.21.15.28:0/1182019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191c00640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f218c0725f0 0x7f218c142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2192600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 -- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 msgr2=0x7f218c145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 -- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f218c0725f0 msgr2=0x7f218c142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f218c0725f0 0x7f218c142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.376+0000 7f2191200640 1 -- 172.21.15.28:0/1182019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c1f7c40 con 0x7f218c147230 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2191200640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f21780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178041020 con 0x7f218c147230 2024-09-10T10:47:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217803d620 con 0x7f218c147230 2024-09-10T10:47:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f218c142680 con 0x7f218c147230 2024-09-10T10:47:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178037040 con 0x7f218c147230 2024-09-10T10:47:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.377+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f218c1e6070 con 0x7f218c147230 2024-09-10T10:47:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.378+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f218c0b4900 con 0x7f218c147230 2024-09-10T10:47:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.379+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2178044020 con 0x7f218c147230 2024-09-10T10:47:00.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.380+0000 7f2189600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.381+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6101+0+0 (secure 0 0 0) 0x7f21780ab880 con 0x7f218c147230 2024-09-10T10:47:00.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.381+0000 7f2191c00640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.382+0000 7f2191c00640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 0x7f216c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2180000f60 tx=0x7f2180009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.385+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f218c0b4900 con 0x7f218c147230 2024-09-10T10:47:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.607+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f218c1456e0 con 0x7f218c147230 2024-09-10T10:47:00.609 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:47:00.607+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.609+0000 7f2189600640 1 -- 172.21.15.28:0/1182019 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) ==== 161+0+0 (secure 0 0 0) 0x7f218c1456e0 con 0x7f218c147230 2024-09-10T10:47:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:47:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 msgr2=0x7f216c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 0x7f216c06e2f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2180000f60 tx=0x7f2180009040 comp rx=0 tx=0).stop 2024-09-10T10:47:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 msgr2=0x7f218c1e3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f21780046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 shutdown_connections 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f216c06be30 0x7f216c06e2f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f218c147e90 0x7f218c145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f218c147230 0x7f218c1e3f90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 --2- 172.21.15.28:0/1182019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f218c0725f0 0x7f218c142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 >> 172.21.15.28:0/1182019 conn(0x7f218c067d70 msgr2=0x7f218c06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 shutdown_connections 2024-09-10T10:47:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.612+0000 7f2194600640 1 -- 172.21.15.28:0/1182019 wait complete. 2024-09-10T10:47:00.623 INFO:tasks.ceph.mds.b.smithi138.stderr: -5323> 2024-09-10T10:46:50.841+0000 7fe03c400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:47:00.629 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_journal_corrupt_dentry_first 1.0 2024-09-10T10:47:00.675 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:47:00.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.789+0000 7ff4c8800640 1 Processor -- start 2024-09-10T10:47:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.789+0000 7ff4c8800640 1 -- start start 2024-09-10T10:47:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c8800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c00b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c8800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff4c00cfea0 0x7ff4c00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c8800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c00b30e0 con 0x7ff4c00b4af0 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c8800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c006edc0 con 0x7ff4c00cfea0 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c8800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c00b33c0 con 0x7ff4c0156740 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c5400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff4c00b4af0 0x7ff4c00cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42118/0 (socket says 172.21.15.28:42118) 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c5400640 1 -- 172.21.15.28:0/3425594718 learned_addr learned my addr 172.21.15.28:0/3425594718 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.791+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3425594718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c00b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c6800640 1 --2- 172.21.15.28:0/3425594718 >> v2:172.21.15.138:3300/0 conn(0x7ff4c00cfea0 0x7ff4c00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3425594718 >> v2:172.21.15.138:3300/0 conn(0x7ff4c00cfea0 msgr2=0x7ff4c00ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3425594718 >> v2:172.21.15.138:3300/0 conn(0x7ff4c00cfea0 0x7ff4c00ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3425594718 >> v1:172.21.15.28:6789/0 conn(0x7ff4c00b4af0 legacy=0x7ff4c00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c4a00640 1 -- 172.21.15.28:0/3425594718 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1901330686 0 0) 0x7ff4c00b30e0 con 0x7ff4c00b4af0 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.792+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3425594718 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c00b36a0 con 0x7ff4c0156740 2024-09-10T10:47:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.793+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3425594718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c00b45b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40097a0 tx=0x7ff4b402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7bbc0b16ff0031 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.793+0000 7ff4c4a00640 1 -- 172.21.15.28:0/3425594718 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037070 con 0x7ff4c0156740 2024-09-10T10:47:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.793+0000 7ff4c4a00640 1 -- 172.21.15.28:0/3425594718 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4032440 con 0x7ff4c0156740 2024-09-10T10:47:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.793+0000 7ff4c4a00640 1 -- 172.21.15.28:0/3425594718 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4032810 con 0x7ff4c0156740 2024-09-10T10:47:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 -- 172.21.15.28:0/3425594718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 msgr2=0x7ff4c00b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3425594718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c00b45b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40097a0 tx=0x7ff4b402a180 comp rx=0 tx=0).stop 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 -- 172.21.15.28:0/3425594718 shutdown_connections 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3425594718 >> v2:172.21.15.138:3300/0 conn(0x7ff4c00cfea0 0x7ff4c00ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3425594718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c00b45b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 -- 172.21.15.28:0/3425594718 >> 172.21.15.28:0/3425594718 conn(0x7ff4c006bdf0 msgr2=0x7ff4c006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 -- 172.21.15.28:0/3425594718 shutdown_connections 2024-09-10T10:47:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.794+0000 7ff4c8800640 1 -- 172.21.15.28:0/3425594718 wait complete. 2024-09-10T10:47:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.796+0000 7ff4c8800640 1 Processor -- start 2024-09-10T10:47:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.796+0000 7ff4c8800640 1 -- start start 2024-09-10T10:47:00.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 0x7ff4c0159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c01570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00d0890 con 0x7ff4c00b4af0 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c00b3c60 con 0x7ff4c00cfea0 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c8800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c00d0a10 con 0x7ff4c0156740 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48158/0 (socket says 172.21.15.28:48158) 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.797+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3348252169 learned_addr learned my addr 172.21.15.28:0/3348252169 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5400640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 0x7ff4c0159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c6800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c01570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 msgr2=0x7ff4c01570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c01570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 msgr2=0x7ff4c0159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 0x7ff4c0159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 -- 172.21.15.28:0/3348252169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c00afd90 con 0x7ff4c00b4af0 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c6800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c01570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5400640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 0x7ff4c0159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c5e00640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098c0 tx=0x7ff4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4037050 con 0x7ff4c00b4af0 2024-09-10T10:47:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4038040 con 0x7ff4c00b4af0 2024-09-10T10:47:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4030b10 con 0x7ff4c00b4af0 2024-09-10T10:47:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.798+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c00b0070 con 0x7ff4c00b4af0 2024-09-10T10:47:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.799+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c00b0630 con 0x7ff4c00b4af0 2024-09-10T10:47:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.800+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c0068f50 con 0x7ff4c00b4af0 2024-09-10T10:47:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.800+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4b4030cb0 con 0x7ff4c00b4af0 2024-09-10T10:47:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.802+0000 7ff4ba000640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 0x7ff4a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.802+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7ff4b40abb50 con 0x7ff4c00b4af0 2024-09-10T10:47:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.802+0000 7ff4c5400640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 0x7ff4a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:00.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.803+0000 7ff4c5400640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 0x7ff4a406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00b1c30 tx=0x7ff4b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:00.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.806+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4b40313a0 con 0x7ff4c00b4af0 2024-09-10T10:47:00.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.995+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}] v 0) -- 0x7ff4c0156c40 con 0x7ff4c00b4af0 2024-09-10T10:47:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:00.999+0000 7ff4ba000640 1 -- 172.21.15.28:0/3348252169 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_journal_corrupt_dentry_first}]=0 v19)=0 v19) ==== 139+0+0 (secure 0 0 0) 0x7ff4b407a280 con 0x7ff4c00b4af0 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 msgr2=0x7ff4a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 0x7ff4a406e2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00b1c30 tx=0x7ff4b0009040 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 msgr2=0x7ff4c00af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff4b40098c0 tx=0x7ff4b40046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 shutdown_connections 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a406be30 0x7ff4a406e2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4c0156740 0x7ff4c01570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4c00cfea0 0x7ff4c0159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 --2- 172.21.15.28:0/3348252169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4c00b4af0 0x7ff4c00af690 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:01.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 >> 172.21.15.28:0/3348252169 conn(0x7ff4c006bdf0 msgr2=0x7ff4c00ce3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:01.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 shutdown_connections 2024-09-10T10:47:01.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:01.001+0000 7ff4c8800640 1 -- 172.21.15.28:0/3348252169 wait complete. 2024-09-10T10:47:06.019 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:47:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.188+0000 7f13ee400640 1 Processor -- start 2024-09-10T10:47:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.188+0000 7f13ee400640 1 -- start start 2024-09-10T10:47:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13ee400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f13e815e900 0x7f13e80b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13ee400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e8072050 0x7f13e80724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13ee400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e80b4620 con 0x7f13e80b4a20 2024-09-10T10:47:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13ee400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13e8072a00 con 0x7f13e815e900 2024-09-10T10:47:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13ee400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13e80ce100 con 0x7f13e8072050 2024-09-10T10:47:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13dfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f13e80b4a20 0x7f13e80ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42506/0 (socket says 172.21.15.28:42506) 2024-09-10T10:47:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.189+0000 7f13dfe00640 1 -- 172.21.15.28:0/1173836395 learned_addr learned my addr 172.21.15.28:0/1173836395 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7400640 1 --2- 172.21.15.28:0/1173836395 >> v2:172.21.15.138:3300/0 conn(0x7f13e815e900 0x7f13e80b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7e00640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e8072050 0x7f13e80724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2787009921 0 0) 0x7f13e80b4620 con 0x7f13e80b4a20 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13cc005730 con 0x7f13e80b4a20 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7400640 1 -- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e8072050 msgr2=0x7f13e80724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1794153485 0 0) 0x7f13cc005730 con 0x7f13e80b4a20 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7400640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e8072050 0x7f13e80724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7400640 1 -- 172.21.15.28:0/1173836395 >> v1:172.21.15.28:6789/0 conn(0x7f13e80b4a20 legacy=0x7f13e80ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.190+0000 7f13e7400640 1 -- 172.21.15.28:0/1173836395 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e80ce3a0 con 0x7f13e815e900 2024-09-10T10:47:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.191+0000 7f13e7400640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e815e900 0x7f13e80b4020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13d80097a0 tx=0x7f13d802a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be15d1f49cc1dffe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8037070 con 0x7f13e815e900 2024-09-10T10:47:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d802ab10 con 0x7f13e815e900 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13e6a00640 1 -- 172.21.15.28:0/1173836395 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0001670 con 0x7f13e815e900 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 -- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e815e900 msgr2=0x7f13e80b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e815e900 0x7f13e80b4020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13d80097a0 tx=0x7f13d802a080 comp rx=0 tx=0).stop 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 -- 172.21.15.28:0/1173836395 shutdown_connections 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e8072050 0x7f13e80724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 --2- 172.21.15.28:0/1173836395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e815e900 0x7f13e80b4020 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 -- 172.21.15.28:0/1173836395 >> 172.21.15.28:0/1173836395 conn(0x7f13e806bdf0 msgr2=0x7f13e806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 -- 172.21.15.28:0/1173836395 shutdown_connections 2024-09-10T10:47:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.192+0000 7f13ee400640 1 -- 172.21.15.28:0/1173836395 wait complete. 2024-09-10T10:47:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.194+0000 7f13ee400640 1 Processor -- start 2024-09-10T10:47:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.194+0000 7f13ee400640 1 -- start start 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e80b4a20 0x7f13e81520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e815e900 0x7f13e80cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13e80b4e50 con 0x7f13e8072050 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13e80ce960 con 0x7f13e80b4a20 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13ee400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f13e80cee80 con 0x7f13e815e900 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13e7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13e7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51002/0 (socket says 172.21.15.28:51002) 2024-09-10T10:47:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.195+0000 7f13e7400640 1 -- 172.21.15.28:0/2336306216 learned_addr learned my addr 172.21.15.28:0/2336306216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 -- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e815e900 msgr2=0x7f13e80cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e815e900 0x7f13e80cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13dfe00640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e80b4a20 0x7f13e81520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 -- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e80b4a20 msgr2=0x7f13e81520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e80b4a20 0x7f13e81520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 -- 172.21.15.28:0/2336306216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e80c9900 con 0x7f13e8072050 2024-09-10T10:47:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13e7400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8041020 con 0x7f13e8072050 2024-09-10T10:47:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d803d620 con 0x7f13e8072050 2024-09-10T10:47:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8037040 con 0x7f13e8072050 2024-09-10T10:47:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.196+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e81525f0 con 0x7f13e8072050 2024-09-10T10:47:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.197+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e80af0e0 con 0x7f13e8072050 2024-09-10T10:47:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.198+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e80724c0 con 0x7f13e8072050 2024-09-10T10:47:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.198+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13d8044020 con 0x7f13e8072050 2024-09-10T10:47:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.199+0000 7f13df400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 0x7f13c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.199+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f13d80ab650 con 0x7f13e8072050 2024-09-10T10:47:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.200+0000 7f13dfe00640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 0x7f13c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.200+0000 7f13dfe00640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 0x7f13c406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e81f7a60 tx=0x7f13d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.204+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f13d804b030 con 0x7f13e8072050 2024-09-10T10:47:06.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.425+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f13e80d05d0 con 0x7f13e8072050 2024-09-10T10:47:06.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.623+0000 7f13df400640 1 -- 172.21.15.28:0/2336306216 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v51) ==== 162+0+0 (secure 0 0 0) 0x7f13d8079d80 con 0x7f13e8072050 2024-09-10T10:47:06.625 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 msgr2=0x7f13c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 0x7f13c406e2f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f13e81f7a60 tx=0x7f13d0009040 comp rx=0 tx=0).stop 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 msgr2=0x7f13e80c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d80046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 shutdown_connections 2024-09-10T10:47:06.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f13c406be30 0x7f13c406e2f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f13e815e900 0x7f13e80cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13e80b4a20 0x7f13e81520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.625+0000 7f13ee400640 1 --2- 172.21.15.28:0/2336306216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13e8072050 0x7f13e80c93c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.626+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 >> 172.21.15.28:0/2336306216 conn(0x7f13e806bdf0 msgr2=0x7f13e815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.626+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 shutdown_connections 2024-09-10T10:47:06.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.626+0000 7f13ee400640 1 -- 172.21.15.28:0/2336306216 wait complete. 2024-09-10T10:47:06.645 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:06.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.800+0000 7f5366600640 1 Processor -- start 2024-09-10T10:47:06.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.800+0000 7f5366600640 1 -- start start 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f5366600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5360072700 0x7f536015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f5366600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5360146140 con 0x7f5360072de0 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f5366600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53600cd740 con 0x7f5360072320 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f5366600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f53600cda20 con 0x7f5360072700 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5360072de0 0x7f536015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42522/0 (socket says 172.21.15.28:42522) 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535ea00640 1 -- 172.21.15.28:0/3642603214 learned_addr learned my addr 172.21.15.28:0/3642603214 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:06.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535fe00640 1 --2- 172.21.15.28:0/3642603214 >> v2:172.21.15.28:3300/0 conn(0x7f5360072700 0x7f536015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535e000640 1 -- 172.21.15.28:0/3642603214 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085669089 0 0) 0x7f5360146140 con 0x7f5360072de0 2024-09-10T10:47:06.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535e000640 1 -- 172.21.15.28:0/3642603214 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5344003520 con 0x7f5360072de0 2024-09-10T10:47:06.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535fe00640 1 -- 172.21.15.28:0/3642603214 >> v1:172.21.15.138:6789/0 conn(0x7f5360072320 legacy=0x7f53600cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535fe00640 1 -- 172.21.15.28:0/3642603214 >> v1:172.21.15.28:6789/0 conn(0x7f5360072de0 legacy=0x7f536015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.802+0000 7f535fe00640 1 -- 172.21.15.28:0/3642603214 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53600cdd00 con 0x7f5360072700 2024-09-10T10:47:06.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.803+0000 7f535fe00640 1 --2- 172.21.15.28:0/3642603214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072700 0x7f536015d8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5358009d90 tx=0x7f535802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37409f440a2e91a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f535e000640 1 -- 172.21.15.28:0/3642603214 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358007740 con 0x7f5360072700 2024-09-10T10:47:06.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f535e000640 1 -- 172.21.15.28:0/3642603214 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53580078e0 con 0x7f5360072700 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f535e000640 1 -- 172.21.15.28:0/3642603214 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358007cb0 con 0x7f5360072700 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 -- 172.21.15.28:0/3642603214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072700 msgr2=0x7f536015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 --2- 172.21.15.28:0/3642603214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072700 0x7f536015d8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5358009d90 tx=0x7f535802a660 comp rx=0 tx=0).stop 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 -- 172.21.15.28:0/3642603214 shutdown_connections 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 --2- 172.21.15.28:0/3642603214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072700 0x7f536015d8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 -- 172.21.15.28:0/3642603214 >> 172.21.15.28:0/3642603214 conn(0x7f5360067d70 msgr2=0x7f536006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 -- 172.21.15.28:0/3642603214 shutdown_connections 2024-09-10T10:47:06.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.804+0000 7f5366600640 1 -- 172.21.15.28:0/3642603214 wait complete. 2024-09-10T10:47:06.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.806+0000 7f5366600640 1 Processor -- start 2024-09-10T10:47:06.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.806+0000 7f5366600640 1 -- start start 2024-09-10T10:47:06.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5360072320 0x7f536013c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 0x7f53600b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f536015dfb0 con 0x7f5360072de0 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f536015de30 con 0x7f5360072700 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f5366600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f536015e130 con 0x7f5360072320 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f535fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f535fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51026/0 (socket says 172.21.15.28:51026) 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f535fe00640 1 -- 172.21.15.28:0/259227660 learned_addr learned my addr 172.21.15.28:0/259227660 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f535ea00640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 0x7f53600b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.807+0000 7f535f400640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5360072320 0x7f536013c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 -- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5360072320 msgr2=0x7f536013c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5360072320 0x7f536013c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 -- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 msgr2=0x7f53600b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:06.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 0x7f53600b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 -- 172.21.15.28:0/259227660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53600ab300 con 0x7f5360072de0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535ea00640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 0x7f53600b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f535fe00640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5358009eb0 tx=0x7f5358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358040070 con 0x7f5360072de0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5358031bc0 con 0x7f5360072de0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358009450 con 0x7f5360072de0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.808+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53600ab5e0 con 0x7f5360072de0 2024-09-10T10:47:06.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.809+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53600b0490 con 0x7f5360072de0 2024-09-10T10:47:06.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.810+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53600cce60 con 0x7f5360072de0 2024-09-10T10:47:06.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.811+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5358039040 con 0x7f5360072de0 2024-09-10T10:47:06.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.811+0000 7f5357400640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 0x7f534006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:06.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.811+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f53580ac960 con 0x7f5360072de0 2024-09-10T10:47:06.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.812+0000 7f535f400640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 0x7f534006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:06.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.813+0000 7f535f400640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 0x7f534006e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5350000f60 tx=0x7f5350009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:06.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:06.816+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f535807b120 con 0x7f5360072de0 2024-09-10T10:47:07.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.041+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53601459b0 con 0x7f5360072de0 2024-09-10T10:47:07.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.041+0000 7f5357400640 1 -- 172.21.15.28:0/259227660 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5224 (secure 0 0 0) 0x7f535807aa20 con 0x7f5360072de0 2024-09-10T10:47:07.043 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:07.044 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":51,"btime":"2024-09-10T10:47:06:623585+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:06.623565+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T10:47:07.044 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 51 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.043+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 msgr2=0x7f534006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.043+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 0x7f534006e2f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5350000f60 tx=0x7f5350009040 comp rx=0 tx=0).stop 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.043+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 msgr2=0x7f53601f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.043+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5358009eb0 tx=0x7f5358004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 shutdown_connections 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f534006be30 0x7f534006e2f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5360072de0 0x7f53601f5000 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:07.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5360072700 0x7f53600b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 --2- 172.21.15.28:0/259227660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5360072320 0x7f536013c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 >> 172.21.15.28:0/259227660 conn(0x7f5360067d70 msgr2=0x7f53600cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 shutdown_connections 2024-09-10T10:47:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:07.044+0000 7f5366600640 1 -- 172.21.15.28:0/259227660 wait complete. 2024-09-10T10:47:07.465 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:06.623565+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5752': {'gid': 5752, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.138:6837/2819175864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 2819175864}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 2819175864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:47:07.465 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5752:up:replay 2024-09-10T10:47:08.466 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:08.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.631+0000 7f678c600640 1 Processor -- start 2024-09-10T10:47:08.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.631+0000 7f678c600640 1 -- start start 2024-09-10T10:47:08.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.632+0000 7f678c600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 0x7f67840cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.632+0000 7f678c600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f678415e1f0 0x7f678406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.632+0000 7f678c600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6784146140 con 0x7f6784072de0 2024-09-10T10:47:08.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.632+0000 7f678c600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f67840cd740 con 0x7f678415e1f0 2024-09-10T10:47:08.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.632+0000 7f678c600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cda20 con 0x7f6784072320 2024-09-10T10:47:08.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f678415e1f0 0x7f678406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f678415e1f0 0x7f678406e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51042/0 (socket says 172.21.15.28:51042) 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 -- 172.21.15.28:0/2524249383 learned_addr learned my addr 172.21.15.28:0/2524249383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f6789c00640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 0x7f67840cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 -- 172.21.15.28:0/2524249383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 msgr2=0x7f67840cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 0x7f67840cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 -- 172.21.15.28:0/2524249383 >> v1:172.21.15.138:6789/0 conn(0x7f6784072de0 legacy=0x7f678406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f678a600640 1 -- 172.21.15.28:0/2524249383 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67840cdd00 con 0x7f678415e1f0 2024-09-10T10:47:08.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.633+0000 7f6789c00640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 0x7f67840cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:08.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.634+0000 7f678a600640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f678406e7c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f677c009e50 tx=0x7f677c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7fba16ef274629 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:08.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f6783e00640 1 -- 172.21.15.28:0/2524249383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c036070 con 0x7f678415e1f0 2024-09-10T10:47:08.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f6783e00640 1 -- 172.21.15.28:0/2524249383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c031600 con 0x7f678415e1f0 2024-09-10T10:47:08.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f6783e00640 1 -- 172.21.15.28:0/2524249383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0319d0 con 0x7f678415e1f0 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 -- 172.21.15.28:0/2524249383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 msgr2=0x7f678406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f678406e7c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f677c009e50 tx=0x7f677c02a680 comp rx=0 tx=0).stop 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 -- 172.21.15.28:0/2524249383 shutdown_connections 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f678406e7c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 --2- 172.21.15.28:0/2524249383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072320 0x7f67840cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 -- 172.21.15.28:0/2524249383 >> 172.21.15.28:0/2524249383 conn(0x7f6784067d70 msgr2=0x7f678406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 -- 172.21.15.28:0/2524249383 shutdown_connections 2024-09-10T10:47:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.635+0000 7f678c600640 1 -- 172.21.15.28:0/2524249383 wait complete. 2024-09-10T10:47:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.638+0000 7f678c600640 1 Processor -- start 2024-09-10T10:47:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.638+0000 7f678c600640 1 -- start start 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 0x7f67841e8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 0x7f67840c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cd520 con 0x7f678415e1f0 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cd3a0 con 0x7f6784072320 2024-09-10T10:47:08.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.639+0000 7f678c600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f678406ed00 con 0x7f6784072de0 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51048/0 (socket says 172.21.15.28:51048) 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 -- 172.21.15.28:0/1792988691 learned_addr learned my addr 172.21.15.28:0/1792988691 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f6789c00640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 0x7f67841e8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f6789200640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 0x7f67840c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 -- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 msgr2=0x7f67840c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 0x7f67840c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 -- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 msgr2=0x7f67841e8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 0x7f67841e8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 -- 172.21.15.28:0/1792988691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67840ca4b0 con 0x7f678415e1f0 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f6789c00640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 0x7f67841e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:08.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f6789200640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 0x7f67840c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:08.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.640+0000 7f678a600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f677c034040 tx=0x7f677c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.643+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c036050 con 0x7f678415e1f0 2024-09-10T10:47:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.643+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c042070 con 0x7f678415e1f0 2024-09-10T10:47:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.643+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67840ca790 con 0x7f678415e1f0 2024-09-10T10:47:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.643+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c009510 con 0x7f678415e1f0 2024-09-10T10:47:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.643+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67841327f0 con 0x7f678415e1f0 2024-09-10T10:47:08.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.644+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6784072700 con 0x7f678415e1f0 2024-09-10T10:47:08.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.645+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f677c007bd0 con 0x7f678415e1f0 2024-09-10T10:47:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.646+0000 7f6781600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 0x7f676406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.646+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f677c0abff0 con 0x7f678415e1f0 2024-09-10T10:47:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.646+0000 7f6789c00640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 0x7f676406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:08.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.647+0000 7f6789c00640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 0x7f676406e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6778006fd0 tx=0x7f6778008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.650+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f677c045360 con 0x7f678415e1f0 2024-09-10T10:47:08.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.872+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67840cacb0 con 0x7f678415e1f0 2024-09-10T10:47:08.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.872+0000 7f6781600640 1 -- 172.21.15.28:0/1792988691 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5224 (secure 0 0 0) 0x7f677c07a720 con 0x7f678415e1f0 2024-09-10T10:47:08.875 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:08.875 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":53,"btime":"2024-09-10T10:47:08:637313+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:07.639768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T10:47:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 53 2024-09-10T10:47:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 msgr2=0x7f676406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 0x7f676406e2f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6778006fd0 tx=0x7f6778008040 comp rx=0 tx=0).stop 2024-09-10T10:47:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 msgr2=0x7f67841e5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f677c034040 tx=0x7f677c004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 shutdown_connections 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676406be30 0x7f676406e2f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678415e1f0 0x7f67841e5320 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6784072de0 0x7f67840c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 --2- 172.21.15.28:0/1792988691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6784072320 0x7f67841e8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 >> 172.21.15.28:0/1792988691 conn(0x7f6784067d70 msgr2=0x7f678406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 shutdown_connections 2024-09-10T10:47:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:08.875+0000 7f678c600640 1 -- 172.21.15.28:0/1792988691 wait complete. 2024-09-10T10:47:09.147 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:47:09.151 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:47:09.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:07.639768+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5752': {'gid': 5752, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.138:6837/2819175864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 2819175864}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 2819175864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:47:09.299 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5752:up:rejoin 2024-09-10T10:47:10.299 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:10.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.464+0000 7f8736c00640 1 Processor -- start 2024-09-10T10:47:10.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.464+0000 7f8736c00640 1 -- start start 2024-09-10T10:47:10.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8736c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87300720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8736c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f87300725f0 0x7f87300aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8736c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873006f480 con 0x7f8730147230 2024-09-10T10:47:10.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8736c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f87300b34d0 con 0x7f87300725f0 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8736c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f87300ae9e0 con 0x7f8730147e90 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f872fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8730147230 0x7f873006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42536/0 (socket says 172.21.15.28:42536) 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f872fe00640 1 -- 172.21.15.28:0/992337265 learned_addr learned my addr 172.21.15.28:0/992337265 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8727e00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87300720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f8734c00640 1 --2- 172.21.15.28:0/992337265 >> v2:172.21.15.138:3300/0 conn(0x7f87300725f0 0x7f87300aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024138097 0 0) 0x7f873006f480 con 0x7f8730147230 2024-09-10T10:47:10.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.466+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8714003520 con 0x7f8730147230 2024-09-10T10:47:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2688477880 0 0) 0x7f8714003520 con 0x7f8730147230 2024-09-10T10:47:10.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 msgr2=0x7f87300720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87300720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 >> v2:172.21.15.138:3300/0 conn(0x7f87300725f0 msgr2=0x7f87300aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 --2- 172.21.15.28:0/992337265 >> v2:172.21.15.138:3300/0 conn(0x7f87300725f0 0x7f87300aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873006e900 con 0x7f8730147230 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f8720001670 con 0x7f8730147230 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 >> v1:172.21.15.28:6789/0 conn(0x7f8730147230 legacy=0x7f873006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87140039b0 0x7f8714003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f871400b9c0 con 0x7f8714004800 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f871400bb40 con 0x7f87140039b0 2024-09-10T10:47:10.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f871400bcc0 con 0x7f8730147e90 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f8727e00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f872fe00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87140039b0 0x7f8714003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.467+0000 7f8734c00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.468+0000 7f8727e00640 1 -- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 msgr2=0x7f87140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.468+0000 7f8727e00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8730147e90 0x7f87140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.468+0000 7f8727e00640 1 -- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87140039b0 msgr2=0x7f8714003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.468+0000 7f8727e00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87140039b0 0x7f8714003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.468+0000 7f8727e00640 1 -- 172.21.15.28:0/992337265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8714003520 con 0x7f8714004800 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8727e00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8714004800 0x7f8714004c80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8718009ac0 tx=0x7f8714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651b91e8d5a3d6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718036070 con 0x7f8714004800 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718031400 con 0x7f8714004800 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f872f400640 1 -- 172.21.15.28:0/992337265 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718009690 con 0x7f8714004800 2024-09-10T10:47:10.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8736c00640 1 -- 172.21.15.28:0/992337265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8714004800 msgr2=0x7f8714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8736c00640 1 --2- 172.21.15.28:0/992337265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8714004800 0x7f8714004c80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8718009ac0 tx=0x7f8714005830 comp rx=0 tx=0).stop 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8734c00640 1 -- 172.21.15.28:0/992337265 reap_dead start 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8736c00640 1 -- 172.21.15.28:0/992337265 shutdown_connections 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.469+0000 7f8736c00640 1 -- 172.21.15.28:0/992337265 >> 172.21.15.28:0/992337265 conn(0x7f8730067d70 msgr2=0x7f873006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.470+0000 7f8736c00640 1 -- 172.21.15.28:0/992337265 shutdown_connections 2024-09-10T10:47:10.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.470+0000 7f8736c00640 1 -- 172.21.15.28:0/992337265 wait complete. 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.471+0000 7f8736c00640 1 Processor -- start 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.471+0000 7f8736c00640 1 -- start start 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 0x7f8730153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 0x7f8730151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f873006eec0 con 0x7f8730152130 2024-09-10T10:47:10.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8730072a40 con 0x7f87301f67f0 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.472+0000 7f8736c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f873006f1a0 con 0x7f8714004800 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51062/0 (socket says 172.21.15.28:51062) 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 -- 172.21.15.28:0/623650403 learned_addr learned my addr 172.21.15.28:0/623650403 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f872fe00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 0x7f8730153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 -- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 msgr2=0x7f8730153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8727e00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 0x7f8730151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 0x7f8730153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 -- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 msgr2=0x7f8730151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 0x7f8730151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 -- 172.21.15.28:0/623650403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87301525d0 con 0x7f8730152130 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f872fe00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 0x7f8730153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:10.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8727e00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 0x7f8730151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f8734c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f87300d03c0 tx=0x7f872800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.473+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728015430 con 0x7f8730152130 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.474+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f872801d070 con 0x7f8730152130 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.474+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728015810 con 0x7f8730152130 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.474+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87301528b0 con 0x7f8730152130 2024-09-10T10:47:10.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.474+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8730153150 con 0x7f8730152130 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.475+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8730152a80 con 0x7f8730152130 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.476+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f87280159b0 con 0x7f8730152130 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.476+0000 7f872cc00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 0x7f870c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.477+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f872808bb30 con 0x7f8730152130 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.477+0000 7f872fe00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 0x7f870c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:10.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.478+0000 7f872fe00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 0x7f870c06e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8730072190 tx=0x7f8720009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:10.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.481+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8728010040 con 0x7f8730152130 2024-09-10T10:47:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.710+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8730153350 con 0x7f8730152130 2024-09-10T10:47:10.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.710+0000 7f872cc00640 1 -- 172.21.15.28:0/623650403 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5231 (secure 0 0 0) 0x7f8730153350 con 0x7f8730152130 2024-09-10T10:47:10.712 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:10.713 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"btime":"2024-09-10T10:47:09:642309+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:09.642303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:10.713 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 54 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 msgr2=0x7f870c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 0x7f870c06e2f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8730072190 tx=0x7f8720009040 comp rx=0 tx=0).stop 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 msgr2=0x7f8730153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f87300d03c0 tx=0x7f872800f580 comp rx=0 tx=0).stop 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 shutdown_connections 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f870c06be30 0x7f870c06e2f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8730152130 0x7f8730153c40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87301f67f0 0x7f8730151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 --2- 172.21.15.28:0/623650403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8714004800 0x7f8730153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:10.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 >> 172.21.15.28:0/623650403 conn(0x7f8730067d70 msgr2=0x7f87300af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:10.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 shutdown_connections 2024-09-10T10:47:10.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:10.713+0000 7f8736c00640 1 -- 172.21.15.28:0/623650403 wait complete. 2024-09-10T10:47:11.133 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:09.642303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5752}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5752': {'gid': 5752, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.138:6837/2819175864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 2819175864}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 2819175864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5752, 'qdb_cluster': [5752]} 2024-09-10T10:47:11.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:47:11.134 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:47:11.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.291+0000 7ff1fbc00640 1 Processor -- start 2024-09-10T10:47:11.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.291+0000 7ff1fbc00640 1 -- start start 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.292+0000 7ff1fbc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff1f4072700 0x7ff1f415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1fbc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f4146140 con 0x7ff1f4072320 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1fbc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f40cd740 con 0x7ff1f4072de0 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1fbc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f40cda20 con 0x7ff1f4072700 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f9200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff1f4072320 0x7ff1f40cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42550/0 (socket says 172.21.15.28:42550) 2024-09-10T10:47:11.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f9200640 1 -- 172.21.15.28:0/210923279 learned_addr learned my addr 172.21.15.28:0/210923279 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:11.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/210923279 >> v2:172.21.15.138:3300/0 conn(0x7ff1f4072700 0x7ff1f415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235577579 0 0) 0x7ff1f4146140 con 0x7ff1f4072320 2024-09-10T10:47:11.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1dc003520 con 0x7ff1f4072320 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.293+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3687250282 0 0) 0x7ff1dc003520 con 0x7ff1f4072320 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 >> v2:172.21.15.138:3300/0 conn(0x7ff1f4072700 msgr2=0x7ff1f415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 --2- 172.21.15.28:0/210923279 >> v2:172.21.15.138:3300/0 conn(0x7ff1f4072700 0x7ff1f415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 >> v1:172.21.15.138:6789/0 conn(0x7ff1f4072de0 legacy=0x7ff1f415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40cdd00 con 0x7ff1f4072320 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7ff1e8001670 con 0x7ff1f4072320 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/210923279 >> v2:172.21.15.138:3300/0 conn(0x7ff1f4072700 0x7ff1f415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 >> v1:172.21.15.28:6789/0 conn(0x7ff1f4072320 legacy=0x7ff1f40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1dc003a00 0x7ff1dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc005210 0x7ff1dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40cda20 con 0x7ff1dc004800 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc00c160 con 0x7ff1dc005210 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc00c360 con 0x7ff1dc003a00 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f9200640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1dc003a00 0x7ff1dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc005210 0x7ff1dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 -- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1dc003a00 msgr2=0x7ff1dc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1dc003a00 0x7ff1dc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 -- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc005210 msgr2=0x7ff1dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc005210 0x7ff1dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f3e00640 1 -- 172.21.15.28:0/210923279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc003520 con 0x7ff1dc004800 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.294+0000 7ff1f9c00640 1 -- 172.21.15.28:0/210923279 reap_dead start 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.295+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001720 tx=0x7ff1dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a1dd87df2800c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0001370 con 0x7ff1dc004800 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e00052d0 con 0x7ff1dc004800 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1f3400640 1 -- 172.21.15.28:0/210923279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e002ab90 con 0x7ff1dc004800 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 -- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 msgr2=0x7ff1dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001720 tx=0x7ff1dc005830 comp rx=0 tx=0).stop 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 -- 172.21.15.28:0/210923279 shutdown_connections 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/210923279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1dc004800 0x7ff1dc004c80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 -- 172.21.15.28:0/210923279 >> 172.21.15.28:0/210923279 conn(0x7ff1f4067d70 msgr2=0x7ff1f406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 -- 172.21.15.28:0/210923279 shutdown_connections 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.296+0000 7ff1fbc00640 1 -- 172.21.15.28:0/210923279 wait complete. 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.298+0000 7ff1fbc00640 1 Processor -- start 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.298+0000 7ff1fbc00640 1 -- start start 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 0x7ff1f4135ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1f41e1ed0 0x7ff1f41e2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40ccfe0 con 0x7ff1f41f7430 2024-09-10T10:47:11.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40cce60 con 0x7ff1dc004800 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1fbc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40cd160 con 0x7ff1f41e1ed0 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51084/0 (socket says 172.21.15.28:51084) 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1f9c00640 1 -- 172.21.15.28:0/4284036660 learned_addr learned my addr 172.21.15.28:0/4284036660 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.299+0000 7ff1f9200640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 0x7ff1f4135ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1f41e1ed0 0x7ff1f41e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 -- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1f41e1ed0 msgr2=0x7ff1f41e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1f41e1ed0 0x7ff1f41e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 -- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 msgr2=0x7ff1f4135ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 0x7ff1f4135ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 -- 172.21.15.28:0/4284036660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f40ad2e0 con 0x7ff1f41f7430 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9200640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 0x7ff1f4135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40705c0 tx=0x7ff1ec00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f0c00640 1 -- 172.21.15.28:0/4284036660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec015430 con 0x7ff1f41f7430 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f0c00640 1 -- 172.21.15.28:0/4284036660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec01d070 con 0x7ff1f41f7430 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1f0c00640 1 -- 172.21.15.28:0/4284036660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec015830 con 0x7ff1f41f7430 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f40ad5c0 con 0x7ff1f41f7430 2024-09-10T10:47:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.300+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f40adb80 con 0x7ff1f41f7430 2024-09-10T10:47:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.303+0000 7ff1f0c00640 1 -- 172.21.15.28:0/4284036660 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1ec0159d0 con 0x7ff1f41f7430 2024-09-10T10:47:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.303+0000 7ff1f0c00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 0x7ff1d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.304+0000 7ff1f0c00640 1 -- 172.21.15.28:0/4284036660 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7ff1ec08be70 con 0x7ff1f41f7430 2024-09-10T10:47:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.304+0000 7ff1f9200640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 0x7ff1d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.304+0000 7ff1fea94740 1 Processor -- start 2024-09-10T10:47:11.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.304+0000 7ff1fea94740 1 -- start start 2024-09-10T10:47:11.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1f9200640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 0x7ff1d406e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1f413ca90 tx=0x7ff1e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07adde0 0x5606a07ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07654b0 0x5606a0791ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5606a06c63b0 con 0x5606a07ae700 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606a06cb410 con 0x5606a07654b0 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.305+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5606a0673620 con 0x5606a07adde0 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51090/0 (socket says 172.21.15.28:51090) 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 -- 172.21.15.28:0/33103295 learned_addr learned my addr 172.21.15.28:0/33103295 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f9200640 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07adde0 0x5606a07ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07654b0 0x5606a0791ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 -- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07adde0 msgr2=0x5606a07ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07adde0 0x5606a07ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 -- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07654b0 msgr2=0x5606a0791ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07654b0 0x5606a0791ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.306+0000 7ff1f3e00640 1 -- 172.21.15.28:0/33103295 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a03de2d0 con 0x5606a07ae700 2024-09-10T10:47:11.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001720 tx=0x7ff1e002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e150a3e8e53313f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1e5600640 1 -- 172.21.15.28:0/33103295 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0037040 con 0x5606a07ae700 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1e5600640 1 -- 172.21.15.28:0/33103295 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0041070 con 0x5606a07ae700 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1e5600640 1 -- 172.21.15.28:0/33103295 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e002a950 con 0x5606a07ae700 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1fea94740 1 -- 172.21.15.28:0/33103295 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 msgr2=0x5606a0791960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.307+0000 7ff1fea94740 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001720 tx=0x7ff1e002c000 comp rx=0 tx=0).stop 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 -- 172.21.15.28:0/33103295 shutdown_connections 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07654b0 0x5606a0791ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791960 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 --2- 172.21.15.28:0/33103295 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07adde0 0x5606a07ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 -- 172.21.15.28:0/33103295 >> 172.21.15.28:0/33103295 conn(0x5606a0764740 msgr2=0x5606a07ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 -- 172.21.15.28:0/33103295 shutdown_connections 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 -- 172.21.15.28:0/33103295 wait complete. 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.308+0000 7ff1fea94740 1 -- start start 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.309+0000 7ff1fea94740 1 Processor -- start 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.309+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a078ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.309+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07a4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1fea94740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5606a0686780 con 0x5606a07ae700 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606a06c73d0 con 0x5606a07adde0 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5606a07680c0 con 0x5606a07954e0 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1fea94740 10 client.0 _ll_register_callbacks cb 0x5606a06c45d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:11.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51098/0 (socket says 172.21.15.28:51098) 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 -- 172.21.15.28:0/656523822 learned_addr learned my addr 172.21.15.28:0/656523822 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f9200640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a078ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 -- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 msgr2=0x5606a07a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 -- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 msgr2=0x5606a078ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a078ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f3e00640 1 -- 172.21.15.28:0/656523822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a04a86b0 con 0x5606a07ae700 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.310+0000 7ff1f9200640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a078ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a540 tx=0x7ff1e0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838272ad4465ff72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e004b070 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7ff1e002ad60 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1e0041070 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff1e0056050 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1fea94740 10 client.6105 fetch_fsmap learned FSMap version 54 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5606a0793c50 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0041070 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.311+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7ff1e0059020 con 0x5606a07ae700 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 10 client.6105 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 10 client.6105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 20 client.6105 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:11.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a07d08b0 0x5606a07d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 4 client.6105 mds_command: new command op to 5752 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:11.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 --> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606a06d8b40 con 0x5606a07d08b0 2024-09-10T10:47:11.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.312+0000 7ff1f9200640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a07d08b0 0x5606a07d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.313+0000 7ff1f9200640 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a07d08b0 0x5606a07d2ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5ef3898815ec7 server_cookie=8bc3c40b93eae8bb in_seq=0 out_seq=0 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.313+0000 7ff1da000640 10 client.6105 ms_handle_connect on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1da000640 1 -- 172.21.15.28:0/656523822 <== mds.0 v2:172.21.15.138:6835/2819175864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606a06d8b40 con 0x5606a07d08b0 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1da000640 10 client.6105 handle_command_reply: tid=0 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 client.6105 shutdown 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 msgr2=0x5606a0794fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff1e002a540 tx=0x7ff1e0001190 comp rx=0 tx=0).stop 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 shutdown_connections 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a07d08b0 0x5606a07d2ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0794fa0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 --2- 172.21.15.28:0/656523822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a078ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 >> 172.21.15.28:0/656523822 conn(0x5606a07654b0 msgr2=0x5606a07afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1da000640 0 client.6105 ms_handle_reset on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 shutdown_connections 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.314+0000 7ff1fea94740 1 -- 172.21.15.28:0/656523822 wait complete. 2024-09-10T10:47:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.315+0000 7ff1fea94740 20 client.6105 trim_cache size 0 max 16384 2024-09-10T10:47:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.333+0000 7ff1fea94740 1 Processor -- start 2024-09-10T10:47:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.333+0000 7ff1fea94740 1 -- start start 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07d88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40c78c0 con 0x5606a07ae700 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f41375f0 con 0x5606a07adde0 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40c7270 con 0x5606a07954e0 2024-09-10T10:47:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51110/0 (socket says 172.21.15.28:51110) 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1f9c00640 1 -- 172.21.15.28:0/1014793693 learned_addr learned my addr 172.21.15.28:0/1014793693 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.334+0000 7ff1f9200640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07d88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 -- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 msgr2=0x5606a07d88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07d88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 -- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 msgr2=0x5606a07afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 -- 172.21.15.28:0/1014793693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606a07c4270 con 0x5606a07ae700 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9200640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0563a0 tx=0x7ff1ec013420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8977c4288601c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1e4c00640 1 -- 172.21.15.28:0/1014793693 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec088070 con 0x5606a07ae700 2024-09-10T10:47:11.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1e4c00640 1 -- 172.21.15.28:0/1014793693 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec062b20 con 0x5606a07ae700 2024-09-10T10:47:11.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.335+0000 7ff1e4c00640 1 -- 172.21.15.28:0/1014793693 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec019030 con 0x5606a07ae700 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- 172.21.15.28:0/1014793693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 msgr2=0x5606a07b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0563a0 tx=0x7ff1ec013420 comp rx=0 tx=0).stop 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- 172.21.15.28:0/1014793693 shutdown_connections 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a07b0310 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 --2- 172.21.15.28:0/1014793693 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a07d88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- 172.21.15.28:0/1014793693 >> 172.21.15.28:0/1014793693 conn(0x5606a07654b0 msgr2=0x5606a07a2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- 172.21.15.28:0/1014793693 shutdown_connections 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- 172.21.15.28:0/1014793693 wait complete. 2024-09-10T10:47:11.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.336+0000 7ff1fea94740 1 -- start start 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.337+0000 7ff1fea94740 1 Processor -- start 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a0794df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5606a07941a0 con 0x5606a07ae700 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606a0686780 con 0x5606a07adde0 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5606a07a58b0 con 0x5606a07954e0 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1fea94740 10 client.0 _ll_register_callbacks cb 0x7ff1ec056260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51112/0 (socket says 172.21.15.28:51112) 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 -- 172.21.15.28:0/3875562493 learned_addr learned my addr 172.21.15.28:0/3875562493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9200640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a0794df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 -- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 msgr2=0x5606a0794df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a0794df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 -- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 msgr2=0x5606a07912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.338+0000 7ff1f9c00640 1 -- 172.21.15.28:0/3875562493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606a07a5b60 con 0x5606a07ae700 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1f9c00640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0563a0 tx=0x7ff1ec08a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c049d67761dd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec088070 con 0x5606a07ae700 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7ff1ec078c80 con 0x5606a07ae700 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1ec062b20 con 0x5606a07ae700 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec079020 con 0x5606a07ae700 2024-09-10T10:47:11.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 10 client.6111 fetch_fsmap learned FSMap version 54 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5606a07b0670 con 0x5606a07ae700 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec062b20 con 0x5606a07ae700 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7ff1ec059390 con 0x5606a07ae700 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 10 client.6111 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 10 client.6111 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 20 client.6111 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a0833120 0x5606a0833500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 4 client.6111 mds_command: new command op to 5752 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.339+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 --> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] -- command(tid 0: {"prefix": "status"}) -- 0x5606a07934b0 con 0x5606a0833120 2024-09-10T10:47:11.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.340+0000 7ff1f9200640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a0833120 0x5606a0833500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.340+0000 7ff1f9200640 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a0833120 0x5606a0833500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49571813c48041f6 server_cookie=70ffa65dfc09ed53 in_seq=0 out_seq=0 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.340+0000 7ff1da000640 10 client.6111 ms_handle_connect on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1da000640 1 -- 172.21.15.28:0/3875562493 <== mds.0 v2:172.21.15.138:6835/2819175864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5606a07934b0 con 0x5606a0833120 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1da000640 10 client.6111 handle_command_reply: tid=0 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 client.6111 shutdown 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 msgr2=0x5606a0791830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0563a0 tx=0x7ff1ec08a090 comp rx=0 tx=0).stop 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 shutdown_connections 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5606a0833120 0x5606a0833500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606a07ae700 0x5606a0791830 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606a07adde0 0x5606a07912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1da000640 0 client.6111 ms_handle_reset on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 --2- 172.21.15.28:0/3875562493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606a07954e0 0x5606a0794df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 >> 172.21.15.28:0/3875562493 conn(0x5606a07654b0 msgr2=0x5606a07afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 shutdown_connections 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 1 -- 172.21.15.28:0/3875562493 wait complete. 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.341+0000 7ff1fea94740 20 client.6111 trim_cache size 0 max 16384 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5752, 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:47:11.348 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 4.7160768370000001, 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 54, 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 71, 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 71, 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 10.664024621999999 2024-09-10T10:47:11.349 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 msgr2=0x7ff1d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 0x7ff1d406e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff1f413ca90 tx=0x7ff1e8009040 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 msgr2=0x7ff1f41f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff1f40705c0 tx=0x7ff1ec00f580 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 shutdown_connections 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1d406be30 0x7ff1d406e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f41f7430 0x7ff1f41f78b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1f41e1ed0 0x7ff1f41e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 --2- 172.21.15.28:0/4284036660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1dc004800 0x7ff1f4135ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 >> 172.21.15.28:0/4284036660 conn(0x7ff1f4067d70 msgr2=0x7ff1f40cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 shutdown_connections 2024-09-10T10:47:11.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.342+0000 7ff1fbc00640 1 -- 172.21.15.28:0/4284036660 wait complete. 2024-09-10T10:47:11.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.512+0000 7f3b47800640 1 Processor -- start 2024-09-10T10:47:11.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.512+0000 7f3b47800640 1 -- start start 2024-09-10T10:47:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b47800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b400b69d0 0x7f3b40162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b47800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b400af280 0x7f3b400afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b47800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4015eb60 con 0x7f3b400affb0 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b47800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400b0400 con 0x7f3b400af280 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b47800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400b6db0 con 0x7f3b400b69d0 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b400af280 0x7f3b400afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b45800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3b400affb0 0x7f3b40154d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42558/0 (socket says 172.21.15.28:42558) 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b400af280 0x7f3b400afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51118/0 (socket says 172.21.15.28:51118) 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 -- 172.21.15.28:0/1651876719 learned_addr learned my addr 172.21.15.28:0/1651876719 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b44e00640 1 --2- 172.21.15.28:0/1651876719 >> v2:172.21.15.138:3300/0 conn(0x7f3b400b69d0 0x7f3b40162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 -- 172.21.15.28:0/1651876719 >> v2:172.21.15.138:3300/0 conn(0x7f3b400b69d0 msgr2=0x7f3b40162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 --2- 172.21.15.28:0/1651876719 >> v2:172.21.15.138:3300/0 conn(0x7f3b400b69d0 0x7f3b40162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 -- 172.21.15.28:0/1651876719 >> v1:172.21.15.28:6789/0 conn(0x7f3b400affb0 legacy=0x7f3b40154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.514+0000 7f3b3fe00640 1 -- 172.21.15.28:0/1651876719 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4015ee40 con 0x7f3b400af280 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.515+0000 7f3b3fe00640 1 --2- 172.21.15.28:0/1651876719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400af280 0x7f3b400afa70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3b40070980 tx=0x7f3b2c02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9cd2d8ef3dc398 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b3f400640 1 -- 172.21.15.28:0/1651876719 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c037070 con 0x7f3b400af280 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b3f400640 1 -- 172.21.15.28:0/1651876719 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2c032440 con 0x7f3b400af280 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b3f400640 1 -- 172.21.15.28:0/1651876719 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c032810 con 0x7f3b400af280 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 -- 172.21.15.28:0/1651876719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400af280 msgr2=0x7f3b400afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 --2- 172.21.15.28:0/1651876719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400af280 0x7f3b400afa70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3b40070980 tx=0x7f3b2c02a150 comp rx=0 tx=0).stop 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 -- 172.21.15.28:0/1651876719 shutdown_connections 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 --2- 172.21.15.28:0/1651876719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400af280 0x7f3b400afa70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 --2- 172.21.15.28:0/1651876719 >> v2:172.21.15.138:3300/0 conn(0x7f3b400b69d0 0x7f3b40162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 -- 172.21.15.28:0/1651876719 >> 172.21.15.28:0/1651876719 conn(0x7f3b4006bc10 msgr2=0x7f3b4006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 -- 172.21.15.28:0/1651876719 shutdown_connections 2024-09-10T10:47:11.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.516+0000 7f3b47800640 1 -- 172.21.15.28:0/1651876719 wait complete. 2024-09-10T10:47:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.518+0000 7f3b47800640 1 Processor -- start 2024-09-10T10:47:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.518+0000 7f3b47800640 1 -- start start 2024-09-10T10:47:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 0x7f3b401486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 0x7f3b401f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400af710 con 0x7f3b400affb0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4015f400 con 0x7f3b400b69d0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b47800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b40155410 con 0x7f3b400af280 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.519+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51130/0 (socket says 172.21.15.28:51130) 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 -- 172.21.15.28:0/994868960 learned_addr learned my addr 172.21.15.28:0/994868960 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b44e00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 0x7f3b401486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b45800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 0x7f3b401f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 -- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 msgr2=0x7f3b401486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 0x7f3b401486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 -- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 msgr2=0x7f3b401f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 0x7f3b401f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 -- 172.21.15.28:0/994868960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4015a660 con 0x7f3b400affb0 2024-09-10T10:47:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b44e00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 0x7f3b401486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b45800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 0x7f3b401f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.520+0000 7f3b3fe00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02a190 tx=0x7f3b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.521+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c041020 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.521+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2c030760 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.521+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c037040 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.521+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4015a940 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.521+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4015af00 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.522+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b401557a0 con 0x7f3b400affb0 2024-09-10T10:47:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.523+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b2c044020 con 0x7f3b400affb0 2024-09-10T10:47:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.524+0000 7f3b3cc00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 0x7f3b2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.524+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f3b2c0ab930 con 0x7f3b400affb0 2024-09-10T10:47:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.524+0000 7f3b44e00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 0x7f3b2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.525+0000 7f3b44e00640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 0x7f3b2006e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.528+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3b2c04b030 con 0x7f3b400affb0 2024-09-10T10:47:11.619 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:47:11.618+0000 7f326b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:47:11.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.752+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b400ae820 con 0x7f3b400affb0 2024-09-10T10:47:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.755+0000 7f3b3cc00640 1 -- 172.21.15.28:0/994868960 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5231 (secure 0 0 0) 0x7f3b2c07a0f0 con 0x7f3b400affb0 2024-09-10T10:47:11.756 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:11.756 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"btime":"2024-09-10T10:47:09:642309+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:09.642303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:11.757 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 54 2024-09-10T10:47:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 msgr2=0x7f3b2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 0x7f3b2006e320 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006fd0 tx=0x7f3b34008040 comp rx=0 tx=0).stop 2024-09-10T10:47:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 msgr2=0x7f3b400cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02a190 tx=0x7f3b2c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 shutdown_connections 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3b2006be60 0x7f3b2006e320 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b400b69d0 0x7f3b401f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b400affb0 0x7f3b400cd650 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 --2- 172.21.15.28:0/994868960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b400af280 0x7f3b401486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 >> 172.21.15.28:0/994868960 conn(0x7f3b4006bc10 msgr2=0x7f3b4006bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 shutdown_connections 2024-09-10T10:47:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.757+0000 7f3b47800640 1 -- 172.21.15.28:0/994868960 wait complete. 2024-09-10T10:47:11.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 true 2024-09-10T10:47:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.932+0000 7fb5d9c00640 1 Processor -- start 2024-09-10T10:47:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.932+0000 7fb5d9c00640 1 -- start start 2024-09-10T10:47:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.933+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d40addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.933+0000 7fb5d9c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5d40d0a80 0x7fb5d415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.933+0000 7fb5d9c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d415c1e0 con 0x7fb5d415b720 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.933+0000 7fb5d9c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d415c4c0 con 0x7fb5d40d0a80 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.933+0000 7fb5d9c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d40b3340 con 0x7fb5d4155930 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5d40d0a80 0x7fb5d415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5d40d0a80 0x7fb5d415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51138/0 (socket says 172.21.15.28:51138) 2024-09-10T10:47:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 -- 172.21.15.28:0/3371711314 learned_addr learned my addr 172.21.15.28:0/3371711314 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d2a00640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d40addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 -- 172.21.15.28:0/3371711314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 msgr2=0x7fb5d40addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d40addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 -- 172.21.15.28:0/3371711314 >> v1:172.21.15.138:6789/0 conn(0x7fb5d415b720 legacy=0x7fb5d40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.934+0000 7fb5d3400640 1 -- 172.21.15.28:0/3371711314 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d40b3620 con 0x7fb5d40d0a80 2024-09-10T10:47:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.935+0000 7fb5d3400640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d40d0a80 0x7fb5d415bb80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aba8b5ccf3d4d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.935+0000 7fb5d2000640 1 -- 172.21.15.28:0/3371711314 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc036070 con 0x7fb5d40d0a80 2024-09-10T10:47:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.935+0000 7fb5d2000640 1 -- 172.21.15.28:0/3371711314 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5cc031600 con 0x7fb5d40d0a80 2024-09-10T10:47:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.935+0000 7fb5d2000640 1 -- 172.21.15.28:0/3371711314 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0319d0 con 0x7fb5d40d0a80 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 -- 172.21.15.28:0/3371711314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d40d0a80 msgr2=0x7fb5d415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d40d0a80 0x7fb5d415bb80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc02a710 comp rx=0 tx=0).stop 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 -- 172.21.15.28:0/3371711314 shutdown_connections 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d40d0a80 0x7fb5d415bb80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/3371711314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d40addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 -- 172.21.15.28:0/3371711314 >> 172.21.15.28:0/3371711314 conn(0x7fb5d406bdf0 msgr2=0x7fb5d406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 -- 172.21.15.28:0/3371711314 shutdown_connections 2024-09-10T10:47:11.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.936+0000 7fb5d9c00640 1 -- 172.21.15.28:0/3371711314 wait complete. 2024-09-10T10:47:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.938+0000 7fb5d9c00640 1 Processor -- start 2024-09-10T10:47:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.938+0000 7fb5d9c00640 1 -- start start 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5d40d0a80 0x7fb5d41f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d4152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40ae490 con 0x7fb5d415b720 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40ae310 con 0x7fb5d40d0a80 2024-09-10T10:47:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d9c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d40b3b40 con 0x7fb5d4155930 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51142/0 (socket says 172.21.15.28:51142) 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d3400640 1 -- 172.21.15.28:0/1388520927 learned_addr learned my addr 172.21.15.28:0/1388520927 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5d2a00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5d40d0a80 0x7fb5d41f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.939+0000 7fb5cbe00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d4152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 -- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 msgr2=0x7fb5d4152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d4152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 -- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5d40d0a80 msgr2=0x7fb5d41f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5d40d0a80 0x7fb5d41f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 -- 172.21.15.28:0/1388520927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d415e430 con 0x7fb5d415b720 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d3400640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc034040 tx=0x7fb5cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc036050 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5cc042070 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d41452a0 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc009510 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.940+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d4145860 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.942+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d415bb80 con 0x7fb5d415b720 2024-09-10T10:47:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.942+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5cc007bd0 con 0x7fb5d415b720 2024-09-10T10:47:11.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.943+0000 7fb5caa00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 0x7fb5b006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:11.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.943+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7fb5cc0abe50 con 0x7fb5d415b720 2024-09-10T10:47:11.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.944+0000 7fb5d2a00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 0x7fb5b006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:11.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.944+0000 7fb5d2a00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 0x7fb5b006e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:11.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:11.948+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5cc079f30 con 0x7fb5d415b720 2024-09-10T10:47:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.205+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"} v 0) -- 0x7fb5d40af270 con 0x7fb5d415b720 2024-09-10T10:47:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.661+0000 7fb5caa00640 1 -- 172.21.15.28:0/1388520927 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "true"}]=0 freezing mds.5752 v55) ==== 112+0+0 (secure 0 0 0) 0x7fb5cc07e230 con 0x7fb5d415b720 2024-09-10T10:47:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:freezing mds.5752 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.663+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 msgr2=0x7fb5b006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.663+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 0x7fb5b006e320 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4006fd0 tx=0x7fb5c4008040 comp rx=0 tx=0).stop 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.663+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 msgr2=0x7fb5d4153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.663+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc034040 tx=0x7fb5cc004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 shutdown_connections 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb5b006be60 0x7fb5b006e320 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5d415b720 0x7fb5d4153120 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5d4155930 0x7fb5d4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 --2- 172.21.15.28:0/1388520927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5d40d0a80 0x7fb5d41f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 >> 172.21.15.28:0/1388520927 conn(0x7fb5d406bdf0 msgr2=0x7fb5d4153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 shutdown_connections 2024-09-10T10:47:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.664+0000 7fb5d9c00640 1 -- 172.21.15.28:0/1388520927 wait complete. 2024-09-10T10:47:12.684 DEBUG:teuthology.orchestra.run.smithi028:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T10:47:12.688 DEBUG:teuthology.orchestra.run.smithi028:> timeout 30 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --connect-timeout=60 tell mds.4:0 flush journal 2024-09-10T10:47:12.692 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-10T10:47:12.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.852+0000 7feaef600640 1 Processor -- start 2024-09-10T10:47:12.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.852+0000 7feaef600640 1 -- start start 2024-09-10T10:47:12.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.853+0000 7feaef600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7feae80d5100 0x7feae80cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.853+0000 7feaef600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae80d3a60 con 0x7feae815e640 2024-09-10T10:47:12.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.853+0000 7feaef600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feae80cec40 con 0x7feae81621b0 2024-09-10T10:47:12.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.853+0000 7feaef600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7feae80cef20 con 0x7feae80d5100 2024-09-10T10:47:12.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.854+0000 7feaecc00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7feae815e640 0x7feae80d31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42570/0 (socket says 172.21.15.28:42570) 2024-09-10T10:47:12.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.854+0000 7feaecc00640 1 -- 172.21.15.28:0/2245307007 learned_addr learned my addr 172.21.15.28:0/2245307007 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:12.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.854+0000 7feae7e00640 1 --2- 172.21.15.28:0/2245307007 >> v2:172.21.15.138:3300/0 conn(0x7feae80d5100 0x7feae80cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.854+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117503883 0 0) 0x7feae80d3a60 con 0x7feae815e640 2024-09-10T10:47:12.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.854+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fead0003520 con 0x7feae815e640 2024-09-10T10:47:12.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2381687381 0 0) 0x7fead0003520 con 0x7feae815e640 2024-09-10T10:47:12.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 >> v2:172.21.15.138:3300/0 conn(0x7feae80d5100 msgr2=0x7feae80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 --2- 172.21.15.28:0/2245307007 >> v2:172.21.15.138:3300/0 conn(0x7feae80d5100 0x7feae80cfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 >> v1:172.21.15.138:6789/0 conn(0x7feae81621b0 legacy=0x7feae806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae80cf200 con 0x7feae815e640 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7feadc001670 con 0x7feae815e640 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 >> v1:172.21.15.28:6789/0 conn(0x7feae815e640 legacy=0x7feae80d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7fead0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead0004800 0x7fead0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead00051c0 0x7fead000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feae80cef20 con 0x7fead00039b0 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fead000c070 con 0x7fead00051c0 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fead000c270 con 0x7fead0004800 2024-09-10T10:47:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feaecc00640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7fead0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feae7e00640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead0004800 0x7fead0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.855+0000 7feaed600640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead00051c0 0x7fead000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaecc00640 1 -- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead0004800 msgr2=0x7fead0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaecc00640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fead0004800 0x7fead0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaecc00640 1 -- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead00051c0 msgr2=0x7fead000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaecc00640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fead00051c0 0x7fead000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaecc00640 1 -- 172.21.15.28:0/2245307007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead0003520 con 0x7fead00039b0 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.856+0000 7feaed600640 1 -- 172.21.15.28:0/2245307007 reap_dead start 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.857+0000 7feaecc00640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7fead0003dd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c210 tx=0x7fead0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a694c6c84a497e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.857+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007d30 con 0x7fead00039b0 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.857+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc005ad0 con 0x7fead00039b0 2024-09-10T10:47:12.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.857+0000 7feae7400640 1 -- 172.21.15.28:0/2245307007 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007070 con 0x7fead00039b0 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 -- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 msgr2=0x7fead0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7fead0003dd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c210 tx=0x7fead0005830 comp rx=0 tx=0).stop 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 -- 172.21.15.28:0/2245307007 shutdown_connections 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 --2- 172.21.15.28:0/2245307007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7fead0003dd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 -- 172.21.15.28:0/2245307007 >> 172.21.15.28:0/2245307007 conn(0x7feae808bfc0 msgr2=0x7feae80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 -- 172.21.15.28:0/2245307007 shutdown_connections 2024-09-10T10:47:12.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.858+0000 7feaef600640 1 -- 172.21.15.28:0/2245307007 wait complete. 2024-09-10T10:47:12.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.861+0000 7feaef600640 1 Processor -- start 2024-09-10T10:47:12.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.861+0000 7feaef600640 1 -- start start 2024-09-10T10:47:12.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7feae81512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815eba0 0x7feae814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae814dd20 0x7feae814e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feae8162620 con 0x7fead00039b0 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feae815ea20 con 0x7feae814dd20 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaef600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feae81627a0 con 0x7feae815eba0 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7feae81512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7feae81512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51174/0 (socket says 172.21.15.28:51174) 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaecc00640 1 -- 172.21.15.28:0/395533772 learned_addr learned my addr 172.21.15.28:0/395533772 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feaed600640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae814dd20 0x7feae814e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.863+0000 7feae7e00640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815eba0 0x7feae814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 -- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815eba0 msgr2=0x7feae814d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feae815eba0 0x7feae814d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 -- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae814dd20 msgr2=0x7feae814e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feae814dd20 0x7feae814e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 -- 172.21.15.28:0/395533772 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae81613b0 con 0x7fead00039b0 2024-09-10T10:47:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaecc00640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fead00039b0 0x7feae81512a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c210 tx=0x7feadc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feae4c00640 1 -- 172.21.15.28:0/395533772 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007d10 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feae4c00640 1 -- 172.21.15.28:0/395533772 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc004e20 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feae4c00640 1 -- 172.21.15.28:0/395533772 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007290 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaef600640 1 -- 172.21.15.28:0/395533772 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae8161690 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.864+0000 7feaef600640 1 -- 172.21.15.28:0/395533772 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae8161c50 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.867+0000 7feae4c00640 1 -- 172.21.15.28:0/395533772 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feadc03b040 con 0x7fead00039b0 2024-09-10T10:47:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.868+0000 7feaf2419740 1 Processor -- start 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.868+0000 7feae4c00640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feac806be30 0x7feac806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.869+0000 7feaf2419740 1 -- start start 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.869+0000 7feae4c00640 1 -- 172.21.15.28:0/395533772 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7feadc0aaf50 con 0x7fead00039b0 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.869+0000 7feaed600640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feac806be30 0x7feac806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5562e1f6b5f0 con 0x5562e2003c50 2024-09-10T10:47:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5562e1f134d0 con 0x5562e2005d60 2024-09-10T10:47:12.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5562e1f675b0 con 0x5562e20085a0 2024-09-10T10:47:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaed600640 1 --2- 172.21.15.28:0/395533772 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feac806be30 0x7feac806e2f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7feae808e120 tx=0x7feae0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaed600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaecc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51190/0 (socket says 172.21.15.28:51190) 2024-09-10T10:47:12.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaed600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:59858/0 (socket says 172.21.15.28:59858) 2024-09-10T10:47:12.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feae7e00640 1 -- 172.21.15.28:0/287707016 learned_addr learned my addr 172.21.15.28:0/287707016 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.870+0000 7feaecc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:52698/0 (socket says 172.21.15.28:52698) 2024-09-10T10:47:12.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 -- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 msgr2=0x5562e2003080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 -- 172.21.15.28:0/287707016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 msgr2=0x5562e2002b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 -- 172.21.15.28:0/287707016 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e1d4aa30 con 0x5562e20085a0 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaed600640 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feae7e00640 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.871+0000 7feaecc00640 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feadc0a9300 tx=0x7feadc078ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44797cd0f5fe5440 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.872+0000 7fead9200640 1 -- 172.21.15.28:0/287707016 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc07d460 con 0x5562e20085a0 2024-09-10T10:47:12.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.872+0000 7fead9200640 1 -- 172.21.15.28:0/287707016 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc081c90 con 0x5562e20085a0 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.872+0000 7fead9200640 1 -- 172.21.15.28:0/287707016 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc07d0e0 con 0x5562e20085a0 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 -- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 msgr2=0x5562e2008980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feadc0a9300 tx=0x7feadc078ad0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 -- 172.21.15.28:0/287707016 shutdown_connections 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2005d60 0x5562e2003080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2003c50 0x5562e2002b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 --2- 172.21.15.28:0/287707016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e20085a0 0x5562e2008980 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 -- 172.21.15.28:0/287707016 >> 172.21.15.28:0/287707016 conn(0x5562e2004d60 msgr2=0x5562e204d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 -- 172.21.15.28:0/287707016 shutdown_connections 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.873+0000 7feaf2419740 1 -- 172.21.15.28:0/287707016 wait complete. 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.874+0000 7feaf2419740 1 -- start start 2024-09-10T10:47:12.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.874+0000 7feaf2419740 1 Processor -- start 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2026b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2048890 0x5562e2002f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5562e1f6b5f0 con 0x5562e2005d60 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5562e1f26750 con 0x5562e20085a0 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5562e1f675b0 con 0x5562e2048890 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaf2419740 10 client.0 _ll_register_callbacks cb 0x5562e1f18230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51200/0 (socket says 172.21.15.28:51200) 2024-09-10T10:47:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaecc00640 1 -- 172.21.15.28:0/3697974754 learned_addr learned my addr 172.21.15.28:0/3697974754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feae7e00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2026b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.876+0000 7feaed600640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2048890 0x5562e2002f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 -- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2048890 msgr2=0x5562e2002f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2048890 0x5562e2002f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 -- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 msgr2=0x5562e2026b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2026b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 -- 172.21.15.28:0/3697974754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e1c7e620 con 0x5562e2005d60 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaecc00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feadc02ae20 tx=0x7feadc039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32570db91964a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feacd600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc07d460 con 0x5562e2005d60 2024-09-10T10:47:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7feadc094ce0 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.877+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feadc081c90 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7feadc081800 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 10 client.6138 fetch_fsmap learned FSMap version 55 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5562e1c3e320 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feadc081c90 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7feadc078500 con 0x5562e2005d60 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 10 client.6138 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 10 client.6138 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 20 client.6138 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e201add0 0x5562e201b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 4 client.6138 mds_command: new command op to 5752 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.878+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 --> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e1f78d20 con 0x5562e201add0 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.879+0000 7feae7e00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e201add0 0x5562e201b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.879+0000 7feae7e00640 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e201add0 0x5562e201b1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543244b10fafcb20 server_cookie=2e314c4e144cab3a in_seq=0 out_seq=0 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.879+0000 7feacd600640 10 client.6138 ms_handle_connect on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.880+0000 7f3aef800640 1 Processor -- start 2024-09-10T10:47:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.880+0000 7f3aef800640 1 -- start start 2024-09-10T10:47:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.880+0000 7feacd600640 1 -- 172.21.15.28:0/3697974754 <== mds.0 v2:172.21.15.138:6835/2819175864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562e1f78d20 con 0x5562e201add0 2024-09-10T10:47:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.880+0000 7feacd600640 10 client.6138 handle_command_reply: tid=0 2024-09-10T10:47:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.880+0000 7feaf2419740 1 client.6138 shutdown 2024-09-10T10:47:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 msgr2=0x5562e2049780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feadc02ae20 tx=0x7feadc039000 comp rx=0 tx=0).stop 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 shutdown_connections 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e201add0 0x5562e201b1f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2048890 0x5562e2002f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2026b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 --2- 172.21.15.28:0/3697974754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2005d60 0x5562e2049780 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feacd600640 0 client.6138 ms_handle_reset on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 >> 172.21.15.28:0/3697974754 conn(0x5562e2003c50 msgr2=0x5562e201cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 shutdown_connections 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 1 -- 172.21.15.28:0/3697974754 wait complete. 2024-09-10T10:47:12.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7feaf2419740 20 client.6138 trim_cache size 0 max 16384 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aef800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae815f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aef800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ae80d2b10 0x7f3ae80d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aef800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae80d51e0 con 0x7f3ae80cf570 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aef800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae815fc60 con 0x7f3ae80d2b10 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aef800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae815eb60 con 0x7f3ae8160010 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aece00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3ae80cf570 0x7f3ae80cf950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:34438/0 (socket says 172.21.15.28:34438) 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aece00640 1 -- 172.21.15.28:0/3150229742 learned_addr learned my addr 172.21.15.28:0/3150229742 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3aed800640 1 --2- 172.21.15.28:0/3150229742 >> v2:172.21.15.138:3300/0 conn(0x7f3ae80d2b10 0x7f3ae80d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.881+0000 7f3ae7e00640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.882+0000 7f3aed800640 1 -- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 msgr2=0x7f3ae815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.882+0000 7f3ae7400640 1 -- 172.21.15.28:0/3150229742 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383260148 0 0) 0x7f3ae80d51e0 con 0x7f3ae80cf570 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.882+0000 7f3aed800640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.882+0000 7f3aed800640 1 -- 172.21.15.28:0/3150229742 >> v1:172.21.15.138:6789/0 conn(0x7f3ae80cf570 legacy=0x7f3ae80cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.882+0000 7f3aed800640 1 -- 172.21.15.28:0/3150229742 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae80cedb0 con 0x7f3ae80d2b10 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.884+0000 7f3aed800640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80d2fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009d90 tx=0x7f3ae002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573010834c59708b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.884+0000 7f3ae7400640 1 -- 172.21.15.28:0/3150229742 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0007740 con 0x7f3ae80d2b10 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.884+0000 7f3ae7400640 1 -- 172.21.15.28:0/3150229742 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae00078e0 con 0x7f3ae80d2b10 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.884+0000 7f3ae7400640 1 -- 172.21.15.28:0/3150229742 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0007cb0 con 0x7f3ae80d2b10 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 -- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80d2b10 msgr2=0x7f3ae80d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80d2fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009d90 tx=0x7f3ae002a660 comp rx=0 tx=0).stop 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 -- 172.21.15.28:0/3150229742 shutdown_connections 2024-09-10T10:47:12.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80d2fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 --2- 172.21.15.28:0/3150229742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 -- 172.21.15.28:0/3150229742 >> 172.21.15.28:0/3150229742 conn(0x7f3ae808bfc0 msgr2=0x7f3ae80d4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 -- 172.21.15.28:0/3150229742 shutdown_connections 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.885+0000 7f3aef800640 1 -- 172.21.15.28:0/3150229742 wait complete. 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.886+0000 7f3aef800640 1 Processor -- start 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.887+0000 7f3aef800640 1 -- start start 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.887+0000 7f3aef800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80cf570 0x7f3ae80b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aef800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aef800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae81f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aef800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80cf090 con 0x7f3ae80d2b10 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aef800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae80ceb60 con 0x7f3ae80cf570 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aef800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae80d2580 con 0x7f3ae8160010 2024-09-10T10:47:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3ae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aece00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80cf570 0x7f3ae80b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3ae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51204/0 (socket says 172.21.15.28:51204) 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3ae7e00640 1 -- 172.21.15.28:0/4134916265 learned_addr learned my addr 172.21.15.28:0/4134916265 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aed800640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae81f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aece00640 1 -- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 msgr2=0x7f3ae81f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aece00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ae8160010 0x7f3ae81f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aece00640 1 -- 172.21.15.28:0/4134916265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 msgr2=0x7f3ae80b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.888+0000 7f3aece00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3aece00640 1 -- 172.21.15.28:0/4134916265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae8159df0 con 0x7f3ae80cf570 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3ae7e00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae80d2b10 0x7f3ae80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3aece00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ae80cf570 0x7f3ae80b1060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00c410 tx=0x7f3adc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc0074e0 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3adc0047e0 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3adc004bd0 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3aef800640 1 -- 172.21.15.28:0/4134916265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae80b8a10 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.889+0000 7f3aef800640 1 -- 172.21.15.28:0/4134916265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae80b8fd0 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.890+0000 7f3aef800640 1 -- 172.21.15.28:0/4134916265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f3ae80b36f0 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.892+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3adc011070 con 0x7f3ae80cf570 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.893+0000 7f3ae4c00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ac806be30 0x7f3ac806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.893+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f3adc08b320 con 0x7f3ae80cf570 2024-09-10T10:47:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.893+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+515 (secure 0 0 0) 0x7f3adc022070 con 0x7f3ae80cf570 2024-09-10T10:47:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.893+0000 7f3ae7e00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ac806be30 0x7f3ac806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.893+0000 7f3aef800640 1 -- 172.21.15.28:0/4134916265 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f3ae80b6f20 con 0x7f3ae80cf570 2024-09-10T10:47:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.894+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 7 ==== log(1 entries from seq 710 at 2024-09-10T10:47:12.207492+0000) ==== 309+0+0 (secure 0 0 0) 0x7f3adc0575f0 con 0x7f3ae80cf570 2024-09-10T10:47:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.894+0000 7f3ae7e00640 1 --2- 172.21.15.28:0/4134916265 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ac806be30 0x7f3ac806e2f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3ae814a520 tx=0x7f3ad4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.898+0000 7feaf2419740 1 Processor -- start 2024-09-10T10:47:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.898+0000 7feaf2419740 1 -- start start 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e2031420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2049640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feae80728a0 con 0x5562e2048890 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feae8071900 con 0x5562e20085a0 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.899+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feae8070c60 con 0x5562e2005d60 2024-09-10T10:47:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51210/0 (socket says 172.21.15.28:51210) 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 -- 172.21.15.28:0/2477359037 learned_addr learned my addr 172.21.15.28:0/2477359037 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaecc00640 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e2031420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feae7e00640 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2049640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 -- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 msgr2=0x5562e2031420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e2031420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 -- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 msgr2=0x5562e2049640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2049640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.900+0000 7feaed600640 1 -- 172.21.15.28:0/2477359037 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e2031fc0 con 0x5562e2048890 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.901+0000 7feaed600640 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae8162590 tx=0x7feae0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fd3b99a6b424a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.901+0000 7feacfe00640 1 -- 172.21.15.28:0/2477359037 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004980 con 0x5562e2048890 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.901+0000 7feacfe00640 1 -- 172.21.15.28:0/2477359037 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae0004b20 con 0x5562e2048890 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.901+0000 7feacfe00640 1 -- 172.21.15.28:0/2477359037 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0004cd0 con 0x5562e2048890 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- 172.21.15.28:0/2477359037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 msgr2=0x5562e2031a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7feae8162590 tx=0x7feae0015000 comp rx=0 tx=0).stop 2024-09-10T10:47:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- 172.21.15.28:0/2477359037 shutdown_connections 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e2048890 0x5562e2031a80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e20085a0 0x5562e2049640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 --2- 172.21.15.28:0/2477359037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e2031420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- 172.21.15.28:0/2477359037 >> 172.21.15.28:0/2477359037 conn(0x5562e2003c50 msgr2=0x5562e2026a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- 172.21.15.28:0/2477359037 shutdown_connections 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- 172.21.15.28:0/2477359037 wait complete. 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.902+0000 7feaf2419740 1 -- start start 2024-09-10T10:47:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.903+0000 7feaf2419740 1 Processor -- start 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e205faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e20085a0 0x5562e20caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2048890 0x5562e2001aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5562e201b610 con 0x5562e20085a0 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feae80712b0 con 0x5562e2048890 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5562e205f970 con 0x5562e2005d60 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaf2419740 10 client.0 _ll_register_callbacks cb 0x7fead40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:12.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e20085a0 0x5562e20caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e20085a0 0x5562e20caac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51226/0 (socket says 172.21.15.28:51226) 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 -- 172.21.15.28:0/1694598718 learned_addr learned my addr 172.21.15.28:0/1694598718 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e205faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaed600640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2048890 0x5562e2001aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 msgr2=0x5562e205faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5562e2005d60 0x5562e205faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2048890 msgr2=0x5562e2001aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2048890 0x5562e2001aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feae7e00640 1 -- 172.21.15.28:0/1694598718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e2002870 con 0x5562e20085a0 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.904+0000 7feaed600640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5562e2048890 0x5562e2001aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feae7e00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5562e20085a0 0x5562e20caac0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x5562e204c9b0 tx=0x7fead4015cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94df9481f1a1d7d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feacd600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feacd600640 1 -- 172.21.15.28:0/1694598718 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4005990 con 0x5562e20085a0 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feacd600640 1 -- 172.21.15.28:0/1694598718 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7fead4005b30 con 0x5562e20085a0 2024-09-10T10:47:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feaf2419740 1 -- 172.21.15.28:0/1694598718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae0004b20 con 0x5562e20085a0 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feacd600640 1 -- 172.21.15.28:0/1694598718 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fead4005d30 con 0x5562e20085a0 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feaf2419740 10 client.6147 fetch_fsmap learned FSMap version 55 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.905+0000 7feaf2419740 1 -- 172.21.15.28:0/1694598718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x5562e20cf730 con 0x5562e20085a0 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feacd600640 1 -- 172.21.15.28:0/1694598718 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae0004b20 con 0x5562e20085a0 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feacd600640 1 -- 172.21.15.28:0/1694598718 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7fead4037080 con 0x5562e20085a0 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 10 client.6147 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 10 client.6147 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2024-09-10T10:47:12.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 20 client.6147 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 4 client.6147 mds_command: new command op to 5752 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaf2419740 1 -- 172.21.15.28:0/1694598718 --> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562e1f73070 con 0x5562e204c110 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.906+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.907+0000 7feacd600640 10 client.6147 ms_handle_connect on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:12.907+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472f2b3936c1f917 server_cookie=d66f05074badb71d in_seq=0 out_seq=0 2024-09-10T10:47:12.920 INFO:teuthology.orchestra.run.smithi028.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-10T10:47:12.920 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-10T10:47:12.921 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:12.922 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:47:12.911+0000 7f3266200640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:47:12.911+0000 7f3266200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f3266200640 time 2024-09-10T10:47:12.912362+0000 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f3275583417] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.923 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.924 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:47:12.912+0000 7f3266200640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f3266200640 time 2024-09-10T10:47:12.912362+0000 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f3275583417] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.925 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.926 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr:*** Caught signal (Aborted) ** 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7f3266200640 thread_name:MR_Finisher 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3274c3e6f0] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3274c8b99c] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f32755834d4] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.927 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.928 INFO:tasks.ceph.mds.b.smithi138.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 21: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 22: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:47:12.914+0000 7f3266200640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7f3266200640 thread_name:MR_Finisher 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3274c3e6f0] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3274c8b99c] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f32755834d4] 2024-09-10T10:47:12.929 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.930 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 21: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 22: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: -190> 2024-09-10T10:47:11.618+0000 7f326b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: -10> 2024-09-10T10:47:12.911+0000 7f3266200640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: -8> 2024-09-10T10:47:12.911+0000 7f3266200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: -2> 2024-09-10T10:47:12.912+0000 7f3266200640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f3266200640 time 2024-09-10T10:47:12.912362+0000 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:47:12.931 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f3275583417] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.932 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 0> 2024-09-10T10:47:12.914+0000 7f3266200640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7f3266200640 thread_name:MR_Finisher 2024-09-10T10:47:12.933 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3274c3e6f0] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3274c8b99c] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f32755834d4] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.934 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 21: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 22: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:47:12.935 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.936 INFO:tasks.ceph.mds.b.smithi138.stderr: -191> 2024-09-10T10:47:11.618+0000 7f326b200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:47:12.936 INFO:tasks.ceph.mds.b.smithi138.stderr: -11> 2024-09-10T10:47:12.911+0000 7f3266200640 -1 mds.0.cache.den(0x1 a) newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: -9> 2024-09-10T10:47:12.911+0000 7f3266200640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: -3> 2024-09-10T10:47:12.912+0000 7f3266200640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7f3266200640 time 2024-09-10T10:47:12.912362+0000 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7f3275583417] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.937 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.938 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: -1> 2024-09-10T10:47:12.914+0000 7f3266200640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7f3266200640 thread_name:MR_Finisher 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7f3274c3e6f0] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7f3274c8b99c] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7f32755834d4] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x55a7878162d1] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x55a787a414e4] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x3cd2fc) [0x55a787a5b2fc] 2024-09-10T10:47:12.939 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (CDir::_omap_commit(int)+0x93a) [0x55a787a5cc2a] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (CDir::commit(unsigned long, MDSContext*, bool, int)+0x265) [0x55a787a5e205] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (LogSegment::try_to_expire(MDSRank*, C_GatherBuilderBase >&, int)+0xaf7) [0x55a787863417] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (MDLog::try_expire(LogSegment*, int)+0x62) [0x55a787ae24d2] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDLog::trim_all()+0x769) [0x55a787ae5399] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: ceph-mds(+0x1bfd71) [0x55a78784dd71] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: ceph-mds(+0x16033d) [0x55a7877ee33d] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: ceph-mds(+0x160602) [0x55a7877ee602] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: (MDSContext::complete(int)+0x5c) [0x55a787ada8ec] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: (MDSIOContextBase::complete(int)+0x31c) [0x55a787adad3c] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 20: (Finisher::finisher_thread_entry()+0x175) [0x7f3275641aa5] 2024-09-10T10:47:12.940 INFO:tasks.ceph.mds.b.smithi138.stderr: 21: /lib64/libc.so.6(+0x89c52) [0x7f3274c89c52] 2024-09-10T10:47:12.941 INFO:tasks.ceph.mds.b.smithi138.stderr: 22: /lib64/libc.so.6(+0x10ec80) [0x7f3274d0ec80] 2024-09-10T10:47:12.941 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:47:12.941 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.001+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.001+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.001+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.001+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.002+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk reading from fd=18 : -104 Unknown error -104 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.002+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.002+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner read peer banner failed r=-1 ((1) Operation not permitted) 2024-09-10T10:47:13.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.002+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T10:47:13.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.090+0000 7f43fa000640 1 Processor -- start 2024-09-10T10:47:13.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.090+0000 7f43fa000640 1 -- start start 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43fa000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f415b300 0x7f43f40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43fa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43f40cd3f0 0x7f43f406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43fa000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f40cdbe0 con 0x7f43f4072d30 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43fa000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f43f40728a0 con 0x7f43f40cd3f0 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43fa000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43f40cd900 con 0x7f43f415b300 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43ebe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43f40cd3f0 0x7f43f406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43ebe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43f40cd3f0 0x7f43f406f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51228/0 (socket says 172.21.15.28:51228) 2024-09-10T10:47:13.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43ebe00640 1 -- 172.21.15.28:0/4210763983 learned_addr learned my addr 172.21.15.28:0/4210763983 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:13.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43f3400640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f415b300 0x7f43f40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.092+0000 7f43ebe00640 1 -- 172.21.15.28:0/4210763983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f415b300 msgr2=0x7f43f40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:13.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.093+0000 7f43ebe00640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f415b300 0x7f43f40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.093+0000 7f43ebe00640 1 -- 172.21.15.28:0/4210763983 >> v1:172.21.15.138:6789/0 conn(0x7f43f4072d30 legacy=0x7f43f40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.093+0000 7f43ebe00640 1 -- 172.21.15.28:0/4210763983 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f4072250 con 0x7f43f40cd3f0 2024-09-10T10:47:13.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43ebe00640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f40cd3f0 0x7f43f406f1c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009870 tx=0x7f43dc02c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69640fa72af0af9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43f2a00640 1 -- 172.21.15.28:0/4210763983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc038070 con 0x7f43f40cd3f0 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43f2a00640 1 -- 172.21.15.28:0/4210763983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43dc031710 con 0x7f43f40cd3f0 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43f2a00640 1 -- 172.21.15.28:0/4210763983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc031ae0 con 0x7f43f40cd3f0 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 -- 172.21.15.28:0/4210763983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f40cd3f0 msgr2=0x7f43f406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f40cd3f0 0x7f43f406f1c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009870 tx=0x7f43dc02c480 comp rx=0 tx=0).stop 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 -- 172.21.15.28:0/4210763983 shutdown_connections 2024-09-10T10:47:13.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f40cd3f0 0x7f43f406f1c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 --2- 172.21.15.28:0/4210763983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f415b300 0x7f43f40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.094+0000 7f43fa000640 1 -- 172.21.15.28:0/4210763983 >> 172.21.15.28:0/4210763983 conn(0x7f43f4067d70 msgr2=0x7f43f406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.095+0000 7f43fa000640 1 -- 172.21.15.28:0/4210763983 shutdown_connections 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.095+0000 7f43fa000640 1 -- 172.21.15.28:0/4210763983 wait complete. 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.096+0000 7f43fa000640 1 Processor -- start 2024-09-10T10:47:13.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.096+0000 7f43fa000640 1 -- start start 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43f40cd3f0 0x7f43f40ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 0x7f43f4153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40d0160 con 0x7f43f415b300 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40cffe0 con 0x7f43f40cd3f0 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43fa000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43f406f700 con 0x7f43f4072d30 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 0x7f43f4153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43f3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43f3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:52754/0 (socket says 172.21.15.28:52754) 2024-09-10T10:47:13.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43f3400640 1 -- 172.21.15.28:0/2347836326 learned_addr learned my addr 172.21.15.28:0/2347836326 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.097+0000 7f43f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 0x7f43f4153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51244/0 (socket says 172.21.15.28:51244) 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43ebe00640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43f40cd3f0 0x7f43f40ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 -- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43f40cd3f0 msgr2=0x7f43f40ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43f40cd3f0 0x7f43f40ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 -- 172.21.15.28:0/2347836326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 msgr2=0x7f43f4153530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 0x7f43f4153530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 -- 172.21.15.28:0/2347836326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f40cae90 con 0x7f43f4072d30 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43f3400640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e8a0 tx=0x7f43e400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e4017070 con 0x7f43f4072d30 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e400c940 con 0x7f43f4072d30 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e400cd30 con 0x7f43f4072d30 2024-09-10T10:47:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.098+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f40cb170 con 0x7f43f4072d30 2024-09-10T10:47:13.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.099+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f41503d0 con 0x7f43f4072d30 2024-09-10T10:47:13.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.100+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f415c340 con 0x7f43f4072d30 2024-09-10T10:47:13.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.101+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43e401a070 con 0x7f43f4072d30 2024-09-10T10:47:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.102+0000 7f43eb400640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 0x7f43d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.102+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f43e408b2f0 con 0x7f43f4072d30 2024-09-10T10:47:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.102+0000 7f43ebe00640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 0x7f43d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.103+0000 7f43ebe00640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 0x7f43d006e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f43dc035000 tx=0x7f43dc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:13.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.106+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43e40593d0 con 0x7f43f4072d30 2024-09-10T10:47:13.184 INFO:tasks.ceph.mds.b.smithi138.stderr:daemon-helper: command crashed with signal 6 2024-09-10T10:47:13.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.203+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:13.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.203+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:13.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.203+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T10:47:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.328+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43f4150c00 con 0x7f43f4072d30 2024-09-10T10:47:13.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.329+0000 7f43eb400640 1 -- 172.21.15.28:0/2347836326 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f43e405d6d0 con 0x7f43f4072d30 2024-09-10T10:47:13.331 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.331 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"btime":"2024-09-10T10:47:12:657299+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:12.207564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:13.332 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 55 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 msgr2=0x7f43d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 0x7f43d006e2f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f43dc035000 tx=0x7f43dc008030 comp rx=0 tx=0).stop 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 msgr2=0x7f43f414fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43e400e8a0 tx=0x7f43e400ed60 comp rx=0 tx=0).stop 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 shutdown_connections 2024-09-10T10:47:13.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43d006be30 0x7f43d006e2f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43f415b300 0x7f43f4153530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43f40cd3f0 0x7f43f40ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 --2- 172.21.15.28:0/2347836326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43f4072d30 0x7f43f414fe90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.331+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 >> 172.21.15.28:0/2347836326 conn(0x7f43f4067d70 msgr2=0x7f43f406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.332+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 shutdown_connections 2024-09-10T10:47:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.332+0000 7f43fa000640 1 -- 172.21.15.28:0/2347836326 wait complete. 2024-09-10T10:47:13.352 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:47:13.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.604+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:13.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.604+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:13.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.604+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T10:47:13.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:13.665+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 8 ==== log(2 entries from seq 711 at 2024-09-10T10:47:12.662591+0000) ==== 635+0+0 (secure 0 0 0) 0x7f3adc057850 con 0x7f3ae80cf570 2024-09-10T10:47:13.667 INFO:teuthology.orchestra.run.smithi028.stdout: cluster: 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: id: f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: health: HEALTH_OK 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: services: 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: mon: 3 daemons, quorum a,b,c (age 3m) 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: mgr: x(active, since 3m), standbys: y 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: data: 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: volumes: 1/1 healthy 2024-09-10T10:47:13.668 INFO:teuthology.orchestra.run.smithi028.stdout: pools: 3 pools, 97 pgs 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: objects: 29 objects, 602 KiB 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: usage: 239 MiB used, 720 GiB / 720 GiB avail 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: pgs: 97 active+clean 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: io: 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: client: 6.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s rd, 2 op/s wr 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:13.669 INFO:teuthology.orchestra.run.smithi028.stdout:2024-09-10T10:47:12.912300+0000 mds.b [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a [fffffffffffffff6,head] auth (dversion lock) v=13 ino=0x10000000000 state=1610612736 | inodepin=1 dirty=1 0x55a791413180] 2024-09-10T10:47:14.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:14.406+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:14.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:14.406+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:14.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:14.406+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T10:47:16.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:16.007+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:16.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:16.007+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:16.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:16.007+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T10:47:18.353 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:18.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.520+0000 7f081ae00640 1 Processor -- start 2024-09-10T10:47:18.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.520+0000 7f081ae00640 1 -- start start 2024-09-10T10:47:18.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f081ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08140d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f081ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f08140d1f90 0x7f08140cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f081ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08140cf000 con 0x7f081415e660 2024-09-10T10:47:18.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f081ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f081406d3e0 con 0x7f08140d1f90 2024-09-10T10:47:18.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f081ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f08140b37d0 con 0x7f08140d0dd0 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f0819000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f081415e660 0x7f081406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46862/0 (socket says 172.21.15.28:46862) 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.521+0000 7f0819000640 1 -- 172.21.15.28:0/1951211437 learned_addr learned my addr 172.21.15.28:0/1951211437 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f081a400640 1 --2- 172.21.15.28:0/1951211437 >> v2:172.21.15.28:3300/0 conn(0x7f08140d1f90 0x7f08140cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f0819a00640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08140d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f081a400640 1 -- 172.21.15.28:0/1951211437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 msgr2=0x7f08140d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f081a400640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08140d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f081a400640 1 -- 172.21.15.28:0/1951211437 >> v1:172.21.15.28:6789/0 conn(0x7f081415e660 legacy=0x7f081406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f081a400640 1 -- 172.21.15.28:0/1951211437 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140d16f0 con 0x7f08140d1f90 2024-09-10T10:47:18.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.522+0000 7f0813e00640 1 -- 172.21.15.28:0/1951211437 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784628281 0 0) 0x7f08140cf000 con 0x7f081415e660 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.523+0000 7f081a400640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f08140cf7c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f080c009d90 tx=0x7f080c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce5466fcd525dba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.523+0000 7f0813e00640 1 -- 172.21.15.28:0/1951211437 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c007740 con 0x7f08140d1f90 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.523+0000 7f0813e00640 1 -- 172.21.15.28:0/1951211437 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f080c0078e0 con 0x7f08140d1f90 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.523+0000 7f0813e00640 1 -- 172.21.15.28:0/1951211437 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080c007cb0 con 0x7f08140d1f90 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 -- 172.21.15.28:0/1951211437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 msgr2=0x7f08140cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f08140cf7c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f080c009d90 tx=0x7f080c02a680 comp rx=0 tx=0).stop 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 -- 172.21.15.28:0/1951211437 shutdown_connections 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f08140cf7c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 --2- 172.21.15.28:0/1951211437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08140d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 -- 172.21.15.28:0/1951211437 >> 172.21.15.28:0/1951211437 conn(0x7f081408bfc0 msgr2=0x7f08140d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:18.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 -- 172.21.15.28:0/1951211437 shutdown_connections 2024-09-10T10:47:18.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.524+0000 7f081ae00640 1 -- 172.21.15.28:0/1951211437 wait complete. 2024-09-10T10:47:18.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.526+0000 7f081ae00640 1 Processor -- start 2024-09-10T10:47:18.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.526+0000 7f081ae00640 1 -- start start 2024-09-10T10:47:18.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08141e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415e660 0x7f08140b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f08140d2c60 con 0x7f08140d1f90 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08140cf330 con 0x7f081415e660 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f08140d2f40 con 0x7f08140d0dd0 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40318/0 (socket says 172.21.15.28:40318) 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 -- 172.21.15.28:0/2032688336 learned_addr learned my addr 172.21.15.28:0/2032688336 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819a00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08141e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f081a400640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415e660 0x7f08140b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 -- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 msgr2=0x7f08141e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08141e5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 -- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415e660 msgr2=0x7f08140b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415e660 0x7f08140b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.527+0000 7f0819000640 1 -- 172.21.15.28:0/2032688336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08140b8af0 con 0x7f08140d1f90 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f0819000640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f080000c410 tx=0x7f0800001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08000074e0 con 0x7f08140d1f90 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08000047e0 con 0x7f08140d1f90 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08140b8dd0 con 0x7f08140d1f90 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004bd0 con 0x7f08140d1f90 2024-09-10T10:47:18.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.528+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08140b9390 con 0x7f08140d1f90 2024-09-10T10:47:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.529+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08140b9870 con 0x7f08140d1f90 2024-09-10T10:47:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.530+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0800011070 con 0x7f08140d1f90 2024-09-10T10:47:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.531+0000 7f0811600640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 0x7f07f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:18.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.531+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7f080008ac60 con 0x7f08140d1f90 2024-09-10T10:47:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.531+0000 7f0819a00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 0x7f07f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:18.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.532+0000 7f0819a00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 0x7f07f406e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0808006fd0 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:18.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.536+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08140b9870 con 0x7f08140d1f90 2024-09-10T10:47:18.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.757+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08140d3220 con 0x7f08140d1f90 2024-09-10T10:47:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.758+0000 7f0811600640 1 -- 172.21.15.28:0/2032688336 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f0800059390 con 0x7f08140d1f90 2024-09-10T10:47:18.759 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:18.759 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"btime":"2024-09-10T10:47:12:657299+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:12.207564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 55 2024-09-10T10:47:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 msgr2=0x7f07f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 0x7f07f406e2f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0808006fd0 tx=0x7f0808008040 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 msgr2=0x7f081415c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f080000c410 tx=0x7f0800001e70 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 shutdown_connections 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f406be30 0x7f07f406e2f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f081415e660 0x7f08140b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08140d1f90 0x7f081415c8e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 --2- 172.21.15.28:0/2032688336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08140d0dd0 0x7f08141e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 >> 172.21.15.28:0/2032688336 conn(0x7f081408bfc0 msgr2=0x7f08140b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 shutdown_connections 2024-09-10T10:47:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:18.760+0000 7f081ae00640 1 -- 172.21.15.28:0/2032688336 wait complete. 2024-09-10T10:47:18.777 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:47:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:19.211+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:19.211+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:19.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:19.211+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-09-10T10:47:19.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T10:47:23.778 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:23.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.947+0000 7fcf20e00640 1 Processor -- start 2024-09-10T10:47:23.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.947+0000 7fcf20e00640 1 -- start start 2024-09-10T10:47:23.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf20e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf20e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf1c0b25c0 0x7fcf1c0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf20e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf1c0b5470 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf20e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1c0b3640 con 0x7fcf1c0b25c0 2024-09-10T10:47:23.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf20e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b5190 con 0x7fcf1c15bef0 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf1aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf1c0b25c0 0x7fcf1c0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:35728/0 (socket says 172.21.15.28:35728) 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.948+0000 7fcf1b400640 1 -- 172.21.15.28:0/831268008 learned_addr learned my addr 172.21.15.28:0/831268008 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.949+0000 7fcf1b400640 1 -- 172.21.15.28:0/831268008 >> v2:172.21.15.138:3300/0 conn(0x7fcf1c0b25c0 msgr2=0x7fcf1c0aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:23.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.949+0000 7fcf1b400640 1 --2- 172.21.15.28:0/831268008 >> v2:172.21.15.138:3300/0 conn(0x7fcf1c0b25c0 0x7fcf1c0aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.949+0000 7fcf1b400640 1 -- 172.21.15.28:0/831268008 >> v1:172.21.15.138:6789/0 conn(0x7fcf1c0af1b0 legacy=0x7fcf1c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:23.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.949+0000 7fcf1b400640 1 -- 172.21.15.28:0/831268008 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c0b38e0 con 0x7fcf1c15bef0 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.950+0000 7fcf1b400640 1 --2- 172.21.15.28:0/831268008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009870 tx=0x7fcf0c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0f6aed602fd2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.950+0000 7fcf1a000640 1 -- 172.21.15.28:0/831268008 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c037070 con 0x7fcf1c15bef0 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.950+0000 7fcf1a000640 1 -- 172.21.15.28:0/831268008 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c032440 con 0x7fcf1c15bef0 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.950+0000 7fcf1a000640 1 -- 172.21.15.28:0/831268008 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c032810 con 0x7fcf1c15bef0 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 -- 172.21.15.28:0/831268008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 msgr2=0x7fcf1c0b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 --2- 172.21.15.28:0/831268008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009870 tx=0x7fcf0c02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 -- 172.21.15.28:0/831268008 shutdown_connections 2024-09-10T10:47:23.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 --2- 172.21.15.28:0/831268008 >> v2:172.21.15.138:3300/0 conn(0x7fcf1c0b25c0 0x7fcf1c0aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 --2- 172.21.15.28:0/831268008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c0b40e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 -- 172.21.15.28:0/831268008 >> 172.21.15.28:0/831268008 conn(0x7fcf1c06c240 msgr2=0x7fcf1c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:23.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 -- 172.21.15.28:0/831268008 shutdown_connections 2024-09-10T10:47:23.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.951+0000 7fcf20e00640 1 -- 172.21.15.28:0/831268008 wait complete. 2024-09-10T10:47:23.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.953+0000 7fcf20e00640 1 Processor -- start 2024-09-10T10:47:23.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.953+0000 7fcf20e00640 1 -- start start 2024-09-10T10:47:23.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 0x7fcf1c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b4e60 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c0af660 con 0x7fcf1c0b25c0 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf20e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf1c0b4b10 con 0x7fcf1c15bef0 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39146/0 (socket says 172.21.15.28:39146) 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 -- 172.21.15.28:0/1559129405 learned_addr learned my addr 172.21.15.28:0/1559129405 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:23.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1aa00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 0x7fcf1c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1be00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 -- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 msgr2=0x7fcf1c159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 -- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 msgr2=0x7fcf1c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 0x7fcf1c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1b400640 1 -- 172.21.15.28:0/1559129405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c1582e0 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1aa00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 0x7fcf1c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.954+0000 7fcf1be00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:23.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf1b400640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a200 tx=0x7fcf0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:23.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c041030 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c030440 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c037040 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c159eb0 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.955+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c15a470 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.956+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf1c0b2a00 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.957+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf0c044070 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.958+0000 7fcf13400640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 0x7fcefc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:23.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.958+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6187+0+0 (secure 0 0 0) 0x7fcf0c0ab6f0 con 0x7fcf1c0af1b0 2024-09-10T10:47:23.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.958+0000 7fcf1aa00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 0x7fcefc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:23.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.959+0000 7fcf1aa00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 0x7fcefc06e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c157f60 tx=0x7fcf08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:23.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:23.963+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf1c1582e0 con 0x7fcf1c0af1b0 2024-09-10T10:47:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.182+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf1c1e8950 con 0x7fcf1c0af1b0 2024-09-10T10:47:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.183+0000 7fcf13400640 1 -- 172.21.15.28:0/1559129405 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7fcf0c079e20 con 0x7fcf1c0af1b0 2024-09-10T10:47:24.184 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:24.185 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"btime":"2024-09-10T10:47:12:657299+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:12.207564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 55 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 msgr2=0x7fcefc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 0x7fcefc06e2f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c157f60 tx=0x7fcf08008040 comp rx=0 tx=0).stop 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 msgr2=0x7fcf1c156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c02a200 tx=0x7fcf0c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 shutdown_connections 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcefc06be30 0x7fcefc06e2f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf1c15bef0 0x7fcf1c159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf1c0b25c0 0x7fcf1c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 --2- 172.21.15.28:0/1559129405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1c0af1b0 0x7fcf1c156e70 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 >> 172.21.15.28:0/1559129405 conn(0x7fcf1c06c240 msgr2=0x7fcf1c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 shutdown_connections 2024-09-10T10:47:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:24.185+0000 7fcf20e00640 1 -- 172.21.15.28:0/1559129405 wait complete. 2024-09-10T10:47:24.202 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:47:25.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:25.618+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:25.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:25.618+0000 7feacd600640 1 client.6147 ms_handle_refused on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:25.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:25.618+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 12.800000 2024-09-10T10:47:26.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-09-10T10:47:27.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:27.746+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 9 ==== log(7 entries from seq 715 at 2024-09-10T10:47:27.607844+0000) ==== 2031+0+0 (secure 0 0 0) 0x7f3adc0896e0 con 0x7f3ae80cf570 2024-09-10T10:47:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:28.760+0000 7f3ae4c00640 1 -- 172.21.15.28:0/4134916265 <== mon.1 v2:172.21.15.138:3300/0 10 ==== log(7 entries from seq 722 at 2024-09-10T10:47:27.751186+0000) ==== 2045+0+0 (secure 0 0 0) 0x7f3adc055bc0 con 0x7f3ae80cf570 2024-09-10T10:47:29.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:29.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.369+0000 7f5c74e00640 1 Processor -- start 2024-09-10T10:47:29.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.369+0000 7f5c74e00640 1 -- start start 2024-09-10T10:47:29.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c74e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c7015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c74e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c70146140 con 0x7f5c70072320 2024-09-10T10:47:29.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c74e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c700cd740 con 0x7f5c70072de0 2024-09-10T10:47:29.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c74e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c700cda20 con 0x7f5c70072700 2024-09-10T10:47:29.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c6de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5c70072320 0x7f5c700cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34404/0 (socket says 172.21.15.28:34404) 2024-09-10T10:47:29.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c6de00640 1 -- 172.21.15.28:0/735144454 learned_addr learned my addr 172.21.15.28:0/735144454 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:29.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c6e800640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c7015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283888849 0 0) 0x7f5c70146140 con 0x7f5c70072320 2024-09-10T10:47:29.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.371+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c54003520 con 0x7f5c70072320 2024-09-10T10:47:29.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3742714015 0 0) 0x7f5c54003520 con 0x7f5c70072320 2024-09-10T10:47:29.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 msgr2=0x7f5c7015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c7015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 >> v1:172.21.15.138:6789/0 conn(0x7f5c70072de0 legacy=0x7f5c7015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700cdd00 con 0x7f5c70072320 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f5c60001670 con 0x7f5c70072320 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 >> v1:172.21.15.28:6789/0 conn(0x7f5c70072320 legacy=0x7f5c700cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c540039e0 0x7f5c54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c54004800 0x7f5c54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700cda20 con 0x7f5c54004800 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5400bac0 con 0x7f5c540039e0 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c5400bcc0 con 0x7f5c70072700 2024-09-10T10:47:29.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c67e00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c54004800 0x7f5c54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6de00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c540039e0 0x7f5c54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.372+0000 7f5c6e800640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.373+0000 7f5c67e00640 1 -- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 msgr2=0x7f5c540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.373+0000 7f5c67e00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c70072700 0x7f5c540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.373+0000 7f5c67e00640 1 -- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c540039e0 msgr2=0x7f5c54003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.373+0000 7f5c67e00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c540039e0 0x7f5c54003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.373+0000 7f5c67e00640 1 -- 172.21.15.28:0/735144454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c54003520 con 0x7f5c54004800 2024-09-10T10:47:29.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c67e00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c54004800 0x7f5c54004c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5c58001860 tx=0x7f5c54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e08cb9e6eea177 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:29.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58037070 con 0x7f5c54004800 2024-09-10T10:47:29.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c5802abf0 con 0x7f5c54004800 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c6d400640 1 -- 172.21.15.28:0/735144454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58001430 con 0x7f5c54004800 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c74e00640 1 -- 172.21.15.28:0/735144454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c54004800 msgr2=0x7f5c54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c74e00640 1 --2- 172.21.15.28:0/735144454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c54004800 0x7f5c54004c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5c58001860 tx=0x7f5c54005830 comp rx=0 tx=0).stop 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.374+0000 7f5c6e800640 1 -- 172.21.15.28:0/735144454 reap_dead start 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.375+0000 7f5c74e00640 1 -- 172.21.15.28:0/735144454 shutdown_connections 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.375+0000 7f5c74e00640 1 -- 172.21.15.28:0/735144454 >> 172.21.15.28:0/735144454 conn(0x7f5c70067d70 msgr2=0x7f5c7006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.375+0000 7f5c74e00640 1 -- 172.21.15.28:0/735144454 shutdown_connections 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.375+0000 7f5c74e00640 1 -- 172.21.15.28:0/735144454 wait complete. 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.376+0000 7f5c74e00640 1 Processor -- start 2024-09-10T10:47:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.376+0000 7f5c74e00640 1 -- start start 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c54004800 0x7f5c7015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 0x7f5c70143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7015da70 con 0x7f5c701e5af0 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7015d8f0 con 0x7f5c701e5f70 2024-09-10T10:47:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c74e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7015dbf0 con 0x7f5c54004800 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.377+0000 7f5c67e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39162/0 (socket says 172.21.15.28:39162) 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 -- 172.21.15.28:0/2580615207 learned_addr learned my addr 172.21.15.28:0/2580615207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c6de00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c54004800 0x7f5c7015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 -- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c54004800 msgr2=0x7f5c7015b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c54004800 0x7f5c7015b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c6e800640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 0x7f5c70143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 -- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 msgr2=0x7f5c70143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 0x7f5c70143840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 -- 172.21.15.28:0/2580615207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7013f180 con 0x7f5c701e5af0 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c6e800640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 0x7f5c70143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c67e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c5802a6c0 tx=0x7f5c58001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5803f020 con 0x7f5c701e5af0 2024-09-10T10:47:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c580067d0 con 0x7f5c701e5af0 2024-09-10T10:47:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.378+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58037040 con 0x7f5c701e5af0 2024-09-10T10:47:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.379+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7013f460 con 0x7f5c701e5af0 2024-09-10T10:47:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.379+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7013fa20 con 0x7f5c701e5af0 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.379+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7015e020 con 0x7f5c701e5af0 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.380+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c58042020 con 0x7f5c701e5af0 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.381+0000 7f5c66000640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 0x7f5c4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.381+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6390+0+0 (secure 0 0 0) 0x7f5c580aaf40 con 0x7f5c701e5af0 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.382+0000 7f5c6de00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 0x7f5c4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.382+0000 7f5c6de00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 0x7f5c4c06e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001790 tx=0x7f5c60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.386+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c58049030 con 0x7f5c701e5af0 2024-09-10T10:47:29.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.604+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c7006bda0 con 0x7f5c701e5af0 2024-09-10T10:47:29.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.605+0000 7f5c66000640 1 -- 172.21.15.28:0/2580615207 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5231 (secure 0 0 0) 0x7f5c58079630 con 0x7f5c701e5af0 2024-09-10T10:47:29.606 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:29.606 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":55,"btime":"2024-09-10T10:47:12:657299+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:12.207564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:29.606 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 55 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 msgr2=0x7f5c4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 0x7f5c4c06e2f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5c60001790 tx=0x7f5c60009040 comp rx=0 tx=0).stop 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 msgr2=0x7f5c701f4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5c5802a6c0 tx=0x7f5c58001230 comp rx=0 tx=0).stop 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 shutdown_connections 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c4c06be30 0x7f5c4c06e2f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c701e5f70 0x7f5c70143840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c701e5af0 0x7f5c701f4120 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 --2- 172.21.15.28:0/2580615207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c54004800 0x7f5c7015b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:29.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 >> 172.21.15.28:0/2580615207 conn(0x7f5c70067d70 msgr2=0x7f5c700b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:29.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 shutdown_connections 2024-09-10T10:47:29.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:29.607+0000 7f5c74e00640 1 -- 172.21.15.28:0/2580615207 wait complete. 2024-09-10T10:47:29.624 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:47:33.060 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2024-09-10T10:47:34.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:34.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.793+0000 7fbb8f000640 1 Processor -- start 2024-09-10T10:47:34.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.793+0000 7fbb8f000640 1 -- start start 2024-09-10T10:47:34.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.794+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb8815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.794+0000 7fbb8f000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb88072700 0x7fbb8815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.794+0000 7fbb8f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb88146140 con 0x7fbb88072320 2024-09-10T10:47:34.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.794+0000 7fbb8f000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbb880cd740 con 0x7fbb88072700 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.794+0000 7fbb8f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb880cda20 con 0x7fbb88072de0 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb88072700 0x7fbb8815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb88072700 0x7fbb8815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36658/0 (socket says 172.21.15.28:36658) 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 -- 172.21.15.28:0/2521085197 learned_addr learned my addr 172.21.15.28:0/2521085197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb87400640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb8815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 -- 172.21.15.28:0/2521085197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 msgr2=0x7fbb8815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb8815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:34.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 -- 172.21.15.28:0/2521085197 >> v1:172.21.15.138:6789/0 conn(0x7fbb88072320 legacy=0x7fbb880cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:34.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.795+0000 7fbb8d000640 1 -- 172.21.15.28:0/2521085197 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb880cdd00 con 0x7fbb88072700 2024-09-10T10:47:34.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.796+0000 7fbb8d000640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072700 0x7fbb8815d720 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009e50 tx=0x7fbb8002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd7ad51740ca38c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:34.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.796+0000 7fbb86a00640 1 -- 172.21.15.28:0/2521085197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80036070 con 0x7fbb88072700 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.796+0000 7fbb86a00640 1 -- 172.21.15.28:0/2521085197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb80031600 con 0x7fbb88072700 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.796+0000 7fbb86a00640 1 -- 172.21.15.28:0/2521085197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb800319d0 con 0x7fbb88072700 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 -- 172.21.15.28:0/2521085197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072700 msgr2=0x7fbb8815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072700 0x7fbb8815d720 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009e50 tx=0x7fbb8002a680 comp rx=0 tx=0).stop 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 -- 172.21.15.28:0/2521085197 shutdown_connections 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072700 0x7fbb8815d720 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 --2- 172.21.15.28:0/2521085197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb8815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 -- 172.21.15.28:0/2521085197 >> 172.21.15.28:0/2521085197 conn(0x7fbb88067d70 msgr2=0x7fbb8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:34.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 -- 172.21.15.28:0/2521085197 shutdown_connections 2024-09-10T10:47:34.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.797+0000 7fbb8f000640 1 -- 172.21.15.28:0/2521085197 wait complete. 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.799+0000 7fbb8f000640 1 Processor -- start 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.799+0000 7fbb8f000640 1 -- start start 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 0x7fbb8815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb880cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8815dde0 con 0x7fbb88072320 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8815dc60 con 0x7fbb88072700 2024-09-10T10:47:34.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8815df60 con 0x7fbb88072de0 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36672/0 (socket says 172.21.15.28:36672) 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb87e00640 1 -- 172.21.15.28:0/3254449706 learned_addr learned my addr 172.21.15.28:0/3254449706 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb8d000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb880cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.800+0000 7fbb87400640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 0x7fbb8815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 -- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 msgr2=0x7fbb880cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb880cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 -- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 msgr2=0x7fbb8815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:34.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 0x7fbb8815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 -- 172.21.15.28:0/3254449706 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8815c390 con 0x7fbb88072320 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb8d000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb880cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87400640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 0x7fbb8815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb87e00640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c400 tx=0x7fbb7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb780052d0 con 0x7fbb88072320 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb78005490 con 0x7fbb88072320 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb78007260 con 0x7fbb88072320 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb880a9b20 con 0x7fbb88072320 2024-09-10T10:47:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.801+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb881f7b00 con 0x7fbb88072320 2024-09-10T10:47:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.802+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb880ce270 con 0x7fbb88072320 2024-09-10T10:47:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.803+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb78007400 con 0x7fbb88072320 2024-09-10T10:47:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.804+0000 7fbb7fe00640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.804+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6390+0+0 (secure 0 0 0) 0x7fbb7808a5b0 con 0x7fbb88072320 2024-09-10T10:47:34.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.805+0000 7fbb87400640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:34.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.806+0000 7fbb87400640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb881e8750 tx=0x7fbb74008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:34.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:34.809+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb880ce270 con 0x7fbb88072320 2024-09-10T10:47:35.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.032+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb88143380 con 0x7fbb88072320 2024-09-10T10:47:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.033+0000 7fbb7fe00640 1 -- 172.21.15.28:0/3254449706 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5279 (secure 0 0 0) 0x7fbb78058650 con 0x7fbb88072320 2024-09-10T10:47:35.035 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:35.035 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":56,"btime":"2024-09-10T10:47:31:628366+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5572,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:31.628310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5752},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5752":{"gid":5752,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":5,"addr":"172.21.15.138:6837/2819175864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2819175864},{"type":"v1","addr":"172.21.15.138:6837","nonce":2819175864}]},"join_fscid":-1,"laggy_since":"2024-09-10T10:47:31.628310+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5752,"qdb_cluster":[5752]},"id":4}]} 2024-09-10T10:47:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 56 2024-09-10T10:47:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 msgr2=0x7fbb6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 0x7fbb6806e2f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbb881e8750 tx=0x7fbb74008040 comp rx=0 tx=0).stop 2024-09-10T10:47:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 msgr2=0x7fbb880a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fbb7800c400 tx=0x7fbb7800c8c0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 shutdown_connections 2024-09-10T10:47:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb6806be30 0x7fbb6806e2f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb88072de0 0x7fbb880cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb88072700 0x7fbb8815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 --2- 172.21.15.28:0/3254449706 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb88072320 0x7fbb880a9490 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 >> 172.21.15.28:0/3254449706 conn(0x7fbb88067d70 msgr2=0x7fbb8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 shutdown_connections 2024-09-10T10:47:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.035+0000 7fbb8f000640 1 -- 172.21.15.28:0/3254449706 wait complete. 2024-09-10T10:47:35.052 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-09-10T10:47:35.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-09-10T10:47:35.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.310+0000 7fc396600640 1 Processor -- start 2024-09-10T10:47:35.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.310+0000 7fc396600640 1 -- start start 2024-09-10T10:47:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.311+0000 7fc396600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc3900d2920 0x7fc3900d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.311+0000 7fc396600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3900d3270 con 0x7fc3900b3400 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.311+0000 7fc396600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3900cee50 con 0x7fc39015e550 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.311+0000 7fc396600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc3900d4a50 con 0x7fc3900d2920 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.311+0000 7fc395c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc3900d2920 0x7fc3900d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.312+0000 7fc395c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc3900d2920 0x7fc3900d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36678/0 (socket says 172.21.15.28:36678) 2024-09-10T10:47:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.312+0000 7fc395c00640 1 -- 172.21.15.28:0/834625011 learned_addr learned my addr 172.21.15.28:0/834625011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.312+0000 7fc395c00640 1 -- 172.21.15.28:0/834625011 >> v1:172.21.15.138:6789/0 conn(0x7fc39015e550 legacy=0x7fc39006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.312+0000 7fc395c00640 1 -- 172.21.15.28:0/834625011 >> v1:172.21.15.28:6789/0 conn(0x7fc3900b3400 legacy=0x7fc3900d1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.312+0000 7fc395c00640 1 -- 172.21.15.28:0/834625011 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900d4ca0 con 0x7fc3900d2920 2024-09-10T10:47:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.313+0000 7fc395c00640 1 --2- 172.21.15.28:0/834625011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3900d2920 0x7fc3900d2d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc38802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabb966a986bf9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.313+0000 7fc38f400640 1 -- 172.21.15.28:0/834625011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388036070 con 0x7fc3900d2920 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.313+0000 7fc38f400640 1 -- 172.21.15.28:0/834625011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388031600 con 0x7fc3900d2920 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.313+0000 7fc38f400640 1 -- 172.21.15.28:0/834625011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3880319d0 con 0x7fc3900d2920 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 -- 172.21.15.28:0/834625011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3900d2920 msgr2=0x7fc3900d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 --2- 172.21.15.28:0/834625011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3900d2920 0x7fc3900d2d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc38802a690 comp rx=0 tx=0).stop 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 -- 172.21.15.28:0/834625011 shutdown_connections 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 --2- 172.21.15.28:0/834625011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3900d2920 0x7fc3900d2d30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 -- 172.21.15.28:0/834625011 >> 172.21.15.28:0/834625011 conn(0x7fc39008bfc0 msgr2=0x7fc3900d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 -- 172.21.15.28:0/834625011 shutdown_connections 2024-09-10T10:47:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.314+0000 7fc396600640 1 -- 172.21.15.28:0/834625011 wait complete. 2024-09-10T10:47:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.315+0000 7fc396600640 1 Processor -- start 2024-09-10T10:47:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 -- start start 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 0x7fc39015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3900d2920 0x7fc3900b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3900cf610 con 0x7fc39015e550 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3900cf180 con 0x7fc3900b3400 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.316+0000 7fc396600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3900cf790 con 0x7fc3900d2920 2024-09-10T10:47:35.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36690/0 (socket says 172.21.15.28:36690) 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 -- 172.21.15.28:0/3945952902 learned_addr learned my addr 172.21.15.28:0/3945952902 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395200640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 0x7fc39015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc38fe00640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3900d2920 0x7fc3900b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 -- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3900d2920 msgr2=0x7fc3900b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3900d2920 0x7fc3900b7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 -- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 msgr2=0x7fc39015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 0x7fc39015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395c00640 1 -- 172.21.15.28:0/3945952902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3900b8b90 con 0x7fc39015e550 2024-09-10T10:47:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.317+0000 7fc395200640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 0x7fc39015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc395c00640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388036050 con 0x7fc39015e550 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388042070 con 0x7fc39015e550 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388007bd0 con 0x7fc39015e550 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3900b8ed0 con 0x7fc39015e550 2024-09-10T10:47:35.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.318+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3900b9710 con 0x7fc39015e550 2024-09-10T10:47:35.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.319+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3900d2d30 con 0x7fc39015e550 2024-09-10T10:47:35.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.320+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc388047460 con 0x7fc39015e550 2024-09-10T10:47:35.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.321+0000 7fc38cc00640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 0x7fc37006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.321+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6390+0+0 (secure 0 0 0) 0x7fc3880ac190 con 0x7fc39015e550 2024-09-10T10:47:35.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.322+0000 7fc395200640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 0x7fc37006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.323+0000 7fc395200640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 0x7fc37006e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc39015dc60 tx=0x7fc384009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.326+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3900b8ed0 con 0x7fc39015e550 2024-09-10T10:47:35.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.513+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7fc3900d18f0 con 0x7fc39015e550 2024-09-10T10:47:35.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.517+0000 7fc38cc00640 1 -- 172.21.15.28:0/3945952902 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fc38807a7f0 con 0x7fc39015e550 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 msgr2=0x7fc37006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 0x7fc37006e2f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc39015dc60 tx=0x7fc384009040 comp rx=0 tx=0).stop 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 msgr2=0x7fc3900b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc388009e50 tx=0x7fc388004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 shutdown_connections 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc37006be30 0x7fc37006e2f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc39015e550 0x7fc3900b8510 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3900d2920 0x7fc3900b7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 --2- 172.21.15.28:0/3945952902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900b3400 0x7fc39015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 >> 172.21.15.28:0/3945952902 conn(0x7fc39008bfc0 msgr2=0x7fc3900b66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 shutdown_connections 2024-09-10T10:47:35.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.519+0000 7fc396600640 1 -- 172.21.15.28:0/3945952902 wait complete. 2024-09-10T10:47:35.537 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 4:0 false 2024-09-10T10:47:35.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.695+0000 7fc4bb200640 1 Processor -- start 2024-09-10T10:47:35.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.695+0000 7fc4bb200640 1 -- start start 2024-09-10T10:47:35.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.696+0000 7fc4bb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc4b406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.696+0000 7fc4bb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4b406f250 0x7fc4b40b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.696+0000 7fc4bb200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4b4071cd0 con 0x7fc4b4158ba0 2024-09-10T10:47:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.696+0000 7fc4bb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b415b4b0 con 0x7fc4b406f250 2024-09-10T10:47:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.696+0000 7fc4bb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b415b710 con 0x7fc4b4073d10 2024-09-10T10:47:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc4b4158ba0 0x7fc4b40732d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38648/0 (socket says 172.21.15.28:38648) 2024-09-10T10:47:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b9e00640 1 -- 172.21.15.28:0/3016550504 learned_addr learned my addr 172.21.15.28:0/3016550504 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4ba800640 1 --2- 172.21.15.28:0/3016550504 >> v2:172.21.15.138:3300/0 conn(0x7fc4b406f250 0x7fc4b40b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b9400640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc4b406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612648039 0 0) 0x7fc4b4071cd0 con 0x7fc4b4158ba0 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc49c003520 con 0x7fc4b4158ba0 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3554902751 0 0) 0x7fc49c003520 con 0x7fc4b4158ba0 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 msgr2=0x7fc4b406ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc4b406ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 >> v2:172.21.15.138:3300/0 conn(0x7fc4b406f250 msgr2=0x7fc4b40b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 --2- 172.21.15.28:0/3016550504 >> v2:172.21.15.138:3300/0 conn(0x7fc4b406f250 0x7fc4b40b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b415b9f0 con 0x7fc4b4158ba0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fc4a8001670 con 0x7fc4b4158ba0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.697+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 >> v1:172.21.15.28:6789/0 conn(0x7fc4b4158ba0 legacy=0x7fc4b40732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0039b0 0x7fc49c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc49c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc49c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b415b4b0 con 0x7fc49c004800 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b415b710 con 0x7fc49c0039b0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc49c00b9c0 con 0x7fc4b4073d10 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc49c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4ba800640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc49c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9e00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0039b0 0x7fc49c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 -- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 msgr2=0x7fc49c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b4073d10 0x7fc49c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 -- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0039b0 msgr2=0x7fc49c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:35.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0039b0 0x7fc49c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9400640 1 -- 172.21.15.28:0/3016550504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c003520 con 0x7fc49c004800 2024-09-10T10:47:35.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.698+0000 7fc4b9e00640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c0039b0 0x7fc49c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.699+0000 7fc4b9400640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc49c004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009a90 tx=0x7fc49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d535ad9e19f8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4036070 con 0x7fc49c004800 2024-09-10T10:47:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a4031400 con 0x7fc49c004800 2024-09-10T10:47:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4b8a00640 1 -- 172.21.15.28:0/3016550504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4009690 con 0x7fc49c004800 2024-09-10T10:47:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 -- 172.21.15.28:0/3016550504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 msgr2=0x7fc49c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 --2- 172.21.15.28:0/3016550504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc49c004c80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009a90 tx=0x7fc49c005830 comp rx=0 tx=0).stop 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4ba800640 1 -- 172.21.15.28:0/3016550504 reap_dead start 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 -- 172.21.15.28:0/3016550504 shutdown_connections 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 -- 172.21.15.28:0/3016550504 >> 172.21.15.28:0/3016550504 conn(0x7fc4b406c240 msgr2=0x7fc4b40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 -- 172.21.15.28:0/3016550504 shutdown_connections 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.700+0000 7fc4bb200640 1 -- 172.21.15.28:0/3016550504 wait complete. 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.702+0000 7fc4bb200640 1 Processor -- start 2024-09-10T10:47:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.702+0000 7fc4bb200640 1 -- start start 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 0x7fc4b40cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 0x7fc4b406e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b40b3410 con 0x7fc49c004800 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b415bfb0 con 0x7fc4b414fba0 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4bb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b40b3590 con 0x7fc4b40cef70 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4b9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4b9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36700/0 (socket says 172.21.15.28:36700) 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4b9e00640 1 -- 172.21.15.28:0/4203591340 learned_addr learned my addr 172.21.15.28:0/4203591340 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4b9400640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 0x7fc4b40cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.703+0000 7fc4ba800640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 0x7fc4b406e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 -- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 msgr2=0x7fc4b40cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 0x7fc4b40cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 -- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 msgr2=0x7fc4b406e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 0x7fc4b406e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 -- 172.21.15.28:0/4203591340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b40cc860 con 0x7fc49c004800 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4ba800640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 0x7fc4b406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9400640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 0x7fc4b40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4b9e00640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8001ab0 tx=0x7fc4a800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800c660 con 0x7fc49c004800 2024-09-10T10:47:35.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a8018070 con 0x7fc49c004800 2024-09-10T10:47:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a800ca60 con 0x7fc49c004800 2024-09-10T10:47:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b40cf3f0 con 0x7fc49c004800 2024-09-10T10:47:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.704+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b40ccde0 con 0x7fc49c004800 2024-09-10T10:47:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.705+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b40b2ed0 con 0x7fc49c004800 2024-09-10T10:47:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.706+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4a800cc00 con 0x7fc49c004800 2024-09-10T10:47:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.707+0000 7fc4ae000640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 0x7fc49806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.707+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6390+0+0 (secure 0 0 0) 0x7fc4a808aec0 con 0x7fc49c004800 2024-09-10T10:47:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.708+0000 7fc4b9400640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 0x7fc49806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.709+0000 7fc4b9400640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 0x7fc49806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4033000 tx=0x7fc4a4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:35.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.711+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc4a80595b0 con 0x7fc49c004800 2024-09-10T10:47:35.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:35.963+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"} v 0) -- 0x7fc4b40afb70 con 0x7fc49c004800 2024-09-10T10:47:36.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.631+0000 7fc4ae000640 1 -- 172.21.15.28:0/4203591340 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "4:0", "val": "false"}]=0 unfreezing mds.5752 v57) ==== 115+0+0 (secure 0 0 0) 0x7fc4a8058eb0 con 0x7fc49c004800 2024-09-10T10:47:36.633 INFO:teuthology.orchestra.run.smithi028.stderr:unfreezing mds.5752 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.635+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 msgr2=0x7fc49806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.635+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 0x7fc49806e2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4033000 tx=0x7fc4a4008030 comp rx=0 tx=0).stop 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.635+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 msgr2=0x7fc4b415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.635+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8001ab0 tx=0x7fc4a800ec10 comp rx=0 tx=0).stop 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 shutdown_connections 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc49806be30 0x7fc49806e2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4b414fba0 0x7fc4b406e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4b40cef70 0x7fc4b40cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:36.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 --2- 172.21.15.28:0/4203591340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc49c004800 0x7fc4b415aca0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:36.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 >> 172.21.15.28:0/4203591340 conn(0x7fc4b406c240 msgr2=0x7fc4b40ae690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:36.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 shutdown_connections 2024-09-10T10:47:36.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:36.636+0000 7fc4bb200640 1 -- 172.21.15.28:0/4203591340 wait complete. 2024-09-10T10:47:36.660 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -n kernel.core_pattern 2024-09-10T10:47:36.717 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:47:36.720 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:47:36.720 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-10T10:47:36.784 INFO:teuthology.orchestra.run.smithi138.stdout:1725965232.42642.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/proc/self/exe', platform: 'x86_64' 2024-09-10T10:47:36.788 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1725965232.42642.core'] 2024-09-10T10:47:36.788 INFO:tasks.cephfs.cephfs_test_case:Found core file 1725965232.42642.core, deleting it 2024-09-10T10:47:36.788 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1725965232.42642.core 2024-09-10T10:47:36.852 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:47:36.852 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:47:36.852 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:47:36.852 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:47:36.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:47:36.853 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi138 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-09-10T10:47:36.853 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:47:36.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:47:36.857 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:47:36.858 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:36.983 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:47:37.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.023+0000 7ff78a600640 1 Processor -- start 2024-09-10T10:47:37.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.023+0000 7ff78a600640 1 -- start start 2024-09-10T10:47:37.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.024+0000 7ff78a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7840cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:37.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff78a600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff784146140 con 0x7ff784072de0 2024-09-10T10:47:37.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff78a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff78406eed0 con 0x7ff78415e1f0 2024-09-10T10:47:37.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff78a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840cd740 con 0x7ff784072320 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff77be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff784072de0 0x7ff78406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38658/0 (socket says 172.21.15.28:38658) 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff77be00640 1 -- 172.21.15.28:0/3506820009 learned_addr learned my addr 172.21.15.28:0/3506820009 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff783400640 1 --2- 172.21.15.28:0/3506820009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7840cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2650377660 0 0) 0x7ff784146140 con 0x7ff784072de0 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff768003520 con 0x7ff784072de0 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107088241 0 0) 0x7ff78406eed0 con 0x7ff78415e1f0 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff783400640 1 -- 172.21.15.28:0/3506820009 >> v1:172.21.15.138:6789/0 conn(0x7ff78415e1f0 legacy=0x7ff78406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff783400640 1 -- 172.21.15.28:0/3506820009 >> v1:172.21.15.28:6789/0 conn(0x7ff784072de0 legacy=0x7ff78406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.025+0000 7ff783400640 1 -- 172.21.15.28:0/3506820009 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7840cda20 con 0x7ff784072320 2024-09-10T10:47:37.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff783400640 1 --2- 172.21.15.28:0/3506820009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7840cce60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff7740097d0 tx=0x7ff77402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4a1b98458ed674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:37.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774037070 con 0x7ff784072320 2024-09-10T10:47:37.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff774032440 con 0x7ff784072320 2024-09-10T10:47:37.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff782a00640 1 -- 172.21.15.28:0/3506820009 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff774032810 con 0x7ff784072320 2024-09-10T10:47:37.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff78a600640 1 -- 172.21.15.28:0/3506820009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 msgr2=0x7ff7840cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.027+0000 7ff78a600640 1 --2- 172.21.15.28:0/3506820009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7840cce60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff7740097d0 tx=0x7ff77402a180 comp rx=0 tx=0).stop 2024-09-10T10:47:37.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.028+0000 7ff78a600640 1 -- 172.21.15.28:0/3506820009 shutdown_connections 2024-09-10T10:47:37.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.028+0000 7ff78a600640 1 --2- 172.21.15.28:0/3506820009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7840cce60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.028+0000 7ff78a600640 1 -- 172.21.15.28:0/3506820009 >> 172.21.15.28:0/3506820009 conn(0x7ff784067d70 msgr2=0x7ff78406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:37.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.028+0000 7ff78a600640 1 -- 172.21.15.28:0/3506820009 shutdown_connections 2024-09-10T10:47:37.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.028+0000 7ff78a600640 1 -- 172.21.15.28:0/3506820009 wait complete. 2024-09-10T10:47:37.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.029+0000 7ff78a600640 1 Processor -- start 2024-09-10T10:47:37.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.029+0000 7ff78a600640 1 -- start start 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7841518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784072de0 0x7ff784140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840cd3a0 con 0x7ff78415e1f0 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff78415c9e0 con 0x7ff784072de0 2024-09-10T10:47:37.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff78a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840cd520 con 0x7ff784072320 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.030+0000 7ff783e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36714/0 (socket says 172.21.15.28:36714) 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 -- 172.21.15.28:0/3154171978 learned_addr learned my addr 172.21.15.28:0/3154171978 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783400640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7841518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff77be00640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784072de0 0x7ff784140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 -- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 msgr2=0x7ff7841518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7841518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 -- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784072de0 msgr2=0x7ff784140ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784072de0 0x7ff784140ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 -- 172.21.15.28:0/3154171978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7841437f0 con 0x7ff78415e1f0 2024-09-10T10:47:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff783e00640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c730 tx=0x7ff77c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.031+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c003da0 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.032+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c00ce40 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.032+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0052b0 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.032+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff784143ad0 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.032+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff784144090 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.033+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff784072700 con 0x7ff78415e1f0 2024-09-10T10:47:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.034+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff77c005450 con 0x7ff78415e1f0 2024-09-10T10:47:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.035+0000 7ff77b400640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 0x7ff76006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.035+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7ff77c08aff0 con 0x7ff78415e1f0 2024-09-10T10:47:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.036+0000 7ff783400640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 0x7ff76006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.036+0000 7ff783400640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 0x7ff76006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff774034000 tx=0x7ff77402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.039+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff77c058fd0 con 0x7ff78415e1f0 2024-09-10T10:47:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.262+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7840cc5a0 con 0x7ff78415e1f0 2024-09-10T10:47:37.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.263+0000 7ff77b400640 1 -- 172.21.15.28:0/3154171978 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+4396 (secure 0 0 0) 0x7ff77c05d2d0 con 0x7ff78415e1f0 2024-09-10T10:47:37.265 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:37.265 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":57,"btime":"2024-09-10T10:47:36:630138+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:36.630132+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T10:47:37.265 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 57 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 msgr2=0x7ff76006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 0x7ff76006e2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff774034000 tx=0x7ff77402e000 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 msgr2=0x7ff78413e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff77c00c730 tx=0x7ff77c00cbf0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 shutdown_connections 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff76006be30 0x7ff76006e2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff78415e1f0 0x7ff78413e110 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff784072de0 0x7ff784140ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 --2- 172.21.15.28:0/3154171978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff784072320 0x7ff7841518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:37.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 >> 172.21.15.28:0/3154171978 conn(0x7ff784067d70 msgr2=0x7ff7841e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:37.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 shutdown_connections 2024-09-10T10:47:37.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:37.265+0000 7ff78a600640 1 -- 172.21.15.28:0/3154171978 wait complete. 2024-09-10T10:47:37.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:36.630132+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 74, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5572': {'gid': 5572, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2942680174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2942680174}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2942680174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:47:37.685 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5572:up:replay 2024-09-10T10:47:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.421+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_RECONNECTING pgs=15 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feacd600640 0 client.6147 ms_handle_remote_reset on v2:172.21.15.138:6835/2819175864 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:47:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.423+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:47:38.686 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:38.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.851+0000 7f34f4c00640 1 Processor -- start 2024-09-10T10:47:38.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.851+0000 7f34f4c00640 1 -- start start 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34f4c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34f0072320 0x7f34f00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34f4c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f0146140 con 0x7f34f0072de0 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34f4c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f006eed0 con 0x7f34f015e1f0 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34f4c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34f00cd740 con 0x7f34f0072320 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34e7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f34f0072de0 0x7f34f006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38670/0 (socket says 172.21.15.28:38670) 2024-09-10T10:47:38.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34e7e00640 1 -- 172.21.15.28:0/2671301505 learned_addr learned my addr 172.21.15.28:0/2671301505 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34ede00640 1 --2- 172.21.15.28:0/2671301505 >> v2:172.21.15.138:3300/0 conn(0x7f34f0072320 0x7f34f00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1275836133 0 0) 0x7f34f0146140 con 0x7f34f0072de0 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.853+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34d4003520 con 0x7f34f0072de0 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.854+0000 7f34ede00640 1 -- 172.21.15.28:0/2671301505 >> v1:172.21.15.138:6789/0 conn(0x7f34f015e1f0 legacy=0x7f34f006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.854+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784317840 0 0) 0x7f34f006eed0 con 0x7f34f015e1f0 2024-09-10T10:47:38.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.854+0000 7f34ede00640 1 -- 172.21.15.28:0/2671301505 >> v1:172.21.15.28:6789/0 conn(0x7f34f0072de0 legacy=0x7f34f006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.854+0000 7f34ede00640 1 -- 172.21.15.28:0/2671301505 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f00cda20 con 0x7f34f0072320 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.855+0000 7f34ede00640 1 --2- 172.21.15.28:0/2671301505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f00cce60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097a0 tx=0x7f34e002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e30b70427bc435c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.855+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0037070 con 0x7f34f0072320 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.855+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e0032440 con 0x7f34f0072320 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.855+0000 7f34ed400640 1 -- 172.21.15.28:0/2671301505 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0032810 con 0x7f34f0072320 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 -- 172.21.15.28:0/2671301505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 msgr2=0x7f34f00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2671301505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f00cce60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f34e00097a0 tx=0x7f34e002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 -- 172.21.15.28:0/2671301505 shutdown_connections 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2671301505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f00cce60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 -- 172.21.15.28:0/2671301505 >> 172.21.15.28:0/2671301505 conn(0x7f34f0067d70 msgr2=0x7f34f006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:38.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 -- 172.21.15.28:0/2671301505 shutdown_connections 2024-09-10T10:47:38.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.856+0000 7f34f4c00640 1 -- 172.21.15.28:0/2671301505 wait complete. 2024-09-10T10:47:38.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.857+0000 7f34f4c00640 1 Processor -- start 2024-09-10T10:47:38.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 -- start start 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 0x7f34f014b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f34f00cd3a0 con 0x7f34f0072de0 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34f015c9e0 con 0x7f34f0072320 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.858+0000 7f34f4c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34f00cd520 con 0x7f34f015e1f0 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36724/0 (socket says 172.21.15.28:36724) 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 -- 172.21.15.28:0/2215177346 learned_addr learned my addr 172.21.15.28:0/2215177346 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34ede00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34ee800640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 0x7f34f014b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 -- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 msgr2=0x7f34f014b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 0x7f34f014b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 -- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 msgr2=0x7f34f013d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f013d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 -- 172.21.15.28:0/2215177346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f0140180 con 0x7f34f0072de0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34ee800640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 0x7f34f014b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.859+0000 7f34e7e00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c500 tx=0x7f34d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.860+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8017070 con 0x7f34f0072de0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.860+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d8005ce0 con 0x7f34f0072de0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.860+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8007330 con 0x7f34f0072de0 2024-09-10T10:47:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.860+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f0140460 con 0x7f34f0072de0 2024-09-10T10:47:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.860+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f015bc20 con 0x7f34f0072de0 2024-09-10T10:47:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.861+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f0149390 con 0x7f34f0072de0 2024-09-10T10:47:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.862+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34d801a070 con 0x7f34f0072de0 2024-09-10T10:47:38.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.863+0000 7f34e6000640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 0x7f34cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:38.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.863+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f34d808a510 con 0x7f34f0072de0 2024-09-10T10:47:38.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.864+0000 7f34ede00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 0x7f34cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:38.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.865+0000 7f34ede00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 0x7f34cc06e320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f34e0034000 tx=0x7f34e002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:38.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:38.868+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34d8058620 con 0x7f34f0072de0 2024-09-10T10:47:39.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.086+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f01455f0 con 0x7f34f0072de0 2024-09-10T10:47:39.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.087+0000 7f34e6000640 1 -- 172.21.15.28:0/2215177346 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5222 (secure 0 0 0) 0x7f34d805c920 con 0x7f34f0072de0 2024-09-10T10:47:39.089 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:39.089 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":59,"btime":"2024-09-10T10:47:38:648794+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:37.654830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:rejoin","state_seq":17,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":4}]} 2024-09-10T10:47:39.090 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 59 2024-09-10T10:47:39.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 msgr2=0x7f34cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:39.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 0x7f34cc06e320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f34e0034000 tx=0x7f34e002e000 comp rx=0 tx=0).stop 2024-09-10T10:47:39.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 msgr2=0x7f34f013f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f34d800c500 tx=0x7f34d800c9c0 comp rx=0 tx=0).stop 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 shutdown_connections 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34cc06be60 0x7f34cc06e320 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34f015e1f0 0x7f34f014b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34f0072de0 0x7f34f013f6c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 --2- 172.21.15.28:0/2215177346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34f0072320 0x7f34f013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 >> 172.21.15.28:0/2215177346 conn(0x7f34f0067d70 msgr2=0x7f34f0143770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 shutdown_connections 2024-09-10T10:47:39.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:39.089+0000 7f34f4c00640 1 -- 172.21.15.28:0/2215177346 wait complete. 2024-09-10T10:47:39.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:47:39.249 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:47:39.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:37.654830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 74, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5572': {'gid': 5572, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:rejoin', 'state_seq': 17, 'addr': '172.21.15.28:6835/2942680174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2942680174}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2942680174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:47:39.509 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5572:up:rejoin 2024-09-10T10:47:39.771 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T10:47:40.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:47:40.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.677+0000 7fdf80400640 1 Processor -- start 2024-09-10T10:47:40.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.677+0000 7fdf80400640 1 -- start start 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.678+0000 7fdf80400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 0x7fdf780cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.678+0000 7fdf80400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 0x7fdf7806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.678+0000 7fdf80400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf78146140 con 0x7fdf78072de0 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.678+0000 7fdf80400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdf780cd740 con 0x7fdf78072320 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.678+0000 7fdf80400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf780cda20 con 0x7fdf7815e1f0 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdf78072de0 0x7fdf7806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38686/0 (socket says 172.21.15.28:38686) 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7d000640 1 -- 172.21.15.28:0/1079948990 learned_addr learned my addr 172.21.15.28:0/1079948990 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7da00640 1 --2- 172.21.15.28:0/1079948990 >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 0x7fdf780cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7e400640 1 --2- 172.21.15.28:0/1079948990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 0x7fdf7806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf77e00640 1 -- 172.21.15.28:0/1079948990 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219968878 0 0) 0x7fdf78146140 con 0x7fdf78072de0 2024-09-10T10:47:40.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf77e00640 1 -- 172.21.15.28:0/1079948990 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf60003520 con 0x7fdf78072de0 2024-09-10T10:47:40.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7e400640 1 -- 172.21.15.28:0/1079948990 >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 msgr2=0x7fdf780cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:40.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7e400640 1 --2- 172.21.15.28:0/1079948990 >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 0x7fdf780cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7e400640 1 -- 172.21.15.28:0/1079948990 >> v1:172.21.15.28:6789/0 conn(0x7fdf78072de0 legacy=0x7fdf7806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7e400640 1 -- 172.21.15.28:0/1079948990 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf780cdd00 con 0x7fdf7815e1f0 2024-09-10T10:47:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.679+0000 7fdf7da00640 1 --2- 172.21.15.28:0/1079948990 >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 0x7fdf780cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:47:40.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf7e400640 1 --2- 172.21.15.28:0/1079948990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 0x7fdf7806e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d90 tx=0x7fdf7002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8870700979d796be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf77e00640 1 -- 172.21.15.28:0/1079948990 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70031600 con 0x7fdf7815e1f0 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf77e00640 1 -- 172.21.15.28:0/1079948990 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf700317a0 con 0x7fdf7815e1f0 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf77e00640 1 -- 172.21.15.28:0/1079948990 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70031b50 con 0x7fdf7815e1f0 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf80400640 1 -- 172.21.15.28:0/1079948990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 msgr2=0x7fdf7806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.681+0000 7fdf80400640 1 --2- 172.21.15.28:0/1079948990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 0x7fdf7806e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009d90 tx=0x7fdf7002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 -- 172.21.15.28:0/1079948990 shutdown_connections 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 --2- 172.21.15.28:0/1079948990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf7815e1f0 0x7fdf7806e7c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 --2- 172.21.15.28:0/1079948990 >> v2:172.21.15.138:3300/0 conn(0x7fdf78072320 0x7fdf780cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 -- 172.21.15.28:0/1079948990 >> 172.21.15.28:0/1079948990 conn(0x7fdf78067d70 msgr2=0x7fdf7806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 -- 172.21.15.28:0/1079948990 shutdown_connections 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.682+0000 7fdf80400640 1 -- 172.21.15.28:0/1079948990 wait complete. 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.683+0000 7fdf80400640 1 Processor -- start 2024-09-10T10:47:40.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.683+0000 7fdf80400640 1 -- start start 2024-09-10T10:47:40.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf78072320 0x7fdf781316c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf78072de0 0x7fdf780c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf780cd520 con 0x7fdf7815e1f0 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf780cd3a0 con 0x7fdf78072320 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf80400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7806ed00 con 0x7fdf78072de0 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.684+0000 7fdf7e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36730/0 (socket says 172.21.15.28:36730) 2024-09-10T10:47:40.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 -- 172.21.15.28:0/20441437 learned_addr learned my addr 172.21.15.28:0/20441437 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:40.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7da00640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf78072320 0x7fdf781316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7d000640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf78072de0 0x7fdf780c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 -- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf78072de0 msgr2=0x7fdf780c8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf78072de0 0x7fdf780c8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 -- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf78072320 msgr2=0x7fdf781316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf78072320 0x7fdf781316c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 -- 172.21.15.28:0/20441437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf78130620 con 0x7fdf7815e1f0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.685+0000 7fdf7e400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009eb0 tx=0x7fdf70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.686+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009450 con 0x7fdf7815e1f0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.686+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70031b50 con 0x7fdf7815e1f0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.686+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf70003e10 con 0x7fdf7815e1f0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.686+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf78130900 con 0x7fdf7815e1f0 2024-09-10T10:47:40.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.686+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf78130ec0 con 0x7fdf7815e1f0 2024-09-10T10:47:40.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.687+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf78072700 con 0x7fdf7815e1f0 2024-09-10T10:47:40.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.688+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf700330d0 con 0x7fdf7815e1f0 2024-09-10T10:47:40.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.689+0000 7fdf75600640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 0x7fdf5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:40.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.689+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7fdf700abef0 con 0x7fdf7815e1f0 2024-09-10T10:47:40.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.689+0000 7fdf7da00640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 0x7fdf5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:40.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.690+0000 7fdf7da00640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 0x7fdf5806e2f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009560 tx=0x7fdf6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:40.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.693+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf7003f030 con 0x7fdf7815e1f0 2024-09-10T10:47:40.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.919+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf78145740 con 0x7fdf7815e1f0 2024-09-10T10:47:40.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.919+0000 7fdf75600640 1 -- 172.21.15.28:0/20441437 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5229 (secure 0 0 0) 0x7fdf7007a5c0 con 0x7fdf7815e1f0 2024-09-10T10:47:40.925 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:47:40.925 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":60,"btime":"2024-09-10T10:47:39:651000+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:39.650991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":18,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5572,"qdb_cluster":[5572]},"id":4}]} 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 60 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 msgr2=0x7fdf5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 0x7fdf5806e2f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009560 tx=0x7fdf6c008040 comp rx=0 tx=0).stop 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 msgr2=0x7fdf7812fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009eb0 tx=0x7fdf70004a40 comp rx=0 tx=0).stop 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 shutdown_connections 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5806be30 0x7fdf5806e2f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7815e1f0 0x7fdf7812fcb0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf78072de0 0x7fdf780c8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 --2- 172.21.15.28:0/20441437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf78072320 0x7fdf781316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:40.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 >> 172.21.15.28:0/20441437 conn(0x7fdf78067d70 msgr2=0x7fdf781449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:40.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 shutdown_connections 2024-09-10T10:47:40.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:40.922+0000 7fdf80400640 1 -- 172.21.15.28:0/20441437 wait complete. 2024-09-10T10:47:41.343 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:46:44.291411+0000', 'modified': '2024-09-10T10:47:39.650991+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 74, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5572': {'gid': 5572, 'name': 'a', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.28:6835/2942680174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2942680174}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2942680174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5572, 'qdb_cluster': [5572]} 2024-09-10T10:47:41.343 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:47:41.344 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:47:41.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.502+0000 7ff83ac00640 1 Processor -- start 2024-09-10T10:47:41.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.502+0000 7ff83ac00640 1 -- start start 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff83ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff834147230 0x7ff83406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff83ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff8340aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff83ac00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83406f480 con 0x7ff834147e90 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff83ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff8340b34d0 con 0x7ff834147230 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff83ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8340ae9e0 con 0x7ff8340727c0 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff833e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff834147230 0x7ff83406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff82b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff834147e90 0x7ff8340720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38696/0 (socket says 172.21.15.28:38696) 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff82b400640 1 -- 172.21.15.28:0/1073164997 learned_addr learned my addr 172.21.15.28:0/1073164997 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff833e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff834147230 0x7ff83406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36742/0 (socket says 172.21.15.28:36742) 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.504+0000 7ff838c00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff8340aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff833e00640 1 -- 172.21.15.28:0/1073164997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 msgr2=0x7ff8340aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff833e00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff8340aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff833e00640 1 -- 172.21.15.28:0/1073164997 >> v1:172.21.15.28:6789/0 conn(0x7ff834147e90 legacy=0x7ff8340720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff833e00640 1 -- 172.21.15.28:0/1073164997 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83406e900 con 0x7ff834147230 2024-09-10T10:47:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff838c00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff8340aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.505+0000 7ff833400640 1 -- 172.21.15.28:0/1073164997 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3803997936 0 0) 0x7ff83406f480 con 0x7ff834147e90 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff833e00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff83406e0b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8240097a0 tx=0x7ff82402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e296840905168029 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff833400640 1 -- 172.21.15.28:0/1073164997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824037070 con 0x7ff834147230 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff833400640 1 -- 172.21.15.28:0/1073164997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824032440 con 0x7ff834147230 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff833400640 1 -- 172.21.15.28:0/1073164997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824032810 con 0x7ff834147230 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 -- 172.21.15.28:0/1073164997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 msgr2=0x7ff83406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff83406e0b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff8240097a0 tx=0x7ff82402a180 comp rx=0 tx=0).stop 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 -- 172.21.15.28:0/1073164997 shutdown_connections 2024-09-10T10:47:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff8340aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1073164997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff83406e0b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.506+0000 7ff83ac00640 1 -- 172.21.15.28:0/1073164997 >> 172.21.15.28:0/1073164997 conn(0x7ff834067d70 msgr2=0x7ff83406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.507+0000 7ff83ac00640 1 -- 172.21.15.28:0/1073164997 shutdown_connections 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.507+0000 7ff83ac00640 1 -- 172.21.15.28:0/1073164997 wait complete. 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.508+0000 7ff83ac00640 1 Processor -- start 2024-09-10T10:47:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.508+0000 7ff83ac00640 1 -- start start 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff83415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff834147e90 0x7ff834146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff83406f1a0 con 0x7ff834147230 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff83406eec0 con 0x7ff834147e90 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff83ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff83406e5f0 con 0x7ff8340727c0 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36748/0 (socket says 172.21.15.28:36748) 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.509+0000 7ff82b400640 1 -- 172.21.15.28:0/1129939253 learned_addr learned my addr 172.21.15.28:0/1129939253 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 -- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 msgr2=0x7ff83415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff83415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 -- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff834147e90 msgr2=0x7ff834146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff834147e90 0x7ff834146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 -- 172.21.15.28:0/1129939253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff834144800 con 0x7ff834147230 2024-09-10T10:47:41.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff82b400640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c430 tx=0x7ff81c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff830c00640 1 -- 172.21.15.28:0/1129939253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0074e0 con 0x7ff834147230 2024-09-10T10:47:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff830c00640 1 -- 172.21.15.28:0/1129939253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c0047e0 con 0x7ff834147230 2024-09-10T10:47:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff830c00640 1 -- 172.21.15.28:0/1129939253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004bd0 con 0x7ff834147230 2024-09-10T10:47:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff834144ae0 con 0x7ff834147230 2024-09-10T10:47:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.510+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff834145380 con 0x7ff834147230 2024-09-10T10:47:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.512+0000 7ff830c00640 1 -- 172.21.15.28:0/1129939253 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff81c011070 con 0x7ff834147230 2024-09-10T10:47:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.513+0000 7ff830c00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 0x7ff81006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.513+0000 7ff830c00640 1 -- 172.21.15.28:0/1129939253 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7ff81c08af30 con 0x7ff834147230 2024-09-10T10:47:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.514+0000 7ff83d9c2740 1 Processor -- start 2024-09-10T10:47:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.514+0000 7ff83d9c2740 1 -- start start 2024-09-10T10:47:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.514+0000 7ff833e00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 0x7ff81006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff833e00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 0x7ff81006e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff824034000 tx=0x7ff82402c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f78c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f6f03b0 con 0x55cf3f78f4b0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f6f5410 con 0x55cf3f793480 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cf3f69d620 con 0x55cf3f791cf0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36754/0 (socket says 172.21.15.28:36754) 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 -- 172.21.15.28:0/1947969332 learned_addr learned my addr 172.21.15.28:0/1947969332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff833e00640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff838c00640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f78c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 -- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 msgr2=0x55cf3f7d7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7d7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 -- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 msgr2=0x55cf3f78c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f78c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff82b400640 1 -- 172.21.15.28:0/1947969332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3f4082d0 con 0x55cf3f78f4b0 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.515+0000 7ff833e00640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff82b400640 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff81c089000 tx=0x7ff81c059570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a6076a85ca597c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff823e00640 1 -- 172.21.15.28:0/1947969332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c061a20 con 0x55cf3f78f4b0 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff823e00640 1 -- 172.21.15.28:0/1947969332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c087070 con 0x55cf3f78f4b0 2024-09-10T10:47:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff823e00640 1 -- 172.21.15.28:0/1947969332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c004d70 con 0x55cf3f78f4b0 2024-09-10T10:47:41.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1947969332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 msgr2=0x55cf3f7920d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff81c089000 tx=0x7ff81c059570 comp rx=0 tx=0).stop 2024-09-10T10:47:41.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1947969332 shutdown_connections 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f78c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f78f4b0 0x55cf3f7920d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1947969332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.516+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1947969332 >> 172.21.15.28:0/1947969332 conn(0x55cf3f78e740 msgr2=0x55cf3f7d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.517+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1947969332 shutdown_connections 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.517+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1947969332 wait complete. 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.517+0000 7ff83d9c2740 1 -- start start 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.517+0000 7ff83d9c2740 1 Processor -- start 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f793480 0x55cf3f7bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7905e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f6b0780 con 0x55cf3f791cf0 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f6f13d0 con 0x55cf3f7b0d20 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cf3f78dbe0 con 0x55cf3f793480 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.518+0000 7ff83d9c2740 10 client.0 _ll_register_callbacks cb 0x55cf3f6ee5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36762/0 (socket says 172.21.15.28:36762) 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 -- 172.21.15.28:0/1249320358 learned_addr learned my addr 172.21.15.28:0/1249320358 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 -- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f793480 msgr2=0x55cf3f7bafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f793480 0x55cf3f7bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 -- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 msgr2=0x55cf3f7905e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7905e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 -- 172.21.15.28:0/1249320358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3f4d26b0 con 0x55cf3f791cf0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff833e00640 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff8240098c0 tx=0x7ff82402c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ff938f7c636a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff820c00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.519+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824037040 con 0x55cf3f791cf0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7ff82403d640 con 0x55cf3f791cf0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81c087070 con 0x55cf3f791cf0 2024-09-10T10:47:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff82403d840 con 0x55cf3f791cf0 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 10 client.6231 fetch_fsmap learned FSMap version 60 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55cf3f7bb580 con 0x55cf3f791cf0 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c087070 con 0x55cf3f791cf0 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7ff82403a030 con 0x55cf3f791cf0 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 10 client.6231 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 10 client.6231 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 20 client.6231 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7afdb0 0x55cf3f7b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 4 client.6231 mds_command: new command op to 5572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.520+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 --> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf3f702b40 con 0x55cf3f7afdb0 2024-09-10T10:47:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.521+0000 7ff82b400640 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7afdb0 0x55cf3f7b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.521+0000 7ff82b400640 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7afdb0 0x55cf3f7b2ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5358bc368235d3 server_cookie=f9cabdd2f3c1aa5a in_seq=0 out_seq=0 2024-09-10T10:47:41.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.521+0000 7ff820c00640 10 client.6231 ms_handle_connect on v2:172.21.15.28:6833/2942680174 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff820c00640 1 -- 172.21.15.28:0/1249320358 <== mds.0 v2:172.21.15.28:6833/2942680174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf3f702b40 con 0x55cf3f7afdb0 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff820c00640 10 client.6231 handle_command_reply: tid=0 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 client.6231 shutdown 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 msgr2=0x55cf3f7b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff8240098c0 tx=0x7ff82402c070 comp rx=0 tx=0).stop 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 shutdown_connections 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7905e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7afdb0 0x55cf3f7b2ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f793480 0x55cf3f7bafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1249320358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f791cf0 0x55cf3f7b1220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 >> 172.21.15.28:0/1249320358 conn(0x55cf3f78f4b0 msgr2=0x55cf3f7d49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff820c00640 0 client.6231 ms_handle_reset on v2:172.21.15.28:6833/2942680174 2024-09-10T10:47:41.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 shutdown_connections 2024-09-10T10:47:41.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.522+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1249320358 wait complete. 2024-09-10T10:47:41.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.523+0000 7ff83d9c2740 20 client.6231 trim_cache size 0 max 16384 2024-09-10T10:47:41.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.541+0000 7ff83d9c2740 1 Processor -- start 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.541+0000 7ff83d9c2740 1 -- start start 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f78ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f7d4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8342004c0 con 0x55cf3f7b0d20 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8341ff820 con 0x55cf3f793480 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8341ffe40 con 0x55cf3f791cf0 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff838c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff838c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36774/0 (socket says 172.21.15.28:36774) 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.542+0000 7ff838c00640 1 -- 172.21.15.28:0/2385008350 learned_addr learned my addr 172.21.15.28:0/2385008350 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff82b400640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f7d4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff833e00640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f78ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 -- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 msgr2=0x55cf3f78ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f78ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 -- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 msgr2=0x55cf3f7d4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f7d4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 -- 172.21.15.28:0/2385008350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3f700a20 con 0x55cf3f7b0d20 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff833e00640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f78ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:47:41.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff838c00640 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009b20 tx=0x7ff82c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6da1987d61f7699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff823400640 1 -- 172.21.15.28:0/2385008350 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c014610 con 0x55cf3f7b0d20 2024-09-10T10:47:41.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff823400640 1 -- 172.21.15.28:0/2385008350 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c019070 con 0x55cf3f7b0d20 2024-09-10T10:47:41.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.543+0000 7ff823400640 1 -- 172.21.15.28:0/2385008350 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0149f0 con 0x55cf3f7b0d20 2024-09-10T10:47:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 -- 172.21.15.28:0/2385008350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 msgr2=0x55cf3f7adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009b20 tx=0x7ff82c004040 comp rx=0 tx=0).stop 2024-09-10T10:47:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 -- 172.21.15.28:0/2385008350 shutdown_connections 2024-09-10T10:47:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7adb20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f793480 0x55cf3f7d4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/2385008350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f78ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 -- 172.21.15.28:0/2385008350 >> 172.21.15.28:0/2385008350 conn(0x55cf3f78f4b0 msgr2=0x55cf3f7e6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 -- 172.21.15.28:0/2385008350 shutdown_connections 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.544+0000 7ff83d9c2740 1 -- 172.21.15.28:0/2385008350 wait complete. 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.545+0000 7ff83d9c2740 1 -- start start 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.545+0000 7ff83d9c2740 1 Processor -- start 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f78c5c0 con 0x55cf3f793480 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf3f78c170 con 0x55cf3f7b0d20 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cf3f7b0270 con 0x55cf3f791cf0 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff83d9c2740 10 client.0 _ll_register_callbacks cb 0x7ff824035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:47:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36788/0 (socket says 172.21.15.28:36788) 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 -- 172.21.15.28:0/1540117907 learned_addr learned my addr 172.21.15.28:0/1540117907 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff838c00640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff833e00640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 -- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 msgr2=0x55cf3f7902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 -- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 msgr2=0x55cf3f7fa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7fa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.546+0000 7ff82b400640 1 -- 172.21.15.28:0/1540117907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf3f7b8de0 con 0x55cf3f793480 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff82b400640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff81c058f70 tx=0x7ff81c06f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8061612f70e373d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff820c00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c061a20 con 0x55cf3f793480 2024-09-10T10:47:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7ff81c0570b0 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82c019070 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7ff81c066050 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff83d9c2740 10 client.6237 fetch_fsmap learned FSMap version 60 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x55cf3f7907f0 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.547+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c019070 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7ff81c069020 con 0x55cf3f793480 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 10 client.6237 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 10 client.6237 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 20 client.6237 populate_metadata read hostname 'smithi028' 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7fa9b0 0x55cf3f7e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:47:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 4 client.6237 mds_command: new command op to 5572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 --> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] -- command(tid 0: {"prefix": "status"}) -- 0x55cf3f6fce90 con 0x55cf3f7fa9b0 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff833e00640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7fa9b0 0x55cf3f7e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff833e00640 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7fa9b0 0x55cf3f7e8b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1404d8aa4470c8a server_cookie=d6ea620d0ff06bb2 in_seq=0 out_seq=0 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.548+0000 7ff820c00640 10 client.6237 ms_handle_connect on v2:172.21.15.28:6833/2942680174 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff820c00640 1 -- 172.21.15.28:0/1540117907 <== mds.0 v2:172.21.15.28:6833/2942680174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55cf3f6fce90 con 0x55cf3f7fa9b0 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff820c00640 10 client.6237 handle_command_reply: tid=0 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 client.6237 shutdown 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 msgr2=0x55cf3f858150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff81c058f70 tx=0x7ff81c06f000 comp rx=0 tx=0).stop 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 shutdown_connections 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf3f7b0d20 0x55cf3f7fa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf3f793480 0x55cf3f858150 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x55cf3f7fa9b0 0x55cf3f7e8b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 --2- 172.21.15.28:0/1540117907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf3f791cf0 0x55cf3f7902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff820c00640 0 client.6237 ms_handle_reset on v2:172.21.15.28:6833/2942680174 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 >> 172.21.15.28:0/1540117907 conn(0x55cf3f78f4b0 msgr2=0x55cf3f7920d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 shutdown_connections 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 1 -- 172.21.15.28:0/1540117907 wait complete. 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.549+0000 7ff83d9c2740 20 client.6237 trim_cache size 0 max 16384 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:47:41.558 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5572, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 4.9166434040000002, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 60, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 74, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 74, 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 61.37348703 2024-09-10T10:47:41.559 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:47:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 msgr2=0x7ff81006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 0x7ff81006e2f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff824034000 tx=0x7ff82402c000 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 msgr2=0x7ff834140840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00c430 tx=0x7ff81c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 shutdown_connections 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff81006be30 0x7ff81006e2f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff834147e90 0x7ff834146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff834147230 0x7ff834140840 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 --2- 172.21.15.28:0/1129939253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff8340727c0 0x7ff83415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 >> 172.21.15.28:0/1129939253 conn(0x7ff834067d70 msgr2=0x7ff8340720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:47:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 shutdown_connections 2024-09-10T10:47:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:41.550+0000 7ff83ac00640 1 -- 172.21.15.28:0/1129939253 wait complete. 2024-09-10T10:47:41.849 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:47:41.847+0000 7f7b3fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:47:53.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:53.432+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:47:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:53.433+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:47:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:53.433+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:47:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:53.433+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:47:53.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:47:53.433+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:48:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:08.437+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:08.438+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:48:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:08.438+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:48:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:08.438+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:48:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:08.438+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:48:09.342 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:48:09.348 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:48:23.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:23.445+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:23.446+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:23.446+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:23.446+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:48:23.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:23.446+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:48:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:38.449+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:38.450+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:48:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:38.450+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:48:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:38.450+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:48:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:38.450+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:48:39.461 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:48:39.467 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:48:53.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:53.457+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:48:53.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:53.458+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:48:53.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:53.458+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:48:53.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:53.458+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:48:53.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:48:53.458+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:49:08.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:08.461+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:08.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:08.462+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2024-09-10T10:49:08.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:08.462+0000 7feaecc00640 1 -- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 msgr2=0x5562e2001fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-10T10:49:08.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:08.462+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 crc :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T10:49:08.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:08.462+0000 7feaecc00640 1 --2- 172.21.15.28:0/1694598718 >> [v2:172.21.15.138:6835/2819175864,v1:172.21.15.138:6837/2819175864] conn(0x5562e204c110 0x5562e2001fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 15.000000 2024-09-10T10:49:09.554 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:49:09.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:49:12.762 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-09-10T10:49:12.763 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:49:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.927+0000 7f54b8600640 1 Processor -- start 2024-09-10T10:49:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.927+0000 7f54b8600640 1 -- start start 2024-09-10T10:49:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.928+0000 7f54b8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 0x7f54b015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.928+0000 7f54b8600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b0146140 con 0x7f54b0072320 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54b8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b00cd740 con 0x7f54b0072de0 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54b8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54b00cda20 con 0x7f54b0072700 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54b5c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54b0072320 0x7f54b00cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59328/0 (socket says 172.21.15.28:59328) 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54b5c00640 1 -- 172.21.15.28:0/1960172300 learned_addr learned my addr 172.21.15.28:0/1960172300 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54b6600640 1 --2- 172.21.15.28:0/1960172300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 0x7f54b015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509888547 0 0) 0x7f54b0146140 con 0x7f54b0072320 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.929+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5498003520 con 0x7f54b0072320 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.930+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1116101689 0 0) 0x7f54b00cd740 con 0x7f54b0072de0 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.930+0000 7f54b6600640 1 -- 172.21.15.28:0/1960172300 >> v1:172.21.15.138:6789/0 conn(0x7f54b0072de0 legacy=0x7f54b015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.930+0000 7f54b6600640 1 -- 172.21.15.28:0/1960172300 >> v1:172.21.15.28:6789/0 conn(0x7f54b0072320 legacy=0x7f54b00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.930+0000 7f54b6600640 1 -- 172.21.15.28:0/1960172300 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b00cdd00 con 0x7f54b0072700 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.931+0000 7f54b6600640 1 --2- 172.21.15.28:0/1960172300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 0x7f54b015d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009dc0 tx=0x7f54a802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd66aa92491aad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.931+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8035020 con 0x7f54b0072700 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.931+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a8007540 con 0x7f54b0072700 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.931+0000 7f54afe00640 1 -- 172.21.15.28:0/1960172300 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40015f0 con 0x7f54b0072700 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 -- 172.21.15.28:0/1960172300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 msgr2=0x7f54b015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 --2- 172.21.15.28:0/1960172300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 0x7f54b015d8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009dc0 tx=0x7f54a802a5b0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 -- 172.21.15.28:0/1960172300 shutdown_connections 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 --2- 172.21.15.28:0/1960172300 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072700 0x7f54b015d8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 -- 172.21.15.28:0/1960172300 >> 172.21.15.28:0/1960172300 conn(0x7f54b0067d70 msgr2=0x7f54b006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:12.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 -- 172.21.15.28:0/1960172300 shutdown_connections 2024-09-10T10:49:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.932+0000 7f54b8600640 1 -- 172.21.15.28:0/1960172300 wait complete. 2024-09-10T10:49:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.933+0000 7f54b8600640 1 Processor -- start 2024-09-10T10:49:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.933+0000 7f54b8600640 1 -- start start 2024-09-10T10:49:12.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b0072700 0x7f54b01e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072de0 0x7f54b015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54b015dfb0 con 0x7f54b0072320 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54b015de30 con 0x7f54b0072700 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54b015e130 con 0x7f54b0072de0 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50398/0 (socket says 172.21.15.28:50398) 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 -- 172.21.15.28:0/1596065922 learned_addr learned my addr 172.21.15.28:0/1596065922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5200640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b0072700 0x7f54b01e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b6600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072de0 0x7f54b015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 -- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072de0 msgr2=0x7f54b015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072de0 0x7f54b015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 -- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b0072700 msgr2=0x7f54b01e61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:12.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b0072700 0x7f54b01e61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.935+0000 7f54b5c00640 1 -- 172.21.15.28:0/1596065922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b01e4d10 con 0x7f54b0072320 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54b5c00640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c780 tx=0x7f54a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4018070 con 0x7f54b0072320 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a40074e0 con 0x7f54b0072320 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40049e0 con 0x7f54b0072320 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01e3730 con 0x7f54b0072320 2024-09-10T10:49:12.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.936+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01e3cf0 con 0x7f54b0072320 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.937+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b00cce60 con 0x7f54b0072320 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.938+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54a4004b80 con 0x7f54b0072320 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.939+0000 7f54ad600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.939+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f54a408a600 con 0x7f54b0072320 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.939+0000 7f54b5200640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.940+0000 7f54b5200640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b01f5110 tx=0x7f549c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:12.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:12.943+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54a4058650 con 0x7f54b0072320 2024-09-10T10:49:13.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.165+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f54b006d090 con 0x7f54b0072320 2024-09-10T10:49:13.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.165+0000 7f54ad600640 1 -- 172.21.15.28:0/1596065922 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f54a405c950 con 0x7f54b0072320 2024-09-10T10:49:13.166 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.167+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 msgr2=0x7f549006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.167+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b01f5110 tx=0x7f549c009040 comp rx=0 tx=0).stop 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.167+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 msgr2=0x7f54b01e47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.167+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f54a400c780 tx=0x7f54a400cc40 comp rx=0 tx=0).stop 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.167+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 shutdown_connections 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b0072de0 0x7f54b015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b0072700 0x7f54b01e61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 --2- 172.21.15.28:0/1596065922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b0072320 0x7f54b01e47d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:13.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 >> 172.21.15.28:0/1596065922 conn(0x7f54b0067d70 msgr2=0x7f54b01449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:13.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 shutdown_connections 2024-09-10T10:49:13.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13.168+0000 7f54b8600640 1 -- 172.21.15.28:0/1596065922 wait complete. 2024-09-10T10:49:13.184 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:49:13.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:49:13.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:49:13.185 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:13.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.185 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:49:13.221 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:49:13.221 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:49:13.221 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:49:13.221 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:49:13.222 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42352sec preferred_lft 42352sec 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:49:13.223 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:13.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.225 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:49:13.314 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:49:13.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.317 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:49:13.353 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:49:13.353 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:49:13.353 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:49:13.353 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:49:13.353 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:49:13.354 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:49:13.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:49:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:13.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.551 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:49:13.586 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:49:13.586 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:49:13.587 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:13.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.587 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:49:13.683 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:49:13.683 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:13.719 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:13.719 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:49:13.719 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:13.794 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:49:13.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:13.922 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:13.923 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:13.923 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:13.938 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:13.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:13.938 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:13.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:49:13.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:49:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:49:13.962 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:49:13.963 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:49:13.963 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:49:13.963 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:49:13.963 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:49:13.965 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:49:13.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:13.966 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:14.021 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:49:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:49:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:14.110 INFO:teuthology.orchestra.run.smithi028.stdout:client5911 2024-09-10T10:49:14.110 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:14.110 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:14.146 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-10T10:49:14.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: File: a/ 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 2ah/42d Inode: 1099511627776 Links: 4 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1333/ ubuntu) 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:46:57.222135804 +0000 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:46:57.228135641 +0000 2024-09-10T10:49:14.268 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:46:57.228135641 +0000 2024-09-10T10:49:14.269 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:46:57.222135804 +0000 2024-09-10T10:49:14.274 INFO:teuthology.orchestra.run.smithi028.stdout:a/ 2024-09-10T10:49:14.274 INFO:teuthology.orchestra.run.smithi028.stdout:a/c 2024-09-10T10:49:14.274 INFO:teuthology.orchestra.run.smithi028.stdout:a/c/d 2024-09-10T10:49:14.275 INFO:teuthology.orchestra.run.smithi028.stdout:a/b 2024-09-10T10:49:14.275 INFO:teuthology.orchestra.run.smithi028.stdout:a/b/d 2024-09-10T10:49:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:14.283 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-10T10:49:14.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.440+0000 7f8823a00640 1 Processor -- start 2024-09-10T10:49:14.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.440+0000 7f8823a00640 1 -- start start 2024-09-10T10:49:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.441+0000 7f8823a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c15c310 0x7f881c160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.441+0000 7f8823a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f881c0b4800 0x7f881c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f8823a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c160e70 con 0x7f881c0d0370 2024-09-10T10:49:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f8823a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f881c0af7a0 con 0x7f881c0b4800 2024-09-10T10:49:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f8823a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f881c0afa80 con 0x7f881c15c310 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f881c0b4800 0x7f881c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f881c0b4800 0x7f881c0aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54422/0 (socket says 172.21.15.28:54422) 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 -- 172.21.15.28:0/2915926118 learned_addr learned my addr 172.21.15.28:0/2915926118 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f8821000640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c15c310 0x7f881c160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 -- 172.21.15.28:0/2915926118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c15c310 msgr2=0x7f881c160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c15c310 0x7f881c160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 -- 172.21.15.28:0/2915926118 >> v1:172.21.15.138:6789/0 conn(0x7f881c0d0370 legacy=0x7f881c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.442+0000 7f881be00640 1 -- 172.21.15.28:0/2915926118 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c160130 con 0x7f881c0b4800 2024-09-10T10:49:14.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.443+0000 7f881be00640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c0b4800 0x7f881c0aef30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4701291a1e5f7ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f881b400640 1 -- 172.21.15.28:0/2915926118 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808037070 con 0x7f881c0b4800 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f881b400640 1 -- 172.21.15.28:0/2915926118 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808032440 con 0x7f881c0b4800 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f881b400640 1 -- 172.21.15.28:0/2915926118 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808032810 con 0x7f881c0b4800 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 -- 172.21.15.28:0/2915926118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c0b4800 msgr2=0x7f881c0aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c0b4800 0x7f881c0aef30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802a270 comp rx=0 tx=0).stop 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 -- 172.21.15.28:0/2915926118 shutdown_connections 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c0b4800 0x7f881c0aef30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 --2- 172.21.15.28:0/2915926118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c15c310 0x7f881c160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 -- 172.21.15.28:0/2915926118 >> 172.21.15.28:0/2915926118 conn(0x7f881c06bdf0 msgr2=0x7f881c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 -- 172.21.15.28:0/2915926118 shutdown_connections 2024-09-10T10:49:14.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.444+0000 7f8823a00640 1 -- 172.21.15.28:0/2915926118 wait complete. 2024-09-10T10:49:14.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.446+0000 7f8823a00640 1 Processor -- start 2024-09-10T10:49:14.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.446+0000 7f8823a00640 1 -- start start 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c0b4800 0x7f881c14d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f881c0d0370 0x7f881c09c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0d0820 con 0x7f881c15c310 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f881c15c6f0 con 0x7f881c0d0370 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8823a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f881c0d09a0 con 0x7f881c0b4800 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54434/0 (socket says 172.21.15.28:54434) 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8821a00640 1 -- 172.21.15.28:0/3433966429 learned_addr learned my addr 172.21.15.28:0/3433966429 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f8821000640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c0b4800 0x7f881c14d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.447+0000 7f881be00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f881c0d0370 0x7f881c09c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 -- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c0b4800 msgr2=0x7f881c14d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c0b4800 0x7f881c14d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 -- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f881c0d0370 msgr2=0x7f881c09c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f881c0d0370 0x7f881c09c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 -- 172.21.15.28:0/3433966429 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c09cee0 con 0x7f881c15c310 2024-09-10T10:49:14.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8821a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f881400c7a0 tx=0x7f881400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8818c00640 1 -- 172.21.15.28:0/3433966429 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814003b70 con 0x7f881c15c310 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8818c00640 1 -- 172.21.15.28:0/3433966429 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8814003d30 con 0x7f881c15c310 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8818c00640 1 -- 172.21.15.28:0/3433966429 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881400aa90 con 0x7f881c15c310 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c09d1c0 con 0x7f881c15c310 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.448+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c09da60 con 0x7f881c15c310 2024-09-10T10:49:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.450+0000 7f8818c00640 1 -- 172.21.15.28:0/3433966429 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f881400ac30 con 0x7f881c15c310 2024-09-10T10:49:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.451+0000 7f8818c00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 0x7f87fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.451+0000 7f8818c00640 1 -- 172.21.15.28:0/3433966429 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f881408b190 con 0x7f881c15c310 2024-09-10T10:49:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.452+0000 7f8826866740 1 Processor -- start 2024-09-10T10:49:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.452+0000 7f8821000640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 0x7f87fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.452+0000 7f8826866740 1 -- start start 2024-09-10T10:49:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8821000640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 0x7f87fc06e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb6b480 0x5627ecbb6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5627ecad3660 con 0x5627ecb85420 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627eca7b680 con 0x5627ecb6b480 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5627ecacf620 con 0x5627ecb85800 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8821000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8821000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54436/0 (socket says 172.21.15.28:54436) 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8821000640 1 -- 172.21.15.28:0/3121094237 learned_addr learned my addr 172.21.15.28:0/3121094237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f881be00640 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.453+0000 7f8821a00640 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb6b480 0x5627ecbb6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 -- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 msgr2=0x5627ecbb6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 -- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb6b480 msgr2=0x5627ecbb6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb6b480 0x5627ecbb6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 -- 172.21.15.28:0/3121094237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ec8b07d0 con 0x5627ecb85420 2024-09-10T10:49:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f8821000640 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f881000fa20 tx=0x7f8810012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b04b7b6599c2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f880d600640 1 -- 172.21.15.28:0/3121094237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881001b050 con 0x5627ecb85420 2024-09-10T10:49:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f880d600640 1 -- 172.21.15.28:0/3121094237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810025070 con 0x5627ecb85420 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.454+0000 7f880d600640 1 -- 172.21.15.28:0/3121094237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8810014de0 con 0x5627ecb85420 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 -- 172.21.15.28:0/3121094237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 msgr2=0x5627ecbb5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f881000fa20 tx=0x7f8810012000 comp rx=0 tx=0).stop 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 -- 172.21.15.28:0/3121094237 shutdown_connections 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb6b480 0x5627ecbb6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 --2- 172.21.15.28:0/3121094237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85420 0x5627ecbb5c40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 -- 172.21.15.28:0/3121094237 >> 172.21.15.28:0/3121094237 conn(0x5627ecb6c220 msgr2=0x5627ecbb4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 -- 172.21.15.28:0/3121094237 shutdown_connections 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.455+0000 7f8826866740 1 -- 172.21.15.28:0/3121094237 wait complete. 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.456+0000 7f8826866740 1 -- start start 2024-09-10T10:49:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.456+0000 7f8826866740 1 Processor -- start 2024-09-10T10:49:14.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecb73ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5627ecb6a3e0 con 0x5627ecb728e0 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627eca5c4c0 con 0x5627ecb85420 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5627ecb6a6c0 con 0x5627ecb85800 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8826866740 10 client.0 _ll_register_callbacks cb 0x5627ecacbe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54440/0 (socket says 172.21.15.28:54440) 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.457+0000 7f8821a00640 1 -- 172.21.15.28:0/2133295610 learned_addr learned my addr 172.21.15.28:0/2133295610 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f881be00640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821000640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecb73ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 -- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 msgr2=0x5627ecbb18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 -- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 msgr2=0x5627ecb73ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecb73ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 -- 172.21.15.28:0/2133295610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ec7e63f0 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f881be00640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821000640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecb73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8821a00640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8814055500 tx=0x7f8814058ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899fee049f09739 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8802000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8814087060 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f8814055710 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8810025070 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8814071070 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8826866740 10 client.6255 fetch_fsmap learned FSMap version 60 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.458+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5627ec7a60f0 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8810025070 con 0x5627ecb728e0 2024-09-10T10:49:14.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f88140584f0 con 0x5627ecb728e0 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 10 client.6255 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 10 client.6255 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 20 client.6255 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecbaf340 0x5627ecbaf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 4 client.6255 mds_command: new command op to 5572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 --> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627ecae0d90 con 0x5627ecbaf340 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.459+0000 7f8821000640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecbaf340 0x5627ecbaf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.460+0000 7f8821000640 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecbaf340 0x5627ecbaf7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5b0ec84ae2f8712 server_cookie=d86b2bd06669ca40 in_seq=0 out_seq=0 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.460+0000 7f8802000640 10 client.6255 ms_handle_connect on v2:172.21.15.28:6833/2942680174 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.460+0000 7f8802000640 1 -- 172.21.15.28:0/2133295610 <== mds.0 v2:172.21.15.28:6833/2942680174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627ecae0d90 con 0x5627ecbaf340 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.460+0000 7f8802000640 10 client.6255 handle_command_reply: tid=0 2024-09-10T10:49:14.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.460+0000 7f8826866740 1 client.6255 shutdown 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 msgr2=0x5627ecbb1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8814055500 tx=0x7f8814058ac0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 shutdown_connections 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb728e0 0x5627ecbb1e00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecbaf340 0x5627ecbaf7b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85800 0x5627ecbb18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 --2- 172.21.15.28:0/2133295610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecb73ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8802000640 0 client.6255 ms_handle_reset on v2:172.21.15.28:6833/2942680174 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 >> 172.21.15.28:0/2133295610 conn(0x5627ecb6b480 msgr2=0x5627ecb68f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 shutdown_connections 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 1 -- 172.21.15.28:0/2133295610 wait complete. 2024-09-10T10:49:14.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.461+0000 7f8826866740 20 client.6255 trim_cache size 0 max 16384 2024-09-10T10:49:14.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.476+0000 7f8826866740 1 Processor -- start 2024-09-10T10:49:14.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.476+0000 7f8826866740 1 -- start start 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb728e0 0x5627ecb69a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85420 0x5627ecbb2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1ff480 con 0x5627ecb85800 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1fe460 con 0x5627ecb728e0 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f881c1f31e0 con 0x5627ecb85420 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54446/0 (socket says 172.21.15.28:54446) 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.477+0000 7f8821a00640 1 -- 172.21.15.28:0/853023711 learned_addr learned my addr 172.21.15.28:0/853023711 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 -- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85420 msgr2=0x5627ecbb2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85420 0x5627ecbb2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 -- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb728e0 msgr2=0x5627ecb69a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb728e0 0x5627ecb69a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 -- 172.21.15.28:0/853023711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627ecbb0d40 con 0x5627ecb85800 2024-09-10T10:49:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f8821a00640 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8814001330 tx=0x7f881405d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47f720bc4358fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f880cc00640 1 -- 172.21.15.28:0/853023711 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881405d450 con 0x5627ecb85800 2024-09-10T10:49:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f880cc00640 1 -- 172.21.15.28:0/853023711 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8814061c80 con 0x5627ecb85800 2024-09-10T10:49:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.478+0000 7f880cc00640 1 -- 172.21.15.28:0/853023711 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140584f0 con 0x5627ecb85800 2024-09-10T10:49:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 -- 172.21.15.28:0/853023711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 msgr2=0x5627ecbd75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8814001330 tx=0x7f881405d000 comp rx=0 tx=0).stop 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 -- 172.21.15.28:0/853023711 shutdown_connections 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbd75b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb85420 0x5627ecbb2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 --2- 172.21.15.28:0/853023711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb728e0 0x5627ecb69a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 -- 172.21.15.28:0/853023711 >> 172.21.15.28:0/853023711 conn(0x5627ecb6b480 msgr2=0x5627ecbae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 -- 172.21.15.28:0/853023711 shutdown_connections 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.479+0000 7f8826866740 1 -- 172.21.15.28:0/853023711 wait complete. 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.480+0000 7f8826866740 1 -- start start 2024-09-10T10:49:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.480+0000 7f8826866740 1 Processor -- start 2024-09-10T10:49:14.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 0x5627ecbf2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecbb0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f881c1fddc0 con 0x5627ecb85800 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5627ecb70b40 con 0x5627ecb85420 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5627ecbf1f80 con 0x5627ecb728e0 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8826866740 10 client.0 _ll_register_callbacks cb 0x7f8814058be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54458/0 (socket says 172.21.15.28:54458) 2024-09-10T10:49:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 -- 172.21.15.28:0/1457066292 learned_addr learned my addr 172.21.15.28:0/1457066292 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f881be00640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecbb0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821000640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 0x5627ecbf2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 -- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 msgr2=0x5627ecbf2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 0x5627ecbf2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 -- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 msgr2=0x5627ecbb0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecbb0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821a00640 1 -- 172.21.15.28:0/1457066292 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627ecc1c790 con 0x5627ecb85800 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.481+0000 7f8821000640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 0x5627ecbf2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8821a00640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8814055500 tx=0x7f88140891f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd13ff2c835eeb36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881405d450 con 0x5627ecb85800 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f8814087b20 con 0x5627ecb85800 2024-09-10T10:49:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8814061c80 con 0x5627ecb85800 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f8814066050 con 0x5627ecb85800 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 10 client.6261 fetch_fsmap learned FSMap version 60 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x5627ecc1c790 con 0x5627ecb85800 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8814061c80 con 0x5627ecb85800 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f8814069020 con 0x5627ecb85800 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 10 client.6261 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 10 client.6261 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.482+0000 7f8826866740 20 client.6261 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecc1b8d0 0x5627ecc1bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8826866740 4 client.6261 mds_command: new command op to 5572 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 --> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627ecadb0e0 con 0x5627ecc1b8d0 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8821000640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecc1b8d0 0x5627ecc1bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8821000640 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecc1b8d0 0x5627ecc1bcb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8fc1d7bbc20e03 server_cookie=bd71084c99ee9cc5 in_seq=0 out_seq=0 2024-09-10T10:49:14.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.483+0000 7f8802000640 10 client.6261 ms_handle_connect on v2:172.21.15.28:6833/2942680174 2024-09-10T10:49:14.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8802000640 1 -- 172.21.15.28:0/1457066292 <== mds.0 v2:172.21.15.28:6833/2942680174 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627ecadb0e0 con 0x5627ecc1b8d0 2024-09-10T10:49:14.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8802000640 10 client.6261 handle_command_reply: tid=0 2024-09-10T10:49:14.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 client.6261 shutdown 2024-09-10T10:49:14.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 msgr2=0x5627ecbb1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8814055500 tx=0x7f88140891f0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 shutdown_connections 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627ecb85800 0x5627ecbb1470 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5627ecb85420 0x5627ecbb0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.28:6833/2942680174,v1:172.21.15.28:6835/2942680174] conn(0x5627ecc1b8d0 0x5627ecc1bcb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 --2- 172.21.15.28:0/1457066292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5627ecb728e0 0x5627ecbf2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8802000640 0 client.6261 ms_handle_reset on v2:172.21.15.28:6833/2942680174 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 >> 172.21.15.28:0/1457066292 conn(0x5627ecb6b480 msgr2=0x5627ecbae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 shutdown_connections 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 1 -- 172.21.15.28:0/1457066292 wait complete. 2024-09-10T10:49:14.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.496+0000 7f8826866740 20 client.6261 trim_cache size 0 max 16384 2024-09-10T10:49:14.499 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:49:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:49:14.499 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:49:14.499 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 msgr2=0x7f87fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 0x7f87fc06e2f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8810006fd0 tx=0x7f8810008040 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 msgr2=0x7f881c0ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f881400c7a0 tx=0x7f881400cc60 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 shutdown_connections 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87fc06be30 0x7f87fc06e2f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f881c15c310 0x7f881c0ad1a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f881c0d0370 0x7f881c09c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 --2- 172.21.15.28:0/3433966429 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f881c0b4800 0x7f881c14d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 >> 172.21.15.28:0/3433966429 conn(0x7f881c06bdf0 msgr2=0x7f881c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:14.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 shutdown_connections 2024-09-10T10:49:14.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:14.497+0000 7f8823a00640 1 -- 172.21.15.28:0/3433966429 wait complete. 2024-09-10T10:49:14.513 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:49:14.514 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:49:14.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:49:14.705 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:49:14.706 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:49:14.706 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:14.706 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:14.741 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:14.741 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:14.817 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:49:15.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.013+0000 7f837f400640 1 Processor -- start 2024-09-10T10:49:15.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.013+0000 7f837f400640 1 -- start start 2024-09-10T10:49:15.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.014+0000 7f837f400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8378072de0 0x7f837815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.014+0000 7f837f400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f837815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.014+0000 7f837f400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8378146140 con 0x7f8378072320 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.014+0000 7f837f400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83780cd740 con 0x7f8378072de0 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.014+0000 7f837f400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83780cda20 con 0x7f8378072700 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f837ca00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8378072320 0x7f83780cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45756/0 (socket says 172.21.15.28:45756) 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f837ca00640 1 -- 172.21.15.28:0/1093050423 learned_addr learned my addr 172.21.15.28:0/1093050423 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f837d400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f837815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377e00640 1 --2- 172.21.15.28:0/1093050423 >> v2:172.21.15.138:3300/0 conn(0x7f8378072de0 0x7f837815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689608111 0 0) 0x7f8378146140 con 0x7f8378072320 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8360003520 con 0x7f8378072320 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1888780214 0 0) 0x7f8360003520 con 0x7f8378072320 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 msgr2=0x7f837815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f837815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 >> v2:172.21.15.138:3300/0 conn(0x7f8378072de0 msgr2=0x7f837815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 --2- 172.21.15.28:0/1093050423 >> v2:172.21.15.138:3300/0 conn(0x7f8378072de0 0x7f837815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780cdd00 con 0x7f8378072320 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f836c001670 con 0x7f8378072320 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 >> v1:172.21.15.28:6789/0 conn(0x7f8378072320 legacy=0x7f83780cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83600039b0 0x7f8360003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.015+0000 7f8377400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f8360004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f8377400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8360004d40 0x7f83600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83780cd740 con 0x7f8360004d40 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83780cda20 con 0x7f83600039b0 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f836000b9f0 con 0x7f8378072700 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8360004d40 0x7f83600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837ca00640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83600039b0 0x7f8360003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f8377e00640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f8360004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 -- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 msgr2=0x7f8360004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8378072700 0x7f8360004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 -- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83600039b0 msgr2=0x7f8360003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83600039b0 0x7f8360003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.016+0000 7f837d400640 1 -- 172.21.15.28:0/1093050423 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8360003520 con 0x7f8360004d40 2024-09-10T10:49:15.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.017+0000 7f837d400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8360004d40 0x7f83600051c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8370009f90 tx=0x7f8360005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65ab643c18095f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.017+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370009c10 con 0x7f8360004d40 2024-09-10T10:49:15.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370036070 con 0x7f8360004d40 2024-09-10T10:49:15.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f8377400640 1 -- 172.21.15.28:0/1093050423 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370031810 con 0x7f8360004d40 2024-09-10T10:49:15.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 -- 172.21.15.28:0/1093050423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8360004d40 msgr2=0x7f83600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 --2- 172.21.15.28:0/1093050423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8360004d40 0x7f83600051c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8370009f90 tx=0x7f8360005830 comp rx=0 tx=0).stop 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837d400640 1 -- 172.21.15.28:0/1093050423 reap_dead start 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 -- 172.21.15.28:0/1093050423 shutdown_connections 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 -- 172.21.15.28:0/1093050423 >> 172.21.15.28:0/1093050423 conn(0x7f8378067d70 msgr2=0x7f837806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 -- 172.21.15.28:0/1093050423 shutdown_connections 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.018+0000 7f837f400640 1 -- 172.21.15.28:0/1093050423 wait complete. 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.020+0000 7f837f400640 1 Processor -- start 2024-09-10T10:49:15.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.020+0000 7f837f400640 1 -- start start 2024-09-10T10:49:15.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8360004d40 0x7f83780a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 0x7f837813fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f837815d8a0 con 0x7f837815b990 2024-09-10T10:49:15.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f837815d720 con 0x7f8360004d40 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837f400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f837815da20 con 0x7f83780aa380 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54476/0 (socket says 172.21.15.28:54476) 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837d400640 1 -- 172.21.15.28:0/1529550663 learned_addr learned my addr 172.21.15.28:0/1529550663 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837ca00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8360004d40 0x7f83780a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f8377e00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 0x7f837813fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.021+0000 7f837d400640 1 -- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 msgr2=0x7f837813fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837d400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 0x7f837813fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837d400640 1 -- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8360004d40 msgr2=0x7f83780a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837d400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8360004d40 0x7f83780a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837d400640 1 -- 172.21.15.28:0/1529550663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837815c350 con 0x7f837815b990 2024-09-10T10:49:15.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f8377e00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 0x7f837813fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837d400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8370033040 tx=0x7f8370009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370009c10 con 0x7f837815b990 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8370045430 con 0x7f837815b990 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83780aabd0 con 0x7f837815b990 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370045800 con 0x7f837815b990 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.022+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83780ab190 con 0x7f837815b990 2024-09-10T10:49:15.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.023+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837815d1e0 con 0x7f837815b990 2024-09-10T10:49:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.024+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8370005d10 con 0x7f837815b990 2024-09-10T10:49:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.025+0000 7f8374c00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 0x7f835806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.025+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f83700ac490 con 0x7f837815b990 2024-09-10T10:49:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.025+0000 7f837ca00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 0x7f835806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.026+0000 7f837ca00640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 0x7f835806e320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f83780cc380 tx=0x7f836c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.029+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f837002e050 con 0x7f837815b990 2024-09-10T10:49:15.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.226+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8378144cd0 con 0x7f837815b990 2024-09-10T10:49:15.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.226+0000 7f8374c00640 1 -- 172.21.15.28:0/1529550663 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v74) ==== 80+0+248 (secure 0 0 0) 0x7f837007aaa0 con 0x7f837815b990 2024-09-10T10:49:15.228 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6837/2819175864 2024-09-11T10:47:36.629863+0000 2024-09-10T10:49:15.228 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6835/2819175864 2024-09-11T10:47:36.629863+0000 2024-09-10T10:49:15.228 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6837/3429810116 2024-09-11T10:47:00.608334+0000 2024-09-10T10:49:15.228 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6835/3429810116 2024-09-11T10:47:00.608334+0000 2024-09-10T10:49:15.228 INFO:teuthology.orchestra.run.smithi028.stderr:listed 4 entries 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.228+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 msgr2=0x7f835806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.228+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 0x7f835806e320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f83780cc380 tx=0x7f836c009040 comp rx=0 tx=0).stop 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.228+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 msgr2=0x7f837815be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.228+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8370033040 tx=0x7f8370009750 comp rx=0 tx=0).stop 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 shutdown_connections 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f835806be60 0x7f835806e320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f837815b990 0x7f837815be10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83780aa380 0x7f837813fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 --2- 172.21.15.28:0/1529550663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8360004d40 0x7f83780a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 >> 172.21.15.28:0/1529550663 conn(0x7f8378067d70 msgr2=0x7f8378143f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 shutdown_connections 2024-09-10T10:49:15.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.229+0000 7f837f400640 1 -- 172.21.15.28:0/1529550663 wait complete. 2024-09-10T10:49:15.246 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:49:15.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:15.246 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:15.309 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:49:15.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:15.309 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:49:15.344 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:49:15.345 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:49:15.345 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:15.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:49:15.385 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:49:15.385 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:15.385 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:15.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:49:15.420 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:49:15.420 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:49:15.420 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:15.462 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:49:15.463 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.617+0000 7fd669800640 1 Processor -- start 2024-09-10T10:49:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.617+0000 7fd669800640 1 -- start start 2024-09-10T10:49:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd669800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd66415e660 0x7fd66406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd669800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6640cf000 con 0x7fd6640d0dd0 2024-09-10T10:49:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd669800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6640d29a0 con 0x7fd6640d1f90 2024-09-10T10:49:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd669800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd6640b37d0 con 0x7fd66415e660 2024-09-10T10:49:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd663e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd6640d0dd0 0x7fd6640d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45764/0 (socket says 172.21.15.28:45764) 2024-09-10T10:49:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd663e00640 1 -- 172.21.15.28:0/1468870876 learned_addr learned my addr 172.21.15.28:0/1468870876 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:15.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.618+0000 7fd663400640 1 --2- 172.21.15.28:0/1468870876 >> v2:172.21.15.28:3300/0 conn(0x7fd66415e660 0x7fd66406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.619+0000 7fd662a00640 1 -- 172.21.15.28:0/1468870876 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654069175 0 0) 0x7fd6640cf000 con 0x7fd6640d0dd0 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.619+0000 7fd662a00640 1 -- 172.21.15.28:0/1468870876 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd648003520 con 0x7fd6640d0dd0 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.619+0000 7fd663400640 1 -- 172.21.15.28:0/1468870876 >> v1:172.21.15.138:6789/0 conn(0x7fd6640d1f90 legacy=0x7fd6640cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.619+0000 7fd663400640 1 -- 172.21.15.28:0/1468870876 >> v1:172.21.15.28:6789/0 conn(0x7fd6640d0dd0 legacy=0x7fd6640d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.619+0000 7fd663400640 1 -- 172.21.15.28:0/1468870876 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6640d2c60 con 0x7fd66415e660 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.620+0000 7fd663400640 1 --2- 172.21.15.28:0/1468870876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd66406ccd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd6500097a0 tx=0x7fd65002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3e8a0fa0bad8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.620+0000 7fd662a00640 1 -- 172.21.15.28:0/1468870876 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650037070 con 0x7fd66415e660 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.620+0000 7fd662a00640 1 -- 172.21.15.28:0/1468870876 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd650032440 con 0x7fd66415e660 2024-09-10T10:49:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.620+0000 7fd662a00640 1 -- 172.21.15.28:0/1468870876 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd650032810 con 0x7fd66415e660 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 -- 172.21.15.28:0/1468870876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 msgr2=0x7fd66406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 --2- 172.21.15.28:0/1468870876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd66406ccd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd6500097a0 tx=0x7fd65002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 -- 172.21.15.28:0/1468870876 shutdown_connections 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 --2- 172.21.15.28:0/1468870876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd66406ccd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 -- 172.21.15.28:0/1468870876 >> 172.21.15.28:0/1468870876 conn(0x7fd66408bfc0 msgr2=0x7fd6640d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 -- 172.21.15.28:0/1468870876 shutdown_connections 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.621+0000 7fd669800640 1 -- 172.21.15.28:0/1468870876 wait complete. 2024-09-10T10:49:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.622+0000 7fd669800640 1 Processor -- start 2024-09-10T10:49:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 -- start start 2024-09-10T10:49:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640d0dd0 0x7fd66415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6640d1f90 0x7fd6640b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640d3220 con 0x7fd66415e660 2024-09-10T10:49:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640cf330 con 0x7fd6640d0dd0 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.623+0000 7fd669800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6640d33a0 con 0x7fd6640d1f90 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54496/0 (socket says 172.21.15.28:54496) 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 -- 172.21.15.28:0/3715585695 learned_addr learned my addr 172.21.15.28:0/3715585695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd663e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640d0dd0 0x7fd66415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd663400640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6640d1f90 0x7fd6640b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 -- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6640d1f90 msgr2=0x7fd6640b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6640d1f90 0x7fd6640b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 -- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640d0dd0 msgr2=0x7fd66415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640d0dd0 0x7fd66415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 -- 172.21.15.28:0/3715585695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6640cb530 con 0x7fd66415e660 2024-09-10T10:49:15.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.624+0000 7fd668e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c780 tx=0x7fd65c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.625+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c004b60 con 0x7fd66415e660 2024-09-10T10:49:15.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.625+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c004d00 con 0x7fd66415e660 2024-09-10T10:49:15.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.625+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00aa90 con 0x7fd66415e660 2024-09-10T10:49:15.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.625+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66415c9d0 con 0x7fd66415e660 2024-09-10T10:49:15.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.625+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6640cbab0 con 0x7fd66415e660 2024-09-10T10:49:15.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.626+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66406d2a0 con 0x7fd66415e660 2024-09-10T10:49:15.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.627+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd65c00ac30 con 0x7fd66415e660 2024-09-10T10:49:15.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.628+0000 7fd65be00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 0x7fd64406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:15.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.628+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7fd65c08b3d0 con 0x7fd66415e660 2024-09-10T10:49:15.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.628+0000 7fd663e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 0x7fd64406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:15.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.629+0000 7fd663e00640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 0x7fd64406e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd654005880 tx=0x7fd654009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:15.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.632+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd65c020070 con 0x7fd66415e660 2024-09-10T10:49:15.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.853+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd664161190 con 0x7fd66415e660 2024-09-10T10:49:15.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.853+0000 7fd65be00640 1 -- 172.21.15.28:0/3715585695 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5229 (secure 0 0 0) 0x7fd65c059390 con 0x7fd66415e660 2024-09-10T10:49:15.855 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:15.855 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":60,"btime":"2024-09-10T10:47:39:651000+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:39.650991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":18,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5572,"qdb_cluster":[5572]},"id":4}]} 2024-09-10T10:49:15.855 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 60 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.855+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 msgr2=0x7fd64406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 0x7fd64406e2f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd654005880 tx=0x7fd654009040 comp rx=0 tx=0).stop 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 msgr2=0x7fd6640b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c780 tx=0x7fd65c00cc40 comp rx=0 tx=0).stop 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 shutdown_connections 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd64406be30 0x7fd64406e2f0 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd654005880 tx=0x7fd654009040 comp rx=0 tx=0).stop 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd66415e660 0x7fd6640b7730 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6640d1f90 0x7fd6640b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 --2- 172.21.15.28:0/3715585695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6640d0dd0 0x7fd66415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:15.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 >> 172.21.15.28:0/3715585695 conn(0x7fd66408bfc0 msgr2=0x7fd66406e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:15.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 shutdown_connections 2024-09-10T10:49:15.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:15.856+0000 7fd669800640 1 -- 172.21.15.28:0/3715585695 wait complete. 2024-09-10T10:49:15.874 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:16.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.029+0000 7f0644c00640 1 Processor -- start 2024-09-10T10:49:16.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.029+0000 7f0644c00640 1 -- start start 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06400d0dd0 0x7f06400d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f064015e660 0x7f064006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06400d1f90 0x7f06400cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f06400b37d0 con 0x7f06400d0dd0 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06400cf330 con 0x7f064015e660 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f0644c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06400cf000 con 0x7f06400d1f90 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f063f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06400d0dd0 0x7f06400d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f063f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06400d0dd0 0x7f06400d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54510/0 (socket says 172.21.15.28:54510) 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.030+0000 7f063f400640 1 -- 172.21.15.28:0/3211548540 learned_addr learned my addr 172.21.15.28:0/3211548540 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063fe00640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06400d1f90 0x7f06400cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063ea00640 1 --2- 172.21.15.28:0/3211548540 >> v2:172.21.15.138:3300/0 conn(0x7f064015e660 0x7f064006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063f400640 1 -- 172.21.15.28:0/3211548540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06400d1f90 msgr2=0x7f06400cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063f400640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06400d1f90 0x7f06400cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063f400640 1 -- 172.21.15.28:0/3211548540 >> v2:172.21.15.138:3300/0 conn(0x7f064015e660 msgr2=0x7f064006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063f400640 1 --2- 172.21.15.28:0/3211548540 >> v2:172.21.15.138:3300/0 conn(0x7f064015e660 0x7f064006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.031+0000 7f063f400640 1 -- 172.21.15.28:0/3211548540 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06400d16f0 con 0x7f06400d0dd0 2024-09-10T10:49:16.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.032+0000 7f063f400640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d0dd0 0x7f06400d11b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f06400b3980 tx=0x7f063002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0b095c6afe6d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.032+0000 7f063e000640 1 -- 172.21.15.28:0/3211548540 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630037070 con 0x7f06400d0dd0 2024-09-10T10:49:16.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.032+0000 7f063e000640 1 -- 172.21.15.28:0/3211548540 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0630032440 con 0x7f06400d0dd0 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.032+0000 7f063e000640 1 -- 172.21.15.28:0/3211548540 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630032810 con 0x7f06400d0dd0 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 -- 172.21.15.28:0/3211548540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d0dd0 msgr2=0x7f06400d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d0dd0 0x7f06400d11b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f06400b3980 tx=0x7f063002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 -- 172.21.15.28:0/3211548540 shutdown_connections 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06400d1f90 0x7f06400cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 --2- 172.21.15.28:0/3211548540 >> v2:172.21.15.138:3300/0 conn(0x7f064015e660 0x7f064006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 --2- 172.21.15.28:0/3211548540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d0dd0 0x7f06400d11b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 -- 172.21.15.28:0/3211548540 >> 172.21.15.28:0/3211548540 conn(0x7f064008bfc0 msgr2=0x7f06400d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 -- 172.21.15.28:0/3211548540 shutdown_connections 2024-09-10T10:49:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.033+0000 7f0644c00640 1 -- 172.21.15.28:0/3211548540 wait complete. 2024-09-10T10:49:16.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.034+0000 7f0644c00640 1 Processor -- start 2024-09-10T10:49:16.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 -- start start 2024-09-10T10:49:16.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400d0dd0 0x7f06400ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064015e660 0x7f0640155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f06400d2f40 con 0x7f06400d1f90 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06400d2c60 con 0x7f06400d0dd0 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.035+0000 7f0644c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06400d3220 con 0x7f064015e660 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54522/0 (socket says 172.21.15.28:54522) 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 -- 172.21.15.28:0/2720531030 learned_addr learned my addr 172.21.15.28:0/2720531030 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063f400640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400d0dd0 0x7f06400ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063fe00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064015e660 0x7f0640155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 -- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064015e660 msgr2=0x7f0640155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064015e660 0x7f0640155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 -- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400d0dd0 msgr2=0x7f06400ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400d0dd0 0x7f06400ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.036+0000 7f063ea00640 1 -- 172.21.15.28:0/2720531030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06400cd430 con 0x7f06400d1f90 2024-09-10T10:49:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f063ea00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f062c00d5f0 tx=0x7f062c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0125f0 con 0x7f06400d1f90 2024-09-10T10:49:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c019070 con 0x7f06400d1f90 2024-09-10T10:49:16.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0129d0 con 0x7f06400d1f90 2024-09-10T10:49:16.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06400cba00 con 0x7f06400d1f90 2024-09-10T10:49:16.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.037+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06400cbfc0 con 0x7f06400d1f90 2024-09-10T10:49:16.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.038+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06400d2440 con 0x7f06400d1f90 2024-09-10T10:49:16.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.039+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f062c012b70 con 0x7f06400d1f90 2024-09-10T10:49:16.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.040+0000 7f0637400640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 0x7f062006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.040+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f062c08b540 con 0x7f06400d1f90 2024-09-10T10:49:16.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.040+0000 7f063f400640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 0x7f062006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.041+0000 7f063f400640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 0x7f062006e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0630034000 tx=0x7f063002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.045+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f062c059470 con 0x7f06400d1f90 2024-09-10T10:49:16.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.266+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06400d1b20 con 0x7f06400d1f90 2024-09-10T10:49:16.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.267+0000 7f0637400640 1 -- 172.21.15.28:0/2720531030 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5229 (secure 0 0 0) 0x7f062c05d770 con 0x7f06400d1f90 2024-09-10T10:49:16.268 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:16.268 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":60,"btime":"2024-09-10T10:47:39:651000+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:39.650991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":18,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5572,"qdb_cluster":[5572]},"id":4}]} 2024-09-10T10:49:16.268 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 60 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 msgr2=0x7f062006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 0x7f062006e2f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0630034000 tx=0x7f063002e000 comp rx=0 tx=0).stop 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 msgr2=0x7f06400ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f062c00d5f0 tx=0x7f062c00dab0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 shutdown_connections 2024-09-10T10:49:16.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f062006be30 0x7f062006e2f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f064015e660 0x7f0640155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06400d1f90 0x7f06400ce010 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 --2- 172.21.15.28:0/2720531030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06400d0dd0 0x7f06400ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 >> 172.21.15.28:0/2720531030 conn(0x7f064008bfc0 msgr2=0x7f06400b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 shutdown_connections 2024-09-10T10:49:16.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.269+0000 7f0644c00640 1 -- 172.21.15.28:0/2720531030 wait complete. 2024-09-10T10:49:16.285 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:16.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.441+0000 7f2edbe00640 1 Processor -- start 2024-09-10T10:49:16.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.441+0000 7f2edbe00640 1 -- start start 2024-09-10T10:49:16.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.442+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 0x7f2edc072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.442+0000 7f2edbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2edc0b49c0 0x7f2edc073320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.442+0000 7f2edbe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc06dc70 con 0x7f2edc15b330 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.442+0000 7f2edbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2edc0b3f50 con 0x7f2edc0b49c0 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.442+0000 7f2edbe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc06df10 con 0x7f2edc15af50 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2edc0b49c0 0x7f2edc073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2edc0b49c0 0x7f2edc073320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54536/0 (socket says 172.21.15.28:54536) 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 -- 172.21.15.28:0/4059286451 learned_addr learned my addr 172.21.15.28:0/4059286451 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edaa00640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 0x7f2edc072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 -- 172.21.15.28:0/4059286451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 msgr2=0x7f2edc072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 0x7f2edc072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 -- 172.21.15.28:0/4059286451 >> v1:172.21.15.138:6789/0 conn(0x7f2edc15b330 legacy=0x7f2edc0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edb400640 1 -- 172.21.15.28:0/4059286451 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc06e1f0 con 0x7f2edc0b49c0 2024-09-10T10:49:16.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.443+0000 7f2edaa00640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 0x7f2edc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.444+0000 7f2edb400640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc0b49c0 0x7f2edc073320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009e50 tx=0x7f2ed402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24f87da3d85df59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2ed9600640 1 -- 172.21.15.28:0/4059286451 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4036070 con 0x7f2edc0b49c0 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2ed9600640 1 -- 172.21.15.28:0/4059286451 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4031600 con 0x7f2edc0b49c0 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2ed9600640 1 -- 172.21.15.28:0/4059286451 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40319d0 con 0x7f2edc0b49c0 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 -- 172.21.15.28:0/4059286451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc0b49c0 msgr2=0x7f2edc073320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc0b49c0 0x7f2edc073320 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009e50 tx=0x7f2ed402a710 comp rx=0 tx=0).stop 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 -- 172.21.15.28:0/4059286451 shutdown_connections 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc0b49c0 0x7f2edc073320 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 --2- 172.21.15.28:0/4059286451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc15af50 0x7f2edc072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 -- 172.21.15.28:0/4059286451 >> 172.21.15.28:0/4059286451 conn(0x7f2edc06c240 msgr2=0x7f2edc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 -- 172.21.15.28:0/4059286451 shutdown_connections 2024-09-10T10:49:16.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.445+0000 7f2edbe00640 1 -- 172.21.15.28:0/4059286451 wait complete. 2024-09-10T10:49:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.447+0000 7f2edbe00640 1 Processor -- start 2024-09-10T10:49:16.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.447+0000 7f2edbe00640 1 -- start start 2024-09-10T10:49:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc0b49c0 0x7f2edc152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2edc15b330 0x7f2edc166e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0adbc0 con 0x7f2edc15af50 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc06e7b0 con 0x7f2edc15b330 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edbe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc0add40 con 0x7f2edc0b49c0 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2eda000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2eda000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54542/0 (socket says 172.21.15.28:54542) 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2eda000640 1 -- 172.21.15.28:0/3950603307 learned_addr learned my addr 172.21.15.28:0/3950603307 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edaa00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc0b49c0 0x7f2edc152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.448+0000 7f2edb400640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2edc15b330 0x7f2edc166e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 -- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc0b49c0 msgr2=0x7f2edc152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc0b49c0 0x7f2edc152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 -- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2edc15b330 msgr2=0x7f2edc166e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2edc15b330 0x7f2edc166e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 -- 172.21.15.28:0/3950603307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc154ab0 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2eda000640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c3c0 tx=0x7f2ec800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8017070 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec80052d0 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc154d90 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8003850 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.449+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc155350 con 0x7f2edc15af50 2024-09-10T10:49:16.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.451+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc070030 con 0x7f2edc15af50 2024-09-10T10:49:16.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.451+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ec801a070 con 0x7f2edc15af50 2024-09-10T10:49:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.453+0000 7f2ed2a00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.453+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f2ec808a330 con 0x7f2edc15af50 2024-09-10T10:49:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.453+0000 7f2edaa00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.454+0000 7f2edaa00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006fd0 tx=0x7f2ecc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.457+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2edc070030 con 0x7f2edc15af50 2024-09-10T10:49:16.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.642+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2edc15ea00 con 0x7f2edc15af50 2024-09-10T10:49:16.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.643+0000 7f2ed2a00640 1 -- 172.21.15.28:0/3950603307 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17555 (secure 0 0 0) 0x7f2ec8058380 con 0x7f2edc15af50 2024-09-10T10:49:16.645 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:16.645 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":74,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:47:36.629915+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:42.078607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:43.284849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.b","mappings":[{"from":4,"to":5}]},{"pgid":"10.10","mappings":[{"from":2,"to":6}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.3b","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6837/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6835/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6837/3429810116":"2024-09-11T10:47:00.608334+0000","172.21.15.138:6835/3429810116":"2024-09-11T10:47:00.608334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 msgr2=0x7f2ebc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc006fd0 tx=0x7f2ecc008040 comp rx=0 tx=0).stop 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 msgr2=0x7f2edc154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c3c0 tx=0x7f2ec800c880 comp rx=0 tx=0).stop 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 shutdown_connections 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ebc06be30 0x7f2ebc06e2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2edc15b330 0x7f2edc166e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2edc15af50 0x7f2edc154160 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 --2- 172.21.15.28:0/3950603307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2edc0b49c0 0x7f2edc152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 >> 172.21.15.28:0/3950603307 conn(0x7f2edc06c240 msgr2=0x7f2edc0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 shutdown_connections 2024-09-10T10:49:16.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.646+0000 7f2edbe00640 1 -- 172.21.15.28:0/3950603307 wait complete. 2024-09-10T10:49:16.665 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:49:16.665 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:16.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.820+0000 7f57ad600640 1 Processor -- start 2024-09-10T10:49:16.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.820+0000 7f57ad600640 1 -- start start 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57ad600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072320 0x7f57a80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57ad600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f57a8072de0 0x7f57a806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57ad600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a8146140 con 0x7f57a8072700 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57ad600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f57a80cd740 con 0x7f57a8072de0 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57ad600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57a80cda20 con 0x7f57a8072320 2024-09-10T10:49:16.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f57a8072de0 0x7f57a806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f57a8072de0 0x7f57a806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54554/0 (socket says 172.21.15.28:54554) 2024-09-10T10:49:16.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 -- 172.21.15.28:0/1753674197 learned_addr learned my addr 172.21.15.28:0/1753674197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f57a6a00640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072320 0x7f57a80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 -- 172.21.15.28:0/1753674197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072320 msgr2=0x7f57a80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072320 0x7f57a80cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.822+0000 7f579fe00640 1 -- 172.21.15.28:0/1753674197 >> v1:172.21.15.138:6789/0 conn(0x7f57a8072700 legacy=0x7f57a806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.823+0000 7f579fe00640 1 -- 172.21.15.28:0/1753674197 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a80cdd00 con 0x7f57a8072de0 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f579fe00640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072de0 0x7f57a806e0b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffcca011a739896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57a6000640 1 -- 172.21.15.28:0/1753674197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790037070 con 0x7f57a8072de0 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57a6000640 1 -- 172.21.15.28:0/1753674197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5790032440 con 0x7f57a8072de0 2024-09-10T10:49:16.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57a6000640 1 -- 172.21.15.28:0/1753674197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790032810 con 0x7f57a8072de0 2024-09-10T10:49:16.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 -- 172.21.15.28:0/1753674197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072de0 msgr2=0x7f57a806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072de0 0x7f57a806e0b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5790009870 tx=0x7f579002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 -- 172.21.15.28:0/1753674197 shutdown_connections 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072de0 0x7f57a806e0b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 --2- 172.21.15.28:0/1753674197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072320 0x7f57a80cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 -- 172.21.15.28:0/1753674197 >> 172.21.15.28:0/1753674197 conn(0x7f57a8067d70 msgr2=0x7f57a806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 -- 172.21.15.28:0/1753674197 shutdown_connections 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.824+0000 7f57ad600640 1 -- 172.21.15.28:0/1753674197 wait complete. 2024-09-10T10:49:16.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.826+0000 7f57ad600640 1 Processor -- start 2024-09-10T10:49:16.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.826+0000 7f57ad600640 1 -- start start 2024-09-10T10:49:16.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a8072700 0x7f57a80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072de0 0x7f57a80adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80cd520 con 0x7f57a8072320 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57a80cd3a0 con 0x7f57a8072700 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57ad600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57a806ed00 con 0x7f57a8072de0 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57a6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54558/0 (socket says 172.21.15.28:54558) 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57a6a00640 1 -- 172.21.15.28:0/725082032 learned_addr learned my addr 172.21.15.28:0/725082032 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f579fe00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a8072700 0x7f57a80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.827+0000 7f57a7400640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072de0 0x7f57a80adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 -- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072de0 msgr2=0x7f57a80adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072de0 0x7f57a80adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 -- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a8072700 msgr2=0x7f57a80b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:16.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a8072700 0x7f57a80b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 -- 172.21.15.28:0/725082032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a815c390 con 0x7f57a8072320 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57a6a00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f579800e9b0 tx=0x7f579800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5798017070 con 0x7f57a8072320 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579800c9a0 con 0x7f57a8072320 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a80b4550 con 0x7f57a8072320 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.828+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800cd90 con 0x7f57a8072320 2024-09-10T10:49:16.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.829+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a80b4b10 con 0x7f57a8072320 2024-09-10T10:49:16.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.830+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a80ca240 con 0x7f57a8072320 2024-09-10T10:49:16.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.831+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f579801a070 con 0x7f57a8072320 2024-09-10T10:49:16.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.832+0000 7f579ea00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:16.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.832+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f579808b0f0 con 0x7f57a8072320 2024-09-10T10:49:16.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.832+0000 7f579fe00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:16.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.833+0000 7f579fe00640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 0x7f578406e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:16.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:16.836+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5798059140 con 0x7f57a8072320 2024-09-10T10:49:17.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.060+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57a8145c10 con 0x7f57a8072320 2024-09-10T10:49:17.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.060+0000 7f579ea00640 1 -- 172.21.15.28:0/725082032 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5229 (secure 0 0 0) 0x7f57a8145c10 con 0x7f57a8072320 2024-09-10T10:49:17.062 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:17.062 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":60,"btime":"2024-09-10T10:47:39:651000+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:39.650991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":18,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5572,"qdb_cluster":[5572]},"id":4}]} 2024-09-10T10:49:17.062 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 60 2024-09-10T10:49:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.062+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 msgr2=0x7f578406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.062+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 0x7f578406e2f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5790034000 tx=0x7f579002e000 comp rx=0 tx=0).stop 2024-09-10T10:49:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.062+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 msgr2=0x7f57a815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.062+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f579800e9b0 tx=0x7f579800ee70 comp rx=0 tx=0).stop 2024-09-10T10:49:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 shutdown_connections 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f578406be30 0x7f578406e2f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57a8072de0 0x7f57a80adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57a8072700 0x7f57a80b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 --2- 172.21.15.28:0/725082032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57a8072320 0x7f57a815be50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 >> 172.21.15.28:0/725082032 conn(0x7f57a8067d70 msgr2=0x7f57a806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 shutdown_connections 2024-09-10T10:49:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.063+0000 7f57ad600640 1 -- 172.21.15.28:0/725082032 wait complete. 2024-09-10T10:49:17.079 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.234+0000 7effeda00640 1 Processor -- start 2024-09-10T10:49:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.234+0000 7effeda00640 1 -- start start 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effeda00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7effe8072d30 0x7effe806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effeda00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe806f700 0x7effe815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effeda00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effe80cdbe0 con 0x7effe815b300 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effeda00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7effe80728a0 con 0x7effe8072d30 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effeda00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effe806fb30 con 0x7effe806f700 2024-09-10T10:49:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effe6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7effe8072d30 0x7effe806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effe6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7effe8072d30 0x7effe806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54572/0 (socket says 172.21.15.28:54572) 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.235+0000 7effe6000640 1 -- 172.21.15.28:0/182152780 learned_addr learned my addr 172.21.15.28:0/182152780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.236+0000 7effe7400640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe806f700 0x7effe815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.236+0000 7effe6000640 1 -- 172.21.15.28:0/182152780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe806f700 msgr2=0x7effe815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.236+0000 7effe6000640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe806f700 0x7effe815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.236+0000 7effe6000640 1 -- 172.21.15.28:0/182152780 >> v1:172.21.15.138:6789/0 conn(0x7effe815b300 legacy=0x7effe80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:17.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.236+0000 7effe6000640 1 -- 172.21.15.28:0/182152780 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe80cd900 con 0x7effe8072d30 2024-09-10T10:49:17.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.237+0000 7effe6000640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe8072d30 0x7effe806f1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7effe80b57c0 tx=0x7effd402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80acd36ba2540df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.237+0000 7effe5600640 1 -- 172.21.15.28:0/182152780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4037070 con 0x7effe8072d30 2024-09-10T10:49:17.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.237+0000 7effe5600640 1 -- 172.21.15.28:0/182152780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd402ae50 con 0x7effe8072d30 2024-09-10T10:49:17.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.237+0000 7effe5600640 1 -- 172.21.15.28:0/182152780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4032660 con 0x7effe8072d30 2024-09-10T10:49:17.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 -- 172.21.15.28:0/182152780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe8072d30 msgr2=0x7effe806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe8072d30 0x7effe806f1c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7effe80b57c0 tx=0x7effd402a140 comp rx=0 tx=0).stop 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 -- 172.21.15.28:0/182152780 shutdown_connections 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe806f700 0x7effe815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 --2- 172.21.15.28:0/182152780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe8072d30 0x7effe806f1c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 -- 172.21.15.28:0/182152780 >> 172.21.15.28:0/182152780 conn(0x7effe8067d70 msgr2=0x7effe806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 -- 172.21.15.28:0/182152780 shutdown_connections 2024-09-10T10:49:17.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.238+0000 7effeda00640 1 -- 172.21.15.28:0/182152780 wait complete. 2024-09-10T10:49:17.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.240+0000 7effeda00640 1 Processor -- start 2024-09-10T10:49:17.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.240+0000 7effeda00640 1 -- start start 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe8072d30 0x7effe8148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 0x7effe80b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7effe80cd5c0 con 0x7effe806f700 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7effe8072530 con 0x7effe815b300 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effeda00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7effe80cffe0 con 0x7effe8072d30 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54588/0 (socket says 172.21.15.28:54588) 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 -- 172.21.15.28:0/1045801081 learned_addr learned my addr 172.21.15.28:0/1045801081 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6000640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe8072d30 0x7effe8148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe7400640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 0x7effe80b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 -- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe8072d30 msgr2=0x7effe8148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe8072d30 0x7effe8148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 -- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 msgr2=0x7effe80b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 0x7effe80b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe6a00640 1 -- 172.21.15.28:0/1045801081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe81678a0 con 0x7effe806f700 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.241+0000 7effe7400640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 0x7effe80b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effe6a00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7effd800b3f0 tx=0x7effd800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8015020 con 0x7effe806f700 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effd8011830 con 0x7effe806f700 2024-09-10T10:49:17.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd8011c20 con 0x7effe806f700 2024-09-10T10:49:17.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe8167b80 con 0x7effe806f700 2024-09-10T10:49:17.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.242+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe8168140 con 0x7effe806f700 2024-09-10T10:49:17.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.243+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe806f180 con 0x7effe806f700 2024-09-10T10:49:17.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.244+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7effd8018020 con 0x7effe806f700 2024-09-10T10:49:17.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.245+0000 7effdea00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 0x7effc806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.245+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7effd808ad40 con 0x7effe806f700 2024-09-10T10:49:17.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.245+0000 7effe6000640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 0x7effc806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.246+0000 7effe6000640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 0x7effc806e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7effd4034000 tx=0x7effd402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.249+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7effd801f030 con 0x7effe806f700 2024-09-10T10:49:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.468+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effe806ebc0 con 0x7effe806f700 2024-09-10T10:49:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.468+0000 7effdea00640 1 -- 172.21.15.28:0/1045801081 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5229 (secure 0 0 0) 0x7effd80593e0 con 0x7effe806f700 2024-09-10T10:49:17.470 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:17.470 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":60,"btime":"2024-09-10T10:47:39:651000+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:46:44.291411+0000","modified":"2024-09-10T10:47:39.650991+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5572":{"gid":5572,"name":"a","rank":0,"incarnation":57,"state":"up:active","state_seq":18,"addr":"172.21.15.28:6835/2942680174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2942680174},{"type":"v1","addr":"172.21.15.28:6835","nonce":2942680174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5572,"qdb_cluster":[5572]},"id":4}]} 2024-09-10T10:49:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 60 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 msgr2=0x7effc806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 0x7effc806e320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7effd4034000 tx=0x7effd402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 msgr2=0x7effe8168940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7effd800b3f0 tx=0x7effd800b8b0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 shutdown_connections 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effc806be60 0x7effc806e320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effe815b300 0x7effe80b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7effe8072d30 0x7effe8148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 --2- 172.21.15.28:0/1045801081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effe806f700 0x7effe8168940 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.470+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 >> 172.21.15.28:0/1045801081 conn(0x7effe8067d70 msgr2=0x7effe81518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:17.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.471+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 shutdown_connections 2024-09-10T10:49:17.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.471+0000 7effeda00640 1 -- 172.21.15.28:0/1045801081 wait complete. 2024-09-10T10:49:17.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:17.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.642+0000 7f0b7be00640 1 Processor -- start 2024-09-10T10:49:17.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.642+0000 7f0b7be00640 1 -- start start 2024-09-10T10:49:17.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 0x7f0b7c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 0x7f0b7c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b7c0cf000 con 0x7f0b7c15e660 2024-09-10T10:49:17.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0b7c06d3e0 con 0x7f0b7c0d0dd0 2024-09-10T10:49:17.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0b37d0 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.643+0000 7f0b7a000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0b7c15e660 0x7f0b7c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45772/0 (socket says 172.21.15.28:45772) 2024-09-10T10:49:17.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7a000640 1 -- 172.21.15.28:0/3486425703 learned_addr learned my addr 172.21.15.28:0/3486425703 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7aa00640 1 --2- 172.21.15.28:0/3486425703 >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 0x7f0b7c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7b400640 1 --2- 172.21.15.28:0/3486425703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 0x7f0b7c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b79600640 1 -- 172.21.15.28:0/3486425703 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443209204 0 0) 0x7f0b7c0cf000 con 0x7f0b7c15e660 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b79600640 1 -- 172.21.15.28:0/3486425703 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b60003520 con 0x7f0b7c15e660 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7b400640 1 -- 172.21.15.28:0/3486425703 >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 msgr2=0x7f0b7c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7b400640 1 --2- 172.21.15.28:0/3486425703 >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 0x7f0b7c0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7b400640 1 -- 172.21.15.28:0/3486425703 >> v1:172.21.15.28:6789/0 conn(0x7f0b7c15e660 legacy=0x7f0b7c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7b400640 1 -- 172.21.15.28:0/3486425703 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c0d16f0 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.644+0000 7f0b7aa00640 1 --2- 172.21.15.28:0/3486425703 >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 0x7f0b7c0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:17.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.645+0000 7f0b7b400640 1 --2- 172.21.15.28:0/3486425703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 0x7f0b7c0cf7c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009d90 tx=0x7f0b7402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a489c4038f23297a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b79600640 1 -- 172.21.15.28:0/3486425703 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74007740 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b79600640 1 -- 172.21.15.28:0/3486425703 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b740078e0 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b79600640 1 -- 172.21.15.28:0/3486425703 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74007cb0 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 -- 172.21.15.28:0/3486425703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 msgr2=0x7f0b7c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 --2- 172.21.15.28:0/3486425703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 0x7f0b7c0cf7c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009d90 tx=0x7f0b7402a660 comp rx=0 tx=0).stop 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 -- 172.21.15.28:0/3486425703 shutdown_connections 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 --2- 172.21.15.28:0/3486425703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c0d1f90 0x7f0b7c0cf7c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 --2- 172.21.15.28:0/3486425703 >> v2:172.21.15.138:3300/0 conn(0x7f0b7c0d0dd0 0x7f0b7c0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 -- 172.21.15.28:0/3486425703 >> 172.21.15.28:0/3486425703 conn(0x7f0b7c08bfc0 msgr2=0x7f0b7c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 -- 172.21.15.28:0/3486425703 shutdown_connections 2024-09-10T10:49:17.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.646+0000 7f0b7be00640 1 -- 172.21.15.28:0/3486425703 wait complete. 2024-09-10T10:49:17.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.648+0000 7f0b7be00640 1 Processor -- start 2024-09-10T10:49:17.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.648+0000 7f0b7be00640 1 -- start start 2024-09-10T10:49:17.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c0d0dd0 0x7f0b7c15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c15e660 0x7f0b7c1e9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d2c60 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c0cf330 con 0x7f0b7c0d0dd0 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c0d2f40 con 0x7f0b7c15e660 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54604/0 (socket says 172.21.15.28:54604) 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7a000640 1 -- 172.21.15.28:0/2246022473 learned_addr learned my addr 172.21.15.28:0/2246022473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7aa00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c0d0dd0 0x7f0b7c15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.649+0000 7f0b7b400640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c15e660 0x7f0b7c1e9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 -- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c15e660 msgr2=0x7f0b7c1e9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c15e660 0x7f0b7c1e9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 -- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c0d0dd0 msgr2=0x7f0b7c15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c0d0dd0 0x7f0b7c15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 -- 172.21.15.28:0/2246022473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c160b20 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7a000640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c400 tx=0x7f0b6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68017070 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b68005d00 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007310 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7c15fa80 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.650+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b7c160040 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.652+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b7c06d200 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.653+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0b6801a070 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.654+0000 7f0b72a00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 0x7f0b5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:17.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.654+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7f0b6808a540 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.654+0000 7f0b7aa00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 0x7f0b5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:17.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.655+0000 7f0b7aa00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 0x7f0b5c06e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009560 tx=0x7f0b6c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:17.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.659+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b7c06d200 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.848+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b7c0d3220 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.849+0000 7f0b72a00640 1 -- 172.21.15.28:0/2246022473 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17555 (secure 0 0 0) 0x7f0b68058590 con 0x7f0b7c0d1f90 2024-09-10T10:49:17.851 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:17.851 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":74,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:47:36.629915+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:46:42.078607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-10T10:46:43.284849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.2","mappings":[{"from":2,"to":7}]},{"pgid":"10.b","mappings":[{"from":4,"to":5}]},{"pgid":"10.10","mappings":[{"from":2,"to":6}]},{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1a","mappings":[{"from":2,"to":5}]},{"pgid":"10.3b","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6837/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6835/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6837/3429810116":"2024-09-11T10:47:00.608334+0000","172.21.15.138:6835/3429810116":"2024-09-11T10:47:00.608334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:17.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 msgr2=0x7f0b5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 0x7f0b5c06e2f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009560 tx=0x7f0b6c009190 comp rx=0 tx=0).stop 2024-09-10T10:49:17.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 msgr2=0x7f0b7c15b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:17.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c400 tx=0x7f0b6800c8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 shutdown_connections 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0b5c06be30 0x7f0b5c06e2f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0b7c15e660 0x7f0b7c1e9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b7c0d1f90 0x7f0b7c15b7a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 --2- 172.21.15.28:0/2246022473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b7c0d0dd0 0x7f0b7c15f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 >> 172.21.15.28:0/2246022473 conn(0x7f0b7c08bfc0 msgr2=0x7f0b7c15d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 shutdown_connections 2024-09-10T10:49:17.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:17.852+0000 7f0b7be00640 1 -- 172.21.15.28:0/2246022473 wait complete. 2024-09-10T10:49:17.869 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:49:18.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.025+0000 7fe07b400640 1 Processor -- start 2024-09-10T10:49:18.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.025+0000 7fe07b400640 1 -- start start 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe07b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe0740d1540 0x7fe07406dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe07b400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0740d1970 con 0x7fe07415e740 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe07b400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0740d0780 con 0x7fe07409b690 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe07b400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe07415eb20 con 0x7fe0740d1540 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe078a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe07415e740 0x7fe07406d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45782/0 (socket says 172.21.15.28:45782) 2024-09-10T10:49:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.026+0000 7fe078a00640 1 -- 172.21.15.28:0/4213074579 learned_addr learned my addr 172.21.15.28:0/4213074579 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:18.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> v2:172.21.15.138:3300/0 conn(0x7fe0740d1540 0x7fe07406dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273020160 0 0) 0x7fe0740d1970 con 0x7fe07415e740 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe058003520 con 0x7fe07415e740 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511368167 0 0) 0x7fe0740d0780 con 0x7fe07409b690 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0740d1970 con 0x7fe07409b690 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 364793993 0 0) 0x7fe058003520 con 0x7fe07415e740 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 >> v2:172.21.15.138:3300/0 conn(0x7fe0740d1540 msgr2=0x7fe07406dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 --2- 172.21.15.28:0/4213074579 >> v2:172.21.15.138:3300/0 conn(0x7fe0740d1540 0x7fe07406dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 >> v1:172.21.15.138:6789/0 conn(0x7fe07409b690 legacy=0x7fe074070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0740d0a60 con 0x7fe07415e740 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fe064001670 con 0x7fe07415e740 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> v2:172.21.15.138:3300/0 conn(0x7fe0740d1540 0x7fe07406dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 >> v1:172.21.15.28:6789/0 conn(0x7fe07415e740 legacy=0x7fe07406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058003b40 0x7fe058003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0580051c0 0x7fe05800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe07415eb20 con 0x7fe058004800 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe05800bfb0 con 0x7fe058003b40 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.027+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe05800c1b0 con 0x7fe0580051c0 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe078a00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058003b40 0x7fe058003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe079400640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0580051c0 0x7fe05800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 -- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0580051c0 msgr2=0x7fe05800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0580051c0 0x7fe05800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 -- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058003b40 msgr2=0x7fe058003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058003b40 0x7fe058003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe06be00640 1 -- 172.21.15.28:0/4213074579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe058003520 con 0x7fe058004800 2024-09-10T10:49:18.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.028+0000 7fe079400640 1 -- 172.21.15.28:0/4213074579 reap_dead start 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.029+0000 7fe06be00640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 0x7fe058004c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009c60 tx=0x7fe058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4687d71be658d10a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.029+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c036070 con 0x7fe058004800 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.029+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05c031400 con 0x7fe058004800 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.029+0000 7fe073e00640 1 -- 172.21.15.28:0/4213074579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c009860 con 0x7fe058004800 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 -- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 msgr2=0x7fe058004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 0x7fe058004c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009c60 tx=0x7fe058005830 comp rx=0 tx=0).stop 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 -- 172.21.15.28:0/4213074579 shutdown_connections 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 --2- 172.21.15.28:0/4213074579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe058004800 0x7fe058004c80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 -- 172.21.15.28:0/4213074579 >> 172.21.15.28:0/4213074579 conn(0x7fe0740d23a0 msgr2=0x7fe0740d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 -- 172.21.15.28:0/4213074579 shutdown_connections 2024-09-10T10:49:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.030+0000 7fe07b400640 1 -- 172.21.15.28:0/4213074579 wait complete. 2024-09-10T10:49:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.032+0000 7fe07b400640 1 Processor -- start 2024-09-10T10:49:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.032+0000 7fe07b400640 1 -- start start 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 0x7fe0740b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0740cdf30 0x7fe0740ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe07409baa0 con 0x7fe074154c80 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0740d1020 con 0x7fe058004800 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe07b400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe07406cd20 con 0x7fe0740cdf30 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe06be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe06be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54620/0 (socket says 172.21.15.28:54620) 2024-09-10T10:49:18.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe06be00640 1 -- 172.21.15.28:0/3596128197 learned_addr learned my addr 172.21.15.28:0/3596128197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe078a00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 0x7fe0740b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe079400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0740cdf30 0x7fe0740ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.033+0000 7fe06be00640 1 -- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0740cdf30 msgr2=0x7fe0740ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe06be00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0740cdf30 0x7fe0740ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe06be00640 1 -- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 msgr2=0x7fe0740b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe06be00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 0x7fe0740b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe06be00640 1 -- 172.21.15.28:0/3596128197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0740c72b0 con 0x7fe074154c80 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe078a00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 0x7fe0740b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe06be00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d80 tx=0x7fe05c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c040070 con 0x7fe074154c80 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05c03b680 con 0x7fe074154c80 2024-09-10T10:49:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c036040 con 0x7fe074154c80 2024-09-10T10:49:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0740c7530 con 0x7fe074154c80 2024-09-10T10:49:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.034+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0740c7af0 con 0x7fe074154c80 2024-09-10T10:49:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.035+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0740d0780 con 0x7fe074154c80 2024-09-10T10:49:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.036+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe05c043070 con 0x7fe074154c80 2024-09-10T10:49:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.037+0000 7fe071600640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 0x7fe05006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.037+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6476+0+0 (secure 0 0 0) 0x7fe05c0ab830 con 0x7fe074154c80 2024-09-10T10:49:18.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.038+0000 7fe078a00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 0x7fe05006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.039+0000 7fe078a00640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 0x7fe05006e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe07406aa80 tx=0x7fe064009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.042+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe0740c72b0 con 0x7fe074154c80 2024-09-10T10:49:18.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.263+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0740b63d0 con 0x7fe074154c80 2024-09-10T10:49:18.265 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:49:18.264+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:49:18.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.266+0000 7fe071600640 1 -- 172.21.15.28:0/3596128197 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) ==== 161+0+0 (secure 0 0 0) 0x7fe05c079880 con 0x7fe074154c80 2024-09-10T10:49:18.267 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.268+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 msgr2=0x7fe05006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.268+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 0x7fe05006e320 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe07406aa80 tx=0x7fe064009040 comp rx=0 tx=0).stop 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 msgr2=0x7fe0740c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe05c009d80 tx=0x7fe05c004730 comp rx=0 tx=0).stop 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 shutdown_connections 2024-09-10T10:49:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe05006be60 0x7fe05006e320 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0740cdf30 0x7fe0740ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe074154c80 0x7fe0740c6d70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 --2- 172.21.15.28:0/3596128197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe058004800 0x7fe0740b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 >> 172.21.15.28:0/3596128197 conn(0x7fe0740d23a0 msgr2=0x7fe07415c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 shutdown_connections 2024-09-10T10:49:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.269+0000 7fe07b400640 1 -- 172.21.15.28:0/3596128197 wait complete. 2024-09-10T10:49:18.281 INFO:tasks.ceph.mds.a.smithi028.stderr: -4093> 2024-09-10T10:47:41.847+0000 7f7b3fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:49:18.286 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:49:18.330 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:49:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.449+0000 7f1cdd400640 1 Processor -- start 2024-09-10T10:49:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.449+0000 7f1cdd400640 1 -- start start 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1cd8158b50 0x7f1cd80b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1cd80b6360 0x7f1cd809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd815f950 0x7f1cd809af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd80b6820 con 0x7f1cd8158b50 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd80b6b00 con 0x7f1cd80b6360 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cdd400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd80b6de0 con 0x7f1cd815f950 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cd6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1cd8158b50 0x7f1cd80b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cd6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1cd8158b50 0x7f1cd80b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54656/0 (socket says 172.21.15.28:54656) 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cd6800640 1 -- 172.21.15.28:0/2520183853 learned_addr learned my addr 172.21.15.28:0/2520183853 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1ccde00640 1 --2- 172.21.15.28:0/2520183853 >> v2:172.21.15.138:3300/0 conn(0x7f1cd80b6360 0x7f1cd809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.450+0000 7f1cd7200640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd815f950 0x7f1cd809af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.451+0000 7f1cd6800640 1 -- 172.21.15.28:0/2520183853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd815f950 msgr2=0x7f1cd809af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.451+0000 7f1cd6800640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd815f950 0x7f1cd809af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.451+0000 7f1cd6800640 1 -- 172.21.15.28:0/2520183853 >> v2:172.21.15.138:3300/0 conn(0x7f1cd80b6360 msgr2=0x7f1cd809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.451+0000 7f1cd6800640 1 --2- 172.21.15.28:0/2520183853 >> v2:172.21.15.138:3300/0 conn(0x7f1cd80b6360 0x7f1cd809aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.451+0000 7f1cd6800640 1 -- 172.21.15.28:0/2520183853 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd80b70c0 con 0x7f1cd8158b50 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.452+0000 7f1cd6800640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd80b5e20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b18c0 tx=0x7f1cc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5aedfc5a11b149 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.452+0000 7f1cd5e00640 1 -- 172.21.15.28:0/2520183853 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1cd8158b50 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.452+0000 7f1cd5e00640 1 -- 172.21.15.28:0/2520183853 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc8032440 con 0x7f1cd8158b50 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.452+0000 7f1cd5e00640 1 -- 172.21.15.28:0/2520183853 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032810 con 0x7f1cd8158b50 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 -- 172.21.15.28:0/2520183853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 msgr2=0x7f1cd80b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd80b5e20 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b18c0 tx=0x7f1cc802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 -- 172.21.15.28:0/2520183853 shutdown_connections 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd815f950 0x7f1cd809af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 --2- 172.21.15.28:0/2520183853 >> v2:172.21.15.138:3300/0 conn(0x7f1cd80b6360 0x7f1cd809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 --2- 172.21.15.28:0/2520183853 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd80b5e20 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 -- 172.21.15.28:0/2520183853 >> 172.21.15.28:0/2520183853 conn(0x7f1cd80b79e0 msgr2=0x7f1cd80b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 -- 172.21.15.28:0/2520183853 shutdown_connections 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.453+0000 7f1cdd400640 1 -- 172.21.15.28:0/2520183853 wait complete. 2024-09-10T10:49:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.454+0000 7f1cdd400640 1 Processor -- start 2024-09-10T10:49:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.454+0000 7f1cdd400640 1 -- start start 2024-09-10T10:49:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 0x7f1cd8159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cd815f950 0x7f1cd814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd81590b0 con 0x7f1cd8158b50 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd8158f30 con 0x7f1cd815f950 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.455+0000 7f1cdd400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd8162220 con 0x7f1cd80b6360 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54668/0 (socket says 172.21.15.28:54668) 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 -- 172.21.15.28:0/1938545140 learned_addr learned my addr 172.21.15.28:0/1938545140 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1cd7200640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cd815f950 0x7f1cd814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1cd6800640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 0x7f1cd8159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 -- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 msgr2=0x7f1cd8159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 0x7f1cd8159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 -- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cd815f950 msgr2=0x7f1cd814ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cd815f950 0x7f1cd814ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 -- 172.21.15.28:0/1938545140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd809eda0 con 0x7f1cd8158b50 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1cd6800640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 0x7f1cd8159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.456+0000 7f1ccde00640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e870 tx=0x7f1cc000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.457+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0017070 con 0x7f1cd8158b50 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.457+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc000c940 con 0x7f1cd8158b50 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.457+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000cd10 con 0x7f1cd8158b50 2024-09-10T10:49:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.457+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd809f080 con 0x7f1cd8158b50 2024-09-10T10:49:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.457+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd809f640 con 0x7f1cd8158b50 2024-09-10T10:49:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.458+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd80b5e20 con 0x7f1cd8158b50 2024-09-10T10:49:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.459+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1cc001a070 con 0x7f1cd8158b50 2024-09-10T10:49:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.460+0000 7f1ccf400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.460+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6562+0+0 (secure 0 0 0) 0x7f1cc008b430 con 0x7f1cd8158b50 2024-09-10T10:49:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.460+0000 7f1cd6800640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:18.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.461+0000 7f1cd6800640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 0x7f1cb406e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:18.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.464+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1cc0059420 con 0x7f1cd8158b50 2024-09-10T10:49:18.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:18.692+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1cd80b77a0 con 0x7f1cd8158b50 2024-09-10T10:49:19.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.271+0000 7f1ccf400640 1 -- 172.21.15.28:0/1938545140 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) ==== 104+0+0 (secure 0 0 0) 0x7f1cc005d720 con 0x7f1cd8158b50 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 msgr2=0x7f1cb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 0x7f1cb406e2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8034000 tx=0x7f1cc802e000 comp rx=0 tx=0).stop 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 msgr2=0x7f1cd809fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000e870 tx=0x7f1cc000ed30 comp rx=0 tx=0).stop 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 shutdown_connections 2024-09-10T10:49:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cb406be30 0x7f1cb406e2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cd815f950 0x7f1cd814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1cd8158b50 0x7f1cd809fe40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 --2- 172.21.15.28:0/1938545140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cd80b6360 0x7f1cd8159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 >> 172.21.15.28:0/1938545140 conn(0x7f1cd80b79e0 msgr2=0x7f1cd80b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 shutdown_connections 2024-09-10T10:49:19.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.274+0000 7f1cdd400640 1 -- 172.21.15.28:0/1938545140 wait complete. 2024-09-10T10:49:19.294 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:49:19.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.496+0000 7f7e0a600640 1 Processor -- start 2024-09-10T10:49:19.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.496+0000 7f7e0a600640 1 -- start start 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e0a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7e040ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e0a600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e0406ee70 con 0x7f7e04072070 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e0a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e0406f130 con 0x7f7e0415c430 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e0a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e0406f410 con 0x7f7e0406ea60 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e03400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7e04072070 0x7f7e04072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45792/0 (socket says 172.21.15.28:45792) 2024-09-10T10:49:19.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e03400640 1 -- 172.21.15.28:0/1113958135 learned_addr learned my addr 172.21.15.28:0/1113958135 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7e040ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052082982 0 0) 0x7f7e0406ee70 con 0x7f7e04072070 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.498+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7de8003520 con 0x7f7e04072070 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2359090178 0 0) 0x7f7de8003520 con 0x7f7e04072070 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 msgr2=0x7f7e040ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7e040ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 >> v1:172.21.15.138:6789/0 conn(0x7f7e0415c430 legacy=0x7f7e0415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0415c9e0 con 0x7f7e04072070 2024-09-10T10:49:19.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f7df4001670 con 0x7f7e04072070 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 >> v1:172.21.15.28:6789/0 conn(0x7f7e04072070 legacy=0x7f7e04072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7de80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de8003fb0 0x7f7de8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0406f410 con 0x7f7de8004d40 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7de800bb00 con 0x7f7de8003fb0 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7de800bd00 con 0x7f7e0406ea60 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7e03400640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7de80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.499+0000 7f7dfbe00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de8003fb0 0x7f7de8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.500+0000 7f7e03e00640 1 -- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 msgr2=0x7f7de80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.500+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e0406ea60 0x7f7de80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.500+0000 7f7e03e00640 1 -- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de8003fb0 msgr2=0x7f7de8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:19.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.500+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7de8003fb0 0x7f7de8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.500+0000 7f7e03e00640 1 -- 172.21.15.28:0/1113958135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de8003520 con 0x7f7de8004d40 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7de80051c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009fc0 tx=0x7f7de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824078672441d3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc009c10 con 0x7f7de8004d40 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dfc036070 con 0x7f7de8004d40 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e02a00640 1 -- 172.21.15.28:0/1113958135 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc031880 con 0x7f7de8004d40 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e0a600640 1 -- 172.21.15.28:0/1113958135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 msgr2=0x7f7de80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1113958135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7de80051c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc009fc0 tx=0x7f7de8005830 comp rx=0 tx=0).stop 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e03e00640 1 -- 172.21.15.28:0/1113958135 reap_dead start 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e0a600640 1 -- 172.21.15.28:0/1113958135 shutdown_connections 2024-09-10T10:49:19.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.501+0000 7f7e0a600640 1 -- 172.21.15.28:0/1113958135 >> 172.21.15.28:0/1113958135 conn(0x7f7e04067d70 msgr2=0x7f7e0406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:19.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.502+0000 7f7e0a600640 1 -- 172.21.15.28:0/1113958135 shutdown_connections 2024-09-10T10:49:19.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.502+0000 7f7e0a600640 1 -- 172.21.15.28:0/1113958135 wait complete. 2024-09-10T10:49:19.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.503+0000 7f7e0a600640 1 Processor -- start 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.503+0000 7f7e0a600640 1 -- start start 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e04152780 0x7f7e041f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e041f1ba0 0x7f7e040b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7e040ce7f0 con 0x7f7de8004d40 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0415cfa0 con 0x7f7e04152780 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.504+0000 7f7e0a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e040ce970 con 0x7f7e041f1ba0 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54694/0 (socket says 172.21.15.28:54694) 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 -- 172.21.15.28:0/1568579428 learned_addr learned my addr 172.21.15.28:0/1568579428 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03e00640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e041f1ba0 0x7f7e040b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7dfbe00640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e04152780 0x7f7e041f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 -- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e041f1ba0 msgr2=0x7f7e040b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e041f1ba0 0x7f7e040b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 -- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e04152780 msgr2=0x7f7e041f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e04152780 0x7f7e041f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 -- 172.21.15.28:0/1568579428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e041f2220 con 0x7f7de8004d40 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7e03400640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7df40072f0 tx=0x7f7df40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.505+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df400f960 con 0x7f7de8004d40 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.506+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df4018070 con 0x7f7de8004d40 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.506+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df400fd60 con 0x7f7de8004d40 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.506+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e041f2500 con 0x7f7de8004d40 2024-09-10T10:49:19.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.506+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e041f2ac0 con 0x7f7de8004d40 2024-09-10T10:49:19.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.507+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e041f1f80 con 0x7f7de8004d40 2024-09-10T10:49:19.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.508+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7df401d460 con 0x7f7de8004d40 2024-09-10T10:49:19.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.509+0000 7f7dfb400640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 0x7f7de006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:19.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.509+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6562+0+0 (secure 0 0 0) 0x7f7df408b380 con 0x7f7de8004d40 2024-09-10T10:49:19.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.509+0000 7f7dfbe00640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 0x7f7de006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:19.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.510+0000 7f7dfbe00640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 0x7f7de006e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7dec000c00 tx=0x7f7dec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:19.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.513+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7df40599c0 con 0x7f7de8004d40 2024-09-10T10:49:19.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:19.697+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e04072450 con 0x7f7de8004d40 2024-09-10T10:49:20.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.662+0000 7f7dfb400640 1 -- 172.21.15.28:0/1568579428 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7f7df40592c0 con 0x7f7de8004d40 2024-09-10T10:49:20.663 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.666+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 msgr2=0x7f7de006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.666+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 0x7f7de006e2f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7dec000c00 tx=0x7f7dec009040 comp rx=0 tx=0).stop 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.666+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 msgr2=0x7f7e04155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7df40072f0 tx=0x7f7df40077b0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 shutdown_connections 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7de006be30 0x7f7de006e2f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e041f1ba0 0x7f7e040b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e04152780 0x7f7e041f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 --2- 172.21.15.28:0/1568579428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7de8004d40 0x7f7e04155340 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 >> 172.21.15.28:0/1568579428 conn(0x7f7e04067d70 msgr2=0x7f7e04160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 shutdown_connections 2024-09-10T10:49:20.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.667+0000 7f7e0a600640 1 -- 172.21.15.28:0/1568579428 wait complete. 2024-09-10T10:49:20.688 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:49:20.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.852+0000 7f7f9c200640 1 Processor -- start 2024-09-10T10:49:20.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.852+0000 7f7f9c200640 1 -- start start 2024-09-10T10:49:20.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.853+0000 7f7f9c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f940b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.853+0000 7f7f9c200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f940cd1f0 0x7f7f94154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.853+0000 7f7f9c200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f940b3f20 con 0x7f7f9415eaa0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.853+0000 7f7f9c200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7f9415f440 con 0x7f7f940cd1f0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.853+0000 7f7f9c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f9415f720 con 0x7f7f940b4ae0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f98e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7f9415eaa0 0x7f7f940ccae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45806/0 (socket says 172.21.15.28:45806) 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f98e00640 1 -- 172.21.15.28:0/1337951689 learned_addr learned my addr 172.21.15.28:0/1337951689 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f9a200640 1 --2- 172.21.15.28:0/1337951689 >> v2:172.21.15.138:3300/0 conn(0x7f7f940cd1f0 0x7f7f94154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f99800640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f940b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264610486 0 0) 0x7f7f940b3f20 con 0x7f7f9415eaa0 2024-09-10T10:49:20.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f7c003520 con 0x7f7f9415eaa0 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4095853441 0 0) 0x7f7f7c003520 con 0x7f7f9415eaa0 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 msgr2=0x7f7f940b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f940b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 >> v2:172.21.15.138:3300/0 conn(0x7f7f940cd1f0 msgr2=0x7f7f94154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 --2- 172.21.15.28:0/1337951689 >> v2:172.21.15.138:3300/0 conn(0x7f7f940cd1f0 0x7f7f94154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f940b31e0 con 0x7f7f9415eaa0 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.854+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f7f80001670 con 0x7f7f9415eaa0 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 >> v1:172.21.15.28:6789/0 conn(0x7f7f9415eaa0 legacy=0x7f7f940ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f7c004800 0x7f7f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9415f440 con 0x7f7f7c0039b0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9415f720 con 0x7f7f7c004800 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c00b9c0 con 0x7f7f940b4ae0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f98e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f7c004800 0x7f7f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f9a200640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 -- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 msgr2=0x7f7f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940b4ae0 0x7f7f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 -- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f7c004800 msgr2=0x7f7f7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f7c004800 0x7f7f7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f99800640 1 -- 172.21.15.28:0/1337951689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c003520 con 0x7f7f7c0039b0 2024-09-10T10:49:20.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.855+0000 7f7f98e00640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f7c004800 0x7f7f7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.856+0000 7f7f99800640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f7c003dd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009ac0 tx=0x7f7f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5ee8de7a9c9d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.856+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88036070 con 0x7f7f7c0039b0 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.856+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f88031400 con 0x7f7f7c0039b0 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.856+0000 7f7f93e00640 1 -- 172.21.15.28:0/1337951689 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88009690 con 0x7f7f7c0039b0 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 -- 172.21.15.28:0/1337951689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 msgr2=0x7f7f7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 --2- 172.21.15.28:0/1337951689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f7c003dd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009ac0 tx=0x7f7f7c005830 comp rx=0 tx=0).stop 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9a200640 1 -- 172.21.15.28:0/1337951689 reap_dead start 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 -- 172.21.15.28:0/1337951689 shutdown_connections 2024-09-10T10:49:20.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 -- 172.21.15.28:0/1337951689 >> 172.21.15.28:0/1337951689 conn(0x7f7f9406bdf0 msgr2=0x7f7f9406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 -- 172.21.15.28:0/1337951689 shutdown_connections 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.857+0000 7f7f9c200640 1 -- 172.21.15.28:0/1337951689 wait complete. 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.859+0000 7f7f9c200640 1 Processor -- start 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.859+0000 7f7f9c200640 1 -- start start 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 0x7f7f94159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 0x7f7f940c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7f940d09a0 con 0x7f7f7c0039b0 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f940d0820 con 0x7f7f940ad3a0 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f9415eec0 con 0x7f7f9415fe00 2024-09-10T10:49:20.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54712/0 (socket says 172.21.15.28:54712) 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 -- 172.21.15.28:0/3300732307 learned_addr learned my addr 172.21.15.28:0/3300732307 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9a200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 0x7f7f940c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f98e00640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 0x7f7f94159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 -- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 msgr2=0x7f7f940c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 0x7f7f940c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 -- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 msgr2=0x7f7f94159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 0x7f7f94159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f99800640 1 -- 172.21.15.28:0/3300732307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f940aef20 con 0x7f7f7c0039b0 2024-09-10T10:49:20.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f9a200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 0x7f7f940c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.860+0000 7f7f98e00640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 0x7f7f94159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f99800640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009ac0 tx=0x7f7f88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88040020 con 0x7f7f7c0039b0 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f88003bc0 con 0x7f7f7c0039b0 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88036040 con 0x7f7f7c0039b0 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f940ad960 con 0x7f7f7c0039b0 2024-09-10T10:49:20.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.861+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f940ae200 con 0x7f7f7c0039b0 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.862+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f940b39e0 con 0x7f7f7c0039b0 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.863+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f88043020 con 0x7f7f7c0039b0 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.864+0000 7f7f91600640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 0x7f7f7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.865+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6124+0+0 (secure 0 0 0) 0x7f7f880ab170 con 0x7f7f7c0039b0 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.865+0000 7f7f98e00640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 0x7f7f7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:20.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.866+0000 7f7f98e00640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 0x7f7f7406e320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f940ad7a0 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:20.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:20.869+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f8804a070 con 0x7f7f7c0039b0 2024-09-10T10:49:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.054+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f941617e0 con 0x7f7f7c0039b0 2024-09-10T10:49:21.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.664+0000 7f7f91600640 1 -- 172.21.15.28:0/3300732307 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7f7f880044e0 con 0x7f7f7c0039b0 2024-09-10T10:49:21.666 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:49:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.668+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 msgr2=0x7f7f7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.668+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 0x7f7f7406e320 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7f940ad7a0 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2024-09-10T10:49:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.668+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 msgr2=0x7f7f940ae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.668+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009ac0 tx=0x7f7f88004730 comp rx=0 tx=0).stop 2024-09-10T10:49:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.669+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 shutdown_connections 2024-09-10T10:49:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.669+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7406be60 0x7f7f7406e320 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.669+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415fe00 0x7f7f940c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.669+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f940ad3a0 0x7f7f94159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.669+0000 7f7f9c200640 1 --2- 172.21.15.28:0/3300732307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f7c0039b0 0x7f7f940ae9e0 secure :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009ac0 tx=0x7f7f88004730 comp rx=0 tx=0).stop 2024-09-10T10:49:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.670+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 >> 172.21.15.28:0/3300732307 conn(0x7f7f9406bdf0 msgr2=0x7f7f9406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.671+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 shutdown_connections 2024-09-10T10:49:21.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.671+0000 7f7f9c200640 1 -- 172.21.15.28:0/3300732307 wait complete. 2024-09-10T10:49:21.695 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:49:21.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.860+0000 7ffa60e00640 1 Processor -- start 2024-09-10T10:49:21.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.860+0000 7ffa60e00640 1 -- start start 2024-09-10T10:49:21.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.861+0000 7ffa60e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa5c158ab0 0x7ffa5c0722b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.861+0000 7ffa60e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 0x7ffa5c0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.861+0000 7ffa60e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5c158e90 con 0x7ffa5c06e310 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.861+0000 7ffa60e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5c161e20 con 0x7ffa5c158ab0 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.861+0000 7ffa60e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c06efb0 con 0x7ffa5c15faf0 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa5c158ab0 0x7ffa5c0722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa5a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffa5c06e310 0x7ffa5c06e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45822/0 (socket says 172.21.15.28:45822) 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa5a800640 1 -- 172.21.15.28:0/1137337339 learned_addr learned my addr 172.21.15.28:0/1137337339 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59e00640 1 -- 172.21.15.28:0/1137337339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 msgr2=0x7ffa5c0727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:21.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59400640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 0x7ffa5c0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59e00640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 0x7ffa5c0727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa58a00640 1 -- 172.21.15.28:0/1137337339 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2920854514 0 0) 0x7ffa5c158e90 con 0x7ffa5c06e310 2024-09-10T10:49:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59e00640 1 -- 172.21.15.28:0/1137337339 >> v1:172.21.15.28:6789/0 conn(0x7ffa5c06e310 legacy=0x7ffa5c06e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59e00640 1 -- 172.21.15.28:0/1137337339 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c06f290 con 0x7ffa5c158ab0 2024-09-10T10:49:21.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.862+0000 7ffa59400640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 0x7ffa5c0727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.863+0000 7ffa59e00640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c158ab0 0x7ffa5c0722b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63cbf02734d90ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.863+0000 7ffa58a00640 1 -- 172.21.15.28:0/1137337339 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c037070 con 0x7ffa5c158ab0 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.863+0000 7ffa58a00640 1 -- 172.21.15.28:0/1137337339 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c032440 con 0x7ffa5c158ab0 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.863+0000 7ffa58a00640 1 -- 172.21.15.28:0/1137337339 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c032810 con 0x7ffa5c158ab0 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 -- 172.21.15.28:0/1137337339 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c158ab0 msgr2=0x7ffa5c0722b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c158ab0 0x7ffa5c0722b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097a0 tx=0x7ffa4c02a180 comp rx=0 tx=0).stop 2024-09-10T10:49:21.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 -- 172.21.15.28:0/1137337339 shutdown_connections 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c15faf0 0x7ffa5c0727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 --2- 172.21.15.28:0/1137337339 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c158ab0 0x7ffa5c0722b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 -- 172.21.15.28:0/1137337339 >> 172.21.15.28:0/1137337339 conn(0x7ffa5c06bc10 msgr2=0x7ffa5c06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 -- 172.21.15.28:0/1137337339 shutdown_connections 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.864+0000 7ffa60e00640 1 -- 172.21.15.28:0/1137337339 wait complete. 2024-09-10T10:49:21.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.866+0000 7ffa60e00640 1 Processor -- start 2024-09-10T10:49:21.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.866+0000 7ffa60e00640 1 -- start start 2024-09-10T10:49:21.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 0x7ffa5c1e9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 0x7ffa5c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0ae970 con 0x7ffa5c15faf0 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0aec60 con 0x7ffa5c06e310 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa60e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c0af180 con 0x7ffa5c158ab0 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54724/0 (socket says 172.21.15.28:54724) 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 -- 172.21.15.28:0/2604298013 learned_addr learned my addr 172.21.15.28:0/2604298013 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa59400640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 0x7ffa5c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa59e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 0x7ffa5c1e9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 -- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 msgr2=0x7ffa5c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 0x7ffa5c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 -- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 msgr2=0x7ffa5c1e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 0x7ffa5c1e9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:21.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa5a800640 1 -- 172.21.15.28:0/2604298013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1e7230 con 0x7ffa5c15faf0 2024-09-10T10:49:21.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa59400640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 0x7ffa5c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:21.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.867+0000 7ffa59e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 0x7ffa5c1e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:21.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa5a800640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c7e0 tx=0x7ffa5400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:21.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54018070 con 0x7ffa5c15faf0 2024-09-10T10:49:21.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5400a640 con 0x7ffa5c15faf0 2024-09-10T10:49:21.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400aa30 con 0x7ffa5c15faf0 2024-09-10T10:49:21.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1e7510 con 0x7ffa5c15faf0 2024-09-10T10:49:21.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.868+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1e7ad0 con 0x7ffa5c15faf0 2024-09-10T10:49:21.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.869+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c06e780 con 0x7ffa5c15faf0 2024-09-10T10:49:21.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.870+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa5401b070 con 0x7ffa5c15faf0 2024-09-10T10:49:21.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.871+0000 7ffa52000640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 0x7ffa3c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:21.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.871+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7ffa5408a4e0 con 0x7ffa5c15faf0 2024-09-10T10:49:21.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.871+0000 7ffa59e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 0x7ffa3c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:21.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.872+0000 7ffa59e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 0x7ffa3c06e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c034000 tx=0x7ffa4c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:21.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:21.875+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa54058eb0 con 0x7ffa5c15faf0 2024-09-10T10:49:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.063+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ffa5c0cd790 con 0x7ffa5c15faf0 2024-09-10T10:49:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.067+0000 7ffa52000640 1 -- 172.21.15.28:0/2604298013 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7ffa540587b0 con 0x7ffa5c15faf0 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.069+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 msgr2=0x7ffa3c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.069+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 0x7ffa3c06e2f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c034000 tx=0x7ffa4c02e000 comp rx=0 tx=0).stop 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.069+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 msgr2=0x7ffa5c0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.069+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400c7e0 tx=0x7ffa5400cca0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 shutdown_connections 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa3c06be30 0x7ffa3c06e2f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c15faf0 0x7ffa5c0b1e80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa5c158ab0 0x7ffa5c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 --2- 172.21.15.28:0/2604298013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa5c06e310 0x7ffa5c1e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 >> 172.21.15.28:0/2604298013 conn(0x7ffa5c06bc10 msgr2=0x7ffa5c0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 shutdown_connections 2024-09-10T10:49:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.070+0000 7ffa60e00640 1 -- 172.21.15.28:0/2604298013 wait complete. 2024-09-10T10:49:22.086 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_journal_corrupt_dentry_first 2024-09-10T10:49:22.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.250+0000 7fefce200640 1 Processor -- start 2024-09-10T10:49:22.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.250+0000 7fefce200640 1 -- start start 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefce200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefc8158ba0 0x7fefc80732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefce200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc8071cd0 con 0x7fefc806f420 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefce200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefc815b710 con 0x7fefc8073d10 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefce200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fefc815b4b0 con 0x7fefc8158ba0 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefcd800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fefc806f420 0x7fefc80b2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45824/0 (socket says 172.21.15.28:45824) 2024-09-10T10:49:22.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefcce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefc8158ba0 0x7fefc80732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.251+0000 7fefcd800640 1 -- 172.21.15.28:0/922811326 learned_addr learned my addr 172.21.15.28:0/922811326 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.252+0000 7fefcce00640 1 -- 172.21.15.28:0/922811326 >> v1:172.21.15.138:6789/0 conn(0x7fefc8073d10 legacy=0x7fefc806ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.252+0000 7fefcce00640 1 -- 172.21.15.28:0/922811326 >> v1:172.21.15.28:6789/0 conn(0x7fefc806f420 legacy=0x7fefc80b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.252+0000 7fefc7400640 1 -- 172.21.15.28:0/922811326 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819518377 0 0) 0x7fefc8071cd0 con 0x7fefc806f420 2024-09-10T10:49:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.252+0000 7fefcce00640 1 -- 172.21.15.28:0/922811326 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc815b9f0 con 0x7fefc8158ba0 2024-09-10T10:49:22.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.253+0000 7fefcce00640 1 --2- 172.21.15.28:0/922811326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc8158ba0 0x7fefc80732d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5566b730bf70655d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.253+0000 7fefc7400640 1 -- 172.21.15.28:0/922811326 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc037070 con 0x7fefc8158ba0 2024-09-10T10:49:22.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.253+0000 7fefc7400640 1 -- 172.21.15.28:0/922811326 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc032440 con 0x7fefc8158ba0 2024-09-10T10:49:22.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.253+0000 7fefc7400640 1 -- 172.21.15.28:0/922811326 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc032810 con 0x7fefc8158ba0 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 -- 172.21.15.28:0/922811326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc8158ba0 msgr2=0x7fefc80732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 --2- 172.21.15.28:0/922811326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc8158ba0 0x7fefc80732d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc02a230 comp rx=0 tx=0).stop 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 -- 172.21.15.28:0/922811326 shutdown_connections 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 --2- 172.21.15.28:0/922811326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc8158ba0 0x7fefc80732d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 -- 172.21.15.28:0/922811326 >> 172.21.15.28:0/922811326 conn(0x7fefc806c240 msgr2=0x7fefc80b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:22.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 -- 172.21.15.28:0/922811326 shutdown_connections 2024-09-10T10:49:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.254+0000 7fefce200640 1 -- 172.21.15.28:0/922811326 wait complete. 2024-09-10T10:49:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.256+0000 7fefce200640 1 Processor -- start 2024-09-10T10:49:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.256+0000 7fefce200640 1 -- start start 2024-09-10T10:49:22.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc8073d10 0x7fefc8150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefc8158ba0 0x7fefc815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fefc8073810 con 0x7fefc806f420 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefc815bfb0 con 0x7fefc8073d10 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefce200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fefc8073990 con 0x7fefc8158ba0 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54742/0 (socket says 172.21.15.28:54742) 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 -- 172.21.15.28:0/1589451897 learned_addr learned my addr 172.21.15.28:0/1589451897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefc7e00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc8073d10 0x7fefc8150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcd800640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefc8158ba0 0x7fefc815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 -- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefc8158ba0 msgr2=0x7fefc815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefc8158ba0 0x7fefc815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 -- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc8073d10 msgr2=0x7fefc8150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc8073d10 0x7fefc8150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.257+0000 7fefcce00640 1 -- 172.21.15.28:0/1589451897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc815ada0 con 0x7fefc806f420 2024-09-10T10:49:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefcce00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc037040 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefbc004250 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefbc032e40 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc8159570 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.258+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc8159b30 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.259+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc8157850 con 0x7fefc806f420 2024-09-10T10:49:22.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.260+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefbc0306f0 con 0x7fefc806f420 2024-09-10T10:49:22.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.261+0000 7fefc4c00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 0x7fefa806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.262+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7fefbc0ac070 con 0x7fefc806f420 2024-09-10T10:49:22.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.262+0000 7fefc7e00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 0x7fefa806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.263+0000 7fefc7e00640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 0x7fefa806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc81f6110 tx=0x7fefb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.266+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fefbc07aad0 con 0x7fefc806f420 2024-09-10T10:49:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.450+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"} v 0) -- 0x7fefc8073250 con 0x7fefc806f420 2024-09-10T10:49:22.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.451+0000 7fefc4c00640 1 -- 172.21.15.28:0/1589451897 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_journal_corrupt_dentry_first"}]=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7fefbc07a3d0 con 0x7fefc806f420 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 msgr2=0x7fefa806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 0x7fefa806e2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc81f6110 tx=0x7fefb4009040 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 msgr2=0x7fefc80cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0097a0 tx=0x7fefbc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 shutdown_connections 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fefa806be30 0x7fefa806e2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefc8158ba0 0x7fefc815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefc8073d10 0x7fefc8150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 --2- 172.21.15.28:0/1589451897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefc806f420 0x7fefc80cf090 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.453+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 >> 172.21.15.28:0/1589451897 conn(0x7fefc806c240 msgr2=0x7fefc8072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:22.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.454+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 shutdown_connections 2024-09-10T10:49:22.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.454+0000 7fefce200640 1 -- 172.21.15.28:0/1589451897 wait complete. 2024-09-10T10:49:22.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit' 2024-09-10T10:49:22.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.626+0000 7fb173600640 1 Processor -- start 2024-09-10T10:49:22.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.626+0000 7fb173600640 1 -- start start 2024-09-10T10:49:22.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb173600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb16c161c80 0x7fb16c06e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb173600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb16c162060 0x7fb16c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb173600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16c06e870 con 0x7fb16c0b64b0 2024-09-10T10:49:22.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb173600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb16c06d9f0 con 0x7fb16c162060 2024-09-10T10:49:22.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb173600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb16c06dcd0 con 0x7fb16c161c80 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb16be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb16c162060 0x7fb16c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb171600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb16c0b64b0 0x7fb16c162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45832/0 (socket says 172.21.15.28:45832) 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.627+0000 7fb171600640 1 -- 172.21.15.28:0/3436860524 learned_addr learned my addr 172.21.15.28:0/3436860524 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb170c00640 1 --2- 172.21.15.28:0/3436860524 >> v2:172.21.15.138:3300/0 conn(0x7fb16c161c80 0x7fb16c06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb16be00640 1 -- 172.21.15.28:0/3436860524 >> v2:172.21.15.138:3300/0 conn(0x7fb16c161c80 msgr2=0x7fb16c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:22.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb16be00640 1 --2- 172.21.15.28:0/3436860524 >> v2:172.21.15.138:3300/0 conn(0x7fb16c161c80 0x7fb16c06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb16be00640 1 -- 172.21.15.28:0/3436860524 >> v1:172.21.15.28:6789/0 conn(0x7fb16c0b64b0 legacy=0x7fb16c162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb16be00640 1 -- 172.21.15.28:0/3436860524 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c06dfb0 con 0x7fb16c162060 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.628+0000 7fb16b400640 1 -- 172.21.15.28:0/3436860524 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898523183 0 0) 0x7fb164001490 con 0x7fb16c0b64b0 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.629+0000 7fb16be00640 1 --2- 172.21.15.28:0/3436860524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c162060 0x7fb16c0b5f70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb16c08d7a0 tx=0x7fb15802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687fe082a4db60a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.629+0000 7fb16b400640 1 -- 172.21.15.28:0/3436860524 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158037070 con 0x7fb16c162060 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.629+0000 7fb16b400640 1 -- 172.21.15.28:0/3436860524 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158032440 con 0x7fb16c162060 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.629+0000 7fb16b400640 1 -- 172.21.15.28:0/3436860524 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158032810 con 0x7fb16c162060 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 -- 172.21.15.28:0/3436860524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c162060 msgr2=0x7fb16c0b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 --2- 172.21.15.28:0/3436860524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c162060 0x7fb16c0b5f70 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb16c08d7a0 tx=0x7fb15802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 -- 172.21.15.28:0/3436860524 shutdown_connections 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 --2- 172.21.15.28:0/3436860524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c162060 0x7fb16c0b5f70 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 --2- 172.21.15.28:0/3436860524 >> v2:172.21.15.138:3300/0 conn(0x7fb16c161c80 0x7fb16c06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 -- 172.21.15.28:0/3436860524 >> 172.21.15.28:0/3436860524 conn(0x7fb16c0d44f0 msgr2=0x7fb16c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:22.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 -- 172.21.15.28:0/3436860524 shutdown_connections 2024-09-10T10:49:22.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.630+0000 7fb173600640 1 -- 172.21.15.28:0/3436860524 wait complete. 2024-09-10T10:49:22.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.632+0000 7fb173600640 1 Processor -- start 2024-09-10T10:49:22.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.632+0000 7fb173600640 1 -- start start 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 0x7fb16c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb16c162060 0x7fb16c1f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0d2a10 con 0x7fb16c0b64b0 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0d2890 con 0x7fb16c162060 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb173600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb16c0b6960 con 0x7fb16c161c80 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54764/0 (socket says 172.21.15.28:54764) 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 -- 172.21.15.28:0/664849688 learned_addr learned my addr 172.21.15.28:0/664849688 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb171600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb16c162060 0x7fb16c1f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb16be00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 0x7fb16c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 -- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 msgr2=0x7fb16c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 0x7fb16c0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 -- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb16c162060 msgr2=0x7fb16c1f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb16c162060 0x7fb16c1f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb170c00640 1 -- 172.21.15.28:0/664849688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c1f85f0 con 0x7fb16c0b64b0 2024-09-10T10:49:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.633+0000 7fb16be00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 0x7fb16c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb170c00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e9b0 tx=0x7fb16000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb160017070 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16000c9a0 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16000cd90 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c0cc370 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.634+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c0cc930 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.635+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c1613e0 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.636+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb16001a070 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.637+0000 7fb168c00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.637+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7fb16008acf0 con 0x7fb16c0b64b0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.637+0000 7fb16be00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.638+0000 7fb16be00640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 0x7fb14c06e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.641+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb16c1613e0 con 0x7fb16c0b64b0 2024-09-10T10:49:22.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:22.897+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]} v 0) -- 0x7fb16c0ba470 con 0x7fb16c0b64b0 2024-09-10T10:49:23.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.678+0000 7fb168c00640 1 -- 172.21.15.28:0/664849688 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_precommit"]}]=0 v273) ==== 142+0+0 (secure 0 0 0) 0x7fb160059100 con 0x7fb16c0b64b0 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 msgr2=0x7fb14c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 0x7fb14c06e2f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb158034000 tx=0x7fb15802e000 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 msgr2=0x7fb16c156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb16000e9b0 tx=0x7fb16000ee70 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 shutdown_connections 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb14c06be30 0x7fb14c06e2f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb16c162060 0x7fb16c1f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb16c161c80 0x7fb16c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 --2- 172.21.15.28:0/664849688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb16c0b64b0 0x7fb16c156660 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 >> 172.21.15.28:0/664849688 conn(0x7fb16c0d44f0 msgr2=0x7fb16c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.681+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 shutdown_connections 2024-09-10T10:49:23.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.682+0000 7fb173600640 1 -- 172.21.15.28:0/664849688 wait complete. 2024-09-10T10:49:23.702 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-10T10:49:23.702 INFO:tasks.cephfs_test_runner:Starting test: test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) That the MDS won't write a dentry with new damage to CDentry::first 2024-09-10T10:49:23.703 INFO:tasks.cephfs_test_runner:test_dentry_first_preflush (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:49:23.703 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.863+0000 7fc67a400640 1 Processor -- start 2024-09-10T10:49:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.863+0000 7fc67a400640 1 -- start start 2024-09-10T10:49:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc67a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc6740b40e0 0x7fc6740aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc67a400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6740b5470 con 0x7fc67415bef0 2024-09-10T10:49:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc67a400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6740b3640 con 0x7fc6740af1b0 2024-09-10T10:49:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc67a400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc6740b5190 con 0x7fc6740b40e0 2024-09-10T10:49:23.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc679000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc67415bef0 0x7fc6740b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48060/0 (socket says 172.21.15.28:48060) 2024-09-10T10:49:23.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc679000640 1 -- 172.21.15.28:0/2279606813 learned_addr learned my addr 172.21.15.28:0/2279606813 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.864+0000 7fc673e00640 1 --2- 172.21.15.28:0/2279606813 >> v2:172.21.15.138:3300/0 conn(0x7fc6740b40e0 0x7fc6740aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 667510867 0 0) 0x7fc6740b5470 con 0x7fc67415bef0 2024-09-10T10:49:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc65c003520 con 0x7fc67415bef0 2024-09-10T10:49:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544406976 0 0) 0x7fc65c003520 con 0x7fc67415bef0 2024-09-10T10:49:23.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 >> v2:172.21.15.138:3300/0 conn(0x7fc6740b40e0 msgr2=0x7fc6740aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 --2- 172.21.15.28:0/2279606813 >> v2:172.21.15.138:3300/0 conn(0x7fc6740b40e0 0x7fc6740aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 >> v1:172.21.15.138:6789/0 conn(0x7fc6740af1b0 legacy=0x7fc674071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6740b38e0 con 0x7fc67415bef0 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fc668001670 con 0x7fc67415bef0 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 >> v1:172.21.15.28:6789/0 conn(0x7fc67415bef0 legacy=0x7fc6740b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 0x7fc65c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c004800 0x7fc65c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c005210 0x7fc65c00fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740b5190 con 0x7fc65c003a00 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc65c010570 con 0x7fc65c004800 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc65c010770 con 0x7fc65c005210 2024-09-10T10:49:23.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc679000640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 0x7fc65c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc673e00640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c004800 0x7fc65c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.865+0000 7fc679a00640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c005210 0x7fc65c00fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679000640 1 -- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c005210 msgr2=0x7fc65c00fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679000640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c005210 0x7fc65c00fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679000640 1 -- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c004800 msgr2=0x7fc65c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679000640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc65c004800 0x7fc65c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679000640 1 -- 172.21.15.28:0/2279606813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65c003520 con 0x7fc65c003a00 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.866+0000 7fc679a00640 1 -- 172.21.15.28:0/2279606813 reap_dead start 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.867+0000 7fc679000640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 0x7fc65c003e20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c0e0 tx=0x7fc65c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8735053737350bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.867+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668007c40 con 0x7fc65c003a00 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.867+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc668005af0 con 0x7fc65c003a00 2024-09-10T10:49:23.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.867+0000 7fc673400640 1 -- 172.21.15.28:0/2279606813 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668007070 con 0x7fc65c003a00 2024-09-10T10:49:23.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 -- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 msgr2=0x7fc65c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 0x7fc65c003e20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc66800c0e0 tx=0x7fc65c005830 comp rx=0 tx=0).stop 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 -- 172.21.15.28:0/2279606813 shutdown_connections 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 --2- 172.21.15.28:0/2279606813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc65c003a00 0x7fc65c003e20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 -- 172.21.15.28:0/2279606813 >> 172.21.15.28:0/2279606813 conn(0x7fc67406c240 msgr2=0x7fc6740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 -- 172.21.15.28:0/2279606813 shutdown_connections 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.868+0000 7fc67a400640 1 -- 172.21.15.28:0/2279606813 wait complete. 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.869+0000 7fc67a400640 1 Processor -- start 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 -- start start 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c003a00 0x7fc67415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 0x7fc67406dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740b4e60 con 0x7fc674155490 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740af640 con 0x7fc674158cc0 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.870+0000 7fc67a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc6740b4b10 con 0x7fc65c003a00 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39436/0 (socket says 172.21.15.28:39436) 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 -- 172.21.15.28:0/177027764 learned_addr learned my addr 172.21.15.28:0/177027764 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679000640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c003a00 0x7fc67415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc673e00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 0x7fc67406dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 -- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c003a00 msgr2=0x7fc67415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c003a00 0x7fc67415b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 -- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 msgr2=0x7fc67406dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 0x7fc67406dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:23.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 -- 172.21.15.28:0/177027764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc674155e50 con 0x7fc674155490 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc673e00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 0x7fc67406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.871+0000 7fc679a00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc67406cb80 tx=0x7fc66c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.872+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c006b50 con 0x7fc674155490 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.872+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc66c006cf0 con 0x7fc674155490 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.872+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c003970 con 0x7fc674155490 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.872+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc674158450 con 0x7fc674155490 2024-09-10T10:49:23.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.872+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6741586f0 con 0x7fc674155490 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.873+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6740b46b0 con 0x7fc674155490 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.874+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc66c01a020 con 0x7fc674155490 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.875+0000 7fc670c00640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 0x7fc65406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.875+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7fc66c08a320 con 0x7fc674155490 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.875+0000 7fc679000640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 0x7fc65406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:23.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.876+0000 7fc679000640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 0x7fc65406e320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc668032000 tx=0x7fc66802d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:23.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:23.879+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc6740b46b0 con 0x7fc674155490 2024-09-10T10:49:24.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.065+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc67406fc60 con 0x7fc674155490 2024-09-10T10:49:24.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.065+0000 7fc670c00640 1 -- 172.21.15.28:0/177027764 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13594 (secure 0 0 0) 0x7fc674155e50 con 0x7fc674155490 2024-09-10T10:49:24.067 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:24.067 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":77,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:49:21.663075+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/2942680174":"2024-09-11T10:49:18.264601+0000","172.21.15.28:6833/2942680174":"2024-09-11T10:49:18.264601+0000","172.21.15.138:6837/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6835/2819175864":"2024-09-11T10:47:36.629863+0000","172.21.15.138:6837/3429810116":"2024-09-11T10:47:00.608334+0000","172.21.15.138:6835/3429810116":"2024-09-11T10:47:00.608334+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:24.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 msgr2=0x7fc65406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 0x7fc65406e320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc668032000 tx=0x7fc66802d000 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 msgr2=0x7fc674155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc67406cb80 tx=0x7fc66c00eb90 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 shutdown_connections 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc65406be60 0x7fc65406e320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc674155490 0x7fc674155910 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc674158cc0 0x7fc67406dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 --2- 172.21.15.28:0/177027764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc65c003a00 0x7fc67415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 >> 172.21.15.28:0/177027764 conn(0x7fc67406c240 msgr2=0x7fc6741576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 shutdown_connections 2024-09-10T10:49:24.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.068+0000 7fc67a400640 1 -- 172.21.15.28:0/177027764 wait complete. 2024-09-10T10:49:24.085 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:49:24.086 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:49:24.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.243+0000 7fc358400640 1 Processor -- start 2024-09-10T10:49:24.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.244+0000 7fc358400640 1 -- start start 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc358400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc350156740 0x7fc3500b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc358400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3500b30e0 con 0x7fc3500cfea0 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc358400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3500b33c0 con 0x7fc3500b4af0 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc358400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc35006edc0 con 0x7fc350156740 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc356400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc3500cfea0 0x7fc3500ce060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48076/0 (socket says 172.21.15.28:48076) 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc356400640 1 -- 172.21.15.28:0/3902527410 learned_addr learned my addr 172.21.15.28:0/3902527410 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:24.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.245+0000 7fc355a00640 1 --2- 172.21.15.28:0/3902527410 >> v2:172.21.15.138:3300/0 conn(0x7fc350156740 0x7fc3500b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953669906 0 0) 0x7fc3500b30e0 con 0x7fc3500cfea0 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc338003520 con 0x7fc3500cfea0 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106888126 0 0) 0x7fc3500b33c0 con 0x7fc3500b4af0 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc355a00640 1 -- 172.21.15.28:0/3902527410 >> v1:172.21.15.138:6789/0 conn(0x7fc3500b4af0 legacy=0x7fc3500cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc355a00640 1 -- 172.21.15.28:0/3902527410 >> v1:172.21.15.28:6789/0 conn(0x7fc3500cfea0 legacy=0x7fc3500ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.246+0000 7fc355a00640 1 -- 172.21.15.28:0/3902527410 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500b36a0 con 0x7fc350156740 2024-09-10T10:49:24.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.247+0000 7fc355a00640 1 --2- 172.21.15.28:0/3902527410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc350156740 0x7fc3500b45b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3440097a0 tx=0x7fc34402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8948db28f737ad37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.247+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344037070 con 0x7fc350156740 2024-09-10T10:49:24.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.247+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc344032440 con 0x7fc350156740 2024-09-10T10:49:24.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.247+0000 7fc34fe00640 1 -- 172.21.15.28:0/3902527410 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440327f0 con 0x7fc350156740 2024-09-10T10:49:24.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 -- 172.21.15.28:0/3902527410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc350156740 msgr2=0x7fc3500b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 --2- 172.21.15.28:0/3902527410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc350156740 0x7fc3500b45b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3440097a0 tx=0x7fc34402a230 comp rx=0 tx=0).stop 2024-09-10T10:49:24.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 -- 172.21.15.28:0/3902527410 shutdown_connections 2024-09-10T10:49:24.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 --2- 172.21.15.28:0/3902527410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc350156740 0x7fc3500b45b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 -- 172.21.15.28:0/3902527410 >> 172.21.15.28:0/3902527410 conn(0x7fc35006bdf0 msgr2=0x7fc35006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:24.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 -- 172.21.15.28:0/3902527410 shutdown_connections 2024-09-10T10:49:24.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.248+0000 7fc358400640 1 -- 172.21.15.28:0/3902527410 wait complete. 2024-09-10T10:49:24.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.249+0000 7fc358400640 1 Processor -- start 2024-09-10T10:49:24.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.250+0000 7fc358400640 1 -- start start 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3500b4af0 0x7fc350159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 0x7fc3501573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500d0890 con 0x7fc3500cfea0 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3500b3c60 con 0x7fc3500b4af0 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc358400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3500d0a10 con 0x7fc350156740 2024-09-10T10:49:24.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39448/0 (socket says 172.21.15.28:39448) 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 -- 172.21.15.28:0/4056231445 learned_addr learned my addr 172.21.15.28:0/4056231445 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355a00640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3500b4af0 0x7fc350159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc356400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 0x7fc3501573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 -- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 msgr2=0x7fc3501573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 0x7fc3501573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 -- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3500b4af0 msgr2=0x7fc350159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3500b4af0 0x7fc350159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc355000640 1 -- 172.21.15.28:0/4056231445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500b1020 con 0x7fc3500cfea0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.251+0000 7fc356400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 0x7fc3501573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc355000640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c4b0 tx=0x7fc33c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c017070 con 0x7fc3500cfea0 2024-09-10T10:49:24.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc33c005480 con 0x7fc3500cfea0 2024-09-10T10:49:24.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33c003830 con 0x7fc3500cfea0 2024-09-10T10:49:24.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3500b04a0 con 0x7fc3500cfea0 2024-09-10T10:49:24.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.252+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3500b0a60 con 0x7fc3500cfea0 2024-09-10T10:49:24.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.253+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc350068f50 con 0x7fc3500cfea0 2024-09-10T10:49:24.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.254+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc33c01a070 con 0x7fc3500cfea0 2024-09-10T10:49:24.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.255+0000 7fc34d600640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.255+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7fc33c08a100 con 0x7fc3500cfea0 2024-09-10T10:49:24.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.255+0000 7fc355a00640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.256+0000 7fc355a00640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc344034000 tx=0x7fc34402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.259+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc33c058510 con 0x7fc3500cfea0 2024-09-10T10:49:24.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.443+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc350156c40 con 0x7fc3500cfea0 2024-09-10T10:49:24.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.443+0000 7fc34d600640 1 -- 172.21.15.28:0/4056231445 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fc33c05c810 con 0x7fc3500cfea0 2024-09-10T10:49:24.444 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 msgr2=0x7fc33006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 0x7fc33006e2f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc344034000 tx=0x7fc34402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 msgr2=0x7fc350156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc33c00c4b0 tx=0x7fc33c00c970 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 shutdown_connections 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc33006be30 0x7fc33006e2f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc350156740 0x7fc3501573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3500cfea0 0x7fc350156e60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 --2- 172.21.15.28:0/4056231445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3500b4af0 0x7fc350159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 >> 172.21.15.28:0/4056231445 conn(0x7fc35006bdf0 msgr2=0x7fc3500b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:24.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 shutdown_connections 2024-09-10T10:49:24.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.445+0000 7fc358400640 1 -- 172.21.15.28:0/4056231445 wait complete. 2024-09-10T10:49:24.462 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-10T10:49:24.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.618+0000 7fb6e1400640 1 Processor -- start 2024-09-10T10:49:24.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.618+0000 7fb6e1400640 1 -- start start 2024-09-10T10:49:24.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6e1400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 0x7fb6dc06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6e1400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0d2700 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6e1400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6dc0cedb0 con 0x7fb6dc0cfcc0 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6e1400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0cf090 con 0x7fb6dc15e760 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6d3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb6dc0d4fd0 0x7fb6dc0cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48080/0 (socket says 172.21.15.28:48080) 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.619+0000 7fb6d3e00640 1 -- 172.21.15.28:0/3488253227 learned_addr learned my addr 172.21.15.28:0/3488253227 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:24.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6da800640 1 --2- 172.21.15.28:0/3488253227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 0x7fb6dc06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506517844 0 0) 0x7fb6c4001290 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6c0003520 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405174532 0 0) 0x7fb6dc0cedb0 con 0x7fb6dc0cfcc0 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6c4001290 con 0x7fb6dc0cfcc0 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6da800640 1 -- 172.21.15.28:0/3488253227 >> v1:172.21.15.138:6789/0 conn(0x7fb6dc0cfcc0 legacy=0x7fb6dc0d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6da800640 1 -- 172.21.15.28:0/3488253227 >> v1:172.21.15.28:6789/0 conn(0x7fb6dc0d4fd0 legacy=0x7fb6dc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.620+0000 7fb6da800640 1 -- 172.21.15.28:0/3488253227 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0b3c80 con 0x7fb6dc15e760 2024-09-10T10:49:24.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.621+0000 7fb6da800640 1 --2- 172.21.15.28:0/3488253227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 0x7fb6dc06cdd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0097d0 tx=0x7fb6cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3a39da7e71515b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc037070 con 0x7fb6dc15e760 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6cc032440 con 0x7fb6dc15e760 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6d9e00640 1 -- 172.21.15.28:0/3488253227 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc032810 con 0x7fb6dc15e760 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 -- 172.21.15.28:0/3488253227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 msgr2=0x7fb6dc06cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 --2- 172.21.15.28:0/3488253227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 0x7fb6dc06cdd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc0097d0 tx=0x7fb6cc02a180 comp rx=0 tx=0).stop 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 -- 172.21.15.28:0/3488253227 shutdown_connections 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 --2- 172.21.15.28:0/3488253227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc15e760 0x7fb6dc06cdd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 -- 172.21.15.28:0/3488253227 >> 172.21.15.28:0/3488253227 conn(0x7fb6dc08bfc0 msgr2=0x7fb6dc0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 -- 172.21.15.28:0/3488253227 shutdown_connections 2024-09-10T10:49:24.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.622+0000 7fb6e1400640 1 -- 172.21.15.28:0/3488253227 wait complete. 2024-09-10T10:49:24.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.624+0000 7fb6e1400640 1 Processor -- start 2024-09-10T10:49:24.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.624+0000 7fb6e1400640 1 -- start start 2024-09-10T10:49:24.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc0cfcc0 0x7fb6dc15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc15e760 0x7fb6dc0b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d35b0 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6dc0b3860 con 0x7fb6dc15e760 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6e1400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb6dc0d3730 con 0x7fb6dc0cfcc0 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39464/0 (socket says 172.21.15.28:39464) 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6d3e00640 1 -- 172.21.15.28:0/25352040 learned_addr learned my addr 172.21.15.28:0/25352040 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6db200640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc15e760 0x7fb6dc0b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.625+0000 7fb6da800640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc0cfcc0 0x7fb6dc15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 -- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc0cfcc0 msgr2=0x7fb6dc15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc0cfcc0 0x7fb6dc15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 -- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc15e760 msgr2=0x7fb6dc0b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:24.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc15e760 0x7fb6dc0b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:24.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 -- 172.21.15.28:0/25352040 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6dc0b77e0 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d3e00640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4001550 tx=0x7fb6c400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4017070 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c40074a0 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40059e0 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6dc0b7ac0 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.626+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6dc0cca60 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.627+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6dc06d590 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.628+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb6c401a070 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.629+0000 7fb6d2a00640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 0x7fb6b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:24.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.629+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7fb6c408a270 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.629+0000 7fb6da800640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 0x7fb6b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:24.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.630+0000 7fb6da800640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 0x7fb6b806e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc034000 tx=0x7fb6cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:24.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.633+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6dc0d2700 con 0x7fb6dc0d4fd0 2024-09-10T10:49:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:24.892+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7fb6dc0b6d20 con 0x7fb6dc0d4fd0 2024-09-10T10:49:25.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.685+0000 7fb6d2a00640 1 -- 172.21.15.28:0/25352040 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v275) ==== 144+0+0 (secure 0 0 0) 0x7fb6c4058680 con 0x7fb6dc0d4fd0 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 msgr2=0x7fb6b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 0x7fb6b806e2f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc034000 tx=0x7fb6cc02e000 comp rx=0 tx=0).stop 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 msgr2=0x7fb6dc0b9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4001550 tx=0x7fb6c400caf0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 shutdown_connections 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb6b806be30 0x7fb6b806e2f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6dc15e760 0x7fb6dc0b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6dc0d4fd0 0x7fb6dc0b9ca0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 --2- 172.21.15.28:0/25352040 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb6dc0cfcc0 0x7fb6dc15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.688+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 >> 172.21.15.28:0/25352040 conn(0x7fb6dc08bfc0 msgr2=0x7fb6dc0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.689+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 shutdown_connections 2024-09-10T10:49:25.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.689+0000 7fb6e1400640 1 -- 172.21.15.28:0/25352040 wait complete. 2024-09-10T10:49:25.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:49:25.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.873+0000 7f7311a00640 1 Processor -- start 2024-09-10T10:49:25.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.873+0000 7f7311a00640 1 -- start start 2024-09-10T10:49:25.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f7311a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f730c15e900 0x7f730c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f7311a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c0b4a20 0x7f730c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f7311a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f730c0b4620 con 0x7f730c072050 2024-09-10T10:49:25.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f7311a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f730c0ce100 con 0x7f730c15e900 2024-09-10T10:49:25.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f7311a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f730c0ce3a0 con 0x7f730c0b4a20 2024-09-10T10:49:25.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f730b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f730c072050 0x7f730c0cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48096/0 (socket says 172.21.15.28:48096) 2024-09-10T10:49:25.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.874+0000 7f730b400640 1 -- 172.21.15.28:0/3435329795 learned_addr learned my addr 172.21.15.28:0/3435329795 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730aa00640 1 --2- 172.21.15.28:0/3435329795 >> v2:172.21.15.138:3300/0 conn(0x7f730c15e900 0x7f730c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f7302000640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c0b4a20 0x7f730c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730a000640 1 -- 172.21.15.28:0/3435329795 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3157095222 0 0) 0x7f730c0b4620 con 0x7f730c072050 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730a000640 1 -- 172.21.15.28:0/3435329795 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72ec005730 con 0x7f730c072050 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730aa00640 1 -- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c0b4a20 msgr2=0x7f730c0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730aa00640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c0b4a20 0x7f730c0ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730aa00640 1 -- 172.21.15.28:0/3435329795 >> v1:172.21.15.28:6789/0 conn(0x7f730c072050 legacy=0x7f730c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.875+0000 7f730aa00640 1 -- 172.21.15.28:0/3435329795 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c0ce680 con 0x7f730c15e900 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.876+0000 7f730aa00640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c15e900 0x7f730c0b4020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae64eb310d88cf3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.876+0000 7f730a000640 1 -- 172.21.15.28:0/3435329795 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x7f730c15e900 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.876+0000 7f730a000640 1 -- 172.21.15.28:0/3435329795 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc02ae40 con 0x7f730c15e900 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.876+0000 7f730a000640 1 -- 172.21.15.28:0/3435329795 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032660 con 0x7f730c15e900 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 -- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c15e900 msgr2=0x7f730c0b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c15e900 0x7f730c0b4020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a0f0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 -- 172.21.15.28:0/3435329795 shutdown_connections 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c0b4a20 0x7f730c0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 --2- 172.21.15.28:0/3435329795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c15e900 0x7f730c0b4020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 -- 172.21.15.28:0/3435329795 >> 172.21.15.28:0/3435329795 conn(0x7f730c06bdf0 msgr2=0x7f730c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 -- 172.21.15.28:0/3435329795 shutdown_connections 2024-09-10T10:49:25.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.877+0000 7f7311a00640 1 -- 172.21.15.28:0/3435329795 wait complete. 2024-09-10T10:49:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.879+0000 7f7311a00640 1 Processor -- start 2024-09-10T10:49:25.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.879+0000 7f7311a00640 1 -- start start 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c0b4a20 0x7f730c06ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c15e900 0x7f730c06f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f730c0ad560 con 0x7f730c072050 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f730c0b4e20 con 0x7f730c0b4a20 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7311a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f730c0cf590 con 0x7f730c15e900 2024-09-10T10:49:25.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39472/0 (socket says 172.21.15.28:39472) 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 -- 172.21.15.28:0/709435153 learned_addr learned my addr 172.21.15.28:0/709435153 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f7302000640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c0b4a20 0x7f730c06ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730b400640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c15e900 0x7f730c06f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 -- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c15e900 msgr2=0x7f730c06f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c15e900 0x7f730c06f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 -- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c0b4a20 msgr2=0x7f730c06ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c0b4a20 0x7f730c06ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.880+0000 7f730aa00640 1 -- 172.21.15.28:0/709435153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c09e0e0 con 0x7f730c072050 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f730aa00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0098c0 tx=0x7f72fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc041030 con 0x7f730c072050 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72fc02ac40 con 0x7f730c072050 2024-09-10T10:49:25.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037040 con 0x7f730c072050 2024-09-10T10:49:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c09d560 con 0x7f730c072050 2024-09-10T10:49:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.881+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c09db20 con 0x7f730c072050 2024-09-10T10:49:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.882+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f730c0ad020 con 0x7f730c072050 2024-09-10T10:49:25.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.883+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72fc044070 con 0x7f730c072050 2024-09-10T10:49:25.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.884+0000 7f7303400640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:25.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.884+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7f72fc0ab260 con 0x7f730c072050 2024-09-10T10:49:25.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.884+0000 7f7302000640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:25.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.885+0000 7f7302000640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f730c0addb0 tx=0x7f72f4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:25.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:25.888+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72fc04b070 con 0x7f730c072050 2024-09-10T10:49:26.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.071+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f730c06c540 con 0x7f730c072050 2024-09-10T10:49:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.075+0000 7f7303400640 1 -- 172.21.15.28:0/709435153 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f72fc0795e0 con 0x7f730c072050 2024-09-10T10:49:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 msgr2=0x7f72e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 0x7f72e806e2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f730c0addb0 tx=0x7f72f4009190 comp rx=0 tx=0).stop 2024-09-10T10:49:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 msgr2=0x7f730c14b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0098c0 tx=0x7f72fc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 shutdown_connections 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72e806be30 0x7f72e806e2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f730c15e900 0x7f730c06f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c0b4a20 0x7f730c06ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 --2- 172.21.15.28:0/709435153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f730c072050 0x7f730c14b2f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 >> 172.21.15.28:0/709435153 conn(0x7f730c06bdf0 msgr2=0x7f730c0adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 shutdown_connections 2024-09-10T10:49:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.077+0000 7f7311a00640 1 -- 172.21.15.28:0/709435153 wait complete. 2024-09-10T10:49:26.095 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:49:26.095 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:49:26.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:49:26.270 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:49:26.271 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:26.271 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:26.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:49:26.307 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:49:26.307 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:26.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:49:26.344 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:26.344 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:26.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:49:26.381 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:49:26.382 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:26.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:49:26.420 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:49:26.421 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.576+0000 7f7615e00640 1 Processor -- start 2024-09-10T10:49:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.577+0000 7f7615e00640 1 -- start start 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7615e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f76100d30f0 0x7f76100b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7615e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76100b64c0 0x7f761006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7615e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76100d3690 con 0x7f761015e5f0 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7615e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f761006d530 con 0x7f76100d30f0 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7615e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f76100d3970 con 0x7f76100b64c0 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f760ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f761015e5f0 0x7f76100d4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48110/0 (socket says 172.21.15.28:48110) 2024-09-10T10:49:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f760ea00640 1 -- 172.21.15.28:0/2272196207 learned_addr learned my addr 172.21.15.28:0/2272196207 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f7606000640 1 --2- 172.21.15.28:0/2272196207 >> v2:172.21.15.138:3300/0 conn(0x7f76100d30f0 0x7f76100b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f760f400640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76100b64c0 0x7f761006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.578+0000 7f760e000640 1 -- 172.21.15.28:0/2272196207 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898624292 0 0) 0x7f76100d3690 con 0x7f761015e5f0 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.579+0000 7f760e000640 1 -- 172.21.15.28:0/2272196207 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f4003520 con 0x7f761015e5f0 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.579+0000 7f7606000640 1 -- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76100b64c0 msgr2=0x7f761006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.579+0000 7f7606000640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76100b64c0 0x7f761006d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.579+0000 7f7606000640 1 -- 172.21.15.28:0/2272196207 >> v1:172.21.15.28:6789/0 conn(0x7f761015e5f0 legacy=0x7f76100d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.579+0000 7f7606000640 1 -- 172.21.15.28:0/2272196207 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761006cee0 con 0x7f76100d30f0 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f7606000640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f76100b5f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f75f80097a0 tx=0x7f75f802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=210934db42b5fd7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f760e000640 1 -- 172.21.15.28:0/2272196207 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8037070 con 0x7f76100d30f0 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f760e000640 1 -- 172.21.15.28:0/2272196207 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f8032440 con 0x7f76100d30f0 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f760e000640 1 -- 172.21.15.28:0/2272196207 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8032810 con 0x7f76100d30f0 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f7615e00640 1 -- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 msgr2=0x7f76100b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.580+0000 7f7615e00640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f76100b5f80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f75f80097a0 tx=0x7f75f802a180 comp rx=0 tx=0).stop 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 -- 172.21.15.28:0/2272196207 shutdown_connections 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f76100b64c0 0x7f761006d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 --2- 172.21.15.28:0/2272196207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f76100b5f80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 -- 172.21.15.28:0/2272196207 >> 172.21.15.28:0/2272196207 conn(0x7f761008bfc0 msgr2=0x7f76100d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 -- 172.21.15.28:0/2272196207 shutdown_connections 2024-09-10T10:49:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.581+0000 7f7615e00640 1 -- 172.21.15.28:0/2272196207 wait complete. 2024-09-10T10:49:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.582+0000 7f7615e00640 1 Processor -- start 2024-09-10T10:49:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.582+0000 7f7615e00640 1 -- start start 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f7610160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 0x7f761015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f761006df00 con 0x7f76100b64c0 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f761015e9d0 con 0x7f76100d30f0 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.583+0000 7f7615e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f761006e080 con 0x7f761015e5f0 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39478/0 (socket says 172.21.15.28:39478) 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 -- 172.21.15.28:0/2298432074 learned_addr learned my addr 172.21.15.28:0/2298432074 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760f400640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 0x7f761015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f7606000640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f7610160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 -- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 msgr2=0x7f761015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 0x7f761015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 -- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 msgr2=0x7f7610160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f7610160090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 -- 172.21.15.28:0/2298432074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76101610d0 con 0x7f76100b64c0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760f400640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 0x7f761015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.584+0000 7f760ea00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f760000c390 tx=0x7f760000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.585+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600017070 con 0x7f76100b64c0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.585+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7600005ce0 con 0x7f76100b64c0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.585+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76101598c0 con 0x7f76100b64c0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.585+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76000072f0 con 0x7f76100b64c0 2024-09-10T10:49:26.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.585+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7610160df0 con 0x7f76100b64c0 2024-09-10T10:49:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.586+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f761006d9c0 con 0x7f76100b64c0 2024-09-10T10:49:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.587+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f760001a070 con 0x7f76100b64c0 2024-09-10T10:49:26.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.588+0000 7f7607e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 0x7f75ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.588+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7f760008a210 con 0x7f76100b64c0 2024-09-10T10:49:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.588+0000 7f7606000640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 0x7f75ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:26.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.589+0000 7f7606000640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 0x7f75ec06e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f75f8034000 tx=0x7f75f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:26.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.592+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f76101610d0 con 0x7f76100b64c0 2024-09-10T10:49:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.813+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7610158fb0 con 0x7f76100b64c0 2024-09-10T10:49:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.814+0000 7f7607e00640 1 -- 172.21.15.28:0/2298432074 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+3914 (secure 0 0 0) 0x7f7600058bf0 con 0x7f76100b64c0 2024-09-10T10:49:26.815 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:26.815 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":62,"btime":"2024-09-10T10:49:19:267269+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1457607439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1457607439},{"type":"v1","addr":"172.21.15.138:6836","nonce":1457607439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1516993971","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1516993971},{"type":"v1","addr":"172.21.15.28:6834","nonce":1516993971}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":5507,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/690708012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":690708012},{"type":"v1","addr":"172.21.15.138:6837","nonce":690708012}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1526303859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1526303859},{"type":"v1","addr":"172.21.15.28:6835","nonce":1526303859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[]} 2024-09-10T10:49:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 62 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 msgr2=0x7f75ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 0x7f75ec06e2f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f75f8034000 tx=0x7f75f802e000 comp rx=0 tx=0).stop 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 msgr2=0x7f7610158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f760000c390 tx=0x7f760000c850 comp rx=0 tx=0).stop 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 shutdown_connections 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f75ec06be30 0x7f75ec06e2f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f761015e5f0 0x7f761015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76100d30f0 0x7f7610160090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 --2- 172.21.15.28:0/2298432074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76100b64c0 0x7f7610158a70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 >> 172.21.15.28:0/2298432074 conn(0x7f761008bfc0 msgr2=0x7f76100d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 shutdown_connections 2024-09-10T10:49:26.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:26.816+0000 7f7615e00640 1 -- 172.21.15.28:0/2298432074 wait complete. 2024-09-10T10:49:26.833 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:49:26.833 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:49:26.833 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:49:26.833 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:49:26.834 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:49:26.834 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:49:26.834 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:49:26.834 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:49:26.834 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:49:26.834 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:49:26.834 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:49:26.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:49:26.835 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:49:26.835 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:49:26.835 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:49:26.835 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:49:26.849 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:49:26.849 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:49:26.850 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:49:26.850 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:49:26.851 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:49:26.851 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:49:26.852 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:49:26.852 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:49:26.893 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:49:26.893 DEBUG:teuthology.parallel:result is None 2024-09-10T10:49:26.895 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:49:26.895 DEBUG:teuthology.parallel:result is None 2024-09-10T10:49:26.897 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:49:26.897 DEBUG:teuthology.parallel:result is None 2024-09-10T10:49:26.899 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:49:26.900 DEBUG:teuthology.parallel:result is None 2024-09-10T10:49:26.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:49:27.023 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:49:27.024 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:49:27.031 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:49:27.031 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:49:27.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T10:49:27.055 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-10T10:49:27.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.063+0000 7f4409000640 1 Processor -- start 2024-09-10T10:49:27.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.063+0000 7f4409000640 1 -- start start 2024-09-10T10:49:27.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.064+0000 7f4409000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f44040b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.064+0000 7f4409000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44040b64b0 0x7f44040b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.064+0000 7f4409000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f440406d9f0 con 0x7f4404161c80 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.064+0000 7f4409000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f440406dcd0 con 0x7f44040b64b0 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.064+0000 7f4409000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f440406dfb0 con 0x7f4404162060 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f4401e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4404161c80 0x7f440406e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48124/0 (socket says 172.21.15.28:48124) 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f4401e00640 1 -- 172.21.15.28:0/144776325 learned_addr learned my addr 172.21.15.28:0/144776325 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f4402800640 1 --2- 172.21.15.28:0/144776325 >> v2:172.21.15.138:3300/0 conn(0x7f44040b64b0 0x7f44040b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f43fbe00640 1 --2- 172.21.15.28:0/144776325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f44040b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f43fbe00640 1 -- 172.21.15.28:0/144776325 >> v2:172.21.15.138:3300/0 conn(0x7f44040b64b0 msgr2=0x7f44040b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:27.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f43fbe00640 1 --2- 172.21.15.28:0/144776325 >> v2:172.21.15.138:3300/0 conn(0x7f44040b64b0 0x7f44040b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:27.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f43fbe00640 1 -- 172.21.15.28:0/144776325 >> v1:172.21.15.28:6789/0 conn(0x7f4404161c80 legacy=0x7f440406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:27.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f43fbe00640 1 -- 172.21.15.28:0/144776325 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44040d01f0 con 0x7f4404162060 2024-09-10T10:49:27.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.065+0000 7f4401400640 1 -- 172.21.15.28:0/144776325 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947101762 0 0) 0x7f440406d9f0 con 0x7f4404161c80 2024-09-10T10:49:27.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.066+0000 7f43fbe00640 1 --2- 172.21.15.28:0/144776325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f44040b5f70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097a0 tx=0x7f43ec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5a153f08ecc664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:27.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4401400640 1 -- 172.21.15.28:0/144776325 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037070 con 0x7f4404162060 2024-09-10T10:49:27.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4401400640 1 -- 172.21.15.28:0/144776325 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec032440 con 0x7f4404162060 2024-09-10T10:49:27.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4401400640 1 -- 172.21.15.28:0/144776325 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec032810 con 0x7f4404162060 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 -- 172.21.15.28:0/144776325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 msgr2=0x7f44040b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 --2- 172.21.15.28:0/144776325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f44040b5f70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f43ec0097a0 tx=0x7f43ec02a180 comp rx=0 tx=0).stop 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 -- 172.21.15.28:0/144776325 shutdown_connections 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 --2- 172.21.15.28:0/144776325 >> v2:172.21.15.138:3300/0 conn(0x7f44040b64b0 0x7f44040b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 --2- 172.21.15.28:0/144776325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f44040b5f70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 -- 172.21.15.28:0/144776325 >> 172.21.15.28:0/144776325 conn(0x7f44040d44f0 msgr2=0x7f44040d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 -- 172.21.15.28:0/144776325 shutdown_connections 2024-09-10T10:49:27.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.067+0000 7f4409000640 1 -- 172.21.15.28:0/144776325 wait complete. 2024-09-10T10:49:27.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.069+0000 7f4409000640 1 Processor -- start 2024-09-10T10:49:27.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.069+0000 7f4409000640 1 -- start start 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44040b64b0 0x7f440415ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f4404159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44040cf1e0 con 0x7f4404161c80 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44040d2aa0 con 0x7f44040b64b0 2024-09-10T10:49:27.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4409000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44040cf360 con 0x7f4404162060 2024-09-10T10:49:27.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f43fbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39504/0 (socket says 172.21.15.28:39504) 2024-09-10T10:49:27.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f43fbe00640 1 -- 172.21.15.28:0/3030841675 learned_addr learned my addr 172.21.15.28:0/3030841675 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:27.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4402800640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f4404159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.070+0000 7f4401e00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44040b64b0 0x7f440415ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 -- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 msgr2=0x7f4404159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f4404159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 -- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44040b64b0 msgr2=0x7f440415ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44040b64b0 0x7f440415ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 -- 172.21.15.28:0/3030841675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44040b9420 con 0x7f4404161c80 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fbe00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f43ec000c00 tx=0x7f43ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037040 con 0x7f4404161c80 2024-09-10T10:49:27.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec042070 con 0x7f4404161c80 2024-09-10T10:49:27.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec030920 con 0x7f4404161c80 2024-09-10T10:49:27.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44040b9700 con 0x7f4404161c80 2024-09-10T10:49:27.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.071+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44040b9cc0 con 0x7f4404161c80 2024-09-10T10:49:27.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.073+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44040b5f70 con 0x7f4404161c80 2024-09-10T10:49:27.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.073+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43ec030ac0 con 0x7f4404161c80 2024-09-10T10:49:27.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.074+0000 7f43fa000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:27.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.074+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5507+0+0 (secure 0 0 0) 0x7f43ec0ab530 con 0x7f4404161c80 2024-09-10T10:49:27.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.075+0000 7f4401e00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:27.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.075+0000 7f4401e00640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 0x7f43e006e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000f60 tx=0x7f43f4007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:27.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.079+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43ec0799e0 con 0x7f4404161c80 2024-09-10T10:49:27.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:27.279+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f44040d3820 con 0x7f4404161c80 2024-09-10T10:49:28.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.093+0000 7f43fa000640 1 -- 172.21.15.28:0/3030841675 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) ==== 97+0+0 (secure 0 0 0) 0x7f43ec07dce0 con 0x7f4404161c80 2024-09-10T10:49:28.095 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:49:28.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 msgr2=0x7f43e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 0x7f43e006e2f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f43f4000f60 tx=0x7f43f4007d50 comp rx=0 tx=0).stop 2024-09-10T10:49:28.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 msgr2=0x7f44040b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f43ec000c00 tx=0x7f43ec0046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 shutdown_connections 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43e006be30 0x7f43e006e2f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4404162060 0x7f4404159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4404161c80 0x7f44040b8ab0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 --2- 172.21.15.28:0/3030841675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44040b64b0 0x7f440415ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 >> 172.21.15.28:0/3030841675 conn(0x7f44040d44f0 msgr2=0x7f44040b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 shutdown_connections 2024-09-10T10:49:28.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.097+0000 7f4409000640 1 -- 172.21.15.28:0/3030841675 wait complete. 2024-09-10T10:49:28.118 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:49:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.277+0000 7f3db4000640 1 Processor -- start 2024-09-10T10:49:28.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.277+0000 7f3db4000640 1 -- start start 2024-09-10T10:49:28.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3dac0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db4000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dac06f480 con 0x7f3dac147230 2024-09-10T10:49:28.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db4000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dac06e900 con 0x7f3dac0725f0 2024-09-10T10:49:28.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac0b34d0 con 0x7f3dac147e90 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db1600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3dac147230 0x7f3dac06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48136/0 (socket says 172.21.15.28:48136) 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.278+0000 7f3db1600640 1 -- 172.21.15.28:0/910874054 learned_addr learned my addr 172.21.15.28:0/910874054 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3db0c00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3dac0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2332469943 0 0) 0x7f3dac06f480 con 0x7f3dac147230 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d94003520 con 0x7f3dac147230 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1782897934 0 0) 0x7f3d94003520 con 0x7f3dac147230 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 msgr2=0x7f3dac0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3dac0720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 >> v1:172.21.15.138:6789/0 conn(0x7f3dac0725f0 legacy=0x7f3dac0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac06ebe0 con 0x7f3dac147230 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f3da0001670 con 0x7f3dac147230 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 >> v1:172.21.15.28:6789/0 conn(0x7f3dac147230 legacy=0x7f3dac06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d940039e0 0x7f3d94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3d94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d94004d40 0x7f3d940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9400ba70 con 0x7f3d940039e0 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9400bc20 con 0x7f3d94004d40 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9400be20 con 0x7f3dac147e90 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3db1600640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d940039e0 0x7f3d94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3db0c00640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3d94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.279+0000 7f3db2000640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d94004d40 0x7f3d940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db1600640 1 -- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 msgr2=0x7f3d94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db1600640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac147e90 0x7f3d94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db1600640 1 -- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d94004d40 msgr2=0x7f3d940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db1600640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d94004d40 0x7f3d940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db1600640 1 -- 172.21.15.28:0/910874054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d94003520 con 0x7f3d940039e0 2024-09-10T10:49:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.280+0000 7f3db2000640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d94004d40 0x7f3d940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3db1600640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d940039e0 0x7f3d94003e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c210 tx=0x7f3d94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa63dfa37c78937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0007cf0 con 0x7f3d940039e0 2024-09-10T10:49:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3da0005af0 con 0x7f3d940039e0 2024-09-10T10:49:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3dabe00640 1 -- 172.21.15.28:0/910874054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0007070 con 0x7f3d940039e0 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3db4000640 1 -- 172.21.15.28:0/910874054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d940039e0 msgr2=0x7f3d94003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3db4000640 1 --2- 172.21.15.28:0/910874054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d940039e0 0x7f3d94003e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3da000c210 tx=0x7f3d94005830 comp rx=0 tx=0).stop 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.281+0000 7f3db2000640 1 -- 172.21.15.28:0/910874054 reap_dead start 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.282+0000 7f3db4000640 1 -- 172.21.15.28:0/910874054 shutdown_connections 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.282+0000 7f3db4000640 1 -- 172.21.15.28:0/910874054 >> 172.21.15.28:0/910874054 conn(0x7f3dac067d70 msgr2=0x7f3dac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.282+0000 7f3db4000640 1 -- 172.21.15.28:0/910874054 shutdown_connections 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.282+0000 7f3db4000640 1 -- 172.21.15.28:0/910874054 wait complete. 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.283+0000 7f3db4000640 1 Processor -- start 2024-09-10T10:49:28.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.283+0000 7f3db4000640 1 -- start start 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 0x7f3dac13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac141d80 0x7f3dac13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac072a40 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac06f1a0 con 0x7f3d940039e0 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.284+0000 7f3db4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3dac147610 con 0x7f3dac141d80 2024-09-10T10:49:28.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39524/0 (socket says 172.21.15.28:39524) 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 -- 172.21.15.28:0/1160833991 learned_addr learned my addr 172.21.15.28:0/1160833991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db2000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac141d80 0x7f3dac13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db1600640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 0x7f3dac13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 -- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac141d80 msgr2=0x7f3dac13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac141d80 0x7f3dac13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 -- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 msgr2=0x7f3dac13d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 0x7f3dac13d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 -- 172.21.15.28:0/1160833991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac13ddb0 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db1600640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 0x7f3dac13d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.285+0000 7f3db0c00640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3d980119d0 tx=0x7f3d98011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.286+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800faf0 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.286+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d9800fc90 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.286+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac13e090 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.286+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98014440 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.286+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac13e650 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.287+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac142160 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.288+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d98012040 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.289+0000 7f3da9600640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 0x7f3d8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.289+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f3d9808b1f0 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.289+0000 7f3db1600640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 0x7f3d8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.290+0000 7f3db1600640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 0x7f3d8c06e320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3da0032000 tx=0x7f3da0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.293+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d98019030 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.543+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3dac072090 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.544+0000 7f3da9600640 1 -- 172.21.15.28:0/1160833991 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f3d98059670 con 0x7f3dac1e4fb0 2024-09-10T10:49:28.545 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:28.545 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.546 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.547 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.548 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.549 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.550 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:49:28.551 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:49:28.552 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:49:28.553 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.554 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.555 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:49:28.556 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-09-10T10:49:28.557 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:49:28.558 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:49:28.559 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:49:28.560 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:49:28.561 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:49:28.562 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:49:28.563 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:49:28.564 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.564 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.564 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:49:28.564 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:49:28.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 msgr2=0x7f3d8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 0x7f3d8c06e320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3da0032000 tx=0x7f3da0009030 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 msgr2=0x7f3dac141840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3d980119d0 tx=0x7f3d98011dd0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 shutdown_connections 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d8c06be60 0x7f3d8c06e320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dac141d80 0x7f3dac13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dac1e4fb0 0x7f3dac141840 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 --2- 172.21.15.28:0/1160833991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d940039e0 0x7f3dac13d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.546+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 >> 172.21.15.28:0/1160833991 conn(0x7f3dac067d70 msgr2=0x7f3dac0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.547+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 shutdown_connections 2024-09-10T10:49:28.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.547+0000 7f3db4000640 1 -- 172.21.15.28:0/1160833991 wait complete. 2024-09-10T10:49:28.567 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:49:28.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.722+0000 7f0ec3e00640 1 Processor -- start 2024-09-10T10:49:28.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.722+0000 7f0ec3e00640 1 -- start start 2024-09-10T10:49:28.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.723+0000 7f0ec3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ec406e190 0x7f0ec40d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.723+0000 7f0ec3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.723+0000 7f0ec3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ec415e800 con 0x7f0ec40d4320 2024-09-10T10:49:28.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.723+0000 7f0ec3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec40d3230 con 0x7f0ec406e190 2024-09-10T10:49:28.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.723+0000 7f0ec3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec40d3510 con 0x7f0ec40cef20 2024-09-10T10:49:28.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0eba000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0ec406e190 0x7f0ec40d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:43378/0 (socket says 172.21.15.28:43378) 2024-09-10T10:49:28.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 -- 172.21.15.28:0/17462682 learned_addr learned my addr 172.21.15.28:0/17462682 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:28.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 -- 172.21.15.28:0/17462682 >> v2:172.21.15.138:3300/0 conn(0x7f0ec406e190 msgr2=0x7f0ec40d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:28.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 --2- 172.21.15.28:0/17462682 >> v2:172.21.15.138:3300/0 conn(0x7f0ec406e190 0x7f0ec40d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 -- 172.21.15.28:0/17462682 >> v1:172.21.15.138:6789/0 conn(0x7f0ec40d4320 legacy=0x7f0ec40d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.724+0000 7f0ec3400640 1 -- 172.21.15.28:0/17462682 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec40d37f0 con 0x7f0ec40cef20 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.725+0000 7f0ec3400640 1 --2- 172.21.15.28:0/17462682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009e50 tx=0x7f0ebc02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86c9cbf839eabee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.725+0000 7f0ec2000640 1 -- 172.21.15.28:0/17462682 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc036070 con 0x7f0ec40cef20 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.725+0000 7f0ec2000640 1 -- 172.21.15.28:0/17462682 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ebc031600 con 0x7f0ec40cef20 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.725+0000 7f0ec2000640 1 -- 172.21.15.28:0/17462682 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0319d0 con 0x7f0ec40cef20 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 -- 172.21.15.28:0/17462682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 msgr2=0x7f0ec40cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/17462682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009e50 tx=0x7f0ebc02a710 comp rx=0 tx=0).stop 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 -- 172.21.15.28:0/17462682 shutdown_connections 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/17462682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec40cef20 0x7f0ec40cf3b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/17462682 >> v2:172.21.15.138:3300/0 conn(0x7f0ec406e190 0x7f0ec40d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 -- 172.21.15.28:0/17462682 >> 172.21.15.28:0/17462682 conn(0x7f0ec409d810 msgr2=0x7f0ec408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 -- 172.21.15.28:0/17462682 shutdown_connections 2024-09-10T10:49:28.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.726+0000 7f0ec3e00640 1 -- 172.21.15.28:0/17462682 wait complete. 2024-09-10T10:49:28.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.728+0000 7f0ec3e00640 1 Processor -- start 2024-09-10T10:49:28.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.728+0000 7f0ec3e00640 1 -- start start 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec406e190 0x7f0ec40cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ec40d4320 0x7f0ec41537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40d2ff0 con 0x7f0ec40cef20 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec40d2e70 con 0x7f0ec40d4320 2024-09-10T10:49:28.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec40b1070 con 0x7f0ec406e190 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39534/0 (socket says 172.21.15.28:39534) 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 -- 172.21.15.28:0/1510527298 learned_addr learned my addr 172.21.15.28:0/1510527298 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec2a00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec406e190 0x7f0ec40cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0ec3400640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ec40d4320 0x7f0ec41537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 -- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec406e190 msgr2=0x7f0ec40cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec406e190 0x7f0ec40cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 -- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ec40d4320 msgr2=0x7f0ec41537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ec40d4320 0x7f0ec41537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.729+0000 7f0eba000640 1 -- 172.21.15.28:0/1510527298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec40cbbe0 con 0x7f0ec40cef20 2024-09-10T10:49:28.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0eba000640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e870 tx=0x7f0eac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac017070 con 0x7f0ec40cef20 2024-09-10T10:49:28.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eac00c940 con 0x7f0ec40cef20 2024-09-10T10:49:28.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec40cbec0 con 0x7f0ec40cef20 2024-09-10T10:49:28.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ccf0 con 0x7f0ec40cef20 2024-09-10T10:49:28.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.730+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec40cc480 con 0x7f0ec40cef20 2024-09-10T10:49:28.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.731+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec40690e0 con 0x7f0ec40cef20 2024-09-10T10:49:28.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.732+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0eac01a070 con 0x7f0ec40cef20 2024-09-10T10:49:28.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.733+0000 7f0ebb400640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 0x7f0ea006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:28.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.733+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7f0eac08aff0 con 0x7f0ec40cef20 2024-09-10T10:49:28.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.733+0000 7f0ec2a00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 0x7f0ea006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:28.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.734+0000 7f0ec2a00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 0x7f0ea006e320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4001640 tx=0x7f0eb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:28.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.737+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0eac059c10 con 0x7f0ec40cef20 2024-09-10T10:49:28.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.923+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0ec408c280 con 0x7f0ec40cef20 2024-09-10T10:49:28.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.923+0000 7f0ebb400640 1 -- 172.21.15.28:0/1510527298 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f0eac059510 con 0x7f0ec40cef20 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:49:28.925 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.925+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 msgr2=0x7f0ea006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.925+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 0x7f0ea006e320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4001640 tx=0x7f0eb4009040 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.925+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 msgr2=0x7f0ec40cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.925+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e870 tx=0x7f0eac00ed30 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.925+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 shutdown_connections 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ea006be60 0x7f0ea006e320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ec40d4320 0x7f0ec41537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ec40cef20 0x7f0ec40cdaf0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 --2- 172.21.15.28:0/1510527298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ec406e190 0x7f0ec40cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 >> 172.21.15.28:0/1510527298 conn(0x7f0ec409d810 msgr2=0x7f0ec40b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:28.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 shutdown_connections 2024-09-10T10:49:28.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:28.926+0000 7f0ec3e00640 1 -- 172.21.15.28:0/1510527298 wait complete. 2024-09-10T10:49:28.943 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:49:28.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:49:29.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.102+0000 7fb0f3e00640 1 Processor -- start 2024-09-10T10:49:29.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.102+0000 7fb0f3e00640 1 -- start start 2024-09-10T10:49:29.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0f40d4360 0x7fb0f4162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f406dc80 0x7fb0f406e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f3e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f40d2b50 con 0x7fb0f406e6b0 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f3e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f40b3af0 con 0x7fb0f40d4360 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f4162ed0 con 0x7fb0f406dc80 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0f40d4360 0x7fb0f4162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0f40d4360 0x7fb0f4162990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39550/0 (socket says 172.21.15.28:39550) 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3432742482 learned_addr learned my addr 172.21.15.28:0/3432742482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:29.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.103+0000 7fb0f2000640 1 --1- 172.21.15.28:0/3432742482 >> v1:172.21.15.28:6789/0 conn(0x7fb0f406e6b0 0x7fb0f40d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48140/0 (socket says 172.21.15.28:48140) 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f3400640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f406dc80 0x7fb0f406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3432742482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f406dc80 msgr2=0x7fb0f406e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f406dc80 0x7fb0f406e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3432742482 >> v1:172.21.15.28:6789/0 conn(0x7fb0f406e6b0 legacy=0x7fb0f40d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3432742482 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f40d2e30 con 0x7fb0f40d4360 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.104+0000 7fb0f1600640 1 -- 172.21.15.28:0/3432742482 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1492142068 0 0) 0x7fb0f40d2b50 con 0x7fb0f406e6b0 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.105+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f40d4360 0x7fb0f4162990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e402a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479937ff99f33e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.105+0000 7fb0f1600640 1 -- 172.21.15.28:0/3432742482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4037070 con 0x7fb0f40d4360 2024-09-10T10:49:29.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.105+0000 7fb0f1600640 1 -- 172.21.15.28:0/3432742482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e4032640 con 0x7fb0f40d4360 2024-09-10T10:49:29.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.105+0000 7fb0f1600640 1 -- 172.21.15.28:0/3432742482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4032a10 con 0x7fb0f40d4360 2024-09-10T10:49:29.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3432742482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f40d4360 msgr2=0x7fb0f4162990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f40d4360 0x7fb0f4162990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e402a320 comp rx=0 tx=0).stop 2024-09-10T10:49:29.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3432742482 shutdown_connections 2024-09-10T10:49:29.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f406dc80 0x7fb0f406e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3432742482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f40d4360 0x7fb0f4162990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3432742482 >> 172.21.15.28:0/3432742482 conn(0x7fb0f409d740 msgr2=0x7fb0f408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:29.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3432742482 shutdown_connections 2024-09-10T10:49:29.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.106+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3432742482 wait complete. 2024-09-10T10:49:29.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.108+0000 7fb0f3e00640 1 Processor -- start 2024-09-10T10:49:29.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.108+0000 7fb0f3e00640 1 -- start start 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f406e6b0 0x7fb0f40cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 0x7fb0f4160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f406d760 con 0x7fb0f406dc80 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f406d480 con 0x7fb0f406e6b0 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f415e6d0 con 0x7fb0f40d4360 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39560/0 (socket says 172.21.15.28:39560) 2024-09-10T10:49:29.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3940526837 learned_addr learned my addr 172.21.15.28:0/3940526837 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3400640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 0x7fb0f4160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 msgr2=0x7fb0f4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 0x7fb0f4160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f406e6b0 msgr2=0x7fb0f40cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f406e6b0 0x7fb0f40cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 -- 172.21.15.28:0/3940526837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f40cc5f0 con 0x7fb0f406dc80 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f3400640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 0x7fb0f4160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.109+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.110+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4037040 con 0x7fb0f406dc80 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.110+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e4032440 con 0x7fb0f406dc80 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.110+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40309b0 con 0x7fb0f406dc80 2024-09-10T10:49:29.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.110+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f40cc8d0 con 0x7fb0f406dc80 2024-09-10T10:49:29.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.110+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f40cce90 con 0x7fb0f406dc80 2024-09-10T10:49:29.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.111+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f406e110 con 0x7fb0f406dc80 2024-09-10T10:49:29.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.112+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0e4044030 con 0x7fb0f406dc80 2024-09-10T10:49:29.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.112+0000 7fb0eaa00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 0x7fb0d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:29.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.113+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5249+0+0 (secure 0 0 0) 0x7fb0e40aac60 con 0x7fb0f406dc80 2024-09-10T10:49:29.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.113+0000 7fb0f2000640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 0x7fb0d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:29.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.114+0000 7fb0f2000640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 0x7fb0d406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.117+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0e404b060 con 0x7fb0f406dc80 2024-09-10T10:49:29.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:29.301+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb0f40b2670 con 0x7fb0f406dc80 2024-09-10T10:49:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.097+0000 7fb0eaa00640 1 -- 172.21.15.28:0/3940526837 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fb0e40797c0 con 0x7fb0f406dc80 2024-09-10T10:49:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:49:30.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 msgr2=0x7fb0d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 0x7fb0d406e2f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007920 comp rx=0 tx=0).stop 2024-09-10T10:49:30.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 msgr2=0x7fb0f4153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40098a0 tx=0x7fb0e40046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 shutdown_connections 2024-09-10T10:49:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406be30 0x7fb0d406e2f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0f40d4360 0x7fb0f4160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0f406e6b0 0x7fb0f40cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 --2- 172.21.15.28:0/3940526837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0f406dc80 0x7fb0f4153b50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.101+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 >> 172.21.15.28:0/3940526837 conn(0x7fb0f409d740 msgr2=0x7fb0f40cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.102+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 shutdown_connections 2024-09-10T10:49:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.102+0000 7fb0f3e00640 1 -- 172.21.15.28:0/3940526837 wait complete. 2024-09-10T10:49:30.127 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:49:30.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.293+0000 7f7ca8c00640 1 Processor -- start 2024-09-10T10:49:30.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.293+0000 7f7ca8c00640 1 -- start start 2024-09-10T10:49:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca8c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ca415e350 0x7f7ca40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ca40d4a90 0x7f7ca40cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca8c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ca40cf1e0 con 0x7f7ca40d2500 2024-09-10T10:49:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca8c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca40d3690 con 0x7f7ca415e350 2024-09-10T10:49:30.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca8c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7ca40d2a80 con 0x7f7ca40d4a90 2024-09-10T10:49:30.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.294+0000 7f7ca2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7ca40d2500 0x7f7ca40d4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48144/0 (socket says 172.21.15.28:48144) 2024-09-10T10:49:30.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca2a00640 1 -- 172.21.15.28:0/3351811933 learned_addr learned my addr 172.21.15.28:0/3351811933 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3351811933 >> v2:172.21.15.28:3300/0 conn(0x7f7ca415e350 0x7f7ca40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3e00640 1 --2- 172.21.15.28:0/3351811933 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40d4a90 0x7f7ca40cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca2000640 1 -- 172.21.15.28:0/3351811933 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867864961 0 0) 0x7f7ca40cf1e0 con 0x7f7ca40d2500 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca2000640 1 -- 172.21.15.28:0/3351811933 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c88003520 con 0x7f7ca40d2500 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3400640 1 -- 172.21.15.28:0/3351811933 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40d4a90 msgr2=0x7f7ca40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3351811933 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40d4a90 0x7f7ca40cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3400640 1 -- 172.21.15.28:0/3351811933 >> v1:172.21.15.28:6789/0 conn(0x7f7ca40d2500 legacy=0x7f7ca40d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.295+0000 7f7ca3400640 1 -- 172.21.15.28:0/3351811933 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca40d2d60 con 0x7f7ca415e350 2024-09-10T10:49:30.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.296+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3351811933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca415e350 0x7f7ca40d1fc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40a2250 tx=0x7f7c9402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40c705ee91bde56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:30.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.296+0000 7f7ca2000640 1 -- 172.21.15.28:0/3351811933 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94037070 con 0x7f7ca415e350 2024-09-10T10:49:30.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.296+0000 7f7ca2000640 1 -- 172.21.15.28:0/3351811933 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c94032440 con 0x7f7ca415e350 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.296+0000 7f7ca2000640 1 -- 172.21.15.28:0/3351811933 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c94032810 con 0x7f7ca415e350 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3351811933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca415e350 msgr2=0x7f7ca40d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3351811933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca415e350 0x7f7ca40d1fc0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40a2250 tx=0x7f7c9402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3351811933 shutdown_connections 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3351811933 >> v2:172.21.15.138:3300/0 conn(0x7f7ca40d4a90 0x7f7ca40cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3351811933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca415e350 0x7f7ca40d1fc0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3351811933 >> 172.21.15.28:0/3351811933 conn(0x7f7ca409d710 msgr2=0x7f7ca408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3351811933 shutdown_connections 2024-09-10T10:49:30.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.297+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3351811933 wait complete. 2024-09-10T10:49:30.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.299+0000 7f7ca8c00640 1 Processor -- start 2024-09-10T10:49:30.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.299+0000 7f7ca8c00640 1 -- start start 2024-09-10T10:49:30.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 0x7f7ca4159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca415e350 0x7f7ca4160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca40b3620 con 0x7f7ca40d4a90 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca40d3320 con 0x7f7ca40d2500 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca8c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca415e730 con 0x7f7ca415e350 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39580/0 (socket says 172.21.15.28:39580) 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 -- 172.21.15.28:0/3326883471 learned_addr learned my addr 172.21.15.28:0/3326883471 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 0x7f7ca4159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca3e00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca415e350 0x7f7ca4160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 -- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca415e350 msgr2=0x7f7ca4160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca415e350 0x7f7ca4160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 -- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 msgr2=0x7f7ca4159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:30.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 0x7f7ca4159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca2a00640 1 -- 172.21.15.28:0/3326883471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca415a240 con 0x7f7ca40d4a90 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.300+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 0x7f7ca4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7ca2a00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c400 tx=0x7f7c9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90017070 con 0x7f7ca40d4a90 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90007500 con 0x7f7ca40d4a90 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900059e0 con 0x7f7ca40d4a90 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca41e7ac0 con 0x7f7ca40d4a90 2024-09-10T10:49:30.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.301+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca41e8080 con 0x7f7ca40d4a90 2024-09-10T10:49:30.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.302+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca40b5520 con 0x7f7ca40d4a90 2024-09-10T10:49:30.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.304+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7c9001a070 con 0x7f7ca40d4a90 2024-09-10T10:49:30.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.304+0000 7f7c9b400640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 0x7f7c8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:30.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.305+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5619+0+0 (secure 0 0 0) 0x7f7c9008a110 con 0x7f7ca40d4a90 2024-09-10T10:49:30.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.305+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 0x7f7c8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:30.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.306+0000 7f7ca3400640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 0x7f7c8406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7c94034000 tx=0x7f7c9402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:30.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.310+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ca40b5520 con 0x7f7ca40d4a90 2024-09-10T10:49:30.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:30.493+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7ca40ba370 con 0x7f7ca40d4a90 2024-09-10T10:49:31.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.103+0000 7f7c9b400640 1 -- 172.21.15.28:0/3326883471 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f7c900584c0 con 0x7f7ca40d4a90 2024-09-10T10:49:31.106 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:49:31.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 msgr2=0x7f7c8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 0x7f7c8406e2f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7c94034000 tx=0x7f7c9402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:31.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 msgr2=0x7f7ca415ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c400 tx=0x7f7c9000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 shutdown_connections 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7c8406be30 0x7f7c8406e2f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ca415e350 0x7f7ca4160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ca40d4a90 0x7f7ca415ffc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 --2- 172.21.15.28:0/3326883471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca40d2500 0x7f7ca4159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 >> 172.21.15.28:0/3326883471 conn(0x7f7ca409d710 msgr2=0x7f7ca40b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 shutdown_connections 2024-09-10T10:49:31.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.107+0000 7f7ca8c00640 1 -- 172.21.15.28:0/3326883471 wait complete. 2024-09-10T10:49:31.142 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:49:31.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.301+0000 7f091b800640 1 Processor -- start 2024-09-10T10:49:31.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.301+0000 7f091b800640 1 -- start start 2024-09-10T10:49:31.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.302+0000 7f091b800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09140d5090 0x7f09140cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.302+0000 7f091b800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140cfee0 con 0x7f091415e6c0 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.302+0000 7f091b800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09140d34f0 con 0x7f09140cfae0 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.302+0000 7f091b800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f09140cf400 con 0x7f09140d5090 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.302+0000 7f0913e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09140d5090 0x7f09140cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0913e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09140d5090 0x7f09140cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39594/0 (socket says 172.21.15.28:39594) 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0913e00640 1 -- 172.21.15.28:0/300765929 learned_addr learned my addr 172.21.15.28:0/300765929 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:31.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0918e00640 1 --1- 172.21.15.28:0/300765929 >> v1:172.21.15.28:6789/0 conn(0x7f091415e6c0 0x7f09140cebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48152/0 (socket says 172.21.15.28:48152) 2024-09-10T10:49:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0913e00640 1 -- 172.21.15.28:0/300765929 >> v1:172.21.15.138:6789/0 conn(0x7f09140cfae0 legacy=0x7f09140b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0913e00640 1 -- 172.21.15.28:0/300765929 >> v1:172.21.15.28:6789/0 conn(0x7f091415e6c0 legacy=0x7f09140cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.303+0000 7f0913e00640 1 -- 172.21.15.28:0/300765929 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09140d3790 con 0x7f09140d5090 2024-09-10T10:49:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.304+0000 7f0913e00640 1 --2- 172.21.15.28:0/300765929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09140d5090 0x7f09140cf5a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe109fc8f295e1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.304+0000 7f0913400640 1 -- 172.21.15.28:0/300765929 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900037070 con 0x7f09140d5090 2024-09-10T10:49:31.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.304+0000 7f0913400640 1 -- 172.21.15.28:0/300765929 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0900032440 con 0x7f09140d5090 2024-09-10T10:49:31.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.304+0000 7f0913400640 1 -- 172.21.15.28:0/300765929 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900032810 con 0x7f09140d5090 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 -- 172.21.15.28:0/300765929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09140d5090 msgr2=0x7f09140cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 --2- 172.21.15.28:0/300765929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09140d5090 0x7f09140cf5a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0900009870 tx=0x7f090002a270 comp rx=0 tx=0).stop 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 -- 172.21.15.28:0/300765929 shutdown_connections 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 --2- 172.21.15.28:0/300765929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09140d5090 0x7f09140cf5a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 -- 172.21.15.28:0/300765929 >> 172.21.15.28:0/300765929 conn(0x7f091408bfc0 msgr2=0x7f09140d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 -- 172.21.15.28:0/300765929 shutdown_connections 2024-09-10T10:49:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.305+0000 7f091b800640 1 -- 172.21.15.28:0/300765929 wait complete. 2024-09-10T10:49:31.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.307+0000 7f091b800640 1 Processor -- start 2024-09-10T10:49:31.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.307+0000 7f091b800640 1 -- start start 2024-09-10T10:49:31.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.307+0000 7f091b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09140cfae0 0x7f0914159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:31.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.307+0000 7f091b800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09140d5090 0x7f09141566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f091b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f091b800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f091415eaa0 con 0x7f091415e6c0 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f091b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09140b10d0 con 0x7f09140cfae0 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f091b800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f091415ec20 con 0x7f09140d5090 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39598/0 (socket says 172.21.15.28:39598) 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 -- 172.21.15.28:0/2804891738 learned_addr learned my addr 172.21.15.28:0/2804891738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0918e00640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09140cfae0 0x7f0914159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0913e00640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09140d5090 0x7f09141566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 -- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09140d5090 msgr2=0x7f09141566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09140d5090 0x7f09141566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 -- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09140cfae0 msgr2=0x7f0914159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:31.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09140cfae0 0x7f0914159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:31.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.308+0000 7f0919800640 1 -- 172.21.15.28:0/2804891738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0914157010 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f0919800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c6b0 tx=0x7f090c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c003b70 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f090c003d30 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c005310 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091415a4a0 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.309+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141f49c0 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.310+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09140cf5a0 con 0x7f091415e6c0 2024-09-10T10:49:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.311+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f090c0054b0 con 0x7f091415e6c0 2024-09-10T10:49:31.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.312+0000 7f0910c00640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:31.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.312+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 6001+0+0 (secure 0 0 0) 0x7f090c08ad40 con 0x7f091415e6c0 2024-09-10T10:49:31.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.312+0000 7f0918e00640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:31.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.313+0000 7f0918e00640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 0x7f08f406e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0908000f60 tx=0x7f0908000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:31.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.317+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f090c058ed0 con 0x7f091415e6c0 2024-09-10T10:49:31.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:31.537+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f09141e82f0 con 0x7f091415e6c0 2024-09-10T10:49:32.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.106+0000 7f0910c00640 1 -- 172.21.15.28:0/2804891738 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v64) ==== 174+0+0 (secure 0 0 0) 0x7f090c05d1d0 con 0x7f091415e6c0 2024-09-10T10:49:32.108 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 11 and data pool 12 2024-09-10T10:49:32.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.109+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 msgr2=0x7f08f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.109+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 0x7f08f406e2f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0908000f60 tx=0x7f0908000ed0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.109+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 msgr2=0x7f09141f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.109+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f090c00c6b0 tx=0x7f090c00cb70 comp rx=0 tx=0).stop 2024-09-10T10:49:32.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 shutdown_connections 2024-09-10T10:49:32.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f08f406be30 0x7f08f406e2f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f091415e6c0 0x7f09141f51c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09140d5090 0x7f09141566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 --2- 172.21.15.28:0/2804891738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09140cfae0 0x7f0914159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 >> 172.21.15.28:0/2804891738 conn(0x7f091408bfc0 msgr2=0x7f0914162260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:32.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.110+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 shutdown_connections 2024-09-10T10:49:32.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.111+0000 7f091b800640 1 -- 172.21.15.28:0/2804891738 wait complete. 2024-09-10T10:49:32.146 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:32.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.307+0000 7f8002200640 1 Processor -- start 2024-09-10T10:49:32.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.307+0000 7f8002200640 1 -- start start 2024-09-10T10:49:32.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8002200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc0d4180 0x7f7ffc0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8002200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ffc15e6a0 0x7f7ffc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8002200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc0d4aa0 con 0x7f7ffc06dd60 2024-09-10T10:49:32.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8002200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7ffc0cefb0 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8002200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0d2c20 con 0x7f7ffc0d4180 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f7ffbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ffc15e6a0 0x7f7ffc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8000e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc0d4180 0x7f7ffc0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8001800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7ffc06dd60 0x7f7ffc0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52198/0 (socket says 172.21.15.28:52198) 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f7ffbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7ffc15e6a0 0x7f7ffc06d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:37976/0 (socket says 172.21.15.28:37976) 2024-09-10T10:49:32.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.308+0000 7f8001800640 1 -- 172.21.15.28:0/646971615 learned_addr learned my addr 172.21.15.28:0/646971615 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.309+0000 7f7ffbe00640 1 -- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc0d4180 msgr2=0x7f7ffc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.309+0000 7f7ffbe00640 1 --2- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc0d4180 0x7f7ffc0d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.309+0000 7f7ffbe00640 1 -- 172.21.15.28:0/646971615 >> v1:172.21.15.138:6789/0 conn(0x7f7ffc06dd60 legacy=0x7f7ffc0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.309+0000 7f7ffbe00640 1 -- 172.21.15.28:0/646971615 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc06e390 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.310+0000 7f7ffbe00640 1 --2- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc06d820 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80097a0 tx=0x7f7fe802a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=799321476e90fa1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.310+0000 7f7ffb400640 1 -- 172.21.15.28:0/646971615 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8037070 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.310+0000 7f7ffb400640 1 -- 172.21.15.28:0/646971615 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe802ae40 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.310+0000 7f7ffb400640 1 -- 172.21.15.28:0/646971615 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8032660 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 -- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc15e6a0 msgr2=0x7f7ffc06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 --2- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc06d820 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80097a0 tx=0x7f7fe802a0f0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 -- 172.21.15.28:0/646971615 shutdown_connections 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 --2- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc06d820 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 --2- 172.21.15.28:0/646971615 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc0d4180 0x7f7ffc0d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 -- 172.21.15.28:0/646971615 >> 172.21.15.28:0/646971615 conn(0x7f7ffc09d6f0 msgr2=0x7f7ffc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 -- 172.21.15.28:0/646971615 shutdown_connections 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.311+0000 7f8002200640 1 -- 172.21.15.28:0/646971615 wait complete. 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.312+0000 7f8002200640 1 Processor -- start 2024-09-10T10:49:32.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.312+0000 7f8002200640 1 -- start start 2024-09-10T10:49:32.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc06dd60 0x7f7ffc15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc0d4180 0x7f7ffc0b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0cebd0 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc06e950 con 0x7f7ffc0d4180 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.313+0000 7f8002200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0d0450 con 0x7f7ffc06dd60 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39602/0 (socket says 172.21.15.28:39602) 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 -- 172.21.15.28:0/2200400840 learned_addr learned my addr 172.21.15.28:0/2200400840 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f7ffbe00640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc0d4180 0x7f7ffc0b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8000e00640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc06dd60 0x7f7ffc15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 -- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc06dd60 msgr2=0x7f7ffc15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc06dd60 0x7f7ffc15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 -- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc0d4180 msgr2=0x7f7ffc0b91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc0d4180 0x7f7ffc0b91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 -- 172.21.15.28:0/2200400840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc09ece0 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.314+0000 7f8001800640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cbf0 tx=0x7f7ff400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.315+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4017070 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.315+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff400ad40 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.315+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4012600 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.315+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc0cc830 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.315+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc0ccdf0 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.316+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc09eeb0 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.317+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ff401a070 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.318+0000 7f7ff8c00640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 0x7f7fdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.318+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7f7ff408ac50 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.318+0000 7f8000e00640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 0x7f7fdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.319+0000 7f8000e00640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 0x7f7fdc06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009b40 tx=0x7f7ff0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.322+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ff4090020 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.506+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ffc0d4560 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.508+0000 7f7ff8c00640 1 -- 172.21.15.28:0/2200400840 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16982 (secure 0 0 0) 0x7f7ff4058e20 con 0x7f7ffc15e6a0 2024-09-10T10:49:32.510 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:32.511 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":85,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:49:31.540093+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:49:29.344267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T10:49:30.537368+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:32.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 msgr2=0x7f7fdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 0x7f7fdc06e2f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009b40 tx=0x7f7ff0009190 comp rx=0 tx=0).stop 2024-09-10T10:49:32.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 msgr2=0x7f7ffc0b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400cbf0 tx=0x7f7ff400a430 comp rx=0 tx=0).stop 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 shutdown_connections 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fdc06be30 0x7f7fdc06e2f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ffc15e6a0 0x7f7ffc0b9720 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ffc0d4180 0x7f7ffc0b91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 --2- 172.21.15.28:0/2200400840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7ffc06dd60 0x7f7ffc15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 >> 172.21.15.28:0/2200400840 conn(0x7f7ffc09d6f0 msgr2=0x7f7ffc0b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 shutdown_connections 2024-09-10T10:49:32.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.510+0000 7f8002200640 1 -- 172.21.15.28:0/2200400840 wait complete. 2024-09-10T10:49:32.528 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:32.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.682+0000 7fccb3000640 1 Processor -- start 2024-09-10T10:49:32.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.682+0000 7fccb3000640 1 -- start start 2024-09-10T10:49:32.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.683+0000 7fccb3000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.683+0000 7fccb3000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccac146140 con 0x7fccac072700 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.683+0000 7fccb3000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccac0cd740 con 0x7fccac072320 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.683+0000 7fccb3000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccac0cda20 con 0x7fccac072de0 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccb1000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fccac072700 0x7fccac15d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48168/0 (socket says 172.21.15.28:48168) 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccb1000640 1 -- 172.21.15.28:0/3286305253 learned_addr learned my addr 172.21.15.28:0/3286305253 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:32.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fcca3400640 1 --2- 172.21.15.28:0/3286305253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395522699 0 0) 0x7fccac146140 con 0x7fccac072700 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc90003520 con 0x7fccac072700 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122380212 0 0) 0x7fccac0cd740 con 0x7fccac072320 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccac146140 con 0x7fccac072320 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fcca3400640 1 -- 172.21.15.28:0/3286305253 >> v1:172.21.15.138:6789/0 conn(0x7fccac072320 legacy=0x7fccac0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fcca3400640 1 -- 172.21.15.28:0/3286305253 >> v1:172.21.15.28:6789/0 conn(0x7fccac072700 legacy=0x7fccac15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.684+0000 7fcca3400640 1 -- 172.21.15.28:0/3286305253 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac0cdd00 con 0x7fccac072de0 2024-09-10T10:49:32.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.685+0000 7fcca3400640 1 --2- 172.21.15.28:0/3286305253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac15d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097d0 tx=0x7fcc9402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a90ddc3af89dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037070 con 0x7fccac072de0 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc94032440 con 0x7fccac072de0 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccab400640 1 -- 172.21.15.28:0/3286305253 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94032810 con 0x7fccac072de0 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 -- 172.21.15.28:0/3286305253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 msgr2=0x7fccac15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 --2- 172.21.15.28:0/3286305253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac15d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097d0 tx=0x7fcc9402a180 comp rx=0 tx=0).stop 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 -- 172.21.15.28:0/3286305253 shutdown_connections 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 --2- 172.21.15.28:0/3286305253 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac15d1e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 -- 172.21.15.28:0/3286305253 >> 172.21.15.28:0/3286305253 conn(0x7fccac067d70 msgr2=0x7fccac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.686+0000 7fccb3000640 1 -- 172.21.15.28:0/3286305253 shutdown_connections 2024-09-10T10:49:32.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.687+0000 7fccb3000640 1 -- 172.21.15.28:0/3286305253 wait complete. 2024-09-10T10:49:32.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.688+0000 7fccb3000640 1 Processor -- start 2024-09-10T10:49:32.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.688+0000 7fccb3000640 1 -- start start 2024-09-10T10:49:32.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac072320 0x7fccac0abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac1e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fccac15dfb0 con 0x7fccac072700 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccac15de30 con 0x7fccac072320 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fccb3000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccac15e130 con 0x7fccac072de0 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fcca3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fcca3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39610/0 (socket says 172.21.15.28:39610) 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.689+0000 7fcca3400640 1 -- 172.21.15.28:0/4178618094 learned_addr learned my addr 172.21.15.28:0/4178618094 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fccb1000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac1e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fccabe00640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac072320 0x7fccac0abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 -- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 msgr2=0x7fccac1e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac1e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 -- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac072320 msgr2=0x7fccac0abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac072320 0x7fccac0abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 -- 172.21.15.28:0/4178618094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac0aa6d0 con 0x7fccac072700 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca3400640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097d0 tx=0x7fcc940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.690+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94041020 con 0x7fccac072700 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.691+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9403d640 con 0x7fccac072700 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.691+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94037040 con 0x7fccac072700 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.691+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac0aa9b0 con 0x7fccac072700 2024-09-10T10:49:32.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.691+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac1e3460 con 0x7fccac072700 2024-09-10T10:49:32.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.692+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac0cce60 con 0x7fccac072700 2024-09-10T10:49:32.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.693+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcc94044020 con 0x7fccac072700 2024-09-10T10:49:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.693+0000 7fcca8c00640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 0x7fcc8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.694+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7fcc940ab800 con 0x7fccac072700 2024-09-10T10:49:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.694+0000 7fccabe00640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 0x7fcc8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:32.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.695+0000 7fccabe00640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 0x7fcc8806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c001650 tx=0x7fcc9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:32.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.698+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc9404b030 con 0x7fccac072700 2024-09-10T10:49:32.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.881+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccac06fed0 con 0x7fccac072700 2024-09-10T10:49:32.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.883+0000 7fcca8c00640 1 -- 172.21.15.28:0/4178618094 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16982 (secure 0 0 0) 0x7fcc9407a0b0 con 0x7fccac072700 2024-09-10T10:49:32.886 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:32.886 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":85,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:49:31.540093+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:49:29.344267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T10:49:30.537368+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 msgr2=0x7fcc8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 0x7fcc8806e2f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c001650 tx=0x7fcc9c009040 comp rx=0 tx=0).stop 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 msgr2=0x7fccac0a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcc940097d0 tx=0x7fcc940046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 shutdown_connections 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcc8806be30 0x7fcc8806e2f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac072de0 0x7fccac1e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccac072700 0x7fccac0a9ea0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 --2- 172.21.15.28:0/4178618094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac072320 0x7fccac0abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:32.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 >> 172.21.15.28:0/4178618094 conn(0x7fccac067d70 msgr2=0x7fccac0cef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:32.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 shutdown_connections 2024-09-10T10:49:32.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:32.885+0000 7fccb3000640 1 -- 172.21.15.28:0/4178618094 wait complete. 2024-09-10T10:49:32.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:49:33.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.066+0000 7ffa17600640 1 Processor -- start 2024-09-10T10:49:33.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.066+0000 7ffa17600640 1 -- start start 2024-09-10T10:49:33.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.067+0000 7ffa17600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa1015e900 0x7ffa100b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.067+0000 7ffa17600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa100b4a20 0x7ffa100ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.067+0000 7ffa17600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa100b4620 con 0x7ffa10072050 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.067+0000 7ffa17600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa100ce100 con 0x7ffa100b4a20 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.067+0000 7ffa17600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa100ce3a0 con 0x7ffa1015e900 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa100b4a20 0x7ffa100ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa100b4a20 0x7ffa100ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39618/0 (socket says 172.21.15.28:39618) 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3581697268 learned_addr learned my addr 172.21.15.28:0/3581697268 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa14c00640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa1015e900 0x7ffa100b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3581697268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa1015e900 msgr2=0x7ffa100b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa1015e900 0x7ffa100b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3581697268 >> v1:172.21.15.138:6789/0 conn(0x7ffa10072050 legacy=0x7ffa100cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:33.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.068+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3581697268 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa100ce680 con 0x7ffa100b4a20 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.069+0000 7ffa0fe00640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100ad020 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00bb10 tx=0x7ff9fc02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53be2da386553842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.069+0000 7ffa0f400640 1 -- 172.21.15.28:0/3581697268 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc032790 con 0x7ffa100b4a20 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa0f400640 1 -- 172.21.15.28:0/3581697268 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc038070 con 0x7ffa100b4a20 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa0f400640 1 -- 172.21.15.28:0/3581697268 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc032b50 con 0x7ffa100b4a20 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 -- 172.21.15.28:0/3581697268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 msgr2=0x7ffa100ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100ad020 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00bb10 tx=0x7ff9fc02c890 comp rx=0 tx=0).stop 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 -- 172.21.15.28:0/3581697268 shutdown_connections 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100ad020 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:33.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 --2- 172.21.15.28:0/3581697268 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa1015e900 0x7ffa100b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 -- 172.21.15.28:0/3581697268 >> 172.21.15.28:0/3581697268 conn(0x7ffa1006bdf0 msgr2=0x7ffa1006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 -- 172.21.15.28:0/3581697268 shutdown_connections 2024-09-10T10:49:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.070+0000 7ffa17600640 1 -- 172.21.15.28:0/3581697268 wait complete. 2024-09-10T10:49:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.072+0000 7ffa17600640 1 Processor -- start 2024-09-10T10:49:33.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.072+0000 7ffa17600640 1 -- start start 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa10072050 0x7ffa10153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 0x7ffa101e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100ad560 con 0x7ffa100b4a20 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100b4e20 con 0x7ffa1015e900 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa17600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa100cf590 con 0x7ffa10072050 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa0fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa0fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:39634/0 (socket says 172.21.15.28:39634) 2024-09-10T10:49:33.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3465471171 learned_addr learned my addr 172.21.15.28:0/3465471171 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.073+0000 7ffa14c00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa10072050 0x7ffa10153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa15600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 0x7ffa101e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa10072050 msgr2=0x7ffa10153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa10072050 0x7ffa10153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 msgr2=0x7ffa101e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 0x7ffa101e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 -- 172.21.15.28:0/3465471171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101e8af0 con 0x7ffa100b4a20 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa15600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 0x7ffa101e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0fe00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc043070 con 0x7ffa100b4a20 2024-09-10T10:49:33.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc03c660 con 0x7ffa100b4a20 2024-09-10T10:49:33.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc03ca50 con 0x7ffa100b4a20 2024-09-10T10:49:33.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.074+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa100b05d0 con 0x7ffa100b4a20 2024-09-10T10:49:33.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.075+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa100b0b90 con 0x7ffa100b4a20 2024-09-10T10:49:33.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.076+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa100ad020 con 0x7ffa100b4a20 2024-09-10T10:49:33.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.076+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9fc046070 con 0x7ffa100b4a20 2024-09-10T10:49:33.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.077+0000 7ffa0cc00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:33.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.077+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 6101+0+0 (secure 0 0 0) 0x7ff9fc0ab3e0 con 0x7ffa100b4a20 2024-09-10T10:49:33.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.078+0000 7ffa14c00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:33.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.078+0000 7ffa14c00640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 0x7ff9f006e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffa100ad450 tx=0x7ffa04009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:33.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.081+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff9fc079510 con 0x7ffa100b4a20 2024-09-10T10:49:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:33.305+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffa1015b5d0 con 0x7ffa100b4a20 2024-09-10T10:49:33.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T10:49:33.767 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-10T10:49:34.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.140+0000 7ffa0cc00640 1 -- 172.21.15.28:0/3465471171 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v67) ==== 118+0+0 (secure 0 0 0) 0x7ff9fc07d810 con 0x7ffa100b4a20 2024-09-10T10:49:34.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 msgr2=0x7ff9f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:34.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 0x7ff9f006e2f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffa100ad450 tx=0x7ffa04009190 comp rx=0 tx=0).stop 2024-09-10T10:49:34.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 msgr2=0x7ffa100b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc004730 comp rx=0 tx=0).stop 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 shutdown_connections 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff9f006be30 0x7ff9f006e2f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa1015e900 0x7ffa101e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa100b4a20 0x7ffa100b1390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 --2- 172.21.15.28:0/3465471171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa10072050 0x7ffa10153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.143+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 >> 172.21.15.28:0/3465471171 conn(0x7ffa1006bdf0 msgr2=0x7ffa100ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.144+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 shutdown_connections 2024-09-10T10:49:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.144+0000 7ffa17600640 1 -- 172.21.15.28:0/3465471171 wait complete. 2024-09-10T10:49:34.164 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:49:34.164 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:49:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.324+0000 7ff4d4a00640 1 Processor -- start 2024-09-10T10:49:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.325+0000 7ff4d4a00640 1 -- start start 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff4d0156740 0x7ff4d00b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d00cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4d00cfcd0 0x7ff4d015fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d006edc0 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d00b30e0 con 0x7ff4d0156740 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4d4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d00b33c0 con 0x7ff4d00b4af0 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4ce800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4d00cfcd0 0x7ff4d015fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4ce800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4d00cfcd0 0x7ff4d015fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50590/0 (socket says 172.21.15.28:50590) 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4ce800640 1 -- 172.21.15.28:0/1396342081 learned_addr learned my addr 172.21.15.28:0/1396342081 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4cde00640 1 --2- 172.21.15.28:0/1396342081 >> v2:172.21.15.138:3300/0 conn(0x7ff4d0156740 0x7ff4d00b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.326+0000 7ff4c7e00640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d00cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.327+0000 7ff4ce800640 1 -- 172.21.15.28:0/1396342081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 msgr2=0x7ff4d00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.327+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d00cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.327+0000 7ff4ce800640 1 -- 172.21.15.28:0/1396342081 >> v2:172.21.15.138:3300/0 conn(0x7ff4d0156740 msgr2=0x7ff4d00b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.327+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1396342081 >> v2:172.21.15.138:3300/0 conn(0x7ff4d0156740 0x7ff4d00b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.327+0000 7ff4ce800640 1 -- 172.21.15.28:0/1396342081 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d00b36a0 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d015fe20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00b4150 tx=0x7ff4c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750eb0715a8ad5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4cd400640 1 -- 172.21.15.28:0/1396342081 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8036070 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4cd400640 1 -- 172.21.15.28:0/1396342081 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c8031600 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4cd400640 1 -- 172.21.15.28:0/1396342081 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80319d0 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1396342081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d00cfcd0 msgr2=0x7ff4d015fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.328+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d015fe20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00b4150 tx=0x7ff4c802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1396342081 shutdown_connections 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d015fe20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1396342081 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d00cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1396342081 >> v2:172.21.15.138:3300/0 conn(0x7ff4d0156740 0x7ff4d00b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1396342081 >> 172.21.15.28:0/1396342081 conn(0x7ff4d006bdf0 msgr2=0x7ff4d006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1396342081 shutdown_connections 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.329+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1396342081 wait complete. 2024-09-10T10:49:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.330+0000 7ff4d4a00640 1 Processor -- start 2024-09-10T10:49:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.330+0000 7ff4d4a00640 1 -- start start 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d009d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00d0890 con 0x7ff4d0156740 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d00b3c60 con 0x7ff4d00cfcd0 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.331+0000 7ff4d4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d00d0a10 con 0x7ff4d00b4af0 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4cde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d009d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50592/0 (socket says 172.21.15.28:50592) 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 -- 172.21.15.28:0/1956776264 learned_addr learned my addr 172.21.15.28:0/1956776264 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4c7e00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 -- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 msgr2=0x7ff4d009d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d009d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 -- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d00cfcd0 msgr2=0x7ff4d014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:34.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 -- 172.21.15.28:0/1956776264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d01507b0 con 0x7ff4d0156740 2024-09-10T10:49:34.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.332+0000 7ff4ce800640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009f40 tx=0x7ff4c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:34.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.333+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8036050 con 0x7ff4d0156740 2024-09-10T10:49:34.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.333+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c8042070 con 0x7ff4d0156740 2024-09-10T10:49:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.333+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8009510 con 0x7ff4d0156740 2024-09-10T10:49:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.333+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d009dd70 con 0x7ff4d0156740 2024-09-10T10:49:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.333+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d0150a70 con 0x7ff4d0156740 2024-09-10T10:49:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.334+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d0150250 con 0x7ff4d0156740 2024-09-10T10:49:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.335+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4c8007bf0 con 0x7ff4d0156740 2024-09-10T10:49:34.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.336+0000 7ff4c6000640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 0x7ff4ac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:34.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.336+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff4c80abc40 con 0x7ff4d0156740 2024-09-10T10:49:34.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.336+0000 7ff4cde00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 0x7ff4ac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:34.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.337+0000 7ff4cde00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 0x7ff4ac06e320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0010c60 tx=0x7ff4c0012040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:34.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.340+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff4c8079e90 con 0x7ff4d0156740 2024-09-10T10:49:34.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:34.563+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff4d00b0190 con 0x7ff4d0156740 2024-09-10T10:49:35.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.145+0000 7ff4c6000640 1 -- 172.21.15.28:0/1956776264 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v68) ==== 122+0+0 (secure 0 0 0) 0x7ff4c807e190 con 0x7ff4d0156740 2024-09-10T10:49:35.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 msgr2=0x7ff4ac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 0x7ff4ac06e320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0010c60 tx=0x7ff4c0012040 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 msgr2=0x7ff4d0158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009f40 tx=0x7ff4c8004a40 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 shutdown_connections 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4ac06be60 0x7ff4ac06e320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4d0156740 0x7ff4d0158010 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4d00cfcd0 0x7ff4d014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 --2- 172.21.15.28:0/1956776264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff4d00b4af0 0x7ff4d009d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 >> 172.21.15.28:0/1956776264 conn(0x7ff4d006bdf0 msgr2=0x7ff4d00ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 shutdown_connections 2024-09-10T10:49:35.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.148+0000 7ff4d4a00640 1 -- 172.21.15.28:0/1956776264 wait complete. 2024-09-10T10:49:35.169 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:49:35.170 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:35.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.327+0000 7fdf8ec00640 1 Processor -- start 2024-09-10T10:49:35.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.327+0000 7fdf8ec00640 1 -- start start 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf8ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf880cfcc0 0x7fdf8806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf8ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880d4fd0 0x7fdf880d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf8ec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf880d2700 con 0x7fdf8815e760 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf8ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdf880cedb0 con 0x7fdf880cfcc0 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf8ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf880cf090 con 0x7fdf880d4fd0 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.328+0000 7fdf87400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf880cfcc0 0x7fdf8806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf880cfcc0 0x7fdf8806cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50606/0 (socket says 172.21.15.28:50606) 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 -- 172.21.15.28:0/1051741380 learned_addr learned my addr 172.21.15.28:0/1051741380 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:35.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880d4fd0 0x7fdf880d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 -- 172.21.15.28:0/1051741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880d4fd0 msgr2=0x7fdf880d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:35.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880d4fd0 0x7fdf880d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 -- 172.21.15.28:0/1051741380 >> v1:172.21.15.138:6789/0 conn(0x7fdf8815e760 legacy=0x7fdf880cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:35.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.329+0000 7fdf87400640 1 -- 172.21.15.28:0/1051741380 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf880b3c80 con 0x7fdf880cfcc0 2024-09-10T10:49:35.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.330+0000 7fdf87400640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880cfcc0 0x7fdf8806cdd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf7402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205f8777bdeff38c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.330+0000 7fdf86a00640 1 -- 172.21.15.28:0/1051741380 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74004030 con 0x7fdf880cfcc0 2024-09-10T10:49:35.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.330+0000 7fdf86a00640 1 -- 172.21.15.28:0/1051741380 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf740041d0 con 0x7fdf880cfcc0 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.330+0000 7fdf86a00640 1 -- 172.21.15.28:0/1051741380 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74003610 con 0x7fdf880cfcc0 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 -- 172.21.15.28:0/1051741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880cfcc0 msgr2=0x7fdf8806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880cfcc0 0x7fdf8806cdd0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009850 tx=0x7fdf7402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 -- 172.21.15.28:0/1051741380 shutdown_connections 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880d4fd0 0x7fdf880d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/1051741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880cfcc0 0x7fdf8806cdd0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 -- 172.21.15.28:0/1051741380 >> 172.21.15.28:0/1051741380 conn(0x7fdf8808bfc0 msgr2=0x7fdf880d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 -- 172.21.15.28:0/1051741380 shutdown_connections 2024-09-10T10:49:35.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.331+0000 7fdf8ec00640 1 -- 172.21.15.28:0/1051741380 wait complete. 2024-09-10T10:49:35.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.332+0000 7fdf8ec00640 1 Processor -- start 2024-09-10T10:49:35.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 -- start start 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 0x7fdf881f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 0x7fdf880d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880d33e0 con 0x7fdf880d4fd0 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf880b38e0 con 0x7fdf8815e760 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.333+0000 7fdf8ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf880d3560 con 0x7fdf880cfcc0 2024-09-10T10:49:35.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50618/0 (socket says 172.21.15.28:50618) 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 -- 172.21.15.28:0/2224623755 learned_addr learned my addr 172.21.15.28:0/2224623755 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87e00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 0x7fdf881f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 0x7fdf880d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 -- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 msgr2=0x7fdf881f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 0x7fdf881f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 -- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 msgr2=0x7fdf880d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 0x7fdf880d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 -- 172.21.15.28:0/2224623755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8815bb60 con 0x7fdf880d4fd0 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87e00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 0x7fdf881f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf8cc00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 0x7fdf880d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:35.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.334+0000 7fdf87400640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf7402a220 tx=0x7fdf740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.335+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74041020 con 0x7fdf880d4fd0 2024-09-10T10:49:35.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.335+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf74003c50 con 0x7fdf880d4fd0 2024-09-10T10:49:35.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.335+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf74004030 con 0x7fdf880d4fd0 2024-09-10T10:49:35.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.335+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8815be40 con 0x7fdf880d4fd0 2024-09-10T10:49:35.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.335+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8815c400 con 0x7fdf880d4fd0 2024-09-10T10:49:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.336+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf880d38e0 con 0x7fdf880d4fd0 2024-09-10T10:49:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.337+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf74044020 con 0x7fdf880d4fd0 2024-09-10T10:49:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.338+0000 7fdf7fe00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.338+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fdf740abc90 con 0x7fdf880d4fd0 2024-09-10T10:49:35.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.338+0000 7fdf87e00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.339+0000 7fdf87e00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.342+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf7404b030 con 0x7fdf880d4fd0 2024-09-10T10:49:35.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.564+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf880b6d20 con 0x7fdf880d4fd0 2024-09-10T10:49:35.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.564+0000 7fdf7fe00640 1 -- 172.21.15.28:0/2224623755 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7fdf880d2700 con 0x7fdf880d4fd0 2024-09-10T10:49:35.566 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:35.567 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"btime":"2024-09-10T10:49:35:142470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:34.565062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 68 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 msgr2=0x7fdf6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf78008040 comp rx=0 tx=0).stop 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 msgr2=0x7fdf880cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fdf7402a220 tx=0x7fdf740046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 shutdown_connections 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf6806be30 0x7fdf6806e2f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf8815e760 0x7fdf880d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf880d4fd0 0x7fdf880cd050 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 --2- 172.21.15.28:0/2224623755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf880cfcc0 0x7fdf881f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.566+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 >> 172.21.15.28:0/2224623755 conn(0x7fdf8808bfc0 msgr2=0x7fdf880b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.567+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 shutdown_connections 2024-09-10T10:49:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.567+0000 7fdf8ec00640 1 -- 172.21.15.28:0/2224623755 wait complete. 2024-09-10T10:49:35.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:49:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.738+0000 7f9292400640 1 Processor -- start 2024-09-10T10:49:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.738+0000 7f9292400640 1 -- start start 2024-09-10T10:49:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f9292400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f928c15e660 0x7f928c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f9292400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f9292400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f928c0cf000 con 0x7f928c0d0dd0 2024-09-10T10:49:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f9292400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f928c0b37d0 con 0x7f928c15e660 2024-09-10T10:49:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f9292400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f928c0cf330 con 0x7f928c0d1f90 2024-09-10T10:49:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f928be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f928c15e660 0x7f928c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f928be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f928c15e660 0x7f928c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50630/0 (socket says 172.21.15.28:50630) 2024-09-10T10:49:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.739+0000 7f928be00640 1 -- 172.21.15.28:0/637309776 learned_addr learned my addr 172.21.15.28:0/637309776 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f9291a00640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f928be00640 1 -- 172.21.15.28:0/637309776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 msgr2=0x7f928c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f928be00640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f928be00640 1 -- 172.21.15.28:0/637309776 >> v1:172.21.15.28:6789/0 conn(0x7f928c0d0dd0 legacy=0x7f928c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f928be00640 1 -- 172.21.15.28:0/637309776 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c0d29a0 con 0x7f928c15e660 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.740+0000 7f9291a00640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.741+0000 7f928be00640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c06ccd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9278009870 tx=0x7f927802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfb7018c10d9dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.741+0000 7f928b400640 1 -- 172.21.15.28:0/637309776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278037070 con 0x7f928c15e660 2024-09-10T10:49:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.741+0000 7f928b400640 1 -- 172.21.15.28:0/637309776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9278032440 con 0x7f928c15e660 2024-09-10T10:49:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.741+0000 7f928b400640 1 -- 172.21.15.28:0/637309776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278032810 con 0x7f928c15e660 2024-09-10T10:49:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 -- 172.21.15.28:0/637309776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 msgr2=0x7f928c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c06ccd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9278009870 tx=0x7f927802a250 comp rx=0 tx=0).stop 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 -- 172.21.15.28:0/637309776 shutdown_connections 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 --2- 172.21.15.28:0/637309776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c06ccd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 -- 172.21.15.28:0/637309776 >> 172.21.15.28:0/637309776 conn(0x7f928c08bfc0 msgr2=0x7f928c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 -- 172.21.15.28:0/637309776 shutdown_connections 2024-09-10T10:49:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.742+0000 7f9292400640 1 -- 172.21.15.28:0/637309776 wait complete. 2024-09-10T10:49:35.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.743+0000 7f9292400640 1 Processor -- start 2024-09-10T10:49:35.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.743+0000 7f9292400640 1 -- start start 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c0d0dd0 0x7f928c1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f928c0d3220 con 0x7f928c15e660 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f928c0d2f40 con 0x7f928c0d0dd0 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.744+0000 7f9292400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f928c0d33a0 con 0x7f928c0d1f90 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50644/0 (socket says 172.21.15.28:50644) 2024-09-10T10:49:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 -- 172.21.15.28:0/2263555032 learned_addr learned my addr 172.21.15.28:0/2263555032 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291000640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c0d0dd0 0x7f928c1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f928be00640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 -- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 msgr2=0x7f928c0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 -- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c0d0dd0 msgr2=0x7f928c1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c0d0dd0 0x7f928c1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 -- 172.21.15.28:0/2263555032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c0b87d0 con 0x7f928c15e660 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.745+0000 7f9291a00640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f928400cf20 tx=0x7f928400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.746+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928401a070 con 0x7f928c15e660 2024-09-10T10:49:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.746+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928400d840 con 0x7f928c15e660 2024-09-10T10:49:35.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.746+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928400dc30 con 0x7f928c15e660 2024-09-10T10:49:35.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.746+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c0b8ab0 con 0x7f928c15e660 2024-09-10T10:49:35.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.746+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c0b9070 con 0x7f928c15e660 2024-09-10T10:49:35.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.747+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c0d2440 con 0x7f928c15e660 2024-09-10T10:49:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.748+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f928401d070 con 0x7f928c15e660 2024-09-10T10:49:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.749+0000 7f9288c00640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 0x7f926c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.749+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f928408b7f0 con 0x7f928c15e660 2024-09-10T10:49:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.749+0000 7f9291000640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 0x7f926c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:35.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.750+0000 7f9291000640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 0x7f926c06e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9280001780 tx=0x7f9280009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.753+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92840599c0 con 0x7f928c15e660 2024-09-10T10:49:35.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.938+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f928c0cab90 con 0x7f928c15e660 2024-09-10T10:49:35.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.939+0000 7f9288c00640 1 -- 172.21.15.28:0/2263555032 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16969 (secure 0 0 0) 0x7f928405dcc0 con 0x7f928c15e660 2024-09-10T10:49:35.941 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:35.942 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":86,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:49:33.123820+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:49:29.344267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T10:49:30.537368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:49:35.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 msgr2=0x7f926c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 0x7f926c06e2f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9280001780 tx=0x7f9280009040 comp rx=0 tx=0).stop 2024-09-10T10:49:35.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 msgr2=0x7f928c156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f928400cf20 tx=0x7f928400a7d0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 shutdown_connections 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f926c06be30 0x7f926c06e2f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f928c15e660 0x7f928c156560 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f928c0d1f90 0x7f928c0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 --2- 172.21.15.28:0/2263555032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f928c0d0dd0 0x7f928c1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 >> 172.21.15.28:0/2263555032 conn(0x7f928c08bfc0 msgr2=0x7f928c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 shutdown_connections 2024-09-10T10:49:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:35.941+0000 7f9292400640 1 -- 172.21.15.28:0/2263555032 wait complete. 2024-09-10T10:49:35.959 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:49:35.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:49:36.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.113+0000 7efc15e00640 1 Processor -- start 2024-09-10T10:49:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.113+0000 7efc15e00640 1 -- start start 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc15e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efc10165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc15e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc100cdbe0 con 0x7efc10072d30 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc15e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc100cd900 con 0x7efc1015b300 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc15e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc100728a0 con 0x7efc1006f8d0 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc07e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efc10072d30 0x7efc1006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38696/0 (socket says 172.21.15.28:38696) 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc07e00640 1 -- 172.21.15.28:0/1500923598 learned_addr learned my addr 172.21.15.28:0/1500923598 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc0f400640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efc10165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241518897 0 0) 0x7efc100cdbe0 con 0x7efc10072d30 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.115+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efbf4005730 con 0x7efc10072d30 2024-09-10T10:49:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3590353189 0 0) 0x7efc100cd900 con 0x7efc1015b300 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc100cdbe0 con 0x7efc1015b300 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3359073066 0 0) 0x7efbf4005730 con 0x7efc10072d30 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 msgr2=0x7efc10165de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efc10165de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 >> v1:172.21.15.138:6789/0 conn(0x7efc1015b300 legacy=0x7efc100cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc10072250 con 0x7efc10072d30 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7efbf8001670 con 0x7efc10072d30 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0f400640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efc10165de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 >> v1:172.21.15.28:6789/0 conn(0x7efc10072d30 legacy=0x7efc1006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf4005d80 0x7efbf4006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf4006a10 0x7efbf4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efbf40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efc100728a0 con 0x7efbf4005d80 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efbf400dce0 con 0x7efbf4006a10 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efbf400dee0 con 0x7efc1006f8d0 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0ea00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf4005d80 0x7efbf4006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc07e00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf4006a10 0x7efbf4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.116+0000 7efc0f400640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efbf40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc0ea00640 1 -- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 msgr2=0x7efbf40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc0ea00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc1006f8d0 0x7efbf40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc0ea00640 1 -- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf4006a10 msgr2=0x7efbf4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc0ea00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf4006a10 0x7efbf4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc0ea00640 1 -- 172.21.15.28:0/1500923598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf4005730 con 0x7efbf4005d80 2024-09-10T10:49:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.117+0000 7efc07e00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efbf4006a10 0x7efbf4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc0ea00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf4005d80 0x7efbf4006180 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efc000018a0 tx=0x7efbf4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddb84fcf684140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000014f0 con 0x7efbf4005d80 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc000052d0 con 0x7efbf4005d80 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc0e000640 1 -- 172.21.15.28:0/1500923598 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0002ac40 con 0x7efbf4005d80 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc15e00640 1 -- 172.21.15.28:0/1500923598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf4005d80 msgr2=0x7efbf4006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc15e00640 1 --2- 172.21.15.28:0/1500923598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efbf4005d80 0x7efbf4006180 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7efc000018a0 tx=0x7efbf4007a40 comp rx=0 tx=0).stop 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.118+0000 7efc0f400640 1 -- 172.21.15.28:0/1500923598 reap_dead start 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.119+0000 7efc15e00640 1 -- 172.21.15.28:0/1500923598 shutdown_connections 2024-09-10T10:49:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.119+0000 7efc15e00640 1 -- 172.21.15.28:0/1500923598 >> 172.21.15.28:0/1500923598 conn(0x7efc10067d70 msgr2=0x7efc1006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.119+0000 7efc15e00640 1 -- 172.21.15.28:0/1500923598 shutdown_connections 2024-09-10T10:49:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.119+0000 7efc15e00640 1 -- 172.21.15.28:0/1500923598 wait complete. 2024-09-10T10:49:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.120+0000 7efc15e00640 1 Processor -- start 2024-09-10T10:49:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.120+0000 7efc15e00640 1 -- start start 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 0x7efc10157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 0x7efc101e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efc100cffe0 con 0x7efc1015d5f0 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc100cd5c0 con 0x7efc101e36f0 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.121+0000 7efc15e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc100d0160 con 0x7efbf4005d80 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50662/0 (socket says 172.21.15.28:50662) 2024-09-10T10:49:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 -- 172.21.15.28:0/2889885688 learned_addr learned my addr 172.21.15.28:0/2889885688 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0ea00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 0x7efc10157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc07e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 0x7efc101e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 -- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 msgr2=0x7efc10157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 0x7efc10157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 -- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 msgr2=0x7efc101e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 0x7efc101e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 -- 172.21.15.28:0/2889885688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc10148340 con 0x7efc1015d5f0 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc07e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 0x7efc101e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0ea00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 0x7efc10157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.122+0000 7efc0f400640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006990 tx=0x7efc08009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.123+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8007ca0 con 0x7efc1015d5f0 2024-09-10T10:49:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.123+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc0801d070 con 0x7efc1015d5f0 2024-09-10T10:49:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.123+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc101e4260 con 0x7efc1015d5f0 2024-09-10T10:49:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.123+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc08015670 con 0x7efc1015d5f0 2024-09-10T10:49:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.123+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101e4820 con 0x7efc1015d5f0 2024-09-10T10:49:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.124+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc10158800 con 0x7efc1015d5f0 2024-09-10T10:49:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.125+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc08015810 con 0x7efc1015d5f0 2024-09-10T10:49:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.125+0000 7efc07400640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 0x7efbec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.126+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7efc0808b8d0 con 0x7efc1015d5f0 2024-09-10T10:49:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.126+0000 7efc0ea00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 0x7efbec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.127+0000 7efc0ea00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 0x7efbec06e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc00032000 tx=0x7efc00009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.130+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efc08010040 con 0x7efc1015d5f0 2024-09-10T10:49:36.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.311+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc10158520 con 0x7efbec06be30 2024-09-10T10:49:36.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.319+0000 7efc07400640 1 -- 172.21.15.28:0/2889885688 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241537 (secure 0 0 0) 0x7efc10158520 con 0x7efbec06be30 2024-09-10T10:49:36.323 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 msgr2=0x7efbec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 0x7efbec06e2f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc00032000 tx=0x7efc00009030 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 msgr2=0x7efc1015da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006990 tx=0x7efc08009f90 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 shutdown_connections 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efbec06be30 0x7efbec06e2f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc1015d5f0 0x7efc1015da70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc101e36f0 0x7efc101e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 --2- 172.21.15.28:0/2889885688 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efbf4005d80 0x7efc10157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.322+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 >> 172.21.15.28:0/2889885688 conn(0x7efc10067d70 msgr2=0x7efc1006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.323+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 shutdown_connections 2024-09-10T10:49:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.323+0000 7efc15e00640 1 -- 172.21.15.28:0/2889885688 wait complete. 2024-09-10T10:49:36.340 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-10T10:49:34.572096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590458,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":175,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258588,"kb_used_data":9060,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716132,"statfs":{"total":773094113280,"available":772829319168,"internally_reserved":0,"allocated":9277440,"data_stored":4891943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.472394"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137467+0000","last_change":"2024-09-10T10:49:32.139173+0000","last_active":"2024-09-10T10:49:33.137467+0000","last_peered":"2024-09-10T10:49:33.137467+0000","last_clean":"2024-09-10T10:49:33.137467+0000","last_became_active":"2024-09-10T10:49:32.134918+0000","last_became_peered":"2024-09-10T10:49:32.134918+0000","last_unstale":"2024-09-10T10:49:33.137467+0000","last_undegraded":"2024-09-10T10:49:33.137467+0000","last_fullsized":"2024-09-10T10:49:33.137467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:11.448134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137760+0000","last_change":"2024-09-10T10:49:32.132374+0000","last_active":"2024-09-10T10:49:33.137760+0000","last_peered":"2024-09-10T10:49:33.137760+0000","last_clean":"2024-09-10T10:49:33.137760+0000","last_became_active":"2024-09-10T10:49:32.131984+0000","last_became_peered":"2024-09-10T10:49:32.131984+0000","last_unstale":"2024-09-10T10:49:33.137760+0000","last_undegraded":"2024-09-10T10:49:33.137760+0000","last_fullsized":"2024-09-10T10:49:33.137760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:05:49.027454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137996+0000","last_change":"2024-09-10T10:49:32.146586+0000","last_active":"2024-09-10T10:49:33.137996+0000","last_peered":"2024-09-10T10:49:33.137996+0000","last_clean":"2024-09-10T10:49:33.137996+0000","last_became_active":"2024-09-10T10:49:32.146300+0000","last_became_peered":"2024-09-10T10:49:32.146300+0000","last_unstale":"2024-09-10T10:49:33.137996+0000","last_undegraded":"2024-09-10T10:49:33.137996+0000","last_fullsized":"2024-09-10T10:49:33.137996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:32.294336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137435+0000","last_change":"2024-09-10T10:49:32.145741+0000","last_active":"2024-09-10T10:49:33.137435+0000","last_peered":"2024-09-10T10:49:33.137435+0000","last_clean":"2024-09-10T10:49:33.137435+0000","last_became_active":"2024-09-10T10:49:32.144277+0000","last_became_peered":"2024-09-10T10:49:32.144277+0000","last_unstale":"2024-09-10T10:49:33.137435+0000","last_undegraded":"2024-09-10T10:49:33.137435+0000","last_fullsized":"2024-09-10T10:49:33.137435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:53.365245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102633+0000","last_change":"2024-09-10T10:49:30.102633+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102633+0000","last_undegraded":"2024-09-10T10:49:30.102633+0000","last_fullsized":"2024-09-10T10:49:30.102633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.103618+0000","last_change":"2024-09-10T10:49:30.103618+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.103618+0000","last_undegraded":"2024-09-10T10:49:30.103618+0000","last_fullsized":"2024-09-10T10:49:30.103618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.107278+0000","last_change":"2024-09-10T10:49:30.107278+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.107278+0000","last_undegraded":"2024-09-10T10:49:30.107278+0000","last_fullsized":"2024-09-10T10:49:30.107278+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.109361+0000","last_change":"2024-09-10T10:49:30.109361+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.109361+0000","last_undegraded":"2024-09-10T10:49:30.109361+0000","last_fullsized":"2024-09-10T10:49:30.109361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137925+0000","last_change":"2024-09-10T10:49:32.135453+0000","last_active":"2024-09-10T10:49:33.137925+0000","last_peered":"2024-09-10T10:49:33.137925+0000","last_clean":"2024-09-10T10:49:33.137925+0000","last_became_active":"2024-09-10T10:49:32.135177+0000","last_became_peered":"2024-09-10T10:49:32.135177+0000","last_unstale":"2024-09-10T10:49:33.137925+0000","last_undegraded":"2024-09-10T10:49:33.137925+0000","last_fullsized":"2024-09-10T10:49:33.137925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:16.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.109485+0000","last_change":"2024-09-10T10:49:30.109485+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.109485+0000","last_undegraded":"2024-09-10T10:49:30.109485+0000","last_fullsized":"2024-09-10T10:49:30.109485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102998+0000","last_change":"2024-09-10T10:49:30.102998+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102998+0000","last_undegraded":"2024-09-10T10:49:30.102998+0000","last_fullsized":"2024-09-10T10:49:30.102998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.108922+0000","last_change":"2024-09-10T10:49:30.108922+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.108922+0000","last_undegraded":"2024-09-10T10:49:30.108922+0000","last_fullsized":"2024-09-10T10:49:30.108922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.111031+0000","last_change":"2024-09-10T10:49:30.111031+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.111031+0000","last_undegraded":"2024-09-10T10:49:30.111031+0000","last_fullsized":"2024-09-10T10:49:30.111031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.107122+0000","last_change":"2024-09-10T10:49:30.107122+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.107122+0000","last_undegraded":"2024-09-10T10:49:30.107122+0000","last_fullsized":"2024-09-10T10:49:30.107122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106829+0000","last_change":"2024-09-10T10:49:30.106829+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106829+0000","last_undegraded":"2024-09-10T10:49:30.106829+0000","last_fullsized":"2024-09-10T10:49:30.106829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137311+0000","last_change":"2024-09-10T10:49:31.130738+0000","last_active":"2024-09-10T10:49:33.137311+0000","last_peered":"2024-09-10T10:49:33.137311+0000","last_clean":"2024-09-10T10:49:33.137311+0000","last_became_active":"2024-09-10T10:49:31.130443+0000","last_became_peered":"2024-09-10T10:49:31.130443+0000","last_unstale":"2024-09-10T10:49:33.137311+0000","last_undegraded":"2024-09-10T10:49:33.137311+0000","last_fullsized":"2024-09-10T10:49:33.137311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137501+0000","last_change":"2024-09-10T10:49:32.138695+0000","last_active":"2024-09-10T10:49:33.137501+0000","last_peered":"2024-09-10T10:49:33.137501+0000","last_clean":"2024-09-10T10:49:33.137501+0000","last_became_active":"2024-09-10T10:49:32.138428+0000","last_became_peered":"2024-09-10T10:49:32.138428+0000","last_unstale":"2024-09-10T10:49:33.137501+0000","last_undegraded":"2024-09-10T10:49:33.137501+0000","last_fullsized":"2024-09-10T10:49:33.137501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:44.642998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.103060+0000","last_change":"2024-09-10T10:49:30.103060+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.103060+0000","last_undegraded":"2024-09-10T10:49:30.103060+0000","last_fullsized":"2024-09-10T10:49:30.103060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106709+0000","last_change":"2024-09-10T10:49:30.106709+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106709+0000","last_undegraded":"2024-09-10T10:49:30.106709+0000","last_fullsized":"2024-09-10T10:49:30.106709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.109272+0000","last_change":"2024-09-10T10:49:30.109272+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.109272+0000","last_undegraded":"2024-09-10T10:49:30.109272+0000","last_fullsized":"2024-09-10T10:49:30.109272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102589+0000","last_change":"2024-09-10T10:49:30.102589+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102589+0000","last_undegraded":"2024-09-10T10:49:30.102589+0000","last_fullsized":"2024-09-10T10:49:30.102589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137225+0000","last_change":"2024-09-10T10:49:31.118143+0000","last_active":"2024-09-10T10:49:33.137225+0000","last_peered":"2024-09-10T10:49:33.137225+0000","last_clean":"2024-09-10T10:49:33.137225+0000","last_became_active":"2024-09-10T10:49:31.116947+0000","last_became_peered":"2024-09-10T10:49:31.116947+0000","last_unstale":"2024-09-10T10:49:33.137225+0000","last_undegraded":"2024-09-10T10:49:33.137225+0000","last_fullsized":"2024-09-10T10:49:33.137225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:58.261531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106619+0000","last_change":"2024-09-10T10:49:30.106619+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106619+0000","last_undegraded":"2024-09-10T10:49:30.106619+0000","last_fullsized":"2024-09-10T10:49:30.106619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106833+0000","last_change":"2024-09-10T10:49:30.106833+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106833+0000","last_undegraded":"2024-09-10T10:49:30.106833+0000","last_fullsized":"2024-09-10T10:49:30.106833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102591+0000","last_change":"2024-09-10T10:49:30.102591+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102591+0000","last_undegraded":"2024-09-10T10:49:30.102591+0000","last_fullsized":"2024-09-10T10:49:30.102591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102604+0000","last_change":"2024-09-10T10:49:30.102604+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102604+0000","last_undegraded":"2024-09-10T10:49:30.102604+0000","last_fullsized":"2024-09-10T10:49:30.102604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.108633+0000","last_change":"2024-09-10T10:49:30.108633+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.108633+0000","last_undegraded":"2024-09-10T10:49:30.108633+0000","last_fullsized":"2024-09-10T10:49:30.108633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.108215+0000","last_change":"2024-09-10T10:49:30.108215+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.108215+0000","last_undegraded":"2024-09-10T10:49:30.108215+0000","last_fullsized":"2024-09-10T10:49:30.108215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.110407+0000","last_change":"2024-09-10T10:49:30.110407+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.110407+0000","last_undegraded":"2024-09-10T10:49:30.110407+0000","last_fullsized":"2024-09-10T10:49:30.110407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.107093+0000","last_change":"2024-09-10T10:49:30.107093+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.107093+0000","last_undegraded":"2024-09-10T10:49:30.107093+0000","last_fullsized":"2024-09-10T10:49:30.107093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137749+0000","last_change":"2024-09-10T10:49:31.125817+0000","last_active":"2024-09-10T10:49:33.137749+0000","last_peered":"2024-09-10T10:49:33.137749+0000","last_clean":"2024-09-10T10:49:33.137749+0000","last_became_active":"2024-09-10T10:49:31.125554+0000","last_became_peered":"2024-09-10T10:49:31.125554+0000","last_unstale":"2024-09-10T10:49:33.137749+0000","last_undegraded":"2024-09-10T10:49:33.137749+0000","last_fullsized":"2024-09-10T10:49:33.137749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:55.513897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106604+0000","last_change":"2024-09-10T10:49:30.106604+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106604+0000","last_undegraded":"2024-09-10T10:49:30.106604+0000","last_fullsized":"2024-09-10T10:49:30.106604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106209+0000","last_change":"2024-09-10T10:49:30.106209+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106209+0000","last_undegraded":"2024-09-10T10:49:30.106209+0000","last_fullsized":"2024-09-10T10:49:30.106209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.102974+0000","last_change":"2024-09-10T10:49:30.102974+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.102974+0000","last_undegraded":"2024-09-10T10:49:30.102974+0000","last_fullsized":"2024-09-10T10:49:30.102974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137703+0000","last_change":"2024-09-10T10:49:31.124068+0000","last_active":"2024-09-10T10:49:33.137703+0000","last_peered":"2024-09-10T10:49:33.137703+0000","last_clean":"2024-09-10T10:49:33.137703+0000","last_became_active":"2024-09-10T10:49:31.123728+0000","last_became_peered":"2024-09-10T10:49:31.123728+0000","last_unstale":"2024-09-10T10:49:33.137703+0000","last_undegraded":"2024-09-10T10:49:33.137703+0000","last_fullsized":"2024-09-10T10:49:33.137703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:20.133803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.106458+0000","last_change":"2024-09-10T10:49:30.106458+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.106458+0000","last_undegraded":"2024-09-10T10:49:30.106458+0000","last_fullsized":"2024-09-10T10:49:30.106458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-10T10:49:30.108654+0000","last_change":"2024-09-10T10:49:30.108654+0000","last_active":"2024-09-10T10:49:30.096128+0000","last_peered":"2024-09-10T10:49:30.096128+0000","last_clean":"2024-09-10T10:49:30.096128+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:30.108654+0000","last_undegraded":"2024-09-10T10:49:30.108654+0000","last_fullsized":"2024-09-10T10:49:30.108654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:49:31.099475+0000","last_change":"2024-09-10T10:49:31.099475+0000","last_active":"2024-09-10T10:49:31.099475+0000","last_peered":"2024-09-10T10:49:31.099475+0000","last_clean":"2024-09-10T10:49:31.099475+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:49:31.099475+0000","last_undegraded":"2024-09-10T10:49:31.099475+0000","last_fullsized":"2024-09-10T10:49:31.099475+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137585+0000","last_change":"2024-09-10T10:49:31.131228+0000","last_active":"2024-09-10T10:49:33.137585+0000","last_peered":"2024-09-10T10:49:33.137585+0000","last_clean":"2024-09-10T10:49:33.137585+0000","last_became_active":"2024-09-10T10:49:31.130375+0000","last_became_peered":"2024-09-10T10:49:31.130375+0000","last_unstale":"2024-09-10T10:49:33.137585+0000","last_undegraded":"2024-09-10T10:49:33.137585+0000","last_fullsized":"2024-09-10T10:49:33.137585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:31.372058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":283,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-10T10:49:30.108558+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:49:30.108558+0000","last_peered":"2024-09-10T10:49:30.108558+0000","last_clean":"2024-09-10T10:49:30.108558+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:49:30.108558+0000","last_undegraded":"2024-09-10T10:49:30.108558+0000","last_fullsized":"2024-09-10T10:49:30.108558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":462100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":462100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":462100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":462100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":995328,"data_stored":455891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509513,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":1052468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":462100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1073084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:49:36.344 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:49:36.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.533+0000 7fce5d000640 1 Processor -- start 2024-09-10T10:49:36.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.533+0000 7fce5d000640 1 -- start start 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce5d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce5d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fce5806f700 0x7fce5815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce5d000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580cdbe0 con 0x7fce5815b300 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce5d000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fce5806fb50 con 0x7fce5806f700 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce5d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce580728a0 con 0x7fce58072d30 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce55e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fce5815b300 0x7fce580cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38708/0 (socket says 172.21.15.28:38708) 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce55e00640 1 -- 172.21.15.28:0/2320038036 learned_addr learned my addr 172.21.15.28:0/2320038036 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:36.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce56800640 1 --2- 172.21.15.28:0/2320038036 >> v2:172.21.15.28:3300/0 conn(0x7fce5806f700 0x7fce5815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce4fe00640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce56800640 1 -- 172.21.15.28:0/2320038036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 msgr2=0x7fce5806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce56800640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce56800640 1 -- 172.21.15.28:0/2320038036 >> v1:172.21.15.28:6789/0 conn(0x7fce5815b300 legacy=0x7fce580cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.536+0000 7fce56800640 1 -- 172.21.15.28:0/2320038036 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580cd900 con 0x7fce5806f700 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.535+0000 7fce55400640 1 -- 172.21.15.28:0/2320038036 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367426297 0 0) 0x7fce580cdbe0 con 0x7fce5815b300 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.536+0000 7fce4fe00640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.536+0000 7fce56800640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5806f700 0x7fce5815b6e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fce50009d90 tx=0x7fce5002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a23463c1b239f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce55400640 1 -- 172.21.15.28:0/2320038036 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50007740 con 0x7fce5806f700 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce55400640 1 -- 172.21.15.28:0/2320038036 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce500078e0 con 0x7fce5806f700 2024-09-10T10:49:36.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce55400640 1 -- 172.21.15.28:0/2320038036 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50007cb0 con 0x7fce5806f700 2024-09-10T10:49:36.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 -- 172.21.15.28:0/2320038036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5806f700 msgr2=0x7fce5815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5806f700 0x7fce5815b6e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fce50009d90 tx=0x7fce5002a660 comp rx=0 tx=0).stop 2024-09-10T10:49:36.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 -- 172.21.15.28:0/2320038036 shutdown_connections 2024-09-10T10:49:36.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5806f700 0x7fce5815b6e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 --2- 172.21.15.28:0/2320038036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 -- 172.21.15.28:0/2320038036 >> 172.21.15.28:0/2320038036 conn(0x7fce58067d70 msgr2=0x7fce5806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.537+0000 7fce5d000640 1 -- 172.21.15.28:0/2320038036 shutdown_connections 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.538+0000 7fce5d000640 1 -- 172.21.15.28:0/2320038036 wait complete. 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.539+0000 7fce5d000640 1 Processor -- start 2024-09-10T10:49:36.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.539+0000 7fce5d000640 1 -- start start 2024-09-10T10:49:36.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 0x7fce5814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5814d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fce580cd5c0 con 0x7fce5815b300 2024-09-10T10:49:36.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce58072530 con 0x7fce5806f700 2024-09-10T10:49:36.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce5d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce580cffe0 con 0x7fce58072d30 2024-09-10T10:49:36.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce56800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce56800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50680/0 (socket says 172.21.15.28:50680) 2024-09-10T10:49:36.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.540+0000 7fce56800640 1 -- 172.21.15.28:0/253483797 learned_addr learned my addr 172.21.15.28:0/253483797 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 -- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 msgr2=0x7fce5814d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5814d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce55e00640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 0x7fce5814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 -- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 msgr2=0x7fce5814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 0x7fce5814d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 -- 172.21.15.28:0/253483797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5814de10 con 0x7fce5815b300 2024-09-10T10:49:36.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce55e00640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 0x7fce5814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce56800640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fce50009eb0 tx=0x7fce50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50040070 con 0x7fce5815b300 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50031c90 con 0x7fce5815b300 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50045660 con 0x7fce5815b300 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.541+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5814e0c0 con 0x7fce5815b300 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.542+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5814e680 con 0x7fce5815b300 2024-09-10T10:49:36.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.543+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce580b0810 con 0x7fce5815b300 2024-09-10T10:49:36.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.544+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce50039040 con 0x7fce5815b300 2024-09-10T10:49:36.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.544+0000 7fce4e000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 0x7fce3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.544+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fce500acb10 con 0x7fce5815b300 2024-09-10T10:49:36.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.545+0000 7fce55e00640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 0x7fce3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.546+0000 7fce55e00640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 0x7fce3406e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce48000f60 tx=0x7fce48009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.549+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce5007b350 con 0x7fce5815b300 2024-09-10T10:49:36.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.732+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce580b0530 con 0x7fce3406be30 2024-09-10T10:49:36.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.739+0000 7fce4e000640 1 -- 172.21.15.28:0/253483797 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252080 (secure 0 0 0) 0x7fce580b0530 con 0x7fce3406be30 2024-09-10T10:49:36.745 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:36.746 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.743+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 msgr2=0x7fce3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.743+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 0x7fce3406e2f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fce48000f60 tx=0x7fce48009040 comp rx=0 tx=0).stop 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 msgr2=0x7fce581356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fce50009eb0 tx=0x7fce50004a40 comp rx=0 tx=0).stop 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 shutdown_connections 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce3406be30 0x7fce3406e2f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce5815b300 0x7fce581356c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce58072d30 0x7fce5814d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 --2- 172.21.15.28:0/253483797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce5806f700 0x7fce5814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 >> 172.21.15.28:0/253483797 conn(0x7fce58067d70 msgr2=0x7fce580b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:36.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 shutdown_connections 2024-09-10T10:49:36.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.744+0000 7fce5d000640 1 -- 172.21.15.28:0/253483797 wait complete. 2024-09-10T10:49:36.761 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-10T10:49:36.572614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258896,"kb_used_data":9384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715824,"statfs":{"total":773094113280,"available":772829003776,"internally_reserved":0,"allocated":9609216,"data_stored":5046564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001324"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339876+0000","last_change":"2024-09-10T10:49:32.151623+0000","last_active":"2024-09-10T10:49:33.339876+0000","last_peered":"2024-09-10T10:49:33.339876+0000","last_clean":"2024-09-10T10:49:33.339876+0000","last_became_active":"2024-09-10T10:49:32.151105+0000","last_became_peered":"2024-09-10T10:49:32.151105+0000","last_unstale":"2024-09-10T10:49:33.339876+0000","last_undegraded":"2024-09-10T10:49:33.339876+0000","last_fullsized":"2024-09-10T10:49:33.339876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:30.592233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120737+0000","last_change":"2024-09-10T10:49:32.144087+0000","last_active":"2024-09-10T10:49:34.120737+0000","last_peered":"2024-09-10T10:49:34.120737+0000","last_clean":"2024-09-10T10:49:34.120737+0000","last_became_active":"2024-09-10T10:49:32.143746+0000","last_became_peered":"2024-09-10T10:49:32.143746+0000","last_unstale":"2024-09-10T10:49:34.120737+0000","last_undegraded":"2024-09-10T10:49:34.120737+0000","last_fullsized":"2024-09-10T10:49:34.120737+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:48.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120789+0000","last_change":"2024-09-10T10:49:32.145360+0000","last_active":"2024-09-10T10:49:34.120789+0000","last_peered":"2024-09-10T10:49:34.120789+0000","last_clean":"2024-09-10T10:49:34.120789+0000","last_became_active":"2024-09-10T10:49:32.144534+0000","last_became_peered":"2024-09-10T10:49:32.144534+0000","last_unstale":"2024-09-10T10:49:34.120789+0000","last_undegraded":"2024-09-10T10:49:34.120789+0000","last_fullsized":"2024-09-10T10:49:34.120789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:14.019464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746778+0000","last_change":"2024-09-10T10:49:32.139332+0000","last_active":"2024-09-10T10:49:34.746778+0000","last_peered":"2024-09-10T10:49:34.746778+0000","last_clean":"2024-09-10T10:49:34.746778+0000","last_became_active":"2024-09-10T10:49:32.139050+0000","last_became_peered":"2024-09-10T10:49:32.139050+0000","last_unstale":"2024-09-10T10:49:34.746778+0000","last_undegraded":"2024-09-10T10:49:34.746778+0000","last_fullsized":"2024-09-10T10:49:34.746778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:04:31.568538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138868+0000","last_change":"2024-09-10T10:49:32.138778+0000","last_active":"2024-09-10T10:49:33.138868+0000","last_peered":"2024-09-10T10:49:33.138868+0000","last_clean":"2024-09-10T10:49:33.138868+0000","last_became_active":"2024-09-10T10:49:32.138327+0000","last_became_peered":"2024-09-10T10:49:32.138327+0000","last_unstale":"2024-09-10T10:49:33.138868+0000","last_undegraded":"2024-09-10T10:49:33.138868+0000","last_fullsized":"2024-09-10T10:49:33.138868+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:30.993648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137467+0000","last_change":"2024-09-10T10:49:32.139173+0000","last_active":"2024-09-10T10:49:33.137467+0000","last_peered":"2024-09-10T10:49:33.137467+0000","last_clean":"2024-09-10T10:49:33.137467+0000","last_became_active":"2024-09-10T10:49:32.134918+0000","last_became_peered":"2024-09-10T10:49:32.134918+0000","last_unstale":"2024-09-10T10:49:33.137467+0000","last_undegraded":"2024-09-10T10:49:33.137467+0000","last_fullsized":"2024-09-10T10:49:33.137467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:08:11.448134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746996+0000","last_change":"2024-09-10T10:49:32.138484+0000","last_active":"2024-09-10T10:49:34.746996+0000","last_peered":"2024-09-10T10:49:34.746996+0000","last_clean":"2024-09-10T10:49:34.746996+0000","last_became_active":"2024-09-10T10:49:32.138185+0000","last_became_peered":"2024-09-10T10:49:32.138185+0000","last_unstale":"2024-09-10T10:49:34.746996+0000","last_undegraded":"2024-09-10T10:49:34.746996+0000","last_fullsized":"2024-09-10T10:49:34.746996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:26.545493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747084+0000","last_change":"2024-09-10T10:49:32.125973+0000","last_active":"2024-09-10T10:49:34.747084+0000","last_peered":"2024-09-10T10:49:34.747084+0000","last_clean":"2024-09-10T10:49:34.747084+0000","last_became_active":"2024-09-10T10:49:32.125712+0000","last_became_peered":"2024-09-10T10:49:32.125712+0000","last_unstale":"2024-09-10T10:49:34.747084+0000","last_undegraded":"2024-09-10T10:49:34.747084+0000","last_fullsized":"2024-09-10T10:49:34.747084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:21:39.267590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747139+0000","last_change":"2024-09-10T10:49:32.141595+0000","last_active":"2024-09-10T10:49:34.747139+0000","last_peered":"2024-09-10T10:49:34.747139+0000","last_clean":"2024-09-10T10:49:34.747139+0000","last_became_active":"2024-09-10T10:49:32.141416+0000","last_became_peered":"2024-09-10T10:49:32.141416+0000","last_unstale":"2024-09-10T10:49:34.747139+0000","last_undegraded":"2024-09-10T10:49:34.747139+0000","last_fullsized":"2024-09-10T10:49:34.747139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:14.019464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339831+0000","last_change":"2024-09-10T10:49:32.137394+0000","last_active":"2024-09-10T10:49:33.339831+0000","last_peered":"2024-09-10T10:49:33.339831+0000","last_clean":"2024-09-10T10:49:33.339831+0000","last_became_active":"2024-09-10T10:49:32.137174+0000","last_became_peered":"2024-09-10T10:49:32.137174+0000","last_unstale":"2024-09-10T10:49:33.339831+0000","last_undegraded":"2024-09-10T10:49:33.339831+0000","last_fullsized":"2024-09-10T10:49:33.339831+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:35.252206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120820+0000","last_change":"2024-09-10T10:49:32.142428+0000","last_active":"2024-09-10T10:49:34.120820+0000","last_peered":"2024-09-10T10:49:34.120820+0000","last_clean":"2024-09-10T10:49:34.120820+0000","last_became_active":"2024-09-10T10:49:32.141608+0000","last_became_peered":"2024-09-10T10:49:32.141608+0000","last_unstale":"2024-09-10T10:49:34.120820+0000","last_undegraded":"2024-09-10T10:49:34.120820+0000","last_fullsized":"2024-09-10T10:49:34.120820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:19.859289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131572+0000","last_change":"2024-09-10T10:49:32.125486+0000","last_active":"2024-09-10T10:49:33.131572+0000","last_peered":"2024-09-10T10:49:33.131572+0000","last_clean":"2024-09-10T10:49:33.131572+0000","last_became_active":"2024-09-10T10:49:32.125166+0000","last_became_peered":"2024-09-10T10:49:32.125166+0000","last_unstale":"2024-09-10T10:49:33.131572+0000","last_undegraded":"2024-09-10T10:49:33.131572+0000","last_fullsized":"2024-09-10T10:49:33.131572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:05.006982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120567+0000","last_change":"2024-09-10T10:49:32.139161+0000","last_active":"2024-09-10T10:49:34.120567+0000","last_peered":"2024-09-10T10:49:34.120567+0000","last_clean":"2024-09-10T10:49:34.120567+0000","last_became_active":"2024-09-10T10:49:32.136719+0000","last_became_peered":"2024-09-10T10:49:32.136719+0000","last_unstale":"2024-09-10T10:49:34.120567+0000","last_undegraded":"2024-09-10T10:49:34.120567+0000","last_fullsized":"2024-09-10T10:49:34.120567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:26.545493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746762+0000","last_change":"2024-09-10T10:49:32.129062+0000","last_active":"2024-09-10T10:49:34.746762+0000","last_peered":"2024-09-10T10:49:34.746762+0000","last_clean":"2024-09-10T10:49:34.746762+0000","last_became_active":"2024-09-10T10:49:32.128791+0000","last_became_peered":"2024-09-10T10:49:32.128791+0000","last_unstale":"2024-09-10T10:49:34.746762+0000","last_undegraded":"2024-09-10T10:49:34.746762+0000","last_fullsized":"2024-09-10T10:49:34.746762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:06.053641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138674+0000","last_change":"2024-09-10T10:49:32.143460+0000","last_active":"2024-09-10T10:49:33.138674+0000","last_peered":"2024-09-10T10:49:33.138674+0000","last_clean":"2024-09-10T10:49:33.138674+0000","last_became_active":"2024-09-10T10:49:32.141550+0000","last_became_peered":"2024-09-10T10:49:32.141550+0000","last_unstale":"2024-09-10T10:49:33.138674+0000","last_undegraded":"2024-09-10T10:49:33.138674+0000","last_fullsized":"2024-09-10T10:49:33.138674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:50.571391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747018+0000","last_change":"2024-09-10T10:49:32.140470+0000","last_active":"2024-09-10T10:49:34.747018+0000","last_peered":"2024-09-10T10:49:34.747018+0000","last_clean":"2024-09-10T10:49:34.747018+0000","last_became_active":"2024-09-10T10:49:32.139275+0000","last_became_peered":"2024-09-10T10:49:32.139275+0000","last_unstale":"2024-09-10T10:49:34.747018+0000","last_undegraded":"2024-09-10T10:49:34.747018+0000","last_fullsized":"2024-09-10T10:49:34.747018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:48.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747199+0000","last_change":"2024-09-10T10:49:32.134828+0000","last_active":"2024-09-10T10:49:34.747199+0000","last_peered":"2024-09-10T10:49:34.747199+0000","last_clean":"2024-09-10T10:49:34.747199+0000","last_became_active":"2024-09-10T10:49:32.134556+0000","last_became_peered":"2024-09-10T10:49:32.134556+0000","last_unstale":"2024-09-10T10:49:34.747199+0000","last_undegraded":"2024-09-10T10:49:34.747199+0000","last_fullsized":"2024-09-10T10:49:34.747199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:45:41.282925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.121041+0000","last_change":"2024-09-10T10:49:32.141091+0000","last_active":"2024-09-10T10:49:34.121041+0000","last_peered":"2024-09-10T10:49:34.121041+0000","last_clean":"2024-09-10T10:49:34.121041+0000","last_became_active":"2024-09-10T10:49:32.139675+0000","last_became_peered":"2024-09-10T10:49:32.139675+0000","last_unstale":"2024-09-10T10:49:34.121041+0000","last_undegraded":"2024-09-10T10:49:34.121041+0000","last_fullsized":"2024-09-10T10:49:34.121041+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:40.662814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747165+0000","last_change":"2024-09-10T10:49:32.135168+0000","last_active":"2024-09-10T10:49:34.747165+0000","last_peered":"2024-09-10T10:49:34.747165+0000","last_clean":"2024-09-10T10:49:34.747165+0000","last_became_active":"2024-09-10T10:49:32.134001+0000","last_became_peered":"2024-09-10T10:49:32.134001+0000","last_unstale":"2024-09-10T10:49:34.747165+0000","last_undegraded":"2024-09-10T10:49:34.747165+0000","last_fullsized":"2024-09-10T10:49:34.747165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:11:27.656010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137760+0000","last_change":"2024-09-10T10:49:32.132374+0000","last_active":"2024-09-10T10:49:33.137760+0000","last_peered":"2024-09-10T10:49:33.137760+0000","last_clean":"2024-09-10T10:49:33.137760+0000","last_became_active":"2024-09-10T10:49:32.131984+0000","last_became_peered":"2024-09-10T10:49:32.131984+0000","last_unstale":"2024-09-10T10:49:33.137760+0000","last_undegraded":"2024-09-10T10:49:33.137760+0000","last_fullsized":"2024-09-10T10:49:33.137760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:05:49.027454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131377+0000","last_change":"2024-09-10T10:49:32.132401+0000","last_active":"2024-09-10T10:49:33.131377+0000","last_peered":"2024-09-10T10:49:33.131377+0000","last_clean":"2024-09-10T10:49:33.131377+0000","last_became_active":"2024-09-10T10:49:32.129629+0000","last_became_peered":"2024-09-10T10:49:32.129629+0000","last_unstale":"2024-09-10T10:49:33.131377+0000","last_undegraded":"2024-09-10T10:49:33.131377+0000","last_fullsized":"2024-09-10T10:49:33.131377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:20.368223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120767+0000","last_change":"2024-09-10T10:49:32.147929+0000","last_active":"2024-09-10T10:49:34.120767+0000","last_peered":"2024-09-10T10:49:34.120767+0000","last_clean":"2024-09-10T10:49:34.120767+0000","last_became_active":"2024-09-10T10:49:32.147629+0000","last_became_peered":"2024-09-10T10:49:32.147629+0000","last_unstale":"2024-09-10T10:49:34.120767+0000","last_undegraded":"2024-09-10T10:49:34.120767+0000","last_fullsized":"2024-09-10T10:49:34.120767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:05.006982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339512+0000","last_change":"2024-09-10T10:49:32.129022+0000","last_active":"2024-09-10T10:49:33.339512+0000","last_peered":"2024-09-10T10:49:33.339512+0000","last_clean":"2024-09-10T10:49:33.339512+0000","last_became_active":"2024-09-10T10:49:32.128637+0000","last_became_peered":"2024-09-10T10:49:32.128637+0000","last_unstale":"2024-09-10T10:49:33.339512+0000","last_undegraded":"2024-09-10T10:49:33.339512+0000","last_fullsized":"2024-09-10T10:49:33.339512+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:06:25.410893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120358+0000","last_change":"2024-09-10T10:49:32.138938+0000","last_active":"2024-09-10T10:49:34.120358+0000","last_peered":"2024-09-10T10:49:34.120358+0000","last_clean":"2024-09-10T10:49:34.120358+0000","last_became_active":"2024-09-10T10:49:32.136947+0000","last_became_peered":"2024-09-10T10:49:32.136947+0000","last_unstale":"2024-09-10T10:49:34.120358+0000","last_undegraded":"2024-09-10T10:49:34.120358+0000","last_fullsized":"2024-09-10T10:49:34.120358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:32.967416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137996+0000","last_change":"2024-09-10T10:49:32.146586+0000","last_active":"2024-09-10T10:49:33.137996+0000","last_peered":"2024-09-10T10:49:33.137996+0000","last_clean":"2024-09-10T10:49:33.137996+0000","last_became_active":"2024-09-10T10:49:32.146300+0000","last_became_peered":"2024-09-10T10:49:32.146300+0000","last_unstale":"2024-09-10T10:49:33.137996+0000","last_undegraded":"2024-09-10T10:49:33.137996+0000","last_fullsized":"2024-09-10T10:49:33.137996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:43:32.294336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747103+0000","last_change":"2024-09-10T10:49:32.149989+0000","last_active":"2024-09-10T10:49:34.747103+0000","last_peered":"2024-09-10T10:49:34.747103+0000","last_clean":"2024-09-10T10:49:34.747103+0000","last_became_active":"2024-09-10T10:49:32.148258+0000","last_became_peered":"2024-09-10T10:49:32.148258+0000","last_unstale":"2024-09-10T10:49:34.747103+0000","last_undegraded":"2024-09-10T10:49:34.747103+0000","last_fullsized":"2024-09-10T10:49:34.747103+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:20.368223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746856+0000","last_change":"2024-09-10T10:49:32.136891+0000","last_active":"2024-09-10T10:49:34.746856+0000","last_peered":"2024-09-10T10:49:34.746856+0000","last_clean":"2024-09-10T10:49:34.746856+0000","last_became_active":"2024-09-10T10:49:32.136607+0000","last_became_peered":"2024-09-10T10:49:32.136607+0000","last_unstale":"2024-09-10T10:49:34.746856+0000","last_undegraded":"2024-09-10T10:49:34.746856+0000","last_fullsized":"2024-09-10T10:49:34.746856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:49.740394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138966+0000","last_change":"2024-09-10T10:49:32.134388+0000","last_active":"2024-09-10T10:49:33.138966+0000","last_peered":"2024-09-10T10:49:33.138966+0000","last_clean":"2024-09-10T10:49:33.138966+0000","last_became_active":"2024-09-10T10:49:32.133970+0000","last_became_peered":"2024-09-10T10:49:32.133970+0000","last_unstale":"2024-09-10T10:49:33.138966+0000","last_undegraded":"2024-09-10T10:49:33.138966+0000","last_fullsized":"2024-09-10T10:49:33.138966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:47:11.781891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137435+0000","last_change":"2024-09-10T10:49:32.145741+0000","last_active":"2024-09-10T10:49:33.137435+0000","last_peered":"2024-09-10T10:49:33.137435+0000","last_clean":"2024-09-10T10:49:33.137435+0000","last_became_active":"2024-09-10T10:49:32.144277+0000","last_became_peered":"2024-09-10T10:49:32.144277+0000","last_unstale":"2024-09-10T10:49:33.137435+0000","last_undegraded":"2024-09-10T10:49:33.137435+0000","last_fullsized":"2024-09-10T10:49:33.137435+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:53.365245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339573+0000","last_change":"2024-09-10T10:49:32.148794+0000","last_active":"2024-09-10T10:49:33.339573+0000","last_peered":"2024-09-10T10:49:33.339573+0000","last_clean":"2024-09-10T10:49:33.339573+0000","last_became_active":"2024-09-10T10:49:32.148524+0000","last_became_peered":"2024-09-10T10:49:32.148524+0000","last_unstale":"2024-09-10T10:49:33.339573+0000","last_undegraded":"2024-09-10T10:49:33.339573+0000","last_fullsized":"2024-09-10T10:49:33.339573+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:18.655429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120727+0000","last_change":"2024-09-10T10:49:32.138701+0000","last_active":"2024-09-10T10:49:34.120727+0000","last_peered":"2024-09-10T10:49:34.120727+0000","last_clean":"2024-09-10T10:49:34.120727+0000","last_became_active":"2024-09-10T10:49:32.138449+0000","last_became_peered":"2024-09-10T10:49:32.138449+0000","last_unstale":"2024-09-10T10:49:34.120727+0000","last_undegraded":"2024-09-10T10:49:34.120727+0000","last_fullsized":"2024-09-10T10:49:34.120727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:20.368223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746889+0000","last_change":"2024-09-10T10:49:32.139947+0000","last_active":"2024-09-10T10:49:34.746889+0000","last_peered":"2024-09-10T10:49:34.746889+0000","last_clean":"2024-09-10T10:49:34.746889+0000","last_became_active":"2024-09-10T10:49:32.138976+0000","last_became_peered":"2024-09-10T10:49:32.138976+0000","last_unstale":"2024-09-10T10:49:34.746889+0000","last_undegraded":"2024-09-10T10:49:34.746889+0000","last_fullsized":"2024-09-10T10:49:34.746889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:47.219873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138481+0000","last_change":"2024-09-10T10:49:32.146539+0000","last_active":"2024-09-10T10:49:33.138481+0000","last_peered":"2024-09-10T10:49:33.138481+0000","last_clean":"2024-09-10T10:49:33.138481+0000","last_became_active":"2024-09-10T10:49:32.146267+0000","last_became_peered":"2024-09-10T10:49:32.146267+0000","last_unstale":"2024-09-10T10:49:33.138481+0000","last_undegraded":"2024-09-10T10:49:33.138481+0000","last_fullsized":"2024-09-10T10:49:33.138481+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:09.560964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"85'4","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.139314+0000","last_change":"2024-09-10T10:49:31.130447+0000","last_active":"2024-09-10T10:49:33.139314+0000","last_peered":"2024-09-10T10:49:33.139314+0000","last_clean":"2024-09-10T10:49:33.139314+0000","last_became_active":"2024-09-10T10:49:31.130189+0000","last_became_peered":"2024-09-10T10:49:31.130189+0000","last_unstale":"2024-09-10T10:49:33.139314+0000","last_undegraded":"2024-09-10T10:49:33.139314+0000","last_fullsized":"2024-09-10T10:49:33.139314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:55.289990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138624+0000","last_change":"2024-09-10T10:49:32.141480+0000","last_active":"2024-09-10T10:49:33.138624+0000","last_peered":"2024-09-10T10:49:33.138624+0000","last_clean":"2024-09-10T10:49:33.138624+0000","last_became_active":"2024-09-10T10:49:32.141143+0000","last_became_peered":"2024-09-10T10:49:32.141143+0000","last_unstale":"2024-09-10T10:49:33.138624+0000","last_undegraded":"2024-09-10T10:49:33.138624+0000","last_fullsized":"2024-09-10T10:49:33.138624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:30.590541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408863+0000","last_change":"2024-09-10T10:49:31.130786+0000","last_active":"2024-09-10T10:49:33.408863+0000","last_peered":"2024-09-10T10:49:33.408863+0000","last_clean":"2024-09-10T10:49:33.408863+0000","last_became_active":"2024-09-10T10:49:31.130458+0000","last_became_peered":"2024-09-10T10:49:31.130458+0000","last_unstale":"2024-09-10T10:49:33.408863+0000","last_undegraded":"2024-09-10T10:49:33.408863+0000","last_fullsized":"2024-09-10T10:49:33.408863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:01.102148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131092+0000","last_change":"2024-09-10T10:49:32.137307+0000","last_active":"2024-09-10T10:49:33.131092+0000","last_peered":"2024-09-10T10:49:33.131092+0000","last_clean":"2024-09-10T10:49:33.131092+0000","last_became_active":"2024-09-10T10:49:32.136597+0000","last_became_peered":"2024-09-10T10:49:32.136597+0000","last_unstale":"2024-09-10T10:49:33.131092+0000","last_undegraded":"2024-09-10T10:49:33.131092+0000","last_fullsized":"2024-09-10T10:49:33.131092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:26.545493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339729+0000","last_change":"2024-09-10T10:49:31.136267+0000","last_active":"2024-09-10T10:49:33.339729+0000","last_peered":"2024-09-10T10:49:33.339729+0000","last_clean":"2024-09-10T10:49:33.339729+0000","last_became_active":"2024-09-10T10:49:31.134051+0000","last_became_peered":"2024-09-10T10:49:31.134051+0000","last_unstale":"2024-09-10T10:49:33.339729+0000","last_undegraded":"2024-09-10T10:49:33.339729+0000","last_fullsized":"2024-09-10T10:49:33.339729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:55.682951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746862+0000","last_change":"2024-09-10T10:49:32.128574+0000","last_active":"2024-09-10T10:49:34.746862+0000","last_peered":"2024-09-10T10:49:34.746862+0000","last_clean":"2024-09-10T10:49:34.746862+0000","last_became_active":"2024-09-10T10:49:32.128229+0000","last_became_peered":"2024-09-10T10:49:32.128229+0000","last_unstale":"2024-09-10T10:49:34.746862+0000","last_undegraded":"2024-09-10T10:49:34.746862+0000","last_fullsized":"2024-09-10T10:49:34.746862+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:47:32.967416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131670+0000","last_change":"2024-09-10T10:49:31.127472+0000","last_active":"2024-09-10T10:49:33.131670+0000","last_peered":"2024-09-10T10:49:33.131670+0000","last_clean":"2024-09-10T10:49:33.131670+0000","last_became_active":"2024-09-10T10:49:31.127136+0000","last_became_peered":"2024-09-10T10:49:31.127136+0000","last_unstale":"2024-09-10T10:49:33.131670+0000","last_undegraded":"2024-09-10T10:49:33.131670+0000","last_fullsized":"2024-09-10T10:49:33.131670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:34.248860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137925+0000","last_change":"2024-09-10T10:49:32.135453+0000","last_active":"2024-09-10T10:49:33.137925+0000","last_peered":"2024-09-10T10:49:33.137925+0000","last_clean":"2024-09-10T10:49:33.137925+0000","last_became_active":"2024-09-10T10:49:32.135177+0000","last_became_peered":"2024-09-10T10:49:32.135177+0000","last_unstale":"2024-09-10T10:49:33.137925+0000","last_undegraded":"2024-09-10T10:49:33.137925+0000","last_fullsized":"2024-09-10T10:49:33.137925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:16.948214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131372+0000","last_change":"2024-09-10T10:49:31.127538+0000","last_active":"2024-09-10T10:49:33.131372+0000","last_peered":"2024-09-10T10:49:33.131372+0000","last_clean":"2024-09-10T10:49:33.131372+0000","last_became_active":"2024-09-10T10:49:31.126319+0000","last_became_peered":"2024-09-10T10:49:31.126319+0000","last_unstale":"2024-09-10T10:49:33.131372+0000","last_undegraded":"2024-09-10T10:49:33.131372+0000","last_fullsized":"2024-09-10T10:49:33.131372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:34.634013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120971+0000","last_change":"2024-09-10T10:49:32.142656+0000","last_active":"2024-09-10T10:49:34.120971+0000","last_peered":"2024-09-10T10:49:34.120971+0000","last_clean":"2024-09-10T10:49:34.120971+0000","last_became_active":"2024-09-10T10:49:32.142046+0000","last_became_peered":"2024-09-10T10:49:32.142046+0000","last_unstale":"2024-09-10T10:49:34.120971+0000","last_undegraded":"2024-09-10T10:49:34.120971+0000","last_fullsized":"2024-09-10T10:49:34.120971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:24.478879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408672+0000","last_change":"2024-09-10T10:49:31.126736+0000","last_active":"2024-09-10T10:49:33.408672+0000","last_peered":"2024-09-10T10:49:33.408672+0000","last_clean":"2024-09-10T10:49:33.408672+0000","last_became_active":"2024-09-10T10:49:31.126406+0000","last_became_peered":"2024-09-10T10:49:31.126406+0000","last_unstale":"2024-09-10T10:49:33.408672+0000","last_undegraded":"2024-09-10T10:49:33.408672+0000","last_fullsized":"2024-09-10T10:49:33.408672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:25:53.654456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131686+0000","last_change":"2024-09-10T10:49:32.133645+0000","last_active":"2024-09-10T10:49:33.131686+0000","last_peered":"2024-09-10T10:49:33.131686+0000","last_clean":"2024-09-10T10:49:33.131686+0000","last_became_active":"2024-09-10T10:49:32.133346+0000","last_became_peered":"2024-09-10T10:49:32.133346+0000","last_unstale":"2024-09-10T10:49:33.131686+0000","last_undegraded":"2024-09-10T10:49:33.131686+0000","last_fullsized":"2024-09-10T10:49:33.131686+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:53:14.019464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746595+0000","last_change":"2024-09-10T10:49:31.134103+0000","last_active":"2024-09-10T10:49:34.746595+0000","last_peered":"2024-09-10T10:49:34.746595+0000","last_clean":"2024-09-10T10:49:34.746595+0000","last_became_active":"2024-09-10T10:49:31.133737+0000","last_became_peered":"2024-09-10T10:49:31.133737+0000","last_unstale":"2024-09-10T10:49:34.746595+0000","last_undegraded":"2024-09-10T10:49:34.746595+0000","last_fullsized":"2024-09-10T10:49:34.746595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:25.542147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408938+0000","last_change":"2024-09-10T10:49:32.141333+0000","last_active":"2024-09-10T10:49:33.408938+0000","last_peered":"2024-09-10T10:49:33.408938+0000","last_clean":"2024-09-10T10:49:33.408938+0000","last_became_active":"2024-09-10T10:49:32.141079+0000","last_became_peered":"2024-09-10T10:49:32.141079+0000","last_unstale":"2024-09-10T10:49:33.408938+0000","last_undegraded":"2024-09-10T10:49:33.408938+0000","last_fullsized":"2024-09-10T10:49:33.408938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:53:23.795175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120434+0000","last_change":"2024-09-10T10:49:31.137889+0000","last_active":"2024-09-10T10:49:34.120434+0000","last_peered":"2024-09-10T10:49:34.120434+0000","last_clean":"2024-09-10T10:49:34.120434+0000","last_became_active":"2024-09-10T10:49:31.137674+0000","last_became_peered":"2024-09-10T10:49:31.137674+0000","last_unstale":"2024-09-10T10:49:34.120434+0000","last_undegraded":"2024-09-10T10:49:34.120434+0000","last_fullsized":"2024-09-10T10:49:34.120434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:04.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120536+0000","last_change":"2024-09-10T10:49:32.138202+0000","last_active":"2024-09-10T10:49:34.120536+0000","last_peered":"2024-09-10T10:49:34.120536+0000","last_clean":"2024-09-10T10:49:34.120536+0000","last_became_active":"2024-09-10T10:49:32.137263+0000","last_became_peered":"2024-09-10T10:49:32.137263+0000","last_unstale":"2024-09-10T10:49:34.120536+0000","last_undegraded":"2024-09-10T10:49:34.120536+0000","last_fullsized":"2024-09-10T10:49:34.120536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:47.219873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747066+0000","last_change":"2024-09-10T10:49:31.133452+0000","last_active":"2024-09-10T10:49:34.747066+0000","last_peered":"2024-09-10T10:49:34.747066+0000","last_clean":"2024-09-10T10:49:34.747066+0000","last_became_active":"2024-09-10T10:49:31.133164+0000","last_became_peered":"2024-09-10T10:49:31.133164+0000","last_unstale":"2024-09-10T10:49:34.747066+0000","last_undegraded":"2024-09-10T10:49:34.747066+0000","last_fullsized":"2024-09-10T10:49:34.747066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:42.373828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131089+0000","last_change":"2024-09-10T10:49:32.125484+0000","last_active":"2024-09-10T10:49:33.131089+0000","last_peered":"2024-09-10T10:49:33.131089+0000","last_clean":"2024-09-10T10:49:33.131089+0000","last_became_active":"2024-09-10T10:49:32.125169+0000","last_became_peered":"2024-09-10T10:49:32.125169+0000","last_unstale":"2024-09-10T10:49:33.131089+0000","last_undegraded":"2024-09-10T10:49:33.131089+0000","last_fullsized":"2024-09-10T10:49:33.131089+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:47.219873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339792+0000","last_change":"2024-09-10T10:49:31.136101+0000","last_active":"2024-09-10T10:49:33.339792+0000","last_peered":"2024-09-10T10:49:33.339792+0000","last_clean":"2024-09-10T10:49:33.339792+0000","last_became_active":"2024-09-10T10:49:31.133544+0000","last_became_peered":"2024-09-10T10:49:31.133544+0000","last_unstale":"2024-09-10T10:49:33.339792+0000","last_undegraded":"2024-09-10T10:49:33.339792+0000","last_fullsized":"2024-09-10T10:49:33.339792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:52:49.568045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747197+0000","last_change":"2024-09-10T10:49:32.125348+0000","last_active":"2024-09-10T10:49:34.747197+0000","last_peered":"2024-09-10T10:49:34.747197+0000","last_clean":"2024-09-10T10:49:34.747197+0000","last_became_active":"2024-09-10T10:49:32.125080+0000","last_became_peered":"2024-09-10T10:49:32.125080+0000","last_unstale":"2024-09-10T10:49:34.747197+0000","last_undegraded":"2024-09-10T10:49:34.747197+0000","last_fullsized":"2024-09-10T10:49:34.747197+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:02.896721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"85'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137311+0000","last_change":"2024-09-10T10:49:31.130738+0000","last_active":"2024-09-10T10:49:33.137311+0000","last_peered":"2024-09-10T10:49:33.137311+0000","last_clean":"2024-09-10T10:49:33.137311+0000","last_became_active":"2024-09-10T10:49:31.130443+0000","last_became_peered":"2024-09-10T10:49:31.130443+0000","last_unstale":"2024-09-10T10:49:33.137311+0000","last_undegraded":"2024-09-10T10:49:33.137311+0000","last_fullsized":"2024-09-10T10:49:33.137311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:26.471681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137501+0000","last_change":"2024-09-10T10:49:32.138695+0000","last_active":"2024-09-10T10:49:33.137501+0000","last_peered":"2024-09-10T10:49:33.137501+0000","last_clean":"2024-09-10T10:49:33.137501+0000","last_became_active":"2024-09-10T10:49:32.138428+0000","last_became_peered":"2024-09-10T10:49:32.138428+0000","last_unstale":"2024-09-10T10:49:33.137501+0000","last_undegraded":"2024-09-10T10:49:33.137501+0000","last_fullsized":"2024-09-10T10:49:33.137501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:44.642998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"85'6","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408852+0000","last_change":"2024-09-10T10:49:31.137475+0000","last_active":"2024-09-10T10:49:33.408852+0000","last_peered":"2024-09-10T10:49:33.408852+0000","last_clean":"2024-09-10T10:49:33.408852+0000","last_became_active":"2024-09-10T10:49:31.137042+0000","last_became_peered":"2024-09-10T10:49:31.137042+0000","last_unstale":"2024-09-10T10:49:33.408852+0000","last_undegraded":"2024-09-10T10:49:33.408852+0000","last_fullsized":"2024-09-10T10:49:33.408852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:55.634182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746804+0000","last_change":"2024-09-10T10:49:32.140349+0000","last_active":"2024-09-10T10:49:34.746804+0000","last_peered":"2024-09-10T10:49:34.746804+0000","last_clean":"2024-09-10T10:49:34.746804+0000","last_became_active":"2024-09-10T10:49:32.137916+0000","last_became_peered":"2024-09-10T10:49:32.137916+0000","last_unstale":"2024-09-10T10:49:34.746804+0000","last_undegraded":"2024-09-10T10:49:34.746804+0000","last_fullsized":"2024-09-10T10:49:34.746804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:52:38.457032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339552+0000","last_change":"2024-09-10T10:49:31.130621+0000","last_active":"2024-09-10T10:49:33.339552+0000","last_peered":"2024-09-10T10:49:33.339552+0000","last_clean":"2024-09-10T10:49:33.339552+0000","last_became_active":"2024-09-10T10:49:31.130317+0000","last_became_peered":"2024-09-10T10:49:31.130317+0000","last_unstale":"2024-09-10T10:49:33.339552+0000","last_undegraded":"2024-09-10T10:49:33.339552+0000","last_fullsized":"2024-09-10T10:49:33.339552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:09:08.557617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339842+0000","last_change":"2024-09-10T10:49:32.147371+0000","last_active":"2024-09-10T10:49:33.339842+0000","last_peered":"2024-09-10T10:49:33.339842+0000","last_clean":"2024-09-10T10:49:33.339842+0000","last_became_active":"2024-09-10T10:49:32.146820+0000","last_became_peered":"2024-09-10T10:49:32.146820+0000","last_unstale":"2024-09-10T10:49:33.339842+0000","last_undegraded":"2024-09-10T10:49:33.339842+0000","last_fullsized":"2024-09-10T10:49:33.339842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:43.377174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131204+0000","last_change":"2024-09-10T10:49:31.124315+0000","last_active":"2024-09-10T10:49:33.131204+0000","last_peered":"2024-09-10T10:49:33.131204+0000","last_clean":"2024-09-10T10:49:33.131204+0000","last_became_active":"2024-09-10T10:49:31.124003+0000","last_became_peered":"2024-09-10T10:49:31.124003+0000","last_unstale":"2024-09-10T10:49:33.131204+0000","last_undegraded":"2024-09-10T10:49:33.131204+0000","last_fullsized":"2024-09-10T10:49:33.131204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:41.120190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131730+0000","last_change":"2024-09-10T10:49:32.136246+0000","last_active":"2024-09-10T10:49:33.131730+0000","last_peered":"2024-09-10T10:49:33.131730+0000","last_clean":"2024-09-10T10:49:33.131730+0000","last_became_active":"2024-09-10T10:49:32.135969+0000","last_became_peered":"2024-09-10T10:49:32.135969+0000","last_unstale":"2024-09-10T10:49:33.131730+0000","last_undegraded":"2024-09-10T10:49:33.131730+0000","last_fullsized":"2024-09-10T10:49:33.131730+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:19.859289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138674+0000","last_change":"2024-09-10T10:49:31.127359+0000","last_active":"2024-09-10T10:49:33.138674+0000","last_peered":"2024-09-10T10:49:33.138674+0000","last_clean":"2024-09-10T10:49:33.138674+0000","last_became_active":"2024-09-10T10:49:31.127011+0000","last_became_peered":"2024-09-10T10:49:31.127011+0000","last_unstale":"2024-09-10T10:49:33.138674+0000","last_undegraded":"2024-09-10T10:49:33.138674+0000","last_fullsized":"2024-09-10T10:49:33.138674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:37.835963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339801+0000","last_change":"2024-09-10T10:49:32.151396+0000","last_active":"2024-09-10T10:49:33.339801+0000","last_peered":"2024-09-10T10:49:33.339801+0000","last_clean":"2024-09-10T10:49:33.339801+0000","last_became_active":"2024-09-10T10:49:32.150754+0000","last_became_peered":"2024-09-10T10:49:32.150754+0000","last_unstale":"2024-09-10T10:49:33.339801+0000","last_undegraded":"2024-09-10T10:49:33.339801+0000","last_fullsized":"2024-09-10T10:49:33.339801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:42.123537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"85'8","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137225+0000","last_change":"2024-09-10T10:49:31.118143+0000","last_active":"2024-09-10T10:49:33.137225+0000","last_peered":"2024-09-10T10:49:33.137225+0000","last_clean":"2024-09-10T10:49:33.137225+0000","last_became_active":"2024-09-10T10:49:31.116947+0000","last_became_peered":"2024-09-10T10:49:31.116947+0000","last_unstale":"2024-09-10T10:49:33.137225+0000","last_undegraded":"2024-09-10T10:49:33.137225+0000","last_fullsized":"2024-09-10T10:49:33.137225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:58.261531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138507+0000","last_change":"2024-09-10T10:49:32.140682+0000","last_active":"2024-09-10T10:49:33.138507+0000","last_peered":"2024-09-10T10:49:33.138507+0000","last_clean":"2024-09-10T10:49:33.138507+0000","last_became_active":"2024-09-10T10:49:32.140349+0000","last_became_peered":"2024-09-10T10:49:32.140349+0000","last_unstale":"2024-09-10T10:49:33.138507+0000","last_undegraded":"2024-09-10T10:49:33.138507+0000","last_fullsized":"2024-09-10T10:49:33.138507+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:56.686298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747160+0000","last_change":"2024-09-10T10:49:31.125550+0000","last_active":"2024-09-10T10:49:34.747160+0000","last_peered":"2024-09-10T10:49:34.747160+0000","last_clean":"2024-09-10T10:49:34.747160+0000","last_became_active":"2024-09-10T10:49:31.125285+0000","last_became_peered":"2024-09-10T10:49:31.125285+0000","last_unstale":"2024-09-10T10:49:34.747160+0000","last_undegraded":"2024-09-10T10:49:34.747160+0000","last_fullsized":"2024-09-10T10:49:34.747160+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:29.588887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120976+0000","last_change":"2024-09-10T10:49:32.141617+0000","last_active":"2024-09-10T10:49:34.120976+0000","last_peered":"2024-09-10T10:49:34.120976+0000","last_clean":"2024-09-10T10:49:34.120976+0000","last_became_active":"2024-09-10T10:49:32.140722+0000","last_became_peered":"2024-09-10T10:49:32.140722+0000","last_unstale":"2024-09-10T10:49:34.120976+0000","last_undegraded":"2024-09-10T10:49:34.120976+0000","last_fullsized":"2024-09-10T10:49:34.120976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:13.913617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746992+0000","last_change":"2024-09-10T10:49:31.122128+0000","last_active":"2024-09-10T10:49:34.746992+0000","last_peered":"2024-09-10T10:49:34.746992+0000","last_clean":"2024-09-10T10:49:34.746992+0000","last_became_active":"2024-09-10T10:49:31.120931+0000","last_became_peered":"2024-09-10T10:49:31.120931+0000","last_unstale":"2024-09-10T10:49:34.746992+0000","last_undegraded":"2024-09-10T10:49:34.746992+0000","last_fullsized":"2024-09-10T10:49:34.746992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:04.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746910+0000","last_change":"2024-09-10T10:49:32.133453+0000","last_active":"2024-09-10T10:49:34.746910+0000","last_peered":"2024-09-10T10:49:34.746910+0000","last_clean":"2024-09-10T10:49:34.746910+0000","last_became_active":"2024-09-10T10:49:32.132324+0000","last_became_peered":"2024-09-10T10:49:32.132324+0000","last_unstale":"2024-09-10T10:49:34.746910+0000","last_undegraded":"2024-09-10T10:49:34.746910+0000","last_fullsized":"2024-09-10T10:49:34.746910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:33.529330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"85'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.139194+0000","last_change":"2024-09-10T10:49:31.127806+0000","last_active":"2024-09-10T10:49:33.139194+0000","last_peered":"2024-09-10T10:49:33.139194+0000","last_clean":"2024-09-10T10:49:33.139194+0000","last_became_active":"2024-09-10T10:49:31.127130+0000","last_became_peered":"2024-09-10T10:49:31.127130+0000","last_unstale":"2024-09-10T10:49:33.139194+0000","last_undegraded":"2024-09-10T10:49:33.139194+0000","last_fullsized":"2024-09-10T10:49:33.139194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:27:18.564825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138967+0000","last_change":"2024-09-10T10:49:32.137081+0000","last_active":"2024-09-10T10:49:33.138967+0000","last_peered":"2024-09-10T10:49:33.138967+0000","last_clean":"2024-09-10T10:49:33.138967+0000","last_became_active":"2024-09-10T10:49:32.135465+0000","last_became_peered":"2024-09-10T10:49:32.135465+0000","last_unstale":"2024-09-10T10:49:33.138967+0000","last_undegraded":"2024-09-10T10:49:33.138967+0000","last_fullsized":"2024-09-10T10:49:33.138967+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:23:51.903000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.139023+0000","last_change":"2024-09-10T10:49:31.128132+0000","last_active":"2024-09-10T10:49:33.139023+0000","last_peered":"2024-09-10T10:49:33.139023+0000","last_clean":"2024-09-10T10:49:33.139023+0000","last_became_active":"2024-09-10T10:49:31.127818+0000","last_became_peered":"2024-09-10T10:49:31.127818+0000","last_unstale":"2024-09-10T10:49:33.139023+0000","last_undegraded":"2024-09-10T10:49:33.139023+0000","last_fullsized":"2024-09-10T10:49:33.139023+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:03:46.692926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.138994+0000","last_change":"2024-09-10T10:49:32.134201+0000","last_active":"2024-09-10T10:49:33.138994+0000","last_peered":"2024-09-10T10:49:33.138994+0000","last_clean":"2024-09-10T10:49:33.138994+0000","last_became_active":"2024-09-10T10:49:32.133555+0000","last_became_peered":"2024-09-10T10:49:32.133555+0000","last_unstale":"2024-09-10T10:49:33.138994+0000","last_undegraded":"2024-09-10T10:49:33.138994+0000","last_fullsized":"2024-09-10T10:49:33.138994+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:23.239499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120586+0000","last_change":"2024-09-10T10:49:31.130113+0000","last_active":"2024-09-10T10:49:34.120586+0000","last_peered":"2024-09-10T10:49:34.120586+0000","last_clean":"2024-09-10T10:49:34.120586+0000","last_became_active":"2024-09-10T10:49:31.129582+0000","last_became_peered":"2024-09-10T10:49:31.129582+0000","last_unstale":"2024-09-10T10:49:34.120586+0000","last_undegraded":"2024-09-10T10:49:34.120586+0000","last_fullsized":"2024-09-10T10:49:34.120586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:53.725260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.139362+0000","last_change":"2024-09-10T10:49:32.137945+0000","last_active":"2024-09-10T10:49:33.139362+0000","last_peered":"2024-09-10T10:49:33.139362+0000","last_clean":"2024-09-10T10:49:33.139362+0000","last_became_active":"2024-09-10T10:49:32.137644+0000","last_became_peered":"2024-09-10T10:49:32.137644+0000","last_unstale":"2024-09-10T10:49:33.139362+0000","last_undegraded":"2024-09-10T10:49:33.139362+0000","last_fullsized":"2024-09-10T10:49:33.139362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:23.965971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746669+0000","last_change":"2024-09-10T10:49:31.128745+0000","last_active":"2024-09-10T10:49:34.746669+0000","last_peered":"2024-09-10T10:49:34.746669+0000","last_clean":"2024-09-10T10:49:34.746669+0000","last_became_active":"2024-09-10T10:49:31.128467+0000","last_became_peered":"2024-09-10T10:49:31.128467+0000","last_unstale":"2024-09-10T10:49:34.746669+0000","last_undegraded":"2024-09-10T10:49:34.746669+0000","last_fullsized":"2024-09-10T10:49:34.746669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:47.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408981+0000","last_change":"2024-09-10T10:49:32.153310+0000","last_active":"2024-09-10T10:49:33.408981+0000","last_peered":"2024-09-10T10:49:33.408981+0000","last_clean":"2024-09-10T10:49:33.408981+0000","last_became_active":"2024-09-10T10:49:32.152991+0000","last_became_peered":"2024-09-10T10:49:32.152991+0000","last_unstale":"2024-09-10T10:49:33.408981+0000","last_undegraded":"2024-09-10T10:49:33.408981+0000","last_fullsized":"2024-09-10T10:49:33.408981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:45.697766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120560+0000","last_change":"2024-09-10T10:49:31.125117+0000","last_active":"2024-09-10T10:49:34.120560+0000","last_peered":"2024-09-10T10:49:34.120560+0000","last_clean":"2024-09-10T10:49:34.120560+0000","last_became_active":"2024-09-10T10:49:31.124375+0000","last_became_peered":"2024-09-10T10:49:31.124375+0000","last_unstale":"2024-09-10T10:49:34.120560+0000","last_undegraded":"2024-09-10T10:49:34.120560+0000","last_fullsized":"2024-09-10T10:49:34.120560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:11.014014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747177+0000","last_change":"2024-09-10T10:49:32.140726+0000","last_active":"2024-09-10T10:49:34.747177+0000","last_peered":"2024-09-10T10:49:34.747177+0000","last_clean":"2024-09-10T10:49:34.747177+0000","last_became_active":"2024-09-10T10:49:32.139549+0000","last_became_peered":"2024-09-10T10:49:32.139549+0000","last_unstale":"2024-09-10T10:49:34.747177+0000","last_undegraded":"2024-09-10T10:49:34.747177+0000","last_fullsized":"2024-09-10T10:49:34.747177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:19.859289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"85'2","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339911+0000","last_change":"2024-09-10T10:49:31.132881+0000","last_active":"2024-09-10T10:49:33.339911+0000","last_peered":"2024-09-10T10:49:33.339911+0000","last_clean":"2024-09-10T10:49:33.339911+0000","last_became_active":"2024-09-10T10:49:31.132536+0000","last_became_peered":"2024-09-10T10:49:31.132536+0000","last_unstale":"2024-09-10T10:49:33.339911+0000","last_undegraded":"2024-09-10T10:49:33.339911+0000","last_fullsized":"2024-09-10T10:49:33.339911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:34:29.990301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339587+0000","last_change":"2024-09-10T10:49:32.145937+0000","last_active":"2024-09-10T10:49:33.339587+0000","last_peered":"2024-09-10T10:49:33.339587+0000","last_clean":"2024-09-10T10:49:33.339587+0000","last_became_active":"2024-09-10T10:49:32.145556+0000","last_became_peered":"2024-09-10T10:49:32.145556+0000","last_unstale":"2024-09-10T10:49:33.339587+0000","last_undegraded":"2024-09-10T10:49:33.339587+0000","last_fullsized":"2024-09-10T10:49:33.339587+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:35.637359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137749+0000","last_change":"2024-09-10T10:49:31.125817+0000","last_active":"2024-09-10T10:49:33.137749+0000","last_peered":"2024-09-10T10:49:33.137749+0000","last_clean":"2024-09-10T10:49:33.137749+0000","last_became_active":"2024-09-10T10:49:31.125554+0000","last_became_peered":"2024-09-10T10:49:31.125554+0000","last_unstale":"2024-09-10T10:49:33.137749+0000","last_undegraded":"2024-09-10T10:49:33.137749+0000","last_fullsized":"2024-09-10T10:49:33.137749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:10:55.513897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131299+0000","last_change":"2024-09-10T10:49:32.137502+0000","last_active":"2024-09-10T10:49:33.131299+0000","last_peered":"2024-09-10T10:49:33.131299+0000","last_clean":"2024-09-10T10:49:33.131299+0000","last_became_active":"2024-09-10T10:49:32.137055+0000","last_became_peered":"2024-09-10T10:49:32.137055+0000","last_unstale":"2024-09-10T10:49:33.131299+0000","last_undegraded":"2024-09-10T10:49:33.131299+0000","last_fullsized":"2024-09-10T10:49:33.131299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:48.432641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339855+0000","last_change":"2024-09-10T10:49:31.133562+0000","last_active":"2024-09-10T10:49:33.339855+0000","last_peered":"2024-09-10T10:49:33.339855+0000","last_clean":"2024-09-10T10:49:33.339855+0000","last_became_active":"2024-09-10T10:49:31.133247+0000","last_became_peered":"2024-09-10T10:49:31.133247+0000","last_unstale":"2024-09-10T10:49:33.339855+0000","last_undegraded":"2024-09-10T10:49:33.339855+0000","last_fullsized":"2024-09-10T10:49:33.339855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:41:29.587194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747087+0000","last_change":"2024-09-10T10:49:32.126766+0000","last_active":"2024-09-10T10:49:34.747087+0000","last_peered":"2024-09-10T10:49:34.747087+0000","last_clean":"2024-09-10T10:49:34.747087+0000","last_became_active":"2024-09-10T10:49:32.126375+0000","last_became_peered":"2024-09-10T10:49:32.126375+0000","last_unstale":"2024-09-10T10:49:34.747087+0000","last_undegraded":"2024-09-10T10:49:34.747087+0000","last_fullsized":"2024-09-10T10:49:34.747087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:46.971506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746838+0000","last_change":"2024-09-10T10:49:31.122923+0000","last_active":"2024-09-10T10:49:34.746838+0000","last_peered":"2024-09-10T10:49:34.746838+0000","last_clean":"2024-09-10T10:49:34.746838+0000","last_became_active":"2024-09-10T10:49:31.122649+0000","last_became_peered":"2024-09-10T10:49:31.122649+0000","last_unstale":"2024-09-10T10:49:34.746838+0000","last_undegraded":"2024-09-10T10:49:34.746838+0000","last_fullsized":"2024-09-10T10:49:34.746838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:11.014014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.131760+0000","last_change":"2024-09-10T10:49:32.128781+0000","last_active":"2024-09-10T10:49:33.131760+0000","last_peered":"2024-09-10T10:49:33.131760+0000","last_clean":"2024-09-10T10:49:33.131760+0000","last_became_active":"2024-09-10T10:49:32.128480+0000","last_became_peered":"2024-09-10T10:49:32.128480+0000","last_unstale":"2024-09-10T10:49:33.131760+0000","last_undegraded":"2024-09-10T10:49:33.131760+0000","last_fullsized":"2024-09-10T10:49:33.131760+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:01:24.478879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408712+0000","last_change":"2024-09-10T10:49:31.128327+0000","last_active":"2024-09-10T10:49:33.408712+0000","last_peered":"2024-09-10T10:49:33.408712+0000","last_clean":"2024-09-10T10:49:33.408712+0000","last_became_active":"2024-09-10T10:49:31.128016+0000","last_became_peered":"2024-09-10T10:49:31.128016+0000","last_unstale":"2024-09-10T10:49:33.408712+0000","last_undegraded":"2024-09-10T10:49:33.408712+0000","last_fullsized":"2024-09-10T10:49:33.408712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:58.431491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.747110+0000","last_change":"2024-09-10T10:49:32.146901+0000","last_active":"2024-09-10T10:49:34.747110+0000","last_peered":"2024-09-10T10:49:34.747110+0000","last_clean":"2024-09-10T10:49:34.747110+0000","last_became_active":"2024-09-10T10:49:32.146549+0000","last_became_peered":"2024-09-10T10:49:32.146549+0000","last_unstale":"2024-09-10T10:49:34.747110+0000","last_undegraded":"2024-09-10T10:49:34.747110+0000","last_fullsized":"2024-09-10T10:49:34.747110+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:05.006982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137703+0000","last_change":"2024-09-10T10:49:31.124068+0000","last_active":"2024-09-10T10:49:33.137703+0000","last_peered":"2024-09-10T10:49:33.137703+0000","last_clean":"2024-09-10T10:49:33.137703+0000","last_became_active":"2024-09-10T10:49:31.123728+0000","last_became_peered":"2024-09-10T10:49:31.123728+0000","last_unstale":"2024-09-10T10:49:33.137703+0000","last_undegraded":"2024-09-10T10:49:33.137703+0000","last_fullsized":"2024-09-10T10:49:33.137703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:43:20.133803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.121087+0000","last_change":"2024-09-10T10:49:32.144016+0000","last_active":"2024-09-10T10:49:34.121087+0000","last_peered":"2024-09-10T10:49:34.121087+0000","last_clean":"2024-09-10T10:49:34.121087+0000","last_became_active":"2024-09-10T10:49:32.143710+0000","last_became_peered":"2024-09-10T10:49:32.143710+0000","last_unstale":"2024-09-10T10:49:34.121087+0000","last_undegraded":"2024-09-10T10:49:34.121087+0000","last_fullsized":"2024-09-10T10:49:34.121087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:05.161295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.746746+0000","last_change":"2024-09-10T10:49:31.134411+0000","last_active":"2024-09-10T10:49:34.746746+0000","last_peered":"2024-09-10T10:49:34.746746+0000","last_clean":"2024-09-10T10:49:34.746746+0000","last_became_active":"2024-09-10T10:49:31.134171+0000","last_became_peered":"2024-09-10T10:49:31.134171+0000","last_unstale":"2024-09-10T10:49:34.746746+0000","last_undegraded":"2024-09-10T10:49:34.746746+0000","last_fullsized":"2024-09-10T10:49:34.746746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:55:22.962625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.139422+0000","last_change":"2024-09-10T10:49:32.138065+0000","last_active":"2024-09-10T10:49:33.139422+0000","last_peered":"2024-09-10T10:49:33.139422+0000","last_clean":"2024-09-10T10:49:33.139422+0000","last_became_active":"2024-09-10T10:49:32.137784+0000","last_became_peered":"2024-09-10T10:49:32.137784+0000","last_unstale":"2024-09-10T10:49:33.139422+0000","last_undegraded":"2024-09-10T10:49:33.139422+0000","last_fullsized":"2024-09-10T10:49:33.139422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:05.205770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:34.120686+0000","last_change":"2024-09-10T10:49:31.126375+0000","last_active":"2024-09-10T10:49:34.120686+0000","last_peered":"2024-09-10T10:49:34.120686+0000","last_clean":"2024-09-10T10:49:34.120686+0000","last_became_active":"2024-09-10T10:49:31.126055+0000","last_became_peered":"2024-09-10T10:49:31.126055+0000","last_unstale":"2024-09-10T10:49:34.120686+0000","last_undegraded":"2024-09-10T10:49:34.120686+0000","last_fullsized":"2024-09-10T10:49:34.120686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:21:24.222829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.408963+0000","last_change":"2024-09-10T10:49:32.135820+0000","last_active":"2024-09-10T10:49:33.408963+0000","last_peered":"2024-09-10T10:49:33.408963+0000","last_clean":"2024-09-10T10:49:33.408963+0000","last_became_active":"2024-09-10T10:49:32.135535+0000","last_became_peered":"2024-09-10T10:49:32.135535+0000","last_unstale":"2024-09-10T10:49:33.408963+0000","last_undegraded":"2024-09-10T10:49:33.408963+0000","last_fullsized":"2024-09-10T10:49:33.408963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:31.099475+0000","last_clean_scrub_stamp":"2024-09-10T10:49:31.099475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:13.585122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.137585+0000","last_change":"2024-09-10T10:49:31.131228+0000","last_active":"2024-09-10T10:49:33.137585+0000","last_peered":"2024-09-10T10:49:33.137585+0000","last_clean":"2024-09-10T10:49:33.137585+0000","last_became_active":"2024-09-10T10:49:31.130375+0000","last_became_peered":"2024-09-10T10:49:31.130375+0000","last_unstale":"2024-09-10T10:49:33.137585+0000","last_undegraded":"2024-09-10T10:49:33.137585+0000","last_fullsized":"2024-09-10T10:49:33.137585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:49:30.096128+0000","last_clean_scrub_stamp":"2024-09-10T10:49:30.096128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:35:31.372058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":289,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-10T10:49:33.339378+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:49:33.339378+0000","last_peered":"2024-09-10T10:49:33.339378+0000","last_clean":"2024-09-10T10:49:33.339378+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:49:33.339378+0000","last_undegraded":"2024-09-10T10:49:33.339378+0000","last_fullsized":"2024-09-10T10:49:33.339378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509514,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":483278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509516,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1052672,"data_stored":483298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":482648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":484342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1044480,"data_stored":482626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1074640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":482648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574922,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1073084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:49:36.765 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:49:36.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:49:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.964+0000 7f26cb200640 1 Processor -- start 2024-09-10T10:49:36.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.964+0000 7f26cb200640 1 -- start start 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f26cc15e350 0x7f26cc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 0x7f26cc0d4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f26cc0d4d30 con 0x7f26cc15e350 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f26cc0d3690 con 0x7f26cc0d2500 2024-09-10T10:49:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26cb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc0cf1e0 con 0x7f26cc0d48c0 2024-09-10T10:49:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26c9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f26cc15e350 0x7f26cc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26c9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f26cc15e350 0x7f26cc0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50692/0 (socket says 172.21.15.28:50692) 2024-09-10T10:49:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.965+0000 7f26c9e00640 1 -- 172.21.15.28:0/1604897245 learned_addr learned my addr 172.21.15.28:0/1604897245 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:36.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c9e00640 1 -- 172.21.15.28:0/1604897245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 msgr2=0x7f26cc0d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26ca800640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c1400640 1 --2- 172.21.15.28:0/1604897245 >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 0x7f26cc0d4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c9e00640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c9e00640 1 -- 172.21.15.28:0/1604897245 >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 msgr2=0x7f26cc0d4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c9e00640 1 --2- 172.21.15.28:0/1604897245 >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 0x7f26cc0d4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c9e00640 1 -- 172.21.15.28:0/1604897245 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc0d2a80 con 0x7f26cc15e350 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26c1400640 1 --2- 172.21.15.28:0/1604897245 >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 0x7f26cc0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.966+0000 7f26ca800640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26c9e00640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc15e350 0x7f26cc0d1fc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f26cc06cb70 tx=0x7f26bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e624f9847262ac02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26c9400640 1 -- 172.21.15.28:0/1604897245 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc037070 con 0x7f26cc15e350 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26c9400640 1 -- 172.21.15.28:0/1604897245 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc032440 con 0x7f26cc15e350 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26c9400640 1 -- 172.21.15.28:0/1604897245 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc032810 con 0x7f26cc15e350 2024-09-10T10:49:36.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26cb200640 1 -- 172.21.15.28:0/1604897245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc15e350 msgr2=0x7f26cc0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26cb200640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc15e350 0x7f26cc0d1fc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f26cc06cb70 tx=0x7f26bc02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26cb200640 1 -- 172.21.15.28:0/1604897245 shutdown_connections 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26cb200640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.967+0000 7f26cb200640 1 --2- 172.21.15.28:0/1604897245 >> v2:172.21.15.138:3300/0 conn(0x7f26cc0d2500 0x7f26cc0d4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.968+0000 7f26cb200640 1 --2- 172.21.15.28:0/1604897245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc15e350 0x7f26cc0d1fc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.968+0000 7f26cb200640 1 -- 172.21.15.28:0/1604897245 >> 172.21.15.28:0/1604897245 conn(0x7f26cc09d710 msgr2=0x7f26cc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.968+0000 7f26cb200640 1 -- 172.21.15.28:0/1604897245 shutdown_connections 2024-09-10T10:49:36.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.968+0000 7f26cb200640 1 -- 172.21.15.28:0/1604897245 wait complete. 2024-09-10T10:49:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.969+0000 7f26cb200640 1 Processor -- start 2024-09-10T10:49:36.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.969+0000 7f26cb200640 1 -- start start 2024-09-10T10:49:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc15e350 0x7f26cc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc0d3320 con 0x7f26cc0d2500 2024-09-10T10:49:36.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc0d3040 con 0x7f26cc15e350 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26cb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f26cc0b3620 con 0x7f26cc0d48c0 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26c9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26c9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50702/0 (socket says 172.21.15.28:50702) 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.970+0000 7f26c9e00640 1 -- 172.21.15.28:0/734639041 learned_addr learned my addr 172.21.15.28:0/734639041 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c1400640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26ca800640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc15e350 0x7f26cc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 -- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 msgr2=0x7f26cc0cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 -- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc15e350 msgr2=0x7f26cc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc15e350 0x7f26cc0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:36.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 -- 172.21.15.28:0/734639041 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc0c80c0 con 0x7f26cc0d2500 2024-09-10T10:49:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.971+0000 7f26c9e00640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02a220 tx=0x7f26bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.972+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc037040 con 0x7f26cc0d2500 2024-09-10T10:49:36.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.972+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26bc042070 con 0x7f26cc0d2500 2024-09-10T10:49:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.972+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc030bb0 con 0x7f26cc0d2500 2024-09-10T10:49:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.972+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc0c83a0 con 0x7f26cc0d2500 2024-09-10T10:49:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.972+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc0ce4b0 con 0x7f26cc0d2500 2024-09-10T10:49:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.973+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26cc15e730 con 0x7f26cc0d2500 2024-09-10T10:49:36.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.974+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26bc047460 con 0x7f26cc0d2500 2024-09-10T10:49:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.975+0000 7f26c2a00640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 0x7f26a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.975+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f26bc0abb10 con 0x7f26cc0d2500 2024-09-10T10:49:36.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.975+0000 7f26c1400640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 0x7f26a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:36.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.976+0000 7f26c1400640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 0x7f26a806e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0cc7b0 tx=0x7f26b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:36.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:36.980+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26bc079c40 con 0x7f26cc0d2500 2024-09-10T10:49:37.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.235+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f26cc0b2d20 con 0x7f26cc0d2500 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.238+0000 7f26c2a00640 1 -- 172.21.15.28:0/734639041 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f26bc07df40 con 0x7f26cc0d2500 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:49:37.240 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:49:37.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 msgr2=0x7f26a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 0x7f26a806e320 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0cc7b0 tx=0x7f26b4008040 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 msgr2=0x7f26cc0ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f26bc02a220 tx=0x7f26bc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 shutdown_connections 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a806be60 0x7f26a806e320 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26cc15e350 0x7f26cc0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f26cc0d48c0 0x7f26cc0cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 --2- 172.21.15.28:0/734639041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26cc0d2500 0x7f26cc0ca840 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 >> 172.21.15.28:0/734639041 conn(0x7f26cc09d710 msgr2=0x7f26cc0b4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 shutdown_connections 2024-09-10T10:49:37.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.240+0000 7f26cb200640 1 -- 172.21.15.28:0/734639041 wait complete. 2024-09-10T10:49:37.258 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:37.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.413+0000 7fb89d200640 1 Processor -- start 2024-09-10T10:49:37.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.413+0000 7fb89d200640 1 -- start start 2024-09-10T10:49:37.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb89d200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb89815e660 0x7fb89806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:37.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb89d200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8980cf000 con 0x7fb8980d1f90 2024-09-10T10:49:37.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb89d200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8980d29a0 con 0x7fb8980d0dd0 2024-09-10T10:49:37.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb89d200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb8980b37d0 con 0x7fb89815e660 2024-09-10T10:49:37.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb897e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb8980d1f90 0x7fb8980cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38710/0 (socket says 172.21.15.28:38710) 2024-09-10T10:49:37.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.414+0000 7fb897e00640 1 -- 172.21.15.28:0/3061936333 learned_addr learned my addr 172.21.15.28:0/3061936333 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:37.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896a00640 1 --2- 172.21.15.28:0/3061936333 >> v2:172.21.15.138:3300/0 conn(0x7fb89815e660 0x7fb89806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:37.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1589915866 0 0) 0x7fb8980cf000 con 0x7fb8980d1f90 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb87c003520 con 0x7fb8980d1f90 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896a00640 1 -- 172.21.15.28:0/3061936333 >> v1:172.21.15.138:6789/0 conn(0x7fb8980d0dd0 legacy=0x7fb8980d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3018500310 0 0) 0x7fb87c003520 con 0x7fb8980d1f90 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896a00640 1 -- 172.21.15.28:0/3061936333 >> v1:172.21.15.28:6789/0 conn(0x7fb8980d1f90 legacy=0x7fb8980cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.415+0000 7fb896a00640 1 -- 172.21.15.28:0/3061936333 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8980d2c60 con 0x7fb89815e660 2024-09-10T10:49:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.416+0000 7fb896a00640 1 --2- 172.21.15.28:0/3061936333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89815e660 0x7fb89806ccd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097a0 tx=0x7fb88402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4afce1bf949f935f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:37.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.417+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884037070 con 0x7fb89815e660 2024-09-10T10:49:37.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.417+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88402ac50 con 0x7fb89815e660 2024-09-10T10:49:37.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.417+0000 7fb896000640 1 -- 172.21.15.28:0/3061936333 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8900019f0 con 0x7fb89815e660 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 -- 172.21.15.28:0/3061936333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89815e660 msgr2=0x7fb89806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 --2- 172.21.15.28:0/3061936333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89815e660 0x7fb89806ccd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8840097a0 tx=0x7fb88402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 -- 172.21.15.28:0/3061936333 shutdown_connections 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 --2- 172.21.15.28:0/3061936333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89815e660 0x7fb89806ccd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 -- 172.21.15.28:0/3061936333 >> 172.21.15.28:0/3061936333 conn(0x7fb89808bfc0 msgr2=0x7fb8980d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 -- 172.21.15.28:0/3061936333 shutdown_connections 2024-09-10T10:49:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.418+0000 7fb89d200640 1 -- 172.21.15.28:0/3061936333 wait complete. 2024-09-10T10:49:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.419+0000 7fb89d200640 1 Processor -- start 2024-09-10T10:49:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.419+0000 7fb89d200640 1 -- start start 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980d1f90 0x7fb8981e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 0x7fb8981e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8980d3220 con 0x7fb8980d0dd0 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8980cf330 con 0x7fb8980d1f90 2024-09-10T10:49:37.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.420+0000 7fb89d200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8980d33a0 con 0x7fb89815e660 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50714/0 (socket says 172.21.15.28:50714) 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 -- 172.21.15.28:0/1940276424 learned_addr learned my addr 172.21.15.28:0/1940276424 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb896a00640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980d1f90 0x7fb8981e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897e00640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 0x7fb8981e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 -- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 msgr2=0x7fb8981e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 0x7fb8981e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 -- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980d1f90 msgr2=0x7fb8981e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980d1f90 0x7fb8981e7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 -- 172.21.15.28:0/1940276424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8980cb610 con 0x7fb8980d0dd0 2024-09-10T10:49:37.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897e00640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 0x7fb8981e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:37.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.421+0000 7fb897400640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c450 tx=0x7fb88800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.422+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888017070 con 0x7fb8980d0dd0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.422+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8880052d0 con 0x7fb8980d0dd0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.422+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888003830 con 0x7fb8980d0dd0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.422+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb898150d40 con 0x7fb8980d0dd0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.422+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb898161400 con 0x7fb8980d0dd0 2024-09-10T10:49:37.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.423+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89806d2a0 con 0x7fb8980d0dd0 2024-09-10T10:49:37.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.424+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb88801a070 con 0x7fb8980d0dd0 2024-09-10T10:49:37.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.425+0000 7fb88f400640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 0x7fb87806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:37.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.425+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fb88808a410 con 0x7fb8980d0dd0 2024-09-10T10:49:37.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.425+0000 7fb896a00640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 0x7fb87806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:37.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.426+0000 7fb896a00640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 0x7fb87806e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb884034000 tx=0x7fb88402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:37.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.429+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb888058540 con 0x7fb8980d0dd0 2024-09-10T10:49:37.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.649+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8980d1b20 con 0x7fb8980d0dd0 2024-09-10T10:49:37.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.650+0000 7fb88f400640 1 -- 172.21.15.28:0/1940276424 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7fb88805c840 con 0x7fb8980d0dd0 2024-09-10T10:49:37.652 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:37.652 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"btime":"2024-09-10T10:49:35:142470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:34.565062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:37.652 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 68 2024-09-10T10:49:37.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 msgr2=0x7fb87806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 0x7fb87806e2f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb884034000 tx=0x7fb88402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:37.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 msgr2=0x7fb898150800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:37.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c450 tx=0x7fb88800c910 comp rx=0 tx=0).stop 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 shutdown_connections 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87806be30 0x7fb87806e2f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb89815e660 0x7fb8981e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8980d1f90 0x7fb8981e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 --2- 172.21.15.28:0/1940276424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8980d0dd0 0x7fb898150800 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 >> 172.21.15.28:0/1940276424 conn(0x7fb89808bfc0 msgr2=0x7fb8980b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 shutdown_connections 2024-09-10T10:49:37.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:37.652+0000 7fb89d200640 1 -- 172.21.15.28:0/1940276424 wait complete. 2024-09-10T10:49:37.699 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:49:37.698+0000 7fe011e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:49:38.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:49:31.540549+0000', 'modified': '2024-09-10T10:49:34.565062+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6411}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6411': {'gid': 6411, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.138:6837/4263014325', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6835', 'nonce': 4263014325}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 4263014325}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6411, 'qdb_cluster': [6411]} 2024-09-10T10:49:38.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:49:38.073 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T10:49:38.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.229+0000 7fcd5b000640 1 Processor -- start 2024-09-10T10:49:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.229+0000 7fcd5b000640 1 -- start start 2024-09-10T10:49:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd5b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd5b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcd5415e1f0 0x7fcd5406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd5b000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd54146140 con 0x7fcd54072de0 2024-09-10T10:49:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd5b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcd540cd740 con 0x7fcd5415e1f0 2024-09-10T10:49:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd5b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540cda20 con 0x7fcd54072320 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd53e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:45436/0 (socket says 172.21.15.28:45436) 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.231+0000 7fcd53e00640 1 -- 172.21.15.28:0/3323159289 learned_addr learned my addr 172.21.15.28:0/3323159289 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.232+0000 7fcd59000640 1 --2- 172.21.15.28:0/3323159289 >> v2:172.21.15.138:3300/0 conn(0x7fcd5415e1f0 0x7fcd5406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.232+0000 7fcd53e00640 1 -- 172.21.15.28:0/3323159289 >> v2:172.21.15.138:3300/0 conn(0x7fcd5415e1f0 msgr2=0x7fcd5406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.232+0000 7fcd53e00640 1 --2- 172.21.15.28:0/3323159289 >> v2:172.21.15.138:3300/0 conn(0x7fcd5415e1f0 0x7fcd5406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.232+0000 7fcd53e00640 1 -- 172.21.15.28:0/3323159289 >> v1:172.21.15.138:6789/0 conn(0x7fcd54072de0 legacy=0x7fcd5406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.232+0000 7fcd53e00640 1 -- 172.21.15.28:0/3323159289 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd540cdd00 con 0x7fcd54072320 2024-09-10T10:49:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.233+0000 7fcd53e00640 1 --2- 172.21.15.28:0/3323159289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30418263f8a991b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.233+0000 7fcd53400640 1 -- 172.21.15.28:0/3323159289 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44037070 con 0x7fcd54072320 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.233+0000 7fcd53400640 1 -- 172.21.15.28:0/3323159289 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd44032440 con 0x7fcd54072320 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.233+0000 7fcd53400640 1 -- 172.21.15.28:0/3323159289 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd44032810 con 0x7fcd54072320 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 -- 172.21.15.28:0/3323159289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 msgr2=0x7fcd540cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 --2- 172.21.15.28:0/3323159289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009870 tx=0x7fcd4402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 -- 172.21.15.28:0/3323159289 shutdown_connections 2024-09-10T10:49:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 --2- 172.21.15.28:0/3323159289 >> v2:172.21.15.138:3300/0 conn(0x7fcd5415e1f0 0x7fcd5406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 --2- 172.21.15.28:0/3323159289 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd540cce60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 -- 172.21.15.28:0/3323159289 >> 172.21.15.28:0/3323159289 conn(0x7fcd54067d70 msgr2=0x7fcd5406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 -- 172.21.15.28:0/3323159289 shutdown_connections 2024-09-10T10:49:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.234+0000 7fcd5b000640 1 -- 172.21.15.28:0/3323159289 wait complete. 2024-09-10T10:49:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.236+0000 7fcd5b000640 1 Processor -- start 2024-09-10T10:49:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.236+0000 7fcd5b000640 1 -- start start 2024-09-10T10:49:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd54141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5415e1f0 0x7fcd54140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540cd520 con 0x7fcd54072de0 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540cd3a0 con 0x7fcd5415e1f0 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd5b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5406ed00 con 0x7fcd54072320 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50720/0 (socket says 172.21.15.28:50720) 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 -- 172.21.15.28:0/484085682 learned_addr learned my addr 172.21.15.28:0/484085682 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd53e00640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd54141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd59000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5415e1f0 0x7fcd54140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 -- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 msgr2=0x7fcd54141ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd54141ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 -- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5415e1f0 msgr2=0x7fcd54140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5415e1f0 0x7fcd54140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.237+0000 7fcd4b400640 1 -- 172.21.15.28:0/484085682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd54142840 con 0x7fcd54072de0 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd4b400640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c440 tx=0x7fcd3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd50c00640 1 -- 172.21.15.28:0/484085682 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0052d0 con 0x7fcd54072de0 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd50c00640 1 -- 172.21.15.28:0/484085682 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c005470 con 0x7fcd54072de0 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd54142b20 con 0x7fcd54072de0 2024-09-10T10:49:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd50c00640 1 -- 172.21.15.28:0/484085682 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c007260 con 0x7fcd54072de0 2024-09-10T10:49:38.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.238+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5412de50 con 0x7fcd54072de0 2024-09-10T10:49:38.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.240+0000 7fcd50c00640 1 -- 172.21.15.28:0/484085682 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd3c007400 con 0x7fcd54072de0 2024-09-10T10:49:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.241+0000 7fcd50c00640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.241+0000 7fcd50c00640 1 -- 172.21.15.28:0/484085682 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fcd3c08a4c0 con 0x7fcd54072de0 2024-09-10T10:49:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.241+0000 7fcd53e00640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.241+0000 7fcd5dea9740 1 Processor -- start 2024-09-10T10:49:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.242+0000 7fcd5dea9740 1 -- start start 2024-09-10T10:49:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.242+0000 7fcd53e00640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd44034000 tx=0x7fcd4402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f3017490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2fcc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5636f2f303b0 con 0x5636f2fcf4b0 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636f2f35410 con 0x5636f2fd3480 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5636f2edd620 con 0x5636f2fd1cf0 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50724/0 (socket says 172.21.15.28:50724) 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 -- 172.21.15.28:0/2279166325 learned_addr learned my addr 172.21.15.28:0/2279166325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd53e00640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f3017490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 -- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 msgr2=0x5636f3017490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd59000640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2fcc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f3017490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 -- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 msgr2=0x5636f2fcc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2fcc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd4b400640 1 -- 172.21.15.28:0/2279166325 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f2c482d0 con 0x5636f2fcf4b0 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd53e00640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f3017490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.243+0000 7fcd59000640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2fcc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd4b400640 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0887c0 tx=0x7fcd3c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75460e033f6d980d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd48c00640 1 -- 172.21.15.28:0/2279166325 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c056900 con 0x5636f2fcf4b0 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd48c00640 1 -- 172.21.15.28:0/2279166325 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c054c70 con 0x5636f2fcf4b0 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd48c00640 1 -- 172.21.15.28:0/2279166325 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c057a50 con 0x5636f2fcf4b0 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 -- 172.21.15.28:0/2279166325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 msgr2=0x5636f2fd20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0887c0 tx=0x7fcd3c00a000 comp rx=0 tx=0).stop 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 -- 172.21.15.28:0/2279166325 shutdown_connections 2024-09-10T10:49:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2fcc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2fcf4b0 0x5636f2fd20d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/2279166325 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f3017490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.244+0000 7fcd5dea9740 1 -- 172.21.15.28:0/2279166325 >> 172.21.15.28:0/2279166325 conn(0x5636f2fce740 msgr2=0x5636f3016960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.245+0000 7fcd5dea9740 1 -- 172.21.15.28:0/2279166325 shutdown_connections 2024-09-10T10:49:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.245+0000 7fcd5dea9740 1 -- 172.21.15.28:0/2279166325 wait complete. 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.245+0000 7fcd5dea9740 1 -- start start 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.245+0000 7fcd5dea9740 1 Processor -- start 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.246+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f2ffbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.246+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.246+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.246+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5636f2ef0780 con 0x5636f2ffc140 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.246+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636f2f313d0 con 0x5636f2fd3480 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5636f2fcdbe0 con 0x5636f2fd1cf0 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd5dea9740 10 client.0 _ll_register_callbacks cb 0x5636f2f2e5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50734/0 (socket says 172.21.15.28:50734) 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 -- 172.21.15.28:0/104061823 learned_addr learned my addr 172.21.15.28:0/104061823 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd53e00640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f2ffbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd4b400640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 -- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 msgr2=0x5636f2ffbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f2ffbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 -- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 msgr2=0x5636f2ff4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.247+0000 7fcd59000640 1 -- 172.21.15.28:0/104061823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f2d126b0 con 0x5636f2ffc140 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd59000640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007d90 tx=0x7fcd4c012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14ddc65784c16d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd41600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c01c020 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd3c054c70 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fcd4c018670 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c026050 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd5dea9740 10 client.6525 fetch_fsmap learned FSMap version 68 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5636f2ffa010 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.248+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd3c054c70 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fcd4c029020 con 0x5636f2ffc140 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 10 client.6525 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 10 client.6525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 20 client.6525 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f2ffaa00 0x5636f2ff3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 4 client.6525 mds_command: new command op to 6411 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 --> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636f2f42b40 con 0x5636f2ffaa00 2024-09-10T10:49:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.249+0000 7fcd53e00640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f2ffaa00 0x5636f2ff3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.250+0000 7fcd53e00640 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f2ffaa00 0x5636f2ff3c00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d640e66127cac6 server_cookie=ac2395c5f7f05aff in_seq=0 out_seq=0 2024-09-10T10:49:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.250+0000 7fcd41600640 10 client.6525 ms_handle_connect on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.251+0000 7fcd41600640 1 -- 172.21.15.28:0/104061823 <== mds.0 v2:172.21.15.138:6835/4263014325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636f2f42b40 con 0x5636f2ffaa00 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.251+0000 7fcd41600640 10 client.6525 handle_command_reply: tid=0 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.251+0000 7fcd5dea9740 1 client.6525 shutdown 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.251+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 msgr2=0x5636f2ff5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.251+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c007d90 tx=0x7fcd4c012610 comp rx=0 tx=0).stop 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 shutdown_connections 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f2ffaa00 0x5636f2ff3c00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff5290 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/104061823 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f2ffbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd41600640 0 client.6525 ms_handle_reset on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:38.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 >> 172.21.15.28:0/104061823 conn(0x5636f2fcf4b0 msgr2=0x5636f30149e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 shutdown_connections 2024-09-10T10:49:38.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 1 -- 172.21.15.28:0/104061823 wait complete. 2024-09-10T10:49:38.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.252+0000 7fcd5dea9740 20 client.6525 trim_cache size 0 max 16384 2024-09-10T10:49:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.271+0000 7fcd5dea9740 1 Processor -- start 2024-09-10T10:49:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.271+0000 7fcd5dea9740 1 -- start start 2024-09-10T10:49:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 0x5636f2fcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 0x5636f3034ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540b4270 con 0x5636f2ffc140 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540b35d0 con 0x5636f2fd1cf0 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540b3c20 con 0x5636f2fd3480 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50748/0 (socket says 172.21.15.28:50748) 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 -- 172.21.15.28:0/1064442416 learned_addr learned my addr 172.21.15.28:0/1064442416 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd53e00640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 0x5636f2fcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd4b400640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 0x5636f3034ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 -- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 msgr2=0x5636f3034ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 0x5636f3034ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 -- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 msgr2=0x5636f2fcc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 0x5636f2fcc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd59000640 1 -- 172.21.15.28:0/1064442416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f2fd2440 con 0x5636f2ffc140 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd53e00640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 0x5636f2fcc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.272+0000 7fcd4b400640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 0x5636f3034ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd59000640 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009c30 tx=0x7fcd4c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656e532646722b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd43e00640 1 -- 172.21.15.28:0/1064442416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c02f070 con 0x5636f2ffc140 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd43e00640 1 -- 172.21.15.28:0/1064442416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c018b70 con 0x5636f2ffc140 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd43e00640 1 -- 172.21.15.28:0/1064442416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c01c040 con 0x5636f2ffc140 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 -- 172.21.15.28:0/1064442416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 msgr2=0x5636f2ff4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009c30 tx=0x7fcd4c018000 comp rx=0 tx=0).stop 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 -- 172.21.15.28:0/1064442416 shutdown_connections 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff4be0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd3480 0x5636f3034ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/1064442416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd1cf0 0x5636f2fcc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.273+0000 7fcd5dea9740 1 -- 172.21.15.28:0/1064442416 >> 172.21.15.28:0/1064442416 conn(0x5636f2fcf4b0 msgr2=0x5636f30138e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.274+0000 7fcd5dea9740 1 -- 172.21.15.28:0/1064442416 shutdown_connections 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.274+0000 7fcd5dea9740 1 -- 172.21.15.28:0/1064442416 wait complete. 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.274+0000 7fcd5dea9740 1 -- start start 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.274+0000 7fcd5dea9740 1 Processor -- start 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f309a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5636f302c290 con 0x5636f2ffc140 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5636f302c450 con 0x5636f2fd3480 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5636f2ffaed0 con 0x5636f2fd1cf0 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.275+0000 7fcd5dea9740 10 client.0 _ll_register_callbacks cb 0x7fcd4c009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50754/0 (socket says 172.21.15.28:50754) 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 -- 172.21.15.28:0/260414779 learned_addr learned my addr 172.21.15.28:0/260414779 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd53e00640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f309a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd4b400640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 -- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 msgr2=0x5636f309a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f309a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 -- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 msgr2=0x5636f2ff1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 -- 172.21.15.28:0/260414779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f2fce290 con 0x5636f2ffc140 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd59000640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02d040 tx=0x7fcd4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282a055e1ce553ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.276+0000 7fcd41600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:38.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c038020 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fcd4c003da0 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4c018b70 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fcd4c042050 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 10 client.6531 fetch_fsmap learned FSMap version 68 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5636f3028200 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c018b70 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fcd4c045020 con 0x5636f2ffc140 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 10 client.6531 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 10 client.6531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 20 client.6531 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f309db40 0x5636f2ff64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 4 client.6531 mds_command: new command op to 6411 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.277+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 --> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] -- command(tid 0: {"prefix": "status"}) -- 0x5636f2f3ce90 con 0x5636f309db40 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.278+0000 7fcd53e00640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f309db40 0x5636f2ff64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.278+0000 7fcd53e00640 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f309db40 0x5636f2ff64f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ed5297c9dfb822 server_cookie=284563e5262dbbcd in_seq=0 out_seq=0 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.278+0000 7fcd41600640 10 client.6531 ms_handle_connect on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd41600640 1 -- 172.21.15.28:0/260414779 <== mds.0 v2:172.21.15.138:6835/4263014325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5636f2f3ce90 con 0x5636f309db40 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd41600640 10 client.6531 handle_command_reply: tid=0 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 client.6531 shutdown 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 msgr2=0x5636f2ff2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c02d040 tx=0x7fcd4c004a40 comp rx=0 tx=0).stop 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 shutdown_connections 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x5636f309db40 0x5636f2ff64f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5636f2ffc140 0x5636f2ff2300 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5636f2fd3480 0x5636f2ff1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd41600640 0 client.6531 ms_handle_reset on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 --2- 172.21.15.28:0/260414779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5636f2fd1cf0 0x5636f309a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 >> 172.21.15.28:0/260414779 conn(0x5636f2fcf4b0 msgr2=0x5636f303ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 shutdown_connections 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 1 -- 172.21.15.28:0/260414779 wait complete. 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.279+0000 7fcd5dea9740 20 client.6531 trim_cache size 0 max 16384 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:49:38.287 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6411, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.1712973849999999, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 68, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 86, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 86, 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 11.245003693999999 2024-09-10T10:49:38.288 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:49:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 msgr2=0x7fcd3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 0x7fcd3006e2f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd44034000 tx=0x7fcd4402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 msgr2=0x7fcd540cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c00c440 tx=0x7fcd3c001e70 comp rx=0 tx=0).stop 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 shutdown_connections 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3006be30 0x7fcd3006e2f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd5415e1f0 0x7fcd54140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd54072de0 0x7fcd540cc110 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 --2- 172.21.15.28:0/484085682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd54072320 0x7fcd54141ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 >> 172.21.15.28:0/484085682 conn(0x7fcd54067d70 msgr2=0x7fcd540c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 shutdown_connections 2024-09-10T10:49:38.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.280+0000 7fcd5b000640 1 -- 172.21.15.28:0/484085682 wait complete. 2024-09-10T10:49:38.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:49:38.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.452+0000 7fd8d8600640 1 Processor -- start 2024-09-10T10:49:38.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.452+0000 7fd8d8600640 1 -- start start 2024-09-10T10:49:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.453+0000 7fd8d8600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd8d0072320 0x7fd8d00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.453+0000 7fd8d8600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d0146140 con 0x7fd8d015e1f0 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.453+0000 7fd8d8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8d006eed0 con 0x7fd8d0072de0 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.453+0000 7fd8d8600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd8d00cd740 con 0x7fd8d0072320 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd8d0072320 0x7fd8d00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd8d0072320 0x7fd8d00cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50768/0 (socket says 172.21.15.28:50768) 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 -- 172.21.15.28:0/31156114 learned_addr learned my addr 172.21.15.28:0/31156114 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d6600640 1 --1- 172.21.15.28:0/31156114 >> v1:172.21.15.28:6789/0 conn(0x7fd8d015e1f0 0x7fd8d006e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38718/0 (socket says 172.21.15.28:38718) 2024-09-10T10:49:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 -- 172.21.15.28:0/31156114 >> v1:172.21.15.138:6789/0 conn(0x7fd8d0072de0 legacy=0x7fd8d006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 -- 172.21.15.28:0/31156114 >> v1:172.21.15.28:6789/0 conn(0x7fd8d015e1f0 legacy=0x7fd8d006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8d5c00640 1 -- 172.21.15.28:0/31156114 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d00cda20 con 0x7fd8d0072320 2024-09-10T10:49:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.454+0000 7fd8cfe00640 1 -- 172.21.15.28:0/31156114 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368932304 0 0) 0x7fd8d0146140 con 0x7fd8d015e1f0 2024-09-10T10:49:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.455+0000 7fd8d5c00640 1 --2- 172.21.15.28:0/31156114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00cce60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfc8120e113a423 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.455+0000 7fd8cfe00640 1 -- 172.21.15.28:0/31156114 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4037070 con 0x7fd8d0072320 2024-09-10T10:49:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.455+0000 7fd8cfe00640 1 -- 172.21.15.28:0/31156114 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4032440 con 0x7fd8d0072320 2024-09-10T10:49:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.455+0000 7fd8cfe00640 1 -- 172.21.15.28:0/31156114 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4032810 con 0x7fd8d0072320 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 -- 172.21.15.28:0/31156114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 msgr2=0x7fd8d00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 --2- 172.21.15.28:0/31156114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00cce60 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c402a210 comp rx=0 tx=0).stop 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 -- 172.21.15.28:0/31156114 shutdown_connections 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 --2- 172.21.15.28:0/31156114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00cce60 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 -- 172.21.15.28:0/31156114 >> 172.21.15.28:0/31156114 conn(0x7fd8d0067d70 msgr2=0x7fd8d006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 -- 172.21.15.28:0/31156114 shutdown_connections 2024-09-10T10:49:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.456+0000 7fd8d8600640 1 -- 172.21.15.28:0/31156114 wait complete. 2024-09-10T10:49:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.458+0000 7fd8d8600640 1 Processor -- start 2024-09-10T10:49:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.458+0000 7fd8d8600640 1 -- start start 2024-09-10T10:49:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 0x7fd8d00b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 0x7fd8d00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d00cd3a0 con 0x7fd8d0072320 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d015c9e0 con 0x7fd8d0072de0 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8d00cd520 con 0x7fd8d015e1f0 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50776/0 (socket says 172.21.15.28:50776) 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 -- 172.21.15.28:0/3990678586 learned_addr learned my addr 172.21.15.28:0/3990678586 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d6600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 0x7fd8d00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 -- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 msgr2=0x7fd8d00b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 0x7fd8d00b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 -- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 msgr2=0x7fd8d00b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5200640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 0x7fd8d00b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 0x7fd8d00b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5c00640 1 -- 172.21.15.28:0/3990678586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d00b54e0 con 0x7fd8d0072320 2024-09-10T10:49:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d6600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 0x7fd8d00b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.459+0000 7fd8d5200640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 0x7fd8d00b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8d5c00640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4037040 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c4004250 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d00b0ac0 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c403c670 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.460+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d00b1080 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.461+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8d0072700 con 0x7fd8d0072320 2024-09-10T10:49:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.462+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8c403a070 con 0x7fd8d0072320 2024-09-10T10:49:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.463+0000 7fd8cd600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 0x7fd8b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.463+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fd8c40ab970 con 0x7fd8d0072320 2024-09-10T10:49:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.464+0000 7fd8d5200640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 0x7fd8b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.464+0000 7fd8d5200640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 0x7fd8b006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd8d006e060 tx=0x7fd8bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:38.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.467+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd8d00b54e0 con 0x7fd8d0072320 2024-09-10T10:49:38.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.687+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd8d01e3c90 con 0x7fd8d0072320 2024-09-10T10:49:38.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.687+0000 7fd8cd600640 1 -- 172.21.15.28:0/3990678586 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7fd8c4041020 con 0x7fd8d0072320 2024-09-10T10:49:38.690 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.689+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 msgr2=0x7fd8b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.689+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 0x7fd8b006e2f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd8d006e060 tx=0x7fd8bc009040 comp rx=0 tx=0).stop 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.689+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 msgr2=0x7fd8d00b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.689+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4009870 tx=0x7fd8c40046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 shutdown_connections 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8b006be30 0x7fd8b006e2f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8d015e1f0 0x7fd8d00b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d0072de0 0x7fd8d00b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 --2- 172.21.15.28:0/3990678586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd8d0072320 0x7fd8d00b1b60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:38.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 >> 172.21.15.28:0/3990678586 conn(0x7fd8d0067d70 msgr2=0x7fd8d01e6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:38.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 shutdown_connections 2024-09-10T10:49:38.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38.690+0000 7fd8d8600640 1 -- 172.21.15.28:0/3990678586 wait complete. 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:49:38.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:49:38.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:38.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:38.709 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:38.745 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42326sec preferred_lft 42326sec 2024-09-10T10:49:38.746 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.747 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:49:38.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:38.749 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:49:38.838 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:49:38.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:38.842 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:49:38.878 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:49:38.878 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:49:38.878 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:49:38.879 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:49:38.879 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:49:38.879 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:49:38.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:49:39.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:39.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:39.077 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:49:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:49:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:49:39.112 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:39.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:39.112 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:49:39.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:49:39.208 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:39.243 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:49:39.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:49:39.243 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:39.318 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:49:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:39.318 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:39.445 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:39.446 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:39.446 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:39.461 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:39.461 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:39.461 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:39.461 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:49:39.461 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:49:39.484 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:49:39.484 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:49:39.484 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:49:39.485 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:49:39.485 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:49:39.485 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:49:39.485 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:49:39.485 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:49:39.487 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:49:39.487 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:49:39.487 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:49:39.487 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:49:39.487 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:49:39.490 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:49:39.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:49:39.490 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:49:39.547 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:49:39.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:49:39.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:39.646 INFO:teuthology.orchestra.run.smithi028.stdout:client6169 2024-09-10T10:49:39.647 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:49:39.647 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:49:39.651 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:49:39.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:49:39.684 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:39.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.878+0000 7fe3dde00640 1 Processor -- start 2024-09-10T10:49:39.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.878+0000 7fe3dde00640 1 -- start start 2024-09-10T10:49:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.879+0000 7fe3dde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3d8072d30 0x7fe3d806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.879+0000 7fe3dde00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3d806f700 0x7fe3d815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.879+0000 7fe3dde00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d806fae0 con 0x7fe3d815b300 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.879+0000 7fe3dde00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d80728a0 con 0x7fe3d806f700 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.879+0000 7fe3dde00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d80cdbe0 con 0x7fe3d8072d30 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe3d815b300 0x7fe3d80cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38722/0 (socket says 172.21.15.28:38722) 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d6a00640 1 -- 172.21.15.28:0/1285323018 learned_addr learned my addr 172.21.15.28:0/1285323018 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d7400640 1 --2- 172.21.15.28:0/1285323018 >> v2:172.21.15.28:3300/0 conn(0x7fe3d806f700 0x7fe3d815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3cfe00640 1 --2- 172.21.15.28:0/1285323018 >> v2:172.21.15.138:3300/0 conn(0x7fe3d8072d30 0x7fe3d806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d6000640 1 -- 172.21.15.28:0/1285323018 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2050718774 0 0) 0x7fe3d806fae0 con 0x7fe3d815b300 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d6000640 1 -- 172.21.15.28:0/1285323018 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3bc003520 con 0x7fe3d815b300 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d7400640 1 -- 172.21.15.28:0/1285323018 >> v2:172.21.15.138:3300/0 conn(0x7fe3d8072d30 msgr2=0x7fe3d806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d7400640 1 --2- 172.21.15.28:0/1285323018 >> v2:172.21.15.138:3300/0 conn(0x7fe3d8072d30 0x7fe3d806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d7400640 1 -- 172.21.15.28:0/1285323018 >> v1:172.21.15.28:6789/0 conn(0x7fe3d815b300 legacy=0x7fe3d80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.880+0000 7fe3d7400640 1 -- 172.21.15.28:0/1285323018 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d80cd900 con 0x7fe3d806f700 2024-09-10T10:49:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.881+0000 7fe3d7400640 1 --2- 172.21.15.28:0/1285323018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d806f700 0x7fe3d815b6e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b57c0 tx=0x7fe3d002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae67b9fd7e558be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3d6000640 1 -- 172.21.15.28:0/1285323018 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0007740 con 0x7fe3d806f700 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3d6000640 1 -- 172.21.15.28:0/1285323018 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d00078e0 con 0x7fe3d806f700 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3d6000640 1 -- 172.21.15.28:0/1285323018 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0007cb0 con 0x7fe3d806f700 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 -- 172.21.15.28:0/1285323018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d806f700 msgr2=0x7fe3d815b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 --2- 172.21.15.28:0/1285323018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d806f700 0x7fe3d815b6e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80b57c0 tx=0x7fe3d002a5f0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 -- 172.21.15.28:0/1285323018 shutdown_connections 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 --2- 172.21.15.28:0/1285323018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d806f700 0x7fe3d815b6e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 --2- 172.21.15.28:0/1285323018 >> v2:172.21.15.138:3300/0 conn(0x7fe3d8072d30 0x7fe3d806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 -- 172.21.15.28:0/1285323018 >> 172.21.15.28:0/1285323018 conn(0x7fe3d8067d70 msgr2=0x7fe3d806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 -- 172.21.15.28:0/1285323018 shutdown_connections 2024-09-10T10:49:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.882+0000 7fe3dde00640 1 -- 172.21.15.28:0/1285323018 wait complete. 2024-09-10T10:49:39.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.884+0000 7fe3dde00640 1 Processor -- start 2024-09-10T10:49:39.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.884+0000 7fe3dde00640 1 -- start start 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d806f700 0x7fe3d8145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8072d30 0x7fe3d8152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d80cd5c0 con 0x7fe3d815b300 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8072530 con 0x7fe3d806f700 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3dde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d80cffe0 con 0x7fe3d8072d30 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3d7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50798/0 (socket says 172.21.15.28:50798) 2024-09-10T10:49:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.885+0000 7fe3d7400640 1 -- 172.21.15.28:0/2479182858 learned_addr learned my addr 172.21.15.28:0/2479182858 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d6a00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d806f700 0x7fe3d8145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3cfe00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8072d30 0x7fe3d8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 -- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8072d30 msgr2=0x7fe3d8152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8072d30 0x7fe3d8152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 -- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d806f700 msgr2=0x7fe3d8145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d806f700 0x7fe3d8145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 -- 172.21.15.28:0/2479182858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d8152da0 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3d7400640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002abe0 tx=0x7fe3d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.886+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d003f070 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.887+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0031bc0 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.887+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d8153070 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.887+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0009450 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.887+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d8153630 con 0x7fe3d815b300 2024-09-10T10:49:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.888+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d80b41e0 con 0x7fe3d815b300 2024-09-10T10:49:39.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.889+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3d003b410 con 0x7fe3d815b300 2024-09-10T10:49:39.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.889+0000 7fe3cf400640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:39.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.890+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fe3d00ac5d0 con 0x7fe3d815b300 2024-09-10T10:49:39.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.890+0000 7fe3d6a00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:39.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.891+0000 7fe3d6a00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 0x7fe3b406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8005880 tx=0x7fe3c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:39.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:39.894+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3d007a700 con 0x7fe3d815b300 2024-09-10T10:49:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.113+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3d806bf90 con 0x7fe3d815b300 2024-09-10T10:49:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.114+0000 7fe3cf400640 1 -- 172.21.15.28:0/2479182858 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5231 (secure 0 0 0) 0x7fe3d007ea00 con 0x7fe3d815b300 2024-09-10T10:49:40.116 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:40.116 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"btime":"2024-09-10T10:49:35:142470+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:34.565062+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 68 2024-09-10T10:49:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 msgr2=0x7fe3b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 0x7fe3b406e2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8005880 tx=0x7fe3c8009040 comp rx=0 tx=0).stop 2024-09-10T10:49:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 msgr2=0x7fe3d80c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe3d002abe0 tx=0x7fe3d0004a40 comp rx=0 tx=0).stop 2024-09-10T10:49:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 shutdown_connections 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3b406be30 0x7fe3b406e2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3d815b300 0x7fe3d80c5440 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3d8072d30 0x7fe3d8152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 --2- 172.21.15.28:0/2479182858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3d806f700 0x7fe3d8145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 >> 172.21.15.28:0/2479182858 conn(0x7fe3d8067d70 msgr2=0x7fe3d8159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 shutdown_connections 2024-09-10T10:49:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.116+0000 7fe3dde00640 1 -- 172.21.15.28:0/2479182858 wait complete. 2024-09-10T10:49:40.134 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 true 2024-09-10T10:49:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.291+0000 7f67cd800640 1 Processor -- start 2024-09-10T10:49:40.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.292+0000 7f67cd800640 1 -- start start 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67cd800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67c80b4800 0x7f67c80aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67cd800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67cd800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c80af7a0 con 0x7f67c815c310 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67cd800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f67c80afa80 con 0x7f67c80b4800 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67cd800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f67c8160130 con 0x7f67c80d0370 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67c80b4800 0x7f67c80aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67c80b4800 0x7f67c80aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50812/0 (socket says 172.21.15.28:50812) 2024-09-10T10:49:40.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 -- 172.21.15.28:0/3046050775 learned_addr learned my addr 172.21.15.28:0/3046050775 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:40.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67c7400640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 -- 172.21.15.28:0/3046050775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 msgr2=0x7f67c80d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 -- 172.21.15.28:0/3046050775 >> v1:172.21.15.138:6789/0 conn(0x7f67c815c310 legacy=0x7f67c8160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.293+0000 7f67be000640 1 -- 172.21.15.28:0/3046050775 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c80af470 con 0x7f67c80b4800 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.294+0000 7f67be000640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c80b4800 0x7f67c80aef30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eccf6518970993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67c6000640 1 -- 172.21.15.28:0/3046050775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0037070 con 0x7f67c80b4800 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67c6000640 1 -- 172.21.15.28:0/3046050775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b0032440 con 0x7f67c80b4800 2024-09-10T10:49:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67c6000640 1 -- 172.21.15.28:0/3046050775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0032810 con 0x7f67c80b4800 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 -- 172.21.15.28:0/3046050775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c80b4800 msgr2=0x7f67c80aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c80b4800 0x7f67c80aef30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009870 tx=0x7f67b002a250 comp rx=0 tx=0).stop 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 -- 172.21.15.28:0/3046050775 shutdown_connections 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 --2- 172.21.15.28:0/3046050775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c80b4800 0x7f67c80aef30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 -- 172.21.15.28:0/3046050775 >> 172.21.15.28:0/3046050775 conn(0x7f67c806bdf0 msgr2=0x7f67c806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 -- 172.21.15.28:0/3046050775 shutdown_connections 2024-09-10T10:49:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.295+0000 7f67cd800640 1 -- 172.21.15.28:0/3046050775 wait complete. 2024-09-10T10:49:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.297+0000 7f67cd800640 1 Processor -- start 2024-09-10T10:49:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.297+0000 7f67cd800640 1 -- start start 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 0x7f67c815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f67c806e090 con 0x7f67c815c310 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67c806df10 con 0x7f67c80b4800 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67cd800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f67c806e210 con 0x7f67c80d0370 2024-09-10T10:49:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67c7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67c7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50826/0 (socket says 172.21.15.28:50826) 2024-09-10T10:49:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67c7400640 1 -- 172.21.15.28:0/1621554206 learned_addr learned my addr 172.21.15.28:0/1621554206 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67be000640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.298+0000 7f67c6a00640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 0x7f67c815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 -- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 msgr2=0x7f67c80cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 -- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 msgr2=0x7f67c815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 0x7f67c815f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 -- 172.21.15.28:0/1621554206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c81f5260 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c6a00640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 0x7f67c815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67c7400640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007e40 tx=0x7f67c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014620 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c0019070 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c81f4f80 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.299+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014a20 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.300+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c815ef00 con 0x7f67c815c310 2024-09-10T10:49:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.301+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c80aef30 con 0x7f67c815c310 2024-09-10T10:49:40.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.302+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f67c0014bc0 con 0x7f67c815c310 2024-09-10T10:49:40.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.302+0000 7f67bf400640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 0x7f67a406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:40.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.303+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f67c008b8b0 con 0x7f67c815c310 2024-09-10T10:49:40.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.303+0000 7f67c6a00640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 0x7f67a406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:40.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.304+0000 7f67c6a00640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 0x7f67a406e320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007fd0 tx=0x7f67b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:40.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.307+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f67c005a030 con 0x7f67c815c310 2024-09-10T10:49:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:40.562+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"} v 0) -- 0x7f67c8075700 con 0x7f67c815c310 2024-09-10T10:49:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.167+0000 7f67bf400640 1 -- 172.21.15.28:0/1621554206 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "true"}]=0 freezing mds.6411 v69) ==== 112+0+0 (secure 0 0 0) 0x7f67c0059930 con 0x7f67c815c310 2024-09-10T10:49:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:freezing mds.6411 2024-09-10T10:49:41.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 msgr2=0x7f67a406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 0x7f67a406e320 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f67b8007fd0 tx=0x7f67b8009040 comp rx=0 tx=0).stop 2024-09-10T10:49:41.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 msgr2=0x7f67c80cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007e40 tx=0x7f67c000e4b0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 shutdown_connections 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f67a406be60 0x7f67a406e320 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67c815c310 0x7f67c80cf150 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c80d0370 0x7f67c80cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 --2- 172.21.15.28:0/1621554206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c80b4800 0x7f67c815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 >> 172.21.15.28:0/1621554206 conn(0x7f67c806bdf0 msgr2=0x7f67c815b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 shutdown_connections 2024-09-10T10:49:41.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.170+0000 7f67cd800640 1 -- 172.21.15.28:0/1621554206 wait complete. 2024-09-10T10:49:41.190 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p a/{b,c}/d') 2024-09-10T10:49:41.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:49:41.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:49:41.304 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-10T10:49:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.462+0000 7ff399200640 1 Processor -- start 2024-09-10T10:49:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.462+0000 7ff399200640 1 -- start start 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff399200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 0x7ff394160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff399200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff394160e70 con 0x7ff3940b4800 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff399200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3940af7a0 con 0x7ff3940d0370 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff399200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940afa80 con 0x7ff39415c310 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff3940b4800 0x7ff3940aef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38728/0 (socket says 172.21.15.28:38728) 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391400640 1 -- 172.21.15.28:0/4180599525 learned_addr learned my addr 172.21.15.28:0/4180599525 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391e00640 1 --2- 172.21.15.28:0/4180599525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 0x7ff394160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380425106 0 0) 0x7ff394160e70 con 0x7ff3940b4800 2024-09-10T10:49:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff378003520 con 0x7ff3940b4800 2024-09-10T10:49:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060575078 0 0) 0x7ff3940af7a0 con 0x7ff3940d0370 2024-09-10T10:49:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff394160e70 con 0x7ff3940d0370 2024-09-10T10:49:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391e00640 1 -- 172.21.15.28:0/4180599525 >> v1:172.21.15.138:6789/0 conn(0x7ff3940d0370 legacy=0x7ff39406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391e00640 1 -- 172.21.15.28:0/4180599525 >> v1:172.21.15.28:6789/0 conn(0x7ff3940b4800 legacy=0x7ff3940aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.464+0000 7ff391e00640 1 -- 172.21.15.28:0/4180599525 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff394160130 con 0x7ff39415c310 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.465+0000 7ff391e00640 1 --2- 172.21.15.28:0/4180599525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 0x7ff394160930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097d0 tx=0x7ff38402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a22c0c96eb2f5a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.465+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384037070 con 0x7ff39415c310 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.465+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384032440 con 0x7ff39415c310 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.465+0000 7ff390a00640 1 -- 172.21.15.28:0/4180599525 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384032810 con 0x7ff39415c310 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 -- 172.21.15.28:0/4180599525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 msgr2=0x7ff394160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 --2- 172.21.15.28:0/4180599525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 0x7ff394160930 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff3840097d0 tx=0x7ff38402a210 comp rx=0 tx=0).stop 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 -- 172.21.15.28:0/4180599525 shutdown_connections 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 --2- 172.21.15.28:0/4180599525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff39415c310 0x7ff394160930 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 -- 172.21.15.28:0/4180599525 >> 172.21.15.28:0/4180599525 conn(0x7ff39406bdf0 msgr2=0x7ff39406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 -- 172.21.15.28:0/4180599525 shutdown_connections 2024-09-10T10:49:41.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.467+0000 7ff399200640 1 -- 172.21.15.28:0/4180599525 wait complete. 2024-09-10T10:49:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 Processor -- start 2024-09-10T10:49:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 -- start start 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3940b4800 0x7ff39414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff39415c310 0x7ff39409c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940d09a0 con 0x7ff3940d0370 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3940d0820 con 0x7ff39415c310 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.468+0000 7ff399200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3940d0b20 con 0x7ff3940b4800 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50834/0 (socket says 172.21.15.28:50834) 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 -- 172.21.15.28:0/72169591 learned_addr learned my addr 172.21.15.28:0/72169591 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391e00640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3940b4800 0x7ff39414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff392800640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff39415c310 0x7ff39409c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 -- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3940b4800 msgr2=0x7ff39414e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3940b4800 0x7ff39414e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 -- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff39415c310 msgr2=0x7ff39409c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff39415c310 0x7ff39409c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 -- 172.21.15.28:0/72169591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941f86c0 con 0x7ff3940d0370 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.469+0000 7ff391400640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c3e0 tx=0x7ff38000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.470+0000 7ff38a000640 1 -- 172.21.15.28:0/72169591 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380017070 con 0x7ff3940d0370 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.470+0000 7ff38a000640 1 -- 172.21.15.28:0/72169591 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff380005ce0 con 0x7ff3940d0370 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.470+0000 7ff38a000640 1 -- 172.21.15.28:0/72169591 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3800072d0 con 0x7ff3940d0370 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.470+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941f7900 con 0x7ff3940d0370 2024-09-10T10:49:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.470+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941f7ec0 con 0x7ff3940d0370 2024-09-10T10:49:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.472+0000 7ff38a000640 1 -- 172.21.15.28:0/72169591 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff38001a070 con 0x7ff3940d0370 2024-09-10T10:49:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.473+0000 7ff38a000640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 0x7ff37406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.473+0000 7ff38a000640 1 -- 172.21.15.28:0/72169591 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff38008a330 con 0x7ff3940d0370 2024-09-10T10:49:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.473+0000 7ff391e00640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 0x7ff37406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.473+0000 7ff39bff9740 1 Processor -- start 2024-09-10T10:49:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.474+0000 7ff39bff9740 1 -- start start 2024-09-10T10:49:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.474+0000 7ff391e00640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 0x7ff37406e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.474+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f806a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff39bff9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555f805be660 con 0x555f80656480 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f80566680 con 0x555f80670420 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555f805ba620 con 0x555f80670800 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50838/0 (socket says 172.21.15.28:50838) 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 -- 172.21.15.28:0/2295042742 learned_addr learned my addr 172.21.15.28:0/2295042742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff391e00640 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f806a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff391400640 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 -- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 msgr2=0x555f806a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 -- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 msgr2=0x555f806a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f806a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.475+0000 7ff392800640 1 -- 172.21.15.28:0/2295042742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8039b7d0 con 0x555f80656480 2024-09-10T10:49:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff392800640 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff38c001c00 tx=0x7ff38c0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6881ab369056167 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff37ea00640 1 -- 172.21.15.28:0/2295042742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c019070 con 0x555f80656480 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff37ea00640 1 -- 172.21.15.28:0/2295042742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c003b80 con 0x555f80656480 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff37ea00640 1 -- 172.21.15.28:0/2295042742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00e7a0 con 0x555f80656480 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 -- 172.21.15.28:0/2295042742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 msgr2=0x555f806a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff38c001c00 tx=0x7ff38c0020c0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 -- 172.21.15.28:0/2295042742 shutdown_connections 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80656480 0x555f806a1740 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2295042742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f806a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.476+0000 7ff39bff9740 1 -- 172.21.15.28:0/2295042742 >> 172.21.15.28:0/2295042742 conn(0x555f80657220 msgr2=0x555f8069f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.477+0000 7ff39bff9740 1 -- 172.21.15.28:0/2295042742 shutdown_connections 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.477+0000 7ff39bff9740 1 -- 172.21.15.28:0/2295042742 wait complete. 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.477+0000 7ff39bff9740 1 -- start start 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.477+0000 7ff39bff9740 1 Processor -- start 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f80659f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555f806553e0 con 0x555f8065a460 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f805474c0 con 0x555f80670420 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555f806556c0 con 0x555f80670800 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.478+0000 7ff39bff9740 10 client.0 _ll_register_callbacks cb 0x555f805b6e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:41.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50840/0 (socket says 172.21.15.28:50840) 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 -- 172.21.15.28:0/2070666653 learned_addr learned my addr 172.21.15.28:0/2070666653 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff391400640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff391e00640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f80659f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 -- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 msgr2=0x555f806b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 -- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 msgr2=0x555f80659f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f80659f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 -- 172.21.15.28:0/2070666653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f802d13f0 con 0x555f8065a460 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff392800640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff38c004f70 tx=0x7ff38c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d4993b77e12c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff373400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c019050 con 0x555f8065a460 2024-09-10T10:49:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.479+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff38c00fcc0 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c003b80 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff38c02d050 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 10 client.6567 fetch_fsmap learned FSMap version 69 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x555f802910f0 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c003b80 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7ff38c030020 con 0x555f8065a460 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 10 client.6567 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 10 client.6567 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 20 client.6567 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f8069c960 0x555f8069cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 4 client.6567 mds_command: new command op to 6411 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:49:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.480+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 --> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f805cbd90 con 0x555f8069c960 2024-09-10T10:49:41.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.481+0000 7ff391e00640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f8069c960 0x555f8069cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.481+0000 7ff391e00640 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f8069c960 0x555f8069cde0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a46bbb2aa80f27a server_cookie=2d941dc4780cda5a in_seq=0 out_seq=0 2024-09-10T10:49:41.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.481+0000 7ff373400640 10 client.6567 ms_handle_connect on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.482+0000 7ff373400640 1 -- 172.21.15.28:0/2070666653 <== mds.0 v2:172.21.15.138:6835/4263014325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f805cbd90 con 0x555f8069c960 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.482+0000 7ff373400640 10 client.6567 handle_command_reply: tid=0 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.482+0000 7ff39bff9740 1 client.6567 shutdown 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 msgr2=0x555f806b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff38c004f70 tx=0x7ff38c001250 comp rx=0 tx=0).stop 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 shutdown_connections 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f8069c960 0x555f8069cde0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f8065a460 0x555f806b1520 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670800 0x555f806b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 --2- 172.21.15.28:0/2070666653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670420 0x555f80659f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff373400640 0 client.6567 ms_handle_reset on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 >> 172.21.15.28:0/2070666653 conn(0x555f80656480 msgr2=0x555f80653f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 shutdown_connections 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 1 -- 172.21.15.28:0/2070666653 wait complete. 2024-09-10T10:49:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.483+0000 7ff39bff9740 20 client.6567 trim_cache size 0 max 16384 2024-09-10T10:49:41.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.498+0000 7ff39bff9740 1 Processor -- start 2024-09-10T10:49:41.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.498+0000 7ff39bff9740 1 -- start start 2024-09-10T10:49:41.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f8065a460 0x555f80694f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 0x555f8069a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff394200fc0 con 0x555f80670800 2024-09-10T10:49:41.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3941fffe0 con 0x555f8065a460 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.499+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff39409ad10 con 0x555f80670420 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50856/0 (socket says 172.21.15.28:50856) 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 -- 172.21.15.28:0/401402330 learned_addr learned my addr 172.21.15.28:0/401402330 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff391e00640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f8065a460 0x555f80694f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff391400640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 0x555f8069a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 -- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 msgr2=0x555f8069a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 0x555f8069a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 -- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f8065a460 msgr2=0x555f80694f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f8065a460 0x555f80694f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 -- 172.21.15.28:0/401402330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f80651050 con 0x555f80670800 2024-09-10T10:49:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff391400640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 0x555f8069a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:41.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff392800640 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0016b0 tx=0x7ff38c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595c57e415ad7cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff37e000640 1 -- 172.21.15.28:0/401402330 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c019040 con 0x555f80670800 2024-09-10T10:49:41.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff37e000640 1 -- 172.21.15.28:0/401402330 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c025070 con 0x555f80670800 2024-09-10T10:49:41.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.500+0000 7ff37e000640 1 -- 172.21.15.28:0/401402330 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00fcc0 con 0x555f80670800 2024-09-10T10:49:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 -- 172.21.15.28:0/401402330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 msgr2=0x555f80653f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0016b0 tx=0x7ff38c01c000 comp rx=0 tx=0).stop 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 -- 172.21.15.28:0/401402330 shutdown_connections 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670800 0x555f80653f60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f80670420 0x555f8069a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 --2- 172.21.15.28:0/401402330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f8065a460 0x555f80694f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 -- 172.21.15.28:0/401402330 >> 172.21.15.28:0/401402330 conn(0x555f80656480 msgr2=0x555f806b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 -- 172.21.15.28:0/401402330 shutdown_connections 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.501+0000 7ff39bff9740 1 -- 172.21.15.28:0/401402330 wait complete. 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.502+0000 7ff39bff9740 1 -- start start 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.502+0000 7ff39bff9740 1 Processor -- start 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f8065a460 0x555f80706620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670800 0x555f806b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff39409b360 con 0x555f80670420 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555f80685aa0 con 0x555f80670800 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555f8071eae0 con 0x555f8065a460 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff39bff9740 10 client.0 _ll_register_callbacks cb 0x7ff38c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50864/0 (socket says 172.21.15.28:50864) 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 -- 172.21.15.28:0/191210128 learned_addr learned my addr 172.21.15.28:0/191210128 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391e00640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f8065a460 0x555f80706620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff392800640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670800 0x555f806b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 -- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f8065a460 msgr2=0x555f80706620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f8065a460 0x555f80706620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 -- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670800 msgr2=0x555f806b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670800 0x555f806b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.503+0000 7ff391400640 1 -- 172.21.15.28:0/191210128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f806a1200 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff391400640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff38000cfd0 tx=0x7ff380001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42383d59ec0674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380086030 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7ff380070430 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c025070 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff3800568b0 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 10 client.6573 fetch_fsmap learned FSMap version 69 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x555f8069cdf0 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c025070 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7ff380007470 con 0x555f80670420 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 10 client.6573 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T10:49:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 10 client.6573 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 20 client.6573 populate_metadata read hostname 'smithi028' 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f80706dc0 0x555f8071d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 4 client.6573 mds_command: new command op to 6411 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.504+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 --> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f805c60e0 con 0x555f80706dc0 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.505+0000 7ff391e00640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f80706dc0 0x555f8071d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.505+0000 7ff391e00640 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f80706dc0 0x555f8071d8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca9c612a97166bc server_cookie=79530a70905ceb4 in_seq=0 out_seq=0 2024-09-10T10:49:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.505+0000 7ff373400640 10 client.6573 ms_handle_connect on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:41.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff373400640 1 -- 172.21.15.28:0/191210128 <== mds.0 v2:172.21.15.138:6835/4263014325 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f805c60e0 con 0x555f80706dc0 2024-09-10T10:49:41.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff373400640 10 client.6573 handle_command_reply: tid=0 2024-09-10T10:49:41.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 client.6573 shutdown 2024-09-10T10:49:41.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 msgr2=0x555f806b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff38000cfd0 tx=0x7ff380001150 comp rx=0 tx=0).stop 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 shutdown_connections 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:6835/4263014325,v1:172.21.15.138:6837/4263014325] conn(0x555f80706dc0 0x555f8071d8b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555f80670800 0x555f806b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555f80670420 0x555f806b2280 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 --2- 172.21.15.28:0/191210128 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555f8065a460 0x555f80706620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff373400640 0 client.6573 ms_handle_reset on v2:172.21.15.138:6835/4263014325 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 >> 172.21.15.28:0/191210128 conn(0x555f80656480 msgr2=0x555f80695270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 shutdown_connections 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 1 -- 172.21.15.28:0/191210128 wait complete. 2024-09-10T10:49:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.518+0000 7ff39bff9740 20 client.6573 trim_cache size 0 max 16384 2024-09-10T10:49:41.522 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:49:41.522 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:49:41.522 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:49:41.522 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 msgr2=0x7ff37406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 0x7ff37406e2f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff384034000 tx=0x7ff38402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 msgr2=0x7ff39409bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff38000c3e0 tx=0x7ff38000c8a0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 shutdown_connections 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff37406be30 0x7ff37406e2f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff39415c310 0x7ff39409c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3940d0370 0x7ff39409bcf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 --2- 172.21.15.28:0/72169591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3940b4800 0x7ff39414e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 >> 172.21.15.28:0/72169591 conn(0x7ff39406bdf0 msgr2=0x7ff39406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 shutdown_connections 2024-09-10T10:49:41.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.519+0000 7ff399200640 1 -- 172.21.15.28:0/72169591 wait complete. 2024-09-10T10:49:41.538 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_inject_rename_corrupt_dentry_first 1.0 2024-09-10T10:49:41.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.697+0000 7f753ae00640 1 Processor -- start 2024-09-10T10:49:41.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.697+0000 7f753ae00640 1 -- start start 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f753ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7534155460 0x7f7534072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f753ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 0x7f753406e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f753ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7534162ed0 con 0x7f75340727b0 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f753ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7534072c10 con 0x7f7534155460 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f753ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f753415f430 con 0x7f7534162870 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f7533e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7534155460 0x7f7534072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f7533e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7534155460 0x7f7534072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50872/0 (socket says 172.21.15.28:50872) 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f7533e00640 1 -- 172.21.15.28:0/1119311805 learned_addr learned my addr 172.21.15.28:0/1119311805 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.698+0000 7f752b400640 1 --1- 172.21.15.28:0/1119311805 >> v1:172.21.15.28:6789/0 conn(0x7f75340727b0 0x7f7534162160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38734/0 (socket says 172.21.15.28:38734) 2024-09-10T10:49:41.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7538e00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 0x7f753406e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7533e00640 1 -- 172.21.15.28:0/1119311805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 msgr2=0x7f753406e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7533e00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 0x7f753406e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7533e00640 1 -- 172.21.15.28:0/1119311805 >> v1:172.21.15.28:6789/0 conn(0x7f75340727b0 legacy=0x7f7534162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7533e00640 1 -- 172.21.15.28:0/1119311805 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753415ede0 con 0x7f7534155460 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7538e00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 0x7f753406e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:41.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.699+0000 7f7533400640 1 -- 172.21.15.28:0/1119311805 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782656351 0 0) 0x7f7534162ed0 con 0x7f75340727b0 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.700+0000 7f7533e00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534155460 0x7f7534072270 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7524009870 tx=0x7f752402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a745f2e3a5855908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.700+0000 7f7533400640 1 -- 172.21.15.28:0/1119311805 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524037070 con 0x7f7534155460 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.700+0000 7f7533400640 1 -- 172.21.15.28:0/1119311805 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7524032440 con 0x7f7534155460 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.700+0000 7f7533400640 1 -- 172.21.15.28:0/1119311805 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7524032810 con 0x7f7534155460 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 -- 172.21.15.28:0/1119311805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534155460 msgr2=0x7f7534072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534155460 0x7f7534072270 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7524009870 tx=0x7f752402a250 comp rx=0 tx=0).stop 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 -- 172.21.15.28:0/1119311805 shutdown_connections 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534162870 0x7f753406e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 --2- 172.21.15.28:0/1119311805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534155460 0x7f7534072270 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 -- 172.21.15.28:0/1119311805 >> 172.21.15.28:0/1119311805 conn(0x7f753406bc10 msgr2=0x7f753406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 -- 172.21.15.28:0/1119311805 shutdown_connections 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.701+0000 7f753ae00640 1 -- 172.21.15.28:0/1119311805 wait complete. 2024-09-10T10:49:41.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.702+0000 7f753ae00640 1 Processor -- start 2024-09-10T10:49:41.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 -- start start 2024-09-10T10:49:41.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 0x7f753414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534155460 0x7f75341f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7534155840 con 0x7f7534162870 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f753415ea90 con 0x7f75340727b0 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.703+0000 7f753ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75341559c0 con 0x7f7534155460 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50876/0 (socket says 172.21.15.28:50876) 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 -- 172.21.15.28:0/3919171167 learned_addr learned my addr 172.21.15.28:0/3919171167 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f752b400640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534155460 0x7f75341f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7533e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 0x7f753414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 -- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534155460 msgr2=0x7f75341f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534155460 0x7f75341f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 -- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 msgr2=0x7f753414a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 0x7f753414a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7538e00640 1 -- 172.21.15.28:0/3919171167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753414dc70 con 0x7f7534162870 2024-09-10T10:49:41.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.704+0000 7f7533e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 0x7f753414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:41.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f7538e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f752c00cd50 tx=0x7f752c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c00f680 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f752c00f840 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c00fc30 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753414df50 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.705+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753414e510 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.706+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753406cba0 con 0x7f7534162870 2024-09-10T10:49:41.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.707+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f752c00e3e0 con 0x7f7534162870 2024-09-10T10:49:41.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.709+0000 7f7530c00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 0x7f751006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:41.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.709+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f752c08be20 con 0x7f7534162870 2024-09-10T10:49:41.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.709+0000 7f7533e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 0x7f751006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:41.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.709+0000 7f7533e00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 0x7f751006e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7524034000 tx=0x7f752402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:41.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.713+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f752c014030 con 0x7f7534162870 2024-09-10T10:49:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.894+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}] v 0) -- 0x7f753406e6f0 con 0x7f7534162870 2024-09-10T10:49:41.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.898+0000 7f7530c00640 1 -- 172.21.15.28:0/3919171167 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_inject_rename_corrupt_dentry_first}]=0 v23)=0 v23) ==== 138+0+0 (secure 0 0 0) 0x7f752c05a630 con 0x7f7534162870 2024-09-10T10:49:41.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 msgr2=0x7f751006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 0x7f751006e2f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7524034000 tx=0x7f752402e000 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 msgr2=0x7f75341f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f752c00cd50 tx=0x7f752c00a650 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 shutdown_connections 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f751006be30 0x7f751006e2f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7534162870 0x7f75341f7e30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7534155460 0x7f75341f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 --2- 172.21.15.28:0/3919171167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75340727b0 0x7f753414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 >> 172.21.15.28:0/3919171167 conn(0x7f753406bc10 msgr2=0x7f753406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 shutdown_connections 2024-09-10T10:49:41.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:41.900+0000 7f753ae00640 1 -- 172.21.15.28:0/3919171167 wait complete. 2024-09-10T10:49:46.919 DEBUG:teuthology.orchestra.run.smithi028:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T10:49:46.924 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'timeout 60 mv a/b a/z') 2024-09-10T10:49:46.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-09-10T10:49:47.024 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:49:47.022+0000 7fe015000640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.024 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:49:47.022+0000 7fe015000640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.024 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fe015000640 time 2024-09-10T10:49:47.023812+0000 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fe01c183417] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.025 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr:*** Caught signal (Aborted) ** 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7fe015000640 thread_name:ms_dispatch 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:49:47.023+0000 7fe015000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fe015000640 time 2024-09-10T10:49:47.023812+0000 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.026 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fe01c183417] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.027 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe01b83e6f0] 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fe01b88b99c] 2024-09-10T10:49:47.028 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fe01c1834d4] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.029 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr:2024-09-10T10:49:47.025+0000 7fe015000640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7fe015000640 thread_name:ms_dispatch 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe01b83e6f0] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fe01b88b99c] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fe01c1834d4] 2024-09-10T10:49:47.030 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.031 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.032 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.032 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:49:47.032 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.037 INFO:tasks.ceph.mds.b.smithi138.stderr: -4762> 2024-09-10T10:49:37.698+0000 7fe011e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:49:47.045 INFO:tasks.ceph.mds.b.smithi138.stderr: -8> 2024-09-10T10:49:47.022+0000 7fe015000640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.045 INFO:tasks.ceph.mds.b.smithi138.stderr: -6> 2024-09-10T10:49:47.022+0000 7fe015000640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.045 INFO:tasks.ceph.mds.b.smithi138.stderr: -1> 2024-09-10T10:49:47.023+0000 7fe015000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fe015000640 time 2024-09-10T10:49:47.023812+0000 2024-09-10T10:49:47.045 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fe01c183417] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.046 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 0> 2024-09-10T10:49:47.025+0000 7fe015000640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7fe015000640 thread_name:ms_dispatch 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe01b83e6f0] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fe01b88b99c] 2024-09-10T10:49:47.047 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fe01c1834d4] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.048 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.049 INFO:tasks.ceph.mds.b.smithi138.stderr: -4765> 2024-09-10T10:49:37.698+0000 7fe011e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: -11> 2024-09-10T10:49:47.022+0000 7fe015000640 -1 mds.0.cache.den(0x10000000000 z) newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: -9> 2024-09-10T10:49:47.022+0000 7fe015000640 -1 log_channel(cluster) log [ERR] : MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: -4> 2024-09-10T10:49:47.023+0000 7fe015000640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: In function 'void MDSRank::abort(std::string_view)' thread 7fe015000640 time 2024-09-10T10:49:47.023812+0000 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.1.1-305-g1cf819b7/rpm/el9/BUILD/ceph-19.1.1-305-g1cf819b7/src/mds/MDSRank.cc: 952: ceph_abort_msg("abort() called") 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd3) [0x7fe01c183417] 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.053 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.054 INFO:tasks.ceph.mds.b.smithi138.stderr: -3> 2024-09-10T10:49:47.025+0000 7fe015000640 -1 *** Caught signal (Aborted) ** 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: in thread 7fe015000640 thread_name:ms_dispatch 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 1: /lib64/libc.so.6(+0x3e6f0) [0x7fe01b83e6f0] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 2: /lib64/libc.so.6(+0x8b99c) [0x7fe01b88b99c] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 3: raise() 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 4: abort() 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 5: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x190) [0x7fe01c1834d4] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 6: (MDSRank::abort(std::basic_string_view >)+0x61) [0x5564408fe2d1] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 7: (CDentry::check_corruption(bool)+0x4d4) [0x556440b294e4] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 8: ceph-mds(+0x1cb3e9) [0x5564409413e9] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 9: (Server::_rename_prepare(boost::intrusive_ptr const&, EMetaBlob*, ceph::buffer::v15_2_0::list*, CDentry*, CDentry*, std::basic_string_view >, CDentry*)+0x207f) [0x5564409cc2df] 2024-09-10T10:49:47.055 INFO:tasks.ceph.mds.b.smithi138.stderr: 10: (Server::handle_client_rename(boost::intrusive_ptr const&)+0x25e2) [0x5564409cf5b2] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 11: (Server::handle_client_request(boost::intrusive_ptr const&)+0x3e3) [0x556440995dc3] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 12: (Server::dispatch(boost::intrusive_ptr const&)+0x13c) [0x556440983cec] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 13: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55644090589b] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 14: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556440905d6c] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 15: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1a6) [0x5564408ea286] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 16: (DispatchQueue::entry()+0x4c2) [0x7fe01c3904d2] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 17: /usr/lib64/ceph/libceph-common.so.2(+0x42a0b1) [0x7fe01c42a0b1] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 18: /lib64/libc.so.6(+0x89c52) [0x7fe01b889c52] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 19: /lib64/libc.so.6(+0x10ec80) [0x7fe01b90ec80] 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-09-10T10:49:47.056 INFO:tasks.ceph.mds.b.smithi138.stderr: 2024-09-10T10:49:47.107 INFO:teuthology.orchestra.run.smithi028.stdout:{"mds_beacon_grace":"15.000000"} 2024-09-10T10:49:47.107 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-09-10T10:49:47.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:47.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.118+0000 7f9171c00640 1 Processor -- start 2024-09-10T10:49:47.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.118+0000 7f9171c00640 1 -- start start 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.119+0000 7f9171c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c165e00 0x7f916c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.119+0000 7f9171c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f916c0cc6e0 0x7f916c0c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f9171c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f916c0c3600 0x7f916c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f9171c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f916c0735c0 con 0x7f916c0c3600 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f9171c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f916c0738a0 con 0x7f916c0cc6e0 2024-09-10T10:49:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f9171c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f916c073b80 con 0x7f916c165e00 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f916c0c3600 0x7f916c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f916c0c3600 0x7f916c0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44164/0 (socket says 172.21.15.28:44164) 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 -- 172.21.15.28:0/1006416222 learned_addr learned my addr 172.21.15.28:0/1006416222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f9163e00640 1 --2- 172.21.15.28:0/1006416222 >> v2:172.21.15.138:3300/0 conn(0x7f916c0cc6e0 0x7f916c0c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916aa00640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c165e00 0x7f916c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 -- 172.21.15.28:0/1006416222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c165e00 msgr2=0x7f916c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c165e00 0x7f916c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 -- 172.21.15.28:0/1006416222 >> v2:172.21.15.138:3300/0 conn(0x7f916c0cc6e0 msgr2=0x7f916c0c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 --2- 172.21.15.28:0/1006416222 >> v2:172.21.15.138:3300/0 conn(0x7f916c0cc6e0 0x7f916c0c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.120+0000 7f916b400640 1 -- 172.21.15.28:0/1006416222 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c0cbdb0 con 0x7f916c0c3600 2024-09-10T10:49:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.122+0000 7f916b400640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c0c3600 0x7f916c0ca740 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f916c0cd0d0 tx=0x7f916402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b5956699f1a345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.122+0000 7f916a000640 1 -- 172.21.15.28:0/1006416222 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164036070 con 0x7f916c0c3600 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.122+0000 7f916a000640 1 -- 172.21.15.28:0/1006416222 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9164031600 con 0x7f916c0c3600 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.122+0000 7f916a000640 1 -- 172.21.15.28:0/1006416222 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91640319d0 con 0x7f916c0c3600 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 -- 172.21.15.28:0/1006416222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c0c3600 msgr2=0x7f916c0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c0c3600 0x7f916c0ca740 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f916c0cd0d0 tx=0x7f916402a6a0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 -- 172.21.15.28:0/1006416222 shutdown_connections 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c0c3600 0x7f916c0ca740 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 --2- 172.21.15.28:0/1006416222 >> v2:172.21.15.138:3300/0 conn(0x7f916c0cc6e0 0x7f916c0c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 --2- 172.21.15.28:0/1006416222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c165e00 0x7f916c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 -- 172.21.15.28:0/1006416222 >> 172.21.15.28:0/1006416222 conn(0x7f916c0b1e20 msgr2=0x7f916c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 -- 172.21.15.28:0/1006416222 shutdown_connections 2024-09-10T10:49:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.123+0000 7f9171c00640 1 -- 172.21.15.28:0/1006416222 wait complete. 2024-09-10T10:49:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.125+0000 7f9171c00640 1 Processor -- start 2024-09-10T10:49:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.125+0000 7f9171c00640 1 -- start start 2024-09-10T10:49:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f916c0c3600 0x7f916c0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c0cc6e0 0x7f916c0b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c165e00 0x7f916c09aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f916c073360 con 0x7f916c165e00 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f916c0731e0 con 0x7f916c0c3600 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9171c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f916c1661e0 con 0x7f916c0cc6e0 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f916aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f916c0c3600 0x7f916c0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f916aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f916c0c3600 0x7f916c0b7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:60352/0 (socket says 172.21.15.28:60352) 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f916aa00640 1 -- 172.21.15.28:0/232236572 learned_addr learned my addr 172.21.15.28:0/232236572 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f916b400640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c165e00 0x7f916c09aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.126+0000 7f9163e00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c0cc6e0 0x7f916c0b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 -- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c0cc6e0 msgr2=0x7f916c0b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f916c0cc6e0 0x7f916c0b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 -- 172.21.15.28:0/232236572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c165e00 msgr2=0x7f916c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c165e00 0x7f916c09aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 -- 172.21.15.28:0/232236572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c09cc90 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916b400640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f916c165e00 0x7f916c09aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.127+0000 7f916aa00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f916c0c3600 0x7f916c0b7d50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e870 tx=0x7f915c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c017070 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c00c940 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c00cd30 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9171c00640 1 -- 172.21.15.28:0/232236572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c09cfd0 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9171c00640 1 -- 172.21.15.28:0/232236572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c0b6780 con 0x7f916c0c3600 2024-09-10T10:49:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.128+0000 7f9171c00640 1 -- 172.21.15.28:0/232236572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f916c0c30c0 con 0x7f916c0c3600 2024-09-10T10:49:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.130+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f915c01a070 con 0x7f916c0c3600 2024-09-10T10:49:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.131+0000 7f9162a00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914806be80 0x7f914806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.131+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f915c08aeb0 con 0x7f916c0c3600 2024-09-10T10:49:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.131+0000 7f9163e00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914806be80 0x7f914806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.131+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+511 (secure 0 0 0) 0x7f916c09cfd0 con 0x7f916c0c3600 2024-09-10T10:49:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.132+0000 7f9171c00640 1 -- 172.21.15.28:0/232236572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f916c06ccd0 con 0x7f916c0c3600 2024-09-10T10:49:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.132+0000 7f9163e00640 1 --2- 172.21.15.28:0/232236572 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f914806be80 0x7f914806e340 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f916c0c4f10 tx=0x7f9154009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.227 INFO:tasks.ceph.mds.b.smithi138.stderr:daemon-helper: command crashed with signal 6 2024-09-10T10:49:47.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.266+0000 7f00b5000640 1 Processor -- start 2024-09-10T10:49:47.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.266+0000 7f00b5000640 1 -- start start 2024-09-10T10:49:47.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00b5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 0x7f00b0169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00b5000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b00d2c20 con 0x7f00b00d4180 2024-09-10T10:49:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00b5000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00b006e390 con 0x7f00b015e6a0 2024-09-10T10:49:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00b5000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00b00cefb0 con 0x7f00b00d0450 2024-09-10T10:49:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00af400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f00b00d4180 0x7f00b00d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53384/0 (socket says 172.21.15.28:53384) 2024-09-10T10:49:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.267+0000 7f00af400640 1 -- 172.21.15.28:0/2967881503 learned_addr learned my addr 172.21.15.28:0/2967881503 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00afe00640 1 --2- 172.21.15.28:0/2967881503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 0x7f00b0169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488058069 0 0) 0x7f00b00d2c20 con 0x7f00b00d4180 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0094003520 con 0x7f00b00d4180 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4191784543 0 0) 0x7f00b006e390 con 0x7f00b015e6a0 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00b00d2c20 con 0x7f00b015e6a0 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00afe00640 1 -- 172.21.15.28:0/2967881503 >> v1:172.21.15.138:6789/0 conn(0x7f00b015e6a0 legacy=0x7f00b006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00afe00640 1 -- 172.21.15.28:0/2967881503 >> v1:172.21.15.28:6789/0 conn(0x7f00b00d4180 legacy=0x7f00b00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.268+0000 7f00afe00640 1 -- 172.21.15.28:0/2967881503 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b006e670 con 0x7f00b00d0450 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.269+0000 7f00afe00640 1 --2- 172.21.15.28:0/2967881503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 0x7f00b0169000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009dc0 tx=0x7f00a802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c30c0671892140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.269+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a802ae60 con 0x7f00b00d0450 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.269+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a8007740 con 0x7f00b00d0450 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.269+0000 7f00aea00640 1 -- 172.21.15.28:0/2967881503 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8007b10 con 0x7f00b00d0450 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 -- 172.21.15.28:0/2967881503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 msgr2=0x7f00b0169000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 --2- 172.21.15.28:0/2967881503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 0x7f00b0169000 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009dc0 tx=0x7f00a802a640 comp rx=0 tx=0).stop 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 -- 172.21.15.28:0/2967881503 shutdown_connections 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 --2- 172.21.15.28:0/2967881503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d0450 0x7f00b0169000 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 -- 172.21.15.28:0/2967881503 >> 172.21.15.28:0/2967881503 conn(0x7f00b009d6f0 msgr2=0x7f00b008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 -- 172.21.15.28:0/2967881503 shutdown_connections 2024-09-10T10:49:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.270+0000 7f00b5000640 1 -- 172.21.15.28:0/2967881503 wait complete. 2024-09-10T10:49:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.272+0000 7f00b5000640 1 Processor -- start 2024-09-10T10:49:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.272+0000 7f00b5000640 1 -- start start 2024-09-10T10:49:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00b00d0450 0x7f00b00b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d4180 0x7f00b00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00b006df30 con 0x7f00b015e6a0 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00b00cebd0 con 0x7f00b00d0450 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00b5000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00b006e0b0 con 0x7f00b00d4180 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44176/0 (socket says 172.21.15.28:44176) 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00afe00640 1 -- 172.21.15.28:0/2622741796 learned_addr learned my addr 172.21.15.28:0/2622741796 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00af400640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00b00d0450 0x7f00b00b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.273+0000 7f00a7e00640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d4180 0x7f00b00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 -- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d4180 msgr2=0x7f00b00c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d4180 0x7f00b00c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 -- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00b00d0450 msgr2=0x7f00b00b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00b00d0450 0x7f00b00b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 -- 172.21.15.28:0/2622741796 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b00b88b0 con 0x7f00b015e6a0 2024-09-10T10:49:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00afe00640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009ee0 tx=0x7f00a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a803f070 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a80319c0 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8031b90 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.274+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b00b8b90 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.275+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b00b9150 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.275+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b00d4560 con 0x7f00b015e6a0 2024-09-10T10:49:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.276+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00a803b410 con 0x7f00b015e6a0 2024-09-10T10:49:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.277+0000 7f00a7400640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 0x7f008c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.277+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f00a80ac490 con 0x7f00b015e6a0 2024-09-10T10:49:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.278+0000 7f00af400640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 0x7f008c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.278+0000 7f00af400640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 0x7f008c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000f60 tx=0x7f00a0007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:47.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.282+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00a807a650 con 0x7f00b015e6a0 2024-09-10T10:49:47.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.503+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00b00b6aa0 con 0x7f00b015e6a0 2024-09-10T10:49:47.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.503+0000 7f00a7400640 1 -- 172.21.15.28:0/2622741796 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5231 (secure 0 0 0) 0x7f00a807e950 con 0x7f00b015e6a0 2024-09-10T10:49:47.505 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.505 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":69,"btime":"2024-09-10T10:49:41:163354+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:40.564032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:47.505 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 69 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 msgr2=0x7f008c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 0x7f008c06e2f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f00a0000f60 tx=0x7f00a0007d70 comp rx=0 tx=0).stop 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 msgr2=0x7f00b00c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009ee0 tx=0x7f00a8004a40 comp rx=0 tx=0).stop 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 shutdown_connections 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f008c06be30 0x7f008c06e2f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b015e6a0 0x7f00b00c8020 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00b00d4180 0x7f00b00c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 --2- 172.21.15.28:0/2622741796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00b00d0450 0x7f00b00b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:47.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.505+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 >> 172.21.15.28:0/2622741796 conn(0x7f00b009d6f0 msgr2=0x7f00b00b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:47.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.506+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 shutdown_connections 2024-09-10T10:49:47.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.506+0000 7f00b5000640 1 -- 172.21.15.28:0/2622741796 wait complete. 2024-09-10T10:49:47.522 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:49:47.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:47.634+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 7 ==== log(1 entries from seq 1 at 2024-09-10T10:49:47.023751+0000) ==== 468+0+0 (secure 0 0 0) 0x7f915c057190 con 0x7f916c0c3600 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: cluster: 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: id: f658987a-dd14-4573-bcd3-ee2a383c9fe0 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: health: HEALTH_OK 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: services: 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: mgr: x(active, since 6m), standbys: y 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T10:49:47.636 INFO:teuthology.orchestra.run.smithi028.stdout: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: data: 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: volumes: 1/1 healthy 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: pools: 3 pools, 97 pgs 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: objects: 29 objects, 596 KiB 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: usage: 253 MiB used, 720 GiB / 720 GiB avail 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: pgs: 97 active+clean 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: io: 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: client: 0 B/s rd, 3.8 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.637 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:47.638 INFO:teuthology.orchestra.run.smithi028.stdout:2024-09-10T10:49:47.023751+0000 mds.b [ERR] MDS abort because newly corrupt dentry to be committed: [dentry #0x1/a/z [fffffffffffffff6,head] auth NULL (dn xlock x=1 by request(client.6169:47 nref=7 cr=0x556462d61c00)) (dversion lock w=1 last_client=6169) pv=11 v=10 ap=2 ino=(nil) state=1073741824 | request=1 lock=2 authpin=1 0x556462e8a780] 2024-09-10T10:49:52.523 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:52.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.689+0000 7f54c9a00640 1 Processor -- start 2024-09-10T10:49:52.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.690+0000 7f54c9a00640 1 -- start start 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c9a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54c415ba30 0x7f54c406f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4073050 0x7f54c40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c9a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54c40b3570 con 0x7f54c406fd50 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c9a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f54c40b4180 con 0x7f54c415ba30 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54c40b3850 con 0x7f54c4073050 2024-09-10T10:49:52.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c2000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54c406fd50 0x7f54c4072940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53394/0 (socket says 172.21.15.28:53394) 2024-09-10T10:49:52.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c2000640 1 -- 172.21.15.28:0/896216020 learned_addr learned my addr 172.21.15.28:0/896216020 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:49:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c3400640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4073050 0x7f54c40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.691+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> v2:172.21.15.138:3300/0 conn(0x7f54c415ba30 0x7f54c406f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189310079 0 0) 0x7f54c40b3570 con 0x7f54c406fd50 2024-09-10T10:49:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a8003520 con 0x7f54c406fd50 2024-09-10T10:49:52.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2679288646 0 0) 0x7f54a8003520 con 0x7f54c406fd50 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4073050 msgr2=0x7f54c40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4073050 0x7f54c40b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 >> v2:172.21.15.138:3300/0 conn(0x7f54c415ba30 msgr2=0x7f54c406f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 --2- 172.21.15.28:0/896216020 >> v2:172.21.15.138:3300/0 conn(0x7f54c415ba30 0x7f54c406f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c40b3b30 con 0x7f54c406fd50 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f54b0001670 con 0x7f54c406fd50 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 >> v1:172.21.15.28:6789/0 conn(0x7f54c406fd50 legacy=0x7f54c4072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54a8004800 0x7f54a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54a80051c0 0x7f54a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54c40b4180 con 0x7f54a80039b0 2024-09-10T10:49:52.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54c40b3850 con 0x7f54a8004800 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54a800bf00 con 0x7f54a80051c0 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c3400640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4073050 0x7f54c40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.692+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> v2:172.21.15.138:3300/0 conn(0x7f54c415ba30 0x7f54c406f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2000640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54a8004800 0x7f54a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c3400640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54a80051c0 0x7f54a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 -- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54a80051c0 msgr2=0x7f54a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54a80051c0 0x7f54a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 -- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54a8004800 msgr2=0x7f54a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54a8004800 0x7f54a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.693+0000 7f54c2a00640 1 -- 172.21.15.28:0/896216020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a8003520 con 0x7f54a80039b0 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.694+0000 7f54c2a00640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54a8003dd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009ca0 tx=0x7f54a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a17e660e5f1120 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.694+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4009840 con 0x7f54a80039b0 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.694+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4004020 con 0x7f54a80039b0 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.694+0000 7f54c1600640 1 -- 172.21.15.28:0/896216020 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40043d0 con 0x7f54a80039b0 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 -- 172.21.15.28:0/896216020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 msgr2=0x7f54a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 --2- 172.21.15.28:0/896216020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54a8003dd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009ca0 tx=0x7f54a8005830 comp rx=0 tx=0).stop 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c3400640 1 -- 172.21.15.28:0/896216020 reap_dead start 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 -- 172.21.15.28:0/896216020 shutdown_connections 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 -- 172.21.15.28:0/896216020 >> 172.21.15.28:0/896216020 conn(0x7f54c4067d70 msgr2=0x7f54c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 -- 172.21.15.28:0/896216020 shutdown_connections 2024-09-10T10:49:52.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.695+0000 7f54c9a00640 1 -- 172.21.15.28:0/896216020 wait complete. 2024-09-10T10:49:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.697+0000 7f54c9a00640 1 Processor -- start 2024-09-10T10:49:52.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.697+0000 7f54c9a00640 1 -- start start 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 0x7f54c4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4151d70 0x7f54c4144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54c40ccfe0 con 0x7f54a80039b0 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54c40ce810 con 0x7f54c40cc8e0 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c9a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54c40cd160 con 0x7f54c4151d70 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44194/0 (socket says 172.21.15.28:44194) 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2a00640 1 -- 172.21.15.28:0/2131156591 learned_addr learned my addr 172.21.15.28:0/2131156591 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c3400640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4151d70 0x7f54c4144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2000640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 0x7f54c4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2a00640 1 -- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4151d70 msgr2=0x7f54c4144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.698+0000 7f54c2a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4151d70 0x7f54c4144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54c2a00640 1 -- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 msgr2=0x7f54c4151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54c2a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 0x7f54c4151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54c2a00640 1 -- 172.21.15.28:0/2131156591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c414bed0 con 0x7f54a80039b0 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54c2000640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 0x7f54c4151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:49:52.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54c2a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009dc0 tx=0x7f54b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4041020 con 0x7f54a80039b0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4005da0 con 0x7f54a80039b0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.699+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003810 con 0x7f54a80039b0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.700+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c414c1b0 con 0x7f54a80039b0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.700+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c415d2e0 con 0x7f54a80039b0 2024-09-10T10:49:52.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.701+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c406f810 con 0x7f54a80039b0 2024-09-10T10:49:52.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.702+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54b4044020 con 0x7f54a80039b0 2024-09-10T10:49:52.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.702+0000 7f54baa00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:52.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.702+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f54b40ab890 con 0x7f54a80039b0 2024-09-10T10:49:52.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.703+0000 7f54c2000640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:52.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.704+0000 7f54c2000640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 0x7f54a406e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f54c415cb00 tx=0x7f54b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:52.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.707+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54b404b030 con 0x7f54a80039b0 2024-09-10T10:49:52.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.926+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c4152150 con 0x7f54a80039b0 2024-09-10T10:49:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.927+0000 7f54baa00640 1 -- 172.21.15.28:0/2131156591 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5231 (secure 0 0 0) 0x7f54b407a010 con 0x7f54a80039b0 2024-09-10T10:49:52.928 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:52.928 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":69,"btime":"2024-09-10T10:49:41:163354+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:40.564032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:52.928 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 69 2024-09-10T10:49:52.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 msgr2=0x7f54a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 0x7f54a406e2f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f54c415cb00 tx=0x7f54b0009040 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 msgr2=0x7f54c40ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009dc0 tx=0x7f54b4004730 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 shutdown_connections 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54a406be30 0x7f54a406e2f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54c4151d70 0x7f54c4144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54c40cc8e0 0x7f54c4151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 --2- 172.21.15.28:0/2131156591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54a80039b0 0x7f54c40ad8a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 >> 172.21.15.28:0/2131156591 conn(0x7f54c4067d70 msgr2=0x7f54c415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.929+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 shutdown_connections 2024-09-10T10:49:52.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:52.930+0000 7f54c9a00640 1 -- 172.21.15.28:0/2131156591 wait complete. 2024-09-10T10:49:52.947 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:49:53.803 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-10T10:49:57.947 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:49:58.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.114+0000 7f9f5fa00640 1 Processor -- start 2024-09-10T10:49:58.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.115+0000 7f9f5fa00640 1 -- start start 2024-09-10T10:49:58.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5fa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f580cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5fa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9f58072de0 0x7f9f5806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f58146140 con 0x7f9f58072700 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5fa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9f580cd740 con 0x7f9f58072de0 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5fa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f580cda20 con 0x7f9f58072320 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9f58072de0 0x7f9f5806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9f58072de0 0x7f9f5806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57966/0 (socket says 172.21.15.28:57966) 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 -- 172.21.15.28:0/2369072101 learned_addr learned my addr 172.21.15.28:0/2369072101 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5d000640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f580cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 -- 172.21.15.28:0/2369072101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 msgr2=0x7f9f580cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f580cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 -- 172.21.15.28:0/2369072101 >> v1:172.21.15.138:6789/0 conn(0x7f9f58072700 legacy=0x7f9f5806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f4fe00640 1 -- 172.21.15.28:0/2369072101 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f580cdd00 con 0x7f9f58072de0 2024-09-10T10:49:58.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.116+0000 7f9f5d000640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f580cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:49:58.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.117+0000 7f9f4fe00640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072de0 0x7f9f5806e0b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea061a6445488f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f57e00640 1 -- 172.21.15.28:0/2369072101 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40037070 con 0x7f9f58072de0 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f57e00640 1 -- 172.21.15.28:0/2369072101 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f40032440 con 0x7f9f58072de0 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f57e00640 1 -- 172.21.15.28:0/2369072101 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40032810 con 0x7f9f58072de0 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 -- 172.21.15.28:0/2369072101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072de0 msgr2=0x7f9f5806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072de0 0x7f9f5806e0b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4002a180 comp rx=0 tx=0).stop 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 -- 172.21.15.28:0/2369072101 shutdown_connections 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072de0 0x7f9f5806e0b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/2369072101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f580cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 -- 172.21.15.28:0/2369072101 >> 172.21.15.28:0/2369072101 conn(0x7f9f58067d70 msgr2=0x7f9f5806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.118+0000 7f9f5fa00640 1 -- 172.21.15.28:0/2369072101 shutdown_connections 2024-09-10T10:49:58.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.119+0000 7f9f5fa00640 1 -- 172.21.15.28:0/2369072101 wait complete. 2024-09-10T10:49:58.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.120+0000 7f9f5fa00640 1 Processor -- start 2024-09-10T10:49:58.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.120+0000 7f9f5fa00640 1 -- start start 2024-09-10T10:49:58.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f5813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f58072de0 0x7f9f5813cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9f580cd520 con 0x7f9f58072700 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f580cd3a0 con 0x7f9f58072de0 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f5fa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5806ed00 con 0x7f9f58072320 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.121+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57980/0 (socket says 172.21.15.28:57980) 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 -- 172.21.15.28:0/4107177021 learned_addr learned my addr 172.21.15.28:0/4107177021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f5d000640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f5813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f5da00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f58072de0 0x7f9f5813cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 -- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 msgr2=0x7f9f5813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f5813d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 -- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f58072de0 msgr2=0x7f9f5813cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f58072de0 0x7f9f5813cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 -- 172.21.15.28:0/4107177021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f58143700 con 0x7f9f58072700 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f4fe00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9f400090b0 tx=0x7f9f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.122+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40041020 con 0x7f9f58072700 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.123+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f40030990 con 0x7f9f58072700 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.123+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5813db60 con 0x7f9f58072700 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.123+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f40037040 con 0x7f9f58072700 2024-09-10T10:49:58.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.123+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5813e400 con 0x7f9f58072700 2024-09-10T10:49:58.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.124+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5813df50 con 0x7f9f58072700 2024-09-10T10:49:58.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.125+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f40044020 con 0x7f9f58072700 2024-09-10T10:49:58.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.126+0000 7f9f55600640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 0x7f9f3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:49:58.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.126+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f9f400aba90 con 0x7f9f58072700 2024-09-10T10:49:58.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.126+0000 7f9f5d000640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 0x7f9f3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:49:58.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.127+0000 7f9f5d000640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 0x7f9f3406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800a990 tx=0x7f9f48008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:49:58.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.130+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f4004b030 con 0x7f9f58072700 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.350+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f580ce600 con 0x7f9f58072700 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.350+0000 7f9f55600640 1 -- 172.21.15.28:0/4107177021 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5231 (secure 0 0 0) 0x7f9f400ab020 con 0x7f9f58072700 2024-09-10T10:49:58.351 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":69,"btime":"2024-09-10T10:49:41:163354+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:49:40.564032+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:49:58.352 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 69 2024-09-10T10:49:58.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 msgr2=0x7f9f3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 0x7f9f3406e2f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800a990 tx=0x7f9f48008040 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 msgr2=0x7f9f581431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9f400090b0 tx=0x7f9f400046c0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 shutdown_connections 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f3406be30 0x7f9f3406e2f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f58072de0 0x7f9f5813cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f58072700 0x7f9f581431c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 --2- 172.21.15.28:0/4107177021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f58072320 0x7f9f5813d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 >> 172.21.15.28:0/4107177021 conn(0x7f9f58067d70 msgr2=0x7f9f5806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.352+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 shutdown_connections 2024-09-10T10:49:58.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:49:58.353+0000 7f9f5fa00640 1 -- 172.21.15.28:0/4107177021 wait complete. 2024-09-10T10:49:58.369 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-09-10T10:50:00.416 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2024-09-10T10:50:00.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:00.666+0000 7f9162a00640 1 -- 172.21.15.28:0/232236572 <== mon.1 v2:172.21.15.138:3300/0 8 ==== log(1 entries from seq 859 at 2024-09-10T10:50:00.000126+0000) ==== 187+0+0 (secure 0 0 0) 0x7f915c0573f0 con 0x7f916c0c3600 2024-09-10T10:50:00.668 INFO:teuthology.orchestra.run.smithi028.stdout:2024-09-10T10:50:00.000126+0000 mon.a [INF] overall HEALTH_OK 2024-09-10T10:50:03.370 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:03.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.540+0000 7fd582000640 1 Processor -- start 2024-09-10T10:50:03.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.540+0000 7fd582000640 1 -- start start 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd582000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd57c0d4180 0x7fd57c0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd582000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 0x7fd57c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd582000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57c0d4aa0 con 0x7fd57c06dd60 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd582000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd57c0cefb0 con 0x7fd57c0d4180 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd582000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c0d2c20 con 0x7fd57c15e6a0 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd580c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd57c0d4180 0x7fd57c0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd580c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd57c0d4180 0x7fd57c0d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57986/0 (socket says 172.21.15.28:57986) 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.541+0000 7fd580c00640 1 -- 172.21.15.28:0/1384444623 learned_addr learned my addr 172.21.15.28:0/1384444623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd57be00640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 0x7fd57c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd580c00640 1 -- 172.21.15.28:0/1384444623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 msgr2=0x7fd57c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:03.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd580c00640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 0x7fd57c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd580c00640 1 -- 172.21.15.28:0/1384444623 >> v1:172.21.15.138:6789/0 conn(0x7fd57c06dd60 legacy=0x7fd57c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:03.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd580c00640 1 -- 172.21.15.28:0/1384444623 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c06e390 con 0x7fd57c0d4180 2024-09-10T10:50:03.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.542+0000 7fd57be00640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 0x7fd57c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.543+0000 7fd580c00640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0d4180 0x7fd57c0d4560 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd570009870 tx=0x7fd57002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2d9d529ab4cae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.543+0000 7fd57b400640 1 -- 172.21.15.28:0/1384444623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570037070 con 0x7fd57c0d4180 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.543+0000 7fd57b400640 1 -- 172.21.15.28:0/1384444623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570032440 con 0x7fd57c0d4180 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.543+0000 7fd57b400640 1 -- 172.21.15.28:0/1384444623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570032810 con 0x7fd57c0d4180 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 -- 172.21.15.28:0/1384444623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0d4180 msgr2=0x7fd57c0d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:03.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0d4180 0x7fd57c0d4560 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd570009870 tx=0x7fd57002a180 comp rx=0 tx=0).stop 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 -- 172.21.15.28:0/1384444623 shutdown_connections 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c15e6a0 0x7fd57c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 --2- 172.21.15.28:0/1384444623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c0d4180 0x7fd57c0d4560 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 -- 172.21.15.28:0/1384444623 >> 172.21.15.28:0/1384444623 conn(0x7fd57c09d6f0 msgr2=0x7fd57c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 -- 172.21.15.28:0/1384444623 shutdown_connections 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.544+0000 7fd582000640 1 -- 172.21.15.28:0/1384444623 wait complete. 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.546+0000 7fd582000640 1 Processor -- start 2024-09-10T10:50:03.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.546+0000 7fd582000640 1 -- start start 2024-09-10T10:50:03.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c06dd60 0x7fd57c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 0x7fd57c1f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c0cebd0 con 0x7fd57c15e6a0 2024-09-10T10:50:03.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c06e950 con 0x7fd57c0d4180 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd582000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd57c0d0450 con 0x7fd57c06dd60 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:57990/0 (socket says 172.21.15.28:57990) 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd580c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c06dd60 0x7fd57c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 -- 172.21.15.28:0/4009444507 learned_addr learned my addr 172.21.15.28:0/4009444507 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd57be00640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 0x7fd57c1f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 -- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c06dd60 msgr2=0x7fd57c0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c06dd60 0x7fd57c0cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 -- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 msgr2=0x7fd57c1f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 0x7fd57c1f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd581600640 1 -- 172.21.15.28:0/4009444507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c0c8730 con 0x7fd57c15e6a0 2024-09-10T10:50:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.547+0000 7fd57be00640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 0x7fd57c1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:03.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd581600640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c780 tx=0x7fd57400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:03.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd574004b60 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd574004d00 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57400aa90 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c0c8a10 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.548+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c0a01a0 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.549+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c0d4830 con 0x7fd57c15e6a0 2024-09-10T10:50:03.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.550+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd57400ac30 con 0x7fd57c15e6a0 2024-09-10T10:50:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.551+0000 7fd578c00640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 0x7fd55c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.551+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7fd57408b2e0 con 0x7fd57c15e6a0 2024-09-10T10:50:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.551+0000 7fd580c00640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 0x7fd55c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.552+0000 7fd580c00640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 0x7fd55c06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd570034000 tx=0x7fd57002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.556+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd57c0d4830 con 0x7fd57c15e6a0 2024-09-10T10:50:03.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.779+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd57c0d4560 con 0x7fd57c15e6a0 2024-09-10T10:50:03.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.779+0000 7fd578c00640 1 -- 172.21.15.28:0/4009444507 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5279 (secure 0 0 0) 0x7fd574020070 con 0x7fd57c15e6a0 2024-09-10T10:50:03.780 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:03.781 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":70,"btime":"2024-09-10T10:50:01:654533+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6408,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:01.654483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6411},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6411":{"gid":6411,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.138:6837/4263014325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":4263014325},{"type":"v1","addr":"172.21.15.138:6837","nonce":4263014325}]},"join_fscid":-1,"laggy_since":"2024-09-10T10:50:01.654483+0000","export_targets":[],"features":4540701547738038271,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6411,"qdb_cluster":[6411]},"id":5}]} 2024-09-10T10:50:03.781 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 70 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 msgr2=0x7fd55c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 0x7fd55c06e2f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd570034000 tx=0x7fd57002e000 comp rx=0 tx=0).stop 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 msgr2=0x7fd57c09e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c780 tx=0x7fd57400cc40 comp rx=0 tx=0).stop 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 shutdown_connections 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd55c06be30 0x7fd55c06e2f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd57c15e6a0 0x7fd57c09e160 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd57c0d4180 0x7fd57c1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 --2- 172.21.15.28:0/4009444507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd57c06dd60 0x7fd57c0cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 >> 172.21.15.28:0/4009444507 conn(0x7fd57c09d6f0 msgr2=0x7fd57c0b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 shutdown_connections 2024-09-10T10:50:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:03.782+0000 7fd582000640 1 -- 172.21.15.28:0/4009444507 wait complete. 2024-09-10T10:50:03.802 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2024-09-10T10:50:03.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T10:50:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.064+0000 7f1daae00640 1 Processor -- start 2024-09-10T10:50:04.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.064+0000 7f1daae00640 1 -- start start 2024-09-10T10:50:04.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.065+0000 7f1daae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 0x7f1da40aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.065+0000 7f1daae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da40d0370 0x7f1da40d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.065+0000 7f1daae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da40af7a0 con 0x7f1da415c310 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.065+0000 7f1daae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1da40afa80 con 0x7f1da40d0370 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.065+0000 7f1daae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1da4160130 con 0x7f1da40b4800 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da40d0370 0x7f1da40d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da40d0370 0x7f1da40d0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47742/0 (socket says 172.21.15.28:47742) 2024-09-10T10:50:04.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 -- 172.21.15.28:0/2720888879 learned_addr learned my addr 172.21.15.28:0/2720888879 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da3400640 1 --2- 172.21.15.28:0/2720888879 >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 0x7f1da40aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 -- 172.21.15.28:0/2720888879 >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 msgr2=0x7f1da40aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 --2- 172.21.15.28:0/2720888879 >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 0x7f1da40aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 -- 172.21.15.28:0/2720888879 >> v1:172.21.15.138:6789/0 conn(0x7f1da415c310 legacy=0x7f1da4160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da8e00640 1 -- 172.21.15.28:0/2720888879 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da40af470 con 0x7f1da40d0370 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.066+0000 7f1da3400640 1 --2- 172.21.15.28:0/2720888879 >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 0x7f1da40aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:04.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.067+0000 7f1da8e00640 1 --2- 172.21.15.28:0/2720888879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da40d0800 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1da40b4e90 tx=0x7f1d9c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe1f80d7147017 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.067+0000 7f1da2a00640 1 -- 172.21.15.28:0/2720888879 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c037070 con 0x7f1da40d0370 2024-09-10T10:50:04.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.067+0000 7f1da2a00640 1 -- 172.21.15.28:0/2720888879 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9c032440 con 0x7f1da40d0370 2024-09-10T10:50:04.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.067+0000 7f1da2a00640 1 -- 172.21.15.28:0/2720888879 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c032810 con 0x7f1da40d0370 2024-09-10T10:50:04.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 -- 172.21.15.28:0/2720888879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 msgr2=0x7f1da40d0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 --2- 172.21.15.28:0/2720888879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da40d0800 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1da40b4e90 tx=0x7f1d9c02a230 comp rx=0 tx=0).stop 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 -- 172.21.15.28:0/2720888879 shutdown_connections 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 --2- 172.21.15.28:0/2720888879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da40d0800 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 --2- 172.21.15.28:0/2720888879 >> v2:172.21.15.138:3300/0 conn(0x7f1da40b4800 0x7f1da40aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 -- 172.21.15.28:0/2720888879 >> 172.21.15.28:0/2720888879 conn(0x7f1da406bdf0 msgr2=0x7f1da406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 -- 172.21.15.28:0/2720888879 shutdown_connections 2024-09-10T10:50:04.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.068+0000 7f1daae00640 1 -- 172.21.15.28:0/2720888879 wait complete. 2024-09-10T10:50:04.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.070+0000 7f1daae00640 1 Processor -- start 2024-09-10T10:50:04.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.070+0000 7f1daae00640 1 -- start start 2024-09-10T10:50:04.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1da40b4800 0x7f1da415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da415c310 0x7f1da409bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1da406e090 con 0x7f1da40d0370 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1da406df10 con 0x7f1da415c310 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1daae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1da406e210 con 0x7f1da40b4800 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47744/0 (socket says 172.21.15.28:47744) 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 -- 172.21.15.28:0/2839197851 learned_addr learned my addr 172.21.15.28:0/2839197851 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3e00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1da40b4800 0x7f1da415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da8e00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da415c310 0x7f1da409bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 -- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1da40b4800 msgr2=0x7f1da415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1da40b4800 0x7f1da415a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 -- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da415c310 msgr2=0x7f1da409bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da415c310 0x7f1da409bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.071+0000 7f1da3400640 1 -- 172.21.15.28:0/2839197851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4159230 con 0x7f1da40d0370 2024-09-10T10:50:04.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1da3400640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c7f0 tx=0x7f1d9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90015070 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d90007a30 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da4159510 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90007e00 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.072+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da4159ad0 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.073+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da40aef30 con 0x7f1da40d0370 2024-09-10T10:50:04.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.074+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d90018070 con 0x7f1da40d0370 2024-09-10T10:50:04.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.075+0000 7f1d9be00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 0x7f1d8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.075+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f1d9008afd0 con 0x7f1da40d0370 2024-09-10T10:50:04.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.076+0000 7f1da3e00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 0x7f1d8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.076+0000 7f1da3e00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 0x7f1d8406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b450 tx=0x7f1d9400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.080+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d90059100 con 0x7f1da40d0370 2024-09-10T10:50:04.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.269+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f1da40cfea0 con 0x7f1da40d0370 2024-09-10T10:50:04.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.272+0000 7f1d9be00640 1 -- 172.21.15.28:0/2839197851 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f1d9005d400 con 0x7f1da40d0370 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 msgr2=0x7f1d8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 0x7f1d8406e2f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400b450 tx=0x7f1d9400d040 comp rx=0 tx=0).stop 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 msgr2=0x7f1da414c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000c7f0 tx=0x7f1d9000cbf0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 shutdown_connections 2024-09-10T10:50:04.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8406be30 0x7f1d8406e2f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1da415c310 0x7f1da409bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da40d0370 0x7f1da414c6f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 --2- 172.21.15.28:0/2839197851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1da40b4800 0x7f1da415a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 >> 172.21.15.28:0/2839197851 conn(0x7f1da406bdf0 msgr2=0x7f1da4157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 shutdown_connections 2024-09-10T10:50:04.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.275+0000 7f1daae00640 1 -- 172.21.15.28:0/2839197851 wait complete. 2024-09-10T10:50:04.292 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 5:0 false 2024-09-10T10:50:04.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.450+0000 7f9599600640 1 Processor -- start 2024-09-10T10:50:04.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.450+0000 7f9599600640 1 -- start start 2024-09-10T10:50:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9599600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 0x7f95940aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9599600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95940af7a0 con 0x7f95940d0370 2024-09-10T10:50:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9599600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95940afa80 con 0x7f959415c310 2024-09-10T10:50:04.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9599600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9594160130 con 0x7f95940b4800 2024-09-10T10:50:04.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9593400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f95940d0370 0x7f959406df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59274/0 (socket says 172.21.15.28:59274) 2024-09-10T10:50:04.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.451+0000 7f9593400640 1 -- 172.21.15.28:0/782875552 learned_addr learned my addr 172.21.15.28:0/782875552 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:04.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f958be00640 1 --2- 172.21.15.28:0/782875552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 0x7f95940aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 426915927 0 0) 0x7f95940af7a0 con 0x7f95940d0370 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9578003520 con 0x7f95940d0370 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981412677 0 0) 0x7f95940afa80 con 0x7f959415c310 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95940af7a0 con 0x7f959415c310 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2363083523 0 0) 0x7f9578003520 con 0x7f95940d0370 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f958be00640 1 -- 172.21.15.28:0/782875552 >> v1:172.21.15.138:6789/0 conn(0x7f959415c310 legacy=0x7f9594160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f958be00640 1 -- 172.21.15.28:0/782875552 >> v1:172.21.15.28:6789/0 conn(0x7f95940d0370 legacy=0x7f959406df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.452+0000 7f958be00640 1 -- 172.21.15.28:0/782875552 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95940af470 con 0x7f95940b4800 2024-09-10T10:50:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.453+0000 7f958be00640 1 --2- 172.21.15.28:0/782875552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 0x7f95940aef30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f957c009800 tx=0x7f957c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1ff1deae01786d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c037070 con 0x7f95940b4800 2024-09-10T10:50:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c02ac50 con 0x7f95940b4800 2024-09-10T10:50:04.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9592000640 1 -- 172.21.15.28:0/782875552 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0019f0 con 0x7f95940b4800 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 -- 172.21.15.28:0/782875552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 msgr2=0x7f95940aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 --2- 172.21.15.28:0/782875552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 0x7f95940aef30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f957c009800 tx=0x7f957c02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 -- 172.21.15.28:0/782875552 shutdown_connections 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 --2- 172.21.15.28:0/782875552 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940b4800 0x7f95940aef30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 -- 172.21.15.28:0/782875552 >> 172.21.15.28:0/782875552 conn(0x7f959406bdf0 msgr2=0x7f959406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 -- 172.21.15.28:0/782875552 shutdown_connections 2024-09-10T10:50:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.454+0000 7f9599600640 1 -- 172.21.15.28:0/782875552 wait complete. 2024-09-10T10:50:04.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.456+0000 7f9599600640 1 Processor -- start 2024-09-10T10:50:04.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.456+0000 7f9599600640 1 -- start start 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 0x7f9594159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d0370 0x7f959409ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f95940d09a0 con 0x7f959415c310 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95940d0820 con 0x7f95940b4800 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9599600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95940d0b20 con 0x7f95940d0370 2024-09-10T10:50:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9593400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9593400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47758/0 (socket says 172.21.15.28:47758) 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9593400640 1 -- 172.21.15.28:0/4095523175 learned_addr learned my addr 172.21.15.28:0/4095523175 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f958be00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d0370 0x7f959409ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9592a00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 0x7f9594159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.457+0000 7f9593400640 1 -- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d0370 msgr2=0x7f959409ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9593400640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d0370 0x7f959409ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9593400640 1 -- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 msgr2=0x7f9594159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9593400640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 0x7f9594159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9593400640 1 -- 172.21.15.28:0/4095523175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959409d8d0 con 0x7f959415c310 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9592a00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 0x7f9594159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9593400640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f958c001ba0 tx=0x7f958c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c00ad20 con 0x7f959415c310 2024-09-10T10:50:04.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f958c012400 con 0x7f959415c310 2024-09-10T10:50:04.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c012830 con 0x7f959415c310 2024-09-10T10:50:04.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959409dbb0 con 0x7f959415c310 2024-09-10T10:50:04.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.458+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959415a1a0 con 0x7f959415c310 2024-09-10T10:50:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.459+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95940ae2a0 con 0x7f959415c310 2024-09-10T10:50:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.460+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f958c0129d0 con 0x7f959415c310 2024-09-10T10:50:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.461+0000 7f958aa00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.461+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6101+0+0 (secure 0 0 0) 0x7f958c08b890 con 0x7f959415c310 2024-09-10T10:50:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.461+0000 7f9592a00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.462+0000 7f9592a00640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 0x7f957006e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f95840011e0 tx=0x7f9584009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:04.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.465+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95940ae2a0 con 0x7f959415c310 2024-09-10T10:50:04.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:04.721+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"} v 0) -- 0x7f959406ce90 con 0x7f959415c310 2024-09-10T10:50:05.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.674+0000 7f958aa00640 1 -- 172.21.15.28:0/4095523175 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "5:0", "val": "false"}]=0 unfreezing mds.6411 v71) ==== 115+0+0 (secure 0 0 0) 0x7f958c0182b0 con 0x7f959415c310 2024-09-10T10:50:05.675 INFO:teuthology.orchestra.run.smithi028.stderr:unfreezing mds.6411 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 msgr2=0x7f957006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 0x7f957006e2f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f95840011e0 tx=0x7f9584009040 comp rx=0 tx=0).stop 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 msgr2=0x7f959415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f958c001ba0 tx=0x7f958c00a430 comp rx=0 tx=0).stop 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 shutdown_connections 2024-09-10T10:50:05.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959415c310 0x7f959415e910 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940d0370 0x7f959409ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 --2- 172.21.15.28:0/4095523175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95940b4800 0x7f9594159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 >> 172.21.15.28:0/4095523175 conn(0x7f959406bdf0 msgr2=0x7f95940731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 shutdown_connections 2024-09-10T10:50:05.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:05.677+0000 7f9599600640 1 -- 172.21.15.28:0/4095523175 wait complete. 2024-09-10T10:50:05.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -n kernel.core_pattern 2024-09-10T10:50:05.758 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T10:50:05.761 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:50:05.761 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2024-09-10T10:50:05.831 INFO:teuthology.orchestra.run.smithi138.stdout:1725965387.43066.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2024-09-10T10:50:05.835 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1725965387.43066.core'] 2024-09-10T10:50:05.835 INFO:tasks.cephfs.cephfs_test_case:Found core file 1725965387.43066.core, deleting it 2024-09-10T10:50:05.835 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1725965387.43066.core 2024-09-10T10:50:05.903 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:50:05.904 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:50:05.904 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:50:05.904 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:50:05.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:50:05.904 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi138 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2024-09-10T10:50:05.905 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:50:05.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:50:05.909 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:50:05.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:06.034 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:50:06.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.073+0000 7f37f4800640 1 Processor -- start 2024-09-10T10:50:06.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.073+0000 7f37f4800640 1 -- start start 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f4800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 0x7f37ec06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f4800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37ec06f700 0x7f37ec15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f4800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ec0cdbe0 con 0x7f37ec15b300 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f4800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f37ec06fb50 con 0x7f37ec06f700 2024-09-10T10:50:06.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f4800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec0728a0 con 0x7f37ec072d30 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f1e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f37ec15b300 0x7f37ec0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59286/0 (socket says 172.21.15.28:59286) 2024-09-10T10:50:06.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f1e00640 1 -- 172.21.15.28:0/1350713064 learned_addr learned my addr 172.21.15.28:0/1350713064 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f2800640 1 --2- 172.21.15.28:0/1350713064 >> v2:172.21.15.138:3300/0 conn(0x7f37ec06f700 0x7f37ec15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f1400640 1 --2- 172.21.15.28:0/1350713064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 0x7f37ec06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095374661 0 0) 0x7f37ec0cdbe0 con 0x7f37ec15b300 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.075+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37d4005730 con 0x7f37ec15b300 2024-09-10T10:50:06.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.076+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 77622364 0 0) 0x7f37d4005730 con 0x7f37ec15b300 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.076+0000 7f37f1400640 1 -- 172.21.15.28:0/1350713064 >> v2:172.21.15.138:3300/0 conn(0x7f37ec06f700 msgr2=0x7f37ec15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.076+0000 7f37f1400640 1 --2- 172.21.15.28:0/1350713064 >> v2:172.21.15.138:3300/0 conn(0x7f37ec06f700 0x7f37ec15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.076+0000 7f37f1400640 1 -- 172.21.15.28:0/1350713064 >> v1:172.21.15.28:6789/0 conn(0x7f37ec15b300 legacy=0x7f37ec0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.076+0000 7f37f1400640 1 -- 172.21.15.28:0/1350713064 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec0cd900 con 0x7f37ec072d30 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.077+0000 7f37f1400640 1 --2- 172.21.15.28:0/1350713064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 0x7f37ec06f1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097a0 tx=0x7f37dc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aae629573140a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.077+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc037070 con 0x7f37ec072d30 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.077+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37dc02aba0 con 0x7f37ec072d30 2024-09-10T10:50:06.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.077+0000 7f37f0a00640 1 -- 172.21.15.28:0/1350713064 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0001670 con 0x7f37ec072d30 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 -- 172.21.15.28:0/1350713064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 msgr2=0x7f37ec06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 --2- 172.21.15.28:0/1350713064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 0x7f37ec06f1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0097a0 tx=0x7f37dc02a110 comp rx=0 tx=0).stop 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 -- 172.21.15.28:0/1350713064 shutdown_connections 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 --2- 172.21.15.28:0/1350713064 >> v2:172.21.15.138:3300/0 conn(0x7f37ec06f700 0x7f37ec15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 --2- 172.21.15.28:0/1350713064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec072d30 0x7f37ec06f1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 -- 172.21.15.28:0/1350713064 >> 172.21.15.28:0/1350713064 conn(0x7f37ec067d70 msgr2=0x7f37ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 -- 172.21.15.28:0/1350713064 shutdown_connections 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.078+0000 7f37f4800640 1 -- 172.21.15.28:0/1350713064 wait complete. 2024-09-10T10:50:06.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.079+0000 7f37f4800640 1 Processor -- start 2024-09-10T10:50:06.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 -- start start 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37ec072d30 0x7f37ec158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec15b300 0x7f37ec158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec0cd5c0 con 0x7f37ec06f700 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec072530 con 0x7f37ec072d30 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.080+0000 7f37f4800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37ec0cffe0 con 0x7f37ec15b300 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47766/0 (socket says 172.21.15.28:47766) 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 -- 172.21.15.28:0/822762418 learned_addr learned my addr 172.21.15.28:0/822762418 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1400640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37ec072d30 0x7f37ec158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f2800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec15b300 0x7f37ec158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 -- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec15b300 msgr2=0x7f37ec158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec15b300 0x7f37ec158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 -- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37ec072d30 msgr2=0x7f37ec158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37ec072d30 0x7f37ec158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.081+0000 7f37f1e00640 1 -- 172.21.15.28:0/822762418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1f44b0 con 0x7f37ec06f700 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37f1e00640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c700 tx=0x7f37e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0017070 con 0x7f37ec06f700 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0005ad0 con 0x7f37ec06f700 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1f4790 con 0x7f37ec06f700 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0007070 con 0x7f37ec06f700 2024-09-10T10:50:06.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.082+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec14e1d0 con 0x7f37ec06f700 2024-09-10T10:50:06.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.083+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ec15dc50 con 0x7f37ec06f700 2024-09-10T10:50:06.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.084+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f37e001a070 con 0x7f37ec06f700 2024-09-10T10:50:06.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.085+0000 7f37e6000640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 0x7f37d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:06.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.085+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f37e008a4f0 con 0x7f37ec06f700 2024-09-10T10:50:06.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.085+0000 7f37f1400640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 0x7f37d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:06.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.086+0000 7f37f1400640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 0x7f37d006e320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37dc034000 tx=0x7f37dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:06.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.089+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37e0058550 con 0x7f37ec06f700 2024-09-10T10:50:06.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.314+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ec0b02a0 con 0x7f37ec06f700 2024-09-10T10:50:06.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.314+0000 7f37e6000640 1 -- 172.21.15.28:0/822762418 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+4397 (secure 0 0 0) 0x7f37e005c850 con 0x7f37ec06f700 2024-09-10T10:50:06.316 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:06.316 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":72,"btime":"2024-09-10T10:50:05:677436+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:05.677432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T10:50:06.316 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 72 2024-09-10T10:50:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 msgr2=0x7f37d006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 0x7f37d006e320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f37dc034000 tx=0x7f37dc02e000 comp rx=0 tx=0).stop 2024-09-10T10:50:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 msgr2=0x7f37ec14dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c700 tx=0x7f37e000cbc0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 shutdown_connections 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37d006be60 0x7f37d006e320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f37ec15b300 0x7f37ec158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37ec072d30 0x7f37ec158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 --2- 172.21.15.28:0/822762418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37ec06f700 0x7f37ec14dc90 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 >> 172.21.15.28:0/822762418 conn(0x7f37ec067d70 msgr2=0x7f37ec0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 shutdown_connections 2024-09-10T10:50:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:06.317+0000 7f37f4800640 1 -- 172.21.15.28:0/822762418 wait complete. 2024-09-10T10:50:06.736 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:49:31.540549+0000', 'modified': '2024-09-10T10:50:05.677432+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6408': {'gid': 6408, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.138:6836/1950306159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1950306159}, {'type': 'v1', 'addr': '172.21.15.138:6836', 'nonce': 1950306159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:50:06.737 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6408:up:replay 2024-09-10T10:50:07.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-10T10:50:07.737 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:07.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.904+0000 7fbfe6000640 1 Processor -- start 2024-09-10T10:50:07.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.904+0000 7fbfe6000640 1 -- start start 2024-09-10T10:50:07.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbfe00d4180 0x7fbfe00d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbfe006dd60 0x7fbfe00d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe00cefb0 con 0x7fbfe006dd60 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe00d4aa0 con 0x7fbfe00d4180 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe00d2c20 con 0x7fbfe015e6a0 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbfe006dd60 0x7fbfe00d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbfe006dd60 0x7fbfe00d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47774/0 (socket says 172.21.15.28:47774) 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.905+0000 7fbfe5600640 1 -- 172.21.15.28:0/4043520212 learned_addr learned my addr 172.21.15.28:0/4043520212 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe4c00640 1 --2- 172.21.15.28:0/4043520212 >> v2:172.21.15.138:3300/0 conn(0x7fbfe00d4180 0x7fbfe00d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfdfe00640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe5600640 1 -- 172.21.15.28:0/4043520212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 msgr2=0x7fbfe006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe5600640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe5600640 1 -- 172.21.15.28:0/4043520212 >> v2:172.21.15.138:3300/0 conn(0x7fbfe00d4180 msgr2=0x7fbfe00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe5600640 1 --2- 172.21.15.28:0/4043520212 >> v2:172.21.15.138:3300/0 conn(0x7fbfe00d4180 0x7fbfe00d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.906+0000 7fbfe5600640 1 -- 172.21.15.28:0/4043520212 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe006e390 con 0x7fbfe006dd60 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.907+0000 7fbfe5600640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00d0d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b6ea0 tx=0x7fbfd802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973f96e14299685f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.907+0000 7fbfdf400640 1 -- 172.21.15.28:0/4043520212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8036070 con 0x7fbfe006dd60 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.907+0000 7fbfdf400640 1 -- 172.21.15.28:0/4043520212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd8031600 con 0x7fbfe006dd60 2024-09-10T10:50:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.907+0000 7fbfdf400640 1 -- 172.21.15.28:0/4043520212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80319d0 con 0x7fbfe006dd60 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 -- 172.21.15.28:0/4043520212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 msgr2=0x7fbfe00d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00d0d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b6ea0 tx=0x7fbfd802a600 comp rx=0 tx=0).stop 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 -- 172.21.15.28:0/4043520212 shutdown_connections 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00d0d40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 --2- 172.21.15.28:0/4043520212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 --2- 172.21.15.28:0/4043520212 >> v2:172.21.15.138:3300/0 conn(0x7fbfe00d4180 0x7fbfe00d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 -- 172.21.15.28:0/4043520212 >> 172.21.15.28:0/4043520212 conn(0x7fbfe009d6f0 msgr2=0x7fbfe008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 -- 172.21.15.28:0/4043520212 shutdown_connections 2024-09-10T10:50:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.908+0000 7fbfe6000640 1 -- 172.21.15.28:0/4043520212 wait complete. 2024-09-10T10:50:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.909+0000 7fbfe6000640 1 Processor -- start 2024-09-10T10:50:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.910+0000 7fbfe6000640 1 -- start start 2024-09-10T10:50:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.910+0000 7fbfe6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe00d4180 0x7fbfe00b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe00b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe6000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe00cebd0 con 0x7fbfe006dd60 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe006e950 con 0x7fbfe00d4180 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe00d1280 con 0x7fbfe015e6a0 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47788/0 (socket says 172.21.15.28:47788) 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 -- 172.21.15.28:0/1123402809 learned_addr learned my addr 172.21.15.28:0/1123402809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe5600640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe00b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfdfe00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe00d4180 0x7fbfe00b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 -- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 msgr2=0x7fbfe00b6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe00b6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 -- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe00d4180 msgr2=0x7fbfe00b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe00d4180 0x7fbfe00b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 -- 172.21.15.28:0/1123402809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe00cc840 con 0x7fbfe006dd60 2024-09-10T10:50:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.911+0000 7fbfe4c00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400d800 tx=0x7fbfd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.912+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4018020 con 0x7fbfe006dd60 2024-09-10T10:50:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.912+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd4009640 con 0x7fbfe006dd60 2024-09-10T10:50:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.912+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4009a10 con 0x7fbfe006dd60 2024-09-10T10:50:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.912+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe00ccb20 con 0x7fbfe006dd60 2024-09-10T10:50:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.912+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe00cd0e0 con 0x7fbfe006dd60 2024-09-10T10:50:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.913+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe00cc5d0 con 0x7fbfe006dd60 2024-09-10T10:50:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.914+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbfd401b020 con 0x7fbfe006dd60 2024-09-10T10:50:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.915+0000 7fbfdcc00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 0x7fbfc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.915+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fbfd408b300 con 0x7fbfe006dd60 2024-09-10T10:50:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.915+0000 7fbfdfe00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 0x7fbfc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.916+0000 7fbfdfe00640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 0x7fbfc006e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:07.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:07.919+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbfd4022030 con 0x7fbfe006dd60 2024-09-10T10:50:08.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.143+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfe00d4560 con 0x7fbfe006dd60 2024-09-10T10:50:08.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.144+0000 7fbfdcc00640 1 -- 172.21.15.28:0/1123402809 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5226 (secure 0 0 0) 0x7fbfd4059a30 con 0x7fbfe006dd60 2024-09-10T10:50:08.146 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:08.146 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":74,"btime":"2024-09-10T10:50:07:684750+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:06.695041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":12,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T10:50:08.147 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 74 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 msgr2=0x7fbfc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 0x7fbfc006e2f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc006fd0 tx=0x7fbfcc008040 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 msgr2=0x7fbfe00ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400d800 tx=0x7fbfd400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 shutdown_connections 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbfc006be30 0x7fbfc006e2f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbfe015e6a0 0x7fbfe00b6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.146+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe00d4180 0x7fbfe00b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.147+0000 7fbfe6000640 1 --2- 172.21.15.28:0/1123402809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbfe006dd60 0x7fbfe00ce150 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.147+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 >> 172.21.15.28:0/1123402809 conn(0x7fbfe009d6f0 msgr2=0x7fbfe00cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:08.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.147+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 shutdown_connections 2024-09-10T10:50:08.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08.147+0000 7fbfe6000640 1 -- 172.21.15.28:0/1123402809 wait complete. 2024-09-10T10:50:08.567 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:49:31.540549+0000', 'modified': '2024-09-10T10:50:06.695041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6408': {'gid': 6408, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 12, 'addr': '172.21.15.138:6836/1950306159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1950306159}, {'type': 'v1', 'addr': '172.21.15.138:6836', 'nonce': 1950306159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:50:08.568 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6408:up:rejoin 2024-09-10T10:50:08.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:09.569 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.736+0000 7f839f200640 1 Processor -- start 2024-09-10T10:50:09.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.736+0000 7f839f200640 1 -- start start 2024-09-10T10:50:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83a015e6a0 0x7f83a006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f83a006dd60 0x7f83a00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839f200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a00d2c20 con 0x7f83a00d4180 2024-09-10T10:50:09.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f83a00cefb0 con 0x7f83a006dd60 2024-09-10T10:50:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83a006e390 con 0x7f83a015e6a0 2024-09-10T10:50:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83a00d4180 0x7f83a00d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59302/0 (socket says 172.21.15.28:59302) 2024-09-10T10:50:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839de00640 1 -- 172.21.15.28:0/3550872027 learned_addr learned my addr 172.21.15.28:0/3550872027 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.737+0000 7f839e800640 1 --2- 172.21.15.28:0/3550872027 >> v2:172.21.15.28:3300/0 conn(0x7f83a006dd60 0x7f83a00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839d400640 1 --2- 172.21.15.28:0/3550872027 >> v2:172.21.15.138:3300/0 conn(0x7f83a015e6a0 0x7f83a006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839ca00640 1 -- 172.21.15.28:0/3550872027 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971984323 0 0) 0x7f83a00d2c20 con 0x7f83a00d4180 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839ca00640 1 -- 172.21.15.28:0/3550872027 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8384003520 con 0x7f83a00d4180 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839e800640 1 -- 172.21.15.28:0/3550872027 >> v2:172.21.15.138:3300/0 conn(0x7f83a015e6a0 msgr2=0x7f83a006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839e800640 1 --2- 172.21.15.28:0/3550872027 >> v2:172.21.15.138:3300/0 conn(0x7f83a015e6a0 0x7f83a006d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839e800640 1 -- 172.21.15.28:0/3550872027 >> v1:172.21.15.28:6789/0 conn(0x7f83a00d4180 legacy=0x7f83a00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.738+0000 7f839e800640 1 -- 172.21.15.28:0/3550872027 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a006e670 con 0x7f83a006dd60 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.739+0000 7f839e800640 1 --2- 172.21.15.28:0/3550872027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a006dd60 0x7f83a00cf440 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f83a00b5e20 tx=0x7f839802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b76877497956b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.739+0000 7f839ca00640 1 -- 172.21.15.28:0/3550872027 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398007740 con 0x7f83a006dd60 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.739+0000 7f839ca00640 1 -- 172.21.15.28:0/3550872027 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83980078e0 con 0x7f83a006dd60 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.739+0000 7f839ca00640 1 -- 172.21.15.28:0/3550872027 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398007c90 con 0x7f83a006dd60 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 -- 172.21.15.28:0/3550872027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a006dd60 msgr2=0x7f83a00cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 --2- 172.21.15.28:0/3550872027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a006dd60 0x7f83a00cf440 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f83a00b5e20 tx=0x7f839802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 -- 172.21.15.28:0/3550872027 shutdown_connections 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 --2- 172.21.15.28:0/3550872027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a006dd60 0x7f83a00cf440 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 --2- 172.21.15.28:0/3550872027 >> v2:172.21.15.138:3300/0 conn(0x7f83a015e6a0 0x7f83a006d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 -- 172.21.15.28:0/3550872027 >> 172.21.15.28:0/3550872027 conn(0x7f83a009d6f0 msgr2=0x7f83a008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 -- 172.21.15.28:0/3550872027 shutdown_connections 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.740+0000 7f839f200640 1 -- 172.21.15.28:0/3550872027 wait complete. 2024-09-10T10:50:09.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.742+0000 7f839f200640 1 Processor -- start 2024-09-10T10:50:09.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.742+0000 7f839f200640 1 -- start start 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a006dd60 0x7f83a01f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a00d4180 0x7f83a00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00cf980 con 0x7f83a015e6a0 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83a00cebd0 con 0x7f83a00d4180 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839f200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83a00cfb00 con 0x7f83a006dd60 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47816/0 (socket says 172.21.15.28:47816) 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 -- 172.21.15.28:0/3223445528 learned_addr learned my addr 172.21.15.28:0/3223445528 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839d400640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a00d4180 0x7f83a00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839de00640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a006dd60 0x7f83a01f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 -- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a006dd60 msgr2=0x7f83a01f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a006dd60 0x7f83a01f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 -- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a00d4180 msgr2=0x7f83a00c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a00d4180 0x7f83a00c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.743+0000 7f839e800640 1 -- 172.21.15.28:0/3223445528 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a00cabb0 con 0x7f83a015e6a0 2024-09-10T10:50:09.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f839e800640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8398009e80 tx=0x7f8398004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398009450 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8398004240 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398004630 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a00ca8d0 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.744+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a00b4f20 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.745+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a00cf440 con 0x7f83a015e6a0 2024-09-10T10:50:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.746+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8398039040 con 0x7f83a015e6a0 2024-09-10T10:50:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.747+0000 7f8396000640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 0x7f838006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.747+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f83980ac050 con 0x7f83a015e6a0 2024-09-10T10:50:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.748+0000 7f839de00640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 0x7f838006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:09.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.748+0000 7f839de00640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 0x7f838006e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8390005880 tx=0x7f8390009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:09.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.751+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8398040030 con 0x7f83a015e6a0 2024-09-10T10:50:09.791 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:50:09.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:50:09.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.978+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a00b9490 con 0x7f83a015e6a0 2024-09-10T10:50:09.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.978+0000 7f8396000640 1 -- 172.21.15.28:0/3223445528 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5233 (secure 0 0 0) 0x7f839807a1c0 con 0x7f83a015e6a0 2024-09-10T10:50:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:09.981 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"btime":"2024-09-10T10:50:08:691575+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:08.691566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":13,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6408,"qdb_cluster":[6408]},"id":5}]} 2024-09-10T10:50:09.981 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 75 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 msgr2=0x7f838006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 0x7f838006e2f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8390005880 tx=0x7f8390009040 comp rx=0 tx=0).stop 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 msgr2=0x7f83a00c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8398009e80 tx=0x7f8398004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 shutdown_connections 2024-09-10T10:50:09.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838006be30 0x7f838006e2f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a015e6a0 0x7f83a00c8020 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a00d4180 0x7f83a00c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 --2- 172.21.15.28:0/3223445528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a006dd60 0x7f83a01f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 >> 172.21.15.28:0/3223445528 conn(0x7f83a009d6f0 msgr2=0x7f83a00b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 shutdown_connections 2024-09-10T10:50:09.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:09.981+0000 7f839f200640 1 -- 172.21.15.28:0/3223445528 wait complete. 2024-09-10T10:50:10.401 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:49:31.540549+0000', 'modified': '2024-09-10T10:50:08.691566+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 87, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6408}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6408': {'gid': 6408, 'name': 'd', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.138:6836/1950306159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 1950306159}, {'type': 'v1', 'addr': '172.21.15.138:6836', 'nonce': 1950306159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6408, 'qdb_cluster': [6408]} 2024-09-10T10:50:10.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:50:10.402 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T10:50:10.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.561+0000 7f1c5b200640 1 Processor -- start 2024-09-10T10:50:10.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.561+0000 7f1c5b200640 1 -- start start 2024-09-10T10:50:10.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.562+0000 7f1c5b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1c540d0dd0 0x7f1c540d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.562+0000 7f1c5b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c540cf000 con 0x7f1c5415e660 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.562+0000 7f1c5b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c540d16f0 con 0x7f1c540d1f90 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.562+0000 7f1c5b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1c5406d3e0 con 0x7f1c540d0dd0 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1c540d0dd0 0x7f1c540d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1c540d0dd0 0x7f1c540d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47826/0 (socket says 172.21.15.28:47826) 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 -- 172.21.15.28:0/3809868883 learned_addr learned my addr 172.21.15.28:0/3809868883 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59400640 1 --1- 172.21.15.28:0/3809868883 >> v1:172.21.15.28:6789/0 conn(0x7f1c5415e660 0x7f1c5406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59306/0 (socket says 172.21.15.28:59306) 2024-09-10T10:50:10.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 -- 172.21.15.28:0/3809868883 >> v1:172.21.15.138:6789/0 conn(0x7f1c540d1f90 legacy=0x7f1c540cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 -- 172.21.15.28:0/3809868883 >> v1:172.21.15.28:6789/0 conn(0x7f1c5415e660 legacy=0x7f1c5406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.563+0000 7f1c59e00640 1 -- 172.21.15.28:0/3809868883 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c540d29a0 con 0x7f1c540d0dd0 2024-09-10T10:50:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.564+0000 7f1c59e00640 1 --2- 172.21.15.28:0/3809868883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c540d0dd0 0x7f1c540d11b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009870 tx=0x7f1c4802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1b6d8534fc79b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.564+0000 7f1c58a00640 1 -- 172.21.15.28:0/3809868883 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48037070 con 0x7f1c540d0dd0 2024-09-10T10:50:10.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.564+0000 7f1c58a00640 1 -- 172.21.15.28:0/3809868883 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4802ae50 con 0x7f1c540d0dd0 2024-09-10T10:50:10.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.564+0000 7f1c58a00640 1 -- 172.21.15.28:0/3809868883 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48032860 con 0x7f1c540d0dd0 2024-09-10T10:50:10.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 -- 172.21.15.28:0/3809868883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c540d0dd0 msgr2=0x7f1c540d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 --2- 172.21.15.28:0/3809868883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c540d0dd0 0x7f1c540d11b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009870 tx=0x7f1c4802a300 comp rx=0 tx=0).stop 2024-09-10T10:50:10.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 -- 172.21.15.28:0/3809868883 shutdown_connections 2024-09-10T10:50:10.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 --2- 172.21.15.28:0/3809868883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c540d0dd0 0x7f1c540d11b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 -- 172.21.15.28:0/3809868883 >> 172.21.15.28:0/3809868883 conn(0x7f1c5408bfc0 msgr2=0x7f1c540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 -- 172.21.15.28:0/3809868883 shutdown_connections 2024-09-10T10:50:10.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.565+0000 7f1c5b200640 1 -- 172.21.15.28:0/3809868883 wait complete. 2024-09-10T10:50:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.567+0000 7f1c5b200640 1 Processor -- start 2024-09-10T10:50:10.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.567+0000 7f1c5b200640 1 -- start start 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c540d0dd0 0x7f1c540ce260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c540d1f90 0x7f1c5414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540cf330 con 0x7f1c5415e660 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540d2f40 con 0x7f1c540d1f90 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c540d3220 con 0x7f1c540d0dd0 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47836/0 (socket says 172.21.15.28:47836) 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c5a800640 1 -- 172.21.15.28:0/1702487425 learned_addr learned my addr 172.21.15.28:0/1702487425 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c540d0dd0 0x7f1c540ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.568+0000 7f1c59400640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c540d1f90 0x7f1c5414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 -- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c540d0dd0 msgr2=0x7f1c540ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c540d0dd0 0x7f1c540ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 -- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c540d1f90 msgr2=0x7f1c5414ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c540d1f90 0x7f1c5414ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 -- 172.21.15.28:0/1702487425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5414e310 con 0x7f1c5415e660 2024-09-10T10:50:10.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c7a0 tx=0x7f1c5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c4e000640 1 -- 172.21.15.28:0/1702487425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50003b70 con 0x7f1c5415e660 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c4e000640 1 -- 172.21.15.28:0/1702487425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c50003d30 con 0x7f1c5415e660 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c4e000640 1 -- 172.21.15.28:0/1702487425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5000aa90 con 0x7f1c5415e660 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5414e5b0 con 0x7f1c5415e660 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.569+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5414ee50 con 0x7f1c5415e660 2024-09-10T10:50:10.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.571+0000 7f1c4e000640 1 -- 172.21.15.28:0/1702487425 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c5000ac30 con 0x7f1c5415e660 2024-09-10T10:50:10.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.572+0000 7f1c4e000640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 0x7f1c3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.572+0000 7f1c4e000640 1 -- 172.21.15.28:0/1702487425 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f1c5008ae70 con 0x7f1c5415e660 2024-09-10T10:50:10.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.573+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 0x7f1c3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.573+0000 7f1c5f7e7740 1 Processor -- start 2024-09-10T10:50:10.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.573+0000 7f1c5f7e7740 1 -- start start 2024-09-10T10:50:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 0x7f1c3806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c48034000 tx=0x7f1c4802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611162ea0 0x5596111a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611160660 0x559611163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5596110c13b0 con 0x559611164630 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596110c6410 con 0x559611162ea0 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55961106e620 con 0x559611160660 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47842/0 (socket says 172.21.15.28:47842) 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.574+0000 7f1c5a800640 1 -- 172.21.15.28:0/3287163101 learned_addr learned my addr 172.21.15.28:0/3287163101 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c59400640 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611160660 0x559611163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c59e00640 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611162ea0 0x5596111a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 -- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611160660 msgr2=0x559611163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611160660 0x559611163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 -- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611162ea0 msgr2=0x5596111a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611162ea0 0x5596111a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 -- 172.21.15.28:0/3287163101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559610dd92d0 con 0x559611164630 2024-09-10T10:50:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c5a800640 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1c50089130 tx=0x7f1c50011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435c32dc26ae9349 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c42600640 1 -- 172.21.15.28:0/3287163101 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50061b20 con 0x559611164630 2024-09-10T10:50:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c42600640 1 -- 172.21.15.28:0/3287163101 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c500572f0 con 0x559611164630 2024-09-10T10:50:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.575+0000 7f1c42600640 1 -- 172.21.15.28:0/3287163101 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50058440 con 0x559611164630 2024-09-10T10:50:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/3287163101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 msgr2=0x55961115d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f1c50089130 tx=0x7f1c50011000 comp rx=0 tx=0).stop 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/3287163101 shutdown_connections 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x55961115d720 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611160660 0x559611163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/3287163101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611162ea0 0x5596111a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/3287163101 >> 172.21.15.28:0/3287163101 conn(0x55961115f8f0 msgr2=0x5596111a7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/3287163101 shutdown_connections 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.576+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/3287163101 wait complete. 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.577+0000 7f1c5f7e7740 1 -- start start 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.577+0000 7f1c5f7e7740 1 Processor -- start 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x55961118d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559611081780 con 0x559611183fa0 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596110c23d0 con 0x559611164630 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55961115ed90 con 0x559611162ea0 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5f7e7740 10 client.0 _ll_register_callbacks cb 0x5596110bf5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.578+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47856/0 (socket says 172.21.15.28:47856) 2024-09-10T10:50:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 -- 172.21.15.28:0/1561373104 learned_addr learned my addr 172.21.15.28:0/1561373104 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 -- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 msgr2=0x559611181c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c59400640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x55961118d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 -- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 msgr2=0x55961118d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x55961118d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 -- 172.21.15.28:0/1561373104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559610ea36b0 con 0x559611183fa0 2024-09-10T10:50:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c59400640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x55961118d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c50089210 tx=0x7f1c50001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d594c09fdb94f9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.579+0000 7f1c36a00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c50058640 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c500572f0 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f1c50082b20 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f1c50081050 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 10 client.6666 fetch_fsmap learned FSMap version 75 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5596111850b0 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c500572f0 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f1c50054d40 con 0x559611183fa0 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 10 client.6666 fetch_fsmap finished waiting for FSMap version 75 2024-09-10T10:50:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 10 client.6666 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 20 client.6666 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x55961115ce80 0x55961115d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 4 client.6666 mds_command: new command op to 6408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.580+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 --> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596110d3b40 con 0x55961115ce80 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.581+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x55961115ce80 0x55961115d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.582+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x55961115ce80 0x55961115d260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8219f88ee5fce4 server_cookie=c320d222b307be1d in_seq=0 out_seq=0 2024-09-10T10:50:10.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.582+0000 7f1c36a00640 10 client.6666 ms_handle_connect on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c36a00640 1 -- 172.21.15.28:0/1561373104 <== mds.0 v2:172.21.15.138:6834/1950306159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596110d3b40 con 0x55961115ce80 2024-09-10T10:50:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c36a00640 10 client.6666 handle_command_reply: tid=0 2024-09-10T10:50:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 client.6666 shutdown 2024-09-10T10:50:10.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 msgr2=0x559611184420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1c50089210 tx=0x7f1c50001250 comp rx=0 tx=0).stop 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 shutdown_connections 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x55961115ce80 0x55961115d260 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611183fa0 0x559611184420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x55961118d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1561373104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c36a00640 0 client.6666 ms_handle_reset on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 >> 172.21.15.28:0/1561373104 conn(0x559611160660 msgr2=0x559611187d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 shutdown_connections 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1561373104 wait complete. 2024-09-10T10:50:10.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.583+0000 7f1c5f7e7740 20 client.6666 trim_cache size 0 max 16384 2024-09-10T10:50:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.602+0000 7f1c5f7e7740 1 Processor -- start 2024-09-10T10:50:10.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.602+0000 7f1c5f7e7740 1 -- start start 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x559611180af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 0x559611189030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54200380 con 0x559611162ea0 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c540cbda0 con 0x559611164630 2024-09-10T10:50:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c541ffed0 con 0x559611183fa0 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c59e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c59e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47862/0 (socket says 172.21.15.28:47862) 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.603+0000 7f1c59e00640 1 -- 172.21.15.28:0/1104088400 learned_addr learned my addr 172.21.15.28:0/1104088400 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 0x559611189030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59400640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x559611180af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 -- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 msgr2=0x559611189030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 0x559611189030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 -- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 msgr2=0x559611180af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x559611180af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 -- 172.21.15.28:0/1104088400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596111a8ca0 con 0x559611162ea0 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c5a800640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 0x559611189030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.604+0000 7f1c59e00640 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009990 tx=0x7f1c48008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9ed125952022 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c41c00640 1 -- 172.21.15.28:0/1104088400 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48037040 con 0x559611162ea0 2024-09-10T10:50:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c41c00640 1 -- 172.21.15.28:0/1104088400 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4804f070 con 0x559611162ea0 2024-09-10T10:50:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c41c00640 1 -- 172.21.15.28:0/1104088400 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4803c660 con 0x559611162ea0 2024-09-10T10:50:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1104088400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 msgr2=0x55961118d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009990 tx=0x7f1c48008030 comp rx=0 tx=0).stop 2024-09-10T10:50:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1104088400 shutdown_connections 2024-09-10T10:50:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611183fa0 0x559611189030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611164630 0x559611180af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/1104088400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611162ea0 0x55961118d240 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1104088400 >> 172.21.15.28:0/1104088400 conn(0x559611160660 msgr2=0x559611158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.605+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1104088400 shutdown_connections 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.606+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/1104088400 wait complete. 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.606+0000 7f1c5f7e7740 1 -- start start 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.606+0000 7f1c5f7e7740 1 Processor -- start 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611183fa0 0x5596111ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559611081780 con 0x559611164630 2024-09-10T10:50:10.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5596110c23d0 con 0x559611183fa0 2024-09-10T10:50:10.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55961115df00 con 0x559611162ea0 2024-09-10T10:50:10.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c5f7e7740 10 client.0 _ll_register_callbacks cb 0x7f1c480096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:10.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.607+0000 7f1c59400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47870/0 (socket says 172.21.15.28:47870) 2024-09-10T10:50:10.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 -- 172.21.15.28:0/935813495 learned_addr learned my addr 172.21.15.28:0/935813495 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59e00640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c5a800640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611183fa0 0x5596111ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 -- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 msgr2=0x559611181860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 -- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611183fa0 msgr2=0x5596111ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611183fa0 0x5596111ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 -- 172.21.15.28:0/935813495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596111b78d0 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c59400640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c440077a0 tx=0x7f1c44016790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2732266fff8a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.608+0000 7f1c36a00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44017640 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c4804f070 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f1c44010040 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f1c44025050 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 10 client.6672 fetch_fsmap learned FSMap version 75 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x559611180af0 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c4804f070 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f1c44028020 con 0x559611164630 2024-09-10T10:50:10.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 10 client.6672 fetch_fsmap finished waiting for FSMap version 75 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 10 client.6672 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 20 client.6672 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x559611226510 0x5596112268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 4 client.6672 mds_command: new command op to 6408 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.609+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 --> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] -- command(tid 0: {"prefix": "status"}) -- 0x559611183b10 con 0x559611226510 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.610+0000 7f1c59e00640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x559611226510 0x5596112268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c59e00640 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x559611226510 0x5596112268f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b5635120438136 server_cookie=91b1cb81188a3747 in_seq=0 out_seq=0 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c36a00640 10 client.6672 ms_handle_connect on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c36a00640 1 -- 172.21.15.28:0/935813495 <== mds.0 v2:172.21.15.138:6834/1950306159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559611183b10 con 0x559611226510 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c36a00640 10 client.6672 handle_command_reply: tid=0 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 client.6672 shutdown 2024-09-10T10:50:10.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 msgr2=0x559611225f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1c440077a0 tx=0x7f1c44016790 comp rx=0 tx=0).stop 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 shutdown_connections 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x559611226510 0x5596112268f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559611183fa0 0x5596111ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559611164630 0x559611225f20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 --2- 172.21.15.28:0/935813495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559611162ea0 0x559611181860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 >> 172.21.15.28:0/935813495 conn(0x559611160660 msgr2=0x5596111a50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c36a00640 0 client.6672 ms_handle_reset on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 shutdown_connections 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.611+0000 7f1c5f7e7740 1 -- 172.21.15.28:0/935813495 wait complete. 2024-09-10T10:50:10.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5f7e7740 20 client.6672 trim_cache size 0 max 16384 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6408, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 4.9326968759999996, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 75, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 87, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T10:50:10.618 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 43.577619413000001 2024-09-10T10:50:10.619 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 msgr2=0x7f1c3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 0x7f1c3806e2f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1c48034000 tx=0x7f1c4802e000 comp rx=0 tx=0).stop 2024-09-10T10:50:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 msgr2=0x7f1c5414f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c7a0 tx=0x7f1c5000cc60 comp rx=0 tx=0).stop 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 shutdown_connections 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c3806be30 0x7f1c3806e2f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c5415e660 0x7f1c5414f650 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c540d1f90 0x7f1c5414ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 --2- 172.21.15.28:0/1702487425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c540d0dd0 0x7f1c540ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 >> 172.21.15.28:0/1702487425 conn(0x7f1c5408bfc0 msgr2=0x7f1c5406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 shutdown_connections 2024-09-10T10:50:10.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.612+0000 7f1c5b200640 1 -- 172.21.15.28:0/1702487425 wait complete. 2024-09-10T10:50:10.630 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat a/ && find a/') 2024-09-10T10:50:10.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout: File: a/ 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 2 Blocks: 0 IO Block: 65536 directory 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 2ah/42d Inode: 1099511627776 Links: 4 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0755/drwxr-xr-x) Uid: ( 1000/ ubuntu) Gid: ( 1333/ ubuntu) 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Context: unconfined_u:object_r:unlabeled_t:s0 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-09-10 10:49:41.283677252 +0000 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-09-10 10:49:47.021522208 +0000 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-09-10 10:49:47.021522208 +0000 2024-09-10T10:50:10.715 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-09-10 10:49:41.283677252 +0000 2024-09-10T10:50:10.724 INFO:teuthology.orchestra.run.smithi028.stdout:a/ 2024-09-10T10:50:10.724 INFO:teuthology.orchestra.run.smithi028.stdout:a/z 2024-09-10T10:50:10.724 INFO:teuthology.orchestra.run.smithi028.stdout:a/z/d 2024-09-10T10:50:10.724 INFO:teuthology.orchestra.run.smithi028.stdout:a/c 2024-09-10T10:50:10.724 INFO:teuthology.orchestra.run.smithi028.stdout:a/c/d 2024-09-10T10:50:10.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:10.733 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-10T10:50:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.891+0000 7f1285e00640 1 Processor -- start 2024-09-10T10:50:10.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.891+0000 7f1285e00640 1 -- start start 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.892+0000 7f1285e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1280161d00 0x7f12800b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.892+0000 7f1285e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f12800b62b0 0x7f12800d2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.892+0000 7f1285e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12800d4300 con 0x7f1280161920 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.892+0000 7f1285e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f12800b49a0 con 0x7f1280161d00 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.892+0000 7f1285e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f12800b4c40 con 0x7f12800b62b0 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1280161d00 0x7f12800b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1280161d00 0x7f12800b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47878/0 (socket says 172.21.15.28:47878) 2024-09-10T10:50:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 -- 172.21.15.28:0/2018501572 learned_addr learned my addr 172.21.15.28:0/2018501572 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f127f400640 1 --2- 172.21.15.28:0/2018501572 >> v2:172.21.15.138:3300/0 conn(0x7f12800b62b0 0x7f12800d2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 -- 172.21.15.28:0/2018501572 >> v2:172.21.15.138:3300/0 conn(0x7f12800b62b0 msgr2=0x7f12800d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 --2- 172.21.15.28:0/2018501572 >> v2:172.21.15.138:3300/0 conn(0x7f12800b62b0 0x7f12800d2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 -- 172.21.15.28:0/2018501572 >> v1:172.21.15.138:6789/0 conn(0x7f1280161920 legacy=0x7f12800b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.893+0000 7f1276000640 1 -- 172.21.15.28:0/2018501572 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12800d4db0 con 0x7f1280161d00 2024-09-10T10:50:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.894+0000 7f1276000640 1 --2- 172.21.15.28:0/2018501572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800b5090 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f12800b5600 tx=0x7f126802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dd2236e26fce25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.894+0000 7f127e000640 1 -- 172.21.15.28:0/2018501572 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268037070 con 0x7f1280161d00 2024-09-10T10:50:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.894+0000 7f127e000640 1 -- 172.21.15.28:0/2018501572 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f126802ae20 con 0x7f1280161d00 2024-09-10T10:50:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.894+0000 7f127e000640 1 -- 172.21.15.28:0/2018501572 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268032660 con 0x7f1280161d00 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 -- 172.21.15.28:0/2018501572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 msgr2=0x7f12800b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 --2- 172.21.15.28:0/2018501572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800b5090 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f12800b5600 tx=0x7f126802a110 comp rx=0 tx=0).stop 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 -- 172.21.15.28:0/2018501572 shutdown_connections 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 --2- 172.21.15.28:0/2018501572 >> v2:172.21.15.138:3300/0 conn(0x7f12800b62b0 0x7f12800d2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 --2- 172.21.15.28:0/2018501572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800b5090 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 -- 172.21.15.28:0/2018501572 >> 172.21.15.28:0/2018501572 conn(0x7f128008bfc0 msgr2=0x7f12800d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 -- 172.21.15.28:0/2018501572 shutdown_connections 2024-09-10T10:50:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.895+0000 7f1285e00640 1 -- 172.21.15.28:0/2018501572 wait complete. 2024-09-10T10:50:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.897+0000 7f1285e00640 1 Processor -- start 2024-09-10T10:50:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.897+0000 7f1285e00640 1 -- start start 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12800b62b0 0x7f128015fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1280161920 0x7f12800b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f12800d2b40 con 0x7f1280161d00 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12800b68d0 con 0x7f12800b62b0 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1285e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12800d2cc0 con 0x7f1280161920 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47890/0 (socket says 172.21.15.28:47890) 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 -- 172.21.15.28:0/2996652579 learned_addr learned my addr 172.21.15.28:0/2996652579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127ea00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12800b62b0 0x7f128015fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f1276000640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1280161920 0x7f12800b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 -- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1280161920 msgr2=0x7f12800b9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1280161920 0x7f12800b9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 -- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12800b62b0 msgr2=0x7f128015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.898+0000 7f127f400640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12800b62b0 0x7f128015fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f127f400640 1 -- 172.21.15.28:0/2996652579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1280160270 con 0x7f1280161d00 2024-09-10T10:50:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f127f400640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1278007e70 tx=0x7f127800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f1277e00640 1 -- 172.21.15.28:0/2996652579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278014620 con 0x7f1280161d00 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f1277e00640 1 -- 172.21.15.28:0/2996652579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1278019070 con 0x7f1280161d00 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f1277e00640 1 -- 172.21.15.28:0/2996652579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1278014a20 con 0x7f1280161d00 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12800a1a90 con 0x7f1280161d00 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.899+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12800c7ee0 con 0x7f1280161d00 2024-09-10T10:50:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.901+0000 7f1277e00640 1 -- 172.21.15.28:0/2996652579 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1278014bc0 con 0x7f1280161d00 2024-09-10T10:50:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.903+0000 7f1277e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 0x7f125c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.903+0000 7f1277e00640 1 -- 172.21.15.28:0/2996652579 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f127808ba30 con 0x7f1280161d00 2024-09-10T10:50:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.903+0000 7f127ea00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 0x7f125c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f1288cb0740 1 Processor -- start 2024-09-10T10:50:10.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f1288cb0740 1 -- start start 2024-09-10T10:50:10.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f127ea00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 0x7f125c06e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f12800c9c90 tx=0x7f1270009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f893c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.904+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f849480 0x56000f894740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1288cb0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56000f7b1660 con 0x56000f863800 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56000f759680 con 0x56000f863420 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56000f7ad620 con 0x56000f849480 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47902/0 (socket says 172.21.15.28:47902) 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 -- 172.21.15.28:0/787833271 learned_addr learned my addr 172.21.15.28:0/787833271 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f127f400640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f849480 0x56000f894740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f127ea00640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f893c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 -- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f849480 msgr2=0x56000f894740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f849480 0x56000f894740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 -- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 msgr2=0x56000f893c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f893c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f1276000640 1 -- 172.21.15.28:0/787833271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000f58e7d0 con 0x56000f863800 2024-09-10T10:50:10.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.905+0000 7f127ea00640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f893c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:10.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f1276000640 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f126802a150 tx=0x7f126802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ef9a0fb3197340 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f126f400640 1 -- 172.21.15.28:0/787833271 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268037040 con 0x56000f863800 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f126f400640 1 -- 172.21.15.28:0/787833271 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12680326c0 con 0x56000f863800 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f126f400640 1 -- 172.21.15.28:0/787833271 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268032a90 con 0x56000f863800 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f1288cb0740 1 -- 172.21.15.28:0/787833271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 msgr2=0x56000f894200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.906+0000 7f1288cb0740 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f126802a150 tx=0x7f126802c000 comp rx=0 tx=0).stop 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 -- 172.21.15.28:0/787833271 shutdown_connections 2024-09-10T10:50:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f849480 0x56000f894740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f894200 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 --2- 172.21.15.28:0/787833271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f893c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 -- 172.21.15.28:0/787833271 >> 172.21.15.28:0/787833271 conn(0x56000f84a220 msgr2=0x56000f892840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 -- 172.21.15.28:0/787833271 shutdown_connections 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 -- 172.21.15.28:0/787833271 wait complete. 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.907+0000 7f1288cb0740 1 -- start start 2024-09-10T10:50:10.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.908+0000 7f1288cb0740 1 Processor -- start 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.908+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863800 0x56000f8a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f8a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56000f8483e0 con 0x56000f863420 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56000f73a4c0 con 0x56000f863800 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56000f8486c0 con 0x56000f878bf0 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1288cb0740 10 client.0 _ll_register_callbacks cb 0x56000f7a9e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47918/0 (socket says 172.21.15.28:47918) 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 -- 172.21.15.28:0/77932178 learned_addr learned my addr 172.21.15.28:0/77932178 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f1276000640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863800 0x56000f8a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127f400640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f8a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 -- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 msgr2=0x56000f8a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f8a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 -- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863800 msgr2=0x56000f8a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863800 0x56000f8a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.909+0000 7f127ea00640 1 -- 172.21.15.28:0/77932178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000f4c43f0 con 0x56000f863420 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f127ea00640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f127000ed90 tx=0x7f12700036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fa35c457d82715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1263e00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270026060 con 0x56000f863420 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f1270021470 con 0x56000f863420 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12680326c0 con 0x56000f863420 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f127002c050 con 0x56000f863420 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1288cb0740 10 client.6684 fetch_fsmap learned FSMap version 75 2024-09-10T10:50:10.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x56000f4840f0 con 0x56000f863420 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.910+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12680326c0 con 0x56000f863420 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f127002f020 con 0x56000f863420 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 10 client.6684 fetch_fsmap finished waiting for FSMap version 75 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 10 client.6684 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 20 client.6684 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f88e990 0x56000f8a6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 4 client.6684 mds_command: new command op to 6408 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 --> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56000f7bed90 con 0x56000f88e990 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.911+0000 7f1276000640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f88e990 0x56000f8a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.912+0000 7f1276000640 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f88e990 0x56000f8a6420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30982465e69b6ca1 server_cookie=ce4072dd196baeef in_seq=0 out_seq=0 2024-09-10T10:50:10.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.912+0000 7f1263e00640 10 client.6684 ms_handle_connect on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1263e00640 1 -- 172.21.15.28:0/77932178 <== mds.0 v2:172.21.15.138:6834/1950306159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56000f7bed90 con 0x56000f88e990 2024-09-10T10:50:10.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1263e00640 10 client.6684 handle_command_reply: tid=0 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 client.6684 shutdown 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 msgr2=0x56000f879ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f127000ed90 tx=0x7f12700036f0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 shutdown_connections 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f88e990 0x56000f8a6420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f8a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863800 0x56000f8a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 --2- 172.21.15.28:0/77932178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863420 0x56000f879ab0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 >> 172.21.15.28:0/77932178 conn(0x56000f849480 msgr2=0x56000f88c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1263e00640 0 client.6684 ms_handle_reset on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 shutdown_connections 2024-09-10T10:50:10.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 1 -- 172.21.15.28:0/77932178 wait complete. 2024-09-10T10:50:10.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.913+0000 7f1288cb0740 20 client.6684 trim_cache size 0 max 16384 2024-09-10T10:50:10.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 Processor -- start 2024-09-10T10:50:10.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 -- start start 2024-09-10T10:50:10.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f843ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f893260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1280201210 con 0x56000f863800 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1280200210 con 0x56000f863420 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.929+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12800c9650 con 0x56000f878bf0 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47934/0 (socket says 172.21.15.28:47934) 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 -- 172.21.15.28:0/3579002232 learned_addr learned my addr 172.21.15.28:0/3579002232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f127ea00640 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f843ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f127f400640 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f893260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 -- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 msgr2=0x56000f893260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f893260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 -- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 msgr2=0x56000f843ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f843ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 -- 172.21.15.28:0/3579002232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000f87b8b0 con 0x56000f863800 2024-09-10T10:50:10.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.930+0000 7f1276000640 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1268009930 tx=0x7f1268040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea0899d2100b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f126ea00640 1 -- 172.21.15.28:0/3579002232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268037040 con 0x56000f863800 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f126ea00640 1 -- 172.21.15.28:0/3579002232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268032b50 con 0x56000f863800 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f126ea00640 1 -- 172.21.15.28:0/3579002232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126803e460 con 0x56000f863800 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 -- 172.21.15.28:0/3579002232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 msgr2=0x56000f8628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1268009930 tx=0x7f1268040040 comp rx=0 tx=0).stop 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 -- 172.21.15.28:0/3579002232 shutdown_connections 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f878bf0 0x56000f893260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f8628e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 --2- 172.21.15.28:0/3579002232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f863420 0x56000f843ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 -- 172.21.15.28:0/3579002232 >> 172.21.15.28:0/3579002232 conn(0x56000f849480 msgr2=0x56000f846f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 -- 172.21.15.28:0/3579002232 shutdown_connections 2024-09-10T10:50:10.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.931+0000 7f1288cb0740 1 -- 172.21.15.28:0/3579002232 wait complete. 2024-09-10T10:50:10.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.932+0000 7f1288cb0740 1 -- start start 2024-09-10T10:50:10.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.932+0000 7f1288cb0740 1 Processor -- start 2024-09-10T10:50:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f863420 0x56000f88b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f878bf0 0x56000f912c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56000f8b7360 con 0x56000f863800 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12801ffbc0 con 0x56000f878bf0 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56000f87a3f0 con 0x56000f863420 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.933+0000 7f1288cb0740 10 client.0 _ll_register_callbacks cb 0x7f1270007e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47936/0 (socket says 172.21.15.28:47936) 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 -- 172.21.15.28:0/2819695918 learned_addr learned my addr 172.21.15.28:0/2819695918 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f127ea00640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f863420 0x56000f88b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f127f400640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f878bf0 0x56000f912c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 -- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f863420 msgr2=0x56000f88b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f863420 0x56000f88b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 -- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f878bf0 msgr2=0x56000f912c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f878bf0 0x56000f912c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 -- 172.21.15.28:0/2819695918 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000f8ad170 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1276000640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f12680090b0 tx=0x7f1268004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942dd73e4a67971b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1263e00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.934+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268037070 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f1268004240 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1268032b50 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f1268004440 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 10 client.6690 fetch_fsmap learned FSMap version 75 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x56000f879430 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1268032b50 con 0x56000f863800 2024-09-10T10:50:10.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f1268058020 con 0x56000f863800 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 10 client.6690 fetch_fsmap finished waiting for FSMap version 75 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 10 client.6690 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 20 client.6690 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f8799c0 0x56000f9131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 4 client.6690 mds_command: new command op to 6408 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.935+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 --> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56000f7b90e0 con 0x56000f8799c0 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.936+0000 7f127ea00640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f8799c0 0x56000f9131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.936+0000 7f127ea00640 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f8799c0 0x56000f9131c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ab352e53b74698 server_cookie=1d3786645bd2a62c in_seq=0 out_seq=0 2024-09-10T10:50:10.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.936+0000 7f1263e00640 10 client.6690 ms_handle_connect on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.947+0000 7f1263e00640 1 -- 172.21.15.28:0/2819695918 <== mds.0 v2:172.21.15.138:6834/1950306159 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56000f7b90e0 con 0x56000f8799c0 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.947+0000 7f1263e00640 10 client.6690 handle_command_reply: tid=0 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.947+0000 7f1288cb0740 1 client.6690 shutdown 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 msgr2=0x56000f88bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f12680090b0 tx=0x7f1268004730 comp rx=0 tx=0).stop 2024-09-10T10:50:10.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 shutdown_connections 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:6834/1950306159,v1:172.21.15.138:6836/1950306159] conn(0x56000f8799c0 0x56000f9131c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56000f878bf0 0x56000f912c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56000f863800 0x56000f88bc10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 --2- 172.21.15.28:0/2819695918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56000f863420 0x56000f88b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 >> 172.21.15.28:0/2819695918 conn(0x56000f849480 msgr2=0x56000f893260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1263e00640 0 client.6690 ms_handle_reset on v2:172.21.15.138:6834/1950306159 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 shutdown_connections 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 1 -- 172.21.15.28:0/2819695918 wait complete. 2024-09-10T10:50:10.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.948+0000 7f1288cb0740 20 client.6690 trim_cache size 0 max 16384 2024-09-10T10:50:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:10.952 INFO:teuthology.orchestra.run.smithi028.stdout: "message": "", 2024-09-10T10:50:10.952 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0 2024-09-10T10:50:10.952 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 msgr2=0x7f125c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 0x7f125c06e2f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f12800c9c90 tx=0x7f1270009040 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 msgr2=0x7f12800c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1278007e70 tx=0x7f127800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 shutdown_connections 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f125c06be30 0x7f125c06e2f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1280161d00 0x7f12800c9e00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1280161920 0x7f12800b9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 --2- 172.21.15.28:0/2996652579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12800b62b0 0x7f128015fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 >> 172.21.15.28:0/2996652579 conn(0x7f128008bfc0 msgr2=0x7f12800d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:10.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 shutdown_connections 2024-09-10T10:50:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:10.949+0000 7f1285e00640 1 -- 172.21.15.28:0/2996652579 wait complete. 2024-09-10T10:50:10.966 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:50:10.967 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:50:11.063 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:50:11.155 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:50:11.156 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:50:11.156 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:11.156 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:11.192 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:11.192 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:11.267 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:50:11.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.475+0000 7fa31d000640 1 Processor -- start 2024-09-10T10:50:11.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.475+0000 7fa31d000640 1 -- start start 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31815b300 0x7fa3180cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3180cd3f0 0x7fa31806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa318072d30 0x7fa31806f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa3180728a0 con 0x7fa3180cd3f0 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3180cdbe0 con 0x7fa318072d30 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa31d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3180cd900 con 0x7fa31815b300 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.476+0000 7fa315400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3180cd3f0 0x7fa31806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3180cd3f0 0x7fa31806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47946/0 (socket says 172.21.15.28:47946) 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 -- 172.21.15.28:0/3645617344 learned_addr learned my addr 172.21.15.28:0/3645617344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315e00640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31815b300 0x7fa3180cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa316800640 1 --2- 172.21.15.28:0/3645617344 >> v2:172.21.15.138:3300/0 conn(0x7fa318072d30 0x7fa31806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 -- 172.21.15.28:0/3645617344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31815b300 msgr2=0x7fa3180cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31815b300 0x7fa3180cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 -- 172.21.15.28:0/3645617344 >> v2:172.21.15.138:3300/0 conn(0x7fa318072d30 msgr2=0x7fa31806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 --2- 172.21.15.28:0/3645617344 >> v2:172.21.15.138:3300/0 conn(0x7fa318072d30 0x7fa31806f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.477+0000 7fa315400640 1 -- 172.21.15.28:0/3645617344 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa318072250 con 0x7fa3180cd3f0 2024-09-10T10:50:11.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.478+0000 7fa315400640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa31806f1c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa318072040 tx=0x7fa30402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83a1dab5dfdce2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.478+0000 7fa314a00640 1 -- 172.21.15.28:0/3645617344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304037070 con 0x7fa3180cd3f0 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.478+0000 7fa314a00640 1 -- 172.21.15.28:0/3645617344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa304032440 con 0x7fa3180cd3f0 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.478+0000 7fa314a00640 1 -- 172.21.15.28:0/3645617344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304032810 con 0x7fa3180cd3f0 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 -- 172.21.15.28:0/3645617344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 msgr2=0x7fa31806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa31806f1c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa318072040 tx=0x7fa30402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 -- 172.21.15.28:0/3645617344 shutdown_connections 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 --2- 172.21.15.28:0/3645617344 >> v2:172.21.15.138:3300/0 conn(0x7fa318072d30 0x7fa31806f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa31806f1c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 --2- 172.21.15.28:0/3645617344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa31815b300 0x7fa3180cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 -- 172.21.15.28:0/3645617344 >> 172.21.15.28:0/3645617344 conn(0x7fa318067d70 msgr2=0x7fa31806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:11.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 -- 172.21.15.28:0/3645617344 shutdown_connections 2024-09-10T10:50:11.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.479+0000 7fa31d000640 1 -- 172.21.15.28:0/3645617344 wait complete. 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.480+0000 7fa31d000640 1 Processor -- start 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 -- start start 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa318072d30 0x7fa318144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31815b300 0x7fa3181e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180d0160 con 0x7fa3180cd3f0 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180cffe0 con 0x7fa31815b300 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.481+0000 7fa31d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3180b3c40 con 0x7fa318072d30 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47952/0 (socket says 172.21.15.28:47952) 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 -- 172.21.15.28:0/1124602893 learned_addr learned my addr 172.21.15.28:0/1124602893 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315e00640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa318072d30 0x7fa318144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa316800640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31815b300 0x7fa3181e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 -- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa318072d30 msgr2=0x7fa318144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa318072d30 0x7fa318144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 -- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31815b300 msgr2=0x7fa3181e6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31815b300 0x7fa3181e6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.482+0000 7fa315400640 1 -- 172.21.15.28:0/1124602893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3181450f0 con 0x7fa3180cd3f0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa315400640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa304000c00 tx=0x7fa3040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304041020 con 0x7fa3180cd3f0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30403d620 con 0x7fa3180cd3f0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304037040 con 0x7fa3180cd3f0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa318147610 con 0x7fa3180cd3f0 2024-09-10T10:50:11.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.483+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa318147bd0 con 0x7fa3180cd3f0 2024-09-10T10:50:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.484+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3180b2fc0 con 0x7fa3180cd3f0 2024-09-10T10:50:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.485+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa304044020 con 0x7fa3180cd3f0 2024-09-10T10:50:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.486+0000 7fa30e000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.486+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fa3040ab640 con 0x7fa3180cd3f0 2024-09-10T10:50:11.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.486+0000 7fa315e00640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:11.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.487+0000 7fa315e00640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 0x7fa2f806e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa308009aa0 tx=0x7fa308009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:11.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.490+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa30404b030 con 0x7fa3180cd3f0 2024-09-10T10:50:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.685+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa31806e250 con 0x7fa3180cd3f0 2024-09-10T10:50:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.685+0000 7fa30e000640 1 -- 172.21.15.28:0/1124602893 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v87) ==== 80+0+124 (secure 0 0 0) 0x7fa304079720 con 0x7fa3180cd3f0 2024-09-10T10:50:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6837/4263014325 2024-09-11T10:50:05.677188+0000 2024-09-10T10:50:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.138:6835/4263014325 2024-09-11T10:50:05.677188+0000 2024-09-10T10:50:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:listed 2 entries 2024-09-10T10:50:11.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 msgr2=0x7fa2f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:11.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 0x7fa2f806e2f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa308009aa0 tx=0x7fa308009190 comp rx=0 tx=0).stop 2024-09-10T10:50:11.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 msgr2=0x7fa3181e6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:11.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa304000c00 tx=0x7fa3040046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 shutdown_connections 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2f806be30 0x7fa2f806e2f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa31815b300 0x7fa3181e6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3180cd3f0 0x7fa3181e6780 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 --2- 172.21.15.28:0/1124602893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa318072d30 0x7fa318144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 >> 172.21.15.28:0/1124602893 conn(0x7fa318067d70 msgr2=0x7fa31815df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 shutdown_connections 2024-09-10T10:50:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:11.688+0000 7fa31d000640 1 -- 172.21.15.28:0/1124602893 wait complete. 2024-09-10T10:50:11.706 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:50:11.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:11.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:11.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:50:11.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:11.768 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:50:11.804 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:50:11.804 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:50:11.804 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:11.840 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:50:11.840 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:50:11.840 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:11.840 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:11.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:50:11.880 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:50:11.880 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:50:11.880 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:11.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:50:11.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:12.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.073+0000 7f639a400640 1 Processor -- start 2024-09-10T10:50:12.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.073+0000 7f639a400640 1 -- start start 2024-09-10T10:50:12.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f639415b300 0x7f63940cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63940cd3f0 0x7f639406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6394072d30 0x7f639406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f63940728a0 con 0x7f639415b300 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f63940cdbe0 con 0x7f6394072d30 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f639a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f63940cd900 con 0x7f63940cd3f0 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f639415b300 0x7f63940cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f639415b300 0x7f63940cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47964/0 (socket says 172.21.15.28:47964) 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393400640 1 -- 172.21.15.28:0/1361298032 learned_addr learned my addr 172.21.15.28:0/1361298032 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f638be00640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63940cd3f0 0x7f639406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393e00640 1 --2- 172.21.15.28:0/1361298032 >> v2:172.21.15.138:3300/0 conn(0x7f6394072d30 0x7f639406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393400640 1 -- 172.21.15.28:0/1361298032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63940cd3f0 msgr2=0x7f639406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.075+0000 7f6393400640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63940cd3f0 0x7f639406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.076+0000 7f6393400640 1 -- 172.21.15.28:0/1361298032 >> v2:172.21.15.138:3300/0 conn(0x7f6394072d30 msgr2=0x7f639406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:12.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.076+0000 7f6393400640 1 --2- 172.21.15.28:0/1361298032 >> v2:172.21.15.138:3300/0 conn(0x7f6394072d30 0x7f639406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.076+0000 7f6393400640 1 -- 172.21.15.28:0/1361298032 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6394072250 con 0x7f639415b300 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.076+0000 7f6393400640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f63940cceb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6394072040 tx=0x7f638402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fb009d21e13acd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f6392a00640 1 -- 172.21.15.28:0/1361298032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384037070 con 0x7f639415b300 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f6392a00640 1 -- 172.21.15.28:0/1361298032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6384032440 con 0x7f639415b300 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f6392a00640 1 -- 172.21.15.28:0/1361298032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384032810 con 0x7f639415b300 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 -- 172.21.15.28:0/1361298032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 msgr2=0x7f63940cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f63940cceb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6394072040 tx=0x7f638402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 -- 172.21.15.28:0/1361298032 shutdown_connections 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 --2- 172.21.15.28:0/1361298032 >> v2:172.21.15.138:3300/0 conn(0x7f6394072d30 0x7f639406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f63940cd3f0 0x7f639406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 --2- 172.21.15.28:0/1361298032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f63940cceb0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 -- 172.21.15.28:0/1361298032 >> 172.21.15.28:0/1361298032 conn(0x7f6394067d70 msgr2=0x7f639406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.077+0000 7f639a400640 1 -- 172.21.15.28:0/1361298032 shutdown_connections 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.078+0000 7f639a400640 1 -- 172.21.15.28:0/1361298032 wait complete. 2024-09-10T10:50:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.079+0000 7f639a400640 1 Processor -- start 2024-09-10T10:50:12.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.079+0000 7f639a400640 1 -- start start 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 0x7f6394156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63940cd3f0 0x7f63941f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f63940d0160 con 0x7f639415b300 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63940cffe0 con 0x7f63940cd3f0 2024-09-10T10:50:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f639a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f63940b3c40 con 0x7f6394072d30 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.080+0000 7f6393e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47978/0 (socket says 172.21.15.28:47978) 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f638be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63940cd3f0 0x7f63941f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 -- 172.21.15.28:0/1206056904 learned_addr learned my addr 172.21.15.28:0/1206056904 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 0x7f6394156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 -- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 msgr2=0x7f6394156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 0x7f6394156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 -- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63940cd3f0 msgr2=0x7f63941f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63940cd3f0 0x7f63941f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 -- 172.21.15.28:0/1206056904 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63941f9000 con 0x7f639415b300 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 0x7f6394156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:12.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.081+0000 7f6393e00640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f638c00eed0 tx=0x7f638c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.082+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c018020 con 0x7f639415b300 2024-09-10T10:50:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.082+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f638c014400 con 0x7f639415b300 2024-09-10T10:50:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.082+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f638c0147f0 con 0x7f639415b300 2024-09-10T10:50:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.082+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63941f5a70 con 0x7f639415b300 2024-09-10T10:50:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.082+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63941f5140 con 0x7f639415b300 2024-09-10T10:50:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.083+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639415b6e0 con 0x7f639415b300 2024-09-10T10:50:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.084+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f638c005d10 con 0x7f639415b300 2024-09-10T10:50:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.084+0000 7f638b400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 0x7f637006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.085+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f638c08b570 con 0x7f639415b300 2024-09-10T10:50:12.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.085+0000 7f6393400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 0x7f637006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.086+0000 7f6393400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 0x7f637006e320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6384034000 tx=0x7f638402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.089+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f638c022070 con 0x7f639415b300 2024-09-10T10:50:12.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.313+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63940c71b0 con 0x7f639415b300 2024-09-10T10:50:12.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.314+0000 7f638b400640 1 -- 172.21.15.28:0/1206056904 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5233 (secure 0 0 0) 0x7f63941f9000 con 0x7f639415b300 2024-09-10T10:50:12.316 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:12.317 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"btime":"2024-09-10T10:50:08:691575+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:08.691566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":13,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6408,"qdb_cluster":[6408]},"id":5}]} 2024-09-10T10:50:12.317 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 75 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 msgr2=0x7f637006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 0x7f637006e320 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6384034000 tx=0x7f638402e000 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 msgr2=0x7f639414fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f638c00eed0 tx=0x7f638c00b480 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 shutdown_connections 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f637006be60 0x7f637006e320 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f639415b300 0x7f639414fe50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63940cd3f0 0x7f63941f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 --2- 172.21.15.28:0/1206056904 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6394072d30 0x7f6394156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 >> 172.21.15.28:0/1206056904 conn(0x7f6394067d70 msgr2=0x7f63941597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 shutdown_connections 2024-09-10T10:50:12.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.316+0000 7f639a400640 1 -- 172.21.15.28:0/1206056904 wait complete. 2024-09-10T10:50:12.335 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:12.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.488+0000 7f6473000640 1 Processor -- start 2024-09-10T10:50:12.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.488+0000 7f6473000640 1 -- start start 2024-09-10T10:50:12.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6473000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 0x7f646c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6473000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f646c1585f0 0x7f646c072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6473000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c0b4330 con 0x7f646c0b5430 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6473000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f646c0b59b0 con 0x7f646c1585f0 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6473000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f646c0b3570 con 0x7f646c158210 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6463e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f646c1585f0 0x7f646c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f646be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 0x7f646c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6471000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f646c0b5430 0x7f646c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59312/0 (socket says 172.21.15.28:59312) 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6463e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f646c1585f0 0x7f646c072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47992/0 (socket says 172.21.15.28:47992) 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.490+0000 7f6471000640 1 -- 172.21.15.28:0/1572574303 learned_addr learned my addr 172.21.15.28:0/1572574303 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f6463e00640 1 -- 172.21.15.28:0/1572574303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 msgr2=0x7f646c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f6463e00640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 0x7f646c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f6463e00640 1 -- 172.21.15.28:0/1572574303 >> v1:172.21.15.28:6789/0 conn(0x7f646c0b5430 legacy=0x7f646c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f6463e00640 1 -- 172.21.15.28:0/1572574303 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c0b3850 con 0x7f646c1585f0 2024-09-10T10:50:12.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f646b400640 1 -- 172.21.15.28:0/1572574303 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 199331933 0 0) 0x7f646c0b4330 con 0x7f646c0b5430 2024-09-10T10:50:12.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.491+0000 7f646be00640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 0x7f646c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6463e00640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c1585f0 0x7f646c072940 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f64540097a0 tx=0x7f645402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68519fd5ecc9b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f646b400640 1 -- 172.21.15.28:0/1572574303 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454037070 con 0x7f646c1585f0 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f646b400640 1 -- 172.21.15.28:0/1572574303 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454032440 con 0x7f646c1585f0 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f646b400640 1 -- 172.21.15.28:0/1572574303 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454032810 con 0x7f646c1585f0 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 -- 172.21.15.28:0/1572574303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c1585f0 msgr2=0x7f646c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c1585f0 0x7f646c072940 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f64540097a0 tx=0x7f645402a180 comp rx=0 tx=0).stop 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 -- 172.21.15.28:0/1572574303 shutdown_connections 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c1585f0 0x7f646c072940 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 --2- 172.21.15.28:0/1572574303 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c158210 0x7f646c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.492+0000 7f6473000640 1 -- 172.21.15.28:0/1572574303 >> 172.21.15.28:0/1572574303 conn(0x7f646c067d70 msgr2=0x7f646c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:12.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.493+0000 7f6473000640 1 -- 172.21.15.28:0/1572574303 shutdown_connections 2024-09-10T10:50:12.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.493+0000 7f6473000640 1 -- 172.21.15.28:0/1572574303 wait complete. 2024-09-10T10:50:12.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.494+0000 7f6473000640 1 Processor -- start 2024-09-10T10:50:12.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.494+0000 7f6473000640 1 -- start start 2024-09-10T10:50:12.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 0x7f646c15c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 0x7f646c0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0cd0e0 con 0x7f646c158210 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0b3e10 con 0x7f646c0b5430 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6473000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f646c072e80 con 0x7f646c1585f0 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6463e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6463e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48008/0 (socket says 172.21.15.28:48008) 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.495+0000 7f6463e00640 1 -- 172.21.15.28:0/1346748931 learned_addr learned my addr 172.21.15.28:0/1346748931 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6471000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 0x7f646c0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f646be00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 0x7f646c15c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 -- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 msgr2=0x7f646c0b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 0x7f646c0b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 -- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 msgr2=0x7f646c15c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 0x7f646c15c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 -- 172.21.15.28:0/1346748931 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c143710 con 0x7f646c158210 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6471000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 0x7f646c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f646be00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 0x7f646c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6463e00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6454000c00 tx=0x7f64540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454037040 con 0x7f646c158210 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454042070 con 0x7f646c158210 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454030920 con 0x7f646c158210 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.496+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c0b12e0 con 0x7f646c158210 2024-09-10T10:50:12.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.497+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c1439b0 con 0x7f646c158210 2024-09-10T10:50:12.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.498+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c0b4ef0 con 0x7f646c158210 2024-09-10T10:50:12.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.498+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6454030ac0 con 0x7f646c158210 2024-09-10T10:50:12.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.500+0000 7f6468c00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 0x7f644806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.500+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f64540ab9e0 con 0x7f646c158210 2024-09-10T10:50:12.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.500+0000 7f646be00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 0x7f644806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.501+0000 7f646be00640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 0x7f644806e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f646c0b1be0 tx=0x7f645c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.504+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6454079b50 con 0x7f646c158210 2024-09-10T10:50:12.700 INFO:tasks.ceph.mds.d.smithi138.stderr:2024-09-10T10:50:12.699+0000 7fd29d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:12.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.725+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f646c06be60 con 0x7f646c158210 2024-09-10T10:50:12.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.726+0000 7f6468c00640 1 -- 172.21.15.28:0/1346748931 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5233 (secure 0 0 0) 0x7f645407de50 con 0x7f646c158210 2024-09-10T10:50:12.728 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:12.728 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"btime":"2024-09-10T10:50:08:691575+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:08.691566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":13,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6408,"qdb_cluster":[6408]},"id":5}]} 2024-09-10T10:50:12.728 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 75 2024-09-10T10:50:12.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 msgr2=0x7f644806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 0x7f644806e2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f646c0b1be0 tx=0x7f645c008040 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 msgr2=0x7f646c15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6454000c00 tx=0x7f64540046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 shutdown_connections 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f644806be30 0x7f644806e2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f646c1585f0 0x7f646c0b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f646c158210 0x7f646c15c770 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 --2- 172.21.15.28:0/1346748931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f646c0b5430 0x7f646c15c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 >> 172.21.15.28:0/1346748931 conn(0x7f646c067d70 msgr2=0x7f646c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:12.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.728+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 shutdown_connections 2024-09-10T10:50:12.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.729+0000 7f6473000640 1 -- 172.21.15.28:0/1346748931 wait complete. 2024-09-10T10:50:12.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.900+0000 7fc82ac00640 1 Processor -- start 2024-09-10T10:50:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.900+0000 7fc82ac00640 1 -- start start 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.901+0000 7fc82ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc8240d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.901+0000 7fc82ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc82406dbd0 0x7fc82406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.901+0000 7fc82ac00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc82406e040 con 0x7fc82409b690 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.901+0000 7fc82ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc82415eb20 con 0x7fc82406dbd0 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.901+0000 7fc82ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc82406d260 con 0x7fc82415e740 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc82406dbd0 0x7fc82406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc82406dbd0 0x7fc82406cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48018/0 (socket says 172.21.15.28:48018) 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 -- 172.21.15.28:0/3080163734 learned_addr learned my addr 172.21.15.28:0/3080163734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc823e00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc8240d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 -- 172.21.15.28:0/3080163734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 msgr2=0x7fc8240d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc8240d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 -- 172.21.15.28:0/3080163734 >> v1:172.21.15.138:6789/0 conn(0x7fc82409b690 legacy=0x7fc82406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.902+0000 7fc828c00640 1 -- 172.21.15.28:0/3080163734 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8240d0780 con 0x7fc82406dbd0 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.903+0000 7fc828c00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82406dbd0 0x7fc82406cd20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009e80 tx=0x7fc81c02a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e758f0db630476a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.903+0000 7fc822a00640 1 -- 172.21.15.28:0/3080163734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c036070 con 0x7fc82406dbd0 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.903+0000 7fc822a00640 1 -- 172.21.15.28:0/3080163734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81c031600 con 0x7fc82406dbd0 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.903+0000 7fc822a00640 1 -- 172.21.15.28:0/3080163734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c0319d0 con 0x7fc82406dbd0 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 -- 172.21.15.28:0/3080163734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82406dbd0 msgr2=0x7fc82406cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82406dbd0 0x7fc82406cd20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009e80 tx=0x7fc81c02a7e0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 -- 172.21.15.28:0/3080163734 shutdown_connections 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82406dbd0 0x7fc82406cd20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 --2- 172.21.15.28:0/3080163734 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc8240d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 -- 172.21.15.28:0/3080163734 >> 172.21.15.28:0/3080163734 conn(0x7fc8240d23a0 msgr2=0x7fc8240d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 -- 172.21.15.28:0/3080163734 shutdown_connections 2024-09-10T10:50:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.904+0000 7fc82ac00640 1 -- 172.21.15.28:0/3080163734 wait complete. 2024-09-10T10:50:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.906+0000 7fc82ac00640 1 Processor -- start 2024-09-10T10:50:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.906+0000 7fc82ac00640 1 -- start start 2024-09-10T10:50:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82406dbd0 0x7fc8240b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc82415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240d1020 con 0x7fc82409b690 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8240d0d40 con 0x7fc82406dbd0 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc82ac00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc82409ba90 con 0x7fc82415e740 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc823400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc823400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48028/0 (socket says 172.21.15.28:48028) 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc823400640 1 -- 172.21.15.28:0/2954316144 learned_addr learned my addr 172.21.15.28:0/2954316144 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc823e00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82406dbd0 0x7fc8240b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.907+0000 7fc828c00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc82415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 -- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 msgr2=0x7fc82415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc82415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 -- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82406dbd0 msgr2=0x7fc8240b79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82406dbd0 0x7fc8240b79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 -- 172.21.15.28:0/2954316144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8240c7f80 con 0x7fc82409b690 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc823400640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c440 tx=0x7fc81000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100052d0 con 0x7fc82409b690 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc810005470 con 0x7fc82409b690 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810007260 con 0x7fc82409b690 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8240b5320 con 0x7fc82409b690 2024-09-10T10:50:12.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.908+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8240b02c0 con 0x7fc82409b690 2024-09-10T10:50:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.909+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82406cd20 con 0x7fc82409b690 2024-09-10T10:50:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.910+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8100074c0 con 0x7fc82409b690 2024-09-10T10:50:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.911+0000 7fc81be00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 0x7fc80406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:12.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.911+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fc81008a470 con 0x7fc82409b690 2024-09-10T10:50:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.911+0000 7fc823e00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 0x7fc80406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:12.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.912+0000 7fc823e00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 0x7fc80406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc814009ad0 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:12.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:12.915+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc810058c40 con 0x7fc82409b690 2024-09-10T10:50:13.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.098+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8240d1cb0 con 0x7fc82409b690 2024-09-10T10:50:13.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.100+0000 7fc81be00640 1 -- 172.21.15.28:0/2954316144 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+17100 (secure 0 0 0) 0x7fc810058540 con 0x7fc82409b690 2024-09-10T10:50:13.102 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:13.102 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":87,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:05.677254+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:49:29.344267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T10:49:30.537368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6837/4263014325":"2024-09-11T10:50:05.677188+0000","172.21.15.138:6835/4263014325":"2024-09-11T10:50:05.677188+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 msgr2=0x7fc80406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 0x7fc80406e2f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc814009ad0 tx=0x7fc814008040 comp rx=0 tx=0).stop 2024-09-10T10:50:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 msgr2=0x7fc8240c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc81000c440 tx=0x7fc81000c900 comp rx=0 tx=0).stop 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 shutdown_connections 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc80406be30 0x7fc80406e2f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc82415e740 0x7fc82415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc82409b690 0x7fc8240c78f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 --2- 172.21.15.28:0/2954316144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc82406dbd0 0x7fc8240b79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 >> 172.21.15.28:0/2954316144 conn(0x7fc8240d23a0 msgr2=0x7fc8241e7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 shutdown_connections 2024-09-10T10:50:13.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.102+0000 7fc82ac00640 1 -- 172.21.15.28:0/2954316144 wait complete. 2024-09-10T10:50:13.120 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:50:13.120 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:13.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.275+0000 7fa2ace00640 1 Processor -- start 2024-09-10T10:50:13.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.276+0000 7fa2ace00640 1 -- start start 2024-09-10T10:50:13.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 0x7fa2a80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa2ace00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a8146140 con 0x7fa2a8072de0 2024-09-10T10:50:13.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa2ace00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a806eed0 con 0x7fa2a815e1f0 2024-09-10T10:50:13.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa2ace00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80cd740 con 0x7fa2a8072320 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa29fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa2a8072de0 0x7fa2a806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59320/0 (socket says 172.21.15.28:59320) 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa29fe00640 1 -- 172.21.15.28:0/678773744 learned_addr learned my addr 172.21.15.28:0/678773744 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.277+0000 7fa2a5e00640 1 --2- 172.21.15.28:0/678773744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 0x7fa2a80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 72706832 0 0) 0x7fa2a8146140 con 0x7fa2a8072de0 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa28c003520 con 0x7fa2a8072de0 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139104441 0 0) 0x7fa2a806eed0 con 0x7fa2a815e1f0 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2a8146140 con 0x7fa2a815e1f0 2024-09-10T10:50:13.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5e00640 1 -- 172.21.15.28:0/678773744 >> v1:172.21.15.138:6789/0 conn(0x7fa2a815e1f0 legacy=0x7fa2a806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5e00640 1 -- 172.21.15.28:0/678773744 >> v1:172.21.15.28:6789/0 conn(0x7fa2a8072de0 legacy=0x7fa2a806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.278+0000 7fa2a5e00640 1 -- 172.21.15.28:0/678773744 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a80cda20 con 0x7fa2a8072320 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2a5e00640 1 --2- 172.21.15.28:0/678773744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 0x7fa2a80cce60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa29802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4dc4426de9130c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298037070 con 0x7fa2a8072320 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa298032440 con 0x7fa2a8072320 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2a5400640 1 -- 172.21.15.28:0/678773744 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298032810 con 0x7fa2a8072320 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2ace00640 1 -- 172.21.15.28:0/678773744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 msgr2=0x7fa2a80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.279+0000 7fa2ace00640 1 --2- 172.21.15.28:0/678773744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 0x7fa2a80cce60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa29802a180 comp rx=0 tx=0).stop 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.280+0000 7fa2ace00640 1 -- 172.21.15.28:0/678773744 shutdown_connections 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.280+0000 7fa2ace00640 1 --2- 172.21.15.28:0/678773744 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072320 0x7fa2a80cce60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.280+0000 7fa2ace00640 1 -- 172.21.15.28:0/678773744 >> 172.21.15.28:0/678773744 conn(0x7fa2a8067d70 msgr2=0x7fa2a806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.280+0000 7fa2ace00640 1 -- 172.21.15.28:0/678773744 shutdown_connections 2024-09-10T10:50:13.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.280+0000 7fa2ace00640 1 -- 172.21.15.28:0/678773744 wait complete. 2024-09-10T10:50:13.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.281+0000 7fa2ace00640 1 Processor -- start 2024-09-10T10:50:13.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.281+0000 7fa2ace00640 1 -- start start 2024-09-10T10:50:13.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072de0 0x7fa2a81e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 0x7fa2a80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a80cd3a0 con 0x7fa2a8072320 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a815c9e0 con 0x7fa2a815e1f0 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.282+0000 7fa2ace00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a80cd520 con 0x7fa2a8072de0 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48032/0 (socket says 172.21.15.28:48032) 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 -- 172.21.15.28:0/46038527 learned_addr learned my addr 172.21.15.28:0/46038527 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a6800640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 0x7fa2a80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa29fe00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072de0 0x7fa2a81e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 -- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072de0 msgr2=0x7fa2a81e33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072de0 0x7fa2a81e33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 -- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 msgr2=0x7fa2a80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 0x7fa2a80cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 -- 172.21.15.28:0/46038527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a81e3d30 con 0x7fa2a8072320 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a6800640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 0x7fa2a80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.283+0000 7fa2a5e00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.284+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298041020 con 0x7fa2a8072320 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.284+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29803d6e0 con 0x7fa2a8072320 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.284+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298037040 con 0x7fa2a8072320 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.284+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81e4010 con 0x7fa2a8072320 2024-09-10T10:50:13.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.284+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81e45d0 con 0x7fa2a8072320 2024-09-10T10:50:13.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.285+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8072700 con 0x7fa2a8072320 2024-09-10T10:50:13.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.286+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa298044020 con 0x7fa2a8072320 2024-09-10T10:50:13.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.286+0000 7fa29e000640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.287+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fa2980ab890 con 0x7fa2a8072320 2024-09-10T10:50:13.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.287+0000 7fa29fe00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.287+0000 7fa29fe00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 0x7fa28406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8143f10 tx=0x7fa290009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.291+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa29804b030 con 0x7fa2a8072320 2024-09-10T10:50:13.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.509+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2a80b4240 con 0x7fa2a8072320 2024-09-10T10:50:13.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.510+0000 7fa29e000640 1 -- 172.21.15.28:0/46038527 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5233 (secure 0 0 0) 0x7fa298079fd0 con 0x7fa2a8072320 2024-09-10T10:50:13.512 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:13.512 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"btime":"2024-09-10T10:50:08:691575+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:08.691566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":13,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6408,"qdb_cluster":[6408]},"id":5}]} 2024-09-10T10:50:13.513 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 75 2024-09-10T10:50:13.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 msgr2=0x7fa28406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 0x7fa28406e2f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8143f10 tx=0x7fa290009040 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 msgr2=0x7fa2a815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa2980097d0 tx=0x7fa2980046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 shutdown_connections 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa28406be30 0x7fa28406e2f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a815e1f0 0x7fa2a80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a8072de0 0x7fa2a81e33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 --2- 172.21.15.28:0/46038527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2a8072320 0x7fa2a815be50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 >> 172.21.15.28:0/46038527 conn(0x7fa2a8067d70 msgr2=0x7fa2a806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 shutdown_connections 2024-09-10T10:50:13.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.512+0000 7fa2ace00640 1 -- 172.21.15.28:0/46038527 wait complete. 2024-09-10T10:50:13.530 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:13.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.683+0000 7fa72ec00640 1 Processor -- start 2024-09-10T10:50:13.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.683+0000 7fa72ec00640 1 -- start start 2024-09-10T10:50:13.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.684+0000 7fa72ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa728072320 0x7fa7280cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.684+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa72806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.684+0000 7fa72ec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa728146140 con 0x7fa728072de0 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.684+0000 7fa72ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7280cd740 con 0x7fa728072320 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.684+0000 7fa72ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280cda20 con 0x7fa72815e1f0 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa728072de0 0x7fa72806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59330/0 (socket says 172.21.15.28:59330) 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727400640 1 -- 172.21.15.28:0/1621265895 learned_addr learned my addr 172.21.15.28:0/1621265895 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727e00640 1 --2- 172.21.15.28:0/1621265895 >> v2:172.21.15.138:3300/0 conn(0x7fa728072320 0x7fa7280cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa72cc00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa72806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa726a00640 1 -- 172.21.15.28:0/1621265895 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 546132844 0 0) 0x7fa728146140 con 0x7fa728072de0 2024-09-10T10:50:13.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa726a00640 1 -- 172.21.15.28:0/1621265895 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70c003520 con 0x7fa728072de0 2024-09-10T10:50:13.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727e00640 1 -- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 msgr2=0x7fa72806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:13.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727e00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa72806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727e00640 1 -- 172.21.15.28:0/1621265895 >> v1:172.21.15.28:6789/0 conn(0x7fa728072de0 legacy=0x7fa72806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.685+0000 7fa727e00640 1 -- 172.21.15.28:0/1621265895 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280cdd00 con 0x7fa728072320 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.686+0000 7fa727e00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7280cce60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097a0 tx=0x7fa71802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4d564eed858f54d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa726a00640 1 -- 172.21.15.28:0/1621265895 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718037070 con 0x7fa728072320 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa726a00640 1 -- 172.21.15.28:0/1621265895 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71802ae50 con 0x7fa728072320 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa726a00640 1 -- 172.21.15.28:0/1621265895 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718032660 con 0x7fa728072320 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 -- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 msgr2=0x7fa7280cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7280cce60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097a0 tx=0x7fa71802a100 comp rx=0 tx=0).stop 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 -- 172.21.15.28:0/1621265895 shutdown_connections 2024-09-10T10:50:13.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa72806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 --2- 172.21.15.28:0/1621265895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7280cce60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 -- 172.21.15.28:0/1621265895 >> 172.21.15.28:0/1621265895 conn(0x7fa728067d70 msgr2=0x7fa72806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.687+0000 7fa72ec00640 1 -- 172.21.15.28:0/1621265895 shutdown_connections 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.688+0000 7fa72ec00640 1 -- 172.21.15.28:0/1621265895 wait complete. 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.689+0000 7fa72ec00640 1 Processor -- start 2024-09-10T10:50:13.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.689+0000 7fa72ec00640 1 -- start start 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7281e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa7281e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280cd520 con 0x7fa728072de0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280cd3a0 con 0x7fa728072320 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa72ec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa72806ed00 con 0x7fa72815e1f0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.690+0000 7fa727400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48036/0 (socket says 172.21.15.28:48036) 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 -- 172.21.15.28:0/559091698 learned_addr learned my addr 172.21.15.28:0/559091698 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa72cc00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa7281e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727e00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7281e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 -- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 msgr2=0x7fa7281e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa7281e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 -- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 msgr2=0x7fa7281e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7281e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 -- 172.21.15.28:0/559091698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa728136080 con 0x7fa728072de0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa727400640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c430 tx=0x7fa71400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.691+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714017070 con 0x7fa728072de0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.692+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714005ce0 con 0x7fa728072de0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.692+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140072f0 con 0x7fa728072de0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.692+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7281e7c40 con 0x7fa728072de0 2024-09-10T10:50:13.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.692+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7281e8200 con 0x7fa728072de0 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.693+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa728072700 con 0x7fa728072de0 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.694+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa71401a070 con 0x7fa728072de0 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.694+0000 7fa71fe00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 0x7fa70806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.695+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fa71408a490 con 0x7fa728072de0 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.695+0000 7fa727e00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 0x7fa70806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.696+0000 7fa727e00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 0x7fa70806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa718034000 tx=0x7fa71802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:13.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.699+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa714058610 con 0x7fa728072de0 2024-09-10T10:50:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.917+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7281e3840 con 0x7fa728072de0 2024-09-10T10:50:13.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.918+0000 7fa71fe00640 1 -- 172.21.15.28:0/559091698 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5233 (secure 0 0 0) 0x7fa71405c910 con 0x7fa728072de0 2024-09-10T10:50:13.920 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:13.920 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":75,"btime":"2024-09-10T10:50:08:691575+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:49:31.540549+0000","modified":"2024-09-10T10:50:08.691566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":87,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6408},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6408":{"gid":6408,"name":"d","rank":0,"incarnation":72,"state":"up:active","state_seq":13,"addr":"172.21.15.138:6836/1950306159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1950306159},{"type":"v1","addr":"172.21.15.138:6836","nonce":1950306159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6408,"qdb_cluster":[6408]},"id":5}]} 2024-09-10T10:50:13.920 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 75 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 msgr2=0x7fa70806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 0x7fa70806e2f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa718034000 tx=0x7fa71802e000 comp rx=0 tx=0).stop 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 msgr2=0x7fa728135b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c430 tx=0x7fa71400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 shutdown_connections 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa70806be30 0x7fa70806e2f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa72815e1f0 0x7fa7281e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa728072de0 0x7fa728135b40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 --2- 172.21.15.28:0/559091698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa728072320 0x7fa7281e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 >> 172.21.15.28:0/559091698 conn(0x7fa728067d70 msgr2=0x7fa72806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.920+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 shutdown_connections 2024-09-10T10:50:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:13.921+0000 7fa72ec00640 1 -- 172.21.15.28:0/559091698 wait complete. 2024-09-10T10:50:13.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.092+0000 7fd4bae00640 1 Processor -- start 2024-09-10T10:50:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.092+0000 7fd4bae00640 1 -- start start 2024-09-10T10:50:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.093+0000 7fd4bae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4b4072d30 0x7fd4b406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.093+0000 7fd4bae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4b406f700 0x7fd4b415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.093+0000 7fd4bae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4b406fae0 con 0x7fd4b415b300 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.093+0000 7fd4bae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b40728a0 con 0x7fd4b4072d30 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.093+0000 7fd4bae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4b40cdbe0 con 0x7fd4b406f700 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4b4072d30 0x7fd4b406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4b4072d30 0x7fd4b406f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41502/0 (socket says 172.21.15.28:41502) 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 -- 172.21.15.28:0/4273709156 learned_addr learned my addr 172.21.15.28:0/4273709156 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b8e00640 1 --2- 172.21.15.28:0/4273709156 >> v2:172.21.15.138:3300/0 conn(0x7fd4b406f700 0x7fd4b415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 -- 172.21.15.28:0/4273709156 >> v2:172.21.15.138:3300/0 conn(0x7fd4b406f700 msgr2=0x7fd4b415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 --2- 172.21.15.28:0/4273709156 >> v2:172.21.15.138:3300/0 conn(0x7fd4b406f700 0x7fd4b415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 -- 172.21.15.28:0/4273709156 >> v1:172.21.15.138:6789/0 conn(0x7fd4b415b300 legacy=0x7fd4b40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.094+0000 7fd4b3400640 1 -- 172.21.15.28:0/4273709156 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b40cd900 con 0x7fd4b4072d30 2024-09-10T10:50:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.095+0000 7fd4b3400640 1 --2- 172.21.15.28:0/4273709156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b406f1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40b38f0 tx=0x7fd4a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db253920e501d7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.095+0000 7fd4b2a00640 1 -- 172.21.15.28:0/4273709156 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0037070 con 0x7fd4b4072d30 2024-09-10T10:50:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.095+0000 7fd4b2a00640 1 -- 172.21.15.28:0/4273709156 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a0032440 con 0x7fd4b4072d30 2024-09-10T10:50:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.095+0000 7fd4b2a00640 1 -- 172.21.15.28:0/4273709156 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0032810 con 0x7fd4b4072d30 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 -- 172.21.15.28:0/4273709156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 msgr2=0x7fd4b406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 --2- 172.21.15.28:0/4273709156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b406f1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd4b40b38f0 tx=0x7fd4a002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 -- 172.21.15.28:0/4273709156 shutdown_connections 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 --2- 172.21.15.28:0/4273709156 >> v2:172.21.15.138:3300/0 conn(0x7fd4b406f700 0x7fd4b415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 --2- 172.21.15.28:0/4273709156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b406f1c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 -- 172.21.15.28:0/4273709156 >> 172.21.15.28:0/4273709156 conn(0x7fd4b4067d70 msgr2=0x7fd4b406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 -- 172.21.15.28:0/4273709156 shutdown_connections 2024-09-10T10:50:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.096+0000 7fd4bae00640 1 -- 172.21.15.28:0/4273709156 wait complete. 2024-09-10T10:50:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.098+0000 7fd4bae00640 1 Processor -- start 2024-09-10T10:50:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.098+0000 7fd4bae00640 1 -- start start 2024-09-10T10:50:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b406f700 0x7fd4b414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 0x7fd4b4159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b40cd5c0 con 0x7fd4b4072d30 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b4072530 con 0x7fd4b415b300 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4bae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4b40cffe0 con 0x7fd4b406f700 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4b3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4b3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41518/0 (socket says 172.21.15.28:41518) 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4b3400640 1 -- 172.21.15.28:0/3645478836 learned_addr learned my addr 172.21.15.28:0/3645478836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4b3e00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b406f700 0x7fd4b414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.099+0000 7fd4b8e00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 0x7fd4b4159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 -- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b406f700 msgr2=0x7fd4b414e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b406f700 0x7fd4b414e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 -- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 msgr2=0x7fd4b4159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 0x7fd4b4159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 -- 172.21.15.28:0/3645478836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b414e5f0 con 0x7fd4b4072d30 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b8e00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 0x7fd4b4159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4b3400640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0041020 con 0x7fd4b4072d30 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a00306c0 con 0x7fd4b4072d30 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0037040 con 0x7fd4b4072d30 2024-09-10T10:50:14.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b415c680 con 0x7fd4b4072d30 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.100+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b415cc40 con 0x7fd4b4072d30 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.101+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b406f1c0 con 0x7fd4b4072d30 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.102+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4a0044020 con 0x7fd4b4072d30 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.103+0000 7fd4abe00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 0x7fd49406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.103+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7fd4a00ab850 con 0x7fd4b4072d30 2024-09-10T10:50:14.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.104+0000 7fd4b3e00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 0x7fd49406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.105+0000 7fd4b3e00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 0x7fd49406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4004f60 tx=0x7fd4a4007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.108+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4a004b030 con 0x7fd4b4072d30 2024-09-10T10:50:14.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.291+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4b40cd1e0 con 0x7fd4b4072d30 2024-09-10T10:50:14.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.292+0000 7fd4abe00640 1 -- 172.21.15.28:0/3645478836 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+17100 (secure 0 0 0) 0x7fd4a007a010 con 0x7fd4b4072d30 2024-09-10T10:50:14.294 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:14.294 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":87,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:05.677254+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:49:29.344267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-10T10:49:30.537368+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6837/4263014325":"2024-09-11T10:50:05.677188+0000","172.21.15.138:6835/4263014325":"2024-09-11T10:50:05.677188+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:14.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 msgr2=0x7fd49406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 0x7fd49406e2f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4004f60 tx=0x7fd4a4007e30 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 msgr2=0x7fd4b4159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0000c00 tx=0x7fd4a00046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 shutdown_connections 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd49406be30 0x7fd49406e2f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4b415b300 0x7fd4b4159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4b4072d30 0x7fd4b4159990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 --2- 172.21.15.28:0/3645478836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4b406f700 0x7fd4b414e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 >> 172.21.15.28:0/3645478836 conn(0x7fd4b4067d70 msgr2=0x7fd4b41e82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 shutdown_connections 2024-09-10T10:50:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.294+0000 7fd4bae00640 1 -- 172.21.15.28:0/3645478836 wait complete. 2024-09-10T10:50:14.312 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:50:14.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.468+0000 7f3cd5600640 1 Processor -- start 2024-09-10T10:50:14.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.469+0000 7f3cd5600640 1 -- start start 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cd5600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3cd0072700 0x7f3cd015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cd5600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd0146140 con 0x7f3cd0072de0 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cd5600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd00cd740 con 0x7f3cd0072320 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cd5600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd00cda20 con 0x7f3cd0072700 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cc6000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3cd0072de0 0x7f3cd015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37024/0 (socket says 172.21.15.28:37024) 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3cc6000640 1 -- 172.21.15.28:0/1318047058 learned_addr learned my addr 172.21.15.28:0/1318047058 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:14.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.470+0000 7f3ccf400640 1 --2- 172.21.15.28:0/1318047058 >> v2:172.21.15.138:3300/0 conn(0x7f3cd0072700 0x7f3cd015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892500703 0 0) 0x7f3cd0146140 con 0x7f3cd0072de0 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3cb4003520 con 0x7f3cd0072de0 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3ccf400640 1 -- 172.21.15.28:0/1318047058 >> v1:172.21.15.138:6789/0 conn(0x7f3cd0072320 legacy=0x7f3cd00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3280642264 0 0) 0x7f3cd00cd740 con 0x7f3cd0072320 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3ccf400640 1 -- 172.21.15.28:0/1318047058 >> v1:172.21.15.28:6789/0 conn(0x7f3cd0072de0 legacy=0x7f3cd015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.471+0000 7f3ccf400640 1 -- 172.21.15.28:0/1318047058 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd00cdd00 con 0x7f3cd0072700 2024-09-10T10:50:14.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.472+0000 7f3ccf400640 1 --2- 172.21.15.28:0/1318047058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd015d8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d90 tx=0x7f3cc802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b810265af84e714c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.472+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8007740 con 0x7f3cd0072700 2024-09-10T10:50:14.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.472+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc80078e0 con 0x7f3cd0072700 2024-09-10T10:50:14.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.472+0000 7f3cce000640 1 -- 172.21.15.28:0/1318047058 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8007c90 con 0x7f3cd0072700 2024-09-10T10:50:14.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 -- 172.21.15.28:0/1318047058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 msgr2=0x7f3cd015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 --2- 172.21.15.28:0/1318047058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd015d8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009d90 tx=0x7f3cc802a660 comp rx=0 tx=0).stop 2024-09-10T10:50:14.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 -- 172.21.15.28:0/1318047058 shutdown_connections 2024-09-10T10:50:14.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 --2- 172.21.15.28:0/1318047058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd015d8f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 -- 172.21.15.28:0/1318047058 >> 172.21.15.28:0/1318047058 conn(0x7f3cd0067d70 msgr2=0x7f3cd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 -- 172.21.15.28:0/1318047058 shutdown_connections 2024-09-10T10:50:14.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.473+0000 7f3cd5600640 1 -- 172.21.15.28:0/1318047058 wait complete. 2024-09-10T10:50:14.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.475+0000 7f3cd5600640 1 Processor -- start 2024-09-10T10:50:14.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.475+0000 7f3cd5600640 1 -- start start 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.475+0000 7f3cd5600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cd0072320 0x7f3cd015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cd5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd00aa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cd5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cd5600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd015dfb0 con 0x7f3cd0072de0 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cd5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd015de30 con 0x7f3cd0072700 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cd5600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd015e130 con 0x7f3cd0072320 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41520/0 (socket says 172.21.15.28:41520) 2024-09-10T10:50:14.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 -- 172.21.15.28:0/2946889653 learned_addr learned my addr 172.21.15.28:0/2946889653 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3cc6000640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd00aa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccea00640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cd0072320 0x7f3cd015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 -- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cd0072320 msgr2=0x7f3cd015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cd0072320 0x7f3cd015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 -- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 msgr2=0x7f3cd00aa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd00aa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.476+0000 7f3ccf400640 1 -- 172.21.15.28:0/2946889653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd00ab730 con 0x7f3cd0072de0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3ccf400640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009eb0 tx=0x7f3cc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8040070 con 0x7f3cd0072de0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc8031bc0 con 0x7f3cd0072de0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8009450 con 0x7f3cd0072de0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd00aba10 con 0x7f3cd0072de0 2024-09-10T10:50:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.477+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd00ac2b0 con 0x7f3cd0072de0 2024-09-10T10:50:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.478+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd00abe00 con 0x7f3cd0072de0 2024-09-10T10:50:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.479+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cc8039040 con 0x7f3cd0072de0 2024-09-10T10:50:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.480+0000 7f3cc7400640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.480+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 6187+0+0 (secure 0 0 0) 0x7f3cc80ac380 con 0x7f3cd0072de0 2024-09-10T10:50:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.480+0000 7f3ccea00640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.481+0000 7f3ccea00640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 0x7f3cac06e320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000f60 tx=0x7f3cc0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.484+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3cc807a4f0 con 0x7f3cd0072de0 2024-09-10T10:50:14.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.703+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3cd00cc4b0 con 0x7f3cd0072de0 2024-09-10T10:50:14.705 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:50:14.703+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:50:14.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.705+0000 7f3cc7400640 1 -- 172.21.15.28:0/2946889653 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f3cd00cc4b0 con 0x7f3cd0072de0 2024-09-10T10:50:14.707 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:50:14.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 msgr2=0x7f3cac06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 0x7f3cac06e320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0000f60 tx=0x7f3cc0009040 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 msgr2=0x7f3cd01e53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8009eb0 tx=0x7f3cc8004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 shutdown_connections 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cac06be60 0x7f3cac06e320 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd0072de0 0x7f3cd01e53a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd0072700 0x7f3cd00aa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 --2- 172.21.15.28:0/2946889653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3cd0072320 0x7f3cd015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 >> 172.21.15.28:0/2946889653 conn(0x7f3cd0067d70 msgr2=0x7f3cd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 shutdown_connections 2024-09-10T10:50:14.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.708+0000 7f3cd5600640 1 -- 172.21.15.28:0/2946889653 wait complete. 2024-09-10T10:50:14.725 INFO:tasks.ceph.mds.d.smithi138.stderr: -67> 2024-09-10T10:50:12.699+0000 7fd29d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:14.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:50:14.768 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:50:14.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.885+0000 7fbea3400640 1 Processor -- start 2024-09-10T10:50:14.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.885+0000 7fbea3400640 1 -- start start 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe9c15b720 0x7fbe9c15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe9c0d0370 0x7fbe9c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea3400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9c0ae4e0 con 0x7fbe9c155930 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea3400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c0d0c90 con 0x7fbe9c15b720 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea3400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9c15c1e0 con 0x7fbe9c0d0370 2024-09-10T10:50:14.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbe9be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe9c15b720 0x7fbe9c15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea0a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbe9c155930 0x7fbe9c0addd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37030/0 (socket says 172.21.15.28:37030) 2024-09-10T10:50:14.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea0a00640 1 -- 172.21.15.28:0/3736381029 learned_addr learned my addr 172.21.15.28:0/3736381029 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:14.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.886+0000 7fbea1400640 1 --2- 172.21.15.28:0/3736381029 >> v2:172.21.15.138:3300/0 conn(0x7fbe9c0d0370 0x7fbe9c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.887+0000 7fbe9be00640 1 -- 172.21.15.28:0/3736381029 >> v2:172.21.15.138:3300/0 conn(0x7fbe9c0d0370 msgr2=0x7fbe9c0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.887+0000 7fbe9be00640 1 --2- 172.21.15.28:0/3736381029 >> v2:172.21.15.138:3300/0 conn(0x7fbe9c0d0370 0x7fbe9c0d0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.887+0000 7fbe9be00640 1 -- 172.21.15.28:0/3736381029 >> v1:172.21.15.28:6789/0 conn(0x7fbe9c155930 legacy=0x7fbe9c0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.887+0000 7fbe9b400640 1 -- 172.21.15.28:0/3736381029 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3529884876 0 0) 0x7fbe90001110 con 0x7fbe9c155930 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.887+0000 7fbe9be00640 1 -- 172.21.15.28:0/3736381029 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c15c4c0 con 0x7fbe9c15b720 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbe9be00640 1 --2- 172.21.15.28:0/3736381029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c15b720 0x7fbe9c15bbb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0d0f30 tx=0x7fbe8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afed55b0cd7719d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbe9b400640 1 -- 172.21.15.28:0/3736381029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88037070 con 0x7fbe9c15b720 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbe9b400640 1 -- 172.21.15.28:0/3736381029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe88032440 con 0x7fbe9c15b720 2024-09-10T10:50:14.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbe9b400640 1 -- 172.21.15.28:0/3736381029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88032810 con 0x7fbe9c15b720 2024-09-10T10:50:14.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbea3400640 1 -- 172.21.15.28:0/3736381029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c15b720 msgr2=0x7fbe9c15bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.888+0000 7fbea3400640 1 --2- 172.21.15.28:0/3736381029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c15b720 0x7fbe9c15bbb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0d0f30 tx=0x7fbe8802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 -- 172.21.15.28:0/3736381029 shutdown_connections 2024-09-10T10:50:14.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 --2- 172.21.15.28:0/3736381029 >> v2:172.21.15.138:3300/0 conn(0x7fbe9c0d0370 0x7fbe9c0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 --2- 172.21.15.28:0/3736381029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c15b720 0x7fbe9c15bbb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 -- 172.21.15.28:0/3736381029 >> 172.21.15.28:0/3736381029 conn(0x7fbe9c06bdf0 msgr2=0x7fbe9c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:14.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 -- 172.21.15.28:0/3736381029 shutdown_connections 2024-09-10T10:50:14.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.889+0000 7fbea3400640 1 -- 172.21.15.28:0/3736381029 wait complete. 2024-09-10T10:50:14.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.890+0000 7fbea3400640 1 Processor -- start 2024-09-10T10:50:14.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.890+0000 7fbea3400640 1 -- start start 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c0d0370 0x7fbe9c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c15b720 0x7fbe9c1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c06f8b0 con 0x7fbe9c155930 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c0b3620 con 0x7fbe9c0d0370 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbea3400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9c06fa30 con 0x7fbe9c15b720 2024-09-10T10:50:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.891+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41530/0 (socket says 172.21.15.28:41530) 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 -- 172.21.15.28:0/2086747521 learned_addr learned my addr 172.21.15.28:0/2086747521 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbea1400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c15b720 0x7fbe9c1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbea0a00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c0d0370 0x7fbe9c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 -- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c15b720 msgr2=0x7fbe9c1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c15b720 0x7fbe9c1f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 -- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c0d0370 msgr2=0x7fbe9c153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c0d0370 0x7fbe9c153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 -- 172.21.15.28:0/2086747521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1f85d0 con 0x7fbe9c155930 2024-09-10T10:50:14.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.892+0000 7fbe9be00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.893+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88037050 con 0x7fbe9c155930 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.893+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe88038040 con 0x7fbe9c155930 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.893+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88004470 con 0x7fbe9c155930 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.893+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1f88b0 con 0x7fbe9c155930 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.893+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1f4b30 con 0x7fbe9c155930 2024-09-10T10:50:14.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.894+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9c1f8a80 con 0x7fbe9c155930 2024-09-10T10:50:14.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.895+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe88030900 con 0x7fbe9c155930 2024-09-10T10:50:14.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.896+0000 7fbe98c00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 0x7fbe7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.896+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6273+0+0 (secure 0 0 0) 0x7fbe880abdb0 con 0x7fbe9c155930 2024-09-10T10:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.896+0000 7fbea0a00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 0x7fbe7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:14.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.897+0000 7fbea0a00640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 0x7fbe7c06e320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe90000f60 tx=0x7fbe90007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:14.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:14.900+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe880b0070 con 0x7fbe9c155930 2024-09-10T10:50:15.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.121+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbe9c0cc460 con 0x7fbe9c155930 2024-09-10T10:50:15.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.722+0000 7fbe98c00640 1 -- 172.21.15.28:0/2086747521 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7fbe9c0cc460 con 0x7fbe9c155930 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 msgr2=0x7fbe7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 0x7fbe7c06e320 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe90000f60 tx=0x7fbe90007970 comp rx=0 tx=0).stop 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 msgr2=0x7fbe9c1e4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fbe88000c00 tx=0x7fbe880046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 shutdown_connections 2024-09-10T10:50:15.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe7c06be60 0x7fbe7c06e320 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe9c15b720 0x7fbe9c1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9c155930 0x7fbe9c1e4a00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 --2- 172.21.15.28:0/2086747521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe9c0d0370 0x7fbe9c153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 >> 172.21.15.28:0/2086747521 conn(0x7fbe9c06bdf0 msgr2=0x7fbe9c15a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 shutdown_connections 2024-09-10T10:50:15.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.726+0000 7fbea3400640 1 -- 172.21.15.28:0/2086747521 wait complete. 2024-09-10T10:50:15.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:50:15.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.907+0000 7f7d81c00640 1 Processor -- start 2024-09-10T10:50:15.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.907+0000 7f7d81c00640 1 -- start start 2024-09-10T10:50:15.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d81c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d7c15b0f0 0x7f7d7c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:15.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d81c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c072070 con 0x7f7d7c15ecf0 2024-09-10T10:50:15.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d81c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c15f260 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d81c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c15f540 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d7aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d7c15b0f0 0x7f7d7c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:15.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d72000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7d7c15ecf0 0x7f7d7c06f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37042/0 (socket says 172.21.15.28:37042) 2024-09-10T10:50:15.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.908+0000 7f7d72000640 1 -- 172.21.15.28:0/1320092865 learned_addr learned my addr 172.21.15.28:0/1320092865 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:15.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.909+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1320092865 >> v1:172.21.15.138:6789/0 conn(0x7f7d7c06f8b0 legacy=0x7f7d7c072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.909+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1320092865 >> v1:172.21.15.28:6789/0 conn(0x7f7d7c15ecf0 legacy=0x7f7d7c06f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.909+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1320092865 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c15f820 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.909+0000 7f7d7a000640 1 -- 172.21.15.28:0/1320092865 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3761131270 0 0) 0x7f7d7c072070 con 0x7f7d7c15ecf0 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.910+0000 7f7d7aa00640 1 --2- 172.21.15.28:0/1320092865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c0b3c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0097a0 tx=0x7f7d6c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10570a244359ee77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.910+0000 7f7d7a000640 1 -- 172.21.15.28:0/1320092865 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c037070 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.910+0000 7f7d7a000640 1 -- 172.21.15.28:0/1320092865 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c02ae50 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.910+0000 7f7d7a000640 1 -- 172.21.15.28:0/1320092865 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c032660 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.910+0000 7f7d81c00640 1 -- 172.21.15.28:0/1320092865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c15b0f0 msgr2=0x7f7d7c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1320092865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c0b3c40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0097a0 tx=0x7f7d6c02a140 comp rx=0 tx=0).stop 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 -- 172.21.15.28:0/1320092865 shutdown_connections 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1320092865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c0b3c40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 -- 172.21.15.28:0/1320092865 >> 172.21.15.28:0/1320092865 conn(0x7f7d7c067d70 msgr2=0x7f7d7c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:15.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 -- 172.21.15.28:0/1320092865 shutdown_connections 2024-09-10T10:50:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.911+0000 7f7d81c00640 1 -- 172.21.15.28:0/1320092865 wait complete. 2024-09-10T10:50:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.912+0000 7f7d81c00640 1 Processor -- start 2024-09-10T10:50:15.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.912+0000 7f7d81c00640 1 -- start start 2024-09-10T10:50:15.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:15.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:15.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d7c15ecf0 0x7f7d7c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:15.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c15b4d0 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c0b4180 con 0x7f7d7c15b0f0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.913+0000 7f7d81c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c0afa00 con 0x7f7d7c15ecf0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41548/0 (socket says 172.21.15.28:41548) 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1906088384 learned_addr learned my addr 172.21.15.28:0/1906088384 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d72000640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7b400640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d7c15ecf0 0x7f7d7c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d7c15ecf0 msgr2=0x7f7d7c1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d7c15ecf0 0x7f7d7c1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d7c15b0f0 msgr2=0x7f7d7c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 -- 172.21.15.28:0/1906088384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1420e0 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.914+0000 7f7d7aa00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0097a0 tx=0x7f7d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.915+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c042070 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.915+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d6c03e670 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.915+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c037040 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.915+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c141320 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.915+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1418e0 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.916+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c0b3f40 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.917+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7d6c045070 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.918+0000 7f7d73400640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:15.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.918+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 6273+0+0 (secure 0 0 0) 0x7f7d6c0ab980 con 0x7f7d7c06f8b0 2024-09-10T10:50:15.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.918+0000 7f7d72000640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:15.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.919+0000 7f7d72000640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 0x7f7d5806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d64007c30 tx=0x7f7d64007b80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:15.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:15.922+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d6c079b20 con 0x7f7d7c06f8b0 2024-09-10T10:50:16.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.103+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d7c0cc780 con 0x7f7d7c06f8b0 2024-09-10T10:50:16.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.731+0000 7f7d73400640 1 -- 172.21.15.28:0/1906088384 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7f7d6c07de20 con 0x7f7d7c06f8b0 2024-09-10T10:50:16.733 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:50:16.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.734+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 msgr2=0x7f7d5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.734+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 0x7f7d5806e2f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7d64007c30 tx=0x7f7d64007b80 comp rx=0 tx=0).stop 2024-09-10T10:50:16.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.734+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 msgr2=0x7f7d7c13e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.734+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0097a0 tx=0x7f7d6c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.734+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 shutdown_connections 2024-09-10T10:50:16.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7d5806be30 0x7f7d5806e2f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d7c15ecf0 0x7f7d7c1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d7c15b0f0 0x7f7d7c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 --2- 172.21.15.28:0/1906088384 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d7c06f8b0 0x7f7d7c13e830 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 >> 172.21.15.28:0/1906088384 conn(0x7f7d7c067d70 msgr2=0x7f7d7c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 shutdown_connections 2024-09-10T10:50:16.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.735+0000 7f7d81c00640 1 -- 172.21.15.28:0/1906088384 wait complete. 2024-09-10T10:50:16.759 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:50:16.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.923+0000 7fed65400640 1 Processor -- start 2024-09-10T10:50:16.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.923+0000 7fed65400640 1 -- start start 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.924+0000 7fed65400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fed60162340 0x7fed6006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.924+0000 7fed65400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed60162e30 0x7fed600d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.924+0000 7fed65400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed6015ff70 con 0x7fed600cf280 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.924+0000 7fed65400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fed601632a0 con 0x7fed60162e30 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.924+0000 7fed65400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fed6006de90 con 0x7fed60162340 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed60162e30 0x7fed600d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed60162e30 0x7fed600d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41560/0 (socket says 172.21.15.28:41560) 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 -- 172.21.15.28:0/1702428883 learned_addr learned my addr 172.21.15.28:0/1702428883 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5e800640 1 --2- 172.21.15.28:0/1702428883 >> v2:172.21.15.138:3300/0 conn(0x7fed60162340 0x7fed6006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 -- 172.21.15.28:0/1702428883 >> v2:172.21.15.138:3300/0 conn(0x7fed60162340 msgr2=0x7fed6006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 --2- 172.21.15.28:0/1702428883 >> v2:172.21.15.138:3300/0 conn(0x7fed60162340 0x7fed6006d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 -- 172.21.15.28:0/1702428883 >> v1:172.21.15.138:6789/0 conn(0x7fed600cf280 legacy=0x7fed60162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.925+0000 7fed5f200640 1 -- 172.21.15.28:0/1702428883 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600d4d10 con 0x7fed60162e30 2024-09-10T10:50:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.926+0000 7fed5f200640 1 --2- 172.21.15.28:0/1702428883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed60162e30 0x7fed600d2990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed6008e4c0 tx=0x7fed5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde19da596e6f812 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.926+0000 7fed5d400640 1 -- 172.21.15.28:0/1702428883 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58036070 con 0x7fed60162e30 2024-09-10T10:50:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.926+0000 7fed5d400640 1 -- 172.21.15.28:0/1702428883 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed58031600 con 0x7fed60162e30 2024-09-10T10:50:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.926+0000 7fed5d400640 1 -- 172.21.15.28:0/1702428883 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580319d0 con 0x7fed60162e30 2024-09-10T10:50:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 -- 172.21.15.28:0/1702428883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed60162e30 msgr2=0x7fed600d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 --2- 172.21.15.28:0/1702428883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed60162e30 0x7fed600d2990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fed6008e4c0 tx=0x7fed5802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 -- 172.21.15.28:0/1702428883 shutdown_connections 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 --2- 172.21.15.28:0/1702428883 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed60162e30 0x7fed600d2990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 --2- 172.21.15.28:0/1702428883 >> v2:172.21.15.138:3300/0 conn(0x7fed60162340 0x7fed6006d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 -- 172.21.15.28:0/1702428883 >> 172.21.15.28:0/1702428883 conn(0x7fed6008bfc0 msgr2=0x7fed600d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 -- 172.21.15.28:0/1702428883 shutdown_connections 2024-09-10T10:50:16.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.927+0000 7fed65400640 1 -- 172.21.15.28:0/1702428883 wait complete. 2024-09-10T10:50:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.929+0000 7fed65400640 1 Processor -- start 2024-09-10T10:50:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.929+0000 7fed65400640 1 -- start start 2024-09-10T10:50:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed60162340 0x7fed6015a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed60162e30 0x7fed6015c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed600d52d0 con 0x7fed600cf280 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed600d2470 con 0x7fed60162e30 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed65400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed600cf6c0 con 0x7fed60162340 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed5e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed5e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41570/0 (socket says 172.21.15.28:41570) 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed5e800640 1 -- 172.21.15.28:0/257082990 learned_addr learned my addr 172.21.15.28:0/257082990 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed5f200640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed60162e30 0x7fed6015c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.930+0000 7fed5de00640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed60162340 0x7fed6015a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 -- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed60162340 msgr2=0x7fed6015a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed60162340 0x7fed6015a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 -- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed60162e30 msgr2=0x7fed6015c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed60162e30 0x7fed6015c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 -- 172.21.15.28:0/257082990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed600b4310 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed5e800640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed5000d830 tx=0x7fed5000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5000b6c0 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed50019070 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5000ba80 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed600b45f0 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.931+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed600b4bb0 con 0x7fed600cf280 2024-09-10T10:50:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.932+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6006d740 con 0x7fed600cf280 2024-09-10T10:50:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.933+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed5001e460 con 0x7fed600cf280 2024-09-10T10:50:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.934+0000 7fed56a00640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.934+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5835+0+0 (secure 0 0 0) 0x7fed5008b450 con 0x7fed600cf280 2024-09-10T10:50:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.935+0000 7fed5de00640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.936+0000 7fed5de00640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fed600d45d0 tx=0x7fed4c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:16.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:16.939+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed50059720 con 0x7fed600cf280 2024-09-10T10:50:17.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.123+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fed6008e640 con 0x7fed600cf280 2024-09-10T10:50:17.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.732+0000 7fed56a00640 1 -- 172.21.15.28:0/257082990 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7fed5005da20 con 0x7fed600cf280 2024-09-10T10:50:17.735 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:50:17.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 msgr2=0x7fed4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 0x7fed4006e2f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fed600d45d0 tx=0x7fed4c009040 comp rx=0 tx=0).stop 2024-09-10T10:50:17.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 msgr2=0x7fed6015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fed5000d830 tx=0x7fed5000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 shutdown_connections 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed4006be30 0x7fed4006e2f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed60162e30 0x7fed6015c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed60162340 0x7fed6015a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 --2- 172.21.15.28:0/257082990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed600cf280 0x7fed6015a360 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.737+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 >> 172.21.15.28:0/257082990 conn(0x7fed6008bfc0 msgr2=0x7fed60162720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:17.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.738+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 shutdown_connections 2024-09-10T10:50:17.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.738+0000 7fed65400640 1 -- 172.21.15.28:0/257082990 wait complete. 2024-09-10T10:50:17.764 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:50:17.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.922+0000 7f21f3e00640 1 Processor -- start 2024-09-10T10:50:17.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.922+0000 7f21f3e00640 1 -- start start 2024-09-10T10:50:17.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.923+0000 7f21f3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21ec0d0a80 0x7f21ec15bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.923+0000 7f21f3e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec0ae4e0 con 0x7f21ec155930 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.923+0000 7f21f3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec15c1e0 con 0x7f21ec15b720 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.923+0000 7f21f3e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec15c4c0 con 0x7f21ec0d0a80 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21ec0d0a80 0x7f21ec15bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21ec155930 0x7f21ec0addd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37044/0 (socket says 172.21.15.28:37044) 2024-09-10T10:50:17.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1400640 1 -- 172.21.15.28:0/2699676416 learned_addr learned my addr 172.21.15.28:0/2699676416 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1e00640 1 -- 172.21.15.28:0/2699676416 >> v1:172.21.15.138:6789/0 conn(0x7f21ec15b720 legacy=0x7f21ec0d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1e00640 1 -- 172.21.15.28:0/2699676416 >> v1:172.21.15.28:6789/0 conn(0x7f21ec155930 legacy=0x7f21ec0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21f1e00640 1 -- 172.21.15.28:0/2699676416 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0b3340 con 0x7f21ec0d0a80 2024-09-10T10:50:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.924+0000 7f21ebe00640 1 -- 172.21.15.28:0/2699676416 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111779414 0 0) 0x7f21e0001110 con 0x7f21ec155930 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.925+0000 7f21f1e00640 1 --2- 172.21.15.28:0/2699676416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec0d0a80 0x7f21ec15bbb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009d90 tx=0x7f21e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84262db7bca5444 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.925+0000 7f21ebe00640 1 -- 172.21.15.28:0/2699676416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4007740 con 0x7f21ec0d0a80 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.925+0000 7f21ebe00640 1 -- 172.21.15.28:0/2699676416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e40078e0 con 0x7f21ec0d0a80 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21ebe00640 1 -- 172.21.15.28:0/2699676416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4007cb0 con 0x7f21ec0d0a80 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 -- 172.21.15.28:0/2699676416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec0d0a80 msgr2=0x7f21ec15bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 --2- 172.21.15.28:0/2699676416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec0d0a80 0x7f21ec15bbb0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009d90 tx=0x7f21e402a6f0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 -- 172.21.15.28:0/2699676416 shutdown_connections 2024-09-10T10:50:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 --2- 172.21.15.28:0/2699676416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec0d0a80 0x7f21ec15bbb0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 -- 172.21.15.28:0/2699676416 >> 172.21.15.28:0/2699676416 conn(0x7f21ec06bdf0 msgr2=0x7f21ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 -- 172.21.15.28:0/2699676416 shutdown_connections 2024-09-10T10:50:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.926+0000 7f21f3e00640 1 -- 172.21.15.28:0/2699676416 wait complete. 2024-09-10T10:50:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.928+0000 7f21f3e00640 1 Processor -- start 2024-09-10T10:50:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.928+0000 7f21f3e00640 1 -- start start 2024-09-10T10:50:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 0x7f21ec1f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 0x7f21ec1f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec06fa30 con 0x7f21ec15b720 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec06f8b0 con 0x7f21ec0d0a80 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0b3b40 con 0x7f21ec155930 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41584/0 (socket says 172.21.15.28:41584) 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 -- 172.21.15.28:0/3913475454 learned_addr learned my addr 172.21.15.28:0/3913475454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f0a00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 0x7f21ec1f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1400640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 0x7f21ec1f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 -- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 msgr2=0x7f21ec1f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 0x7f21ec1f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 -- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 msgr2=0x7f21ec1f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:17.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 0x7f21ec1f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:17.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1e00640 1 -- 172.21.15.28:0/3913475454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0ccdf0 con 0x7f21ec15b720 2024-09-10T10:50:17.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f0a00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 0x7f21ec1f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:17.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.929+0000 7f21f1400640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 0x7f21ec1f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:17.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21f1e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009eb0 tx=0x7f21e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4040070 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e4031c40 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4003990 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec0cd0d0 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.930+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec0cd690 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.931+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec152b00 con 0x7f21ec15b720 2024-09-10T10:50:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.932+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21e4039040 con 0x7f21ec15b720 2024-09-10T10:50:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.933+0000 7f21e9600640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 0x7f21cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.934+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f21e40ac160 con 0x7f21ec15b720 2024-09-10T10:50:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.934+0000 7f21f1400640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 0x7f21cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.935+0000 7f21f1400640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 0x7f21cc06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000f60 tx=0x7f21e0007a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:17.938+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f21ec152b00 con 0x7f21ec15b720 2024-09-10T10:50:18.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.122+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f21ec072ab0 con 0x7f21ec15b720 2024-09-10T10:50:18.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.126+0000 7f21e9600640 1 -- 172.21.15.28:0/3913475454 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7f21ec072ab0 con 0x7f21ec15b720 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 msgr2=0x7f21cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 0x7f21cc06e2f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000f60 tx=0x7f21e0007a10 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 msgr2=0x7f21ec151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009eb0 tx=0x7f21e4004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 shutdown_connections 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be30 0x7f21cc06e2f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec15b720 0x7f21ec151150 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec155930 0x7f21ec1f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 --2- 172.21.15.28:0/3913475454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec0d0a80 0x7f21ec1f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.128+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 >> 172.21.15.28:0/3913475454 conn(0x7f21ec06bdf0 msgr2=0x7f21ec0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.129+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 shutdown_connections 2024-09-10T10:50:18.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.129+0000 7f21f3e00640 1 -- 172.21.15.28:0/3913475454 wait complete. 2024-09-10T10:50:18.146 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_inject_rename_corrupt_dentry_first 2024-09-10T10:50:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.303+0000 7f1e12a00640 1 Processor -- start 2024-09-10T10:50:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.303+0000 7f1e12a00640 1 -- start start 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e12a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e0c0b4800 0x7f1e0c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e12a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0c0af7a0 con 0x7f1e0c15c310 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e12a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e0c0afa80 con 0x7f1e0c0d0370 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e12a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1e0c160130 con 0x7f1e0c0b4800 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e0be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1e0c15c310 0x7f1e0c160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37052/0 (socket says 172.21.15.28:37052) 2024-09-10T10:50:18.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e0be00640 1 -- 172.21.15.28:0/496346890 learned_addr learned my addr 172.21.15.28:0/496346890 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.304+0000 7f1e03400640 1 --2- 172.21.15.28:0/496346890 >> v2:172.21.15.138:3300/0 conn(0x7f1e0c0b4800 0x7f1e0c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104217604 0 0) 0x7f1e0c0af7a0 con 0x7f1e0c15c310 2024-09-10T10:50:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1df0003520 con 0x7f1e0c15c310 2024-09-10T10:50:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751584604 0 0) 0x7f1e0c0afa80 con 0x7f1e0c0d0370 2024-09-10T10:50:18.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e0c0af7a0 con 0x7f1e0c0d0370 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3413697052 0 0) 0x7f1df0003520 con 0x7f1e0c15c310 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 >> v2:172.21.15.138:3300/0 conn(0x7f1e0c0b4800 msgr2=0x7f1e0c0aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 --2- 172.21.15.28:0/496346890 >> v2:172.21.15.138:3300/0 conn(0x7f1e0c0b4800 0x7f1e0c0aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 >> v1:172.21.15.138:6789/0 conn(0x7f1e0c0d0370 legacy=0x7f1e0c06df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c0af470 con 0x7f1e0c15c310 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f1dfc001670 con 0x7f1e0c15c310 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 >> v1:172.21.15.28:6789/0 conn(0x7f1e0c15c310 legacy=0x7f1e0c160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 0x7f1df0003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1df0004800 0x7f1df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df00051c0 0x7f1df000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c160130 con 0x7f1df0003b90 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1df000c090 con 0x7f1df00051c0 2024-09-10T10:50:18.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1df000c290 con 0x7f1df0004800 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e03400640 1 --2- 172.21.15.28:0/496346890 >> v2:172.21.15.138:3300/0 conn(0x7f1e0c0b4800 0x7f1e0c0aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e0be00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 0x7f1df0003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.305+0000 7f1e10a00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df00051c0 0x7f1df000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e03400640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1df0004800 0x7f1df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e0be00640 1 -- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1df0004800 msgr2=0x7f1df0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e0be00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1df0004800 0x7f1df0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e0be00640 1 -- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df00051c0 msgr2=0x7f1df000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e0be00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df00051c0 0x7f1df000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e0be00640 1 -- 172.21.15.28:0/496346890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df0003520 con 0x7f1df0003b90 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e10a00640 1 -- 172.21.15.28:0/496346890 reap_dead start 2024-09-10T10:50:18.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.306+0000 7f1e03400640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1df0004800 0x7f1df0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:18.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.307+0000 7f1e0be00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 0x7f1df0003f90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fb0 tx=0x7f1df0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f180d0ecbd9b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.307+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc007c00 con 0x7f1df0003b90 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.307+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dfc005af0 con 0x7f1df0003b90 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.307+0000 7f1e0b400640 1 -- 172.21.15.28:0/496346890 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc007070 con 0x7f1df0003b90 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 -- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 msgr2=0x7f1df0003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 0x7f1df0003f90 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc007fb0 tx=0x7f1df0005830 comp rx=0 tx=0).stop 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 -- 172.21.15.28:0/496346890 shutdown_connections 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 --2- 172.21.15.28:0/496346890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1df0003b90 0x7f1df0003f90 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 -- 172.21.15.28:0/496346890 >> 172.21.15.28:0/496346890 conn(0x7f1e0c06bdf0 msgr2=0x7f1e0c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 -- 172.21.15.28:0/496346890 shutdown_connections 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.308+0000 7f1e12a00640 1 -- 172.21.15.28:0/496346890 wait complete. 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.309+0000 7f1e12a00640 1 Processor -- start 2024-09-10T10:50:18.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 -- start start 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df0003b90 0x7f1e0c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e0c15eec0 0x7f1e0c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0af0b0 con 0x7f1e0c159d40 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c0aef30 con 0x7f1df0003b90 2024-09-10T10:50:18.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.310+0000 7f1e12a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0c0af230 con 0x7f1e0c15eec0 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41596/0 (socket says 172.21.15.28:41596) 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 -- 172.21.15.28:0/4162907407 learned_addr learned my addr 172.21.15.28:0/4162907407 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e10a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e0c15eec0 0x7f1e0c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e0be00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df0003b90 0x7f1e0c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 -- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e0c15eec0 msgr2=0x7f1e0c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e0c15eec0 0x7f1e0c15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 -- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df0003b90 msgr2=0x7f1e0c1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df0003b90 0x7f1e0c1579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 -- 172.21.15.28:0/4162907407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c1580a0 con 0x7f1e0c159d40 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.311+0000 7f1e03400640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011860 tx=0x7f1df4011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.312+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df400f9c0 con 0x7f1e0c159d40 2024-09-10T10:50:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.312+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df400fb60 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.312+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c158380 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.312+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4015460 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.312+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c158940 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.313+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0c15f880 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.314+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1df4015600 con 0x7f1e0c159d40 2024-09-10T10:50:18.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.315+0000 7f1e08c00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 0x7f1de806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.315+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f1df408b890 con 0x7f1e0c159d40 2024-09-10T10:50:18.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.315+0000 7f1e0be00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 0x7f1de806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.316+0000 7f1e0be00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 0x7f1de806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc032000 tx=0x7f1dfc02d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.319+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1df4019030 con 0x7f1e0c159d40 2024-09-10T10:50:18.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.502+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"} v 0) -- 0x7f1e0c0d0800 con 0x7f1e0c159d40 2024-09-10T10:50:18.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.503+0000 7f1e08c00640 1 -- 172.21.15.28:0/4162907407 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_inject_rename_corrupt_dentry_first"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f1df405a230 con 0x7f1e0c159d40 2024-09-10T10:50:18.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 msgr2=0x7f1de806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 0x7f1de806e2f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc032000 tx=0x7f1dfc02d000 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 msgr2=0x7f1e0c14c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1df4011860 tx=0x7f1df4011c60 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 shutdown_connections 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1de806be30 0x7f1de806e2f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e0c15eec0 0x7f1e0c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e0c159d40 0x7f1e0c14c330 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 --2- 172.21.15.28:0/4162907407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1df0003b90 0x7f1e0c1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 >> 172.21.15.28:0/4162907407 conn(0x7f1e0c06bdf0 msgr2=0x7f1e0c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 shutdown_connections 2024-09-10T10:50:18.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.505+0000 7f1e12a00640 1 -- 172.21.15.28:0/4162907407 wait complete. 2024-09-10T10:50:18.523 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush' 2024-09-10T10:50:18.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.679+0000 7f1ea0000640 1 Processor -- start 2024-09-10T10:50:18.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.679+0000 7f1ea0000640 1 -- start start 2024-09-10T10:50:18.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1ea0000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 0x7f1e9806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1ea0000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1ea0000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e9806e870 con 0x7f1e98162060 2024-09-10T10:50:18.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1ea0000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1e9806d9f0 con 0x7f1e98161c80 2024-09-10T10:50:18.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1ea0000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9806dcd0 con 0x7f1e980b6680 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:44310/0 (socket says 172.21.15.28:44310) 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 -- 172.21.15.28:0/2912334643 learned_addr learned my addr 172.21.15.28:0/2912334643 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9d600640 1 --2- 172.21.15.28:0/2912334643 >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 0x7f1e9806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 -- 172.21.15.28:0/2912334643 >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 msgr2=0x7f1e9806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 --2- 172.21.15.28:0/2912334643 >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 0x7f1e9806e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 -- 172.21.15.28:0/2912334643 >> v1:172.21.15.138:6789/0 conn(0x7f1e98162060 legacy=0x7f1e980b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9e000640 1 -- 172.21.15.28:0/2912334643 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9806dfb0 con 0x7f1e980b6680 2024-09-10T10:50:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.680+0000 7f1e9d600640 1 --2- 172.21.15.28:0/2912334643 >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 0x7f1e9806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:18.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.681+0000 7f1e9e000640 1 --2- 172.21.15.28:0/2912334643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009e50 tx=0x7f1e9002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e630a3af6dd86e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1e97e00640 1 -- 172.21.15.28:0/2912334643 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e90036070 con 0x7f1e980b6680 2024-09-10T10:50:18.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1e97e00640 1 -- 172.21.15.28:0/2912334643 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e90031600 con 0x7f1e980b6680 2024-09-10T10:50:18.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1e97e00640 1 -- 172.21.15.28:0/2912334643 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e900319d0 con 0x7f1e980b6680 2024-09-10T10:50:18.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 -- 172.21.15.28:0/2912334643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 msgr2=0x7f1e980cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 --2- 172.21.15.28:0/2912334643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009e50 tx=0x7f1e9002a710 comp rx=0 tx=0).stop 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 -- 172.21.15.28:0/2912334643 shutdown_connections 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 --2- 172.21.15.28:0/2912334643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e980b6680 0x7f1e980cf1e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 --2- 172.21.15.28:0/2912334643 >> v2:172.21.15.138:3300/0 conn(0x7f1e98161c80 0x7f1e9806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 -- 172.21.15.28:0/2912334643 >> 172.21.15.28:0/2912334643 conn(0x7f1e980d44f0 msgr2=0x7f1e980d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 -- 172.21.15.28:0/2912334643 shutdown_connections 2024-09-10T10:50:18.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.682+0000 7f1ea0000640 1 -- 172.21.15.28:0/2912334643 wait complete. 2024-09-10T10:50:18.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.684+0000 7f1ea0000640 1 Processor -- start 2024-09-10T10:50:18.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.684+0000 7f1ea0000640 1 -- start start 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e98161c80 0x7f1e980b9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e98162060 0x7f1e980b9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e980b6cb0 con 0x7f1e980b6680 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e980b6b30 con 0x7f1e98161c80 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1ea0000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e980cf720 con 0x7f1e98162060 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1e9d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1e9d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41608/0 (socket says 172.21.15.28:41608) 2024-09-10T10:50:18.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1e9d600640 1 -- 172.21.15.28:0/3377056291 learned_addr learned my addr 172.21.15.28:0/3377056291 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1e9cc00640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e98161c80 0x7f1e980b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.685+0000 7f1e9e000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e98162060 0x7f1e980b9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 -- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e98162060 msgr2=0x7f1e980b9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e98162060 0x7f1e980b9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 -- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e98161c80 msgr2=0x7f1e980b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e98161c80 0x7f1e980b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 -- 172.21.15.28:0/3377056291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e980cb7d0 con 0x7f1e980b6680 2024-09-10T10:50:18.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e9d600640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00ebe0 tx=0x7f1e8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c017070 con 0x7f1e980b6680 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c00cc30 con 0x7f1e980b6680 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e980cbab0 con 0x7f1e980b6680 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.686+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c013470 con 0x7f1e980b6680 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.687+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e980b9f40 con 0x7f1e980b6680 2024-09-10T10:50:18.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.688+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e980cf1e0 con 0x7f1e980b6680 2024-09-10T10:50:18.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.689+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e8c01a070 con 0x7f1e980b6680 2024-09-10T10:50:18.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.689+0000 7f1e95600640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 0x7f1e7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:18.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.689+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f1e8c08afa0 con 0x7f1e980b6680 2024-09-10T10:50:18.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.690+0000 7f1e9cc00640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 0x7f1e7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:18.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.692+0000 7f1e9cc00640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 0x7f1e7806e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1e98160570 tx=0x7f1e84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:18.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.694+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e8c059410 con 0x7f1e980b6680 2024-09-10T10:50:18.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:18.953+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]} v 0) -- 0x7f1e980ba470 con 0x7f1e980b6680 2024-09-10T10:50:19.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.737+0000 7f1e95600640 1 -- 172.21.15.28:0/3377056291 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_dentry_first_preflush"]}]=0 v320) ==== 141+0+0 (secure 0 0 0) 0x7f1e8c05d710 con 0x7f1e980b6680 2024-09-10T10:50:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 msgr2=0x7f1e7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 0x7f1e7806e320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1e98160570 tx=0x7f1e84009040 comp rx=0 tx=0).stop 2024-09-10T10:50:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 msgr2=0x7f1e98155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00ebe0 tx=0x7f1e8c00c520 comp rx=0 tx=0).stop 2024-09-10T10:50:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 shutdown_connections 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e7806be60 0x7f1e7806e320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e98162060 0x7f1e980b9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e98161c80 0x7f1e980b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 --2- 172.21.15.28:0/3377056291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e980b6680 0x7f1e98155910 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 >> 172.21.15.28:0/3377056291 conn(0x7f1e980d44f0 msgr2=0x7f1e98162a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 shutdown_connections 2024-09-10T10:50:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.741+0000 7f1ea0000640 1 -- 172.21.15.28:0/3377056291 wait complete. 2024-09-10T10:50:19.760 INFO:tasks.cephfs_test_runner:That the MDS won't write a dentry with new damage to CDentry::first ... ok 2024-09-10T10:50:19.760 INFO:tasks.cephfs_test_runner:Starting test: test_object_deletion (tasks.cephfs.test_damage.TestDamage) That the MDS has a clean 'damaged' response to loss of any single metadata object 2024-09-10T10:50:19.760 INFO:tasks.cephfs_test_runner:test_object_deletion (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:50:19.761 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.921+0000 7f890c200640 1 Processor -- start 2024-09-10T10:50:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.921+0000 7f890c200640 1 -- start start 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.922+0000 7f890c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904161c80 0x7f890406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.922+0000 7f890c200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8904162060 0x7f89040b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.922+0000 7f890c200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f890406e870 con 0x7f89040b64b0 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.922+0000 7f890c200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f890406d9f0 con 0x7f8904162060 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.922+0000 7f890c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f890406dcd0 con 0x7f8904161c80 2024-09-10T10:50:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f890a200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f89040b64b0 0x7f8904162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37056/0 (socket says 172.21.15.28:37056) 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f890a200640 1 -- 172.21.15.28:0/2634790219 learned_addr learned my addr 172.21.15.28:0/2634790219 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8908e00640 1 --2- 172.21.15.28:0/2634790219 >> v2:172.21.15.28:3300/0 conn(0x7f8904162060 0x7f89040b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8909800640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904161c80 0x7f890406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8908e00640 1 -- 172.21.15.28:0/2634790219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904161c80 msgr2=0x7f890406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8908e00640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904161c80 0x7f890406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8908e00640 1 -- 172.21.15.28:0/2634790219 >> v1:172.21.15.28:6789/0 conn(0x7f89040b64b0 legacy=0x7f8904162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8908e00640 1 -- 172.21.15.28:0/2634790219 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890406dfb0 con 0x7f8904162060 2024-09-10T10:50:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.923+0000 7f8903e00640 1 -- 172.21.15.28:0/2634790219 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524720541 0 0) 0x7f88fc001490 con 0x7f89040b64b0 2024-09-10T10:50:19.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.924+0000 7f8908e00640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8904162060 0x7f89040b5f70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f88f00097a0 tx=0x7f88f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfc5a04ad05b694 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.924+0000 7f8903e00640 1 -- 172.21.15.28:0/2634790219 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0037070 con 0x7f8904162060 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.924+0000 7f8903e00640 1 -- 172.21.15.28:0/2634790219 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0032440 con 0x7f8904162060 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.924+0000 7f8903e00640 1 -- 172.21.15.28:0/2634790219 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f0032810 con 0x7f8904162060 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 -- 172.21.15.28:0/2634790219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8904162060 msgr2=0x7f89040b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8904162060 0x7f89040b5f70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f88f00097a0 tx=0x7f88f002a180 comp rx=0 tx=0).stop 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 -- 172.21.15.28:0/2634790219 shutdown_connections 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8904162060 0x7f89040b5f70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 --2- 172.21.15.28:0/2634790219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904161c80 0x7f890406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 -- 172.21.15.28:0/2634790219 >> 172.21.15.28:0/2634790219 conn(0x7f89040d44f0 msgr2=0x7f89040d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 -- 172.21.15.28:0/2634790219 shutdown_connections 2024-09-10T10:50:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.925+0000 7f890c200640 1 -- 172.21.15.28:0/2634790219 wait complete. 2024-09-10T10:50:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.927+0000 7f890c200640 1 Processor -- start 2024-09-10T10:50:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.927+0000 7f890c200640 1 -- start start 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8904161c80 0x7f89040ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904162060 0x7f89040ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89040d2a10 con 0x7f89040b64b0 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89040d2890 con 0x7f8904161c80 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f89040b6960 con 0x7f8904162060 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41620/0 (socket says 172.21.15.28:41620) 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 -- 172.21.15.28:0/153974280 learned_addr learned my addr 172.21.15.28:0/153974280 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8908e00640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8904161c80 0x7f89040ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f890a200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904162060 0x7f89040ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 -- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904162060 msgr2=0x7f89040ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904162060 0x7f89040ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 -- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8904161c80 msgr2=0x7f89040ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8904161c80 0x7f89040ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.928+0000 7f8909800640 1 -- 172.21.15.28:0/153974280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89041f5dc0 con 0x7f89040b64b0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f8909800640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f88f800d800 tx=0x7f88f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f800b6c0 con 0x7f89040b64b0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f8019070 con 0x7f89040b64b0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f800ba80 con 0x7f89040b64b0 2024-09-10T10:50:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89041f5ae0 con 0x7f89040b64b0 2024-09-10T10:50:19.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.929+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041f8820 con 0x7f89040b64b0 2024-09-10T10:50:19.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.930+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89041f8c60 con 0x7f89040b64b0 2024-09-10T10:50:19.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.931+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f88f800bc20 con 0x7f89040b64b0 2024-09-10T10:50:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.932+0000 7f8901600640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 0x7f88e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.932+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f88f808b0d0 con 0x7f89040b64b0 2024-09-10T10:50:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.933+0000 7f8908e00640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 0x7f88e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.933+0000 7f8908e00640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 0x7f88e406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f88f0034000 tx=0x7f88f002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:19.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:19.936+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f88f8059540 con 0x7f89040b64b0 2024-09-10T10:50:20.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.122+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f890406e330 con 0x7f89040b64b0 2024-09-10T10:50:20.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.123+0000 7f8901600640 1 -- 172.21.15.28:0/153974280 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+13464 (secure 0 0 0) 0x7f88f805d840 con 0x7f89040b64b0 2024-09-10T10:50:20.124 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:20.125 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":90,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:17.732551+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6836/1950306159":"2024-09-11T10:50:14.705296+0000","172.21.15.138:6834/1950306159":"2024-09-11T10:50:14.705296+0000","172.21.15.138:6837/4263014325":"2024-09-11T10:50:05.677188+0000","172.21.15.138:6835/4263014325":"2024-09-11T10:50:05.677188+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 msgr2=0x7f88e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 0x7f88e406e2f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f88f0034000 tx=0x7f88f002e000 comp rx=0 tx=0).stop 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 msgr2=0x7f89041560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f88f800d800 tx=0x7f88f800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 shutdown_connections 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f88e406be30 0x7f88e406e2f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8904162060 0x7f89040ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8904161c80 0x7f89040ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 --2- 172.21.15.28:0/153974280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89040b64b0 0x7f89041560c0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 >> 172.21.15.28:0/153974280 conn(0x7f89040d44f0 msgr2=0x7f89040d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 shutdown_connections 2024-09-10T10:50:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.126+0000 7f890c200640 1 -- 172.21.15.28:0/153974280 wait complete. 2024-09-10T10:50:20.143 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:50:20.143 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:50:20.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.301+0000 7ff3af200640 1 Processor -- start 2024-09-10T10:50:20.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.301+0000 7ff3af200640 1 -- start start 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff3a80cf280 0x7ff3a8162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 0x7ff3a80d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a806de90 con 0x7ff3a80cf280 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a815ff70 con 0x7ff3a8162c60 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3af200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a80d4d10 con 0x7ff3a8162340 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.302+0000 7ff3a7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff3a80cf280 0x7ff3a8162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff3a80cf280 0x7ff3a8162720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41626/0 (socket says 172.21.15.28:41626) 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 -- 172.21.15.28:0/2634990988 learned_addr learned my addr 172.21.15.28:0/2634990988 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3ad200640 1 --2- 172.21.15.28:0/2634990988 >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 0x7ff3a80d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 -- 172.21.15.28:0/2634990988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 msgr2=0x7ff3a806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a806d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 -- 172.21.15.28:0/2634990988 >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 msgr2=0x7ff3a80d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 --2- 172.21.15.28:0/2634990988 >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 0x7ff3a80d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7400640 1 -- 172.21.15.28:0/2634990988 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a80d4ff0 con 0x7ff3a80cf280 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:20.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.303+0000 7ff3ad200640 1 --2- 172.21.15.28:0/2634990988 >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 0x7ff3a80d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:20.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.304+0000 7ff3a7400640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a80cf280 0x7ff3a8162720 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80c7f30 tx=0x7ff39402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40df034bdf27df76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.304+0000 7ff3a6a00640 1 -- 172.21.15.28:0/2634990988 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394004030 con 0x7ff3a80cf280 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.304+0000 7ff3a6a00640 1 -- 172.21.15.28:0/2634990988 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3940041d0 con 0x7ff3a80cf280 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.304+0000 7ff3a6a00640 1 -- 172.21.15.28:0/2634990988 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394003610 con 0x7ff3a80cf280 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 -- 172.21.15.28:0/2634990988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a80cf280 msgr2=0x7ff3a8162720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a80cf280 0x7ff3a8162720 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80c7f30 tx=0x7ff39402a130 comp rx=0 tx=0).stop 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 -- 172.21.15.28:0/2634990988 shutdown_connections 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 --2- 172.21.15.28:0/2634990988 >> v2:172.21.15.138:3300/0 conn(0x7ff3a8162c60 0x7ff3a80d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a80cf280 0x7ff3a8162720 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 --2- 172.21.15.28:0/2634990988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 -- 172.21.15.28:0/2634990988 >> 172.21.15.28:0/2634990988 conn(0x7ff3a808bfc0 msgr2=0x7ff3a80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 -- 172.21.15.28:0/2634990988 shutdown_connections 2024-09-10T10:50:20.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.305+0000 7ff3af200640 1 -- 172.21.15.28:0/2634990988 wait complete. 2024-09-10T10:50:20.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.307+0000 7ff3af200640 1 Processor -- start 2024-09-10T10:50:20.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.307+0000 7ff3af200640 1 -- start start 2024-09-10T10:50:20.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.307+0000 7ff3af200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 0x7ff3a80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.307+0000 7ff3af200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a80b8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3af200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3af200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a80cf6c0 con 0x7ff3a8162c60 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3af200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a80d52d0 con 0x7ff3a80cf280 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3af200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a8163110 con 0x7ff3a8162340 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41634/0 (socket says 172.21.15.28:41634) 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 -- 172.21.15.28:0/4155632116 learned_addr learned my addr 172.21.15.28:0/4155632116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 0x7ff3a80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3a7400640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a80b8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 -- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 msgr2=0x7ff3a80b8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a80b8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 -- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 msgr2=0x7ff3a80cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 0x7ff3a80cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3ad200640 1 -- 172.21.15.28:0/4155632116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a80b8d70 con 0x7ff3a8162c60 2024-09-10T10:50:20.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.308+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 0x7ff3a80cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:20.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff3ad200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0007ef0 tx=0x7ff3a000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a001a070 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a0003b90 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0004030 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a80b9010 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.309+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a80b7680 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.310+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a8162720 con 0x7ff3a8162c60 2024-09-10T10:50:20.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.311+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff3a001d070 con 0x7ff3a8162c60 2024-09-10T10:50:20.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.312+0000 7ff39fe00640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 0x7ff38806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.312+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7ff3a008af90 con 0x7ff3a8162c60 2024-09-10T10:50:20.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.312+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 0x7ff38806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.313+0000 7ff3a7e00640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 0x7ff38806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80cd100 tx=0x7ff398008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.316+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3a0010070 con 0x7ff3a8162c60 2024-09-10T10:50:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.500+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3a80ba450 con 0x7ff3a8162c60 2024-09-10T10:50:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.500+0000 7ff39fe00640 1 -- 172.21.15.28:0/4155632116 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ff3a0059370 con 0x7ff3a8162c60 2024-09-10T10:50:20.501 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 msgr2=0x7ff38806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 0x7ff38806e2f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff3a80cd100 tx=0x7ff398008040 comp rx=0 tx=0).stop 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 msgr2=0x7ff3a80b7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0007ef0 tx=0x7ff3a000d4a0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 shutdown_connections 2024-09-10T10:50:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff38806be30 0x7ff38806e2f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3a8162c60 0x7ff3a80b7e80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff3a8162340 0x7ff3a80b8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 --2- 172.21.15.28:0/4155632116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3a80cf280 0x7ff3a80cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 >> 172.21.15.28:0/4155632116 conn(0x7ff3a808bfc0 msgr2=0x7ff3a8161730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 shutdown_connections 2024-09-10T10:50:20.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.503+0000 7ff3af200640 1 -- 172.21.15.28:0/4155632116 wait complete. 2024-09-10T10:50:20.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-10T10:50:20.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.673+0000 7fc402800640 1 Processor -- start 2024-09-10T10:50:20.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.674+0000 7fc402800640 1 -- start start 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc402800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 0x7fc3fc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc402800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3fc146140 con 0x7fc3fc072de0 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc402800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3fc0cd740 con 0x7fc3fc072320 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc402800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc0cda20 con 0x7fc3fc072700 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc3f3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc3fc072de0 0x7fc3fc15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37068/0 (socket says 172.21.15.28:37068) 2024-09-10T10:50:20.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc3f3e00640 1 -- 172.21.15.28:0/2213098999 learned_addr learned my addr 172.21.15.28:0/2213098999 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc3fbe00640 1 --2- 172.21.15.28:0/2213098999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 0x7fc3fc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.675+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4286745817 0 0) 0x7fc3fc146140 con 0x7fc3fc072de0 2024-09-10T10:50:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3e0003520 con 0x7fc3fc072de0 2024-09-10T10:50:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895374308 0 0) 0x7fc3fc0cd740 con 0x7fc3fc072320 2024-09-10T10:50:20.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3fc146140 con 0x7fc3fc072320 2024-09-10T10:50:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3778929336 0 0) 0x7fc3e0003520 con 0x7fc3fc072de0 2024-09-10T10:50:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3fbe00640 1 -- 172.21.15.28:0/2213098999 >> v1:172.21.15.138:6789/0 conn(0x7fc3fc072320 legacy=0x7fc3fc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3fbe00640 1 -- 172.21.15.28:0/2213098999 >> v1:172.21.15.28:6789/0 conn(0x7fc3fc072de0 legacy=0x7fc3fc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.676+0000 7fc3fbe00640 1 -- 172.21.15.28:0/2213098999 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc0cdd00 con 0x7fc3fc072700 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.677+0000 7fc3fbe00640 1 --2- 172.21.15.28:0/2213098999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 0x7fc3fc15d8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009dc0 tx=0x7fc3f402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1c965cd3352bbef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.677+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4001670 con 0x7fc3fc072700 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.677+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f4007540 con 0x7fc3fc072700 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.677+0000 7fc3faa00640 1 -- 172.21.15.28:0/2213098999 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4007910 con 0x7fc3fc072700 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 -- 172.21.15.28:0/2213098999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 msgr2=0x7fc3fc15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 --2- 172.21.15.28:0/2213098999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 0x7fc3fc15d8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009dc0 tx=0x7fc3f402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 -- 172.21.15.28:0/2213098999 shutdown_connections 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 --2- 172.21.15.28:0/2213098999 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072700 0x7fc3fc15d8f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 -- 172.21.15.28:0/2213098999 >> 172.21.15.28:0/2213098999 conn(0x7fc3fc067d70 msgr2=0x7fc3fc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 -- 172.21.15.28:0/2213098999 shutdown_connections 2024-09-10T10:50:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.678+0000 7fc402800640 1 -- 172.21.15.28:0/2213098999 wait complete. 2024-09-10T10:50:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.680+0000 7fc402800640 1 Processor -- start 2024-09-10T10:50:20.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.680+0000 7fc402800640 1 -- start start 2024-09-10T10:50:20.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.680+0000 7fc402800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072320 0x7fc3fc138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc402800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc402800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3fc072de0 0x7fc3fc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc402800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc15dfb0 con 0x7fc3fc072700 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc402800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc15de30 con 0x7fc3fc072de0 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc402800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3fc15e130 con 0x7fc3fc072320 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41644/0 (socket says 172.21.15.28:41644) 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 -- 172.21.15.28:0/1772443864 learned_addr learned my addr 172.21.15.28:0/1772443864 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3fb400640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072320 0x7fc3fc138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3fbe00640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3fc072de0 0x7fc3fc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 -- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072320 msgr2=0x7fc3fc138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072320 0x7fc3fc138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 -- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3fc072de0 msgr2=0x7fc3fc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3fc072de0 0x7fc3fc0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:20.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.681+0000 7fc3f3e00640 1 -- 172.21.15.28:0/1772443864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc138560 con 0x7fc3fc072700 2024-09-10T10:50:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc3f3e00640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c780 tx=0x7fc3e400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4005ad0 con 0x7fc3fc072700 2024-09-10T10:50:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e4005c70 con 0x7fc3fc072700 2024-09-10T10:50:20.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc1e55a0 con 0x7fc3fc072700 2024-09-10T10:50:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e40072b0 con 0x7fc3fc072700 2024-09-10T10:50:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.682+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc1e5b60 con 0x7fc3fc072700 2024-09-10T10:50:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.683+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc0cce60 con 0x7fc3fc072700 2024-09-10T10:50:20.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.684+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3e4007450 con 0x7fc3fc072700 2024-09-10T10:50:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.685+0000 7fc3f3400640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 0x7fc3d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:20.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.685+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7fc3e408ab00 con 0x7fc3fc072700 2024-09-10T10:50:20.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.686+0000 7fc3fb400640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 0x7fc3d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:20.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.687+0000 7fc3fb400640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 0x7fc3d806e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007fd0 tx=0x7fc3ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:20.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.690+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3e4005e20 con 0x7fc3fc072700 2024-09-10T10:50:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:20.946+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7fc3fc06d090 con 0x7fc3fc072700 2024-09-10T10:50:21.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.743+0000 7fc3f3400640 1 -- 172.21.15.28:0/1772443864 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v322) ==== 138+0+0 (secure 0 0 0) 0x7fc3e408ae20 con 0x7fc3fc072700 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 msgr2=0x7fc3d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 0x7fc3d806e2f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007fd0 tx=0x7fc3ec009040 comp rx=0 tx=0).stop 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 msgr2=0x7fc3fc1e4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc3e400c780 tx=0x7fc3e400cc40 comp rx=0 tx=0).stop 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 shutdown_connections 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc3d806be30 0x7fc3d806e2f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3fc072de0 0x7fc3fc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3fc072700 0x7fc3fc1e4c30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 --2- 172.21.15.28:0/1772443864 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3fc072320 0x7fc3fc138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 >> 172.21.15.28:0/1772443864 conn(0x7fc3fc067d70 msgr2=0x7fc3fc0b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 shutdown_connections 2024-09-10T10:50:21.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.746+0000 7fc402800640 1 -- 172.21.15.28:0/1772443864 wait complete. 2024-09-10T10:50:21.766 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:50:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.927+0000 7f987be00640 1 Processor -- start 2024-09-10T10:50:21.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.928+0000 7f987be00640 1 -- start start 2024-09-10T10:50:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f987415e900 0x7f98740b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98740b4a20 0x7f98740ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9874072050 0x7f9874072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f98740b4620 con 0x7f9874072050 2024-09-10T10:50:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98740ce100 con 0x7f98740b4a20 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f987be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98740ce3a0 con 0x7f987415e900 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f9879e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9874072050 0x7f9874072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f9879e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9874072050 0x7f9874072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41656/0 (socket says 172.21.15.28:41656) 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f9879400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f987415e900 0x7f98740b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f9879e00640 1 -- 172.21.15.28:0/376200938 learned_addr learned my addr 172.21.15.28:0/376200938 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:21.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.929+0000 7f9878a00640 1 --2- 172.21.15.28:0/376200938 >> v2:172.21.15.138:3300/0 conn(0x7f98740b4a20 0x7f98740ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.930+0000 7f9879e00640 1 -- 172.21.15.28:0/376200938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f987415e900 msgr2=0x7f98740b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.930+0000 7f9879e00640 1 --2- 172.21.15.28:0/376200938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f987415e900 0x7f98740b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.930+0000 7f9879e00640 1 -- 172.21.15.28:0/376200938 >> v2:172.21.15.138:3300/0 conn(0x7f98740b4a20 msgr2=0x7f98740ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.930+0000 7f9879e00640 1 --2- 172.21.15.28:0/376200938 >> v2:172.21.15.138:3300/0 conn(0x7f98740b4a20 0x7f98740ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.930+0000 7f9879e00640 1 -- 172.21.15.28:0/376200938 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98740ce680 con 0x7f9874072050 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f9879e00640 1 --2- 172.21.15.28:0/376200938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f9874072500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f98740cfe80 tx=0x7f986c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d73d6b26a5b1f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f9873e00640 1 -- 172.21.15.28:0/376200938 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c036070 con 0x7f9874072050 2024-09-10T10:50:21.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f9873e00640 1 -- 172.21.15.28:0/376200938 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c031600 con 0x7f9874072050 2024-09-10T10:50:21.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f9873e00640 1 -- 172.21.15.28:0/376200938 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c0319d0 con 0x7f9874072050 2024-09-10T10:50:21.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f987be00640 1 -- 172.21.15.28:0/376200938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 msgr2=0x7f9874072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.931+0000 7f987be00640 1 --2- 172.21.15.28:0/376200938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f9874072500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f98740cfe80 tx=0x7f986c02a610 comp rx=0 tx=0).stop 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 -- 172.21.15.28:0/376200938 shutdown_connections 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 --2- 172.21.15.28:0/376200938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f9874072500 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 --2- 172.21.15.28:0/376200938 >> v2:172.21.15.138:3300/0 conn(0x7f98740b4a20 0x7f98740ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 --2- 172.21.15.28:0/376200938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f987415e900 0x7f98740b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 -- 172.21.15.28:0/376200938 >> 172.21.15.28:0/376200938 conn(0x7f987406bdf0 msgr2=0x7f987406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 -- 172.21.15.28:0/376200938 shutdown_connections 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.932+0000 7f987be00640 1 -- 172.21.15.28:0/376200938 wait complete. 2024-09-10T10:50:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.933+0000 7f987be00640 1 Processor -- start 2024-09-10T10:50:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.933+0000 7f987be00640 1 -- start start 2024-09-10T10:50:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 0x7f987409cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 0x7f9874155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98740ad560 con 0x7f9874072050 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98740b4e20 con 0x7f987415e900 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f987be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98740cee80 con 0x7f98740b4a20 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.934+0000 7f9879400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41672/0 (socket says 172.21.15.28:41672) 2024-09-10T10:50:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 -- 172.21.15.28:0/2983046365 learned_addr learned my addr 172.21.15.28:0/2983046365 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879e00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 0x7f9874155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9878a00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 0x7f987409cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 -- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 msgr2=0x7f987409cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 0x7f987409cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 -- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 msgr2=0x7f9874155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 0x7f9874155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 -- 172.21.15.28:0/2983046365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987409ad10 con 0x7f9874072050 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879e00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 0x7f9874155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9878a00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 0x7f987409cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9879400640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f986800e8a0 tx=0x7f986800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.935+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9868017070 con 0x7f9874072050 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.936+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986800c940 con 0x7f9874072050 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.936+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986800cd30 con 0x7f9874072050 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.936+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987409d760 con 0x7f9874072050 2024-09-10T10:50:21.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.936+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98740af950 con 0x7f9874072050 2024-09-10T10:50:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.937+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98740ad020 con 0x7f9874072050 2024-09-10T10:50:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.938+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f986801a070 con 0x7f9874072050 2024-09-10T10:50:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.938+0000 7f9871600640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 0x7f985406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.939+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f986808af60 con 0x7f9874072050 2024-09-10T10:50:21.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.939+0000 7f9878a00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 0x7f985406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:21.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.940+0000 7f9878a00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 0x7f985406e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98740af580 tx=0x7f9860008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:21.943+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9868059a20 con 0x7f9874072050 2024-09-10T10:50:22.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.125+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9874152730 con 0x7f9874072050 2024-09-10T10:50:22.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.129+0000 7f9871600640 1 -- 172.21.15.28:0/2983046365 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f9868059320 con 0x7f9874072050 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 msgr2=0x7f985406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 0x7f985406e2f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f98740af580 tx=0x7f9860008040 comp rx=0 tx=0).stop 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 msgr2=0x7f987409a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f986800e8a0 tx=0x7f986800ed60 comp rx=0 tx=0).stop 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 shutdown_connections 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f985406be30 0x7f985406e2f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f987415e900 0x7f9874155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98740b4a20 0x7f987409cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 --2- 172.21.15.28:0/2983046365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9874072050 0x7f987409a7d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 >> 172.21.15.28:0/2983046365 conn(0x7f987406bdf0 msgr2=0x7f98740b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 shutdown_connections 2024-09-10T10:50:22.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.132+0000 7f987be00640 1 -- 172.21.15.28:0/2983046365 wait complete. 2024-09-10T10:50:22.150 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:50:22.150 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:50:22.240 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:50:22.332 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:50:22.333 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:22.333 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:22.368 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:50:22.368 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:50:22.368 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:22.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:50:22.405 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:22.406 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:22.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:50:22.443 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:50:22.443 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:22.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:50:22.482 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:50:22.483 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:22.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.636+0000 7f6915800640 1 Processor -- start 2024-09-10T10:50:22.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.636+0000 7f6915800640 1 -- start start 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6915800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6910072d30 0x7f691006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6915800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f691006f700 0x7f691015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6915800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f691006fae0 con 0x7f691015b300 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6915800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f69100728a0 con 0x7f6910072d30 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6915800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f69100cdbe0 con 0x7f691006f700 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6910072d30 0x7f691006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6910072d30 0x7f691006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41674/0 (socket says 172.21.15.28:41674) 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 -- 172.21.15.28:0/3922837830 learned_addr learned my addr 172.21.15.28:0/3922837830 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:22.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f690f400640 1 --2- 172.21.15.28:0/3922837830 >> v2:172.21.15.138:3300/0 conn(0x7f691006f700 0x7f691015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 -- 172.21.15.28:0/3922837830 >> v2:172.21.15.138:3300/0 conn(0x7f691006f700 msgr2=0x7f691015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:22.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 --2- 172.21.15.28:0/3922837830 >> v2:172.21.15.138:3300/0 conn(0x7f691006f700 0x7f691015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 -- 172.21.15.28:0/3922837830 >> v1:172.21.15.138:6789/0 conn(0x7f691015b300 legacy=0x7f69100cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.638+0000 7f6907e00640 1 -- 172.21.15.28:0/3922837830 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69100cd900 con 0x7f6910072d30 2024-09-10T10:50:22.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.639+0000 7f6907e00640 1 --2- 172.21.15.28:0/3922837830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6910072d30 0x7f691006f1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69100b38f0 tx=0x7f68f802c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f394f4d7225c9723 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f690e000640 1 -- 172.21.15.28:0/3922837830 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8038070 con 0x7f6910072d30 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f690e000640 1 -- 172.21.15.28:0/3922837830 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8031710 con 0x7f6910072d30 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f690e000640 1 -- 172.21.15.28:0/3922837830 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8031ae0 con 0x7f6910072d30 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 -- 172.21.15.28:0/3922837830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6910072d30 msgr2=0x7f691006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 --2- 172.21.15.28:0/3922837830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6910072d30 0x7f691006f1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f69100b38f0 tx=0x7f68f802c410 comp rx=0 tx=0).stop 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 -- 172.21.15.28:0/3922837830 shutdown_connections 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 --2- 172.21.15.28:0/3922837830 >> v2:172.21.15.138:3300/0 conn(0x7f691006f700 0x7f691015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 --2- 172.21.15.28:0/3922837830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6910072d30 0x7f691006f1c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 -- 172.21.15.28:0/3922837830 >> 172.21.15.28:0/3922837830 conn(0x7f6910067d70 msgr2=0x7f691006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 -- 172.21.15.28:0/3922837830 shutdown_connections 2024-09-10T10:50:22.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.640+0000 7f6915800640 1 -- 172.21.15.28:0/3922837830 wait complete. 2024-09-10T10:50:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.642+0000 7f6915800640 1 Processor -- start 2024-09-10T10:50:22.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.642+0000 7f6915800640 1 -- start start 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691006f700 0x7f69100ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 0x7f69100cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f69100cd5c0 con 0x7f691015b300 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6910072530 con 0x7f6910072d30 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6915800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f69100cffe0 con 0x7f691006f700 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f690f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.643+0000 7f6907e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 0x7f69100cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f690f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41684/0 (socket says 172.21.15.28:41684) 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f690f400640 1 -- 172.21.15.28:0/786687322 learned_addr learned my addr 172.21.15.28:0/786687322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f690ea00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691006f700 0x7f69100ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 -- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691006f700 msgr2=0x7f69100ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691006f700 0x7f69100ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 -- 172.21.15.28:0/786687322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 msgr2=0x7f69101f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 -- 172.21.15.28:0/786687322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69101f53f0 con 0x7f6910072d30 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f690f400640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:22.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6907e00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 0x7f69100cbc50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68f8033000 tx=0x7f68f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.644+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8038040 con 0x7f6910072d30 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.645+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8033830 con 0x7f6910072d30 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.645+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8033c20 con 0x7f6910072d30 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.645+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69100cb190 con 0x7f6910072d30 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.645+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69101548b0 con 0x7f6910072d30 2024-09-10T10:50:22.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.646+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691015c190 con 0x7f6910072d30 2024-09-10T10:50:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.647+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f68f803e430 con 0x7f6910072d30 2024-09-10T10:50:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.648+0000 7f6906a00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 0x7f68ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.648+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5421+0+0 (secure 0 0 0) 0x7f68f80ab620 con 0x7f6910072d30 2024-09-10T10:50:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.648+0000 7f690ea00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 0x7f68ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.649+0000 7f690ea00640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 0x7f68ec06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6900001780 tx=0x7f6900009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.654+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f69101f53f0 con 0x7f6910072d30 2024-09-10T10:50:22.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.881+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6910068400 con 0x7f6910072d30 2024-09-10T10:50:22.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.882+0000 7f6906a00640 1 -- 172.21.15.28:0/786687322 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3917 (secure 0 0 0) 0x7f68f8079a00 con 0x7f6910072d30 2024-09-10T10:50:22.883 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:22.884 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":77,"btime":"2024-09-10T10:50:15:718449+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6058,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1686229354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1686229354},{"type":"v1","addr":"172.21.15.28:6834","nonce":1686229354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2383704388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":2383704388},{"type":"v1","addr":"172.21.15.138:6837","nonce":2383704388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73},{"gid":6340,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/1236259085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1236259085},{"type":"v1","addr":"172.21.15.138:6836","nonce":1236259085}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3070777557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3070777557},{"type":"v1","addr":"172.21.15.28:6835","nonce":3070777557}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-09-10T10:50:22.884 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 77 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.884+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 msgr2=0x7f68ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.884+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 0x7f68ec06e2f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6900001780 tx=0x7f6900009040 comp rx=0 tx=0).stop 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.884+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 msgr2=0x7f69100cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.884+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 0x7f69100cbc50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68f8033000 tx=0x7f68f80046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 shutdown_connections 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f68ec06be30 0x7f68ec06e2f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f691015b300 0x7f69101f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6910072d30 0x7f69100cbc50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 --2- 172.21.15.28:0/786687322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f691006f700 0x7f69100ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:22.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 >> 172.21.15.28:0/786687322 conn(0x7f6910067d70 msgr2=0x7f691015dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:22.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 shutdown_connections 2024-09-10T10:50:22.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:22.885+0000 7f6915800640 1 -- 172.21.15.28:0/786687322 wait complete. 2024-09-10T10:50:22.901 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:50:22.901 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:50:22.902 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:50:22.902 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:50:22.902 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:50:22.902 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:50:22.902 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:50:22.902 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:50:22.902 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:50:22.902 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:50:22.903 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:50:22.903 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:50:22.903 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:50:22.903 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:50:22.903 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:50:22.903 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:50:22.916 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:50:22.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:50:22.917 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:50:22.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:50:22.918 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:50:22.918 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:50:22.919 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:50:22.919 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:50:22.962 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:50:22.962 DEBUG:teuthology.parallel:result is None 2024-09-10T10:50:22.963 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:50:22.963 DEBUG:teuthology.parallel:result is None 2024-09-10T10:50:22.964 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:50:22.964 DEBUG:teuthology.parallel:result is None 2024-09-10T10:50:22.966 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:50:22.966 DEBUG:teuthology.parallel:result is None 2024-09-10T10:50:22.966 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:50:23.089 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:50:23.090 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:50:23.095 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:50:23.095 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:50:23.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.130+0000 7f5b7a000640 1 Processor -- start 2024-09-10T10:50:23.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.130+0000 7f5b7a000640 1 -- start start 2024-09-10T10:50:23.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b7415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b7a000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b74146140 con 0x7f5b74072700 2024-09-10T10:50:23.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b7a000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740cd740 con 0x7f5b74072320 2024-09-10T10:50:23.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b7a000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b740cda20 con 0x7f5b74072de0 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b73400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5b74072320 0x7f5b740cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:35212/0 (socket says 172.21.15.28:35212) 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b73400640 1 -- 172.21.15.28:0/2061485116 learned_addr learned my addr 172.21.15.28:0/2061485116 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.132+0000 7f5b72a00640 1 --2- 172.21.15.28:0/2061485116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b7415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.133+0000 7f5b72000640 1 -- 172.21.15.28:0/2061485116 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584348722 0 0) 0x7f5b740cd740 con 0x7f5b74072320 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.133+0000 7f5b72000640 1 -- 172.21.15.28:0/2061485116 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b58003520 con 0x7f5b74072320 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.133+0000 7f5b72a00640 1 -- 172.21.15.28:0/2061485116 >> v1:172.21.15.138:6789/0 conn(0x7f5b74072320 legacy=0x7f5b740cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.133+0000 7f5b72a00640 1 -- 172.21.15.28:0/2061485116 >> v1:172.21.15.28:6789/0 conn(0x7f5b74072700 legacy=0x7f5b7415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.133+0000 7f5b72a00640 1 -- 172.21.15.28:0/2061485116 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740cdd00 con 0x7f5b74072de0 2024-09-10T10:50:23.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.135+0000 7f5b72a00640 1 --2- 172.21.15.28:0/2061485116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b7415d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b600097d0 tx=0x7f5b6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5963029300f159c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:23.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.137+0000 7f5b72000640 1 -- 172.21.15.28:0/2061485116 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60037070 con 0x7f5b74072de0 2024-09-10T10:50:23.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.137+0000 7f5b72000640 1 -- 172.21.15.28:0/2061485116 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b60032440 con 0x7f5b74072de0 2024-09-10T10:50:23.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.137+0000 7f5b72000640 1 -- 172.21.15.28:0/2061485116 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60032810 con 0x7f5b74072de0 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 -- 172.21.15.28:0/2061485116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 msgr2=0x7f5b7415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2061485116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b7415d1e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5b600097d0 tx=0x7f5b6002a180 comp rx=0 tx=0).stop 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 -- 172.21.15.28:0/2061485116 shutdown_connections 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2061485116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b7415d1e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 -- 172.21.15.28:0/2061485116 >> 172.21.15.28:0/2061485116 conn(0x7f5b74067d70 msgr2=0x7f5b7406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 -- 172.21.15.28:0/2061485116 shutdown_connections 2024-09-10T10:50:23.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.138+0000 7f5b7a000640 1 -- 172.21.15.28:0/2061485116 wait complete. 2024-09-10T10:50:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.141+0000 7f5b7a000640 1 Processor -- start 2024-09-10T10:50:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.141+0000 7f5b7a000640 1 -- start start 2024-09-10T10:50:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 0x7f5b741e4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b740b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7415dfb0 con 0x7f5b74072700 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7415de30 con 0x7f5b74072320 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.142+0000 7f5b7a000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7415e130 con 0x7f5b74072de0 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b72a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41720/0 (socket says 172.21.15.28:41720) 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b72a00640 1 -- 172.21.15.28:0/2914533876 learned_addr learned my addr 172.21.15.28:0/2914533876 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73400640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 0x7f5b741e4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b740b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 -- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 msgr2=0x7f5b741e4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 0x7f5b741e4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 -- 172.21.15.28:0/2914533876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 msgr2=0x7f5b74135bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73e00640 1 -- 172.21.15.28:0/2914533876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b741e8e50 con 0x7f5b74072de0 2024-09-10T10:50:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b72a00640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.143+0000 7f5b73400640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 0x7f5b741e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.144+0000 7f5b73e00640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b740b36d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c690 tx=0x7f5b6c00cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.146+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c003b70 con 0x7f5b74072de0 2024-09-10T10:50:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.146+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c003d10 con 0x7f5b74072de0 2024-09-10T10:50:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.146+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c005310 con 0x7f5b74072de0 2024-09-10T10:50:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.146+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b741e32a0 con 0x7f5b74072de0 2024-09-10T10:50:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.146+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b741e3860 con 0x7f5b74072de0 2024-09-10T10:50:23.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.147+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b740cce60 con 0x7f5b74072de0 2024-09-10T10:50:23.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.148+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b6c0054b0 con 0x7f5b74072de0 2024-09-10T10:50:23.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.149+0000 7f5b6b400640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 0x7f5b5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:23.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.149+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(94..94 src has 1..94) ==== 5421+0+0 (secure 0 0 0) 0x7f5b6c08aaf0 con 0x7f5b74072de0 2024-09-10T10:50:23.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.149+0000 7f5b73400640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 0x7f5b5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:23.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.150+0000 7f5b73400640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 0x7f5b5406e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b640011e0 tx=0x7f5b64007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:23.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.154+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b6c058ed0 con 0x7f5b74072de0 2024-09-10T10:50:23.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:23.347+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5b740cf050 con 0x7f5b74072de0 2024-09-10T10:50:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.148+0000 7f5b6b400640 1 -- 172.21.15.28:0/2914533876 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) ==== 97+0+0 (secure 0 0 0) 0x7f5b6c05d1d0 con 0x7f5b74072de0 2024-09-10T10:50:24.149 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:50:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 msgr2=0x7f5b5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 0x7f5b5406e2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b640011e0 tx=0x7f5b64007d30 comp rx=0 tx=0).stop 2024-09-10T10:50:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 msgr2=0x7f5b740b36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b740b36d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c690 tx=0x7f5b6c00cb50 comp rx=0 tx=0).stop 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 shutdown_connections 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b5406be30 0x7f5b5406e2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b74072de0 0x7f5b740b36d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b74072700 0x7f5b74135bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 --2- 172.21.15.28:0/2914533876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b74072320 0x7f5b741e4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 >> 172.21.15.28:0/2914533876 conn(0x7f5b74067d70 msgr2=0x7f5b741454a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 shutdown_connections 2024-09-10T10:50:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.151+0000 7f5b7a000640 1 -- 172.21.15.28:0/2914533876 wait complete. 2024-09-10T10:50:24.171 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:50:24.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.331+0000 7f7059600640 1 Processor -- start 2024-09-10T10:50:24.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.331+0000 7f7059600640 1 -- start start 2024-09-10T10:50:24.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.332+0000 7f7059600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f705415b300 0x7f70540cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.332+0000 7f7059600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f705406f8d0 0x7f705415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.332+0000 7f7059600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70540cdbe0 con 0x7f70540cd3f0 2024-09-10T10:50:24.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.332+0000 7f7059600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f70540728a0 con 0x7f705406f8d0 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.332+0000 7f7059600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f70540cd900 con 0x7f705415b300 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f705406f8d0 0x7f705415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f705406f8d0 0x7f705415b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37366/0 (socket says 172.21.15.28:37366) 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 -- 172.21.15.28:0/918445061 learned_addr learned my addr 172.21.15.28:0/918445061 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f704a000640 1 --1- 172.21.15.28:0/918445061 >> v1:172.21.15.28:6789/0 conn(0x7f70540cd3f0 0x7f705406f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51048/0 (socket says 172.21.15.28:51048) 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7052a00640 1 --2- 172.21.15.28:0/918445061 >> v2:172.21.15.138:3300/0 conn(0x7f705415b300 0x7f70540cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 -- 172.21.15.28:0/918445061 >> v2:172.21.15.138:3300/0 conn(0x7f705415b300 msgr2=0x7f70540cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 --2- 172.21.15.28:0/918445061 >> v2:172.21.15.138:3300/0 conn(0x7f705415b300 0x7f70540cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 -- 172.21.15.28:0/918445061 >> v1:172.21.15.28:6789/0 conn(0x7f70540cd3f0 legacy=0x7f705406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.333+0000 7f7053400640 1 -- 172.21.15.28:0/918445061 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7054072250 con 0x7f705406f8d0 2024-09-10T10:50:24.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.334+0000 7f7053400640 1 --2- 172.21.15.28:0/918445061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f705406f8d0 0x7f705415b6e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7054070250 tx=0x7f704c02c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2b30c528719993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7052000640 1 -- 172.21.15.28:0/918445061 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00b2e0 con 0x7f705406f8d0 2024-09-10T10:50:24.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7052000640 1 -- 172.21.15.28:0/918445061 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c037070 con 0x7f705406f8d0 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7052000640 1 -- 172.21.15.28:0/918445061 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c00b480 con 0x7f705406f8d0 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 -- 172.21.15.28:0/918445061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f705406f8d0 msgr2=0x7f705415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 --2- 172.21.15.28:0/918445061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f705406f8d0 0x7f705415b6e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7054070250 tx=0x7f704c02c940 comp rx=0 tx=0).stop 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 -- 172.21.15.28:0/918445061 shutdown_connections 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 --2- 172.21.15.28:0/918445061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f705406f8d0 0x7f705415b6e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 --2- 172.21.15.28:0/918445061 >> v2:172.21.15.138:3300/0 conn(0x7f705415b300 0x7f70540cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 -- 172.21.15.28:0/918445061 >> 172.21.15.28:0/918445061 conn(0x7f7054067d70 msgr2=0x7f705406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 -- 172.21.15.28:0/918445061 shutdown_connections 2024-09-10T10:50:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.335+0000 7f7059600640 1 -- 172.21.15.28:0/918445061 wait complete. 2024-09-10T10:50:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.337+0000 7f7059600640 1 Processor -- start 2024-09-10T10:50:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.337+0000 7f7059600640 1 -- start start 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705406f8d0 0x7f7054140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 0x7f7054155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f70540d0160 con 0x7f70540cd3f0 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70540cffe0 con 0x7f705406f8d0 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7059600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7054072d30 con 0x7f705415b300 2024-09-10T10:50:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f704a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f704a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37372/0 (socket says 172.21.15.28:37372) 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f704a000640 1 -- 172.21.15.28:0/933927854 learned_addr learned my addr 172.21.15.28:0/933927854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7052a00640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705406f8d0 0x7f7054140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.338+0000 7f7053400640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 0x7f7054155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 -- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 msgr2=0x7f7054155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 0x7f7054155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 -- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705406f8d0 msgr2=0x7f7054140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705406f8d0 0x7f7054140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 -- 172.21.15.28:0/933927854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7054156520 con 0x7f70540cd3f0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f7053400640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 0x7f7054155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704a000640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c400 tx=0x7f703c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c0074e0 con 0x7f70540cd3f0 2024-09-10T10:50:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f703c007680 con 0x7f70540cd3f0 2024-09-10T10:50:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c004a00 con 0x7f70540cd3f0 2024-09-10T10:50:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7054156800 con 0x7f70540cd3f0 2024-09-10T10:50:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.339+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7054156dc0 con 0x7f70540cd3f0 2024-09-10T10:50:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.341+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70540b0da0 con 0x7f70540cd3f0 2024-09-10T10:50:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.341+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f703c01a020 con 0x7f70540cd3f0 2024-09-10T10:50:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.342+0000 7f704b400640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 0x7f703006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.342+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7f703c08a500 con 0x7f70540cd3f0 2024-09-10T10:50:24.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.343+0000 7f7052a00640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 0x7f703006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.344+0000 7f7052a00640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 0x7f703006e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7044009aa0 tx=0x7f7044009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.346+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f703c058fd0 con 0x7f70540cd3f0 2024-09-10T10:50:24.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.598+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f705415a8a0 con 0x7f70540cd3f0 2024-09-10T10:50:24.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.598+0000 7f704b400640 1 -- 172.21.15.28:0/933927854 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f703c0588d0 con 0x7f70540cd3f0 2024-09-10T10:50:24.599 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:24.599 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:24.599 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:50:24.599 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:50:24.600 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:50:24.601 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:50:24.602 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.603 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:50:24.604 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.605 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.606 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.607 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.608 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:50:24.609 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:50:24.610 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.611 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.612 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.613 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.614 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:50:24.614 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:24.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 msgr2=0x7f703006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 0x7f703006e2f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7044009aa0 tx=0x7f7044009190 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 msgr2=0x7f7054155960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f703c00c400 tx=0x7f703c00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 shutdown_connections 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f703006be30 0x7f703006e2f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f705415b300 0x7f7054155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70540cd3f0 0x7f7054155960 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 --2- 172.21.15.28:0/933927854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705406f8d0 0x7f7054140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 >> 172.21.15.28:0/933927854 conn(0x7f7054067d70 msgr2=0x7f70540b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 shutdown_connections 2024-09-10T10:50:24.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.600+0000 7f7059600640 1 -- 172.21.15.28:0/933927854 wait complete. 2024-09-10T10:50:24.618 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:50:24.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.774+0000 7fd98aa00640 1 Processor -- start 2024-09-10T10:50:24.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.774+0000 7fd98aa00640 1 -- start start 2024-09-10T10:50:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd98aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 0x7fd9840b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd98aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd9840b64e0 0x7fd98406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd98aa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9840d37d0 con 0x7fd98415e610 2024-09-10T10:50:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd98aa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd9840cfeb0 con 0x7fd9840b64e0 2024-09-10T10:50:24.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd98aa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd9840cecf0 con 0x7fd9840cfab0 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.775+0000 7fd988a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd9840b64e0 0x7fd98406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd9840b64e0 0x7fd98406cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37374/0 (socket says 172.21.15.28:37374) 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 -- 172.21.15.28:0/388716292 learned_addr learned my addr 172.21.15.28:0/388716292 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd983e00640 1 --1- 172.21.15.28:0/388716292 >> v1:172.21.15.28:6789/0 conn(0x7fd98415e610 0x7fd9840d30c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51056/0 (socket says 172.21.15.28:51056) 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 -- 172.21.15.28:0/388716292 >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 msgr2=0x7fd9840b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd983400640 1 --2- 172.21.15.28:0/388716292 >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 0x7fd9840b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 --2- 172.21.15.28:0/388716292 >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 0x7fd9840b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 -- 172.21.15.28:0/388716292 >> v1:172.21.15.28:6789/0 conn(0x7fd98415e610 legacy=0x7fd9840d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd982a00640 1 -- 172.21.15.28:0/388716292 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546910913 0 0) 0x7fd9840d37d0 con 0x7fd98415e610 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd988a00640 1 -- 172.21.15.28:0/388716292 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9840cefd0 con 0x7fd9840b64e0 2024-09-10T10:50:24.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.776+0000 7fd983400640 1 --2- 172.21.15.28:0/388716292 >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 0x7fd9840b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.777+0000 7fd988a00640 1 --2- 172.21.15.28:0/388716292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9840b64e0 0x7fd98406cdd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b3f50 tx=0x7fd97c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda675c2606457a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.777+0000 7fd982a00640 1 -- 172.21.15.28:0/388716292 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c036070 con 0x7fd9840b64e0 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.777+0000 7fd982a00640 1 -- 172.21.15.28:0/388716292 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c031600 con 0x7fd9840b64e0 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.777+0000 7fd982a00640 1 -- 172.21.15.28:0/388716292 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0319f0 con 0x7fd9840b64e0 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 -- 172.21.15.28:0/388716292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9840b64e0 msgr2=0x7fd98406cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 --2- 172.21.15.28:0/388716292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9840b64e0 0x7fd98406cdd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fd9840b3f50 tx=0x7fd97c02a7c0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 -- 172.21.15.28:0/388716292 shutdown_connections 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 --2- 172.21.15.28:0/388716292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9840b64e0 0x7fd98406cdd0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 --2- 172.21.15.28:0/388716292 >> v2:172.21.15.138:3300/0 conn(0x7fd9840cfab0 0x7fd9840b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 -- 172.21.15.28:0/388716292 >> 172.21.15.28:0/388716292 conn(0x7fd98408bfc0 msgr2=0x7fd9840d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 -- 172.21.15.28:0/388716292 shutdown_connections 2024-09-10T10:50:24.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.778+0000 7fd98aa00640 1 -- 172.21.15.28:0/388716292 wait complete. 2024-09-10T10:50:24.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.780+0000 7fd98aa00640 1 Processor -- start 2024-09-10T10:50:24.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.780+0000 7fd98aa00640 1 -- start start 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 0x7fd9840cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9840cfab0 0x7fd9840b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd98415e9f0 con 0x7fd98415e610 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9840cf590 con 0x7fd9840cfab0 2024-09-10T10:50:24.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd98aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9840d4f90 con 0x7fd9840b64e0 2024-09-10T10:50:24.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd988a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd988a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37378/0 (socket says 172.21.15.28:37378) 2024-09-10T10:50:24.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd988a00640 1 -- 172.21.15.28:0/1176623418 learned_addr learned my addr 172.21.15.28:0/1176623418 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:24.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd983e00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 0x7fd9840cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.781+0000 7fd983400640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9840cfab0 0x7fd9840b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 -- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 msgr2=0x7fd9840cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 0x7fd9840cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 -- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9840cfab0 msgr2=0x7fd9840b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9840cfab0 0x7fd9840b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 -- 172.21.15.28:0/1176623418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9840c8290 con 0x7fd98415e610 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd983e00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 0x7fd9840cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd988a00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009f40 tx=0x7fd97c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c036050 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c042070 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9840cb790 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c009510 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.782+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9840cbd50 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.783+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9840cb230 con 0x7fd98415e610 2024-09-10T10:50:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.784+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd97c007c50 con 0x7fd98415e610 2024-09-10T10:50:24.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.785+0000 7fd97be00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 0x7fd96406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:24.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.785+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7fd97c0abc40 con 0x7fd98415e610 2024-09-10T10:50:24.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.786+0000 7fd983e00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 0x7fd96406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:24.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.786+0000 7fd983e00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 0x7fd96406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd974000f60 tx=0x7fd974007ca0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:24.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.789+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9840cb230 con 0x7fd98415e610 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.977+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd9840b5fa0 con 0x7fd98415e610 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.978+0000 7fd97be00640 1 -- 172.21.15.28:0/1176623418 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v95) ==== 84+0+65 (secure 0 0 0) 0x7fd97c045360 con 0x7fd98415e610 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:50:24.979 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:50:24.980 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:50:24.980 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:50:24.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 msgr2=0x7fd96406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 0x7fd96406e2f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd974000f60 tx=0x7fd974007ca0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 msgr2=0x7fd9840c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009f40 tx=0x7fd97c004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 shutdown_connections 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd96406be30 0x7fd96406e2f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd98415e610 0x7fd9840c8e20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9840cfab0 0x7fd9840b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 --2- 172.21.15.28:0/1176623418 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9840b64e0 0x7fd9840cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 >> 172.21.15.28:0/1176623418 conn(0x7fd98408bfc0 msgr2=0x7fd9840d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.980+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 shutdown_connections 2024-09-10T10:50:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:24.981+0000 7fd98aa00640 1 -- 172.21.15.28:0/1176623418 wait complete. 2024-09-10T10:50:24.997 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:50:24.998 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:50:25.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.158+0000 7f1a35800640 1 Processor -- start 2024-09-10T10:50:25.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.158+0000 7f1a35800640 1 -- start start 2024-09-10T10:50:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.159+0000 7f1a35800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 0x7f1a30162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.159+0000 7f1a35800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a300d2b50 con 0x7f1a3006dc80 2024-09-10T10:50:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.159+0000 7f1a35800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a300d2e30 con 0x7f1a300d4360 2024-09-10T10:50:25.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.159+0000 7f1a35800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a300b3af0 con 0x7f1a3006e6b0 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a34e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1a3006dc80 0x7f1a300ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51066/0 (socket says 172.21.15.28:51066) 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a34e00640 1 -- 172.21.15.28:0/735405536 learned_addr learned my addr 172.21.15.28:0/735405536 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a27e00640 1 --2- 172.21.15.28:0/735405536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 0x7f1a30162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842365611 0 0) 0x7f1a300d2b50 con 0x7f1a3006dc80 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a14003520 con 0x7f1a3006dc80 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4142488340 0 0) 0x7f1a300d2e30 con 0x7f1a300d4360 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a300d2b50 con 0x7f1a300d4360 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a27e00640 1 -- 172.21.15.28:0/735405536 >> v1:172.21.15.138:6789/0 conn(0x7f1a300d4360 legacy=0x7f1a300d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a27e00640 1 -- 172.21.15.28:0/735405536 >> v1:172.21.15.28:6789/0 conn(0x7f1a3006dc80 legacy=0x7f1a300ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:25.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.160+0000 7f1a27e00640 1 -- 172.21.15.28:0/735405536 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3006d1a0 con 0x7f1a3006e6b0 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a27e00640 1 --2- 172.21.15.28:0/735405536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 0x7f1a30162990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a180097d0 tx=0x7f1a1802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5fa1a98f60e163 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18037070 con 0x7f1a3006e6b0 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18032440 con 0x7f1a3006e6b0 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a2f400640 1 -- 172.21.15.28:0/735405536 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a180327f0 con 0x7f1a3006e6b0 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a35800640 1 -- 172.21.15.28:0/735405536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 msgr2=0x7f1a30162990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a35800640 1 --2- 172.21.15.28:0/735405536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 0x7f1a30162990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1a180097d0 tx=0x7f1a1802a290 comp rx=0 tx=0).stop 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a35800640 1 -- 172.21.15.28:0/735405536 shutdown_connections 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.162+0000 7f1a35800640 1 --2- 172.21.15.28:0/735405536 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a3006e6b0 0x7f1a30162990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.163+0000 7f1a35800640 1 -- 172.21.15.28:0/735405536 >> 172.21.15.28:0/735405536 conn(0x7f1a3009d740 msgr2=0x7f1a3008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.163+0000 7f1a35800640 1 -- 172.21.15.28:0/735405536 shutdown_connections 2024-09-10T10:50:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.163+0000 7f1a35800640 1 -- 172.21.15.28:0/735405536 wait complete. 2024-09-10T10:50:25.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.164+0000 7f1a35800640 1 Processor -- start 2024-09-10T10:50:25.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.164+0000 7f1a35800640 1 -- start start 2024-09-10T10:50:25.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 0x7f1a300b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 0x7f1a300ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3006d760 con 0x7f1a3006dc80 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30162ed0 con 0x7f1a3006e6b0 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a35800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3015e6d0 con 0x7f1a300d4360 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.165+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37382/0 (socket says 172.21.15.28:37382) 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 -- 172.21.15.28:0/847862197 learned_addr learned my addr 172.21.15.28:0/847862197 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a34e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 0x7f1a300ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a27e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 0x7f1a300b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 -- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 msgr2=0x7f1a300ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:25.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 0x7f1a300ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 -- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 msgr2=0x7f1a300b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 0x7f1a300b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 -- 172.21.15.28:0/847862197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a300c7a60 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a34e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 0x7f1a300ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a27e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 0x7f1a300b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.166+0000 7f1a2fe00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c460 tx=0x7f1a2000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.167+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20017070 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.167+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a200052f0 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.167+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20003830 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.167+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a300c7d00 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.167+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a300c9a20 con 0x7f1a3006dc80 2024-09-10T10:50:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.168+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a30162990 con 0x7f1a3006dc80 2024-09-10T10:50:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.169+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a2001a070 con 0x7f1a3006dc80 2024-09-10T10:50:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.170+0000 7f1a2cc00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 0x7f1a0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.170+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5249+0+0 (secure 0 0 0) 0x7f1a2008a130 con 0x7f1a3006dc80 2024-09-10T10:50:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.170+0000 7f1a27e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 0x7f1a0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.171+0000 7f1a27e00640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 0x7f1a0c06e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1a18034000 tx=0x7f1a1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.174+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a20058c10 con 0x7f1a3006dc80 2024-09-10T10:50:25.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:25.356+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1a300b50e0 con 0x7f1a3006dc80 2024-09-10T10:50:26.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.149+0000 7f1a2cc00640 1 -- 172.21.15.28:0/847862197 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v96) ==== 138+0+0 (secure 0 0 0) 0x7f1a20077020 con 0x7f1a3006dc80 2024-09-10T10:50:26.150 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.152+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 msgr2=0x7f1a0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.152+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 0x7f1a0c06e2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1a18034000 tx=0x7f1a1802e000 comp rx=0 tx=0).stop 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.152+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 msgr2=0x7f1a301f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.152+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c460 tx=0x7f1a2000c920 comp rx=0 tx=0).stop 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 shutdown_connections 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a0c06be30 0x7f1a0c06e2f0 secure :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1a18034000 tx=0x7f1a1802e000 comp rx=0 tx=0).stop 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a300d4360 0x7f1a300ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a3006e6b0 0x7f1a300b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 --2- 172.21.15.28:0/847862197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a3006dc80 0x7f1a301f8830 secure :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c460 tx=0x7f1a2000c920 comp rx=0 tx=0).stop 2024-09-10T10:50:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 >> 172.21.15.28:0/847862197 conn(0x7f1a3009d740 msgr2=0x7f1a300cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 shutdown_connections 2024-09-10T10:50:26.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.153+0000 7f1a35800640 1 -- 172.21.15.28:0/847862197 wait complete. 2024-09-10T10:50:26.177 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:50:26.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.343+0000 7f5c73e00640 1 Processor -- start 2024-09-10T10:50:26.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.343+0000 7f5c73e00640 1 -- start start 2024-09-10T10:50:26.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.344+0000 7f5c73e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5c7406e6b0 0x7f5c74162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.344+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.344+0000 7f5c73e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c740d2b50 con 0x7f5c740d4360 2024-09-10T10:50:26.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.344+0000 7f5c73e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5c740b3af0 con 0x7f5c7406e6b0 2024-09-10T10:50:26.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.344+0000 7f5c73e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c74162ed0 con 0x7f5c7406dc80 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:55312/0 (socket says 172.21.15.28:55312) 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c72a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5c740d4360 0x7f5c740d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52630/0 (socket says 172.21.15.28:52630) 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 -- 172.21.15.28:0/3191552936 learned_addr learned my addr 172.21.15.28:0/3191552936 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c6be00640 1 --2- 172.21.15.28:0/3191552936 >> v2:172.21.15.138:3300/0 conn(0x7f5c7406e6b0 0x7f5c74162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 -- 172.21.15.28:0/3191552936 >> v2:172.21.15.138:3300/0 conn(0x7f5c7406e6b0 msgr2=0x7f5c74162990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 --2- 172.21.15.28:0/3191552936 >> v2:172.21.15.138:3300/0 conn(0x7f5c7406e6b0 0x7f5c74162990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 -- 172.21.15.28:0/3191552936 >> v1:172.21.15.138:6789/0 conn(0x7f5c740d4360 legacy=0x7f5c740d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c73400640 1 -- 172.21.15.28:0/3191552936 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c740d2e30 con 0x7f5c7406dc80 2024-09-10T10:50:26.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.345+0000 7f5c72000640 1 -- 172.21.15.28:0/3191552936 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1577932730 0 0) 0x7f5c740d2b50 con 0x7f5c740d4360 2024-09-10T10:50:26.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.346+0000 7f5c73400640 1 --2- 172.21.15.28:0/3191552936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e01ac4307a24fde2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:26.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c72000640 1 -- 172.21.15.28:0/3191552936 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c036070 con 0x7f5c7406dc80 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c72000640 1 -- 172.21.15.28:0/3191552936 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c031600 con 0x7f5c7406dc80 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c72000640 1 -- 172.21.15.28:0/3191552936 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0319d0 con 0x7f5c7406dc80 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 -- 172.21.15.28:0/3191552936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 msgr2=0x7f5c7406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 --2- 172.21.15.28:0/3191552936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c009e50 tx=0x7f5c6c02a7a0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 -- 172.21.15.28:0/3191552936 shutdown_connections 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 --2- 172.21.15.28:0/3191552936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c7406dc80 0x7f5c7406e110 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 --2- 172.21.15.28:0/3191552936 >> v2:172.21.15.138:3300/0 conn(0x7f5c7406e6b0 0x7f5c74162990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 -- 172.21.15.28:0/3191552936 >> 172.21.15.28:0/3191552936 conn(0x7f5c7409d740 msgr2=0x7f5c7408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 -- 172.21.15.28:0/3191552936 shutdown_connections 2024-09-10T10:50:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.347+0000 7f5c73e00640 1 -- 172.21.15.28:0/3191552936 wait complete. 2024-09-10T10:50:26.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.349+0000 7f5c73e00640 1 Processor -- start 2024-09-10T10:50:26.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.349+0000 7f5c73e00640 1 -- start start 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7406e6b0 0x7f5c740c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c740d4360 0x7f5c74151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7406d760 con 0x7f5c7406dc80 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7406d480 con 0x7f5c7406e6b0 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c73e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7415e6d0 con 0x7f5c740d4360 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c72a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c72a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37388/0 (socket says 172.21.15.28:37388) 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c72a00640 1 -- 172.21.15.28:0/2094123684 learned_addr learned my addr 172.21.15.28:0/2094123684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.350+0000 7f5c6be00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7406e6b0 0x7f5c740c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c73400640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c740d4360 0x7f5c74151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 -- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c740d4360 msgr2=0x7f5c74151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c740d4360 0x7f5c74151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 -- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7406e6b0 msgr2=0x7f5c740c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7406e6b0 0x7f5c740c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 -- 172.21.15.28:0/2094123684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c74161180 con 0x7f5c7406dc80 2024-09-10T10:50:26.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c72a00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c460 tx=0x7f5c6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64018070 con 0x7f5c7406dc80 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c64007500 con 0x7f5c7406dc80 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004a20 con 0x7f5c7406dc80 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.351+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c74161460 con 0x7f5c7406dc80 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.352+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c740b8380 con 0x7f5c7406dc80 2024-09-10T10:50:26.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.353+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7406e110 con 0x7f5c7406dc80 2024-09-10T10:50:26.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.354+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c6401b360 con 0x7f5c7406dc80 2024-09-10T10:50:26.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.355+0000 7f5c6b400640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:26.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.355+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5619+0+0 (secure 0 0 0) 0x7f5c6408a2a0 con 0x7f5c7406dc80 2024-09-10T10:50:26.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.355+0000 7f5c6be00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:26.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.356+0000 7f5c6be00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 0x7f5c5006e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c740b7470 tx=0x7f5c5c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:26.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.359+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c64021080 con 0x7f5c7406dc80 2024-09-10T10:50:26.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:26.544+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5c740b5c30 con 0x7f5c7406dc80 2024-09-10T10:50:27.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.159+0000 7f5c6b400640 1 -- 172.21.15.28:0/2094123684 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v97) ==== 170+0+0 (secure 0 0 0) 0x7f5c64058730 con 0x7f5c7406dc80 2024-09-10T10:50:27.160 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:50:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 msgr2=0x7f5c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 0x7f5c5006e2f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c740b7470 tx=0x7f5c5c009190 comp rx=0 tx=0).stop 2024-09-10T10:50:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 msgr2=0x7f5c741608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c460 tx=0x7f5c6400c920 comp rx=0 tx=0).stop 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 shutdown_connections 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c5006be30 0x7f5c5006e2f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5c740d4360 0x7f5c74151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7406e6b0 0x7f5c740c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 --2- 172.21.15.28:0/2094123684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c7406dc80 0x7f5c741608e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 >> 172.21.15.28:0/2094123684 conn(0x7f5c7409d740 msgr2=0x7f5c740d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.162+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 shutdown_connections 2024-09-10T10:50:27.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.163+0000 7f5c73e00640 1 -- 172.21.15.28:0/2094123684 wait complete. 2024-09-10T10:50:27.197 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:50:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.356+0000 7fca76c00640 1 Processor -- start 2024-09-10T10:50:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.356+0000 7fca76c00640 1 -- start start 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.357+0000 7fca76c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca700d15f0 0x7fca700cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.357+0000 7fca76c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca700cf160 0x7fca7006d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.357+0000 7fca76c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca700cf980 con 0x7fca70161a20 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.357+0000 7fca76c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fca70161e00 con 0x7fca700d15f0 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.357+0000 7fca76c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca7006d050 con 0x7fca700cf160 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca700d15f0 0x7fca700cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca700d15f0 0x7fca700cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37404/0 (socket says 172.21.15.28:37404) 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 -- 172.21.15.28:0/131794929 learned_addr learned my addr 172.21.15.28:0/131794929 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca74c00640 1 --2- 172.21.15.28:0/131794929 >> v2:172.21.15.138:3300/0 conn(0x7fca700cf160 0x7fca7006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 -- 172.21.15.28:0/131794929 >> v2:172.21.15.138:3300/0 conn(0x7fca700cf160 msgr2=0x7fca7006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:27.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 --2- 172.21.15.28:0/131794929 >> v2:172.21.15.138:3300/0 conn(0x7fca700cf160 0x7fca7006d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 -- 172.21.15.28:0/131794929 >> v1:172.21.15.138:6789/0 conn(0x7fca70161a20 legacy=0x7fca700d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.358+0000 7fca67400640 1 -- 172.21.15.28:0/131794929 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca700cfc20 con 0x7fca700d15f0 2024-09-10T10:50:27.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.359+0000 7fca67400640 1 --2- 172.21.15.28:0/131794929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca700d15f0 0x7fca700cec20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fca700d4410 tx=0x7fca5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ae85dd68f47866 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:27.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.359+0000 7fca6f400640 1 -- 172.21.15.28:0/131794929 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58037070 con 0x7fca700d15f0 2024-09-10T10:50:27.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.359+0000 7fca6f400640 1 -- 172.21.15.28:0/131794929 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca58032440 con 0x7fca700d15f0 2024-09-10T10:50:27.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.359+0000 7fca6f400640 1 -- 172.21.15.28:0/131794929 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58032810 con 0x7fca700d15f0 2024-09-10T10:50:27.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 -- 172.21.15.28:0/131794929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca700d15f0 msgr2=0x7fca700cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 --2- 172.21.15.28:0/131794929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca700d15f0 0x7fca700cec20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fca700d4410 tx=0x7fca5802a270 comp rx=0 tx=0).stop 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 -- 172.21.15.28:0/131794929 shutdown_connections 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 --2- 172.21.15.28:0/131794929 >> v2:172.21.15.138:3300/0 conn(0x7fca700cf160 0x7fca7006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 --2- 172.21.15.28:0/131794929 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca700d15f0 0x7fca700cec20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 -- 172.21.15.28:0/131794929 >> 172.21.15.28:0/131794929 conn(0x7fca7009d850 msgr2=0x7fca7008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 -- 172.21.15.28:0/131794929 shutdown_connections 2024-09-10T10:50:27.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.360+0000 7fca76c00640 1 -- 172.21.15.28:0/131794929 wait complete. 2024-09-10T10:50:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.362+0000 7fca76c00640 1 Processor -- start 2024-09-10T10:50:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.362+0000 7fca76c00640 1 -- start start 2024-09-10T10:50:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca700cf160 0x7fca700b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca700d15f0 0x7fca700b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca700b1040 con 0x7fca70161a20 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca7006d2f0 con 0x7fca700d15f0 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca76c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca700cf590 con 0x7fca700cf160 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37406/0 (socket says 172.21.15.28:37406) 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 -- 172.21.15.28:0/2603269923 learned_addr learned my addr 172.21.15.28:0/2603269923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca67400640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca700d15f0 0x7fca700b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca6fe00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca700cf160 0x7fca700b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 -- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca700cf160 msgr2=0x7fca700b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca700cf160 0x7fca700b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 -- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca700d15f0 msgr2=0x7fca700b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca700d15f0 0x7fca700b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.363+0000 7fca74c00640 1 -- 172.21.15.28:0/2603269923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca700b4060 con 0x7fca70161a20 2024-09-10T10:50:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca74c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fca6800efc0 tx=0x7fca6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68018020 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca68014480 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68014870 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca700b4340 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.364+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca700b7320 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.365+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca700cec20 con 0x7fca70161a20 2024-09-10T10:50:27.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.366+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca68005d10 con 0x7fca70161a20 2024-09-10T10:50:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.367+0000 7fca6cc00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.367+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6001+0+0 (secure 0 0 0) 0x7fca6808bce0 con 0x7fca70161a20 2024-09-10T10:50:27.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.367+0000 7fca6fe00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:27.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.368+0000 7fca6fe00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fca60000f60 tx=0x7fca60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:27.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.372+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca68022030 con 0x7fca70161a20 2024-09-10T10:50:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:27.592+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fca700b75b0 con 0x7fca70161a20 2024-09-10T10:50:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.161+0000 7fca6cc00640 1 -- 172.21.15.28:0/2603269923 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v79) ==== 174+0+0 (secure 0 0 0) 0x7fca68010040 con 0x7fca70161a20 2024-09-10T10:50:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 13 and data pool 14 2024-09-10T10:50:28.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.168+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 msgr2=0x7fca4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.168+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fca60000f60 tx=0x7fca60009040 comp rx=0 tx=0).stop 2024-09-10T10:50:28.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.168+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 msgr2=0x7fca700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.168+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fca6800efc0 tx=0x7fca6800c490 comp rx=0 tx=0).stop 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 shutdown_connections 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca70161a20 0x7fca700b4e20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca700d15f0 0x7fca700b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 --2- 172.21.15.28:0/2603269923 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca700cf160 0x7fca700b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.169+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 >> 172.21.15.28:0/2603269923 conn(0x7fca7009d850 msgr2=0x7fca700d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.170+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 shutdown_connections 2024-09-10T10:50:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.170+0000 7fca76c00640 1 -- 172.21.15.28:0/2603269923 wait complete. 2024-09-10T10:50:28.212 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:28.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.368+0000 7ff20b200640 1 Processor -- start 2024-09-10T10:50:28.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.368+0000 7ff20b200640 1 -- start start 2024-09-10T10:50:28.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff20b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff20c15e660 0x7ff20c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff20b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff20c0d1f90 0x7ff20c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff20b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff20c0cf000 con 0x7ff20c0d0dd0 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff20b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff20c0b37d0 con 0x7ff20c15e660 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff20b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff20c0cf330 con 0x7ff20c0d1f90 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff209400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff20c15e660 0x7ff20c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff209e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff20c0d0dd0 0x7ff20c0d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51082/0 (socket says 172.21.15.28:51082) 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff209400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff20c15e660 0x7ff20c06ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37412/0 (socket says 172.21.15.28:37412) 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.369+0000 7ff209e00640 1 -- 172.21.15.28:0/3388295227 learned_addr learned my addr 172.21.15.28:0/3388295227 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:28.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.370+0000 7ff20a800640 1 --2- 172.21.15.28:0/3388295227 >> v2:172.21.15.138:3300/0 conn(0x7ff20c0d1f90 0x7ff20c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.370+0000 7ff209400640 1 -- 172.21.15.28:0/3388295227 >> v2:172.21.15.138:3300/0 conn(0x7ff20c0d1f90 msgr2=0x7ff20c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.370+0000 7ff209400640 1 --2- 172.21.15.28:0/3388295227 >> v2:172.21.15.138:3300/0 conn(0x7ff20c0d1f90 0x7ff20c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.370+0000 7ff209400640 1 -- 172.21.15.28:0/3388295227 >> v1:172.21.15.28:6789/0 conn(0x7ff20c0d0dd0 legacy=0x7ff20c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.370+0000 7ff209400640 1 -- 172.21.15.28:0/3388295227 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c0d29a0 con 0x7ff20c15e660 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.371+0000 7ff209400640 1 --2- 172.21.15.28:0/3388295227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c15e660 0x7ff20c06ccd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0cb420 tx=0x7ff1f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbe1f770f10c492 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.371+0000 7ff208a00640 1 -- 172.21.15.28:0/3388295227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037070 con 0x7ff20c15e660 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.371+0000 7ff208a00640 1 -- 172.21.15.28:0/3388295227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f8032440 con 0x7ff20c15e660 2024-09-10T10:50:28.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.371+0000 7ff208a00640 1 -- 172.21.15.28:0/3388295227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8032810 con 0x7ff20c15e660 2024-09-10T10:50:28.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 -- 172.21.15.28:0/3388295227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c15e660 msgr2=0x7ff20c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 --2- 172.21.15.28:0/3388295227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c15e660 0x7ff20c06ccd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0cb420 tx=0x7ff1f802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 -- 172.21.15.28:0/3388295227 shutdown_connections 2024-09-10T10:50:28.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 --2- 172.21.15.28:0/3388295227 >> v2:172.21.15.138:3300/0 conn(0x7ff20c0d1f90 0x7ff20c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 --2- 172.21.15.28:0/3388295227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c15e660 0x7ff20c06ccd0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 -- 172.21.15.28:0/3388295227 >> 172.21.15.28:0/3388295227 conn(0x7ff20c08bfc0 msgr2=0x7ff20c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 -- 172.21.15.28:0/3388295227 shutdown_connections 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.372+0000 7ff20b200640 1 -- 172.21.15.28:0/3388295227 wait complete. 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.374+0000 7ff20b200640 1 Processor -- start 2024-09-10T10:50:28.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.374+0000 7ff20b200640 1 -- start start 2024-09-10T10:50:28.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff20c0d0dd0 0x7ff20c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff20c15e660 0x7ff20c0b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c0d3220 con 0x7ff20c0d1f90 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c0d2f40 con 0x7ff20c0d0dd0 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff20c0d33a0 con 0x7ff20c15e660 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37424/0 (socket says 172.21.15.28:37424) 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 -- 172.21.15.28:0/3483141986 learned_addr learned my addr 172.21.15.28:0/3483141986 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209e00640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff20c0d0dd0 0x7ff20c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff20a800640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff20c15e660 0x7ff20c0b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 -- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff20c15e660 msgr2=0x7ff20c0b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff20c15e660 0x7ff20c0b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 -- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff20c0d0dd0 msgr2=0x7ff20c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff20c0d0dd0 0x7ff20c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.375+0000 7ff209400640 1 -- 172.21.15.28:0/3483141986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c14dc50 con 0x7ff20c0d1f90 2024-09-10T10:50:28.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff209400640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009890 tx=0x7ff1f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8037040 con 0x7ff20c0d1f90 2024-09-10T10:50:28.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f8042070 con 0x7ff20c0d1f90 2024-09-10T10:50:28.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8030bd0 con 0x7ff20c0d1f90 2024-09-10T10:50:28.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c1e5b90 con 0x7ff20c0d1f90 2024-09-10T10:50:28.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.376+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c14b440 con 0x7ff20c0d1f90 2024-09-10T10:50:28.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.378+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff20c1e60b0 con 0x7ff20c0d1f90 2024-09-10T10:50:28.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.379+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1f8047460 con 0x7ff20c0d1f90 2024-09-10T10:50:28.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.379+0000 7ff202000640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.379+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7ff1f80abae0 con 0x7ff20c0d1f90 2024-09-10T10:50:28.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.380+0000 7ff209e00640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.381+0000 7ff209e00640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc007fd0 tx=0x7ff1fc007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.384+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff20c1e60b0 con 0x7ff20c0d1f90 2024-09-10T10:50:28.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.570+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff20c0d2440 con 0x7ff20c0d1f90 2024-09-10T10:50:28.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.571+0000 7ff202000640 1 -- 172.21.15.28:0/3483141986 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+17030 (secure 0 0 0) 0x7ff20c14dc50 con 0x7ff20c0d1f90 2024-09-10T10:50:28.572 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:28.573 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":98,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:27.593162+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:50:25.399081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T10:50:26.587379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.573+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 msgr2=0x7ff1ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.573+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc007fd0 tx=0x7ff1fc007e80 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.573+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 msgr2=0x7ff20c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.573+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8009890 tx=0x7ff1f80046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 shutdown_connections 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff1ec06be30 0x7ff1ec06e2f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff20c15e660 0x7ff20c0b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff20c0d1f90 0x7ff20c0cb5e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 --2- 172.21.15.28:0/3483141986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff20c0d0dd0 0x7ff20c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 >> 172.21.15.28:0/3483141986 conn(0x7ff20c08bfc0 msgr2=0x7ff20c15ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:28.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 shutdown_connections 2024-09-10T10:50:28.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.574+0000 7ff20b200640 1 -- 172.21.15.28:0/3483141986 wait complete. 2024-09-10T10:50:28.591 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:28.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.747+0000 7f8281800640 1 Processor -- start 2024-09-10T10:50:28.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.747+0000 7f8281800640 1 -- start start 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8281800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f827c0d0dd0 0x7f827c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8281800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f827c0d1f90 0x7f827c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8281800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f827c0cf000 con 0x7f827c15e660 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8281800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f827c06d3e0 con 0x7f827c0d1f90 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8281800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f827c0b37d0 con 0x7f827c0d0dd0 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8280e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f827c0d1f90 0x7f827c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8280e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f827c0d1f90 0x7f827c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37438/0 (socket says 172.21.15.28:37438) 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.748+0000 7f8280e00640 1 -- 172.21.15.28:0/604295906 learned_addr learned my addr 172.21.15.28:0/604295906 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.749+0000 7f827be00640 1 --2- 172.21.15.28:0/604295906 >> v2:172.21.15.138:3300/0 conn(0x7f827c0d0dd0 0x7f827c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.749+0000 7f8280e00640 1 -- 172.21.15.28:0/604295906 >> v2:172.21.15.138:3300/0 conn(0x7f827c0d0dd0 msgr2=0x7f827c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.749+0000 7f8280e00640 1 --2- 172.21.15.28:0/604295906 >> v2:172.21.15.138:3300/0 conn(0x7f827c0d0dd0 0x7f827c0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.749+0000 7f8280e00640 1 -- 172.21.15.28:0/604295906 >> v1:172.21.15.138:6789/0 conn(0x7f827c15e660 legacy=0x7f827c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.749+0000 7f8280e00640 1 -- 172.21.15.28:0/604295906 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c0d16f0 con 0x7f827c0d1f90 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.750+0000 7f8280e00640 1 --2- 172.21.15.28:0/604295906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d1f90 0x7f827c0cf7c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b6fb0 tx=0x7f827402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6310213be43c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.750+0000 7f827aa00640 1 -- 172.21.15.28:0/604295906 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274036070 con 0x7f827c0d1f90 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.750+0000 7f827aa00640 1 -- 172.21.15.28:0/604295906 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8274031600 con 0x7f827c0d1f90 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.750+0000 7f827aa00640 1 -- 172.21.15.28:0/604295906 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740319d0 con 0x7f827c0d1f90 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 -- 172.21.15.28:0/604295906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d1f90 msgr2=0x7f827c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 --2- 172.21.15.28:0/604295906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d1f90 0x7f827c0cf7c0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b6fb0 tx=0x7f827402a730 comp rx=0 tx=0).stop 2024-09-10T10:50:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 -- 172.21.15.28:0/604295906 shutdown_connections 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 --2- 172.21.15.28:0/604295906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d1f90 0x7f827c0cf7c0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 --2- 172.21.15.28:0/604295906 >> v2:172.21.15.138:3300/0 conn(0x7f827c0d0dd0 0x7f827c0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 -- 172.21.15.28:0/604295906 >> 172.21.15.28:0/604295906 conn(0x7f827c08bfc0 msgr2=0x7f827c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 -- 172.21.15.28:0/604295906 shutdown_connections 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.751+0000 7f8281800640 1 -- 172.21.15.28:0/604295906 wait complete. 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.752+0000 7f8281800640 1 Processor -- start 2024-09-10T10:50:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 -- start start 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f827c0d1f90 0x7f827c0b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f827c15e660 0x7f827c0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f827c0d2c60 con 0x7f827c0d0dd0 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f827c0cf330 con 0x7f827c0d1f90 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.753+0000 7f8281800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f827c0d2f40 con 0x7f827c15e660 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37454/0 (socket says 172.21.15.28:37454) 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 -- 172.21.15.28:0/2096465674 learned_addr learned my addr 172.21.15.28:0/2096465674 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827b400640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f827c0d1f90 0x7f827c0b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f8280e00640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f827c15e660 0x7f827c0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 -- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f827c15e660 msgr2=0x7f827c0cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f827c15e660 0x7f827c0cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 -- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f827c0d1f90 msgr2=0x7f827c0b4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f827c0d1f90 0x7f827c0b4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.754+0000 7f827be00640 1 -- 172.21.15.28:0/2096465674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c152d10 con 0x7f827c0d0dd0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f827be00640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d800 tx=0x7f826c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00b6c0 con 0x7f827c0d0dd0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c019070 con 0x7f827c0d0dd0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c00ba80 con 0x7f827c0d0dd0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c15bf70 con 0x7f827c0d0dd0 2024-09-10T10:50:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.755+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c15c530 con 0x7f827c0d0dd0 2024-09-10T10:50:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.756+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c0d3220 con 0x7f827c0d0dd0 2024-09-10T10:50:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.757+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f826c00bc20 con 0x7f827c0d0dd0 2024-09-10T10:50:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.758+0000 7f8273e00640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.758+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7f826c08b3e0 con 0x7f827c0d0dd0 2024-09-10T10:50:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.758+0000 7f827b400640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.759+0000 7f827b400640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 0x7f825c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b2160 tx=0x7f8268009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:28.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.762+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f826c059510 con 0x7f827c0d0dd0 2024-09-10T10:50:28.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.949+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f827c0cadc0 con 0x7f827c0d0dd0 2024-09-10T10:50:28.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.950+0000 7f8273e00640 1 -- 172.21.15.28:0/2096465674 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+17030 (secure 0 0 0) 0x7f826c05d810 con 0x7f827c0d0dd0 2024-09-10T10:50:28.952 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:28.952 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":98,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:27.593162+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:50:25.399081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T10:50:26.587379+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 msgr2=0x7f825c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 0x7f825c06e2f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b2160 tx=0x7f8268009040 comp rx=0 tx=0).stop 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 msgr2=0x7f827c1527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f826c00d800 tx=0x7f826c00dcc0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 shutdown_connections 2024-09-10T10:50:28.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f825c06be30 0x7f825c06e2f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f827c15e660 0x7f827c0cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f827c0d1f90 0x7f827c0b4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 --2- 172.21.15.28:0/2096465674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f827c0d0dd0 0x7f827c1527d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 >> 172.21.15.28:0/2096465674 conn(0x7f827c08bfc0 msgr2=0x7f827c0b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 shutdown_connections 2024-09-10T10:50:28.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:28.953+0000 7f8281800640 1 -- 172.21.15.28:0/2096465674 wait complete. 2024-09-10T10:50:28.971 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:50:29.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.128+0000 7fe7c3e00640 1 Processor -- start 2024-09-10T10:50:29.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.128+0000 7fe7c3e00640 1 -- start start 2024-09-10T10:50:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.129+0000 7fe7c3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7c4161ac0 0x7fe7c406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.129+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 0x7fe7c40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.129+0000 7fe7c3e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7c40b1170 con 0x7fe7c40b3640 2024-09-10T10:50:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.129+0000 7fe7c3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7c40ceba0 con 0x7fe7c4161ac0 2024-09-10T10:50:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.129+0000 7fe7c3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c406dfb0 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 0x7fe7c40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7bbe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe7c40b3640 0x7fe7c40d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52658/0 (socket says 172.21.15.28:52658) 2024-09-10T10:50:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7c4161ac0 0x7fe7c406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7bbe00640 1 -- 172.21.15.28:0/2596017101 learned_addr learned my addr 172.21.15.28:0/2596017101 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c3400640 1 -- 172.21.15.28:0/2596017101 >> v2:172.21.15.138:3300/0 conn(0x7fe7c4161ac0 msgr2=0x7fe7c406e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c3400640 1 --2- 172.21.15.28:0/2596017101 >> v2:172.21.15.138:3300/0 conn(0x7fe7c4161ac0 0x7fe7c406e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c3400640 1 -- 172.21.15.28:0/2596017101 >> v1:172.21.15.138:6789/0 conn(0x7fe7c40b3640 legacy=0x7fe7c40d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c3400640 1 -- 172.21.15.28:0/2596017101 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c40b1450 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.130+0000 7fe7c2000640 1 -- 172.21.15.28:0/2596017101 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 625765522 0 0) 0x7fe7c40b1170 con 0x7fe7c40b3640 2024-09-10T10:50:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.131+0000 7fe7c3400640 1 --2- 172.21.15.28:0/2596017101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 0x7fe7c40b1f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70d3c9d8fdad692 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c2000640 1 -- 172.21.15.28:0/2596017101 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc007740 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c2000640 1 -- 172.21.15.28:0/2596017101 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0078e0 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c2000640 1 -- 172.21.15.28:0/2596017101 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc007cb0 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 -- 172.21.15.28:0/2596017101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 msgr2=0x7fe7c40b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/2596017101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 0x7fe7c40b1f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc02a6f0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 -- 172.21.15.28:0/2596017101 shutdown_connections 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/2596017101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c40d2ac0 0x7fe7c40b1f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/2596017101 >> v2:172.21.15.138:3300/0 conn(0x7fe7c4161ac0 0x7fe7c406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 -- 172.21.15.28:0/2596017101 >> 172.21.15.28:0/2596017101 conn(0x7fe7c408bfc0 msgr2=0x7fe7c40d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 -- 172.21.15.28:0/2596017101 shutdown_connections 2024-09-10T10:50:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.132+0000 7fe7c3e00640 1 -- 172.21.15.28:0/2596017101 wait complete. 2024-09-10T10:50:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.134+0000 7fe7c3e00640 1 Processor -- start 2024-09-10T10:50:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.134+0000 7fe7c3e00640 1 -- start start 2024-09-10T10:50:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c40d2ac0 0x7fe7c415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c4161ac0 0x7fe7c40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c4161ea0 con 0x7fe7c40b3640 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c40b1a10 con 0x7fe7c40d2ac0 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c4162020 con 0x7fe7c4161ac0 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37456/0 (socket says 172.21.15.28:37456) 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c2a00640 1 -- 172.21.15.28:0/3154369783 learned_addr learned my addr 172.21.15.28:0/3154369783 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7c3400640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c4161ac0 0x7fe7c40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.135+0000 7fe7bbe00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c40d2ac0 0x7fe7c415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 -- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c4161ac0 msgr2=0x7fe7c40b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c4161ac0 0x7fe7c40b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 -- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c40d2ac0 msgr2=0x7fe7c415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:29.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c40d2ac0 0x7fe7c415be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:29.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 -- 172.21.15.28:0/3154369783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c40cb540 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c2a00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e870 tx=0x7fe7b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b400c940 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400cd30 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c40cb820 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.136+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c40cbde0 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.138+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c40d5490 con 0x7fe7c40b3640 2024-09-10T10:50:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.138+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7b401a070 con 0x7fe7c40b3640 2024-09-10T10:50:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.139+0000 7fe7baa00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 0x7fe7a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.139+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6101+0+0 (secure 0 0 0) 0x7fe7b408b120 con 0x7fe7c40b3640 2024-09-10T10:50:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.140+0000 7fe7bbe00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 0x7fe7a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.141+0000 7fe7bbe00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 0x7fe7a006e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40cb140 tx=0x7fe7ac007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:29.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.144+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7c40d5490 con 0x7fe7c40b3640 2024-09-10T10:50:29.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:29.370+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe7c406e6c0 con 0x7fe7c40b3640 2024-09-10T10:50:30.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.185+0000 7fe7baa00640 1 -- 172.21.15.28:0/3154369783 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7fe7b4059380 con 0x7fe7c40b3640 2024-09-10T10:50:30.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 msgr2=0x7fe7a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 0x7fe7a006e2f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40cb140 tx=0x7fe7ac007d50 comp rx=0 tx=0).stop 2024-09-10T10:50:30.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 msgr2=0x7fe7c40a16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e870 tx=0x7fe7b400ed30 comp rx=0 tx=0).stop 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 shutdown_connections 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe7a006be30 0x7fe7a006e2f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7c4161ac0 0x7fe7c40b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7c40d2ac0 0x7fe7c415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 --2- 172.21.15.28:0/3154369783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7c40b3640 0x7fe7c40a16c0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 >> 172.21.15.28:0/3154369783 conn(0x7fe7c408bfc0 msgr2=0x7fe7c40b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 shutdown_connections 2024-09-10T10:50:30.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.189+0000 7fe7c3e00640 1 -- 172.21.15.28:0/3154369783 wait complete. 2024-09-10T10:50:30.209 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:50:30.209 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:50:30.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.373+0000 7f20c7e00640 1 Processor -- start 2024-09-10T10:50:30.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.373+0000 7f20c7e00640 1 -- start start 2024-09-10T10:50:30.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f20c80b3640 0x7f20c806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:30.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c80b1170 con 0x7f20c8161ac0 2024-09-10T10:50:30.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20c80b1450 con 0x7f20c80d2ac0 2024-09-10T10:50:30.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f20c80ceba0 con 0x7f20c80b3640 2024-09-10T10:50:30.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f20c8161ac0 0x7f20c80d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51096/0 (socket says 172.21.15.28:51096) 2024-09-10T10:50:30.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c6a00640 1 -- 172.21.15.28:0/1958720881 learned_addr learned my addr 172.21.15.28:0/1958720881 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:30.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.374+0000 7f20c7400640 1 --2- 172.21.15.28:0/1958720881 >> v2:172.21.15.28:3300/0 conn(0x7f20c80b3640 0x7f20c806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:30.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.375+0000 7f20c7400640 1 -- 172.21.15.28:0/1958720881 >> v1:172.21.15.138:6789/0 conn(0x7f20c80d2ac0 legacy=0x7f20c80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:30.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.375+0000 7f20c7400640 1 -- 172.21.15.28:0/1958720881 >> v1:172.21.15.28:6789/0 conn(0x7f20c8161ac0 legacy=0x7f20c80d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.375+0000 7f20c7400640 1 -- 172.21.15.28:0/1958720881 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c80b1730 con 0x7f20c80b3640 2024-09-10T10:50:30.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.375+0000 7f20c5600640 1 -- 172.21.15.28:0/1958720881 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698707048 0 0) 0x7f20c80b1170 con 0x7f20c8161ac0 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.376+0000 7f20c7400640 1 --2- 172.21.15.28:0/1958720881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c80b3640 0x7f20c806e6c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009d90 tx=0x7f20c002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2323151b1ea3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.376+0000 7f20c5600640 1 -- 172.21.15.28:0/1958720881 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0007740 con 0x7f20c80b3640 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.376+0000 7f20c5600640 1 -- 172.21.15.28:0/1958720881 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c00078e0 con 0x7f20c80b3640 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.376+0000 7f20c5600640 1 -- 172.21.15.28:0/1958720881 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0007cb0 con 0x7f20c80b3640 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 -- 172.21.15.28:0/1958720881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c80b3640 msgr2=0x7f20c806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 --2- 172.21.15.28:0/1958720881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c80b3640 0x7f20c806e6c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009d90 tx=0x7f20c002a730 comp rx=0 tx=0).stop 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 -- 172.21.15.28:0/1958720881 shutdown_connections 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 --2- 172.21.15.28:0/1958720881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c80b3640 0x7f20c806e6c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 -- 172.21.15.28:0/1958720881 >> 172.21.15.28:0/1958720881 conn(0x7f20c808bfc0 msgr2=0x7f20c80d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:30.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 -- 172.21.15.28:0/1958720881 shutdown_connections 2024-09-10T10:50:30.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.377+0000 7f20c7e00640 1 -- 172.21.15.28:0/1958720881 wait complete. 2024-09-10T10:50:30.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.378+0000 7f20c7e00640 1 Processor -- start 2024-09-10T10:50:30.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 -- start start 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20c80b3640 0x7f20c80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 0x7f20c81f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8161ea0 con 0x7f20c8161ac0 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20c806dfb0 con 0x7f20c80d2ac0 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.379+0000 7f20c7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f20c8162020 con 0x7f20c80b3640 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37472/0 (socket says 172.21.15.28:37472) 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 -- 172.21.15.28:0/653324407 learned_addr learned my addr 172.21.15.28:0/653324407 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c6a00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20c80b3640 0x7f20c80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c6000640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 0x7f20c81f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:30.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 -- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20c80b3640 msgr2=0x7f20c80b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20c80b3640 0x7f20c80b6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 -- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 msgr2=0x7f20c81f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 0x7f20c81f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c7400640 1 -- 172.21.15.28:0/653324407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c81f77a0 con 0x7f20c8161ac0 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.380+0000 7f20c6000640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 0x7f20c81f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20c7400640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009eb0 tx=0x7f20c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0040070 con 0x7f20c8161ac0 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c0031c90 con 0x7f20c8161ac0 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0045680 con 0x7f20c8161ac0 2024-09-10T10:50:30.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c80c7f60 con 0x7f20c8161ac0 2024-09-10T10:50:30.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.381+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c80b57a0 con 0x7f20c8161ac0 2024-09-10T10:50:30.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.382+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c806e6c0 con 0x7f20c8161ac0 2024-09-10T10:50:30.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.383+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f20c0039040 con 0x7f20c8161ac0 2024-09-10T10:50:30.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.384+0000 7f20bf400640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 0x7f20a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:30.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.384+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f20c00aca00 con 0x7f20c8161ac0 2024-09-10T10:50:30.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.384+0000 7f20c6a00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 0x7f20a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:30.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.385+0000 7f20c6a00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 0x7f20a806e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20b8007fd0 tx=0x7f20b8007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:30.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.388+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f20c007b240 con 0x7f20c8161ac0 2024-09-10T10:50:30.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:30.608+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f20c80d0b90 con 0x7f20c8161ac0 2024-09-10T10:50:31.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.191+0000 7f20bf400640 1 -- 172.21.15.28:0/653324407 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f20c007ab40 con 0x7f20c8161ac0 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 msgr2=0x7f20a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 0x7f20a806e2f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f20b8007fd0 tx=0x7f20b8007de0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 msgr2=0x7f20c814b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009eb0 tx=0x7f20c0004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 shutdown_connections 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f20a806be30 0x7f20a806e2f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f20c8161ac0 0x7f20c814b910 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20c80d2ac0 0x7f20c81f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 --2- 172.21.15.28:0/653324407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f20c80b3640 0x7f20c80b6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 >> 172.21.15.28:0/653324407 conn(0x7f20c808bfc0 msgr2=0x7f20c80b7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:31.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 shutdown_connections 2024-09-10T10:50:31.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.194+0000 7f20c7e00640 1 -- 172.21.15.28:0/653324407 wait complete. 2024-09-10T10:50:31.214 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:50:31.214 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.374+0000 7f72f4a00640 1 Processor -- start 2024-09-10T10:50:31.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.374+0000 7f72f4a00640 1 -- start start 2024-09-10T10:50:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72f4a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f72f00b47b0 0x7f72f006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72f4a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f006ddd0 con 0x7f72f009aa30 2024-09-10T10:50:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72f4a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72f006e0b0 con 0x7f72f01629e0 2024-09-10T10:50:31.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72f4a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f72f00d1b60 con 0x7f72f00b47b0 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72e7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f72f00b47b0 0x7f72f006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72efe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f72f009aa30 0x7f72f00d0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51108/0 (socket says 172.21.15.28:51108) 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72e7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f72f00b47b0 0x7f72f006e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37486/0 (socket says 172.21.15.28:37486) 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.375+0000 7f72efe00640 1 -- 172.21.15.28:0/3784923474 learned_addr learned my addr 172.21.15.28:0/3784923474 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.376+0000 7f72e7e00640 1 -- 172.21.15.28:0/3784923474 >> v1:172.21.15.138:6789/0 conn(0x7f72f01629e0 legacy=0x7f72f0070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.376+0000 7f72e7e00640 1 -- 172.21.15.28:0/3784923474 >> v1:172.21.15.28:6789/0 conn(0x7f72f009aa30 legacy=0x7f72f00d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.376+0000 7f72eea00640 1 -- 172.21.15.28:0/3784923474 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037294813 0 0) 0x7f72f006e0b0 con 0x7f72f01629e0 2024-09-10T10:50:31.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.376+0000 7f72e7e00640 1 -- 172.21.15.28:0/3784923474 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f006e6a0 con 0x7f72f00b47b0 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.377+0000 7f72e7e00640 1 --2- 172.21.15.28:0/3784923474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f006e9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097a0 tx=0x7f72d802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e587fe0af0f084 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.377+0000 7f72eea00640 1 -- 172.21.15.28:0/3784923474 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8037070 con 0x7f72f00b47b0 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.377+0000 7f72eea00640 1 -- 172.21.15.28:0/3784923474 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d802ad30 con 0x7f72f00b47b0 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.377+0000 7f72eea00640 1 -- 172.21.15.28:0/3784923474 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8032660 con 0x7f72f00b47b0 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 -- 172.21.15.28:0/3784923474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 msgr2=0x7f72f006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 --2- 172.21.15.28:0/3784923474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f006e9d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f72d80097a0 tx=0x7f72d802a0a0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 -- 172.21.15.28:0/3784923474 shutdown_connections 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 --2- 172.21.15.28:0/3784923474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f006e9d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 -- 172.21.15.28:0/3784923474 >> 172.21.15.28:0/3784923474 conn(0x7f72f00b3c90 msgr2=0x7f72f00b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 -- 172.21.15.28:0/3784923474 shutdown_connections 2024-09-10T10:50:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.378+0000 7f72f4a00640 1 -- 172.21.15.28:0/3784923474 wait complete. 2024-09-10T10:50:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.379+0000 7f72f4a00640 1 Processor -- start 2024-09-10T10:50:31.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.379+0000 7f72f4a00640 1 -- start start 2024-09-10T10:50:31.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 0x7f72f01f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 0x7f72f015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f72f00d0a00 con 0x7f72f00b47b0 2024-09-10T10:50:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72f00d0880 con 0x7f72f01629e0 2024-09-10T10:50:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.380+0000 7f72f4a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f72f00d0b80 con 0x7f72f009aa30 2024-09-10T10:50:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37488/0 (socket says 172.21.15.28:37488) 2024-09-10T10:50:31.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 -- 172.21.15.28:0/4281901635 learned_addr learned my addr 172.21.15.28:0/4281901635 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72ef400640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 0x7f72f01f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72efe00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 0x7f72f015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 -- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 msgr2=0x7f72f01f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 0x7f72f01f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 -- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 msgr2=0x7f72f015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 0x7f72f015f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 -- 172.21.15.28:0/4281901635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f00c70d0 con 0x7f72f00b47b0 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72ef400640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 0x7f72f01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:31.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72efe00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 0x7f72f015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.381+0000 7f72e7e00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72d802a0e0 tx=0x7f72d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.382+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8037040 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.382+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72d8032e10 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.382+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004270 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.382+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f00c6df0 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.382+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01f7930 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.383+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f009fe60 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.384+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f72d803a070 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.384+0000 7f72e7400640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 0x7f72cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.385+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f72d80ab550 con 0x7f72f00b47b0 2024-09-10T10:50:31.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.385+0000 7f72ef400640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 0x7f72cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.386+0000 7f72ef400640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 0x7f72cc06e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f60 tx=0x7f72e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.389+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f72f009fe60 con 0x7f72f00b47b0 2024-09-10T10:50:31.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.611+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72f006e0b0 con 0x7f72f00b47b0 2024-09-10T10:50:31.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.611+0000 7f72e7400640 1 -- 172.21.15.28:0/4281901635 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5225 (secure 0 0 0) 0x7f72d8041020 con 0x7f72f00b47b0 2024-09-10T10:50:31.612 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:31.612 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":83,"btime":"2024-09-10T10:50:31:186469+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:30.609334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6849":{"gid":6849,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/3446889436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3446889436},{"type":"v1","addr":"172.21.15.28:6835","nonce":3446889436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6849,"qdb_cluster":[6849]},"id":6}]} 2024-09-10T10:50:31.613 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 83 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 msgr2=0x7f72cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 0x7f72cc06e2f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f72e0000f60 tx=0x7f72e0009040 comp rx=0 tx=0).stop 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 msgr2=0x7f72f009c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f72d802a0e0 tx=0x7f72d80046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 shutdown_connections 2024-09-10T10:50:31.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f72cc06be30 0x7f72cc06e2f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72f01629e0 0x7f72f015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72f00b47b0 0x7f72f009c3b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 --2- 172.21.15.28:0/4281901635 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f72f009aa30 0x7f72f01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 >> 172.21.15.28:0/4281901635 conn(0x7f72f00b3c90 msgr2=0x7f72f00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 shutdown_connections 2024-09-10T10:50:31.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.614+0000 7f72f4a00640 1 -- 172.21.15.28:0/4281901635 wait complete. 2024-09-10T10:50:31.630 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:50:31.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.786+0000 7f5d4fe00640 1 Processor -- start 2024-09-10T10:50:31.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.786+0000 7f5d4fe00640 1 -- start start 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d500d0dd0 0x7f5d500d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 0x7f5d5006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d500d1f90 0x7f5d500cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5d500b37d0 con 0x7f5d500d1f90 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5d500cf330 con 0x7f5d500d0dd0 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d500cf000 con 0x7f5d5015e660 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d500d1f90 0x7f5d500cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d500d1f90 0x7f5d500cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37500/0 (socket says 172.21.15.28:37500) 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.787+0000 7f5d4f400640 1 -- 172.21.15.28:0/1706933254 learned_addr learned my addr 172.21.15.28:0/1706933254 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:31.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4e000640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 0x7f5d5006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4ea00640 1 --2- 172.21.15.28:0/1706933254 >> v2:172.21.15.138:3300/0 conn(0x7f5d500d0dd0 0x7f5d500d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4f400640 1 -- 172.21.15.28:0/1706933254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 msgr2=0x7f5d5006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4f400640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 0x7f5d5006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4f400640 1 -- 172.21.15.28:0/1706933254 >> v2:172.21.15.138:3300/0 conn(0x7f5d500d0dd0 msgr2=0x7f5d500d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4f400640 1 --2- 172.21.15.28:0/1706933254 >> v2:172.21.15.138:3300/0 conn(0x7f5d500d0dd0 0x7f5d500d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4f400640 1 -- 172.21.15.28:0/1706933254 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d500d16f0 con 0x7f5d500d1f90 2024-09-10T10:50:31.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.788+0000 7f5d4e000640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 0x7f5d5006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:31.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.789+0000 7f5d4f400640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d500d1f90 0x7f5d500cf7c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b6fb0 tx=0x7f5d4802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c432316465cbcca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.789+0000 7f5d4d600640 1 -- 172.21.15.28:0/1706933254 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48036070 con 0x7f5d500d1f90 2024-09-10T10:50:31.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.789+0000 7f5d4d600640 1 -- 172.21.15.28:0/1706933254 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d48031600 con 0x7f5d500d1f90 2024-09-10T10:50:31.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.789+0000 7f5d4d600640 1 -- 172.21.15.28:0/1706933254 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d480319d0 con 0x7f5d500d1f90 2024-09-10T10:50:31.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 -- 172.21.15.28:0/1706933254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d500d1f90 msgr2=0x7f5d500cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d500d1f90 0x7f5d500cf7c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b6fb0 tx=0x7f5d4802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 -- 172.21.15.28:0/1706933254 shutdown_connections 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d500d1f90 0x7f5d500cf7c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/1706933254 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d5015e660 0x7f5d5006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/1706933254 >> v2:172.21.15.138:3300/0 conn(0x7f5d500d0dd0 0x7f5d500d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 -- 172.21.15.28:0/1706933254 >> 172.21.15.28:0/1706933254 conn(0x7f5d5008bfc0 msgr2=0x7f5d500d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 -- 172.21.15.28:0/1706933254 shutdown_connections 2024-09-10T10:50:31.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.790+0000 7f5d4fe00640 1 -- 172.21.15.28:0/1706933254 wait complete. 2024-09-10T10:50:31.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.791+0000 7f5d4fe00640 1 Processor -- start 2024-09-10T10:50:31.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 -- start start 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d500d0dd0 0x7f5d50155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d500d1f90 0x7f5d500ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5d500d2f40 con 0x7f5d5015e660 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d500d2c60 con 0x7f5d500d1f90 2024-09-10T10:50:31.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.792+0000 7f5d4fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d500d3220 con 0x7f5d500d0dd0 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37510/0 (socket says 172.21.15.28:37510) 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 -- 172.21.15.28:0/3909396404 learned_addr learned my addr 172.21.15.28:0/3909396404 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4ea00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d500d0dd0 0x7f5d50155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4e000640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d500d1f90 0x7f5d500ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 -- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d500d0dd0 msgr2=0x7f5d50155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d500d0dd0 0x7f5d50155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 -- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d500d1f90 msgr2=0x7f5d500ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d500d1f90 0x7f5d500ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 -- 172.21.15.28:0/3909396404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d500cbe70 con 0x7f5d5015e660 2024-09-10T10:50:31.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.793+0000 7f5d4f400640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034040 tx=0x7f5d48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.794+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48041020 con 0x7f5d5015e660 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.794+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d480077a0 con 0x7f5d5015e660 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.794+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48036040 con 0x7f5d5015e660 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.794+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d500cc150 con 0x7f5d5015e660 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.794+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d500cc710 con 0x7f5d5015e660 2024-09-10T10:50:31.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.795+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d500d2440 con 0x7f5d5015e660 2024-09-10T10:50:31.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.796+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d48044020 con 0x7f5d5015e660 2024-09-10T10:50:31.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.797+0000 7f5d46a00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 0x7f5d3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.797+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f5d480abe70 con 0x7f5d5015e660 2024-09-10T10:50:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.797+0000 7f5d4ea00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 0x7f5d3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.798+0000 7f5d4ea00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 0x7f5d3006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b2990 tx=0x7f5d40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.801+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5d4804b030 con 0x7f5d5015e660 2024-09-10T10:50:31.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.987+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d500d1b20 con 0x7f5d5015e660 2024-09-10T10:50:31.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.988+0000 7f5d46a00640 1 -- 172.21.15.28:0/3909396404 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+17017 (secure 0 0 0) 0x7f5d4807a5f0 con 0x7f5d5015e660 2024-09-10T10:50:31.989 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:31.989 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":99,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:50:29.173268+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:50:25.399081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T10:50:26.587379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:50:31.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 msgr2=0x7f5d3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 0x7f5d3006e2f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b2990 tx=0x7f5d40008040 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 msgr2=0x7f5d500cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f5d48034040 tx=0x7f5d48004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 shutdown_connections 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d3006be30 0x7f5d3006e2f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d5015e660 0x7f5d500cb490 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d500d1f90 0x7f5d500ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 --2- 172.21.15.28:0/3909396404 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d500d0dd0 0x7f5d50155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 >> 172.21.15.28:0/3909396404 conn(0x7f5d5008bfc0 msgr2=0x7f5d50157710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 shutdown_connections 2024-09-10T10:50:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:31.990+0000 7f5d4fe00640 1 -- 172.21.15.28:0/3909396404 wait complete. 2024-09-10T10:50:32.008 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:50:32.008 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:50:32.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.163+0000 7f1fbdc00640 1 Processor -- start 2024-09-10T10:50:32.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.163+0000 7f1fbdc00640 1 -- start start 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 0x7f1fb80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fbdc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb8146140 con 0x7f1fb8072de0 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fbdc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb806eed0 con 0x7f1fb815e1f0 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb80cd740 con 0x7f1fb8072320 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb6000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1fb8072de0 0x7f1fb806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51116/0 (socket says 172.21.15.28:51116) 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb6000640 1 -- 172.21.15.28:0/4076970929 learned_addr learned my addr 172.21.15.28:0/4076970929 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb6a00640 1 --2- 172.21.15.28:0/4076970929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 0x7f1fb80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2327420934 0 0) 0x7f1fb8146140 con 0x7f1fb8072de0 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f9c003520 con 0x7f1fb8072de0 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3059581435 0 0) 0x7f1fb806eed0 con 0x7f1fb815e1f0 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.165+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fb8146140 con 0x7f1fb815e1f0 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.166+0000 7f1fb6a00640 1 -- 172.21.15.28:0/4076970929 >> v1:172.21.15.138:6789/0 conn(0x7f1fb815e1f0 legacy=0x7f1fb806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.166+0000 7f1fb6a00640 1 -- 172.21.15.28:0/4076970929 >> v1:172.21.15.28:6789/0 conn(0x7f1fb8072de0 legacy=0x7f1fb806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.166+0000 7f1fb6a00640 1 -- 172.21.15.28:0/4076970929 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb80cda20 con 0x7f1fb8072320 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fb6a00640 1 --2- 172.21.15.28:0/4076970929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 0x7f1fb80cce60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097d0 tx=0x7f1fa802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2a6853d7a6896b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8037070 con 0x7f1fb8072320 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa8032440 con 0x7f1fb8072320 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fb5600640 1 -- 172.21.15.28:0/4076970929 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8032810 con 0x7f1fb8072320 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fbdc00640 1 -- 172.21.15.28:0/4076970929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 msgr2=0x7f1fb80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.167+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/4076970929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 0x7f1fb80cce60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097d0 tx=0x7f1fa802a180 comp rx=0 tx=0).stop 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.168+0000 7f1fbdc00640 1 -- 172.21.15.28:0/4076970929 shutdown_connections 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.168+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/4076970929 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072320 0x7f1fb80cce60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.168+0000 7f1fbdc00640 1 -- 172.21.15.28:0/4076970929 >> 172.21.15.28:0/4076970929 conn(0x7f1fb8067d70 msgr2=0x7f1fb806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.168+0000 7f1fbdc00640 1 -- 172.21.15.28:0/4076970929 shutdown_connections 2024-09-10T10:50:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.168+0000 7f1fbdc00640 1 -- 172.21.15.28:0/4076970929 wait complete. 2024-09-10T10:50:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.169+0000 7f1fbdc00640 1 Processor -- start 2024-09-10T10:50:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.169+0000 7f1fbdc00640 1 -- start start 2024-09-10T10:50:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072de0 0x7f1fb81e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 0x7f1fb80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb80cd3a0 con 0x7f1fb8072320 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb815c9e0 con 0x7f1fb815e1f0 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.170+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb80cd520 con 0x7f1fb8072de0 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37512/0 (socket says 172.21.15.28:37512) 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 -- 172.21.15.28:0/2308910844 learned_addr learned my addr 172.21.15.28:0/2308910844 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6000640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072de0 0x7f1fb81e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 -- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072de0 msgr2=0x7f1fb81e33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072de0 0x7f1fb81e33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb7400640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 0x7f1fb80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 -- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 msgr2=0x7f1fb80cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 0x7f1fb80cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 -- 172.21.15.28:0/2308910844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb81e3d30 con 0x7f1fb8072320 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb7400640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 0x7f1fb80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.171+0000 7f1fb6a00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097d0 tx=0x7f1fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.172+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8041020 con 0x7f1fb8072320 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.172+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa803d6e0 con 0x7f1fb8072320 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.172+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb81e4010 con 0x7f1fb8072320 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.172+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8037040 con 0x7f1fb8072320 2024-09-10T10:50:32.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.172+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb81e45d0 con 0x7f1fb8072320 2024-09-10T10:50:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.173+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb8072700 con 0x7f1fb8072320 2024-09-10T10:50:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.174+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fa8044020 con 0x7f1fb8072320 2024-09-10T10:50:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.175+0000 7f1faea00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.175+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f1fa80ab750 con 0x7f1fb8072320 2024-09-10T10:50:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.175+0000 7f1fb6000640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.176+0000 7f1fb6000640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 0x7f1f9806e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8143f10 tx=0x7f1fa4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.180+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1fa804b030 con 0x7f1fb8072320 2024-09-10T10:50:32.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.365+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fb80b3f60 con 0x7f1f9806be30 2024-09-10T10:50:32.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.371+0000 7f1faea00640 1 -- 172.21.15.28:0/2308910844 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+238786 (secure 0 0 0) 0x7f1fb80b3f60 con 0x7f1f9806be30 2024-09-10T10:50:32.375 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:32.376 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 msgr2=0x7f1f9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 0x7f1f9806e2f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8143f10 tx=0x7f1fa4009040 comp rx=0 tx=0).stop 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 msgr2=0x7f1fb815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80097d0 tx=0x7f1fa80046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 shutdown_connections 2024-09-10T10:50:32.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fb815e1f0 0x7f1fb80cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fb8072de0 0x7f1fb81e33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 --2- 172.21.15.28:0/2308910844 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fb8072320 0x7f1fb815be50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 >> 172.21.15.28:0/2308910844 conn(0x7f1fb8067d70 msgr2=0x7f1fb806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 shutdown_connections 2024-09-10T10:50:32.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.375+0000 7f1fbdc00640 1 -- 172.21.15.28:0/2308910844 wait complete. 2024-09-10T10:50:32.392 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-10T10:50:30.586783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":20,"acting":20,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259836,"kb_used_data":10284,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714884,"statfs":{"total":773094113280,"available":772828041216,"internally_reserved":0,"allocated":10530816,"data_stored":5490819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.436277"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202026+0000","last_change":"2024-09-10T10:50:28.184346+0000","last_active":"2024-09-10T10:50:28.202026+0000","last_peered":"2024-09-10T10:50:28.202026+0000","last_clean":"2024-09-10T10:50:28.202026+0000","last_became_active":"2024-09-10T10:50:28.184068+0000","last_became_peered":"2024-09-10T10:50:28.184068+0000","last_unstale":"2024-09-10T10:50:28.202026+0000","last_undegraded":"2024-09-10T10:50:28.202026+0000","last_fullsized":"2024-09-10T10:50:28.202026+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:45.188246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.204948+0000","last_change":"2024-09-10T10:50:28.193766+0000","last_active":"2024-09-10T10:50:28.204948+0000","last_peered":"2024-09-10T10:50:28.204948+0000","last_clean":"2024-09-10T10:50:28.204948+0000","last_became_active":"2024-09-10T10:50:28.193289+0000","last_became_peered":"2024-09-10T10:50:28.193289+0000","last_unstale":"2024-09-10T10:50:28.204948+0000","last_undegraded":"2024-09-10T10:50:28.204948+0000","last_fullsized":"2024-09-10T10:50:28.204948+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:58.731770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202908+0000","last_change":"2024-09-10T10:50:28.188444+0000","last_active":"2024-09-10T10:50:28.202908+0000","last_peered":"2024-09-10T10:50:28.202908+0000","last_clean":"2024-09-10T10:50:28.202908+0000","last_became_active":"2024-09-10T10:50:28.188169+0000","last_became_peered":"2024-09-10T10:50:28.188169+0000","last_unstale":"2024-09-10T10:50:28.202908+0000","last_undegraded":"2024-09-10T10:50:28.202908+0000","last_fullsized":"2024-09-10T10:50:28.202908+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:24.216712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.203930+0000","last_change":"2024-09-10T10:50:27.191277+0000","last_active":"2024-09-10T10:50:28.203930+0000","last_peered":"2024-09-10T10:50:28.203930+0000","last_clean":"2024-09-10T10:50:28.203930+0000","last_became_active":"2024-09-10T10:50:27.190992+0000","last_became_peered":"2024-09-10T10:50:27.190992+0000","last_unstale":"2024-09-10T10:50:28.203930+0000","last_undegraded":"2024-09-10T10:50:28.203930+0000","last_fullsized":"2024-09-10T10:50:28.203930+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:00.947632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.206563+0000","last_change":"2024-09-10T10:50:28.201838+0000","last_active":"2024-09-10T10:50:28.206563+0000","last_peered":"2024-09-10T10:50:28.206563+0000","last_clean":"2024-09-10T10:50:28.206563+0000","last_became_active":"2024-09-10T10:50:28.197464+0000","last_became_peered":"2024-09-10T10:50:28.197464+0000","last_unstale":"2024-09-10T10:50:28.206563+0000","last_undegraded":"2024-09-10T10:50:28.206563+0000","last_fullsized":"2024-09-10T10:50:28.206563+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:39.909923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.204973+0000","last_change":"2024-09-10T10:50:28.194356+0000","last_active":"2024-09-10T10:50:28.204973+0000","last_peered":"2024-09-10T10:50:28.204973+0000","last_clean":"2024-09-10T10:50:28.204973+0000","last_became_active":"2024-09-10T10:50:28.194087+0000","last_became_peered":"2024-09-10T10:50:28.194087+0000","last_unstale":"2024-09-10T10:50:28.204973+0000","last_undegraded":"2024-09-10T10:50:28.204973+0000","last_fullsized":"2024-09-10T10:50:28.204973+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:42.144671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.175178+0000","last_change":"2024-09-10T10:50:27.183793+0000","last_active":"2024-09-10T10:50:28.175178+0000","last_peered":"2024-09-10T10:50:28.175178+0000","last_clean":"2024-09-10T10:50:28.175178+0000","last_became_active":"2024-09-10T10:50:27.183501+0000","last_became_peered":"2024-09-10T10:50:27.183501+0000","last_unstale":"2024-09-10T10:50:28.175178+0000","last_undegraded":"2024-09-10T10:50:28.175178+0000","last_fullsized":"2024-09-10T10:50:28.175178+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:33.239560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202898+0000","last_change":"2024-09-10T10:50:28.190397+0000","last_active":"2024-09-10T10:50:28.202898+0000","last_peered":"2024-09-10T10:50:28.202898+0000","last_clean":"2024-09-10T10:50:28.202898+0000","last_became_active":"2024-09-10T10:50:28.189002+0000","last_became_peered":"2024-09-10T10:50:28.189002+0000","last_unstale":"2024-09-10T10:50:28.202898+0000","last_undegraded":"2024-09-10T10:50:28.202898+0000","last_fullsized":"2024-09-10T10:50:28.202898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:54.061184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.184845+0000","last_change":"2024-09-10T10:50:28.179368+0000","last_active":"2024-09-10T10:50:28.184845+0000","last_peered":"2024-09-10T10:50:28.184845+0000","last_clean":"2024-09-10T10:50:28.184845+0000","last_became_active":"2024-09-10T10:50:28.179083+0000","last_became_peered":"2024-09-10T10:50:28.179083+0000","last_unstale":"2024-09-10T10:50:28.184845+0000","last_undegraded":"2024-09-10T10:50:28.184845+0000","last_fullsized":"2024-09-10T10:50:28.184845+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:13.712524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:27.150272+0000","last_change":"2024-09-10T10:50:27.150272+0000","last_active":"2024-09-10T10:50:27.150272+0000","last_peered":"2024-09-10T10:50:27.150272+0000","last_clean":"2024-09-10T10:50:27.150272+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:27.150272+0000","last_undegraded":"2024-09-10T10:50:27.150272+0000","last_fullsized":"2024-09-10T10:50:27.150272+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T10:50:26.146982+0000","last_change":"2024-09-10T10:50:26.146982+0000","last_active":"2024-09-10T10:50:26.146982+0000","last_peered":"2024-09-10T10:50:26.146982+0000","last_clean":"2024-09-10T10:50:26.146982+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:50:26.146982+0000","last_undegraded":"2024-09-10T10:50:26.146982+0000","last_fullsized":"2024-09-10T10:50:26.146982+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":307,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-10T10:50:24.147766+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:50:24.147766+0000","last_peered":"2024-09-10T10:50:24.147766+0000","last_clean":"2024-09-10T10:50:24.147766+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:50:24.147766+0000","last_undegraded":"2024-09-10T10:50:24.147766+0000","last_fullsized":"2024-09-10T10:50:24.147766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":4,"acting":4,"num_store_stats":1},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32324,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32332,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32332,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32328,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509524,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1757184,"data_stored":1126623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32332,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1163264,"data_stored":536255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1794048,"data_stored":1146666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:50:32.396 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:50:32.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.588+0000 7f088ee00640 1 Processor -- start 2024-09-10T10:50:32.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.588+0000 7f088ee00640 1 -- start start 2024-09-10T10:50:32.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.589+0000 7f088ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 0x7f08880d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.589+0000 7f088ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f088806bc10 0x7f088806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.589+0000 7f088ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08880d0980 con 0x7f088815e5d0 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.589+0000 7f088ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f088815e9b0 con 0x7f088806bc10 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.589+0000 7f088ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f08880d0c60 con 0x7f08880d1b70 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f088806bc10 0x7f088806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f088806bc10 0x7f088806c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37526/0 (socket says 172.21.15.28:37526) 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 -- 172.21.15.28:0/1520317424 learned_addr learned my addr 172.21.15.28:0/1520317424 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:32.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 -- 172.21.15.28:0/1520317424 >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 msgr2=0x7f08880d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088d000640 1 --2- 172.21.15.28:0/1520317424 >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 0x7f08880d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 --2- 172.21.15.28:0/1520317424 >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 0x7f08880d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 -- 172.21.15.28:0/1520317424 >> v1:172.21.15.138:6789/0 conn(0x7f088815e5d0 legacy=0x7f0888073780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088e400640 1 -- 172.21.15.28:0/1520317424 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0888072f80 con 0x7f088806bc10 2024-09-10T10:50:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.590+0000 7f088d000640 1 --2- 172.21.15.28:0/1520317424 >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 0x7f08880d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.591+0000 7f088e400640 1 --2- 172.21.15.28:0/1520317424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f088806bc10 0x7f088806c0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0880009e50 tx=0x7f088002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58435cc07bf0b247 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.591+0000 7f0887e00640 1 -- 172.21.15.28:0/1520317424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880036070 con 0x7f088806bc10 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.591+0000 7f0887e00640 1 -- 172.21.15.28:0/1520317424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0880031600 con 0x7f088806bc10 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.591+0000 7f0887e00640 1 -- 172.21.15.28:0/1520317424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088002ae60 con 0x7f088806bc10 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 -- 172.21.15.28:0/1520317424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f088806bc10 msgr2=0x7f088806c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 --2- 172.21.15.28:0/1520317424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f088806bc10 0x7f088806c0a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0880009e50 tx=0x7f088002a680 comp rx=0 tx=0).stop 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 -- 172.21.15.28:0/1520317424 shutdown_connections 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 --2- 172.21.15.28:0/1520317424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f088806bc10 0x7f088806c0a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 --2- 172.21.15.28:0/1520317424 >> v2:172.21.15.138:3300/0 conn(0x7f08880d1b70 0x7f08880d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 -- 172.21.15.28:0/1520317424 >> 172.21.15.28:0/1520317424 conn(0x7f08880d1080 msgr2=0x7f08880d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 -- 172.21.15.28:0/1520317424 shutdown_connections 2024-09-10T10:50:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.592+0000 7f088ee00640 1 -- 172.21.15.28:0/1520317424 wait complete. 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.594+0000 7f088ee00640 1 Processor -- start 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.594+0000 7f088ee00640 1 -- start start 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 0x7f08880b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f088815e5d0 0x7f08880cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f088806cb60 con 0x7f08880d1b70 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f088806c9e0 con 0x7f088806bc10 2024-09-10T10:50:32.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0888162030 con 0x7f088815e5d0 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37530/0 (socket says 172.21.15.28:37530) 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 -- 172.21.15.28:0/3722506097 learned_addr learned my addr 172.21.15.28:0/3722506097 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088e400640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f088815e5d0 0x7f08880cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088da00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 0x7f08880b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 -- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f088815e5d0 msgr2=0x7f08880cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f088815e5d0 0x7f08880cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 -- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 msgr2=0x7f08880b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 0x7f08880b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088d000640 1 -- 172.21.15.28:0/3722506097 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08880c8c00 con 0x7f08880d1b70 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.595+0000 7f088da00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 0x7f08880b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f088d000640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f087400c7f0 tx=0x7f087400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874015070 con 0x7f08880d1b70 2024-09-10T10:50:32.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08740079d0 con 0x7f08880d1b70 2024-09-10T10:50:32.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08880c8ee0 con 0x7f08880d1b70 2024-09-10T10:50:32.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874007da0 con 0x7f08880d1b70 2024-09-10T10:50:32.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.596+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08880cc5b0 con 0x7f08880d1b70 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.597+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088815dcd0 con 0x7f08880d1b70 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.598+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0874018070 con 0x7f08880d1b70 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.599+0000 7f0885600640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 0x7f086806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.599+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f087408aed0 con 0x7f08880d1b70 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.600+0000 7f088da00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 0x7f086806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:32.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.600+0000 7f088da00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 0x7f086806e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:32.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.603+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f088815dcd0 con 0x7f08880d1b70 2024-09-10T10:50:32.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.786+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08880d0310 con 0x7f086806be30 2024-09-10T10:50:32.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.793+0000 7f0885600640 1 -- 172.21.15.28:0/3722506097 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252069 (secure 0 0 0) 0x7f08880d0310 con 0x7f086806be30 2024-09-10T10:50:32.797 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:32.798 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 msgr2=0x7f086806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 0x7f086806e2f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f087c006fd0 tx=0x7f087c008040 comp rx=0 tx=0).stop 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 msgr2=0x7f08880ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f087400c7f0 tx=0x7f087400cbf0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 shutdown_connections 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f086806be30 0x7f086806e2f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f088815e5d0 0x7f08880cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08880d1b70 0x7f08880ca840 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 --2- 172.21.15.28:0/3722506097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f088806bc10 0x7f08880b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.796+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 >> 172.21.15.28:0/3722506097 conn(0x7f08880d1080 msgr2=0x7f08880af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.797+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 shutdown_connections 2024-09-10T10:50:32.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:32.797+0000 7f088ee00640 1 -- 172.21.15.28:0/3722506097 wait complete. 2024-09-10T10:50:32.814 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-10T10:50:32.587232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260216,"kb_used_data":10684,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714504,"statfs":{"total":773094113280,"available":772827652096,"internally_reserved":0,"allocated":10940416,"data_stored":5682784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001346"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223640+0000","last_change":"2024-09-10T10:50:28.184919+0000","last_active":"2024-09-10T10:50:29.223640+0000","last_peered":"2024-09-10T10:50:29.223640+0000","last_clean":"2024-09-10T10:50:29.223640+0000","last_became_active":"2024-09-10T10:50:28.184096+0000","last_became_peered":"2024-09-10T10:50:28.184096+0000","last_unstale":"2024-09-10T10:50:29.223640+0000","last_undegraded":"2024-09-10T10:50:29.223640+0000","last_fullsized":"2024-09-10T10:50:29.223640+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:03.781090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223496+0000","last_change":"2024-09-10T10:50:28.201041+0000","last_active":"2024-09-10T10:50:29.223496+0000","last_peered":"2024-09-10T10:50:29.223496+0000","last_clean":"2024-09-10T10:50:29.223496+0000","last_became_active":"2024-09-10T10:50:28.200728+0000","last_became_peered":"2024-09-10T10:50:28.200728+0000","last_unstale":"2024-09-10T10:50:29.223496+0000","last_undegraded":"2024-09-10T10:50:29.223496+0000","last_fullsized":"2024-09-10T10:50:29.223496+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:56.246919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223835+0000","last_change":"2024-09-10T10:50:28.198574+0000","last_active":"2024-09-10T10:50:29.223835+0000","last_peered":"2024-09-10T10:50:29.223835+0000","last_clean":"2024-09-10T10:50:29.223835+0000","last_became_active":"2024-09-10T10:50:28.198299+0000","last_became_peered":"2024-09-10T10:50:28.198299+0000","last_unstale":"2024-09-10T10:50:29.223835+0000","last_undegraded":"2024-09-10T10:50:29.223835+0000","last_fullsized":"2024-09-10T10:50:29.223835+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:48.357764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223394+0000","last_change":"2024-09-10T10:50:28.199527+0000","last_active":"2024-09-10T10:50:29.223394+0000","last_peered":"2024-09-10T10:50:29.223394+0000","last_clean":"2024-09-10T10:50:29.223394+0000","last_became_active":"2024-09-10T10:50:28.199212+0000","last_became_peered":"2024-09-10T10:50:28.199212+0000","last_unstale":"2024-09-10T10:50:29.223394+0000","last_undegraded":"2024-09-10T10:50:29.223394+0000","last_fullsized":"2024-09-10T10:50:29.223394+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:37.333722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223189+0000","last_change":"2024-09-10T10:50:28.179800+0000","last_active":"2024-09-10T10:50:29.223189+0000","last_peered":"2024-09-10T10:50:29.223189+0000","last_clean":"2024-09-10T10:50:29.223189+0000","last_became_active":"2024-09-10T10:50:28.178835+0000","last_became_peered":"2024-09-10T10:50:28.178835+0000","last_unstale":"2024-09-10T10:50:29.223189+0000","last_undegraded":"2024-09-10T10:50:29.223189+0000","last_fullsized":"2024-09-10T10:50:29.223189+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:32.417029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.183629+0000","last_change":"2024-09-10T10:50:28.180851+0000","last_active":"2024-09-10T10:50:29.183629+0000","last_peered":"2024-09-10T10:50:29.183629+0000","last_clean":"2024-09-10T10:50:29.183629+0000","last_became_active":"2024-09-10T10:50:28.180479+0000","last_became_peered":"2024-09-10T10:50:28.180479+0000","last_unstale":"2024-09-10T10:50:29.183629+0000","last_undegraded":"2024-09-10T10:50:29.183629+0000","last_fullsized":"2024-09-10T10:50:29.183629+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:21.392883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180575+0000","last_change":"2024-09-10T10:50:28.201802+0000","last_active":"2024-09-10T10:50:29.180575+0000","last_peered":"2024-09-10T10:50:29.180575+0000","last_clean":"2024-09-10T10:50:29.180575+0000","last_became_active":"2024-09-10T10:50:28.201478+0000","last_became_peered":"2024-09-10T10:50:28.201478+0000","last_unstale":"2024-09-10T10:50:29.180575+0000","last_undegraded":"2024-09-10T10:50:29.180575+0000","last_fullsized":"2024-09-10T10:50:29.180575+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:55.203478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182231+0000","last_change":"2024-09-10T10:50:28.185815+0000","last_active":"2024-09-10T10:50:29.182231+0000","last_peered":"2024-09-10T10:50:29.182231+0000","last_clean":"2024-09-10T10:50:29.182231+0000","last_became_active":"2024-09-10T10:50:28.185484+0000","last_became_peered":"2024-09-10T10:50:28.185484+0000","last_unstale":"2024-09-10T10:50:29.182231+0000","last_undegraded":"2024-09-10T10:50:29.182231+0000","last_fullsized":"2024-09-10T10:50:29.182231+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:31.403400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.181121+0000","last_change":"2024-09-10T10:50:28.181268+0000","last_active":"2024-09-10T10:50:29.181121+0000","last_peered":"2024-09-10T10:50:29.181121+0000","last_clean":"2024-09-10T10:50:29.181121+0000","last_became_active":"2024-09-10T10:50:28.181002+0000","last_became_peered":"2024-09-10T10:50:28.181002+0000","last_unstale":"2024-09-10T10:50:29.181121+0000","last_undegraded":"2024-09-10T10:50:29.181121+0000","last_fullsized":"2024-09-10T10:50:29.181121+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:39:36.378322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223230+0000","last_change":"2024-09-10T10:50:28.199265+0000","last_active":"2024-09-10T10:50:29.223230+0000","last_peered":"2024-09-10T10:50:29.223230+0000","last_clean":"2024-09-10T10:50:29.223230+0000","last_became_active":"2024-09-10T10:50:28.198957+0000","last_became_peered":"2024-09-10T10:50:28.198957+0000","last_unstale":"2024-09-10T10:50:29.223230+0000","last_undegraded":"2024-09-10T10:50:29.223230+0000","last_fullsized":"2024-09-10T10:50:29.223230+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:23.706808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180930+0000","last_change":"2024-09-10T10:50:28.199088+0000","last_active":"2024-09-10T10:50:29.180930+0000","last_peered":"2024-09-10T10:50:29.180930+0000","last_clean":"2024-09-10T10:50:29.180930+0000","last_became_active":"2024-09-10T10:50:28.197752+0000","last_became_peered":"2024-09-10T10:50:28.197752+0000","last_unstale":"2024-09-10T10:50:29.180930+0000","last_undegraded":"2024-09-10T10:50:29.180930+0000","last_fullsized":"2024-09-10T10:50:29.180930+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:34.033217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223571+0000","last_change":"2024-09-10T10:50:28.190540+0000","last_active":"2024-09-10T10:50:29.223571+0000","last_peered":"2024-09-10T10:50:29.223571+0000","last_clean":"2024-09-10T10:50:29.223571+0000","last_became_active":"2024-09-10T10:50:28.190271+0000","last_became_peered":"2024-09-10T10:50:28.190271+0000","last_unstale":"2024-09-10T10:50:29.223571+0000","last_undegraded":"2024-09-10T10:50:29.223571+0000","last_fullsized":"2024-09-10T10:50:29.223571+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:03.485383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202026+0000","last_change":"2024-09-10T10:50:28.184346+0000","last_active":"2024-09-10T10:50:28.202026+0000","last_peered":"2024-09-10T10:50:28.202026+0000","last_clean":"2024-09-10T10:50:28.202026+0000","last_became_active":"2024-09-10T10:50:28.184068+0000","last_became_peered":"2024-09-10T10:50:28.184068+0000","last_unstale":"2024-09-10T10:50:28.202026+0000","last_undegraded":"2024-09-10T10:50:28.202026+0000","last_fullsized":"2024-09-10T10:50:28.202026+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:28:45.188246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203454+0000","last_change":"2024-09-10T10:50:28.189756+0000","last_active":"2024-09-10T10:50:29.203454+0000","last_peered":"2024-09-10T10:50:29.203454+0000","last_clean":"2024-09-10T10:50:29.203454+0000","last_became_active":"2024-09-10T10:50:28.184730+0000","last_became_peered":"2024-09-10T10:50:28.184730+0000","last_unstale":"2024-09-10T10:50:29.203454+0000","last_undegraded":"2024-09-10T10:50:29.203454+0000","last_fullsized":"2024-09-10T10:50:29.203454+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:20.637581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223100+0000","last_change":"2024-09-10T10:50:28.200233+0000","last_active":"2024-09-10T10:50:29.223100+0000","last_peered":"2024-09-10T10:50:29.223100+0000","last_clean":"2024-09-10T10:50:29.223100+0000","last_became_active":"2024-09-10T10:50:28.199822+0000","last_became_peered":"2024-09-10T10:50:28.199822+0000","last_unstale":"2024-09-10T10:50:29.223100+0000","last_undegraded":"2024-09-10T10:50:29.223100+0000","last_fullsized":"2024-09-10T10:50:29.223100+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:29:43.022303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182248+0000","last_change":"2024-09-10T10:50:28.202258+0000","last_active":"2024-09-10T10:50:29.182248+0000","last_peered":"2024-09-10T10:50:29.182248+0000","last_clean":"2024-09-10T10:50:29.182248+0000","last_became_active":"2024-09-10T10:50:28.201617+0000","last_became_peered":"2024-09-10T10:50:28.201617+0000","last_unstale":"2024-09-10T10:50:29.182248+0000","last_undegraded":"2024-09-10T10:50:29.182248+0000","last_fullsized":"2024-09-10T10:50:29.182248+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:28.044165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.183937+0000","last_change":"2024-09-10T10:50:28.184666+0000","last_active":"2024-09-10T10:50:29.183937+0000","last_peered":"2024-09-10T10:50:29.183937+0000","last_clean":"2024-09-10T10:50:29.183937+0000","last_became_active":"2024-09-10T10:50:28.184239+0000","last_became_peered":"2024-09-10T10:50:28.184239+0000","last_unstale":"2024-09-10T10:50:29.183937+0000","last_undegraded":"2024-09-10T10:50:29.183937+0000","last_fullsized":"2024-09-10T10:50:29.183937+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:47.189007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223397+0000","last_change":"2024-09-10T10:50:28.199657+0000","last_active":"2024-09-10T10:50:29.223397+0000","last_peered":"2024-09-10T10:50:29.223397+0000","last_clean":"2024-09-10T10:50:29.223397+0000","last_became_active":"2024-09-10T10:50:28.199335+0000","last_became_peered":"2024-09-10T10:50:28.199335+0000","last_unstale":"2024-09-10T10:50:29.223397+0000","last_undegraded":"2024-09-10T10:50:29.223397+0000","last_fullsized":"2024-09-10T10:50:29.223397+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:11.262762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222360+0000","last_change":"2024-09-10T10:50:28.202808+0000","last_active":"2024-09-10T10:50:29.222360+0000","last_peered":"2024-09-10T10:50:29.222360+0000","last_clean":"2024-09-10T10:50:29.222360+0000","last_became_active":"2024-09-10T10:50:28.198282+0000","last_became_peered":"2024-09-10T10:50:28.198282+0000","last_unstale":"2024-09-10T10:50:29.222360+0000","last_undegraded":"2024-09-10T10:50:29.222360+0000","last_fullsized":"2024-09-10T10:50:29.222360+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:22.853710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182923+0000","last_change":"2024-09-10T10:50:28.182460+0000","last_active":"2024-09-10T10:50:29.182923+0000","last_peered":"2024-09-10T10:50:29.182923+0000","last_clean":"2024-09-10T10:50:29.182923+0000","last_became_active":"2024-09-10T10:50:28.181932+0000","last_became_peered":"2024-09-10T10:50:28.181932+0000","last_unstale":"2024-09-10T10:50:29.182923+0000","last_undegraded":"2024-09-10T10:50:29.182923+0000","last_fullsized":"2024-09-10T10:50:29.182923+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:52.992086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223215+0000","last_change":"2024-09-10T10:50:28.177625+0000","last_active":"2024-09-10T10:50:29.223215+0000","last_peered":"2024-09-10T10:50:29.223215+0000","last_clean":"2024-09-10T10:50:29.223215+0000","last_became_active":"2024-09-10T10:50:28.177352+0000","last_became_peered":"2024-09-10T10:50:28.177352+0000","last_unstale":"2024-09-10T10:50:29.223215+0000","last_undegraded":"2024-09-10T10:50:29.223215+0000","last_fullsized":"2024-09-10T10:50:29.223215+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:01:48.078804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222340+0000","last_change":"2024-09-10T10:50:28.185298+0000","last_active":"2024-09-10T10:50:29.222340+0000","last_peered":"2024-09-10T10:50:29.222340+0000","last_clean":"2024-09-10T10:50:29.222340+0000","last_became_active":"2024-09-10T10:50:28.184554+0000","last_became_peered":"2024-09-10T10:50:28.184554+0000","last_unstale":"2024-09-10T10:50:29.222340+0000","last_undegraded":"2024-09-10T10:50:29.222340+0000","last_fullsized":"2024-09-10T10:50:29.222340+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:47.352205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222785+0000","last_change":"2024-09-10T10:50:28.199293+0000","last_active":"2024-09-10T10:50:29.222785+0000","last_peered":"2024-09-10T10:50:29.222785+0000","last_clean":"2024-09-10T10:50:29.222785+0000","last_became_active":"2024-09-10T10:50:28.197114+0000","last_became_peered":"2024-09-10T10:50:28.197114+0000","last_unstale":"2024-09-10T10:50:29.222785+0000","last_undegraded":"2024-09-10T10:50:29.222785+0000","last_fullsized":"2024-09-10T10:50:29.222785+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:22.853710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223120+0000","last_change":"2024-09-10T10:50:28.185953+0000","last_active":"2024-09-10T10:50:29.223120+0000","last_peered":"2024-09-10T10:50:29.223120+0000","last_clean":"2024-09-10T10:50:29.223120+0000","last_became_active":"2024-09-10T10:50:28.185660+0000","last_became_peered":"2024-09-10T10:50:28.185660+0000","last_unstale":"2024-09-10T10:50:29.223120+0000","last_undegraded":"2024-09-10T10:50:29.223120+0000","last_fullsized":"2024-09-10T10:50:29.223120+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:31.175994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223597+0000","last_change":"2024-09-10T10:50:28.179242+0000","last_active":"2024-09-10T10:50:29.223597+0000","last_peered":"2024-09-10T10:50:29.223597+0000","last_clean":"2024-09-10T10:50:29.223597+0000","last_became_active":"2024-09-10T10:50:28.178958+0000","last_became_peered":"2024-09-10T10:50:28.178958+0000","last_unstale":"2024-09-10T10:50:29.223597+0000","last_undegraded":"2024-09-10T10:50:29.223597+0000","last_fullsized":"2024-09-10T10:50:29.223597+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:29:31.403400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223618+0000","last_change":"2024-09-10T10:50:28.200982+0000","last_active":"2024-09-10T10:50:29.223618+0000","last_peered":"2024-09-10T10:50:29.223618+0000","last_clean":"2024-09-10T10:50:29.223618+0000","last_became_active":"2024-09-10T10:50:28.200683+0000","last_became_peered":"2024-09-10T10:50:28.200683+0000","last_unstale":"2024-09-10T10:50:29.223618+0000","last_undegraded":"2024-09-10T10:50:29.223618+0000","last_fullsized":"2024-09-10T10:50:29.223618+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:28.044165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223993+0000","last_change":"2024-09-10T10:50:28.188973+0000","last_active":"2024-09-10T10:50:29.223993+0000","last_peered":"2024-09-10T10:50:29.223993+0000","last_clean":"2024-09-10T10:50:29.223993+0000","last_became_active":"2024-09-10T10:50:28.188574+0000","last_became_peered":"2024-09-10T10:50:28.188574+0000","last_unstale":"2024-09-10T10:50:29.223993+0000","last_undegraded":"2024-09-10T10:50:29.223993+0000","last_fullsized":"2024-09-10T10:50:29.223993+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:20:15.415473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.204948+0000","last_change":"2024-09-10T10:50:28.193766+0000","last_active":"2024-09-10T10:50:28.204948+0000","last_peered":"2024-09-10T10:50:28.204948+0000","last_clean":"2024-09-10T10:50:28.204948+0000","last_became_active":"2024-09-10T10:50:28.193289+0000","last_became_peered":"2024-09-10T10:50:28.193289+0000","last_unstale":"2024-09-10T10:50:28.204948+0000","last_undegraded":"2024-09-10T10:50:28.204948+0000","last_fullsized":"2024-09-10T10:50:28.204948+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:58.731770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223338+0000","last_change":"2024-09-10T10:50:28.187853+0000","last_active":"2024-09-10T10:50:29.223338+0000","last_peered":"2024-09-10T10:50:29.223338+0000","last_clean":"2024-09-10T10:50:29.223338+0000","last_became_active":"2024-09-10T10:50:28.187466+0000","last_became_peered":"2024-09-10T10:50:28.187466+0000","last_unstale":"2024-09-10T10:50:29.223338+0000","last_undegraded":"2024-09-10T10:50:29.223338+0000","last_fullsized":"2024-09-10T10:50:29.223338+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:07:34.846864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223088+0000","last_change":"2024-09-10T10:50:28.187770+0000","last_active":"2024-09-10T10:50:29.223088+0000","last_peered":"2024-09-10T10:50:29.223088+0000","last_clean":"2024-09-10T10:50:29.223088+0000","last_became_active":"2024-09-10T10:50:28.187396+0000","last_became_peered":"2024-09-10T10:50:28.187396+0000","last_unstale":"2024-09-10T10:50:29.223088+0000","last_undegraded":"2024-09-10T10:50:29.223088+0000","last_fullsized":"2024-09-10T10:50:29.223088+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:30.166347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222226+0000","last_change":"2024-09-10T10:50:28.191962+0000","last_active":"2024-09-10T10:50:29.222226+0000","last_peered":"2024-09-10T10:50:29.222226+0000","last_clean":"2024-09-10T10:50:29.222226+0000","last_became_active":"2024-09-10T10:50:28.191086+0000","last_became_peered":"2024-09-10T10:50:28.191086+0000","last_unstale":"2024-09-10T10:50:29.222226+0000","last_undegraded":"2024-09-10T10:50:29.222226+0000","last_fullsized":"2024-09-10T10:50:29.222226+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:20.637581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182418+0000","last_change":"2024-09-10T10:50:28.205394+0000","last_active":"2024-09-10T10:50:29.182418+0000","last_peered":"2024-09-10T10:50:29.182418+0000","last_clean":"2024-09-10T10:50:29.182418+0000","last_became_active":"2024-09-10T10:50:28.202014+0000","last_became_peered":"2024-09-10T10:50:28.202014+0000","last_unstale":"2024-09-10T10:50:29.182418+0000","last_undegraded":"2024-09-10T10:50:29.182418+0000","last_fullsized":"2024-09-10T10:50:29.182418+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:31.175994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223714+0000","last_change":"2024-09-10T10:50:28.191255+0000","last_active":"2024-09-10T10:50:29.223714+0000","last_peered":"2024-09-10T10:50:29.223714+0000","last_clean":"2024-09-10T10:50:29.223714+0000","last_became_active":"2024-09-10T10:50:28.189438+0000","last_became_peered":"2024-09-10T10:50:28.189438+0000","last_unstale":"2024-09-10T10:50:29.223714+0000","last_undegraded":"2024-09-10T10:50:29.223714+0000","last_fullsized":"2024-09-10T10:50:29.223714+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:22:52.992086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"98'4","reported_seq":18,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223880+0000","last_change":"2024-09-10T10:50:27.187237+0000","last_active":"2024-09-10T10:50:29.223880+0000","last_peered":"2024-09-10T10:50:29.223880+0000","last_clean":"2024-09-10T10:50:29.223880+0000","last_became_active":"2024-09-10T10:50:27.185655+0000","last_became_peered":"2024-09-10T10:50:27.185655+0000","last_unstale":"2024-09-10T10:50:29.223880+0000","last_undegraded":"2024-09-10T10:50:29.223880+0000","last_fullsized":"2024-09-10T10:50:29.223880+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:14.759772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.183078+0000","last_change":"2024-09-10T10:50:28.188138+0000","last_active":"2024-09-10T10:50:29.183078+0000","last_peered":"2024-09-10T10:50:29.183078+0000","last_clean":"2024-09-10T10:50:29.183078+0000","last_became_active":"2024-09-10T10:50:28.187838+0000","last_became_peered":"2024-09-10T10:50:28.187838+0000","last_unstale":"2024-09-10T10:50:29.183078+0000","last_undegraded":"2024-09-10T10:50:29.183078+0000","last_fullsized":"2024-09-10T10:50:29.183078+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:13.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203605+0000","last_change":"2024-09-10T10:50:27.175301+0000","last_active":"2024-09-10T10:50:29.203605+0000","last_peered":"2024-09-10T10:50:29.203605+0000","last_clean":"2024-09-10T10:50:29.203605+0000","last_became_active":"2024-09-10T10:50:27.174963+0000","last_became_peered":"2024-09-10T10:50:27.174963+0000","last_unstale":"2024-09-10T10:50:29.203605+0000","last_undegraded":"2024-09-10T10:50:29.203605+0000","last_fullsized":"2024-09-10T10:50:29.203605+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:50.280764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180964+0000","last_change":"2024-09-10T10:50:28.188043+0000","last_active":"2024-09-10T10:50:29.180964+0000","last_peered":"2024-09-10T10:50:29.180964+0000","last_clean":"2024-09-10T10:50:29.180964+0000","last_became_active":"2024-09-10T10:50:28.187725+0000","last_became_peered":"2024-09-10T10:50:28.187725+0000","last_unstale":"2024-09-10T10:50:29.180964+0000","last_undegraded":"2024-09-10T10:50:29.180964+0000","last_fullsized":"2024-09-10T10:50:29.180964+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:18:01.482371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223400+0000","last_change":"2024-09-10T10:50:27.189906+0000","last_active":"2024-09-10T10:50:29.223400+0000","last_peered":"2024-09-10T10:50:29.223400+0000","last_clean":"2024-09-10T10:50:29.223400+0000","last_became_active":"2024-09-10T10:50:27.189622+0000","last_became_peered":"2024-09-10T10:50:27.189622+0000","last_unstale":"2024-09-10T10:50:29.223400+0000","last_undegraded":"2024-09-10T10:50:29.223400+0000","last_fullsized":"2024-09-10T10:50:29.223400+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:09:10.259471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.224032+0000","last_change":"2024-09-10T10:50:28.197503+0000","last_active":"2024-09-10T10:50:29.224032+0000","last_peered":"2024-09-10T10:50:29.224032+0000","last_clean":"2024-09-10T10:50:29.224032+0000","last_became_active":"2024-09-10T10:50:28.197209+0000","last_became_peered":"2024-09-10T10:50:28.197209+0000","last_unstale":"2024-09-10T10:50:29.224032+0000","last_undegraded":"2024-09-10T10:50:29.224032+0000","last_fullsized":"2024-09-10T10:50:29.224032+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:53:15.763062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203452+0000","last_change":"2024-09-10T10:50:27.184761+0000","last_active":"2024-09-10T10:50:29.203452+0000","last_peered":"2024-09-10T10:50:29.203452+0000","last_clean":"2024-09-10T10:50:29.203452+0000","last_became_active":"2024-09-10T10:50:27.184459+0000","last_became_peered":"2024-09-10T10:50:27.184459+0000","last_unstale":"2024-09-10T10:50:29.203452+0000","last_undegraded":"2024-09-10T10:50:29.203452+0000","last_fullsized":"2024-09-10T10:50:29.203452+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:32:27.701480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202908+0000","last_change":"2024-09-10T10:50:28.188444+0000","last_active":"2024-09-10T10:50:28.202908+0000","last_peered":"2024-09-10T10:50:28.202908+0000","last_clean":"2024-09-10T10:50:28.202908+0000","last_became_active":"2024-09-10T10:50:28.188169+0000","last_became_peered":"2024-09-10T10:50:28.188169+0000","last_unstale":"2024-09-10T10:50:28.202908+0000","last_undegraded":"2024-09-10T10:50:28.202908+0000","last_fullsized":"2024-09-10T10:50:28.202908+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:24.216712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222704+0000","last_change":"2024-09-10T10:50:27.179618+0000","last_active":"2024-09-10T10:50:29.222704+0000","last_peered":"2024-09-10T10:50:29.222704+0000","last_clean":"2024-09-10T10:50:29.222704+0000","last_became_active":"2024-09-10T10:50:27.179306+0000","last_became_peered":"2024-09-10T10:50:27.179306+0000","last_unstale":"2024-09-10T10:50:29.222704+0000","last_undegraded":"2024-09-10T10:50:29.222704+0000","last_fullsized":"2024-09-10T10:50:29.222704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:20.343923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180626+0000","last_change":"2024-09-10T10:50:28.199350+0000","last_active":"2024-09-10T10:50:29.180626+0000","last_peered":"2024-09-10T10:50:29.180626+0000","last_clean":"2024-09-10T10:50:29.180626+0000","last_became_active":"2024-09-10T10:50:28.199087+0000","last_became_peered":"2024-09-10T10:50:28.199087+0000","last_unstale":"2024-09-10T10:50:29.180626+0000","last_undegraded":"2024-09-10T10:50:29.180626+0000","last_fullsized":"2024-09-10T10:50:29.180626+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:23.190222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"98'1","reported_seq":13,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.203930+0000","last_change":"2024-09-10T10:50:27.191277+0000","last_active":"2024-09-10T10:50:28.203930+0000","last_peered":"2024-09-10T10:50:28.203930+0000","last_clean":"2024-09-10T10:50:28.203930+0000","last_became_active":"2024-09-10T10:50:27.190992+0000","last_became_peered":"2024-09-10T10:50:27.190992+0000","last_unstale":"2024-09-10T10:50:28.203930+0000","last_undegraded":"2024-09-10T10:50:28.203930+0000","last_fullsized":"2024-09-10T10:50:28.203930+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:00.947632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182846+0000","last_change":"2024-09-10T10:50:28.192994+0000","last_active":"2024-09-10T10:50:29.182846+0000","last_peered":"2024-09-10T10:50:29.182846+0000","last_clean":"2024-09-10T10:50:29.182846+0000","last_became_active":"2024-09-10T10:50:28.192710+0000","last_became_peered":"2024-09-10T10:50:28.192710+0000","last_unstale":"2024-09-10T10:50:29.182846+0000","last_undegraded":"2024-09-10T10:50:29.182846+0000","last_fullsized":"2024-09-10T10:50:29.182846+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:52.383485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223441+0000","last_change":"2024-09-10T10:50:27.178210+0000","last_active":"2024-09-10T10:50:29.223441+0000","last_peered":"2024-09-10T10:50:29.223441+0000","last_clean":"2024-09-10T10:50:29.223441+0000","last_became_active":"2024-09-10T10:50:27.177930+0000","last_became_peered":"2024-09-10T10:50:27.177930+0000","last_unstale":"2024-09-10T10:50:29.223441+0000","last_undegraded":"2024-09-10T10:50:29.223441+0000","last_fullsized":"2024-09-10T10:50:29.223441+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:43.480148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223476+0000","last_change":"2024-09-10T10:50:28.191493+0000","last_active":"2024-09-10T10:50:29.223476+0000","last_peered":"2024-09-10T10:50:29.223476+0000","last_clean":"2024-09-10T10:50:29.223476+0000","last_became_active":"2024-09-10T10:50:28.191106+0000","last_became_peered":"2024-09-10T10:50:28.191106+0000","last_unstale":"2024-09-10T10:50:29.223476+0000","last_undegraded":"2024-09-10T10:50:29.223476+0000","last_fullsized":"2024-09-10T10:50:29.223476+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:04.379276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222423+0000","last_change":"2024-09-10T10:50:27.188910+0000","last_active":"2024-09-10T10:50:29.222423+0000","last_peered":"2024-09-10T10:50:29.222423+0000","last_clean":"2024-09-10T10:50:29.222423+0000","last_became_active":"2024-09-10T10:50:27.188566+0000","last_became_peered":"2024-09-10T10:50:27.188566+0000","last_unstale":"2024-09-10T10:50:29.222423+0000","last_undegraded":"2024-09-10T10:50:29.222423+0000","last_fullsized":"2024-09-10T10:50:29.222423+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:12:22.703518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223302+0000","last_change":"2024-09-10T10:50:28.191076+0000","last_active":"2024-09-10T10:50:29.223302+0000","last_peered":"2024-09-10T10:50:29.223302+0000","last_clean":"2024-09-10T10:50:29.223302+0000","last_became_active":"2024-09-10T10:50:28.188604+0000","last_became_peered":"2024-09-10T10:50:28.188604+0000","last_unstale":"2024-09-10T10:50:29.223302+0000","last_undegraded":"2024-09-10T10:50:29.223302+0000","last_fullsized":"2024-09-10T10:50:29.223302+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:05.120513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223882+0000","last_change":"2024-09-10T10:50:27.190965+0000","last_active":"2024-09-10T10:50:29.223882+0000","last_peered":"2024-09-10T10:50:29.223882+0000","last_clean":"2024-09-10T10:50:29.223882+0000","last_became_active":"2024-09-10T10:50:27.190506+0000","last_became_peered":"2024-09-10T10:50:27.190506+0000","last_unstale":"2024-09-10T10:50:29.223882+0000","last_undegraded":"2024-09-10T10:50:29.223882+0000","last_fullsized":"2024-09-10T10:50:29.223882+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:46.847232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.181025+0000","last_change":"2024-09-10T10:50:28.181343+0000","last_active":"2024-09-10T10:50:29.181025+0000","last_peered":"2024-09-10T10:50:29.181025+0000","last_clean":"2024-09-10T10:50:29.181025+0000","last_became_active":"2024-09-10T10:50:28.180554+0000","last_became_peered":"2024-09-10T10:50:28.180554+0000","last_unstale":"2024-09-10T10:50:29.181025+0000","last_undegraded":"2024-09-10T10:50:29.181025+0000","last_fullsized":"2024-09-10T10:50:29.181025+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:38:17.946911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223640+0000","last_change":"2024-09-10T10:50:27.184552+0000","last_active":"2024-09-10T10:50:29.223640+0000","last_peered":"2024-09-10T10:50:29.223640+0000","last_clean":"2024-09-10T10:50:29.223640+0000","last_became_active":"2024-09-10T10:50:27.184063+0000","last_became_peered":"2024-09-10T10:50:27.184063+0000","last_unstale":"2024-09-10T10:50:29.223640+0000","last_undegraded":"2024-09-10T10:50:29.223640+0000","last_fullsized":"2024-09-10T10:50:29.223640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:00.054489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223830+0000","last_change":"2024-09-10T10:50:28.195521+0000","last_active":"2024-09-10T10:50:29.223830+0000","last_peered":"2024-09-10T10:50:29.223830+0000","last_clean":"2024-09-10T10:50:29.223830+0000","last_became_active":"2024-09-10T10:50:28.195008+0000","last_became_peered":"2024-09-10T10:50:28.195008+0000","last_unstale":"2024-09-10T10:50:29.223830+0000","last_undegraded":"2024-09-10T10:50:29.223830+0000","last_fullsized":"2024-09-10T10:50:29.223830+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:09:13.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"98'3","reported_seq":17,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.181174+0000","last_change":"2024-09-10T10:50:27.191029+0000","last_active":"2024-09-10T10:50:29.181174+0000","last_peered":"2024-09-10T10:50:29.181174+0000","last_clean":"2024-09-10T10:50:29.181174+0000","last_became_active":"2024-09-10T10:50:27.190730+0000","last_became_peered":"2024-09-10T10:50:27.190730+0000","last_unstale":"2024-09-10T10:50:29.181174+0000","last_undegraded":"2024-09-10T10:50:29.181174+0000","last_fullsized":"2024-09-10T10:50:29.181174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:55.546071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.184036+0000","last_change":"2024-09-10T10:50:28.189190+0000","last_active":"2024-09-10T10:50:29.184036+0000","last_peered":"2024-09-10T10:50:29.184036+0000","last_clean":"2024-09-10T10:50:29.184036+0000","last_became_active":"2024-09-10T10:50:28.188873+0000","last_became_peered":"2024-09-10T10:50:28.188873+0000","last_unstale":"2024-09-10T10:50:29.184036+0000","last_undegraded":"2024-09-10T10:50:29.184036+0000","last_fullsized":"2024-09-10T10:50:29.184036+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:18.576619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"98'6","reported_seq":20,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223841+0000","last_change":"2024-09-10T10:50:27.189980+0000","last_active":"2024-09-10T10:50:29.223841+0000","last_peered":"2024-09-10T10:50:29.223841+0000","last_clean":"2024-09-10T10:50:29.223841+0000","last_became_active":"2024-09-10T10:50:27.188777+0000","last_became_peered":"2024-09-10T10:50:27.188777+0000","last_unstale":"2024-09-10T10:50:29.223841+0000","last_undegraded":"2024-09-10T10:50:29.223841+0000","last_fullsized":"2024-09-10T10:50:29.223841+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:17:31.413739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223138+0000","last_change":"2024-09-10T10:50:28.185450+0000","last_active":"2024-09-10T10:50:29.223138+0000","last_peered":"2024-09-10T10:50:29.223138+0000","last_clean":"2024-09-10T10:50:29.223138+0000","last_became_active":"2024-09-10T10:50:28.185189+0000","last_became_peered":"2024-09-10T10:50:28.185189+0000","last_unstale":"2024-09-10T10:50:29.223138+0000","last_undegraded":"2024-09-10T10:50:29.223138+0000","last_fullsized":"2024-09-10T10:50:29.223138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:15.515889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223308+0000","last_change":"2024-09-10T10:50:27.190720+0000","last_active":"2024-09-10T10:50:29.223308+0000","last_peered":"2024-09-10T10:50:29.223308+0000","last_clean":"2024-09-10T10:50:29.223308+0000","last_became_active":"2024-09-10T10:50:27.190403+0000","last_became_peered":"2024-09-10T10:50:27.190403+0000","last_unstale":"2024-09-10T10:50:29.223308+0000","last_undegraded":"2024-09-10T10:50:29.223308+0000","last_fullsized":"2024-09-10T10:50:29.223308+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:02.482092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222890+0000","last_change":"2024-09-10T10:50:28.186636+0000","last_active":"2024-09-10T10:50:29.222890+0000","last_peered":"2024-09-10T10:50:29.222890+0000","last_clean":"2024-09-10T10:50:29.222890+0000","last_became_active":"2024-09-10T10:50:28.186303+0000","last_became_peered":"2024-09-10T10:50:28.186303+0000","last_unstale":"2024-09-10T10:50:29.222890+0000","last_undegraded":"2024-09-10T10:50:29.222890+0000","last_fullsized":"2024-09-10T10:50:29.222890+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:14.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223294+0000","last_change":"2024-09-10T10:50:27.189562+0000","last_active":"2024-09-10T10:50:29.223294+0000","last_peered":"2024-09-10T10:50:29.223294+0000","last_clean":"2024-09-10T10:50:29.223294+0000","last_became_active":"2024-09-10T10:50:27.188413+0000","last_became_peered":"2024-09-10T10:50:27.188413+0000","last_unstale":"2024-09-10T10:50:29.223294+0000","last_undegraded":"2024-09-10T10:50:29.223294+0000","last_fullsized":"2024-09-10T10:50:29.223294+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:19.634290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223101+0000","last_change":"2024-09-10T10:50:28.190612+0000","last_active":"2024-09-10T10:50:29.223101+0000","last_peered":"2024-09-10T10:50:29.223101+0000","last_clean":"2024-09-10T10:50:29.223101+0000","last_became_active":"2024-09-10T10:50:28.190334+0000","last_became_peered":"2024-09-10T10:50:28.190334+0000","last_unstale":"2024-09-10T10:50:29.223101+0000","last_undegraded":"2024-09-10T10:50:29.223101+0000","last_fullsized":"2024-09-10T10:50:29.223101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:28.044165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222968+0000","last_change":"2024-09-10T10:50:27.189980+0000","last_active":"2024-09-10T10:50:29.222968+0000","last_peered":"2024-09-10T10:50:29.222968+0000","last_clean":"2024-09-10T10:50:29.222968+0000","last_became_active":"2024-09-10T10:50:27.189192+0000","last_became_peered":"2024-09-10T10:50:27.189192+0000","last_unstale":"2024-09-10T10:50:29.222968+0000","last_undegraded":"2024-09-10T10:50:29.222968+0000","last_fullsized":"2024-09-10T10:50:29.222968+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.428926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.206563+0000","last_change":"2024-09-10T10:50:28.201838+0000","last_active":"2024-09-10T10:50:28.206563+0000","last_peered":"2024-09-10T10:50:28.206563+0000","last_clean":"2024-09-10T10:50:28.206563+0000","last_became_active":"2024-09-10T10:50:28.197464+0000","last_became_peered":"2024-09-10T10:50:28.197464+0000","last_unstale":"2024-09-10T10:50:28.206563+0000","last_undegraded":"2024-09-10T10:50:28.206563+0000","last_fullsized":"2024-09-10T10:50:28.206563+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:05:39.909923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"98'8","reported_seq":22,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223044+0000","last_change":"2024-09-10T10:50:27.186350+0000","last_active":"2024-09-10T10:50:29.223044+0000","last_peered":"2024-09-10T10:50:29.223044+0000","last_clean":"2024-09-10T10:50:29.223044+0000","last_became_active":"2024-09-10T10:50:27.185921+0000","last_became_peered":"2024-09-10T10:50:27.185921+0000","last_unstale":"2024-09-10T10:50:29.223044+0000","last_undegraded":"2024-09-10T10:50:29.223044+0000","last_fullsized":"2024-09-10T10:50:29.223044+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:15.415730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.204973+0000","last_change":"2024-09-10T10:50:28.194356+0000","last_active":"2024-09-10T10:50:28.204973+0000","last_peered":"2024-09-10T10:50:28.204973+0000","last_clean":"2024-09-10T10:50:28.204973+0000","last_became_active":"2024-09-10T10:50:28.194087+0000","last_became_peered":"2024-09-10T10:50:28.194087+0000","last_unstale":"2024-09-10T10:50:28.204973+0000","last_undegraded":"2024-09-10T10:50:28.204973+0000","last_fullsized":"2024-09-10T10:50:28.204973+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:21:42.144671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222535+0000","last_change":"2024-09-10T10:50:27.180001+0000","last_active":"2024-09-10T10:50:29.222535+0000","last_peered":"2024-09-10T10:50:29.222535+0000","last_clean":"2024-09-10T10:50:29.222535+0000","last_became_active":"2024-09-10T10:50:27.179711+0000","last_became_peered":"2024-09-10T10:50:27.179711+0000","last_unstale":"2024-09-10T10:50:29.222535+0000","last_undegraded":"2024-09-10T10:50:29.222535+0000","last_fullsized":"2024-09-10T10:50:29.222535+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.227839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.181083+0000","last_change":"2024-09-10T10:50:28.190581+0000","last_active":"2024-09-10T10:50:29.181083+0000","last_peered":"2024-09-10T10:50:29.181083+0000","last_clean":"2024-09-10T10:50:29.181083+0000","last_became_active":"2024-09-10T10:50:28.188600+0000","last_became_peered":"2024-09-10T10:50:28.188600+0000","last_unstale":"2024-09-10T10:50:29.181083+0000","last_undegraded":"2024-09-10T10:50:29.181083+0000","last_fullsized":"2024-09-10T10:50:29.181083+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:17.055970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203668+0000","last_change":"2024-09-10T10:50:27.177545+0000","last_active":"2024-09-10T10:50:29.203668+0000","last_peered":"2024-09-10T10:50:29.203668+0000","last_clean":"2024-09-10T10:50:29.203668+0000","last_became_active":"2024-09-10T10:50:27.177215+0000","last_became_peered":"2024-09-10T10:50:27.177215+0000","last_unstale":"2024-09-10T10:50:29.203668+0000","last_undegraded":"2024-09-10T10:50:29.203668+0000","last_fullsized":"2024-09-10T10:50:29.203668+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:46.095780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223534+0000","last_change":"2024-09-10T10:50:28.191509+0000","last_active":"2024-09-10T10:50:29.223534+0000","last_peered":"2024-09-10T10:50:29.223534+0000","last_clean":"2024-09-10T10:50:29.223534+0000","last_became_active":"2024-09-10T10:50:28.191212+0000","last_became_peered":"2024-09-10T10:50:28.191212+0000","last_unstale":"2024-09-10T10:50:29.223534+0000","last_undegraded":"2024-09-10T10:50:29.223534+0000","last_fullsized":"2024-09-10T10:50:29.223534+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:24:20.992544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"98'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203580+0000","last_change":"2024-09-10T10:50:27.182791+0000","last_active":"2024-09-10T10:50:29.203580+0000","last_peered":"2024-09-10T10:50:29.203580+0000","last_clean":"2024-09-10T10:50:29.203580+0000","last_became_active":"2024-09-10T10:50:27.181051+0000","last_became_peered":"2024-09-10T10:50:27.181051+0000","last_unstale":"2024-09-10T10:50:29.203580+0000","last_undegraded":"2024-09-10T10:50:29.203580+0000","last_fullsized":"2024-09-10T10:50:29.203580+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:56:44.227839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182440+0000","last_change":"2024-09-10T10:50:28.196398+0000","last_active":"2024-09-10T10:50:29.182440+0000","last_peered":"2024-09-10T10:50:29.182440+0000","last_clean":"2024-09-10T10:50:29.182440+0000","last_became_active":"2024-09-10T10:50:28.196095+0000","last_became_peered":"2024-09-10T10:50:28.196095+0000","last_unstale":"2024-09-10T10:50:29.182440+0000","last_undegraded":"2024-09-10T10:50:29.182440+0000","last_fullsized":"2024-09-10T10:50:29.182440+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:15.515889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180545+0000","last_change":"2024-09-10T10:50:27.190351+0000","last_active":"2024-09-10T10:50:29.180545+0000","last_peered":"2024-09-10T10:50:29.180545+0000","last_clean":"2024-09-10T10:50:29.180545+0000","last_became_active":"2024-09-10T10:50:27.190089+0000","last_became_peered":"2024-09-10T10:50:27.190089+0000","last_unstale":"2024-09-10T10:50:29.180545+0000","last_undegraded":"2024-09-10T10:50:29.180545+0000","last_fullsized":"2024-09-10T10:50:29.180545+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:10:32.744485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203397+0000","last_change":"2024-09-10T10:50:28.196336+0000","last_active":"2024-09-10T10:50:29.203397+0000","last_peered":"2024-09-10T10:50:29.203397+0000","last_clean":"2024-09-10T10:50:29.203397+0000","last_became_active":"2024-09-10T10:50:28.196067+0000","last_became_peered":"2024-09-10T10:50:28.196067+0000","last_unstale":"2024-09-10T10:50:29.203397+0000","last_undegraded":"2024-09-10T10:50:29.203397+0000","last_fullsized":"2024-09-10T10:50:29.203397+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:34.093896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182985+0000","last_change":"2024-09-10T10:50:27.190765+0000","last_active":"2024-09-10T10:50:29.182985+0000","last_peered":"2024-09-10T10:50:29.182985+0000","last_clean":"2024-09-10T10:50:29.182985+0000","last_became_active":"2024-09-10T10:50:27.190168+0000","last_became_peered":"2024-09-10T10:50:27.190168+0000","last_unstale":"2024-09-10T10:50:29.182985+0000","last_undegraded":"2024-09-10T10:50:29.182985+0000","last_fullsized":"2024-09-10T10:50:29.182985+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:03.428926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222482+0000","last_change":"2024-09-10T10:50:28.197580+0000","last_active":"2024-09-10T10:50:29.222482+0000","last_peered":"2024-09-10T10:50:29.222482+0000","last_clean":"2024-09-10T10:50:29.222482+0000","last_became_active":"2024-09-10T10:50:28.197400+0000","last_became_peered":"2024-09-10T10:50:28.197400+0000","last_unstale":"2024-09-10T10:50:29.222482+0000","last_undegraded":"2024-09-10T10:50:29.222482+0000","last_fullsized":"2024-09-10T10:50:29.222482+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:52:14.016985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.203345+0000","last_change":"2024-09-10T10:50:27.183922+0000","last_active":"2024-09-10T10:50:29.203345+0000","last_peered":"2024-09-10T10:50:29.203345+0000","last_clean":"2024-09-10T10:50:29.203345+0000","last_became_active":"2024-09-10T10:50:27.183637+0000","last_became_peered":"2024-09-10T10:50:27.183637+0000","last_unstale":"2024-09-10T10:50:29.203345+0000","last_undegraded":"2024-09-10T10:50:29.203345+0000","last_fullsized":"2024-09-10T10:50:29.203345+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:31.639866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223631+0000","last_change":"2024-09-10T10:50:28.182100+0000","last_active":"2024-09-10T10:50:29.223631+0000","last_peered":"2024-09-10T10:50:29.223631+0000","last_clean":"2024-09-10T10:50:29.223631+0000","last_became_active":"2024-09-10T10:50:28.181392+0000","last_became_peered":"2024-09-10T10:50:28.181392+0000","last_unstale":"2024-09-10T10:50:29.223631+0000","last_undegraded":"2024-09-10T10:50:29.223631+0000","last_fullsized":"2024-09-10T10:50:29.223631+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:42:52.383485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.175178+0000","last_change":"2024-09-10T10:50:27.183793+0000","last_active":"2024-09-10T10:50:28.175178+0000","last_peered":"2024-09-10T10:50:28.175178+0000","last_clean":"2024-09-10T10:50:28.175178+0000","last_became_active":"2024-09-10T10:50:27.183501+0000","last_became_peered":"2024-09-10T10:50:27.183501+0000","last_unstale":"2024-09-10T10:50:28.175178+0000","last_undegraded":"2024-09-10T10:50:28.175178+0000","last_fullsized":"2024-09-10T10:50:28.175178+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:33.239560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222210+0000","last_change":"2024-09-10T10:50:28.203098+0000","last_active":"2024-09-10T10:50:29.222210+0000","last_peered":"2024-09-10T10:50:29.222210+0000","last_clean":"2024-09-10T10:50:29.222210+0000","last_became_active":"2024-09-10T10:50:28.202490+0000","last_became_peered":"2024-09-10T10:50:28.202490+0000","last_unstale":"2024-09-10T10:50:29.222210+0000","last_undegraded":"2024-09-10T10:50:29.222210+0000","last_fullsized":"2024-09-10T10:50:29.222210+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:34.093896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"98'2","reported_seq":16,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.181145+0000","last_change":"2024-09-10T10:50:27.172716+0000","last_active":"2024-09-10T10:50:29.181145+0000","last_peered":"2024-09-10T10:50:29.181145+0000","last_clean":"2024-09-10T10:50:29.181145+0000","last_became_active":"2024-09-10T10:50:27.171686+0000","last_became_peered":"2024-09-10T10:50:27.171686+0000","last_unstale":"2024-09-10T10:50:29.181145+0000","last_undegraded":"2024-09-10T10:50:29.181145+0000","last_fullsized":"2024-09-10T10:50:29.181145+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:51:58.282518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.202898+0000","last_change":"2024-09-10T10:50:28.190397+0000","last_active":"2024-09-10T10:50:28.202898+0000","last_peered":"2024-09-10T10:50:28.202898+0000","last_clean":"2024-09-10T10:50:28.202898+0000","last_became_active":"2024-09-10T10:50:28.189002+0000","last_became_peered":"2024-09-10T10:50:28.189002+0000","last_unstale":"2024-09-10T10:50:28.202898+0000","last_undegraded":"2024-09-10T10:50:28.202898+0000","last_fullsized":"2024-09-10T10:50:28.202898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:25:54.061184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"99'1","reported_seq":15,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:30.174024+0000","last_change":"2024-09-10T10:50:27.182026+0000","last_active":"2024-09-10T10:50:30.174024+0000","last_peered":"2024-09-10T10:50:30.174024+0000","last_clean":"2024-09-10T10:50:30.174024+0000","last_became_active":"2024-09-10T10:50:27.181066+0000","last_became_peered":"2024-09-10T10:50:27.181066+0000","last_unstale":"2024-09-10T10:50:30.174024+0000","last_undegraded":"2024-09-10T10:50:30.174024+0000","last_fullsized":"2024-09-10T10:50:30.174024+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:34:29.768086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-10T10:50:28.184845+0000","last_change":"2024-09-10T10:50:28.179368+0000","last_active":"2024-09-10T10:50:28.184845+0000","last_peered":"2024-09-10T10:50:28.184845+0000","last_clean":"2024-09-10T10:50:28.184845+0000","last_became_active":"2024-09-10T10:50:28.179083+0000","last_became_peered":"2024-09-10T10:50:28.179083+0000","last_unstale":"2024-09-10T10:50:28.184845+0000","last_undegraded":"2024-09-10T10:50:28.184845+0000","last_fullsized":"2024-09-10T10:50:28.184845+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:07:13.712524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222442+0000","last_change":"2024-09-10T10:50:27.179717+0000","last_active":"2024-09-10T10:50:29.222442+0000","last_peered":"2024-09-10T10:50:29.222442+0000","last_clean":"2024-09-10T10:50:29.222442+0000","last_became_active":"2024-09-10T10:50:27.179448+0000","last_became_peered":"2024-09-10T10:50:27.179448+0000","last_unstale":"2024-09-10T10:50:29.222442+0000","last_undegraded":"2024-09-10T10:50:29.222442+0000","last_fullsized":"2024-09-10T10:50:29.222442+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:09:31.639866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223760+0000","last_change":"2024-09-10T10:50:28.197936+0000","last_active":"2024-09-10T10:50:29.223760+0000","last_peered":"2024-09-10T10:50:29.223760+0000","last_clean":"2024-09-10T10:50:29.223760+0000","last_became_active":"2024-09-10T10:50:28.191216+0000","last_became_peered":"2024-09-10T10:50:28.191216+0000","last_unstale":"2024-09-10T10:50:29.223760+0000","last_undegraded":"2024-09-10T10:50:29.223760+0000","last_fullsized":"2024-09-10T10:50:29.223760+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:48:15.515889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.180885+0000","last_change":"2024-09-10T10:50:27.183689+0000","last_active":"2024-09-10T10:50:29.180885+0000","last_peered":"2024-09-10T10:50:29.180885+0000","last_clean":"2024-09-10T10:50:29.180885+0000","last_became_active":"2024-09-10T10:50:27.183408+0000","last_became_peered":"2024-09-10T10:50:27.183408+0000","last_unstale":"2024-09-10T10:50:29.180885+0000","last_undegraded":"2024-09-10T10:50:29.180885+0000","last_fullsized":"2024-09-10T10:50:29.180885+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:40:52.969316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182600+0000","last_change":"2024-09-10T10:50:28.183290+0000","last_active":"2024-09-10T10:50:29.182600+0000","last_peered":"2024-09-10T10:50:29.182600+0000","last_clean":"2024-09-10T10:50:29.182600+0000","last_became_active":"2024-09-10T10:50:28.182995+0000","last_became_peered":"2024-09-10T10:50:28.182995+0000","last_unstale":"2024-09-10T10:50:29.182600+0000","last_undegraded":"2024-09-10T10:50:29.182600+0000","last_fullsized":"2024-09-10T10:50:29.182600+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:20:04.379276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223738+0000","last_change":"2024-09-10T10:50:27.189598+0000","last_active":"2024-09-10T10:50:29.223738+0000","last_peered":"2024-09-10T10:50:29.223738+0000","last_clean":"2024-09-10T10:50:29.223738+0000","last_became_active":"2024-09-10T10:50:27.189244+0000","last_became_peered":"2024-09-10T10:50:27.189244+0000","last_unstale":"2024-09-10T10:50:29.223738+0000","last_undegraded":"2024-09-10T10:50:29.223738+0000","last_fullsized":"2024-09-10T10:50:29.223738+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:33.090606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223259+0000","last_change":"2024-09-10T10:50:28.199685+0000","last_active":"2024-09-10T10:50:29.223259+0000","last_peered":"2024-09-10T10:50:29.223259+0000","last_clean":"2024-09-10T10:50:29.223259+0000","last_became_active":"2024-09-10T10:50:28.189133+0000","last_became_peered":"2024-09-10T10:50:28.189133+0000","last_unstale":"2024-09-10T10:50:29.223259+0000","last_undegraded":"2024-09-10T10:50:29.223259+0000","last_fullsized":"2024-09-10T10:50:29.223259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:58.947518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223076+0000","last_change":"2024-09-10T10:50:27.185603+0000","last_active":"2024-09-10T10:50:29.223076+0000","last_peered":"2024-09-10T10:50:29.223076+0000","last_clean":"2024-09-10T10:50:29.223076+0000","last_became_active":"2024-09-10T10:50:27.183721+0000","last_became_peered":"2024-09-10T10:50:27.183721+0000","last_unstale":"2024-09-10T10:50:29.223076+0000","last_undegraded":"2024-09-10T10:50:29.223076+0000","last_fullsized":"2024-09-10T10:50:29.223076+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:12:47.354473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222614+0000","last_change":"2024-09-10T10:50:28.189172+0000","last_active":"2024-09-10T10:50:29.222614+0000","last_peered":"2024-09-10T10:50:29.222614+0000","last_clean":"2024-09-10T10:50:29.222614+0000","last_became_active":"2024-09-10T10:50:28.188906+0000","last_became_peered":"2024-09-10T10:50:28.188906+0000","last_unstale":"2024-09-10T10:50:29.222614+0000","last_undegraded":"2024-09-10T10:50:29.222614+0000","last_fullsized":"2024-09-10T10:50:29.222614+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:30.166347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222874+0000","last_change":"2024-09-10T10:50:27.184981+0000","last_active":"2024-09-10T10:50:29.222874+0000","last_peered":"2024-09-10T10:50:29.222874+0000","last_clean":"2024-09-10T10:50:29.222874+0000","last_became_active":"2024-09-10T10:50:27.184727+0000","last_became_peered":"2024-09-10T10:50:27.184727+0000","last_unstale":"2024-09-10T10:50:29.222874+0000","last_undegraded":"2024-09-10T10:50:29.222874+0000","last_fullsized":"2024-09-10T10:50:29.222874+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:15:57.944228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.182921+0000","last_change":"2024-09-10T10:50:28.188155+0000","last_active":"2024-09-10T10:50:29.182921+0000","last_peered":"2024-09-10T10:50:29.182921+0000","last_clean":"2024-09-10T10:50:29.182921+0000","last_became_active":"2024-09-10T10:50:28.187749+0000","last_became_peered":"2024-09-10T10:50:28.187749+0000","last_unstale":"2024-09-10T10:50:29.182921+0000","last_undegraded":"2024-09-10T10:50:29.182921+0000","last_fullsized":"2024-09-10T10:50:29.182921+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:03.781090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222550+0000","last_change":"2024-09-10T10:50:27.185390+0000","last_active":"2024-09-10T10:50:29.222550+0000","last_peered":"2024-09-10T10:50:29.222550+0000","last_clean":"2024-09-10T10:50:29.222550+0000","last_became_active":"2024-09-10T10:50:27.184135+0000","last_became_peered":"2024-09-10T10:50:27.184135+0000","last_unstale":"2024-09-10T10:50:29.222550+0000","last_undegraded":"2024-09-10T10:50:29.222550+0000","last_fullsized":"2024-09-10T10:50:29.222550+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:46:36.330432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223685+0000","last_change":"2024-09-10T10:50:28.186113+0000","last_active":"2024-09-10T10:50:29.223685+0000","last_peered":"2024-09-10T10:50:29.223685+0000","last_clean":"2024-09-10T10:50:29.223685+0000","last_became_active":"2024-09-10T10:50:28.185849+0000","last_became_peered":"2024-09-10T10:50:28.185849+0000","last_unstale":"2024-09-10T10:50:29.223685+0000","last_undegraded":"2024-09-10T10:50:29.223685+0000","last_fullsized":"2024-09-10T10:50:29.223685+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":97,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:27.150272+0000","last_clean_scrub_stamp":"2024-09-10T10:50:27.150272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:31.175994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.223976+0000","last_change":"2024-09-10T10:50:27.189324+0000","last_active":"2024-09-10T10:50:29.223976+0000","last_peered":"2024-09-10T10:50:29.223976+0000","last_clean":"2024-09-10T10:50:29.223976+0000","last_became_active":"2024-09-10T10:50:27.188864+0000","last_became_peered":"2024-09-10T10:50:27.188864+0000","last_unstale":"2024-09-10T10:50:29.223976+0000","last_undegraded":"2024-09-10T10:50:29.223976+0000","last_fullsized":"2024-09-10T10:50:29.223976+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:50:26.146982+0000","last_clean_scrub_stamp":"2024-09-10T10:50:26.146982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:45:04.117222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":315,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-10T10:50:29.222637+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:50:29.222637+0000","last_peered":"2024-09-10T10:50:29.222637+0000","last_clean":"2024-09-10T10:50:29.222637+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:50:29.222637+0000","last_undegraded":"2024-09-10T10:50:29.222637+0000","last_fullsized":"2024-09-10T10:50:29.222637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1212416,"data_stored":562990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1232896,"data_stored":564726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":562990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1232896,"data_stored":564714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509527,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1220608,"data_stored":563594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32972,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338868,"statfs":{"total":96636764160,"available":96603000832,"internally_reserved":0,"allocated":1818624,"data_stored":1154044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1216512,"data_stored":563060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1794048,"data_stored":1146666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:50:32.818 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:50:32.818 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:50:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.016+0000 7fde00e00640 1 Processor -- start 2024-09-10T10:50:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.016+0000 7fde00e00640 1 -- start start 2024-09-10T10:50:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.017+0000 7fde00e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fddfc0d4360 0x7fddfc162990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.017+0000 7fde00e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddfc0d2b50 con 0x7fddfc06dc80 2024-09-10T10:50:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.017+0000 7fde00e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddfc0d2e30 con 0x7fddfc06e6b0 2024-09-10T10:50:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.017+0000 7fde00e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fddfc0b3af0 con 0x7fddfc0d4360 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfbe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fddfc06dc80 0x7fddfc0ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51132/0 (socket says 172.21.15.28:51132) 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfbe00640 1 -- 172.21.15.28:0/3068328095 learned_addr learned my addr 172.21.15.28:0/3068328095 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfb400640 1 --2- 172.21.15.28:0/3068328095 >> v2:172.21.15.138:3300/0 conn(0x7fddfc0d4360 0x7fddfc162990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462765462 0 0) 0x7fddfc0d2b50 con 0x7fddfc06dc80 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdde0003520 con 0x7fddfc06dc80 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 644725786 0 0) 0x7fdde0003520 con 0x7fddfc06dc80 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 >> v2:172.21.15.138:3300/0 conn(0x7fddfc0d4360 msgr2=0x7fddfc162990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 --2- 172.21.15.28:0/3068328095 >> v2:172.21.15.138:3300/0 conn(0x7fddfc0d4360 0x7fddfc162990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 >> v1:172.21.15.138:6789/0 conn(0x7fddfc06e6b0 legacy=0x7fddfc0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc06d1a0 con 0x7fddfc06dc80 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fddf40019f0 con 0x7fddfc06dc80 2024-09-10T10:50:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 >> v1:172.21.15.28:6789/0 conn(0x7fddfc06dc80 legacy=0x7fddfc0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde00039b0 0x7fdde0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.018+0000 7fddfa000640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fdde0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfa000640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde00051c0 0x7fdde000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc0b3af0 con 0x7fdde0004800 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000c070 con 0x7fdde00039b0 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdde000c270 con 0x7fdde00051c0 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fdde0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfb400640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde00039b0 0x7fdde0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfbe00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde00051c0 0x7fdde000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 -- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde00051c0 msgr2=0x7fdde000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdde00051c0 0x7fdde000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 -- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde00039b0 msgr2=0x7fdde0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:33.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde00039b0 0x7fdde0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfaa00640 1 -- 172.21.15.28:0/3068328095 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde0003520 con 0x7fdde0004800 2024-09-10T10:50:33.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.019+0000 7fddfbe00640 1 -- 172.21.15.28:0/3068328095 reap_dead start 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.020+0000 7fddfaa00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fdde0004c80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdde8001750 tx=0x7fdde0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef63c2b8bf7e7a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.020+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde8001370 con 0x7fdde0004800 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde80052d0 con 0x7fdde0004800 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fddfa000640 1 -- 172.21.15.28:0/3068328095 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde802abc0 con 0x7fdde0004800 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 -- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 msgr2=0x7fdde0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fdde0004c80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdde8001750 tx=0x7fdde0005830 comp rx=0 tx=0).stop 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 -- 172.21.15.28:0/3068328095 shutdown_connections 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 --2- 172.21.15.28:0/3068328095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fdde0004c80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 -- 172.21.15.28:0/3068328095 >> 172.21.15.28:0/3068328095 conn(0x7fddfc09d740 msgr2=0x7fddfc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 -- 172.21.15.28:0/3068328095 shutdown_connections 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.021+0000 7fde00e00640 1 -- 172.21.15.28:0/3068328095 wait complete. 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.023+0000 7fde00e00640 1 Processor -- start 2024-09-10T10:50:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.023+0000 7fde00e00640 1 -- start start 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddfc153fe0 0x7fddfc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc0cbe10 0x7fddfc0cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc15e6d0 con 0x7fdde0004800 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc06d760 con 0x7fddfc0cbe10 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fde00e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc15e850 con 0x7fddfc153fe0 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37546/0 (socket says 172.21.15.28:37546) 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 -- 172.21.15.28:0/2719775810 learned_addr learned my addr 172.21.15.28:0/2719775810 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 -- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddfc153fe0 msgr2=0x7fddfc153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddfc153fe0 0x7fddfc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 -- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc0cbe10 msgr2=0x7fddfc0cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc0cbe10 0x7fddfc0cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.024+0000 7fddfb400640 1 -- 172.21.15.28:0/2719775810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc0cb9e0 con 0x7fdde0004800 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fddfb400640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fddec011890 tx=0x7fddec011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec019070 con 0x7fdde0004800 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec004240 con 0x7fdde0004800 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec00f7a0 con 0x7fdde0004800 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddfc160e50 con 0x7fdde0004800 2024-09-10T10:50:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.025+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddfc161410 con 0x7fdde0004800 2024-09-10T10:50:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.026+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddfc06e110 con 0x7fdde0004800 2024-09-10T10:50:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.027+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddec01c070 con 0x7fdde0004800 2024-09-10T10:50:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.028+0000 7fddf3400640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 0x7fdddc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.028+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fddec08b460 con 0x7fdde0004800 2024-09-10T10:50:33.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.028+0000 7fddfaa00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 0x7fdddc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.029+0000 7fddfaa00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 0x7fdddc06e320 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdde8032000 tx=0x7fdde802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.033+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fddec059bf0 con 0x7fdde0004800 2024-09-10T10:50:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.285+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fddfc0d1900 con 0x7fdde0004800 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.288+0000 7fddf3400640 1 -- 172.21.15.28:0/2719775810 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fddec0594f0 con 0x7fdde0004800 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:50:33.289 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:50:33.290 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:50:33.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 msgr2=0x7fdddc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 0x7fdddc06e320 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdde8032000 tx=0x7fdde802c000 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 msgr2=0x7fddfc0cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fddec011890 tx=0x7fddec011d50 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 shutdown_connections 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdddc06be60 0x7fdddc06e320 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddfc0cbe10 0x7fddfc0cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddfc153fe0 0x7fddfc153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 --2- 172.21.15.28:0/2719775810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdde0004800 0x7fddfc0cb4a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 >> 172.21.15.28:0/2719775810 conn(0x7fddfc09d740 msgr2=0x7fddfc154650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 shutdown_connections 2024-09-10T10:50:33.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.290+0000 7fde00e00640 1 -- 172.21.15.28:0/2719775810 wait complete. 2024-09-10T10:50:33.308 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:33.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.463+0000 7f40b8a00640 1 Processor -- start 2024-09-10T10:50:33.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.463+0000 7f40b8a00640 1 -- start start 2024-09-10T10:50:33.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.464+0000 7f40b8a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f40b4161c80 0x7f40b406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.464+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.464+0000 7f40b8a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40b406e870 con 0x7f40b40b64b0 2024-09-10T10:50:33.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.464+0000 7f40b8a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f40b406d9f0 con 0x7f40b4161c80 2024-09-10T10:50:33.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.464+0000 7f40b8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f40b406dcd0 con 0x7f40b4162060 2024-09-10T10:50:33.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:55438/0 (socket says 172.21.15.28:55438) 2024-09-10T10:50:33.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 -- 172.21.15.28:0/3862431460 learned_addr learned my addr 172.21.15.28:0/3862431460 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1e00640 1 --2- 172.21.15.28:0/3862431460 >> v2:172.21.15.138:3300/0 conn(0x7f40b4161c80 0x7f40b406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b2800640 1 --1- 172.21.15.28:0/3862431460 >> v1:172.21.15.138:6789/0 conn(0x7f40b40b64b0 0x7f40b4162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:52722/0 (socket says 172.21.15.28:52722) 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 -- 172.21.15.28:0/3862431460 >> v2:172.21.15.138:3300/0 conn(0x7f40b4161c80 msgr2=0x7f40b406e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 --2- 172.21.15.28:0/3862431460 >> v2:172.21.15.138:3300/0 conn(0x7f40b4161c80 0x7f40b406e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 -- 172.21.15.28:0/3862431460 >> v1:172.21.15.138:6789/0 conn(0x7f40b40b64b0 legacy=0x7f40b4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.465+0000 7f40b1400640 1 -- 172.21.15.28:0/3862431460 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b406dfb0 con 0x7f40b4162060 2024-09-10T10:50:33.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b1400640 1 --2- 172.21.15.28:0/3862431460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d4642d9eaaf4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b0a00640 1 -- 172.21.15.28:0/3862431460 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0037070 con 0x7f40b4162060 2024-09-10T10:50:33.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b0a00640 1 -- 172.21.15.28:0/3862431460 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a0032440 con 0x7f40b4162060 2024-09-10T10:50:33.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b0a00640 1 -- 172.21.15.28:0/3862431460 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0032810 con 0x7f40b4162060 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b8a00640 1 -- 172.21.15.28:0/3862431460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 msgr2=0x7f40b40b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.467+0000 7f40b8a00640 1 --2- 172.21.15.28:0/3862431460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009870 tx=0x7f40a002a270 comp rx=0 tx=0).stop 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 -- 172.21.15.28:0/3862431460 shutdown_connections 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 --2- 172.21.15.28:0/3862431460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b5f70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 --2- 172.21.15.28:0/3862431460 >> v2:172.21.15.138:3300/0 conn(0x7f40b4161c80 0x7f40b406e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 -- 172.21.15.28:0/3862431460 >> 172.21.15.28:0/3862431460 conn(0x7f40b40d44f0 msgr2=0x7f40b40d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:33.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 -- 172.21.15.28:0/3862431460 shutdown_connections 2024-09-10T10:50:33.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.468+0000 7f40b8a00640 1 -- 172.21.15.28:0/3862431460 wait complete. 2024-09-10T10:50:33.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.469+0000 7f40b8a00640 1 Processor -- start 2024-09-10T10:50:33.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.469+0000 7f40b8a00640 1 -- start start 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 0x7f40b415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d2a10 con 0x7f40b4161c80 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40b40d2890 con 0x7f40b40b64b0 2024-09-10T10:50:33.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.470+0000 7f40b8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f40b40b6960 con 0x7f40b4162060 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 0x7f40b415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37560/0 (socket says 172.21.15.28:37560) 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 -- 172.21.15.28:0/1122525416 learned_addr learned my addr 172.21.15.28:0/1122525416 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b2800640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 -- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 msgr2=0x7f40b40b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 -- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 msgr2=0x7f40b415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 0x7f40b415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1400640 1 -- 172.21.15.28:0/1122525416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b41591d0 con 0x7f40b4161c80 2024-09-10T10:50:33.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.471+0000 7f40b1e00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 0x7f40b415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:33.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40b1400640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f40a002a2b0 tx=0x7f40a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0041030 con 0x7f40b4161c80 2024-09-10T10:50:33.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40a0030440 con 0x7f40b4161c80 2024-09-10T10:50:33.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0037040 con 0x7f40b4161c80 2024-09-10T10:50:33.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b41594b0 con 0x7f40b4161c80 2024-09-10T10:50:33.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.472+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b4159a70 con 0x7f40b4161c80 2024-09-10T10:50:33.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.473+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4068ea0 con 0x7f40b4161c80 2024-09-10T10:50:33.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.474+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40a0044070 con 0x7f40b4161c80 2024-09-10T10:50:33.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.475+0000 7f40aa000640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 0x7f409406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:33.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.475+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f40a00ab230 con 0x7f40b4161c80 2024-09-10T10:50:33.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.475+0000 7f40b1e00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 0x7f409406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:33.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.476+0000 7f40b1e00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 0x7f409406e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40b40cb160 tx=0x7f40a4009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:33.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.479+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40b4068ea0 con 0x7f40b4161c80 2024-09-10T10:50:33.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.698+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b406e330 con 0x7f40b4161c80 2024-09-10T10:50:33.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.698+0000 7f40aa000640 1 -- 172.21.15.28:0/1122525416 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5225 (secure 0 0 0) 0x7f40a004b070 con 0x7f40b4161c80 2024-09-10T10:50:33.699 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:33.699 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":83,"btime":"2024-09-10T10:50:31:186469+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:30.609334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6849":{"gid":6849,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/3446889436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3446889436},{"type":"v1","addr":"172.21.15.28:6835","nonce":3446889436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6849,"qdb_cluster":[6849]},"id":6}]} 2024-09-10T10:50:33.700 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 83 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 msgr2=0x7f409406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 0x7f409406e320 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40b40cb160 tx=0x7f40a4009190 comp rx=0 tx=0).stop 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 msgr2=0x7f40b415f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f40a002a2b0 tx=0x7f40a00046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 shutdown_connections 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f409406be60 0x7f409406e320 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40b4162060 0x7f40b40b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40b4161c80 0x7f40b415f1f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 --2- 172.21.15.28:0/1122525416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40b40b64b0 0x7f40b415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 >> 172.21.15.28:0/1122525416 conn(0x7f40b40d44f0 msgr2=0x7f40b406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 shutdown_connections 2024-09-10T10:50:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:33.701+0000 7f40b8a00640 1 -- 172.21.15.28:0/1122525416 wait complete. 2024-09-10T10:50:33.759 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:50:33.758+0000 7fb7d1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:34.120 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:50:30.609334+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6849}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6849': {'gid': 6849, 'name': 'a', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6835/3446889436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 3446889436}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 3446889436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6849, 'qdb_cluster': [6849]} 2024-09-10T10:50:34.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:50:34.121 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:50:34.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.280+0000 7f1a9f800640 1 Processor -- start 2024-09-10T10:50:34.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.280+0000 7f1a9f800640 1 -- start start 2024-09-10T10:50:34.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.281+0000 7f1a9f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a98072de0 0x7f1a9815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.281+0000 7f1a9f800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a98072700 0x7f1a9815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.281+0000 7f1a9f800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a98146140 con 0x7f1a98072320 2024-09-10T10:50:34.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.281+0000 7f1a9f800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1a980cd740 con 0x7f1a98072700 2024-09-10T10:50:34.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.281+0000 7f1a9f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a980cda20 con 0x7f1a98072de0 2024-09-10T10:50:34.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a9ce00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1a98072320 0x7f1a980cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56182/0 (socket says 172.21.15.28:56182) 2024-09-10T10:50:34.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a9ce00640 1 -- 172.21.15.28:0/976245123 learned_addr learned my addr 172.21.15.28:0/976245123 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:34.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97e00640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a98072de0 0x7f1a9815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a9d800640 1 --2- 172.21.15.28:0/976245123 >> v2:172.21.15.138:3300/0 conn(0x7f1a98072700 0x7f1a9815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658232438 0 0) 0x7f1a98146140 con 0x7f1a98072320 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a80003520 con 0x7f1a98072320 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3706439165 0 0) 0x7f1a80003520 con 0x7f1a98072320 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a98072de0 msgr2=0x7f1a9815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a98072de0 0x7f1a9815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 >> v2:172.21.15.138:3300/0 conn(0x7f1a98072700 msgr2=0x7f1a9815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.282+0000 7f1a97400640 1 --2- 172.21.15.28:0/976245123 >> v2:172.21.15.138:3300/0 conn(0x7f1a98072700 0x7f1a9815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a980cdd00 con 0x7f1a98072320 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f1a8c001670 con 0x7f1a98072320 2024-09-10T10:50:34.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 >> v1:172.21.15.28:6789/0 conn(0x7f1a98072320 legacy=0x7f1a980cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a800043a0 0x7f1a80004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980cd740 con 0x7f1a80004f90 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980cda20 con 0x7f1a800043a0 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8000bf30 con 0x7f1a800039b0 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97e00640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a98072de0 0x7f1a9815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a97e00640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a800043a0 0x7f1a80004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 -- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a800039b0 msgr2=0x7f1a80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a800039b0 0x7f1a80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 -- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a800043a0 msgr2=0x7f1a80004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a800043a0 0x7f1a80004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.283+0000 7f1a9d800640 1 -- 172.21.15.28:0/976245123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80003520 con 0x7f1a80004f90 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.284+0000 7f1a9d800640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a8000b9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009fc0 tx=0x7f1a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc2b6f7b1ecefa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90009c10 con 0x7f1a80004f90 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a90036070 con 0x7f1a80004f90 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a97400640 1 -- 172.21.15.28:0/976245123 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9002d070 con 0x7f1a80004f90 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 -- 172.21.15.28:0/976245123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 msgr2=0x7f1a8000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 --2- 172.21.15.28:0/976245123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a8000b9c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009fc0 tx=0x7f1a80005830 comp rx=0 tx=0).stop 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9d800640 1 -- 172.21.15.28:0/976245123 reap_dead start 2024-09-10T10:50:34.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 -- 172.21.15.28:0/976245123 shutdown_connections 2024-09-10T10:50:34.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 -- 172.21.15.28:0/976245123 >> 172.21.15.28:0/976245123 conn(0x7f1a98067d70 msgr2=0x7f1a9806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 -- 172.21.15.28:0/976245123 shutdown_connections 2024-09-10T10:50:34.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.285+0000 7f1a9f800640 1 -- 172.21.15.28:0/976245123 wait complete. 2024-09-10T10:50:34.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.287+0000 7f1a9f800640 1 Processor -- start 2024-09-10T10:50:34.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.287+0000 7f1a9f800640 1 -- start start 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a98136e20 0x7f1a981372a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a981e5460 0x7f1a981e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9815d360 con 0x7f1a80004f90 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9815d1e0 con 0x7f1a98136e20 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9815d4e0 con 0x7f1a981e5460 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44130/0 (socket says 172.21.15.28:44130) 2024-09-10T10:50:34.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9ce00640 1 -- 172.21.15.28:0/1023184296 learned_addr learned my addr 172.21.15.28:0/1023184296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a9d800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a981e5460 0x7f1a981e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.288+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a98136e20 0x7f1a981372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 -- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a981e5460 msgr2=0x7f1a981e58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a981e5460 0x7f1a981e58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 -- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a98136e20 msgr2=0x7f1a981372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a98136e20 0x7f1a981372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 -- 172.21.15.28:0/1023184296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a981378e0 con 0x7f1a80004f90 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c007340 tx=0x7f1a8c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a94c00640 1 -- 172.21.15.28:0/1023184296 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00f940 con 0x7f1a80004f90 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a94c00640 1 -- 172.21.15.28:0/1023184296 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c018070 con 0x7f1a80004f90 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a98137bc0 con 0x7f1a80004f90 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.289+0000 7f1a94c00640 1 -- 172.21.15.28:0/1023184296 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c00fd00 con 0x7f1a80004f90 2024-09-10T10:50:34.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.290+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a98144020 con 0x7f1a80004f90 2024-09-10T10:50:34.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.292+0000 7f1a94c00640 1 -- 172.21.15.28:0/1023184296 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a8c01d460 con 0x7f1a80004f90 2024-09-10T10:50:34.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.292+0000 7f1a94c00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.293+0000 7f1a94c00640 1 -- 172.21.15.28:0/1023184296 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f1a8c08b020 con 0x7f1a80004f90 2024-09-10T10:50:34.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.293+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.293+0000 7f1aa2668740 1 Processor -- start 2024-09-10T10:50:34.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.293+0000 7f1aa2668740 1 -- start start 2024-09-10T10:50:34.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 0x7f1a7806e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a981f7330 tx=0x7f1a84009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196a84b0 0x55ad196ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ad196093b0 con 0x55ad196aacf0 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad1960e410 con 0x55ad196a84b0 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ad195b6620 con 0x55ad196ac480 2024-09-10T10:50:34.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a9ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44146/0 (socket says 172.21.15.28:44146) 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a9ce00640 1 -- 172.21.15.28:0/2228340261 learned_addr learned my addr 172.21.15.28:0/2228340261 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a97e00640 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196a84b0 0x55ad196ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.294+0000 7f1a9d800640 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 -- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 msgr2=0x55ad196a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 -- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196a84b0 msgr2=0x55ad196ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196a84b0 0x55ad196ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 -- 172.21.15.28:0/2228340261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad193212d0 con 0x55ad196aacf0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c089300 tx=0x7f1a8c0591f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e969973083830c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a89600640 1 -- 172.21.15.28:0/2228340261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c071180 con 0x55ad196aacf0 2024-09-10T10:50:34.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a89600640 1 -- 172.21.15.28:0/2228340261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c061d20 con 0x55ad196aacf0 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.295+0000 7f1a89600640 1 -- 172.21.15.28:0/2228340261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c05d160 con 0x55ad196aacf0 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 -- 172.21.15.28:0/2228340261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 msgr2=0x55ad196f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c089300 tx=0x7f1a8c0591f0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 -- 172.21.15.28:0/2228340261 shutdown_connections 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196a84b0 0x55ad196ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2228340261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196aacf0 0x55ad196f0490 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 -- 172.21.15.28:0/2228340261 >> 172.21.15.28:0/2228340261 conn(0x55ad196a7740 msgr2=0x55ad196ef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 -- 172.21.15.28:0/2228340261 shutdown_connections 2024-09-10T10:50:34.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.296+0000 7f1aa2668740 1 -- 172.21.15.28:0/2228340261 wait complete. 2024-09-10T10:50:34.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.297+0000 7f1aa2668740 1 -- start start 2024-09-10T10:50:34.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.297+0000 7f1aa2668740 1 Processor -- start 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196ac480 0x55ad196d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ad195c9780 con 0x55ad196cf670 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad1960a3d0 con 0x55ad196ac480 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ad196a6be0 con 0x55ad196aacf0 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1aa2668740 10 client.0 _ll_register_callbacks cb 0x55ad196075d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44152/0 (socket says 172.21.15.28:44152) 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1a9d800640 1 -- 172.21.15.28:0/3449403062 learned_addr learned my addr 172.21.15.28:0/3449403062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196ac480 0x55ad196d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.298+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 -- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 msgr2=0x55ad196d53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196d53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 -- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196ac480 msgr2=0x55ad196d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196ac480 0x55ad196d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 -- 172.21.15.28:0/3449403062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad193eb6b0 con 0x55ad196cf670 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a9d800640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a9002af10 tx=0x7f1a90004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9622c919b2f8d510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a7e000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90003fa0 con 0x55ad196cf670 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f1a90004140 con 0x55ad196cf670 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a8c061d20 con 0x55ad196cf670 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1a9004f050 con 0x55ad196cf670 2024-09-10T10:50:34.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1aa2668740 10 client.6963 fetch_fsmap learned FSMap version 83 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.299+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55ad196d2f00 con 0x55ad196cf670 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c061d20 con 0x55ad196cf670 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 83) ==== 3502+0+0 (secure 0 0 0) 0x7f1a90052020 con 0x55ad196cf670 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 10 client.6963 fetch_fsmap finished waiting for FSMap version 83 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 10 client.6963 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 20 client.6963 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196cfab0 0x55ad197034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 4 client.6963 mds_command: new command op to 6849 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 --> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad1961bb40 con 0x55ad196cfab0 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196cfab0 0x55ad197034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196cfab0 0x55ad197034b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c767e699cf2908e server_cookie=a07ef2e4daffd55e in_seq=0 out_seq=0 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.300+0000 7f1a7e000640 10 client.6963 ms_handle_connect on v2:172.21.15.28:6833/3446889436 2024-09-10T10:50:34.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.301+0000 7f1a7e000640 1 -- 172.21.15.28:0/3449403062 <== mds.0 v2:172.21.15.28:6833/3446889436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad1961bb40 con 0x55ad196cfab0 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.301+0000 7f1a7e000640 10 client.6963 handle_command_reply: tid=0 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.301+0000 7f1aa2668740 1 client.6963 shutdown 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.301+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 msgr2=0x55ad196eadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.301+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1a9002af10 tx=0x7f1a90004ab0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 shutdown_connections 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196cfab0 0x55ad197034b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196eadf0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196ac480 0x55ad196d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 --2- 172.21.15.28:0/3449403062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196d53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1a7e000640 0 client.6963 ms_handle_reset on v2:172.21.15.28:6833/3446889436 2024-09-10T10:50:34.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 >> 172.21.15.28:0/3449403062 conn(0x55ad196a84b0 msgr2=0x55ad196eeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 shutdown_connections 2024-09-10T10:50:34.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 1 -- 172.21.15.28:0/3449403062 wait complete. 2024-09-10T10:50:34.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.302+0000 7f1aa2668740 20 client.6963 trim_cache size 0 max 16384 2024-09-10T10:50:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 Processor -- start 2024-09-10T10:50:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 -- start start 2024-09-10T10:50:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196ff810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 0x55ad196d36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98200610 con 0x55ad196ac480 2024-09-10T10:50:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a981ff9a0 con 0x55ad196cf670 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.321+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a981ffff0 con 0x55ad196aacf0 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44160/0 (socket says 172.21.15.28:44160) 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 -- 172.21.15.28:0/1618788043 learned_addr learned my addr 172.21.15.28:0/1618788043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a9d800640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 0x55ad196d36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 -- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 msgr2=0x55ad196ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 -- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 msgr2=0x55ad196d36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 0x55ad196d36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 -- 172.21.15.28:0/1618788043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad196ecc20 con 0x55ad196ac480 2024-09-10T10:50:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:34.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a9d800640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 0x55ad196d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:34.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.322+0000 7f1a97e00640 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1a84021610 tx=0x7f1a84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc14f43c571784d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1a88c00640 1 -- 172.21.15.28:0/1618788043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8401b070 con 0x55ad196ac480 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1a88c00640 1 -- 172.21.15.28:0/1618788043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a84031070 con 0x55ad196ac480 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1a88c00640 1 -- 172.21.15.28:0/1618788043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402c660 con 0x55ad196ac480 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 -- 172.21.15.28:0/1618788043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 msgr2=0x55ad196d3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1a84021610 tx=0x7f1a84017000 comp rx=0 tx=0).stop 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 -- 172.21.15.28:0/1618788043 shutdown_connections 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196cf670 0x55ad196d36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196ac480 0x55ad196d3180 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 --2- 172.21.15.28:0/1618788043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196aacf0 0x55ad196ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 -- 172.21.15.28:0/1618788043 >> 172.21.15.28:0/1618788043 conn(0x55ad196a84b0 msgr2=0x55ad196cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 -- 172.21.15.28:0/1618788043 shutdown_connections 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.323+0000 7f1aa2668740 1 -- 172.21.15.28:0/1618788043 wait complete. 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.324+0000 7f1aa2668740 1 -- start start 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.324+0000 7f1aa2668740 1 Processor -- start 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196aacf0 0x55ad1976da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ad196eb560 con 0x55ad196cf670 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad196d01f0 con 0x55ad196aacf0 2024-09-10T10:50:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ad196ec9d0 con 0x55ad196ac480 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1aa2668740 10 client.0 _ll_register_callbacks cb 0x7f1a84009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1a9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44162/0 (socket says 172.21.15.28:44162) 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1a9d800640 1 -- 172.21.15.28:0/2663801180 learned_addr learned my addr 172.21.15.28:0/2663801180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.325+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196aacf0 0x55ad1976da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a97e00640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 -- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 msgr2=0x55ad196cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 -- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196aacf0 msgr2=0x55ad1976da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196aacf0 0x55ad1976da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 -- 172.21.15.28:0/2663801180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad19700060 con 0x55ad196cf670 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a9d800640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009fd0 tx=0x7f1a9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c00e4484d13d74e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a7e000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90059060 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f1a90047630 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.326+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a84031070 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1a9005f050 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 10 client.6969 fetch_fsmap learned FSMap version 83 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x55ad19709900 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a84031070 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 83) ==== 3502+0+0 (secure 0 0 0) 0x7f1a90062020 con 0x55ad196cf670 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 10 client.6969 fetch_fsmap finished waiting for FSMap version 83 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 10 client.6969 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:50:34.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 20 client.6969 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196f0140 0x55ad196e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 4 client.6969 mds_command: new command op to 6849 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 --> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] -- command(tid 0: {"prefix": "status"}) -- 0x55ad196d67e0 con 0x55ad196f0140 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.327+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196f0140 0x55ad196e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1a7e000640 10 client.6969 ms_handle_connect on v2:172.21.15.28:6833/3446889436 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1a9ce00640 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196f0140 0x55ad196e8b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d406bfaf3e0da135 server_cookie=2449217ce9470c01 in_seq=0 out_seq=0 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1a7e000640 1 -- 172.21.15.28:0/2663801180 <== mds.0 v2:172.21.15.28:6833/3446889436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55ad196d67e0 con 0x55ad196f0140 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1a7e000640 10 client.6969 handle_command_reply: tid=0 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 client.6969 shutdown 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 msgr2=0x55ad196cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009fd0 tx=0x7f1a9000b040 comp rx=0 tx=0).stop 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 shutdown_connections 2024-09-10T10:50:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:6833/3446889436,v1:172.21.15.28:6835/3446889436] conn(0x55ad196f0140 0x55ad196e8b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad196cf670 0x55ad196cbcb0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad196ac480 0x55ad196cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 --2- 172.21.15.28:0/2663801180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad196aacf0 0x55ad1976da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1a7e000640 0 client.6969 ms_handle_reset on v2:172.21.15.28:6833/3446889436 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.328+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 >> 172.21.15.28:0/2663801180 conn(0x55ad196a84b0 msgr2=0x55ad196c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 shutdown_connections 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1aa2668740 1 -- 172.21.15.28:0/2663801180 wait complete. 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1aa2668740 20 client.6969 trim_cache size 0 max 16384 2024-09-10T10:50:34.334 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6849, 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:50:34.335 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.1657665120000003, 2024-09-10T10:50:34.336 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 83, 2024-09-10T10:50:34.336 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 99, 2024-09-10T10:50:34.336 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 99, 2024-09-10T10:50:34.336 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 11.227946563 2024-09-10T10:50:34.336 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 msgr2=0x7f1a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 0x7f1a7806e2f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1a981f7330 tx=0x7f1a84009190 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 msgr2=0x7f1a9815bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c007340 tx=0x7f1a8c007800 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 shutdown_connections 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806be30 0x7f1a7806e2f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a981e5460 0x7f1a981e58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a98136e20 0x7f1a981372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 --2- 172.21.15.28:0/1023184296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a80004f90 0x7f1a9815bce0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.329+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 >> 172.21.15.28:0/1023184296 conn(0x7f1a98067d70 msgr2=0x7f1a9806f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.330+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 shutdown_connections 2024-09-10T10:50:34.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.330+0000 7f1a9f800640 1 -- 172.21.15.28:0/1023184296 wait complete. 2024-09-10T10:50:34.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:50:34.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.501+0000 7f6c7b000640 1 Processor -- start 2024-09-10T10:50:34.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.501+0000 7f6c7b000640 1 -- start start 2024-09-10T10:50:34.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c7b000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6c74162ad0 0x7f6c740cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c7b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c7406f6f0 0x7f6c740b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c7b000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c740afa20 con 0x7f6c740720b0 2024-09-10T10:50:34.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c7b000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6c740afce0 con 0x7f6c74162ad0 2024-09-10T10:50:34.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c7b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c740b34d0 con 0x7f6c7406f6f0 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6c74162ad0 0x7f6c740cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6c74162ad0 0x7f6c740cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44166/0 (socket says 172.21.15.28:44166) 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c79000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c7406f6f0 0x7f6c740b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 -- 172.21.15.28:0/577449207 learned_addr learned my addr 172.21.15.28:0/577449207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 -- 172.21.15.28:0/577449207 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406f6f0 msgr2=0x7f6c740b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 --2- 172.21.15.28:0/577449207 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406f6f0 0x7f6c740b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.503+0000 7f6c6b400640 1 -- 172.21.15.28:0/577449207 >> v1:172.21.15.138:6789/0 conn(0x7f6c740720b0 legacy=0x7f6c74072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.504+0000 7f6c6b400640 1 -- 172.21.15.28:0/577449207 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c740b37b0 con 0x7f6c74162ad0 2024-09-10T10:50:34.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.504+0000 7f6c6b400640 1 --2- 172.21.15.28:0/577449207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c74162ad0 0x7f6c740cfca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c740affc0 tx=0x7f6c5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2fa07d90e54b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c73400640 1 -- 172.21.15.28:0/577449207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c037070 con 0x7f6c74162ad0 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c73400640 1 -- 172.21.15.28:0/577449207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c5c032440 con 0x7f6c74162ad0 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c73400640 1 -- 172.21.15.28:0/577449207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c032810 con 0x7f6c74162ad0 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 -- 172.21.15.28:0/577449207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c74162ad0 msgr2=0x7f6c740cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 --2- 172.21.15.28:0/577449207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c74162ad0 0x7f6c740cfca0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f6c740affc0 tx=0x7f6c5c02a270 comp rx=0 tx=0).stop 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 -- 172.21.15.28:0/577449207 shutdown_connections 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 --2- 172.21.15.28:0/577449207 >> v2:172.21.15.138:3300/0 conn(0x7f6c7406f6f0 0x7f6c740b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 --2- 172.21.15.28:0/577449207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c74162ad0 0x7f6c740cfca0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.505+0000 7f6c7b000640 1 -- 172.21.15.28:0/577449207 >> 172.21.15.28:0/577449207 conn(0x7f6c74067d70 msgr2=0x7f6c7406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.506+0000 7f6c7b000640 1 -- 172.21.15.28:0/577449207 shutdown_connections 2024-09-10T10:50:34.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.506+0000 7f6c7b000640 1 -- 172.21.15.28:0/577449207 wait complete. 2024-09-10T10:50:34.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.507+0000 7f6c7b000640 1 Processor -- start 2024-09-10T10:50:34.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.507+0000 7f6c7b000640 1 -- start start 2024-09-10T10:50:34.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c740720b0 0x7f6c7415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 0x7f6c7415bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740b47d0 con 0x7f6c7406f6f0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740b3d70 con 0x7f6c74162ad0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c7b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c740b4950 con 0x7f6c740720b0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44170/0 (socket says 172.21.15.28:44170) 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.508+0000 7f6c73e00640 1 -- 172.21.15.28:0/3329562196 learned_addr learned my addr 172.21.15.28:0/3329562196 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c6b400640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c740720b0 0x7f6c7415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c79000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 0x7f6c7415bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 -- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c740720b0 msgr2=0x7f6c7415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c740720b0 0x7f6c7415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 -- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 msgr2=0x7f6c7415bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 0x7f6c7415bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 -- 172.21.15.28:0/3329562196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7415c870 con 0x7f6c7406f6f0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c79000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 0x7f6c7415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c73e00640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c430 tx=0x7f6c6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640052d0 con 0x7f6c7406f6f0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c64005490 con 0x7f6c7406f6f0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64007260 con 0x7f6c7406f6f0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.509+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7415cb50 con 0x7f6c7406f6f0 2024-09-10T10:50:34.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.510+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7415d110 con 0x7f6c7406f6f0 2024-09-10T10:50:34.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.510+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c740b4290 con 0x7f6c7406f6f0 2024-09-10T10:50:34.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.511+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c64007400 con 0x7f6c7406f6f0 2024-09-10T10:50:34.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.512+0000 7f6c70c00640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 0x7f6c5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:34.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.512+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f6c6408a470 con 0x7f6c7406f6f0 2024-09-10T10:50:34.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.513+0000 7f6c6b400640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 0x7f6c5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:34.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.513+0000 7f6c6b400640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 0x7f6c5006e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c034000 tx=0x7f6c5c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:34.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.516+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c64058630 con 0x7f6c7406f6f0 2024-09-10T10:50:34.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.734+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6c740af700 con 0x7f6c7406f6f0 2024-09-10T10:50:34.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.734+0000 7f6c70c00640 1 -- 172.21.15.28:0/3329562196 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f6c6405c930 con 0x7f6c7406f6f0 2024-09-10T10:50:34.736 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.736+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 msgr2=0x7f6c5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.736+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 0x7f6c5006e2f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c034000 tx=0x7f6c5c02e000 comp rx=0 tx=0).stop 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.736+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 msgr2=0x7f6c7414d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.736+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400c430 tx=0x7f6c6400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 shutdown_connections 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c5006be30 0x7f6c5006e2f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c74162ad0 0x7f6c7415bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c740720b0 0x7f6c7415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 --2- 172.21.15.28:0/3329562196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c7406f6f0 0x7f6c7414d5d0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:34.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 >> 172.21.15.28:0/3329562196 conn(0x7f6c74067d70 msgr2=0x7f6c7406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:34.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 shutdown_connections 2024-09-10T10:50:34.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:34.737+0000 7f6c7b000640 1 -- 172.21.15.28:0/3329562196 wait complete. 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:50:34.754 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:50:34.755 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:34.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:34.755 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:50:34.791 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:50:34.791 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:50:34.791 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:50:34.791 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:50:34.792 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42270sec preferred_lft 42270sec 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.793 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:34.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:34.794 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:50:34.888 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:50:34.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:34.891 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:50:34.927 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:50:34.927 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:50:34.927 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:50:34.928 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:50:34.928 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:50:34.928 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:50:35.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:50:35.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:35.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.126 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:50:35.160 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:50:35.161 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:50:35.161 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:35.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:50:35.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:50:35.259 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:35.293 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:35.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:50:35.294 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:35.368 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:50:35.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:35.496 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:35.497 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:35.497 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:35.512 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:35.512 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:35.512 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:35.512 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:50:35.512 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:50:35.537 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:50:35.538 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:50:35.540 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:50:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:50:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:50:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:50:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:50:35.543 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:50:35.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.543 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:35.599 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:50:35.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:50:35.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:35.695 INFO:teuthology.orchestra.run.smithi028.stdout:client6559 2024-09-10T10:50:35.695 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:35.695 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:35.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.730 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-09-10T10:50:35.811 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:35.812 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:35.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:35.887 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-10T10:50:36.035 INFO:teuthology.orchestra.run.smithi028.stderr:6+0 records in 2024-09-10T10:50:36.036 INFO:teuthology.orchestra.run.smithi028.stderr:6+0 records out 2024-09-10T10:50:36.036 INFO:teuthology.orchestra.run.smithi028.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0726902 s, 86.6 MB/s 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> import json 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> except OSError as e: 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(e.errno) 2024-09-10T10:50:36.037 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:50:36.038 DEBUG:teuthology.orchestra.run.smithi028:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T10:50:36.038 DEBUG:teuthology.orchestra.run.smithi028:> print(json.dumps( 2024-09-10T10:50:36.038 DEBUG:teuthology.orchestra.run.smithi028:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T10:50:36.038 DEBUG:teuthology.orchestra.run.smithi028:> indent=2)) 2024-09-10T10:50:36.038 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:50:36.139 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:36.139 INFO:teuthology.orchestra.run.smithi028.stdout: "st_mode": 33188, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_ino": 1099511627777, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_dev": 42, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_nlink": 1, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_uid": 1000, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_gid": 1333, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_size": 6291456, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_atime": 1725965435.9602, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_mtime": 1725965435.9961991, 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout: "st_ctime": 1725965435.9961991 2024-09-10T10:50:36.140 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:36.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:36.151 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:36.151 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:36.227 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:36.228 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:36.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:50:36.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.500+0000 7f68a8600640 1 Processor -- start 2024-09-10T10:50:36.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.500+0000 7f68a8600640 1 -- start start 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.501+0000 7f68a8600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f68a0147e90 0x7f68a00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a8600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a0072a00 con 0x7f68a00725f0 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68a006f480 con 0x7f68a0147230 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a8600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f68a00b34d0 con 0x7f68a0147e90 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f68a0147e90 0x7f68a00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a6600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f68a00725f0 0x7f68a00aee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56196/0 (socket says 172.21.15.28:56196) 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f68a0147e90 0x7f68a00720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44186/0 (socket says 172.21.15.28:44186) 2024-09-10T10:50:36.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a6600640 1 -- 172.21.15.28:0/612954280 learned_addr learned my addr 172.21.15.28:0/612954280 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:36.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a5200640 1 -- 172.21.15.28:0/612954280 >> v1:172.21.15.138:6789/0 conn(0x7f68a0147230 legacy=0x7f68a006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:36.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a5200640 1 -- 172.21.15.28:0/612954280 >> v1:172.21.15.28:6789/0 conn(0x7f68a00725f0 legacy=0x7f68a00aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.502+0000 7f68a5200640 1 -- 172.21.15.28:0/612954280 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a006e900 con 0x7f68a0147e90 2024-09-10T10:50:36.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.503+0000 7f68a5200640 1 --2- 172.21.15.28:0/612954280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a0147e90 0x7f68a00720b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097a0 tx=0x7f688c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4552c59265081fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:36.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f689fe00640 1 -- 172.21.15.28:0/612954280 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c037070 con 0x7f68a0147e90 2024-09-10T10:50:36.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f689fe00640 1 -- 172.21.15.28:0/612954280 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c032440 con 0x7f68a0147e90 2024-09-10T10:50:36.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f689fe00640 1 -- 172.21.15.28:0/612954280 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c032810 con 0x7f68a0147e90 2024-09-10T10:50:36.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 -- 172.21.15.28:0/612954280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a0147e90 msgr2=0x7f68a00720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 --2- 172.21.15.28:0/612954280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a0147e90 0x7f68a00720b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f688c0097a0 tx=0x7f688c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 -- 172.21.15.28:0/612954280 shutdown_connections 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 --2- 172.21.15.28:0/612954280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a0147e90 0x7f68a00720b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 -- 172.21.15.28:0/612954280 >> 172.21.15.28:0/612954280 conn(0x7f68a0067d70 msgr2=0x7f68a006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 -- 172.21.15.28:0/612954280 shutdown_connections 2024-09-10T10:50:36.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.504+0000 7f68a8600640 1 -- 172.21.15.28:0/612954280 wait complete. 2024-09-10T10:50:36.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.506+0000 7f68a8600640 1 Processor -- start 2024-09-10T10:50:36.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.506+0000 7f68a8600640 1 -- start start 2024-09-10T10:50:36.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 0x7f68a01e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 0x7f68a01e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f68a006eec0 con 0x7f68a00725f0 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68a006f1a0 con 0x7f68a0147230 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68a0147610 con 0x7f68a0147e90 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44200/0 (socket says 172.21.15.28:44200) 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 -- 172.21.15.28:0/3643405038 learned_addr learned my addr 172.21.15.28:0/3643405038 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5200640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 0x7f68a01e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a6600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 0x7f68a01e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 -- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 msgr2=0x7f68a01e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 0x7f68a01e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 -- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 msgr2=0x7f68a01e7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 0x7f68a01e7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5c00640 1 -- 172.21.15.28:0/3643405038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a00c55e0 con 0x7f68a00725f0 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a6600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 0x7f68a01e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.507+0000 7f68a5200640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 0x7f68a01e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f68a5c00640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f689400c460 tx=0x7f689400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894016020 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894012870 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894012c60 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a00c58c0 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.508+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a015e340 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.509+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a0072090 con 0x7f68a00725f0 2024-09-10T10:50:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.510+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6894019020 con 0x7f68a00725f0 2024-09-10T10:50:36.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.511+0000 7f689d600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:36.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.511+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f689408ae20 con 0x7f68a00725f0 2024-09-10T10:50:36.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.512+0000 7f68a5200640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:36.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.513+0000 7f68a5200640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.515+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6894020030 con 0x7f68a00725f0 2024-09-10T10:50:36.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.714+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68a00addc0 con 0x7f68a00725f0 2024-09-10T10:50:36.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.714+0000 7f689d600640 1 -- 172.21.15.28:0/3643405038 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) ==== 80+0+0 (secure 0 0 0) 0x7f6894019310 con 0x7f68a00725f0 2024-09-10T10:50:36.715 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.716+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 msgr2=0x7f688006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.716+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 0x7f688006e2f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f688c034000 tx=0x7f688c02e000 comp rx=0 tx=0).stop 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.716+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 msgr2=0x7f68a015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.716+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f689400c460 tx=0x7f689400c920 comp rx=0 tx=0).stop 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 shutdown_connections 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f688006be30 0x7f688006e2f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68a0147e90 0x7f68a01e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a0147230 0x7f68a01e7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 --2- 172.21.15.28:0/3643405038 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68a00725f0 0x7f68a015de00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:36.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 >> 172.21.15.28:0/3643405038 conn(0x7f68a0067d70 msgr2=0x7f68a006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:36.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 shutdown_connections 2024-09-10T10:50:36.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:36.717+0000 7f68a8600640 1 -- 172.21.15.28:0/3643405038 wait complete. 2024-09-10T10:50:36.733 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:50:36.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:36.733 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:36.830 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:50:36.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:36.830 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:50:36.874 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:37.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.070+0000 7f004b200640 1 Processor -- start 2024-09-10T10:50:37.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.070+0000 7f004b200640 1 -- start start 2024-09-10T10:50:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.071+0000 7f004b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044072d30 0x7f004406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.071+0000 7f004b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f004406f700 0x7f004415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.071+0000 7f004b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00440cdbe0 con 0x7f004415b300 2024-09-10T10:50:37.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.071+0000 7f004b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f004406fb50 con 0x7f004406f700 2024-09-10T10:50:37.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.071+0000 7f004b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00440728a0 con 0x7f0044072d30 2024-09-10T10:50:37.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0043e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f004415b300 0x7f00440cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56204/0 (socket says 172.21.15.28:56204) 2024-09-10T10:50:37.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0043e00640 1 -- 172.21.15.28:0/1170007237 learned_addr learned my addr 172.21.15.28:0/1170007237 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0043400640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044072d30 0x7f004406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> v2:172.21.15.138:3300/0 conn(0x7f004406f700 0x7f004415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095631251 0 0) 0x7f00440cdbe0 con 0x7f004415b300 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0028003520 con 0x7f004415b300 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1994958615 0 0) 0x7f0028003520 con 0x7f004415b300 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044072d30 msgr2=0x7f004406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044072d30 0x7f004406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 >> v2:172.21.15.138:3300/0 conn(0x7f004406f700 msgr2=0x7f004415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 --2- 172.21.15.28:0/1170007237 >> v2:172.21.15.138:3300/0 conn(0x7f004406f700 0x7f004415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00440cd900 con 0x7f004415b300 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f0034001670 con 0x7f004415b300 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.072+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 >> v1:172.21.15.28:6789/0 conn(0x7f004415b300 legacy=0x7f00440cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00280039b0 0x7f0028003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0028004800 0x7f0028004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f002800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f004406fb50 con 0x7f00280051c0 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00440728a0 con 0x7f00280039b0 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f002800bf00 con 0x7f0028004800 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> v2:172.21.15.138:3300/0 conn(0x7f004406f700 0x7f004415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0043400640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044072d30 0x7f004406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:37.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f002800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0043e00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00280039b0 0x7f0028003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0043400640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0028004800 0x7f0028004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 -- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0028004800 msgr2=0x7f0028004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0028004800 0x7f0028004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 -- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00280039b0 msgr2=0x7f0028003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00280039b0 0x7f0028003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0049200640 1 -- 172.21.15.28:0/1170007237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0028003520 con 0x7f00280051c0 2024-09-10T10:50:37.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.073+0000 7f0043e00640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00280039b0 0x7f0028003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:37.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.074+0000 7f0049200640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f002800b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e380 tx=0x7f0028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c191615e58dc9636 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c009dc0 con 0x7f00280051c0 2024-09-10T10:50:37.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c003b90 con 0x7f00280051c0 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f0042a00640 1 -- 172.21.15.28:0/1170007237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c004020 con 0x7f00280051c0 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 -- 172.21.15.28:0/1170007237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 msgr2=0x7f002800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 --2- 172.21.15.28:0/1170007237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f002800b9c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e380 tx=0x7f0028005830 comp rx=0 tx=0).stop 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f0049200640 1 -- 172.21.15.28:0/1170007237 reap_dead start 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 -- 172.21.15.28:0/1170007237 shutdown_connections 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 -- 172.21.15.28:0/1170007237 >> 172.21.15.28:0/1170007237 conn(0x7f0044067d70 msgr2=0x7f004406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 -- 172.21.15.28:0/1170007237 shutdown_connections 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.075+0000 7f004b200640 1 -- 172.21.15.28:0/1170007237 wait complete. 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.077+0000 7f004b200640 1 Processor -- start 2024-09-10T10:50:37.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.077+0000 7f004b200640 1 -- start start 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044168900 0x7f0044147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044150dc0 0x7f00441479a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00440cd5c0 con 0x7f00280051c0 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0044072530 con 0x7f0044150dc0 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f004b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00440cffe0 con 0x7f0044168900 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f0043e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f0043e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44224/0 (socket says 172.21.15.28:44224) 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f0043e00640 1 -- 172.21.15.28:0/2040096192 learned_addr learned my addr 172.21.15.28:0/2040096192 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:37.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f0043400640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044168900 0x7f0044147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.078+0000 7f0049200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044150dc0 0x7f00441479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 -- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044168900 msgr2=0x7f0044147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044168900 0x7f0044147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 -- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044150dc0 msgr2=0x7f00441479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044150dc0 0x7f00441479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 -- 172.21.15.28:0/2040096192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004415e5e0 con 0x7f00280051c0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f0043e00640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0034007310 tx=0x7f00340077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400fa00 con 0x7f00280051c0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0034018070 con 0x7f00280051c0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400fda0 con 0x7f00280051c0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004415da60 con 0x7f00280051c0 2024-09-10T10:50:37.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.079+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004415e020 con 0x7f00280051c0 2024-09-10T10:50:37.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.080+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0044166c80 con 0x7f00280051c0 2024-09-10T10:50:37.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.081+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f003401d460 con 0x7f00280051c0 2024-09-10T10:50:37.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.082+0000 7f003be00640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.082+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7f003408b2b0 con 0x7f00280051c0 2024-09-10T10:50:37.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.083+0000 7f0043400640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.083+0000 7f0043400640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0044148240 tx=0x7f0030008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.087+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0034059ac0 con 0x7f00280051c0 2024-09-10T10:50:37.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.307+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00440cc9a0 con 0x7f00280051c0 2024-09-10T10:50:37.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.308+0000 7f003be00640 1 -- 172.21.15.28:0/2040096192 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5225 (secure 0 0 0) 0x7f00340593c0 con 0x7f00280051c0 2024-09-10T10:50:37.309 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:37.310 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":83,"btime":"2024-09-10T10:50:31:186469+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:30.609334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6849":{"gid":6849,"name":"a","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/3446889436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3446889436},{"type":"v1","addr":"172.21.15.28:6835","nonce":3446889436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6849,"qdb_cluster":[6849]},"id":6}]} 2024-09-10T10:50:37.310 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 83 2024-09-10T10:50:37.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 msgr2=0x7f002406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 0x7f002406e2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0044148240 tx=0x7f0030008040 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 msgr2=0x7f0044142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0034007310 tx=0x7f00340077d0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 shutdown_connections 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f002406be30 0x7f002406e2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0044150dc0 0x7f00441479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0044168900 0x7f0044147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 --2- 172.21.15.28:0/2040096192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00280051c0 0x7f0044142a50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.310+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 >> 172.21.15.28:0/2040096192 conn(0x7f0044067d70 msgr2=0x7f0044159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.311+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 shutdown_connections 2024-09-10T10:50:37.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.311+0000 7f004b200640 1 -- 172.21.15.28:0/2040096192 wait complete. 2024-09-10T10:50:37.327 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T10:50:37.510 INFO:teuthology.orchestra.run.smithi028.stdout:{"message":"","return_code":0} 2024-09-10T10:50:37.510 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T10:50:37.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:50:37.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.667+0000 7fd361400640 1 Processor -- start 2024-09-10T10:50:37.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.667+0000 7fd361400640 1 -- start start 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.668+0000 7fd361400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.668+0000 7fd361400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd35c06f8d0 0x7fd35c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.668+0000 7fd361400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c0cdbe0 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.668+0000 7fd361400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd35c0728a0 con 0x7fd35c06f8d0 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.668+0000 7fd361400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c0cd900 con 0x7fd35c15b300 2024-09-10T10:50:37.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd35c06f8d0 0x7fd35c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd353e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd35c0cd3f0 0x7fd35c06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56214/0 (socket says 172.21.15.28:56214) 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd35c06f8d0 0x7fd35c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44236/0 (socket says 172.21.15.28:44236) 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd353e00640 1 -- 172.21.15.28:0/3376833329 learned_addr learned my addr 172.21.15.28:0/3376833329 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35a800640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 -- 172.21.15.28:0/3376833329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 msgr2=0x7fd35c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 -- 172.21.15.28:0/3376833329 >> v1:172.21.15.28:6789/0 conn(0x7fd35c0cd3f0 legacy=0x7fd35c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd35b200640 1 -- 172.21.15.28:0/3376833329 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c072250 con 0x7fd35c06f8d0 2024-09-10T10:50:37.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.669+0000 7fd359e00640 1 -- 172.21.15.28:0/3376833329 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4012387418 0 0) 0x7fd35c0cdbe0 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.670+0000 7fd35b200640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c06f8d0 0x7fd35c15b6e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd354009d90 tx=0x7fd35402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b3d3b579527cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.670+0000 7fd359e00640 1 -- 172.21.15.28:0/3376833329 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354007740 con 0x7fd35c06f8d0 2024-09-10T10:50:37.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.670+0000 7fd359e00640 1 -- 172.21.15.28:0/3376833329 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3540078e0 con 0x7fd35c06f8d0 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.670+0000 7fd359e00640 1 -- 172.21.15.28:0/3376833329 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354007cb0 con 0x7fd35c06f8d0 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 -- 172.21.15.28:0/3376833329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c06f8d0 msgr2=0x7fd35c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c06f8d0 0x7fd35c15b6e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd354009d90 tx=0x7fd35402a660 comp rx=0 tx=0).stop 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 -- 172.21.15.28:0/3376833329 shutdown_connections 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c06f8d0 0x7fd35c15b6e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 --2- 172.21.15.28:0/3376833329 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 -- 172.21.15.28:0/3376833329 >> 172.21.15.28:0/3376833329 conn(0x7fd35c067d70 msgr2=0x7fd35c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 -- 172.21.15.28:0/3376833329 shutdown_connections 2024-09-10T10:50:37.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.671+0000 7fd361400640 1 -- 172.21.15.28:0/3376833329 wait complete. 2024-09-10T10:50:37.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.673+0000 7fd361400640 1 Processor -- start 2024-09-10T10:50:37.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.673+0000 7fd361400640 1 -- start start 2024-09-10T10:50:37.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd35c06f8d0 0x7fd35c0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0d0160 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0cffe0 con 0x7fd35c06f8d0 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd361400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c072d30 con 0x7fd35c15b300 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd353e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd353e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44244/0 (socket says 172.21.15.28:44244) 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd353e00640 1 -- 172.21.15.28:0/2676436995 learned_addr learned my addr 172.21.15.28:0/2676436995 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd35b200640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.674+0000 7fd35a800640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd35c06f8d0 0x7fd35c0aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 -- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 msgr2=0x7fd35c1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 -- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd35c06f8d0 msgr2=0x7fd35c0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd35c06f8d0 0x7fd35c0aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 -- 172.21.15.28:0/2676436995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c15d000 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd35b200640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd353e00640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c4b0 tx=0x7fd34400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344018070 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344007500 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344004a20 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c155700 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.675+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c155cc0 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.676+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c06ed30 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.677+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd34401b070 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.678+0000 7fd352a00640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 0x7fd33806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:37.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.678+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 6101+0+0 (secure 0 0 0) 0x7fd34408a200 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.679+0000 7fd35a800640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 0x7fd33806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:37.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.679+0000 7fd35a800640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 0x7fd33806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0079a0 tx=0x7fd34c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:37.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.683+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd344058490 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.901+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd35c156000 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.902 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:50:37.902+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:50:37.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.904+0000 7fd352a00640 1 -- 172.21.15.28:0/2676436995 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fd34405c790 con 0x7fd35c0cd3f0 2024-09-10T10:50:37.905 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:50:37.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 msgr2=0x7fd33806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 0x7fd33806e2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0079a0 tx=0x7fd34c008040 comp rx=0 tx=0).stop 2024-09-10T10:50:37.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 msgr2=0x7fd35c154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:37.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd34400c4b0 tx=0x7fd34400c970 comp rx=0 tx=0).stop 2024-09-10T10:50:37.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 shutdown_connections 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd33806be30 0x7fd33806e2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd35c15b300 0x7fd35c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd35c0cd3f0 0x7fd35c154b90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 --2- 172.21.15.28:0/2676436995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd35c06f8d0 0x7fd35c0aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.906+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 >> 172.21.15.28:0/2676436995 conn(0x7fd35c067d70 msgr2=0x7fd35c0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:37.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.907+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 shutdown_connections 2024-09-10T10:50:37.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:37.907+0000 7fd361400640 1 -- 172.21.15.28:0/2676436995 wait complete. 2024-09-10T10:50:37.916 INFO:tasks.ceph.mds.a.smithi028.stderr: -4272> 2024-09-10T10:50:33.758+0000 7fb7d1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:37.924 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata export - 2024-09-10T10:50:37.966 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:50:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.001+0000 7f0f2c261f40 1 Processor -- start 2024-09-10T10:50:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.001+0000 7f0f2c261f40 1 -- start start 2024-09-10T10:50:38.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df9091b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df9099a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55df90919530 con 0x55df908ce470 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df90876b30 con 0x55df909a79c0 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55df909a6ee0 con 0x55df909a43e0 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44276/0 (socket says 172.21.15.28:44276) 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 -- 172.21.15.28:0/2196712577 learned_addr learned my addr 172.21.15.28:0/2196712577 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f28a00640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df9091b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29e00640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df9099a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 -- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 msgr2=0x55df9091b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df9091b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 -- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 msgr2=0x55df9099a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df9099a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29400640 1 -- 172.21.15.28:0/2196712577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df90901db0 con 0x55df908ce470 2024-09-10T10:50:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.002+0000 7f0f29e00640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df9099a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f29400640 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c001e60 tx=0x7f0f1c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d624d1cdbe047bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f23400640 1 -- 172.21.15.28:0/2196712577 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c012020 con 0x55df908ce470 2024-09-10T10:50:38.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f23400640 1 -- 172.21.15.28:0/2196712577 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00e670 con 0x55df908ce470 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f23400640 1 -- 172.21.15.28:0/2196712577 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00eaa0 con 0x55df908ce470 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f2c261f40 1 -- 172.21.15.28:0/2196712577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 msgr2=0x55df909a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.003+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c001e60 tx=0x7f0f1c00b780 comp rx=0 tx=0).stop 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 -- 172.21.15.28:0/2196712577 shutdown_connections 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df9099a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df9091b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/2196712577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a56d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 -- 172.21.15.28:0/2196712577 >> 172.21.15.28:0/2196712577 conn(0x55df9091d0a0 msgr2=0x55df909a4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 -- 172.21.15.28:0/2196712577 shutdown_connections 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 -- 172.21.15.28:0/2196712577 wait complete. 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 Processor -- start 2024-09-10T10:50:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.004+0000 7f0f2c261f40 1 -- start start 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df909a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df90912720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55df908d0c20 con 0x55df908ce470 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df90902090 con 0x55df909a79c0 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f2c261f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55df9091d7b0 con 0x55df909a43e0 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.005+0000 7f0f29400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44286/0 (socket says 172.21.15.28:44286) 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 learned_addr learned my addr 172.21.15.28:0/5918673 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df909a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df90912720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 msgr2=0x55df909a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55df909a43e0 0x55df909a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 msgr2=0x55df90912720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df909a79c0 0x55df90912720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df909a97f0 con 0x55df908ce470 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c001ed0 tx=0x7f0f1c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.006+0000 7f0f22000640 1 -- 172.21.15.28:0/5918673 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00e470 con 0x55df908ce470 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.007+0000 7f0f22000640 1 -- 172.21.15.28:0/5918673 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f1c0197f0 con 0x55df908ce470 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.007+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df9099e5b0 con 0x55df908ce470 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.007+0000 7f0f22000640 1 -- 172.21.15.28:0/5918673 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c019be0 con 0x55df908ce470 2024-09-10T10:50:38.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.007+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df9091c120 con 0x55df908ce470 2024-09-10T10:50:38.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.008+0000 7f0f22000640 1 -- 172.21.15.28:0/5918673 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0f1c015020 con 0x55df908ce470 2024-09-10T10:50:38.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.009+0000 7f0f22000640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 0x7f0f0c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.009+0000 7f0f22000640 1 -- 172.21.15.28:0/5918673 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f0f1c08b790 con 0x55df908ce470 2024-09-10T10:50:38.012 INFO:teuthology.orchestra.run.smithi028.stdout:���� 2024-09-10T10:50:38.012 INFO:teuthology.orchestra.run.smithi028.stdout: �� 2024-09-10T10:50:38.012 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.010+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55df909a1760 0x55df90a40740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.010+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x55df909a1760 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.010+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 0x7f0f0c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.010+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55df909a1760 0x55df90a40740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.010+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55df909a1760 0x55df90a40740 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.011+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 0x7f0f0c06ccd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55df908ceba0 tx=0x7f0f1800c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.011+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.011+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f0f24050900 0x7f0f2404f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.011+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:08000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24050900 2024-09-10T10:50:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.011+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f0f24050900 0x7f0f2404f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.012+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f0f24050900 0x7f0f2404f370 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.015+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(2 [pgnls start_epoch 100 out=177b] v99'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.015+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f0f1c017040 0x7f0f1c058710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.015+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:3 13.8 13:10000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c017040 2024-09-10T10:50:38.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.015+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f0f1c017040 0x7f0f1c058710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.016+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f0f1c017040 0x7f0f1c058710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.018+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.5 v2:172.21.15.138:6802/2049132176 1 ==== osd_op_reply(3 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c017040 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.019+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:4 13.18 13:18000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x55df909a1760 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.019+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(4 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.019+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f0f24056300 0x7f0f24056780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.019+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:5 13.4 13:20000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24056300 2024-09-10T10:50:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.019+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f0f24056300 0x7f0f24056780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.020+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f0f24056300 0x7f0f24056780 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.022+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(5 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.022+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f0f2406b940 0x7f0f240683a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.022+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:6 13.14 13:28000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f2406b940 2024-09-10T10:50:38.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.023+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f0f2406b940 0x7f0f240683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.023+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f0f2406b940 0x7f0f240683a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.026+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(6 [pgnls start_epoch 100 out=149b] v99'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.026+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:7 13.c 13:30000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f2406b940 2024-09-10T10:50:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.027+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(7 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.027+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f0f1c059ab0 0x7f0f1c05fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.027+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:8 13.1c 13:38000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.027+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f0f1c059ab0 0x7f0f1c05fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.027+0000 7f0f28a00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f0f1c059ab0 0x7f0f1c05fda0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.029+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(8 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.029+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:9 13.2 13:40000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x55df909a1760 2024-09-10T10:50:38.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.030+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(9 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.030+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:10 13.12 13:48000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f2406b940 2024-09-10T10:50:38.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.030+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(10 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.030+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f0f1c061fa0 0x7f0f1c0891a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.030+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- osd_op(unknown.0.0:11 13.a 13:50000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c061fa0 2024-09-10T10:50:38.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.031+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f0f1c061fa0 0x7f0f1c0891a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.032+0000 7f0f29e00640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f0f1c061fa0 0x7f0f1c0891a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.032+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.4 v2:172.21.15.138:6810/1083898337 1 ==== osd_op_reply(11 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f1c061fa0 2024-09-10T10:50:38.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.032+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:12 13.1a 13:58000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c017040 2024-09-10T10:50:38.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.5 v2:172.21.15.138:6802/2049132176 2 ==== osd_op_reply(12 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c017040 2024-09-10T10:50:38.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:13 13.6 13:60000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24056300 2024-09-10T10:50:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(13 [pgnls start_epoch 100 out=77b] v99'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.16 13:68000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24050900 2024-09-10T10:50:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(14 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.033+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:15 13.e 13:70000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(15 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:16 13.1e 13:78000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(16 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:17 13.1 13:80000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x55df909a1760 2024-09-10T10:50:38.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(17 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.034+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:18 13.11 13:88000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f2406b940 2024-09-10T10:50:38.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.035+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(18 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.035+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:19 13.9 13:90000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.036+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(19 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.036+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.19 13:98000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24050900 2024-09-10T10:50:38.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.036+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(20 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.036+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f0f1c05ab80 0x7f0f1c057e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.036+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:21 13.5 13:a0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f1c05ab80 2024-09-10T10:50:38.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.037+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f0f1c05ab80 0x7f0f1c057e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.037+0000 7f0f29400640 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f0f1c05ab80 0x7f0f1c057e10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.037+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(21 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.037+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:22 13.15 13:a8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55df909a2ac0 con 0x7f0f24056300 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(22 [pgnls start_epoch 100 out=97b] v98'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:23 13.d 13:b0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f1c059ab0 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(23 [pgnls start_epoch 100 out=74b] v98'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:24 13.1d 13:b8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x55df909a1760 2024-09-10T10:50:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(24 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.038+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:25 13.3 13:c0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f2406b940 2024-09-10T10:50:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.039+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(25 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.039+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:26 13.13 13:c8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x55df909a1760 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.039+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(26 [pgnls start_epoch 100 out=98b] v99'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.039+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:27 13.b 13:d0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f24056300 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(27 [pgnls start_epoch 100 out=76b] v99'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:28 13.1b 13:d8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f1c05ab80 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(28 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:29 13.7 13:e0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f24056300 2024-09-10T10:50:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(29 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.040+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:30 13.17 13:e8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x55df909a1760 2024-09-10T10:50:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(30 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:31 13.f 13:f0000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x7f0f1c05ab80 2024-09-10T10:50:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(31 [pgnls start_epoch 100 out=73b] v98'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:32 13.1f 13:f8000000:%01:::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x7f0f24081070 con 0x55df909a1760 2024-09-10T10:50:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(32 [pgnls start_epoch 100 out=101b] v99'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.041+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909aa510 con 0x7f0f24050900 2024-09-10T10:50:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.042+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(33 601.00000000 [read 0~0] v0'0 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.042+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:34 13.10 13:09401a81:::601.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a40c80 con 0x7f0f24050900 2024-09-10T10:50:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.043+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(34 601.00000000 [getxattrs out=124b] v0'0 uv4 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.043+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:35 13.10 13:09401a81:::601.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a41050 con 0x7f0f24050900 2024-09-10T10:50:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.043+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(35 601.00000000 [omap-get-header out=274b] v0'0 uv4 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.043+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:36 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a49a10 con 0x7f0f24050900 2024-09-10T10:50:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.044+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(36 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.044+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:37 13.10 13:0bd6d154:::602.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a7dd0 con 0x7f0f24050900 2024-09-10T10:50:38.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.044+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(37 602.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.045+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:38 13.10 13:0bd6d154:::602.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8170 con 0x7f0f24050900 2024-09-10T10:50:38.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.045+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(38 602.00000000 [getxattrs out=124b] v0'0 uv6 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.045+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:39 13.10 13:0bd6d154:::602.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8590 con 0x7f0f24050900 2024-09-10T10:50:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.045+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(39 602.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.046+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:40 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8590 con 0x7f0f24050900 2024-09-10T10:50:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.046+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(40 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.046+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:41 13.10 13:0cf6ad0f:::10000000000.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.046+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(41 10000000000.00000000 [read 0~0] v0'0 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.046+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:42 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.047+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(42 10000000000.00000000 [getxattrs out=124b] v0'0 uv10 ondisk = 0) ==== 164+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.047+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:43 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.047+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(43 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.047+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:44 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.048+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals out=594b] v0'0 uv10 ondisk = 0) ==== 164+0+594 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.050 INFO:teuthology.orchestra.run.smithi028.stdout: ��*$- 601.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray1 ���� �� ((88���� �� ��*$- 602.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray2 ���� �� ((88���� �� ��2,510000000000.00000000�������������������!�������������������������� ���~_layout��������_parent>8subdir ���� ��  ({$�f�};9({$�f�};98`{$�f��`;8`{$�f��`;���� ��WQ sixmegs_head9i* �{$�f��`;���5@@`��������{$�f��`;{$�f�};9(8`{$�f��`;8`{$�f��`; 2024-09-10T10:50:38.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.048+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:45 13.10 13:0d82a743:::600.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.049+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(45 600.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.049+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:46 13.10 13:0d82a743:::600.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.049+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(46 600.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.049+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:47 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.050+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(47 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.050+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:48 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.051+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(48 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.051+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:49 13.10 13:0d89b25e:::603.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.051+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(49 603.00000000 [read 0~0] v0'0 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.051+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:50 13.10 13:0d89b25e:::603.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.052+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(50 603.00000000 [getxattrs out=124b] v0'0 uv8 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.052+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:51 13.10 13:0d89b25e:::603.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.052+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(51 603.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.052+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:52 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f24050900 2024-09-10T10:50:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.053+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(52 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f24050900 2024-09-10T10:50:38.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.053+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:53 13.14 13:287034c4:::1.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.054+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(53 1.00000000.inode [read 0~582 out=582b] v0'0 uv15 ondisk = 0) ==== 160+0+582 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.054+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:54 13.14 13:287034c4:::1.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.056 INFO:teuthology.orchestra.run.smithi028.stdout:��������{$�f�};9����`security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� ��*$- 600.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray0 ���� �� ((88���� �� ��*$- 603.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray3 ���� �� ((88���� �� ��.(11.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)�{$�f`� 0�C@@��������{$�f`� 0({$�f`� 08`{$�f��`;8t$�f:�� t$�f:�� ����NHt$�f:�� ������������ �� 2024-09-10T10:50:38.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.054+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(54 1.00000000.inode [getxattrs out=4b] v0'0 uv15 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.054+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:55 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.057 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.055+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(55 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.055+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:56 13.14 13:287034c4:::1.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.055+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(56 1.00000000.inode [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.055+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:57 13.14 13:292cf221:::200.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(57 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:58 13.14 13:292cf221:::200.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.059 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��*$- 200.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@�>@�>@�F@@@ ���� �� 2024-09-10T10:50:38.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(58 200.00000000 [getxattrs out=4b] v0'0 uv19 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:59 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(59 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.060 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.056+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:60 13.14 13:292cf221:::200.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.057+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(60 200.00000000 [omap-get-vals out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.057+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:61 13.14 13:2e2fa760:::200.00000001:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.061 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��*$- 200.00000001�������������������!�������������������������� ���F�F�F�F��XS&YA1e@��XS&YA1h^Tt$�f ` 2024-09-10T10:50:38.061 INFO:teuthology.orchestra.run.smithi028.stdout:2 ((88 2 ( 2024-09-10T10:50:38.061 INFO:teuthology.orchestra.run.smithi028.stdout:( 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:8 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� 8 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� ��������@0@��XS&YA1�� 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:�{$�f��������� entity_id0hostname smithi028kernel_version6.11.0-rc5-g2ca44a2f35caroot/��7 ����@��XS&YA1���{$�fP��#n���������@��XS&YA1{$�f�#setattr� C���������{$�f�.z#�C@@��������t$�f:�� (8{$�f�.z#8t$�f:�� t$�f:�� ����NHt$�f:�� �����������������[@��XS&YA1��{$�f��&0mkdir�p ({$�f`� 0({$�f`� 08{$�f`� 08{$�f`� 0J 8subdir���������{$�f`� 0�A�5����������������{$�f`� 0{$�f`� 0(88��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������2 ((88  C���������{$�f`� 0�C@@��������{$�f`� 0({$�f`� 08{$�f`� 08t$�f:�� t$�f:�� ����NHt$�f:�� ��������������������@��XS&YA1�{$�f�-Z9openc�p ({$�f`� 0({$�f`� 08{$�f�};98{$�f`� 0J 8subdir���������{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98{$�f�};98{$�f�};9��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� ({$�f�};9({$�f�};98{$�f�};98{$�f�};9m [sixmegs���������{$�f�};9���5@@��������{$�f�};9{$�f�};9�@(88��������{$�f�};9����security.selinux%unconfined_u:object_r:unlabeled_t:s0����������!!���������@��XS&YA1{$�f[pK: 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:cap update�p ({$�f`� 0({$�f`� 08 {$�f�] :8{$�f`� 0J 8subdir���������{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98 {$�f�] :8 {$�f�] :��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� ({$�f�};9({$�f�};98 {$�f�] :8 {$�f�] :u csixmegs���������{$�f�] :���5@@ ��������{$�f�] :{$�f�};9��(8 {$�f�] :8 {$�f�] :��������{$�f�};9���� security.selinux%unconfined_u:object_r:unlabeled_t:s0������������������@��XS&YA1{$�f� 2; 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:cap update�p  ({$�f`� 0({$�f`� 08P{$�f�=;8{$�f`� 0J 8subdir���������{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98P{$�f�=;8P{$�f�=;��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0��������� ({$�f�};9({$�f�};98P{$�f�=;8P{$�f�=;u csixmegs���������{$�f�=;���5@@P��������{$�f�=;{$�f�};9��(8P{$�f�=;8P{$�f�=;��������{$�f�};9����Psecurity.selinux%unconfined_u:object_r:unlabeled_t:s0������������������ @��XS&YA1|$�f���. 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:cap update�p  ({$�f`� 0({$�f`� 08`{$�f��`;8{$�f`� 0J 8subdir�������� 2024-09-10T10:50:38.062 INFO:teuthology.orchestra.run.smithi028.stdout:�{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98`{$�f��`;8`{$�f��`; 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  ({$�f�};9({$�f�};98`{$�f��`;8`{$�f��`;u csixmegs���������{$�f��`;���5@@`��������{$�f��`;{$�f�};9�(8`{$�f��`;8`{$�f��`;��������{$�f�};9����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�����������������()@��XS&YA1�� 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:�|$�f�D/��������X1@��XS&YA1s i _ |$�fX~�0check_inode_max_size p  ({$�f`� 0({$�f`� 08`{$�f��`;8`{$�f��`;J 8subdir�������� �{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98`{$�f��`;8`{$�f��`; ��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0���������  ({$�f�};9({$�f�};98`{$�f��`;8`{$�f��`;� =sixmegs�������� 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:�{$�f��`;���5@@`��������{$�f��`;{$�f�};9(8`{$�f��`;8`{$�f��`; 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:��������{$�f�};9����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� =sixmegs�������� 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:�{$�f��`;���5@@`��������{$�f��`;{$�f�};9(8`{$�f��`;8`{$�f��`; 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.058+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(61 200.00000001 [read 0~18118 out=18118b] v0'0 uv16 ondisk = 0) ==== 156+0+18118 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.063 INFO:teuthology.orchestra.run.smithi028.stdout:��������{$�f�};9����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� C���������{$�f`� 0�C@@��������{$�f`� 0({$�f`� 08`{$�f��`;8t$�f:�� t$�f:�� ����NHt$�f:�� ����������������2@��XS&YA18.$��>@��XS&YA1h^T}$�f��%2  ({$�f`� 0({$�f`� 08`{$�f��`;8`{$�f��`; 2 ( 2024-09-10T10:50:38.064 INFO:teuthology.orchestra.run.smithi028.stdout:( 2024-09-10T10:50:38.064 INFO:teuthology.orchestra.run.smithi028.stdout:8 2024-09-10T10:50:38.064 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� 8 2024-09-10T10:50:38.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.058+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:62 13.14 13:2e2fa760:::200.00000001:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.064 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� ��������@�>@��XS&YA1^TJ}$�fÓ¤Mscatter_writebehind2 ({$�f`� 0({$�f`� 08`{$�f��`;8`{$�f��`;  C���������{$�f`� 0�C@@��������{$�f`� 0({$�f`� 08`{$�f��`;8t$�f:�� t$�f:�� ����NHt$�f:�� ����������������TB@���� �� 2024-09-10T10:50:38.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.058+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(62 200.00000001 [getxattrs out=4b] v0'0 uv16 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.058+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:63 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(63 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.066 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:64 13.14 13:2e2fa760:::200.00000001:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(64 200.00000001 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:65 13.14 13:2f55791f:::606.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(65 606.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.059+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:66 13.14 13:2f55791f:::606.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.060+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(66 606.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.060+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:67 13.14 13:2f55791f:::606.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.060+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(67 606.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.060+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:68 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f2406b940 2024-09-10T10:50:38.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.061+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(68 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f2406b940 2024-09-10T10:50:38.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.061+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:69 13.1c 13:3ed09add:::607.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8510 con 0x7f0f1c059ab0 2024-09-10T10:50:38.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.061+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(69 607.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.061+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:70 13.1c 13:3ed09add:::607.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df909a8900 con 0x7f0f1c059ab0 2024-09-10T10:50:38.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(70 607.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:71 13.1c 13:3ed09add:::607.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a49db0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(71 607.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:72 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4b160 con 0x7f0f1c059ab0 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(72 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.062+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.6 13:654134d2:::mds0_openfiles.0:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4a150 con 0x7f0f24056300 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(73 mds0_openfiles.0 [read 0~0] v0'0 uv4 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.6 13:654134d2:::mds0_openfiles.0:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4a570 con 0x7f0f24056300 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(74 mds0_openfiles.0 [getxattrs out=4b] v0'0 uv4 ondisk = 0) ==== 160+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.072 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��*$- 606.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray6 ���� �� ((88���� �� ��*$- 607.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray7 ���� �� ((88���� �� ��.(1mds0_openfiles.0�������������������!�������������������������� �� 2024-09-10T10:50:38.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4a9b0 con 0x7f0f24056300 2024-09-10T10:50:38.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(75 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4ad50 con 0x7f0f24056300 2024-09-10T10:50:38.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.063+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(76 mds0_openfiles.0 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:77 13.e 13:717a0223:::608.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4b500 con 0x7f0f1c059ab0 2024-09-10T10:50:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(77 608.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.e 13:717a0223:::608.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4b8a0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(78 608.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.e 13:717a0223:::608.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bc40 con 0x7f0f1c059ab0 2024-09-10T10:50:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(79 608.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.064+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:81 13.15 13:a93a17c2:::604.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(81 604.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:82 13.15 13:a93a17c2:::604.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(82 604.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.065+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:83 13.15 13:a93a17c2:::604.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.066+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(83 604.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.066+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:84 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x7f0f24056300 2024-09-10T10:50:38.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.066+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(84 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.066+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:85 13.15 13:aa448500:::500.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.066+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(85 500.00000000 [read 0~90 out=90b] v0'0 uv3 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:86 13.15 13:aa448500:::500.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(86 500.00000000 [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.080 INFO:teuthology.orchestra.run.smithi028.stdout:���� ��4.*ceph fs volume v011 ���� �� ��*$- 608.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray8 ���� �� ((88���� �� ��*$- 604.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray4 ���� �� ((88���� �� ��*$- 500.00000000�������������������!�������������������������� ��tnZZTceph fs volume v011@@@@@@ ���� �� 2024-09-10T10:50:38.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:87 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f24056300 2024-09-10T10:50:38.081 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(87 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:88 13.15 13:aa448500:::500.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x7f0f24056300 2024-09-10T10:50:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.067+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(88 500.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:89 13.d 13:b50e409b:::mds_snaptable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(89 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:90 13.d 13:b50e409b:::mds_snaptable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(90 mds_snaptable [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.083 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��+%. mds_snaptable�������������������!�������������������������� ��`ZFF4���� �� 2024-09-10T10:50:38.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:91 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.068+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(91 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.084 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:92 13.d 13:b50e409b:::mds_snaptable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x7f0f1c059ab0 2024-09-10T10:50:38.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f28a00640 1 -- 172.21.15.28:0/5918673 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(92 mds_snaptable [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f0f18021040 con 0x7f0f1c059ab0 2024-09-10T10:50:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.1d 13:b871830b:::605.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 605.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.1d 13:b871830b:::605.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 605.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.069+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.1d 13:b871830b:::605.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 605.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x55df909a1760 2024-09-10T10:50:38.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.070+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 mds0_inotable [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 157+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.089 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��*$- 605.00000000�������������������!�������������������������� ���~_layout��������_parent>8stray5 ���� �� ((88���� �� ��+%. mds0_inotable�������������������!�������������������������� ��<6""��������� �� 2024-09-10T10:50:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.090 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x55df909a1760 2024-09-10T10:50:38.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 mds0_inotable [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:101 13.13 13:cd5a64a3:::100.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(101 100.00000000 [read 0~0] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.071+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:102 13.13 13:cd5a64a3:::100.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(102 100.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.091 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��*$- 100.00000000�������������������!�������������������������� �� 2024-09-10T10:50:38.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:103 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.092 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ( 2024-09-10T10:50:38.092 INFO:teuthology.orchestra.run.smithi028.stdout:( 2024-09-10T10:50:38.092 INFO:teuthology.orchestra.run.smithi028.stdout:8 2024-09-10T10:50:38.092 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� 8 2024-09-10T10:50:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(103 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:104 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x55df909a1760 2024-09-10T10:50:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(104 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.093 INFO:teuthology.orchestra.run.smithi028.stdout:t$�f��� ���� ��PJ 2024-09-10T10:50:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.072+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.b 13:d0630e4c:::mds0_sessionmap:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(105 mds0_sessionmap [read 0~0] v0'0 uv3 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.b 13:d0630e4c:::mds0_sessionmap:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.094 INFO:teuthology.orchestra.run.smithi028.stdout: stray0_head�i���t$�f��� @A����������������t$�f��� (8t$�f��� 8t$�f��� t$�f��� ������������ stray1_head�i���t$�fU�� @A����������������t$�fU�� (8t$�fU�� 8t$�fU�� t$�fU�� ������������ stray2_head�i���t$�f}� @A����������������t$�f}� (8t$�f}� 8t$�f}� t$�f}� ������������ stray3_head�i���t$�f�� @A����������������t$�f�� (8t$�f�� 8t$�f�� t$�f�� ������������ stray4_head�i���t$�f�� @A����������������t$�f�� (8t$�f�� 8t$�f�� t$�f�� ������������ stray5_head�i���t$�f��� @A����������������t$�f��� (8t$�f��� 8t$�f��� t$�f��� ������������ stray6_head�i���t$�f��� @A����������������t$�f��� (8t$�f��� 8t$�f��� t$�f��� ������������ stray7_head�i���t$�f&�� @A����������������t$�f&�� (8t$�f&�� 8t$�f&�� t$�f&�� ������������ stray8_head�i���t$�f��� @A����������������t$�f��� (8t$�f��� 8t$�f��� t$�f��� ������������ stray9_head�i��� t$�f��� @A����������������t$�f��� (8t$�f��� 8t$�f��� t$�f��� ���������������� �� ��-'0mds0_sessionmap�������������������!�������������������������� �� 2024-09-10T10:50:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(106 mds0_sessionmap [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 159+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(107 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.073+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(108 mds0_sessionmap [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(109 609.00000000 [read 0~0] v0'0 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(110 609.00000000 [getxattrs out=124b] v0'0 uv2 ondisk = 0) ==== 156+0+124 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:111 13.7 13:e0b41d06:::609.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900cc0 con 0x7f0f24056300 2024-09-10T10:50:38.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(111 609.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.074+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:112 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x7f0f24056300 2024-09-10T10:50:38.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.075+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(112 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x7f0f24056300 2024-09-10T10:50:38.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.075+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x7f0f1c05ab80 2024-09-10T10:50:38.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.075+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(113 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.075+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x7f0f1c05ab80 2024-09-10T10:50:38.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(114 400.00000000 [getxattrs out=4b] v0'0 uv1 ondisk = 0) ==== 156+0+4 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.099 INFO:teuthology.orchestra.run.smithi028.stdout:���� ������ �� ��*$- 609.00000000�������������������!�������������������������� ���~_layout��������_parent>8 stray9 ���� �� ((88���� �� ��*$- 400.00000000�������������������!�������������������������� ��0*���� �� 2024-09-10T10:50:38.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x7f0f1c05ab80 2024-09-10T10:50:38.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 6 ==== osd_op_reply(115 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.100 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:116 13.f 13:f1f69726:::400.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x7f0f1c05ab80 2024-09-10T10:50:38.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f29400640 1 -- 172.21.15.28:0/5918673 <== osd.2 v2:172.21.15.28:6824/1014290416 7 ==== osd_op_reply(116 400.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f0f1c029020 con 0x7f0f1c05ab80 2024-09-10T10:50:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.076+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x55df909a1760 2024-09-10T10:50:38.101 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��0*3100.00000000.inode�������������������!�������������������������� ��`ZFFceph fs volume v011)�t$�f՚� @A����������������t$�f՚� ( 2024-09-10T10:50:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(117 100.00000000.inode [read 0~582 out=582b] v0'0 uv3 ondisk = 0) ==== 162+0+582 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x55df909a1760 2024-09-10T10:50:38.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(118 100.00000000.inode [getxattrs out=4b] v0'0 uv3 ondisk = 0) ==== 162+0+4 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.102 INFO:teuthology.orchestra.run.smithi028.stdout:8 t$�f��� 8 t$�f��� t$�f՚� ����NH������������ �� 2024-09-10T10:50:38.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x55df909a1760 2024-09-10T10:50:38.102 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� 2024-09-10T10:50:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(119 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x55df909a1760 2024-09-10T10:50:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(120 100.00000000.inode [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.103 INFO:teuthology.orchestra.run.smithi028.stdout:���� �� ��("+ 2024-09-10T10:50:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.077+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90900ca0 con 0x55df909a1760 2024-09-10T10:50:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(121 1.00000000 [read 0~0] v0'0 uv6 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(122 1.00000000 [getxattrs out=92b] v0'0 uv6 ondisk = 0) ==== 154+0+92 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4bfe0 con 0x55df909a1760 2024-09-10T10:50:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(123 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.078+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55df90a4cff0 con 0x55df909a1760 2024-09-10T10:50:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(124 1.00000000 [omap-get-vals out=585b] v0'0 uv6 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x7f0f2404e040 con 0x55df909a1760 2024-09-10T10:50:38.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f0f1c059ab0 msgr2=0x7f0f1c05fda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f0f1c059ab0 0x7f0f1c05fda0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55df909a1760 msgr2=0x55df90a40740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55df909a1760 0x55df90a40740 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f0f1c05ab80 msgr2=0x7f0f1c057e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f0f1c05ab80 0x7f0f1c057e10 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f0f24056300 msgr2=0x7f0f24056780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f0f24056300 0x7f0f24056780 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f0f1c061fa0 msgr2=0x7f0f1c0891a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f0f1c061fa0 0x7f0f1c0891a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f0f1c017040 msgr2=0x7f0f1c058710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f0f1c017040 0x7f0f1c058710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f0f2406b940 msgr2=0x7f0f240683a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f0f2406b940 0x7f0f240683a0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f0f24050900 msgr2=0x7f0f2404f370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f0f24050900 0x7f0f2404f370 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 reap_dead start 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 reap_dead start 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 reap_dead start 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 msgr2=0x7f0f0c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f29e00640 1 -- 172.21.15.28:0/5918673 reap_dead start 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 0x7f0f0c06ccd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55df908ceba0 tx=0x7f0f1800c040 comp rx=0 tx=0).stop 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 msgr2=0x55df909a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c001ed0 tx=0x7f0f1c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 shutdown_connections 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0f0c06a810 0x7f0f0c06ccd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 --2- 172.21.15.28:0/5918673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55df908ce470 0x55df909a1220 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.079+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 >> 172.21.15.28:0/5918673 conn(0x55df9091d0a0 msgr2=0x55df9091aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.080+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 shutdown_connections 2024-09-10T10:50:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.080+0000 7f0f2c261f40 1 -- 172.21.15.28:0/5918673 wait complete. 2024-09-10T10:50:38.110 INFO:teuthology.orchestra.run.smithi028.stdout:1.00000000�������������������!�������������������������� ��d^_layout@@_parent ���� ��  ({$�f`� 0({$�f`� 08`{$�f��`;8`{$�f��`;���� ��NH subdir_head1i"�{$�f�};9�A�5����������������{$�f�};9{$�f`� 0({$�f�};98`{$�f��`;8`{$�f��`; ��������{$�f`� 0����security.selinux%unconfined_u:object_r:unlabeled_t:s0������������ �� �� 2024-09-10T10:50:38.110 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata ls 2024-09-10T10:50:38.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.221+0000 7f4ccdcacf40 1 Processor -- start 2024-09-10T10:50:38.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.221+0000 7f4ccdcacf40 1 -- start start 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 0x55c81b145820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1439c0 0x55c81b145d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c81b0b5530 con 0x55c81b0b48e0 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c81b0b52d0 con 0x55c81b1403e0 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c81b142ee0 con 0x55c81b1439c0 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44306/0 (socket says 172.21.15.28:44306) 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 -- 172.21.15.28:0/3738679505 learned_addr learned my addr 172.21.15.28:0/3738679505 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccb800640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1439c0 0x55c81b145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4cca400640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 0x55c81b145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 -- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1439c0 msgr2=0x55c81b145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1439c0 0x55c81b145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 -- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 msgr2=0x55c81b145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 0x55c81b145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4ccae00640 1 -- 172.21.15.28:0/3738679505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81af96b30 con 0x55c81b0b48e0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.222+0000 7f4cca400640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 0x55c81b145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccae00640 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009860 tx=0x7f4cbc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4faebc7372f2ec38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4cc9000640 1 -- 172.21.15.28:0/3738679505 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0088d0 con 0x55c81b0b48e0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4cc9000640 1 -- 172.21.15.28:0/3738679505 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc018070 con 0x55c81b0b48e0 2024-09-10T10:50:38.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4cc9000640 1 -- 172.21.15.28:0/3738679505 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc008cb0 con 0x55c81b0b48e0 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/3738679505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 msgr2=0x55c81b13f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009860 tx=0x7f4cbc005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/3738679505 shutdown_connections 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1439c0 0x55c81b145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1403e0 0x55c81b145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/3738679505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b13f700 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/3738679505 >> 172.21.15.28:0/3738679505 conn(0x55c81b0b90a0 msgr2=0x55c81b0b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.223+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/3738679505 shutdown_connections 2024-09-10T10:50:38.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.224+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/3738679505 wait complete. 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.224+0000 7f4ccdcacf40 1 Processor -- start 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.224+0000 7f4ccdcacf40 1 -- start start 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1403e0 0x55c81b1d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1439c0 0x55c81b140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c81b142750 con 0x55c81b0b48e0 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c81b142470 con 0x55c81b1439c0 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccdcacf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c81b09e570 con 0x55c81b1403e0 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44318/0 (socket says 172.21.15.28:44318) 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.225+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 learned_addr learned my addr 172.21.15.28:0/4063529714 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1403e0 0x55c81b1d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1439c0 0x55c81b140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1403e0 msgr2=0x55c81b1d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1403e0 0x55c81b1d1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1439c0 msgr2=0x55c81b140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1439c0 0x55c81b140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c81b09d1c0 con 0x55c81b0b48e0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c81b1403e0 0x55c81b1d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c81b1439c0 0x55c81b140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc015040 tx=0x7f4cbc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4cc3400640 1 -- 172.21.15.28:0/4063529714 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc027430 con 0x55c81b0b48e0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4cc3400640 1 -- 172.21.15.28:0/4063529714 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cbc0275f0 con 0x55c81b0b48e0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4cc3400640 1 -- 172.21.15.28:0/4063529714 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc0279e0 con 0x55c81b0b48e0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.226+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c81b09d4a0 con 0x55c81b0b48e0 2024-09-10T10:50:38.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.227+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c81b13c5c0 con 0x55c81b0b48e0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.228+0000 7f4cc3400640 1 -- 172.21.15.28:0/4063529714 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cbc030440 con 0x55c81b0b48e0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.229+0000 7f4cc3400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 0x7f4cac06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.229+0000 7f4cc3400640 1 -- 172.21.15.28:0/4063529714 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f4cbc08c080 con 0x55c81b0b48e0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.229+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c81b09de70 0x55c81b1d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.229+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.230+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 0x7f4cac06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.230+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c81b09de70 0x55c81b1d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.230+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c81b09de70 0x55c81b1d3c30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.230+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 0x7f4cac06ccd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55c81b1dc320 tx=0x7f4cb80058a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.230+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.231+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f4cc4059070 0x7f4cc4059960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.231+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4059070 2024-09-10T10:50:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.231+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f4cc4059070 0x7f4cc4059960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.232+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f4cc4059070 0x7f4cc4059960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.232+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(2 [pgnls start_epoch 100 out=177b] v99'10 uv10 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc4059070 2024-09-10T10:50:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.232+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f4cbc05a4c0 0x7f4cbc059120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.232+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc05a4c0 2024-09-10T10:50:38.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.233+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f4cbc05a4c0 0x7f4cbc059120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.233+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f4cbc05a4c0 0x7f4cbc059120 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.233+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.5 v2:172.21.15.138:6802/2049132176 1 ==== osd_op_reply(3 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc05a4c0 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.233+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.234+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(4 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.234+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f4cc4050070 0x7f4cc404f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.234+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4050070 2024-09-10T10:50:38.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.234+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f4cc4050070 0x7f4cc404f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.235+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f4cc4050070 0x7f4cc404f8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.235+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(5 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cc4050070 2024-09-10T10:50:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.235+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f4cc406a1b0 0x7f4cc4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.235+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc406a1b0 2024-09-10T10:50:38.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.235+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f4cc406a1b0 0x7f4cc4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.236+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f4cc406a1b0 0x7f4cc4069410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.236+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(6 [pgnls start_epoch 100 out=149b] v99'19 uv19 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc406a1b0 2024-09-10T10:50:38.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.236+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc406a1b0 2024-09-10T10:50:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.237+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(7 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc406a1b0 2024-09-10T10:50:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.237+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f4cbc0629b0 0x7f4cbc0607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.237+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc0629b0 2024-09-10T10:50:38.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.237+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f4cbc0629b0 0x7f4cbc0607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.238+0000 7f4cca400640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f4cbc0629b0 0x7f4cbc0607b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.238+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(8 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc0629b0 2024-09-10T10:50:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.238+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.238+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(9 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.238+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc406a1b0 2024-09-10T10:50:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.239+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(10 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc406a1b0 2024-09-10T10:50:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.239+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f4cbc027b80 0x7f4cbc058820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.239+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc027b80 2024-09-10T10:50:38.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.239+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f4cbc027b80 0x7f4cbc058820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.240+0000 7f4ccb800640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f4cbc027b80 0x7f4cbc058820 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.240+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.4 v2:172.21.15.138:6810/1083898337 1 ==== osd_op_reply(11 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cbc027b80 2024-09-10T10:50:38.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.240+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc05a4c0 2024-09-10T10:50:38.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.241+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.5 v2:172.21.15.138:6802/2049132176 2 ==== osd_op_reply(12 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc05a4c0 2024-09-10T10:50:38.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.241+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4050070 2024-09-10T10:50:38.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(13 [pgnls start_epoch 100 out=77b] v99'4 uv4 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cc4050070 2024-09-10T10:50:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4059070 2024-09-10T10:50:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(14 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc4059070 2024-09-10T10:50:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc0629b0 2024-09-10T10:50:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(15 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc0629b0 2024-09-10T10:50:38.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.242+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc0629b0 2024-09-10T10:50:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.243+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(16 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc0629b0 2024-09-10T10:50:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.243+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.243+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(17 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.243+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc406a1b0 2024-09-10T10:50:38.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.243+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(18 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc406a1b0 2024-09-10T10:50:38.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.244+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc0629b0 2024-09-10T10:50:38.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.244+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(19 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc0629b0 2024-09-10T10:50:38.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.244+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4059070 2024-09-10T10:50:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.245+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(20 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc4059070 2024-09-10T10:50:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.245+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f4cbc06e040 0x7f4cbc05abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.245+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc06e040 2024-09-10T10:50:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.245+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f4cbc06e040 0x7f4cbc05abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.245+0000 7f4ccae00640 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f4cbc06e040 0x7f4cbc05abf0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(21 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cbc06e040 2024-09-10T10:50:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4050070 2024-09-10T10:50:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(22 [pgnls start_epoch 100 out=97b] v98'3 uv3 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cc4050070 2024-09-10T10:50:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc0629b0 2024-09-10T10:50:38.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(23 [pgnls start_epoch 100 out=74b] v98'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4cb8013030 con 0x7f4cbc0629b0 2024-09-10T10:50:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.246+0000 7f4cca400640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.247+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(24 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.247+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc406a1b0 2024-09-10T10:50:38.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.247+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(25 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cc406a1b0 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.247+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(26 [pgnls start_epoch 100 out=98b] v99'3 uv3 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4050070 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(27 [pgnls start_epoch 100 out=76b] v99'3 uv3 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cc4050070 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc06e040 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(28 [pgnls start_epoch 100 out=49b] v98'1 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cbc02d020 con 0x7f4cbc06e040 2024-09-10T10:50:38.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.248+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cc4050070 2024-09-10T10:50:38.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(29 [pgnls start_epoch 100 out=73b] v98'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4cc4056030 con 0x7f4cc4050070 2024-09-10T10:50:38.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(30 [pgnls start_epoch 100 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x7f4cbc06e040 2024-09-10T10:50:38.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(31 [pgnls start_epoch 100 out=73b] v98'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x7f4cbc030d70 con 0x7f4cbc06e040 2024-09-10T10:50:38.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.249+0000 7f4ccae00640 1 -- 172.21.15.28:0/4063529714 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 100 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e100) -- 0x55c81b1d4170 con 0x55c81b09de70 2024-09-10T10:50:38.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(32 [pgnls start_epoch 100 out=101b] v99'6 uv6 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x7f4cc4056030 con 0x55c81b09de70 2024-09-10T10:50:38.251 INFO:teuthology.orchestra.run.smithi028.stdout:601.00000000 2024-09-10T10:50:38.251 INFO:teuthology.orchestra.run.smithi028.stdout:602.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:10000000000.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:600.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:603.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:1.00000000.inode 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:200.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:200.00000001 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:606.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:607.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:mds0_openfiles.0 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:608.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:604.00000000 2024-09-10T10:50:38.252 INFO:teuthology.orchestra.run.smithi028.stdout:500.00000000 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:mds_snaptable 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:605.00000000 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:mds0_inotable 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:100.00000000 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:mds0_sessionmap 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:609.00000000 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:400.00000000 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:100.00000000.inode 2024-09-10T10:50:38.253 INFO:teuthology.orchestra.run.smithi028.stdout:1.00000000 2024-09-10T10:50:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f4cbc0629b0 msgr2=0x7f4cbc0607b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f4cbc0629b0 0x7f4cbc0607b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c81b09de70 msgr2=0x55c81b1d3c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c81b09de70 0x55c81b1d3c30 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f4cbc06e040 msgr2=0x7f4cbc05abf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7f4cbc06e040 0x7f4cbc05abf0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f4cc4050070 msgr2=0x7f4cc404f8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f4cc4050070 0x7f4cc404f8c0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f4cbc027b80 msgr2=0x7f4cbc058820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7f4cbc027b80 0x7f4cbc058820 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f4cbc05a4c0 msgr2=0x7f4cbc059120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f4cbc05a4c0 0x7f4cbc059120 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f4cc406a1b0 msgr2=0x7f4cc4069410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f4cc406a1b0 0x7f4cc4069410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f4cc4059070 msgr2=0x7f4cc4059960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f4cc4059070 0x7f4cc4059960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 reap_dead start 2024-09-10T10:50:38.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 reap_dead start 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 reap_dead start 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccb800640 1 -- 172.21.15.28:0/4063529714 reap_dead start 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 msgr2=0x7f4cac06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 0x7f4cac06ccd0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55c81b1dc320 tx=0x7f4cb80058a0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 msgr2=0x55c81b136bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc015040 tx=0x7f4cbc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 shutdown_connections 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4cac06a810 0x7f4cac06ccd0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 --2- 172.21.15.28:0/4063529714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c81b0b48e0 0x55c81b136bc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 >> 172.21.15.28:0/4063529714 conn(0x55c81b0b90a0 msgr2=0x55c81b146a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 shutdown_connections 2024-09-10T10:50:38.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.250+0000 7f4ccdcacf40 1 -- 172.21.15.28:0/4063529714 wait complete. 2024-09-10T10:50:38.258 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 10000000000.00000000 2024-09-10T10:50:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.332+0000 7fe8f4d61f40 1 Processor -- start 2024-09-10T10:50:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.332+0000 7fe8f4d61f40 1 -- start start 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f17833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5582f16f7530 con 0x5582f16f68e0 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582f1654b30 con 0x5582f17859c0 2024-09-10T10:50:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5582f1784ee0 con 0x5582f17823e0 2024-09-10T10:50:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44338/0 (socket says 172.21.15.28:44338) 2024-09-10T10:50:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.333+0000 7fe8f2000640 1 -- 172.21.15.28:0/4012072080 learned_addr learned my addr 172.21.15.28:0/4012072080 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 -- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 msgr2=0x5582f16df060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f1600640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2a00640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f17833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 -- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 msgr2=0x5582f17833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f17833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 -- 172.21.15.28:0/4012072080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f1780e90 con 0x5582f16f68e0 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f1600640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2a00640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f17833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8f2000640 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7b57e5cfa2dacc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8ebe00640 1 -- 172.21.15.28:0/4012072080 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4053940 con 0x5582f16f68e0 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8ebe00640 1 -- 172.21.15.28:0/4012072080 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e4062070 con 0x5582f16f68e0 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.334+0000 7fe8ebe00640 1 -- 172.21.15.28:0/4012072080 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4053d20 con 0x5582f16f68e0 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/4012072080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 msgr2=0x5582f177d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4000c00 tx=0x7fe8e4051000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/4012072080 shutdown_connections 2024-09-10T10:50:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f17833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/4012072080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f16f68e0 0x5582f177d410 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/4012072080 >> 172.21.15.28:0/4012072080 conn(0x5582f16fb0a0 msgr2=0x5582f1782850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/4012072080 shutdown_connections 2024-09-10T10:50:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.335+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/4012072080 wait complete. 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.336+0000 7fe8f4d61f40 1 Processor -- start 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.336+0000 7fe8f4d61f40 1 -- start start 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f16f68e0 0x5582f17798d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f16df5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5582f16f99d0 con 0x5582f17823e0 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582f16f9d20 con 0x5582f17859c0 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f4d61f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5582f16ac430 con 0x5582f16f68e0 2024-09-10T10:50:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 0x5582f16df060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44346/0 (socket says 172.21.15.28:44346) 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 -- 172.21.15.28:0/2076377750 learned_addr learned my addr 172.21.15.28:0/2076377750 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f2000640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f16f68e0 0x5582f17798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f2a00640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f16df5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 -- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f16f68e0 msgr2=0x5582f17798d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582f16f68e0 0x5582f17798d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 -- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 msgr2=0x5582f16df5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582f17859c0 0x5582f16df5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.337+0000 7fe8f1600640 1 -- 172.21.15.28:0/2076377750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582f177ac40 con 0x5582f17823e0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8f1600640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 0x5582f16df060 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000be00 tx=0x7fe8e000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8eaa00640 1 -- 172.21.15.28:0/2076377750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0019070 con 0x5582f17823e0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8eaa00640 1 -- 172.21.15.28:0/2076377750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e0007a70 con 0x5582f17823e0 2024-09-10T10:50:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5582f177af20 con 0x5582f17823e0 2024-09-10T10:50:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8eaa00640 1 -- 172.21.15.28:0/2076377750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0007e20 con 0x5582f17823e0 2024-09-10T10:50:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.338+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582f177b4e0 con 0x5582f17823e0 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.340+0000 7fe8eaa00640 1 -- 172.21.15.28:0/2076377750 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8e001c070 con 0x5582f17823e0 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.341+0000 7fe8eaa00640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8d406a840 0x7fe8d406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.341+0000 7fe8eaa00640 1 -- 172.21.15.28:0/2076377750 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fe8e0088c60 con 0x5582f17823e0 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.341+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5582f177a0c0 0x5582f177c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.341+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5582f177cbb0 con 0x5582f177a0c0 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.341+0000 7fe8f2000640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8d406a840 0x7fe8d406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.342+0000 7fe8f2a00640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5582f177a0c0 0x5582f177c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.342+0000 7fe8f2000640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8d406a840 0x7fe8d406cd00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5582f15d22b0 tx=0x7fe8e4053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.342+0000 7fe8f2a00640 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5582f177a0c0 0x5582f177c670 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f2a00640 1 -- 172.21.15.28:0/2076377750 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x7fe8ec0016a0 con 0x5582f177a0c0 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stdout:header (274 bytes) : 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 04 03 0c 01 00 00 0b 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T10:50:38.344 INFO:teuthology.orchestra.run.smithi028.stdout:00000020 00 00 00 00 7b 24 e0 66 a1 7d 3b 39 01 00 00 00 |....{$.f.};9....| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000030 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000050 00 00 7b 24 e0 66 a1 7d 3b 39 01 00 00 00 00 00 |..{$.f.};9......| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000060 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000a0 00 00 00 00 00 00 00 00 7b 24 e0 66 d5 ca 60 3b |........{$.f..`;| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |`...............| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000e0 00 00 00 00 00 00 7b 24 e0 66 d5 ca 60 3b 00 00 |......{$.f..`;..| 2024-09-10T10:50:38.345 INFO:teuthology.orchestra.run.smithi028.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.346 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:38.346 INFO:teuthology.orchestra.run.smithi028.stdout:00000110 00 00 |..| 2024-09-10T10:50:38.346 INFO:teuthology.orchestra.run.smithi028.stdout:00000112 2024-09-10T10:50:38.346 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5582f177a0c0 msgr2=0x5582f177c670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5582f177a0c0 0x5582f177c670 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8d406a840 msgr2=0x7fe8d406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8d406a840 0x7fe8d406cd00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5582f15d22b0 tx=0x7fe8e4053000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 msgr2=0x5582f16df060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 --2- 172.21.15.28:0/2076377750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582f17823e0 0x5582f16df060 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000be00 tx=0x7fe8e000bef0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f2a00640 1 -- 172.21.15.28:0/2076377750 reap_dead start 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 shutdown_connections 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 >> 172.21.15.28:0/2076377750 conn(0x5582f16fb0a0 msgr2=0x5582f1782850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 shutdown_connections 2024-09-10T10:50:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.343+0000 7fe8f4d61f40 1 -- 172.21.15.28:0/2076377750 wait complete. 2024-09-10T10:50:38.349 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 600.00000000 2024-09-10T10:50:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.423+0000 7f3d0b693f40 1 Processor -- start 2024-09-10T10:50:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.423+0000 7f3d0b693f40 1 -- start start 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d49b8e0 0x55677d450d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d52a9c0 0x55677d49df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55677d49c530 con 0x55677d5273e0 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55677d3f9b30 con 0x55677d52a9c0 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55677d529ee0 con 0x55677d49b8e0 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44364/0 (socket says 172.21.15.28:44364) 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d07e00640 1 -- 172.21.15.28:0/2907830718 learned_addr learned my addr 172.21.15.28:0/2907830718 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.424+0000 7f3d08800640 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d49b8e0 0x55677d450d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d09200640 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d52a9c0 0x55677d49df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 -- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d49b8e0 msgr2=0x55677d450d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d49b8e0 0x55677d450d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 -- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d52a9c0 msgr2=0x55677d49df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d52a9c0 0x55677d49df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 -- 172.21.15.28:0/2907830718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677d510390 con 0x55677d5273e0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400a740 tx=0x7f3cf400a830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196095a9c2819cd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d06a00640 1 -- 172.21.15.28:0/2907830718 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4019070 con 0x55677d5273e0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d06a00640 1 -- 172.21.15.28:0/2907830718 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf4007600 con 0x55677d5273e0 2024-09-10T10:50:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.425+0000 7f3d06a00640 1 -- 172.21.15.28:0/2907830718 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40079f0 con 0x55677d5273e0 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2907830718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 msgr2=0x55677d49d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400a740 tx=0x7f3cf400a830 comp rx=0 tx=0).stop 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2907830718 shutdown_connections 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d52a9c0 0x55677d49df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d5273e0 0x55677d49d9c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2907830718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d49b8e0 0x55677d450d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2907830718 >> 172.21.15.28:0/2907830718 conn(0x55677d4a00a0 msgr2=0x55677d527850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2907830718 shutdown_connections 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.426+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2907830718 wait complete. 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.427+0000 7f3d0b693f40 1 Processor -- start 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.427+0000 7f3d0b693f40 1 -- start start 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.427+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 0x55677d50f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d5273e0 0x55677d499d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d0b693f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d52a9c0 0x55677d528570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55677d525c00 con 0x55677d49b8e0 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55677d4a0970 con 0x55677d5273e0 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d0b693f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55677d4a0690 con 0x55677d52a9c0 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 0x55677d50f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 0x55677d50f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44378/0 (socket says 172.21.15.28:44378) 2024-09-10T10:50:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 -- 172.21.15.28:0/2258710774 learned_addr learned my addr 172.21.15.28:0/2258710774 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d09200640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d52a9c0 0x55677d528570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d5273e0 0x55677d499d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 -- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d52a9c0 msgr2=0x55677d528570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d52a9c0 0x55677d528570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 -- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d5273e0 msgr2=0x55677d499d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d5273e0 0x55677d499d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d08800640 1 -- 172.21.15.28:0/2258710774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55677d49a290 con 0x55677d49b8e0 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d09200640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55677d52a9c0 0x55677d528570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.428+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55677d5273e0 0x55677d499d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d08800640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 0x55677d50f470 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc055040 tx=0x7f3cfc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d05600640 1 -- 172.21.15.28:0/2258710774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc06c070 con 0x55677d49b8e0 2024-09-10T10:50:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d05600640 1 -- 172.21.15.28:0/2258710774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cfc05fa70 con 0x55677d49b8e0 2024-09-10T10:50:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d05600640 1 -- 172.21.15.28:0/2258710774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc05fe20 con 0x55677d49b8e0 2024-09-10T10:50:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55677d498780 con 0x55677d49b8e0 2024-09-10T10:50:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.429+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55677d498d40 con 0x55677d49b8e0 2024-09-10T10:50:38.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.431+0000 7f3d05600640 1 -- 172.21.15.28:0/2258710774 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3cfc06f070 con 0x55677d49b8e0 2024-09-10T10:50:38.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d05600640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ce806a810 0x7f3ce806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d05600640 1 -- 172.21.15.28:0/2258710774 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f3cfc0cdc60 con 0x55677d49b8e0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55677d52cfc0 0x55677d52d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55677d52d8e0 con 0x55677d52cfc0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ce806a810 0x7f3ce806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.432+0000 7f3d09200640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55677d52cfc0 0x55677d52d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d07e00640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ce806a810 0x7f3ce806ccd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55677d5284a0 tx=0x7f3cf400c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d09200640 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55677d52cfc0 0x55677d52d3a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d09200640 1 -- 172.21.15.28:0/2258710774 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 600.00000000 [omap-get-header out=274b] v0'0 uv2 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f3d000044c0 con 0x55677d52cfc0 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stdout:header (274 bytes) : 2024-09-10T10:50:38.435 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:38.436 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:38.437 INFO:teuthology.orchestra.run.smithi028.stdout:00000110 00 00 |..| 2024-09-10T10:50:38.437 INFO:teuthology.orchestra.run.smithi028.stdout:00000112 2024-09-10T10:50:38.437 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55677d52cfc0 msgr2=0x55677d52d3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55677d52cfc0 0x55677d52d3a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ce806a810 msgr2=0x7f3ce806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.433+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ce806a810 0x7f3ce806ccd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55677d5284a0 tx=0x7f3cf400c000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 msgr2=0x55677d50f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 --2- 172.21.15.28:0/2258710774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55677d49b8e0 0x55677d50f470 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc055040 tx=0x7f3cfc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d09200640 1 -- 172.21.15.28:0/2258710774 reap_dead start 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 shutdown_connections 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 >> 172.21.15.28:0/2258710774 conn(0x55677d4a00a0 msgr2=0x55677d49d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 shutdown_connections 2024-09-10T10:50:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.434+0000 7f3d0b693f40 1 -- 172.21.15.28:0/2258710774 wait complete. 2024-09-10T10:50:38.439 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000.inode 2024-09-10T10:50:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 Processor -- start 2024-09-10T10:50:38.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 -- start start 2024-09-10T10:50:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.514+0000 7f614a864f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568a5843dc0 con 0x5568a57fe5a0 2024-09-10T10:50:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f614a864f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568a57a2b30 con 0x5568a58d04b0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f614a864f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568a58431f0 con 0x5568a58442d0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44406/0 (socket says 172.21.15.28:44406) 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 -- 172.21.15.28:0/3151634171 learned_addr learned my addr 172.21.15.28:0/3151634171 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6148400640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147000640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 -- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 msgr2=0x5568a58d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 -- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 msgr2=0x5568a58d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 -- 172.21.15.28:0/3151634171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a58264c0 con 0x5568a57fe5a0 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6148400640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.515+0000 7f6147a00640 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f61380098b0 tx=0x7f6138005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cce35d4f9283fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f6145c00640 1 -- 172.21.15.28:0/3151634171 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138008930 con 0x5568a57fe5a0 2024-09-10T10:50:38.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f6145c00640 1 -- 172.21.15.28:0/3151634171 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6138018070 con 0x5568a57fe5a0 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f6145c00640 1 -- 172.21.15.28:0/3151634171 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138008cd0 con 0x5568a57fe5a0 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 -- 172.21.15.28:0/3151634171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 msgr2=0x5568a5844930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f61380098b0 tx=0x7f6138005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 -- 172.21.15.28:0/3151634171 shutdown_connections 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 --2- 172.21.15.28:0/3151634171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a5844930 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 -- 172.21.15.28:0/3151634171 >> 172.21.15.28:0/3151634171 conn(0x5568a5828b90 msgr2=0x5568a5828f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 -- 172.21.15.28:0/3151634171 shutdown_connections 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.516+0000 7f614a864f40 1 -- 172.21.15.28:0/3151634171 wait complete. 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.517+0000 7f614a864f40 1 Processor -- start 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.517+0000 7f614a864f40 1 -- start start 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a58d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568a58cf290 con 0x5568a57fe5a0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568a58cefb0 con 0x5568a58d04b0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f614a864f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568a58d2ec0 con 0x5568a58442d0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f6147a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a58d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f6147a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a58d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44412/0 (socket says 172.21.15.28:44412) 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.518+0000 7f6147a00640 1 -- 172.21.15.28:0/2045511424 learned_addr learned my addr 172.21.15.28:0/2045511424 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147000640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6148400640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 -- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 msgr2=0x5568a58c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568a58442d0 0x5568a58c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 -- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 msgr2=0x5568a58d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568a58d04b0 0x5568a58d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 -- 172.21.15.28:0/2045511424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568a58c4cf0 con 0x5568a57fe5a0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f6147a00640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a58d12d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6138000c00 tx=0x7f6138003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f613fe00640 1 -- 172.21.15.28:0/2045511424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138013830 con 0x5568a57fe5a0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f613fe00640 1 -- 172.21.15.28:0/2045511424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6138022070 con 0x5568a57fe5a0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568a58d1810 con 0x5568a57fe5a0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.519+0000 7f613fe00640 1 -- 172.21.15.28:0/2045511424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138013bd0 con 0x5568a57fe5a0 2024-09-10T10:50:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.520+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568a58c4360 con 0x5568a57fe5a0 2024-09-10T10:50:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.521+0000 7f613fe00640 1 -- 172.21.15.28:0/2045511424 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f613801d410 con 0x5568a57fe5a0 2024-09-10T10:50:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.522+0000 7f613fe00640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f612806a810 0x7f612806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.522+0000 7f613fe00640 1 -- 172.21.15.28:0/2045511424 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f613808c040 con 0x5568a57fe5a0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.522+0000 7f614a864f40 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568a58d27f0 0x5568a58c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.522+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5568a58c6730 con 0x5568a58d27f0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.522+0000 7f6147000640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f612806a810 0x7f612806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.523+0000 7f6148400640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568a58d27f0 0x5568a58c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.523+0000 7f6147000640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f612806a810 0x7f612806ccd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5568a5829320 tx=0x7f6134003050 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.523+0000 7f6148400640 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568a58d27f0 0x5568a58c61f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f6148400640 1 -- 172.21.15.28:0/2045511424 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-header] v0'0 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f614004f990 con 0x5568a58d27f0 2024-09-10T10:50:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:38.526 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568a58d27f0 msgr2=0x5568a58c61f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568a58d27f0 0x5568a58c61f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f612806a810 msgr2=0x7f612806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f612806a810 0x7f612806ccd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5568a5829320 tx=0x7f6134003050 comp rx=0 tx=0).stop 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 msgr2=0x5568a58d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 --2- 172.21.15.28:0/2045511424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568a57fe5a0 0x5568a58d12d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6138000c00 tx=0x7f6138003020 comp rx=0 tx=0).stop 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f6148400640 1 -- 172.21.15.28:0/2045511424 reap_dead start 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 shutdown_connections 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 >> 172.21.15.28:0/2045511424 conn(0x5568a5828b90 msgr2=0x5568a5844930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 shutdown_connections 2024-09-10T10:50:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.524+0000 7f614a864f40 1 -- 172.21.15.28:0/2045511424 wait complete. 2024-09-10T10:50:38.529 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000000 2024-09-10T10:50:38.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.603+0000 7fec3651ef40 1 Processor -- start 2024-09-10T10:50:38.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.603+0000 7fec3651ef40 1 -- start start 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 0x55c5d1d883d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1cf74b0 0x55c5d1c67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c5d1c6adc0 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5d1bc9b30 con 0x55c5d1c25cb0 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec3651ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c5d1c6a1f0 con 0x55c5d1cf74b0 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec32400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec32400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44438/0 (socket says 172.21.15.28:44438) 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.604+0000 7fec32400640 1 -- 172.21.15.28:0/2406175243 learned_addr learned my addr 172.21.15.28:0/2406175243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32e00640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 0x55c5d1d883d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec33800640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1cf74b0 0x55c5d1c67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 -- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1cf74b0 msgr2=0x55c5d1c67210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1cf74b0 0x55c5d1c67210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 -- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 msgr2=0x55c5d1d883d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 0x55c5d1d883d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 -- 172.21.15.28:0/2406175243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d1cf8fd0 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32e00640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 0x55c5d1d883d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec32400640 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fec200097b0 tx=0x7fec20005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4c359e6d79630f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec31000640 1 -- 172.21.15.28:0/2406175243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20019070 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec31000640 1 -- 172.21.15.28:0/2406175243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20008890 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.605+0000 7fec31000640 1 -- 172.21.15.28:0/2406175243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20008c60 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 -- 172.21.15.28:0/2406175243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 msgr2=0x55c5d1cf5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fec200097b0 tx=0x7fec20005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 -- 172.21.15.28:0/2406175243 shutdown_connections 2024-09-10T10:50:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1cf74b0 0x55c5d1c67210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf5a50 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2406175243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c25cb0 0x55c5d1d883d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 -- 172.21.15.28:0/2406175243 >> 172.21.15.28:0/2406175243 conn(0x55c5d1c4fb90 msgr2=0x55c5d1c6b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 -- 172.21.15.28:0/2406175243 shutdown_connections 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.606+0000 7fec3651ef40 1 -- 172.21.15.28:0/2406175243 wait complete. 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.607+0000 7fec3651ef40 1 Processor -- start 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.607+0000 7fec3651ef40 1 -- start start 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1c25cb0 0x55c5d1d8b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 0x55c5d1ced420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c5d1cf69e0 con 0x55c5d1cf74b0 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5d1cf6370 con 0x55c5d1c6b2d0 2024-09-10T10:50:38.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec3651ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c5d1cf5f90 con 0x55c5d1c25cb0 2024-09-10T10:50:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec33800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 0x55c5d1ced420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec33800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 0x55c5d1ced420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44440/0 (socket says 172.21.15.28:44440) 2024-09-10T10:50:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec33800640 1 -- 172.21.15.28:0/2042592830 learned_addr learned my addr 172.21.15.28:0/2042592830 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec32400640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec32e00640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1c25cb0 0x55c5d1d8b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.608+0000 7fec33800640 1 -- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1c25cb0 msgr2=0x55c5d1d8b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec33800640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5d1c25cb0 0x55c5d1d8b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec33800640 1 -- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c6b2d0 msgr2=0x55c5d1cf9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec33800640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec33800640 1 -- 172.21.15.28:0/2042592830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5d1d8aa00 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec32400640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5d1c6b2d0 0x55c5d1cf9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec33800640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 0x55c5d1ced420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fec2c007000 tx=0x7fec2c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec2b400640 1 -- 172.21.15.28:0/2042592830 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c005b40 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec2b400640 1 -- 172.21.15.28:0/2042592830 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2c067070 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c5d1c6bc20 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec2b400640 1 -- 172.21.15.28:0/2042592830 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c062640 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.609+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5d1c6bf00 con 0x55c5d1cf74b0 2024-09-10T10:50:38.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.611+0000 7fec2b400640 1 -- 172.21.15.28:0/2042592830 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec2c061070 con 0x55c5d1cf74b0 2024-09-10T10:50:38.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.612+0000 7fec2b400640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a840 0x7fec1406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.612+0000 7fec2b400640 1 -- 172.21.15.28:0/2042592830 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fec2c0cd8d0 con 0x55c5d1cf74b0 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.612+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c5d1cf9e80 0x55c5d1cf4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.612+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55c5d1cf2070 con 0x55c5d1cf9e80 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.612+0000 7fec32e00640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a840 0x7fec1406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.613+0000 7fec32400640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c5d1cf9e80 0x55c5d1cf4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.613+0000 7fec32e00640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a840 0x7fec1406cd00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec24003f10 tx=0x7fec24003870 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.613+0000 7fec32400640 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c5d1cf9e80 0x55c5d1cf4260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec32400640 1 -- 172.21.15.28:0/2042592830 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [omap-get-header] v0'0 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fec2001c040 con 0x55c5d1cf9e80 2024-09-10T10:50:38.615 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:38.616 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c5d1cf9e80 msgr2=0x55c5d1cf4260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c5d1cf9e80 0x55c5d1cf4260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a840 msgr2=0x7fec1406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a840 0x7fec1406cd00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fec24003f10 tx=0x7fec24003870 comp rx=0 tx=0).stop 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 msgr2=0x55c5d1ced420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 --2- 172.21.15.28:0/2042592830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c5d1cf74b0 0x55c5d1ced420 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fec2c007000 tx=0x7fec2c003330 comp rx=0 tx=0).stop 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec33800640 1 -- 172.21.15.28:0/2042592830 reap_dead start 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 shutdown_connections 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 >> 172.21.15.28:0/2042592830 conn(0x55c5d1c4fb90 msgr2=0x55c5d1c4ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 shutdown_connections 2024-09-10T10:50:38.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.614+0000 7fec3651ef40 1 -- 172.21.15.28:0/2042592830 wait complete. 2024-09-10T10:50:38.619 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 200.00000001 2024-09-10T10:50:38.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.694+0000 7f26c698df40 1 Processor -- start 2024-09-10T10:50:38.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.694+0000 7f26c698df40 1 -- start start 2024-09-10T10:50:38.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad5fdd470 0x55aad60b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b69c0 0x55aad60b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55aad6028530 con 0x55aad60b33e0 2024-09-10T10:50:38.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aad5f85b30 con 0x55aad5fdd470 2024-09-10T10:50:38.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c698df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55aad60b5ee0 con 0x55aad60b69c0 2024-09-10T10:50:38.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44450/0 (socket says 172.21.15.28:44450) 2024-09-10T10:50:38.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.695+0000 7f26c3200640 1 -- 172.21.15.28:0/1193915520 learned_addr learned my addr 172.21.15.28:0/1193915520 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3c00640 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad5fdd470 0x55aad60b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c4600640 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b69c0 0x55aad60b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 -- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b69c0 msgr2=0x55aad60b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b69c0 0x55aad60b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 -- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad5fdd470 msgr2=0x55aad60b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad5fdd470 0x55aad60b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 -- 172.21.15.28:0/1193915520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad600d8e0 con 0x55aad60b33e0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c3200640 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f26b00099d0 tx=0x7f26b0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f514204a245b00ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c1e00640 1 -- 172.21.15.28:0/1193915520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0008a30 con 0x55aad60b33e0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c1e00640 1 -- 172.21.15.28:0/1193915520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b0018070 con 0x55aad60b33e0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.696+0000 7f26c1e00640 1 -- 172.21.15.28:0/1193915520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0008dd0 con 0x55aad60b33e0 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 -- 172.21.15.28:0/1193915520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 msgr2=0x55aad60b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f26b00099d0 tx=0x7f26b0005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 -- 172.21.15.28:0/1193915520 shutdown_connections 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b69c0 0x55aad60b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad60b33e0 0x55aad60b55d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 --2- 172.21.15.28:0/1193915520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad5fdd470 0x55aad60b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 -- 172.21.15.28:0/1193915520 >> 172.21.15.28:0/1193915520 conn(0x55aad602c0a0 msgr2=0x55aad602c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 -- 172.21.15.28:0/1193915520 shutdown_connections 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.697+0000 7f26c698df40 1 -- 172.21.15.28:0/1193915520 wait complete. 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.698+0000 7f26c698df40 1 Processor -- start 2024-09-10T10:50:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.698+0000 7f26c698df40 1 -- start start 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.698+0000 7f26c698df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 0x55aad60b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c698df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b33e0 0x55aad6025a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c698df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad60b69c0 0x55aad60b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c698df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55aad600cf40 con 0x55aad5fdd470 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c698df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aad600d600 con 0x55aad60b69c0 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c698df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55aad60acbd0 con 0x55aad60b33e0 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 0x55aad60b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 0x55aad60b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44452/0 (socket says 172.21.15.28:44452) 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 -- 172.21.15.28:0/1255797169 learned_addr learned my addr 172.21.15.28:0/1255797169 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3200640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b33e0 0x55aad6025a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c4600640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad60b69c0 0x55aad60b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 -- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b33e0 msgr2=0x55aad6025a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aad60b33e0 0x55aad6025a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 -- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad60b69c0 msgr2=0x55aad60b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad60b69c0 0x55aad60b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 -- 172.21.15.28:0/1255797169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad60b8fb0 con 0x55aad5fdd470 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c4600640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aad60b69c0 0x55aad60b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.699+0000 7f26c3c00640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 0x55aad60b4440 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f26b4001c70 tx=0x7f26b400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.700+0000 7f26c0a00640 1 -- 172.21.15.28:0/1255797169 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b400f650 con 0x55aad5fdd470 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.700+0000 7f26c0a00640 1 -- 172.21.15.28:0/1255797169 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b400f7f0 con 0x55aad5fdd470 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.700+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aad60b9290 con 0x55aad5fdd470 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.700+0000 7f26c0a00640 1 -- 172.21.15.28:0/1255797169 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b400fbe0 con 0x55aad5fdd470 2024-09-10T10:50:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.700+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad60ac390 con 0x55aad5fdd470 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.702+0000 7f26c0a00640 1 -- 172.21.15.28:0/1255797169 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f26b4017400 con 0x55aad5fdd470 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.702+0000 7f26c0a00640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a406a810 0x7f26a406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.703+0000 7f26c0a00640 1 -- 172.21.15.28:0/1255797169 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f26b408a430 con 0x55aad5fdd470 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.703+0000 7f26c698df40 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55aad600ec90 0x55aad60ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.703+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55aad6025f80 con 0x55aad600ec90 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.703+0000 7f26c3200640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a406a810 0x7f26a406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.703+0000 7f26c4600640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55aad600ec90 0x55aad60ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c3200640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a406a810 0x7f26a406ccd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55aad60b2a50 tx=0x7f26b0012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c4600640 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55aad600ec90 0x55aad60ae640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c4600640 1 -- 172.21.15.28:0/1255797169 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [omap-get-header] v0'0 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f26bc04e040 con 0x55aad600ec90 2024-09-10T10:50:38.706 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:38.706 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55aad600ec90 msgr2=0x55aad60ae640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55aad600ec90 0x55aad60ae640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a406a810 msgr2=0x7f26a406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f26a406a810 0x7f26a406ccd0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55aad60b2a50 tx=0x7f26b0012000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 msgr2=0x55aad60b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c698df40 1 --2- 172.21.15.28:0/1255797169 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aad5fdd470 0x55aad60b4440 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f26b4001c70 tx=0x7f26b400b6c0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.704+0000 7f26c4600640 1 -- 172.21.15.28:0/1255797169 reap_dead start 2024-09-10T10:50:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.705+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 shutdown_connections 2024-09-10T10:50:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.705+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 >> 172.21.15.28:0/1255797169 conn(0x55aad602c0a0 msgr2=0x55aad602c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.705+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 shutdown_connections 2024-09-10T10:50:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.705+0000 7f26c698df40 1 -- 172.21.15.28:0/1255797169 wait complete. 2024-09-10T10:50:38.710 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_openfiles.0 2024-09-10T10:50:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.784+0000 7ff4c935ff40 1 Processor -- start 2024-09-10T10:50:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.784+0000 7ff4c935ff40 1 -- start start 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aacd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55820aa4b530 con 0x55820aad63e0 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55820a9a8b30 con 0x55820aa4aff0 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55820aad8ee0 con 0x55820aad99c0 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44466/0 (socket says 172.21.15.28:44466) 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.785+0000 7ff4c5c00640 1 -- 172.21.15.28:0/1192529221 learned_addr learned my addr 172.21.15.28:0/1192529221 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c7000640 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c6600640 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aacd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 -- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 msgr2=0x55820aacd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 -- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 msgr2=0x55820aacd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aacd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 -- 172.21.15.28:0/1192529221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820aad0be0 con 0x55820aad63e0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4001ea0 tx=0x7ff4b400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16317ef9be7d096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4bfe00640 1 -- 172.21.15.28:0/1192529221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b4008070 con 0x55820aad63e0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4bfe00640 1 -- 172.21.15.28:0/1192529221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b400d500 con 0x55820aad63e0 2024-09-10T10:50:38.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.786+0000 7ff4bfe00640 1 -- 172.21.15.28:0/1192529221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b400d950 con 0x55820aad63e0 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 -- 172.21.15.28:0/1192529221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 msgr2=0x55820aa33850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff4b4001ea0 tx=0x7ff4b400b6a0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 -- 172.21.15.28:0/1192529221 shutdown_connections 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aa33850 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/1192529221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aacd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 -- 172.21.15.28:0/1192529221 >> 172.21.15.28:0/1192529221 conn(0x55820aa4f0a0 msgr2=0x55820aad6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 -- 172.21.15.28:0/1192529221 shutdown_connections 2024-09-10T10:50:38.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.787+0000 7ff4c935ff40 1 -- 172.21.15.28:0/1192529221 wait complete. 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.788+0000 7ff4c935ff40 1 Processor -- start 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.788+0000 7ff4c935ff40 1 -- start start 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aadb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aad57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55820aa327b0 con 0x55820aad63e0 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55820aa321f0 con 0x55820aa4aff0 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c935ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55820aad84f0 con 0x55820aad99c0 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aad57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aad57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44470/0 (socket says 172.21.15.28:44470) 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 -- 172.21.15.28:0/940228034 learned_addr learned my addr 172.21.15.28:0/940228034 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c7000640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c6600640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aadb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 -- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 msgr2=0x55820aacdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55820aad99c0 0x55820aacdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 -- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 msgr2=0x55820aadb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aadb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c5c00640 1 -- 172.21.15.28:0/940228034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820aac9110 con 0x55820aad63e0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.789+0000 7ff4c6600640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55820aa4aff0 0x55820aadb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4c5c00640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aad57a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55820a926260 tx=0x7ff4b4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4bea00640 1 -- 172.21.15.28:0/940228034 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401b850 con 0x55820aad63e0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4bea00640 1 -- 172.21.15.28:0/940228034 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b4020030 con 0x55820aad63e0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4bea00640 1 -- 172.21.15.28:0/940228034 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b401bc10 con 0x55820aad63e0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55820aa00010 con 0x55820aad63e0 2024-09-10T10:50:38.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.790+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820aace180 con 0x55820aad63e0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.792+0000 7ff4bea00640 1 -- 172.21.15.28:0/940228034 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff4b4017040 con 0x55820aad63e0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4bea00640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a806a840 0x7ff4a806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4bea00640 1 -- 172.21.15.28:0/940228034 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff4b408bc80 con 0x55820aad63e0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55820aac93f0 0x55820aac88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55820aac9a10 con 0x55820aac93f0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4c6600640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a806a840 0x7ff4a806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.793+0000 7ff4c7000640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55820aac93f0 0x55820aac88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c6600640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a806a840 0x7ff4a806cd00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8057000 tx=0x7ff4b8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c7000640 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55820aac93f0 0x55820aac88c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c7000640 1 -- 172.21.15.28:0/940228034 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-header out=42b] v0'0 uv4 ondisk = 0) ==== 160+0+42 (crc 0 0 0) 0x7ff4c000e960 con 0x55820aac93f0 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout:header (42 bytes) : 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 13 00 00 00 63 65 70 68 20 66 73 20 76 6f 6c 75 |....ceph fs volu| 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout:00000010 6d 65 20 76 30 31 31 01 01 0d 00 00 00 04 00 00 |me v011.........| 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout:00000020 00 00 00 00 00 01 00 00 00 00 |..........| 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout:0000002a 2024-09-10T10:50:38.796 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55820aac93f0 msgr2=0x55820aac88c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55820aac93f0 0x55820aac88c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a806a840 msgr2=0x7ff4a806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff4a806a840 0x7ff4a806cd00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8057000 tx=0x7ff4b8052000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 msgr2=0x55820aad57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 --2- 172.21.15.28:0/940228034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55820aad63e0 0x55820aad57a0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55820a926260 tx=0x7ff4b4000ed0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c7000640 1 -- 172.21.15.28:0/940228034 reap_dead start 2024-09-10T10:50:38.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.794+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 shutdown_connections 2024-09-10T10:50:38.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.795+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 >> 172.21.15.28:0/940228034 conn(0x55820aa4f0a0 msgr2=0x55820aad76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.795+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 shutdown_connections 2024-09-10T10:50:38.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.795+0000 7ff4c935ff40 1 -- 172.21.15.28:0/940228034 wait complete. 2024-09-10T10:50:38.799 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 500.00000000 2024-09-10T10:50:38.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.875+0000 7f7ad120ff40 1 Processor -- start 2024-09-10T10:50:38.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.875+0000 7f7ad120ff40 1 -- start start 2024-09-10T10:50:38.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c779d5a0 0x55e0c7901580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c77e32d0 0x55e0c786d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0c77e2dc0 con 0x55e0c786f4b0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0c7741b30 con 0x55e0c779d5a0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e0c77e21f0 con 0x55e0c77e32d0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44476/0 (socket says 172.21.15.28:44476) 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 -- 172.21.15.28:0/460430318 learned_addr learned my addr 172.21.15.28:0/460430318 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7acd800640 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c779d5a0 0x55e0c7901580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7acce00640 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c77e32d0 0x55e0c786d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 -- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c77e32d0 msgr2=0x55e0c786d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c77e32d0 0x55e0c786d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 -- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c779d5a0 msgr2=0x55e0c7901580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c779d5a0 0x55e0c7901580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.876+0000 7f7ace200640 1 -- 172.21.15.28:0/460430318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c786de70 con 0x55e0c786f4b0 2024-09-10T10:50:38.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ace200640 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800a4b0 tx=0x7f7ac800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdc75b645f72f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ac7400640 1 -- 172.21.15.28:0/460430318 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8015070 con 0x55e0c786f4b0 2024-09-10T10:50:38.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ac7400640 1 -- 172.21.15.28:0/460430318 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac800e730 con 0x55e0c786f4b0 2024-09-10T10:50:38.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ac7400640 1 -- 172.21.15.28:0/460430318 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800eb20 con 0x55e0c786f4b0 2024-09-10T10:50:38.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ad120ff40 1 -- 172.21.15.28:0/460430318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 msgr2=0x55e0c7872470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800a4b0 tx=0x7f7ac800a970 comp rx=0 tx=0).stop 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.877+0000 7f7ad120ff40 1 -- 172.21.15.28:0/460430318 shutdown_connections 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c7872470 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c77e32d0 0x55e0c786d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/460430318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c779d5a0 0x55e0c7901580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 -- 172.21.15.28:0/460430318 >> 172.21.15.28:0/460430318 conn(0x55e0c77c7b90 msgr2=0x55e0c77c7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 -- 172.21.15.28:0/460430318 shutdown_connections 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 -- 172.21.15.28:0/460430318 wait complete. 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 Processor -- start 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.878+0000 7f7ad120ff40 1 -- start start 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c779d5a0 0x55e0c7872860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c77e32d0 0x55e0c7871650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c790b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0c77e12d0 con 0x55e0c786f4b0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0c786e120 con 0x55e0c77e32d0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ad120ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e0c77e0ff0 con 0x55e0c779d5a0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ace200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c790b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.879+0000 7f7ace200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c790b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44480/0 (socket says 172.21.15.28:44480) 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 -- 172.21.15.28:0/459549049 learned_addr learned my addr 172.21.15.28:0/459549049 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7acd800640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c779d5a0 0x55e0c7872860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7acce00640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c77e32d0 0x55e0c7871650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 -- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c779d5a0 msgr2=0x55e0c7872860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e0c779d5a0 0x55e0c7872860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 -- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c77e32d0 msgr2=0x55e0c7871650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0c77e32d0 0x55e0c7871650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 -- 172.21.15.28:0/459549049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c7872da0 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ace200640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c790b9f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800ae70 tx=0x7f7ac8003350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ac6000640 1 -- 172.21.15.28:0/459549049 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8020020 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ac6000640 1 -- 172.21.15.28:0/459549049 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac801c430 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.880+0000 7f7ac6000640 1 -- 172.21.15.28:0/459549049 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8015030 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.881+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e0c7902ee0 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.881+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c7860500 con 0x55e0c786f4b0 2024-09-10T10:50:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.882+0000 7f7ac6000640 1 -- 172.21.15.28:0/459549049 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ac8023020 con 0x55e0c786f4b0 2024-09-10T10:50:38.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.883+0000 7f7ac6000640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ab006a810 0x7f7ab006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.883+0000 7f7ac6000640 1 -- 172.21.15.28:0/459549049 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f7ac808a810 con 0x55e0c786f4b0 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.884+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e0c7871b90 0x55e0c786b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.884+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55e0c786b670 con 0x55e0c7871b90 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.884+0000 7f7acd800640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ab006a810 0x7f7ab006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.884+0000 7f7acce00640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e0c7871b90 0x55e0c786b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.884+0000 7f7acce00640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e0c7871b90 0x55e0c786b130 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7acd800640 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ab006a810 0x7f7ab006ccd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0057000 tx=0x7f7ac0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7acce00640 1 -- 172.21.15.28:0/459549049 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [omap-get-header] v0'0 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7abc0107c0 con 0x55e0c7871b90 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:38.888 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e0c7871b90 msgr2=0x55e0c786b130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e0c7871b90 0x55e0c786b130 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ab006a810 msgr2=0x7f7ab006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7ab006a810 0x7f7ab006ccd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0057000 tx=0x7f7ac0052000 comp rx=0 tx=0).stop 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 msgr2=0x55e0c790b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 --2- 172.21.15.28:0/459549049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e0c786f4b0 0x55e0c790b9f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800ae70 tx=0x7f7ac8003350 comp rx=0 tx=0).stop 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ace200640 1 -- 172.21.15.28:0/459549049 reap_dead start 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 shutdown_connections 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 >> 172.21.15.28:0/459549049 conn(0x55e0c77c7b90 msgr2=0x55e0c77c7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 shutdown_connections 2024-09-10T10:50:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.885+0000 7f7ad120ff40 1 -- 172.21.15.28:0/459549049 wait complete. 2024-09-10T10:50:38.891 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds_snaptable 2024-09-10T10:50:38.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.967+0000 7f9f4c07bf40 1 Processor -- start 2024-09-10T10:50:38.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.967+0000 7f9f4c07bf40 1 -- start start 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98ba8ff0 0x55bc98b5dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc98ba9530 con 0x55bc98c343e0 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc98b06b30 con 0x55bc98c379c0 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bc98c36ee0 con 0x55bc98ba8ff0 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44508/0 (socket says 172.21.15.28:44508) 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 -- 172.21.15.28:0/2468158376 learned_addr learned my addr 172.21.15.28:0/2468158376 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f49200640 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98ba8ff0 0x55bc98b5dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f49c00640 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 -- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98ba8ff0 msgr2=0x55bc98b5dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98ba8ff0 0x55bc98b5dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 -- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 msgr2=0x55bc98c32bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c32bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.968+0000 7f9f43e00640 1 -- 172.21.15.28:0/2468158376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc98c34cc0 con 0x55bc98c343e0 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f43e00640 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400a430 tx=0x7f9f3400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11ef2b6e929454e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f42a00640 1 -- 172.21.15.28:0/2468158376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400b920 con 0x55bc98c343e0 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f42a00640 1 -- 172.21.15.28:0/2468158376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3400f030 con 0x55bc98c343e0 2024-09-10T10:50:38.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f42a00640 1 -- 172.21.15.28:0/2468158376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400bd20 con 0x55bc98c343e0 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/2468158376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 msgr2=0x55bc98c32680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400a430 tx=0x7f9f3400a8f0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/2468158376 shutdown_connections 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c32bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98c343e0 0x55bc98c32680 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/2468158376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98ba8ff0 0x55bc98b5dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.969+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/2468158376 >> 172.21.15.28:0/2468158376 conn(0x55bc98bad0a0 msgr2=0x55bc98bad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.970+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/2468158376 shutdown_connections 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.970+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/2468158376 wait complete. 2024-09-10T10:50:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.970+0000 7f9f4c07bf40 1 Processor -- start 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.970+0000 7f9f4c07bf40 1 -- start start 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 0x55bc98baa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98c343e0 0x55bc98baaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc98c35bf0 con 0x55bc98ba8ff0 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc98c35910 con 0x55bc98c379c0 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.971+0000 7f9f4c07bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bc98c35ed0 con 0x55bc98c343e0 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 0x55bc98baa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 0x55bc98baa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44514/0 (socket says 172.21.15.28:44514) 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 -- 172.21.15.28:0/3418300340 learned_addr learned my addr 172.21.15.28:0/3418300340 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f43e00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98c343e0 0x55bc98baaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49c00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 -- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98c343e0 msgr2=0x55bc98baaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98c343e0 0x55bc98baaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 -- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 msgr2=0x55bc98c1cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c1cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 -- 172.21.15.28:0/3418300340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc98c2fc90 con 0x55bc98ba8ff0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f43e00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc98c343e0 0x55bc98baaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49c00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc98c379c0 0x55bc98c1cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.972+0000 7f9f49200640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 0x55bc98baa9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c025dc0 tx=0x7f9f3c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.973+0000 7f9f41600640 1 -- 172.21.15.28:0/3418300340 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c068070 con 0x55bc98ba8ff0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.973+0000 7f9f41600640 1 -- 172.21.15.28:0/3418300340 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3c058b50 con 0x55bc98ba8ff0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.973+0000 7f9f41600640 1 -- 172.21.15.28:0/3418300340 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c063400 con 0x55bc98ba8ff0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.973+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc98c2ff70 con 0x55bc98ba8ff0 2024-09-10T10:50:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.973+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc98c2b810 con 0x55bc98ba8ff0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.974+0000 7f9f41600640 1 -- 172.21.15.28:0/3418300340 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f3c06b070 con 0x55bc98ba8ff0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.975+0000 7f9f41600640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f2806a810 0x7f9f2806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.975+0000 7f9f41600640 1 -- 172.21.15.28:0/3418300340 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f9f3c0cd750 con 0x55bc98ba8ff0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.976+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bc98c193d0 0x55bc98c1d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.976+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55bc98ba76a0 con 0x55bc98c193d0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.976+0000 7f9f49c00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bc98c193d0 0x55bc98c1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.976+0000 7f9f43e00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f2806a810 0x7f9f2806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.976+0000 7f9f49c00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bc98c193d0 0x55bc98c1d760 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f43e00640 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f2806a810 0x7f9f2806ccd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55bc98baba00 tx=0x7f9f34016040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f49c00640 1 -- 172.21.15.28:0/3418300340 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [omap-get-header] v0'0 uv1 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9f4400b920 con 0x55bc98c193d0 2024-09-10T10:50:38.979 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:38.979 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bc98c193d0 msgr2=0x55bc98c1d760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bc98c193d0 0x55bc98c1d760 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f2806a810 msgr2=0x7f9f2806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9f2806a810 0x7f9f2806ccd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55bc98baba00 tx=0x7f9f34016040 comp rx=0 tx=0).stop 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 msgr2=0x55bc98baa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 --2- 172.21.15.28:0/3418300340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc98ba8ff0 0x55bc98baa9c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c025dc0 tx=0x7f9f3c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f49c00640 1 -- 172.21.15.28:0/3418300340 reap_dead start 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 shutdown_connections 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 >> 172.21.15.28:0/3418300340 conn(0x55bc98bad0a0 msgr2=0x55bc98bad480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 shutdown_connections 2024-09-10T10:50:38.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:38.977+0000 7f9f4c07bf40 1 -- 172.21.15.28:0/3418300340 wait complete. 2024-09-10T10:50:38.982 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_inotable 2024-09-10T10:50:39.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.056+0000 7ff93cc2df40 1 Processor -- start 2024-09-10T10:50:39.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 -- start start 2024-09-10T10:50:39.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d01ccb0 0x55964d0e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 0x55964d0e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55964cfc0b30 con 0x55964d0ee4b0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55964cfc4b70 con 0x55964d0622d0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.057+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55964d061dc0 con 0x55964d01ccb0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44526/0 (socket says 172.21.15.28:44526) 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 -- 172.21.15.28:0/1398722470 learned_addr learned my addr 172.21.15.28:0/1398722470 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff939400640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 0x55964d0e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff939e00640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d01ccb0 0x55964d0e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 -- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d01ccb0 msgr2=0x55964d0e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d01ccb0 0x55964d0e7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 -- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 msgr2=0x55964d0e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 0x55964d0e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 -- 172.21.15.28:0/1398722470 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964d0edc50 con 0x55964d0ee4b0 2024-09-10T10:50:39.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff939400640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 0x55964d0e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.058+0000 7ff93a800640 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff93400a4e0 tx=0x7ff93400a9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92ddbd61ec36cb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff933e00640 1 -- 172.21.15.28:0/1398722470 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934015070 con 0x55964d0ee4b0 2024-09-10T10:50:39.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff933e00640 1 -- 172.21.15.28:0/1398722470 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93400e730 con 0x55964d0ee4b0 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff933e00640 1 -- 172.21.15.28:0/1398722470 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93400eb20 con 0x55964d0ee4b0 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 -- 172.21.15.28:0/1398722470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 msgr2=0x55964d0ecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff93400a4e0 tx=0x7ff93400a9a0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 -- 172.21.15.28:0/1398722470 shutdown_connections 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0ee4b0 0x55964d0ecb10 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d0622d0 0x55964d0e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/1398722470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d01ccb0 0x55964d0e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 -- 172.21.15.28:0/1398722470 >> 172.21.15.28:0/1398722470 conn(0x55964d046b90 msgr2=0x55964d0ef380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 -- 172.21.15.28:0/1398722470 shutdown_connections 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.059+0000 7ff93cc2df40 1 -- 172.21.15.28:0/1398722470 wait complete. 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.060+0000 7ff93cc2df40 1 Processor -- start 2024-09-10T10:50:39.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.060+0000 7ff93cc2df40 1 -- start start 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d01ccb0 0x55964d17f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 0x55964d17fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d0ee4b0 0x55964d0ea9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55964d0ed800 con 0x55964d0622d0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55964d047440 con 0x55964d01ccb0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff93cc2df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55964d0e76c0 con 0x55964d0ee4b0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff939400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 0x55964d17fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff939400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 0x55964d17fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44540/0 (socket says 172.21.15.28:44540) 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff939400640 1 -- 172.21.15.28:0/631626819 learned_addr learned my addr 172.21.15.28:0/631626819 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.061+0000 7ff939e00640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d01ccb0 0x55964d17f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff93a800640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d0ee4b0 0x55964d0ea9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 -- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d0ee4b0 msgr2=0x55964d0ea9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d0ee4b0 0x55964d0ea9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 -- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d01ccb0 msgr2=0x55964d17f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55964d01ccb0 0x55964d17f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 -- 172.21.15.28:0/631626819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55964d0e69a0 con 0x55964d0622d0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff93a800640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55964d0ee4b0 0x55964d0ea9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff939400640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 0x55964d17fcb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff928025e40 tx=0x7ff928002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff932a00640 1 -- 172.21.15.28:0/631626819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928068070 con 0x55964d0622d0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff932a00640 1 -- 172.21.15.28:0/631626819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff928058b50 con 0x55964d0622d0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff932a00640 1 -- 172.21.15.28:0/631626819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928063400 con 0x55964d0622d0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.062+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55964d0ea1c0 con 0x55964d0622d0 2024-09-10T10:50:39.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.063+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55964d0e8f50 con 0x55964d0622d0 2024-09-10T10:50:39.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.064+0000 7ff932a00640 1 -- 172.21.15.28:0/631626819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff9280635a0 con 0x55964d0622d0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.065+0000 7ff932a00640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff91c06a810 0x7ff91c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.065+0000 7ff932a00640 1 -- 172.21.15.28:0/631626819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff9280cd1a0 con 0x55964d0622d0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.065+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55964d049a90 0x55964d0eb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.065+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55964d0eb950 con 0x55964d049a90 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.066+0000 7ff939e00640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff91c06a810 0x7ff91c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.066+0000 7ff93a800640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55964d049a90 0x55964d0eb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.066+0000 7ff93a800640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55964d049a90 0x55964d0eb410 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.066+0000 7ff939e00640 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff91c06a810 0x7ff91c06ccd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0104f0 tx=0x7ff92c0037d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93a800640 1 -- 172.21.15.28:0/631626819 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [omap-get-header] v0'0 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff934018040 con 0x55964d049a90 2024-09-10T10:50:39.069 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:39.069 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55964d049a90 msgr2=0x55964d0eb410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55964d049a90 0x55964d0eb410 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff91c06a810 msgr2=0x7ff91c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff91c06a810 0x7ff91c06ccd0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0104f0 tx=0x7ff92c0037d0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 msgr2=0x55964d17fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 --2- 172.21.15.28:0/631626819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55964d0622d0 0x55964d17fcb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff928025e40 tx=0x7ff928002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93a800640 1 -- 172.21.15.28:0/631626819 reap_dead start 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 shutdown_connections 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 >> 172.21.15.28:0/631626819 conn(0x55964d046b90 msgr2=0x55964d0ef680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 shutdown_connections 2024-09-10T10:50:39.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.067+0000 7ff93cc2df40 1 -- 172.21.15.28:0/631626819 wait complete. 2024-09-10T10:50:39.072 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000 2024-09-10T10:50:39.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.147+0000 7f9d21120f40 1 Processor -- start 2024-09-10T10:50:39.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.147+0000 7f9d21120f40 1 -- start start 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c172c470 0x5611c18035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 0x5611c17ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5611c16d4b30 con 0x5611c18023e0 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611c16d8b70 con 0x5611c18059c0 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d21120f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5611c1804ee0 con 0x5611c172c470 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d1ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d1ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44560/0 (socket says 172.21.15.28:44560) 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d1ce00640 1 -- 172.21.15.28:0/617050043 learned_addr learned my addr 172.21.15.28:0/617050043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d1e200640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 0x5611c17ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.148+0000 7f9d1d800640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c172c470 0x5611c18035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 -- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c172c470 msgr2=0x5611c18035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c172c470 0x5611c18035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 -- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 msgr2=0x5611c17ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 0x5611c17ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 -- 172.21.15.28:0/617050043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c1760570 con 0x5611c18023e0 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1e200640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 0x5611c17ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00a440 tx=0x7f9d0c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2400bb67c81ccaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d17400640 1 -- 172.21.15.28:0/617050043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c068070 con 0x5611c18023e0 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d17400640 1 -- 172.21.15.28:0/617050043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0c050980 con 0x5611c18023e0 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.149+0000 7f9d17400640 1 -- 172.21.15.28:0/617050043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c050d30 con 0x5611c18023e0 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 -- 172.21.15.28:0/617050043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 msgr2=0x5611c17ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00a440 tx=0x7f9d0c051000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 -- 172.21.15.28:0/617050043 shutdown_connections 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c18059c0 0x5611c17ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17ff790 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 --2- 172.21.15.28:0/617050043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c172c470 0x5611c18035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 -- 172.21.15.28:0/617050043 >> 172.21.15.28:0/617050043 conn(0x5611c177b0a0 msgr2=0x5611c1802880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 -- 172.21.15.28:0/617050043 shutdown_connections 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.150+0000 7f9d21120f40 1 -- 172.21.15.28:0/617050043 wait complete. 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.151+0000 7f9d21120f40 1 Processor -- start 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.151+0000 7f9d21120f40 1 -- start start 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.151+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c172c470 0x5611c17ff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d21120f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c18059c0 0x5611c17fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d21120f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5611c175bff0 con 0x5611c18023e0 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d21120f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5611c1800210 con 0x5611c172c470 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d21120f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5611c175b6a0 con 0x5611c18059c0 2024-09-10T10:50:39.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44566/0 (socket says 172.21.15.28:44566) 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 -- 172.21.15.28:0/882664201 learned_addr learned my addr 172.21.15.28:0/882664201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1d800640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c172c470 0x5611c17ff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1e200640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c18059c0 0x5611c17fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 -- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c18059c0 msgr2=0x5611c17fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5611c18059c0 0x5611c17fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 -- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c172c470 msgr2=0x5611c17ff750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c172c470 0x5611c17ff750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1ce00640 1 -- 172.21.15.28:0/882664201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611c1774b00 con 0x5611c18023e0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.152+0000 7f9d1d800640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5611c172c470 0x5611c17ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d1ce00640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17feba0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00a440 tx=0x7f9d0c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d16000640 1 -- 172.21.15.28:0/882664201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c068040 con 0x5611c18023e0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d16000640 1 -- 172.21.15.28:0/882664201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0c050cf0 con 0x5611c18023e0 2024-09-10T10:50:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d16000640 1 -- 172.21.15.28:0/882664201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c06c610 con 0x5611c18023e0 2024-09-10T10:50:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5611c17fe0c0 con 0x5611c18023e0 2024-09-10T10:50:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.153+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611c17fe680 con 0x5611c18023e0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.155+0000 7f9d16000640 1 -- 172.21.15.28:0/882664201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d0c073060 con 0x5611c18023e0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d16000640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0006a840 0x7f9d0006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d16000640 1 -- 172.21.15.28:0/882664201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f9d0c0ce5d0 con 0x5611c18023e0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d21120f40 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5611c172dca0 0x5611c189ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5611c17ffc90 con 0x5611c172dca0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d1d800640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0006a840 0x7f9d0006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.156+0000 7f9d1e200640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5611c172dca0 0x5611c189ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d1e200640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5611c172dca0 0x5611c189ad70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d1d800640 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0006a840 0x7f9d0006cd00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000f60 tx=0x7f9d10001700 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d1e200640 1 -- 172.21.15.28:0/882664201 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [omap-get-header out=274b] v0'0 uv1 ondisk = 0) ==== 156+0+274 (crc 0 0 0) 0x7f9d18001580 con 0x5611c172dca0 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:header (274 bytes) : 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 04 03 0c 01 00 00 01 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000030 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T10:50:39.159 INFO:teuthology.orchestra.run.smithi028.stdout:00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:00000060 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:00000090 0a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000a0 00 00 00 00 00 00 00 00 74 24 e0 66 aa ef ff 09 |........t$.f....| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000e0 00 00 00 00 00 00 74 24 e0 66 aa ef ff 09 00 00 |......t$.f......| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:39.160 INFO:teuthology.orchestra.run.smithi028.stdout:00000110 00 00 |..| 2024-09-10T10:50:39.161 INFO:teuthology.orchestra.run.smithi028.stdout:00000112 2024-09-10T10:50:39.161 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5611c172dca0 msgr2=0x5611c189ad70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5611c172dca0 0x5611c189ad70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0006a840 msgr2=0x7f9d0006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d0006a840 0x7f9d0006cd00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000f60 tx=0x7f9d10001700 comp rx=0 tx=0).stop 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 msgr2=0x5611c17feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 --2- 172.21.15.28:0/882664201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5611c18023e0 0x5611c17feba0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c00a440 tx=0x7f9d0c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d1e200640 1 -- 172.21.15.28:0/882664201 reap_dead start 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 shutdown_connections 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.157+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 >> 172.21.15.28:0/882664201 conn(0x5611c177b0a0 msgr2=0x5611c1804790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.158+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 shutdown_connections 2024-09-10T10:50:39.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.158+0000 7f9d21120f40 1 -- 172.21.15.28:0/882664201 wait complete. 2024-09-10T10:50:39.163 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader mds0_sessionmap 2024-09-10T10:50:39.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.237+0000 7f054e598f40 1 Processor -- start 2024-09-10T10:50:39.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.237+0000 7f054e598f40 1 -- start start 2024-09-10T10:50:39.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 0x555e8e1c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 0x555e8e1455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555e8e143530 con 0x555e8e1d19c0 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555e8e0a0b30 con 0x555e8e1ce3e0 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054e598f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555e8e1d0ee0 con 0x555e8e1428e0 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.238+0000 7f054c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44576/0 (socket says 172.21.15.28:44576) 2024-09-10T10:50:39.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 -- 172.21.15.28:0/3525376975 learned_addr learned my addr 172.21.15.28:0/3525376975 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054b800640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 0x555e8e1c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054ae00640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 0x555e8e1455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 -- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 msgr2=0x555e8e1c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 0x555e8e1c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 -- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 msgr2=0x555e8e1455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 0x555e8e1455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 -- 172.21.15.28:0/3525376975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8e1479f0 con 0x555e8e1d19c0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054b800640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 0x555e8e1c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054ae00640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 0x555e8e1455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f054c200640 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0544001600 tx=0x7f0544001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa35356b0dbb04b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f0549a00640 1 -- 172.21.15.28:0/3525376975 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0544016070 con 0x555e8e1d19c0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f0549a00640 1 -- 172.21.15.28:0/3525376975 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054400f710 con 0x555e8e1d19c0 2024-09-10T10:50:39.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.239+0000 7f0549a00640 1 -- 172.21.15.28:0/3525376975 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054400fb00 con 0x555e8e1d19c0 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 -- 172.21.15.28:0/3525376975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 msgr2=0x555e8e1c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0544001600 tx=0x7f0544001ac0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 -- 172.21.15.28:0/3525376975 shutdown_connections 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1d19c0 0x555e8e1c8ff0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1ce3e0 0x555e8e1455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 --2- 172.21.15.28:0/3525376975 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1428e0 0x555e8e1c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 -- 172.21.15.28:0/3525376975 >> 172.21.15.28:0/3525376975 conn(0x555e8e1470a0 msgr2=0x555e8e0f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 -- 172.21.15.28:0/3525376975 shutdown_connections 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.240+0000 7f054e598f40 1 -- 172.21.15.28:0/3525376975 wait complete. 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.241+0000 7f054e598f40 1 Processor -- start 2024-09-10T10:50:39.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.241+0000 7f054e598f40 1 -- start start 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 0x555e8e144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1ce3e0 0x555e8e261660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1d19c0 0x555e8e261ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555e8e1d09a0 con 0x555e8e1428e0 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555e8e1ced60 con 0x555e8e1d19c0 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054e598f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555e8e1c5370 con 0x555e8e1ce3e0 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 0x555e8e144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 0x555e8e144bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44584/0 (socket says 172.21.15.28:44584) 2024-09-10T10:50:39.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 -- 172.21.15.28:0/1690092469 learned_addr learned my addr 172.21.15.28:0/1690092469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054ae00640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1ce3e0 0x555e8e261660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054c200640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1d19c0 0x555e8e261ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 -- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1ce3e0 msgr2=0x555e8e261660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555e8e1ce3e0 0x555e8e261660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 -- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1d19c0 msgr2=0x555e8e261ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555e8e1d19c0 0x555e8e261ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.242+0000 7f054b800640 1 -- 172.21.15.28:0/1690092469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8e147720 con 0x555e8e1428e0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f054b800640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 0x555e8e144bb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f053c045b60 tx=0x7f053c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f0543e00640 1 -- 172.21.15.28:0/1690092469 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c055070 con 0x555e8e1428e0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f0543e00640 1 -- 172.21.15.28:0/1690092469 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053c051820 con 0x555e8e1428e0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f0543e00640 1 -- 172.21.15.28:0/1690092469 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c051bf0 con 0x555e8e1428e0 2024-09-10T10:50:39.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e8e0f98a0 con 0x555e8e1428e0 2024-09-10T10:50:39.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.243+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8e1d3db0 con 0x555e8e1428e0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.245+0000 7f0543e00640 1 -- 172.21.15.28:0/1690092469 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f053c058070 con 0x555e8e1428e0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f0543e00640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f052c06a840 0x7f052c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f0543e00640 1 -- 172.21.15.28:0/1690092469 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f053c0cd870 con 0x555e8e1428e0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f054e598f40 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555e8e1450f0 0x555e8e2620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555e8e26a7d0 con 0x555e8e1450f0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f054ae00640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f052c06a840 0x7f052c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.246+0000 7f054c200640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555e8e1450f0 0x555e8e2620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054ae00640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f052c06a840 0x7f052c06cd00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x555e8e261300 tx=0x7f0538005990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054c200640 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555e8e1450f0 0x555e8e2620e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054c200640 1 -- 172.21.15.28:0/1690092469 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-header out=14b] v0'0 uv3 ondisk = 0) ==== 159+0+14 (crc 0 0 0) 0x7f054400f070 con 0x555e8e1450f0 2024-09-10T10:50:39.249 INFO:teuthology.orchestra.run.smithi028.stdout:header (14 bytes) : 2024-09-10T10:50:39.249 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 01 01 08 00 00 00 04 00 00 00 00 00 00 00 |..............| 2024-09-10T10:50:39.249 INFO:teuthology.orchestra.run.smithi028.stdout:0000000e 2024-09-10T10:50:39.249 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555e8e1450f0 msgr2=0x555e8e2620e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555e8e1450f0 0x555e8e2620e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f052c06a840 msgr2=0x7f052c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f052c06a840 0x7f052c06cd00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x555e8e261300 tx=0x7f0538005990 comp rx=0 tx=0).stop 2024-09-10T10:50:39.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 msgr2=0x555e8e144bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054e598f40 1 --2- 172.21.15.28:0/1690092469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e8e1428e0 0x555e8e144bb0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f053c045b60 tx=0x7f053c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.247+0000 7f054c200640 1 -- 172.21.15.28:0/1690092469 reap_dead start 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.248+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 shutdown_connections 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.248+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 >> 172.21.15.28:0/1690092469 conn(0x555e8e1470a0 msgr2=0x555e8e1cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.248+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 shutdown_connections 2024-09-10T10:50:39.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.248+0000 7f054e598f40 1 -- 172.21.15.28:0/1690092469 wait complete. 2024-09-10T10:50:39.253 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 400.00000000 2024-09-10T10:50:39.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.328+0000 7fca84520f40 1 Processor -- start 2024-09-10T10:50:39.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.328+0000 7fca84520f40 1 -- start start 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf30f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 0x5606cf30fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5606cf283530 con 0x5606cf3119c0 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606cf1e0b30 con 0x5606cf30e3e0 2024-09-10T10:50:39.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca84520f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5606cf310ee0 con 0x5606cf2828e0 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca81800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 0x5606cf30fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca80e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf30f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca81800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44594/0 (socket says 172.21.15.28:44594) 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca80e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf30f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50086/0 (socket says 172.21.15.28:50086) 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca81800640 1 -- 172.21.15.28:0/1622078087 learned_addr learned my addr 172.21.15.28:0/1622078087 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.329+0000 7fca7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 0x5606cf30fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:53772/0 (socket says 172.21.15.28:53772) 2024-09-10T10:50:39.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 -- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 msgr2=0x5606cf30f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf30f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 -- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 msgr2=0x5606cf30fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 0x5606cf30fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 -- 172.21.15.28:0/1622078087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cf23aba0 con 0x5606cf3119c0 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca81800640 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0015d0 tx=0x7fca7c001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ac28e366b1fe3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca7aa00640 1 -- 172.21.15.28:0/1622078087 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c013620 con 0x5606cf3119c0 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca7aa00640 1 -- 172.21.15.28:0/1622078087 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca7c0137c0 con 0x5606cf3119c0 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.330+0000 7fca7aa00640 1 -- 172.21.15.28:0/1622078087 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7c013bb0 con 0x5606cf3119c0 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 -- 172.21.15.28:0/1622078087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 msgr2=0x5606cf3a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0015d0 tx=0x7fca7c001a90 comp rx=0 tx=0).stop 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 -- 172.21.15.28:0/1622078087 shutdown_connections 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf3119c0 0x5606cf3a2c00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf30e3e0 0x5606cf30fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 --2- 172.21.15.28:0/1622078087 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf30f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 -- 172.21.15.28:0/1622078087 >> 172.21.15.28:0/1622078087 conn(0x5606cf2870a0 msgr2=0x5606cf238070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 -- 172.21.15.28:0/1622078087 shutdown_connections 2024-09-10T10:50:39.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.331+0000 7fca84520f40 1 -- 172.21.15.28:0/1622078087 wait complete. 2024-09-10T10:50:39.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.332+0000 7fca84520f40 1 Processor -- start 2024-09-10T10:50:39.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.332+0000 7fca84520f40 1 -- start start 2024-09-10T10:50:39.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf26bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 0x5606cf30bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf3119c0 0x5606cf3022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5606cf308ba0 con 0x5606cf30e3e0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606cf3102f0 con 0x5606cf3119c0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca84520f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5606cf305400 con 0x5606cf2828e0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 0x5606cf30bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 0x5606cf30bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44600/0 (socket says 172.21.15.28:44600) 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 -- 172.21.15.28:0/470424131 learned_addr learned my addr 172.21.15.28:0/470424131 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca80e00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf26bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca81800640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf3119c0 0x5606cf3022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 -- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 msgr2=0x5606cf26bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5606cf2828e0 0x5606cf26bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 -- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf3119c0 msgr2=0x5606cf3022d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606cf3119c0 0x5606cf3022d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.333+0000 7fca7be00640 1 -- 172.21.15.28:0/470424131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606cf30fba0 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca7be00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 0x5606cf30bf90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0086b0 tx=0x7fca6c0087a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca79600640 1 -- 172.21.15.28:0/470424131 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0147e0 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca79600640 1 -- 172.21.15.28:0/470424131 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6c01a070 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca79600640 1 -- 172.21.15.28:0/470424131 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c014ba0 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5606cf305fd0 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.334+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606cf306590 con 0x5606cf30e3e0 2024-09-10T10:50:39.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.336+0000 7fca79600640 1 -- 172.21.15.28:0/470424131 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca6c015470 con 0x5606cf30e3e0 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca79600640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca6006a840 0x7fca6006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca79600640 1 -- 172.21.15.28:0/470424131 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fca6c08a460 con 0x5606cf30e3e0 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca84520f40 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5606cf3091f0 0x5606cf3095d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5606cf3029d0 con 0x5606cf3091f0 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca80e00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca6006a840 0x7fca6006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.337+0000 7fca81800640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5606cf3091f0 0x5606cf3095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca81800640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5606cf3091f0 0x5606cf3095d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca80e00640 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca6006a840 0x7fca6006cd00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fca74057000 tx=0x7fca74052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca81800640 1 -- 172.21.15.28:0/470424131 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [omap-get-header] v0'0 uv1 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fca7c01b040 con 0x5606cf3091f0 2024-09-10T10:50:39.340 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:39.340 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5606cf3091f0 msgr2=0x5606cf3095d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5606cf3091f0 0x5606cf3095d0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca6006a840 msgr2=0x7fca6006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca6006a840 0x7fca6006cd00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fca74057000 tx=0x7fca74052000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 msgr2=0x5606cf30bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca84520f40 1 --2- 172.21.15.28:0/470424131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5606cf30e3e0 0x5606cf30bf90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0086b0 tx=0x7fca6c0087a0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.338+0000 7fca81800640 1 -- 172.21.15.28:0/470424131 reap_dead start 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.339+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 shutdown_connections 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.339+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 >> 172.21.15.28:0/470424131 conn(0x5606cf2870a0 msgr2=0x5606cf238070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.339+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 shutdown_connections 2024-09-10T10:50:39.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.339+0000 7fca84520f40 1 -- 172.21.15.28:0/470424131 wait complete. 2024-09-10T10:50:39.344 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 100.00000000.inode 2024-09-10T10:50:39.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.418+0000 7fd90df68f40 1 Processor -- start 2024-09-10T10:50:39.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.418+0000 7fd90df68f40 1 -- start start 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52daeff0 0x55ca52d97af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 0x55ca52d64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ca52daf530 con 0x55ca52e3a3e0 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca52d0cb30 con 0x55ca52e3d9c0 2024-09-10T10:50:39.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90df68f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ca52e3cee0 con 0x55ca52daeff0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44620/0 (socket says 172.21.15.28:44620) 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90a800640 1 -- 172.21.15.28:0/1550621933 learned_addr learned my addr 172.21.15.28:0/1550621933 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90b200640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52daeff0 0x55ca52d97af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.419+0000 7fd90bc00640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 0x55ca52d64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 -- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52daeff0 msgr2=0x55ca52d97af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52daeff0 0x55ca52d97af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 -- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 msgr2=0x55ca52d64000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 0x55ca52d64000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 -- 172.21.15.28:0/1550621933 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca52e3c0f0 con 0x55ca52e3a3e0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90bc00640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 0x55ca52d64000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd90a800640 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001d30 tx=0x7fd8f800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3da0ed992a45d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd909400640 1 -- 172.21.15.28:0/1550621933 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800bd20 con 0x55ca52e3a3e0 2024-09-10T10:50:39.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd909400640 1 -- 172.21.15.28:0/1550621933 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f800f400 con 0x55ca52e3a3e0 2024-09-10T10:50:39.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.420+0000 7fd909400640 1 -- 172.21.15.28:0/1550621933 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800f7f0 con 0x55ca52e3a3e0 2024-09-10T10:50:39.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 -- 172.21.15.28:0/1550621933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 msgr2=0x55ca52d63ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001d30 tx=0x7fd8f800b5b0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 -- 172.21.15.28:0/1550621933 shutdown_connections 2024-09-10T10:50:39.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3d9c0 0x55ca52d64000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52e3a3e0 0x55ca52d63ac0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 --2- 172.21.15.28:0/1550621933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52daeff0 0x55ca52d97af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 -- 172.21.15.28:0/1550621933 >> 172.21.15.28:0/1550621933 conn(0x55ca52db30a0 msgr2=0x55ca52db3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 -- 172.21.15.28:0/1550621933 shutdown_connections 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 -- 172.21.15.28:0/1550621933 wait complete. 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.421+0000 7fd90df68f40 1 Processor -- start 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.422+0000 7fd90df68f40 1 -- start start 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.422+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 0x55ca52e27190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.422+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3a3e0 0x55ca52d95e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.422+0000 7fd90df68f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52e3d9c0 0x55ca52ed67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90df68f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ca52e3abc0 con 0x55ca52daeff0 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90df68f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ca52e3bc60 con 0x55ca52e3a3e0 2024-09-10T10:50:39.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90df68f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ca52e3b610 con 0x55ca52e3d9c0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 0x55ca52e27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 0x55ca52e27190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44624/0 (socket says 172.21.15.28:44624) 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 -- 172.21.15.28:0/3192668948 learned_addr learned my addr 172.21.15.28:0/3192668948 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90bc00640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52e3d9c0 0x55ca52ed67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90a800640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3a3e0 0x55ca52d95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 -- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52e3d9c0 msgr2=0x55ca52ed67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ca52e3d9c0 0x55ca52ed67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 -- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3a3e0 msgr2=0x55ca52d95e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ca52e3a3e0 0x55ca52d95e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.423+0000 7fd90b200640 1 -- 172.21.15.28:0/3192668948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca52e276d0 con 0x55ca52daeff0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd90b200640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 0x55ca52e27190 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc000c00 tx=0x7fd8fc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd903e00640 1 -- 172.21.15.28:0/3192668948 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc055040 con 0x55ca52daeff0 2024-09-10T10:50:39.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd903e00640 1 -- 172.21.15.28:0/3192668948 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc051a00 con 0x55ca52daeff0 2024-09-10T10:50:39.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd903e00640 1 -- 172.21.15.28:0/3192668948 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc051db0 con 0x55ca52daeff0 2024-09-10T10:50:39.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ca52e2b7d0 con 0x55ca52daeff0 2024-09-10T10:50:39.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.424+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca52e2bd70 con 0x55ca52daeff0 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.426+0000 7fd903e00640 1 -- 172.21.15.28:0/3192668948 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8fc058070 con 0x55ca52daeff0 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd903e00640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8ec06a840 0x7fd8ec06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd903e00640 1 -- 172.21.15.28:0/3192668948 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd8fc0cda70 con 0x55ca52daeff0 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd90df68f40 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ca52d963c0 0x55ca52e2c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ca52e2cab0 con 0x55ca52d963c0 2024-09-10T10:50:39.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd90a800640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8ec06a840 0x7fd8ec06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.427+0000 7fd90bc00640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ca52d963c0 0x55ca52e2c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90bc00640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ca52d963c0 0x55ca52e2c570 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90a800640 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8ec06a840 0x7fd8ec06cd00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55ca52ece620 tx=0x7fd8f8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90bc00640 1 -- 172.21.15.28:0/3192668948 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-header] v0'0 uv3 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd904011040 con 0x55ca52d963c0 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stdout:header (0 bytes) : 2024-09-10T10:50:39.430 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ca52d963c0 msgr2=0x55ca52e2c570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ca52d963c0 0x55ca52e2c570 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8ec06a840 msgr2=0x7fd8ec06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd8ec06a840 0x7fd8ec06cd00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55ca52ece620 tx=0x7fd8f8009000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 msgr2=0x55ca52e27190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 --2- 172.21.15.28:0/3192668948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ca52daeff0 0x55ca52e27190 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc000c00 tx=0x7fd8fc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90bc00640 1 -- 172.21.15.28:0/3192668948 reap_dead start 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 shutdown_connections 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.428+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 >> 172.21.15.28:0/3192668948 conn(0x55ca52db30a0 msgr2=0x55ca52db3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.429+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 shutdown_connections 2024-09-10T10:50:39.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.429+0000 7fd90df68f40 1 -- 172.21.15.28:0/3192668948 wait complete. 2024-09-10T10:50:39.434 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapheader 1.00000000 2024-09-10T10:50:39.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.508+0000 7f0ee4046f40 1 Processor -- start 2024-09-10T10:50:39.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 -- start start 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41e54b0 0x557bb41e7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557bb40b7b30 con 0x557bb41592d0 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bb4158dc0 con 0x557bb41135a0 2024-09-10T10:50:39.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.509+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557bb41e8410 con 0x557bb41e54b0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44640/0 (socket says 172.21.15.28:44640) 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 -- 172.21.15.28:0/2352853297 learned_addr learned my addr 172.21.15.28:0/2352853297 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0ee1200640 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0ee1c00640 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41e54b0 0x557bb41e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 -- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41e54b0 msgr2=0x557bb41e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41e54b0 0x557bb41e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 -- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41135a0 msgr2=0x557bb41de3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 -- 172.21.15.28:0/2352853297 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb410eb40 con 0x557bb41592d0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.510+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc045b60 tx=0x7f0ecc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970b28225ec5a7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0edaa00640 1 -- 172.21.15.28:0/2352853297 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc055070 con 0x557bb41592d0 2024-09-10T10:50:39.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0edaa00640 1 -- 172.21.15.28:0/2352853297 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ecc051730 con 0x557bb41592d0 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0edaa00640 1 -- 172.21.15.28:0/2352853297 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc051b00 con 0x557bb41592d0 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2352853297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 msgr2=0x557bb413d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc045b60 tx=0x7f0ecc051000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2352853297 shutdown_connections 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41e54b0 0x557bb41e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb413d090 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2352853297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2352853297 >> 172.21.15.28:0/2352853297 conn(0x557bb413db90 msgr2=0x557bb41e6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2352853297 shutdown_connections 2024-09-10T10:50:39.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.511+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2352853297 wait complete. 2024-09-10T10:50:39.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.512+0000 7f0ee4046f40 1 Processor -- start 2024-09-10T10:50:39.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.512+0000 7f0ee4046f40 1 -- start start 2024-09-10T10:50:39.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb41e4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41e54b0 0x557bb41d9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557bb41e8640 con 0x557bb41592d0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bb41568e0 con 0x557bb41e54b0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee4046f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557bb41e8920 con 0x557bb41135a0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb41e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0edbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb41e4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44652/0 (socket says 172.21.15.28:44652) 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0edbe00640 1 -- 172.21.15.28:0/2675061677 learned_addr learned my addr 172.21.15.28:0/2675061677 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee1200640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.513+0000 7f0ee1c00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41e54b0 0x557bb41d9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 -- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41135a0 msgr2=0x557bb41de3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557bb41135a0 0x557bb41de3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 -- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41e54b0 msgr2=0x557bb41d9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bb41e54b0 0x557bb41d9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 -- 172.21.15.28:0/2675061677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb41d9b70 con 0x557bb41592d0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0edbe00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb41e4350 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc045b60 tx=0x7f0ecc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0ed9600640 1 -- 172.21.15.28:0/2675061677 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc06d070 con 0x557bb41592d0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.514+0000 7f0ed9600640 1 -- 172.21.15.28:0/2675061677 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ecc051d90 con 0x557bb41592d0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.515+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557bb41d9e10 con 0x557bb41592d0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.515+0000 7f0ed9600640 1 -- 172.21.15.28:0/2675061677 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc055070 con 0x557bb41592d0 2024-09-10T10:50:39.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.515+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb41da820 con 0x557bb41592d0 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.516+0000 7f0ed9600640 1 -- 172.21.15.28:0/2675061677 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ecc070070 con 0x557bb41592d0 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.517+0000 7f0ed9600640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ec006a810 0x7f0ec006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.517+0000 7f0ed9600640 1 -- 172.21.15.28:0/2675061677 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f0ecc0cde50 con 0x557bb41592d0 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557bb41d9760 0x557bb41e7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x557bb41db4d0 con 0x557bb41d9760 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee1200640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ec006a810 0x7f0ec006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee1c00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557bb41d9760 0x557bb41e7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee1200640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ec006a810 0x7f0ec006ccd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40055f0 tx=0x7f0ed400c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.518+0000 7f0ee1c00640 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557bb41d9760 0x557bb41e7610 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee1c00640 1 -- 172.21.15.28:0/2675061677 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) ==== 154+0+274 (crc 0 0 0) 0x7f0edc0027c0 con 0x557bb41d9760 2024-09-10T10:50:39.520 INFO:teuthology.orchestra.run.smithi028.stdout:header (274 bytes) : 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000000 04 03 0c 01 00 00 0d 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000010 00 00 00 00 00 00 03 02 28 00 00 00 00 00 00 00 |........(.......| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000020 00 00 00 00 7b 24 e0 66 60 b3 0d 30 00 00 00 00 |....{$.f`..0....| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000030 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 00 |................| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000040 00 00 00 00 03 02 28 00 00 00 00 00 00 00 00 00 |......(.........| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000050 00 00 7b 24 e0 66 60 b3 0d 30 00 00 00 00 00 00 |..{$.f`..0......| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000060 00 00 01 00 00 00 00 00 00 00 01 00 00 00 00 00 |................| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000070 00 00 03 02 38 00 00 00 00 00 00 00 00 00 00 00 |....8...........| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000080 00 00 60 00 00 00 00 00 01 00 00 00 00 00 00 00 |..`.............| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:00000090 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:000000a0 00 00 00 00 00 00 00 00 7b 24 e0 66 d5 ca 60 3b |........{$.f..`;| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:000000b0 03 02 38 00 00 00 00 00 00 00 00 00 00 00 00 00 |..8.............| 2024-09-10T10:50:39.521 INFO:teuthology.orchestra.run.smithi028.stdout:000000c0 60 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 |`...............| 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:000000d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:000000e0 00 00 00 00 00 00 7b 24 e0 66 d5 ca 60 3b 00 00 |......{$.f..`;..| 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:000000f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:* 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:00000110 00 00 |..| 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout:00000112 2024-09-10T10:50:39.522 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557bb41d9760 msgr2=0x557bb41e7610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557bb41d9760 0x557bb41e7610 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ec006a810 msgr2=0x7f0ec006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ec006a810 0x7f0ec006ccd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40055f0 tx=0x7f0ed400c040 comp rx=0 tx=0).stop 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 msgr2=0x557bb41e4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 --2- 172.21.15.28:0/2675061677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557bb41592d0 0x557bb41e4350 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc045b60 tx=0x7f0ecc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee1c00640 1 -- 172.21.15.28:0/2675061677 reap_dead start 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 shutdown_connections 2024-09-10T10:50:39.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 >> 172.21.15.28:0/2675061677 conn(0x557bb413db90 msgr2=0x557bb4159760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 shutdown_connections 2024-09-10T10:50:39.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.519+0000 7f0ee4046f40 1 -- 172.21.15.28:0/2675061677 wait complete. 2024-09-10T10:50:39.525 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.00000000 2024-09-10T10:50:39.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.599+0000 7f467fe3cf40 1 Processor -- start 2024-09-10T10:50:39.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.599+0000 7f467fe3cf40 1 -- start start 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd93d2d0 0x5653dd93aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5653dd93cdc0 con 0x5653dd8f7cb0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653dd89bb30 con 0x5653dd93d2d0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5653dd93c1f0 con 0x5653dd9c94b0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44676/0 (socket says 172.21.15.28:44676) 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 -- 172.21.15.28:0/3648449768 learned_addr learned my addr 172.21.15.28:0/3648449768 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467da00640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f4677e00640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd93d2d0 0x5653dd93aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 -- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 msgr2=0x5653dd9c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 -- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd93d2d0 msgr2=0x5653dd93aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd93d2d0 0x5653dd93aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467d000640 1 -- 172.21.15.28:0/3648449768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dd9cb930 con 0x5653dd8f7cb0 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.600+0000 7f467da00640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467d000640 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4670001db0 tx=0x7f467000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc447e3998674b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f4676a00640 1 -- 172.21.15.28:0/3648449768 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000bd80 con 0x5653dd8f7cb0 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f4676a00640 1 -- 172.21.15.28:0/3648449768 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467000f400 con 0x5653dd8f7cb0 2024-09-10T10:50:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f4676a00640 1 -- 172.21.15.28:0/3648449768 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467000f7f0 con 0x5653dd8f7cb0 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3648449768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 msgr2=0x5653dd93a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f4670001db0 tx=0x7f467000b5b0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3648449768 shutdown_connections 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd93d2d0 0x5653dd93aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3648449768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd8f7cb0 0x5653dd93a990 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3648449768 >> 172.21.15.28:0/3648449768 conn(0x5653dd921b90 msgr2=0x5653dd921f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3648449768 shutdown_connections 2024-09-10T10:50:39.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.601+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3648449768 wait complete. 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.602+0000 7f467fe3cf40 1 Processor -- start 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.602+0000 7f467fe3cf40 1 -- start start 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd8f7cb0 0x5653dd93d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 0x5653dd9c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5653dd9cbef0 con 0x5653dd93d2d0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653dd9cbc10 con 0x5653dd8f7cb0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467fe3cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5653dd920c20 con 0x5653dd9c94b0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f4677e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 0x5653dd9c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f4677e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 0x5653dd9c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44692/0 (socket says 172.21.15.28:44692) 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f4677e00640 1 -- 172.21.15.28:0/3730252148 learned_addr learned my addr 172.21.15.28:0/3730252148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467da00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.603+0000 7f467d000640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd8f7cb0 0x5653dd93d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 -- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 msgr2=0x5653dd9c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653dd9c94b0 0x5653dd9c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 -- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd8f7cb0 msgr2=0x5653dd93d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd8f7cb0 0x5653dd93d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 -- 172.21.15.28:0/3730252148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dd9c34c0 con 0x5653dd93d2d0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f467d000640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dd8f7cb0 0x5653dd93d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4677e00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 0x5653dd9c6330 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4668000c00 tx=0x7f4668002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4675600640 1 -- 172.21.15.28:0/3730252148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668055070 con 0x5653dd93d2d0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4675600640 1 -- 172.21.15.28:0/3730252148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46680517f0 con 0x5653dd93d2d0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653dd9c37a0 con 0x5653dd93d2d0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f4675600640 1 -- 172.21.15.28:0/3730252148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668051c00 con 0x5653dd93d2d0 2024-09-10T10:50:39.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.604+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dd93b0c0 con 0x5653dd93d2d0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.606+0000 7f4675600640 1 -- 172.21.15.28:0/3730252148 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4668058070 con 0x5653dd93d2d0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.607+0000 7f4675600640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f465c06a810 0x7f465c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.607+0000 7f4675600640 1 -- 172.21.15.28:0/3730252148 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f46680cd610 con 0x5653dd93d2d0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.607+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5653dda691a0 0x5653dda6b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.607+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5653dd93dca0 con 0x5653dda691a0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.607+0000 7f467d000640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f465c06a810 0x7f465c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.608+0000 7f467da00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5653dda691a0 0x5653dda6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.608+0000 7f467d000640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f465c06a810 0x7f465c06ccd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4670003040 tx=0x7f4670009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.608+0000 7f467da00640 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5653dda691a0 0x5653dda6b590 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467da00640 1 -- 172.21.15.28:0/3730252148 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-keys out=21b] v0'0 uv10 ondisk = 0) ==== 164+0+21 (crc 0 0 0) 0x7f4678002a80 con 0x5653dda691a0 2024-09-10T10:50:39.611 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs_head 2024-09-10T10:50:39.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5653dda691a0 msgr2=0x5653dda6b590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5653dda691a0 0x5653dda6b590 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f465c06a810 msgr2=0x7f465c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f465c06a810 0x7f465c06ccd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4670003040 tx=0x7f4670009000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 msgr2=0x5653dd9c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 --2- 172.21.15.28:0/3730252148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653dd93d2d0 0x5653dd9c6330 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4668000c00 tx=0x7f4668002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467da00640 1 -- 172.21.15.28:0/3730252148 reap_dead start 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 shutdown_connections 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 >> 172.21.15.28:0/3730252148 conn(0x5653dd921b90 msgr2=0x5653dd921f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 shutdown_connections 2024-09-10T10:50:39.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.609+0000 7f467fe3cf40 1 -- 172.21.15.28:0/3730252148 wait complete. 2024-09-10T10:50:39.614 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 600.00000000 2024-09-10T10:50:39.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 Processor -- start 2024-09-10T10:50:39.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 -- start start 2024-09-10T10:50:39.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adb90c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adc1d9c0 0x55b4adc1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4adb8f530 con 0x55b4adb44470 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4adaecb30 con 0x55b4adc1d9c0 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.690+0000 7f54cf91df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b4adc1cee0 con 0x55b4adc1a3e0 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44708/0 (socket says 172.21.15.28:44708) 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 -- 172.21.15.28:0/3950301371 learned_addr learned my addr 172.21.15.28:0/3950301371 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cca00640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adc1d9c0 0x55b4adc1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cb600640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adb90c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 -- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 msgr2=0x55b4adb90c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adb90c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 -- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adc1d9c0 msgr2=0x55b4adc1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adc1d9c0 0x55b4adc1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 -- 172.21.15.28:0/3950301371 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4adb90760 con 0x55b4adb44470 2024-09-10T10:50:39.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cb600640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adb90c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:39.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54cc000640 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00b310 tx=0x7f54bc00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584f846937bfa42e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54ca200640 1 -- 172.21.15.28:0/3950301371 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc008070 con 0x55b4adb44470 2024-09-10T10:50:39.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54ca200640 1 -- 172.21.15.28:0/3950301371 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54bc00d500 con 0x55b4adb44470 2024-09-10T10:50:39.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.691+0000 7f54ca200640 1 -- 172.21.15.28:0/3950301371 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc00d8b0 con 0x55b4adb44470 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 -- 172.21.15.28:0/3950301371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 msgr2=0x55b4adb90220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00b310 tx=0x7f54bc00b7d0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 -- 172.21.15.28:0/3950301371 shutdown_connections 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adc1d9c0 0x55b4adc1b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adb90c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 --2- 172.21.15.28:0/3950301371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adb44470 0x55b4adb90220 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 -- 172.21.15.28:0/3950301371 >> 172.21.15.28:0/3950301371 conn(0x55b4adb930a0 msgr2=0x55b4adc1c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 -- 172.21.15.28:0/3950301371 shutdown_connections 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.692+0000 7f54cf91df40 1 -- 172.21.15.28:0/3950301371 wait complete. 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.693+0000 7f54cf91df40 1 Processor -- start 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.693+0000 7f54cf91df40 1 -- start start 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adb44470 0x55b4adc20a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adc0d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 0x55b4adb76e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4adc19370 con 0x55b4adc1d9c0 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4adc19630 con 0x55b4adb44470 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cf91df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b4adc199f0 con 0x55b4adc1a3e0 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 0x55b4adb76e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 0x55b4adb76e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44722/0 (socket says 172.21.15.28:44722) 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 -- 172.21.15.28:0/2388541562 learned_addr learned my addr 172.21.15.28:0/2388541562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cc000640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adb44470 0x55b4adc20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cb600640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adc0d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 -- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 msgr2=0x55b4adc0d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4adc1a3e0 0x55b4adc0d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 -- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adb44470 msgr2=0x55b4adc20a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4adb44470 0x55b4adc20a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.694+0000 7f54cca00640 1 -- 172.21.15.28:0/2388541562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4adc1be10 con 0x55b4adc1d9c0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54cca00640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 0x55b4adb76e10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005fd0 tx=0x7f54c4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54c8e00640 1 -- 172.21.15.28:0/2388541562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4065070 con 0x55b4adc1d9c0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54c8e00640 1 -- 172.21.15.28:0/2388541562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c404dc00 con 0x55b4adc1d9c0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54c8e00640 1 -- 172.21.15.28:0/2388541562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4005b40 con 0x55b4adc1d9c0 2024-09-10T10:50:39.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b4adc1e030 con 0x55b4adc1d9c0 2024-09-10T10:50:39.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.695+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4adc1e5f0 con 0x55b4adc1d9c0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.697+0000 7f54c8e00640 1 -- 172.21.15.28:0/2388541562 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54c4068070 con 0x55b4adc1d9c0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.698+0000 7f54c8e00640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ac06a840 0x7f54ac06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.698+0000 7f54c8e00640 1 -- 172.21.15.28:0/2388541562 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f54c40ce070 con 0x55b4adc1d9c0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.698+0000 7f54cf91df40 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b4adc18ee0 0x55b4adcb3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.698+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55b4adc0ded0 con 0x55b4adc18ee0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.698+0000 7f54cc000640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ac06a840 0x7f54ac06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.699+0000 7f54cb600640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b4adc18ee0 0x55b4adcb3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.699+0000 7f54cc000640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ac06a840 0x7f54ac06cd00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00b310 tx=0x7f54bc008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.699+0000 7f54cb600640 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b4adc18ee0 0x55b4adcb3070 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cb600640 1 -- 172.21.15.28:0/2388541562 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 600.00000000 [omap-get-keys out=5b] v0'0 uv2 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f54b80078d0 con 0x55b4adc18ee0 2024-09-10T10:50:39.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b4adc18ee0 msgr2=0x55b4adcb3070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b4adc18ee0 0x55b4adcb3070 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ac06a840 msgr2=0x7f54ac06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ac06a840 0x7f54ac06cd00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f54bc00b310 tx=0x7f54bc008000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 msgr2=0x55b4adb76e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 --2- 172.21.15.28:0/2388541562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4adc1d9c0 0x55b4adb76e10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005fd0 tx=0x7f54c4003330 comp rx=0 tx=0).stop 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cca00640 1 -- 172.21.15.28:0/2388541562 reap_dead start 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 shutdown_connections 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 >> 172.21.15.28:0/2388541562 conn(0x55b4adb930a0 msgr2=0x55b4adc17fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.700+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 shutdown_connections 2024-09-10T10:50:39.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.701+0000 7f54cf91df40 1 -- 172.21.15.28:0/2388541562 wait complete. 2024-09-10T10:50:39.705 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000.inode 2024-09-10T10:50:39.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.780+0000 7f6dd011bf40 1 Processor -- start 2024-09-10T10:50:39.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.780+0000 7f6dd011bf40 1 -- start start 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc1277ff0 0x564fc1303850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc13069c0 0x564fc122cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564fc1278530 con 0x564fc13033e0 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fc11d5b30 con 0x564fc13069c0 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564fc1305ee0 con 0x564fc1277ff0 2024-09-10T10:50:39.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44748/0 (socket says 172.21.15.28:44748) 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 -- 172.21.15.28:0/2256915407 learned_addr learned my addr 172.21.15.28:0/2256915407 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcc800640 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc1277ff0 0x564fc1303850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcd200640 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc13069c0 0x564fc122cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 -- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc1277ff0 msgr2=0x564fc1303850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc1277ff0 0x564fc1303850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 -- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc13069c0 msgr2=0x564fc122cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc13069c0 0x564fc122cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.781+0000 7f6dcbe00640 1 -- 172.21.15.28:0/2256915407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1303e30 con 0x564fc13033e0 2024-09-10T10:50:39.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6db8001df0 tx=0x7f6db800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8daa07f6f5701df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dcaa00640 1 -- 172.21.15.28:0/2256915407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800be30 con 0x564fc13033e0 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dcaa00640 1 -- 172.21.15.28:0/2256915407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8006070 con 0x564fc13033e0 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dcaa00640 1 -- 172.21.15.28:0/2256915407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800e7e0 con 0x564fc13033e0 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 -- 172.21.15.28:0/2256915407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 msgr2=0x564fc1260c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6db8001df0 tx=0x7f6db800b600 comp rx=0 tx=0).stop 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 -- 172.21.15.28:0/2256915407 shutdown_connections 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc13069c0 0x564fc122cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13033e0 0x564fc1260c90 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/2256915407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc1277ff0 0x564fc1303850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 -- 172.21.15.28:0/2256915407 >> 172.21.15.28:0/2256915407 conn(0x564fc127c0a0 msgr2=0x564fc127c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.782+0000 7f6dd011bf40 1 -- 172.21.15.28:0/2256915407 shutdown_connections 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.783+0000 7f6dd011bf40 1 -- 172.21.15.28:0/2256915407 wait complete. 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.783+0000 7f6dd011bf40 1 Processor -- start 2024-09-10T10:50:39.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.783+0000 7f6dd011bf40 1 -- start start 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc1277ff0 0x564fc12fb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc13033e0 0x564fc12f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 0x564fc12fb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564fc1304e00 con 0x564fc13069c0 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564fc1304110 con 0x564fc1277ff0 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dd011bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564fc13047b0 con 0x564fc13033e0 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.784+0000 7f6dcd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 0x564fc12fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcc800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc1277ff0 0x564fc12fb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 0x564fc12fb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44762/0 (socket says 172.21.15.28:44762) 2024-09-10T10:50:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 -- 172.21.15.28:0/4012805257 learned_addr learned my addr 172.21.15.28:0/4012805257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc13033e0 0x564fc12f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 -- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc13033e0 msgr2=0x564fc12f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564fc13033e0 0x564fc12f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 -- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc1277ff0 msgr2=0x564fc12fb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564fc1277ff0 0x564fc12fb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 -- 172.21.15.28:0/4012805257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc139cd40 con 0x564fc13069c0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dcd200640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 0x564fc12fb000 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400f8a0 tx=0x7f6dc400fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dc9600640 1 -- 172.21.15.28:0/4012805257 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4019020 con 0x564fc13069c0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.785+0000 7f6dc9600640 1 -- 172.21.15.28:0/4012805257 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc4002e30 con 0x564fc13069c0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.786+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564fc139c0c0 con 0x564fc13069c0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.786+0000 7f6dc9600640 1 -- 172.21.15.28:0/4012805257 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400db40 con 0x564fc13069c0 2024-09-10T10:50:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.786+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc139c960 con 0x564fc13069c0 2024-09-10T10:50:39.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.787+0000 7f6dc9600640 1 -- 172.21.15.28:0/4012805257 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6dc401c020 con 0x564fc13069c0 2024-09-10T10:50:39.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.788+0000 7f6dc9600640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6dac06a810 0x7f6dac06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.788+0000 7f6dc9600640 1 -- 172.21.15.28:0/4012805257 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f6dc408a3e0 con 0x564fc13069c0 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.789+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564fc12f7970 0x564fc12f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.789+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x564fc12f4380 con 0x564fc12f7970 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.789+0000 7f6dcc800640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6dac06a810 0x7f6dac06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.789+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564fc12f7970 0x564fc12f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dcc800640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6dac06a810 0x7f6dac06ccd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0057000 tx=0x7f6dc005e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dcbe00640 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564fc12f7970 0x564fc12f5cd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dcbe00640 1 -- 172.21.15.28:0/4012805257 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [omap-get-keys out=5b] v0'0 uv15 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7f6db800e640 con 0x564fc12f7970 2024-09-10T10:50:39.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564fc12f7970 msgr2=0x564fc12f5cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564fc12f7970 0x564fc12f5cd0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6dac06a810 msgr2=0x7f6dac06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6dac06a810 0x7f6dac06ccd0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0057000 tx=0x7f6dc005e040 comp rx=0 tx=0).stop 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 msgr2=0x564fc12fb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 --2- 172.21.15.28:0/4012805257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564fc13069c0 0x564fc12fb000 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400f8a0 tx=0x7f6dc400fd60 comp rx=0 tx=0).stop 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dcd200640 1 -- 172.21.15.28:0/4012805257 reap_dead start 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 shutdown_connections 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 >> 172.21.15.28:0/4012805257 conn(0x564fc127c0a0 msgr2=0x564fc127c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 shutdown_connections 2024-09-10T10:50:39.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.790+0000 7f6dd011bf40 1 -- 172.21.15.28:0/4012805257 wait complete. 2024-09-10T10:50:39.795 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000000 2024-09-10T10:50:39.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 Processor -- start 2024-09-10T10:50:39.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 -- start start 2024-09-10T10:50:39.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 0x55dff8bbf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c59de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dff8b3d530 con 0x55dff8af2470 2024-09-10T10:50:39.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dff8a9ab30 con 0x55dff8bcb9c0 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.870+0000 7f02a39caf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dff8bcaee0 con 0x55dff8bc83e0 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44772/0 (socket says 172.21.15.28:44772) 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 -- 172.21.15.28:0/2068957709 learned_addr learned my addr 172.21.15.28:0/2068957709 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 -- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 msgr2=0x55dff8bbf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a1600640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c59de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f029be00640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 0x55dff8bbf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 0x55dff8bbf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 -- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 msgr2=0x55dff8c59de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c59de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 -- 172.21.15.28:0/2068957709 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff8bbf890 con 0x55dff8af2470 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a1600640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c59de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f029be00640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 0x55dff8bbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.871+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0294002e10 tx=0x7f029400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c1db70d356b635 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f029aa00640 1 -- 172.21.15.28:0/2068957709 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294013020 con 0x55dff8af2470 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f029aa00640 1 -- 172.21.15.28:0/2068957709 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f029400f400 con 0x55dff8af2470 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f029aa00640 1 -- 172.21.15.28:0/2068957709 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400f7f0 con 0x55dff8af2470 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 -- 172.21.15.28:0/2068957709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 msgr2=0x55dff8bc9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0294002e10 tx=0x7f029400b5b0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 -- 172.21.15.28:0/2068957709 shutdown_connections 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c59de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8bc83e0 0x55dff8bbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2068957709 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8af2470 0x55dff8bc9c80 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 -- 172.21.15.28:0/2068957709 >> 172.21.15.28:0/2068957709 conn(0x55dff8b410a0 msgr2=0x55dff8bc8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 -- 172.21.15.28:0/2068957709 shutdown_connections 2024-09-10T10:50:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.872+0000 7f02a39caf40 1 -- 172.21.15.28:0/2068957709 wait complete. 2024-09-10T10:50:39.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.873+0000 7f02a39caf40 1 Processor -- start 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.873+0000 7f02a39caf40 1 -- start start 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8af2470 0x55dff8c640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 0x55dff8c61350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c61890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dff8b3fe60 con 0x55dff8bc83e0 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dff8bc9640 con 0x55dff8bcb9c0 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a39caf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dff8c54100 con 0x55dff8af2470 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f029be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 0x55dff8c61350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f029be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 0x55dff8c61350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44776/0 (socket says 172.21.15.28:44776) 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f029be00640 1 -- 172.21.15.28:0/2332384375 learned_addr learned my addr 172.21.15.28:0/2332384375 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8af2470 0x55dff8c640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.874+0000 7f02a1600640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c61890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 -- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8af2470 msgr2=0x55dff8c640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dff8af2470 0x55dff8c640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 -- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 msgr2=0x55dff8c61890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dff8bcb9c0 0x55dff8c61890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 -- 172.21.15.28:0/2332384375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff8bce370 con 0x55dff8bc83e0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f029be00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 0x55dff8c61350 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f028c007dd0 tx=0x7f028c006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f0299600640 1 -- 172.21.15.28:0/2332384375 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c010510 con 0x55dff8bc83e0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f0299600640 1 -- 172.21.15.28:0/2332384375 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028c0106b0 con 0x55dff8bc83e0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dff8bce650 con 0x55dff8bc83e0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f0299600640 1 -- 172.21.15.28:0/2332384375 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c010a80 con 0x55dff8bc83e0 2024-09-10T10:50:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.875+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff8c645f0 con 0x55dff8bc83e0 2024-09-10T10:50:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.877+0000 7f0299600640 1 -- 172.21.15.28:0/2332384375 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f028c010c20 con 0x55dff8bc83e0 2024-09-10T10:50:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.878+0000 7f0299600640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f028006a810 0x7f028006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.878+0000 7f0299600640 1 -- 172.21.15.28:0/2332384375 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f028c089520 con 0x55dff8bc83e0 2024-09-10T10:50:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.878+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dff8bbeb20 0x55dff8c622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.879+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55dff8bcce30 con 0x55dff8bbeb20 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.879+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f028006a810 0x7f028006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.879+0000 7f02a1600640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dff8bbeb20 0x55dff8c622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.879+0000 7f02a0c00640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f028006a810 0x7f028006ccd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f029400b5f0 tx=0x7f0294009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.879+0000 7f02a1600640 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dff8bbeb20 0x55dff8c622e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a1600640 1 -- 172.21.15.28:0/2332384375 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [omap-get-keys out=5b] v0'0 uv19 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f029c04e960 con 0x55dff8bbeb20 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dff8bbeb20 msgr2=0x55dff8c622e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dff8bbeb20 0x55dff8c622e0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f028006a810 msgr2=0x7f028006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f028006a810 0x7f028006ccd0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f029400b5f0 tx=0x7f0294009000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 msgr2=0x55dff8c61350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 --2- 172.21.15.28:0/2332384375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dff8bc83e0 0x55dff8c61350 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f028c007dd0 tx=0x7f028c006750 comp rx=0 tx=0).stop 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a1600640 1 -- 172.21.15.28:0/2332384375 reap_dead start 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 shutdown_connections 2024-09-10T10:50:39.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 >> 172.21.15.28:0/2332384375 conn(0x55dff8b410a0 msgr2=0x55dff8b23c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 shutdown_connections 2024-09-10T10:50:39.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.880+0000 7f02a39caf40 1 -- 172.21.15.28:0/2332384375 wait complete. 2024-09-10T10:50:39.885 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 200.00000001 2024-09-10T10:50:39.888 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:50:39.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:50:39.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.961+0000 7f1d16f22f40 1 Processor -- start 2024-09-10T10:50:39.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.961+0000 7f1d16f22f40 1 -- start start 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 0x55613d1a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1aa3e0 0x55613d1af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55613d11f530 con 0x55613d1ad9c0 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55613d11f2d0 con 0x55613d11e8e0 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55613d1acee0 con 0x55613d1aa3e0 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44796/0 (socket says 172.21.15.28:44796) 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 -- 172.21.15.28:0/1556419894 learned_addr learned my addr 172.21.15.28:0/1556419894 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d13800640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1aa3e0 0x55613d1af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14200640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 0x55613d1a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 -- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1aa3e0 msgr2=0x55613d1af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1aa3e0 0x55613d1af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 -- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 msgr2=0x55613d1a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 0x55613d1a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14c00640 1 -- 172.21.15.28:0/1556419894 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613d000b30 con 0x55613d1ad9c0 2024-09-10T10:50:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.962+0000 7f1d14200640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 0x55613d1a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d14c00640 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c002800 tx=0x7f1d0c002cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdaeeef7d91784f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d12400640 1 -- 172.21.15.28:0/1556419894 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c016020 con 0x55613d1ad9c0 2024-09-10T10:50:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d12400640 1 -- 172.21.15.28:0/1556419894 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00d9c0 con 0x55613d1ad9c0 2024-09-10T10:50:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d12400640 1 -- 172.21.15.28:0/1556419894 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00ddf0 con 0x55613d1ad9c0 2024-09-10T10:50:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 -- 172.21.15.28:0/1556419894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 msgr2=0x55613d1afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c002800 tx=0x7f1d0c002cc0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 -- 172.21.15.28:0/1556419894 shutdown_connections 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d1ad9c0 0x55613d1afed0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1aa3e0 0x55613d1af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/1556419894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d11e8e0 0x55613d1a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 -- 172.21.15.28:0/1556419894 >> 172.21.15.28:0/1556419894 conn(0x55613d1230a0 msgr2=0x55613d121210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 -- 172.21.15.28:0/1556419894 shutdown_connections 2024-09-10T10:50:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.963+0000 7f1d16f22f40 1 -- 172.21.15.28:0/1556419894 wait complete. 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.964+0000 7f1d16f22f40 1 Processor -- start 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.964+0000 7f1d16f22f40 1 -- start start 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 0x55613d102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d1aa3e0 0x55613d19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1ad9c0 0x55613d1044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55613d108860 con 0x55613d11e8e0 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55613d1aec10 con 0x55613d1aa3e0 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d16f22f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55613d1aeeb0 con 0x55613d1ad9c0 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d14200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 0x55613d102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d14200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 0x55613d102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44806/0 (socket says 172.21.15.28:44806) 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d14200640 1 -- 172.21.15.28:0/4220704780 learned_addr learned my addr 172.21.15.28:0/4220704780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d13800640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d1aa3e0 0x55613d19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.965+0000 7f1d14c00640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1ad9c0 0x55613d1044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 -- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1ad9c0 msgr2=0x55613d1044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55613d1ad9c0 0x55613d1044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 -- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d1aa3e0 msgr2=0x55613d19f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55613d1aa3e0 0x55613d19f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 -- 172.21.15.28:0/4220704780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613d1a0370 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d14200640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 0x55613d102af0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1d04008430 tx=0x7f1d04008520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d11000640 1 -- 172.21.15.28:0/4220704780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04008e10 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d11000640 1 -- 172.21.15.28:0/4220704780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d040145d0 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55613d1a0650 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.966+0000 7f1d11000640 1 -- 172.21.15.28:0/4220704780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04014980 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.967+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613d106b00 con 0x55613d11e8e0 2024-09-10T10:50:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.968+0000 7f1d11000640 1 -- 172.21.15.28:0/4220704780 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d0401c020 con 0x55613d11e8e0 2024-09-10T10:50:39.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.969+0000 7f1d11000640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cf406a810 0x7f1cf406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.969+0000 7f1d11000640 1 -- 172.21.15.28:0/4220704780 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f1d04089b60 con 0x55613d11e8e0 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.970+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55613d1aad60 0x55613d107020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.970+0000 7f1d13800640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cf406a810 0x7f1cf406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.970+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55613d107560 con 0x55613d1aad60 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.970+0000 7f1d14c00640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55613d1aad60 0x55613d107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.970+0000 7f1d13800640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cf406a810 0x7f1cf406ccd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1d00057000 tx=0x7f1d00052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d14c00640 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55613d1aad60 0x55613d107020 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d14c00640 1 -- 172.21.15.28:0/4220704780 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [omap-get-keys out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f1d0c00dcf0 con 0x55613d1aad60 2024-09-10T10:50:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55613d1aad60 msgr2=0x55613d107020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55613d1aad60 0x55613d107020 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cf406a810 msgr2=0x7f1cf406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1cf406a810 0x7f1cf406ccd0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1d00057000 tx=0x7f1d00052000 comp rx=0 tx=0).stop 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 msgr2=0x55613d102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 --2- 172.21.15.28:0/4220704780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55613d11e8e0 0x55613d102af0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1d04008430 tx=0x7f1d04008520 comp rx=0 tx=0).stop 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d14c00640 1 -- 172.21.15.28:0/4220704780 reap_dead start 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 shutdown_connections 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 >> 172.21.15.28:0/4220704780 conn(0x55613d1230a0 msgr2=0x55613d1a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 shutdown_connections 2024-09-10T10:50:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:39.971+0000 7f1d16f22f40 1 -- 172.21.15.28:0/4220704780 wait complete. 2024-09-10T10:50:39.977 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_openfiles.0 2024-09-10T10:50:40.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 Processor -- start 2024-09-10T10:50:40.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 -- start start 2024-09-10T10:50:40.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccb71ff0 0x5635ccbfaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccb27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5635ccb72530 con 0x5635ccbfd3e0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635ccacfb30 con 0x5635ccb71ff0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.107+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5635ccbffee0 con 0x5635ccc009c0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44834/0 (socket says 172.21.15.28:44834) 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 -- 172.21.15.28:0/182160563 learned_addr learned my addr 172.21.15.28:0/182160563 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64be00640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccb27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64b400640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccb71ff0 0x5635ccbfaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 -- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 msgr2=0x5635ccb27000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccb27000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 -- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccb71ff0 msgr2=0x5635ccbfaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccb71ff0 0x5635ccbfaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 -- 172.21.15.28:0/182160563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ccbff7c0 con 0x5635ccbfd3e0 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64be00640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccb27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.108+0000 7fd64aa00640 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd638001d10 tx=0x7fd63800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fc27bf853cde2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd649600640 1 -- 172.21.15.28:0/182160563 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd638001e60 con 0x5635ccbfd3e0 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd649600640 1 -- 172.21.15.28:0/182160563 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63800bd80 con 0x5635ccbfd3e0 2024-09-10T10:50:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd649600640 1 -- 172.21.15.28:0/182160563 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63800f640 con 0x5635ccbfd3e0 2024-09-10T10:50:40.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 -- 172.21.15.28:0/182160563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 msgr2=0x5635ccb26ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fd638001d10 tx=0x7fd63800b5b0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 -- 172.21.15.28:0/182160563 shutdown_connections 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccb27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccbfd3e0 0x5635ccb26ac0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/182160563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccb71ff0 0x5635ccbfaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 -- 172.21.15.28:0/182160563 >> 172.21.15.28:0/182160563 conn(0x5635ccb760a0 msgr2=0x5635ccbfd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 -- 172.21.15.28:0/182160563 shutdown_connections 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.109+0000 7fd64e2fef40 1 -- 172.21.15.28:0/182160563 wait complete. 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.110+0000 7fd64e2fef40 1 Processor -- start 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.110+0000 7fd64e2fef40 1 -- start start 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 0x5635ccbed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccbfd3e0 0x5635ccc03ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccbfda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5635ccbff090 con 0x5635ccb71ff0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635ccbff2e0 con 0x5635ccbfd3e0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64e2fef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5635ccb59570 con 0x5635ccc009c0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 0x5635ccbed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 0x5635ccbed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44848/0 (socket says 172.21.15.28:44848) 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64b400640 1 -- 172.21.15.28:0/3712365952 learned_addr learned my addr 172.21.15.28:0/3712365952 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64aa00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccbfd3e0 0x5635ccc03ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.111+0000 7fd64be00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccbfda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 -- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 msgr2=0x5635ccbfda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccbfda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 -- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccbfd3e0 msgr2=0x5635ccc03ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5635ccbfd3e0 0x5635ccc03ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 -- 172.21.15.28:0/3712365952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ccc04020 con 0x5635ccb71ff0 2024-09-10T10:50:40.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64be00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5635ccc009c0 0x5635ccbfda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64b400640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 0x5635ccbed180 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd63c025d20 tx=0x7fd63c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd643e00640 1 -- 172.21.15.28:0/3712365952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c055070 con 0x5635ccb71ff0 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd643e00640 1 -- 172.21.15.28:0/3712365952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63c0519c0 con 0x5635ccb71ff0 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd643e00640 1 -- 172.21.15.28:0/3712365952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c051db0 con 0x5635ccb71ff0 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5635ccbedeb0 con 0x5635ccb71ff0 2024-09-10T10:50:40.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.112+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ccbee470 con 0x5635ccb71ff0 2024-09-10T10:50:40.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.114+0000 7fd643e00640 1 -- 172.21.15.28:0/3712365952 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd63c058070 con 0x5635ccb71ff0 2024-09-10T10:50:40.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.115+0000 7fd643e00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd62c06a810 0x7fd62c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.115+0000 7fd643e00640 1 -- 172.21.15.28:0/3712365952 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd63c0cd940 con 0x5635ccb71ff0 2024-09-10T10:50:40.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.115+0000 7fd64aa00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd62c06a810 0x7fd62c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.115+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5635ccbfdf80 0x5635ccbf21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.115+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5635ccbf26e0 con 0x5635ccbfdf80 2024-09-10T10:50:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.116+0000 7fd64be00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5635ccbfdf80 0x5635ccbf21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.116+0000 7fd64aa00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd62c06a810 0x7fd62c06ccd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5635ccb59450 tx=0x7fd638009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.116+0000 7fd64be00640 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5635ccbfdf80 0x5635ccbf21a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64be00640 1 -- 172.21.15.28:0/3712365952 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-get-keys out=5b] v0'0 uv4 ondisk = 0) ==== 160+0+5 (crc 0 0 0) 0x7fd644011040 con 0x5635ccbfdf80 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5635ccbfdf80 msgr2=0x5635ccbf21a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5635ccbfdf80 0x5635ccbf21a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd62c06a810 msgr2=0x7fd62c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd62c06a810 0x7fd62c06ccd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x5635ccb59450 tx=0x7fd638009000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 msgr2=0x5635ccbed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 --2- 172.21.15.28:0/3712365952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635ccb71ff0 0x5635ccbed180 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fd63c025d20 tx=0x7fd63c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64be00640 1 -- 172.21.15.28:0/3712365952 reap_dead start 2024-09-10T10:50:40.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 shutdown_connections 2024-09-10T10:50:40.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 >> 172.21.15.28:0/3712365952 conn(0x5635ccb760a0 msgr2=0x5635ccb58c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 shutdown_connections 2024-09-10T10:50:40.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.117+0000 7fd64e2fef40 1 -- 172.21.15.28:0/3712365952 wait complete. 2024-09-10T10:50:40.122 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 500.00000000 2024-09-10T10:50:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.197+0000 7f60e04eaf40 1 Processor -- start 2024-09-10T10:50:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.197+0000 7f60e04eaf40 1 -- start start 2024-09-10T10:50:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909902bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559098fa2b30 con 0x559098ffa470 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559098fa6b70 con 0x5590990d39c0 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5590990d2ee0 con 0x5590990d03e0 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44858/0 (socket says 172.21.15.28:44858) 2024-09-10T10:50:40.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 -- 172.21.15.28:0/2774684863 learned_addr learned my addr 172.21.15.28:0/2774684863 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dcc00640 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60de000640 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909902bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 -- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 msgr2=0x5590990ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 -- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 msgr2=0x55909902bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909902bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.198+0000 7f60dd600640 1 -- 172.21.15.28:0/2774684863 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559098ffcc20 con 0x559098ffa470 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60dd600640 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f60d000b4a0 tx=0x7f60d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e106f70a1c713fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60d7400640 1 -- 172.21.15.28:0/2774684863 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0012070 con 0x559098ffa470 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60d7400640 1 -- 172.21.15.28:0/2774684863 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d000d6b0 con 0x559098ffa470 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60d7400640 1 -- 172.21.15.28:0/2774684863 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000daa0 con 0x559098ffa470 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2774684863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 msgr2=0x559099049650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f60d000b4a0 tx=0x7f60d000b960 comp rx=0 tx=0).stop 2024-09-10T10:50:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2774684863 shutdown_connections 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909902bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2774684863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x559099049650 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2774684863 >> 172.21.15.28:0/2774684863 conn(0x5590990490a0 msgr2=0x5590990d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.199+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2774684863 shutdown_connections 2024-09-10T10:50:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.200+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2774684863 wait complete. 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.200+0000 7f60e04eaf40 1 Processor -- start 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.200+0000 7f60e04eaf40 1 -- start start 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x5590990c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909916c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5590990cfc20 con 0x559098ffa470 2024-09-10T10:50:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5590990d62a0 con 0x5590990d39c0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60e04eaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5590990cf940 con 0x5590990d03e0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60dd600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x5590990c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60dd600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x5590990c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44868/0 (socket says 172.21.15.28:44868) 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.201+0000 7f60dd600640 1 -- 172.21.15.28:0/2831075205 learned_addr learned my addr 172.21.15.28:0/2831075205 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dcc00640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60de000640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909916c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 -- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 msgr2=0x5590990c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5590990d03e0 0x5590990c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 -- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 msgr2=0x55909916c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590990d39c0 0x55909916c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 -- 172.21.15.28:0/2831075205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590990d6960 con 0x559098ffa470 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60dd600640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x5590990c73c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f60d0000c00 tx=0x7f60d0001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60d6000640 1 -- 172.21.15.28:0/2831075205 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d0012050 con 0x559098ffa470 2024-09-10T10:50:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.202+0000 7f60d6000640 1 -- 172.21.15.28:0/2831075205 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60d000e680 con 0x559098ffa470 2024-09-10T10:50:40.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.203+0000 7f60d6000640 1 -- 172.21.15.28:0/2831075205 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d000ea70 con 0x559098ffa470 2024-09-10T10:50:40.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.203+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5590990cb490 con 0x559098ffa470 2024-09-10T10:50:40.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.203+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590990c6b90 con 0x559098ffa470 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.205+0000 7f60d6000640 1 -- 172.21.15.28:0/2831075205 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60d0015070 con 0x559098ffa470 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.205+0000 7f60d6000640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60c006a810 0x7f60c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.205+0000 7f60d6000640 1 -- 172.21.15.28:0/2831075205 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f60d008b5a0 con 0x559098ffa470 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.206+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5590990c6450 0x5590990cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.206+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5590990cba30 con 0x5590990c6450 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.206+0000 7f60dcc00640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60c006a810 0x7f60c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.206+0000 7f60de000640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5590990c6450 0x5590990cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.206+0000 7f60de000640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5590990c6450 0x5590990cc880 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60dcc00640 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60c006a810 0x7f60c006ccd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f60cc056000 tx=0x7f60cc050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60de000640 1 -- 172.21.15.28:0/2831075205 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f60d8006020 con 0x5590990c6450 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5590990c6450 msgr2=0x5590990cc880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5590990c6450 0x5590990cc880 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60c006a810 msgr2=0x7f60c006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60c006a810 0x7f60c006ccd0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f60cc056000 tx=0x7f60cc050000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 msgr2=0x5590990c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 --2- 172.21.15.28:0/2831075205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559098ffa470 0x5590990c73c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f60d0000c00 tx=0x7f60d0001150 comp rx=0 tx=0).stop 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60de000640 1 -- 172.21.15.28:0/2831075205 reap_dead start 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 shutdown_connections 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 >> 172.21.15.28:0/2831075205 conn(0x5590990490a0 msgr2=0x5590990d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 shutdown_connections 2024-09-10T10:50:40.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.207+0000 7f60e04eaf40 1 -- 172.21.15.28:0/2831075205 wait complete. 2024-09-10T10:50:40.212 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds_snaptable 2024-09-10T10:50:40.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.289+0000 7f0af3feaf40 1 Processor -- start 2024-09-10T10:50:40.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.289+0000 7f0af3feaf40 1 -- start start 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 0x55bf354086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 0x55bf3540fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bf352dfb30 con 0x55bf3540d4b0 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf352e3b70 con 0x55bf353812d0 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bf353681e0 con 0x55bf3533b5a0 2024-09-10T10:50:40.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44874/0 (socket says 172.21.15.28:44874) 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 -- 172.21.15.28:0/784503492 learned_addr learned my addr 172.21.15.28:0/784503492 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0aebe00640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 0x55bf3540fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1200640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 0x55bf354086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 -- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 msgr2=0x55bf354086d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 0x55bf354086d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 -- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 msgr2=0x55bf3540fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 0x55bf3540fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1c00640 1 -- 172.21.15.28:0/784503492 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf35368710 con 0x55bf3540d4b0 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0af1200640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 0x55bf354086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.290+0000 7f0aebe00640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 0x55bf3540fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.291+0000 7f0af1c00640 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0aec006f80 tx=0x7f0aec001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1318b3b929adde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.291+0000 7f0aeaa00640 1 -- 172.21.15.28:0/784503492 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec017070 con 0x55bf3540d4b0 2024-09-10T10:50:40.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.291+0000 7f0aeaa00640 1 -- 172.21.15.28:0/784503492 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec0026b0 con 0x55bf3540d4b0 2024-09-10T10:50:40.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.291+0000 7f0aeaa00640 1 -- 172.21.15.28:0/784503492 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec002aa0 con 0x55bf3540d4b0 2024-09-10T10:50:40.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 -- 172.21.15.28:0/784503492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 msgr2=0x55bf35337e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0aec006f80 tx=0x7f0aec001ac0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 -- 172.21.15.28:0/784503492 shutdown_connections 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf3540d4b0 0x55bf35337e70 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf353812d0 0x55bf3540fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/784503492 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3533b5a0 0x55bf354086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 -- 172.21.15.28:0/784503492 >> 172.21.15.28:0/784503492 conn(0x55bf35365b90 msgr2=0x55bf3533b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 -- 172.21.15.28:0/784503492 shutdown_connections 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 -- 172.21.15.28:0/784503492 wait complete. 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.292+0000 7f0af3feaf40 1 Processor -- start 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 -- start start 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf3533b5a0 0x55bf3540bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 0x55bf35404970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3540d4b0 0x55bf354073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bf3540c280 con 0x55bf353812d0 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf35410e40 con 0x55bf3533b5a0 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.293+0000 7f0af3feaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bf35368430 con 0x55bf3540d4b0 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 0x55bf35404970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 0x55bf35404970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44890/0 (socket says 172.21.15.28:44890) 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 -- 172.21.15.28:0/124043083 learned_addr learned my addr 172.21.15.28:0/124043083 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 -- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3540d4b0 msgr2=0x55bf354073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bf3540d4b0 0x55bf354073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 -- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf3533b5a0 msgr2=0x55bf3540bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0af1200640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf3533b5a0 0x55bf3540bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf3533b5a0 0x55bf3540bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 -- 172.21.15.28:0/124043083 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf354aa670 con 0x55bf353812d0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0af1200640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf3533b5a0 0x55bf3540bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.294+0000 7f0aebe00640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 0x55bf35404970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55bf35366c50 tx=0x7f0adc00a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.295+0000 7f0ae9600640 1 -- 172.21.15.28:0/124043083 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc008040 con 0x55bf353812d0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.295+0000 7f0ae9600640 1 -- 172.21.15.28:0/124043083 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc014020 con 0x55bf353812d0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.295+0000 7f0ae9600640 1 -- 172.21.15.28:0/124043083 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00f620 con 0x55bf353812d0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.295+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bf35406be0 con 0x55bf353812d0 2024-09-10T10:50:40.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.295+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3540fa80 con 0x55bf353812d0 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.297+0000 7f0ae9600640 1 -- 172.21.15.28:0/124043083 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0adc00f7c0 con 0x55bf353812d0 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0ae9600640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006a810 0x7f0ad006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0ae9600640 1 -- 172.21.15.28:0/124043083 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f0adc089df0 con 0x55bf353812d0 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bf354aa190 0x55bf35409360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55bf3540e3d0 con 0x55bf354aa190 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0af1200640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006a810 0x7f0ad006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.298+0000 7f0af1c00640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bf354aa190 0x55bf35409360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af1c00640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bf354aa190 0x55bf35409360 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af1200640 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006a810 0x7f0ad006ccd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4057000 tx=0x7f0ae4052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af1c00640 1 -- 172.21.15.28:0/124043083 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f0aec00f070 con 0x55bf354aa190 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bf354aa190 msgr2=0x55bf35409360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bf354aa190 0x55bf35409360 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006a810 msgr2=0x7f0ad006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006a810 0x7f0ad006ccd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4057000 tx=0x7f0ae4052000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 msgr2=0x55bf35404970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 --2- 172.21.15.28:0/124043083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bf353812d0 0x55bf35404970 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x55bf35366c50 tx=0x7f0adc00a690 comp rx=0 tx=0).stop 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af1c00640 1 -- 172.21.15.28:0/124043083 reap_dead start 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 shutdown_connections 2024-09-10T10:50:40.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 >> 172.21.15.28:0/124043083 conn(0x55bf35365b90 msgr2=0x55bf3540c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 shutdown_connections 2024-09-10T10:50:40.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.299+0000 7f0af3feaf40 1 -- 172.21.15.28:0/124043083 wait complete. 2024-09-10T10:50:40.305 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_inotable 2024-09-10T10:50:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.380+0000 7f2eaa5e3f40 1 Processor -- start 2024-09-10T10:50:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.380+0000 7f2eaa5e3f40 1 -- start start 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f42a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f42abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558b3f429dc0 con 0x558b3f3e45a0 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b3f388b30 con 0x558b3f4b64b0 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558b3f4291f0 con 0x558b3f42a2d0 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2ea7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2ea7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44900/0 (socket says 172.21.15.28:44900) 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2ea7800640 1 -- 172.21.15.28:0/2206187738 learned_addr learned my addr 172.21.15.28:0/2206187738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2ea6e00640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f42a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.381+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f42abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 -- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 msgr2=0x558b3f42a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f42a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 -- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 msgr2=0x558b3f42abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f42abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 -- 172.21.15.28:0/2206187738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3f422e10 con 0x558b3f3e45a0 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f42abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c5544aad309105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea5a00640 1 -- 172.21.15.28:0/2206187738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98055070 con 0x558b3f3e45a0 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea5a00640 1 -- 172.21.15.28:0/2206187738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e980517d0 con 0x558b3f3e45a0 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.382+0000 7f2ea5a00640 1 -- 172.21.15.28:0/2206187738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98051b80 con 0x558b3f3e45a0 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2206187738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 msgr2=0x558b3f4b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98051000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2206187738 shutdown_connections 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f42abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f42a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2206187738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f4b51c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2206187738 >> 172.21.15.28:0/2206187738 conn(0x558b3f40eb90 msgr2=0x558b3f4b9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2206187738 shutdown_connections 2024-09-10T10:50:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2206187738 wait complete. 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.383+0000 7f2eaa5e3f40 1 Processor -- start 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 -- start start 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f425cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f551c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f423e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558b3f4233d0 con 0x558b3f3e45a0 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b3f4230f0 con 0x558b3f4b64b0 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.384+0000 7f2eaa5e3f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558b3f4236b0 con 0x558b3f42a2d0 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f425cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f425cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44916/0 (socket says 172.21.15.28:44916) 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 -- 172.21.15.28:0/2802432439 learned_addr learned my addr 172.21.15.28:0/2802432439 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea6e00640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f551c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f423e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 -- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 msgr2=0x558b3f551c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f551c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 -- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 msgr2=0x558b3f423e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f423e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 -- 172.21.15.28:0/2802432439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3f40c0f0 con 0x558b3f3e45a0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b3f4b64b0 0x558b3f423e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea6e00640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b3f42a2d0 0x558b3f551c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.385+0000 7f2ea7800640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f425cd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2e98046070 tx=0x7f2e98002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.386+0000 7f2e9fe00640 1 -- 172.21.15.28:0/2802432439 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98055070 con 0x558b3f3e45a0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.386+0000 7f2e9fe00640 1 -- 172.21.15.28:0/2802432439 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e98051db0 con 0x558b3f3e45a0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.386+0000 7f2e9fe00640 1 -- 172.21.15.28:0/2802432439 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980656a0 con 0x558b3f3e45a0 2024-09-10T10:50:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.386+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558b3f40c3d0 con 0x558b3f3e45a0 2024-09-10T10:50:40.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.386+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3f40c6b0 con 0x558b3f3e45a0 2024-09-10T10:50:40.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.387+0000 7f2e9fe00640 1 -- 172.21.15.28:0/2802432439 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e98058070 con 0x558b3f3e45a0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.388+0000 7f2e9fe00640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e8806a810 0x7f2e8806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.388+0000 7f2e9fe00640 1 -- 172.21.15.28:0/2802432439 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f2e980ceb00 con 0x558b3f3e45a0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b3f3e1e80 0x558b3f3e2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x558b3f3e27d0 con 0x558b3f3e1e80 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2ea6e00640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e8806a810 0x7f2e8806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b3f3e1e80 0x558b3f3e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2ea6e00640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e8806a810 0x7f2e8806ccd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x558b3f4251f0 tx=0x7f2e94003420 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.389+0000 7f2ea8200640 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b3f3e1e80 0x558b3f3e2290 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2ea8200640 1 -- 172.21.15.28:0/2802432439 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 157+0+5 (crc 0 0 0) 0x7f2ea00029e0 con 0x558b3f3e1e80 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b3f3e1e80 msgr2=0x558b3f3e2290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b3f3e1e80 0x558b3f3e2290 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e8806a810 msgr2=0x7f2e8806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e8806a810 0x7f2e8806ccd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x558b3f4251f0 tx=0x7f2e94003420 comp rx=0 tx=0).stop 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 msgr2=0x558b3f425cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 --2- 172.21.15.28:0/2802432439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b3f3e45a0 0x558b3f425cd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2e98046070 tx=0x7f2e98002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2ea8200640 1 -- 172.21.15.28:0/2802432439 reap_dead start 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 shutdown_connections 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 >> 172.21.15.28:0/2802432439 conn(0x558b3f40eb90 msgr2=0x558b3f4b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 shutdown_connections 2024-09-10T10:50:40.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.390+0000 7f2eaa5e3f40 1 -- 172.21.15.28:0/2802432439 wait complete. 2024-09-10T10:50:40.396 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000 2024-09-10T10:50:40.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.470+0000 7fe77191ef40 1 Processor -- start 2024-09-10T10:50:40.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.470+0000 7fe77191ef40 1 -- start start 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda70fff0 0x562eda6c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda79e9c0 0x562eda799bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562eda710530 con 0x562eda79b3e0 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562eda66db30 con 0x562eda70fff0 2024-09-10T10:50:40.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe77191ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562eda79dee0 con 0x562eda79e9c0 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe76d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe76d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44940/0 (socket says 172.21.15.28:44940) 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe76d600640 1 -- 172.21.15.28:0/2225693707 learned_addr learned my addr 172.21.15.28:0/2225693707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe76ea00640 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda79e9c0 0x562eda799bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.471+0000 7fe76e000640 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda70fff0 0x562eda6c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 -- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda79e9c0 msgr2=0x562eda799bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda79e9c0 0x562eda799bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 -- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda70fff0 msgr2=0x562eda6c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda70fff0 0x562eda6c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 -- 172.21.15.28:0/2225693707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eda79bcc0 con 0x562eda79b3e0 2024-09-10T10:50:40.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe76d600640 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00a430 tx=0x7fe75c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2589795c32d7d265 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe767e00640 1 -- 172.21.15.28:0/2225693707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b920 con 0x562eda79b3e0 2024-09-10T10:50:40.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe767e00640 1 -- 172.21.15.28:0/2225693707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c00f030 con 0x562eda79b3e0 2024-09-10T10:50:40.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.472+0000 7fe767e00640 1 -- 172.21.15.28:0/2225693707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00bd20 con 0x562eda79b3e0 2024-09-10T10:50:40.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 -- 172.21.15.28:0/2225693707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 msgr2=0x562eda799680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00a430 tx=0x7fe75c00a8f0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 -- 172.21.15.28:0/2225693707 shutdown_connections 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda79e9c0 0x562eda799bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda799680 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 --2- 172.21.15.28:0/2225693707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda70fff0 0x562eda6c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 -- 172.21.15.28:0/2225693707 >> 172.21.15.28:0/2225693707 conn(0x562eda7140a0 msgr2=0x562eda714480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 -- 172.21.15.28:0/2225693707 shutdown_connections 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.473+0000 7fe77191ef40 1 -- 172.21.15.28:0/2225693707 wait complete. 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.474+0000 7fe77191ef40 1 Processor -- start 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.474+0000 7fe77191ef40 1 -- start start 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.474+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda70fff0 0x562eda794c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.474+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda7951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.474+0000 7fe77191ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda79e9c0 0x562eda7a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe77191ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562eda79cbf0 con 0x562eda79b3e0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe77191ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562eda79c910 con 0x562eda79e9c0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe77191ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562eda79ced0 con 0x562eda70fff0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda7951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda7951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44946/0 (socket says 172.21.15.28:44946) 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 -- 172.21.15.28:0/539261173 learned_addr learned my addr 172.21.15.28:0/539261173 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76e000640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda70fff0 0x562eda794c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76ea00640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda79e9c0 0x562eda7a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 -- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda70fff0 msgr2=0x562eda794c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562eda70fff0 0x562eda794c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 -- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda79e9c0 msgr2=0x562eda7a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562eda79e9c0 0x562eda7a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.475+0000 7fe76d600640 1 -- 172.21.15.28:0/539261173 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eda792530 con 0x562eda79b3e0 2024-09-10T10:50:40.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe76d600640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda7951c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x562eda6f86d0 tx=0x7fe75c002f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe766a00640 1 -- 172.21.15.28:0/539261173 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c024070 con 0x562eda79b3e0 2024-09-10T10:50:40.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe766a00640 1 -- 172.21.15.28:0/539261173 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe75c014420 con 0x562eda79b3e0 2024-09-10T10:50:40.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe766a00640 1 -- 172.21.15.28:0/539261173 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c014810 con 0x562eda79b3e0 2024-09-10T10:50:40.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562eda792810 con 0x562eda79b3e0 2024-09-10T10:50:40.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.476+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eda712120 con 0x562eda79b3e0 2024-09-10T10:50:40.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.478+0000 7fe766a00640 1 -- 172.21.15.28:0/539261173 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe75c027070 con 0x562eda79b3e0 2024-09-10T10:50:40.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe766a00640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75006a810 0x7fe75006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe766a00640 1 -- 172.21.15.28:0/539261173 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fe75c08aad0 con 0x562eda79b3e0 2024-09-10T10:50:40.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe77191ef40 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562eda7803d0 0x562eda840330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x562eda7966a0 con 0x562eda7803d0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe76e000640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75006a810 0x7fe75006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.479+0000 7fe76ea00640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562eda7803d0 0x562eda840330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe76ea00640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562eda7803d0 0x562eda840330 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe76e000640 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75006a810 0x7fe75006ccd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe760057000 tx=0x7fe760052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe76ea00640 1 -- 172.21.15.28:0/539261173 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [omap-get-keys out=155b] v0'0 uv1 ondisk = 0) ==== 156+0+155 (crc 0 0 0) 0x7fe768002940 con 0x562eda7803d0 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stdout:stray0_head 2024-09-10T10:50:40.482 INFO:teuthology.orchestra.run.smithi028.stdout:stray1_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray2_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray3_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray4_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray5_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray6_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray7_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray8_head 2024-09-10T10:50:40.483 INFO:teuthology.orchestra.run.smithi028.stdout:stray9_head 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562eda7803d0 msgr2=0x562eda840330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562eda7803d0 0x562eda840330 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75006a810 msgr2=0x7fe75006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe75006a810 0x7fe75006ccd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe760057000 tx=0x7fe760052000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 msgr2=0x562eda7951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe77191ef40 1 --2- 172.21.15.28:0/539261173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562eda79b3e0 0x562eda7951c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x562eda6f86d0 tx=0x7fe75c002f70 comp rx=0 tx=0).stop 2024-09-10T10:50:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.480+0000 7fe76ea00640 1 -- 172.21.15.28:0/539261173 reap_dead start 2024-09-10T10:50:40.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.481+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 shutdown_connections 2024-09-10T10:50:40.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.481+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 >> 172.21.15.28:0/539261173 conn(0x562eda7140a0 msgr2=0x562eda714480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.481+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 shutdown_connections 2024-09-10T10:50:40.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.481+0000 7fe77191ef40 1 -- 172.21.15.28:0/539261173 wait complete. 2024-09-10T10:50:40.486 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys mds0_sessionmap 2024-09-10T10:50:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.561+0000 7fdfb6192f40 1 Processor -- start 2024-09-10T10:50:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.561+0000 7fdfb6192f40 1 -- start start 2024-09-10T10:50:40.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d20bcff0 0x55f6d20a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d213ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f6d201ab30 con 0x55f6d21483e0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6d201eb70 con 0x55f6d214b9c0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f6d214aee0 con 0x55f6d20bcff0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44954/0 (socket says 172.21.15.28:44954) 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb2a00640 1 -- 172.21.15.28:0/1939860565 learned_addr learned my addr 172.21.15.28:0/1939860565 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb3400640 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d20bcff0 0x55f6d20a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.562+0000 7fdfb3e00640 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d213ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 -- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d20bcff0 msgr2=0x55f6d20a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d20bcff0 0x55f6d20a5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 -- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 msgr2=0x55f6d213ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d213ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 -- 172.21.15.28:0/1939860565 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d20a40a0 con 0x55f6d21483e0 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e00 tx=0x7fdfa000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33984e0bda25112 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb1600640 1 -- 172.21.15.28:0/1939860565 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0012020 con 0x55f6d21483e0 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb1600640 1 -- 172.21.15.28:0/1939860565 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa000e670 con 0x55f6d21483e0 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb1600640 1 -- 172.21.15.28:0/1939860565 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000ea60 con 0x55f6d21483e0 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1939860565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 msgr2=0x55f6d20bf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.563+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e00 tx=0x7fdfa000b780 comp rx=0 tx=0).stop 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1939860565 shutdown_connections 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d213ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d21483e0 0x55f6d20bf460 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1939860565 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d20bcff0 0x55f6d20a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1939860565 >> 172.21.15.28:0/1939860565 conn(0x55f6d20c10a0 msgr2=0x55f6d21487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1939860565 shutdown_connections 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1939860565 wait complete. 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.564+0000 7fdfb6192f40 1 Processor -- start 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 -- start start 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 0x55f6d20bada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d21483e0 0x55f6d21465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d21e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f6d214a020 con 0x55f6d20bcff0 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6d20a47a0 con 0x55f6d214b9c0 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.565+0000 7fdfb6192f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f6d214a620 con 0x55f6d21483e0 2024-09-10T10:50:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 0x55f6d20bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 0x55f6d20bada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44962/0 (socket says 172.21.15.28:44962) 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 -- 172.21.15.28:0/1248886331 learned_addr learned my addr 172.21.15.28:0/1248886331 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d21483e0 0x55f6d21465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3e00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d21e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 -- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d21483e0 msgr2=0x55f6d21465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f6d21483e0 0x55f6d21465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 -- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 msgr2=0x55f6d21e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6d214b9c0 0x55f6d21e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 -- 172.21.15.28:0/1248886331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d20bb2e0 con 0x55f6d20bcff0 2024-09-10T10:50:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.566+0000 7fdfb3400640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 0x55f6d20bada0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4000c00 tx=0x7fdfa4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.567+0000 7fdfabe00640 1 -- 172.21.15.28:0/1248886331 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4053460 con 0x55f6d20bcff0 2024-09-10T10:50:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.567+0000 7fdfabe00640 1 -- 172.21.15.28:0/1248886331 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa4053600 con 0x55f6d20bcff0 2024-09-10T10:50:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.567+0000 7fdfabe00640 1 -- 172.21.15.28:0/1248886331 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40539b0 con 0x55f6d20bcff0 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.567+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f6d2145d90 con 0x55f6d20bcff0 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.567+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d213b8c0 con 0x55f6d20bcff0 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.569+0000 7fdfabe00640 1 -- 172.21.15.28:0/1248886331 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfa4053b50 con 0x55f6d20bcff0 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.569+0000 7fdfabe00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf9406a810 0x7fdf9406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.570+0000 7fdfabe00640 1 -- 172.21.15.28:0/1248886331 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fdfa40cdf90 con 0x55f6d20bcff0 2024-09-10T10:50:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.570+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf9406a810 0x7fdf9406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.570+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f6d214ef60 0x55f6d21e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.570+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55f6d2146b10 con 0x55f6d214ef60 2024-09-10T10:50:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.570+0000 7fdfb3e00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f6d214ef60 0x55f6d21e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb2a00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf9406a810 0x7fdf9406ccd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e00 tx=0x7fdfa001b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb3e00640 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f6d214ef60 0x55f6d21e5200 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb3e00640 1 -- 172.21.15.28:0/1248886331 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_sessionmap [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 159+0+5 (crc 0 0 0) 0x7fdfac002840 con 0x55f6d214ef60 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f6d214ef60 msgr2=0x55f6d21e5200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f6d214ef60 0x55f6d21e5200 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf9406a810 msgr2=0x7fdf9406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf9406a810 0x7fdf9406ccd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0001e00 tx=0x7fdfa001b040 comp rx=0 tx=0).stop 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 msgr2=0x55f6d20bada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 --2- 172.21.15.28:0/1248886331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f6d20bcff0 0x55f6d20bada0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4000c00 tx=0x7fdfa4002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb3e00640 1 -- 172.21.15.28:0/1248886331 reap_dead start 2024-09-10T10:50:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 shutdown_connections 2024-09-10T10:50:40.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 >> 172.21.15.28:0/1248886331 conn(0x55f6d20c10a0 msgr2=0x55f6d20bf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 shutdown_connections 2024-09-10T10:50:40.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.571+0000 7fdfb6192f40 1 -- 172.21.15.28:0/1248886331 wait complete. 2024-09-10T10:50:40.576 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 400.00000000 2024-09-10T10:50:40.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.654+0000 7f3011a1ef40 1 Processor -- start 2024-09-10T10:50:40.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 -- start start 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 0x555a9c6c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a9c645c40 con 0x555a9c641730 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a9c5a1b30 con 0x555a9c642170 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.655+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555a9c6d09c0 con 0x555a9c643af0 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44984/0 (socket says 172.21.15.28:44984) 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 -- 172.21.15.28:0/2342567958 learned_addr learned my addr 172.21.15.28:0/2342567958 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300d600640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 0x555a9c6c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300ea00640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 -- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 msgr2=0x555a9c6c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 -- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 msgr2=0x555a9c6c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 0x555a9c6c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 -- 172.21.15.28:0/2342567958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9c5fcd10 con 0x555a9c641730 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300d600640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 0x555a9c6c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300ea00640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.656+0000 7f300e000640 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30000098e0 tx=0x7f3000005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb3001018c6e7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3007e00640 1 -- 172.21.15.28:0/2342567958 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000008950 con 0x555a9c641730 2024-09-10T10:50:40.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3007e00640 1 -- 172.21.15.28:0/2342567958 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3000018070 con 0x555a9c641730 2024-09-10T10:50:40.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3007e00640 1 -- 172.21.15.28:0/2342567958 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000008d30 con 0x555a9c641730 2024-09-10T10:50:40.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2342567958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 msgr2=0x555a9c6c1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30000098e0 tx=0x7f3000005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2342567958 shutdown_connections 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c642170 0x555a9c6c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2342567958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c641730 0x555a9c6c1af0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2342567958 >> 172.21.15.28:0/2342567958 conn(0x555a9c625bf0 msgr2=0x555a9c629080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2342567958 shutdown_connections 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.657+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2342567958 wait complete. 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.658+0000 7f3011a1ef40 1 Processor -- start 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.658+0000 7f3011a1ef40 1 -- start start 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c641730 0x555a9c6bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 0x555a9c6ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555a9c5fcff0 con 0x555a9c642170 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a9c6c25a0 con 0x555a9c641730 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f3011a1ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555a9c62b1e0 con 0x555a9c643af0 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f300d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 0x555a9c6ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f300d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 0x555a9c6ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44996/0 (socket says 172.21.15.28:44996) 2024-09-10T10:50:40.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f300d600640 1 -- 172.21.15.28:0/2484940136 learned_addr learned my addr 172.21.15.28:0/2484940136 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f300e000640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c641730 0x555a9c6bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.659+0000 7f300ea00640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 -- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 msgr2=0x555a9c6ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555a9c643af0 0x555a9c6ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 -- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c641730 msgr2=0x555a9c6bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a9c641730 0x555a9c6bd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 -- 172.21.15.28:0/2484940136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a9c6c4c30 con 0x555a9c642170 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f300d600640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 0x555a9c6ce620 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00cf70 tx=0x7f2ffc00a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f3006a00640 1 -- 172.21.15.28:0/2484940136 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc008040 con 0x555a9c642170 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f3006a00640 1 -- 172.21.15.28:0/2484940136 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ffc014020 con 0x555a9c642170 2024-09-10T10:50:40.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a9c629660 con 0x555a9c642170 2024-09-10T10:50:40.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f3006a00640 1 -- 172.21.15.28:0/2484940136 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc00f400 con 0x555a9c642170 2024-09-10T10:50:40.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.660+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a9c6bdef0 con 0x555a9c642170 2024-09-10T10:50:40.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.662+0000 7f3006a00640 1 -- 172.21.15.28:0/2484940136 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ffc00f5a0 con 0x555a9c642170 2024-09-10T10:50:40.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.663+0000 7f3006a00640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ff006a840 0x7f2ff006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.663+0000 7f3006a00640 1 -- 172.21.15.28:0/2484940136 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f2ffc0898d0 con 0x555a9c642170 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.663+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555a9c6d23a0 0x555a9c5fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.663+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x555a9c6c8530 con 0x555a9c6d23a0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.664+0000 7f300ea00640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555a9c6d23a0 0x555a9c5fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.664+0000 7f300e000640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ff006a840 0x7f2ff006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.664+0000 7f300ea00640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555a9c6d23a0 0x555a9c5fab70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.664+0000 7f300e000640 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ff006a840 0x7f2ff006cd00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f3000011000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.664+0000 7f300ea00640 1 -- 172.21.15.28:0/2484940136 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [omap-get-keys out=5b] v0'0 uv1 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x7f3008057030 con 0x555a9c6d23a0 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555a9c6d23a0 msgr2=0x555a9c5fab70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555a9c6d23a0 0x555a9c5fab70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ff006a840 msgr2=0x7f2ff006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2ff006a840 0x7f2ff006cd00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f3000011000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 msgr2=0x555a9c6ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 --2- 172.21.15.28:0/2484940136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555a9c642170 0x555a9c6ce620 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc00cf70 tx=0x7f2ffc00a690 comp rx=0 tx=0).stop 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f300ea00640 1 -- 172.21.15.28:0/2484940136 reap_dead start 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 shutdown_connections 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 >> 172.21.15.28:0/2484940136 conn(0x555a9c625bf0 msgr2=0x555a9c6c7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 shutdown_connections 2024-09-10T10:50:40.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.665+0000 7f3011a1ef40 1 -- 172.21.15.28:0/2484940136 wait complete. 2024-09-10T10:50:40.670 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 100.00000000.inode 2024-09-10T10:50:40.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.744+0000 7f3d9178bf40 1 Processor -- start 2024-09-10T10:50:40.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.744+0000 7f3d9178bf40 1 -- start start 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd37110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecc9a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5594eccb2530 con 0x5594ecd3d3e0 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594ecc0fb30 con 0x5594ecd409c0 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5594ecd3fee0 con 0x5594ecc67470 2024-09-10T10:50:40.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.745+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45012/0 (socket says 172.21.15.28:45012) 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 -- 172.21.15.28:0/266297445 learned_addr learned my addr 172.21.15.28:0/266297445 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8ea00640 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd37110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8f400640 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecc9a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 -- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 msgr2=0x5594ecd37110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd37110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 -- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 msgr2=0x5594ecc9a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecc9a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 -- 172.21.15.28:0/266297445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ecc69930 con 0x5594ecd3d3e0 2024-09-10T10:50:40.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8e000640 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8d36bdae079fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.746+0000 7f3d8cc00640 1 -- 172.21.15.28:0/266297445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c055070 con 0x5594ecd3d3e0 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d8cc00640 1 -- 172.21.15.28:0/266297445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c051850 con 0x5594ecd3d3e0 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d8cc00640 1 -- 172.21.15.28:0/266297445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c051c20 con 0x5594ecd3d3e0 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 -- 172.21.15.28:0/266297445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 msgr2=0x5594ecd33de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c000c00 tx=0x7f3d7c051000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 -- 172.21.15.28:0/266297445 shutdown_connections 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecc9a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594ecd33de0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/266297445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd37110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 -- 172.21.15.28:0/266297445 >> 172.21.15.28:0/266297445 conn(0x5594eccb60a0 msgr2=0x5594eccb6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 -- 172.21.15.28:0/266297445 shutdown_connections 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.747+0000 7f3d9178bf40 1 -- 172.21.15.28:0/266297445 wait complete. 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.748+0000 7f3d9178bf40 1 Processor -- start 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.748+0000 7f3d9178bf40 1 -- start start 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594eccb3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecd430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5594ecc69c10 con 0x5594ecd3d3e0 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594ecc98ea0 con 0x5594ecd409c0 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d9178bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5594ecd3f900 con 0x5594ecc67470 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594eccb3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d8e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594eccb3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45014/0 (socket says 172.21.15.28:45014) 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d8e000640 1 -- 172.21.15.28:0/2090218471 learned_addr learned my addr 172.21.15.28:0/2090218471 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d8f400640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecd430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.749+0000 7f3d8ea00640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 -- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 msgr2=0x5594ecd2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5594ecc67470 0x5594ecd2c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 -- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 msgr2=0x5594ecd430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594ecd409c0 0x5594ecd430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 -- 172.21.15.28:0/2090218471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ecd32790 con 0x5594ecd3d3e0 2024-09-10T10:50:40.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d8e000640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594eccb3bb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c046070 tx=0x7f3d7c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d87400640 1 -- 172.21.15.28:0/2090218471 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c055070 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d87400640 1 -- 172.21.15.28:0/2090218471 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c051d30 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5594ecd32a70 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d87400640 1 -- 172.21.15.28:0/2090218471 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c06a070 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.750+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ecd33030 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.752+0000 7f3d87400640 1 -- 172.21.15.28:0/2090218471 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d7c075020 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.753+0000 7f3d87400640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d7006a810 0x7f3d7006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.753+0000 7f3d87400640 1 -- 172.21.15.28:0/2090218471 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f3d7c0ceab0 con 0x5594ecd3d3e0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.753+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5594eccb40f0 0x5594ecd435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.753+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5594ecd34630 con 0x5594eccb40f0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.753+0000 7f3d8ea00640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d7006a810 0x7f3d7006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.754+0000 7f3d8f400640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5594eccb40f0 0x5594ecd435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.754+0000 7f3d8ea00640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d7006a810 0x7f3d7006ccd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001ed0 tx=0x7f3d80001850 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.754+0000 7f3d8f400640 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5594eccb40f0 0x5594ecd435e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d8f400640 1 -- 172.21.15.28:0/2090218471 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [omap-get-keys out=5b] v0'0 uv3 ondisk = 0) ==== 162+0+5 (crc 0 0 0) 0x7f3d8800f850 con 0x5594eccb40f0 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5594eccb40f0 msgr2=0x5594ecd435e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5594eccb40f0 0x5594ecd435e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d7006a810 msgr2=0x7f3d7006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d7006a810 0x7f3d7006ccd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001ed0 tx=0x7f3d80001850 comp rx=0 tx=0).stop 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 msgr2=0x5594eccb3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 --2- 172.21.15.28:0/2090218471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5594ecd3d3e0 0x5594eccb3bb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c046070 tx=0x7f3d7c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d8f400640 1 -- 172.21.15.28:0/2090218471 reap_dead start 2024-09-10T10:50:40.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 shutdown_connections 2024-09-10T10:50:40.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 >> 172.21.15.28:0/2090218471 conn(0x5594eccb60a0 msgr2=0x5594ecd33de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 shutdown_connections 2024-09-10T10:50:40.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.755+0000 7f3d9178bf40 1 -- 172.21.15.28:0/2090218471 wait complete. 2024-09-10T10:50:40.760 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 1.00000000 2024-09-10T10:50:40.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.835+0000 7fbcd180ef40 1 Processor -- start 2024-09-10T10:50:40.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.835+0000 7fbcd180ef40 1 -- start start 2024-09-10T10:50:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 0x562f273007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f27303d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562f27277530 con 0x562f273023e0 2024-09-10T10:50:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f271d4b30 con 0x562f272768e0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562f27304ee0 con 0x562f273059c0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbccd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbccd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45018/0 (socket says 172.21.15.28:45018) 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.836+0000 7fbccd400640 1 -- 172.21.15.28:0/2954822550 learned_addr learned my addr 172.21.15.28:0/2954822550 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccde00640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 0x562f273007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 -- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 msgr2=0x562f27303d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbcce800640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f27303d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f27303d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 -- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 msgr2=0x562f273007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 0x562f273007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 -- 172.21.15.28:0/2954822550 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2722ca00 con 0x562f273023e0 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccde00640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 0x562f273007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbcce800640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f27303d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:40.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbccd400640 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009720 tx=0x7fbcbc005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1205d4fab1ac140c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbcc7e00640 1 -- 172.21.15.28:0/2954822550 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc019070 con 0x562f273023e0 2024-09-10T10:50:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbcc7e00640 1 -- 172.21.15.28:0/2954822550 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc008890 con 0x562f273023e0 2024-09-10T10:50:40.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.837+0000 7fbcc7e00640 1 -- 172.21.15.28:0/2954822550 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc008c80 con 0x562f273023e0 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 -- 172.21.15.28:0/2954822550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 msgr2=0x562f27300d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009720 tx=0x7fbcbc005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 -- 172.21.15.28:0/2954822550 shutdown_connections 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f27303d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f273023e0 0x562f27300d30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/2954822550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f272768e0 0x562f273007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 -- 172.21.15.28:0/2954822550 >> 172.21.15.28:0/2954822550 conn(0x562f2727b0a0 msgr2=0x562f2722b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 -- 172.21.15.28:0/2954822550 shutdown_connections 2024-09-10T10:50:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.838+0000 7fbcd180ef40 1 -- 172.21.15.28:0/2954822550 wait complete. 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 Processor -- start 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 -- start start 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 0x562f272fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f273023e0 0x562f272fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f2725e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562f27301990 con 0x562f272768e0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562f273016b0 con 0x562f273023e0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.839+0000 7fbcd180ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562f27308920 con 0x562f273059c0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 0x562f272fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 0x562f272fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45028/0 (socket says 172.21.15.28:45028) 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 -- 172.21.15.28:0/3554265211 learned_addr learned my addr 172.21.15.28:0/3554265211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbcce800640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f2725e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccd400640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f273023e0 0x562f272fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 -- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 msgr2=0x562f2725e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562f273059c0 0x562f2725e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 -- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f273023e0 msgr2=0x562f272fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562f273023e0 0x562f272fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.840+0000 7fbccde00640 1 -- 172.21.15.28:0/3554265211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f272f9400 con 0x562f272768e0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbccde00640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 0x562f272fe9c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0025a80 tx=0x7fbcc0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbcc6a00640 1 -- 172.21.15.28:0/3554265211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0064020 con 0x562f272768e0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbcc6a00640 1 -- 172.21.15.28:0/3554265211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc005a040 con 0x562f272768e0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbcc6a00640 1 -- 172.21.15.28:0/3554265211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0059070 con 0x562f272768e0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f272fd3b0 con 0x562f272768e0 2024-09-10T10:50:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.841+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f272f9e70 con 0x562f272768e0 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.843+0000 7fbcc6a00640 1 -- 172.21.15.28:0/3554265211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcc0067020 con 0x562f272768e0 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbcc6a00640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcb006a810 0x7fbcb006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbcc6a00640 1 -- 172.21.15.28:0/3554265211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fbcc00cd670 con 0x562f272768e0 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbccd400640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcb006a810 0x7fbcb006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562f272fef00 0x562f2739e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x562f272fcc40 con 0x562f272fef00 2024-09-10T10:50:40.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.844+0000 7fbcce800640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562f272fef00 0x562f2739e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbccd400640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcb006a810 0x7fbcb006ccd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x562f27277000 tx=0x7fbcbc008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcce800640 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562f272fef00 0x562f2739e840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcce800640 1 -- 172.21.15.28:0/3554265211 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [omap-get-keys out=20b] v0'0 uv6 ondisk = 0) ==== 154+0+20 (crc 0 0 0) 0x7fbcc8005020 con 0x562f272fef00 2024-09-10T10:50:40.846 INFO:teuthology.orchestra.run.smithi028.stdout:subdir_head 2024-09-10T10:50:40.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562f272fef00 msgr2=0x562f2739e840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562f272fef00 0x562f2739e840 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcb006a810 msgr2=0x7fbcb006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcb006a810 0x7fbcb006ccd0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x562f27277000 tx=0x7fbcbc008000 comp rx=0 tx=0).stop 2024-09-10T10:50:40.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 msgr2=0x562f272fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcd180ef40 1 --2- 172.21.15.28:0/3554265211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562f272768e0 0x562f272fe9c0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0025a80 tx=0x7fbcc0002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.845+0000 7fbcce800640 1 -- 172.21.15.28:0/3554265211 reap_dead start 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.846+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 shutdown_connections 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.846+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 >> 172.21.15.28:0/3554265211 conn(0x562f2727b0a0 msgr2=0x562f27303d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.846+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 shutdown_connections 2024-09-10T10:50:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.846+0000 7fbcd180ef40 1 -- 172.21.15.28:0/3554265211 wait complete. 2024-09-10T10:50:40.851 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-09-10T10:50:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.925+0000 7f7b6bb26f40 1 Processor -- start 2024-09-10T10:50:40.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.925+0000 7f7b6bb26f40 1 -- start start 2024-09-10T10:50:40.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00b3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f0029740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5642f0027530 con 0x5642f00b59c0 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642eff84b30 con 0x5642effdc470 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5642f00b4ee0 con 0x5642f00b23e0 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b68c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b68c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45048/0 (socket says 172.21.15.28:45048) 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.926+0000 7f7b68c00640 1 -- 172.21.15.28:0/207888119 learned_addr learned my addr 172.21.15.28:0/207888119 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b67800640 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f0029740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68200640 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 -- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 msgr2=0x5642f0029740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f0029740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 -- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 msgr2=0x5642f00b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 -- 172.21.15.28:0/207888119 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642effdeae0 con 0x5642f00b59c0 2024-09-10T10:50:40.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b68c00640 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c440 tx=0x7f7b6000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8008fe161ca1033c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b66400640 1 -- 172.21.15.28:0/207888119 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b600116b0 con 0x5642f00b59c0 2024-09-10T10:50:40.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b66400640 1 -- 172.21.15.28:0/207888119 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60017070 con 0x5642f00b59c0 2024-09-10T10:50:40.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.927+0000 7f7b66400640 1 -- 172.21.15.28:0/207888119 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60011ab0 con 0x5642f00b59c0 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/207888119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 msgr2=0x5642f00a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000c440 tx=0x7f7b6000c900 comp rx=0 tx=0).stop 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/207888119 shutdown_connections 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f00a8840 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f0029740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/207888119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/207888119 >> 172.21.15.28:0/207888119 conn(0x5642f002b0a0 msgr2=0x5642f00b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/207888119 shutdown_connections 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.928+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/207888119 wait complete. 2024-09-10T10:50:40.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.929+0000 7f7b6bb26f40 1 Processor -- start 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.929+0000 7f7b6bb26f40 1 -- start start 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00a8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5642f00a9490 con 0x5642f00b59c0 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642effde890 con 0x5642effdc470 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b6bb26f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5642f00b46c0 con 0x5642f00b23e0 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f014eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45064/0 (socket says 172.21.15.28:45064) 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 -- 172.21.15.28:0/2823179114 learned_addr learned my addr 172.21.15.28:0/2823179114 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:40.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b67800640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 -- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 msgr2=0x5642f014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5642f00b23e0 0x5642f014e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 -- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 msgr2=0x5642f00a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642effdc470 0x5642f00a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.930+0000 7f7b68c00640 1 -- 172.21.15.28:0/2823179114 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f0024d90 con 0x5642f00b59c0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b68c00640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f014eef0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000ce00 tx=0x7f7b600033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b65000640 1 -- 172.21.15.28:0/2823179114 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60012bf0 con 0x5642f00b59c0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5642f00a8240 con 0x5642f00b59c0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b65000640 1 -- 172.21.15.28:0/2823179114 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b60012d90 con 0x5642f00b59c0 2024-09-10T10:50:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b65000640 1 -- 172.21.15.28:0/2823179114 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b60026650 con 0x5642f00b59c0 2024-09-10T10:50:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.931+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f00a7710 con 0x5642f00b59c0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.933+0000 7f7b65000640 1 -- 172.21.15.28:0/2823179114 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b60024020 con 0x5642f00b59c0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.934+0000 7f7b65000640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7b4806a840 0x7f7b4806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.934+0000 7f7b65000640 1 -- 172.21.15.28:0/2823179114 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f7b6000e020 con 0x5642f00b59c0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.934+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5642f00a96d0 0x5642f0156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.934+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5642f00a8d40 con 0x5642f00a96d0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.934+0000 7f7b68200640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7b4806a840 0x7f7b4806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.935+0000 7f7b67800640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5642f00a96d0 0x5642f0156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.935+0000 7f7b68200640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7b4806a840 0x7f7b4806cd00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003f10 tx=0x7f7b58003870 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.935+0000 7f7b67800640 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5642f00a96d0 0x5642f0156180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b67800640 1 -- 172.21.15.28:0/2823179114 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x7f7b54057030 con 0x5642f00a96d0 2024-09-10T10:50:40.937 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-09-10T10:50:37.000000+0000, size 0 2024-09-10T10:50:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5642f00a96d0 msgr2=0x5642f0156180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5642f00a96d0 0x5642f0156180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7b4806a840 msgr2=0x7f7b4806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7b4806a840 0x7f7b4806cd00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7b58003f10 tx=0x7f7b58003870 comp rx=0 tx=0).stop 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 msgr2=0x5642f014eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 --2- 172.21.15.28:0/2823179114 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5642f00b59c0 0x5642f014eef0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000ce00 tx=0x7f7b600033a0 comp rx=0 tx=0).stop 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b68c00640 1 -- 172.21.15.28:0/2823179114 reap_dead start 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 shutdown_connections 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 >> 172.21.15.28:0/2823179114 conn(0x5642f002b0a0 msgr2=0x5642effdd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 shutdown_connections 2024-09-10T10:50:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:40.936+0000 7f7b6bb26f40 1 -- 172.21.15.28:0/2823179114 wait complete. 2024-09-10T10:50:40.942 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 600.00000000 2024-09-10T10:50:41.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.016+0000 7fd85e0f1f40 1 Processor -- start 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 -- start start 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348862d0 0x55cb3499f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb349124b0 0x55cb3499f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cb347e4b30 con 0x55cb348405a0 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb34885dc0 con 0x55cb348862d0 2024-09-10T10:50:41.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.017+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cb34886760 con 0x55cb349124b0 2024-09-10T10:50:41.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45080/0 (socket says 172.21.15.28:45080) 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 -- 172.21.15.28:0/7385871 learned_addr learned my addr 172.21.15.28:0/7385871 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85bc00640 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb349124b0 0x55cb3499f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85a800640 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348862d0 0x55cb3499f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 -- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb349124b0 msgr2=0x55cb3499f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb349124b0 0x55cb3499f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 -- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348862d0 msgr2=0x55cb3499f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348862d0 0x55cb3499f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 -- 172.21.15.28:0/7385871 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb34911db0 con 0x55cb348405a0 2024-09-10T10:50:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.018+0000 7fd85b200640 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098b0 tx=0x7fd84c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f73d7d55ef00cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd859400640 1 -- 172.21.15.28:0/7385871 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c008930 con 0x55cb348405a0 2024-09-10T10:50:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd859400640 1 -- 172.21.15.28:0/7385871 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84c018070 con 0x55cb348405a0 2024-09-10T10:50:41.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd859400640 1 -- 172.21.15.28:0/7385871 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c008cd0 con 0x55cb348405a0 2024-09-10T10:50:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/7385871 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 msgr2=0x55cb3490b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098b0 tx=0x7fd84c005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/7385871 shutdown_connections 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb349124b0 0x55cb3499f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348862d0 0x55cb3499f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/7385871 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb348405a0 0x55cb3490b560 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/7385871 >> 172.21.15.28:0/7385871 conn(0x55cb3486ab90 msgr2=0x55cb34912c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/7385871 shutdown_connections 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.019+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/7385871 wait complete. 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.020+0000 7fd85e0f1f40 1 Processor -- start 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.020+0000 7fd85e0f1f40 1 -- start start 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348405a0 0x55cb3490a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb348862d0 0x55cb3483e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 0x55cb349ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cb34914bc0 con 0x55cb349124b0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb34914730 con 0x55cb348405a0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85e0f1f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cb3483ce70 con 0x55cb348862d0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 0x55cb349ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 0x55cb349ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45086/0 (socket says 172.21.15.28:45086) 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85bc00640 1 -- 172.21.15.28:0/1462523627 learned_addr learned my addr 172.21.15.28:0/1462523627 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85b200640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348405a0 0x55cb3490a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.021+0000 7fd85a800640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb348862d0 0x55cb3483e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 -- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb348862d0 msgr2=0x55cb3483e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cb348862d0 0x55cb3483e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 -- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348405a0 msgr2=0x55cb3490a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb348405a0 0x55cb3490a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 -- 172.21.15.28:0/1462523627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb34910bb0 con 0x55cb349124b0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85bc00640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 0x55cb349ae7c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd854006f50 tx=0x7fd85400e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd853e00640 1 -- 172.21.15.28:0/1462523627 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540060d0 con 0x55cb349124b0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd853e00640 1 -- 172.21.15.28:0/1462523627 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd854017070 con 0x55cb349124b0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb3490d3e0 con 0x55cb349124b0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd853e00640 1 -- 172.21.15.28:0/1462523627 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854004520 con 0x55cb349124b0 2024-09-10T10:50:41.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.022+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb3490d100 con 0x55cb349124b0 2024-09-10T10:50:41.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.024+0000 7fd853e00640 1 -- 172.21.15.28:0/1462523627 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8540046c0 con 0x55cb349124b0 2024-09-10T10:50:41.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.025+0000 7fd853e00640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83c06a840 0x7fd83c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.025+0000 7fd853e00640 1 -- 172.21.15.28:0/1462523627 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd854089a10 con 0x55cb349124b0 2024-09-10T10:50:41.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.025+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cb349af210 0x55cb349a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.025+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55cb349a2740 con 0x55cb349af210 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.025+0000 7fd85b200640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83c06a840 0x7fd83c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.026+0000 7fd85a800640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cb349af210 0x55cb349a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.026+0000 7fd85b200640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83c06a840 0x7fd83c06cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd84c015040 tx=0x7fd84c012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.026+0000 7fd85a800640 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cb349af210 0x55cb349a2200 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85a800640 1 -- 172.21.15.28:0/1462523627 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 600.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fd84800a020 con 0x55cb349af210 2024-09-10T10:50:41.028 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/600.00000000 mtime 2024-09-10T10:50:28.000000+0000, size 0 2024-09-10T10:50:41.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cb349af210 msgr2=0x55cb349a2200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cb349af210 0x55cb349a2200 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83c06a840 msgr2=0x7fd83c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83c06a840 0x7fd83c06cd00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd84c015040 tx=0x7fd84c012000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 msgr2=0x55cb349ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 --2- 172.21.15.28:0/1462523627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cb349124b0 0x55cb349ae7c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd854006f50 tx=0x7fd85400e660 comp rx=0 tx=0).stop 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85bc00640 1 -- 172.21.15.28:0/1462523627 reap_dead start 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 shutdown_connections 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 >> 172.21.15.28:0/1462523627 conn(0x55cb3486ab90 msgr2=0x55cb34912c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 shutdown_connections 2024-09-10T10:50:41.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.027+0000 7fd85e0f1f40 1 -- 172.21.15.28:0/1462523627 wait complete. 2024-09-10T10:50:41.033 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000.inode 2024-09-10T10:50:41.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 Processor -- start 2024-09-10T10:50:41.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 -- start start 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3c2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cf8c33f530 con 0x55cf8c2f4470 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf8c29cb30 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.108+0000 7f649d814f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cf8c3ccee0 con 0x55cf8c3cd9c0 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45098/0 (socket says 172.21.15.28:45098) 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 -- 172.21.15.28:0/743460969 learned_addr learned my addr 172.21.15.28:0/743460969 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f649a800640 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499400640 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 -- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 msgr2=0x55cf8c3cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 -- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3ca3e0 msgr2=0x55cf8c3c2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3c2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 -- 172.21.15.28:0/743460969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8c3cb6f0 con 0x55cf8c2f4470 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.109+0000 7f6499e00640 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f648c001db0 tx=0x7f648c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9769b4fadc2d2869 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f6493e00640 1 -- 172.21.15.28:0/743460969 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00bde0 con 0x55cf8c2f4470 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f6493e00640 1 -- 172.21.15.28:0/743460969 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c00f400 con 0x55cf8c2f4470 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f6493e00640 1 -- 172.21.15.28:0/743460969 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c00f7f0 con 0x55cf8c2f4470 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 -- 172.21.15.28:0/743460969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 msgr2=0x55cf8c3c1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f648c001db0 tx=0x7f648c00b5b0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 -- 172.21.15.28:0/743460969 shutdown_connections 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3c2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 --2- 172.21.15.28:0/743460969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c2f4470 0x55cf8c3c1580 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 -- 172.21.15.28:0/743460969 >> 172.21.15.28:0/743460969 conn(0x55cf8c3430a0 msgr2=0x55cf8c3ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 -- 172.21.15.28:0/743460969 shutdown_connections 2024-09-10T10:50:41.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.110+0000 7f649d814f40 1 -- 172.21.15.28:0/743460969 wait complete. 2024-09-10T10:50:41.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.111+0000 7f649d814f40 1 Processor -- start 2024-09-10T10:50:41.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.111+0000 7f649d814f40 1 -- start start 2024-09-10T10:50:41.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3bfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3c0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3c45d0 0x55cf8c3ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cf8c3288d0 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf8c2f6a90 con 0x55cf8c3c45d0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649d814f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cf8c2f6d70 con 0x55cf8c3cd9c0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f6499e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3bfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f6499e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3bfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45108/0 (socket says 172.21.15.28:45108) 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f6499e00640 1 -- 172.21.15.28:0/521086916 learned_addr learned my addr 172.21.15.28:0/521086916 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f6499400640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.112+0000 7f649a800640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3c45d0 0x55cf8c3ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 -- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 msgr2=0x55cf8c3c0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cf8c3cd9c0 0x55cf8c3c0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 -- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3c45d0 msgr2=0x55cf8c3ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf8c3c45d0 0x55cf8c3ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 -- 172.21.15.28:0/521086916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf8c3c3470 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6499e00640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3bfc90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55cf8c45e790 tx=0x7f648c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6492a00640 1 -- 172.21.15.28:0/521086916 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c001120 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6492a00640 1 -- 172.21.15.28:0/521086916 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c021020 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f6492a00640 1 -- 172.21.15.28:0/521086916 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c01c440 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cf8c3c3710 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.113+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf8c3c3cd0 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.115+0000 7f6492a00640 1 -- 172.21.15.28:0/521086916 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f648c017070 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.116+0000 7f6492a00640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f647c06a810 0x7f647c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.116+0000 7f6492a00640 1 -- 172.21.15.28:0/521086916 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f648c08b0a0 con 0x55cf8c3ca3e0 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.116+0000 7f649d814f40 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cf8c3ce7b0 0x55cf8c3c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.116+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55cf8c3c0a50 con 0x55cf8c3ce7b0 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.116+0000 7f6499400640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f647c06a810 0x7f647c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.117+0000 7f649a800640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cf8c3ce7b0 0x55cf8c3c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.117+0000 7f6499400640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f647c06a810 0x7f647c06ccd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55cf8c45e4a0 tx=0x7f6488005b30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.117+0000 7f649a800640 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cf8c3ce7b0 0x55cf8c3c55e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649a800640 1 -- 172.21.15.28:0/521086916 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv15 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7f649404f990 con 0x55cf8c3ce7b0 2024-09-10T10:50:41.119 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/1.00000000.inode mtime 2024-09-10T10:50:37.000000+0000, size 582 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cf8c3ce7b0 msgr2=0x55cf8c3c55e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cf8c3ce7b0 0x55cf8c3c55e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f647c06a810 msgr2=0x7f647c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f647c06a810 0x7f647c06ccd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55cf8c45e4a0 tx=0x7f6488005b30 comp rx=0 tx=0).stop 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 msgr2=0x55cf8c3bfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 --2- 172.21.15.28:0/521086916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cf8c3ca3e0 0x55cf8c3bfc90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55cf8c45e790 tx=0x7f648c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649a800640 1 -- 172.21.15.28:0/521086916 reap_dead start 2024-09-10T10:50:41.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 shutdown_connections 2024-09-10T10:50:41.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 >> 172.21.15.28:0/521086916 conn(0x55cf8c3430a0 msgr2=0x55cf8c3cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 shutdown_connections 2024-09-10T10:50:41.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.118+0000 7f649d814f40 1 -- 172.21.15.28:0/521086916 wait complete. 2024-09-10T10:50:41.123 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000000 2024-09-10T10:50:41.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.198+0000 7f1f9063cf40 1 Processor -- start 2024-09-10T10:50:41.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.198+0000 7f1f9063cf40 1 -- start start 2024-09-10T10:50:41.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58dc53e0 0x55ea58dc6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dc69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ea58d3a530 con 0x55ea58d398e0 2024-09-10T10:50:41.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea58c97b30 con 0x55ea58dc89c0 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ea58dc7ee0 con 0x55ea58dc53e0 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f8d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f8d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45110/0 (socket says 172.21.15.28:45110) 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f8d800640 1 -- 172.21.15.28:0/2408511456 learned_addr learned my addr 172.21.15.28:0/2408511456 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f8ce00640 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58dc53e0 0x55ea58dc6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.199+0000 7f1f8e200640 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dc69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 -- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58dc53e0 msgr2=0x55ea58dc6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58dc53e0 0x55ea58dc6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 -- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 msgr2=0x55ea58dc69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dc69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 -- 172.21.15.28:0/2408511456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea58d23060 con 0x55ea58d398e0 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f8d800640 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000a440 tx=0x7f1f80051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb9834f3bd30931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f87400640 1 -- 172.21.15.28:0/2408511456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80063070 con 0x55ea58d398e0 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f87400640 1 -- 172.21.15.28:0/2408511456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f800559f0 con 0x55ea58d398e0 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.200+0000 7f1f87400640 1 -- 172.21.15.28:0/2408511456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80055dc0 con 0x55ea58d398e0 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 -- 172.21.15.28:0/2408511456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 msgr2=0x55ea58dc75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000a440 tx=0x7f1f80051000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 -- 172.21.15.28:0/2408511456 shutdown_connections 2024-09-10T10:50:41.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dc69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58dc53e0 0x55ea58dc6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/2408511456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58d398e0 0x55ea58dc75a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 -- 172.21.15.28:0/2408511456 >> 172.21.15.28:0/2408511456 conn(0x55ea58d3e0a0 msgr2=0x55ea58cef070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 -- 172.21.15.28:0/2408511456 shutdown_connections 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.201+0000 7f1f9063cf40 1 -- 172.21.15.28:0/2408511456 wait complete. 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.202+0000 7f1f9063cf40 1 Processor -- start 2024-09-10T10:50:41.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.202+0000 7f1f9063cf40 1 -- start start 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.202+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58d398e0 0x55ea58dbab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.202+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 0x55ea58dbb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f9063cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dbd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ea58d39dc0 con 0x55ea58dc53e0 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ea58d3cb70 con 0x55ea58dc89c0 2024-09-10T10:50:41.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f9063cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ea58d22ca0 con 0x55ea58d398e0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 0x55ea58dbb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 0x55ea58dbb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45118/0 (socket says 172.21.15.28:45118) 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 -- 172.21.15.28:0/3075349793 learned_addr learned my addr 172.21.15.28:0/3075349793 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8d800640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58d398e0 0x55ea58dbab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8e200640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dbd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 -- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58d398e0 msgr2=0x55ea58dbab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ea58d398e0 0x55ea58dbab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 -- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 msgr2=0x55ea58dbd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ea58dc89c0 0x55ea58dbd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.203+0000 7f1f8ce00640 1 -- 172.21.15.28:0/3075349793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea58d21420 con 0x55ea58dc53e0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f8ce00640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 0x55ea58dbb0c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0086d0 tx=0x7f1f7c0087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f86000640 1 -- 172.21.15.28:0/3075349793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0147e0 con 0x55ea58dc53e0 2024-09-10T10:50:41.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f86000640 1 -- 172.21.15.28:0/3075349793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c01a070 con 0x55ea58dc53e0 2024-09-10T10:50:41.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f86000640 1 -- 172.21.15.28:0/3075349793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014b80 con 0x55ea58dc53e0 2024-09-10T10:50:41.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ea58dc5940 con 0x55ea58dc53e0 2024-09-10T10:50:41.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.204+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea58dba160 con 0x55ea58dc53e0 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.206+0000 7f1f86000640 1 -- 172.21.15.28:0/3075349793 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f7c015470 con 0x55ea58dc53e0 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f86000640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f7006a810 0x7f1f7006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f86000640 1 -- 172.21.15.28:0/3075349793 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f1f7c08a510 con 0x55ea58dc53e0 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ea58e61820 0x55ea58e61c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ea58cef410 con 0x55ea58e61820 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f8d800640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f7006a810 0x7f1f7006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.207+0000 7f1f8e200640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ea58e61820 0x55ea58e61c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.208+0000 7f1f8d800640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f7006a810 0x7f1f7006ccd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1f80064000 tx=0x7f1f80053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.208+0000 7f1f8e200640 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ea58e61820 0x55ea58e61c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.208+0000 7f1f8e200640 1 -- 172.21.15.28:0/3075349793 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f1f88001a00 con 0x55ea58e61820 2024-09-10T10:50:41.210 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/200.00000000 mtime 2024-09-10T10:50:37.000000+0000, size 90 2024-09-10T10:50:41.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ea58e61820 msgr2=0x55ea58e61c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ea58e61820 0x55ea58e61c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f7006a810 msgr2=0x7f1f7006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f7006a810 0x7f1f7006ccd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f1f80064000 tx=0x7f1f80053000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 msgr2=0x55ea58dbb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 --2- 172.21.15.28:0/3075349793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ea58dc53e0 0x55ea58dbb0c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0086d0 tx=0x7f1f7c0087c0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f8e200640 1 -- 172.21.15.28:0/3075349793 reap_dead start 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 shutdown_connections 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 >> 172.21.15.28:0/3075349793 conn(0x55ea58d3e0a0 msgr2=0x55ea58dc6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 shutdown_connections 2024-09-10T10:50:41.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.209+0000 7f1f9063cf40 1 -- 172.21.15.28:0/3075349793 wait complete. 2024-09-10T10:50:41.214 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 200.00000001 2024-09-10T10:50:41.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.288+0000 7ffbebd1ff40 1 Processor -- start 2024-09-10T10:50:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.288+0000 7ffbebd1ff40 1 -- start start 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7ae8d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af10e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560a7ae8b530 con 0x560a7ae8a8e0 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a7ade8b30 con 0x560a7af199c0 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x560a7af18ee0 con 0x560a7af163e0 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbe8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbe8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45132/0 (socket says 172.21.15.28:45132) 2024-09-10T10:50:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.289+0000 7ffbe8400640 1 -- 172.21.15.28:0/3044188685 learned_addr learned my addr 172.21.15.28:0/3044188685 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 -- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 msgr2=0x560a7ae8d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7ae8d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af10e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 -- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 msgr2=0x560a7af10e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af10e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 -- 172.21.15.28:0/3044188685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7ae42b20 con 0x560a7ae8a8e0 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af10e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe8400640 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800a4f0 tx=0x7ffbd8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c478065db5f7098b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe6600640 1 -- 172.21.15.28:0/3044188685 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8063070 con 0x560a7ae8a8e0 2024-09-10T10:50:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe6600640 1 -- 172.21.15.28:0/3044188685 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd8055910 con 0x560a7ae8a8e0 2024-09-10T10:50:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.290+0000 7ffbe6600640 1 -- 172.21.15.28:0/3044188685 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8055cc0 con 0x560a7ae8a8e0 2024-09-10T10:50:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/3044188685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 msgr2=0x560a7af0cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800a4f0 tx=0x7ffbd8051000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/3044188685 shutdown_connections 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af10e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7ae8d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/3044188685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0cb50 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/3044188685 >> 172.21.15.28:0/3044188685 conn(0x560a7ae8f0a0 msgr2=0x560a7ae8f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/3044188685 shutdown_connections 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.291+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/3044188685 wait complete. 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.292+0000 7ffbebd1ff40 1 Processor -- start 2024-09-10T10:50:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.292+0000 7ffbebd1ff40 1 -- start start 2024-09-10T10:50:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7af1c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af173e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560a7ae73e10 con 0x560a7ae8a8e0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560a7af0d090 con 0x560a7af199c0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbebd1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x560a7af0d210 con 0x560a7af163e0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbe8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbe8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45148/0 (socket says 172.21.15.28:45148) 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbe8400640 1 -- 172.21.15.28:0/1322890064 learned_addr learned my addr 172.21.15.28:0/1322890064 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.293+0000 7ffbe7a00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7af1c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 -- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 msgr2=0x560a7af1c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7af1c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 -- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 msgr2=0x560a7af173e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af173e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 -- 172.21.15.28:0/1322890064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7ae84bd0 con 0x560a7ae8a8e0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560a7af199c0 0x560a7af173e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe7a00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x560a7af163e0 0x560a7af1c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe8400640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0bde0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800a4f0 tx=0x7ffbd8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe5200640 1 -- 172.21.15.28:0/1322890064 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8077070 con 0x560a7ae8a8e0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe5200640 1 -- 172.21.15.28:0/1322890064 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd806e070 con 0x560a7ae8a8e0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560a7ae84eb0 con 0x560a7ae8a8e0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbe5200640 1 -- 172.21.15.28:0/1322890064 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8063040 con 0x560a7ae8a8e0 2024-09-10T10:50:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.294+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7ae85470 con 0x560a7ae8a8e0 2024-09-10T10:50:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.296+0000 7ffbe5200640 1 -- 172.21.15.28:0/1322890064 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbd807a070 con 0x560a7ae8a8e0 2024-09-10T10:50:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.297+0000 7ffbe5200640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffbc806a810 0x7ffbc806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.297+0000 7ffbe5200640 1 -- 172.21.15.28:0/1322890064 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ffbd80cf6b0 con 0x560a7ae8a8e0 2024-09-10T10:50:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.297+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x560a7af1ae30 0x560a7af1b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.297+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x560a7af0c320 con 0x560a7af1ae30 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.297+0000 7ffbe7a00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffbc806a810 0x7ffbc806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.298+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x560a7af1ae30 0x560a7af1b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.298+0000 7ffbe7a00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffbc806a810 0x7ffbc806ccd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560a7af13a40 tx=0x7ffbd400c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.298+0000 7ffbe8e00640 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x560a7af1ae30 0x560a7af1b210 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbe8e00640 1 -- 172.21.15.28:0/1322890064 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7ffbe0003450 con 0x560a7af1ae30 2024-09-10T10:50:41.300 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/200.00000001 mtime 2024-09-10T10:50:37.000000+0000, size 18118 2024-09-10T10:50:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x560a7af1ae30 msgr2=0x560a7af1b210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x560a7af1ae30 0x560a7af1b210 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffbc806a810 msgr2=0x7ffbc806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffbc806a810 0x7ffbc806ccd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x560a7af13a40 tx=0x7ffbd400c040 comp rx=0 tx=0).stop 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 msgr2=0x560a7af0bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 --2- 172.21.15.28:0/1322890064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560a7ae8a8e0 0x560a7af0bde0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800a4f0 tx=0x7ffbd8002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbe8e00640 1 -- 172.21.15.28:0/1322890064 reap_dead start 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 shutdown_connections 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 >> 172.21.15.28:0/1322890064 conn(0x560a7ae8f0a0 msgr2=0x560a7ae8f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 shutdown_connections 2024-09-10T10:50:41.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.299+0000 7ffbebd1ff40 1 -- 172.21.15.28:0/1322890064 wait complete. 2024-09-10T10:50:41.304 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.0 2024-09-10T10:50:41.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.378+0000 7ff9462d0f40 1 Processor -- start 2024-09-10T10:50:41.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 -- start start 2024-09-10T10:50:41.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e738728e0 0x557e738fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73875210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557e73873530 con 0x557e739019c0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e737d0b30 con 0x557e738fe3e0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.379+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557e73900ee0 con 0x557e738728e0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45166/0 (socket says 172.21.15.28:45166) 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 -- 172.21.15.28:0/2146373433 learned_addr learned my addr 172.21.15.28:0/2146373433 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943400640 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e738728e0 0x557e738fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff93be00640 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73875210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 -- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e738728e0 msgr2=0x557e738fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e738728e0 0x557e738fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 -- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 msgr2=0x557e73875210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73875210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 -- 172.21.15.28:0/2146373433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e73828350 con 0x557e739019c0 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.380+0000 7ff943e00640 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0016b0 tx=0x7ff93c001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701a192cc26f093e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff942000640 1 -- 172.21.15.28:0/2146373433 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0062e0 con 0x557e739019c0 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff942000640 1 -- 172.21.15.28:0/2146373433 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff93c003650 con 0x557e739019c0 2024-09-10T10:50:41.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff942000640 1 -- 172.21.15.28:0/2146373433 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c003a60 con 0x557e739019c0 2024-09-10T10:50:41.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 -- 172.21.15.28:0/2146373433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 msgr2=0x557e73875750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0016b0 tx=0x7ff93c001b70 comp rx=0 tx=0).stop 2024-09-10T10:50:41.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 -- 172.21.15.28:0/2146373433 shutdown_connections 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e739019c0 0x557e73875750 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73875210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/2146373433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e738728e0 0x557e738fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 -- 172.21.15.28:0/2146373433 >> 172.21.15.28:0/2146373433 conn(0x557e738770a0 msgr2=0x557e73877480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 -- 172.21.15.28:0/2146373433 shutdown_connections 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.381+0000 7ff9462d0f40 1 -- 172.21.15.28:0/2146373433 wait complete. 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.382+0000 7ff9462d0f40 1 Processor -- start 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.382+0000 7ff9462d0f40 1 -- start start 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 0x557e738f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73874be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e739019c0 0x557e73859ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557e739003e0 con 0x557e738728e0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557e7385c5f0 con 0x557e738fe3e0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff9462d0f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557e73871840 con 0x557e739019c0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff943400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 0x557e738f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff943400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 0x557e738f8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45178/0 (socket says 172.21.15.28:45178) 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff943400640 1 -- 172.21.15.28:0/3944436431 learned_addr learned my addr 172.21.15.28:0/3944436431 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff943e00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e739019c0 0x557e73859ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.383+0000 7ff93be00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73874be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 -- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e739019c0 msgr2=0x557e73859ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557e739019c0 0x557e73859ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 -- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 msgr2=0x557e73874be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557e738fe3e0 0x557e73874be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 -- 172.21.15.28:0/3944436431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e738f89c0 con 0x557e738728e0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff943400640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 0x557e738f8480 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff934008400 tx=0x7ff9340084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff940c00640 1 -- 172.21.15.28:0/3944436431 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93401a070 con 0x557e738728e0 2024-09-10T10:50:41.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff940c00640 1 -- 172.21.15.28:0/3944436431 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff934015620 con 0x557e738728e0 2024-09-10T10:50:41.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557e738f73e0 con 0x557e738728e0 2024-09-10T10:50:41.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff940c00640 1 -- 172.21.15.28:0/3944436431 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9340159d0 con 0x557e738728e0 2024-09-10T10:50:41.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.384+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e738f79a0 con 0x557e738728e0 2024-09-10T10:50:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.386+0000 7ff940c00640 1 -- 172.21.15.28:0/3944436431 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff93401d070 con 0x557e738728e0 2024-09-10T10:50:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.387+0000 7ff940c00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92006a840 0x7ff92006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.387+0000 7ff940c00640 1 -- 172.21.15.28:0/3944436431 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff934089360 con 0x557e738728e0 2024-09-10T10:50:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.387+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557e73875120 0x557e738f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.387+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x557e738f1fe0 con 0x557e73875120 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.387+0000 7ff93be00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92006a840 0x7ff92006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff943e00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557e73875120 0x557e738f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff943e00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557e73875120 0x557e738f9820 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff93be00640 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92006a840 0x7ff92006cd00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff92c057000 tx=0x7ff92c061040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff943e00640 1 -- 172.21.15.28:0/3944436431 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_openfiles.0 [stat out=16b] v0'0 uv4 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x7ff93c00f030 con 0x557e73875120 2024-09-10T10:50:41.390 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/mds0_openfiles.0 mtime 2024-09-10T10:50:37.000000+0000, size 0 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557e73875120 msgr2=0x557e738f9820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.388+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557e73875120 0x557e738f9820 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92006a840 msgr2=0x7ff92006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff92006a840 0x7ff92006cd00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff92c057000 tx=0x7ff92c061040 comp rx=0 tx=0).stop 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 msgr2=0x557e738f8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 --2- 172.21.15.28:0/3944436431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557e738728e0 0x557e738f8480 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff934008400 tx=0x7ff9340084f0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff943e00640 1 -- 172.21.15.28:0/3944436431 reap_dead start 2024-09-10T10:50:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 shutdown_connections 2024-09-10T10:50:41.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 >> 172.21.15.28:0/3944436431 conn(0x557e738770a0 msgr2=0x557e73877480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 shutdown_connections 2024-09-10T10:50:41.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.389+0000 7ff9462d0f40 1 -- 172.21.15.28:0/3944436431 wait complete. 2024-09-10T10:50:41.394 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 500.00000000 2024-09-10T10:50:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.468+0000 7f31a9d8ff40 1 Processor -- start 2024-09-10T10:50:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.468+0000 7f31a9d8ff40 1 -- start start 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba8e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba8e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5608ba7b8b30 con 0x5608ba811d00 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608ba7bcb70 con 0x5608ba8120e0 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5608ba810210 con 0x5608ba8431c0 2024-09-10T10:50:41.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45188/0 (socket says 172.21.15.28:45188) 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.469+0000 7f31a7000640 1 -- 172.21.15.28:0/2328172507 learned_addr learned my addr 172.21.15.28:0/2328172507 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a6600640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba8e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7a00640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba8e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 -- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 msgr2=0x5608ba8e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba8e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 -- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 msgr2=0x5608ba8e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba8e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 -- 172.21.15.28:0/2328172507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ba8e9d20 con 0x5608ba811d00 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7a00640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba8e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a7000640 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3198001d90 tx=0x7f319800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae032c71ecc3865 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a5200640 1 -- 172.21.15.28:0/2328172507 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3198012020 con 0x5608ba811d00 2024-09-10T10:50:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a5200640 1 -- 172.21.15.28:0/2328172507 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319800e670 con 0x5608ba811d00 2024-09-10T10:50:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.470+0000 7f31a5200640 1 -- 172.21.15.28:0/2328172507 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800eac0 con 0x5608ba811d00 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/2328172507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 msgr2=0x5608ba8db310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3198001d90 tx=0x7f319800b6c0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/2328172507 shutdown_connections 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba8e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba8e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/2328172507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8db310 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/2328172507 >> 172.21.15.28:0/2328172507 conn(0x5608ba85f130 msgr2=0x5608ba8e8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/2328172507 shutdown_connections 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/2328172507 wait complete. 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.471+0000 7f31a9d8ff40 1 Processor -- start 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 -- start start 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8e1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba85d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba85d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5608ba85f7d0 con 0x5608ba811d00 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608ba85f510 con 0x5608ba8120e0 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.472+0000 7f31a9d8ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5608ba8e6850 con 0x5608ba8431c0 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8e1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8e1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45198/0 (socket says 172.21.15.28:45198) 2024-09-10T10:50:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 -- 172.21.15.28:0/1645278735 learned_addr learned my addr 172.21.15.28:0/1645278735 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7a00640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba85d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a6600640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba85d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 -- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 msgr2=0x5608ba85d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5608ba8431c0 0x5608ba85d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 -- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 msgr2=0x5608ba85d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba85d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 -- 172.21.15.28:0/1645278735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ba8d8970 con 0x5608ba811d00 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a6600640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608ba8120e0 0x5608ba85d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.473+0000 7f31a7000640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8e1a30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3198011040 tx=0x7f3198000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.474+0000 7f319f400640 1 -- 172.21.15.28:0/1645278735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319800be90 con 0x5608ba811d00 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.474+0000 7f319f400640 1 -- 172.21.15.28:0/1645278735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3198001470 con 0x5608ba811d00 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.474+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608ba8d8c50 con 0x5608ba811d00 2024-09-10T10:50:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.474+0000 7f319f400640 1 -- 172.21.15.28:0/1645278735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980197f0 con 0x5608ba811d00 2024-09-10T10:50:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.474+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ba8d9210 con 0x5608ba811d00 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.476+0000 7f319f400640 1 -- 172.21.15.28:0/1645278735 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3198015020 con 0x5608ba811d00 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.476+0000 7f319f400640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318806a9b0 0x7f318806ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.476+0000 7f319f400640 1 -- 172.21.15.28:0/1645278735 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f319808b740 con 0x5608ba811d00 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.477+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5608ba8e4db0 0x5608ba8e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.477+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5608ba8e56d0 con 0x5608ba8e4db0 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.477+0000 7f31a6600640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318806a9b0 0x7f318806ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.477+0000 7f31a7a00640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5608ba8e4db0 0x5608ba8e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.477+0000 7f31a6600640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318806a9b0 0x7f318806ce70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3194057000 tx=0x7f3194052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a7a00640 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5608ba8e4db0 0x5608ba8e5190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a7a00640 1 -- 172.21.15.28:0/1645278735 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f31a0003850 con 0x5608ba8e4db0 2024-09-10T10:50:41.480 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/500.00000000 mtime 2024-09-10T10:50:28.000000+0000, size 90 2024-09-10T10:50:41.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5608ba8e4db0 msgr2=0x5608ba8e5190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5608ba8e4db0 0x5608ba8e5190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318806a9b0 msgr2=0x7f318806ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318806a9b0 0x7f318806ce70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3194057000 tx=0x7f3194052000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 msgr2=0x5608ba8e1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 --2- 172.21.15.28:0/1645278735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5608ba811d00 0x5608ba8e1a30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3198011040 tx=0x7f3198000bc0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a7a00640 1 -- 172.21.15.28:0/1645278735 reap_dead start 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 shutdown_connections 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 >> 172.21.15.28:0/1645278735 conn(0x5608ba85f130 msgr2=0x5608ba8eb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 shutdown_connections 2024-09-10T10:50:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.478+0000 7f31a9d8ff40 1 -- 172.21.15.28:0/1645278735 wait complete. 2024-09-10T10:50:41.483 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds_snaptable 2024-09-10T10:50:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 Processor -- start 2024-09-10T10:50:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 -- start start 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 0x55ba6c530580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c52e3e0 0x55ba6c48b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ba6c4a3530 con 0x55ba6c5319c0 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba6c400b30 con 0x55ba6c4a2ff0 2024-09-10T10:50:41.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.558+0000 7f00ca38df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ba6c530ee0 con 0x55ba6c52e3e0 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45210/0 (socket says 172.21.15.28:45210) 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 -- 172.21.15.28:0/2158650085 learned_addr learned my addr 172.21.15.28:0/2158650085 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c7600640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 0x55ba6c530580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c6c00640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c52e3e0 0x55ba6c48b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 -- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c52e3e0 msgr2=0x55ba6c48b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c52e3e0 0x55ba6c48b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 -- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 msgr2=0x55ba6c530580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 0x55ba6c530580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 -- 172.21.15.28:0/2158650085 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6c48bd90 con 0x55ba6c5319c0 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c7600640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 0x55ba6c530580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.559+0000 7f00c8000640 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b650 tx=0x7f00c000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2826d9bb7ca68f2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00c5800640 1 -- 172.21.15.28:0/2158650085 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0014020 con 0x55ba6c5319c0 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00c5800640 1 -- 172.21.15.28:0/2158650085 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c0009930 con 0x55ba6c5319c0 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00c5800640 1 -- 172.21.15.28:0/2158650085 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0009d20 con 0x55ba6c5319c0 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 -- 172.21.15.28:0/2158650085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 msgr2=0x55ba6c525110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f00c000b650 tx=0x7f00c000bb10 comp rx=0 tx=0).stop 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 -- 172.21.15.28:0/2158650085 shutdown_connections 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c5319c0 0x55ba6c525110 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c52e3e0 0x55ba6c48b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2158650085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c4a2ff0 0x55ba6c530580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 -- 172.21.15.28:0/2158650085 >> 172.21.15.28:0/2158650085 conn(0x55ba6c4a70a0 msgr2=0x55ba6c4a7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 -- 172.21.15.28:0/2158650085 shutdown_connections 2024-09-10T10:50:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.560+0000 7f00ca38df40 1 -- 172.21.15.28:0/2158650085 wait complete. 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.561+0000 7f00ca38df40 1 Processor -- start 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.561+0000 7f00ca38df40 1 -- start start 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c4a2ff0 0x55ba6c488260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 0x55ba6c525a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c5319c0 0x55ba6c5ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ba6c4a5c20 con 0x55ba6c52e3e0 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba6c48a040 con 0x55ba6c5319c0 2024-09-10T10:50:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00ca38df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ba6c48a690 con 0x55ba6c4a2ff0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00c6c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 0x55ba6c525a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00c6c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 0x55ba6c525a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45222/0 (socket says 172.21.15.28:45222) 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00c6c00640 1 -- 172.21.15.28:0/2141732460 learned_addr learned my addr 172.21.15.28:0/2141732460 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00c7600640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c4a2ff0 0x55ba6c488260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.562+0000 7f00c8000640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c5319c0 0x55ba6c5ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 -- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c4a2ff0 msgr2=0x55ba6c488260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba6c4a2ff0 0x55ba6c488260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 -- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c5319c0 msgr2=0x55ba6c5ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c5319c0 0x55ba6c5ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 -- 172.21.15.28:0/2141732460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6c52e850 con 0x55ba6c52e3e0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c8000640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba6c5319c0 0x55ba6c5ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00c6c00640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 0x55ba6c525a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55ba6c531df0 tx=0x7f00b400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00bfe00640 1 -- 172.21.15.28:0/2141732460 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4014020 con 0x55ba6c52e3e0 2024-09-10T10:50:41.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00bfe00640 1 -- 172.21.15.28:0/2141732460 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b4010410 con 0x55ba6c52e3e0 2024-09-10T10:50:41.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba6c52eaf0 con 0x55ba6c52e3e0 2024-09-10T10:50:41.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00bfe00640 1 -- 172.21.15.28:0/2141732460 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4008030 con 0x55ba6c52e3e0 2024-09-10T10:50:41.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.563+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6c5252b0 con 0x55ba6c52e3e0 2024-09-10T10:50:41.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.565+0000 7f00bfe00640 1 -- 172.21.15.28:0/2141732460 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00b4017020 con 0x55ba6c52e3e0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.566+0000 7f00bfe00640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00a806a810 0x7f00a806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.566+0000 7f00bfe00640 1 -- 172.21.15.28:0/2141732460 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f00b4089c80 con 0x55ba6c52e3e0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.566+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ba6c4887a0 0x55ba6c5cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.566+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x55ba6c4a77a0 con 0x55ba6c4887a0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.566+0000 7f00c7600640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00a806a810 0x7f00a806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.567+0000 7f00c8000640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ba6c4887a0 0x55ba6c5cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.567+0000 7f00c7600640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00a806a810 0x7f00a806ccd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f00b8057000 tx=0x7f00b8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.567+0000 7f00c8000640 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ba6c4887a0 0x55ba6c5cb120 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.567+0000 7f00c8000640 1 -- 172.21.15.28:0/2141732460 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [stat out=16b] v0'0 uv1 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f00c0009ae0 con 0x55ba6c4887a0 2024-09-10T10:50:41.570 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/mds_snaptable mtime 2024-09-10T10:50:28.000000+0000, size 70 2024-09-10T10:50:41.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ba6c4887a0 msgr2=0x55ba6c5cb120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ba6c4887a0 0x55ba6c5cb120 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00a806a810 msgr2=0x7f00a806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00a806a810 0x7f00a806ccd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f00b8057000 tx=0x7f00b8052000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 msgr2=0x55ba6c525a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 --2- 172.21.15.28:0/2141732460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba6c52e3e0 0x55ba6c525a30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x55ba6c531df0 tx=0x7f00b400df90 comp rx=0 tx=0).stop 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00c8000640 1 -- 172.21.15.28:0/2141732460 reap_dead start 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 shutdown_connections 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 >> 172.21.15.28:0/2141732460 conn(0x55ba6c4a70a0 msgr2=0x55ba6c51fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 shutdown_connections 2024-09-10T10:50:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.568+0000 7f00ca38df40 1 -- 172.21.15.28:0/2141732460 wait complete. 2024-09-10T10:50:41.573 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_inotable 2024-09-10T10:50:41.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.649+0000 7f8ea6998f40 1 Processor -- start 2024-09-10T10:50:41.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.649+0000 7f8ea6998f40 1 -- start start 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91a3e0 0x5646ca91b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 0x5646ca891060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5646ca88f530 con 0x5646ca91d9c0 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646ca7ecb30 con 0x5646ca88e8e0 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5646ca91cee0 con 0x5646ca91a3e0 2024-09-10T10:50:41.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 0x5646ca891060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:54394/0 (socket says 172.21.15.28:54394) 2024-09-10T10:50:41.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea3c00640 1 -- 172.21.15.28:0/1482755202 learned_addr learned my addr 172.21.15.28:0/1482755202 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.650+0000 7f8ea3200640 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91a3e0 0x5646ca91b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 -- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91a3e0 msgr2=0x5646ca91b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91a3e0 0x5646ca91b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 -- 172.21.15.28:0/1482755202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 msgr2=0x5646ca891060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 0x5646ca891060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 -- 172.21.15.28:0/1482755202 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca911140 con 0x5646ca88e8e0 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea4600640 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 0x5646ca891060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:41.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.651+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a440 tx=0x7f8e94051000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a8b6e1005ccbab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea1e00640 1 -- 172.21.15.28:0/1482755202 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94063070 con 0x5646ca88e8e0 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea1e00640 1 -- 172.21.15.28:0/1482755202 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e94055aa0 con 0x5646ca88e8e0 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea1e00640 1 -- 172.21.15.28:0/1482755202 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94055e70 con 0x5646ca88e8e0 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 msgr2=0x5646ca91de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a440 tx=0x7f8e94051000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1482755202 shutdown_connections 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91d9c0 0x5646ca891060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91a3e0 0x5646ca91b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1482755202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca91de90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1482755202 >> 172.21.15.28:0/1482755202 conn(0x5646ca8930a0 msgr2=0x5646ca844070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1482755202 shutdown_connections 2024-09-10T10:50:41.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.652+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1482755202 wait complete. 2024-09-10T10:50:41.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.653+0000 7f8ea6998f40 1 Processor -- start 2024-09-10T10:50:41.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.653+0000 7f8ea6998f40 1 -- start start 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca9113b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 0x5646ca90fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91d9c0 0x5646ca9a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5646ca920ce0 con 0x5646ca91a3e0 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646ca890bd0 con 0x5646ca88e8e0 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea6998f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5646ca920fc0 con 0x5646ca91d9c0 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 0x5646ca90fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 0x5646ca90fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45242/0 (socket says 172.21.15.28:45242) 2024-09-10T10:50:41.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea3200640 1 -- 172.21.15.28:0/1422769029 learned_addr learned my addr 172.21.15.28:0/1422769029 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea4600640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91d9c0 0x5646ca9a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.654+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca9113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 -- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91d9c0 msgr2=0x5646ca9a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646ca91d9c0 0x5646ca9a99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 -- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 msgr2=0x5646ca9113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646ca88e8e0 0x5646ca9113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 -- 172.21.15.28:0/1422769029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca91e3d0 con 0x5646ca91a3e0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea3200640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 0x5646ca90fde0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8e90008680 tx=0x7f8e900086b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea0a00640 1 -- 172.21.15.28:0/1422769029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90019020 con 0x5646ca91a3e0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea0a00640 1 -- 172.21.15.28:0/1422769029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e90015880 con 0x5646ca91a3e0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea0a00640 1 -- 172.21.15.28:0/1422769029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90015c30 con 0x5646ca91a3e0 2024-09-10T10:50:41.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646ca9a6ef0 con 0x5646ca91a3e0 2024-09-10T10:50:41.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.655+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ca891b20 con 0x5646ca91a3e0 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.657+0000 7f8ea0a00640 1 -- 172.21.15.28:0/1422769029 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e9001c020 con 0x5646ca91a3e0 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.658+0000 7f8ea0a00640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e8406a840 0x7f8e8406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.658+0000 7f8ea0a00640 1 -- 172.21.15.28:0/1422769029 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f8e9008a5d0 con 0x5646ca91a3e0 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.658+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646ca9a9f30 0x5646ca9a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.658+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x5646ca910320 con 0x5646ca9a9f30 2024-09-10T10:50:41.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.659+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e8406a840 0x7f8e8406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.659+0000 7f8ea4600640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646ca9a9f30 0x5646ca9a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.659+0000 7f8ea4600640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646ca9a9f30 0x5646ca9a7a90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.659+0000 7f8ea3c00640 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e8406a840 0x7f8e8406cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a4f0 tx=0x7f8e94053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea4600640 1 -- 172.21.15.28:0/1422769029 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [stat out=16b] v0'0 uv3 ondisk = 0) ==== 157+0+16 (crc 0 0 0) 0x7f8e9c003530 con 0x5646ca9a9f30 2024-09-10T10:50:41.662 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/mds0_inotable mtime 2024-09-10T10:50:37.000000+0000, size 34 2024-09-10T10:50:41.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646ca9a9f30 msgr2=0x5646ca9a7a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646ca9a9f30 0x5646ca9a7a90 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e8406a840 msgr2=0x7f8e8406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e8406a840 0x7f8e8406cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400a4f0 tx=0x7f8e94053000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 msgr2=0x5646ca90fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 --2- 172.21.15.28:0/1422769029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646ca91a3e0 0x5646ca90fde0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8e90008680 tx=0x7f8e900086b0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea4600640 1 -- 172.21.15.28:0/1422769029 reap_dead start 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 shutdown_connections 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 >> 172.21.15.28:0/1422769029 conn(0x5646ca8930a0 msgr2=0x5646ca844070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 shutdown_connections 2024-09-10T10:50:41.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.660+0000 7f8ea6998f40 1 -- 172.21.15.28:0/1422769029 wait complete. 2024-09-10T10:50:41.665 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000 2024-09-10T10:50:41.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.740+0000 7f7186e29f40 1 Processor -- start 2024-09-10T10:50:41.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.740+0000 7f7186e29f40 1 -- start start 2024-09-10T10:50:41.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.740+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 0x562a72b2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7186e29f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562a72aa9dc0 con 0x562a72a645a0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7186e29f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a72a08b30 con 0x562a72aaa2d0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7186e29f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562a72aa91f0 con 0x562a72b364b0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45264/0 (socket says 172.21.15.28:45264) 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 -- 172.21.15.28:0/778957139 learned_addr learned my addr 172.21.15.28:0/778957139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7183600640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 0x562a72b2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184a00640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 -- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72b364b0 msgr2=0x562a72b38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 -- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 msgr2=0x562a72b2e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 0x562a72b2e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 -- 172.21.15.28:0/778957139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72aaadc0 con 0x562a72a645a0 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7183600640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 0x562a72b2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.741+0000 7f7184000640 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7174009750 tx=0x7f7174005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ca13ec7e9c20e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7182200640 1 -- 172.21.15.28:0/778957139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174019070 con 0x562a72a645a0 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7182200640 1 -- 172.21.15.28:0/778957139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7174008890 con 0x562a72a645a0 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7182200640 1 -- 172.21.15.28:0/778957139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7174008c60 con 0x562a72a645a0 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 -- 172.21.15.28:0/778957139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 msgr2=0x562a72b34b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7174009750 tx=0x7f7174005a00 comp rx=0 tx=0).stop 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 -- 172.21.15.28:0/778957139 shutdown_connections 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72aaa2d0 0x562a72b2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 --2- 172.21.15.28:0/778957139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72a645a0 0x562a72b34b70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 -- 172.21.15.28:0/778957139 >> 172.21.15.28:0/778957139 conn(0x562a72a8eb90 msgr2=0x562a72aaa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 -- 172.21.15.28:0/778957139 shutdown_connections 2024-09-10T10:50:41.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.742+0000 7f7186e29f40 1 -- 172.21.15.28:0/778957139 wait complete. 2024-09-10T10:50:41.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.743+0000 7f7186e29f40 1 Processor -- start 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.743+0000 7f7186e29f40 1 -- start start 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72aaa2d0 0x562a72bc5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 0x562a72b38dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562a72b31c00 con 0x562a72b276d0 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a72aab060 con 0x562a72b364b0 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7186e29f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562a72b33f70 con 0x562a72aaa2d0 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7184a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 0x562a72b38dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7184a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 0x562a72b38dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45276/0 (socket says 172.21.15.28:45276) 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7184a00640 1 -- 172.21.15.28:0/2651306855 learned_addr learned my addr 172.21.15.28:0/2651306855 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.744+0000 7f7184000640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72aaa2d0 0x562a72bc5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7183600640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 -- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72aaa2d0 msgr2=0x562a72bc5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562a72aaa2d0 0x562a72bc5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 -- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72b364b0 msgr2=0x562a72b38880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 -- 172.21.15.28:0/2651306855 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a72b2a1f0 con 0x562a72b276d0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7183600640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a72b364b0 0x562a72b38880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7184a00640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 0x562a72b38dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f717c025ad0 tx=0x7f717c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7180e00640 1 -- 172.21.15.28:0/2651306855 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c005b40 con 0x562a72b276d0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7180e00640 1 -- 172.21.15.28:0/2651306855 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f717c062400 con 0x562a72b276d0 2024-09-10T10:50:41.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.745+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562a72bc64e0 con 0x562a72b276d0 2024-09-10T10:50:41.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.746+0000 7f7180e00640 1 -- 172.21.15.28:0/2651306855 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717c04f660 con 0x562a72b276d0 2024-09-10T10:50:41.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.746+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a72b2afa0 con 0x562a72b276d0 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.747+0000 7f7180e00640 1 -- 172.21.15.28:0/2651306855 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f717c0627f0 con 0x562a72b276d0 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.748+0000 7f7180e00640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f716406a810 0x7f716406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.748+0000 7f7180e00640 1 -- 172.21.15.28:0/2651306855 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f717c0cdd20 con 0x562a72b276d0 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.749+0000 7f7186e29f40 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562a72b27280 0x562a72b2c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.749+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x562a72b2cd00 con 0x562a72b27280 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.749+0000 7f7184000640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f716406a810 0x7f716406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.749+0000 7f7183600640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562a72b27280 0x562a72b2c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.749+0000 7f7184000640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f716406a810 0x7f716406ccd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562a72a900d0 tx=0x7f7174008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7183600640 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562a72b27280 0x562a72b2c7c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7183600640 1 -- 172.21.15.28:0/2651306855 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7f717000e040 con 0x562a72b27280 2024-09-10T10:50:41.752 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/100.00000000 mtime 2024-09-10T10:50:28.000000+0000, size 0 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562a72b27280 msgr2=0x562a72b2c7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x562a72b27280 0x562a72b2c7c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f716406a810 msgr2=0x7f716406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f716406a810 0x7f716406ccd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x562a72a900d0 tx=0x7f7174008000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 msgr2=0x562a72b38dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 --2- 172.21.15.28:0/2651306855 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562a72b276d0 0x562a72b38dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f717c025ad0 tx=0x7f717c003330 comp rx=0 tx=0).stop 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7184a00640 1 -- 172.21.15.28:0/2651306855 reap_dead start 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 shutdown_connections 2024-09-10T10:50:41.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 >> 172.21.15.28:0/2651306855 conn(0x562a72a8eb90 msgr2=0x562a72a911e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 shutdown_connections 2024-09-10T10:50:41.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.750+0000 7f7186e29f40 1 -- 172.21.15.28:0/2651306855 wait complete. 2024-09-10T10:50:41.756 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_sessionmap 2024-09-10T10:50:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.830+0000 7fd747f0ef40 1 Processor -- start 2024-09-10T10:50:41.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.830+0000 7fd747f0ef40 1 -- start start 2024-09-10T10:50:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184cf4470 0x562184d245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184dc0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562184d3f530 con 0x562184dcd9c0 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562184c9cb30 con 0x562184dca3e0 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562184dccee0 con 0x562184cf4470 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.831+0000 7fd745c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45288/0 (socket says 172.21.15.28:45288) 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 -- 172.21.15.28:0/2805964783 learned_addr learned my addr 172.21.15.28:0/2805964783 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745200640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184cf4470 0x562184d245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd73fe00640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184dc0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 -- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184cf4470 msgr2=0x562184d245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184cf4470 0x562184d245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 -- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 msgr2=0x562184dc0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184dc0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 -- 172.21.15.28:0/2805964783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562184d27ad0 con 0x562184dcd9c0 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd73fe00640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184dc0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.832+0000 7fd745c00640 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b6c0 tx=0x7fd74000bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581491a47c7aa7ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd73ea00640 1 -- 172.21.15.28:0/2805964783 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740016020 con 0x562184dcd9c0 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd73ea00640 1 -- 172.21.15.28:0/2805964783 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74000d750 con 0x562184dcd9c0 2024-09-10T10:50:41.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd73ea00640 1 -- 172.21.15.28:0/2805964783 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74000db20 con 0x562184dcd9c0 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 -- 172.21.15.28:0/2805964783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 msgr2=0x562184d416b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b6c0 tx=0x7fd74000bb80 comp rx=0 tx=0).stop 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 -- 172.21.15.28:0/2805964783 shutdown_connections 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184dcd9c0 0x562184d416b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184dc0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/2805964783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184cf4470 0x562184d245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 -- 172.21.15.28:0/2805964783 >> 172.21.15.28:0/2805964783 conn(0x562184d430a0 msgr2=0x562184d43480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 -- 172.21.15.28:0/2805964783 shutdown_connections 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.833+0000 7fd747f0ef40 1 -- 172.21.15.28:0/2805964783 wait complete. 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.834+0000 7fd747f0ef40 1 Processor -- start 2024-09-10T10:50:41.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.834+0000 7fd747f0ef40 1 -- start start 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 0x562184dbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184d23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184dcd9c0 0x562184d38000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562184d27d40 con 0x562184cf4470 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562184cf6a90 con 0x562184dca3e0 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd747f0ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562184d28020 con 0x562184dcd9c0 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd745200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 0x562184dbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd745200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 0x562184dbfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45292/0 (socket says 172.21.15.28:45292) 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd745200640 1 -- 172.21.15.28:0/892109489 learned_addr learned my addr 172.21.15.28:0/892109489 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd73fe00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184d23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.835+0000 7fd745c00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184dcd9c0 0x562184d38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 -- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184dcd9c0 msgr2=0x562184d38000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562184dcd9c0 0x562184d38000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 -- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 msgr2=0x562184d23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184d23ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 -- 172.21.15.28:0/892109489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562184d3d120 con 0x562184cf4470 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd73fe00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562184dca3e0 0x562184d23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd745200640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 0x562184dbfdf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd73800efd0 tx=0x7fd73800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd73d600640 1 -- 172.21.15.28:0/892109489 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800f860 con 0x562184cf4470 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd73d600640 1 -- 172.21.15.28:0/892109489 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd738013030 con 0x562184cf4470 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562184d38b90 con 0x562184cf4470 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd73d600640 1 -- 172.21.15.28:0/892109489 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800fc20 con 0x562184cf4470 2024-09-10T10:50:41.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.836+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562184dc0330 con 0x562184cf4470 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.838+0000 7fd73d600640 1 -- 172.21.15.28:0/892109489 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7380173d0 con 0x562184cf4470 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.839+0000 7fd73d600640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd72406a840 0x7fd72406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.839+0000 7fd73d600640 1 -- 172.21.15.28:0/892109489 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd73808a480 con 0x562184cf4470 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.839+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x562184d24010 0x562184dbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.839+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x562184d41bd0 con 0x562184d24010 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.840+0000 7fd73fe00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd72406a840 0x7fd72406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.840+0000 7fd745c00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x562184d24010 0x562184dbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.840+0000 7fd745c00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x562184d24010 0x562184dbe3d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.840+0000 7fd73fe00640 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd72406a840 0x7fd72406cd00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd730057000 tx=0x7fd730052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd745c00640 1 -- 172.21.15.28:0/892109489 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_sessionmap [stat out=16b] v0'0 uv3 ondisk = 0) ==== 159+0+16 (crc 0 0 0) 0x7fd74000dc60 con 0x562184d24010 2024-09-10T10:50:41.843 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/mds0_sessionmap mtime 2024-09-10T10:50:37.000000+0000, size 0 2024-09-10T10:50:41.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x562184d24010 msgr2=0x562184dbe3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x562184d24010 0x562184dbe3d0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd72406a840 msgr2=0x7fd72406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd72406a840 0x7fd72406cd00 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd730057000 tx=0x7fd730052000 comp rx=0 tx=0).stop 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 msgr2=0x562184dbfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 --2- 172.21.15.28:0/892109489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562184cf4470 0x562184dbfdf0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd73800efd0 tx=0x7fd73800c730 comp rx=0 tx=0).stop 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd745c00640 1 -- 172.21.15.28:0/892109489 reap_dead start 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 shutdown_connections 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 >> 172.21.15.28:0/892109489 conn(0x562184d430a0 msgr2=0x562184d43480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 shutdown_connections 2024-09-10T10:50:41.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.841+0000 7fd747f0ef40 1 -- 172.21.15.28:0/892109489 wait complete. 2024-09-10T10:50:41.847 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 400.00000000 2024-09-10T10:50:41.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.921+0000 7fe33f3c5f40 1 Processor -- start 2024-09-10T10:50:41.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.921+0000 7fe33f3c5f40 1 -- start start 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029c9fff0 0x559029d22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029d2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559029ca0530 con 0x559029d2e9c0 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559029bfdb30 con 0x559029c9fff0 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559029d2dee0 con 0x559029d2b3e0 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45312/0 (socket says 172.21.15.28:45312) 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 -- 172.21.15.28:0/2124831447 learned_addr learned my addr 172.21.15.28:0/2124831447 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33bc00640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029d2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33c600640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029c9fff0 0x559029d22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 -- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 msgr2=0x559029d2c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029d2c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 -- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029c9fff0 msgr2=0x559029d22230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029c9fff0 0x559029d22230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33d000640 1 -- 172.21.15.28:0/2124831447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559029c87040 con 0x559029d2e9c0 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.922+0000 7fe33bc00640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029d2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33d000640 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe334002820 tx=0x7fe334002ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed317873e0f3f2f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33a800640 1 -- 172.21.15.28:0/2124831447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334016020 con 0x559029d2e9c0 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33a800640 1 -- 172.21.15.28:0/2124831447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe33400d9c0 con 0x559029d2e9c0 2024-09-10T10:50:41.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33a800640 1 -- 172.21.15.28:0/2124831447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33400dd90 con 0x559029d2e9c0 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/2124831447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 msgr2=0x559029c54ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe334002820 tx=0x7fe334002ce0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/2124831447 shutdown_connections 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029d2e9c0 0x559029c54ac0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029d2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/2124831447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029c9fff0 0x559029d22230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/2124831447 >> 172.21.15.28:0/2124831447 conn(0x559029ca40a0 msgr2=0x559029ca4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/2124831447 shutdown_connections 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.923+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/2124831447 wait complete. 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.924+0000 7fe33f3c5f40 1 Processor -- start 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.924+0000 7fe33f3c5f40 1 -- start start 2024-09-10T10:50:41.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 0x559029d1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029c549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029d2e9c0 0x559029dc7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559029c87320 con 0x559029c9fff0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559029c87600 con 0x559029d2e9c0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33f3c5f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559029d2ce20 con 0x559029d2b3e0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33c600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 0x559029d1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33c600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 0x559029d1d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45324/0 (socket says 172.21.15.28:45324) 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33c600640 1 -- 172.21.15.28:0/1308597139 learned_addr learned my addr 172.21.15.28:0/1308597139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.925+0000 7fe33bc00640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029c549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33d000640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029d2e9c0 0x559029dc7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 -- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 msgr2=0x559029c549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559029d2b3e0 0x559029c549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 -- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029d2e9c0 msgr2=0x559029dc7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029d2e9c0 0x559029dc7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 -- 172.21.15.28:0/1308597139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559029d27690 con 0x559029c9fff0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33d000640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559029d2e9c0 0x559029dc7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33c600640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 0x559029d1d2a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe32c000c00 tx=0x7fe32c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe339400640 1 -- 172.21.15.28:0/1308597139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c054ac0 con 0x559029c9fff0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe339400640 1 -- 172.21.15.28:0/1308597139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c064070 con 0x559029c9fff0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe339400640 1 -- 172.21.15.28:0/1308597139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c069470 con 0x559029c9fff0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.926+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559029d1ff00 con 0x559029c9fff0 2024-09-10T10:50:41.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.927+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559029d204c0 con 0x559029c9fff0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.928+0000 7fe339400640 1 -- 172.21.15.28:0/1308597139 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe32c05c040 con 0x559029c9fff0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.929+0000 7fe339400640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe31c06a840 0x7fe31c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.929+0000 7fe339400640 1 -- 172.21.15.28:0/1308597139 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fe32c0ce440 con 0x559029c9fff0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.929+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559029c54ee0 0x559029d20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.930+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x559029d22230 con 0x559029c54ee0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.930+0000 7fe33bc00640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe31c06a840 0x7fe31c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.930+0000 7fe33d000640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559029c54ee0 0x559029d20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.930+0000 7fe33bc00640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe31c06a840 0x7fe31c06cd00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x559029d1e120 tx=0x7fe328001860 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.930+0000 7fe33d000640 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559029c54ee0 0x559029d20cc0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:41.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33d000640 1 -- 172.21.15.28:0/1308597139 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x7fe33400dd50 con 0x559029c54ee0 2024-09-10T10:50:41.933 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/400.00000000 mtime 2024-09-10T10:50:28.000000+0000, size 22 2024-09-10T10:50:41.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559029c54ee0 msgr2=0x559029d20cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559029c54ee0 0x559029d20cc0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe31c06a840 msgr2=0x7fe31c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe31c06a840 0x7fe31c06cd00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x559029d1e120 tx=0x7fe328001860 comp rx=0 tx=0).stop 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 msgr2=0x559029d1d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 --2- 172.21.15.28:0/1308597139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559029c9fff0 0x559029d1d2a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe32c000c00 tx=0x7fe32c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33d000640 1 -- 172.21.15.28:0/1308597139 reap_dead start 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 shutdown_connections 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 >> 172.21.15.28:0/1308597139 conn(0x559029ca40a0 msgr2=0x559029ca4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 shutdown_connections 2024-09-10T10:50:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:41.931+0000 7fe33f3c5f40 1 -- 172.21.15.28:0/1308597139 wait complete. 2024-09-10T10:50:41.937 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 100.00000000.inode 2024-09-10T10:50:42.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.012+0000 7ff82b946f40 1 Processor -- start 2024-09-10T10:50:42.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.012+0000 7ff82b946f40 1 -- start start 2024-09-10T10:50:42.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 0x558477be8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477bf13e0 0x558477bf29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558477ac3b30 con 0x558477bf49c0 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558477ac7b70 con 0x558477b1b470 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff82b946f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558477bf3ee0 con 0x558477bf13e0 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45334/0 (socket says 172.21.15.28:45334) 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 -- 172.21.15.28:0/1907027427 learned_addr learned my addr 172.21.15.28:0/1907027427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff828c00640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 0x558477be8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff823e00640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477bf13e0 0x558477bf29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 -- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477bf13e0 msgr2=0x558477bf29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477bf13e0 0x558477bf29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 -- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 msgr2=0x558477be8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 0x558477be8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff829600640 1 -- 172.21.15.28:0/1907027427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558477bf2420 con 0x558477bf49c0 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.013+0000 7ff828c00640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 0x558477be8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff829600640 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff8240015e0 tx=0x7ff824001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4025e071ed07f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff822a00640 1 -- 172.21.15.28:0/1907027427 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824016070 con 0x558477bf49c0 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff822a00640 1 -- 172.21.15.28:0/1907027427 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82400f710 con 0x558477bf49c0 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff822a00640 1 -- 172.21.15.28:0/1907027427 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82400fae0 con 0x558477bf49c0 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff82b946f40 1 -- 172.21.15.28:0/1907027427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 msgr2=0x558477bf57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff82b946f40 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff8240015e0 tx=0x7ff824001aa0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff82b946f40 1 -- 172.21.15.28:0/1907027427 shutdown_connections 2024-09-10T10:50:42.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff82b946f40 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477bf57e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.014+0000 7ff82b946f40 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477bf13e0 0x558477bf29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 --2- 172.21.15.28:0/1907027427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477b1b470 0x558477be8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 -- 172.21.15.28:0/1907027427 >> 172.21.15.28:0/1907027427 conn(0x558477b6a0a0 msgr2=0x558477b06f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:42.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 -- 172.21.15.28:0/1907027427 shutdown_connections 2024-09-10T10:50:42.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 -- 172.21.15.28:0/1907027427 wait complete. 2024-09-10T10:50:42.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 Processor -- start 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.015+0000 7ff82b946f40 1 -- start start 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477b1b470 0x558477beffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477bf13e0 0x558477b4ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477be65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558477b4f280 con 0x558477bf49c0 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558477bf2690 con 0x558477bf13e0 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.016+0000 7ff82b946f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558477be8890 con 0x558477b1b470 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477be65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477be65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45336/0 (socket says 172.21.15.28:45336) 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 -- 172.21.15.28:0/4084276746 learned_addr learned my addr 172.21.15.28:0/4084276746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff823e00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477bf13e0 0x558477b4ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff828c00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477b1b470 0x558477beffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 -- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477b1b470 msgr2=0x558477beffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477b1b470 0x558477beffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 -- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477bf13e0 msgr2=0x558477b4ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477bf13e0 0x558477b4ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 -- 172.21.15.28:0/4084276746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558477be7660 con 0x558477bf49c0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff828c00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558477b1b470 0x558477beffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff823e00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558477bf13e0 0x558477b4ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff829600640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477be65e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff824014040 tx=0x7ff824001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.017+0000 7ff821600640 1 -- 172.21.15.28:0/4084276746 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824016040 con 0x558477bf49c0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.018+0000 7ff821600640 1 -- 172.21.15.28:0/4084276746 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff824011660 con 0x558477bf49c0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.018+0000 7ff821600640 1 -- 172.21.15.28:0/4084276746 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff824011a50 con 0x558477bf49c0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.018+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558477bec040 con 0x558477bf49c0 2024-09-10T10:50:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.018+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558477be78d0 con 0x558477bf49c0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.019+0000 7ff821600640 1 -- 172.21.15.28:0/4084276746 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff824011bf0 con 0x558477bf49c0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.020+0000 7ff821600640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff80806a840 0x7ff80806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.020+0000 7ff821600640 1 -- 172.21.15.28:0/4084276746 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff82408b480 con 0x558477bf49c0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff82b946f40 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558477b4d3c0 0x558477be8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x558477b1cc50 con 0x558477b4d3c0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff828c00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff80806a840 0x7ff80806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff823e00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558477b4d3c0 0x558477be8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff823e00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558477b4d3c0 0x558477be8f80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.021+0000 7ff828c00640 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff80806a840 0x7ff80806cd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0011e0 tx=0x7ff81c001650 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff823e00640 1 -- 172.21.15.28:0/4084276746 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x7ff81400a040 con 0x558477b4d3c0 2024-09-10T10:50:42.024 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/100.00000000.inode mtime 2024-09-10T10:50:28.000000+0000, size 582 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558477b4d3c0 msgr2=0x558477be8f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558477b4d3c0 0x558477be8f80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff80806a840 msgr2=0x7ff80806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff80806a840 0x7ff80806cd00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0011e0 tx=0x7ff81c001650 comp rx=0 tx=0).stop 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 msgr2=0x558477be65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 --2- 172.21.15.28:0/4084276746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558477bf49c0 0x558477be65e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff824014040 tx=0x7ff824001250 comp rx=0 tx=0).stop 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff829600640 1 -- 172.21.15.28:0/4084276746 reap_dead start 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 shutdown_connections 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 >> 172.21.15.28:0/4084276746 conn(0x558477b6a0a0 msgr2=0x558477b06f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 shutdown_connections 2024-09-10T10:50:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.022+0000 7ff82b946f40 1 -- 172.21.15.28:0/4084276746 wait complete. 2024-09-10T10:50:42.027 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 1.00000000 2024-09-10T10:50:42.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.102+0000 7f5f61e10f40 1 Processor -- start 2024-09-10T10:50:42.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.102+0000 7f5f61e10f40 1 -- start start 2024-09-10T10:50:42.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268c9fd00 0x563268d731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268cd11c0 0x563268d6de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563268c9df30 con 0x563268ca00e0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563268c46b30 con 0x563268c9fd00 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563268c9e210 con 0x563268cd11c0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f5d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f5d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45338/0 (socket says 172.21.15.28:45338) 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f5d800640 1 -- 172.21.15.28:0/1899684098 learned_addr learned my addr 172.21.15.28:0/1899684098 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f5e200640 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268c9fd00 0x563268d731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.103+0000 7f5f5ec00640 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268cd11c0 0x563268d6de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 -- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268cd11c0 msgr2=0x563268d6de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268cd11c0 0x563268d6de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 -- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268c9fd00 msgr2=0x563268d731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268c9fd00 0x563268d731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 -- 172.21.15.28:0/1899684098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563268d69520 con 0x563268ca00e0 2024-09-10T10:50:42.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f5d800640 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c001d80 tx=0x7f5f4c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c466b4c35448edd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f57e00640 1 -- 172.21.15.28:0/1899684098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00bd70 con 0x563268ca00e0 2024-09-10T10:50:42.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f57e00640 1 -- 172.21.15.28:0/1899684098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00f400 con 0x563268ca00e0 2024-09-10T10:50:42.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.104+0000 7f5f57e00640 1 -- 172.21.15.28:0/1899684098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00f7f0 con 0x563268ca00e0 2024-09-10T10:50:42.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 -- 172.21.15.28:0/1899684098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 msgr2=0x563268d68ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c001d80 tx=0x7f5f4c001e70 comp rx=0 tx=0).stop 2024-09-10T10:50:42.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 -- 172.21.15.28:0/1899684098 shutdown_connections 2024-09-10T10:50:42.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268cd11c0 0x563268d6de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268ca00e0 0x563268d68ba0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/1899684098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268c9fd00 0x563268d731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 -- 172.21.15.28:0/1899684098 >> 172.21.15.28:0/1899684098 conn(0x563268ced130 msgr2=0x563268ceb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 -- 172.21.15.28:0/1899684098 shutdown_connections 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 -- 172.21.15.28:0/1899684098 wait complete. 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.105+0000 7f5f61e10f40 1 Processor -- start 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 -- start start 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 0x563268d6e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268ca00e0 0x563268d760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268cd11c0 0x563268d794e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563268ceb910 con 0x563268c9fd00 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563268d73ed0 con 0x563268cd11c0 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.106+0000 7f5f61e10f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563268d73bf0 con 0x563268ca00e0 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 0x563268d6e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 0x563268d6e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45346/0 (socket says 172.21.15.28:45346) 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 -- 172.21.15.28:0/772227749 learned_addr learned my addr 172.21.15.28:0/772227749 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5d800640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268ca00e0 0x563268d760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5ec00640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268cd11c0 0x563268d794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 -- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268ca00e0 msgr2=0x563268d760a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563268ca00e0 0x563268d760a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 -- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268cd11c0 msgr2=0x563268d794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268cd11c0 0x563268d794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 -- 172.21.15.28:0/772227749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563268d6f670 con 0x563268c9fd00 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5ec00640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563268cd11c0 0x563268d794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.107+0000 7f5f5e200640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 0x563268d6e7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5f50008840 tx=0x7f5f50008930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.108+0000 7f5f56a00640 1 -- 172.21.15.28:0/772227749 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f500147e0 con 0x563268c9fd00 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.108+0000 7f5f56a00640 1 -- 172.21.15.28:0/772227749 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5001a070 con 0x563268c9fd00 2024-09-10T10:50:42.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.108+0000 7f5f56a00640 1 -- 172.21.15.28:0/772227749 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f50014bc0 con 0x563268c9fd00 2024-09-10T10:50:42.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.108+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563268d6cc30 con 0x563268c9fd00 2024-09-10T10:50:42.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.108+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563268d737a0 con 0x563268c9fd00 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.110+0000 7f5f56a00640 1 -- 172.21.15.28:0/772227749 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5f50015470 con 0x563268c9fd00 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.110+0000 7f5f56a00640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5f4006a810 0x7f5f4006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.110+0000 7f5f56a00640 1 -- 172.21.15.28:0/772227749 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f5f5008a7c0 con 0x563268c9fd00 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.111+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563268d6ed30 0x563268e10c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.111+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100) -- 0x563268d665b0 con 0x563268d6ed30 2024-09-10T10:50:42.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.111+0000 7f5f5d800640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5f4006a810 0x7f5f4006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.111+0000 7f5f5ec00640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563268d6ed30 0x563268e10c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f5ec00640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563268d6ed30 0x563268e10c20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f5d800640 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5f4006a810 0x7f5f4006ccd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563268cd1f00 tx=0x7f5f4c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f5ec00640 1 -- 172.21.15.28:0/772227749 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 154+0+16 (crc 0 0 0) 0x7f5f58056030 con 0x563268d6ed30 2024-09-10T10:50:42.114 INFO:teuthology.orchestra.run.smithi028.stdout:cephfs_metadata/1.00000000 mtime 2024-09-10T10:50:37.000000+0000, size 0 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563268d6ed30 msgr2=0x563268e10c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563268d6ed30 0x563268e10c20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5f4006a810 msgr2=0x7f5f4006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5f4006a810 0x7f5f4006ccd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x563268cd1f00 tx=0x7f5f4c009000 comp rx=0 tx=0).stop 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 msgr2=0x563268d6e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 --2- 172.21.15.28:0/772227749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563268c9fd00 0x563268d6e7f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f5f50008840 tx=0x7f5f50008930 comp rx=0 tx=0).stop 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f5ec00640 1 -- 172.21.15.28:0/772227749 reap_dead start 2024-09-10T10:50:42.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 shutdown_connections 2024-09-10T10:50:42.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 >> 172.21.15.28:0/772227749 conn(0x563268ced130 msgr2=0x563268ced510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:42.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 shutdown_connections 2024-09-10T10:50:42.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.112+0000 7f5f61e10f40 1 -- 172.21.15.28:0/772227749 wait complete. 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '10000000000.00000000' will be 'no damage' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '600.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000.inode' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '200.00000001' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_openfiles.0' will be 'no damage' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '500.00000000' will be 'no damage' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds_snaptable' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_inotable' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '400.00000000' will be 'no damage' 2024-09-10T10:50:42.118 INFO:tasks.cephfs.test_damage:Expectation on rm '100.00000000.inode' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on rm '1.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '10000000000.00000000' will be 'no damage' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '600.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_openfiles.0' will be 'no damage' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '100.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header 'mds0_sessionmap' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Expectation on corrupt header '1.00000000' will be 'damaged_on_start' 2024-09-10T10:50:42.119 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 10000000000.00000000' 2024-09-10T10:50:42.119 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:42.119 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:42.155 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:50:42.155 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:50:42.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:42.155 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:50:42.229 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:50:42.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:50:42.230 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:50:42.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.426+0000 7ff743200640 1 Processor -- start 2024-09-10T10:50:42.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.426+0000 7ff743200640 1 -- start start 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff743200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440ad8a0 0x7ff7440adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff743200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7440b5340 0x7ff7440b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff743200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff744070040 con 0x7ff74415e300 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff743200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff7440ae220 con 0x7ff7440b5340 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff743200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff74406f4c0 con 0x7ff7440ad8a0 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff741e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff74415e300 0x7ff744070610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56464/0 (socket says 172.21.15.28:56464) 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff741e00640 1 -- 172.21.15.28:0/604370991 learned_addr learned my addr 172.21.15.28:0/604370991 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:42.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.427+0000 7ff742800640 1 --2- 172.21.15.28:0/604370991 >> v2:172.21.15.28:3300/0 conn(0x7ff7440b5340 0x7ff7440b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff741400640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440ad8a0 0x7ff7440adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff740a00640 1 -- 172.21.15.28:0/604370991 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313109530 0 0) 0x7ff744070040 con 0x7ff74415e300 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff742800640 1 -- 172.21.15.28:0/604370991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440ad8a0 msgr2=0x7ff7440adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff742800640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440ad8a0 0x7ff7440adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff742800640 1 -- 172.21.15.28:0/604370991 >> v1:172.21.15.28:6789/0 conn(0x7ff74415e300 legacy=0x7ff744070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.428+0000 7ff742800640 1 -- 172.21.15.28:0/604370991 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff744070d20 con 0x7ff7440b5340 2024-09-10T10:50:42.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.429+0000 7ff742800640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7440b5340 0x7ff7440b57a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009d90 tx=0x7ff73c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e39e824bec1899 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.429+0000 7ff740a00640 1 -- 172.21.15.28:0/604370991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c007740 con 0x7ff7440b5340 2024-09-10T10:50:42.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.429+0000 7ff740a00640 1 -- 172.21.15.28:0/604370991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c0078e0 con 0x7ff7440b5340 2024-09-10T10:50:42.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.429+0000 7ff740a00640 1 -- 172.21.15.28:0/604370991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c007cb0 con 0x7ff7440b5340 2024-09-10T10:50:42.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 -- 172.21.15.28:0/604370991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7440b5340 msgr2=0x7ff7440b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7440b5340 0x7ff7440b57a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009d90 tx=0x7ff73c02a660 comp rx=0 tx=0).stop 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 -- 172.21.15.28:0/604370991 shutdown_connections 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7440b5340 0x7ff7440b57a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 --2- 172.21.15.28:0/604370991 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440ad8a0 0x7ff7440adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 -- 172.21.15.28:0/604370991 >> 172.21.15.28:0/604370991 conn(0x7ff744073470 msgr2=0x7ff74406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 -- 172.21.15.28:0/604370991 shutdown_connections 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.430+0000 7ff743200640 1 -- 172.21.15.28:0/604370991 wait complete. 2024-09-10T10:50:42.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.431+0000 7ff743200640 1 Processor -- start 2024-09-10T10:50:42.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 -- start start 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7440ad8a0 0x7ff744160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440b5340 0x7ff74414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff74406fd60 con 0x7ff74415e300 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff74406fa80 con 0x7ff7440ad8a0 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.432+0000 7ff743200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff74406f0e0 con 0x7ff7440b5340 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45354/0 (socket says 172.21.15.28:45354) 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 -- 172.21.15.28:0/194667833 learned_addr learned my addr 172.21.15.28:0/194667833 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff741400640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440b5340 0x7ff74414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff741e00640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7440ad8a0 0x7ff744160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 -- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440b5340 msgr2=0x7ff74414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440b5340 0x7ff74414f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 -- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7440ad8a0 msgr2=0x7ff744160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7440ad8a0 0x7ff744160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 -- 172.21.15.28:0/194667833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff744150660 con 0x7ff74415e300 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.433+0000 7ff742800640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009eb0 tx=0x7ff73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.434+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c040070 con 0x7ff74415e300 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.434+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73c031bc0 con 0x7ff74415e300 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.434+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73c009450 con 0x7ff74415e300 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.434+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74415f660 con 0x7ff74415e300 2024-09-10T10:50:42.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.434+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74415fc20 con 0x7ff74415e300 2024-09-10T10:50:42.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.435+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff744150830 con 0x7ff74415e300 2024-09-10T10:50:42.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.436+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff73c039040 con 0x7ff74415e300 2024-09-10T10:50:42.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.437+0000 7ff73a000640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:42.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.437+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7ff73c0ac290 con 0x7ff74415e300 2024-09-10T10:50:42.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.438+0000 7ff741e00640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:42.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.439+0000 7ff741e00640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 0x7ff72406e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff734000f60 tx=0x7ff734007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:42.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.442+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff73c04c070 con 0x7ff74415e300 2024-09-10T10:50:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:42.666+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7440adce0 con 0x7ff74415e300 2024-09-10T10:50:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.656+0000 7ff73a000640 1 -- 172.21.15.28:0/194667833 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7ff73c07a930 con 0x7ff74415e300 2024-09-10T10:50:43.658 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 msgr2=0x7ff72406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 0x7ff72406e320 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff734000f60 tx=0x7ff734007c00 comp rx=0 tx=0).stop 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 msgr2=0x7ff7440cebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009eb0 tx=0x7ff73c004a40 comp rx=0 tx=0).stop 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 shutdown_connections 2024-09-10T10:50:43.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff72406be60 0x7ff72406e320 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74415e300 0x7ff7440cebc0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7440b5340 0x7ff74414f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 --2- 172.21.15.28:0/194667833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7440ad8a0 0x7ff744160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 >> 172.21.15.28:0/194667833 conn(0x7ff744073470 msgr2=0x7ff74406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 shutdown_connections 2024-09-10T10:50:43.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.660+0000 7ff743200640 1 -- 172.21.15.28:0/194667833 wait complete. 2024-09-10T10:50:43.682 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:50:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.845+0000 7f8ebfe00640 1 Processor -- start 2024-09-10T10:50:43.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.845+0000 7f8ebfe00640 1 -- start start 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.846+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 0x7f8ec0072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebfe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec006dc70 con 0x7f8ec00b49c0 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec006df10 con 0x7f8ec015b330 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebfe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec006e1f0 con 0x7f8ec015af50 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebf400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8ec00b49c0 0x7f8ec00b2f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52990/0 (socket says 172.21.15.28:52990) 2024-09-10T10:50:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebf400640 1 -- 172.21.15.28:0/1677041871 learned_addr learned my addr 172.21.15.28:0/1677041871 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebea00640 1 --2- 172.21.15.28:0/1677041871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 0x7f8ec0072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 945192695 0 0) 0x7f8ec006dc70 con 0x7f8ec00b49c0 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.847+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ea4003520 con 0x7f8ec00b49c0 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.848+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2158786727 0 0) 0x7f8ec006df10 con 0x7f8ec015b330 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.848+0000 7f8ebea00640 1 -- 172.21.15.28:0/1677041871 >> v1:172.21.15.138:6789/0 conn(0x7f8ec015b330 legacy=0x7f8ec00b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.848+0000 7f8ebea00640 1 -- 172.21.15.28:0/1677041871 >> v1:172.21.15.28:6789/0 conn(0x7f8ec00b49c0 legacy=0x7f8ec00b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.848+0000 7f8ebea00640 1 -- 172.21.15.28:0/1677041871 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec006e4d0 con 0x7f8ec015af50 2024-09-10T10:50:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebea00640 1 --2- 172.21.15.28:0/1677041871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 0x7f8ec0072de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00097d0 tx=0x7f8eb002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3006aa614b1dd93f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0037070 con 0x7f8ec015af50 2024-09-10T10:50:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb002ac10 con 0x7f8ec015af50 2024-09-10T10:50:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebe000640 1 -- 172.21.15.28:0/1677041871 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8001a10 con 0x7f8ec015af50 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebfe00640 1 -- 172.21.15.28:0/1677041871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 msgr2=0x7f8ec0072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.849+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/1677041871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 0x7f8ec0072de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00097d0 tx=0x7f8eb002a180 comp rx=0 tx=0).stop 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.850+0000 7f8ebfe00640 1 -- 172.21.15.28:0/1677041871 shutdown_connections 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.850+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/1677041871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015af50 0x7f8ec0072de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.850+0000 7f8ebfe00640 1 -- 172.21.15.28:0/1677041871 >> 172.21.15.28:0/1677041871 conn(0x7f8ec006c240 msgr2=0x7f8ec00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.850+0000 7f8ebfe00640 1 -- 172.21.15.28:0/1677041871 shutdown_connections 2024-09-10T10:50:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.850+0000 7f8ebfe00640 1 -- 172.21.15.28:0/1677041871 wait complete. 2024-09-10T10:50:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.851+0000 7f8ebfe00640 1 Processor -- start 2024-09-10T10:50:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.852+0000 7f8ebfe00640 1 -- start start 2024-09-10T10:50:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.852+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec00b49c0 0x7f8ec0151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.852+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebfe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 0x7f8ec015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebfe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec00add40 con 0x7f8ec015af50 2024-09-10T10:50:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec00adbc0 con 0x7f8ec00b49c0 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebfe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec0073320 con 0x7f8ec015b330 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34808/0 (socket says 172.21.15.28:34808) 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 -- 172.21.15.28:0/3683929030 learned_addr learned my addr 172.21.15.28:0/3683929030 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebea00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec00b49c0 0x7f8ec0151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebf400640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 0x7f8ec015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 -- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 msgr2=0x7f8ec015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 0x7f8ec015c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 -- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec00b49c0 msgr2=0x7f8ec0151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec00b49c0 0x7f8ec0151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8eb7e00640 1 -- 172.21.15.28:0/3683929030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec015d140 con 0x7f8ec015af50 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.853+0000 7f8ebf400640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 0x7f8ec015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c440 tx=0x7f8ea800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8017070 con 0x7f8ec015af50 2024-09-10T10:50:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea80035f0 con 0x7f8ec015af50 2024-09-10T10:50:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80039e0 con 0x7f8ec015af50 2024-09-10T10:50:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec009bc20 con 0x7f8ec015af50 2024-09-10T10:50:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.854+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec009c1e0 con 0x7f8ec015af50 2024-09-10T10:50:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.855+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec0072de0 con 0x7f8ec015af50 2024-09-10T10:50:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.856+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ea801a070 con 0x7f8ec015af50 2024-09-10T10:50:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.857+0000 7f8eb7400640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 0x7f8e9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.857+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f8ea808a4a0 con 0x7f8ec015af50 2024-09-10T10:50:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.857+0000 7f8ebea00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 0x7f8e9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.858+0000 7f8ebea00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 0x7f8e9c06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0034000 tx=0x7f8eb002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:43.861+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ea8058bd0 con 0x7f8ec015af50 2024-09-10T10:50:44.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.112+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8ec015ea00 con 0x7f8ec015af50 2024-09-10T10:50:44.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.661+0000 7f8eb7400640 1 -- 172.21.15.28:0/3683929030 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v87) ==== 107+0+0 (secure 0 0 0) 0x7f8ea80584d0 con 0x7f8ec015af50 2024-09-10T10:50:44.664 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:50:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 msgr2=0x7f8e9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 0x7f8e9c06e2f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0034000 tx=0x7f8eb002e000 comp rx=0 tx=0).stop 2024-09-10T10:50:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 msgr2=0x7f8ec009c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c440 tx=0x7f8ea800c900 comp rx=0 tx=0).stop 2024-09-10T10:50:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 shutdown_connections 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9c06be30 0x7f8e9c06e2f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8ec015b330 0x7f8ec015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ec015af50 0x7f8ec009c700 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 --2- 172.21.15.28:0/3683929030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ec00b49c0 0x7f8ec0151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 >> 172.21.15.28:0/3683929030 conn(0x7f8ec006c240 msgr2=0x7f8ec01563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 shutdown_connections 2024-09-10T10:50:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.664+0000 7f8ebfe00640 1 -- 172.21.15.28:0/3683929030 wait complete. 2024-09-10T10:50:44.684 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:50:44.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.760+0000 7fd850c93f40 1 Processor -- start 2024-09-10T10:50:44.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.760+0000 7fd850c93f40 1 -- start start 2024-09-10T10:50:44.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 0x55839de8e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 0x55839de8e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55839de02530 con 0x55839ddb7470 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55839dd5fb30 con 0x55839de8d3e0 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd850c93f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55839de8fee0 con 0x55839de909c0 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.761+0000 7fd84de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34810/0 (socket says 172.21.15.28:34810) 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 -- 172.21.15.28:0/38950257 learned_addr learned my addr 172.21.15.28:0/38950257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84e800640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 0x55839de8e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84d400640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 0x55839de8e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 -- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 msgr2=0x55839de8e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 0x55839de8e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 -- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 msgr2=0x55839de8e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 0x55839de8e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 -- 172.21.15.28:0/38950257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839de8b370 con 0x55839ddb7470 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84e800640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 0x55839de8e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84d400640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 0x55839de8e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd84de00640 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b3c0 tx=0x7fd84000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e7814d4e27bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd847e00640 1 -- 172.21.15.28:0/38950257 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840015070 con 0x55839ddb7470 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd847e00640 1 -- 172.21.15.28:0/38950257 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84000d7c0 con 0x55839ddb7470 2024-09-10T10:50:44.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.762+0000 7fd847e00640 1 -- 172.21.15.28:0/38950257 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840008070 con 0x55839ddb7470 2024-09-10T10:50:44.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 -- 172.21.15.28:0/38950257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 msgr2=0x55839de8ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b3c0 tx=0x7fd84000b880 comp rx=0 tx=0).stop 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 -- 172.21.15.28:0/38950257 shutdown_connections 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839de909c0 0x55839de8e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de8d3e0 0x55839de8e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 --2- 172.21.15.28:0/38950257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839ddb7470 0x55839de8ae30 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 -- 172.21.15.28:0/38950257 >> 172.21.15.28:0/38950257 conn(0x55839de060a0 msgr2=0x55839de8f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 -- 172.21.15.28:0/38950257 shutdown_connections 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.763+0000 7fd850c93f40 1 -- 172.21.15.28:0/38950257 wait complete. 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.764+0000 7fd850c93f40 1 Processor -- start 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.764+0000 7fd850c93f40 1 -- start start 2024-09-10T10:50:44.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839ddb7470 0x55839de8a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de909c0 0x55839de89420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55839dde5ae0 con 0x55839de8d3e0 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55839de06480 con 0x55839de909c0 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd850c93f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55839ddeb900 con 0x55839ddb7470 2024-09-10T10:50:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd84d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd84d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34824/0 (socket says 172.21.15.28:34824) 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 learned_addr learned my addr 172.21.15.28:0/2338638351 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de909c0 0x55839de89420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.765+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839ddb7470 0x55839de8a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839ddb7470 msgr2=0x55839de8a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839ddb7470 0x55839de8a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de909c0 msgr2=0x55839de89420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de909c0 0x55839de89420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839de91910 con 0x55839de8d3e0 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55839de909c0 0x55839de89420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55839ddb7470 0x55839de8a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd83c008450 tx=0x7fd83c008540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd846a00640 1 -- 172.21.15.28:0/2338638351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0143f0 con 0x55839de8d3e0 2024-09-10T10:50:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd846a00640 1 -- 172.21.15.28:0/2338638351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd83c014590 con 0x55839de8d3e0 2024-09-10T10:50:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55839de91bf0 con 0x55839de8d3e0 2024-09-10T10:50:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd846a00640 1 -- 172.21.15.28:0/2338638351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c014980 con 0x55839de8d3e0 2024-09-10T10:50:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.766+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839de921b0 con 0x55839de8d3e0 2024-09-10T10:50:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.768+0000 7fd846a00640 1 -- 172.21.15.28:0/2338638351 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd83c01b020 con 0x55839de8d3e0 2024-09-10T10:50:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.769+0000 7fd846a00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 0x7fd83006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.769+0000 7fd846a00640 1 -- 172.21.15.28:0/2338638351 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd83c089920 con 0x55839de8d3e0 2024-09-10T10:50:44.773 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.769+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55839df28df0 0x55839de929b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.769+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839de92ef0 con 0x55839df28df0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.770+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 0x7fd83006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.770+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55839df28df0 0x55839de929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.770+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55839df28df0 0x55839de929b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.770+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 0x7fd83006ccd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b3c0 tx=0x7fd840005000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.772+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v100'11 uv4 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.772+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839de932c0 con 0x55839df28df0 2024-09-10T10:50:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.774+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v100'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.774+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df324a0 con 0x55839df28df0 2024-09-10T10:50:44.778 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:50:44.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.776+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v100'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.776+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df33850 con 0x55839df28df0 2024-09-10T10:50:44.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.777+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v100'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.777+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df33bf0 con 0x55839df28df0 2024-09-10T10:50:44.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.779+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v100'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.779+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32840 con 0x55839df28df0 2024-09-10T10:50:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.780+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v100'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.780+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.782+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v100'17 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.782+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.784+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v100'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.784+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.786+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v100'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.788 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:50:44.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.786+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.787+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v100'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.787+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.789+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v100'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.789+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.792+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v100'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.792+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.793+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v100'23 uv10 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.793+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.794+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v100'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.794+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.796+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v100'25 uv25 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.797 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:50:44.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.796+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.797+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v100'26 uv26 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.797+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.799+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v100'27 uv27 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.799+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32c30 con 0x55839df28df0 2024-09-10T10:50:44.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.800+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v100'28 uv28 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.800+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ff0 con 0x55839df28df0 2024-09-10T10:50:44.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.802+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v100'29 uv29 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.802+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.804+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v100'30 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.804+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.805+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.805+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.806+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.809 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:50:44.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.807+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.808+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v100'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.808+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.810+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v100'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.810+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.811+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v100'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.811+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.815+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v100'36 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.815+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.816+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v100'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.816+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.819 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:50:44.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.818+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v100'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.818+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.820+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v100'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.820+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.821+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.821+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df28df0 2024-09-10T10:50:44.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.824+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v100'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df28df0 2024-09-10T10:50:44.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.824+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55839df33f90 0x55839df36380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.824+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df32ef0 con 0x55839df33f90 2024-09-10T10:50:44.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.824+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55839df33f90 0x55839df36380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.824+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55839df33f90 0x55839df36380 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.826+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v100'20 uv15 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.826+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df368c0 con 0x55839df33f90 2024-09-10T10:50:44.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.827+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v100'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.827+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df36c60 con 0x55839df33f90 2024-09-10T10:50:44.830 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:50:44.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.828+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v100'22 uv22 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.828+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37000 con 0x55839df33f90 2024-09-10T10:50:44.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.830+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v100'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.830+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df373a0 con 0x55839df33f90 2024-09-10T10:50:44.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.832+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v100'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.832+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37740 con 0x55839df33f90 2024-09-10T10:50:44.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.833+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v100'25 uv19 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.833+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.834+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v100'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.834+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.838 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:50:44.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.836+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v100'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.836+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.837+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v100'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.837+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.839+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v100'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.839+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.840+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v100'30 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.840+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.841+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v100'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.841+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.845 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:50:44.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.843+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v100'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.843+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.845+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v100'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.845+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.846+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v100'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.846+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.848+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v100'35 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.848+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.849+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v100'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.849+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.852 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:50:44.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.850+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v100'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.850+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.851+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v100'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.851+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.852+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v100'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.852+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df37ae0 con 0x55839df33f90 2024-09-10T10:50:44.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.854+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v100'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df33f90 2024-09-10T10:50:44.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.854+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55839df384d0 0x55839df3a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.854+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3ae00 con 0x55839df384d0 2024-09-10T10:50:44.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.854+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55839df384d0 0x55839df3a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.854+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55839df384d0 0x55839df3a8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.856+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v100'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.856+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df3b1a0 con 0x55839df384d0 2024-09-10T10:50:44.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.857+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v100'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.857+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3b680 con 0x55839df384d0 2024-09-10T10:50:44.860 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:50:44.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.858+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.858+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3cf30 con 0x55839df384d0 2024-09-10T10:50:44.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.860+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.860+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3d2d0 con 0x55839df384d0 2024-09-10T10:50:44.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.861+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v100'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.861+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3bf20 con 0x55839df384d0 2024-09-10T10:50:44.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.862+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v100'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.862+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55839df3c820 0x55839df3f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.862+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3fbc0 con 0x55839df3c820 2024-09-10T10:50:44.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.863+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55839df3c820 0x55839df3f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.863+0000 7fd84e800640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55839df3c820 0x55839df3f680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.864+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v100'5 uv4 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.864+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df3fbc0 con 0x55839df3c820 2024-09-10T10:50:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.866+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v100'6 uv6 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.866+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df3fc40 con 0x55839df3c820 2024-09-10T10:50:44.868 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:50:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.867+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v100'7 uv7 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.867+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.868+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v100'8 uv8 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.868+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.869+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v100'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.869+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.870+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v100'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.870+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.873 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:50:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.871+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.871+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df41040 con 0x55839df384d0 2024-09-10T10:50:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.873+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.873+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df41040 con 0x55839df384d0 2024-09-10T10:50:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.874+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v100'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.874+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.875+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v100'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.875+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.877+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v100'4 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.877+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.878+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.878+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.881 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:50:44.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.879+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.879+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.881+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v100'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.881+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.882+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v100'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.882+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.884+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v100'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.884+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.885+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v100'10 uv3 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.885+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.886+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v100'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.886+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.890 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:50:44.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.888+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v100'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.888+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.889+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v100'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.889+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df3c820 2024-09-10T10:50:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.892+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v100'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.892+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.893+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v100'2 uv1 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.893+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.894+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v100'3 uv3 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.894+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.897 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:50:44.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.896+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v100'4 uv4 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.896+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.897+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v100'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.897+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df40030 con 0x55839df384d0 2024-09-10T10:50:44.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.898+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v100'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df384d0 2024-09-10T10:50:44.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.898+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55839df40950 0x55839df42d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.898+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43280 con 0x55839df40950 2024-09-10T10:50:44.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.898+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55839df40950 0x55839df42d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.899+0000 7fd84de00640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55839df40950 0x55839df42d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.900+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v100'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.901+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df43280 con 0x55839df40950 2024-09-10T10:50:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.903+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v100'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.903+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:50:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.906+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.906+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44680 con 0x55839df40950 2024-09-10T10:50:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.909+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.909+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44680 con 0x55839df40950 2024-09-10T10:50:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.911+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v100'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.911+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.912+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v100'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.912+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.914+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v100'4 uv3 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.914+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.915+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v100'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.915+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.918 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:50:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.916+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v100'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.916+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.918+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v100'7 uv7 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.918+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.919+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v100'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.919+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.921+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v100'9 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.921+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.922+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v100'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.922+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.925 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:50:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.923+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v100'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.923+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43670 con 0x55839df40950 2024-09-10T10:50:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.925+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v100'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.925+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df46480 con 0x55839df40950 2024-09-10T10:50:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.926+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v100'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.926+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.927+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v100'4 uv3 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.927+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.929+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v100'5 uv5 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.929+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.932 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:50:44.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.930+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v100'6 uv6 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.930+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.931+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v100'7 uv7 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.931+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.933+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v100'3 uv2 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.933+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.934+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v100'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.934+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.937 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:50:44.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.935+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.935+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.936+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.936+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44b30 con 0x55839df3c820 2024-09-10T10:50:44.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.937+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v100'7 uv7 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.937+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df43b20 con 0x55839df3c820 2024-09-10T10:50:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.938+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v100'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd8480527f0 con 0x55839df3c820 2024-09-10T10:50:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.938+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55839df441a0 0x55839df44580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.938+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df441a0 2024-09-10T10:50:44.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.939+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55839df441a0 0x55839df44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:44.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.939+0000 7fd84d400640 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55839df441a0 0x55839df44580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:44.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.940+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v100'2 uv1 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df441a0 2024-09-10T10:50:44.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.940+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df441a0 2024-09-10T10:50:44.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.941+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v100'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df441a0 2024-09-10T10:50:44.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.941+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df441a0 2024-09-10T10:50:44.944 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:50:44.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.943+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v100'4 uv4 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df441a0 2024-09-10T10:50:44.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.943+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df441a0 2024-09-10T10:50:44.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.944+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v100'5 uv5 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df441a0 2024-09-10T10:50:44.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.944+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df441a0 2024-09-10T10:50:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.945+0000 7fd84d400640 1 -- 172.21.15.28:0/2338638351 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v100'6 uv6 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd83c053fd0 con 0x55839df441a0 2024-09-10T10:50:44.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.945+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.946+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v100'7 uv3 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.946+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.947+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v100'8 uv8 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.947+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.950 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:50:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.948+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v100'9 uv9 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.948+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.949+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v100'10 uv10 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.949+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.951+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v100'11 uv11 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.951+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.952+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v100'12 uv6 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.952+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.953+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v100'13 uv13 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.953+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.956 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:50:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.954+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v100'14 uv14 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.954+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.955+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v100'15 uv15 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.955+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.956+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v100'16 uv16 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.956+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df44ac0 con 0x55839df40950 2024-09-10T10:50:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.957+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v100'17 uv17 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.957+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e100) -- 0x55839df453a0 con 0x55839df40950 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd84de00640 1 -- 172.21.15.28:0/2338638351 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v100'18 uv18 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fd840025040 con 0x55839df40950 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55839df384d0 msgr2=0x55839df3a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55839df384d0 0x55839df3a8c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55839df40950 msgr2=0x55839df42d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55839df40950 0x55839df42d40 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55839df441a0 msgr2=0x55839df44580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55839df441a0 0x55839df44580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55839df3c820 msgr2=0x55839df3f680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55839df3c820 0x55839df3f680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55839df33f90 msgr2=0x55839df36380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55839df33f90 0x55839df36380 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55839df28df0 msgr2=0x55839de929b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55839df28df0 0x55839de929b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 reap_dead start 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.958+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 reap_dead start 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 reap_dead start 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd84e800640 1 -- 172.21.15.28:0/2338638351 reap_dead start 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 msgr2=0x7fd83006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 0x7fd83006ccd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd84000b3c0 tx=0x7fd840005000 comp rx=0 tx=0).stop 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 msgr2=0x55839df25de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd83c008450 tx=0x7fd83c008540 comp rx=0 tx=0).stop 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 shutdown_connections 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd83006a810 0x7fd83006ccd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 --2- 172.21.15.28:0/2338638351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55839de8d3e0 0x55839df25de0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 >> 172.21.15.28:0/2338638351 conn(0x55839de060a0 msgr2=0x55839de8f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 shutdown_connections 2024-09-10T10:50:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:44.959+0000 7fd850c93f40 1 -- 172.21.15.28:0/2338638351 wait complete. 2024-09-10T10:50:44.966 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-10T10:50:45.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 Processor -- start 2024-09-10T10:50:45.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 -- start start 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e06ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d791d81530 con 0x55d791e0f9c0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d791cdeb30 con 0x55d791e0c3e0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.080+0000 7f6835e21f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d791e0eee0 con 0x55d791d808e0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e06ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:53990/0 (socket says 172.21.15.28:53990) 2024-09-10T10:50:45.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:33914/0 (socket says 172.21.15.28:33914) 2024-09-10T10:50:45.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833000640 1 -- 172.21.15.28:0/2051138680 learned_addr learned my addr 172.21.15.28:0/2051138680 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:45.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 -- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 msgr2=0x55d791e02ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:45.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 -- 172.21.15.28:0/2051138680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 msgr2=0x55d791e06ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e06ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6832600640 1 -- 172.21.15.28:0/2051138680 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d791d859f0 con 0x55d791e0c3e0 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833000640 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.081+0000 7f6833a00640 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6832600640 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f682000a5e0 tx=0x7f6820005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06b7dc2007068e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6831200640 1 -- 172.21.15.28:0/2051138680 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820019070 con 0x55d791e0c3e0 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6831200640 1 -- 172.21.15.28:0/2051138680 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6820007600 con 0x55d791e0c3e0 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6831200640 1 -- 172.21.15.28:0/2051138680 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68200079f0 con 0x55d791e0c3e0 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6835e21f40 1 -- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 msgr2=0x55d791d835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.082+0000 7f6835e21f40 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f682000a5e0 tx=0x7f6820005f90 comp rx=0 tx=0).stop 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 -- 172.21.15.28:0/2051138680 shutdown_connections 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e06ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d835d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 --2- 172.21.15.28:0/2051138680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e02ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 -- 172.21.15.28:0/2051138680 >> 172.21.15.28:0/2051138680 conn(0x55d791d850a0 msgr2=0x55d791d36070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 -- 172.21.15.28:0/2051138680 shutdown_connections 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 -- 172.21.15.28:0/2051138680 wait complete. 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 Processor -- start 2024-09-10T10:50:45.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.083+0000 7f6835e21f40 1 -- start start 2024-09-10T10:50:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d82bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e129c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d791e0e9a0 con 0x55d791e0f9c0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d791e0cd60 con 0x55d791e0c3e0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.084+0000 7f6835e21f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d791e03370 con 0x55d791d808e0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e129c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34858/0 (socket says 172.21.15.28:34858) 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 -- 172.21.15.28:0/3986667273 learned_addr learned my addr 172.21.15.28:0/3986667273 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6832600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d82bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 -- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 msgr2=0x55d791e000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d791d808e0 0x55d791e000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 -- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 msgr2=0x55d791d82bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d791e0c3e0 0x55d791d82bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 -- 172.21.15.28:0/3986667273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d791d83560 con 0x55d791e0f9c0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6833a00640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e129c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f682c00d3f0 tx=0x7f682c00d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f682b400640 1 -- 172.21.15.28:0/3986667273 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c017070 con 0x55d791e0f9c0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f682b400640 1 -- 172.21.15.28:0/3986667273 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f682c006030 con 0x55d791e0f9c0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f682b400640 1 -- 172.21.15.28:0/3986667273 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c011590 con 0x55d791e0f9c0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.085+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d791d83840 con 0x55d791e0f9c0 2024-09-10T10:50:45.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.086+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d791e124a0 con 0x55d791e0f9c0 2024-09-10T10:50:45.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.087+0000 7f682b400640 1 -- 172.21.15.28:0/3986667273 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f682c01a070 con 0x55d791e0f9c0 2024-09-10T10:50:45.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.088+0000 7f682b400640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681406a810 0x7f681406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.088+0000 7f682b400640 1 -- 172.21.15.28:0/3986667273 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f682c088ec0 con 0x55d791e0f9c0 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.088+0000 7f6835e21f40 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d791d830f0 0x55d791e9e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.089+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e100) -- 0x55d791e007f0 con 0x55d791d830f0 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.089+0000 7f6833000640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681406a810 0x7f681406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.089+0000 7f6832600640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d791d830f0 0x55d791e9e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.089+0000 7f6832600640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d791d830f0 0x55d791e9e7e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.089+0000 7f6833000640 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681406a810 0x7f681406ccd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f682400a5a0 tx=0x7f6824052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6832600640 1 -- 172.21.15.28:0/3986667273 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v100'42 uv42 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f682001c040 con 0x55d791d830f0 2024-09-10T10:50:45.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d791d830f0 msgr2=0x55d791e9e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d791d830f0 0x55d791e9e7e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681406a810 msgr2=0x7f681406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681406a810 0x7f681406ccd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f682400a5a0 tx=0x7f6824052000 comp rx=0 tx=0).stop 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 msgr2=0x55d791e129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 --2- 172.21.15.28:0/3986667273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d791e0f9c0 0x55d791e129c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f682c00d3f0 tx=0x7f682c00d8b0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6833a00640 1 -- 172.21.15.28:0/3986667273 reap_dead start 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 shutdown_connections 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 >> 172.21.15.28:0/3986667273 conn(0x55d791d850a0 msgr2=0x55d791d36070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 shutdown_connections 2024-09-10T10:50:45.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.091+0000 7f6835e21f40 1 -- 172.21.15.28:0/3986667273 wait complete. 2024-09-10T10:50:45.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:50:45.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.255+0000 7f4127000640 1 Processor -- start 2024-09-10T10:50:45.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.255+0000 7f4127000640 1 -- start start 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41200cf280 0x7f4120162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4120162c60 0x7f41200d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f412006de90 con 0x7f4120162c60 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f412015ff70 con 0x7f41200cf280 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.256+0000 7f4127000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41200d4d10 con 0x7f4120162340 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4120162c60 0x7f41200d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4120162c60 0x7f41200d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34870/0 (socket says 172.21.15.28:34870) 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 -- 172.21.15.28:0/4034058235 learned_addr learned my addr 172.21.15.28:0/4034058235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f411fe00640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4117e00640 1 --2- 172.21.15.28:0/4034058235 >> v2:172.21.15.138:3300/0 conn(0x7f41200cf280 0x7f4120162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 -- 172.21.15.28:0/4034058235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 msgr2=0x7f412006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:45.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412006d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 -- 172.21.15.28:0/4034058235 >> v2:172.21.15.138:3300/0 conn(0x7f41200cf280 msgr2=0x7f4120162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 --2- 172.21.15.28:0/4034058235 >> v2:172.21.15.138:3300/0 conn(0x7f41200cf280 0x7f4120162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.257+0000 7f4125000640 1 -- 172.21.15.28:0/4034058235 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41200d4ff0 con 0x7f4120162c60 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.258+0000 7f4125000640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4120162c60 0x7f41200d2990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f412008d200 tx=0x7f411802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a4a94e043d38fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.258+0000 7f411f400640 1 -- 172.21.15.28:0/4034058235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118036070 con 0x7f4120162c60 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.258+0000 7f411f400640 1 -- 172.21.15.28:0/4034058235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4118031600 con 0x7f4120162c60 2024-09-10T10:50:45.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.258+0000 7f411f400640 1 -- 172.21.15.28:0/4034058235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41180319d0 con 0x7f4120162c60 2024-09-10T10:50:45.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 -- 172.21.15.28:0/4034058235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4120162c60 msgr2=0x7f41200d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4120162c60 0x7f41200d2990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f412008d200 tx=0x7f411802a610 comp rx=0 tx=0).stop 2024-09-10T10:50:45.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 -- 172.21.15.28:0/4034058235 shutdown_connections 2024-09-10T10:50:45.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4120162c60 0x7f41200d2990 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 --2- 172.21.15.28:0/4034058235 >> v2:172.21.15.138:3300/0 conn(0x7f41200cf280 0x7f4120162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 --2- 172.21.15.28:0/4034058235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412006d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 -- 172.21.15.28:0/4034058235 >> 172.21.15.28:0/4034058235 conn(0x7f412008bfc0 msgr2=0x7f41200d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 -- 172.21.15.28:0/4034058235 shutdown_connections 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.259+0000 7f4127000640 1 -- 172.21.15.28:0/4034058235 wait complete. 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.261+0000 7f4127000640 1 Processor -- start 2024-09-10T10:50:45.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.261+0000 7f4127000640 1 -- start start 2024-09-10T10:50:45.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4120162c60 0x7f412015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41200cf6c0 con 0x7f41200cf280 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41200d52d0 con 0x7f4120162c60 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4127000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4120163110 con 0x7f4120162340 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f411fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f411fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34884/0 (socket says 172.21.15.28:34884) 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f411fe00640 1 -- 172.21.15.28:0/833826629 learned_addr learned my addr 172.21.15.28:0/833826629 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4117e00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.262+0000 7f4125000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4120162c60 0x7f412015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 -- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 msgr2=0x7f412015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 -- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4120162c60 msgr2=0x7f412015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4120162c60 0x7f412015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 -- 172.21.15.28:0/833826629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41200cbb60 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411fe00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f411000e870 tx=0x7f411000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110017070 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411000c940 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411000cd30 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41200cbe40 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.263+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41200cc400 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.264+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4120162720 con 0x7f41200cf280 2024-09-10T10:50:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.265+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f411001a070 con 0x7f41200cf280 2024-09-10T10:50:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.266+0000 7f411cc00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 0x7f40fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.266+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f411008b120 con 0x7f41200cf280 2024-09-10T10:50:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.267+0000 7f4117e00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 0x7f40fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.267+0000 7f4117e00640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 0x7f40fc06e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f41200c7b50 tx=0x7f4108009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.270+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4110059330 con 0x7f41200cf280 2024-09-10T10:50:45.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.489+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f412008e640 con 0x7f41200cf280 2024-09-10T10:50:45.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.663+0000 7f411cc00640 1 -- 172.21.15.28:0/833826629 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v88) ==== 162+0+0 (secure 0 0 0) 0x7f411005d630 con 0x7f41200cf280 2024-09-10T10:50:45.666 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 msgr2=0x7f40fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 0x7f40fc06e2f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f41200c7b50 tx=0x7f4108009110 comp rx=0 tx=0).stop 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 msgr2=0x7f4120153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f411000e870 tx=0x7f411000ed30 comp rx=0 tx=0).stop 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 shutdown_connections 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40fc06be30 0x7f40fc06e2f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4120162c60 0x7f412015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4120162340 0x7f412015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 --2- 172.21.15.28:0/833826629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41200cf280 0x7f4120153b80 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 >> 172.21.15.28:0/833826629 conn(0x7f412008bfc0 msgr2=0x7f41200cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 shutdown_connections 2024-09-10T10:50:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:45.667+0000 7f4127000640 1 -- 172.21.15.28:0/833826629 wait complete. 2024-09-10T10:50:46.091 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:46.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.249+0000 7fefb3200640 1 Processor -- start 2024-09-10T10:50:46.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.249+0000 7fefb3200640 1 -- start start 2024-09-10T10:50:46.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.250+0000 7fefb3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefac0d4180 0x7fefac0d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.250+0000 7fefb3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 0x7fefac0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefac0d4aa0 con 0x7fefac15e6a0 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fefac0cefb0 con 0x7fefac0d4180 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fefac0d2c20 con 0x7fefac0cf440 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefac0d4180 0x7fefac0d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefac0d4180 0x7fefac0d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34900/0 (socket says 172.21.15.28:34900) 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 -- 172.21.15.28:0/3589406193 learned_addr learned my addr 172.21.15.28:0/3589406193 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb2800640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 0x7fefac0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb0a00640 1 -- 172.21.15.28:0/3589406193 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539061836 0 0) 0x7fefac0d4aa0 con 0x7fefac15e6a0 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 -- 172.21.15.28:0/3589406193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 msgr2=0x7fefac0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 0x7fefac0cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 -- 172.21.15.28:0/3589406193 >> v1:172.21.15.138:6789/0 conn(0x7fefac15e6a0 legacy=0x7fefac06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb1e00640 1 -- 172.21.15.28:0/3589406193 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac06e390 con 0x7fefac0d4180 2024-09-10T10:50:46.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.251+0000 7fefb2800640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 0x7fefac0cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.252+0000 7fefb1e00640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0d4180 0x7fefac0d4560 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009870 tx=0x7fefa002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0687c1555b54ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb0a00640 1 -- 172.21.15.28:0/3589406193 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0037070 con 0x7fefac0d4180 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb0a00640 1 -- 172.21.15.28:0/3589406193 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0032440 con 0x7fefac0d4180 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb0a00640 1 -- 172.21.15.28:0/3589406193 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0032810 con 0x7fefac0d4180 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 -- 172.21.15.28:0/3589406193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0d4180 msgr2=0x7fefac0d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0d4180 0x7fefac0d4560 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009870 tx=0x7fefa002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 -- 172.21.15.28:0/3589406193 shutdown_connections 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0cf440 0x7fefac0cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 --2- 172.21.15.28:0/3589406193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0d4180 0x7fefac0d4560 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 -- 172.21.15.28:0/3589406193 >> 172.21.15.28:0/3589406193 conn(0x7fefac09d6f0 msgr2=0x7fefac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:46.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.253+0000 7fefb3200640 1 -- 172.21.15.28:0/3589406193 shutdown_connections 2024-09-10T10:50:46.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.254+0000 7fefb3200640 1 -- 172.21.15.28:0/3589406193 wait complete. 2024-09-10T10:50:46.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.255+0000 7fefb3200640 1 Processor -- start 2024-09-10T10:50:46.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.255+0000 7fefb3200640 1 -- start start 2024-09-10T10:50:46.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0d4180 0x7fefac1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefac15e6a0 0x7fefac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0cebd0 con 0x7fefac0cf440 2024-09-10T10:50:46.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefac06e950 con 0x7fefac15e6a0 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.256+0000 7fefb3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fefac06df30 con 0x7fefac0d4180 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34914/0 (socket says 172.21.15.28:34914) 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 -- 172.21.15.28:0/284764057 learned_addr learned my addr 172.21.15.28:0/284764057 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1400640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0d4180 0x7fefac1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb2800640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefac15e6a0 0x7fefac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 -- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0d4180 msgr2=0x7fefac1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0d4180 0x7fefac1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 -- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefac15e6a0 msgr2=0x7fefac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefac15e6a0 0x7fefac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 -- 172.21.15.28:0/284764057 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac1f4890 con 0x7fefac0cf440 2024-09-10T10:50:46.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.257+0000 7fefb1e00640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fefa0000c00 tx=0x7fefa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:46.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.258+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0037040 con 0x7fefac0cf440 2024-09-10T10:50:46.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.258+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0042070 con 0x7fefac0cf440 2024-09-10T10:50:46.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.258+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa00308a0 con 0x7fefac0cf440 2024-09-10T10:50:46.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.258+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac1f4b70 con 0x7fefac0cf440 2024-09-10T10:50:46.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.258+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1f7190 con 0x7fefac0cf440 2024-09-10T10:50:46.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.259+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac0d4560 con 0x7fefac0cf440 2024-09-10T10:50:46.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.260+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fefa002ad10 con 0x7fefac0cf440 2024-09-10T10:50:46.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.261+0000 7fefa6000640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 0x7fef9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:46.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.261+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fefa00ab440 con 0x7fefac0cf440 2024-09-10T10:50:46.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.262+0000 7fefb1400640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 0x7fef9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:46.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.262+0000 7fefb1400640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 0x7fef9006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f60 tx=0x7fef9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:46.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.266+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fefa0079c00 con 0x7fefac0cf440 2024-09-10T10:50:46.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.484+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefac08c160 con 0x7fefac0cf440 2024-09-10T10:50:46.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.485+0000 7fefa6000640 1 -- 172.21.15.28:0/284764057 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5220 (secure 0 0 0) 0x7fefa0079500 con 0x7fefac0cf440 2024-09-10T10:50:46.487 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:46.487 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":89,"btime":"2024-09-10T10:50:45:668334+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:45.668320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2814091387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2814091387},{"type":"v1","addr":"172.21.15.28:6835","nonce":2814091387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:50:46.487 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 89 2024-09-10T10:50:46.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 msgr2=0x7fef9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 0x7fef9006e2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000f60 tx=0x7fef9c009040 comp rx=0 tx=0).stop 2024-09-10T10:50:46.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 msgr2=0x7fefac1f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:46.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fefa0000c00 tx=0x7fefa00046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 shutdown_connections 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef9006be30 0x7fef9006e2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefac15e6a0 0x7fefac15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefac0d4180 0x7fefac1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 --2- 172.21.15.28:0/284764057 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefac0cf440 0x7fefac1f6c50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 >> 172.21.15.28:0/284764057 conn(0x7fefac09d6f0 msgr2=0x7fefac0b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 shutdown_connections 2024-09-10T10:50:46.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:46.487+0000 7fefb3200640 1 -- 172.21.15.28:0/284764057 wait complete. 2024-09-10T10:50:46.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:50:45.668320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2814091387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2814091387}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2814091387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:50:46.505 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7008:up:replay 2024-09-10T10:50:46.505 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:50:48.919 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:50:48.917+0000 7f0dab600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:51.910 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:52.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.076+0000 7f387ce00640 1 Processor -- start 2024-09-10T10:50:52.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.077+0000 7f387ce00640 1 -- start start 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f387ce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3878072de0 0x7f387815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f387ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 0x7f387815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f387ce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3878146140 con 0x7f3878072320 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f387ce00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f38780cd740 con 0x7f3878072de0 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f387ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f38780cda20 con 0x7f3878072700 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f386fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3878072de0 0x7f387815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f386fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3878072de0 0x7f387815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34926/0 (socket says 172.21.15.28:34926) 2024-09-10T10:50:52.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f386fe00640 1 -- 172.21.15.28:0/1529446660 learned_addr learned my addr 172.21.15.28:0/1529446660 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f3876800640 1 --2- 172.21.15.28:0/1529446660 >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 0x7f387815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.078+0000 7f386fe00640 1 -- 172.21.15.28:0/1529446660 >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 msgr2=0x7f387815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.079+0000 7f386fe00640 1 --2- 172.21.15.28:0/1529446660 >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 0x7f387815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.079+0000 7f386fe00640 1 -- 172.21.15.28:0/1529446660 >> v1:172.21.15.28:6789/0 conn(0x7f3878072320 legacy=0x7f38780cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.079+0000 7f386fe00640 1 -- 172.21.15.28:0/1529446660 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780cdd00 con 0x7f3878072de0 2024-09-10T10:50:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.079+0000 7f3875400640 1 -- 172.21.15.28:0/1529446660 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896775271 0 0) 0x7f3878146140 con 0x7f3878072320 2024-09-10T10:50:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.079+0000 7f3876800640 1 --2- 172.21.15.28:0/1529446660 >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 0x7f387815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f386fe00640 1 --2- 172.21.15.28:0/1529446660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072de0 0x7f387815d1e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f38780cfce0 tx=0x7f386002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76a79dd612609cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f3875400640 1 -- 172.21.15.28:0/1529446660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860037070 con 0x7f3878072de0 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f3875400640 1 -- 172.21.15.28:0/1529446660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3860032440 con 0x7f3878072de0 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f3875400640 1 -- 172.21.15.28:0/1529446660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860032810 con 0x7f3878072de0 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 -- 172.21.15.28:0/1529446660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072de0 msgr2=0x7f387815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 --2- 172.21.15.28:0/1529446660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072de0 0x7f387815d1e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f38780cfce0 tx=0x7f386002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 -- 172.21.15.28:0/1529446660 shutdown_connections 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 --2- 172.21.15.28:0/1529446660 >> v2:172.21.15.138:3300/0 conn(0x7f3878072700 0x7f387815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 --2- 172.21.15.28:0/1529446660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072de0 0x7f387815d1e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.080+0000 7f387ce00640 1 -- 172.21.15.28:0/1529446660 >> 172.21.15.28:0/1529446660 conn(0x7f3878067d70 msgr2=0x7f387806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.081+0000 7f387ce00640 1 -- 172.21.15.28:0/1529446660 shutdown_connections 2024-09-10T10:50:52.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.081+0000 7f387ce00640 1 -- 172.21.15.28:0/1529446660 wait complete. 2024-09-10T10:50:52.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.082+0000 7f387ce00640 1 Processor -- start 2024-09-10T10:50:52.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.082+0000 7f387ce00640 1 -- start start 2024-09-10T10:50:52.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 0x7f387815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3878072de0 0x7f387813c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f387815dde0 con 0x7f3878072320 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f387815dc60 con 0x7f3878072700 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f387ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f387815df60 con 0x7f3878072de0 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.083+0000 7f3875e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34930/0 (socket says 172.21.15.28:34930) 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 -- 172.21.15.28:0/1569687994 learned_addr learned my addr 172.21.15.28:0/1569687994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f386fe00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 0x7f387815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3876800640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3878072de0 0x7f387813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 -- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3878072de0 msgr2=0x7f387813c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3878072de0 0x7f387813c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 -- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 msgr2=0x7f387815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 0x7f387815b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 -- 172.21.15.28:0/1569687994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38780b1e90 con 0x7f3878072320 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f386fe00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 0x7f387815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f3875e00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f386800c4e0 tx=0x7f386800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.084+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868017070 con 0x7f3878072320 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.085+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38680035f0 con 0x7f3878072320 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.085+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680039e0 con 0x7f3878072320 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.085+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387813df10 con 0x7f3878072320 2024-09-10T10:50:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.085+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387813e4d0 con 0x7f3878072320 2024-09-10T10:50:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.086+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387815e320 con 0x7f3878072320 2024-09-10T10:50:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.087+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f386801a070 con 0x7f3878072320 2024-09-10T10:50:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.088+0000 7f386e000640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 0x7f385406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.088+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f386808a3a0 con 0x7f3878072320 2024-09-10T10:50:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.088+0000 7f386fe00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 0x7f385406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.089+0000 7f386fe00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 0x7f385406e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3860034000 tx=0x7f386002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.092+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3868058480 con 0x7f3878072320 2024-09-10T10:50:52.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.318+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38780b2a50 con 0x7f3878072320 2024-09-10T10:50:52.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.319+0000 7f386e000640 1 -- 172.21.15.28:0/1569687994 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f386805c780 con 0x7f3878072320 2024-09-10T10:50:52.321 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:52.321 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":92,"btime":"2024-09-10T10:50:48:682113+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:48.682104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7008":{"gid":7008,"name":"a","rank":0,"incarnation":89,"state":"up:active","state_seq":5,"addr":"172.21.15.28:6835/2814091387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2814091387},{"type":"v1","addr":"172.21.15.28:6835","nonce":2814091387}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7008,"qdb_cluster":[7008]},"id":6}]} 2024-09-10T10:50:52.321 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 92 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 msgr2=0x7f385406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 0x7f385406e320 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3860034000 tx=0x7f386002e000 comp rx=0 tx=0).stop 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 msgr2=0x7f38780b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f386800c4e0 tx=0x7f386800c9a0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 shutdown_connections 2024-09-10T10:50:52.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f385406be60 0x7f385406e320 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3878072de0 0x7f387813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878072700 0x7f387815b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 --2- 172.21.15.28:0/1569687994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3878072320 0x7f38780b1720 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 >> 172.21.15.28:0/1569687994 conn(0x7f3878067d70 msgr2=0x7f3878144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 shutdown_connections 2024-09-10T10:50:52.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.321+0000 7f387ce00640 1 -- 172.21.15.28:0/1569687994 wait complete. 2024-09-10T10:50:52.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:50:48.682104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7008': {'gid': 7008, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.28:6835/2814091387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2814091387}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2814091387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7008, 'qdb_cluster': [7008]} 2024-09-10T10:50:52.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:50:52.340 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:50:52.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.495+0000 7fac90a00640 1 Processor -- start 2024-09-10T10:50:52.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.495+0000 7fac90a00640 1 -- start start 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac90a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fac8c15b300 0x7fac8c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac90a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac90a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8c0cdbe0 con 0x7fac8c0cd3f0 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac90a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fac8c0728a0 con 0x7fac8c15b300 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac90a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c0cd900 con 0x7fac8c06f8d0 2024-09-10T10:50:52.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.496+0000 7fac89e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fac8c15b300 0x7fac8c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fac8c15b300 0x7fac8c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34938/0 (socket says 172.21.15.28:34938) 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 -- 172.21.15.28:0/525094354 learned_addr learned my addr 172.21.15.28:0/525094354 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac8a800640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 -- 172.21.15.28:0/525094354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 msgr2=0x7fac8c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 -- 172.21.15.28:0/525094354 >> v1:172.21.15.138:6789/0 conn(0x7fac8c0cd3f0 legacy=0x7fac8c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac89e00640 1 -- 172.21.15.28:0/525094354 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c072250 con 0x7fac8c15b300 2024-09-10T10:50:52.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.497+0000 7fac8a800640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.498+0000 7fac89e00640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c0cceb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009870 tx=0x7fac7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c2326109eb8131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.498+0000 7fac88a00640 1 -- 172.21.15.28:0/525094354 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c037070 con 0x7fac8c15b300 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.498+0000 7fac88a00640 1 -- 172.21.15.28:0/525094354 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c032440 con 0x7fac8c15b300 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.498+0000 7fac88a00640 1 -- 172.21.15.28:0/525094354 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c032810 con 0x7fac8c15b300 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 -- 172.21.15.28:0/525094354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 msgr2=0x7fac8c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c0cceb0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009870 tx=0x7fac7c02a270 comp rx=0 tx=0).stop 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 -- 172.21.15.28:0/525094354 shutdown_connections 2024-09-10T10:50:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 --2- 172.21.15.28:0/525094354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c0cceb0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 -- 172.21.15.28:0/525094354 >> 172.21.15.28:0/525094354 conn(0x7fac8c067d70 msgr2=0x7fac8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 -- 172.21.15.28:0/525094354 shutdown_connections 2024-09-10T10:50:52.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.499+0000 7fac90a00640 1 -- 172.21.15.28:0/525094354 wait complete. 2024-09-10T10:50:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.501+0000 7fac90a00640 1 Processor -- start 2024-09-10T10:50:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.501+0000 7fac90a00640 1 -- start start 2024-09-10T10:50:52.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8c0cd3f0 0x7fac8c1460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0d0160 con 0x7fac8c15b300 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0cffe0 con 0x7fac8c0cd3f0 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac90a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c072d30 con 0x7fac8c06f8d0 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34942/0 (socket says 172.21.15.28:34942) 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 -- 172.21.15.28:0/4052619516 learned_addr learned my addr 172.21.15.28:0/4052619516 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac89e00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac89400640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8c0cd3f0 0x7fac8c1460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 -- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 msgr2=0x7fac8c15e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 -- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8c0cd3f0 msgr2=0x7fac8c1460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8c0cd3f0 0x7fac8c1460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.502+0000 7fac8a800640 1 -- 172.21.15.28:0/4052619516 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c15d050 con 0x7fac8c15b300 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac8a800640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fac84007e70 tx=0x7fac8400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac82000640 1 -- 172.21.15.28:0/4052619516 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8401a070 con 0x7fac8c15b300 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac82000640 1 -- 172.21.15.28:0/4052619516 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84015600 con 0x7fac8c15b300 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac82000640 1 -- 172.21.15.28:0/4052619516 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac840159f0 con 0x7fac8c15b300 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c15d330 con 0x7fac8c15b300 2024-09-10T10:50:52.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.503+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c15d8f0 con 0x7fac8c15b300 2024-09-10T10:50:52.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.505+0000 7fac82000640 1 -- 172.21.15.28:0/4052619516 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac8401d070 con 0x7fac8c15b300 2024-09-10T10:50:52.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.506+0000 7fac82000640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 0x7fac6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.506+0000 7fac82000640 1 -- 172.21.15.28:0/4052619516 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fac8408b3d0 con 0x7fac8c15b300 2024-09-10T10:50:52.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.506+0000 7fac9387a740 1 Processor -- start 2024-09-10T10:50:52.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.506+0000 7fac89e00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 0x7fac6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 -- start start 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac89e00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 0x7fac6c06e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac7c034000 tx=0x7fac7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582ef9ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5582ef9083b0 con 0x5582ef9a74b0 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582ef90d410 con 0x5582ef9a9cf0 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.507+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5582ef8b5620 con 0x5582ef9ab480 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582ef9ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34954/0 (socket says 172.21.15.28:34954) 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 -- 172.21.15.28:0/4241588484 learned_addr learned my addr 172.21.15.28:0/4241588484 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac8a800640 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 -- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 msgr2=0x5582ef9a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 -- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 msgr2=0x5582ef9ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582ef9ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 -- 172.21.15.28:0/4241588484 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ef6202d0 con 0x5582ef9a74b0 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.508+0000 7fac89400640 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fac780015a0 tx=0x7fac78001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09f1ad878e6149 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.509+0000 7fac76600640 1 -- 172.21.15.28:0/4241588484 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78019070 con 0x5582ef9a74b0 2024-09-10T10:50:52.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.509+0000 7fac76600640 1 -- 172.21.15.28:0/4241588484 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac780050d0 con 0x5582ef9a74b0 2024-09-10T10:50:52.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.509+0000 7fac76600640 1 -- 172.21.15.28:0/4241588484 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac78007030 con 0x5582ef9a74b0 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.509+0000 7fac9387a740 1 -- 172.21.15.28:0/4241588484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 msgr2=0x5582ef9aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.509+0000 7fac9387a740 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fac780015a0 tx=0x7fac78001a60 comp rx=0 tx=0).stop 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 -- 172.21.15.28:0/4241588484 shutdown_connections 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a74b0 0x5582ef9aa0d0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 --2- 172.21.15.28:0/4241588484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582ef9ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 -- 172.21.15.28:0/4241588484 >> 172.21.15.28:0/4241588484 conn(0x5582ef9a6740 msgr2=0x5582ef9ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 -- 172.21.15.28:0/4241588484 shutdown_connections 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 -- 172.21.15.28:0/4241588484 wait complete. 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.510+0000 7fac9387a740 1 -- start start 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.511+0000 7fac9387a740 1 Processor -- start 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 0x5582ef9a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 0x5582ef9a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5582ef8c8780 con 0x5582ef9a9cf0 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582ef9093d0 con 0x5582ef9ab480 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5582ef9a5be0 con 0x5582ef9c8470 2024-09-10T10:50:52.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac9387a740 10 client.0 _ll_register_callbacks cb 0x5582ef9065d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34956/0 (socket says 172.21.15.28:34956) 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 -- 172.21.15.28:0/9231338 learned_addr learned my addr 172.21.15.28:0/9231338 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89400640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 0x5582ef9a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac8a800640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 0x5582ef9a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 -- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 msgr2=0x5582ef9a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 0x5582ef9a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 -- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 msgr2=0x5582ef9a3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 0x5582ef9a3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89e00640 1 -- 172.21.15.28:0/9231338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ef6ea6b0 con 0x5582ef9a9cf0 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac89400640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 0x5582ef9a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:52.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.512+0000 7fac8a800640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 0x5582ef9a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac89e00640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fac7c035040 tx=0x7fac7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf79c5d6a97306bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c04e070 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fac7c03c850 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac780050d0 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fac7c037040 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac9387a740 10 client.7500 fetch_fsmap learned FSMap version 92 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5582ef9ca640 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac780050d0 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.513+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7fac7c046040 con 0x5582ef9a9cf0 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 10 client.7500 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T10:50:52.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 10 client.7500 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 20 client.7500 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582efa027b0 0x5582efa02b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 4 client.7500 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 --> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ef91ab40 con 0x5582efa027b0 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac89400640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582efa027b0 0x5582efa02b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac89400640 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582efa027b0 0x5582efa02b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faebd606b6b5468b server_cookie=20de71f3549efb8 in_seq=0 out_seq=0 2024-09-10T10:50:52.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.514+0000 7fac6aa00640 10 client.7500 ms_handle_connect on v2:172.21.15.28:6833/2814091387 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.515+0000 7fac6aa00640 1 -- 172.21.15.28:0/9231338 <== mds.0 v2:172.21.15.28:6833/2814091387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582ef91ab40 con 0x5582efa027b0 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.515+0000 7fac6aa00640 10 client.7500 handle_command_reply: tid=0 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.515+0000 7fac9387a740 1 client.7500 shutdown 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 msgr2=0x5582ef9cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fac7c035040 tx=0x7fac7c004730 comp rx=0 tx=0).stop 2024-09-10T10:50:52.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 shutdown_connections 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582efa027b0 0x5582efa02b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9c8470 0x5582ef9a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9ab480 0x5582ef9a3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac6aa00640 0 client.7500 ms_handle_reset on v2:172.21.15.28:6833/2814091387 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 --2- 172.21.15.28:0/9231338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582ef9cae40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 >> 172.21.15.28:0/9231338 conn(0x5582ef9a74b0 msgr2=0x5582ef9edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 shutdown_connections 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 1 -- 172.21.15.28:0/9231338 wait complete. 2024-09-10T10:50:52.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.516+0000 7fac9387a740 20 client.7500 trim_cache size 0 max 16384 2024-09-10T10:50:52.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.535+0000 7fac9387a740 1 Processor -- start 2024-09-10T10:50:52.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.535+0000 7fac9387a740 1 -- start start 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9c8470 0x5582ef9ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c201130 con 0x5582ef9a9cf0 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c2001b0 con 0x5582ef9c8470 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c1e8080 con 0x5582ef9ab480 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac89e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac89e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34960/0 (socket says 172.21.15.28:34960) 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac89e00640 1 -- 172.21.15.28:0/2397033290 learned_addr learned my addr 172.21.15.28:0/2397033290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac8a800640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9c8470 0x5582ef9ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.536+0000 7fac89400640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 -- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 msgr2=0x5582ef9fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 -- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9c8470 msgr2=0x5582ef9ec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9c8470 0x5582ef9ec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 -- 172.21.15.28:0/2397033290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ef9d0e20 con 0x5582ef9a9cf0 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89400640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac89e00640 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032c00 tx=0x7fac7c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93aa7203e79f642b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac75c00640 1 -- 172.21.15.28:0/2397033290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c04e040 con 0x5582ef9a9cf0 2024-09-10T10:50:52.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac75c00640 1 -- 172.21.15.28:0/2397033290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c03ce40 con 0x5582ef9a9cf0 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.537+0000 7fac75c00640 1 -- 172.21.15.28:0/2397033290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c03c870 con 0x5582ef9a9cf0 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- 172.21.15.28:0/2397033290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 msgr2=0x5582efa08d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fac7c032c00 tx=0x7fac7c047000 comp rx=0 tx=0).stop 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- 172.21.15.28:0/2397033290 shutdown_connections 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9c8470 0x5582ef9ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582ef9fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 --2- 172.21.15.28:0/2397033290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9a9cf0 0x5582efa08d70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- 172.21.15.28:0/2397033290 >> 172.21.15.28:0/2397033290 conn(0x5582ef9a74b0 msgr2=0x5582ef9d1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- 172.21.15.28:0/2397033290 shutdown_connections 2024-09-10T10:50:52.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- 172.21.15.28:0/2397033290 wait complete. 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.538+0000 7fac9387a740 1 -- start start 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.539+0000 7fac9387a740 1 Processor -- start 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582efa712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582efa6cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5582ef9fee50 con 0x5582ef9c8470 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c1e86d0 con 0x5582ef9a9cf0 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5582ef9a85e0 con 0x5582ef9ab480 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac9387a740 10 client.0 _ll_register_callbacks cb 0x7fac7c0097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac89e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582efa712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac89400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582efa6cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34972/0 (socket says 172.21.15.28:34972) 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 -- 172.21.15.28:0/4013729804 learned_addr learned my addr 172.21.15.28:0/4013729804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 -- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 msgr2=0x5582efa6cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582efa6cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 -- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 msgr2=0x5582efa712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582efa712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.540+0000 7fac8a800640 1 -- 172.21.15.28:0/4013729804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582efa11b70 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac8a800640 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fac8400d9a0 tx=0x7fac84061fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab8ce562b0400b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84076060 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fac840598a0 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5582ef914e90 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fac84071030 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 10 client.7506 fetch_fsmap learned FSMap version 92 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5582efa11600 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5582ef914e90 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7fac840891f0 con 0x5582ef9c8470 2024-09-10T10:50:52.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 10 client.7506 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 10 client.7506 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 20 client.7506 populate_metadata read hostname 'smithi028' 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582ef9c8000 0x5582ef9fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 4 client.7506 mds_command: new command op to 7008 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.541+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 --> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] -- command(tid 0: {"prefix": "status"}) -- 0x5582ef891a40 con 0x5582ef9c8000 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac89e00640 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582ef9c8000 0x5582ef9fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac89e00640 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582ef9c8000 0x5582ef9fd020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b13a60281a2fa4d server_cookie=1e23f3b09c1864bc in_seq=0 out_seq=0 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac6aa00640 10 client.7506 ms_handle_connect on v2:172.21.15.28:6833/2814091387 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac6aa00640 1 -- 172.21.15.28:0/4013729804 <== mds.0 v2:172.21.15.28:6833/2814091387 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5582ef891a40 con 0x5582ef9c8000 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac6aa00640 10 client.7506 handle_command_reply: tid=0 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.542+0000 7fac9387a740 1 client.7506 shutdown 2024-09-10T10:50:52.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 msgr2=0x5582ef9f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fac8400d9a0 tx=0x7fac84061fe0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 shutdown_connections 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:6833/2814091387,v1:172.21.15.28:6835/2814091387] conn(0x5582ef9c8000 0x5582ef9fd020 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5582ef9c8470 0x5582ef9f1df0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5582ef9ab480 0x5582efa6cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac6aa00640 0 client.7506 ms_handle_reset on v2:172.21.15.28:6833/2814091387 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 --2- 172.21.15.28:0/4013729804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582ef9a9cf0 0x5582efa712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 >> 172.21.15.28:0/4013729804 conn(0x5582ef9a74b0 msgr2=0x5582ef9ec9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 shutdown_connections 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 1 -- 172.21.15.28:0/4013729804 wait complete. 2024-09-10T10:50:52.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac9387a740 20 client.7506 trim_cache size 0 max 16384 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 7008, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8737532789999998, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 92, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 100, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 100, 2024-09-10T10:50:52.551 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 14.574878429 2024-09-10T10:50:52.552 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:50:52.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 msgr2=0x7fac6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.543+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 0x7fac6c06e2f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac7c034000 tx=0x7fac7c02e000 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 msgr2=0x7fac8c15c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fac84007e70 tx=0x7fac8400d4a0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 shutdown_connections 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac6c06be30 0x7fac6c06e2f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c15b300 0x7fac8c15c6e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8c0cd3f0 0x7fac8c1460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 --2- 172.21.15.28:0/4052619516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac8c06f8d0 0x7fac8c15e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 >> 172.21.15.28:0/4052619516 conn(0x7fac8c067d70 msgr2=0x7fac8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 shutdown_connections 2024-09-10T10:50:52.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.544+0000 7fac90a00640 1 -- 172.21.15.28:0/4052619516 wait complete. 2024-09-10T10:50:52.561 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:50:52.561 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 10000000000.00000000', proceeding to ls 2024-09-10T10:50:52.561 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:50:52.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.718+0000 7fe372e00640 1 Processor -- start 2024-09-10T10:50:52.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.719+0000 7fe372e00640 1 -- start start 2024-09-10T10:50:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe372e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe36c072320 0x7fe36c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe372e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe36c072de0 0x7fe36c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe372e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe36c146140 con 0x7fe36c072700 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe372e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe36c0cd740 con 0x7fe36c072320 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe372e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe36c0cda20 con 0x7fe36c072de0 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe36c072320 0x7fe36c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe36c072320 0x7fe36c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34980/0 (socket says 172.21.15.28:34980) 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 -- 172.21.15.28:0/1849822694 learned_addr learned my addr 172.21.15.28:0/1849822694 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe363e00640 1 --2- 172.21.15.28:0/1849822694 >> v2:172.21.15.138:3300/0 conn(0x7fe36c072de0 0x7fe36c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 -- 172.21.15.28:0/1849822694 >> v2:172.21.15.138:3300/0 conn(0x7fe36c072de0 msgr2=0x7fe36c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 --2- 172.21.15.28:0/1849822694 >> v2:172.21.15.138:3300/0 conn(0x7fe36c072de0 0x7fe36c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 -- 172.21.15.28:0/1849822694 >> v1:172.21.15.138:6789/0 conn(0x7fe36c072700 legacy=0x7fe36c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.720+0000 7fe36be00640 1 -- 172.21.15.28:0/1849822694 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0cdd00 con 0x7fe36c072320 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.721+0000 7fe36be00640 1 --2- 172.21.15.28:0/1849822694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0cce60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe36c06f8c0 tx=0x7fe35c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53e74ed39e5ca32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe36b400640 1 -- 172.21.15.28:0/1849822694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c037070 con 0x7fe36c072320 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe36b400640 1 -- 172.21.15.28:0/1849822694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c032440 con 0x7fe36c072320 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe36b400640 1 -- 172.21.15.28:0/1849822694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c032810 con 0x7fe36c072320 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 -- 172.21.15.28:0/1849822694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 msgr2=0x7fe36c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 --2- 172.21.15.28:0/1849822694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0cce60 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe36c06f8c0 tx=0x7fe35c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 -- 172.21.15.28:0/1849822694 shutdown_connections 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 --2- 172.21.15.28:0/1849822694 >> v2:172.21.15.138:3300/0 conn(0x7fe36c072de0 0x7fe36c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 --2- 172.21.15.28:0/1849822694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0cce60 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 -- 172.21.15.28:0/1849822694 >> 172.21.15.28:0/1849822694 conn(0x7fe36c067d70 msgr2=0x7fe36c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 -- 172.21.15.28:0/1849822694 shutdown_connections 2024-09-10T10:50:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.722+0000 7fe372e00640 1 -- 172.21.15.28:0/1849822694 wait complete. 2024-09-10T10:50:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.724+0000 7fe372e00640 1 Processor -- start 2024-09-10T10:50:52.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.724+0000 7fe372e00640 1 -- start start 2024-09-10T10:50:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 0x7fe36c141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 0x7fe36c142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0cd520 con 0x7fe36c072320 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0cd3a0 con 0x7fe36c072700 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe372e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c06ed00 con 0x7fe36c072de0 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe36be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe36be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34982/0 (socket says 172.21.15.28:34982) 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe36be00640 1 -- 172.21.15.28:0/937888562 learned_addr learned my addr 172.21.15.28:0/937888562 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe363e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 0x7fe36c141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.725+0000 7fe370e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 0x7fe36c142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 -- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 msgr2=0x7fe36c142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 0x7fe36c142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 -- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 msgr2=0x7fe36c141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 0x7fe36c141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 -- 172.21.15.28:0/937888562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0b3eb0 con 0x7fe36c072320 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe370e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 0x7fe36c142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe363e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 0x7fe36c141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe36be00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe35c02a220 tx=0x7fe35c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c041020 con 0x7fe36c072320 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe35c0306c0 con 0x7fe36c072320 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c037040 con 0x7fe36c072320 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c0b4190 con 0x7fe36c072320 2024-09-10T10:50:52.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.726+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c0b4750 con 0x7fe36c072320 2024-09-10T10:50:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.728+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c0cc4b0 con 0x7fe36c072320 2024-09-10T10:50:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.728+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe35c044020 con 0x7fe36c072320 2024-09-10T10:50:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.729+0000 7fe368c00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 0x7fe34806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.729+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fe35c0ab830 con 0x7fe36c072320 2024-09-10T10:50:52.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.730+0000 7fe363e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 0x7fe34806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:52.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.730+0000 7fe363e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 0x7fe34806e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe354006fd0 tx=0x7fe354008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:52.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.733+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe36c0cc4b0 con 0x7fe36c072320 2024-09-10T10:50:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.953+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe36c0b4f50 con 0x7fe36c072320 2024-09-10T10:50:52.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.953+0000 7fe368c00640 1 -- 172.21.15.28:0/937888562 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v92) ==== 53+0+73 (secure 0 0 0) 0x7fe35c04b030 con 0x7fe36c072320 2024-09-10T10:50:52.955 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:50:52.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 msgr2=0x7fe34806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 0x7fe34806e2f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe354006fd0 tx=0x7fe354008040 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 msgr2=0x7fe36c0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fe35c02a220 tx=0x7fe35c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 shutdown_connections 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe34806be30 0x7fe34806e2f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe36c072de0 0x7fe36c142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c072700 0x7fe36c141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 --2- 172.21.15.28:0/937888562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe36c072320 0x7fe36c0b35d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 >> 172.21.15.28:0/937888562 conn(0x7fe36c067d70 msgr2=0x7fe36c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 shutdown_connections 2024-09-10T10:50:52.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:52.955+0000 7fe372e00640 1 -- 172.21.15.28:0/937888562 wait complete. 2024-09-10T10:50:52.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:50:52.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:50:52.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:50:52.975 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:52.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:52.975 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:50:53.010 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:50:53.011 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42252sec preferred_lft 42252sec 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.012 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:50:53.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.013 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:50:53.105 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:50:53.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.108 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:50:53.144 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:50:53.144 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:50:53.145 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:50:53.145 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:50:53.145 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:50:53.145 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:50:53.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:50:53.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:53.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.343 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:50:53.378 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:50:53.378 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:50:53.378 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:53.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.379 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:50:53.475 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:50:53.475 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:53.510 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:50:53.510 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:50:53.510 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:53.583 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:50:53.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:53.712 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:53.712 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:53.712 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:53.727 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:53.727 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:53.727 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:53.728 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:50:53.728 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:50:53.749 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:50:53.750 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:50:53.750 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:50:53.751 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:50:53.751 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:50:53.751 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:50:53.751 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:50:53.751 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:50:53.754 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:50:53.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.754 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:50:53.809 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:50:53.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:50:53.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:50:53.899 INFO:teuthology.orchestra.run.smithi028.stdout:client7515 2024-09-10T10:50:53.899 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:53.899 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:53.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:53.935 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:50:53.980 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:50:54.012 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:50:54.010+0000 7f0da6600640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-10T10:50:54.013 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:50:54.013 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:50:54.013 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:54.013 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:50:54.014 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 10000000000.00000000' did not caused DAMAGED state 2024-09-10T10:50:54.014 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 600.00000000' 2024-09-10T10:50:54.014 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:54.014 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:54.090 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:50:54.090 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:50:54.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:50:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.361+0000 7f3648000640 1 Processor -- start 2024-09-10T10:50:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.361+0000 7f3648000640 1 -- start start 2024-09-10T10:50:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3648000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f36400d30f0 0x7f36400b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3648000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36400d3690 con 0x7f364015e5f0 2024-09-10T10:50:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3648000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36400d3970 con 0x7f36400b64c0 2024-09-10T10:50:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3648000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f364006d530 con 0x7f36400d30f0 2024-09-10T10:50:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3645600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f364015e5f0 0x7f36400d4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53870/0 (socket says 172.21.15.28:53870) 2024-09-10T10:50:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.362+0000 7f3645600640 1 -- 172.21.15.28:0/787731935 learned_addr learned my addr 172.21.15.28:0/787731935 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f3644c00640 1 --2- 172.21.15.28:0/787731935 >> v2:172.21.15.138:3300/0 conn(0x7f36400d30f0 0x7f36400b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1115887500 0 0) 0x7f36400d3690 con 0x7f364015e5f0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3628003520 con 0x7f364015e5f0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017267042 0 0) 0x7f36400d3970 con 0x7f36400b64c0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36400d3690 con 0x7f36400b64c0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f3644c00640 1 -- 172.21.15.28:0/787731935 >> v1:172.21.15.138:6789/0 conn(0x7f36400b64c0 legacy=0x7f364006d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1279140074 0 0) 0x7f3628003520 con 0x7f364015e5f0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f3644c00640 1 -- 172.21.15.28:0/787731935 >> v1:172.21.15.28:6789/0 conn(0x7f364015e5f0 legacy=0x7f36400d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.363+0000 7f3644c00640 1 -- 172.21.15.28:0/787731935 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364006cee0 con 0x7f36400d30f0 2024-09-10T10:50:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.365+0000 7f3644c00640 1 --2- 172.21.15.28:0/787731935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b5f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc97a91e9f8e5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.365+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c037070 con 0x7f36400d30f0 2024-09-10T10:50:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.365+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c02ac50 con 0x7f36400d30f0 2024-09-10T10:50:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.365+0000 7f363fe00640 1 -- 172.21.15.28:0/787731935 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634001670 con 0x7f36400d30f0 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 -- 172.21.15.28:0/787731935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 msgr2=0x7f36400b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 --2- 172.21.15.28:0/787731935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b5f80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f362c0097a0 tx=0x7f362c02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 -- 172.21.15.28:0/787731935 shutdown_connections 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 --2- 172.21.15.28:0/787731935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b5f80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 -- 172.21.15.28:0/787731935 >> 172.21.15.28:0/787731935 conn(0x7f364008bfc0 msgr2=0x7f36400d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 -- 172.21.15.28:0/787731935 shutdown_connections 2024-09-10T10:50:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.366+0000 7f3648000640 1 -- 172.21.15.28:0/787731935 wait complete. 2024-09-10T10:50:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.367+0000 7f3648000640 1 Processor -- start 2024-09-10T10:50:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.368+0000 7f3648000640 1 -- start start 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.368+0000 7f3648000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36400b64c0 0x7f36401508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.368+0000 7f3648000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.368+0000 7f3648000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3648000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f364006e0d0 con 0x7f364015e5f0 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3648000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f364015e9d0 con 0x7f36400d30f0 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3648000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f364006e250 con 0x7f36400b64c0 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36130/0 (socket says 172.21.15.28:36130) 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 -- 172.21.15.28:0/4279803382 learned_addr learned my addr 172.21.15.28:0/4279803382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:54.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3644c00640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3645600640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36400b64c0 0x7f36401508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 -- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36400b64c0 msgr2=0x7f36401508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36400b64c0 0x7f36401508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 -- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 msgr2=0x7f36400b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.369+0000 7f3646000640 1 -- 172.21.15.28:0/4279803382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36400cad40 con 0x7f364015e5f0 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f3646000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f363800c7c0 tx=0x7f363800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638003b70 con 0x7f364015e5f0 2024-09-10T10:50:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3638003d10 con 0x7f364015e5f0 2024-09-10T10:50:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36400b7890 con 0x7f364015e5f0 2024-09-10T10:50:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800aa50 con 0x7f364015e5f0 2024-09-10T10:50:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.370+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36400ba240 con 0x7f364015e5f0 2024-09-10T10:50:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.371+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36400b5f80 con 0x7f364015e5f0 2024-09-10T10:50:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.372+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f363800abf0 con 0x7f364015e5f0 2024-09-10T10:50:54.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.373+0000 7f363d600640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 0x7f362006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.373+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7f363808aed0 con 0x7f364015e5f0 2024-09-10T10:50:54.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.374+0000 7f3645600640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 0x7f362006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.374+0000 7f3645600640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 0x7f362006e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f36340017e0 tx=0x7f3634001610 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.378+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f36400cad40 con 0x7f364015e5f0 2024-09-10T10:50:54.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.572+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f36400b2f00 con 0x7f364015e5f0 2024-09-10T10:50:54.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.573+0000 7f363d600640 1 -- 172.21.15.28:0/4279803382 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v100) ==== 80+0+122 (secure 0 0 0) 0x7f3638058fb0 con 0x7f364015e5f0 2024-09-10T10:50:54.575 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:50:54.575 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:50:54.576 INFO:teuthology.orchestra.run.smithi028.stderr:listed 2 entries 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 msgr2=0x7f362006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 0x7f362006e320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f36340017e0 tx=0x7f3634001610 comp rx=0 tx=0).stop 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 msgr2=0x7f36401e8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f363800c7c0 tx=0x7f363800cc80 comp rx=0 tx=0).stop 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 shutdown_connections 2024-09-10T10:50:54.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f362006be60 0x7f362006e320 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f364015e5f0 0x7f36401e8e10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36400d30f0 0x7f36400b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 --2- 172.21.15.28:0/4279803382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36400b64c0 0x7f36401508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.575+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 >> 172.21.15.28:0/4279803382 conn(0x7f364008bfc0 msgr2=0x7f36400b5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.576+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 shutdown_connections 2024-09-10T10:50:54.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.576+0000 7f3648000640 1 -- 172.21.15.28:0/4279803382 wait complete. 2024-09-10T10:50:54.594 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:50:54.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:54.594 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:50:54.663 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:50:54.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:50:54.664 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:50:54.699 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:50:54.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.894+0000 7fd879800640 1 Processor -- start 2024-09-10T10:50:54.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.895+0000 7fd879800640 1 -- start start 2024-09-10T10:50:54.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd879800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd8740cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd879800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd87415e1f0 0x7fd87406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd879800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd874146140 con 0x7fd874072de0 2024-09-10T10:50:54.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd879800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd8740cd740 con 0x7fd87415e1f0 2024-09-10T10:50:54.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd879800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8740cda20 con 0x7fd874072320 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd872a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd8740cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd86be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd874072de0 0x7fd87406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:45256/0 (socket says 172.21.15.28:45256) 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd86be00640 1 -- 172.21.15.28:0/3270824277 learned_addr learned my addr 172.21.15.28:0/3270824277 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.896+0000 7fd873400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd87415e1f0 0x7fd87406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.897+0000 7fd872a00640 1 -- 172.21.15.28:0/3270824277 >> v2:172.21.15.138:3300/0 conn(0x7fd87415e1f0 msgr2=0x7fd87406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.897+0000 7fd872000640 1 -- 172.21.15.28:0/3270824277 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1705341807 0 0) 0x7fd874146140 con 0x7fd874072de0 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.897+0000 7fd872a00640 1 --2- 172.21.15.28:0/3270824277 >> v2:172.21.15.138:3300/0 conn(0x7fd87415e1f0 0x7fd87406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.897+0000 7fd872a00640 1 -- 172.21.15.28:0/3270824277 >> v1:172.21.15.138:6789/0 conn(0x7fd874072de0 legacy=0x7fd87406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.897+0000 7fd872a00640 1 -- 172.21.15.28:0/3270824277 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8740cdd00 con 0x7fd874072320 2024-09-10T10:50:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.898+0000 7fd872a00640 1 --2- 172.21.15.28:0/3270824277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd8740cce60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601891f759cd4136 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.898+0000 7fd872000640 1 -- 172.21.15.28:0/3270824277 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864037070 con 0x7fd874072320 2024-09-10T10:50:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.898+0000 7fd872000640 1 -- 172.21.15.28:0/3270824277 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86402ae50 con 0x7fd874072320 2024-09-10T10:50:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.898+0000 7fd872000640 1 -- 172.21.15.28:0/3270824277 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864032660 con 0x7fd874072320 2024-09-10T10:50:54.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 -- 172.21.15.28:0/3270824277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 msgr2=0x7fd8740cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 --2- 172.21.15.28:0/3270824277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd8740cce60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd8640097a0 tx=0x7fd86402a100 comp rx=0 tx=0).stop 2024-09-10T10:50:54.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 -- 172.21.15.28:0/3270824277 shutdown_connections 2024-09-10T10:50:54.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 --2- 172.21.15.28:0/3270824277 >> v2:172.21.15.138:3300/0 conn(0x7fd87415e1f0 0x7fd87406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 --2- 172.21.15.28:0/3270824277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd8740cce60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 -- 172.21.15.28:0/3270824277 >> 172.21.15.28:0/3270824277 conn(0x7fd874067d70 msgr2=0x7fd87406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:54.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 -- 172.21.15.28:0/3270824277 shutdown_connections 2024-09-10T10:50:54.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.899+0000 7fd879800640 1 -- 172.21.15.28:0/3270824277 wait complete. 2024-09-10T10:50:54.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.901+0000 7fd879800640 1 Processor -- start 2024-09-10T10:50:54.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.901+0000 7fd879800640 1 -- start start 2024-09-10T10:50:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd874141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd87415e1f0 0x7fd8741e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740cd520 con 0x7fd874072de0 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8740cd3a0 con 0x7fd87415e1f0 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd879800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd87406ed00 con 0x7fd874072320 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36140/0 (socket says 172.21.15.28:36140) 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 -- 172.21.15.28:0/2383424225 learned_addr learned my addr 172.21.15.28:0/2383424225 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd872a00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd874141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd873400640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd87415e1f0 0x7fd8741e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 -- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 msgr2=0x7fd874141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd874141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 -- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd87415e1f0 msgr2=0x7fd8741e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd87415e1f0 0x7fd8741e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.902+0000 7fd86be00640 1 -- 172.21.15.28:0/2383424225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741fba10 con 0x7fd874072de0 2024-09-10T10:50:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd86be00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c4b0 tx=0x7fd85c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c018070 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd85c007500 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c004a20 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741fbcf0 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.903+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741fc2b0 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.904+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd874072700 con 0x7fd874072de0 2024-09-10T10:50:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.905+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd85c01b070 con 0x7fd874072de0 2024-09-10T10:50:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.906+0000 7fd86aa00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 0x7fd85006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.906+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 6187+0+0 (secure 0 0 0) 0x7fd85c08a390 con 0x7fd874072de0 2024-09-10T10:50:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.906+0000 7fd872a00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 0x7fd85006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.907+0000 7fd872a00640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 0x7fd85006e320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd864034000 tx=0x7fd86402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:54.910+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd85c058530 con 0x7fd874072de0 2024-09-10T10:50:55.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.133+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd874144540 con 0x7fd874072de0 2024-09-10T10:50:55.136 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:50:55.134+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:50:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.136+0000 7fd86aa00640 1 -- 172.21.15.28:0/2383424225 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7fd85c05c830 con 0x7fd874072de0 2024-09-10T10:50:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 msgr2=0x7fd85006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 0x7fd85006e320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd864034000 tx=0x7fd86402e000 comp rx=0 tx=0).stop 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 msgr2=0x7fd8741e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c4b0 tx=0x7fd85c00c970 comp rx=0 tx=0).stop 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 shutdown_connections 2024-09-10T10:50:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85006be60 0x7fd85006e320 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd87415e1f0 0x7fd8741e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd874072de0 0x7fd8741e8570 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 --2- 172.21.15.28:0/2383424225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd874072320 0x7fd874141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 >> 172.21.15.28:0/2383424225 conn(0x7fd874067d70 msgr2=0x7fd874142a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 shutdown_connections 2024-09-10T10:50:55.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.139+0000 7fd879800640 1 -- 172.21.15.28:0/2383424225 wait complete. 2024-09-10T10:50:55.146 INFO:tasks.ceph.mds.a.smithi028.stderr: -665> 2024-09-10T10:50:48.917+0000 7f0dab600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:50:55.147 INFO:tasks.ceph.mds.a.smithi028.stderr: -124> 2024-09-10T10:50:54.010+0000 7f0da6600640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/subdir) 2024-09-10T10:50:55.159 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:50:55.190 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:50:55.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.317+0000 7fae75800640 1 Processor -- start 2024-09-10T10:50:55.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.317+0000 7fae75800640 1 -- start start 2024-09-10T10:50:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.318+0000 7fae75800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fae7006dbd0 0x7fae7009b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.318+0000 7fae75800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae700d1970 con 0x7fae7015e740 2024-09-10T10:50:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.318+0000 7fae75800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae700d0780 con 0x7fae700d1540 2024-09-10T10:50:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.318+0000 7fae75800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fae7015eb20 con 0x7fae7006dbd0 2024-09-10T10:50:55.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fae7006dbd0 0x7fae7009b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:55.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fae7015e740 0x7fae7006d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53872/0 (socket says 172.21.15.28:53872) 2024-09-10T10:50:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6ea00640 1 -- 172.21.15.28:0/2865572224 learned_addr learned my addr 172.21.15.28:0/2865572224 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6e000640 1 -- 172.21.15.28:0/2865572224 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1984819430 0 0) 0x7fae700d1970 con 0x7fae7015e740 2024-09-10T10:50:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6f400640 1 -- 172.21.15.28:0/2865572224 >> v1:172.21.15.138:6789/0 conn(0x7fae700d1540 legacy=0x7fae70070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6f400640 1 -- 172.21.15.28:0/2865572224 >> v1:172.21.15.28:6789/0 conn(0x7fae7015e740 legacy=0x7fae7006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.319+0000 7fae6f400640 1 -- 172.21.15.28:0/2865572224 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae700d0a60 con 0x7fae7006dbd0 2024-09-10T10:50:55.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.320+0000 7fae6f400640 1 --2- 172.21.15.28:0/2865572224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae7009b690 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fae68009d90 tx=0x7fae6802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a441946b09f6297 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.320+0000 7fae6e000640 1 -- 172.21.15.28:0/2865572224 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68037070 con 0x7fae7006dbd0 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.320+0000 7fae6e000640 1 -- 172.21.15.28:0/2865572224 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae68007740 con 0x7fae7006dbd0 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.320+0000 7fae6e000640 1 -- 172.21.15.28:0/2865572224 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae68007b10 con 0x7fae7006dbd0 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 -- 172.21.15.28:0/2865572224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 msgr2=0x7fae7009b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 --2- 172.21.15.28:0/2865572224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae7009b690 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fae68009d90 tx=0x7fae6802a780 comp rx=0 tx=0).stop 2024-09-10T10:50:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 -- 172.21.15.28:0/2865572224 shutdown_connections 2024-09-10T10:50:55.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 --2- 172.21.15.28:0/2865572224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae7009b690 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 -- 172.21.15.28:0/2865572224 >> 172.21.15.28:0/2865572224 conn(0x7fae700d23a0 msgr2=0x7fae700d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:55.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 -- 172.21.15.28:0/2865572224 shutdown_connections 2024-09-10T10:50:55.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.321+0000 7fae75800640 1 -- 172.21.15.28:0/2865572224 wait complete. 2024-09-10T10:50:55.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.323+0000 7fae75800640 1 Processor -- start 2024-09-10T10:50:55.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.323+0000 7fae75800640 1 -- start start 2024-09-10T10:50:55.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.323+0000 7fae75800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae75800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae700d1540 0x7fae700c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae75800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 0x7fae7015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae75800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fae7006dfe0 con 0x7fae7006dbd0 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae75800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae700d1020 con 0x7fae7015e740 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae75800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fae7006cd20 con 0x7fae700d1540 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36184/0 (socket says 172.21.15.28:36184) 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 -- 172.21.15.28:0/2271722845 learned_addr learned my addr 172.21.15.28:0/2271722845 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6f400640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 0x7fae7015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae66000640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae700d1540 0x7fae700c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 -- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae700d1540 msgr2=0x7fae700c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae700d1540 0x7fae700c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 -- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 msgr2=0x7fae7015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 0x7fae7015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 -- 172.21.15.28:0/2271722845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae701f45b0 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6f400640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 0x7fae7015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.324+0000 7fae6ea00640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c3f0 tx=0x7fae6000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.325+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae600074e0 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.325+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae600047e0 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.325+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60004c10 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.325+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae701f4870 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.325+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701f4e30 con 0x7fae7006dbd0 2024-09-10T10:50:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.326+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7006d0b0 con 0x7fae7006dbd0 2024-09-10T10:50:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.327+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae60011070 con 0x7fae7006dbd0 2024-09-10T10:50:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.328+0000 7fae67400640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 0x7fae4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.328+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6273+0+0 (secure 0 0 0) 0x7fae6008acf0 con 0x7fae7006dbd0 2024-09-10T10:50:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.328+0000 7fae66000640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 0x7fae4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:55.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.329+0000 7fae66000640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 0x7fae4c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fae701580f0 tx=0x7fae58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:55.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.333+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae6001a2b0 con 0x7fae7006dbd0 2024-09-10T10:50:55.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:55.585+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fae700b8320 con 0x7fae7006dbd0 2024-09-10T10:50:56.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.139+0000 7fae67400640 1 -- 172.21.15.28:0/2271722845 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v94) ==== 107+0+0 (secure 0 0 0) 0x7fae60059450 con 0x7fae7006dbd0 2024-09-10T10:50:56.140 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 msgr2=0x7fae4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 0x7fae4c06e2f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fae701580f0 tx=0x7fae58009040 comp rx=0 tx=0).stop 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 msgr2=0x7fae70156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fae6000c3f0 tx=0x7fae6000c8b0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 shutdown_connections 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae4c06be30 0x7fae4c06e2f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae7015e740 0x7fae7015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae700d1540 0x7fae700c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 --2- 172.21.15.28:0/2271722845 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae7006dbd0 0x7fae70156750 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 >> 172.21.15.28:0/2271722845 conn(0x7fae700d23a0 msgr2=0x7fae700b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 shutdown_connections 2024-09-10T10:50:56.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.142+0000 7fae75800640 1 -- 172.21.15.28:0/2271722845 wait complete. 2024-09-10T10:50:56.161 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:50:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.281+0000 7f9c7351ef40 1 Processor -- start 2024-09-10T10:50:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.281+0000 7f9c7351ef40 1 -- start start 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190da9730 0x55f190dacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190e51250 0x55f190e485b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f190d21b30 con 0x7f9c58003ab0 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f190d25b70 con 0x55f190da9730 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f190e51ee0 con 0x55f190e51250 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36194/0 (socket says 172.21.15.28:36194) 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 -- 172.21.15.28:0/3803750772 learned_addr learned my addr 172.21.15.28:0/3803750772 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:56.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c71200640 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190e51250 0x55f190e485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190da9730 0x55f190dacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 -- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190e51250 msgr2=0x55f190e485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190e51250 0x55f190e485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 -- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190da9730 msgr2=0x55f190dacc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190da9730 0x55f190dacc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.282+0000 7f9c70800640 1 -- 172.21.15.28:0/3803750772 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f190e4c9f0 con 0x7f9c58003ab0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c70800640 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9c640096b0 tx=0x7f9c64009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36732145a74f2b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c6ea00640 1 -- 172.21.15.28:0/3803750772 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400e8e0 con 0x7f9c58003ab0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c6ea00640 1 -- 172.21.15.28:0/3803750772 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c64013030 con 0x7f9c58003ab0 2024-09-10T10:50:56.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c6ea00640 1 -- 172.21.15.28:0/3803750772 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400ece0 con 0x7f9c58003ab0 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 -- 172.21.15.28:0/3803750772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 msgr2=0x55f190daaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9c640096b0 tx=0x7f9c64009b70 comp rx=0 tx=0).stop 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 -- 172.21.15.28:0/3803750772 shutdown_connections 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190e51250 0x55f190e485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190da9730 0x55f190dacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/3803750772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190daaea0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 -- 172.21.15.28:0/3803750772 >> 172.21.15.28:0/3803750772 conn(0x55f190dc80a0 msgr2=0x55f190dc8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 -- 172.21.15.28:0/3803750772 shutdown_connections 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.283+0000 7f9c7351ef40 1 -- 172.21.15.28:0/3803750772 wait complete. 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.284+0000 7f9c7351ef40 1 Processor -- start 2024-09-10T10:50:56.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.284+0000 7f9c7351ef40 1 -- start start 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190da9730 0x55f190dc6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190e51250 0x55f190e3f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f190e50c90 con 0x7f9c58003ab0 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f190dac330 con 0x55f190e51250 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c7351ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f190e4c6a0 con 0x55f190da9730 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.285+0000 7f9c71200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c70800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190da9730 0x55f190dc6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36210/0 (socket says 172.21.15.28:36210) 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 learned_addr learned my addr 172.21.15.28:0/724623521 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190e51250 0x55f190e3f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190da9730 msgr2=0x55f190dc6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f190da9730 0x55f190dc6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190e51250 msgr2=0x55f190e3f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f190e51250 0x55f190e3f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f190dc5bb0 con 0x7f9c58003ab0 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.286+0000 7f9c71200640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9c68025e80 tx=0x7f9c680033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.287+0000 7f9c6d600640 1 -- 172.21.15.28:0/724623521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6801a460 con 0x7f9c58003ab0 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.287+0000 7f9c6d600640 1 -- 172.21.15.28:0/724623521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c68005030 con 0x7f9c58003ab0 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.287+0000 7f9c6d600640 1 -- 172.21.15.28:0/724623521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68005a80 con 0x7f9c58003ab0 2024-09-10T10:50:56.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.287+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f190dc5e90 con 0x7f9c58003ab0 2024-09-10T10:50:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.287+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f190e3fa70 con 0x7f9c58003ab0 2024-09-10T10:50:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.289+0000 7f9c6d600640 1 -- 172.21.15.28:0/724623521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c68005400 con 0x7f9c58003ab0 2024-09-10T10:50:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.289+0000 7f9c6d600640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 0x7f9c5006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.290+0000 7f9c6d600640 1 -- 172.21.15.28:0/724623521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6273+0+0 (secure 0 0 0) 0x7f9c680cd330 con 0x7f9c58003ab0 2024-09-10T10:50:56.292 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:50:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.290+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 0x55f190e4b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.290+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190e4b620 con 0x55f190e47fc0 2024-09-10T10:50:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.290+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 0x7f9c5006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.290+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 0x55f190e4b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.291+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 0x55f190e4b0e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.291+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 0x7f9c5006ccd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009bb0 tx=0x7f9c64005000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.293+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v101'43 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.293+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190e4ba10 con 0x55f190e47fc0 2024-09-10T10:50:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.295+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v101'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.295+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef4b10 con 0x55f190e47fc0 2024-09-10T10:50:56.297 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:50:56.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.297+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v101'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.297+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190e4bdb0 con 0x55f190e47fc0 2024-09-10T10:50:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.298+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v101'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.298+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5ec0 con 0x55f190e47fc0 2024-09-10T10:50:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.300+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v101'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.300+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef4eb0 con 0x55f190e47fc0 2024-09-10T10:50:56.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.302+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v101'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.302+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5250 con 0x55f190e47fc0 2024-09-10T10:50:56.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.304+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v101'49 uv22 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.304+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5640 con 0x55f190e47fc0 2024-09-10T10:50:56.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.305+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v101'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.305+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5640 con 0x55f190e47fc0 2024-09-10T10:50:56.308 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:50:56.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.307+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.307+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.309+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.309+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.311+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.311+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.312+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.313+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.315 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:50:56.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.314+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v101'55 uv55 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.314+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.316+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [setxattr (30)] v101'56 uv56 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.316+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.318+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [setxattr (62)] v101'57 uv57 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.318+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef55f0 con 0x55f190e47fc0 2024-09-10T10:50:56.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.320+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [omap-set-header] v101'58 uv58 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.320+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5a20 con 0x55f190e47fc0 2024-09-10T10:50:56.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.322+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v101'59 uv59 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.322+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5a20 con 0x55f190e47fc0 2024-09-10T10:50:56.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.323+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 600.00000000 [create] v101'60 uv35 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.323+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5a20 con 0x55f190e47fc0 2024-09-10T10:50:56.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.325+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 600.00000000 [delete] v101'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.325+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef5a20 con 0x55f190e47fc0 2024-09-10T10:50:56.328 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:50:56.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.327+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v101'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.327+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.329+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v101'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.329+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.331+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v101'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.331+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.333+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v101'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.333+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.334+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 603.00000000 [create] v101'66 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.334+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.336+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 603.00000000 [delete] v101'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.336+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.338 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:50:56.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.338+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v101'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.338+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.339+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v101'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.339+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.341+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v101'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.341+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef59f0 con 0x55f190e47fc0 2024-09-10T10:50:56.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.342+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v101'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190e47fc0 2024-09-10T10:50:56.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.342+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f190ef6770 0x55f190ef8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.342+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef90a0 con 0x55f190ef6770 2024-09-10T10:50:56.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.342+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f190ef6770 0x55f190ef8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.343+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f190ef6770 0x55f190ef8b60 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.345+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(30 1.00000000.inode [create] v101'45 uv24 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.345+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef9440 con 0x55f190ef6770 2024-09-10T10:50:56.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.346+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(31 1.00000000.inode [delete] v101'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.346+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef97e0 con 0x55f190ef6770 2024-09-10T10:50:56.348 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:50:56.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.347+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(32 1.00000000.inode [create] v101'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.347+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef9b80 con 0x55f190ef6770 2024-09-10T10:50:56.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.349+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v101'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.349+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190ef9f20 con 0x55f190ef6770 2024-09-10T10:50:56.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.351+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v101'49 uv49 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.351+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efa2c0 con 0x55f190ef6770 2024-09-10T10:50:56.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.352+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(35 200.00000000 [create] v101'50 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.352+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190efa660 con 0x55f190ef6770 2024-09-10T10:50:56.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.353+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(36 200.00000000 [delete] v101'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.353+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.355 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:50:56.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.355+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(37 200.00000000 [create] v101'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.355+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.356+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v101'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.356+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.358+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v101'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.358+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.359+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(40 200.00000001 [create] v101'55 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.359+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.361+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(41 200.00000001 [delete] v101'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.361+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.363 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:50:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.362+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(42 200.00000001 [create] v101'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.362+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190eff0f0 con 0x55f190ef6770 2024-09-10T10:50:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.364+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(43 200.00000001 [write 0~18118] v101'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.364+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190eff0f0 con 0x55f190ef6770 2024-09-10T10:50:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.365+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v101'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.365+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.366+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(45 606.00000000 [create] v101'60 uv40 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.366+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.368+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(46 606.00000000 [delete] v101'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.368+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.370 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:50:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.369+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(47 606.00000000 [create] v101'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.369+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efba10 con 0x55f190ef6770 2024-09-10T10:50:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.370+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v101'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.370+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efba10 con 0x55f190ef6770 2024-09-10T10:50:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.371+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v101'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.371+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efaa00 con 0x55f190ef6770 2024-09-10T10:50:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.372+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v101'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190ef6770 2024-09-10T10:50:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.373+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f190efb080 0x55f190efd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.373+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efd9b0 con 0x55f190efb080 2024-09-10T10:50:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.373+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f190efb080 0x55f190efd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.373+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f190efb080 0x55f190efd470 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.375+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(51 607.00000000 [create] v101'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.375+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190efda30 con 0x55f190efb080 2024-09-10T10:50:56.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.377+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(52 607.00000000 [delete] v101'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.377+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efde50 con 0x55f190efb080 2024-09-10T10:50:56.379 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:50:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.378+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(53 607.00000000 [create] v101'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.378+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efee60 con 0x55f190efb080 2024-09-10T10:50:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.380+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v101'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.380+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efee60 con 0x55f190efb080 2024-09-10T10:50:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.381+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v101'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.381+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190efde50 con 0x55f190efb080 2024-09-10T10:50:56.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.383+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v101'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.383+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f190efe4d0 0x55f190f008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.383+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f00ed0 con 0x55f190efe4d0 2024-09-10T10:50:56.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.383+0000 7f9c71200640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f190efe4d0 0x55f190f008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.384+0000 7f9c71200640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f190efe4d0 0x55f190f008c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.385+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v101'10 uv9 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.385+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f013d0 con 0x55f190efe4d0 2024-09-10T10:50:56.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.386+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v101'11 uv11 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.386+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f018d0 con 0x55f190efe4d0 2024-09-10T10:50:56.388 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:50:56.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.387+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v101'12 uv12 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.387+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02190 con 0x55f190efe4d0 2024-09-10T10:50:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.388+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v101'13 uv13 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.388+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02530 con 0x55f190efb080 2024-09-10T10:50:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.390+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(61 608.00000000 [create] v101'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.390+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02530 con 0x55f190efb080 2024-09-10T10:50:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.391+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(62 608.00000000 [delete] v101'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.391+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02530 con 0x55f190efb080 2024-09-10T10:50:56.393 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:50:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.392+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(63 608.00000000 [create] v101'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.392+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03540 con 0x55f190efb080 2024-09-10T10:50:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.393+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v101'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.393+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03540 con 0x55f190efb080 2024-09-10T10:50:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.394+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v101'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.394+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02530 con 0x55f190efb080 2024-09-10T10:50:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.395+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v101'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.395+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02530 con 0x55f190efe4d0 2024-09-10T10:50:56.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.397+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(67 604.00000000 [create] v101'15 uv9 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.397+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f028d0 con 0x55f190efe4d0 2024-09-10T10:50:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.398+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(68 604.00000000 [delete] v101'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.398+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f02c70 con 0x55f190efe4d0 2024-09-10T10:50:56.400 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:50:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.399+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(69 604.00000000 [create] v101'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.400+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f04020 con 0x55f190efe4d0 2024-09-10T10:50:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.401+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v101'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.401+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f04020 con 0x55f190efe4d0 2024-09-10T10:50:56.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.402+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v101'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.402+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.403+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v101'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.403+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.405+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(73 500.00000000 [create] v101'21 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.405+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.406+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(74 500.00000000 [delete] v101'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.406+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.407+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(75 500.00000000 [create] v101'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.408 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:50:56.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.407+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.409+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v101'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.409+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efe4d0 2024-09-10T10:50:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.410+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v101'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.410+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efb080 2024-09-10T10:50:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.411+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(78 mds_snaptable [create] v101'7 uv6 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.411+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efb080 2024-09-10T10:50:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.412+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(79 mds_snaptable [delete] v101'8 uv8 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.412+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efb080 2024-09-10T10:50:56.414 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:50:56.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.413+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(80 mds_snaptable [create] v101'9 uv9 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.413+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efb080 2024-09-10T10:50:56.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.415+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v101'10 uv10 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.415+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f03010 con 0x55f190efb080 2024-09-10T10:50:56.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.416+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v101'11 uv11 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190efb080 2024-09-10T10:50:56.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.416+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f190f03690 0x55f190f05a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.416+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f060e0 con 0x55f190f03690 2024-09-10T10:50:56.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.416+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f190f03690 0x55f190f05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.417+0000 7f9c70800640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f190f03690 0x55f190f05a80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.418+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v101'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.418+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.420+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v101'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.420+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.422 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:50:56.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.421+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v101'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.421+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f074c0 con 0x55f190f03690 2024-09-10T10:50:56.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.422+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v101'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.422+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f074c0 con 0x55f190f03690 2024-09-10T10:50:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.423+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v101'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.423+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.425+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v101'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.425+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.426+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v101'14 uv8 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.426+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.427+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v101'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.427+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.429 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:50:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.429+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v101'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.429+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064f0 con 0x55f190f03690 2024-09-10T10:50:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.430+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v101'17 uv17 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.430+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.432+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v101'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.432+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.433+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v101'19 uv13 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.433+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.434+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v101'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.434+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.436+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v101'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.437 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:50:56.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.436+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f064b0 con 0x55f190f03690 2024-09-10T10:50:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.437+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v101'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.437+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09300 con 0x55f190f03690 2024-09-10T10:50:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.439+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v101'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.439+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.442+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v101'9 uv7 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.442+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.443+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v101'10 uv10 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.443+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.445 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:50:56.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.445+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v101'11 uv11 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.445+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.446+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v101'12 uv12 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.446+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.447+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v101'9 uv8 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.447+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.448+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v101'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.448+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.449+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v101'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.450 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:50:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.450+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.451+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v101'12 uv12 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.451+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.452+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v101'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.452+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f06940 con 0x55f190efe4d0 2024-09-10T10:50:56.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.453+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v101'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c68097980 con 0x55f190efe4d0 2024-09-10T10:50:56.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.453+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f190f06fc0 0x55f190f093b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.453+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f06fc0 2024-09-10T10:50:56.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.453+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f190f06fc0 0x55f190f093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.454+0000 7f9c6fe00640 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f190f06fc0 0x55f190f093b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.455+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v101'7 uv6 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190f06fc0 2024-09-10T10:50:56.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.455+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f06fc0 2024-09-10T10:50:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.456+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v101'8 uv8 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190f06fc0 2024-09-10T10:50:56.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.456+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f06fc0 2024-09-10T10:50:56.458 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:50:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.457+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v101'9 uv9 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190f06fc0 2024-09-10T10:50:56.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.457+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f06fc0 2024-09-10T10:50:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.458+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v101'10 uv10 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190f06fc0 2024-09-10T10:50:56.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.458+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f06fc0 2024-09-10T10:50:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.459+0000 7f9c6fe00640 1 -- 172.21.15.28:0/724623521 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v101'11 uv11 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9c5c0113c0 con 0x55f190f06fc0 2024-09-10T10:50:56.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.459+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f03690 2024-09-10T10:50:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.461+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v101'20 uv11 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.461+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099c0 con 0x55f190f03690 2024-09-10T10:50:56.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.462+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v101'21 uv21 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.462+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09990 con 0x55f190f03690 2024-09-10T10:50:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.463+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v101'22 uv22 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.464 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:50:56.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.463+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f099b0 con 0x55f190f03690 2024-09-10T10:50:56.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.464+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v101'23 uv23 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.464+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09f30 con 0x55f190f03690 2024-09-10T10:50:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.465+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v101'24 uv24 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.466+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09f30 con 0x55f190f03690 2024-09-10T10:50:56.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.467+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v101'25 uv18 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.467+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09e00 con 0x55f190f03690 2024-09-10T10:50:56.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.468+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v101'26 uv26 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.468+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09db0 con 0x55f190f03690 2024-09-10T10:50:56.469 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:50:56.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.469+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v101'27 uv27 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.469+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f0ae00 con 0x55f190f03690 2024-09-10T10:50:56.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.470+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v101'28 uv28 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.470+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f0adc0 con 0x55f190f03690 2024-09-10T10:50:56.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.471+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v101'29 uv29 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.471+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f09db0 con 0x55f190f03690 2024-09-10T10:50:56.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.472+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v101'30 uv30 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.472+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e101) -- 0x55f190f0a650 con 0x55f190f03690 2024-09-10T10:50:56.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c70800640 1 -- 172.21.15.28:0/724623521 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v101'31 uv31 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9c6400ebc0 con 0x55f190f03690 2024-09-10T10:50:56.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f190efb080 msgr2=0x55f190efd470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f190efb080 0x55f190efd470 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f190f03690 msgr2=0x55f190f05a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f190f03690 0x55f190f05a80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f190f06fc0 msgr2=0x55f190f093b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f190f06fc0 0x55f190f093b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f190efe4d0 msgr2=0x55f190f008c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f190efe4d0 0x55f190f008c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f190ef6770 msgr2=0x55f190ef8b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f190ef6770 0x55f190ef8b60 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 reap_dead start 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c71200640 1 -- 172.21.15.28:0/724623521 reap_dead start 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 msgr2=0x55f190e4b0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 0x55f190e4b0e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 msgr2=0x7f9c5006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 0x7f9c5006ccd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009bb0 tx=0x7f9c64005000 comp rx=0 tx=0).stop 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 msgr2=0x55f190e54f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9c68025e80 tx=0x7f9c680033a0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 shutdown_connections 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f190e47fc0 0x55f190e4b0e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9c5006a810 0x7f9c5006ccd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 --2- 172.21.15.28:0/724623521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c58003ab0 0x55f190e54f60 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 >> 172.21.15.28:0/724623521 conn(0x55f190dc80a0 msgr2=0x55f190e51630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 shutdown_connections 2024-09-10T10:50:56.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.474+0000 7f9c7351ef40 1 -- 172.21.15.28:0/724623521 wait complete. 2024-09-10T10:50:56.480 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 600.00000000 2024-09-10T10:50:56.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 Processor -- start 2024-09-10T10:50:56.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 -- start start 2024-09-10T10:50:56.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a5137cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a5138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5578a5007b30 con 0x5578a5060d00 2024-09-10T10:50:56.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578a500bb70 con 0x5578a50610e0 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.555+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5578a505f210 con 0x5578a50921c0 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36222/0 (socket says 172.21.15.28:36222) 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 -- 172.21.15.28:0/2043150858 learned_addr learned my addr 172.21.15.28:0/2043150858 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42da00640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a5137cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a5138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 -- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 msgr2=0x5578a5138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a5138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 -- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 msgr2=0x5578a5137cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a5137cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 -- 172.21.15.28:0/2043150858 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a512a050 con 0x5578a5060d00 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a5138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:56.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.556+0000 7fd42e400640 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd420025bc0 tx=0x7fd420051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad54d5cc8782ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd427e00640 1 -- 172.21.15.28:0/2043150858 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420056660 con 0x5578a5060d00 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd427e00640 1 -- 172.21.15.28:0/2043150858 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd420056800 con 0x5578a5060d00 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd427e00640 1 -- 172.21.15.28:0/2043150858 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420056bd0 con 0x5578a5060d00 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2043150858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 msgr2=0x5578a5129b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd420025bc0 tx=0x7fd420051000 comp rx=0 tx=0).stop 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2043150858 shutdown_connections 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a5138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a5137cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2043150858 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a5129b10 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2043150858 >> 172.21.15.28:0/2043150858 conn(0x5578a50ae130 msgr2=0x5578a505e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2043150858 shutdown_connections 2024-09-10T10:50:56.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.557+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2043150858 wait complete. 2024-09-10T10:50:56.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.558+0000 7fd4312e5f40 1 Processor -- start 2024-09-10T10:50:56.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.558+0000 7fd4312e5f40 1 -- start start 2024-09-10T10:50:56.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a51282b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a512db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a51356e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5578a5131430 con 0x5578a5060d00 2024-09-10T10:50:56.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578a508dc50 con 0x5578a50610e0 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd4312e5f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5578a508dea0 con 0x5578a50921c0 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd42e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a51282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd42e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a51282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36236/0 (socket says 172.21.15.28:36236) 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.559+0000 7fd42e400640 1 -- 172.21.15.28:0/2282584702 learned_addr learned my addr 172.21.15.28:0/2282584702 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a51356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42da00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a512db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 -- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 msgr2=0x5578a51356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a51356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 -- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 msgr2=0x5578a512db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a512db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 -- 172.21.15.28:0/2282584702 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a51287f0 con 0x5578a5060d00 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578a50921c0 0x5578a51356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42da00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578a50610e0 0x5578a512db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:56.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd42e400640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a51282b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd42000a4f0 tx=0x7fd420002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd426a00640 1 -- 172.21.15.28:0/2282584702 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200569b0 con 0x5578a5060d00 2024-09-10T10:50:56.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd426a00640 1 -- 172.21.15.28:0/2282584702 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42006c070 con 0x5578a5060d00 2024-09-10T10:50:56.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd426a00640 1 -- 172.21.15.28:0/2282584702 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420056d90 con 0x5578a5060d00 2024-09-10T10:50:56.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.560+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5578a512bce0 con 0x5578a5060d00 2024-09-10T10:50:56.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.561+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a512c2a0 con 0x5578a5060d00 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.562+0000 7fd426a00640 1 -- 172.21.15.28:0/2282584702 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd420054440 con 0x5578a5060d00 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.563+0000 7fd426a00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd41006a810 0x7fd41006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.563+0000 7fd426a00640 1 -- 172.21.15.28:0/2282584702 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6273+0+0 (secure 0 0 0) 0x7fd4200cfa40 con 0x5578a5060d00 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.563+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5578a5130880 0x5578a51338d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.563+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e101) -- 0x5578a5129480 con 0x5578a5130880 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.564+0000 7fd42da00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd41006a810 0x7fd41006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.564+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5578a5130880 0x5578a51338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.564+0000 7fd42da00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd41006a810 0x7fd41006ccd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x5578a508ea80 tx=0x7fd41c000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.564+0000 7fd42ee00640 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5578a5130880 0x5578a51338d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd42ee00640 1 -- 172.21.15.28:0/2282584702 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 600.00000000 [delete] v101'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fd428001790 con 0x5578a5130880 2024-09-10T10:50:56.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5578a5130880 msgr2=0x5578a51338d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5578a5130880 0x5578a51338d0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd41006a810 msgr2=0x7fd41006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd41006a810 0x7fd41006ccd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x5578a508ea80 tx=0x7fd41c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 msgr2=0x5578a51282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 --2- 172.21.15.28:0/2282584702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578a5060d00 0x5578a51282b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fd42000a4f0 tx=0x7fd420002fb0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd42ee00640 1 -- 172.21.15.28:0/2282584702 reap_dead start 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 shutdown_connections 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 >> 172.21.15.28:0/2282584702 conn(0x5578a50ae130 msgr2=0x5578a505e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 shutdown_connections 2024-09-10T10:50:56.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.566+0000 7fd4312e5f40 1 -- 172.21.15.28:0/2282584702 wait complete. 2024-09-10T10:50:56.571 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:50:56.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.729+0000 7f0383400640 1 Processor -- start 2024-09-10T10:50:56.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.729+0000 7f0383400640 1 -- start start 2024-09-10T10:50:56.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.730+0000 7f0383400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f037c0cf790 0x7f037c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.730+0000 7f0383400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037c0b30e0 con 0x7f037c0ce060 2024-09-10T10:50:56.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.730+0000 7f0383400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037c0b33c0 con 0x7f037c156740 2024-09-10T10:50:56.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.730+0000 7f0383400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f037c0d0140 con 0x7f037c0cf790 2024-09-10T10:50:56.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f037be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f037c0cf790 0x7f037c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f0381400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f037c0ce060 0x7f037c0ce480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53890/0 (socket says 172.21.15.28:53890) 2024-09-10T10:50:56.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f0381400640 1 -- 172.21.15.28:0/2685977797 learned_addr learned my addr 172.21.15.28:0/2685977797 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:56.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f037be00640 1 -- 172.21.15.28:0/2685977797 >> v1:172.21.15.138:6789/0 conn(0x7f037c156740 legacy=0x7f037c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f037be00640 1 -- 172.21.15.28:0/2685977797 >> v1:172.21.15.28:6789/0 conn(0x7f037c0ce060 legacy=0x7f037c0ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f037be00640 1 -- 172.21.15.28:0/2685977797 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c0b36a0 con 0x7f037c0cf790 2024-09-10T10:50:56.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.731+0000 7f037b400640 1 -- 172.21.15.28:0/2685977797 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999166536 0 0) 0x7f037c0b30e0 con 0x7f037c0ce060 2024-09-10T10:50:56.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.732+0000 7f037be00640 1 --2- 172.21.15.28:0/2685977797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c0cf790 0x7f037c0cfc00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfc23421ced9715 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f037b400640 1 -- 172.21.15.28:0/2685977797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368037070 con 0x7f037c0cf790 2024-09-10T10:50:56.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f037b400640 1 -- 172.21.15.28:0/2685977797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0368032440 con 0x7f037c0cf790 2024-09-10T10:50:56.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f037b400640 1 -- 172.21.15.28:0/2685977797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368032810 con 0x7f037c0cf790 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 -- 172.21.15.28:0/2685977797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c0cf790 msgr2=0x7f037c0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 --2- 172.21.15.28:0/2685977797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c0cf790 0x7f037c0cfc00 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f03680097a0 tx=0x7f036802a230 comp rx=0 tx=0).stop 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 -- 172.21.15.28:0/2685977797 shutdown_connections 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 --2- 172.21.15.28:0/2685977797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c0cf790 0x7f037c0cfc00 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 -- 172.21.15.28:0/2685977797 >> 172.21.15.28:0/2685977797 conn(0x7f037c06bdf0 msgr2=0x7f037c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 -- 172.21.15.28:0/2685977797 shutdown_connections 2024-09-10T10:50:56.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.733+0000 7f0383400640 1 -- 172.21.15.28:0/2685977797 wait complete. 2024-09-10T10:50:56.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.735+0000 7f0383400640 1 Processor -- start 2024-09-10T10:50:56.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.735+0000 7f0383400640 1 -- start start 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037c0ce060 0x7f037c149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 0x7f037c14b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f037c0b3c60 con 0x7f037c156740 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f037c06edc0 con 0x7f037c0cf790 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0383400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f037c0b4cc0 con 0x7f037c0ce060 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0381400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0381400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36248/0 (socket says 172.21.15.28:36248) 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f037be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 0x7f037c14b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0381400640 1 -- 172.21.15.28:0/3237298628 learned_addr learned my addr 172.21.15.28:0/3237298628 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.736+0000 7f0380a00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037c0ce060 0x7f037c149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 -- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037c0ce060 msgr2=0x7f037c149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037c0ce060 0x7f037c149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 -- 172.21.15.28:0/3237298628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 msgr2=0x7f037c1e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 -- 172.21.15.28:0/3237298628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037c149d50 con 0x7f037c0cf790 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0381400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:50:56.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f037be00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 0x7f037c14b320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0368031000 tx=0x7f03680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368032e60 con 0x7f037c0cf790 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0368030770 con 0x7f037c0cf790 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368037040 con 0x7f037c0cf790 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037c149ff0 con 0x7f037c0cf790 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.737+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f037c14a5b0 con 0x7f037c0cf790 2024-09-10T10:50:56.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.738+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f037c0d0920 con 0x7f037c0cf790 2024-09-10T10:50:56.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.740+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0368044020 con 0x7f037c0cf790 2024-09-10T10:50:56.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.740+0000 7f0378c00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 0x7f035c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:56.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.740+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6273+0+0 (secure 0 0 0) 0x7f03680aafe0 con 0x7f037c0cf790 2024-09-10T10:50:56.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.741+0000 7f0380a00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 0x7f035c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:56.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.741+0000 7f0380a00640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 0x7f035c06e320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0370000f60 tx=0x7f0370009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:56.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.746+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f036804b020 con 0x7f037c0cf790 2024-09-10T10:50:56.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:56.968+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f037c15c1e0 con 0x7f037c0cf790 2024-09-10T10:50:57.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.147+0000 7f0378c00640 1 -- 172.21.15.28:0/3237298628 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v95) ==== 162+0+0 (secure 0 0 0) 0x7f0368079110 con 0x7f037c0cf790 2024-09-10T10:50:57.149 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:50:57.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 msgr2=0x7f035c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 0x7f035c06e320 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0370000f60 tx=0x7f0370009040 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 msgr2=0x7f037c14b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 0x7f037c14b320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0368031000 tx=0x7f03680046c0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 shutdown_connections 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f035c06be60 0x7f035c06e320 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f037c156740 0x7f037c1e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f037c0cf790 0x7f037c14b320 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 --2- 172.21.15.28:0/3237298628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f037c0ce060 0x7f037c149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.150+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 >> 172.21.15.28:0/3237298628 conn(0x7f037c06bdf0 msgr2=0x7f037c06ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.151+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 shutdown_connections 2024-09-10T10:50:57.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.151+0000 7f0383400640 1 -- 172.21.15.28:0/3237298628 wait complete. 2024-09-10T10:50:57.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:50:57.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.324+0000 7fce7e000640 1 Processor -- start 2024-09-10T10:50:57.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.324+0000 7fce7e000640 1 -- start start 2024-09-10T10:50:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.325+0000 7fce7e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce780d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.325+0000 7fce7e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce780d1f90 0x7fce780cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.325+0000 7fce7e000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce780cf000 con 0x7fce7815e660 2024-09-10T10:50:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.325+0000 7fce7e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce7806d3e0 con 0x7fce780d1f90 2024-09-10T10:50:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.325+0000 7fce7e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce780b37d0 con 0x7fce780d0dd0 2024-09-10T10:50:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fce7815e660 0x7fce7806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53898/0 (socket says 172.21.15.28:53898) 2024-09-10T10:50:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77e00640 1 -- 172.21.15.28:0/832228733 learned_addr learned my addr 172.21.15.28:0/832228733 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce7d600640 1 --2- 172.21.15.28:0/832228733 >> v2:172.21.15.138:3300/0 conn(0x7fce780d1f90 0x7fce780cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce7cc00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce780d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197467906 0 0) 0x7fce780cf000 con 0x7fce7815e660 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce60003520 con 0x7fce7815e660 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 508657580 0 0) 0x7fce60003520 con 0x7fce7815e660 2024-09-10T10:50:57.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 msgr2=0x7fce780d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce780d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 >> v2:172.21.15.138:3300/0 conn(0x7fce780d1f90 msgr2=0x7fce780cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 --2- 172.21.15.28:0/832228733 >> v2:172.21.15.138:3300/0 conn(0x7fce780d1f90 0x7fce780cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce780d16f0 con 0x7fce7815e660 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fce64001670 con 0x7fce7815e660 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 >> v1:172.21.15.28:6789/0 conn(0x7fce7815e660 legacy=0x7fce7806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce60004800 0x7fce60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fce7806d3e0 con 0x7fce600039b0 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce6000b9c0 con 0x7fce60004800 2024-09-10T10:50:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.326+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce6000bb40 con 0x7fce780d0dd0 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce77e00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce60004800 0x7fce60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7d600640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 -- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 msgr2=0x7fce600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce780d0dd0 0x7fce600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 -- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce60004800 msgr2=0x7fce60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce60004800 0x7fce60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.327+0000 7fce7cc00640 1 -- 172.21.15.28:0/832228733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce60003520 con 0x7fce600039b0 2024-09-10T10:50:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.328+0000 7fce7cc00640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009ac0 tx=0x7fce60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6471d4fb48e5b0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.328+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c036070 con 0x7fce600039b0 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce6c031400 con 0x7fce600039b0 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce77400640 1 -- 172.21.15.28:0/832228733 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6c009690 con 0x7fce600039b0 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 -- 172.21.15.28:0/832228733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce600039b0 msgr2=0x7fce60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 --2- 172.21.15.28:0/832228733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce600039b0 0x7fce60003dd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009ac0 tx=0x7fce60005830 comp rx=0 tx=0).stop 2024-09-10T10:50:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7d600640 1 -- 172.21.15.28:0/832228733 reap_dead start 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 -- 172.21.15.28:0/832228733 shutdown_connections 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 -- 172.21.15.28:0/832228733 >> 172.21.15.28:0/832228733 conn(0x7fce7808bfc0 msgr2=0x7fce780d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 -- 172.21.15.28:0/832228733 shutdown_connections 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.329+0000 7fce7e000640 1 -- 172.21.15.28:0/832228733 wait complete. 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.331+0000 7fce7e000640 1 Processor -- start 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.331+0000 7fce7e000640 1 -- start start 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce600039b0 0x7fce780b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 0x7fce780cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fce780d2c60 con 0x7fce7815b510 2024-09-10T10:50:57.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce780cf330 con 0x7fce780cd6f0 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce780d2f40 con 0x7fce600039b0 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36280/0 (socket says 172.21.15.28:36280) 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce77e00640 1 -- 172.21.15.28:0/2816647302 learned_addr learned my addr 172.21.15.28:0/2816647302 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7cc00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce600039b0 0x7fce780b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.332+0000 7fce7d600640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 0x7fce780cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 -- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce600039b0 msgr2=0x7fce780b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce600039b0 0x7fce780b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 -- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 msgr2=0x7fce780cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 0x7fce780cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 -- 172.21.15.28:0/2816647302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce780ce0c0 con 0x7fce7815b510 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce7d600640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 0x7fce780cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce77e00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fce64007370 tx=0x7fce64007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64007e10 con 0x7fce7815b510 2024-09-10T10:50:57.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce64018070 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce781f7d90 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6400fc00 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.333+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce781f8350 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.334+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce780b3bc0 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.335+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce6401d460 con 0x7fce7815b510 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.336+0000 7fce74c00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 0x7fce5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:50:57.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.336+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 6273+0+0 (secure 0 0 0) 0x7fce6408b070 con 0x7fce7815b510 2024-09-10T10:50:57.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.336+0000 7fce7cc00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 0x7fce5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:50:57.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.337+0000 7fce7cc00640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 0x7fce5806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fce6c033000 tx=0x7fce6c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:50:57.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.341+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce780b3bc0 con 0x7fce7815b510 2024-09-10T10:50:57.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.559+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce780d3220 con 0x7fce7815b510 2024-09-10T10:50:57.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.560+0000 7fce74c00640 1 -- 172.21.15.28:0/2816647302 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+5220 (secure 0 0 0) 0x7fce64059740 con 0x7fce7815b510 2024-09-10T10:50:57.561 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:50:57.561 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":96,"btime":"2024-09-10T10:50:57:147340+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:50:57.147330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7530":{"gid":7530,"name":"a","rank":0,"incarnation":96,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/1408648899","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1408648899},{"type":"v1","addr":"172.21.15.28:6835","nonce":1408648899}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:50:57.562 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 96 2024-09-10T10:50:57.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 msgr2=0x7fce5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 0x7fce5806e2f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fce6c033000 tx=0x7fce6c008030 comp rx=0 tx=0).stop 2024-09-10T10:50:57.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 msgr2=0x7fce7815b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:50:57.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fce64007370 tx=0x7fce64007830 comp rx=0 tx=0).stop 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 shutdown_connections 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce5806be30 0x7fce5806e2f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce780cd6f0 0x7fce780cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce7815b510 0x7fce7815b990 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 --2- 172.21.15.28:0/2816647302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce600039b0 0x7fce780b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 >> 172.21.15.28:0/2816647302 conn(0x7fce7808bfc0 msgr2=0x7fce780b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 shutdown_connections 2024-09-10T10:50:57.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:50:57.562+0000 7fce7e000640 1 -- 172.21.15.28:0/2816647302 wait complete. 2024-09-10T10:50:57.578 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:51:00.169 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:00.168+0000 7fd727600640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-10T10:51:00.169 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:00.168+0000 7fd727600640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T10:51:00.169 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:00.168+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:00.169 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:00.168+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:01.148 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:01.147+0000 7fd72c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:02.579 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:02.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.747+0000 7f3d12600640 1 Processor -- start 2024-09-10T10:51:02.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.747+0000 7f3d12600640 1 -- start start 2024-09-10T10:51:02.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d12600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d0c15b330 0x7f3d0c0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d12600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c0734f0 0x7f3d0c0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d12600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d0c0b4d30 con 0x7f3d0c15af50 2024-09-10T10:51:02.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d12600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3d0c06dc70 con 0x7f3d0c15b330 2024-09-10T10:51:02.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d12600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c0b3f50 con 0x7f3d0c0734f0 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d0be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d0c15b330 0x7f3d0c0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d11200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3d0c15af50 0x7f3d0c072de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53902/0 (socket says 172.21.15.28:53902) 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.748+0000 7f3d11200640 1 -- 172.21.15.28:0/1210567386 learned_addr learned my addr 172.21.15.28:0/1210567386 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.749+0000 7f3d11c00640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c0734f0 0x7f3d0c0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.749+0000 7f3d0be00640 1 -- 172.21.15.28:0/1210567386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c0734f0 msgr2=0x7f3d0c0b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.749+0000 7f3d0be00640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c0734f0 0x7f3d0c0b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.749+0000 7f3d0be00640 1 -- 172.21.15.28:0/1210567386 >> v1:172.21.15.28:6789/0 conn(0x7f3d0c15af50 legacy=0x7f3d0c072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.749+0000 7f3d0be00640 1 -- 172.21.15.28:0/1210567386 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c06df10 con 0x7f3d0c15b330 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.750+0000 7f3d0be00640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0b42b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80097a0 tx=0x7f3cf802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03f2346415007db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.750+0000 7f3d0b400640 1 -- 172.21.15.28:0/1210567386 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8037070 con 0x7f3d0c15b330 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.750+0000 7f3d0b400640 1 -- 172.21.15.28:0/1210567386 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cf8032440 con 0x7f3d0c15b330 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.750+0000 7f3d0b400640 1 -- 172.21.15.28:0/1210567386 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8032810 con 0x7f3d0c15b330 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 -- 172.21.15.28:0/1210567386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15b330 msgr2=0x7f3d0c0b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0b42b0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80097a0 tx=0x7f3cf802a180 comp rx=0 tx=0).stop 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 -- 172.21.15.28:0/1210567386 shutdown_connections 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c0734f0 0x7f3d0c0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 --2- 172.21.15.28:0/1210567386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0b42b0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 -- 172.21.15.28:0/1210567386 >> 172.21.15.28:0/1210567386 conn(0x7f3d0c06c240 msgr2=0x7f3d0c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 -- 172.21.15.28:0/1210567386 shutdown_connections 2024-09-10T10:51:02.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.751+0000 7f3d12600640 1 -- 172.21.15.28:0/1210567386 wait complete. 2024-09-10T10:51:02.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.752+0000 7f3d12600640 1 Processor -- start 2024-09-10T10:51:02.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 -- start start 2024-09-10T10:51:02.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15af50 0x7f3d0c15e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e7b0 con 0x7f3d0c0734f0 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c06e4d0 con 0x7f3d0c15b330 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.753+0000 7f3d12600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c0adbc0 con 0x7f3d0c15af50 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36292/0 (socket says 172.21.15.28:36292) 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 -- 172.21.15.28:0/3109753687 learned_addr learned my addr 172.21.15.28:0/3109753687 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d0be00640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15af50 0x7f3d0c15e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11c00640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 -- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15af50 msgr2=0x7f3d0c15e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15af50 0x7f3d0c15e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 -- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c15b330 msgr2=0x7f3d0c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 -- 172.21.15.28:0/3109753687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c156830 con 0x7f3d0c0734f0 2024-09-10T10:51:02.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.754+0000 7f3d11200640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c400 tx=0x7f3d0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.755+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00017070 con 0x7f3d0c0734f0 2024-09-10T10:51:02.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.755+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d00005ce0 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.755+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000072f0 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.755+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c156b10 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.755+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1570d0 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.756+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c0b42b0 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.757+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d0001a070 con 0x7f3d0c0734f0 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.758+0000 7f3d08c00640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:02.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.758+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7f3d0008a530 con 0x7f3d0c0734f0 2024-09-10T10:51:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.758+0000 7f3d0be00640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:02.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.759+0000 7f3d0be00640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8034000 tx=0x7f3cf802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:02.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.762+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d00058c40 con 0x7f3d0c0734f0 2024-09-10T10:51:02.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.985+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d0c15ea00 con 0x7f3d0c0734f0 2024-09-10T10:51:02.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.986+0000 7f3d08c00640 1 -- 172.21.15.28:0/3109753687 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4388 (secure 0 0 0) 0x7f3d00058540 con 0x7f3d0c0734f0 2024-09-10T10:51:02.987 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:02.987 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":100,"btime":"2024-09-10T10:51:00:169198+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:00.169195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:02.987 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 100 2024-09-10T10:51:02.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 msgr2=0x7f3cec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8034000 tx=0x7f3cf802e000 comp rx=0 tx=0).stop 2024-09-10T10:51:02.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 msgr2=0x7f3d0c15c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000c400 tx=0x7f3d0000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 shutdown_connections 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3cec06be30 0x7f3cec06e2f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d0c15b330 0x7f3d0c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d0c15af50 0x7f3d0c15e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 --2- 172.21.15.28:0/3109753687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d0c0734f0 0x7f3d0c15c070 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 >> 172.21.15.28:0/3109753687 conn(0x7f3d0c06c240 msgr2=0x7f3d0c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 shutdown_connections 2024-09-10T10:51:02.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:02.988+0000 7f3d12600640 1 -- 172.21.15.28:0/3109753687 wait complete. 2024-09-10T10:51:03.005 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:51:03.005 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:03.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.160+0000 7fac79000640 1 Processor -- start 2024-09-10T10:51:03.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.160+0000 7fac79000640 1 -- start start 2024-09-10T10:51:03.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.161+0000 7fac79000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac7415b300 0x7fac740cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac79000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac79000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac740cdbe0 con 0x7fac74072d30 2024-09-10T10:51:03.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac79000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac740728a0 con 0x7fac7415b300 2024-09-10T10:51:03.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac79000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac740cd900 con 0x7fac740cd3f0 2024-09-10T10:51:03.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac69400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac69400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:51206/0 (socket says 172.21.15.28:51206) 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac69400640 1 -- 172.21.15.28:0/3505058787 learned_addr learned my addr 172.21.15.28:0/3505058787 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac72800640 1 --1- 172.21.15.28:0/3505058787 >> v1:172.21.15.138:6789/0 conn(0x7fac74072d30 0x7fac740b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:45312/0 (socket says 172.21.15.28:45312) 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.162+0000 7fac71e00640 1 --2- 172.21.15.28:0/3505058787 >> v2:172.21.15.138:3300/0 conn(0x7fac7415b300 0x7fac740cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.163+0000 7fac69400640 1 -- 172.21.15.28:0/3505058787 >> v2:172.21.15.138:3300/0 conn(0x7fac7415b300 msgr2=0x7fac740cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.163+0000 7fac69400640 1 --2- 172.21.15.28:0/3505058787 >> v2:172.21.15.138:3300/0 conn(0x7fac7415b300 0x7fac740cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.163+0000 7fac69400640 1 -- 172.21.15.28:0/3505058787 >> v1:172.21.15.138:6789/0 conn(0x7fac74072d30 legacy=0x7fac740b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.163+0000 7fac69400640 1 -- 172.21.15.28:0/3505058787 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74072250 con 0x7fac740cd3f0 2024-09-10T10:51:03.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac69400640 1 --2- 172.21.15.28:0/3505058787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00bb10 tx=0x7fac5c02c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe88de661d62945b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac71400640 1 -- 172.21.15.28:0/3505058787 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c036020 con 0x7fac740cd3f0 2024-09-10T10:51:03.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac71400640 1 -- 172.21.15.28:0/3505058787 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5c032820 con 0x7fac740cd3f0 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac71400640 1 -- 172.21.15.28:0/3505058787 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5c032bf0 con 0x7fac740cd3f0 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac79000640 1 -- 172.21.15.28:0/3505058787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 msgr2=0x7fac7406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.164+0000 7fac79000640 1 --2- 172.21.15.28:0/3505058787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fac5c00bb10 tx=0x7fac5c02c920 comp rx=0 tx=0).stop 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 -- 172.21.15.28:0/3505058787 shutdown_connections 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 --2- 172.21.15.28:0/3505058787 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac740cd3f0 0x7fac7406f1c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 --2- 172.21.15.28:0/3505058787 >> v2:172.21.15.138:3300/0 conn(0x7fac7415b300 0x7fac740cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 -- 172.21.15.28:0/3505058787 >> 172.21.15.28:0/3505058787 conn(0x7fac74067d70 msgr2=0x7fac7406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 -- 172.21.15.28:0/3505058787 shutdown_connections 2024-09-10T10:51:03.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.165+0000 7fac79000640 1 -- 172.21.15.28:0/3505058787 wait complete. 2024-09-10T10:51:03.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.166+0000 7fac79000640 1 Processor -- start 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.166+0000 7fac79000640 1 -- start start 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac74072d30 0x7fac74152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740cd3f0 0x7fac740b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac740d0160 con 0x7fac7415b300 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac740cffe0 con 0x7fac740cd3f0 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac79000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac7406f700 con 0x7fac74072d30 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.167+0000 7fac72800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36308/0 (socket says 172.21.15.28:36308) 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 -- 172.21.15.28:0/1186391732 learned_addr learned my addr 172.21.15.28:0/1186391732 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac71e00640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac74072d30 0x7fac74152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac69400640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740cd3f0 0x7fac740b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 -- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac74072d30 msgr2=0x7fac74152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac74072d30 0x7fac74152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 -- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740cd3f0 msgr2=0x7fac740b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740cd3f0 0x7fac740b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 -- 172.21.15.28:0/1186391732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac74153600 con 0x7fac7415b300 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.168+0000 7fac72800640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c780 tx=0x7fac6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.169+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c017070 con 0x7fac7415b300 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.169+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac6c00a640 con 0x7fac7415b300 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.169+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6c00aa30 con 0x7fac7415b300 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.169+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac741538e0 con 0x7fac7415b300 2024-09-10T10:51:03.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.169+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac740b2f00 con 0x7fac7415b300 2024-09-10T10:51:03.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.170+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac740ce990 con 0x7fac7415b300 2024-09-10T10:51:03.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.171+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fac6c01a070 con 0x7fac7415b300 2024-09-10T10:51:03.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.172+0000 7fac6aa00640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 0x7fac5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.172+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7fac6c08a700 con 0x7fac7415b300 2024-09-10T10:51:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.172+0000 7fac71e00640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 0x7fac5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.173+0000 7fac71e00640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 0x7fac5006e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac64009aa0 tx=0x7fac64009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.176+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac740ce990 con 0x7fac7415b300 2024-09-10T10:51:03.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.394+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac7406fac0 con 0x7fac7415b300 2024-09-10T10:51:03.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.394+0000 7fac6aa00640 1 -- 172.21.15.28:0/1186391732 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4388 (secure 0 0 0) 0x7fac6c058730 con 0x7fac7415b300 2024-09-10T10:51:03.396 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:03.396 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":100,"btime":"2024-09-10T10:51:00:169198+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:00.169195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:03.396 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 100 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.396+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 msgr2=0x7fac5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.396+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 0x7fac5006e320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac64009aa0 tx=0x7fac64009190 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.396+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 msgr2=0x7fac7414ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.396+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c780 tx=0x7fac6c00cc40 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 shutdown_connections 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fac5006be60 0x7fac5006e320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac7415b300 0x7fac7414ff00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac740cd3f0 0x7fac740b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 --2- 172.21.15.28:0/1186391732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac74072d30 0x7fac74152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 >> 172.21.15.28:0/1186391732 conn(0x7fac74067d70 msgr2=0x7fac7415da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 shutdown_connections 2024-09-10T10:51:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.397+0000 7fac79000640 1 -- 172.21.15.28:0/1186391732 wait complete. 2024-09-10T10:51:03.413 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:03.413 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.568+0000 7fdfe1a00640 1 Processor -- start 2024-09-10T10:51:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.569+0000 7fdfe1a00640 1 -- start start 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfe1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfdc0b0180 0x7fdfdc162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfe1a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfdc15eb60 con 0x7fdfdc0af280 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfe1a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfdc15ee40 con 0x7fdfdc0b69d0 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfe1a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdfdc0b6db0 con 0x7fdfdc0b0180 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfdb400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdfdc0af280 0x7fdfdc154d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53914/0 (socket says 172.21.15.28:53914) 2024-09-10T10:51:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfdb400640 1 -- 172.21.15.28:0/1318161086 learned_addr learned my addr 172.21.15.28:0/1318161086 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:03.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfd3e00640 1 --2- 172.21.15.28:0/1318161086 >> v2:172.21.15.138:3300/0 conn(0x7fdfdc0b0180 0x7fdfdc162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.570+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804998929 0 0) 0x7fdfdc15eb60 con 0x7fdfdc0af280 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfc0003520 con 0x7fdfdc0af280 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389365344 0 0) 0x7fdfdc15ee40 con 0x7fdfdc0b69d0 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfdc15eb60 con 0x7fdfdc0b69d0 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfd3e00640 1 -- 172.21.15.28:0/1318161086 >> v1:172.21.15.138:6789/0 conn(0x7fdfdc0b69d0 legacy=0x7fdfdc0afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3780879570 0 0) 0x7fdfc0003520 con 0x7fdfdc0af280 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfd3e00640 1 -- 172.21.15.28:0/1318161086 >> v1:172.21.15.28:6789/0 conn(0x7fdfdc0af280 legacy=0x7fdfdc154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.571+0000 7fdfd3e00640 1 -- 172.21.15.28:0/1318161086 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc15f120 con 0x7fdfdc0b0180 2024-09-10T10:51:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.572+0000 7fdfd3e00640 1 --2- 172.21.15.28:0/1318161086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc162a50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40097a0 tx=0x7fdfc402a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ea62cb264fbdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.572+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4037070 con 0x7fdfdc0b0180 2024-09-10T10:51:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.572+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc402ab30 con 0x7fdfdc0b0180 2024-09-10T10:51:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.572+0000 7fdfda000640 1 -- 172.21.15.28:0/1318161086 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40019f0 con 0x7fdfdc0b0180 2024-09-10T10:51:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1318161086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 msgr2=0x7fdfdc162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1318161086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc162a50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40097a0 tx=0x7fdfc402a0a0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1318161086 shutdown_connections 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1318161086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc162a50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1318161086 >> 172.21.15.28:0/1318161086 conn(0x7fdfdc06bc10 msgr2=0x7fdfdc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1318161086 shutdown_connections 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.573+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1318161086 wait complete. 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.575+0000 7fdfe1a00640 1 Processor -- start 2024-09-10T10:51:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.575+0000 7fdfe1a00640 1 -- start start 2024-09-10T10:51:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc0c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 0x7fdfdc1f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc06f870 con 0x7fdfdc0af280 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc0af6d0 con 0x7fdfdc0b0180 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfe1a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfdc06f9f0 con 0x7fdfdc0b69d0 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36312/0 (socket says 172.21.15.28:36312) 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 -- 172.21.15.28:0/1675738104 learned_addr learned my addr 172.21.15.28:0/1675738104 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfd3e00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc0c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdb400640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 0x7fdfdc1f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 -- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 msgr2=0x7fdfdc1f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 0x7fdfdc1f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 -- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 msgr2=0x7fdfdc0c65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc0c65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdaa00640 1 -- 172.21.15.28:0/1675738104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc1f4a50 con 0x7fdfdc0af280 2024-09-10T10:51:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.576+0000 7fdfdb400640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 0x7fdfdc1f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfdaa00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c460 tx=0x7fdfcc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0052d0 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfcc005490 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc1f4d30 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc012870 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.577+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc1f7710 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.578+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc0c5be0 con 0x7fdfdc0af280 2024-09-10T10:51:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.579+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfcc012a10 con 0x7fdfdc0af280 2024-09-10T10:51:03.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.580+0000 7fdfd2a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 0x7fdfb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:03.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.580+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7fdfcc08acb0 con 0x7fdfdc0af280 2024-09-10T10:51:03.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.581+0000 7fdfd3e00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 0x7fdfb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:03.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.581+0000 7fdfd3e00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 0x7fdfb806e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4034000 tx=0x7fdfc402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:03.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.584+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfcc059330 con 0x7fdfdc0af280 2024-09-10T10:51:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.808+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfdc06d750 con 0x7fdfdc0af280 2024-09-10T10:51:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.808+0000 7fdfd2a00640 1 -- 172.21.15.28:0/1675738104 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+4388 (secure 0 0 0) 0x7fdfcc058c30 con 0x7fdfdc0af280 2024-09-10T10:51:03.810 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:03.810 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":100,"btime":"2024-09-10T10:51:00:169198+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:00.169195+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:03.810 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 100 2024-09-10T10:51:03.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 msgr2=0x7fdfb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 0x7fdfb806e2f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4034000 tx=0x7fdfc402e000 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 msgr2=0x7fdfdc0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c460 tx=0x7fdfcc00c920 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 shutdown_connections 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfb806be30 0x7fdfb806e2f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfdc0b69d0 0x7fdfdc1f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfdc0b0180 0x7fdfdc0c65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 --2- 172.21.15.28:0/1675738104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfdc0af280 0x7fdfdc0cfbf0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.810+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 >> 172.21.15.28:0/1675738104 conn(0x7fdfdc06bc10 msgr2=0x7fdfdc06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.811+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 shutdown_connections 2024-09-10T10:51:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:03.811+0000 7fdfe1a00640 1 -- 172.21.15.28:0/1675738104 wait complete. 2024-09-10T10:51:03.827 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 600.00000000' led to DAMAGED state 2024-09-10T10:51:03.828 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000.inode' 2024-09-10T10:51:03.828 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:03.828 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:03.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:03.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:03.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:03.864 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:03.938 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:03.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:03.939 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:04.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.135+0000 7f755d600640 1 Processor -- start 2024-09-10T10:51:04.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.135+0000 7f755d600640 1 -- start start 2024-09-10T10:51:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.136+0000 7f755d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7558072320 0x7f75580cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.136+0000 7f755d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f755815e1f0 0x7f755806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.136+0000 7f755d600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7558146140 con 0x7f7558072de0 2024-09-10T10:51:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.136+0000 7f755d600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f75580cd740 con 0x7f7558072320 2024-09-10T10:51:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.136+0000 7f755d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f75580cda20 con 0x7f755815e1f0 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7558072320 0x7f75580cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7558072320 0x7f75580cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35124/0 (socket says 172.21.15.28:35124) 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 -- 172.21.15.28:0/1028292663 learned_addr learned my addr 172.21.15.28:0/1028292663 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7557400640 1 --2- 172.21.15.28:0/1028292663 >> v2:172.21.15.138:3300/0 conn(0x7f755815e1f0 0x7f755806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 -- 172.21.15.28:0/1028292663 >> v2:172.21.15.138:3300/0 conn(0x7f755815e1f0 msgr2=0x7f755806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 --2- 172.21.15.28:0/1028292663 >> v2:172.21.15.138:3300/0 conn(0x7f755815e1f0 0x7f755806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 -- 172.21.15.28:0/1028292663 >> v1:172.21.15.138:6789/0 conn(0x7f7558072de0 legacy=0x7f755806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.137+0000 7f7556a00640 1 -- 172.21.15.28:0/1028292663 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75580cdd00 con 0x7f7558072320 2024-09-10T10:51:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.138+0000 7f7556a00640 1 --2- 172.21.15.28:0/1028292663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072320 0x7f75580cce60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f75580cfce0 tx=0x7f754802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb9fa344ecca5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.138+0000 7f7556000640 1 -- 172.21.15.28:0/1028292663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548037070 con 0x7f7558072320 2024-09-10T10:51:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f7556000640 1 -- 172.21.15.28:0/1028292663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548032440 con 0x7f7558072320 2024-09-10T10:51:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f7556000640 1 -- 172.21.15.28:0/1028292663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548032810 con 0x7f7558072320 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 -- 172.21.15.28:0/1028292663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072320 msgr2=0x7f75580cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 --2- 172.21.15.28:0/1028292663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072320 0x7f75580cce60 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f75580cfce0 tx=0x7f754802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 -- 172.21.15.28:0/1028292663 shutdown_connections 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 --2- 172.21.15.28:0/1028292663 >> v2:172.21.15.138:3300/0 conn(0x7f755815e1f0 0x7f755806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 --2- 172.21.15.28:0/1028292663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072320 0x7f75580cce60 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 -- 172.21.15.28:0/1028292663 >> 172.21.15.28:0/1028292663 conn(0x7f7558067d70 msgr2=0x7f755806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 -- 172.21.15.28:0/1028292663 shutdown_connections 2024-09-10T10:51:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.139+0000 7f755d600640 1 -- 172.21.15.28:0/1028292663 wait complete. 2024-09-10T10:51:04.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.141+0000 7f755d600640 1 Processor -- start 2024-09-10T10:51:04.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.141+0000 7f755d600640 1 -- start start 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 0x7f755813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f755815e1f0 0x7f7558142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f75580cd520 con 0x7f7558072de0 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75580cd3a0 con 0x7f7558072320 2024-09-10T10:51:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f755d600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f755806ed00 con 0x7f755815e1f0 2024-09-10T10:51:04.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f754e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 0x7f755813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f7556a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f7556a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:34730/0 (socket says 172.21.15.28:34730) 2024-09-10T10:51:04.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f7556a00640 1 -- 172.21.15.28:0/2440520302 learned_addr learned my addr 172.21.15.28:0/2440520302 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:04.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.142+0000 7f7557400640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f755815e1f0 0x7f7558142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 -- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f755815e1f0 msgr2=0x7f7558142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f755815e1f0 0x7f7558142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 -- 172.21.15.28:0/2440520302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 msgr2=0x7f755813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 0x7f755813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 -- 172.21.15.28:0/2440520302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7558160d20 con 0x7f7558072320 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f754e000640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 0x7f755813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f7556a00640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f754802a220 tx=0x7f75480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548041070 con 0x7f7558072320 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f754803c870 con 0x7f7558072320 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548037040 con 0x7f7558072320 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.143+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755815ef20 con 0x7f7558072320 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.144+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755815f4e0 con 0x7f7558072320 2024-09-10T10:51:04.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.145+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7558072700 con 0x7f7558072320 2024-09-10T10:51:04.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.146+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7548044070 con 0x7f7558072320 2024-09-10T10:51:04.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.147+0000 7f754f400640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 0x7f753406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.147+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7f75480abed0 con 0x7f7558072320 2024-09-10T10:51:04.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.147+0000 7f754e000640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 0x7f753406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.148+0000 7f754e000640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 0x7f753406e340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f755813ee30 tx=0x7f7540009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.151+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7558160d20 con 0x7f7558072320 2024-09-10T10:51:04.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.372+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f75580b4280 con 0x7f7558072320 2024-09-10T10:51:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.658+0000 7f754f400640 1 -- 172.21.15.28:0/2440520302 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f754807a0a0 con 0x7f7558072320 2024-09-10T10:51:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.660+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 msgr2=0x7f753406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.660+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 0x7f753406e340 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f755813ee30 tx=0x7f7540009040 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.660+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 msgr2=0x7f755815b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.660+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f754802a220 tx=0x7f75480046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.660+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 shutdown_connections 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753406be80 0x7f753406e340 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f755815e1f0 0x7f7558142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7558072de0 0x7f755813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 --2- 172.21.15.28:0/2440520302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7558072320 0x7f755815b540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 >> 172.21.15.28:0/2440520302 conn(0x7f7558067d70 msgr2=0x7f75580ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 shutdown_connections 2024-09-10T10:51:04.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.661+0000 7f755d600640 1 -- 172.21.15.28:0/2440520302 wait complete. 2024-09-10T10:51:04.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:04.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.835+0000 7fd50c200640 1 Processor -- start 2024-09-10T10:51:04.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.835+0000 7fd50c200640 1 -- start start 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.836+0000 7fd50c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 0x7fd5040cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50c200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd504072de0 0x7fd50406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50c200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd504146140 con 0x7fd504072700 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50c200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040cd740 con 0x7fd504072de0 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd5040cda20 con 0x7fd504072320 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50a200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd504072700 0x7fd50406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33990/0 (socket says 172.21.15.28:33990) 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd50a200640 1 -- 172.21.15.28:0/254879655 learned_addr learned my addr 172.21.15.28:0/254879655 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:04.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd509800640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 0x7fd5040cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd508e00640 1 --2- 172.21.15.28:0/254879655 >> v2:172.21.15.138:3300/0 conn(0x7fd504072de0 0x7fd50406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd503e00640 1 -- 172.21.15.28:0/254879655 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638454338 0 0) 0x7fd504146140 con 0x7fd504072700 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.837+0000 7fd503e00640 1 -- 172.21.15.28:0/254879655 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4ec003520 con 0x7fd504072700 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.838+0000 7fd508e00640 1 -- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 msgr2=0x7fd5040cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.838+0000 7fd508e00640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 0x7fd5040cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.838+0000 7fd508e00640 1 -- 172.21.15.28:0/254879655 >> v1:172.21.15.28:6789/0 conn(0x7fd504072700 legacy=0x7fd50406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.838+0000 7fd508e00640 1 -- 172.21.15.28:0/254879655 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040cdd00 con 0x7fd504072de0 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.838+0000 7fd509800640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 0x7fd5040cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:04.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.839+0000 7fd508e00640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072de0 0x7fd50406e0b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b398917b005798 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.839+0000 7fd503e00640 1 -- 172.21.15.28:0/254879655 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0037070 con 0x7fd504072de0 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.839+0000 7fd503e00640 1 -- 172.21.15.28:0/254879655 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f002ae40 con 0x7fd504072de0 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.839+0000 7fd503e00640 1 -- 172.21.15.28:0/254879655 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0032660 con 0x7fd504072de0 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 -- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072de0 msgr2=0x7fd50406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072de0 0x7fd50406e0b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00097a0 tx=0x7fd4f002a0f0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 -- 172.21.15.28:0/254879655 shutdown_connections 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072de0 0x7fd50406e0b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 --2- 172.21.15.28:0/254879655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072320 0x7fd5040cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 -- 172.21.15.28:0/254879655 >> 172.21.15.28:0/254879655 conn(0x7fd504067d70 msgr2=0x7fd50406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:04.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 -- 172.21.15.28:0/254879655 shutdown_connections 2024-09-10T10:51:04.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.840+0000 7fd50c200640 1 -- 172.21.15.28:0/254879655 wait complete. 2024-09-10T10:51:04.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.841+0000 7fd50c200640 1 Processor -- start 2024-09-10T10:51:04.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.842+0000 7fd50c200640 1 -- start start 2024-09-10T10:51:04.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.842+0000 7fd50c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072700 0x7fd5040b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072de0 0x7fd5040b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50c200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040cd520 con 0x7fd504072320 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040cd3a0 con 0x7fd504072700 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd50406ed00 con 0x7fd504072de0 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35148/0 (socket says 172.21.15.28:35148) 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 -- 172.21.15.28:0/603830943 learned_addr learned my addr 172.21.15.28:0/603830943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd50a200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072de0 0x7fd5040b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd508e00640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072700 0x7fd5040b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 -- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072de0 msgr2=0x7fd5040b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072de0 0x7fd5040b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 -- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072700 msgr2=0x7fd5040b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072700 0x7fd5040b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:04.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.843+0000 7fd509800640 1 -- 172.21.15.28:0/603830943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd504141270 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd509800640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800ebe0 tx=0x7fd4f800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8017070 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f800cc30 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8013470 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5041e2980 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.844+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5041e2f40 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.845+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5040cce60 con 0x7fd504072320 2024-09-10T10:51:04.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.846+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4f801a070 con 0x7fd504072320 2024-09-10T10:51:04.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.847+0000 7fd501600640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 0x7fd4e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:04.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.847+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7fd4f808afe0 con 0x7fd504072320 2024-09-10T10:51:04.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.847+0000 7fd508e00640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 0x7fd4e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:04.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.848+0000 7fd508e00640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 0x7fd4e406e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0034000 tx=0x7fd4f002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:04.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:04.852+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4f80590c0 con 0x7fd504072320 2024-09-10T10:51:04.923 INFO:tasks.ceph.mds.a.smithi028.stderr: -48> 2024-09-10T10:51:00.168+0000 7fd727600640 -1 log_channel(cluster) log [ERR] : dir 0x600 object missing on disk; some files may be lost (~mds0/stray0) 2024-09-10T10:51:04.923 INFO:tasks.ceph.mds.a.smithi028.stderr: -46> 2024-09-10T10:51:00.168+0000 7fd727600640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T10:51:04.923 INFO:tasks.ceph.mds.a.smithi028.stderr: -31> 2024-09-10T10:51:01.147+0000 7fd72c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:04.964 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:51:05.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.115+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fd50406bf90 con 0x7fd504072320 2024-09-10T10:51:05.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.656+0000 7fd501600640 1 -- 172.21.15.28:0/603830943 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v102) ==== 101+0+0 (secure 0 0 0) 0x7fd4f805d3c0 con 0x7fd504072320 2024-09-10T10:51:05.657 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 msgr2=0x7fd4e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 0x7fd4e406e2f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0034000 tx=0x7fd4f002e000 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 msgr2=0x7fd504134bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800ebe0 tx=0x7fd4f800c520 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 shutdown_connections 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4e406be30 0x7fd4e406e2f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd504072de0 0x7fd5040b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd504072700 0x7fd5040b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 --2- 172.21.15.28:0/603830943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd504072320 0x7fd504134bb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.659+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 >> 172.21.15.28:0/603830943 conn(0x7fd504067d70 msgr2=0x7fd50406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:05.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.660+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 shutdown_connections 2024-09-10T10:51:05.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.660+0000 7fd50c200640 1 -- 172.21.15.28:0/603830943 wait complete. 2024-09-10T10:51:05.680 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:05.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.758+0000 7fef5efd3f40 1 Processor -- start 2024-09-10T10:51:05.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.758+0000 7fef5efd3f40 1 -- start start 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2870b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2871584b0 0x55d287084300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d28702ab30 con 0x55d2870cc2d0 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2870cbdc0 con 0x55d2870865a0 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d287086b40 con 0x55d2871584b0 2024-09-10T10:51:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35174/0 (socket says 172.21.15.28:35174) 2024-09-10T10:51:05.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5b800640 1 -- 172.21.15.28:0/348163508 learned_addr learned my addr 172.21.15.28:0/348163508 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5cc00640 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2871584b0 0x55d287084300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.759+0000 7fef5c200640 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2870b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 -- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2871584b0 msgr2=0x55d287084300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2871584b0 0x55d287084300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 -- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 msgr2=0x55d2870b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2870b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 -- 172.21.15.28:0/348163508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d287157a20 con 0x55d2870cc2d0 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5b800640 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fef4800a440 tx=0x7fef48051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0110834bc22922d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5a400640 1 -- 172.21.15.28:0/348163508 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48063070 con 0x55d2870cc2d0 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5a400640 1 -- 172.21.15.28:0/348163508 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef480559d0 con 0x55d2870cc2d0 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.760+0000 7fef5a400640 1 -- 172.21.15.28:0/348163508 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48055da0 con 0x55d2870cc2d0 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 -- 172.21.15.28:0/348163508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 msgr2=0x55d2870aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fef4800a440 tx=0x7fef48051000 comp rx=0 tx=0).stop 2024-09-10T10:51:05.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 -- 172.21.15.28:0/348163508 shutdown_connections 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2871584b0 0x55d287084300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2870cc2d0 0x55d2870aed80 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/348163508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2870b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 -- 172.21.15.28:0/348163508 >> 172.21.15.28:0/348163508 conn(0x55d2870b0b90 msgr2=0x55d2870cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 -- 172.21.15.28:0/348163508 shutdown_connections 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 -- 172.21.15.28:0/348163508 wait complete. 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.761+0000 7fef5efd3f40 1 Processor -- start 2024-09-10T10:51:05.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 -- start start 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2871506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2870cc2d0 0x55d2870842c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d287157320 con 0x55d2871584b0 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d2871577d0 con 0x55d2870865a0 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.762+0000 7fef5efd3f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d2870832f0 con 0x55d2870cc2d0 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35176/0 (socket says 172.21.15.28:35176) 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 learned_addr learned my addr 172.21.15.28:0/1896845920 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2870cc2d0 0x55d2870842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2871506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2870cc2d0 msgr2=0x55d2870842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d2870cc2d0 0x55d2870842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 msgr2=0x55d2871506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d2870865a0 0x55d2871506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2870cc800 con 0x55d2871584b0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.763+0000 7fef5cc00640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fef54002190 tx=0x7fef5400d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.764+0000 7fef59000640 1 -- 172.21.15.28:0/1896845920 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54018070 con 0x55d2871584b0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.764+0000 7fef59000640 1 -- 172.21.15.28:0/1896845920 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef540060d0 con 0x55d2871584b0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.764+0000 7fef59000640 1 -- 172.21.15.28:0/1896845920 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54004520 con 0x55d2871584b0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.764+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d2870af650 con 0x55d2871584b0 2024-09-10T10:51:05.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.764+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2871530f0 con 0x55d2871584b0 2024-09-10T10:51:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.766+0000 7fef59000640 1 -- 172.21.15.28:0/1896845920 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef5401b070 con 0x55d2871584b0 2024-09-10T10:51:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef59000640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 0x7fef3c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef59000640 1 -- 172.21.15.28:0/1896845920 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7fef54089370 con 0x55d2871584b0 2024-09-10T10:51:05.768 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:05.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d287156e10 0x55d2871538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871f1af0 con 0x55d287156e10 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 0x7fef3c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.767+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d287156e10 0x55d2871538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.768+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 0x7fef3c06cd00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005fd0 tx=0x7fef4c00c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.768+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d287156e10 0x55d2871538e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.770+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v102'73 uv48 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.770+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2871f1ed0 con 0x55d287156e10 2024-09-10T10:51:05.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.772+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v102'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.772+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871f2300 con 0x55d287156e10 2024-09-10T10:51:05.775 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:05.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.774+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v102'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.774+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fe6a0 con 0x55d287156e10 2024-09-10T10:51:05.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.777+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.777+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fea40 con 0x55d287156e10 2024-09-10T10:51:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.779+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v102'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.780+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fd690 con 0x55d287156e10 2024-09-10T10:51:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.782+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.782+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fda80 con 0x55d287156e10 2024-09-10T10:51:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.785+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v102'79 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.785+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.790+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.790+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.792 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.791+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v102'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.791+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.794+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v102'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.794+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.799+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v102'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.799+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.800+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v102'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.800+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.803+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v102'85 uv59 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.803+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.808+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v102'86 uv86 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.808+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.810 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:05.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.809+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v102'87 uv87 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.809+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.812+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v102'88 uv88 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.812+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.814+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v102'89 uv89 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.814+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fde20 con 0x55d287156e10 2024-09-10T10:51:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.816+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v102'90 uv90 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.816+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.818+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v102'91 uv91 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.818+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.819+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v102'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.820 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:05.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.819+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871ffdf0 con 0x55d287156e10 2024-09-10T10:51:05.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.821+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [setxattr (30)] v102'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.821+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871ffdf0 con 0x55d287156e10 2024-09-10T10:51:05.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.822+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [setxattr (62)] v102'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.822+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.824+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [omap-set-header] v102'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.824+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.825+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 603.00000000 [create] v102'96 uv71 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.826+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.827+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 603.00000000 [delete] v102'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.827+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.830 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:05.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.829+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v102'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.829+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871ffdf0 con 0x55d287156e10 2024-09-10T10:51:05.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.830+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [setxattr (30)] v102'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.830+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871ffdf0 con 0x55d287156e10 2024-09-10T10:51:05.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.832+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [setxattr (62)] v102'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.832+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2871fede0 con 0x55d287156e10 2024-09-10T10:51:05.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.833+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [omap-set-header] v102'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287156e10 2024-09-10T10:51:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.833+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d2871ff2f0 0x55d2872016e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.833+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:30 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287201c20 con 0x55d2871ff2f0 2024-09-10T10:51:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.833+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d2871ff2f0 0x55d2872016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.834+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d2871ff2f0 0x55d2872016e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.836+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(30 1.00000000.inode [create] v102'67 uv49 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fef4c0188c0 con 0x55d2871ff2f0 2024-09-10T10:51:05.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.836+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:31 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287201fc0 con 0x55d2871ff2f0 2024-09-10T10:51:05.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.837+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(31 1.00000000.inode [delete] v102'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c0188c0 con 0x55d2871ff2f0 2024-09-10T10:51:05.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.837+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287202360 con 0x55d2871ff2f0 2024-09-10T10:51:05.840 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:05.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.839+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(32 1.00000000.inode [create] v102'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.839+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287202700 con 0x55d2871ff2f0 2024-09-10T10:51:05.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.840+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v102'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.840+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287202aa0 con 0x55d2871ff2f0 2024-09-10T10:51:05.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.842+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v102'71 uv71 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.842+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287202e40 con 0x55d2871ff2f0 2024-09-10T10:51:05.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.843+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(35 200.00000000 [create] v102'72 uv54 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.843+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2872031e0 con 0x55d2871ff2f0 2024-09-10T10:51:05.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.845+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(36 200.00000000 [delete] v102'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.845+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.847 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:05.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.846+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(37 200.00000000 [create] v102'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.847+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.848+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(38 200.00000000 [write 0~90] v102'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.848+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.850+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(39 200.00000000 [omap-set-header] v102'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.850+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.851+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(40 200.00000001 [create] v102'77 uv59 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.851+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.852+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(41 200.00000001 [delete] v102'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.852+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.854 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:05.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.853+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(42 200.00000001 [create] v102'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.853+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287207c70 con 0x55d2871ff2f0 2024-09-10T10:51:05.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.855+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(43 200.00000001 [write 0~18118] v102'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.855+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287207c70 con 0x55d2871ff2f0 2024-09-10T10:51:05.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.857+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(44 200.00000001 [omap-set-header] v102'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.857+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.858+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(45 606.00000000 [create] v102'82 uv65 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.858+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.860+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(46 606.00000000 [delete] v102'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.860+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.862 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:05.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.862+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(47 606.00000000 [create] v102'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.862+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287204590 con 0x55d2871ff2f0 2024-09-10T10:51:05.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.863+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v102'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.863+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287204590 con 0x55d2871ff2f0 2024-09-10T10:51:05.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.864+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v102'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.864+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287203580 con 0x55d2871ff2f0 2024-09-10T10:51:05.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.865+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(50 606.00000000 [omap-set-header] v102'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d2871ff2f0 2024-09-10T10:51:05.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.865+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d287203c50 0x55d287204030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.866+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.866+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d287203c50 0x55d287204030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.866+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d287203c50 0x55d287204030 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.868+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(51 607.00000000 [create] v102'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.868+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.869+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(52 607.00000000 [delete] v102'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.869+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.871 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:05.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.870+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(53 607.00000000 [create] v102'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.871+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.872+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v102'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.872+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.873+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v102'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.873+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872069a0 con 0x55d287203c50 2024-09-10T10:51:05.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.874+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v102'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.874+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d287207280 0x55d2872076e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.874+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720a040 con 0x55d287207280 2024-09-10T10:51:05.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.875+0000 7fef5cc00640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d287207280 0x55d2872076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.875+0000 7fef5cc00640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d287207280 0x55d2872076e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.876+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v102'14 uv13 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.876+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720a3e0 con 0x55d287207280 2024-09-10T10:51:05.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.878+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v102'15 uv15 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.878+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720a780 con 0x55d287207280 2024-09-10T10:51:05.879 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:05.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.879+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v102'16 uv16 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.879+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2870ce9d0 con 0x55d287207280 2024-09-10T10:51:05.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.880+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v102'17 uv17 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.880+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b020 con 0x55d287203c50 2024-09-10T10:51:05.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.881+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(61 608.00000000 [create] v102'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.881+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b020 con 0x55d287203c50 2024-09-10T10:51:05.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.882+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(62 608.00000000 [delete] v102'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.882+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b020 con 0x55d287203c50 2024-09-10T10:51:05.884 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:05.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.884+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(63 608.00000000 [create] v102'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.884+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720c030 con 0x55d287203c50 2024-09-10T10:51:05.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.885+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v102'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.885+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720c030 con 0x55d287203c50 2024-09-10T10:51:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.886+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v102'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.886+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b020 con 0x55d287203c50 2024-09-10T10:51:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.887+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v102'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.887+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b020 con 0x55d287207280 2024-09-10T10:51:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.888+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(67 604.00000000 [create] v102'26 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.888+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b3e0 con 0x55d287207280 2024-09-10T10:51:05.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.890+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(68 604.00000000 [delete] v102'27 uv27 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.890+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720b7d0 con 0x55d287207280 2024-09-10T10:51:05.892 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:05.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.891+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(69 604.00000000 [create] v102'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.891+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bbc0 con 0x55d287207280 2024-09-10T10:51:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.893+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v102'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.893+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bbc0 con 0x55d287207280 2024-09-10T10:51:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.894+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v102'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.894+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bff0 con 0x55d287207280 2024-09-10T10:51:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.895+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v102'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.895+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bbc0 con 0x55d287207280 2024-09-10T10:51:05.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.896+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(73 500.00000000 [create] v102'32 uv25 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.897+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bbc0 con 0x55d287207280 2024-09-10T10:51:05.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.898+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(74 500.00000000 [delete] v102'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.898+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bff0 con 0x55d287207280 2024-09-10T10:51:05.900 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:05.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.899+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(75 500.00000000 [create] v102'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.899+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287207280 2024-09-10T10:51:05.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.900+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v102'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.900+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287207280 2024-09-10T10:51:05.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.902+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v102'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.902+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287203c50 2024-09-10T10:51:05.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.903+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(78 mds_snaptable [create] v102'12 uv11 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.903+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287203c50 2024-09-10T10:51:05.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.904+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(79 mds_snaptable [delete] v102'13 uv13 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.904+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287203c50 2024-09-10T10:51:05.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:05.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.905+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(80 mds_snaptable [create] v102'14 uv14 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.905+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287203c50 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.906+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v102'15 uv15 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.906+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720bb70 con 0x55d287203c50 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.908+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v102'16 uv16 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d287203c50 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.908+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d28720c920 0x55d28720ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.908+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720c040 con 0x55d28720c920 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.908+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d28720c920 0x55d28720ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.909+0000 7fef5c200640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d28720c920 0x55d28720ed10 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.910+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v102'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.910+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f290 con 0x55d28720c920 2024-09-10T10:51:05.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.911+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v102'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.911+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.913 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:05.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.913+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v102'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.913+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872106c0 con 0x55d28720c920 2024-09-10T10:51:05.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.915+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v102'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.915+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872106c0 con 0x55d28720c920 2024-09-10T10:51:05.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.916+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v102'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.916+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.918+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v102'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.918+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.919+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v102'24 uv18 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.919+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.921+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v102'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.921+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.922+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v102'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.923 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:05.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.922+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.924+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v102'27 uv27 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.924+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.926+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v102'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.926+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.927+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v102'29 uv23 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.927+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.929+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v102'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.929+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.930+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v102'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.931 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:05.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.930+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720f6b0 con 0x55d28720c920 2024-09-10T10:51:05.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.932+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v102'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.932+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d2872124c0 con 0x55d28720c920 2024-09-10T10:51:05.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.933+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v102'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.933+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.935+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v102'14 uv12 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.935+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.936+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v102'15 uv15 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.936+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.938 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:05.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.938+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v102'16 uv16 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.938+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.939+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v102'17 uv17 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.939+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.940+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v102'15 uv14 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.940+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.941+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v102'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.941+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:05.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.943+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v102'17 uv17 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.943+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b70 con 0x55d287207280 2024-09-10T10:51:05.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.944+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v102'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.944+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b70 con 0x55d287207280 2024-09-10T10:51:05.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.945+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v102'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.945+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d28720fb60 con 0x55d287207280 2024-09-10T10:51:05.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.946+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v102'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef5401a030 con 0x55d287207280 2024-09-10T10:51:05.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.946+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d2872101e0 0x55d2872105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:05.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.946+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d2872101e0 2024-09-10T10:51:05.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.946+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d2872101e0 0x55d2872105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:05.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.947+0000 7fef5b800640 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d2872101e0 0x55d2872105c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:05.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.948+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v102'12 uv11 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d2872101e0 2024-09-10T10:51:05.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.948+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d2872101e0 2024-09-10T10:51:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.949+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v102'13 uv13 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d2872101e0 2024-09-10T10:51:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.949+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d2872101e0 2024-09-10T10:51:05.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.950+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v102'14 uv14 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d2872101e0 2024-09-10T10:51:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.950+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d2872101e0 2024-09-10T10:51:05.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.951+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v102'15 uv15 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d2872101e0 2024-09-10T10:51:05.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.951+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d2872101e0 2024-09-10T10:51:05.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.952+0000 7fef5b800640 1 -- 172.21.15.28:0/1896845920 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v102'16 uv16 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fef48055d00 con 0x55d2872101e0 2024-09-10T10:51:05.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.952+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.953+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v102'33 uv24 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.953+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.955+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v102'34 uv34 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.955+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.957 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:05.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.956+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v102'35 uv35 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.956+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.957+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v102'36 uv36 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.957+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.958+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v102'37 uv37 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.958+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.959+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v102'38 uv31 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.959+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.961+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v102'39 uv39 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.961+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.962+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v102'40 uv40 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.963 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:05.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.962+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287211f60 con 0x55d28720c920 2024-09-10T10:51:05.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.963+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v102'41 uv41 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.963+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.964+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v102'42 uv42 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.964+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210b00 con 0x55d28720c920 2024-09-10T10:51:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.965+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v102'43 uv43 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.965+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d287210f50 con 0x55d28720c920 2024-09-10T10:51:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5c200640 1 -- 172.21.15.28:0/1896845920 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v102'44 uv44 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fef4c021040 con 0x55d28720c920 2024-09-10T10:51:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d287203c50 msgr2=0x55d287204030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d287203c50 0x55d287204030 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d28720c920 msgr2=0x55d28720ed10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d28720c920 0x55d28720ed10 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d2872101e0 msgr2=0x55d2872105c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d2872101e0 0x55d2872105c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d287207280 msgr2=0x55d2872076e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d287207280 0x55d2872076e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d2871ff2f0 msgr2=0x55d2872016e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d2871ff2f0 0x55d2872016e0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.966+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 reap_dead start 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d287156e10 msgr2=0x55d2871538e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d287156e10 0x55d2871538e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 reap_dead start 2024-09-10T10:51:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5cc00640 1 -- 172.21.15.28:0/1896845920 reap_dead start 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 msgr2=0x7fef3c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 0x7fef3c06cd00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fef4c005fd0 tx=0x7fef4c00c040 comp rx=0 tx=0).stop 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 msgr2=0x55d287084800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fef54002190 tx=0x7fef5400d530 comp rx=0 tx=0).stop 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 shutdown_connections 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef3c06a840 0x7fef3c06cd00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 --2- 172.21.15.28:0/1896845920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d2871584b0 0x55d287084800 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 >> 172.21.15.28:0/1896845920 conn(0x55d2870b0b90 msgr2=0x55d2870b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 shutdown_connections 2024-09-10T10:51:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:05.967+0000 7fef5efd3f40 1 -- 172.21.15.28:0/1896845920 wait complete. 2024-09-10T10:51:05.972 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000.inode 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.046+0000 7fa9f4d5ff40 1 Processor -- start 2024-09-10T10:51:06.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.046+0000 7fa9f4d5ff40 1 -- start start 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b39ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6b40360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6ab1530 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6a0eb30 con 0x55f8c6ab08e0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f8c6b3eee0 con 0x55f8c6b3f9c0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35186/0 (socket says 172.21.15.28:35186) 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.047+0000 7fa9f1600640 1 -- 172.21.15.28:0/288919311 learned_addr learned my addr 172.21.15.28:0/288919311 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f2000640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b39ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f2a00640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6b40360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 -- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 msgr2=0x55f8c6b40360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6b40360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 -- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 msgr2=0x55f8c6b39ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b39ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 -- 172.21.15.28:0/288919311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c6b3cb40 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f2000640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b39ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9f1600640 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000a770 tx=0x7fa9e000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21ae4721eaf126c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9ebe00640 1 -- 172.21.15.28:0/288919311 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0019070 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9ebe00640 1 -- 172.21.15.28:0/288919311 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e0007600 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.048+0000 7fa9ebe00640 1 -- 172.21.15.28:0/288919311 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0007a10 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/288919311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 msgr2=0x55f8c6b3fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000a770 tx=0x7fa9e000a860 comp rx=0 tx=0).stop 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/288919311 shutdown_connections 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6b40360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6b3fe20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/288919311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b39ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/288919311 >> 172.21.15.28:0/288919311 conn(0x55f8c6ab50a0 msgr2=0x55f8c6ab5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/288919311 shutdown_connections 2024-09-10T10:51:06.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.049+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/288919311 wait complete. 2024-09-10T10:51:06.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.050+0000 7fa9f4d5ff40 1 Processor -- start 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.050+0000 7fa9f4d5ff40 1 -- start start 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b2d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6a97c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6a981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6a663c0 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f8c6a96630 con 0x55f8c6ab08e0 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f4d5ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f8c6b3cdb0 con 0x55f8c6b3f9c0 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6a97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6a97c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35202/0 (socket says 172.21.15.28:35202) 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 -- 172.21.15.28:0/3313331207 learned_addr learned my addr 172.21.15.28:0/3313331207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:06.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f2000640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b2d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f2a00640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6a981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 -- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 msgr2=0x55f8c6a981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f8c6b3f9c0 0x55f8c6a981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 -- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 msgr2=0x55f8c6b2d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b2d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f1600640 1 -- 172.21.15.28:0/3313331207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c6a963c0 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.051+0000 7fa9f2000640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f8c6ab08e0 0x55f8c6b2d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9f1600640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6a97c90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0013000 tx=0x7fa9e0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9eaa00640 1 -- 172.21.15.28:0/3313331207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0022020 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9eaa00640 1 -- 172.21.15.28:0/3313331207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e001a040 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9eaa00640 1 -- 172.21.15.28:0/3313331207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0019070 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f8c6b41d00 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.052+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c6b422c0 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.054+0000 7fa9eaa00640 1 -- 172.21.15.28:0/3313331207 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9e0025020 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.055+0000 7fa9eaa00640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa9d406a840 0x7fa9d406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.055+0000 7fa9eaa00640 1 -- 172.21.15.28:0/3313331207 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7fa9e008ac20 con 0x55f8c6b3c3e0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.055+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f8c6b42eb0 0x55f8c6a993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.055+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e102) -- 0x55f8c6a99900 con 0x55f8c6b42eb0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.055+0000 7fa9f2000640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa9d406a840 0x7fa9d406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.056+0000 7fa9f2a00640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f8c6b42eb0 0x55f8c6a993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.056+0000 7fa9f2000640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa9d406a840 0x7fa9d406cd00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4057000 tx=0x7fa9e4052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.056+0000 7fa9f2a00640 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f8c6b42eb0 0x55f8c6a993c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f2a00640 1 -- 172.21.15.28:0/3313331207 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [delete] v102'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa9ec001630 con 0x55f8c6b42eb0 2024-09-10T10:51:06.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f8c6b42eb0 msgr2=0x55f8c6a993c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f8c6b42eb0 0x55f8c6a993c0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa9d406a840 msgr2=0x7fa9d406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa9d406a840 0x7fa9d406cd00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4057000 tx=0x7fa9e4052000 comp rx=0 tx=0).stop 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 msgr2=0x55f8c6a97c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 --2- 172.21.15.28:0/3313331207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f8c6b3c3e0 0x55f8c6a97c90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0013000 tx=0x7fa9e0003020 comp rx=0 tx=0).stop 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f2a00640 1 -- 172.21.15.28:0/3313331207 reap_dead start 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 shutdown_connections 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 >> 172.21.15.28:0/3313331207 conn(0x55f8c6ab50a0 msgr2=0x55f8c6ab5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 shutdown_connections 2024-09-10T10:51:06.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.058+0000 7fa9f4d5ff40 1 -- 172.21.15.28:0/3313331207 wait complete. 2024-09-10T10:51:06.064 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.221+0000 7f1b15400640 1 Processor -- start 2024-09-10T10:51:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.221+0000 7f1b15400640 1 -- start start 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.222+0000 7f1b15400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b10159ce0 0x7f1b1015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.222+0000 7f1b15400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b100b5f10 0x7f1b100b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.222+0000 7f1b15400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b100b6850 con 0x7f1b1006e6c0 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.222+0000 7f1b15400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1b1015d730 con 0x7f1b100b5f10 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.222+0000 7f1b15400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b100700b0 con 0x7f1b10159ce0 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b100b5f10 0x7f1b100b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b100b5f10 0x7f1b100b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35216/0 (socket says 172.21.15.28:35216) 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 -- 172.21.15.28:0/1412466043 learned_addr learned my addr 172.21.15.28:0/1412466043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0e800640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b10159ce0 0x7f1b1015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 -- 172.21.15.28:0/1412466043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b10159ce0 msgr2=0x7f1b1015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b10159ce0 0x7f1b1015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 -- 172.21.15.28:0/1412466043 >> v1:172.21.15.138:6789/0 conn(0x7f1b1006e6c0 legacy=0x7f1b1006eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.223+0000 7f1b0de00640 1 -- 172.21.15.28:0/1412466043 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b101601b0 con 0x7f1b100b5f10 2024-09-10T10:51:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.224+0000 7f1b0de00640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b6310 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009870 tx=0x7f1afc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2f1c271f09cf57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.224+0000 7f1b0d400640 1 -- 172.21.15.28:0/1412466043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc037070 con 0x7f1b100b5f10 2024-09-10T10:51:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.224+0000 7f1b0d400640 1 -- 172.21.15.28:0/1412466043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc032440 con 0x7f1b100b5f10 2024-09-10T10:51:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.224+0000 7f1b0d400640 1 -- 172.21.15.28:0/1412466043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc032810 con 0x7f1b100b5f10 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 -- 172.21.15.28:0/1412466043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 msgr2=0x7f1b100b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b6310 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1afc009870 tx=0x7f1afc02a130 comp rx=0 tx=0).stop 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 -- 172.21.15.28:0/1412466043 shutdown_connections 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b6310 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 --2- 172.21.15.28:0/1412466043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b10159ce0 0x7f1b1015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 -- 172.21.15.28:0/1412466043 >> 172.21.15.28:0/1412466043 conn(0x7f1b10073590 msgr2=0x7f1b1006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 -- 172.21.15.28:0/1412466043 shutdown_connections 2024-09-10T10:51:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.225+0000 7f1b15400640 1 -- 172.21.15.28:0/1412466043 wait complete. 2024-09-10T10:51:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.227+0000 7f1b15400640 1 Processor -- start 2024-09-10T10:51:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.227+0000 7f1b15400640 1 -- start start 2024-09-10T10:51:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b1006e6c0 0x7f1b101518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b10159ce0 0x7f1b101f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10160a50 con 0x7f1b100b5f10 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b10160770 con 0x7f1b10159ce0 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b15400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b1015a0c0 con 0x7f1b1006e6c0 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0e800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b1006e6c0 0x7f1b101518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35228/0 (socket says 172.21.15.28:35228) 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 -- 172.21.15.28:0/3158958756 learned_addr learned my addr 172.21.15.28:0/3158958756 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0f200640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b10159ce0 0x7f1b101f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 -- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b1006e6c0 msgr2=0x7f1b101518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b1006e6c0 0x7f1b101518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 -- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b10159ce0 msgr2=0x7f1b101f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b10159ce0 0x7f1b101f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.228+0000 7f1b0de00640 1 -- 172.21.15.28:0/3158958756 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b10152fa0 con 0x7f1b100b5f10 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b0de00640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1afc02a170 tx=0x7f1afc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc037040 con 0x7f1b100b5f10 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc042070 con 0x7f1b100b5f10 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b10153280 con 0x7f1b100b5f10 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc030880 con 0x7f1b100b5f10 2024-09-10T10:51:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.229+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b10155ea0 con 0x7f1b100b5f10 2024-09-10T10:51:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.230+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1015fda0 con 0x7f1b100b5f10 2024-09-10T10:51:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.231+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1afc030a20 con 0x7f1b100b5f10 2024-09-10T10:51:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.232+0000 7f1b06a00640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 0x7f1af006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.232+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6359+0+0 (secure 0 0 0) 0x7f1afc0ab810 con 0x7f1b100b5f10 2024-09-10T10:51:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.232+0000 7f1b0e800640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 0x7f1af006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.233+0000 7f1b0e800640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 0x7f1af006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009b40 tx=0x7f1b00009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.236+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1afc0798e0 con 0x7f1b100b5f10 2024-09-10T10:51:06.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.456+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1b10067d60 con 0x7f1b100b5f10 2024-09-10T10:51:06.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.667+0000 7f1b06a00640 1 -- 172.21.15.28:0/3158958756 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) ==== 162+0+0 (secure 0 0 0) 0x7f1afc07dbe0 con 0x7f1b100b5f10 2024-09-10T10:51:06.668 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 msgr2=0x7f1af006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 0x7f1af006e2f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009b40 tx=0x7f1b00009190 comp rx=0 tx=0).stop 2024-09-10T10:51:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 msgr2=0x7f1b100b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1afc02a170 tx=0x7f1afc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 shutdown_connections 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1af006be30 0x7f1af006e2f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b10159ce0 0x7f1b101f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b100b5f10 0x7f1b100b4010 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 --2- 172.21.15.28:0/3158958756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b1006e6c0 0x7f1b101518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 >> 172.21.15.28:0/3158958756 conn(0x7f1b10073590 msgr2=0x7f1b1015efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 shutdown_connections 2024-09-10T10:51:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.669+0000 7f1b15400640 1 -- 172.21.15.28:0/3158958756 wait complete. 2024-09-10T10:51:06.680 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:06.679+0000 7f12abe00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T10:51:06.680 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:06.679+0000 7f12abe00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:51:06.680 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:06.680+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:06.681 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:06.680+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:06.688 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:06.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.845+0000 7f47c5600640 1 Processor -- start 2024-09-10T10:51:06.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.845+0000 7f47c5600640 1 -- start start 2024-09-10T10:51:06.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c5600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47c015e6a0 0x7f47c006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c5600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c00d4aa0 con 0x7f47c006dd60 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c5600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f47c00cefb0 con 0x7f47c015e6a0 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c5600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00d2c20 con 0x7f47c00d4180 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47c015e6a0 0x7f47c006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c4c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f47c006dd60 0x7f47c00d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34020/0 (socket says 172.21.15.28:34020) 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.846+0000 7f47c4c00640 1 -- 172.21.15.28:0/2627323637 learned_addr learned my addr 172.21.15.28:0/2627323637 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:06.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47bfe00640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47b7e00640 1 -- 172.21.15.28:0/2627323637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 msgr2=0x7f47c00d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47b7e00640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47b7e00640 1 -- 172.21.15.28:0/2627323637 >> v1:172.21.15.28:6789/0 conn(0x7f47c006dd60 legacy=0x7f47c00d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47b7e00640 1 -- 172.21.15.28:0/2627323637 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c006e390 con 0x7f47c015e6a0 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.847+0000 7f47bf400640 1 -- 172.21.15.28:0/2627323637 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422638952 0 0) 0x7f47c00d4aa0 con 0x7f47c006dd60 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.848+0000 7f47b7e00640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c015e6a0 0x7f47c006d820 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f47a80097a0 tx=0x7f47a802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5729b1fe88240f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.848+0000 7f47bf400640 1 -- 172.21.15.28:0/2627323637 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8037070 con 0x7f47c015e6a0 2024-09-10T10:51:06.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.848+0000 7f47bf400640 1 -- 172.21.15.28:0/2627323637 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a802ae50 con 0x7f47c015e6a0 2024-09-10T10:51:06.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.848+0000 7f47bf400640 1 -- 172.21.15.28:0/2627323637 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a8032660 con 0x7f47c015e6a0 2024-09-10T10:51:06.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 -- 172.21.15.28:0/2627323637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c015e6a0 msgr2=0x7f47c006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c015e6a0 0x7f47c006d820 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f47a80097a0 tx=0x7f47a802a100 comp rx=0 tx=0).stop 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 -- 172.21.15.28:0/2627323637 shutdown_connections 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c015e6a0 0x7f47c006d820 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 --2- 172.21.15.28:0/2627323637 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 -- 172.21.15.28:0/2627323637 >> 172.21.15.28:0/2627323637 conn(0x7f47c009d6f0 msgr2=0x7f47c008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 -- 172.21.15.28:0/2627323637 shutdown_connections 2024-09-10T10:51:06.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.849+0000 7f47c5600640 1 -- 172.21.15.28:0/2627323637 wait complete. 2024-09-10T10:51:06.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.850+0000 7f47c5600640 1 Processor -- start 2024-09-10T10:51:06.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.851+0000 7f47c5600640 1 -- start start 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.851+0000 7f47c5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.851+0000 7f47c5600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.851+0000 7f47c5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c015e6a0 0x7f47c00b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.851+0000 7f47c5600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00cebd0 con 0x7f47c006dd60 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47c5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47c006e950 con 0x7f47c015e6a0 2024-09-10T10:51:06.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47c5600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00d0450 con 0x7f47c00d4180 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35254/0 (socket says 172.21.15.28:35254) 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 -- 172.21.15.28:0/4153991398 learned_addr learned my addr 172.21.15.28:0/4153991398 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47b7e00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47c4c00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c015e6a0 0x7f47c00b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 -- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 msgr2=0x7f47c00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 -- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c015e6a0 msgr2=0x7f47c00b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c015e6a0 0x7f47c00b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 -- 172.21.15.28:0/4153991398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c01f74f0 con 0x7f47c006dd60 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47b7e00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:06.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.852+0000 7f47bfe00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f47b000d830 tx=0x7f47b000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.853+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000b6c0 con 0x7f47c006dd60 2024-09-10T10:51:06.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.853+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b0019070 con 0x7f47c006dd60 2024-09-10T10:51:06.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.853+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ba80 con 0x7f47c006dd60 2024-09-10T10:51:06.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.853+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c01f77d0 con 0x7f47c006dd60 2024-09-10T10:51:06.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.853+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01f7d90 con 0x7f47c006dd60 2024-09-10T10:51:06.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.854+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c006d820 con 0x7f47c006dd60 2024-09-10T10:51:06.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.855+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f47b001e460 con 0x7f47c006dd60 2024-09-10T10:51:06.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.856+0000 7f47bcc00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 0x7f479c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:06.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.856+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7f47b008b830 con 0x7f47c006dd60 2024-09-10T10:51:06.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.856+0000 7f47b7e00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 0x7f479c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:06.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.857+0000 7f47b7e00640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 0x7f479c06e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f47a8034000 tx=0x7f47a802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:06.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:06.860+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f47b0059ef0 con 0x7f47c006dd60 2024-09-10T10:51:07.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.080+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47c00b3d70 con 0x7f47c006dd60 2024-09-10T10:51:07.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.080+0000 7f47bcc00640 1 -- 172.21.15.28:0/4153991398 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7f47b00600c0 con 0x7f47c006dd60 2024-09-10T10:51:07.081 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:07.082 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":105,"btime":"2024-09-10T10:51:06:680721+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:06.680637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:07.082 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 105 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 msgr2=0x7f479c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 0x7f479c06e2f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f47a8034000 tx=0x7f47a802e000 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 msgr2=0x7f47c00b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f47b000d830 tx=0x7f47b000dcf0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 shutdown_connections 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f479c06be30 0x7f479c06e2f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47c015e6a0 0x7f47c00b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47c00d4180 0x7f47c00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 --2- 172.21.15.28:0/4153991398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47c006dd60 0x7f47c00b87c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.082+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 >> 172.21.15.28:0/4153991398 conn(0x7f47c009d6f0 msgr2=0x7f47c00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:07.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.083+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 shutdown_connections 2024-09-10T10:51:07.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.083+0000 7f47c5600640 1 -- 172.21.15.28:0/4153991398 wait complete. 2024-09-10T10:51:07.099 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:07.099 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:07.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.256+0000 7fe58e600640 1 Processor -- start 2024-09-10T10:51:07.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.256+0000 7fe58e600640 1 -- start start 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.257+0000 7fe58e600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe5880cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.257+0000 7fe58e600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe588072de0 0x7fe58806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.257+0000 7fe58e600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe588146140 con 0x7fe588072700 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.257+0000 7fe58e600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe5880cd740 con 0x7fe588072de0 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.257+0000 7fe58e600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe5880cda20 con 0x7fe588072320 2024-09-10T10:51:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe588072de0 0x7fe58806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe588072de0 0x7fe58806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35268/0 (socket says 172.21.15.28:35268) 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 -- 172.21.15.28:0/1913931793 learned_addr learned my addr 172.21.15.28:0/1913931793 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe587400640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe5880cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 -- 172.21.15.28:0/1913931793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 msgr2=0x7fe5880cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe5880cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 -- 172.21.15.28:0/1913931793 >> v1:172.21.15.138:6789/0 conn(0x7fe588072700 legacy=0x7fe58806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:07.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.258+0000 7fe586a00640 1 -- 172.21.15.28:0/1913931793 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5880cdd00 con 0x7fe588072de0 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.259+0000 7fe586a00640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe58806e0b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe574009870 tx=0x7fe57402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3f0c1d776d5b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.259+0000 7fe586000640 1 -- 172.21.15.28:0/1913931793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574037070 con 0x7fe588072de0 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.259+0000 7fe586000640 1 -- 172.21.15.28:0/1913931793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe574032440 con 0x7fe588072de0 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.259+0000 7fe586000640 1 -- 172.21.15.28:0/1913931793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574032810 con 0x7fe588072de0 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 -- 172.21.15.28:0/1913931793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 msgr2=0x7fe58806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe58806e0b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe574009870 tx=0x7fe57402a180 comp rx=0 tx=0).stop 2024-09-10T10:51:07.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 -- 172.21.15.28:0/1913931793 shutdown_connections 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe58806e0b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 --2- 172.21.15.28:0/1913931793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe5880cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 -- 172.21.15.28:0/1913931793 >> 172.21.15.28:0/1913931793 conn(0x7fe588067d70 msgr2=0x7fe58806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 -- 172.21.15.28:0/1913931793 shutdown_connections 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.260+0000 7fe58e600640 1 -- 172.21.15.28:0/1913931793 wait complete. 2024-09-10T10:51:07.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.262+0000 7fe58e600640 1 Processor -- start 2024-09-10T10:51:07.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.262+0000 7fe58e600640 1 -- start start 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe58813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe588072700 0x7fe588142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe5880cd520 con 0x7fe588072de0 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5880cd3a0 con 0x7fe588072700 2024-09-10T10:51:07.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe58e600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe58806ed00 con 0x7fe588072320 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35274/0 (socket says 172.21.15.28:35274) 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 -- 172.21.15.28:0/1712455216 learned_addr learned my addr 172.21.15.28:0/1712455216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587400640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe58813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe586a00640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe588072700 0x7fe588142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 -- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 msgr2=0x7fe58813fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe58813fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 -- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe588072700 msgr2=0x7fe588142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.263+0000 7fe587e00640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe588072700 0x7fe588142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe587e00640 1 -- 172.21.15.28:0/1712455216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe588140290 con 0x7fe588072de0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe587e00640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c7b0 tx=0x7fe58000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580004b60 con 0x7fe588072de0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe580004d00 con 0x7fe588072de0 2024-09-10T10:51:07.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58000aa30 con 0x7fe588072de0 2024-09-10T10:51:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5881428a0 con 0x7fe588072de0 2024-09-10T10:51:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.264+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe588142e60 con 0x7fe588072de0 2024-09-10T10:51:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.265+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5880cce60 con 0x7fe588072de0 2024-09-10T10:51:07.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.266+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe58000abd0 con 0x7fe588072de0 2024-09-10T10:51:07.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.267+0000 7fe57f400640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.267+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7fe58008b200 con 0x7fe588072de0 2024-09-10T10:51:07.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.267+0000 7fe587400640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.268+0000 7fe587400640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 0x7fe56806e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.271+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe580020070 con 0x7fe588072de0 2024-09-10T10:51:07.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.491+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe588075790 con 0x7fe588072de0 2024-09-10T10:51:07.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.491+0000 7fe57f400640 1 -- 172.21.15.28:0/1712455216 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7fe580059270 con 0x7fe588072de0 2024-09-10T10:51:07.492 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:07.492 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":105,"btime":"2024-09-10T10:51:06:680721+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:06.680637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:07.493 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 105 2024-09-10T10:51:07.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 msgr2=0x7fe56806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 0x7fe56806e2f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe578006fd0 tx=0x7fe578008040 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 msgr2=0x7fe5881e2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fe58000c7b0 tx=0x7fe58000cc70 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 shutdown_connections 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe56806be30 0x7fe56806e2f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe588072de0 0x7fe5881e2850 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe588072700 0x7fe588142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 --2- 172.21.15.28:0/1712455216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe588072320 0x7fe58813fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 >> 172.21.15.28:0/1712455216 conn(0x7fe588067d70 msgr2=0x7fe5881449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.493+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 shutdown_connections 2024-09-10T10:51:07.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.494+0000 7fe58e600640 1 -- 172.21.15.28:0/1712455216 wait complete. 2024-09-10T10:51:07.510 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:07.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:07.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.665+0000 7fcccd600640 1 Processor -- start 2024-09-10T10:51:07.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.665+0000 7fcccd600640 1 -- start start 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.666+0000 7fcccd600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fccc815af50 0x7fccc8072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.666+0000 7fcccd600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc80b49c0 0x7fccc8073320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.666+0000 7fcccd600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccc806dc70 con 0x7fccc815b330 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fcccd600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fccc80b3f50 con 0x7fccc815af50 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fcccd600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccc806df10 con 0x7fccc80b49c0 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccbfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fccc815b330 0x7fccc80b42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34036/0 (socket says 172.21.15.28:34036) 2024-09-10T10:51:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccbfe00640 1 -- 172.21.15.28:0/473550069 learned_addr learned my addr 172.21.15.28:0/473550069 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccc7e00640 1 --2- 172.21.15.28:0/473550069 >> v2:172.21.15.138:3300/0 conn(0x7fccc815af50 0x7fccc8072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccccc00640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc80b49c0 0x7fccc8073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206553993 0 0) 0x7fccc806dc70 con 0x7fccc815b330 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.667+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccac003520 con 0x7fccc815b330 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.668+0000 7fccc7e00640 1 -- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc80b49c0 msgr2=0x7fccc8073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.668+0000 7fccc7e00640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc80b49c0 0x7fccc8073320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.668+0000 7fccc7e00640 1 -- 172.21.15.28:0/473550069 >> v1:172.21.15.28:6789/0 conn(0x7fccc815b330 legacy=0x7fccc80b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.668+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3530437745 0 0) 0x7fccac003520 con 0x7fccc815b330 2024-09-10T10:51:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.668+0000 7fccc7e00640 1 -- 172.21.15.28:0/473550069 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc806e1f0 con 0x7fccc815af50 2024-09-10T10:51:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.669+0000 7fccc7e00640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815af50 0x7fccc8072de0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097a0 tx=0x7fccb802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f228d4d464ba0d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.672+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8037070 con 0x7fccc815af50 2024-09-10T10:51:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.672+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb802aba0 con 0x7fccc815af50 2024-09-10T10:51:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.672+0000 7fccc7400640 1 -- 172.21.15.28:0/473550069 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb0001690 con 0x7fccc815af50 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 -- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815af50 msgr2=0x7fccc8072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815af50 0x7fccc8072de0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fccb80097a0 tx=0x7fccb802a110 comp rx=0 tx=0).stop 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 -- 172.21.15.28:0/473550069 shutdown_connections 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc80b49c0 0x7fccc8073320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 --2- 172.21.15.28:0/473550069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815af50 0x7fccc8072de0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 -- 172.21.15.28:0/473550069 >> 172.21.15.28:0/473550069 conn(0x7fccc806c240 msgr2=0x7fccc80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 -- 172.21.15.28:0/473550069 shutdown_connections 2024-09-10T10:51:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.673+0000 7fcccd600640 1 -- 172.21.15.28:0/473550069 wait complete. 2024-09-10T10:51:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.675+0000 7fcccd600640 1 Processor -- start 2024-09-10T10:51:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.675+0000 7fcccd600640 1 -- start start 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc815af50 0x7fccc81ee160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815b330 0x7fccc80cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fccc80adbc0 con 0x7fccc80b49c0 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccc806e7b0 con 0x7fccc815b330 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fcccd600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccc80add40 con 0x7fccc815af50 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fccc7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35286/0 (socket says 172.21.15.28:35286) 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fccc7e00640 1 -- 172.21.15.28:0/1421155978 learned_addr learned my addr 172.21.15.28:0/1421155978 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fccbfe00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc815af50 0x7fccc81ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.676+0000 7fccccc00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815b330 0x7fccc80cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 -- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc815af50 msgr2=0x7fccc81ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc815af50 0x7fccc81ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 -- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815b330 msgr2=0x7fccc80cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815b330 0x7fccc80cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 -- 172.21.15.28:0/1421155978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc81f4820 con 0x7fccc80b49c0 2024-09-10T10:51:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc7e00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fccb8035040 tx=0x7fccb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8037050 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb8043070 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81f4b00 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb8006e10 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.677+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81f50c0 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.678+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc81f5540 con 0x7fccc80b49c0 2024-09-10T10:51:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.679+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccb803ca10 con 0x7fccc80b49c0 2024-09-10T10:51:07.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.680+0000 7fccc4c00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 0x7fcca406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:07.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.681+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7fccb80aba30 con 0x7fccc80b49c0 2024-09-10T10:51:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.681+0000 7fccbfe00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 0x7fcca406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:07.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.682+0000 7fccbfe00640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 0x7fcca406e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fccc815a960 tx=0x7fccb0001e00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:07.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.685+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fccb8079c60 con 0x7fccc80b49c0 2024-09-10T10:51:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.909+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc80b42b0 con 0x7fccc80b49c0 2024-09-10T10:51:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.909+0000 7fccc4c00640 1 -- 172.21.15.28:0/1421155978 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+4388 (secure 0 0 0) 0x7fccb807df60 con 0x7fccc80b49c0 2024-09-10T10:51:07.910 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:07.910 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":105,"btime":"2024-09-10T10:51:06:680721+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6846,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/236564669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":236564669},{"type":"v1","addr":"172.21.15.28:6834","nonce":236564669}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:06.680637+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 105 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.911+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 msgr2=0x7fcca406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.911+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 0x7fcca406e2f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fccc815a960 tx=0x7fccb0001e00 comp rx=0 tx=0).stop 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.911+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 msgr2=0x7fccc80cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.911+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fccb8035040 tx=0x7fccb80046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 shutdown_connections 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcca406be30 0x7fcca406e2f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc815b330 0x7fccc80cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccc815af50 0x7fccc81ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 --2- 172.21.15.28:0/1421155978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccc80b49c0 0x7fccc80cbb00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 >> 172.21.15.28:0/1421155978 conn(0x7fccc806c240 msgr2=0x7fccc80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 shutdown_connections 2024-09-10T10:51:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:07.912+0000 7fcccd600640 1 -- 172.21.15.28:0/1421155978 wait complete. 2024-09-10T10:51:07.928 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000.inode' led to DAMAGED state 2024-09-10T10:51:07.928 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000000' 2024-09-10T10:51:07.929 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:07.929 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:07.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:07.965 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:07.965 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:08.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:08.041 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:08.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.240+0000 7f54ba000640 1 Processor -- start 2024-09-10T10:51:08.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.240+0000 7f54ba000640 1 -- start start 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.241+0000 7f54ba000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f54b4072320 0x7f54b40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.241+0000 7f54ba000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f54b415e1f0 0x7f54b406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.241+0000 7f54ba000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b4146140 con 0x7f54b4072de0 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54ba000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f54b40cd740 con 0x7f54b4072320 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54ba000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f54b40cda20 con 0x7f54b415e1f0 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f54b4072320 0x7f54b40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f54b4072320 0x7f54b40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35288/0 (socket says 172.21.15.28:35288) 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 -- 172.21.15.28:0/2132000578 learned_addr learned my addr 172.21.15.28:0/2132000578 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3e00640 1 --2- 172.21.15.28:0/2132000578 >> v2:172.21.15.138:3300/0 conn(0x7f54b415e1f0 0x7f54b406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 -- 172.21.15.28:0/2132000578 >> v2:172.21.15.138:3300/0 conn(0x7f54b415e1f0 msgr2=0x7f54b406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 --2- 172.21.15.28:0/2132000578 >> v2:172.21.15.138:3300/0 conn(0x7f54b415e1f0 0x7f54b406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 -- 172.21.15.28:0/2132000578 >> v1:172.21.15.28:6789/0 conn(0x7f54b4072de0 legacy=0x7f54b406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.242+0000 7f54b3400640 1 -- 172.21.15.28:0/2132000578 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b40cdd00 con 0x7f54b4072320 2024-09-10T10:51:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.243+0000 7f54b3400640 1 --2- 172.21.15.28:0/2132000578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b4072320 0x7f54b40cce60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f54b40cfce0 tx=0x7f54a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2b045213a8a5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54b2a00640 1 -- 172.21.15.28:0/2132000578 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4037070 con 0x7f54b4072320 2024-09-10T10:51:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54b2a00640 1 -- 172.21.15.28:0/2132000578 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54a4032440 con 0x7f54b4072320 2024-09-10T10:51:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54b2a00640 1 -- 172.21.15.28:0/2132000578 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4032810 con 0x7f54b4072320 2024-09-10T10:51:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 -- 172.21.15.28:0/2132000578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b4072320 msgr2=0x7f54b40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 --2- 172.21.15.28:0/2132000578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b4072320 0x7f54b40cce60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f54b40cfce0 tx=0x7f54a402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 -- 172.21.15.28:0/2132000578 shutdown_connections 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 --2- 172.21.15.28:0/2132000578 >> v2:172.21.15.138:3300/0 conn(0x7f54b415e1f0 0x7f54b406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 --2- 172.21.15.28:0/2132000578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b4072320 0x7f54b40cce60 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 -- 172.21.15.28:0/2132000578 >> 172.21.15.28:0/2132000578 conn(0x7f54b4067d70 msgr2=0x7f54b406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 -- 172.21.15.28:0/2132000578 shutdown_connections 2024-09-10T10:51:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.244+0000 7f54ba000640 1 -- 172.21.15.28:0/2132000578 wait complete. 2024-09-10T10:51:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.246+0000 7f54ba000640 1 Processor -- start 2024-09-10T10:51:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.246+0000 7f54ba000640 1 -- start start 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b4072320 0x7f54b413da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b4072de0 0x7f54b40b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54b40cd520 con 0x7f54b415e1f0 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f54b40cd3a0 con 0x7f54b4072320 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54ba000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f54b406ed00 con 0x7f54b4072de0 2024-09-10T10:51:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35298/0 (socket says 172.21.15.28:35298) 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 -- 172.21.15.28:0/3100068345 learned_addr learned my addr 172.21.15.28:0/3100068345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3400640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b4072320 0x7f54b413da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54abe00640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b4072de0 0x7f54b40b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 -- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b4072de0 msgr2=0x7f54b40b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b4072de0 0x7f54b40b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 -- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b4072320 msgr2=0x7f54b413da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.247+0000 7f54b3e00640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b4072320 0x7f54b413da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54b3e00640 1 -- 172.21.15.28:0/3100068345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b41f79d0 con 0x7f54b415e1f0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54b3e00640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f54ac007d80 tx=0x7f54ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac014420 con 0x7f54b415e1f0 2024-09-10T10:51:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ac019070 con 0x7f54b415e1f0 2024-09-10T10:51:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac014800 con 0x7f54b415e1f0 2024-09-10T10:51:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b41f7cb0 con 0x7f54b415e1f0 2024-09-10T10:51:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.248+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b41f8270 con 0x7f54b415e1f0 2024-09-10T10:51:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.249+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b4072700 con 0x7f54b415e1f0 2024-09-10T10:51:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.250+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54ac0149a0 con 0x7f54b415e1f0 2024-09-10T10:51:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.251+0000 7f54ab400640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.251+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7f54ac08b920 con 0x7f54b415e1f0 2024-09-10T10:51:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.251+0000 7f54b3400640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.252+0000 7f54b3400640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f54a4034000 tx=0x7f54a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.255+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54ac059f50 con 0x7f54b415e1f0 2024-09-10T10:51:08.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.473+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f54b40cc870 con 0x7f54b415e1f0 2024-09-10T10:51:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.674+0000 7f54ab400640 1 -- 172.21.15.28:0/3100068345 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f54ac059850 con 0x7f54b415e1f0 2024-09-10T10:51:08.675 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.676+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 msgr2=0x7f549006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.676+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f54a4034000 tx=0x7f54a402e000 comp rx=0 tx=0).stop 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.676+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 msgr2=0x7f54b41e7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.676+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f54ac007d80 tx=0x7f54ac00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 shutdown_connections 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f549006be30 0x7f549006e2f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b415e1f0 0x7f54b41e7b20 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f54b4072de0 0x7f54b40b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 --2- 172.21.15.28:0/3100068345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54b4072320 0x7f54b413da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 >> 172.21.15.28:0/3100068345 conn(0x7f54b4067d70 msgr2=0x7f54b4145600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:08.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 shutdown_connections 2024-09-10T10:51:08.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.677+0000 7f54ba000640 1 -- 172.21.15.28:0/3100068345 wait complete. 2024-09-10T10:51:08.694 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.850+0000 7ff7f7600640 1 Processor -- start 2024-09-10T10:51:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.850+0000 7ff7f7600640 1 -- start start 2024-09-10T10:51:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.851+0000 7ff7f7600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f01690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f7600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f006f480 con 0x7ff7f0147230 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f7600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7f006e900 con 0x7ff7f0147e90 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f7600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f00b34d0 con 0x7ff7f00727c0 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f4c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff7f0147230 0x7ff7f006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34054/0 (socket says 172.21.15.28:34054) 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f4c00640 1 -- 172.21.15.28:0/2719513856 learned_addr learned my addr 172.21.15.28:0/2719513856 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7f5600640 1 --2- 172.21.15.28:0/2719513856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f01690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 705601638 0 0) 0x7ff7f006f480 con 0x7ff7f0147230 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7d8003520 con 0x7ff7f0147230 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.852+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668715838 0 0) 0x7ff7f006e900 con 0x7ff7f0147e90 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.853+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7f006f480 con 0x7ff7f0147e90 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.853+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3033252728 0 0) 0x7ff7d8003520 con 0x7ff7f0147230 2024-09-10T10:51:08.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.853+0000 7ff7f5600640 1 -- 172.21.15.28:0/2719513856 >> v1:172.21.15.138:6789/0 conn(0x7ff7f0147e90 legacy=0x7ff7f00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.853+0000 7ff7f5600640 1 -- 172.21.15.28:0/2719513856 >> v1:172.21.15.28:6789/0 conn(0x7ff7f0147230 legacy=0x7ff7f006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.853+0000 7ff7f5600640 1 -- 172.21.15.28:0/2719513856 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f006ebe0 con 0x7ff7f00727c0 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.854+0000 7ff7f5600640 1 --2- 172.21.15.28:0/2719513856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f01690f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009dc0 tx=0x7ff7e802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87bd50751d18728 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.854+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4001670 con 0x7ff7f00727c0 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.854+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e8007540 con 0x7ff7f00727c0 2024-09-10T10:51:08.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.854+0000 7ff7ef400640 1 -- 172.21.15.28:0/2719513856 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e80078f0 con 0x7ff7f00727c0 2024-09-10T10:51:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 -- 172.21.15.28:0/2719513856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 msgr2=0x7ff7f01690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 --2- 172.21.15.28:0/2719513856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f01690f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009dc0 tx=0x7ff7e802a660 comp rx=0 tx=0).stop 2024-09-10T10:51:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 -- 172.21.15.28:0/2719513856 shutdown_connections 2024-09-10T10:51:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 --2- 172.21.15.28:0/2719513856 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f01690f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 -- 172.21.15.28:0/2719513856 >> 172.21.15.28:0/2719513856 conn(0x7ff7f0067d70 msgr2=0x7ff7f006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 -- 172.21.15.28:0/2719513856 shutdown_connections 2024-09-10T10:51:08.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.855+0000 7ff7f7600640 1 -- 172.21.15.28:0/2719513856 wait complete. 2024-09-10T10:51:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.857+0000 7ff7f7600640 1 Processor -- start 2024-09-10T10:51:08.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.857+0000 7ff7f7600640 1 -- start start 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.857+0000 7ff7f7600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f0161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f7600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f7600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f0147e90 0x7ff7f01e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f7600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f006f1a0 con 0x7ff7f0147230 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f7600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00afe20 con 0x7ff7f0147e90 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f7600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7f0147610 con 0x7ff7f00727c0 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35300/0 (socket says 172.21.15.28:35300) 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 -- 172.21.15.28:0/236054055 learned_addr learned my addr 172.21.15.28:0/236054055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:08.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f4c00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f0161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7f5600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f0147e90 0x7ff7f01e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 -- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 msgr2=0x7ff7f0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f0161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 -- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f0147e90 msgr2=0x7ff7f01e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f0147e90 0x7ff7f01e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.858+0000 7ff7efe00640 1 -- 172.21.15.28:0/236054055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f01619d0 con 0x7ff7f0147230 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7efe00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c460 tx=0x7ff7dc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0052d0 con 0x7ff7f0147230 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7dc005490 con 0x7ff7f0147230 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc012870 con 0x7ff7f0147230 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f0140920 con 0x7ff7f0147230 2024-09-10T10:51:08.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.859+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f015f690 con 0x7ff7f0147230 2024-09-10T10:51:08.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.860+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f006e900 con 0x7ff7f0147230 2024-09-10T10:51:08.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.861+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff7dc012a10 con 0x7ff7f0147230 2024-09-10T10:51:08.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.862+0000 7ff7ecc00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 0x7ff7d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:08.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.862+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7ff7dc08ad50 con 0x7ff7f0147230 2024-09-10T10:51:08.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.862+0000 7ff7f4c00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 0x7ff7d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:08.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.863+0000 7ff7f4c00640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 0x7ff7d006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40011e0 tx=0x7ff7e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:08.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:08.866+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7f006e900 con 0x7ff7f0147230 2024-09-10T10:51:09.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.123+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7ff7f006e0b0 con 0x7ff7f0147230 2024-09-10T10:51:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.681+0000 7ff7ecc00640 1 -- 172.21.15.28:0/236054055 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v107) ==== 101+0+0 (secure 0 0 0) 0x7ff7f01619d0 con 0x7ff7f0147230 2024-09-10T10:51:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.683+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 msgr2=0x7ff7d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.683+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 0x7ff7d006e2f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40011e0 tx=0x7ff7e4009040 comp rx=0 tx=0).stop 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.683+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 msgr2=0x7ff7f01403e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.683+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00c460 tx=0x7ff7dc00c920 comp rx=0 tx=0).stop 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 shutdown_connections 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff7d006be30 0x7ff7d006e2f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f0147e90 0x7ff7f01e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7f0147230 0x7ff7f01403e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 --2- 172.21.15.28:0/236054055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7f00727c0 0x7ff7f0161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 >> 172.21.15.28:0/236054055 conn(0x7ff7f0067d70 msgr2=0x7ff7f00b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 shutdown_connections 2024-09-10T10:51:09.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.684+0000 7ff7f7600640 1 -- 172.21.15.28:0/236054055 wait complete. 2024-09-10T10:51:09.703 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.778+0000 7f5d0a7ecf40 1 Processor -- start 2024-09-10T10:51:09.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.778+0000 7f5d0a7ecf40 1 -- start start 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b39c0 0x55d5954a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d595425530 con 0x55d5954b03e0 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d595382b30 con 0x55d5954b39c0 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d5954b2ee0 con 0x55d5953da470 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35324/0 (socket says 172.21.15.28:35324) 2024-09-10T10:51:09.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 -- 172.21.15.28:0/4102941880 learned_addr learned my addr 172.21.15.28:0/4102941880 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07a00640 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d08400640 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b39c0 0x55d5954a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 -- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 msgr2=0x55d5954269c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954269c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 -- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b39c0 msgr2=0x55d5954a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b39c0 0x55d5954a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.779+0000 7f5d07000640 1 -- 172.21.15.28:0/4102941880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5954a7840 con 0x55d5954b03e0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d07000640 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400a7a0 tx=0x7f5cf400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7711a3a0f1d671eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d05c00640 1 -- 172.21.15.28:0/4102941880 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4019070 con 0x55d5954b03e0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d05c00640 1 -- 172.21.15.28:0/4102941880 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf4007600 con 0x55d5954b03e0 2024-09-10T10:51:09.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d05c00640 1 -- 172.21.15.28:0/4102941880 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf40079d0 con 0x55d5954b03e0 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/4102941880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 msgr2=0x55d5954a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400a7a0 tx=0x7f5cf400a7d0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/4102941880 shutdown_connections 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b39c0 0x55d5954a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b03e0 0x55d5954a6330 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/4102941880 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.780+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/4102941880 >> 172.21.15.28:0/4102941880 conn(0x55d5954290a0 msgr2=0x55d5954b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.781+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/4102941880 shutdown_connections 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.781+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/4102941880 wait complete. 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.781+0000 7f5d0a7ecf40 1 Processor -- start 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.781+0000 7f5d0a7ecf40 1 -- start start 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b03e0 0x55d5954a1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5954a73b0 con 0x55d5954b39c0 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5954b2520 con 0x55d5954b03e0 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d0a7ecf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d595427e70 con 0x55d5953da470 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.782+0000 7f5d08400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35342/0 (socket says 172.21.15.28:35342) 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 learned_addr learned my addr 172.21.15.28:0/2983440943 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b03e0 0x55d5954a1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 msgr2=0x55d5954a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b03e0 msgr2=0x55d5954a1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5954b03e0 0x55d5954a1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5954a6fb0 con 0x55d5954b39c0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5953da470 0x55d5954a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.783+0000 7f5d08400640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007000 tx=0x7f5d00003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.784+0000 7f5cffe00640 1 -- 172.21.15.28:0/2983440943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0004f640 con 0x55d5954b39c0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.784+0000 7f5cffe00640 1 -- 172.21.15.28:0/2983440943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d00067070 con 0x55d5954b39c0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.784+0000 7f5cffe00640 1 -- 172.21.15.28:0/2983440943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00062660 con 0x55d5954b39c0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.784+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d5954a2390 con 0x55d5954b39c0 2024-09-10T10:51:09.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.784+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d59540d680 con 0x55d5954b39c0 2024-09-10T10:51:09.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.786+0000 7f5cffe00640 1 -- 172.21.15.28:0/2983440943 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d00061070 con 0x55d5954b39c0 2024-09-10T10:51:09.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5cffe00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 0x7f5ce806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5cffe00640 1 -- 172.21.15.28:0/2983440943 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7f5d000cdca0 con 0x55d5954b39c0 2024-09-10T10:51:09.788 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:09.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d5954b6500 0x55d5955556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595422a90 con 0x55d5954b6500 2024-09-10T10:51:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 0x7f5ce806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.787+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d5954b6500 0x55d5955556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.788+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 0x7f5ce806ccd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d5954afd90 tx=0x7f5cf8003050 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.788+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d5954b6500 0x55d5955556f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.790+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v103'102 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.790+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d5954b19c0 con 0x55d5954b6500 2024-09-10T10:51:09.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.792+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v103'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.792+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595555c30 con 0x55d5954b6500 2024-09-10T10:51:09.794 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:09.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.793+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v103'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.793+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556fe0 con 0x55d5954b6500 2024-09-10T10:51:09.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.795+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v103'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.795+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595557380 con 0x55d5954b6500 2024-09-10T10:51:09.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.797+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v103'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.797+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595555fd0 con 0x55d5954b6500 2024-09-10T10:51:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.798+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v103'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.798+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d5955563e0 con 0x55d5954b6500 2024-09-10T10:51:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.800+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v103'108 uv84 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.800+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d5955563e0 con 0x55d5954b6500 2024-09-10T10:51:09.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.802+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v103'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.802+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556390 con 0x55d5954b6500 2024-09-10T10:51:09.804 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:09.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.803+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v103'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.803+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556390 con 0x55d5954b6500 2024-09-10T10:51:09.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.805+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v103'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.805+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.806+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v103'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.806+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.808+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v103'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.808+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.810+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v103'114 uv91 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.810+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.812+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v103'115 uv115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.812+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.814 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:09.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.813+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v103'116 uv116 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.813+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.815+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v103'117 uv117 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.815+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.816+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v103'118 uv118 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.817+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.819+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v103'119 uv119 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.819+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.821+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v103'120 uv120 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.821+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.822+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v103'121 uv95 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.822+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.824+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v103'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.824+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.826 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:09.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.825+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v103'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.825+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.827+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v103'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.827+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.829+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v103'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.829+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.830+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v103'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.830+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.832+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v103'127 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.832+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.833+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v103'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.833+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.835 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:09.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.835+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v103'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.835+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.836+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v103'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.836+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.838+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v103'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.838+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d5954b6500 2024-09-10T10:51:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.840+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v103'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d5954b6500 2024-09-10T10:51:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.840+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d595557720 0x55d595559b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.840+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595556370 con 0x55d595557720 2024-09-10T10:51:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.840+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d595557720 0x55d595559b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.841+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d595557720 0x55d595559b10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.842+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v103'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.843 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.842+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555a050 con 0x55d595557720 2024-09-10T10:51:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.843+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [write 0~582] v103'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.843+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555a3f0 con 0x55d595557720 2024-09-10T10:51:09.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.845+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [omap-set-header] v103'91 uv91 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.845+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555a790 con 0x55d595557720 2024-09-10T10:51:09.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.846+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 200.00000000 [create] v103'92 uv76 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.846+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d59555ab30 con 0x55d595557720 2024-09-10T10:51:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.848+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 200.00000000 [delete] v103'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.848+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555aed0 con 0x55d595557720 2024-09-10T10:51:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.849+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v103'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.850 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.849+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b270 con 0x55d595557720 2024-09-10T10:51:09.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.850+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v103'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.850+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.852+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v103'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.852+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.853+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000001 [create] v103'97 uv81 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.853+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.855+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000001 [delete] v103'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.855+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.856+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v103'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.857 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:09.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.856+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555fd00 con 0x55d595557720 2024-09-10T10:51:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.858+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [write 0~18118] v103'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.858+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555fd00 con 0x55d595557720 2024-09-10T10:51:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.859+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v103'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.859+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.860+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 606.00000000 [create] v103'102 uv87 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.860+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.862+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 606.00000000 [delete] v103'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.862+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.864 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:09.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.863+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v103'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.863+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555c620 con 0x55d595557720 2024-09-10T10:51:09.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.864+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v103'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.864+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555c620 con 0x55d595557720 2024-09-10T10:51:09.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.865+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v103'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.865+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555b610 con 0x55d595557720 2024-09-10T10:51:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.867+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v103'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d595557720 2024-09-10T10:51:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.867+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d59555bb20 0x55d59555df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.867+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555e450 con 0x55d59555bb20 2024-09-10T10:51:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.867+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d59555bb20 0x55d59555df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.867+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d59555bb20 0x55d59555df10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.869+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(51 607.00000000 [create] v103'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.869+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d59555e4d0 con 0x55d59555bb20 2024-09-10T10:51:09.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.870+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(52 607.00000000 [delete] v103'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.870+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555e8f0 con 0x55d59555bb20 2024-09-10T10:51:09.872 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:09.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.872+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(53 607.00000000 [create] v103'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.872+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555f900 con 0x55d59555bb20 2024-09-10T10:51:09.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.873+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v103'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.873+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555f900 con 0x55d59555bb20 2024-09-10T10:51:09.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.875+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v103'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.875+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d59555e8f0 con 0x55d59555bb20 2024-09-10T10:51:09.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.876+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v103'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.876+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d59555ef70 0x55d595561360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.876+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595561910 con 0x55d59555ef70 2024-09-10T10:51:09.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.877+0000 7f5d08400640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d59555ef70 0x55d595561360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.877+0000 7f5d08400640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d59555ef70 0x55d595561360 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.878+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v103'18 uv17 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.878+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595561e20 con 0x55d59555ef70 2024-09-10T10:51:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.879+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v103'19 uv19 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.880+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562320 con 0x55d59555ef70 2024-09-10T10:51:09.881 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.881+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v103'20 uv20 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.881+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562be0 con 0x55d59555ef70 2024-09-10T10:51:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.882+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v103'21 uv21 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.882+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562f80 con 0x55d59555bb20 2024-09-10T10:51:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.883+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(61 608.00000000 [create] v103'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.883+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595562f80 con 0x55d59555bb20 2024-09-10T10:51:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.884+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(62 608.00000000 [delete] v103'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.884+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562f80 con 0x55d59555bb20 2024-09-10T10:51:09.886 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.885+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(63 608.00000000 [create] v103'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.885+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563f90 con 0x55d59555bb20 2024-09-10T10:51:09.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.886+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v103'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.886+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563f90 con 0x55d59555bb20 2024-09-10T10:51:09.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.887+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v103'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.887+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562f80 con 0x55d59555bb20 2024-09-10T10:51:09.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.888+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v103'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.888+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595562f80 con 0x55d59555ef70 2024-09-10T10:51:09.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.890+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(67 604.00000000 [create] v103'37 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.890+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595563320 con 0x55d59555ef70 2024-09-10T10:51:09.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.891+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(68 604.00000000 [delete] v103'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.891+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d5955636c0 con 0x55d59555ef70 2024-09-10T10:51:09.893 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:09.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.892+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(69 604.00000000 [create] v103'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.892+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595564a70 con 0x55d59555ef70 2024-09-10T10:51:09.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.894+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.894+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595564a70 con 0x55d59555ef70 2024-09-10T10:51:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.895+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.895+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.896+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.896+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.898+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(73 500.00000000 [create] v103'43 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.898+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.899+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(74 500.00000000 [delete] v103'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.899+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.901 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.901+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(75 500.00000000 [create] v103'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.901+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.902+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v103'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.902+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555ef70 2024-09-10T10:51:09.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.904+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v103'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.904+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555bb20 2024-09-10T10:51:09.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.905+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(78 mds_snaptable [create] v103'17 uv16 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.905+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555bb20 2024-09-10T10:51:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.906+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(79 mds_snaptable [delete] v103'18 uv18 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.906+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555bb20 2024-09-10T10:51:09.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.907+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(80 mds_snaptable [create] v103'19 uv19 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.907+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555bb20 2024-09-10T10:51:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.908+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v103'20 uv20 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.908+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595563a60 con 0x55d59555bb20 2024-09-10T10:51:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.909+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v103'21 uv21 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d59555bb20 2024-09-10T10:51:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.909+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5955640e0 0x55d5955664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.909+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566aa0 con 0x55d5955640e0 2024-09-10T10:51:09.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.909+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5955640e0 0x55d5955664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.910+0000 7f5d07a00640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5955640e0 0x55d5955664d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.911+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v103'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.911+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595566b60 con 0x55d5955640e0 2024-09-10T10:51:09.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.913+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v103'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.913+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.915 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:09.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.914+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v103'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.914+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567f90 con 0x55d5955640e0 2024-09-10T10:51:09.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.915+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v103'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.915+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567f90 con 0x55d5955640e0 2024-09-10T10:51:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.917+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v103'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.917+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.918+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v103'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.918+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.919+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v103'34 uv28 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.919+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.921+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v103'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.921+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.923 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.922+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v103'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.922+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.923+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v103'37 uv37 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.923+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.925+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v103'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.925+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.926+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v103'39 uv33 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.926+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.927+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v103'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.927+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:09.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.929+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v103'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.929+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595566f80 con 0x55d5955640e0 2024-09-10T10:51:09.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.930+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v103'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.930+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595569d90 con 0x55d5955640e0 2024-09-10T10:51:09.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.932+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v103'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.932+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.934+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v103'18 uv17 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.934+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.935+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v103'19 uv19 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.935+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.937+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v103'20 uv20 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.937 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:09.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.937+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.938+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v103'21 uv21 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.938+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.939+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v103'21 uv20 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.939+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.941+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v103'22 uv22 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.941+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.942 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.942+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v103'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.942+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.943+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v103'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.943+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568440 con 0x55d59555ef70 2024-09-10T10:51:09.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.944+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v103'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.944+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595567430 con 0x55d59555ef70 2024-09-10T10:51:09.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.945+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v103'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5d00060030 con 0x55d59555ef70 2024-09-10T10:51:09.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.945+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d595567d00 0x55d595568100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:09.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.945+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d595567d00 2024-09-10T10:51:09.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.945+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d595567d00 0x55d595568100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:09.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.946+0000 7f5d07000640 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d595567d00 0x55d595568100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:09.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.947+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v103'17 uv16 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d595567d00 2024-09-10T10:51:09.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.947+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d595567d00 2024-09-10T10:51:09.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.948+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v103'18 uv18 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d595567d00 2024-09-10T10:51:09.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.948+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d595567d00 2024-09-10T10:51:09.950 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:09.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.949+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v103'19 uv19 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d595567d00 2024-09-10T10:51:09.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.949+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d595567d00 2024-09-10T10:51:09.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.951+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v103'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d595567d00 2024-09-10T10:51:09.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.951+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d595567d00 2024-09-10T10:51:09.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.952+0000 7f5d07000640 1 -- 172.21.15.28:0/2983440943 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v103'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5cf401c040 con 0x55d595567d00 2024-09-10T10:51:09.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.952+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.953+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v103'45 uv37 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.953+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.954+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v103'46 uv46 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.954+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.956 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:09.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.955+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v103'47 uv47 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.955+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.957+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v103'48 uv48 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.957+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.958+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v103'49 uv49 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.958+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.959+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v103'50 uv44 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.959+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.960+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v103'51 uv51 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.960+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.962 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:09.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.961+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v103'52 uv52 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.961+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.962+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v103'53 uv53 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.962+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.963+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v103'54 uv54 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.963+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595568830 con 0x55d5955640e0 2024-09-10T10:51:09.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.964+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v103'55 uv55 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.964+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103) -- 0x55d595569100 con 0x55d5955640e0 2024-09-10T10:51:09.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d07a00640 1 -- 172.21.15.28:0/2983440943 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v103'56 uv56 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5cf800e040 con 0x55d5955640e0 2024-09-10T10:51:09.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d59555bb20 msgr2=0x55d59555df10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55d59555bb20 0x55d59555df10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5955640e0 msgr2=0x55d5955664d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5955640e0 0x55d5955664d0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d595567d00 msgr2=0x55d595568100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55d595567d00 0x55d595568100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d59555ef70 msgr2=0x55d595561360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55d59555ef70 0x55d595561360 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d595557720 msgr2=0x55d595559b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55d595557720 0x55d595559b10 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d5954b6500 msgr2=0x55d5955556f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55d5954b6500 0x55d5955556f0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 reap_dead start 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 reap_dead start 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 reap_dead start 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d08400640 1 -- 172.21.15.28:0/2983440943 reap_dead start 2024-09-10T10:51:09.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 msgr2=0x7f5ce806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 0x7f5ce806ccd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55d5954afd90 tx=0x7f5cf8003050 comp rx=0 tx=0).stop 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 msgr2=0x55d5954b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5d00007000 tx=0x7f5d00003330 comp rx=0 tx=0).stop 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 shutdown_connections 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ce806a810 0x7f5ce806ccd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 --2- 172.21.15.28:0/2983440943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5954b39c0 0x55d5954b1480 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 >> 172.21.15.28:0/2983440943 conn(0x55d5954290a0 msgr2=0x55d595426f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 shutdown_connections 2024-09-10T10:51:09.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:09.966+0000 7f5d0a7ecf40 1 -- 172.21.15.28:0/2983440943 wait complete. 2024-09-10T10:51:09.972 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000000 2024-09-10T10:51:09.985 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:51:10.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:51:10.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.089+0000 7f1a7671ef40 1 Processor -- start 2024-09-10T10:51:10.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.089+0000 7f1a7671ef40 1 -- start start 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee08260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ed38ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ecddb30 con 0x55bc0ee09ff0 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ece1b70 con 0x7f1a58002ca0 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bc0ed7edc0 con 0x55bc0ed66500 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35356/0 (socket says 172.21.15.28:35356) 2024-09-10T10:51:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 -- 172.21.15.28:0/2580755151 learned_addr learned my addr 172.21.15.28:0/2580755151 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72e00640 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee08260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a73800640 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ed38ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 -- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 msgr2=0x55bc0ed38ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:10.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ed38ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 -- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 msgr2=0x55bc0ee08260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee08260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.090+0000 7f1a72400640 1 -- 172.21.15.28:0/2580755151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0ee0c230 con 0x55bc0ee09ff0 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a72400640 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009760 tx=0x7f1a60051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a25371279f2bc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a71000640 1 -- 172.21.15.28:0/2580755151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6004f860 con 0x55bc0ee09ff0 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a71000640 1 -- 172.21.15.28:0/2580755151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a6004fa20 con 0x55bc0ee09ff0 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a71000640 1 -- 172.21.15.28:0/2580755151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6004fe10 con 0x55bc0ee09ff0 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a7671ef40 1 -- 172.21.15.28:0/2580755151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 msgr2=0x55bc0ed38570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.091+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009760 tx=0x7f1a60051000 comp rx=0 tx=0).stop 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 -- 172.21.15.28:0/2580755151 shutdown_connections 2024-09-10T10:51:10.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ed38ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ed38570 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/2580755151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee08260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 -- 172.21.15.28:0/2580755151 >> 172.21.15.28:0/2580755151 conn(0x55bc0ed63b90 msgr2=0x55bc0ed63f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 -- 172.21.15.28:0/2580755151 shutdown_connections 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 -- 172.21.15.28:0/2580755151 wait complete. 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.092+0000 7f1a7671ef40 1 Processor -- start 2024-09-10T10:51:10.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 -- start start 2024-09-10T10:51:10.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ee0dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ee092b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee097f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ee9d3e0 con 0x55bc0ee09ff0 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ee0c510 con 0x7f1a58002ca0 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.093+0000 7f1a7671ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bc0ee0c7f0 con 0x55bc0ed66500 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ee092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ee092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35364/0 (socket says 172.21.15.28:35364) 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 -- 172.21.15.28:0/1812937266 learned_addr learned my addr 172.21.15.28:0/1812937266 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72e00640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ee0dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a73800640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 -- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 msgr2=0x55bc0ee0dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bc0ed66500 0x55bc0ee0dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 -- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 msgr2=0x55bc0ee097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58002ca0 0x55bc0ee097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 -- 172.21.15.28:0/1812937266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0ee0d020 con 0x55bc0ee09ff0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.094+0000 7f1a72400640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ee092b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1a60025e30 tx=0x7f1a60003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.095+0000 7f1a6b400640 1 -- 172.21.15.28:0/1812937266 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6004fbd0 con 0x55bc0ee09ff0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.095+0000 7f1a6b400640 1 -- 172.21.15.28:0/1812937266 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a60068020 con 0x55bc0ee09ff0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.095+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc0ee08260 con 0x55bc0ee09ff0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.095+0000 7f1a6b400640 1 -- 172.21.15.28:0/1812937266 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6004fdf0 con 0x55bc0ee09ff0 2024-09-10T10:51:10.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.095+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc0ee0d5a0 con 0x55bc0ee09ff0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.097+0000 7f1a6b400640 1 -- 172.21.15.28:0/1812937266 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a60063410 con 0x55bc0ee09ff0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.097+0000 7f1a6b400640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a5406a810 0x7f1a5406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.098+0000 7f1a6b400640 1 -- 172.21.15.28:0/1812937266 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7f1a600cd160 con 0x55bc0ee09ff0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.098+0000 7f1a72e00640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a5406a810 0x7f1a5406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.098+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55bc0ee0e2e0 0x55bc0ed7b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.098+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e103) -- 0x55bc0ee02170 con 0x55bc0ee0e2e0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.098+0000 7f1a73800640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55bc0ee0e2e0 0x55bc0ed7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.099+0000 7f1a72e00640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a5406a810 0x7f1a5406ccd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55bc0ed61530 tx=0x7f1a64011040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.099+0000 7f1a73800640 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55bc0ee0e2e0 0x55bc0ed7b720 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a73800640 1 -- 172.21.15.28:0/1812937266 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [delete] v103'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a6c006770 con 0x55bc0ee0e2e0 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55bc0ee0e2e0 msgr2=0x55bc0ed7b720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55bc0ee0e2e0 0x55bc0ed7b720 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a5406a810 msgr2=0x7f1a5406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a5406a810 0x7f1a5406ccd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x55bc0ed61530 tx=0x7f1a64011040 comp rx=0 tx=0).stop 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.100+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 msgr2=0x55bc0ee092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a7671ef40 1 --2- 172.21.15.28:0/1812937266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bc0ee09ff0 0x55bc0ee092b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f1a60025e30 tx=0x7f1a60003020 comp rx=0 tx=0).stop 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a73800640 1 -- 172.21.15.28:0/1812937266 reap_dead start 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 shutdown_connections 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 >> 172.21.15.28:0/1812937266 conn(0x55bc0ed63b90 msgr2=0x55bc0ed63f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 shutdown_connections 2024-09-10T10:51:10.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.101+0000 7f1a7671ef40 1 -- 172.21.15.28:0/1812937266 wait complete. 2024-09-10T10:51:10.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.304+0000 7f7429800640 1 Processor -- start 2024-09-10T10:51:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.304+0000 7f7429800640 1 -- start start 2024-09-10T10:51:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7429800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7424161ac0 0x7f742406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7429800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f74240b3640 0x7f74240d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7429800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74240b1170 con 0x7f74240d28f0 2024-09-10T10:51:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7429800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f74240d2d60 con 0x7f74240b3640 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7429800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f74240ceba0 con 0x7f7424161ac0 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f74240b3640 0x7f74240d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7428e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f74240d28f0 0x7f742406cd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34064/0 (socket says 172.21.15.28:34064) 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f74240b3640 0x7f74240d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35378/0 (socket says 172.21.15.28:35378) 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7428e00640 1 -- 172.21.15.28:0/207484150 learned_addr learned my addr 172.21.15.28:0/207484150 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423e00640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7424161ac0 0x7f742406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 -- 172.21.15.28:0/207484150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7424161ac0 msgr2=0x7f742406e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7424161ac0 0x7f742406e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 -- 172.21.15.28:0/207484150 >> v1:172.21.15.28:6789/0 conn(0x7f74240d28f0 legacy=0x7f742406cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.306+0000 7f7423400640 1 -- 172.21.15.28:0/207484150 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74240b1450 con 0x7f74240b3640 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.307+0000 7f7423400640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74240b3640 0x7f74240d23b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f74100097a0 tx=0x7f741002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a358bbda1fc98fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7422a00640 1 -- 172.21.15.28:0/207484150 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410037070 con 0x7f74240b3640 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7422a00640 1 -- 172.21.15.28:0/207484150 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410032440 con 0x7f74240b3640 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7422a00640 1 -- 172.21.15.28:0/207484150 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410032810 con 0x7f74240b3640 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 -- 172.21.15.28:0/207484150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74240b3640 msgr2=0x7f74240d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74240b3640 0x7f74240d23b0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f74100097a0 tx=0x7f741002a210 comp rx=0 tx=0).stop 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 -- 172.21.15.28:0/207484150 shutdown_connections 2024-09-10T10:51:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74240b3640 0x7f74240d23b0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 --2- 172.21.15.28:0/207484150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7424161ac0 0x7f742406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 -- 172.21.15.28:0/207484150 >> 172.21.15.28:0/207484150 conn(0x7f742408bfc0 msgr2=0x7f74240d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 -- 172.21.15.28:0/207484150 shutdown_connections 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.308+0000 7f7429800640 1 -- 172.21.15.28:0/207484150 wait complete. 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.310+0000 7f7429800640 1 Processor -- start 2024-09-10T10:51:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.310+0000 7f7429800640 1 -- start start 2024-09-10T10:51:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74240b3640 0x7f74241575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74240d28f0 0x7f74240cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74240b1a10 con 0x7f7424161ac0 2024-09-10T10:51:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f742406dfb0 con 0x7f74240d28f0 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7429800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7424161ea0 con 0x7f74240b3640 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7428e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7428e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35382/0 (socket says 172.21.15.28:35382) 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7428e00640 1 -- 172.21.15.28:0/2599882668 learned_addr learned my addr 172.21.15.28:0/2599882668 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7423e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74240b3640 0x7f74241575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.311+0000 7f7423400640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74240d28f0 0x7f74240cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 -- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74240b3640 msgr2=0x7f74241575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74240b3640 0x7f74241575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 -- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74240d28f0 msgr2=0x7f74240cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74240d28f0 0x7f74240cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 -- 172.21.15.28:0/2599882668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7424157b10 con 0x7f7424161ac0 2024-09-10T10:51:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7428e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f741c00cbc0 tx=0x7f741c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c017070 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741c00ad40 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c012600 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241f4dc0 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.312+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241f5380 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.314+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74240d23b0 con 0x7f7424161ac0 2024-09-10T10:51:10.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.314+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f741c01a070 con 0x7f7424161ac0 2024-09-10T10:51:10.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.316+0000 7f741be00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 0x7f740406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.316+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6445+0+0 (secure 0 0 0) 0x7f741c08acc0 con 0x7f7424161ac0 2024-09-10T10:51:10.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.316+0000 7f7423e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 0x7f740406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.317+0000 7f7423e00640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 0x7f740406e320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f742415eb10 tx=0x7f7414009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.320+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f741c058dd0 con 0x7f7424161ac0 2024-09-10T10:51:10.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.540+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f7424075670 con 0x7f7424161ac0 2024-09-10T10:51:10.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.685+0000 7f741be00640 1 -- 172.21.15.28:0/2599882668 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7f741c05d0d0 con 0x7f7424161ac0 2024-09-10T10:51:10.687 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 msgr2=0x7f740406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 0x7f740406e320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f742415eb10 tx=0x7f7414009190 comp rx=0 tx=0).stop 2024-09-10T10:51:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 msgr2=0x7f74240c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f741c00cbc0 tx=0x7f741c00a430 comp rx=0 tx=0).stop 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 shutdown_connections 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f740406be60 0x7f740406e320 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7424161ac0 0x7f74240c8660 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74240d28f0 0x7f74240cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 --2- 172.21.15.28:0/2599882668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f74240b3640 0x7f74241575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 >> 172.21.15.28:0/2599882668 conn(0x7f742408bfc0 msgr2=0x7f74240d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 shutdown_connections 2024-09-10T10:51:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.688+0000 7f7429800640 1 -- 172.21.15.28:0/2599882668 wait complete. 2024-09-10T10:51:10.698 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:10.698+0000 7f6f9aa00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-10T10:51:10.699 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:10.698+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:10.699 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:10.698+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:10.708 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:10.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.863+0000 7f0d47200640 1 Processor -- start 2024-09-10T10:51:10.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.863+0000 7f0d47200640 1 -- start start 2024-09-10T10:51:10.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d47200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0d40072d30 0x7f0d4006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d47200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d47200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d400cdbe0 con 0x7f0d4015b300 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d47200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0d400728a0 con 0x7f0d40072d30 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d47200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4006fb30 con 0x7f0d4006f700 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d3fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0d4015b300 0x7f0d400cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34068/0 (socket says 172.21.15.28:34068) 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d3fe00640 1 -- 172.21.15.28:0/432542539 learned_addr learned my addr 172.21.15.28:0/432542539 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d45200640 1 --2- 172.21.15.28:0/432542539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d3f400640 1 --2- 172.21.15.28:0/432542539 >> v2:172.21.15.138:3300/0 conn(0x7f0d40072d30 0x7f0d4006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d3ea00640 1 -- 172.21.15.28:0/432542539 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610732009 0 0) 0x7f0d400cdbe0 con 0x7f0d4015b300 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.865+0000 7f0d3ea00640 1 -- 172.21.15.28:0/432542539 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d24003520 con 0x7f0d4015b300 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.866+0000 7f0d45200640 1 -- 172.21.15.28:0/432542539 >> v2:172.21.15.138:3300/0 conn(0x7f0d40072d30 msgr2=0x7f0d4006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.866+0000 7f0d45200640 1 --2- 172.21.15.28:0/432542539 >> v2:172.21.15.138:3300/0 conn(0x7f0d40072d30 0x7f0d4006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.866+0000 7f0d45200640 1 -- 172.21.15.28:0/432542539 >> v1:172.21.15.28:6789/0 conn(0x7f0d4015b300 legacy=0x7f0d400cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.866+0000 7f0d45200640 1 -- 172.21.15.28:0/432542539 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d400cd900 con 0x7f0d4006f700 2024-09-10T10:51:10.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d45200640 1 --2- 172.21.15.28:0/432542539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4015b6e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009d90 tx=0x7f0d3802c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658a2bbc0e5388be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d3ea00640 1 -- 172.21.15.28:0/432542539 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007740 con 0x7f0d4006f700 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d3ea00640 1 -- 172.21.15.28:0/432542539 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d380078e0 con 0x7f0d4006f700 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d3ea00640 1 -- 172.21.15.28:0/432542539 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38007cb0 con 0x7f0d4006f700 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 -- 172.21.15.28:0/432542539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 msgr2=0x7f0d4015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 --2- 172.21.15.28:0/432542539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4015b6e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009d90 tx=0x7f0d3802c7e0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 -- 172.21.15.28:0/432542539 shutdown_connections 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 --2- 172.21.15.28:0/432542539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4015b6e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 --2- 172.21.15.28:0/432542539 >> v2:172.21.15.138:3300/0 conn(0x7f0d40072d30 0x7f0d4006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.867+0000 7f0d47200640 1 -- 172.21.15.28:0/432542539 >> 172.21.15.28:0/432542539 conn(0x7f0d40067d70 msgr2=0x7f0d4006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.868+0000 7f0d47200640 1 -- 172.21.15.28:0/432542539 shutdown_connections 2024-09-10T10:51:10.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.868+0000 7f0d47200640 1 -- 172.21.15.28:0/432542539 wait complete. 2024-09-10T10:51:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.869+0000 7f0d47200640 1 Processor -- start 2024-09-10T10:51:10.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.869+0000 7f0d47200640 1 -- start start 2024-09-10T10:51:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4015b300 0x7f0d4014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400cd5c0 con 0x7f0d40072d30 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40072530 con 0x7f0d4015b300 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d47200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400cffe0 con 0x7f0d4006f700 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d3f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d3f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35384/0 (socket says 172.21.15.28:35384) 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.870+0000 7f0d3f400640 1 -- 172.21.15.28:0/3744577613 learned_addr learned my addr 172.21.15.28:0/3744577613 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3fe00640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d45200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4015b300 0x7f0d4014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 -- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 msgr2=0x7f0d4014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 -- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4015b300 msgr2=0x7f0d4014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4015b300 0x7f0d4014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 -- 172.21.15.28:0/3744577613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4014ffd0 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d3f400640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9b0 tx=0x7f0d2c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c017070 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d2c00c9a0 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.871+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d40150a70 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.872+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c00cd90 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.872+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d40151030 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.873+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d40159580 con 0x7f0d40072d30 2024-09-10T10:51:10.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.874+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d2c01a070 con 0x7f0d40072d30 2024-09-10T10:51:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.874+0000 7f0d37e00640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 0x7f0d2006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:10.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.875+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7f0d2c08b650 con 0x7f0d40072d30 2024-09-10T10:51:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.875+0000 7f0d3fe00640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 0x7f0d2006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:10.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.875+0000 7f0d3fe00640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 0x7f0d2006e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d30001200 tx=0x7f0d30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:10.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:10.879+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d2c059b90 con 0x7f0d40072d30 2024-09-10T10:51:11.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.099+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d400b4730 con 0x7f0d40072d30 2024-09-10T10:51:11.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.099+0000 7f0d37e00640 1 -- 172.21.15.28:0/3744577613 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3566 (secure 0 0 0) 0x7f0d2c059490 con 0x7f0d40072d30 2024-09-10T10:51:11.100 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:11.101 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":110,"btime":"2024-09-10T10:51:10:699250+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:10.699191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:11.101 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 110 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.101+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 msgr2=0x7f0d2006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.101+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 0x7f0d2006e320 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0d30001200 tx=0x7f0d30009040 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.101+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 msgr2=0x7f0d40153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.101+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c00e9b0 tx=0x7f0d2c00ee70 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 shutdown_connections 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d2006be60 0x7f0d2006e320 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d4015b300 0x7f0d4014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d40072d30 0x7f0d40153ff0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 --2- 172.21.15.28:0/3744577613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0d4006f700 0x7f0d4014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 >> 172.21.15.28:0/3744577613 conn(0x7f0d40067d70 msgr2=0x7f0d4006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 shutdown_connections 2024-09-10T10:51:11.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.102+0000 7f0d47200640 1 -- 172.21.15.28:0/3744577613 wait complete. 2024-09-10T10:51:11.118 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:11.119 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.273+0000 7fea2d800640 1 Processor -- start 2024-09-10T10:51:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.274+0000 7fea2d800640 1 -- start start 2024-09-10T10:51:11.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2d800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 0x7fea2806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2d800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 0x7fea2809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2d800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2806ddd0 con 0x7fea280b47b0 2024-09-10T10:51:11.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2d800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fea2809b3c0 con 0x7fea2809aa30 2024-09-10T10:51:11.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2d800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fea2806e0b0 con 0x7fea281629e0 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea27e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 0x7fea2806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea2ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 0x7fea2809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea1fe00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fea280b47b0 0x7fea28070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:39352/0 (socket says 172.21.15.28:39352) 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.275+0000 7fea1fe00640 1 -- 172.21.15.28:0/1517813677 learned_addr learned my addr 172.21.15.28:0/1517813677 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.276+0000 7fea27e00640 1 -- 172.21.15.28:0/1517813677 >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 msgr2=0x7fea2809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.276+0000 7fea27e00640 1 --2- 172.21.15.28:0/1517813677 >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 0x7fea2809ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.276+0000 7fea27e00640 1 -- 172.21.15.28:0/1517813677 >> v1:172.21.15.138:6789/0 conn(0x7fea280b47b0 legacy=0x7fea28070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.276+0000 7fea27e00640 1 -- 172.21.15.28:0/1517813677 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea280d1b60 con 0x7fea281629e0 2024-09-10T10:51:11.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.276+0000 7fea2ce00640 1 --2- 172.21.15.28:0/1517813677 >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 0x7fea2809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:11.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.277+0000 7fea27e00640 1 --2- 172.21.15.28:0/1517813677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 0x7fea2806e9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fea180097a0 tx=0x7fea1802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529bbb1391c06978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.277+0000 7fea27400640 1 -- 172.21.15.28:0/1517813677 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18037070 con 0x7fea281629e0 2024-09-10T10:51:11.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.277+0000 7fea27400640 1 -- 172.21.15.28:0/1517813677 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1802ae50 con 0x7fea281629e0 2024-09-10T10:51:11.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.277+0000 7fea27400640 1 -- 172.21.15.28:0/1517813677 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18032660 con 0x7fea281629e0 2024-09-10T10:51:11.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 -- 172.21.15.28:0/1517813677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 msgr2=0x7fea2806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 --2- 172.21.15.28:0/1517813677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 0x7fea2806e9d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fea180097a0 tx=0x7fea1802a100 comp rx=0 tx=0).stop 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 -- 172.21.15.28:0/1517813677 shutdown_connections 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 --2- 172.21.15.28:0/1517813677 >> v2:172.21.15.138:3300/0 conn(0x7fea2809aa30 0x7fea2809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 --2- 172.21.15.28:0/1517813677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea281629e0 0x7fea2806e9d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 -- 172.21.15.28:0/1517813677 >> 172.21.15.28:0/1517813677 conn(0x7fea280b3c90 msgr2=0x7fea280b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 -- 172.21.15.28:0/1517813677 shutdown_connections 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.278+0000 7fea2d800640 1 -- 172.21.15.28:0/1517813677 wait complete. 2024-09-10T10:51:11.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.280+0000 7fea2d800640 1 Processor -- start 2024-09-10T10:51:11.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.280+0000 7fea2d800640 1 -- start start 2024-09-10T10:51:11.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea280b47b0 0x7fea280b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea281629e0 0x7fea2815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fea28162f40 con 0x7fea2809aa30 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fea28162dc0 con 0x7fea281629e0 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2d800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fea280d0170 con 0x7fea280b47b0 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35386/0 (socket says 172.21.15.28:35386) 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 -- 172.21.15.28:0/2474708279 learned_addr learned my addr 172.21.15.28:0/2474708279 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea1fe00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea280b47b0 0x7fea280b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea2ce00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea281629e0 0x7fea2815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 -- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea280b47b0 msgr2=0x7fea280b7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea280b47b0 0x7fea280b7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 -- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea281629e0 msgr2=0x7fea2815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea281629e0 0x7fea2815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.281+0000 7fea27e00640 1 -- 172.21.15.28:0/2474708279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea280b5d50 con 0x7fea2809aa30 2024-09-10T10:51:11.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea27e00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea18035040 tx=0x7fea180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18041070 con 0x7fea2809aa30 2024-09-10T10:51:11.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1803c460 con 0x7fea2809aa30 2024-09-10T10:51:11.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18037040 con 0x7fea2809aa30 2024-09-10T10:51:11.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea280b6030 con 0x7fea2809aa30 2024-09-10T10:51:11.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.282+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea280b6b10 con 0x7fea2809aa30 2024-09-10T10:51:11.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.283+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2809ae80 con 0x7fea2809aa30 2024-09-10T10:51:11.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.284+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fea1803a070 con 0x7fea2809aa30 2024-09-10T10:51:11.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.285+0000 7fea24c00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 0x7fea0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.285+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7fea180ab570 con 0x7fea2809aa30 2024-09-10T10:51:11.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.285+0000 7fea1fe00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 0x7fea0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.286+0000 7fea1fe00640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 0x7fea0406e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fea2815efe0 tx=0x7fea10007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.289+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fea180796a0 con 0x7fea2809aa30 2024-09-10T10:51:11.433 INFO:tasks.ceph.mds.a.smithi028.stderr: -35> 2024-09-10T10:51:06.679+0000 7f12abe00640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T10:51:11.433 INFO:tasks.ceph.mds.a.smithi028.stderr: -33> 2024-09-10T10:51:06.679+0000 7f12abe00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:51:11.471 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:51:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.512+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea280b8120 con 0x7fea2809aa30 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.513+0000 7fea24c00640 1 -- 172.21.15.28:0/2474708279 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+3566 (secure 0 0 0) 0x7fea1807d9a0 con 0x7fea2809aa30 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":110,"btime":"2024-09-10T10:51:10:699250+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:10.699191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 110 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 msgr2=0x7fea0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 0x7fea0406e2f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fea2815efe0 tx=0x7fea10007e80 comp rx=0 tx=0).stop 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 msgr2=0x7fea2809c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fea18035040 tx=0x7fea180046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 shutdown_connections 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fea0406be30 0x7fea0406e2f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fea281629e0 0x7fea2815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fea280b47b0 0x7fea280b7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 --2- 172.21.15.28:0/2474708279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea2809aa30 0x7fea2809c290 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.515+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 >> 172.21.15.28:0/2474708279 conn(0x7fea280b3c90 msgr2=0x7fea280b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.516+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 shutdown_connections 2024-09-10T10:51:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.516+0000 7fea2d800640 1 -- 172.21.15.28:0/2474708279 wait complete. 2024-09-10T10:51:11.532 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:11.533 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:11.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.688+0000 7f0af4c00640 1 Processor -- start 2024-09-10T10:51:11.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.688+0000 7f0af4c00640 1 -- start start 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0af4c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0af0072d30 0x7f0af006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0af4c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0af006f700 0x7f0af015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0af4c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af006fae0 con 0x7f0af015b300 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0af4c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0af00728a0 con 0x7f0af0072d30 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0af4c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0af00cdbe0 con 0x7f0af006f700 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0aede00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0af015b300 0x7f0af00cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34088/0 (socket says 172.21.15.28:34088) 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0aede00640 1 -- 172.21.15.28:0/2290810841 learned_addr learned my addr 172.21.15.28:0/2290810841 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0aed400640 1 --2- 172.21.15.28:0/2290810841 >> v2:172.21.15.28:3300/0 conn(0x7f0af0072d30 0x7f0af006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.689+0000 7f0aee800640 1 --2- 172.21.15.28:0/2290810841 >> v2:172.21.15.138:3300/0 conn(0x7f0af006f700 0x7f0af015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.690+0000 7f0aed400640 1 -- 172.21.15.28:0/2290810841 >> v2:172.21.15.138:3300/0 conn(0x7f0af006f700 msgr2=0x7f0af015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:11.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.690+0000 7f0aed400640 1 --2- 172.21.15.28:0/2290810841 >> v2:172.21.15.138:3300/0 conn(0x7f0af006f700 0x7f0af015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.690+0000 7f0aed400640 1 -- 172.21.15.28:0/2290810841 >> v1:172.21.15.28:6789/0 conn(0x7f0af015b300 legacy=0x7f0af00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.690+0000 7f0aed400640 1 -- 172.21.15.28:0/2290810841 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af00cd900 con 0x7f0af0072d30 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.691+0000 7f0aed400640 1 --2- 172.21.15.28:0/2290810841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af0072d30 0x7f0af006f1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0af00b38f0 tx=0x7f0adc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bba2f446a026774 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.691+0000 7f0aeca00640 1 -- 172.21.15.28:0/2290810841 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc037070 con 0x7f0af0072d30 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.691+0000 7f0aeca00640 1 -- 172.21.15.28:0/2290810841 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0adc032440 con 0x7f0af0072d30 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.691+0000 7f0aeca00640 1 -- 172.21.15.28:0/2290810841 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc02ae60 con 0x7f0af0072d30 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 -- 172.21.15.28:0/2290810841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af0072d30 msgr2=0x7f0af006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 --2- 172.21.15.28:0/2290810841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af0072d30 0x7f0af006f1c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0af00b38f0 tx=0x7f0adc02a110 comp rx=0 tx=0).stop 2024-09-10T10:51:11.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 -- 172.21.15.28:0/2290810841 shutdown_connections 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 --2- 172.21.15.28:0/2290810841 >> v2:172.21.15.138:3300/0 conn(0x7f0af006f700 0x7f0af015b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 --2- 172.21.15.28:0/2290810841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af0072d30 0x7f0af006f1c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 -- 172.21.15.28:0/2290810841 >> 172.21.15.28:0/2290810841 conn(0x7f0af0067d70 msgr2=0x7f0af006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 -- 172.21.15.28:0/2290810841 shutdown_connections 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.692+0000 7f0af4c00640 1 -- 172.21.15.28:0/2290810841 wait complete. 2024-09-10T10:51:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.694+0000 7f0af4c00640 1 Processor -- start 2024-09-10T10:51:11.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.694+0000 7f0af4c00640 1 -- start start 2024-09-10T10:51:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af0072d30 0x7f0af0155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 0x7f0af01e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00cd5c0 con 0x7f0af006f700 2024-09-10T10:51:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0072530 con 0x7f0af0072d30 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0af4c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0af00cffe0 con 0x7f0af015b300 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35414/0 (socket says 172.21.15.28:35414) 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 -- 172.21.15.28:0/574667068 learned_addr learned my addr 172.21.15.28:0/574667068 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aed400640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af0072d30 0x7f0af0155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aee800640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 0x7f0af01e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 -- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 msgr2=0x7f0af01e4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 0x7f0af01e4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 -- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af0072d30 msgr2=0x7f0af0155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af0072d30 0x7f0af0155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aede00640 1 -- 172.21.15.28:0/574667068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af015d220 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.695+0000 7f0aee800640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 0x7f0af01e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0aede00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c400 tx=0x7f0ae000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0017070 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae0005ce0 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00072f0 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af0155b60 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.696+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af0156120 con 0x7f0af006f700 2024-09-10T10:51:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.697+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af015d3f0 con 0x7f0af006f700 2024-09-10T10:51:11.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.698+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0ae001a070 con 0x7f0af006f700 2024-09-10T10:51:11.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.699+0000 7f0ae6000640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 0x7f0ad006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:11.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.700+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7f0ae008a530 con 0x7f0af006f700 2024-09-10T10:51:11.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.700+0000 7f0aed400640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 0x7f0ad006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:11.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.701+0000 7f0aed400640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 0x7f0ad006e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0adc034000 tx=0x7f0adc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:11.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.704+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ae0058540 con 0x7f0af006f700 2024-09-10T10:51:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.925+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0af006f1c0 con 0x7f0af006f700 2024-09-10T10:51:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.925+0000 7f0ae6000640 1 -- 172.21.15.28:0/574667068 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+4392 (secure 0 0 0) 0x7f0ae005c840 con 0x7f0af006f700 2024-09-10T10:51:11.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:11.927 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":111,"btime":"2024-09-10T10:51:11:704945+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7189,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1234809424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1234809424},{"type":"v1","addr":"172.21.15.28:6835","nonce":1234809424}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:10.699191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":104,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 111 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 msgr2=0x7f0ad006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 0x7f0ad006e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0adc034000 tx=0x7f0adc02e000 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 msgr2=0x7f0af0152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c400 tx=0x7f0ae000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 shutdown_connections 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ad006be30 0x7f0ad006e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0af015b300 0x7f0af01e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.927+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0af0072d30 0x7f0af0155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.928+0000 7f0af4c00640 1 --2- 172.21.15.28:0/574667068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0af006f700 0x7f0af0152680 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.928+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 >> 172.21.15.28:0/574667068 conn(0x7f0af0067d70 msgr2=0x7f0af006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.928+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 shutdown_connections 2024-09-10T10:51:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:11.928+0000 7f0af4c00640 1 -- 172.21.15.28:0/574667068 wait complete. 2024-09-10T10:51:11.946 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000000' led to DAMAGED state 2024-09-10T10:51:11.946 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 200.00000001' 2024-09-10T10:51:11.946 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:11.946 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:11.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:11.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:11.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:11.982 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:12.057 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:12.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:12.058 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:12.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.256+0000 7f2d6b000640 1 Processor -- start 2024-09-10T10:51:12.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.256+0000 7f2d6b000640 1 -- start start 2024-09-10T10:51:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.257+0000 7f2d6b000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2d64072dc0 0x7f2d640b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.257+0000 7f2d6b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d640b3c90 0x7f2d6406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.257+0000 7f2d6b000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d640b44c0 con 0x7f2d640afd60 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.257+0000 7f2d6b000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2d640b0140 con 0x7f2d64072dc0 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.257+0000 7f2d6b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6415ead0 con 0x7f2d640b3c90 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d63e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2d640afd60 0x7f2d640726b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34104/0 (socket says 172.21.15.28:34104) 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d63e00640 1 -- 172.21.15.28:0/1739961191 learned_addr learned my addr 172.21.15.28:0/1739961191 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d5be00640 1 --2- 172.21.15.28:0/1739961191 >> v2:172.21.15.28:3300/0 conn(0x7f2d64072dc0 0x7f2d640b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d69000640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d640b3c90 0x7f2d6406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d63400640 1 -- 172.21.15.28:0/1739961191 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 214326492 0 0) 0x7f2d640b44c0 con 0x7f2d640afd60 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d63400640 1 -- 172.21.15.28:0/1739961191 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d48003520 con 0x7f2d640afd60 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d5be00640 1 -- 172.21.15.28:0/1739961191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d640b3c90 msgr2=0x7f2d6406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d5be00640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d640b3c90 0x7f2d6406f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d5be00640 1 -- 172.21.15.28:0/1739961191 >> v1:172.21.15.28:6789/0 conn(0x7f2d640afd60 legacy=0x7f2d640726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.258+0000 7f2d5be00640 1 -- 172.21.15.28:0/1739961191 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6415edb0 con 0x7f2d64072dc0 2024-09-10T10:51:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.259+0000 7f2d5be00640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d64072dc0 0x7f2d640b3750 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d9bb9ee61efd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.259+0000 7f2d63400640 1 -- 172.21.15.28:0/1739961191 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037070 con 0x7f2d64072dc0 2024-09-10T10:51:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.259+0000 7f2d63400640 1 -- 172.21.15.28:0/1739961191 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c032440 con 0x7f2d64072dc0 2024-09-10T10:51:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.259+0000 7f2d63400640 1 -- 172.21.15.28:0/1739961191 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0327f0 con 0x7f2d64072dc0 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 -- 172.21.15.28:0/1739961191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d64072dc0 msgr2=0x7f2d640b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d64072dc0 0x7f2d640b3750 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0097a0 tx=0x7f2d4c02a180 comp rx=0 tx=0).stop 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 -- 172.21.15.28:0/1739961191 shutdown_connections 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d640b3c90 0x7f2d6406f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 --2- 172.21.15.28:0/1739961191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d64072dc0 0x7f2d640b3750 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 -- 172.21.15.28:0/1739961191 >> 172.21.15.28:0/1739961191 conn(0x7f2d6406c030 msgr2=0x7f2d6406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 -- 172.21.15.28:0/1739961191 shutdown_connections 2024-09-10T10:51:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.260+0000 7f2d6b000640 1 -- 172.21.15.28:0/1739961191 wait complete. 2024-09-10T10:51:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.262+0000 7f2d6b000640 1 Processor -- start 2024-09-10T10:51:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.262+0000 7f2d6b000640 1 -- start start 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d64072dc0 0x7f2d641f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 0x7f2d64151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6415f890 con 0x7f2d640afd60 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6415f370 con 0x7f2d640b3c90 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d6b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2d640b4120 con 0x7f2d64072dc0 2024-09-10T10:51:12.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35438/0 (socket says 172.21.15.28:35438) 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 -- 172.21.15.28:0/3862843809 learned_addr learned my addr 172.21.15.28:0/3862843809 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d63e00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d64072dc0 0x7f2d641f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d69000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 0x7f2d64151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 -- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d64072dc0 msgr2=0x7f2d641f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d64072dc0 0x7f2d641f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 -- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 msgr2=0x7f2d64151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 0x7f2d64151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.263+0000 7f2d5be00640 1 -- 172.21.15.28:0/3862843809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d641f45f0 con 0x7f2d640afd60 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d69000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 0x7f2d64151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d5be00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c02a1c0 tx=0x7f2d4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c041020 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d4c030700 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c037040 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d641f48d0 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.264+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d641f4cb0 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.265+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d640adfb0 con 0x7f2d640afd60 2024-09-10T10:51:12.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.266+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d4c044020 con 0x7f2d640afd60 2024-09-10T10:51:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.267+0000 7f2d60c00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.267+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7f2d4c0ab980 con 0x7f2d640afd60 2024-09-10T10:51:12.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.268+0000 7f2d63e00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.268+0000 7f2d63e00640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 0x7f2d4006e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005880 tx=0x7f2d54009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.271+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d640adfb0 con 0x7f2d640afd60 2024-09-10T10:51:12.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.491+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2d640b3750 con 0x7f2d640afd60 2024-09-10T10:51:12.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.711+0000 7f2d60c00640 1 -- 172.21.15.28:0/3862843809 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v112) ==== 161+0+0 (secure 0 0 0) 0x7f2d4c07ee20 con 0x7f2d640afd60 2024-09-10T10:51:12.713 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.713+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 msgr2=0x7f2d4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.713+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 0x7f2d4006e2f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005880 tx=0x7f2d54009040 comp rx=0 tx=0).stop 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 msgr2=0x7f2d641f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c02a1c0 tx=0x7f2d4c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 shutdown_connections 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2d4006be30 0x7f2d4006e2f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d640b3c90 0x7f2d64151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d640afd60 0x7f2d641f4010 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 --2- 172.21.15.28:0/3862843809 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2d64072dc0 0x7f2d641f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 >> 172.21.15.28:0/3862843809 conn(0x7f2d6406c030 msgr2=0x7f2d640ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:12.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 shutdown_connections 2024-09-10T10:51:12.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.714+0000 7f2d6b000640 1 -- 172.21.15.28:0/3862843809 wait complete. 2024-09-10T10:51:12.730 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:12.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.886+0000 7fe2c8200640 1 Processor -- start 2024-09-10T10:51:12.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.886+0000 7fe2c8200640 1 -- start start 2024-09-10T10:51:12.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c8200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe2c0072de0 0x7fe2c015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c8200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2c0072700 0x7fe2c015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c8200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c0146140 con 0x7fe2c0072320 2024-09-10T10:51:12.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c8200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c00cd740 con 0x7fe2c0072700 2024-09-10T10:51:12.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c8200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c00cda20 con 0x7fe2c0072de0 2024-09-10T10:51:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.887+0000 7fe2c6200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2c0072700 0x7fe2c015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2c0072700 0x7fe2c015d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35440/0 (socket says 172.21.15.28:35440) 2024-09-10T10:51:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 -- 172.21.15.28:0/1136484906 learned_addr learned my addr 172.21.15.28:0/1136484906 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:12.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c4e00640 1 --2- 172.21.15.28:0/1136484906 >> v2:172.21.15.138:3300/0 conn(0x7fe2c0072de0 0x7fe2c015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 -- 172.21.15.28:0/1136484906 >> v2:172.21.15.138:3300/0 conn(0x7fe2c0072de0 msgr2=0x7fe2c015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 --2- 172.21.15.28:0/1136484906 >> v2:172.21.15.138:3300/0 conn(0x7fe2c0072de0 0x7fe2c015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 -- 172.21.15.28:0/1136484906 >> v1:172.21.15.138:6789/0 conn(0x7fe2c0072320 legacy=0x7fe2c00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:12.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.888+0000 7fe2c6200640 1 -- 172.21.15.28:0/1136484906 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c00cdd00 con 0x7fe2c0072700 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.889+0000 7fe2c6200640 1 --2- 172.21.15.28:0/1136484906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072700 0x7fe2c015d720 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe2c006f8c0 tx=0x7fe2b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fd356af98aeded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.889+0000 7fe2bfe00640 1 -- 172.21.15.28:0/1136484906 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8036070 con 0x7fe2c0072700 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.889+0000 7fe2bfe00640 1 -- 172.21.15.28:0/1136484906 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b8031600 con 0x7fe2c0072700 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.889+0000 7fe2bfe00640 1 -- 172.21.15.28:0/1136484906 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80319d0 con 0x7fe2c0072700 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 -- 172.21.15.28:0/1136484906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072700 msgr2=0x7fe2c015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 --2- 172.21.15.28:0/1136484906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072700 0x7fe2c015d720 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe2c006f8c0 tx=0x7fe2b802a660 comp rx=0 tx=0).stop 2024-09-10T10:51:12.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 -- 172.21.15.28:0/1136484906 shutdown_connections 2024-09-10T10:51:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 --2- 172.21.15.28:0/1136484906 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072700 0x7fe2c015d720 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 --2- 172.21.15.28:0/1136484906 >> v2:172.21.15.138:3300/0 conn(0x7fe2c0072de0 0x7fe2c015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 -- 172.21.15.28:0/1136484906 >> 172.21.15.28:0/1136484906 conn(0x7fe2c0067d70 msgr2=0x7fe2c006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 -- 172.21.15.28:0/1136484906 shutdown_connections 2024-09-10T10:51:12.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.890+0000 7fe2c8200640 1 -- 172.21.15.28:0/1136484906 wait complete. 2024-09-10T10:51:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.892+0000 7fe2c8200640 1 Processor -- start 2024-09-10T10:51:12.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.892+0000 7fe2c8200640 1 -- start start 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0072320 0x7fe2c015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2c0072700 0x7fe2c00aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015dde0 con 0x7fe2c0072de0 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c015dc60 con 0x7fe2c0072320 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c8200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c015df60 con 0x7fe2c0072700 2024-09-10T10:51:12.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35444/0 (socket says 172.21.15.28:35444) 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 -- 172.21.15.28:0/646620406 learned_addr learned my addr 172.21.15.28:0/646620406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c5800640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0072320 0x7fe2c015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c4e00640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2c0072700 0x7fe2c00aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 -- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2c0072700 msgr2=0x7fe2c00aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2c0072700 0x7fe2c00aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 -- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0072320 msgr2=0x7fe2c015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0072320 0x7fe2c015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.893+0000 7fe2c6200640 1 -- 172.21.15.28:0/646620406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c00a9e10 con 0x7fe2c0072de0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2c6200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe2b802ac50 tx=0x7fe2b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8036050 con 0x7fe2c0072de0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2b8042070 con 0x7fe2c0072de0 2024-09-10T10:51:12.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8009510 con 0x7fe2c0072de0 2024-09-10T10:51:12.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c00aa0f0 con 0x7fe2c0072de0 2024-09-10T10:51:12.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.894+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c00aa6b0 con 0x7fe2c0072de0 2024-09-10T10:51:12.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.895+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c015e320 con 0x7fe2c0072de0 2024-09-10T10:51:12.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.896+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe2b8007bf0 con 0x7fe2c0072de0 2024-09-10T10:51:12.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.897+0000 7fe2bd600640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 0x7fe2a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:12.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.897+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7fe2b80abff0 con 0x7fe2c0072de0 2024-09-10T10:51:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.898+0000 7fe2c5800640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 0x7fe2a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:12.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.899+0000 7fe2c5800640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 0x7fe2a006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4004ea0 tx=0x7fe2b4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:12.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:12.902+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe2b807a600 con 0x7fe2c0072de0 2024-09-10T10:51:13.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.159+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe2c006d090 con 0x7fe2c0072de0 2024-09-10T10:51:13.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.718+0000 7fe2bd600640 1 -- 172.21.15.28:0/646620406 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v113) ==== 101+0+0 (secure 0 0 0) 0x7fe2b8079f00 con 0x7fe2c0072de0 2024-09-10T10:51:13.719 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.720+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 msgr2=0x7fe2a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.720+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 0x7fe2a006e2f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4004ea0 tx=0x7fe2b4009040 comp rx=0 tx=0).stop 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.720+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 msgr2=0x7fe2c00a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe2b802ac50 tx=0x7fe2b8004a40 comp rx=0 tx=0).stop 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 shutdown_connections 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe2a006be30 0x7fe2a006e2f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2c0072de0 0x7fe2c00a9490 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2c0072700 0x7fe2c00aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 --2- 172.21.15.28:0/646620406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0072320 0x7fe2c015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 >> 172.21.15.28:0/646620406 conn(0x7fe2c0067d70 msgr2=0x7fe2c00c55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:13.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 shutdown_connections 2024-09-10T10:51:13.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.721+0000 7fe2c8200640 1 -- 172.21.15.28:0/646620406 wait complete. 2024-09-10T10:51:13.741 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:13.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.818+0000 7f04d3662f40 1 Processor -- start 2024-09-10T10:51:13.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.818+0000 7f04d3662f40 1 -- start start 2024-09-10T10:51:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 0x5621cdeb09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdf3d9c0 0x5621cdeb0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5621cdeaf530 con 0x5621cdeaeff0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621cde0cb30 con 0x5621cdf3a3e0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d3662f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5621cdf3cee0 con 0x5621cdf3d9c0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.819+0000 7f04d0800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43754/0 (socket says 172.21.15.28:43754) 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 -- 172.21.15.28:0/74529396 learned_addr learned my addr 172.21.15.28:0/74529396 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04cfe00640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 0x5621cdeb09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d1200640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdf3d9c0 0x5621cdeb0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 -- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdf3d9c0 msgr2=0x5621cdeb0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdf3d9c0 0x5621cdeb0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 -- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 msgr2=0x5621cdeb09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 0x5621cdeb09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 -- 172.21.15.28:0/74529396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621cde64340 con 0x5621cdeaeff0 2024-09-10T10:51:13.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04cfe00640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 0x5621cdeb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:13.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04d0800640 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f04c40098e0 tx=0x7f04c4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163276cbbd611364 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.820+0000 7f04cea00640 1 -- 172.21.15.28:0/74529396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4008950 con 0x5621cdeaeff0 2024-09-10T10:51:13.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04cea00640 1 -- 172.21.15.28:0/74529396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c4018070 con 0x5621cdeaeff0 2024-09-10T10:51:13.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04cea00640 1 -- 172.21.15.28:0/74529396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4008d30 con 0x5621cdeaeff0 2024-09-10T10:51:13.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 -- 172.21.15.28:0/74529396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 msgr2=0x5621cdf3bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:13.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f04c40098e0 tx=0x7f04c4005a00 comp rx=0 tx=0).stop 2024-09-10T10:51:13.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 -- 172.21.15.28:0/74529396 shutdown_connections 2024-09-10T10:51:13.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdf3d9c0 0x5621cdeb0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3a3e0 0x5621cdeb09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 --2- 172.21.15.28:0/74529396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdeaeff0 0x5621cdf3bb90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 -- 172.21.15.28:0/74529396 >> 172.21.15.28:0/74529396 conn(0x5621cdeb30a0 msgr2=0x5621cdf3aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 -- 172.21.15.28:0/74529396 shutdown_connections 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.821+0000 7f04d3662f40 1 -- 172.21.15.28:0/74529396 wait complete. 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.822+0000 7f04d3662f40 1 Processor -- start 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.822+0000 7f04d3662f40 1 -- start start 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdeaeff0 0x5621cdf2c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3d9c0 0x5621cdf40820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5621cdf31420 con 0x5621cdf3a3e0 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5621cdf34930 con 0x5621cdf3d9c0 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d3662f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5621cdf31db0 con 0x5621cdeaeff0 2024-09-10T10:51:13.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43762/0 (socket says 172.21.15.28:43762) 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 learned_addr learned my addr 172.21.15.28:0/1120934707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdeaeff0 0x5621cdf2c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04d1200640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3d9c0 0x5621cdf40820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdeaeff0 msgr2=0x5621cdf2c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.823+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdeaeff0 0x5621cdf2c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3d9c0 msgr2=0x5621cdf40820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621cdf3d9c0 0x5621cdf40820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621cdf33f30 con 0x5621cdf3a3e0 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5621cdeaeff0 0x5621cdf2c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00dcd0 tx=0x7f04bc00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cd600640 1 -- 172.21.15.28:0/1120934707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00f650 con 0x5621cdf3a3e0 2024-09-10T10:51:13.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cd600640 1 -- 172.21.15.28:0/1120934707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc00f7f0 con 0x5621cdf3a3e0 2024-09-10T10:51:13.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04cd600640 1 -- 172.21.15.28:0/1120934707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00fbe0 con 0x5621cdf3a3e0 2024-09-10T10:51:13.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5621cdf2ff10 con 0x5621cdf3a3e0 2024-09-10T10:51:13.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.824+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621cdf304d0 con 0x5621cdf3a3e0 2024-09-10T10:51:13.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.826+0000 7f04cd600640 1 -- 172.21.15.28:0/1120934707 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f04bc017400 con 0x5621cdf3a3e0 2024-09-10T10:51:13.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.827+0000 7f04cd600640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 0x7f04b006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.827+0000 7f04cd600640 1 -- 172.21.15.28:0/1120934707 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7f04bc08a3b0 con 0x5621cdf3a3e0 2024-09-10T10:51:13.829 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.827+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5621cdf31870 0x5621cdf3e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.827+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdf3e830 con 0x5621cdf31870 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.828+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 0x7f04b006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.828+0000 7f04d1200640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5621cdf31870 0x5621cdf3e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.828+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 0x7f04b006ccd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f04c4015040 tx=0x7f04c400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.829+0000 7f04d1200640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5621cdf31870 0x5621cdf3e2f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.830+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v104'133 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.830+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdf3ec20 con 0x5621cdf31870 2024-09-10T10:51:13.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.832+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v104'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.832+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf500 con 0x5621cdf31870 2024-09-10T10:51:13.834 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.834+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v104'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.834+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe08b0 con 0x5621cdf31870 2024-09-10T10:51:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.835+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v104'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.835+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe0c50 con 0x5621cdf31870 2024-09-10T10:51:13.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.837+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v104'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.837+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.838+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v104'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.838+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.840+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v104'139 uv113 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.840+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.841+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v104'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.841+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.844 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:13.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.844+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v104'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.844+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe0ff0 con 0x5621cdf31870 2024-09-10T10:51:13.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.845+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v104'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.845+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe0ff0 con 0x5621cdf31870 2024-09-10T10:51:13.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.846+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v104'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.846+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.848+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v104'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.848+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.850+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v104'145 uv120 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.850+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.852+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v104'146 uv146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.852+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.854 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:13.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.853+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v104'147 uv147 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.853+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe0ff0 con 0x5621cdf31870 2024-09-10T10:51:13.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.855+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v104'148 uv148 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.855+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe0ff0 con 0x5621cdf31870 2024-09-10T10:51:13.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.856+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v104'149 uv149 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.856+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.858+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v104'150 uv150 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.858+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.860+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v104'151 uv151 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.860+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.862+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v104'152 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.862+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.863+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v104'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.863+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.865+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v104'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.866 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:13.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.865+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe14a0 con 0x5621cdf31870 2024-09-10T10:51:13.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.866+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v104'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.866+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe14a0 con 0x5621cdf31870 2024-09-10T10:51:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.868+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v104'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.868+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.869+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v104'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.869+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.871+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v104'158 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.871+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdfd50 con 0x5621cdf31870 2024-09-10T10:51:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.873+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v104'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.873+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.875 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.874+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v104'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.874+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.876+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v104'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.876+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.878+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v104'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.878+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfdf8a0 con 0x5621cdf31870 2024-09-10T10:51:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.880+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v104'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdf31870 2024-09-10T10:51:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.880+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5621cdfe0260 0x5621cdfe3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.880+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe3540 con 0x5621cdfe0260 2024-09-10T10:51:13.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.880+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5621cdfe0260 0x5621cdfe3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.880+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5621cdfe0260 0x5621cdfe3000 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.882+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v104'109 uv91 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.882+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe38e0 con 0x5621cdfe0260 2024-09-10T10:51:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.883+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v104'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.883+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe3c80 con 0x5621cdfe0260 2024-09-10T10:51:13.886 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.885+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v104'111 uv111 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.885+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4020 con 0x5621cdfe0260 2024-09-10T10:51:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.887+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v104'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.887+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe43c0 con 0x5621cdfe0260 2024-09-10T10:51:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.888+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v104'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.889+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4760 con 0x5621cdfe0260 2024-09-10T10:51:13.891 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.890+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v104'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.890+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4b00 con 0x5621cdfe0260 2024-09-10T10:51:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.891+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [write 0~90] v104'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.891+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.893+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [omap-set-header] v104'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.893+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.895+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000001 [create] v104'117 uv101 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.895+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.897+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000001 [delete] v104'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.897+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.899 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.898+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v104'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.898+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe9590 con 0x5621cdfe0260 2024-09-10T10:51:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.900+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [write 0~18118] v104'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.900+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe9590 con 0x5621cdfe0260 2024-09-10T10:51:13.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.901+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v104'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.901+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.903+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 606.00000000 [create] v104'122 uv107 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.903+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.904+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 606.00000000 [delete] v104'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.904+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.906 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:13.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.905+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v104'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.905+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe5eb0 con 0x5621cdfe0260 2024-09-10T10:51:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.907+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v104'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.907+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe5eb0 con 0x5621cdfe0260 2024-09-10T10:51:13.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.908+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v104'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.908+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe4ea0 con 0x5621cdfe0260 2024-09-10T10:51:13.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.909+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v104'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfe0260 2024-09-10T10:51:13.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.909+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5621cdfe53b0 0x5621cdfe77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.909+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe7d20 con 0x5621cdfe53b0 2024-09-10T10:51:13.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.910+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5621cdfe53b0 0x5621cdfe77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.910+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5621cdfe53b0 0x5621cdfe77a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.912+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(51 607.00000000 [create] v104'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.912+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe81f0 con 0x5621cdfe53b0 2024-09-10T10:51:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.913+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(52 607.00000000 [delete] v104'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.913+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe86d0 con 0x5621cdfe53b0 2024-09-10T10:51:13.916 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.915+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(53 607.00000000 [create] v104'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.915+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe9f80 con 0x5621cdfe53b0 2024-09-10T10:51:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.918+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v104'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.918+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfea320 con 0x5621cdfe53b0 2024-09-10T10:51:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.920+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v104'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.920+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfe8f70 con 0x5621cdfe53b0 2024-09-10T10:51:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.923+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v104'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.923+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5621cdfe9870 0x5621cdfec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.923+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfecc10 con 0x5621cdfe9870 2024-09-10T10:51:13.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.924+0000 7f04d1200640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5621cdfe9870 0x5621cdfec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.924+0000 7f04d1200640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5621cdfe9870 0x5621cdfec6d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.925+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v104'22 uv21 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.925+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfecc10 con 0x5621cdfe9870 2024-09-10T10:51:13.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.926+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v104'23 uv23 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.926+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfecc90 con 0x5621cdfe9870 2024-09-10T10:51:13.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.927+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v104'24 uv24 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:13.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.928+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.928+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v104'25 uv25 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.929+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.930+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(61 608.00000000 [create] v104'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.930+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.931+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(62 608.00000000 [delete] v104'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.931+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.933 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:13.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.932+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(63 608.00000000 [create] v104'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.932+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfee0c0 con 0x5621cdfe53b0 2024-09-10T10:51:13.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.933+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v104'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.933+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfee0c0 con 0x5621cdfe53b0 2024-09-10T10:51:13.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.934+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v104'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.934+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.935+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v104'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.935+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.937+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(67 604.00000000 [create] v104'48 uv42 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.937+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.938+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(68 604.00000000 [delete] v104'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.938+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.939+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(69 604.00000000 [create] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.941 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:13.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.940+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfee0c0 con 0x5621cdfe9870 2024-09-10T10:51:13.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.941+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v104'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.941+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfee0c0 con 0x5621cdfe9870 2024-09-10T10:51:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.942+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.942+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.943+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.943+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.944+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(73 500.00000000 [create] v104'54 uv47 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.944+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.946+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(74 500.00000000 [delete] v104'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.946+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.947+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(75 500.00000000 [create] v104'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.948 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:13.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.947+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.949+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v104'57 uv57 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.949+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe9870 2024-09-10T10:51:13.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.950+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v104'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.950+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.951+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(78 mds_snaptable [create] v104'22 uv21 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.951+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.952+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(79 mds_snaptable [delete] v104'23 uv23 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.952+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.954 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.953+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(80 mds_snaptable [create] v104'24 uv24 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.953+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.955+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v104'25 uv25 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.955+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfed0b0 con 0x5621cdfe53b0 2024-09-10T10:51:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.956+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v104'26 uv26 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdfe53b0 2024-09-10T10:51:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.956+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5621cdfed5c0 0x5621cdfef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.956+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdfeff80 con 0x5621cdfed5c0 2024-09-10T10:51:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.956+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5621cdfed5c0 0x5621cdfef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.957+0000 7f04d0800640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5621cdfed5c0 0x5621cdfef9b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.958+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v104'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.958+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0060 con 0x5621cdfed5c0 2024-09-10T10:51:13.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.959+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v104'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.959+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.960+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v104'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.961 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:13.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.960+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff1490 con 0x5621cdfed5c0 2024-09-10T10:51:13.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.961+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v104'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.961+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff1490 con 0x5621cdfed5c0 2024-09-10T10:51:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.963+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v104'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.963+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.964+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v104'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.964+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.965+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v104'44 uv38 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.966+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.967+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v104'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.967+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.969 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:13.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.968+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v104'46 uv46 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.968+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.969+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v104'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.969+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.971+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v104'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.971+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.972+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v104'49 uv43 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.972+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.974+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v104'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.974+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.975+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v104'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:13.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.975+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0480 con 0x5621cdfed5c0 2024-09-10T10:51:13.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.976+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v104'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.976+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff3290 con 0x5621cdfed5c0 2024-09-10T10:51:13.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.978+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v104'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.978+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.979+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v104'22 uv21 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.979+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.980+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v104'23 uv23 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.980+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.981+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v104'24 uv24 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.982 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:13.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.981+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.982+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v104'25 uv25 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.982+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.984+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v104'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.984+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.985+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v104'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.985+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.987 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:13.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.986+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v104'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.986+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff1940 con 0x5621cdfe9870 2024-09-10T10:51:13.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.987+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v104'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.987+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff1940 con 0x5621cdfe9870 2024-09-10T10:51:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.988+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v104'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.988+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff0930 con 0x5621cdfe9870 2024-09-10T10:51:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.989+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v104'32 uv32 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04c804e040 con 0x5621cdfe9870 2024-09-10T10:51:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.989+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5621cdff0fb0 0x5621cdff38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.989+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdff0fb0 2024-09-10T10:51:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.990+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5621cdff0fb0 0x5621cdff38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.990+0000 7f04cfe00640 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5621cdff0fb0 0x5621cdff38b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:13.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.991+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v104'22 uv21 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdff0fb0 2024-09-10T10:51:13.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.991+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdff0fb0 2024-09-10T10:51:13.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.992+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v104'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdff0fb0 2024-09-10T10:51:13.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.992+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdff0fb0 2024-09-10T10:51:13.994 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.993+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v104'24 uv24 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdff0fb0 2024-09-10T10:51:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.993+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdff0fb0 2024-09-10T10:51:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.994+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v104'25 uv25 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdff0fb0 2024-09-10T10:51:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.994+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdff0fb0 2024-09-10T10:51:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.995+0000 7f04cfe00640 1 -- 172.21.15.28:0/1120934707 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v104'26 uv26 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f04bc08a050 con 0x5621cdff0fb0 2024-09-10T10:51:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.995+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.997+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v104'57 uv49 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.997+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.998+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v104'58 uv58 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:13.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.998+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.000 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.999+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v104'59 uv59 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:13.999+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.000+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v104'60 uv60 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.001+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.002+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v104'61 uv61 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.002+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.003+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v104'62 uv56 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.003+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.004+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v104'63 uv63 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.004+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.006 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.005+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v104'64 uv64 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.005+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.006+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v104'65 uv65 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.006+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.007+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v104'66 uv66 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.008+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff33a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.009+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v104'67 uv67 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.009+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) -- 0x5621cdff42a0 con 0x5621cdfed5c0 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d0800640 1 -- 172.21.15.28:0/1120934707 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v104'68 uv68 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f04c4013420 con 0x5621cdfed5c0 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5621cdfe53b0 msgr2=0x5621cdfe77a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5621cdfe53b0 0x5621cdfe77a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5621cdfed5c0 msgr2=0x5621cdfef9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5621cdfed5c0 0x5621cdfef9b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5621cdff0fb0 msgr2=0x5621cdff38b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5621cdff0fb0 0x5621cdff38b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5621cdfe9870 msgr2=0x5621cdfec6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5621cdfe9870 0x5621cdfec6d0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5621cdfe0260 msgr2=0x5621cdfe3000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5621cdfe0260 0x5621cdfe3000 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 reap_dead start 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5621cdf31870 msgr2=0x5621cdf3e2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5621cdf31870 0x5621cdf3e2f0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 reap_dead start 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d1200640 1 -- 172.21.15.28:0/1120934707 reap_dead start 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 msgr2=0x7f04b006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 0x7f04b006ccd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f04c4015040 tx=0x7f04c400b040 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 msgr2=0x5621cdf339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00dcd0 tx=0x7f04bc00b6c0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 shutdown_connections 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f04b006a810 0x7f04b006ccd0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 --2- 172.21.15.28:0/1120934707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5621cdf3a3e0 0x5621cdf339f0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 >> 172.21.15.28:0/1120934707 conn(0x5621cdeb30a0 msgr2=0x5621cde97400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 shutdown_connections 2024-09-10T10:51:14.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.010+0000 7f04d3662f40 1 -- 172.21.15.28:0/1120934707 wait complete. 2024-09-10T10:51:14.016 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 200.00000001 2024-09-10T10:51:14.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 Processor -- start 2024-09-10T10:51:14.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 -- start start 2024-09-10T10:51:14.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4991ff0 0x5653e4a1ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 0x5653e4946ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5653e4992530 con 0x5653e4a209c0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653e48efb30 con 0x5653e4991ff0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.091+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5653e4a1fee0 con 0x5653e4a1d3e0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43782/0 (socket says 172.21.15.28:43782) 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 -- 172.21.15.28:0/583582127 learned_addr learned my addr 172.21.15.28:0/583582127 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dce00640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4991ff0 0x5653e4a1ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30d7e00640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 0x5653e4946ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 -- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 msgr2=0x5653e4946ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 0x5653e4946ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 -- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4991ff0 msgr2=0x5653e4a1ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4991ff0 0x5653e4a1ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 -- 172.21.15.28:0/583582127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e4a1c850 con 0x5653e4a209c0 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30d7e00640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 0x5653e4946ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.092+0000 7f30dd800640 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c480 tx=0x7f30d800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd0bb116e1067cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30d6a00640 1 -- 172.21.15.28:0/583582127 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80116b0 con 0x5653e4a209c0 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30d6a00640 1 -- 172.21.15.28:0/583582127 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d8017070 con 0x5653e4a209c0 2024-09-10T10:51:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30d6a00640 1 -- 172.21.15.28:0/583582127 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8011a90 con 0x5653e4a209c0 2024-09-10T10:51:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/583582127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 msgr2=0x5653e4947000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f30d800c480 tx=0x7f30d800c940 comp rx=0 tx=0).stop 2024-09-10T10:51:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/583582127 shutdown_connections 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4947000 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4a1d3e0 0x5653e4946ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/583582127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4991ff0 0x5653e4a1ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/583582127 >> 172.21.15.28:0/583582127 conn(0x5653e49960a0 msgr2=0x5653e4996480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/583582127 shutdown_connections 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.093+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/583582127 wait complete. 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.094+0000 7f30dfcb2f40 1 Processor -- start 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.094+0000 7f30dfcb2f40 1 -- start start 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4991ff0 0x5653e4a178a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4a1d3e0 0x5653e4a0f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4a19050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5653e4949b70 con 0x5653e4a209c0 2024-09-10T10:51:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653e4a1f120 con 0x5653e4a1d3e0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dfcb2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5653e4a1dca0 con 0x5653e4991ff0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4a19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4a19050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43788/0 (socket says 172.21.15.28:43788) 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dd800640 1 -- 172.21.15.28:0/1262478066 learned_addr learned my addr 172.21.15.28:0/1262478066 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30dce00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4991ff0 0x5653e4a178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.095+0000 7f30d7e00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4a1d3e0 0x5653e4a0f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 -- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4991ff0 msgr2=0x5653e4a178a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5653e4991ff0 0x5653e4a178a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 -- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4a1d3e0 msgr2=0x5653e4a0f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653e4a1d3e0 0x5653e4a0f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 -- 172.21.15.28:0/1262478066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e4a17100 con 0x5653e4a209c0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dd800640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4a19050 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f30d8013000 tx=0x7f30d80033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30d5600640 1 -- 172.21.15.28:0/1262478066 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d80129c0 con 0x5653e4a209c0 2024-09-10T10:51:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30d5600640 1 -- 172.21.15.28:0/1262478066 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30d8012b80 con 0x5653e4a209c0 2024-09-10T10:51:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5653e4a17de0 con 0x5653e4a209c0 2024-09-10T10:51:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30d5600640 1 -- 172.21.15.28:0/1262478066 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8024400 con 0x5653e4a209c0 2024-09-10T10:51:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.096+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e4947160 con 0x5653e4a209c0 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.098+0000 7f30d5600640 1 -- 172.21.15.28:0/1262478066 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f30d801c400 con 0x5653e4a209c0 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.099+0000 7f30d5600640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f30bc06a810 0x7f30bc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.099+0000 7f30d5600640 1 -- 172.21.15.28:0/1262478066 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7f30d808be80 con 0x5653e4a209c0 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.099+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5653e4a0ff30 0x5653e4aae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.099+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e104) -- 0x5653e4a1f5f0 con 0x5653e4a0ff30 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.100+0000 7f30dce00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f30bc06a810 0x7f30bc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.100+0000 7f30d7e00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5653e4a0ff30 0x5653e4aae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.100+0000 7f30dce00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f30bc06a810 0x7f30bc06ccd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f30d0057000 tx=0x7f30d0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.100+0000 7f30d7e00640 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5653e4a0ff30 0x5653e4aae1c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30d7e00640 1 -- 172.21.15.28:0/1262478066 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [delete] v104'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30c800b690 con 0x5653e4a0ff30 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5653e4a0ff30 msgr2=0x5653e4aae1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5653e4a0ff30 0x5653e4aae1c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f30bc06a810 msgr2=0x7f30bc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f30bc06a810 0x7f30bc06ccd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f30d0057000 tx=0x7f30d0052000 comp rx=0 tx=0).stop 2024-09-10T10:51:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 msgr2=0x5653e4a19050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 --2- 172.21.15.28:0/1262478066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5653e4a209c0 0x5653e4a19050 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f30d8013000 tx=0x7f30d80033a0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dd800640 1 -- 172.21.15.28:0/1262478066 reap_dead start 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 shutdown_connections 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 >> 172.21.15.28:0/1262478066 conn(0x5653e49960a0 msgr2=0x5653e49462a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 shutdown_connections 2024-09-10T10:51:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.102+0000 7f30dfcb2f40 1 -- 172.21.15.28:0/1262478066 wait complete. 2024-09-10T10:51:14.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:14.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.265+0000 7fa656e00640 1 Processor -- start 2024-09-10T10:51:14.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.265+0000 7fa656e00640 1 -- start start 2024-09-10T10:51:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 0x7fa65006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6500b3640 0x7fa6500b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6500b1170 con 0x7fa650161ac0 2024-09-10T10:51:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa6500ceba0 con 0x7fa6500b3640 2024-09-10T10:51:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa65006dfb0 con 0x7fa6500d2ac0 2024-09-10T10:51:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa656400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6500b3640 0x7fa6500b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa655a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa650161ac0 0x7fa6500d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60242/0 (socket says 172.21.15.28:60242) 2024-09-10T10:51:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.266+0000 7fa655a00640 1 -- 172.21.15.28:0/2083431095 learned_addr learned my addr 172.21.15.28:0/2083431095 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa655000640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 0x7fa65006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa656400640 1 -- 172.21.15.28:0/2083431095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 msgr2=0x7fa65006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa656400640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 0x7fa65006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa656400640 1 -- 172.21.15.28:0/2083431095 >> v1:172.21.15.28:6789/0 conn(0x7fa650161ac0 legacy=0x7fa6500d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa656400640 1 -- 172.21.15.28:0/2083431095 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6500b1450 con 0x7fa6500b3640 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa64fe00640 1 -- 172.21.15.28:0/2083431095 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479079696 0 0) 0x7fa6500b1170 con 0x7fa650161ac0 2024-09-10T10:51:14.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.267+0000 7fa655000640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 0x7fa65006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:14.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.268+0000 7fa656400640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500b1f30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7bf58aa07bdb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.268+0000 7fa64fe00640 1 -- 172.21.15.28:0/2083431095 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648007740 con 0x7fa6500b3640 2024-09-10T10:51:14.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.268+0000 7fa64fe00640 1 -- 172.21.15.28:0/2083431095 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6480078e0 con 0x7fa6500b3640 2024-09-10T10:51:14.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.268+0000 7fa64fe00640 1 -- 172.21.15.28:0/2083431095 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648007cb0 con 0x7fa6500b3640 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 -- 172.21.15.28:0/2083431095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 msgr2=0x7fa6500b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500b1f30 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 -- 172.21.15.28:0/2083431095 shutdown_connections 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500b1f30 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 --2- 172.21.15.28:0/2083431095 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6500d2ac0 0x7fa65006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 -- 172.21.15.28:0/2083431095 >> 172.21.15.28:0/2083431095 conn(0x7fa65008bfc0 msgr2=0x7fa6500d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 -- 172.21.15.28:0/2083431095 shutdown_connections 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.269+0000 7fa656e00640 1 -- 172.21.15.28:0/2083431095 wait complete. 2024-09-10T10:51:14.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.271+0000 7fa656e00640 1 Processor -- start 2024-09-10T10:51:14.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.271+0000 7fa656e00640 1 -- start start 2024-09-10T10:51:14.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6500d2ac0 0x7fa6500b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa650161ac0 0x7fa6500ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa650161ea0 con 0x7fa6500b3640 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6500b1a10 con 0x7fa6500d2ac0 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa650162020 con 0x7fa650161ac0 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43814/0 (socket says 172.21.15.28:43814) 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 -- 172.21.15.28:0/1019079878 learned_addr learned my addr 172.21.15.28:0/1019079878 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa656400640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa650161ac0 0x7fa6500ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655000640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6500d2ac0 0x7fa6500b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 -- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa650161ac0 msgr2=0x7fa6500ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa650161ac0 0x7fa6500ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 -- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6500d2ac0 msgr2=0x7fa6500b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6500d2ac0 0x7fa6500b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.272+0000 7fa655a00640 1 -- 172.21.15.28:0/1019079878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6500b5540 con 0x7fa6500b3640 2024-09-10T10:51:14.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa655a00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa64400c430 tx=0x7fa64400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6440074e0 con 0x7fa6500b3640 2024-09-10T10:51:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6440047e0 con 0x7fa6500b3640 2024-09-10T10:51:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa644004bd0 con 0x7fa6500b3640 2024-09-10T10:51:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6500b5820 con 0x7fa6500b3640 2024-09-10T10:51:14.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.273+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6500b5de0 con 0x7fa6500b3640 2024-09-10T10:51:14.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.274+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa65006e6c0 con 0x7fa6500b3640 2024-09-10T10:51:14.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.275+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa644011070 con 0x7fa6500b3640 2024-09-10T10:51:14.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.276+0000 7fa64d600640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 0x7fa63006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.276+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6531+0+0 (secure 0 0 0) 0x7fa64408ad60 con 0x7fa6500b3640 2024-09-10T10:51:14.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.276+0000 7fa655000640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 0x7fa63006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.277+0000 7fa655000640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 0x7fa63006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa6500c7b10 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.280+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa644058ce0 con 0x7fa6500b3640 2024-09-10T10:51:14.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.501+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa65015eb30 con 0x7fa6500b3640 2024-09-10T10:51:14.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.727+0000 7fa64d600640 1 -- 172.21.15.28:0/1019079878 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v114) ==== 162+0+0 (secure 0 0 0) 0x7fa64405cfe0 con 0x7fa6500b3640 2024-09-10T10:51:14.728 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 msgr2=0x7fa63006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 0x7fa63006e2f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa6500c7b10 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 msgr2=0x7fa6500c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa64400c430 tx=0x7fa64400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 shutdown_connections 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa63006be30 0x7fa63006e2f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa650161ac0 0x7fa6500ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6500d2ac0 0x7fa6500b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 --2- 172.21.15.28:0/1019079878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6500b3640 0x7fa6500c9f40 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.729+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 >> 172.21.15.28:0/1019079878 conn(0x7fa65008bfc0 msgr2=0x7fa6500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.730+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 shutdown_connections 2024-09-10T10:51:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.730+0000 7fa656e00640 1 -- 172.21.15.28:0/1019079878 wait complete. 2024-09-10T10:51:14.742 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:14.741+0000 7f43d9800640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:51:14.743 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:14.742+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:14.743 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:14.742+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:14.750 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:14.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.907+0000 7faa87800640 1 Processor -- start 2024-09-10T10:51:14.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.907+0000 7faa87800640 1 -- start start 2024-09-10T10:51:14.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.908+0000 7faa87800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 0x7faa8015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.908+0000 7faa87800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa80146140 con 0x7faa80072de0 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.908+0000 7faa87800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa800cd740 con 0x7faa80072320 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.908+0000 7faa87800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa800cda20 con 0x7faa80072700 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faa80072de0 0x7faa8015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60258/0 (socket says 172.21.15.28:60258) 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7fe00640 1 -- 172.21.15.28:0/1153458549 learned_addr learned my addr 172.21.15.28:0/1153458549 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:14.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa85800640 1 --2- 172.21.15.28:0/1153458549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 0x7faa8015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974306154 0 0) 0x7faa80146140 con 0x7faa80072de0 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa68003520 con 0x7faa80072de0 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752511453 0 0) 0x7faa800cd740 con 0x7faa80072320 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa80146140 con 0x7faa80072320 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa85800640 1 -- 172.21.15.28:0/1153458549 >> v1:172.21.15.138:6789/0 conn(0x7faa80072320 legacy=0x7faa800cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa85800640 1 -- 172.21.15.28:0/1153458549 >> v1:172.21.15.28:6789/0 conn(0x7faa80072de0 legacy=0x7faa8015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.909+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2697603329 0 0) 0x7faa68003520 con 0x7faa80072de0 2024-09-10T10:51:14.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.910+0000 7faa85800640 1 -- 172.21.15.28:0/1153458549 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa800cdd00 con 0x7faa80072700 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa85800640 1 --2- 172.21.15.28:0/1153458549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 0x7faa8015d8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa78009dc0 tx=0x7faa7802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee80db0d615b012f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c001670 con 0x7faa80072700 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa78007540 con 0x7faa80072700 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa7f400640 1 -- 172.21.15.28:0/1153458549 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78007910 con 0x7faa80072700 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa87800640 1 -- 172.21.15.28:0/1153458549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 msgr2=0x7faa8015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.911+0000 7faa87800640 1 --2- 172.21.15.28:0/1153458549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 0x7faa8015d8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa78009dc0 tx=0x7faa7802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.912+0000 7faa87800640 1 -- 172.21.15.28:0/1153458549 shutdown_connections 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.912+0000 7faa87800640 1 --2- 172.21.15.28:0/1153458549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072700 0x7faa8015d8f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.912+0000 7faa87800640 1 -- 172.21.15.28:0/1153458549 >> 172.21.15.28:0/1153458549 conn(0x7faa80067d70 msgr2=0x7faa8006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:14.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.912+0000 7faa87800640 1 -- 172.21.15.28:0/1153458549 shutdown_connections 2024-09-10T10:51:14.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.912+0000 7faa87800640 1 -- 172.21.15.28:0/1153458549 wait complete. 2024-09-10T10:51:14.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.913+0000 7faa87800640 1 Processor -- start 2024-09-10T10:51:14.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.913+0000 7faa87800640 1 -- start start 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072320 0x7faa80142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa80072de0 0x7faa800aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa8015dfb0 con 0x7faa80072700 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa8015de30 con 0x7faa80072de0 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa87800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa8015e130 con 0x7faa80072320 2024-09-10T10:51:14.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.914+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43826/0 (socket says 172.21.15.28:43826) 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 -- 172.21.15.28:0/524205009 learned_addr learned my addr 172.21.15.28:0/524205009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa84e00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072320 0x7faa80142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa85800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa80072de0 0x7faa800aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 -- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072320 msgr2=0x7faa80142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072320 0x7faa80142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 -- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa80072de0 msgr2=0x7faa800aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa80072de0 0x7faa800aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 -- 172.21.15.28:0/524205009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa800a9c90 con 0x7faa80072700 2024-09-10T10:51:14.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.915+0000 7faa7fe00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c750 tx=0x7faa6c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.916+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c005ad0 con 0x7faa80072700 2024-09-10T10:51:14.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.916+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c005c70 con 0x7faa80072700 2024-09-10T10:51:14.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.916+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa800a9f70 con 0x7faa80072700 2024-09-10T10:51:14.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.916+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c005e60 con 0x7faa80072700 2024-09-10T10:51:14.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.916+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa800aa530 con 0x7faa80072700 2024-09-10T10:51:14.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.917+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa800cce60 con 0x7faa80072700 2024-09-10T10:51:14.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.918+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa6c0072b0 con 0x7faa80072700 2024-09-10T10:51:14.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.918+0000 7faa7cc00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 0x7faa6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:14.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.919+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7faa6c08aff0 con 0x7faa80072700 2024-09-10T10:51:14.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.919+0000 7faa84e00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 0x7faa6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:14.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.920+0000 7faa84e00640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 0x7faa6006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faa74001650 tx=0x7faa74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:14.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:14.923+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faa6c059700 con 0x7faa80072700 2024-09-10T10:51:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.141+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa8006beb0 con 0x7faa80072700 2024-09-10T10:51:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.142+0000 7faa7cc00640 1 -- 172.21.15.28:0/524205009 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3566 (secure 0 0 0) 0x7faa6c059000 con 0x7faa80072700 2024-09-10T10:51:15.143 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:15.143 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":116,"btime":"2024-09-10T10:51:14:743488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:14.743434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 116 2024-09-10T10:51:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 msgr2=0x7faa6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 0x7faa6006e2f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faa74001650 tx=0x7faa74009040 comp rx=0 tx=0).stop 2024-09-10T10:51:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 msgr2=0x7faa8015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c750 tx=0x7faa6c00cc10 comp rx=0 tx=0).stop 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 shutdown_connections 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa6006be30 0x7faa6006e2f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa80072de0 0x7faa800aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa80072700 0x7faa8015bee0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 --2- 172.21.15.28:0/524205009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80072320 0x7faa80142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 >> 172.21.15.28:0/524205009 conn(0x7faa80067d70 msgr2=0x7faa800b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 shutdown_connections 2024-09-10T10:51:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.144+0000 7faa87800640 1 -- 172.21.15.28:0/524205009 wait complete. 2024-09-10T10:51:15.162 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:15.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:15.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.318+0000 7fbd74000640 1 Processor -- start 2024-09-10T10:51:15.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.318+0000 7fbd74000640 1 -- start start 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.319+0000 7fbd74000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd6c072700 0x7fbd6c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.319+0000 7fbd74000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6c146140 con 0x7fbd6c072320 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.319+0000 7fbd74000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6c0cd740 con 0x7fbd6c072de0 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.319+0000 7fbd74000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd6c0cda20 con 0x7fbd6c072700 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd71600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbd6c072320 0x7fbd6c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60272/0 (socket says 172.21.15.28:60272) 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd71600640 1 -- 172.21.15.28:0/3765789170 learned_addr learned my addr 172.21.15.28:0/3765789170 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:15.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd72000640 1 --2- 172.21.15.28:0/3765789170 >> v2:172.21.15.28:3300/0 conn(0x7fbd6c072700 0x7fbd6c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd6be00640 1 -- 172.21.15.28:0/3765789170 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3795121131 0 0) 0x7fbd6c146140 con 0x7fbd6c072320 2024-09-10T10:51:15.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd6be00640 1 -- 172.21.15.28:0/3765789170 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd54003520 con 0x7fbd6c072320 2024-09-10T10:51:15.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd72000640 1 -- 172.21.15.28:0/3765789170 >> v1:172.21.15.138:6789/0 conn(0x7fbd6c072de0 legacy=0x7fbd6c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd72000640 1 -- 172.21.15.28:0/3765789170 >> v1:172.21.15.28:6789/0 conn(0x7fbd6c072320 legacy=0x7fbd6c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.320+0000 7fbd72000640 1 -- 172.21.15.28:0/3765789170 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c0cdd00 con 0x7fbd6c072700 2024-09-10T10:51:15.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.321+0000 7fbd72000640 1 --2- 172.21.15.28:0/3765789170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072700 0x7fbd6c15d8f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009d90 tx=0x7fbd6402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37df0dcf4a8be917 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd6be00640 1 -- 172.21.15.28:0/3765789170 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64007740 con 0x7fbd6c072700 2024-09-10T10:51:15.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd6be00640 1 -- 172.21.15.28:0/3765789170 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd640078e0 con 0x7fbd6c072700 2024-09-10T10:51:15.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd6be00640 1 -- 172.21.15.28:0/3765789170 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64007cb0 con 0x7fbd6c072700 2024-09-10T10:51:15.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 -- 172.21.15.28:0/3765789170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072700 msgr2=0x7fbd6c15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 --2- 172.21.15.28:0/3765789170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072700 0x7fbd6c15d8f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009d90 tx=0x7fbd6402a660 comp rx=0 tx=0).stop 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 -- 172.21.15.28:0/3765789170 shutdown_connections 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 --2- 172.21.15.28:0/3765789170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072700 0x7fbd6c15d8f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 -- 172.21.15.28:0/3765789170 >> 172.21.15.28:0/3765789170 conn(0x7fbd6c067d70 msgr2=0x7fbd6c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 -- 172.21.15.28:0/3765789170 shutdown_connections 2024-09-10T10:51:15.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.322+0000 7fbd74000640 1 -- 172.21.15.28:0/3765789170 wait complete. 2024-09-10T10:51:15.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.324+0000 7fbd74000640 1 Processor -- start 2024-09-10T10:51:15.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.324+0000 7fbd74000640 1 -- start start 2024-09-10T10:51:15.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c072700 0x7fbd6c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c072de0 0x7fbd6c1e7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c15dfb0 con 0x7fbd6c072320 2024-09-10T10:51:15.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c15de30 con 0x7fbd6c072700 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd74000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6c15e130 con 0x7fbd6c072de0 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd71600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd71600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43834/0 (socket says 172.21.15.28:43834) 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.325+0000 7fbd71600640 1 -- 172.21.15.28:0/3254356386 learned_addr learned my addr 172.21.15.28:0/3254356386 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd70c00640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c072700 0x7fbd6c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd72000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c072de0 0x7fbd6c1e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 -- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c072de0 msgr2=0x7fbd6c1e7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c072de0 0x7fbd6c1e7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 -- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c072700 msgr2=0x7fbd6c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c072700 0x7fbd6c0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 -- 172.21.15.28:0/3254356386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c0c9630 con 0x7fbd6c072320 2024-09-10T10:51:15.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd71600640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000c500 tx=0x7fbd6000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60017070 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd60005ce0 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60007330 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c0c9910 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.326+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c0c9ed0 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.328+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c0cce60 con 0x7fbd6c072320 2024-09-10T10:51:15.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.329+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd6001a070 con 0x7fbd6c072320 2024-09-10T10:51:15.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.329+0000 7fbd69600640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 0x7fbd4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.330+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7fbd6008a5d0 con 0x7fbd6c072320 2024-09-10T10:51:15.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.330+0000 7fbd70c00640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 0x7fbd4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.331+0000 7fbd70c00640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 0x7fbd4c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007fd0 tx=0x7fbd58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.334+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd60058d40 con 0x7fbd6c072320 2024-09-10T10:51:15.452 INFO:tasks.ceph.mds.c.smithi028.stderr: -36> 2024-09-10T10:51:10.698+0000 7f6f9aa00640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (2) No such file or directory 2024-09-10T10:51:15.517 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:51:15.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.582+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd6c144440 con 0x7fbd6c072320 2024-09-10T10:51:15.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.583+0000 7fbd69600640 1 -- 172.21.15.28:0/3254356386 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3566 (secure 0 0 0) 0x7fbd60058640 con 0x7fbd6c072320 2024-09-10T10:51:15.584 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:15.584 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":116,"btime":"2024-09-10T10:51:14:743488+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:14.743434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:15.585 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 116 2024-09-10T10:51:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 msgr2=0x7fbd4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 0x7fbd4c06e2f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbd58007fd0 tx=0x7fbd58009040 comp rx=0 tx=0).stop 2024-09-10T10:51:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 msgr2=0x7fbd6c15bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fbd6000c500 tx=0x7fbd6000c9c0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 shutdown_connections 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd4c06be30 0x7fbd4c06e2f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c072de0 0x7fbd6c1e7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c072700 0x7fbd6c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 --2- 172.21.15.28:0/3254356386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd6c072320 0x7fbd6c15bc00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 >> 172.21.15.28:0/3254356386 conn(0x7fbd6c067d70 msgr2=0x7fbd6c0c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 shutdown_connections 2024-09-10T10:51:15.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.585+0000 7fbd74000640 1 -- 172.21.15.28:0/3254356386 wait complete. 2024-09-10T10:51:15.601 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:15.602 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:15.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.758+0000 7fceeb800640 1 Processor -- start 2024-09-10T10:51:15.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.758+0000 7fceeb800640 1 -- start start 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.759+0000 7fceeb800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcee4072320 0x7fcee40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.759+0000 7fceeb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.759+0000 7fceeb800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee4146140 con 0x7fcee4072700 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fceeb800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fcee40cd740 con 0x7fcee4072320 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fceeb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcee40cda20 con 0x7fcee4072de0 2024-09-10T10:51:15.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcee4072320 0x7fcee40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcee4072320 0x7fcee40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43848/0 (socket says 172.21.15.28:43848) 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 -- 172.21.15.28:0/2319490799 learned_addr learned my addr 172.21.15.28:0/2319490799 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee3e00640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 -- 172.21.15.28:0/2319490799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 msgr2=0x7fcee406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 -- 172.21.15.28:0/2319490799 >> v1:172.21.15.138:6789/0 conn(0x7fcee4072700 legacy=0x7fcee406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee8e00640 1 -- 172.21.15.28:0/2319490799 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee40cdd00 con 0x7fcee4072320 2024-09-10T10:51:15.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.760+0000 7fcee3e00640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:15.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.761+0000 7fcee8e00640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee40cce60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fced8009870 tx=0x7fced802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43d968ddddb993e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fcee3400640 1 -- 172.21.15.28:0/2319490799 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8037070 con 0x7fcee4072320 2024-09-10T10:51:15.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fcee3400640 1 -- 172.21.15.28:0/2319490799 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8032440 con 0x7fcee4072320 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fcee3400640 1 -- 172.21.15.28:0/2319490799 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8032810 con 0x7fcee4072320 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 -- 172.21.15.28:0/2319490799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 msgr2=0x7fcee40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee40cce60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fced8009870 tx=0x7fced802a180 comp rx=0 tx=0).stop 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 -- 172.21.15.28:0/2319490799 shutdown_connections 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 --2- 172.21.15.28:0/2319490799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee40cce60 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 -- 172.21.15.28:0/2319490799 >> 172.21.15.28:0/2319490799 conn(0x7fcee4067d70 msgr2=0x7fcee406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 -- 172.21.15.28:0/2319490799 shutdown_connections 2024-09-10T10:51:15.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.762+0000 7fceeb800640 1 -- 172.21.15.28:0/2319490799 wait complete. 2024-09-10T10:51:15.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.764+0000 7fceeb800640 1 Processor -- start 2024-09-10T10:51:15.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.764+0000 7fceeb800640 1 -- start start 2024-09-10T10:51:15.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4072700 0x7fcee40a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee4143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cd520 con 0x7fcee4072320 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cd3a0 con 0x7fcee4072700 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fceeb800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcee406ed00 con 0x7fcee4072de0 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fcee8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fcee8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43858/0 (socket says 172.21.15.28:43858) 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fcee8e00640 1 -- 172.21.15.28:0/2089884103 learned_addr learned my addr 172.21.15.28:0/2089884103 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fcee3e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4072700 0x7fcee40a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.765+0000 7fcee9800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee4143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 -- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 msgr2=0x7fcee4143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee4143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 -- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4072700 msgr2=0x7fcee40a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:15.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4072700 0x7fcee40a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:15.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 -- 172.21.15.28:0/2089884103 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee40caf40 con 0x7fcee4072320 2024-09-10T10:51:15.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee9800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee4143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee8e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fced8009870 tx=0x7fced80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8037040 con 0x7fcee4072320 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced8042070 con 0x7fcee4072320 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced8030bb0 con 0x7fcee4072320 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee40aa7d0 con 0x7fcee4072320 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.766+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee40aad90 con 0x7fcee4072320 2024-09-10T10:51:15.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.767+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee40cce60 con 0x7fcee4072320 2024-09-10T10:51:15.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.768+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fced8047460 con 0x7fcee4072320 2024-09-10T10:51:15.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.770+0000 7fcee0c00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:15.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.770+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7fced80abe60 con 0x7fcee4072320 2024-09-10T10:51:15.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.770+0000 7fcee3e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:15.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.771+0000 7fcee3e00640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 0x7fcec406e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b3180 tx=0x7fced0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:15.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.774+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fced807a3e0 con 0x7fcee4072320 2024-09-10T10:51:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.997+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcee406d090 con 0x7fcee4072320 2024-09-10T10:51:15.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:15.998+0000 7fcee0c00640 1 -- 172.21.15.28:0/2089884103 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+4392 (secure 0 0 0) 0x7fced8079ce0 con 0x7fcee4072320 2024-09-10T10:51:15.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:15.999 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":117,"btime":"2024-09-10T10:51:15:745013+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7797,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3452166242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3452166242},{"type":"v1","addr":"172.21.15.28:6834","nonce":3452166242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:14.743434+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:15.999 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 117 2024-09-10T10:51:16.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 msgr2=0x7fcec406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 0x7fcec406e2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcee40b3180 tx=0x7fced0008040 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 msgr2=0x7fcee413f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fced8009870 tx=0x7fced80046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 shutdown_connections 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcec406be30 0x7fcec406e2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcee4072de0 0x7fcee4143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcee4072700 0x7fcee40a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 --2- 172.21.15.28:0/2089884103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee4072320 0x7fcee413f0a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.000+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 >> 172.21.15.28:0/2089884103 conn(0x7fcee4067d70 msgr2=0x7fcee40c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.001+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 shutdown_connections 2024-09-10T10:51:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.001+0000 7fceeb800640 1 -- 172.21.15.28:0/2089884103 wait complete. 2024-09-10T10:51:16.017 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 200.00000001' led to DAMAGED state 2024-09-10T10:51:16.017 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_openfiles.0' 2024-09-10T10:51:16.018 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:16.018 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:16.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:16.054 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:16.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:16.054 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:16.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:16.129 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:16.130 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:16.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.327+0000 7fd281e00640 1 Processor -- start 2024-09-10T10:51:16.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.327+0000 7fd281e00640 1 -- start start 2024-09-10T10:51:16.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.328+0000 7fd281e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c0cfcc0 0x7fd27c06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.328+0000 7fd281e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.328+0000 7fd281e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd27c0d2700 con 0x7fd27c15e760 2024-09-10T10:51:16.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.328+0000 7fd281e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd27c0cedb0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.328+0000 7fd281e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c0cf090 con 0x7fd27c0cfcc0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43870/0 (socket says 172.21.15.28:43870) 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 -- 172.21.15.28:0/3505375364 learned_addr learned my addr 172.21.15.28:0/3505375364 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27a000640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c0cfcc0 0x7fd27c06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 -- 172.21.15.28:0/3505375364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c0cfcc0 msgr2=0x7fd27c06cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c0cfcc0 0x7fd27c06cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 -- 172.21.15.28:0/3505375364 >> v1:172.21.15.138:6789/0 conn(0x7fd27c15e760 legacy=0x7fd27c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.329+0000 7fd27b400640 1 -- 172.21.15.28:0/3505375364 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0b3c80 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.330+0000 7fd27b400640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd274009e50 tx=0x7fd27402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6c31c1cabe0c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.330+0000 7fd279600640 1 -- 172.21.15.28:0/3505375364 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274036070 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.330+0000 7fd279600640 1 -- 172.21.15.28:0/3505375364 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274031600 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.330+0000 7fd279600640 1 -- 172.21.15.28:0/3505375364 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740319d0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 -- 172.21.15.28:0/3505375364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 msgr2=0x7fd27c0d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd274009e50 tx=0x7fd27402a730 comp rx=0 tx=0).stop 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 -- 172.21.15.28:0/3505375364 shutdown_connections 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0d2ea0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 --2- 172.21.15.28:0/3505375364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c0cfcc0 0x7fd27c06cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 -- 172.21.15.28:0/3505375364 >> 172.21.15.28:0/3505375364 conn(0x7fd27c08bfc0 msgr2=0x7fd27c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 -- 172.21.15.28:0/3505375364 shutdown_connections 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.331+0000 7fd281e00640 1 -- 172.21.15.28:0/3505375364 wait complete. 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.333+0000 7fd281e00640 1 Processor -- start 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.333+0000 7fd281e00640 1 -- start start 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd27c0cfcc0 0x7fd27c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c15e760 0x7fd27c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c0d33e0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd27c0b38a0 con 0x7fd27c0cfcc0 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd281e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd27c0d3560 con 0x7fd27c15e760 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd27c0cfcc0 0x7fd27c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43876/0 (socket says 172.21.15.28:43876) 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 -- 172.21.15.28:0/2186338788 learned_addr learned my addr 172.21.15.28:0/2186338788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27b400640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c15e760 0x7fd27c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 -- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c15e760 msgr2=0x7fd27c0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c15e760 0x7fd27c0c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 -- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd27c0cfcc0 msgr2=0x7fd27c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd27c0cfcc0 0x7fd27c0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.334+0000 7fd27a000640 1 -- 172.21.15.28:0/2186338788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27c0c8310 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd27a000640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d7d0 tx=0x7fd26800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26800b6c0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd268019070 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26800ba80 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27c0c85f0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.335+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27c0c8bb0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.336+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27c0d38e0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.337+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd26800bc20 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.338+0000 7fd273400640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 0x7fd25c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.338+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7fd26808b5d0 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.338+0000 7fd27aa00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 0x7fd25c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.339+0000 7fd27aa00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 0x7fd25c06e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000f60 tx=0x7fd26c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.342+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd268059500 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.561+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd27c0b6d20 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.751+0000 7fd273400640 1 -- 172.21.15.28:0/2186338788 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v118) ==== 161+0+0 (secure 0 0 0) 0x7fd26805d800 con 0x7fd27c0d4fd0 2024-09-10T10:51:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:16.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 msgr2=0x7fd25c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 0x7fd25c06e2f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000f60 tx=0x7fd26c009040 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 msgr2=0x7fd27c0b96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd26800d7d0 tx=0x7fd26800dc90 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 shutdown_connections 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd25c06be30 0x7fd25c06e2f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd27c15e760 0x7fd27c0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd27c0d4fd0 0x7fd27c0b96a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 --2- 172.21.15.28:0/2186338788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd27c0cfcc0 0x7fd27c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 >> 172.21.15.28:0/2186338788 conn(0x7fd27c08bfc0 msgr2=0x7fd27c0b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 shutdown_connections 2024-09-10T10:51:16.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.753+0000 7fd281e00640 1 -- 172.21.15.28:0/2186338788 wait complete. 2024-09-10T10:51:16.772 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.928+0000 7f2e53600640 1 Processor -- start 2024-09-10T10:51:16.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.928+0000 7f2e53600640 1 -- start start 2024-09-10T10:51:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.929+0000 7f2e53600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e4c15b300 0x7f2e4c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.929+0000 7f2e53600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e53600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4c0cdbe0 con 0x7f2e4c0cd3f0 2024-09-10T10:51:16.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e53600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4c0728a0 con 0x7f2e4c15b300 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e53600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cd900 con 0x7f2e4c06f8d0 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e4c15b300 0x7f2e4c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e4c15b300 0x7f2e4c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43882/0 (socket says 172.21.15.28:43882) 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 -- 172.21.15.28:0/1680798065 learned_addr learned my addr 172.21.15.28:0/1680798065 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e51600640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 -- 172.21.15.28:0/1680798065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 msgr2=0x7f2e4c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 -- 172.21.15.28:0/1680798065 >> v1:172.21.15.138:6789/0 conn(0x7f2e4c0cd3f0 legacy=0x7f2e4c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e50c00640 1 -- 172.21.15.28:0/1680798065 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c072250 con 0x7f2e4c15b300 2024-09-10T10:51:16.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.930+0000 7f2e51600640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.931+0000 7f2e50c00640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c0cceb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bb10 tx=0x7f2e4002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254bb8c90772eea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e4b400640 1 -- 172.21.15.28:0/1680798065 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40036020 con 0x7f2e4c15b300 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e4b400640 1 -- 172.21.15.28:0/1680798065 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e40032820 con 0x7f2e4c15b300 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e4b400640 1 -- 172.21.15.28:0/1680798065 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e40032bf0 con 0x7f2e4c15b300 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 -- 172.21.15.28:0/1680798065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 msgr2=0x7f2e4c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c0cceb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000bb10 tx=0x7f2e4002c920 comp rx=0 tx=0).stop 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 -- 172.21.15.28:0/1680798065 shutdown_connections 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 --2- 172.21.15.28:0/1680798065 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c0cceb0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 -- 172.21.15.28:0/1680798065 >> 172.21.15.28:0/1680798065 conn(0x7f2e4c067d70 msgr2=0x7f2e4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 -- 172.21.15.28:0/1680798065 shutdown_connections 2024-09-10T10:51:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.932+0000 7f2e53600640 1 -- 172.21.15.28:0/1680798065 wait complete. 2024-09-10T10:51:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.934+0000 7f2e53600640 1 Processor -- start 2024-09-10T10:51:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.934+0000 7f2e53600640 1 -- start start 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 0x7f2e4c1e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0d0160 con 0x7f2e4c15b300 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0cffe0 con 0x7f2e4c0cd3f0 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e53600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c072d30 con 0x7f2e4c06f8d0 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e51600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e51600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43896/0 (socket says 172.21.15.28:43896) 2024-09-10T10:51:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.935+0000 7f2e51600640 1 -- 172.21.15.28:0/2490815062 learned_addr learned my addr 172.21.15.28:0/2490815062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e50c00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e4be00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 0x7f2e4c1e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 -- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 msgr2=0x7f2e4c1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 -- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 msgr2=0x7f2e4c1e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 0x7f2e4c1e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 -- 172.21.15.28:0/2490815062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c142bf0 con 0x7f2e4c15b300 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e4be00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 0x7f2e4c1e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e51600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2e44007ea0 tx=0x7f2e44007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4401a070 con 0x7f2e4c15b300 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e44015600 con 0x7f2e4c15b300 2024-09-10T10:51:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.936+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e440159f0 con 0x7f2e4c15b300 2024-09-10T10:51:16.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.937+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c155c00 con 0x7f2e4c15b300 2024-09-10T10:51:16.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.937+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c0cc470 con 0x7f2e4c15b300 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.938+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4c158f70 con 0x7f2e4c15b300 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.938+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e4401d070 con 0x7f2e4c15b300 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.939+0000 7f2e48c00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 0x7f2e2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.939+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f2e4408b4c0 con 0x7f2e4c15b300 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.940+0000 7f2e50c00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 0x7f2e2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:16.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.941+0000 7f2e50c00640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 0x7f2e2c06e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2e40037000 tx=0x7f2e40032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:16.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:16.944+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e44010070 con 0x7f2e4c15b300 2024-09-10T10:51:17.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.198+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f2e4c158c90 con 0x7f2e4c15b300 2024-09-10T10:51:17.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.757+0000 7f2e48c00640 1 -- 172.21.15.28:0/2490815062 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v119) ==== 101+0+0 (secure 0 0 0) 0x7f2e440593f0 con 0x7f2e4c15b300 2024-09-10T10:51:17.759 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:17.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 msgr2=0x7f2e2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:17.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 0x7f2e2c06e2f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2e40037000 tx=0x7f2e40032000 comp rx=0 tx=0).stop 2024-09-10T10:51:17.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 msgr2=0x7f2e4c1426b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f2e44007ea0 tx=0x7f2e44007f90 comp rx=0 tx=0).stop 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 shutdown_connections 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e2c06be30 0x7f2e2c06e2f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e4c15b300 0x7f2e4c1426b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e4c0cd3f0 0x7f2e4c1e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 --2- 172.21.15.28:0/2490815062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e4c06f8d0 0x7f2e4c1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 >> 172.21.15.28:0/2490815062 conn(0x7f2e4c067d70 msgr2=0x7f2e4c0af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 shutdown_connections 2024-09-10T10:51:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.761+0000 7f2e53600640 1 -- 172.21.15.28:0/2490815062 wait complete. 2024-09-10T10:51:17.780 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:17.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.857+0000 7f855ad26f40 1 Processor -- start 2024-09-10T10:51:17.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 -- start start 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edcb5470 0x5623edd884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 0x5623edce45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5623edc5db30 con 0x5623edd8e9c0 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623edc61b70 con 0x5623edd8b3e0 2024-09-10T10:51:17.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.858+0000 7f855ad26f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5623edd8dee0 con 0x5623edcb5470 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43906/0 (socket says 172.21.15.28:43906) 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 -- 172.21.15.28:0/3170342292 learned_addr learned my addr 172.21.15.28:0/3170342292 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557400640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edcb5470 0x5623edd884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8556a00640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 0x5623edce45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 -- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edcb5470 msgr2=0x5623edd884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edcb5470 0x5623edd884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 -- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 msgr2=0x5623edce45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 0x5623edce45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 -- 172.21.15.28:0/3170342292 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623edce9900 con 0x5623edd8e9c0 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8556a00640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 0x5623edce45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:17.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.859+0000 7f8557e00640 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8550025a70 tx=0x7f8550051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebba2b3c68181fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f8555600640 1 -- 172.21.15.28:0/3170342292 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550058b50 con 0x5623edd8e9c0 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f8555600640 1 -- 172.21.15.28:0/3170342292 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550066070 con 0x5623edd8e9c0 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f8555600640 1 -- 172.21.15.28:0/3170342292 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550061400 con 0x5623edd8e9c0 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 -- 172.21.15.28:0/3170342292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 msgr2=0x5623edd8edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8550025a70 tx=0x7f8550051000 comp rx=0 tx=0).stop 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 -- 172.21.15.28:0/3170342292 shutdown_connections 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623edd8edd0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edd8b3e0 0x5623edce45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 --2- 172.21.15.28:0/3170342292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edcb5470 0x5623edd884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 -- 172.21.15.28:0/3170342292 >> 172.21.15.28:0/3170342292 conn(0x5623edd040a0 msgr2=0x5623edd8b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 -- 172.21.15.28:0/3170342292 shutdown_connections 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.860+0000 7f855ad26f40 1 -- 172.21.15.28:0/3170342292 wait complete. 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.861+0000 7f855ad26f40 1 Processor -- start 2024-09-10T10:51:17.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.861+0000 7f855ad26f40 1 -- start start 2024-09-10T10:51:17.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edcb5470 0x5623edd8c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edd8b3e0 0x5623edce76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5623edd04520 con 0x5623edd8e9c0 2024-09-10T10:51:17.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5623edd8d9e0 con 0x5623edcb5470 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f855ad26f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5623edd88db0 con 0x5623edd8b3e0 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f8557e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f8557e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43922/0 (socket says 172.21.15.28:43922) 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.862+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 learned_addr learned my addr 172.21.15.28:0/1462694672 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edcb5470 0x5623edd8c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edd8b3e0 0x5623edce76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edd8b3e0 msgr2=0x5623edce76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edd8b3e0 0x5623edce76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edcb5470 msgr2=0x5623edd8c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edcb5470 0x5623edd8c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623edd81360 con 0x5623edd8e9c0 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5623edcb5470 0x5623edd8c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:17.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5623edd8b3e0 0x5623edce76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f8557e00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8550063040 tx=0x7f8550003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f854fe00640 1 -- 172.21.15.28:0/1462694672 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855006e070 con 0x5623edd8e9c0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f854fe00640 1 -- 172.21.15.28:0/1462694672 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8550005250 con 0x5623edd8e9c0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f854fe00640 1 -- 172.21.15.28:0/1462694672 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8550004600 con 0x5623edd8e9c0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.863+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623edce6dc0 con 0x5623edd8e9c0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.864+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623edd00280 con 0x5623edd8e9c0 2024-09-10T10:51:17.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.865+0000 7f854fe00640 1 -- 172.21.15.28:0/1462694672 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8550071070 con 0x5623edd8e9c0 2024-09-10T10:51:17.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.866+0000 7f854fe00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 0x7f853806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.866+0000 7f854fe00640 1 -- 172.21.15.28:0/1462694672 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f85500cf1d0 con 0x5623edd8e9c0 2024-09-10T10:51:17.868 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.866+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 0x5623ede27e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.867+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623edd854d0 con 0x5623edd03600 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.867+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 0x7f853806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.867+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 0x5623ede27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.867+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 0x7f853806cd00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f854800e270 tx=0x7f85480025f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.868+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 0x5623ede27e60 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.869+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v105'164 uv138 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.869+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623edce7be0 con 0x5623edd03600 2024-09-10T10:51:17.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.871+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v105'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.871+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623edd8ca30 con 0x5623edd03600 2024-09-10T10:51:17.874 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:17.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.873+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v105'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.873+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623edd84850 con 0x5623edd03600 2024-09-10T10:51:17.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.874+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v105'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.874+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede316f0 con 0x5623edd03600 2024-09-10T10:51:17.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.875+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v105'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.876+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede306e0 con 0x5623edd03600 2024-09-10T10:51:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.878+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v105'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.878+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30ad0 con 0x5623edd03600 2024-09-10T10:51:17.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.879+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v105'170 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.879+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30ec0 con 0x5623edd03600 2024-09-10T10:51:17.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.881+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v105'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.881+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30ec0 con 0x5623edd03600 2024-09-10T10:51:17.884 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.882+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v105'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.883+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.884+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v105'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.884+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.886+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v105'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.886+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.888+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v105'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.888+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.890+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v105'176 uv151 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.890+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.891+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v105'177 uv177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.891+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.893 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:17.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.892+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v105'178 uv178 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.892+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.894+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v105'179 uv179 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.894+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.895+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v105'180 uv180 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.895+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede30e70 con 0x5623edd03600 2024-09-10T10:51:17.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.897+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v105'181 uv181 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.897+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.899+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v105'182 uv182 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.899+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.900+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v105'183 uv157 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.900+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.902+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v105'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.902+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.904 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:17.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.903+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v105'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.903+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.905+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v105'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.905+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.906+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v105'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.906+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.908+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v105'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.908+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.909+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v105'189 uv163 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.909+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.911+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v105'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.911+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.913 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:17.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.912+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v105'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.912+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31230 con 0x5623edd03600 2024-09-10T10:51:17.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.914+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v105'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.914+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31130 con 0x5623edd03600 2024-09-10T10:51:17.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.916+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v105'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.916+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede31130 con 0x5623edd03600 2024-09-10T10:51:17.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.918+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v105'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623edd03600 2024-09-10T10:51:17.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.918+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 0x5623ede34390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.918+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede348d0 con 0x5623ede31fa0 2024-09-10T10:51:17.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.918+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 0x5623ede34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.918+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 0x5623ede34390 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.920+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v105'129 uv113 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.920+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede34c70 con 0x5623ede31fa0 2024-09-10T10:51:17.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.921+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v105'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.922+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede35010 con 0x5623ede31fa0 2024-09-10T10:51:17.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.923+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v105'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.924 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:17.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.923+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede353b0 con 0x5623ede31fa0 2024-09-10T10:51:17.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.924+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v105'132 uv132 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.925+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede35750 con 0x5623ede31fa0 2024-09-10T10:51:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.926+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v105'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.926+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede35af0 con 0x5623ede31fa0 2024-09-10T10:51:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.927+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v105'134 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.927+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede35e90 con 0x5623ede31fa0 2024-09-10T10:51:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.929+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v105'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.929+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.930+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v105'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.931 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.930+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.931+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v105'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.931+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.933+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v105'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.933+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.934+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v105'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.935 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.934+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3a920 con 0x5623ede31fa0 2024-09-10T10:51:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.936+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [write 0~18118] v105'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.936+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3a920 con 0x5623ede31fa0 2024-09-10T10:51:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.937+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [omap-set-header] v105'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.937+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.939+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 606.00000000 [create] v105'142 uv127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.939+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.940+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 606.00000000 [delete] v105'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.940+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.942+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v105'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:17.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.942+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede37240 con 0x5623ede31fa0 2024-09-10T10:51:17.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.943+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [setxattr (30)] v105'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.943+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede37240 con 0x5623ede31fa0 2024-09-10T10:51:17.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.944+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [setxattr (62)] v105'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.944+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede36230 con 0x5623ede31fa0 2024-09-10T10:51:17.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.946+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [omap-set-header] v105'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede31fa0 2024-09-10T10:51:17.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.946+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5623ede36740 0x5623ede38b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.946+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:51 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede39190 con 0x5623ede36740 2024-09-10T10:51:17.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.946+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5623ede36740 0x5623ede38b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.947+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5623ede36740 0x5623ede38b30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.948+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(51 607.00000000 [create] v105'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.948+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:52 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede39250 con 0x5623ede36740 2024-09-10T10:51:17.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.950+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(52 607.00000000 [delete] v105'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.950+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede39670 con 0x5623ede36740 2024-09-10T10:51:17.953 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:17.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.952+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(53 607.00000000 [create] v105'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.952+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3a680 con 0x5623ede36740 2024-09-10T10:51:17.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.954+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(54 607.00000000 [setxattr (30)] v105'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.954+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3a680 con 0x5623ede36740 2024-09-10T10:51:17.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.957+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(55 607.00000000 [setxattr (62)] v105'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.957+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede39670 con 0x5623ede36740 2024-09-10T10:51:17.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.960+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(56 607.00000000 [omap-set-header] v105'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.960+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5623ede39cf0 0x5623ede3c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.960+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:57 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3c6f0 con 0x5623ede39cf0 2024-09-10T10:51:17.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.960+0000 7f8557e00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5623ede39cf0 0x5623ede3c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.960+0000 7f8557e00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5623ede39cf0 0x5623ede3c0e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.962+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(57 mds0_openfiles.0 [create] v105'26 uv25 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.962+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:58 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3cc00 con 0x5623ede39cf0 2024-09-10T10:51:17.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.963+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(58 mds0_openfiles.0 [delete] v105'27 uv27 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.963+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d0c0 con 0x5623ede39cf0 2024-09-10T10:51:17.964 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:17.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.964+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(59 mds0_openfiles.0 [create] v105'28 uv28 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.964+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d9b0 con 0x5623ede39cf0 2024-09-10T10:51:17.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.965+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(60 mds0_openfiles.0 [omap-set-header] v105'29 uv29 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.965+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:61 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3dd50 con 0x5623ede36740 2024-09-10T10:51:17.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.966+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(61 608.00000000 [create] v105'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.966+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:62 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3dd50 con 0x5623ede36740 2024-09-10T10:51:17.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.967+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(62 608.00000000 [delete] v105'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.967+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3dd50 con 0x5623ede36740 2024-09-10T10:51:17.969 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:17.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.968+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(63 608.00000000 [create] v105'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.968+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3ed60 con 0x5623ede36740 2024-09-10T10:51:17.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.969+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(64 608.00000000 [setxattr (30)] v105'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.969+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3ed60 con 0x5623ede36740 2024-09-10T10:51:17.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.970+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(65 608.00000000 [setxattr (62)] v105'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.970+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3dd50 con 0x5623ede36740 2024-09-10T10:51:17.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.971+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(66 608.00000000 [omap-set-header] v105'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.971+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:67 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3dd50 con 0x5623ede39cf0 2024-09-10T10:51:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.973+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(67 604.00000000 [create] v105'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.973+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:68 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e0f0 con 0x5623ede39cf0 2024-09-10T10:51:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.974+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(68 604.00000000 [delete] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.974+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e490 con 0x5623ede39cf0 2024-09-10T10:51:17.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.976+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(69 604.00000000 [create] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.976+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3f840 con 0x5623ede39cf0 2024-09-10T10:51:17.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.977+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(70 604.00000000 [setxattr (30)] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.977+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3f840 con 0x5623ede39cf0 2024-09-10T10:51:17.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.979+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(71 604.00000000 [setxattr (62)] v105'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.979+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.980+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(72 604.00000000 [omap-set-header] v105'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.980+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.981+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(73 500.00000000 [create] v105'65 uv58 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.981+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.983+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(74 500.00000000 [delete] v105'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.983+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:17.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.984+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(75 500.00000000 [create] v105'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.984+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.986+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(76 500.00000000 [write 0~90] v105'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.986+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede39cf0 2024-09-10T10:51:17.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.987+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(77 500.00000000 [omap-set-header] v105'69 uv69 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:17.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.987+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:78 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede36740 2024-09-10T10:51:17.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.989+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(78 mds_snaptable [create] v105'27 uv26 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.989+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:79 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede36740 2024-09-10T10:51:17.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.990+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(79 mds_snaptable [delete] v105'28 uv28 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.990+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede36740 2024-09-10T10:51:17.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.991+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(80 mds_snaptable [create] v105'29 uv29 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.992 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:17.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.991+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede36740 2024-09-10T10:51:17.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.992+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(81 mds_snaptable [write 0~70] v105'30 uv30 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.992+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3e830 con 0x5623ede36740 2024-09-10T10:51:17.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.993+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v105'31 uv31 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede36740 2024-09-10T10:51:17.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.993+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5623ede3eeb0 0x5623ede412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:17.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.994+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41830 con 0x5623ede3eeb0 2024-09-10T10:51:17.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.994+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5623ede3eeb0 0x5623ede412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:17.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.994+0000 7f8557400640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5623ede3eeb0 0x5623ede412a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:17.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.996+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v105'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:17.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.996+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:17.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.997+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v105'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:17.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.997+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:17.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.998+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v105'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.000 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:18.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:17.998+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede42c10 con 0x5623ede3eeb0 2024-09-10T10:51:18.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.000+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v105'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.000+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede42c10 con 0x5623ede3eeb0 2024-09-10T10:51:18.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.001+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v105'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.001+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:18.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.002+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v105'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.002+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:18.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.004+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v105'54 uv48 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.004+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:18.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.005+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v105'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.005+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede3eeb0 2024-09-10T10:51:18.007 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:18.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.006+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v105'56 uv56 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.006+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.008+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v105'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.008+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.009+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v105'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.009+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.010+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v105'59 uv53 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.011+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.012+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v105'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.012+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.014 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.013+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v105'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.013+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede3d5a0 con 0x5623ede3eeb0 2024-09-10T10:51:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.015+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v105'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.015+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede44a70 con 0x5623ede3eeb0 2024-09-10T10:51:18.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.016+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v105'63 uv63 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.016+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.018+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v105'26 uv25 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.018+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.020+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v105'27 uv27 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.020+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.022 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:18.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.021+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v105'28 uv28 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.021+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.022+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v105'29 uv29 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.022+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.023+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v105'33 uv32 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.023+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.024+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v105'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.024+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.027 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.026+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v105'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.026+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.027+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v105'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.027+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.028+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v105'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.028+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede41c00 con 0x5623ede39cf0 2024-09-10T10:51:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.029+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v105'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f855007a070 con 0x5623ede39cf0 2024-09-10T10:51:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.029+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5623ede429b0 0x5623ede45130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.029+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede429b0 2024-09-10T10:51:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.029+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5623ede429b0 0x5623ede45130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.030+0000 7f8556a00640 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5623ede429b0 0x5623ede45130 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.031+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v105'27 uv26 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede429b0 2024-09-10T10:51:18.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.031+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede429b0 2024-09-10T10:51:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.032+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v105'28 uv28 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede429b0 2024-09-10T10:51:18.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.032+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede429b0 2024-09-10T10:51:18.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.033+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v105'29 uv29 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede429b0 2024-09-10T10:51:18.035 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:18.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.033+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede429b0 2024-09-10T10:51:18.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.035+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v105'30 uv30 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede429b0 2024-09-10T10:51:18.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.035+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede429b0 2024-09-10T10:51:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.036+0000 7f8556a00640 1 -- 172.21.15.28:0/1462694672 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v105'31 uv31 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f854400d580 con 0x5623ede429b0 2024-09-10T10:51:18.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.036+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.037+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v105'69 uv61 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.037+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.038+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v105'70 uv70 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.038+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.040 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:18.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.039+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v105'71 uv71 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.039+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.041+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v105'72 uv72 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.041+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.042+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v105'73 uv73 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.042+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.043+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v105'74 uv68 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.043+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.044+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v105'75 uv75 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.044+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.046 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:18.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.045+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v105'76 uv76 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.045+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede46680 con 0x5623ede3eeb0 2024-09-10T10:51:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.046+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v105'77 uv77 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.046+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede46680 con 0x5623ede3eeb0 2024-09-10T10:51:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.047+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v105'78 uv78 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.047+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45670 con 0x5623ede3eeb0 2024-09-10T10:51:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.049+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v105'79 uv79 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.049+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105) -- 0x5623ede45b20 con 0x5623ede3eeb0 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f8557400640 1 -- 172.21.15.28:0/1462694672 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v105'80 uv80 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f8548011730 con 0x5623ede3eeb0 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5623ede36740 msgr2=0x5623ede38b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5623ede36740 0x5623ede38b30 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5623ede3eeb0 msgr2=0x5623ede412a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5623ede3eeb0 0x5623ede412a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5623ede429b0 msgr2=0x5623ede45130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5623ede429b0 0x5623ede45130 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5623ede39cf0 msgr2=0x5623ede3c0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5623ede39cf0 0x5623ede3c0e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 msgr2=0x5623ede34390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 0x5623ede34390 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 reap_dead start 2024-09-10T10:51:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f8557e00640 1 -- 172.21.15.28:0/1462694672 reap_dead start 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 msgr2=0x5623ede27e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 0x5623ede27e60 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 msgr2=0x7f853806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 0x7f853806cd00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f854800e270 tx=0x7f85480025f0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 msgr2=0x5623ede27410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8550063040 tx=0x7f8550003330 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 shutdown_connections 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5623ede31fa0 0x5623ede34390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5623edd03600 0x5623ede27e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f853806a840 0x7f853806cd00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 --2- 172.21.15.28:0/1462694672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5623edd8e9c0 0x5623ede27410 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 >> 172.21.15.28:0/1462694672 conn(0x5623edd040a0 msgr2=0x5623edd8bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 shutdown_connections 2024-09-10T10:51:18.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.050+0000 7f855ad26f40 1 -- 172.21.15.28:0/1462694672 wait complete. 2024-09-10T10:51:18.056 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_openfiles.0 2024-09-10T10:51:18.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.170+0000 7f4f80e64f40 1 Processor -- start 2024-09-10T10:51:18.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.170+0000 7f4f80e64f40 1 -- start start 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4ae185a0 0x55aa4aee1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4ae16840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa4ae18f00 con 0x55aa4aeea4b0 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa4ae5ddc0 con 0x55aa4ae5e2d0 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55aa4ae5d1f0 con 0x55aa4ae185a0 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f7ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f7ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43942/0 (socket says 172.21.15.28:43942) 2024-09-10T10:51:18.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.171+0000 7f4f7ea00640 1 -- 172.21.15.28:0/4043002888 learned_addr learned my addr 172.21.15.28:0/4043002888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7d600640 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4ae16840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7e000640 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4ae185a0 0x55aa4aee1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 -- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4ae185a0 msgr2=0x55aa4aee1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4ae185a0 0x55aa4aee1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 -- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 msgr2=0x55aa4ae16840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4ae16840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 -- 172.21.15.28:0/4043002888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4ad40b30 con 0x55aa4aeea4b0 2024-09-10T10:51:18.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4f78025a20 tx=0x7f4f78050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86103c510f913779 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f77e00640 1 -- 172.21.15.28:0/4043002888 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78054040 con 0x55aa4aeea4b0 2024-09-10T10:51:18.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f77e00640 1 -- 172.21.15.28:0/4043002888 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f78050800 con 0x55aa4aeea4b0 2024-09-10T10:51:18.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.172+0000 7f4f77e00640 1 -- 172.21.15.28:0/4043002888 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f78050bf0 con 0x55aa4aeea4b0 2024-09-10T10:51:18.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 -- 172.21.15.28:0/4043002888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 msgr2=0x55aa4ae5e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4f78025a20 tx=0x7f4f78050000 comp rx=0 tx=0).stop 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 -- 172.21.15.28:0/4043002888 shutdown_connections 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4aeea4b0 0x55aa4ae5e6b0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4ae16840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/4043002888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4ae185a0 0x55aa4aee1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 -- 172.21.15.28:0/4043002888 >> 172.21.15.28:0/4043002888 conn(0x55aa4ae42b90 msgr2=0x55aa4ae45060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 -- 172.21.15.28:0/4043002888 shutdown_connections 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.173+0000 7f4f80e64f40 1 -- 172.21.15.28:0/4043002888 wait complete. 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.174+0000 7f4f80e64f40 1 Processor -- start 2024-09-10T10:51:18.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.174+0000 7f4f80e64f40 1 -- start start 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 0x55aa4aee31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4aeecd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4aeea4b0 0x55aa4af86630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa4aee2490 con 0x55aa4ae185a0 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa4aeeac80 con 0x55aa4ae5e2d0 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f80e64f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55aa4aeec5d0 con 0x55aa4aeea4b0 2024-09-10T10:51:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 0x55aa4aee31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 0x55aa4aee31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43958/0 (socket says 172.21.15.28:43958) 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 -- 172.21.15.28:0/2635056737 learned_addr learned my addr 172.21.15.28:0/2635056737 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4aeea4b0 0x55aa4af86630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7d600640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4aeecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 -- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4aeea4b0 msgr2=0x55aa4af86630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4aeea4b0 0x55aa4af86630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 -- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 msgr2=0x55aa4aeecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa4ae5e2d0 0x55aa4aeecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7e000640 1 -- 172.21.15.28:0/2635056737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4aede2e0 con 0x55aa4ae185a0 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.175+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55aa4aeea4b0 0x55aa4af86630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f7e000640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 0x55aa4aee31e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4f70008790 tx=0x7f4f70008880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f76a00640 1 -- 172.21.15.28:0/2635056737 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70019020 con 0x55aa4ae185a0 2024-09-10T10:51:18.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f76a00640 1 -- 172.21.15.28:0/2635056737 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f70015880 con 0x55aa4ae185a0 2024-09-10T10:51:18.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f76a00640 1 -- 172.21.15.28:0/2635056737 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f70015c30 con 0x55aa4ae185a0 2024-09-10T10:51:18.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa4ae5ed60 con 0x55aa4ae185a0 2024-09-10T10:51:18.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.176+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4aee3720 con 0x55aa4ae185a0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.178+0000 7f4f76a00640 1 -- 172.21.15.28:0/2635056737 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f7001c020 con 0x55aa4ae185a0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f76a00640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f6006a810 0x7f4f6006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f76a00640 1 -- 172.21.15.28:0/2635056737 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f4f7008a990 con 0x55aa4ae185a0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55aa4aeed260 0x55aa4af87080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e105) -- 0x55aa4ae15ec0 con 0x55aa4aeed260 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f7d600640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f6006a810 0x7f4f6006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.179+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55aa4aeed260 0x55aa4af87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.180+0000 7f4f7d600640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f6006a810 0x7f4f6006ccd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55aa4aee5b90 tx=0x7f4f6c00c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.180+0000 7f4f7ea00640 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55aa4aeed260 0x55aa4af87080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.181+0000 7f4f7ea00640 1 -- 172.21.15.28:0/2635056737 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_openfiles.0 [delete] v105'30 uv30 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4f78056030 con 0x55aa4aeed260 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.181+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55aa4aeed260 msgr2=0x55aa4af87080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.181+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55aa4aeed260 0x55aa4af87080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f6006a810 msgr2=0x7f4f6006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f6006a810 0x7f4f6006ccd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55aa4aee5b90 tx=0x7f4f6c00c040 comp rx=0 tx=0).stop 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 msgr2=0x55aa4aee31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 --2- 172.21.15.28:0/2635056737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aa4ae185a0 0x55aa4aee31e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f4f70008790 tx=0x7f4f70008880 comp rx=0 tx=0).stop 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f7ea00640 1 -- 172.21.15.28:0/2635056737 reap_dead start 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 shutdown_connections 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 >> 172.21.15.28:0/2635056737 conn(0x55aa4ae42b90 msgr2=0x55aa4aee9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:18.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 shutdown_connections 2024-09-10T10:51:18.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.182+0000 7f4f80e64f40 1 -- 172.21.15.28:0/2635056737 wait complete. 2024-09-10T10:51:18.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:18.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.345+0000 7f0777a00640 1 Processor -- start 2024-09-10T10:51:18.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.345+0000 7f0777a00640 1 -- start start 2024-09-10T10:51:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.346+0000 7f0777a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0770162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.346+0000 7f0777a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07701626a0 0x7f077006e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.346+0000 7f0777a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0770162ed0 con 0x7f0770155460 2024-09-10T10:51:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.346+0000 7f0777a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f077015f430 con 0x7f07701626a0 2024-09-10T10:51:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.346+0000 7f0777a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f077015ede0 con 0x7f0770072980 2024-09-10T10:51:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f0775000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0770155460 0x7f0770072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60312/0 (socket says 172.21.15.28:60312) 2024-09-10T10:51:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f0775000640 1 -- 172.21.15.28:0/258933722 learned_addr learned my addr 172.21.15.28:0/258933722 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f0775a00640 1 --2- 172.21.15.28:0/258933722 >> v2:172.21.15.138:3300/0 conn(0x7f07701626a0 0x7f077006e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076fe00640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0770162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276016405 0 0) 0x7f0770162ed0 con 0x7f0770155460 2024-09-10T10:51:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0758003520 con 0x7f0770155460 2024-09-10T10:51:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 459013898 0 0) 0x7f0758003520 con 0x7f0770155460 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 msgr2=0x7f0770162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0770162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 >> v2:172.21.15.138:3300/0 conn(0x7f07701626a0 msgr2=0x7f077006e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 --2- 172.21.15.28:0/258933722 >> v2:172.21.15.138:3300/0 conn(0x7f07701626a0 0x7f077006e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077015f0c0 con 0x7f0770155460 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f0764001670 con 0x7f0770155460 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.347+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 >> v1:172.21.15.28:6789/0 conn(0x7f0770155460 legacy=0x7f0770072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07580039b0 0x7f0758003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0758004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758004d40 0x7f07580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f077015f430 con 0x7f07580039b0 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f077015ede0 con 0x7f0758004d40 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f075800b9c0 con 0x7f0770072980 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07580039b0 0x7f0758003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f076fe00640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0758004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775a00640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758004d40 0x7f07580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 -- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 msgr2=0x7f0758004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0770072980 0x7f0758004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 -- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758004d40 msgr2=0x7f07580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758004d40 0x7f07580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.348+0000 7f0775000640 1 -- 172.21.15.28:0/258933722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0758003520 con 0x7f07580039b0 2024-09-10T10:51:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.349+0000 7f0775000640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07580039b0 0x7f0758003dd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f076400c270 tx=0x7f0758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5497e7f62fd4b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.349+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764007d30 con 0x7f07580039b0 2024-09-10T10:51:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.349+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0764005ad0 con 0x7f07580039b0 2024-09-10T10:51:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.349+0000 7f076f400640 1 -- 172.21.15.28:0/258933722 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0764007070 con 0x7f07580039b0 2024-09-10T10:51:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 -- 172.21.15.28:0/258933722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07580039b0 msgr2=0x7f0758003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 --2- 172.21.15.28:0/258933722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07580039b0 0x7f0758003dd0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f076400c270 tx=0x7f0758005830 comp rx=0 tx=0).stop 2024-09-10T10:51:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0775a00640 1 -- 172.21.15.28:0/258933722 reap_dead start 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 -- 172.21.15.28:0/258933722 shutdown_connections 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 -- 172.21.15.28:0/258933722 >> 172.21.15.28:0/258933722 conn(0x7f077006bc10 msgr2=0x7f077006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 -- 172.21.15.28:0/258933722 shutdown_connections 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.350+0000 7f0777a00640 1 -- 172.21.15.28:0/258933722 wait complete. 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.352+0000 7f0777a00640 1 Processor -- start 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.352+0000 7f0777a00640 1 -- start start 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07701f3a60 0x7f077006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f077006f080 0x7f0770159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0770162cb0 con 0x7f07701599d0 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0770162b30 con 0x7f07701f3a60 2024-09-10T10:51:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0777a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0770155840 con 0x7f077006f080 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0775a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0775000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07701f3a60 0x7f077006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0775a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43970/0 (socket says 172.21.15.28:43970) 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f0775a00640 1 -- 172.21.15.28:0/3566846973 learned_addr learned my addr 172.21.15.28:0/3566846973 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.353+0000 7f076fe00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f077006f080 0x7f0770159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 -- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f077006f080 msgr2=0x7f0770159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f077006f080 0x7f0770159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 -- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07701f3a60 msgr2=0x7f077006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07701f3a60 0x7f077006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 -- 172.21.15.28:0/3566846973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07700aeb90 con 0x7f07701599d0 2024-09-10T10:51:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0775a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f07700ae730 tx=0x7f076800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076800fcb0 con 0x7f07701599d0 2024-09-10T10:51:18.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076801d070 con 0x7f07701599d0 2024-09-10T10:51:18.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768015670 con 0x7f07701599d0 2024-09-10T10:51:18.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07700aee70 con 0x7f07701599d0 2024-09-10T10:51:18.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.354+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07700af430 con 0x7f07701599d0 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.355+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077006cba0 con 0x7f07701599d0 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.356+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0768015810 con 0x7f07701599d0 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.357+0000 7f076cc00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 0x7f075006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.357+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f076808b960 con 0x7f07701599d0 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.358+0000 7f0775000640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 0x7f075006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:18.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.358+0000 7f0775000640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 0x7f075006e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0764033000 tx=0x7f0764009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:18.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.361+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0768010040 con 0x7f07701599d0 2024-09-10T10:51:18.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.583+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f077015f8c0 con 0x7f07701599d0 2024-09-10T10:51:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.760+0000 7f076cc00640 1 -- 172.21.15.28:0/3566846973 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v120) ==== 162+0+0 (secure 0 0 0) 0x7f0768059ee0 con 0x7f07701599d0 2024-09-10T10:51:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:18.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 msgr2=0x7f075006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 0x7f075006e320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0764033000 tx=0x7f0764009030 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 msgr2=0x7f07701f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f07700ae730 tx=0x7f076800f1b0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 shutdown_connections 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f075006be60 0x7f075006e320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07701599d0 0x7f07701f74f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f077006f080 0x7f0770159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 --2- 172.21.15.28:0/3566846973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07701f3a60 0x7f077006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 >> 172.21.15.28:0/3566846973 conn(0x7f077006bc10 msgr2=0x7f07701577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:18.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 shutdown_connections 2024-09-10T10:51:18.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:18.762+0000 7f0777a00640 1 -- 172.21.15.28:0/3566846973 wait complete. 2024-09-10T10:51:18.768 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:18.768+0000 7f02da400640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-10T10:51:19.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:19.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.342+0000 7f7844400640 1 Processor -- start 2024-09-10T10:51:19.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.343+0000 7f7844400640 1 -- start start 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7844400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f783c147e90 0x7f783c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7844400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 0x7f783c0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7844400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f783c072a00 con 0x7f783c147230 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7844400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f783c0b34d0 con 0x7f783c147e90 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7844400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f783c0ae9e0 con 0x7f783c0725f0 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7841000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f783c147e90 0x7f783c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7841000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f783c147e90 0x7f783c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43978/0 (socket says 172.21.15.28:43978) 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7841000640 1 -- 172.21.15.28:0/354748398 learned_addr learned my addr 172.21.15.28:0/354748398 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.344+0000 7f7842400640 1 --2- 172.21.15.28:0/354748398 >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 0x7f783c0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.345+0000 7f7841000640 1 -- 172.21.15.28:0/354748398 >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 msgr2=0x7f783c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.345+0000 7f7841000640 1 --2- 172.21.15.28:0/354748398 >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 0x7f783c0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.345+0000 7f7841000640 1 -- 172.21.15.28:0/354748398 >> v1:172.21.15.138:6789/0 conn(0x7f783c147230 legacy=0x7f783c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.345+0000 7f7841000640 1 -- 172.21.15.28:0/354748398 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c06f480 con 0x7f783c147e90 2024-09-10T10:51:19.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.345+0000 7f7842400640 1 --2- 172.21.15.28:0/354748398 >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 0x7f783c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f7841000640 1 --2- 172.21.15.28:0/354748398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147e90 0x7f783c0720b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f783c148320 tx=0x7f782802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645dd77e8867b543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f783be00640 1 -- 172.21.15.28:0/354748398 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828037070 con 0x7f783c147e90 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f783be00640 1 -- 172.21.15.28:0/354748398 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7828032440 con 0x7f783c147e90 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f783be00640 1 -- 172.21.15.28:0/354748398 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828032810 con 0x7f783c147e90 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f7844400640 1 -- 172.21.15.28:0/354748398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147e90 msgr2=0x7f783c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.346+0000 7f7844400640 1 --2- 172.21.15.28:0/354748398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147e90 0x7f783c0720b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f783c148320 tx=0x7f782802a270 comp rx=0 tx=0).stop 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 -- 172.21.15.28:0/354748398 shutdown_connections 2024-09-10T10:51:19.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 --2- 172.21.15.28:0/354748398 >> v2:172.21.15.138:3300/0 conn(0x7f783c0725f0 0x7f783c0aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 --2- 172.21.15.28:0/354748398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147e90 0x7f783c0720b0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 -- 172.21.15.28:0/354748398 >> 172.21.15.28:0/354748398 conn(0x7f783c067d70 msgr2=0x7f783c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 -- 172.21.15.28:0/354748398 shutdown_connections 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.347+0000 7f7844400640 1 -- 172.21.15.28:0/354748398 wait complete. 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.348+0000 7f7844400640 1 Processor -- start 2024-09-10T10:51:19.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.348+0000 7f7844400640 1 -- start start 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 0x7f783c1410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c147e90 0x7f783c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f783c06eec0 con 0x7f783c147230 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f783c06ebe0 con 0x7f783c147e90 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7844400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f783c06f1a0 con 0x7f783c0725f0 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7841000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7841000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43988/0 (socket says 172.21.15.28:43988) 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.349+0000 7f7841000640 1 -- 172.21.15.28:0/3894153560 learned_addr learned my addr 172.21.15.28:0/3894153560 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:19.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7842400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c147e90 0x7f783c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841a00640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 0x7f783c1410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 -- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 msgr2=0x7f783c1410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 0x7f783c1410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 -- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c147e90 msgr2=0x7f783c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c147e90 0x7f783c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 -- 172.21.15.28:0/3894153560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c0c75f0 con 0x7f783c147230 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841a00640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 0x7f783c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.350+0000 7f7841000640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f782802a2b0 tx=0x7f78280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.351+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828041020 con 0x7f783c147230 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.351+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78280306c0 con 0x7f783c147230 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.351+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783c0c78d0 con 0x7f783c147230 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.351+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828037040 con 0x7f783c147230 2024-09-10T10:51:19.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.351+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783c0c7e90 con 0x7f783c147230 2024-09-10T10:51:19.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.352+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f783c147610 con 0x7f783c147230 2024-09-10T10:51:19.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.353+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7828044020 con 0x7f783c147230 2024-09-10T10:51:19.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.353+0000 7f7839600640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:19.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.354+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f78280aba90 con 0x7f783c147230 2024-09-10T10:51:19.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.354+0000 7f7841a00640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:19.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.355+0000 7f7841a00640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 0x7f781c06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7830001680 tx=0x7f7830009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:19.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.358+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f782804b030 con 0x7f783c147230 2024-09-10T10:51:19.494 INFO:tasks.ceph.mds.a.smithi028.stderr: -36> 2024-09-10T10:51:14.741+0000 7f43d9800640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:51:19.533 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:51:19.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.578+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f783c0731d0 con 0x7f783c147230 2024-09-10T10:51:19.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.579+0000 7f7839600640 1 -- 172.21.15.28:0/3894153560 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+4401 (secure 0 0 0) 0x7f782807a010 con 0x7f783c147230 2024-09-10T10:51:19.580 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:19.580 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":121,"btime":"2024-09-10T10:51:18:762472+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:18.762460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7797},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7797":{"gid":7797,"name":"c","rank":0,"incarnation":121,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/3452166242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3452166242},{"type":"v1","addr":"172.21.15.28:6834","nonce":3452166242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:19.580 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 121 2024-09-10T10:51:19.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 msgr2=0x7f781c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 0x7f781c06e2f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7830001680 tx=0x7f7830009040 comp rx=0 tx=0).stop 2024-09-10T10:51:19.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 msgr2=0x7f783c142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f782802a2b0 tx=0x7f78280046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 shutdown_connections 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f781c06be30 0x7f781c06e2f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f783c147e90 0x7f783c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f783c147230 0x7f783c142d00 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 --2- 172.21.15.28:0/3894153560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f783c0725f0 0x7f783c1410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 >> 172.21.15.28:0/3894153560 conn(0x7f783c067d70 msgr2=0x7f783c0af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 shutdown_connections 2024-09-10T10:51:19.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:19.581+0000 7f7844400640 1 -- 172.21.15.28:0/3894153560 wait complete. 2024-09-10T10:51:19.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:51:18.762460+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 105, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7797}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7797': {'gid': 7797, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6834/3452166242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 3452166242}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 3452166242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:51:19.599 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7797:up:replay 2024-09-10T10:51:19.599 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:51:25.002 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:25.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.168+0000 7f3d5dc00640 1 Processor -- start 2024-09-10T10:51:25.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.168+0000 7f3d5dc00640 1 -- start start 2024-09-10T10:51:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d5dc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d58072320 0x7f3d580cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d5dc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072de0 0x7f3d5806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d5dc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d58146140 con 0x7f3d58072700 2024-09-10T10:51:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d5dc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3d580cd740 con 0x7f3d58072320 2024-09-10T10:51:25.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d5dc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d580cda20 con 0x7f3d58072de0 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d4fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072de0 0x7f3d5806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d56a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d58072320 0x7f3d580cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d57400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3d58072700 0x7f3d5806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:57496/0 (socket says 172.21.15.28:57496) 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.170+0000 7f3d57400640 1 -- 172.21.15.28:0/2505807676 learned_addr learned my addr 172.21.15.28:0/2505807676 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.171+0000 7f3d56a00640 1 -- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072de0 msgr2=0x7f3d5806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.171+0000 7f3d56a00640 1 --2- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072de0 0x7f3d5806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.171+0000 7f3d56a00640 1 -- 172.21.15.28:0/2505807676 >> v1:172.21.15.138:6789/0 conn(0x7f3d58072700 legacy=0x7f3d5806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.171+0000 7f3d56a00640 1 -- 172.21.15.28:0/2505807676 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d580cdd00 con 0x7f3d58072320 2024-09-10T10:51:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.172+0000 7f3d56a00640 1 --2- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d580cce60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d480097a0 tx=0x7f3d4802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1edb8d9bc4ee2466 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.172+0000 7f3d56000640 1 -- 172.21.15.28:0/2505807676 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48037070 con 0x7f3d58072320 2024-09-10T10:51:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.172+0000 7f3d56000640 1 -- 172.21.15.28:0/2505807676 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4802ae50 con 0x7f3d58072320 2024-09-10T10:51:25.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.172+0000 7f3d56000640 1 -- 172.21.15.28:0/2505807676 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48032660 con 0x7f3d58072320 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 -- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 msgr2=0x7f3d580cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d580cce60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3d480097a0 tx=0x7f3d4802a100 comp rx=0 tx=0).stop 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 -- 172.21.15.28:0/2505807676 shutdown_connections 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072de0 0x7f3d5806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/2505807676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d580cce60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 -- 172.21.15.28:0/2505807676 >> 172.21.15.28:0/2505807676 conn(0x7f3d58067d70 msgr2=0x7f3d5806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 -- 172.21.15.28:0/2505807676 shutdown_connections 2024-09-10T10:51:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.173+0000 7f3d5dc00640 1 -- 172.21.15.28:0/2505807676 wait complete. 2024-09-10T10:51:25.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.175+0000 7f3d5dc00640 1 Processor -- start 2024-09-10T10:51:25.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.175+0000 7f3d5dc00640 1 -- start start 2024-09-10T10:51:25.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d58140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072700 0x7f3d581e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d580cd520 con 0x7f3d58072de0 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d580cd3a0 con 0x7f3d58072320 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d5dc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5806ed00 con 0x7f3d58072700 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41056/0 (socket says 172.21.15.28:41056) 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 -- 172.21.15.28:0/3874689541 learned_addr learned my addr 172.21.15.28:0/3874689541 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d56a00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d58140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d4fe00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072700 0x7f3d581e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 -- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072700 msgr2=0x7f3d581e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072700 0x7f3d581e6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 -- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 msgr2=0x7f3d58140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.176+0000 7f3d57400640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d58140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d57400640 1 -- 172.21.15.28:0/3874689541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d581e7370 con 0x7f3d58072de0 2024-09-10T10:51:25.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d56a00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d58140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:25.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d57400640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000cbc0 tx=0x7f3d5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50017070 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5000ad40 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50012600 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d581e7650 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.177+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5813f1a0 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.178+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d581e7b70 con 0x7f3d58072de0 2024-09-10T10:51:25.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.180+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d5001a070 con 0x7f3d58072de0 2024-09-10T10:51:25.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.180+0000 7f3d4ea00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 0x7f3d3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.181+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f3d5008ae70 con 0x7f3d58072de0 2024-09-10T10:51:25.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.181+0000 7f3d56a00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 0x7f3d3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.182+0000 7f3d56a00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 0x7f3d3406e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3d48034000 tx=0x7f3d4802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.185+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d50058e30 con 0x7f3d58072de0 2024-09-10T10:51:25.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.404+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d580cce60 con 0x7f3d58072de0 2024-09-10T10:51:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.405+0000 7f3d4ea00640 1 -- 172.21.15.28:0/3874689541 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+5234 (secure 0 0 0) 0x7f3d5005d130 con 0x7f3d58072de0 2024-09-10T10:51:25.407 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:25.407 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":124,"btime":"2024-09-10T10:51:21:773483+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:21.773476+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":105,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7797},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7797":{"gid":7797,"name":"c","rank":0,"incarnation":121,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6834/3452166242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3452166242},{"type":"v1","addr":"172.21.15.28:6834","nonce":3452166242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7797,"qdb_cluster":[7797]},"id":6}]} 2024-09-10T10:51:25.407 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 124 2024-09-10T10:51:25.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 msgr2=0x7f3d3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 0x7f3d3406e2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3d48034000 tx=0x7f3d4802e000 comp rx=0 tx=0).stop 2024-09-10T10:51:25.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 msgr2=0x7f3d5813ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000cbc0 tx=0x7f3d5000a430 comp rx=0 tx=0).stop 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 shutdown_connections 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d3406be30 0x7f3d3406e2f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d58072de0 0x7f3d5813ec60 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58072700 0x7f3d581e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 --2- 172.21.15.28:0/3874689541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d58072320 0x7f3d58140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 >> 172.21.15.28:0/3874689541 conn(0x7f3d58067d70 msgr2=0x7f3d580c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 shutdown_connections 2024-09-10T10:51:25.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.407+0000 7f3d5dc00640 1 -- 172.21.15.28:0/3874689541 wait complete. 2024-09-10T10:51:25.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 124, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:51:21.773476+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 105, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7797}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7797': {'gid': 7797, 'name': 'c', 'rank': 0, 'incarnation': 121, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6834/3452166242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 3452166242}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 3452166242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7797, 'qdb_cluster': [7797]} 2024-09-10T10:51:25.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:51:25.425 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:51:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.580+0000 7f18c0a00640 1 Processor -- start 2024-09-10T10:51:25.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.581+0000 7f18c0a00640 1 -- start start 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18c0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18bc15bef0 0x7f18bc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18c0a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18bc0af380 0x7f18bc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18c0a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc0b5470 con 0x7f18bc0b25c0 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18c0a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0b3640 con 0x7f18bc15bef0 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18c0a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0b5190 con 0x7f18bc0af380 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18bb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18bc15bef0 0x7f18bc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18bb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f18bc15bef0 0x7f18bc0b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41064/0 (socket says 172.21.15.28:41064) 2024-09-10T10:51:25.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18bb400640 1 -- 172.21.15.28:0/1477161478 learned_addr learned my addr 172.21.15.28:0/1477161478 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.582+0000 7f18bbe00640 1 --2- 172.21.15.28:0/1477161478 >> v2:172.21.15.138:3300/0 conn(0x7f18bc0af380 0x7f18bc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.583+0000 7f18bb400640 1 -- 172.21.15.28:0/1477161478 >> v2:172.21.15.138:3300/0 conn(0x7f18bc0af380 msgr2=0x7f18bc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.583+0000 7f18bb400640 1 --2- 172.21.15.28:0/1477161478 >> v2:172.21.15.138:3300/0 conn(0x7f18bc0af380 0x7f18bc071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.583+0000 7f18bb400640 1 -- 172.21.15.28:0/1477161478 >> v1:172.21.15.138:6789/0 conn(0x7f18bc0b25c0 legacy=0x7f18bc0aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.583+0000 7f18bb400640 1 -- 172.21.15.28:0/1477161478 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0b38e0 con 0x7f18bc15bef0 2024-09-10T10:51:25.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18bb400640 1 --2- 172.21.15.28:0/1477161478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc15bef0 0x7f18bc0b40e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b48c0 tx=0x7f18ac02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85192c3e5b17280 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18baa00640 1 -- 172.21.15.28:0/1477161478 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037070 con 0x7f18bc15bef0 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18baa00640 1 -- 172.21.15.28:0/1477161478 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac032440 con 0x7f18bc15bef0 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18baa00640 1 -- 172.21.15.28:0/1477161478 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac032810 con 0x7f18bc15bef0 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 -- 172.21.15.28:0/1477161478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc15bef0 msgr2=0x7f18bc0b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 --2- 172.21.15.28:0/1477161478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc15bef0 0x7f18bc0b40e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b48c0 tx=0x7f18ac02a200 comp rx=0 tx=0).stop 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 -- 172.21.15.28:0/1477161478 shutdown_connections 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 --2- 172.21.15.28:0/1477161478 >> v2:172.21.15.138:3300/0 conn(0x7f18bc0af380 0x7f18bc071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 --2- 172.21.15.28:0/1477161478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc15bef0 0x7f18bc0b40e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.584+0000 7f18c0a00640 1 -- 172.21.15.28:0/1477161478 >> 172.21.15.28:0/1477161478 conn(0x7f18bc06c240 msgr2=0x7f18bc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.585+0000 7f18c0a00640 1 -- 172.21.15.28:0/1477161478 shutdown_connections 2024-09-10T10:51:25.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.585+0000 7f18c0a00640 1 -- 172.21.15.28:0/1477161478 wait complete. 2024-09-10T10:51:25.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.586+0000 7f18c0a00640 1 Processor -- start 2024-09-10T10:51:25.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.586+0000 7f18c0a00640 1 -- start start 2024-09-10T10:51:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 0x7f18bc1f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 0x7f18bc0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0b4b10 con 0x7f18bc0af380 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0b4e60 con 0x7f18bc15bef0 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18c0a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0b2a30 con 0x7f18bc0b25c0 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41072/0 (socket says 172.21.15.28:41072) 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.587+0000 7f18bb400640 1 -- 172.21.15.28:0/4262002464 learned_addr learned my addr 172.21.15.28:0/4262002464 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18b3e00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 0x7f18bc1f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bbe00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 0x7f18bc0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 -- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 msgr2=0x7f18bc1f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 0x7f18bc1f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 -- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 msgr2=0x7f18bc0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 0x7f18bc0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 -- 172.21.15.28:0/4262002464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0cdd70 con 0x7f18bc0af380 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18b3e00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 0x7f18bc1f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bbe00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 0x7f18bc0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18bb400640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02a240 tx=0x7f18ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18b3400640 1 -- 172.21.15.28:0/4262002464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac041020 con 0x7f18bc0af380 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18b3400640 1 -- 172.21.15.28:0/4262002464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0306c0 con 0x7f18bc0af380 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18b3400640 1 -- 172.21.15.28:0/4262002464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037040 con 0x7f18bc0af380 2024-09-10T10:51:25.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.588+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc15b350 con 0x7f18bc0af380 2024-09-10T10:51:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.589+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc15da20 con 0x7f18bc0af380 2024-09-10T10:51:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.590+0000 7f18b3400640 1 -- 172.21.15.28:0/4262002464 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18ac044020 con 0x7f18bc0af380 2024-09-10T10:51:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.591+0000 7f18b3400640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 0x7f189806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.591+0000 7f18b3400640 1 -- 172.21.15.28:0/4262002464 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f18ac0aba40 con 0x7f18bc0af380 2024-09-10T10:51:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.592+0000 7f18b3e00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 0x7f189806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.592+0000 7f18c5036740 1 Processor -- start 2024-09-10T10:51:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.592+0000 7f18c5036740 1 -- start start 2024-09-10T10:51:25.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.592+0000 7f18b3e00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 0x7f189806e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18bc1ebc50 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 0x5571db27e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 0x5571db233720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5571db1973b0 con 0x5571db236660 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571db19c410 con 0x5571db23a630 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5571db144620 con 0x5571db238ea0 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41088/0 (socket says 172.21.15.28:41088) 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.593+0000 7f18b3e00640 1 -- 172.21.15.28:0/2495772460 learned_addr learned my addr 172.21.15.28:0/2495772460 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18bb400640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 0x5571db27e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 -- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 msgr2=0x5571db27e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18bbe00640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 0x5571db233720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 0x5571db27e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 -- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 msgr2=0x5571db233720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 0x5571db233720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 -- 172.21.15.28:0/2495772460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571daeaf2d0 con 0x5571db236660 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18bb400640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 0x5571db27e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18bbe00640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 0x5571db233720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:25.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18b3e00640 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d860 tx=0x7f18a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3991d4684651be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18ab400640 1 -- 172.21.15.28:0/2495772460 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4024070 con 0x5571db236660 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18ab400640 1 -- 172.21.15.28:0/2495772460 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4014ba0 con 0x5571db236660 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.594+0000 7f18ab400640 1 -- 172.21.15.28:0/2495772460 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4007050 con 0x5571db236660 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 -- 172.21.15.28:0/2495772460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 msgr2=0x5571db239280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f18a400d860 tx=0x7f18a4010000 comp rx=0 tx=0).stop 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 -- 172.21.15.28:0/2495772460 shutdown_connections 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db23a630 0x5571db233720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db236660 0x5571db239280 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 --2- 172.21.15.28:0/2495772460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db238ea0 0x5571db27e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 -- 172.21.15.28:0/2495772460 >> 172.21.15.28:0/2495772460 conn(0x5571db2358f0 msgr2=0x5571db27db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 -- 172.21.15.28:0/2495772460 shutdown_connections 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.595+0000 7f18c5036740 1 -- 172.21.15.28:0/2495772460 wait complete. 2024-09-10T10:51:25.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.596+0000 7f18c5036740 1 -- start start 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.596+0000 7f18c5036740 1 Processor -- start 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5571db157780 con 0x5571db23a630 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571db1983d0 con 0x5571db238ea0 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5571db234d90 con 0x5571db2325e0 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18c5036740 10 client.0 _ll_register_callbacks cb 0x5571db1955d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18bb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:42752/0 (socket says 172.21.15.28:42752) 2024-09-10T10:51:25.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18bb400640 1 -- 172.21.15.28:0/3214728723 learned_addr learned my addr 172.21.15.28:0/3214728723 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41090/0 (socket says 172.21.15.28:41090) 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.597+0000 7f18bbe00640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 -- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 msgr2=0x5571db2496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 -- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 msgr2=0x5571db25b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 -- 172.21.15.28:0/3214728723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571daf796b0 con 0x5571db23a630 2024-09-10T10:51:25.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18bb400640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:25.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18b3e00640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f18a4021000 tx=0x7f18a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8e4116d893948 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f189fe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:51:25.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a402f060 con 0x5571db23a630 2024-09-10T10:51:25.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a4014ba0 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.598+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f18a4007450 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f18a4035050 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 10 client.7866 fetch_fsmap learned FSMap version 124 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) -- 0x5571db25a940 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a4014ba0 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 124) ==== 3502+0+0 (secure 0 0 0) 0x7f18a4038020 con 0x5571db23a630 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 10 client.7866 fetch_fsmap finished waiting for FSMap version 124 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 10 client.7866 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 20 client.7866 populate_metadata read hostname 'smithi028' 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db233450 0x5571db233830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 4 client.7866 mds_command: new command op to 7797 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 --> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571db1a9b40 con 0x5571db233450 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.599+0000 7f18bb400640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db233450 0x5571db233830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.600+0000 7f18bb400640 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db233450 0x5571db233830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=552d79e6e48b66e server_cookie=3a6f7eddf2270569 in_seq=0 out_seq=0 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.600+0000 7f189fe00640 10 client.7866 ms_handle_connect on v2:172.21.15.28:6832/3452166242 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f189fe00640 1 -- 172.21.15.28:0/3214728723 <== mds.0 v2:172.21.15.28:6832/3452166242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571db1a9b40 con 0x5571db233450 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f189fe00640 10 client.7866 handle_command_reply: tid=0 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 client.7866 shutdown 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 msgr2=0x5571db27b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f18a4021000 tx=0x7f18a4014000 comp rx=0 tx=0).stop 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 shutdown_connections 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db233450 0x5571db233830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db27b030 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 --2- 172.21.15.28:0/3214728723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db25b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f189fe00640 0 client.7866 ms_handle_reset on v2:172.21.15.28:6832/3452166242 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 >> 172.21.15.28:0/3214728723 conn(0x5571db236660 msgr2=0x5571db27d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 shutdown_connections 2024-09-10T10:51:25.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 1 -- 172.21.15.28:0/3214728723 wait complete. 2024-09-10T10:51:25.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.601+0000 7f18c5036740 20 client.7866 trim_cache size 0 max 16384 2024-09-10T10:51:25.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.620+0000 7f18c5036740 1 Processor -- start 2024-09-10T10:51:25.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.620+0000 7f18c5036740 1 -- start start 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db258b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 0x5571db280a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc2003d0 con 0x5571db2325e0 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ff7d0 con 0x5571db238ea0 2024-09-10T10:51:25.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc1ffd80 con 0x5571db23a630 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41106/0 (socket says 172.21.15.28:41106) 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.621+0000 7f18bb400640 1 -- 172.21.15.28:0/835486539 learned_addr learned my addr 172.21.15.28:0/835486539 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18b3e00640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db258b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bbe00640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 0x5571db280a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 -- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 msgr2=0x5571db280a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 0x5571db280a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 -- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 msgr2=0x5571db258b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db258b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 -- 172.21.15.28:0/835486539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571db28ed30 con 0x5571db2325e0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bbe00640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 0x5571db280a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18bb400640 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac0a9aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d2bdb954022e09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18aaa00640 1 -- 172.21.15.28:0/835486539 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0a8070 con 0x5571db2325e0 2024-09-10T10:51:25.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18aaa00640 1 -- 172.21.15.28:0/835486539 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac082530 con 0x5571db2325e0 2024-09-10T10:51:25.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.622+0000 7f18aaa00640 1 -- 172.21.15.28:0/835486539 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac077c50 con 0x5571db2325e0 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 -- 172.21.15.28:0/835486539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 msgr2=0x5571db2585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac0a9aa0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 -- 172.21.15.28:0/835486539 shutdown_connections 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db23a630 0x5571db280a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db258b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 --2- 172.21.15.28:0/835486539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db2325e0 0x5571db2585d0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 -- 172.21.15.28:0/835486539 >> 172.21.15.28:0/835486539 conn(0x5571db236660 msgr2=0x5571db249120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 -- 172.21.15.28:0/835486539 shutdown_connections 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.623+0000 7f18c5036740 1 -- 172.21.15.28:0/835486539 wait complete. 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.624+0000 7f18c5036740 1 -- start start 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.624+0000 7f18c5036740 1 Processor -- start 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db280d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5571db157780 con 0x5571db23a630 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571db25a720 con 0x5571db238ea0 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5571db28ea50 con 0x5571db2325e0 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18c5036740 10 client.0 _ll_register_callbacks cb 0x7f18a4022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41112/0 (socket says 172.21.15.28:41112) 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 -- 172.21.15.28:0/432655551 learned_addr learned my addr 172.21.15.28:0/432655551 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18b3e00640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db280d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bb400640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 -- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 msgr2=0x5571db2f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 -- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 msgr2=0x5571db280d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db280d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.625+0000 7f18bbe00640 1 -- 172.21.15.28:0/432655551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571db235100 con 0x5571db23a630 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f18bbe00640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f18b4007b00 tx=0x7f18b40142c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394c39d991a3c371 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f189fe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b401b030 con 0x5571db23a630 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f18b4014c80 con 0x5571db23a630 2024-09-10T10:51:25.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac082530 con 0x5571db23a630 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f18b4027050 con 0x5571db23a630 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f18c5036740 10 client.7872 fetch_fsmap learned FSMap version 124 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.626+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=2+}) -- 0x5571db236c00 con 0x5571db23a630 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac082530 con 0x5571db23a630 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 124) ==== 3502+0+0 (secure 0 0 0) 0x7f18b402a020 con 0x5571db23a630 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 10 client.7872 fetch_fsmap finished waiting for FSMap version 124 2024-09-10T10:51:25.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 10 client.7872 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 20 client.7872 populate_metadata read hostname 'smithi028' 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db28abe0 0x5571db3074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 4 client.7872 mds_command: new command op to 7797 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 --> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] -- command(tid 0: {"prefix": "status"}) -- 0x5571db27ceb0 con 0x5571db28abe0 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18bb400640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db28abe0 0x5571db3074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f18bb400640 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db28abe0 0x5571db3074b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ce2e575b529f5c server_cookie=2f50d87d8c27809c in_seq=0 out_seq=0 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.627+0000 7f189fe00640 10 client.7872 ms_handle_connect on v2:172.21.15.28:6832/3452166242 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f189fe00640 1 -- 172.21.15.28:0/432655551 <== mds.0 v2:172.21.15.28:6832/3452166242 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5571db27ceb0 con 0x5571db28abe0 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f189fe00640 10 client.7872 handle_command_reply: tid=0 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 client.7872 shutdown 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 msgr2=0x5571db2812d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f18b4007b00 tx=0x7f18b40142c0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 shutdown_connections 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:6832/3452166242,v1:172.21.15.28:6834/3452166242] conn(0x5571db28abe0 0x5571db3074b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5571db23a630 0x5571db2812d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571db238ea0 0x5571db280d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 --2- 172.21.15.28:0/432655551 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5571db2325e0 0x5571db2f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f189fe00640 0 client.7872 ms_handle_reset on v2:172.21.15.28:6832/3452166242 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 >> 172.21.15.28:0/432655551 conn(0x5571db236660 msgr2=0x5571db27f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 shutdown_connections 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 1 -- 172.21.15.28:0/432655551 wait complete. 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.628+0000 7f18c5036740 20 client.7872 trim_cache size 0 max 16384 2024-09-10T10:51:25.634 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 7797, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8652312689999997, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 124, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 105, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 105, 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 10.111825369 2024-09-10T10:51:25.635 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 msgr2=0x7f189806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 0x7f189806e2f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f18bc1ebc50 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 msgr2=0x7f18bc15ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f18ac02a240 tx=0x7f18ac0046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 shutdown_connections 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f189806be30 0x7f189806e2f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18bc15bef0 0x7f18bc0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18bc0b25c0 0x7f18bc1f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 --2- 172.21.15.28:0/4262002464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18bc0af380 0x7f18bc15ad70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 >> 172.21.15.28:0/4262002464 conn(0x7f18bc06c240 msgr2=0x7f18bc06dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 shutdown_connections 2024-09-10T10:51:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.629+0000 7f18c0a00640 1 -- 172.21.15.28:0/4262002464 wait complete. 2024-09-10T10:51:25.646 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:51:25.646 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete mds0_openfiles.0', proceeding to ls 2024-09-10T10:51:25.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:51:25.754 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:25.752+0000 7f02df400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.801+0000 7f5774c00640 1 Processor -- start 2024-09-10T10:51:25.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.801+0000 7f5774c00640 1 -- start start 2024-09-10T10:51:25.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5774c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f577015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5774c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5770146140 con 0x7f5770072de0 2024-09-10T10:51:25.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5774c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57700cd740 con 0x7f5770072320 2024-09-10T10:51:25.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5774c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57700cda20 con 0x7f5770072700 2024-09-10T10:51:25.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5767e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5770072de0 0x7f577015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53554/0 (socket says 172.21.15.28:53554) 2024-09-10T10:51:25.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.802+0000 7f5767e00640 1 -- 172.21.15.28:0/1460931250 learned_addr learned my addr 172.21.15.28:0/1460931250 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:25.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576e800640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f577015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234987783 0 0) 0x7f5770146140 con 0x7f5770072de0 2024-09-10T10:51:25.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5754003520 con 0x7f5770072de0 2024-09-10T10:51:25.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2713507316 0 0) 0x7f5754003520 con 0x7f5770072de0 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 msgr2=0x7f577015d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f577015d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 >> v1:172.21.15.138:6789/0 conn(0x7f5770072320 legacy=0x7f57700cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57700cdd00 con 0x7f5770072de0 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f5758001670 con 0x7f5770072de0 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 >> v1:172.21.15.28:6789/0 conn(0x7f5770072de0 legacy=0x7f577015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57540039e0 0x7f5754003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f5754004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754004d40 0x7f57540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f57700cda20 con 0x7f57540039e0 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f575400bb50 con 0x7f5754004d40 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.803+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f575400bd50 con 0x7f5770072700 2024-09-10T10:51:25.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f576de00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57540039e0 0x7f5754003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f5754004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f576e800640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754004d40 0x7f57540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 -- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754004d40 msgr2=0x7f57540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5754004d40 0x7f57540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 -- 172.21.15.28:0/1460931250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57540039e0 msgr2=0x7f5754003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57540039e0 0x7f5754003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.804+0000 7f5767e00640 1 -- 172.21.15.28:0/1460931250 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5754003520 con 0x7f5770072700 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.805+0000 7f5767e00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f5754004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f575800c0e0 tx=0x7f5754005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8986439377b7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.805+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758007c20 con 0x7f5770072700 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.805+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5758005af0 con 0x7f5770072700 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.805+0000 7f576d400640 1 -- 172.21.15.28:0/1460931250 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5758007070 con 0x7f5770072700 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 -- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 msgr2=0x7f5754004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 --2- 172.21.15.28:0/1460931250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5770072700 0x7f5754004800 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f575800c0e0 tx=0x7f5754005830 comp rx=0 tx=0).stop 2024-09-10T10:51:25.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f576e800640 1 -- 172.21.15.28:0/1460931250 reap_dead start 2024-09-10T10:51:25.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 -- 172.21.15.28:0/1460931250 shutdown_connections 2024-09-10T10:51:25.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 -- 172.21.15.28:0/1460931250 >> 172.21.15.28:0/1460931250 conn(0x7f5770067d70 msgr2=0x7f577006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:25.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 -- 172.21.15.28:0/1460931250 shutdown_connections 2024-09-10T10:51:25.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.806+0000 7f5774c00640 1 -- 172.21.15.28:0/1460931250 wait complete. 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.808+0000 7f5774c00640 1 Processor -- start 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.808+0000 7f5774c00640 1 -- start start 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f577013fca0 0x7f5770140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5770140660 0x7f5770143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f577015da70 con 0x7f5770072700 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f577015d8f0 con 0x7f5770140660 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5774c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f577015dbf0 con 0x7f577013fca0 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41128/0 (socket says 172.21.15.28:41128) 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 -- 172.21.15.28:0/654959793 learned_addr learned my addr 172.21.15.28:0/654959793 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576e800640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5770140660 0x7f5770143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f5767e00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f577013fca0 0x7f5770140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 -- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f577013fca0 msgr2=0x7f5770140120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f577013fca0 0x7f5770140120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 -- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5770140660 msgr2=0x7f5770143e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5770140660 0x7f5770143e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:25.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.809+0000 7f576de00640 1 -- 172.21.15.28:0/654959793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5770140ae0 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f576de00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f576000f5f0 tx=0x7f576000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57600052e0 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5760018070 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5760003810 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f5774c00640 1 -- 172.21.15.28:0/654959793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5770142d60 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.810+0000 7f5774c00640 1 -- 172.21.15.28:0/654959793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5770143320 con 0x7f5770072700 2024-09-10T10:51:25.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.811+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f577015dfd0 con 0x7f5770072700 2024-09-10T10:51:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.812+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57600039b0 con 0x7f5770072700 2024-09-10T10:51:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.813+0000 7f5766000640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 0x7f574c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:25.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.813+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f576008af90 con 0x7f5770072700 2024-09-10T10:51:25.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.814+0000 7f5767e00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 0x7f574c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:25.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.815+0000 7f5767e00640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 0x7f574c06e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5758032000 tx=0x7f5758034040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:25.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:25.817+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5760058ec0 con 0x7f5770072700 2024-09-10T10:51:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.037+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f57700c6260 con 0x7f5770072700 2024-09-10T10:51:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.038+0000 7f5766000640 1 -- 172.21.15.28:0/654959793 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v124) ==== 53+0+73 (secure 0 0 0) 0x7f576005d1c0 con 0x7f5770072700 2024-09-10T10:51:26.040 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 msgr2=0x7f574c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 0x7f574c06e320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5758032000 tx=0x7f5758034040 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 msgr2=0x7f57700a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f576000f5f0 tx=0x7f576000fab0 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 shutdown_connections 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f574c06be60 0x7f574c06e320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5770140660 0x7f5770143e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f577013fca0 0x7f5770140120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 --2- 172.21.15.28:0/654959793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5770072700 0x7f57700a9e80 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:26.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 >> 172.21.15.28:0/654959793 conn(0x7f5770067d70 msgr2=0x7f577006f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:26.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 shutdown_connections 2024-09-10T10:51:26.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26.040+0000 7f575f400640 1 -- 172.21.15.28:0/654959793 wait complete. 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:51:26.058 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:51:26.059 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:51:26.059 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:26.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.059 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:51:26.095 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:51:26.095 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:51:26.095 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:51:26.095 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.095 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:51:26.096 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42219sec preferred_lft 42219sec 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.097 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:26.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.099 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:51:26.189 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:51:26.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.192 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:51:26.228 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:51:26.228 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:51:26.228 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:51:26.228 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:51:26.228 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:51:26.229 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:51:26.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:51:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:51:26.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.426 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:51:26.462 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:51:26.462 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:51:26.462 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:26.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.462 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:51:26.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:51:26.558 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:26.593 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:26.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:51:26.594 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:26.669 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:51:26.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.670 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:26.800 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:26.800 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:26.800 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:26.816 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:26.816 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:26.816 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:26.816 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:51:26.816 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:51:26.839 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:51:26.840 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:51:26.842 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:51:26.842 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:51:26.842 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:51:26.842 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:51:26.842 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:51:26.845 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:51:26.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:26.845 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:26.900 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:51:26.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:51:26.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:51:26.990 INFO:teuthology.orchestra.run.smithi028.stdout:client7312 2024-09-10T10:51:26.991 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:26.991 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:27.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:27.027 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:51:27.071 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:51:27.104 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:51:27.104 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:51:27.104 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:27.104 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:51:27.104 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:51:27.105 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_openfiles.0' did not caused DAMAGED state 2024-09-10T10:51:27.105 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 500.00000000' 2024-09-10T10:51:27.105 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:27.105 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:27.181 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:27.181 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:27.257 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:51:27.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.452+0000 7fbd38200640 1 Processor -- start 2024-09-10T10:51:27.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.452+0000 7fbd38200640 1 -- start start 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd30147230 0x7fbd3006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd30147e90 0x7fbd300720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 0x7fbd30161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd300b34d0 con 0x7fbd30147230 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd300ae9e0 con 0x7fbd30147e90 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd38200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd30072a20 con 0x7fbd300725f0 2024-09-10T10:51:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd30147230 0x7fbd3006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd30147230 0x7fbd3006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41136/0 (socket says 172.21.15.28:41136) 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 -- 172.21.15.28:0/3900078055 learned_addr learned my addr 172.21.15.28:0/3900078055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd36200640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 0x7fbd30161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd34e00640 1 --2- 172.21.15.28:0/3900078055 >> v2:172.21.15.138:3300/0 conn(0x7fbd30147e90 0x7fbd300720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 -- 172.21.15.28:0/3900078055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 msgr2=0x7fbd30161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 0x7fbd30161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 -- 172.21.15.28:0/3900078055 >> v2:172.21.15.138:3300/0 conn(0x7fbd30147e90 msgr2=0x7fbd300720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 --2- 172.21.15.28:0/3900078055 >> v2:172.21.15.138:3300/0 conn(0x7fbd30147e90 0x7fbd300720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd35800640 1 -- 172.21.15.28:0/3900078055 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3006f480 con 0x7fbd30147230 2024-09-10T10:51:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.454+0000 7fbd36200640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 0x7fbd30161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.455+0000 7fbd35800640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd30147230 0x7fbd3006e0b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbd3006ccb0 tx=0x7fbd2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775c4391b74100e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd2fe00640 1 -- 172.21.15.28:0/3900078055 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24037070 con 0x7fbd30147230 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd2fe00640 1 -- 172.21.15.28:0/3900078055 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd24032440 con 0x7fbd30147230 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd2fe00640 1 -- 172.21.15.28:0/3900078055 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24032810 con 0x7fbd30147230 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 -- 172.21.15.28:0/3900078055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd30147230 msgr2=0x7fbd3006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd30147230 0x7fbd3006e0b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbd3006ccb0 tx=0x7fbd2402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 -- 172.21.15.28:0/3900078055 shutdown_connections 2024-09-10T10:51:27.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd300725f0 0x7fbd30161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 --2- 172.21.15.28:0/3900078055 >> v2:172.21.15.138:3300/0 conn(0x7fbd30147e90 0x7fbd300720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 --2- 172.21.15.28:0/3900078055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd30147230 0x7fbd3006e0b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 -- 172.21.15.28:0/3900078055 >> 172.21.15.28:0/3900078055 conn(0x7fbd30067d70 msgr2=0x7fbd3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 -- 172.21.15.28:0/3900078055 shutdown_connections 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.456+0000 7fbd38200640 1 -- 172.21.15.28:0/3900078055 wait complete. 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.458+0000 7fbd38200640 1 Processor -- start 2024-09-10T10:51:27.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.458+0000 7fbd38200640 1 -- start start 2024-09-10T10:51:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 0x7fbd3015df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd30147e90 0x7fbd301335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3006eec0 con 0x7fbd300725f0 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3006ebe0 con 0x7fbd30147e90 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd38200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3006f1a0 con 0x7fbd30147230 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd35800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd35800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41146/0 (socket says 172.21.15.28:41146) 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd35800640 1 -- 172.21.15.28:0/2195576493 learned_addr learned my addr 172.21.15.28:0/2195576493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd34e00640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 0x7fbd3015df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.459+0000 7fbd36200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd30147e90 0x7fbd301335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 -- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 msgr2=0x7fbd3015df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 0x7fbd3015df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 -- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd30147e90 msgr2=0x7fbd301335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd30147e90 0x7fbd301335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 -- 172.21.15.28:0/2195576493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3015e440 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd34e00640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 0x7fbd3015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd35800640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbd2402a220 tx=0x7fbd240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24041020 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd240306c0 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd301e6c10 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24037040 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.460+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd30134120 con 0x7fbd300725f0 2024-09-10T10:51:27.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.461+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd30147610 con 0x7fbd300725f0 2024-09-10T10:51:27.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.462+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd24044020 con 0x7fbd300725f0 2024-09-10T10:51:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.463+0000 7fbd2d600640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 0x7fbd1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.463+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7fbd240aba90 con 0x7fbd300725f0 2024-09-10T10:51:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.464+0000 7fbd34e00640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 0x7fbd1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.464+0000 7fbd34e00640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 0x7fbd1006e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd30143520 tx=0x7fbd1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:27.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.468+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd2404b030 con 0x7fbd300725f0 2024-09-10T10:51:27.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.668+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbd300b5470 con 0x7fbd300725f0 2024-09-10T10:51:27.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.668+0000 7fbd2d600640 1 -- 172.21.15.28:0/2195576493 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 12 entries v105) ==== 81+0+730 (secure 0 0 0) 0x7fbd3015e440 con 0x7fbd300725f0 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:51:27.671 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:51:27.672 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:51:27.672 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:51:27.672 INFO:teuthology.orchestra.run.smithi028.stderr:listed 12 entries 2024-09-10T10:51:27.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 msgr2=0x7fbd1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 0x7fbd1006e2f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbd30143520 tx=0x7fbd1c008040 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 msgr2=0x7fbd301f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fbd2402a220 tx=0x7fbd240046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 shutdown_connections 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd1006be30 0x7fbd1006e2f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd30147e90 0x7fbd301335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd30147230 0x7fbd3015df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 --2- 172.21.15.28:0/2195576493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd300725f0 0x7fbd301f4d50 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 >> 172.21.15.28:0/2195576493 conn(0x7fbd30067d70 msgr2=0x7fbd3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:27.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 shutdown_connections 2024-09-10T10:51:27.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.671+0000 7fbd38200640 1 -- 172.21.15.28:0/2195576493 wait complete. 2024-09-10T10:51:27.689 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:51:27.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:27.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:51:27.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:27.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:27.759 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:27.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:27.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.992+0000 7f7cd9200640 1 Processor -- start 2024-09-10T10:51:27.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.992+0000 7f7cd9200640 1 -- start start 2024-09-10T10:51:27.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd9200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7cd415e660 0x7f7cd406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd9200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd40cf000 con 0x7f7cd40d1f90 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd9200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd40d29a0 con 0x7f7cd40d0dd0 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd9200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7cd40b37d0 con 0x7f7cd415e660 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7cd40d1f90 0x7f7cd40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53568/0 (socket says 172.21.15.28:53568) 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.993+0000 7f7cd3e00640 1 -- 172.21.15.28:0/3059152949 learned_addr learned my addr 172.21.15.28:0/3059152949 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2a00640 1 --2- 172.21.15.28:0/3059152949 >> v2:172.21.15.138:3300/0 conn(0x7f7cd415e660 0x7f7cd406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:27.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2000640 1 -- 172.21.15.28:0/3059152949 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1834921534 0 0) 0x7f7cd40cf000 con 0x7f7cd40d1f90 2024-09-10T10:51:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2000640 1 -- 172.21.15.28:0/3059152949 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb8003520 con 0x7f7cd40d1f90 2024-09-10T10:51:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2a00640 1 -- 172.21.15.28:0/3059152949 >> v1:172.21.15.138:6789/0 conn(0x7f7cd40d0dd0 legacy=0x7f7cd40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2a00640 1 -- 172.21.15.28:0/3059152949 >> v1:172.21.15.28:6789/0 conn(0x7f7cd40d1f90 legacy=0x7f7cd40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.994+0000 7f7cd2a00640 1 -- 172.21.15.28:0/3059152949 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd40d2c60 con 0x7f7cd415e660 2024-09-10T10:51:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.995+0000 7f7cd2a00640 1 --2- 172.21.15.28:0/3059152949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd415e660 0x7f7cd406ccd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00097a0 tx=0x7f7cc002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fceff097657ce7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd2000640 1 -- 172.21.15.28:0/3059152949 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0037070 con 0x7f7cd415e660 2024-09-10T10:51:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd2000640 1 -- 172.21.15.28:0/3059152949 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc0032440 con 0x7f7cd415e660 2024-09-10T10:51:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd2000640 1 -- 172.21.15.28:0/3059152949 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0032810 con 0x7f7cd415e660 2024-09-10T10:51:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 -- 172.21.15.28:0/3059152949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd415e660 msgr2=0x7f7cd406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 --2- 172.21.15.28:0/3059152949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd415e660 0x7f7cd406ccd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00097a0 tx=0x7f7cc002a230 comp rx=0 tx=0).stop 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 -- 172.21.15.28:0/3059152949 shutdown_connections 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 --2- 172.21.15.28:0/3059152949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd415e660 0x7f7cd406ccd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 -- 172.21.15.28:0/3059152949 >> 172.21.15.28:0/3059152949 conn(0x7f7cd408bfc0 msgr2=0x7f7cd40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 -- 172.21.15.28:0/3059152949 shutdown_connections 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.996+0000 7f7cd9200640 1 -- 172.21.15.28:0/3059152949 wait complete. 2024-09-10T10:51:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.998+0000 7f7cd9200640 1 Processor -- start 2024-09-10T10:51:28.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.998+0000 7f7cd9200640 1 -- start start 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd40d1f90 0x7f7cd41e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 0x7f7cd415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd40d3220 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd40cf330 con 0x7f7cd40d1f90 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd9200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd40d33a0 con 0x7f7cd415e660 2024-09-10T10:51:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41162/0 (socket says 172.21.15.28:41162) 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd3400640 1 -- 172.21.15.28:0/872340251 learned_addr learned my addr 172.21.15.28:0/872340251 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd2a00640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd40d1f90 0x7f7cd41e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:27.999+0000 7f7cd3e00640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 0x7f7cd415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 -- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 msgr2=0x7f7cd415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 0x7f7cd415d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 -- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd40d1f90 msgr2=0x7f7cd41e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd40d1f90 0x7f7cd41e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 -- 172.21.15.28:0/872340251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd41e5e40 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3e00640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 0x7f7cd415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd3400640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c430 tx=0x7f7cc400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40052d0 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cc4005490 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4007260 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd41e6120 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.000+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41e3880 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.001+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd406d2a0 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.002+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cc4007400 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.003+0000 7f7ccb400640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.003+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6617+0+0 (secure 0 0 0) 0x7f7cc408a670 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.004+0000 7f7cd2a00640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.004+0000 7f7cd2a00640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 0x7f7cb406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.007+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7cc4058630 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.231+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7cd40d1b20 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.234 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:28.232+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.234+0000 7f7ccb400640 1 -- 172.21.15.28:0/872340251 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v125) ==== 161+0+0 (secure 0 0 0) 0x7f7cc405c930 con 0x7f7cd40d0dd0 2024-09-10T10:51:28.236 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:28.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.237+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 msgr2=0x7f7cb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.237+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 0x7f7cb406e2f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0034000 tx=0x7f7cc002e000 comp rx=0 tx=0).stop 2024-09-10T10:51:28.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.237+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 msgr2=0x7f7cd40cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.237+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f7cc400c430 tx=0x7f7cc400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 shutdown_connections 2024-09-10T10:51:28.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7cb406be30 0x7f7cb406e2f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cd415e660 0x7f7cd415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cd40d1f90 0x7f7cd41e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 --2- 172.21.15.28:0/872340251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7cd40d0dd0 0x7f7cd40cc5d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.238+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 >> 172.21.15.28:0/872340251 conn(0x7f7cd408bfc0 msgr2=0x7f7cd40b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.239+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 shutdown_connections 2024-09-10T10:51:28.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.239+0000 7f7cd9200640 1 -- 172.21.15.28:0/872340251 wait complete. 2024-09-10T10:51:28.243 INFO:tasks.ceph.mds.c.smithi028.stderr: -1798> 2024-09-10T10:51:18.768+0000 7f02da400640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2024-09-10T10:51:28.246 INFO:tasks.ceph.mds.c.smithi028.stderr: -543> 2024-09-10T10:51:25.752+0000 7f02df400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:28.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:28.293 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:51:28.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.420+0000 7f15e1a00640 1 Processor -- start 2024-09-10T10:51:28.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.420+0000 7f15e1a00640 1 -- start start 2024-09-10T10:51:28.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.421+0000 7f15e1a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15dc0d0dd0 0x7f15dc0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.421+0000 7f15e1a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f15dc15e660 0x7f15dc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.421+0000 7f15e1a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15dc0cf000 con 0x7f15dc0d1f90 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.421+0000 7f15e1a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f15dc0b37d0 con 0x7f15dc15e660 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.421+0000 7f15e1a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f15dc0cf330 con 0x7f15dc0d0dd0 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15e1000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f15dc0d1f90 0x7f15dc0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53576/0 (socket says 172.21.15.28:53576) 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15e1000640 1 -- 172.21.15.28:0/2799683960 learned_addr learned my addr 172.21.15.28:0/2799683960 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15dbe00640 1 --2- 172.21.15.28:0/2799683960 >> v2:172.21.15.138:3300/0 conn(0x7f15dc0d0dd0 0x7f15dc0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15d3e00640 1 --2- 172.21.15.28:0/2799683960 >> v2:172.21.15.28:3300/0 conn(0x7f15dc15e660 0x7f15dc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15d3e00640 1 -- 172.21.15.28:0/2799683960 >> v2:172.21.15.138:3300/0 conn(0x7f15dc0d0dd0 msgr2=0x7f15dc0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:28.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15db400640 1 -- 172.21.15.28:0/2799683960 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267832984 0 0) 0x7f15dc0cf000 con 0x7f15dc0d1f90 2024-09-10T10:51:28.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15d3e00640 1 --2- 172.21.15.28:0/2799683960 >> v2:172.21.15.138:3300/0 conn(0x7f15dc0d0dd0 0x7f15dc0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15d3e00640 1 -- 172.21.15.28:0/2799683960 >> v1:172.21.15.28:6789/0 conn(0x7f15dc0d1f90 legacy=0x7f15dc0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.422+0000 7f15d3e00640 1 -- 172.21.15.28:0/2799683960 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc0d16f0 con 0x7f15dc15e660 2024-09-10T10:51:28.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.423+0000 7f15d3e00640 1 --2- 172.21.15.28:0/2799683960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc15e660 0x7f15dc06ccd0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0b6fb0 tx=0x7f15c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc94d8854e0b5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.423+0000 7f15db400640 1 -- 172.21.15.28:0/2799683960 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4037070 con 0x7f15dc15e660 2024-09-10T10:51:28.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.423+0000 7f15db400640 1 -- 172.21.15.28:0/2799683960 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c4032440 con 0x7f15dc15e660 2024-09-10T10:51:28.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.423+0000 7f15db400640 1 -- 172.21.15.28:0/2799683960 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4032810 con 0x7f15dc15e660 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 -- 172.21.15.28:0/2799683960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc15e660 msgr2=0x7f15dc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 --2- 172.21.15.28:0/2799683960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc15e660 0x7f15dc06ccd0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0b6fb0 tx=0x7f15c402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 -- 172.21.15.28:0/2799683960 shutdown_connections 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 --2- 172.21.15.28:0/2799683960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc15e660 0x7f15dc06ccd0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 --2- 172.21.15.28:0/2799683960 >> v2:172.21.15.138:3300/0 conn(0x7f15dc0d0dd0 0x7f15dc0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 -- 172.21.15.28:0/2799683960 >> 172.21.15.28:0/2799683960 conn(0x7f15dc08bfc0 msgr2=0x7f15dc0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 -- 172.21.15.28:0/2799683960 shutdown_connections 2024-09-10T10:51:28.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.424+0000 7f15e1a00640 1 -- 172.21.15.28:0/2799683960 wait complete. 2024-09-10T10:51:28.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.426+0000 7f15e1a00640 1 Processor -- start 2024-09-10T10:51:28.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.426+0000 7f15e1a00640 1 -- start start 2024-09-10T10:51:28.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15dc0d0dd0 0x7f15dc1f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15dc15e660 0x7f15dc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc0d2f40 con 0x7f15dc0d1f90 2024-09-10T10:51:28.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc0d2c60 con 0x7f15dc15e660 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15dc0d3220 con 0x7f15dc0d0dd0 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41206/0 (socket says 172.21.15.28:41206) 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15d3e00640 1 -- 172.21.15.28:0/1717468636 learned_addr learned my addr 172.21.15.28:0/1717468636 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15dbe00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15dc0d0dd0 0x7f15dc1f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.427+0000 7f15e1000640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15dc15e660 0x7f15dc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 -- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15dc0d0dd0 msgr2=0x7f15dc1f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15dc0d0dd0 0x7f15dc1f7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 -- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15dc15e660 msgr2=0x7f15dc0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15dc15e660 0x7f15dc0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 -- 172.21.15.28:0/1717468636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc1f77b0 con 0x7f15dc0d1f90 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d3e00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f15c4035040 tx=0x7f15c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4037050 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c4038040 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c4004450 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc0b7700 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.428+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc0b7cc0 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.429+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15dc0d2420 con 0x7f15dc0d1f90 2024-09-10T10:51:28.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.430+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f15c403c6d0 con 0x7f15dc0d1f90 2024-09-10T10:51:28.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.431+0000 7f15d8c00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:28.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.431+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6906+0+0 (secure 0 0 0) 0x7f15c40abdd0 con 0x7f15dc0d1f90 2024-09-10T10:51:28.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.431+0000 7f15dbe00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:28.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.432+0000 7f15dbe00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 0x7f15b806e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009aa0 tx=0x7f15cc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:28.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.435+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f15c4079b50 con 0x7f15dc0d1f90 2024-09-10T10:51:28.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:28.689+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f15dc0b5f70 con 0x7f15dc0d1f90 2024-09-10T10:51:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.240+0000 7f15d8c00640 1 -- 172.21.15.28:0/1717468636 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v126) ==== 107+0+0 (secure 0 0 0) 0x7f15c407de50 con 0x7f15dc0d1f90 2024-09-10T10:51:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:51:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.243+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 msgr2=0x7f15b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.243+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 0x7f15b806e2f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f15cc009aa0 tx=0x7f15cc009190 comp rx=0 tx=0).stop 2024-09-10T10:51:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.243+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 msgr2=0x7f15dc0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.243+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f15c4035040 tx=0x7f15c40046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 shutdown_connections 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f15b806be30 0x7f15b806e2f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15dc15e660 0x7f15dc0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15dc0d1f90 0x7f15dc0cc4e0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 --2- 172.21.15.28:0/1717468636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15dc0d0dd0 0x7f15dc1f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 >> 172.21.15.28:0/1717468636 conn(0x7f15dc08bfc0 msgr2=0x7f15dc0b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 shutdown_connections 2024-09-10T10:51:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.244+0000 7f15e1a00640 1 -- 172.21.15.28:0/1717468636 wait complete. 2024-09-10T10:51:29.268 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.385+0000 7f63ccb3ff40 1 Processor -- start 2024-09-10T10:51:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.385+0000 7f63ccb3ff40 1 -- start start 2024-09-10T10:51:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b19176d8e0 0x55b1917259f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917f93e0 0x55b1917fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b19176e530 con 0x55b1917fc9c0 2024-09-10T10:51:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b1916cbb30 con 0x55b19176d8e0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b1917fbee0 con 0x55b1917f93e0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ca800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ca800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41228/0 (socket says 172.21.15.28:41228) 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.386+0000 7f63ca800640 1 -- 172.21.15.28:0/3677409954 learned_addr learned my addr 172.21.15.28:0/3677409954 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63c9e00640 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b19176d8e0 0x55b1917259f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63c9400640 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917f93e0 0x55b1917fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 -- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917f93e0 msgr2=0x55b1917fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917f93e0 0x55b1917fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 -- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b19176d8e0 msgr2=0x55b1917259f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b19176d8e0 0x55b1917259f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 -- 172.21.15.28:0/3677409954 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1917708f0 con 0x55b1917fc9c0 2024-09-10T10:51:29.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63ca800640 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f63c40016e0 tx=0x7f63c4001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386cb65570bb3264 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63c3e00640 1 -- 172.21.15.28:0/3677409954 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c40062e0 con 0x55b1917fc9c0 2024-09-10T10:51:29.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63c3e00640 1 -- 172.21.15.28:0/3677409954 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63c4003650 con 0x55b1917fc9c0 2024-09-10T10:51:29.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.387+0000 7f63c3e00640 1 -- 172.21.15.28:0/3677409954 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4003a00 con 0x55b1917fc9c0 2024-09-10T10:51:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/3677409954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 msgr2=0x55b1917fab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f63c40016e0 tx=0x7f63c4001ba0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/3677409954 shutdown_connections 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b1917fc9c0 0x55b1917fab20 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917f93e0 0x55b1917fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/3677409954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b19176d8e0 0x55b1917259f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/3677409954 >> 172.21.15.28:0/3677409954 conn(0x55b1917720a0 msgr2=0x55b191772480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/3677409954 shutdown_connections 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.388+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/3677409954 wait complete. 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.389+0000 7f63ccb3ff40 1 Processor -- start 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.389+0000 7f63ccb3ff40 1 -- start start 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1917f93e0 0x55b1918957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917fc9c0 0x55b191895d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b191770610 con 0x55b19176d8e0 2024-09-10T10:51:29.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b191770d00 con 0x55b1917f93e0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ccb3ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b19170f160 con 0x55b1917fc9c0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41230/0 (socket says 172.21.15.28:41230) 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 learned_addr learned my addr 172.21.15.28:0/4012737175 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1917f93e0 0x55b1918957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63ca800640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917fc9c0 0x55b191895d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917fc9c0 msgr2=0x55b191895d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.390+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b1917fc9c0 0x55b191895d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1917f93e0 msgr2=0x55b1918957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1917f93e0 0x55b1918957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1917ec7f0 con 0x55b19176d8e0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f63bc055040 tx=0x7f63bc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c2a00640 1 -- 172.21.15.28:0/4012737175 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc068640 con 0x55b19176d8e0 2024-09-10T10:51:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c2a00640 1 -- 172.21.15.28:0/4012737175 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63bc0687e0 con 0x55b19176d8e0 2024-09-10T10:51:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b1917efe30 con 0x55b19176d8e0 2024-09-10T10:51:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63c2a00640 1 -- 172.21.15.28:0/4012737175 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc068bb0 con 0x55b19176d8e0 2024-09-10T10:51:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.391+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1917f38d0 con 0x55b19176d8e0 2024-09-10T10:51:29.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.393+0000 7f63c2a00640 1 -- 172.21.15.28:0/4012737175 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f63bc052070 con 0x55b19176d8e0 2024-09-10T10:51:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.394+0000 7f63c2a00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 0x7f63ac06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.394+0000 7f63c2a00640 1 -- 172.21.15.28:0/4012737175 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f63bc0ce670 con 0x55b19176d8e0 2024-09-10T10:51:29.397 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.394+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b1917fec20 0x55b1917f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.394+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1917f2100 con 0x55b1917fec20 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.394+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 0x7f63ac06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.395+0000 7f63ca800640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b1917fec20 0x55b1917f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.395+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 0x7f63ac06ccd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005fd0 tx=0x7f63b8005b10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.395+0000 7f63ca800640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b1917fec20 0x55b1917f1bc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.397+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v107'195 uv169 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.397+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1917f2510 con 0x55b1917fec20 2024-09-10T10:51:29.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.400+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v107'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.400+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1917f28e0 con 0x55b1917fec20 2024-09-10T10:51:29.403 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:29.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.401+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v107'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.401+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1917f2c80 con 0x55b1917fec20 2024-09-10T10:51:29.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.403+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v107'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.403+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189fab0 con 0x55b1917fec20 2024-09-10T10:51:29.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.405+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v107'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.405+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189eaa0 con 0x55b1917fec20 2024-09-10T10:51:29.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.406+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v107'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.407+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.408+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v107'201 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.408+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.410+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v107'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.410+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.414 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:29.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.412+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v107'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.412+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.414+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v107'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.414+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.415+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v107'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.415+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.416+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v107'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.416+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.418+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v107'207 uv182 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.418+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.419+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v107'208 uv208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.419+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.423 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:29.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.421+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v107'209 uv209 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.421+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.422+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v107'210 uv210 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.422+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.425+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v107'211 uv211 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.425+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189ee40 con 0x55b1917fec20 2024-09-10T10:51:29.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.426+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v107'212 uv212 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.426+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f6d0 con 0x55b1917fec20 2024-09-10T10:51:29.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.427+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v107'213 uv213 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.428+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f6d0 con 0x55b1917fec20 2024-09-10T10:51:29.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.430+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v107'214 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.430+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.431+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v107'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.431+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.434 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:29.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.432+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v107'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.432+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.433+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v107'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.433+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.435+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v107'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.435+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.436+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v107'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.436+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.438+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v107'220 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.438+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.440+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v107'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.440+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.441+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v107'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.443 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:29.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.442+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.443+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v107'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.443+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.445+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v107'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.445+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b19189f5b0 con 0x55b1917fec20 2024-09-10T10:51:29.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.446+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v107'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1917fec20 2024-09-10T10:51:29.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.446+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b19189f5b0 0x55b1918a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.446+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a28b0 con 0x55b19189f5b0 2024-09-10T10:51:29.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.447+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b19189f5b0 0x55b1918a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.447+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b19189f5b0 0x55b1918a2370 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.449+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v107'152 uv133 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.449+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a2c50 con 0x55b19189f5b0 2024-09-10T10:51:29.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.451+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v107'153 uv153 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.451+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a2ff0 con 0x55b19189f5b0 2024-09-10T10:51:29.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.452+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v107'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.454 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:29.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.452+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a3390 con 0x55b19189f5b0 2024-09-10T10:51:29.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.454+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v107'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.454+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a3730 con 0x55b19189f5b0 2024-09-10T10:51:29.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.455+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v107'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.455+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a3ad0 con 0x55b19189f5b0 2024-09-10T10:51:29.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.456+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v107'157 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.456+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a3e70 con 0x55b19189f5b0 2024-09-10T10:51:29.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.458+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v107'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.458+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.461 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:29.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.459+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v107'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.459+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.460+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v107'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.460+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.462+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v107'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.462+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.464+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v107'162 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.464+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.465+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v107'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.465+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.469 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:29.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.467+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v107'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.467+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a8900 con 0x55b19189f5b0 2024-09-10T10:51:29.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.469+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v107'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.469+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a8900 con 0x55b19189f5b0 2024-09-10T10:51:29.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.470+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v107'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.470+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.472+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v107'167 uv147 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.472+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.473+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v107'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.473+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.477 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:29.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.475+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v107'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.475+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a5220 con 0x55b19189f5b0 2024-09-10T10:51:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.476+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v107'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.476+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a5220 con 0x55b19189f5b0 2024-09-10T10:51:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.478+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v107'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.478+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a4210 con 0x55b19189f5b0 2024-09-10T10:51:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.479+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v107'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b19189f5b0 2024-09-10T10:51:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.480+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b1918a4720 0x55b1918a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.480+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7050 con 0x55b1918a4720 2024-09-10T10:51:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.480+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b1918a4720 0x55b1918a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.480+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b1918a4720 0x55b1918a6b10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.482+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v107'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63b8016480 con 0x55b1918a4720 2024-09-10T10:51:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.482+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a73f0 con 0x55b1918a4720 2024-09-10T10:51:29.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.483+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v107'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.483+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a78a0 con 0x55b1918a4720 2024-09-10T10:51:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.486+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v107'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.488 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.486+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a9180 con 0x55b1918a4720 2024-09-10T10:51:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.489+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v107'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.489+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a9520 con 0x55b1918a4720 2024-09-10T10:51:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.490+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v107'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.490+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a8170 con 0x55b1918a4720 2024-09-10T10:51:29.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.493+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v107'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.493+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b1918a8a90 0x55b1918ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.493+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918abe10 con 0x55b1918a8a90 2024-09-10T10:51:29.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.493+0000 7f63ca800640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b1918a8a90 0x55b1918ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.494+0000 7f63ca800640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b1918a8a90 0x55b1918ab8d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.495+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v107'32 uv31 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.495+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918abe10 con 0x55b1918a8a90 2024-09-10T10:51:29.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.496+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v107'33 uv33 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.496+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918abe10 con 0x55b1918a8a90 2024-09-10T10:51:29.499 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:29.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.497+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v107'34 uv34 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.497+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac220 con 0x55b1918a8a90 2024-09-10T10:51:29.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.498+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v107'35 uv35 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.499+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac220 con 0x55b1918a4720 2024-09-10T10:51:29.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.500+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v107'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.500+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.501+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v107'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.501+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.504 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:29.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.502+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v107'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.502+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ad5d0 con 0x55b1918a4720 2024-09-10T10:51:29.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.503+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v107'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.503+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ad5d0 con 0x55b1918a4720 2024-09-10T10:51:29.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.504+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v107'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.504+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.505+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v107'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.505+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.507+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v107'70 uv64 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.507+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.508+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.508+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.511 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:29.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.509+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.510+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ad5d0 con 0x55b1918a8a90 2024-09-10T10:51:29.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.511+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.511+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ad5d0 con 0x55b1918a8a90 2024-09-10T10:51:29.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.512+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v107'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.512+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.514+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v107'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.514+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.515+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v107'76 uv69 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.515+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.517+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v107'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.517+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.520 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:29.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.518+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v107'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.518+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.520+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v107'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.520+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a8a90 2024-09-10T10:51:29.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.521+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v107'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.521+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.522+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v107'32 uv31 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.522+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.523+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v107'33 uv33 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.523+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.526 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:29.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.524+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v107'34 uv34 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.524+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.526+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v107'35 uv35 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.526+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918ac5c0 con 0x55b1918a4720 2024-09-10T10:51:29.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.527+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v107'36 uv36 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918a4720 2024-09-10T10:51:29.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.527+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b1918acc40 0x55b1918af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.527+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af640 con 0x55b1918acc40 2024-09-10T10:51:29.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.527+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b1918acc40 0x55b1918af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.527+0000 7f63c9e00640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b1918acc40 0x55b1918af030 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.529+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v107'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.529+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.530+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v107'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.530+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.533 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:29.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.532+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v107'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.532+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b0a20 con 0x55b1918acc40 2024-09-10T10:51:29.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.533+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v107'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.533+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b0a20 con 0x55b1918acc40 2024-09-10T10:51:29.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.534+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v107'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.534+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.535+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v107'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.535+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.537+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v107'64 uv58 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.537+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.538+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v107'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.538+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.541 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:29.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.539+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v107'66 uv66 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.539+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7da0 con 0x55b1918acc40 2024-09-10T10:51:29.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.540+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v107'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.540+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7da0 con 0x55b1918acc40 2024-09-10T10:51:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.542+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v107'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.542+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7da0 con 0x55b1918acc40 2024-09-10T10:51:29.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.543+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v107'69 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.543+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7da0 con 0x55b1918acc40 2024-09-10T10:51:29.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.544+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v107'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.544+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7d80 con 0x55b1918acc40 2024-09-10T10:51:29.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.546+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v107'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.548 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:29.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.546+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918a7d80 con 0x55b1918acc40 2024-09-10T10:51:29.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.547+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v107'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.547+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b28a0 con 0x55b1918acc40 2024-09-10T10:51:29.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.549+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v107'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.549+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.550+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v107'31 uv29 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.550+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.552+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v107'32 uv32 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.552+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.555 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:29.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.553+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v107'33 uv33 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.553+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.555+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v107'34 uv34 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.555+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.556+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v107'39 uv38 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.556+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.557+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v107'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.557+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.558+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v107'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.560 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:29.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.558+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b0a20 con 0x55b1918a8a90 2024-09-10T10:51:29.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.559+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v107'42 uv42 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.559+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b0a20 con 0x55b1918a8a90 2024-09-10T10:51:29.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.560+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v107'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.560+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918a8a90 2024-09-10T10:51:29.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.562+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v107'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63c400f030 con 0x55b1918a8a90 2024-09-10T10:51:29.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.562+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b1918b02e0 0x55b1918b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.562+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918b02e0 2024-09-10T10:51:29.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.562+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b1918b02e0 0x55b1918b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.563+0000 7f63c9400640 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b1918b02e0 0x55b1918b06c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.566+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v107'32 uv31 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918b02e0 2024-09-10T10:51:29.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.566+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918b02e0 2024-09-10T10:51:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.567+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v107'33 uv33 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918b02e0 2024-09-10T10:51:29.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.567+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918b02e0 2024-09-10T10:51:29.569 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.568+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v107'34 uv34 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918b02e0 2024-09-10T10:51:29.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.568+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918b02e0 2024-09-10T10:51:29.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.569+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v107'35 uv35 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918b02e0 2024-09-10T10:51:29.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.569+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918b02e0 2024-09-10T10:51:29.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.570+0000 7f63c9400640 1 -- 172.21.15.28:0/4012737175 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v107'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f63b80168e0 con 0x55b1918b02e0 2024-09-10T10:51:29.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.570+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.571+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v107'82 uv73 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.571+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.573+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v107'83 uv83 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.573+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.576 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:29.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.574+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v107'84 uv84 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.574+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.575+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v107'85 uv85 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.575+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918afa10 con 0x55b1918acc40 2024-09-10T10:51:29.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.576+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v107'86 uv86 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.576+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.577+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v107'87 uv80 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.578+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.579+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v107'88 uv88 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.579+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.582 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.580+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v107'89 uv89 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.580+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.581+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v107'90 uv90 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.581+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.582+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v107'91 uv91 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.582+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918af9e0 con 0x55b1918acc40 2024-09-10T10:51:29.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.583+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v107'92 uv92 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.584+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107) -- 0x55b1918b12d0 con 0x55b1918acc40 2024-09-10T10:51:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63c9e00640 1 -- 172.21.15.28:0/4012737175 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v107'93 uv93 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f63bc06e020 con 0x55b1918acc40 2024-09-10T10:51:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b1918a4720 msgr2=0x55b1918a6b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b1918a4720 0x55b1918a6b10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b1918acc40 msgr2=0x55b1918af030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b1918acc40 0x55b1918af030 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b1918b02e0 msgr2=0x55b1918b06c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b1918b02e0 0x55b1918b06c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b1918a8a90 msgr2=0x55b1918ab8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b1918a8a90 0x55b1918ab8d0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b19189f5b0 msgr2=0x55b1918a2370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b19189f5b0 0x55b1918a2370 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 reap_dead start 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b1917fec20 msgr2=0x55b1917f1bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b1917fec20 0x55b1917f1bc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 reap_dead start 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ca800640 1 -- 172.21.15.28:0/4012737175 reap_dead start 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 msgr2=0x7f63ac06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 0x7f63ac06ccd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f63b8005fd0 tx=0x7f63b8005b10 comp rx=0 tx=0).stop 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 msgr2=0x55b19176fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f63bc055040 tx=0x7f63bc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 shutdown_connections 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f63ac06a810 0x7f63ac06ccd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 --2- 172.21.15.28:0/4012737175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b19176d8e0 0x55b19176fbb0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 >> 172.21.15.28:0/4012737175 conn(0x55b1917720a0 msgr2=0x55b191724700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 shutdown_connections 2024-09-10T10:51:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.585+0000 7f63ccb3ff40 1 -- 172.21.15.28:0/4012737175 wait complete. 2024-09-10T10:51:29.592 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 500.00000000 2024-09-10T10:51:29.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.706+0000 7f3d26745f40 1 Processor -- start 2024-09-10T10:51:29.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.706+0000 7f3d26745f40 1 -- start start 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868bd3a470 0x55868bea11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868be103e0 0x55868be0b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55868bce2b30 con 0x55868be139c0 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55868bce6b70 con 0x55868bd3a470 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d26745f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55868be12ee0 con 0x55868be103e0 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41234/0 (socket says 172.21.15.28:41234) 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 -- 172.21.15.28:0/857900121 learned_addr learned my addr 172.21.15.28:0/857900121 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:29.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d23a00640 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868bd3a470 0x55868bea11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d23000640 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868be103e0 0x55868be0b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 -- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868be103e0 msgr2=0x55868be0b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868be103e0 0x55868be0b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 -- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868bd3a470 msgr2=0x55868bea11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868bd3a470 0x55868bea11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.707+0000 7f3d24400640 1 -- 172.21.15.28:0/857900121 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868bea1700 con 0x55868be139c0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d24400640 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0015e0 tx=0x7f3d1c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cb2449e24dc41a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d21c00640 1 -- 172.21.15.28:0/857900121 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004520 con 0x55868be139c0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d21c00640 1 -- 172.21.15.28:0/857900121 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1c0046e0 con 0x55868be139c0 2024-09-10T10:51:29.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d21c00640 1 -- 172.21.15.28:0/857900121 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004ad0 con 0x55868be139c0 2024-09-10T10:51:29.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 -- 172.21.15.28:0/857900121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 msgr2=0x55868be16790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0015e0 tx=0x7f3d1c001aa0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 -- 172.21.15.28:0/857900121 shutdown_connections 2024-09-10T10:51:29.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be139c0 0x55868be16790 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868be103e0 0x55868be0b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.708+0000 7f3d26745f40 1 --2- 172.21.15.28:0/857900121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868bd3a470 0x55868bea11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.709+0000 7f3d26745f40 1 -- 172.21.15.28:0/857900121 >> 172.21.15.28:0/857900121 conn(0x55868bd890a0 msgr2=0x55868bd6dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.709+0000 7f3d26745f40 1 -- 172.21.15.28:0/857900121 shutdown_connections 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.709+0000 7f3d26745f40 1 -- 172.21.15.28:0/857900121 wait complete. 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.709+0000 7f3d26745f40 1 Processor -- start 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.709+0000 7f3d26745f40 1 -- start start 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868bd3a470 0x55868be06000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 0x55868be0df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868be139c0 0x55868be0e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55868be14ff0 con 0x55868be103e0 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55868be0fa70 con 0x55868be139c0 2024-09-10T10:51:29.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d26745f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55868be0f1b0 con 0x55868bd3a470 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.710+0000 7f3d23000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 0x55868be0df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 0x55868be0df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41240/0 (socket says 172.21.15.28:41240) 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 -- 172.21.15.28:0/1633995579 learned_addr learned my addr 172.21.15.28:0/1633995579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d24400640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868be139c0 0x55868be0e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23a00640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868bd3a470 0x55868be06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 -- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868bd3a470 msgr2=0x55868be06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55868bd3a470 0x55868be06000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 -- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868be139c0 msgr2=0x55868be0e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868be139c0 0x55868be0e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 -- 172.21.15.28:0/1633995579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868be0a610 con 0x55868be103e0 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d24400640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55868be139c0 0x55868be0e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:29.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.711+0000 7f3d23000640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 0x55868be0df60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3d10025c60 tx=0x7f3d10002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.712+0000 7f3d1be00640 1 -- 172.21.15.28:0/1633995579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1006b070 con 0x55868be103e0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.712+0000 7f3d1be00640 1 -- 172.21.15.28:0/1633995579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d10052960 con 0x55868be103e0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.712+0000 7f3d1be00640 1 -- 172.21.15.28:0/1633995579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10056070 con 0x55868be103e0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.712+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55868be0a8f0 con 0x55868be103e0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.712+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55868be0a2c0 con 0x55868be103e0 2024-09-10T10:51:29.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.714+0000 7f3d1be00640 1 -- 172.21.15.28:0/1633995579 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3d1006e070 con 0x55868be103e0 2024-09-10T10:51:29.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.714+0000 7f3d1be00640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d0406a840 0x7f3d0406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.715+0000 7f3d1be00640 1 -- 172.21.15.28:0/1633995579 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f3d100ce0f0 con 0x55868be103e0 2024-09-10T10:51:29.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.715+0000 7f3d26745f40 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55868be0be40 0x55868be0c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.715+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e107) -- 0x55868be0c790 con 0x55868be0be40 2024-09-10T10:51:29.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.715+0000 7f3d23a00640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d0406a840 0x7f3d0406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.715+0000 7f3d24400640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55868be0be40 0x55868be0c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.716+0000 7f3d23a00640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d0406a840 0x7f3d0406cd00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d14002fd0 tx=0x7f3d14001810 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.716+0000 7f3d24400640 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55868be0be40 0x55868be0c250 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d24400640 1 -- 172.21.15.28:0/1633995579 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [delete] v107'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3d1c00f030 con 0x55868be0be40 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55868be0be40 msgr2=0x55868be0c250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55868be0be40 0x55868be0c250 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d0406a840 msgr2=0x7f3d0406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3d0406a840 0x7f3d0406cd00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d14002fd0 tx=0x7f3d14001810 comp rx=0 tx=0).stop 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 msgr2=0x55868be0df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 --2- 172.21.15.28:0/1633995579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55868be103e0 0x55868be0df60 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f3d10025c60 tx=0x7f3d10002fb0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d24400640 1 -- 172.21.15.28:0/1633995579 reap_dead start 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 shutdown_connections 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.717+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 >> 172.21.15.28:0/1633995579 conn(0x55868bd890a0 msgr2=0x55868bd6ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.718+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 shutdown_connections 2024-09-10T10:51:29.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.718+0000 7f3d26745f40 1 -- 172.21.15.28:0/1633995579 wait complete. 2024-09-10T10:51:29.724 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:29.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.880+0000 7f461ee00640 1 Processor -- start 2024-09-10T10:51:29.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.880+0000 7f461ee00640 1 -- start start 2024-09-10T10:51:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 0x7f4618071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4618073980 con 0x7f461815d2a0 2024-09-10T10:51:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4618073c20 con 0x7f46180cc280 2024-09-10T10:51:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f461806ecb0 con 0x7f46180af5a0 2024-09-10T10:51:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f461815d2a0 0x7f461806e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53608/0 (socket says 172.21.15.28:53608) 2024-09-10T10:51:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.881+0000 7f461d000640 1 -- 172.21.15.28:0/1777277501 learned_addr learned my addr 172.21.15.28:0/1777277501 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f461da00640 1 --2- 172.21.15.28:0/1777277501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 0x7f4618071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699607934 0 0) 0x7f4618073980 con 0x7f461815d2a0 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4600003520 con 0x7f461815d2a0 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720138736 0 0) 0x7f4618073c20 con 0x7f46180cc280 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4618073980 con 0x7f46180cc280 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f461da00640 1 -- 172.21.15.28:0/1777277501 >> v1:172.21.15.138:6789/0 conn(0x7f46180cc280 legacy=0x7f46180cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f461da00640 1 -- 172.21.15.28:0/1777277501 >> v1:172.21.15.28:6789/0 conn(0x7f461815d2a0 legacy=0x7f461806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.882+0000 7f461da00640 1 -- 172.21.15.28:0/1777277501 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461806dda0 con 0x7f46180af5a0 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.883+0000 7f461da00640 1 --2- 172.21.15.28:0/1777277501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 0x7f4618071cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097d0 tx=0x7f460c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3dbb5fd4ba5c766 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c037070 con 0x7f46180af5a0 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c032440 con 0x7f46180af5a0 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f4617e00640 1 -- 172.21.15.28:0/1777277501 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c032810 con 0x7f46180af5a0 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 -- 172.21.15.28:0/1777277501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 msgr2=0x7f4618071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 --2- 172.21.15.28:0/1777277501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 0x7f4618071cf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f460c0097d0 tx=0x7f460c02a210 comp rx=0 tx=0).stop 2024-09-10T10:51:29.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 -- 172.21.15.28:0/1777277501 shutdown_connections 2024-09-10T10:51:29.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 --2- 172.21.15.28:0/1777277501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180af5a0 0x7f4618071cf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 -- 172.21.15.28:0/1777277501 >> 172.21.15.28:0/1777277501 conn(0x7f46180b1bd0 msgr2=0x7f46180b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:29.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 -- 172.21.15.28:0/1777277501 shutdown_connections 2024-09-10T10:51:29.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.884+0000 7f461ee00640 1 -- 172.21.15.28:0/1777277501 wait complete. 2024-09-10T10:51:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.886+0000 7f461ee00640 1 Processor -- start 2024-09-10T10:51:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.886+0000 7f461ee00640 1 -- start start 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 0x7f4618156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 0x7f46180d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4618073f00 con 0x7f461815d2a0 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46180af980 con 0x7f46180af5a0 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4618072230 con 0x7f46180cc280 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41250/0 (socket says 172.21.15.28:41250) 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461e400640 1 -- 172.21.15.28:0/417930350 learned_addr learned my addr 172.21.15.28:0/417930350 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461da00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 0x7f4618156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.887+0000 7f461d000640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 0x7f46180d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 -- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 msgr2=0x7f46180d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 0x7f46180d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 -- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 msgr2=0x7f4618156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 0x7f4618156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 -- 172.21.15.28:0/417930350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4618156ac0 con 0x7f461815d2a0 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461da00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 0x7f4618156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461d000640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 0x7f46180d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:29.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461e400640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f461000c790 tx=0x7f461000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610003b70 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4610003d10 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461000aa50 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4618156da0 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.888+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4618155da0 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.889+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4618071cf0 con 0x7f461815d2a0 2024-09-10T10:51:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.890+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f461000abf0 con 0x7f461815d2a0 2024-09-10T10:51:29.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.891+0000 7f4615600640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 0x7f45f806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:29.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.891+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f461008b2b0 con 0x7f461815d2a0 2024-09-10T10:51:29.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.892+0000 7f461da00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 0x7f45f806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:29.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.893+0000 7f461da00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 0x7f45f806e320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f460c034000 tx=0x7f460c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:29.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:29.896+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4610059710 con 0x7f461815d2a0 2024-09-10T10:51:30.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.117+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f46180cc940 con 0x7f461815d2a0 2024-09-10T10:51:30.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.671+0000 7f4615600640 1 -- 172.21.15.28:0/417930350 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v127) ==== 162+0+0 (secure 0 0 0) 0x7f4610059010 con 0x7f461815d2a0 2024-09-10T10:51:30.673 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 msgr2=0x7f45f806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 0x7f45f806e320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f460c034000 tx=0x7f460c02e000 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 msgr2=0x7f46181f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f461000c790 tx=0x7f461000cc50 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 shutdown_connections 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f45f806be60 0x7f45f806e320 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f461815d2a0 0x7f46181f3f00 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46180cc280 0x7f46180d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 --2- 172.21.15.28:0/417930350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46180af5a0 0x7f4618156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:30.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.673+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 >> 172.21.15.28:0/417930350 conn(0x7f46180b1bd0 msgr2=0x7f46180aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:30.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.674+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 shutdown_connections 2024-09-10T10:51:30.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:30.674+0000 7f461ee00640 1 -- 172.21.15.28:0/417930350 wait complete. 2024-09-10T10:51:31.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:31.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.266+0000 7f1a92400640 1 Processor -- start 2024-09-10T10:51:31.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.266+0000 7f1a92400640 1 -- start start 2024-09-10T10:51:31.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.267+0000 7f1a92400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a8c072de0 0x7f1a8c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:31.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.267+0000 7f1a92400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c146140 con 0x7f1a8c072320 2024-09-10T10:51:31.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.267+0000 7f1a92400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8c0cd740 con 0x7f1a8c072700 2024-09-10T10:51:31.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.267+0000 7f1a92400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8c0cda20 con 0x7f1a8c072de0 2024-09-10T10:51:31.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.268+0000 7f1a8b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1a8c072320 0x7f1a8c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53620/0 (socket says 172.21.15.28:53620) 2024-09-10T10:51:31.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.268+0000 7f1a8b400640 1 -- 172.21.15.28:0/3745203446 learned_addr learned my addr 172.21.15.28:0/3745203446 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.268+0000 7f1a82a00640 1 --2- 172.21.15.28:0/3745203446 >> v2:172.21.15.138:3300/0 conn(0x7f1a8c072de0 0x7f1a8c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.268+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781655098 0 0) 0x7f1a8c146140 con 0x7f1a8c072320 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.268+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a70003520 con 0x7f1a8c072320 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062402238 0 0) 0x7f1a8c0cd740 con 0x7f1a8c072700 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a8c146140 con 0x7f1a8c072700 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a82a00640 1 -- 172.21.15.28:0/3745203446 >> v1:172.21.15.138:6789/0 conn(0x7f1a8c072700 legacy=0x7f1a8c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a82a00640 1 -- 172.21.15.28:0/3745203446 >> v1:172.21.15.28:6789/0 conn(0x7f1a8c072320 legacy=0x7f1a8c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 336504086 0 0) 0x7f1a70003520 con 0x7f1a8c072320 2024-09-10T10:51:31.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.269+0000 7f1a82a00640 1 -- 172.21.15.28:0/3745203446 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0cdd00 con 0x7f1a8c072de0 2024-09-10T10:51:31.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a82a00640 1 --2- 172.21.15.28:0/3745203446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c15d1e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a740097a0 tx=0x7f1a7402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c44b925669bda8b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74037070 con 0x7f1a8c072de0 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a74032440 con 0x7f1a8c072de0 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a8aa00640 1 -- 172.21.15.28:0/3745203446 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a740327f0 con 0x7f1a8c072de0 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a92400640 1 -- 172.21.15.28:0/3745203446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 msgr2=0x7f1a8c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.270+0000 7f1a92400640 1 --2- 172.21.15.28:0/3745203446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c15d1e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1a740097a0 tx=0x7f1a7402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.271+0000 7f1a92400640 1 -- 172.21.15.28:0/3745203446 shutdown_connections 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.271+0000 7f1a92400640 1 --2- 172.21.15.28:0/3745203446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c15d1e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.271+0000 7f1a92400640 1 -- 172.21.15.28:0/3745203446 >> 172.21.15.28:0/3745203446 conn(0x7f1a8c067d70 msgr2=0x7f1a8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.271+0000 7f1a92400640 1 -- 172.21.15.28:0/3745203446 shutdown_connections 2024-09-10T10:51:31.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.271+0000 7f1a92400640 1 -- 172.21.15.28:0/3745203446 wait complete. 2024-09-10T10:51:31.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.272+0000 7f1a92400640 1 Processor -- start 2024-09-10T10:51:31.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.272+0000 7f1a92400640 1 -- start start 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 0x7f1a8c0c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c1e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c15dfb0 con 0x7f1a8c072700 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c15de30 con 0x7f1a8c072de0 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.273+0000 7f1a92400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8c15e130 con 0x7f1a8c072320 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:31.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41254/0 (socket says 172.21.15.28:41254) 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 -- 172.21.15.28:0/3126599082 learned_addr learned my addr 172.21.15.28:0/3126599082 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a8b400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 0x7f1a8c0c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a8be00640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c1e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 -- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 msgr2=0x7f1a8c0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 0x7f1a8c0c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 -- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 msgr2=0x7f1a8c1e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c1e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 -- 172.21.15.28:0/3126599082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c0c8990 con 0x7f1a8c072700 2024-09-10T10:51:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a8b400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 0x7f1a8c0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:31.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.274+0000 7f1a82a00640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1a7402a1e0 tx=0x7f1a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:31.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.275+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74041020 con 0x7f1a8c072700 2024-09-10T10:51:31.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.275+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a7403d620 con 0x7f1a8c072700 2024-09-10T10:51:31.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.275+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74037040 con 0x7f1a8c072700 2024-09-10T10:51:31.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.275+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c0c8c70 con 0x7f1a8c072700 2024-09-10T10:51:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.275+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c0c9230 con 0x7f1a8c072700 2024-09-10T10:51:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.276+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a8c0cd350 con 0x7f1a8c072700 2024-09-10T10:51:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.277+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a74044020 con 0x7f1a8c072700 2024-09-10T10:51:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.277+0000 7f1a83e00640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 0x7f1a6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.278+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f1a740ab7f0 con 0x7f1a8c072700 2024-09-10T10:51:31.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.278+0000 7f1a8b400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 0x7f1a6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:31.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.279+0000 7f1a8b400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 0x7f1a6806e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005880 tx=0x7f1a7c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:31.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.282+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a7404b070 con 0x7f1a8c072700 2024-09-10T10:51:31.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.504+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a8c0008d0 con 0x7f1a8c072700 2024-09-10T10:51:31.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.504+0000 7f1a83e00640 1 -- 172.21.15.28:0/3126599082 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) ==== 95+0+5224 (secure 0 0 0) 0x7f1a74079c60 con 0x7f1a8c072700 2024-09-10T10:51:31.506 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:31.506 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":128,"btime":"2024-09-10T10:51:30:674793+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:30.674779+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"c","rank":0,"incarnation":128,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/786875524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":786875524},{"type":"v1","addr":"172.21.15.28:6834","nonce":786875524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:31.507 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 128 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 msgr2=0x7f1a6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 0x7f1a6806e2f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c005880 tx=0x7f1a7c009040 comp rx=0 tx=0).stop 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 msgr2=0x7f1a8c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1a7402a1e0 tx=0x7f1a740046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 shutdown_connections 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a6806be30 0x7f1a6806e2f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a8c072de0 0x7f1a8c1e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1a8c072700 0x7f1a8c15bdc0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 --2- 172.21.15.28:0/3126599082 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a8c072320 0x7f1a8c0c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:31.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 >> 172.21.15.28:0/3126599082 conn(0x7f1a8c067d70 msgr2=0x7f1a8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:31.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 shutdown_connections 2024-09-10T10:51:31.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:31.507+0000 7f1a92400640 1 -- 172.21.15.28:0/3126599082 wait complete. 2024-09-10T10:51:31.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:51:30.674779+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'c', 'rank': 0, 'incarnation': 128, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6834/786875524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 786875524}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 786875524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:51:31.525 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7896:up:replay 2024-09-10T10:51:31.525 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:51:34.257 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:34.256+0000 7f7aef200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:36.929 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.094+0000 7fa5b9e00640 1 Processor -- start 2024-09-10T10:51:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.095+0000 7fa5b9e00640 1 -- start start 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa5b4072700 0x7fa5b415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b9e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b4146140 con 0x7fa5b4072320 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b9e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b40cd740 con 0x7fa5b4072de0 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b9e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b40cda20 con 0x7fa5b4072700 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa5b4072320 0x7fa5b40cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56524/0 (socket says 172.21.15.28:56524) 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b2a00640 1 -- 172.21.15.28:0/2079698852 learned_addr learned my addr 172.21.15.28:0/2079698852 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.096+0000 7fa5b3400640 1 --2- 172.21.15.28:0/2079698852 >> v2:172.21.15.28:3300/0 conn(0x7fa5b4072700 0x7fa5b415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.097+0000 7fa5b2000640 1 -- 172.21.15.28:0/2079698852 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 125176122 0 0) 0x7fa5b4146140 con 0x7fa5b4072320 2024-09-10T10:51:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.097+0000 7fa5b2000640 1 -- 172.21.15.28:0/2079698852 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa598003520 con 0x7fa5b4072320 2024-09-10T10:51:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.097+0000 7fa5b3400640 1 -- 172.21.15.28:0/2079698852 >> v1:172.21.15.138:6789/0 conn(0x7fa5b4072de0 legacy=0x7fa5b415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.097+0000 7fa5b3400640 1 -- 172.21.15.28:0/2079698852 >> v1:172.21.15.28:6789/0 conn(0x7fa5b4072320 legacy=0x7fa5b40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.097+0000 7fa5b3400640 1 -- 172.21.15.28:0/2079698852 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b40cdd00 con 0x7fa5b4072700 2024-09-10T10:51:37.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.098+0000 7fa5b3400640 1 --2- 172.21.15.28:0/2079698852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b415d8f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009d90 tx=0x7fa5ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a04a8ad07d75b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.098+0000 7fa5b2000640 1 -- 172.21.15.28:0/2079698852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac007740 con 0x7fa5b4072700 2024-09-10T10:51:37.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.098+0000 7fa5b2000640 1 -- 172.21.15.28:0/2079698852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ac0078e0 con 0x7fa5b4072700 2024-09-10T10:51:37.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.098+0000 7fa5b2000640 1 -- 172.21.15.28:0/2079698852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac007c90 con 0x7fa5b4072700 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 -- 172.21.15.28:0/2079698852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 msgr2=0x7fa5b415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/2079698852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b415d8f0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009d90 tx=0x7fa5ac02a660 comp rx=0 tx=0).stop 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 -- 172.21.15.28:0/2079698852 shutdown_connections 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/2079698852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b415d8f0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 -- 172.21.15.28:0/2079698852 >> 172.21.15.28:0/2079698852 conn(0x7fa5b4067d70 msgr2=0x7fa5b406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 -- 172.21.15.28:0/2079698852 shutdown_connections 2024-09-10T10:51:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.099+0000 7fa5b9e00640 1 -- 172.21.15.28:0/2079698852 wait complete. 2024-09-10T10:51:37.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.100+0000 7fa5b9e00640 1 Processor -- start 2024-09-10T10:51:37.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.100+0000 7fa5b9e00640 1 -- start start 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b4072320 0x7fa5b41e7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b4072de0 0x7fa5b40c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b415dfb0 con 0x7fa5b4072700 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b415de30 con 0x7fa5b4072de0 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.101+0000 7fa5b9e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b415e130 con 0x7fa5b4072320 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47948/0 (socket says 172.21.15.28:47948) 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 -- 172.21.15.28:0/1718031780 learned_addr learned my addr 172.21.15.28:0/1718031780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5b3400640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b4072de0 0x7fa5b40c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5b2a00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b4072320 0x7fa5b41e7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 -- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b4072320 msgr2=0x7fa5b41e7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b4072320 0x7fa5b41e7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 -- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b4072de0 msgr2=0x7fa5b40c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b4072de0 0x7fa5b40c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.102+0000 7fa5aa000640 1 -- 172.21.15.28:0/1718031780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b41e7fb0 con 0x7fa5b4072700 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c430 tx=0x7fa59c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c017070 con 0x7fa5b4072700 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c0052d0 con 0x7fa5b4072700 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b40cb1e0 con 0x7fa5b4072700 2024-09-10T10:51:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c003830 con 0x7fa5b4072700 2024-09-10T10:51:37.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.103+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b40cb7a0 con 0x7fa5b4072700 2024-09-10T10:51:37.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.104+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b40cce60 con 0x7fa5b4072700 2024-09-10T10:51:37.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.105+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa59c01a070 con 0x7fa5b4072700 2024-09-10T10:51:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.106+0000 7fa5abe00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.106+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7fa59c08a650 con 0x7fa5b4072700 2024-09-10T10:51:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.106+0000 7fa5b2a00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.107+0000 7fa5b2a00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 0x7fa59006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40011e0 tx=0x7fa5a4007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.110+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa59c058470 con 0x7fa5b4072700 2024-09-10T10:51:37.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.330+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5b41e83a0 con 0x7fa5b4072700 2024-09-10T10:51:37.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.330+0000 7fa5abe00640 1 -- 172.21.15.28:0/1718031780 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) ==== 95+0+5231 (secure 0 0 0) 0x7fa59c05c770 con 0x7fa5b4072700 2024-09-10T10:51:37.331 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:37.332 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":131,"btime":"2024-09-10T10:51:33:685164+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:33.685154+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":106,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7896},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7896":{"gid":7896,"name":"c","rank":0,"incarnation":128,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6834/786875524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":786875524},{"type":"v1","addr":"172.21.15.28:6834","nonce":786875524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7896,"qdb_cluster":[7896]},"id":6}]} 2024-09-10T10:51:37.332 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 131 2024-09-10T10:51:37.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 msgr2=0x7fa59006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 0x7fa59006e2f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40011e0 tx=0x7fa5a4007d30 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 msgr2=0x7fa5b40c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c430 tx=0x7fa59c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 shutdown_connections 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa59006be30 0x7fa59006e2f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5b4072de0 0x7fa5b40c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5b4072700 0x7fa5b40c8450 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 --2- 172.21.15.28:0/1718031780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5b4072320 0x7fa5b41e7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 >> 172.21.15.28:0/1718031780 conn(0x7fa5b4067d70 msgr2=0x7fa5b40c5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 shutdown_connections 2024-09-10T10:51:37.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.332+0000 7fa5b9e00640 1 -- 172.21.15.28:0/1718031780 wait complete. 2024-09-10T10:51:37.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:51:33.685154+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 106, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7896}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7896': {'gid': 7896, 'name': 'c', 'rank': 0, 'incarnation': 128, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6834/786875524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 786875524}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 786875524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7896, 'qdb_cluster': [7896]} 2024-09-10T10:51:37.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:51:37.350 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:51:37.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.506+0000 7f5d30400640 1 Processor -- start 2024-09-10T10:51:37.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.506+0000 7f5d30400640 1 -- start start 2024-09-10T10:51:37.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.507+0000 7f5d30400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.507+0000 7f5d30400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d280d0bb0 0x7f5d280ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.507+0000 7f5d30400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d2806cb30 con 0x7f5d28070dd0 2024-09-10T10:51:37.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.507+0000 7f5d30400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5d2806bfb0 con 0x7f5d280d0bb0 2024-09-10T10:51:37.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.507+0000 7f5d30400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d2806c290 con 0x7f5d28147360 2024-09-10T10:51:37.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d280d0bb0 0x7f5d280ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5d280d0bb0 0x7f5d280ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47958/0 (socket says 172.21.15.28:47958) 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 -- 172.21.15.28:0/574671217 learned_addr learned my addr 172.21.15.28:0/574671217 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2da00640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 -- 172.21.15.28:0/574671217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 msgr2=0x7f5d28070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28070890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 -- 172.21.15.28:0/574671217 >> v1:172.21.15.138:6789/0 conn(0x7f5d28070dd0 legacy=0x7f5d280d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.508+0000 7f5d2e400640 1 -- 172.21.15.28:0/574671217 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2806c570 con 0x7f5d280d0bb0 2024-09-10T10:51:37.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.509+0000 7f5d2e400640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d280d0bb0 0x7f5d280ae550 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009e50 tx=0x7f5d2002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13de4232fa1f91c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d27e00640 1 -- 172.21.15.28:0/574671217 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20036070 con 0x7f5d280d0bb0 2024-09-10T10:51:37.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d27e00640 1 -- 172.21.15.28:0/574671217 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d20031600 con 0x7f5d280d0bb0 2024-09-10T10:51:37.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d27e00640 1 -- 172.21.15.28:0/574671217 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d200319d0 con 0x7f5d280d0bb0 2024-09-10T10:51:37.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 -- 172.21.15.28:0/574671217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d280d0bb0 msgr2=0x7f5d280ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d280d0bb0 0x7f5d280ae550 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009e50 tx=0x7f5d2002a680 comp rx=0 tx=0).stop 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 -- 172.21.15.28:0/574671217 shutdown_connections 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d280d0bb0 0x7f5d280ae550 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 --2- 172.21.15.28:0/574671217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 -- 172.21.15.28:0/574671217 >> 172.21.15.28:0/574671217 conn(0x7f5d280b2f40 msgr2=0x7f5d280b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 -- 172.21.15.28:0/574671217 shutdown_connections 2024-09-10T10:51:37.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.510+0000 7f5d30400640 1 -- 172.21.15.28:0/574671217 wait complete. 2024-09-10T10:51:37.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.512+0000 7f5d30400640 1 Processor -- start 2024-09-10T10:51:37.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.512+0000 7f5d30400640 1 -- start start 2024-09-10T10:51:37.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 0x7f5d28143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28140620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2806bd90 con 0x7f5d28070dd0 2024-09-10T10:51:37.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2806bc10 con 0x7f5d280d0bb0 2024-09-10T10:51:37.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d30400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d280aea90 con 0x7f5d28147360 2024-09-10T10:51:37.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47960/0 (socket says 172.21.15.28:47960) 2024-09-10T10:51:37.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.513+0000 7f5d2da00640 1 -- 172.21.15.28:0/4234856134 learned_addr learned my addr 172.21.15.28:0/4234856134 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2e400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28140620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2d000640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 0x7f5d28143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 -- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 msgr2=0x7f5d28140620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28140620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 -- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 msgr2=0x7f5d28143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 0x7f5d28143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 -- 172.21.15.28:0/4234856134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d280b5d00 con 0x7f5d28070dd0 2024-09-10T10:51:37.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2e400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2d000640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 0x7f5d28143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e8a0 tx=0x7f5d1c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d25600640 1 -- 172.21.15.28:0/4234856134 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017070 con 0x7f5d28070dd0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d25600640 1 -- 172.21.15.28:0/4234856134 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00c8e0 con 0x7f5d28070dd0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.514+0000 7f5d25600640 1 -- 172.21.15.28:0/4234856134 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00ccd0 con 0x7f5d28070dd0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.515+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d280b5fe0 con 0x7f5d28070dd0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.515+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d280b65a0 con 0x7f5d28070dd0 2024-09-10T10:51:37.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.517+0000 7f5d25600640 1 -- 172.21.15.28:0/4234856134 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d28070dd0 2024-09-10T10:51:37.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.518+0000 7f5d25600640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 0x7f5d0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.518+0000 7f5d25600640 1 -- 172.21.15.28:0/4234856134 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f5d1c08b5a0 con 0x7f5d28070dd0 2024-09-10T10:51:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.518+0000 7f5d2d000640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 0x7f5d0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.518+0000 7f5d33401740 1 Processor -- start 2024-09-10T10:51:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.518+0000 7f5d33401740 1 -- start start 2024-09-10T10:51:37.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d2d000640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 0x7f5d0806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5d28075930 tx=0x7f5d14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d8961624b0 0x55d8961650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d89615f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d8960c33b0 con 0x55d896164cf0 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8960c8410 con 0x55d896166480 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.519+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d896070620 con 0x55d8961624b0 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47974/0 (socket says 172.21.15.28:47974) 2024-09-10T10:51:37.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 -- 172.21.15.28:0/4218819754 learned_addr learned my addr 172.21.15.28:0/4218819754 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2d000640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d8961624b0 0x55d8961650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2e400640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d89615f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 -- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d8961624b0 msgr2=0x55d8961650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d8961624b0 0x55d8961650d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 -- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 msgr2=0x55d89615f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d89615f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 -- 172.21.15.28:0/4218819754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d895ddb2d0 con 0x55d896164cf0 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2e400640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d89615f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:37.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d2da00640 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c055760 tx=0x7f5d1c059ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805cf1392a5034b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d1a000640 1 -- 172.21.15.28:0/4218819754 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c021040 con 0x55d896164cf0 2024-09-10T10:51:37.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d1a000640 1 -- 172.21.15.28:0/4218819754 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c061ee0 con 0x55d896164cf0 2024-09-10T10:51:37.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.520+0000 7f5d1a000640 1 -- 172.21.15.28:0/4218819754 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c055970 con 0x55d896164cf0 2024-09-10T10:51:37.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 -- 172.21.15.28:0/4218819754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 msgr2=0x55d8961aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c055760 tx=0x7f5d1c059ac0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 -- 172.21.15.28:0/4218819754 shutdown_connections 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d89615f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d8961624b0 0x55d8961650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 --2- 172.21.15.28:0/4218819754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d8961aa490 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 -- 172.21.15.28:0/4218819754 >> 172.21.15.28:0/4218819754 conn(0x55d896161740 msgr2=0x55d8961a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 -- 172.21.15.28:0/4218819754 shutdown_connections 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.521+0000 7f5d33401740 1 -- 172.21.15.28:0/4218819754 wait complete. 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.522+0000 7f5d33401740 1 -- start start 2024-09-10T10:51:37.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.522+0000 7f5d33401740 1 Processor -- start 2024-09-10T10:51:37.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d89618c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89618efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d896083780 con 0x55d896166480 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8960c43d0 con 0x55d896164cf0 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d896160be0 con 0x55d89618cc10 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d33401740 10 client.0 _ll_register_callbacks cb 0x55d8960c15d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d2d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d2d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47986/0 (socket says 172.21.15.28:47986) 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d2d000640 1 -- 172.21.15.28:0/1779546353 learned_addr learned my addr 172.21.15.28:0/1779546353 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d2e400640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89618efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.523+0000 7f5d2da00640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d89618c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 -- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 msgr2=0x55d89618efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89618efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 -- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 msgr2=0x55d89618c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d89618c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 -- 172.21.15.28:0/1779546353 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d895ea56b0 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d2d000640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400dce0 tx=0x7f5d14000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83731959e541fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14020420 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f5d140205c0 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.524+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c061ee0 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) ==== 24+0+0 (secure 0 0 0) 0x7f5d1402d050 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 10 client.7959 fetch_fsmap learned FSMap version 131 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) -- 0x55d89618dc90 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c061ee0 con 0x55d896166480 2024-09-10T10:51:37.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 131) ==== 3502+0+0 (secure 0 0 0) 0x7f5d14030020 con 0x55d896166480 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 10 client.7959 fetch_fsmap finished waiting for FSMap version 131 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 10 client.7959 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 20 client.7959 populate_metadata read hostname 'smithi028' 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d8961bb1a0 0x55d896186e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 4 client.7959 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 --> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8960d5b40 con 0x55d8961bb1a0 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.525+0000 7f5d2da00640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d8961bb1a0 0x55d896186e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.526+0000 7f5d2da00640 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d8961bb1a0 0x55d896186e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9030c6c8a64d54 server_cookie=38fa1cc5f50660fa in_seq=0 out_seq=0 2024-09-10T10:51:37.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.526+0000 7f5d0ea00640 10 client.7959 ms_handle_connect on v2:172.21.15.28:6832/786875524 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d0ea00640 1 -- 172.21.15.28:0/1779546353 <== mds.0 v2:172.21.15.28:6832/786875524 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8960d5b40 con 0x55d8961bb1a0 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d0ea00640 10 client.7959 handle_command_reply: tid=0 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 client.7959 shutdown 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 msgr2=0x55d89618eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400dce0 tx=0x7f5d14000bc0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 shutdown_connections 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d8961bb1a0 0x55d896186e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89618efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896166480 0x55d89618eaa0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 --2- 172.21.15.28:0/1779546353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d89618c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d0ea00640 0 client.7959 ms_handle_reset on v2:172.21.15.28:6832/786875524 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 >> 172.21.15.28:0/1779546353 conn(0x55d8961624b0 msgr2=0x55d8961a8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 shutdown_connections 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 1 -- 172.21.15.28:0/1779546353 wait complete. 2024-09-10T10:51:37.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.527+0000 7f5d33401740 20 client.7959 trim_cache size 0 max 16384 2024-09-10T10:51:37.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.546+0000 7f5d33401740 1 Processor -- start 2024-09-10T10:51:37.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.546+0000 7f5d33401740 1 -- start start 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d8961c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89617cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5d28200c50 con 0x55d896164cf0 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d281fff30 con 0x55d896166480 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d28200580 con 0x55d89618cc10 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47990/0 (socket says 172.21.15.28:47990) 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d2da00640 1 -- 172.21.15.28:0/3953703861 learned_addr learned my addr 172.21.15.28:0/3953703861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.547+0000 7f5d2d000640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d8961c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2e400640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89617cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 -- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 msgr2=0x55d89617cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89617cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 -- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 msgr2=0x55d8961c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d8961c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 -- 172.21.15.28:0/3953703861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89618ca40 con 0x55d896164cf0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2d000640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d8961c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d2da00640 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c055760 tx=0x7f5d1c059200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc60c9fdbb5acff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d19600640 1 -- 172.21.15.28:0/3953703861 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c021040 con 0x55d896164cf0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d19600640 1 -- 172.21.15.28:0/3953703861 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d1c057420 con 0x55d896164cf0 2024-09-10T10:51:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.548+0000 7f5d19600640 1 -- 172.21.15.28:0/3953703861 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c061ee0 con 0x55d896164cf0 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- 172.21.15.28:0/3953703861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 msgr2=0x55d896183d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c055760 tx=0x7f5d1c059200 comp rx=0 tx=0).stop 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- 172.21.15.28:0/3953703861 shutdown_connections 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d89618cc10 0x55d89617cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896166480 0x55d8961c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 --2- 172.21.15.28:0/3953703861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d896164cf0 0x55d896183d90 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- 172.21.15.28:0/3953703861 >> 172.21.15.28:0/3953703861 conn(0x55d8961624b0 msgr2=0x55d8961a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- 172.21.15.28:0/3953703861 shutdown_connections 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- 172.21.15.28:0/3953703861 wait complete. 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.549+0000 7f5d33401740 1 -- start start 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.550+0000 7f5d33401740 1 Processor -- start 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d8961a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d896166480 0x55d8961ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 0x55d89620d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d8961bb020 con 0x55d89618cc10 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d8961bb910 con 0x55d896164cf0 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d8961bb5b0 con 0x55d896166480 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d33401740 10 client.0 _ll_register_callbacks cb 0x7f5d1400d930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 0x55d89620d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d8961a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 0x55d89620d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48000/0 (socket says 172.21.15.28:48000) 2024-09-10T10:51:37.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d896166480 0x55d8961ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2e400640 1 -- 172.21.15.28:0/903171563 learned_addr learned my addr 172.21.15.28:0/903171563 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 -- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d896166480 msgr2=0x55d8961ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d896166480 0x55d8961ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 -- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 msgr2=0x55d89620d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 0x55d89620d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.551+0000 7f5d2da00640 1 -- 172.21.15.28:0/903171563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8961c2e70 con 0x55d896164cf0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d2da00640 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d8961a9e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c059000 tx=0x7f5d1c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17f759bf006d0872 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d0ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c076060 con 0x55d896164cf0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f5d1c071170 con 0x55d896164cf0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d8960cfe90 con 0x55d896164cf0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c07d050 con 0x55d896164cf0 2024-09-10T10:51:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d33401740 10 client.7396 fetch_fsmap learned FSMap version 131 2024-09-10T10:51:37.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.552+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) -- 0x55d896228190 con 0x55d896164cf0 2024-09-10T10:51:37.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d8960cfe90 con 0x55d896164cf0 2024-09-10T10:51:37.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 131) ==== 3502+0+0 (secure 0 0 0) 0x7f5d1c08b070 con 0x55d896164cf0 2024-09-10T10:51:37.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 10 client.7396 fetch_fsmap finished waiting for FSMap version 131 2024-09-10T10:51:37.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 10 client.7396 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:51:37.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 20 client.7396 populate_metadata read hostname 'smithi028' 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d89618e8b0 0x55d89618ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 4 client.7396 mds_command: new command op to 7896 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 --> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] -- command(tid 0: {"prefix": "status"}) -- 0x55d89604ca40 con 0x55d89618e8b0 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.553+0000 7f5d2d000640 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d89618e8b0 0x55d89618ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d0ea00640 10 client.7396 ms_handle_connect on v2:172.21.15.28:6832/786875524 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d2d000640 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d89618e8b0 0x55d89618ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c43418695f964ab server_cookie=74a670c6e3d42e6c in_seq=0 out_seq=0 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d0ea00640 1 -- 172.21.15.28:0/903171563 <== mds.0 v2:172.21.15.28:6832/786875524 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55d89604ca40 con 0x55d89618e8b0 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d0ea00640 10 client.7396 handle_command_reply: tid=0 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 client.7396 shutdown 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 msgr2=0x55d8961a9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d8961a9e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c059000 tx=0x7f5d1c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 shutdown_connections 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d89618cc10 0x55d89620d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.28:6832/786875524,v1:172.21.15.28:6834/786875524] conn(0x55d89618e8b0 0x55d89618ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d896166480 0x55d8961ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 --2- 172.21.15.28:0/903171563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d896164cf0 0x55d8961a9e00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d0ea00640 0 client.7396 ms_handle_reset on v2:172.21.15.28:6832/786875524 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 >> 172.21.15.28:0/903171563 conn(0x55d8961624b0 msgr2=0x55d89617cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 shutdown_connections 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 1 -- 172.21.15.28:0/903171563 wait complete. 2024-09-10T10:51:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.554+0000 7f5d33401740 20 client.7396 trim_cache size 0 max 16384 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 7896, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8775481159999998, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 131, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 107, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 107, 2024-09-10T10:51:37.560 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 9.2574291679999998 2024-09-10T10:51:37.561 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:51:37.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 msgr2=0x7f5d0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 0x7f5d0806e2f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5d28075930 tx=0x7f5d14009040 comp rx=0 tx=0).stop 2024-09-10T10:51:37.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 msgr2=0x7f5d280b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00e8a0 tx=0x7f5d1c00ed60 comp rx=0 tx=0).stop 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 shutdown_connections 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5d0806be30 0x7f5d0806e2f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d28147360 0x7f5d28140620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d280d0bb0 0x7f5d28143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 --2- 172.21.15.28:0/4234856134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5d28070dd0 0x7f5d280b5620 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 >> 172.21.15.28:0/4234856134 conn(0x7f5d280b2f40 msgr2=0x7f5d280afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 shutdown_connections 2024-09-10T10:51:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.555+0000 7f5d30400640 1 -- 172.21.15.28:0/4234856134 wait complete. 2024-09-10T10:51:37.572 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:51:37.572 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 500.00000000', proceeding to ls 2024-09-10T10:51:37.572 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:51:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.728+0000 7f0cad400640 1 Processor -- start 2024-09-10T10:51:37.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.728+0000 7f0cad400640 1 -- start start 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.729+0000 7f0cad400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.729+0000 7f0cad400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0cad400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ca80d2700 con 0x7f0ca815e760 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0cad400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0ca80cedb0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0cad400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80cf090 con 0x7f0ca80cfcc0 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48010/0 (socket says 172.21.15.28:48010) 2024-09-10T10:51:37.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 -- 172.21.15.28:0/3740190218 learned_addr learned my addr 172.21.15.28:0/3740190218 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0c9fe00640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 -- 172.21.15.28:0/3740190218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 msgr2=0x7f0ca806cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca806cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 -- 172.21.15.28:0/3740190218 >> v1:172.21.15.138:6789/0 conn(0x7f0ca815e760 legacy=0x7f0ca80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0ca7200640 1 -- 172.21.15.28:0/3740190218 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80b3c80 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.730+0000 7f0c9fe00640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:37.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.731+0000 7f0ca7200640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009e50 tx=0x7f0ca002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81880d020ffe18fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0ca5e00640 1 -- 172.21.15.28:0/3740190218 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca0036070 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0ca5e00640 1 -- 172.21.15.28:0/3740190218 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca0031600 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0ca5e00640 1 -- 172.21.15.28:0/3740190218 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00319d0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 -- 172.21.15.28:0/3740190218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 msgr2=0x7f0ca80d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009e50 tx=0x7f0ca002a710 comp rx=0 tx=0).stop 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 -- 172.21.15.28:0/3740190218 shutdown_connections 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80d2ea0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 --2- 172.21.15.28:0/3740190218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca806cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 -- 172.21.15.28:0/3740190218 >> 172.21.15.28:0/3740190218 conn(0x7f0ca808bfc0 msgr2=0x7f0ca80d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 -- 172.21.15.28:0/3740190218 shutdown_connections 2024-09-10T10:51:37.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.732+0000 7f0cad400640 1 -- 172.21.15.28:0/3740190218 wait complete. 2024-09-10T10:51:37.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.734+0000 7f0cad400640 1 Processor -- start 2024-09-10T10:51:37.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.734+0000 7f0cad400640 1 -- start start 2024-09-10T10:51:37.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca80b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca815e760 0x7f0ca80b9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80d33e0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca80b38a0 con 0x7f0ca815e760 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0cad400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca80d3560 con 0x7f0ca80cfcc0 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0c9fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0c9fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48022/0 (socket says 172.21.15.28:48022) 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.735+0000 7f0c9fe00640 1 -- 172.21.15.28:0/2666266824 learned_addr learned my addr 172.21.15.28:0/2666266824 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 -- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 msgr2=0x7f0ca80b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0ca6800640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca80b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0ca7200640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca815e760 0x7f0ca80b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca80b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 -- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca815e760 msgr2=0x7f0ca80b9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca815e760 0x7f0ca80b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 -- 172.21.15.28:0/2666266824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca80b7e00 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0ca6800640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca80b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9fe00640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c7f0 tx=0x7f0c9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90005b90 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c90005d30 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca80b80e0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90007a50 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.736+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca80b86a0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.737+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca80b8a80 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.738+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0c90007bf0 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.739+0000 7f0c9ea00640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 0x7f0c8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:37.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.739+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f0c9008b460 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.740+0000 7f0ca6800640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 0x7f0c8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:37.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.740+0000 7f0ca6800640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 0x7f0c8406e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0ca806d4c0 tx=0x7f0c98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:37.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.743+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ca80b8a80 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.967+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0ca80b6d20 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.968+0000 7f0c9ea00640 1 -- 172.21.15.28:0/2666266824 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) ==== 53+0+73 (secure 0 0 0) 0x7f0c90015020 con 0x7f0ca80d4fd0 2024-09-10T10:51:37.969 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 msgr2=0x7f0c8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 0x7f0c8406e2f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f0ca806d4c0 tx=0x7f0c98009040 comp rx=0 tx=0).stop 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 msgr2=0x7f0ca80b8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000c7f0 tx=0x7f0c9000ccb0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 shutdown_connections 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0c8406be30 0x7f0c8406e2f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ca815e760 0x7f0ca80b9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ca80d4fd0 0x7f0ca80b8ec0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 --2- 172.21.15.28:0/2666266824 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0ca80cfcc0 0x7f0ca80b7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:37.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 >> 172.21.15.28:0/2666266824 conn(0x7f0ca808bfc0 msgr2=0x7f0ca80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:37.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 shutdown_connections 2024-09-10T10:51:37.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:37.970+0000 7f0cad400640 1 -- 172.21.15.28:0/2666266824 wait complete. 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:51:37.987 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:51:37.988 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:37.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:37.988 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:51:38.024 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:51:38.024 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:51:38.024 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:51:38.024 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.024 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:51:38.025 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42207sec preferred_lft 42207sec 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:51:38.026 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:51:38.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.027 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:51:38.118 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:51:38.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.121 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:51:38.156 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:51:38.157 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:51:38.157 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:51:38.157 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:51:38.157 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:51:38.157 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:51:38.157 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:51:38.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:51:38.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:51:38.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:51:38.158 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:51:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:51:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:51:38.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.355 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:51:38.390 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:51:38.390 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:51:38.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:38.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.391 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:51:38.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:51:38.487 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:38.522 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:51:38.522 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:51:38.522 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:38.597 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:51:38.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.597 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:38.725 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:38.725 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:38.725 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:38.741 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:38.741 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:38.741 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:38.741 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:51:38.741 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:51:38.763 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:51:38.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:51:38.767 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:51:38.767 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:51:38.767 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:51:38.767 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:51:38.767 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:51:38.769 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:51:38.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.770 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:51:38.825 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:51:38.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:51:38.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:51:38.915 INFO:teuthology.orchestra.run.smithi028.stdout:client7402 2024-09-10T10:51:38.915 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:38.915 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:38.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:38.950 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:51:38.995 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:51:39.028 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:51:39.028 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:51:39.028 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:39.028 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:51:39.028 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:51:39.029 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 500.00000000' did not caused DAMAGED state 2024-09-10T10:51:39.029 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds_snaptable' 2024-09-10T10:51:39.029 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:39.029 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:39.105 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:39.105 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:39.181 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:51:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.378+0000 7f51bf200640 1 Processor -- start 2024-09-10T10:51:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.378+0000 7f51bf200640 1 -- start start 2024-09-10T10:51:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51bf200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51c00d0dd0 0x7f51c00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51bf200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c00cf000 con 0x7f51c00d1f90 2024-09-10T10:51:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51bf200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c00d16f0 con 0x7f51c015e660 2024-09-10T10:51:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51bf200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f51c006d3e0 con 0x7f51c00d0dd0 2024-09-10T10:51:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51be800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f51c00d1f90 0x7f51c00cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56538/0 (socket says 172.21.15.28:56538) 2024-09-10T10:51:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51be800640 1 -- 172.21.15.28:0/1936455580 learned_addr learned my addr 172.21.15.28:0/1936455580 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.379+0000 7f51bde00640 1 --2- 172.21.15.28:0/1936455580 >> v2:172.21.15.138:3300/0 conn(0x7f51c00d0dd0 0x7f51c00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288326279 0 0) 0x7f51c00cf000 con 0x7f51c00d1f90 2024-09-10T10:51:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51a4003520 con 0x7f51c00d1f90 2024-09-10T10:51:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bde00640 1 -- 172.21.15.28:0/1936455580 >> v1:172.21.15.138:6789/0 conn(0x7f51c015e660 legacy=0x7f51c006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038592255 0 0) 0x7f51c00d16f0 con 0x7f51c015e660 2024-09-10T10:51:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bde00640 1 -- 172.21.15.28:0/1936455580 >> v1:172.21.15.28:6789/0 conn(0x7f51c00d1f90 legacy=0x7f51c00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.380+0000 7f51bde00640 1 -- 172.21.15.28:0/1936455580 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c00d29a0 con 0x7f51c00d0dd0 2024-09-10T10:51:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.381+0000 7f51bde00640 1 --2- 172.21.15.28:0/1936455580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d0dd0 0x7f51c00d11b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fba2b873017b056 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.381+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b0037070 con 0x7f51c00d0dd0 2024-09-10T10:51:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.381+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b0032440 con 0x7f51c00d0dd0 2024-09-10T10:51:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.381+0000 7f51bd400640 1 -- 172.21.15.28:0/1936455580 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b00327f0 con 0x7f51c00d0dd0 2024-09-10T10:51:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 -- 172.21.15.28:0/1936455580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d0dd0 msgr2=0x7f51c00d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 --2- 172.21.15.28:0/1936455580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d0dd0 0x7f51c00d11b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f51b00097a0 tx=0x7f51b002a230 comp rx=0 tx=0).stop 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 -- 172.21.15.28:0/1936455580 shutdown_connections 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 --2- 172.21.15.28:0/1936455580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d0dd0 0x7f51c00d11b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 -- 172.21.15.28:0/1936455580 >> 172.21.15.28:0/1936455580 conn(0x7f51c008bfc0 msgr2=0x7f51c00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 -- 172.21.15.28:0/1936455580 shutdown_connections 2024-09-10T10:51:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.382+0000 7f51bf200640 1 -- 172.21.15.28:0/1936455580 wait complete. 2024-09-10T10:51:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.384+0000 7f51bf200640 1 Processor -- start 2024-09-10T10:51:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.384+0000 7f51bf200640 1 -- start start 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c00d0dd0 0x7f51c00c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d1f90 0x7f51c01f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00cf330 con 0x7f51c015e660 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00d2f40 con 0x7f51c00d1f90 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bf200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51c00d3220 con 0x7f51c00d0dd0 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48028/0 (socket says 172.21.15.28:48028) 2024-09-10T10:51:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 -- 172.21.15.28:0/2476118843 learned_addr learned my addr 172.21.15.28:0/2476118843 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51b7e00640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d1f90 0x7f51c01f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51bde00640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c00d0dd0 0x7f51c00c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 -- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c00d0dd0 msgr2=0x7f51c00c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c00d0dd0 0x7f51c00c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 -- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d1f90 msgr2=0x7f51c01f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d1f90 0x7f51c01f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.385+0000 7f51be800640 1 -- 172.21.15.28:0/2476118843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c014e400 con 0x7f51c015e660 2024-09-10T10:51:39.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51be800640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f51b800cb40 tx=0x7f51b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8017070 con 0x7f51c015e660 2024-09-10T10:51:39.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51b800aca0 con 0x7f51c015e660 2024-09-10T10:51:39.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8012620 con 0x7f51c015e660 2024-09-10T10:51:39.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c014e6e0 con 0x7f51c015e660 2024-09-10T10:51:39.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.386+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c014eca0 con 0x7f51c015e660 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.387+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c00d2420 con 0x7f51c015e660 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.388+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f51b801a070 con 0x7f51c015e660 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.389+0000 7f51b6000640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.389+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7f51b808b100 con 0x7f51c015e660 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.390+0000 7f51bde00640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.391+0000 7f51bde00640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 0x7f519c06e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f51b0034000 tx=0x7f51b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.393+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51b8058fa0 con 0x7f51c015e660 2024-09-10T10:51:39.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.589+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f51c00d11b0 con 0x7f51c015e660 2024-09-10T10:51:39.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.589+0000 7f51b6000640 1 -- 172.21.15.28:0/2476118843 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v107) ==== 81+0+852 (secure 0 0 0) 0x7f51b805d2a0 con 0x7f51c015e660 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:51:39.591 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:51:39.592 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:51:39.592 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:51:39.592 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:51:39.592 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:listed 14 entries 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.591+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 msgr2=0x7f519c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.591+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 0x7f519c06e2f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f51b0034000 tx=0x7f51b002e000 comp rx=0 tx=0).stop 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.591+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 msgr2=0x7f51c00b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.591+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f51b800cb40 tx=0x7f51b800a060 comp rx=0 tx=0).stop 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 shutdown_connections 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f519c06be30 0x7f519c06e2f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51c015e660 0x7f51c00b1b20 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c00d1f90 0x7f51c01f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 --2- 172.21.15.28:0/2476118843 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c00d0dd0 0x7f51c00c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 >> 172.21.15.28:0/2476118843 conn(0x7f51c008bfc0 msgr2=0x7f51c00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:39.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 shutdown_connections 2024-09-10T10:51:39.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.592+0000 7f51bf200640 1 -- 172.21.15.28:0/2476118843 wait complete. 2024-09-10T10:51:39.608 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:51:39.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:39.608 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:51:39.688 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:39.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:39.688 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:39.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:39.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.921+0000 7fa89ee00640 1 Processor -- start 2024-09-10T10:51:39.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.921+0000 7fa89ee00640 1 -- start start 2024-09-10T10:51:39.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.922+0000 7fa89ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa89806f870 0x7fa898162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.922+0000 7fa89ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89815eb60 con 0x7fa8980b0180 2024-09-10T10:51:39.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.922+0000 7fa89ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89815ee40 con 0x7fa8980b69d0 2024-09-10T10:51:39.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.922+0000 7fa89ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa8980b6db0 con 0x7fa89806f870 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa897400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa8980b0180 0x7fa898154d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56540/0 (socket says 172.21.15.28:56540) 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa897400640 1 -- 172.21.15.28:0/2084464155 learned_addr learned my addr 172.21.15.28:0/2084464155 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa89ce00640 1 --2- 172.21.15.28:0/2084464155 >> v2:172.21.15.28:3300/0 conn(0x7fa89806f870 0x7fa898162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa89ce00640 1 -- 172.21.15.28:0/2084464155 >> v1:172.21.15.138:6789/0 conn(0x7fa8980b69d0 legacy=0x7fa8980afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa89ce00640 1 -- 172.21.15.28:0/2084464155 >> v1:172.21.15.28:6789/0 conn(0x7fa8980b0180 legacy=0x7fa898154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.923+0000 7fa89ce00640 1 -- 172.21.15.28:0/2084464155 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89815f120 con 0x7fa89806f870 2024-09-10T10:51:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.924+0000 7fa89ce00640 1 --2- 172.21.15.28:0/2084464155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa89806f870 0x7fa898162a50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa890009dc0 tx=0x7fa89002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec0e7403a1a5259 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.924+0000 7fa896a00640 1 -- 172.21.15.28:0/2084464155 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890031600 con 0x7fa89806f870 2024-09-10T10:51:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.924+0000 7fa896a00640 1 -- 172.21.15.28:0/2084464155 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8900317a0 con 0x7fa89806f870 2024-09-10T10:51:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.924+0000 7fa896a00640 1 -- 172.21.15.28:0/2084464155 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890031b70 con 0x7fa89806f870 2024-09-10T10:51:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 -- 172.21.15.28:0/2084464155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa89806f870 msgr2=0x7fa898162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 --2- 172.21.15.28:0/2084464155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa89806f870 0x7fa898162a50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa890009dc0 tx=0x7fa89002a610 comp rx=0 tx=0).stop 2024-09-10T10:51:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 -- 172.21.15.28:0/2084464155 shutdown_connections 2024-09-10T10:51:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 --2- 172.21.15.28:0/2084464155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa89806f870 0x7fa898162a50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 -- 172.21.15.28:0/2084464155 >> 172.21.15.28:0/2084464155 conn(0x7fa89806bc10 msgr2=0x7fa89806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 -- 172.21.15.28:0/2084464155 shutdown_connections 2024-09-10T10:51:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.925+0000 7fa89ee00640 1 -- 172.21.15.28:0/2084464155 wait complete. 2024-09-10T10:51:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.927+0000 7fa89ee00640 1 Processor -- start 2024-09-10T10:51:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.927+0000 7fa89ee00640 1 -- start start 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa89806f870 0x7fa8981460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 0x7fa898148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980af400 con 0x7fa8980b0180 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8980af280 con 0x7fa8980b69d0 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8980af580 con 0x7fa89806f870 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48042/0 (socket says 172.21.15.28:48042) 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 -- 172.21.15.28:0/4051699193 learned_addr learned my addr 172.21.15.28:0/4051699193 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897e00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa89806f870 0x7fa8981460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ce00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 0x7fa898148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 -- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa89806f870 msgr2=0x7fa8981460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa89806f870 0x7fa8981460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 -- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 msgr2=0x7fa898148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 0x7fa898148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa897400640 1 -- 172.21.15.28:0/4051699193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa898149720 con 0x7fa8980b0180 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.928+0000 7fa89ce00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 0x7fa898148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa897400640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa88400c4d0 tx=0x7fa88400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884017070 con 0x7fa8980b0180 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa884005ce0 con 0x7fa8980b0180 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884007270 con 0x7fa8980b0180 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa898147420 con 0x7fa8980b0180 2024-09-10T10:51:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.929+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981479e0 con 0x7fa8980b0180 2024-09-10T10:51:39.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.930+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898162a50 con 0x7fa8980b0180 2024-09-10T10:51:39.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.931+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa88401a070 con 0x7fa8980b0180 2024-09-10T10:51:39.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.932+0000 7fa88fe00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:39.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.932+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6906+0+0 (secure 0 0 0) 0x7fa88408a630 con 0x7fa8980b0180 2024-09-10T10:51:39.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.933+0000 7fa897e00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:39.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.933+0000 7fa897e00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 0x7fa87806e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8880016e0 tx=0x7fa888009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:39.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:39.936+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa884058440 con 0x7fa8980b0180 2024-09-10T10:51:40.104 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:51:40.108 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:51:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.158+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa8980aec80 con 0x7fa8980b0180 2024-09-10T10:51:40.160 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:40.159+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:40.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.161+0000 7fa88fe00640 1 -- 172.21.15.28:0/4051699193 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v132) ==== 161+0+0 (secure 0 0 0) 0x7fa88405c740 con 0x7fa8980b0180 2024-09-10T10:51:40.162 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 msgr2=0x7fa87806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 0x7fa87806e2f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8880016e0 tx=0x7fa888009040 comp rx=0 tx=0).stop 2024-09-10T10:51:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 msgr2=0x7fa8981488d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa88400c4d0 tx=0x7fa88400c990 comp rx=0 tx=0).stop 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 shutdown_connections 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa87806be30 0x7fa87806e2f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8980b69d0 0x7fa898148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8980b0180 0x7fa8981488d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 --2- 172.21.15.28:0/4051699193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa89806f870 0x7fa8981460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.164+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 >> 172.21.15.28:0/4051699193 conn(0x7fa89806bc10 msgr2=0x7fa8980ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.165+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 shutdown_connections 2024-09-10T10:51:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.165+0000 7fa89ee00640 1 -- 172.21.15.28:0/4051699193 wait complete. 2024-09-10T10:51:40.169 INFO:tasks.ceph.mds.c.smithi028.stderr: -681> 2024-09-10T10:51:34.256+0000 7f7aef200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:40.182 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:40.215 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:51:40.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.342+0000 7fb2a5400640 1 Processor -- start 2024-09-10T10:51:40.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.342+0000 7fb2a5400640 1 -- start start 2024-09-10T10:51:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb2a5400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 0x7fb2a00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb2a5400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 0x7fb2a015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb2a5400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a00cdbe0 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb2a5400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a00728a0 con 0x7fb2a015b300 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb2a5400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a00cd900 con 0x7fb2a006f8d0 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb29de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb2a00cd3f0 0x7fb2a006f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56560/0 (socket says 172.21.15.28:56560) 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.343+0000 7fb29de00640 1 -- 172.21.15.28:0/1714633902 learned_addr learned my addr 172.21.15.28:0/1714633902 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29f200640 1 --2- 172.21.15.28:0/1714633902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 0x7fb2a015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29e800640 1 --2- 172.21.15.28:0/1714633902 >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 0x7fb2a00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440071350 0 0) 0x7fb2a00cdbe0 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb284003520 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29f200640 1 -- 172.21.15.28:0/1714633902 >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 msgr2=0x7fb2a00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29f200640 1 --2- 172.21.15.28:0/1714633902 >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 0x7fb2a00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29f200640 1 -- 172.21.15.28:0/1714633902 >> v1:172.21.15.28:6789/0 conn(0x7fb2a00cd3f0 legacy=0x7fb2a006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3344791335 0 0) 0x7fb284003520 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29f200640 1 -- 172.21.15.28:0/1714633902 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a0072250 con 0x7fb2a006f8d0 2024-09-10T10:51:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.344+0000 7fb29e800640 1 --2- 172.21.15.28:0/1714633902 >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 0x7fb2a00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.345+0000 7fb29f200640 1 --2- 172.21.15.28:0/1714633902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 0x7fb2a015b6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6184f4abb9a42859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c001670 con 0x7fb2a006f8d0 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb298007540 con 0x7fb2a006f8d0 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb29d400640 1 -- 172.21.15.28:0/1714633902 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb298007910 con 0x7fb2a006f8d0 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 -- 172.21.15.28:0/1714633902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 msgr2=0x7fb2a015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 --2- 172.21.15.28:0/1714633902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 0x7fb2a015b6e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 -- 172.21.15.28:0/1714633902 shutdown_connections 2024-09-10T10:51:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 --2- 172.21.15.28:0/1714633902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a006f8d0 0x7fb2a015b6e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 --2- 172.21.15.28:0/1714633902 >> v2:172.21.15.138:3300/0 conn(0x7fb2a015b300 0x7fb2a00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.346+0000 7fb2a5400640 1 -- 172.21.15.28:0/1714633902 >> 172.21.15.28:0/1714633902 conn(0x7fb2a0067d70 msgr2=0x7fb2a006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.347+0000 7fb2a5400640 1 -- 172.21.15.28:0/1714633902 shutdown_connections 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.347+0000 7fb2a5400640 1 -- 172.21.15.28:0/1714633902 wait complete. 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.348+0000 7fb2a5400640 1 Processor -- start 2024-09-10T10:51:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.348+0000 7fb2a5400640 1 -- start start 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 0x7fb2a0140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 0x7fb2a0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00d0160 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00cffe0 con 0x7fb2a006f8d0 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb2a5400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a0072d10 con 0x7fb2a015b300 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb29de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb29de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48062/0 (socket says 172.21.15.28:48062) 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.349+0000 7fb29de00640 1 -- 172.21.15.28:0/883010501 learned_addr learned my addr 172.21.15.28:0/883010501 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29f200640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 0x7fb2a0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29e800640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 0x7fb2a0140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 -- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 msgr2=0x7fb2a0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 0x7fb2a0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 -- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 msgr2=0x7fb2a0140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 0x7fb2a0140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 -- 172.21.15.28:0/883010501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a01514c0 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29e800640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 0x7fb2a0140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29f200640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 0x7fb2a0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb29de00640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0011e0 tx=0x7fb28c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c017070 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb28c005ad0 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c007070 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.350+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a01517a0 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.351+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a0151d60 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.352+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a006f380 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.353+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb28c01a070 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.353+0000 7fb296a00640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 0x7fb28006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:40.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.354+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6992+0+0 (secure 0 0 0) 0x7fb28c08a900 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.354+0000 7fb29e800640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 0x7fb28006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:40.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.355+0000 7fb29e800640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 0x7fb28006e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb290009af0 tx=0x7fb290008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:40.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.358+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb28c058d30 con 0x7fb2a00cd3f0 2024-09-10T10:51:40.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:40.613+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb2a01411e0 con 0x7fb2a00cd3f0 2024-09-10T10:51:41.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.163+0000 7fb296a00640 1 -- 172.21.15.28:0/883010501 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v133) ==== 107+0+0 (secure 0 0 0) 0x7fb28c058630 con 0x7fb2a00cd3f0 2024-09-10T10:51:41.164 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 msgr2=0x7fb28006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 0x7fb28006e2f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb290009af0 tx=0x7fb290008040 comp rx=0 tx=0).stop 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 msgr2=0x7fb2a0150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0011e0 tx=0x7fb28c00cbe0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 shutdown_connections 2024-09-10T10:51:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb28006be30 0x7fb28006e2f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb2a015b300 0x7fb2a0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a00cd3f0 0x7fb2a0150b00 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 --2- 172.21.15.28:0/883010501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2a006f8d0 0x7fb2a0140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.166+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 >> 172.21.15.28:0/883010501 conn(0x7fb2a0067d70 msgr2=0x7fb2a00af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.167+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 shutdown_connections 2024-09-10T10:51:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.167+0000 7fb2a5400640 1 -- 172.21.15.28:0/883010501 wait complete. 2024-09-10T10:51:41.186 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:41.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.302+0000 7fdc0a153f40 1 Processor -- start 2024-09-10T10:51:41.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.302+0000 7fdc0a153f40 1 -- start start 2024-09-10T10:51:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.302+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.302+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e62d2d0 0x559f2e6142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.302+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 0x559f2e5e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559f2e62cdc0 con 0x559f2e5e7cb0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f2e58bb30 con 0x559f2e6b94b0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559f2e62c1f0 con 0x559f2e62d2d0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48082/0 (socket says 172.21.15.28:48082) 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 -- 172.21.15.28:0/643780872 learned_addr learned my addr 172.21.15.28:0/643780872 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07e00640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 0x559f2e5e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc06a00640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e62d2d0 0x559f2e6142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 -- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e62d2d0 msgr2=0x559f2e6142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e62d2d0 0x559f2e6142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 -- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 msgr2=0x559f2e5e56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 0x559f2e5e56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 -- 172.21.15.28:0/643780872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2e612000 con 0x559f2e5e7cb0 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07e00640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 0x559f2e5e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.303+0000 7fdc07400640 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001d10 tx=0x7fdbf800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2c6e28ed378169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc05600640 1 -- 172.21.15.28:0/643780872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8001e60 con 0x559f2e5e7cb0 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc05600640 1 -- 172.21.15.28:0/643780872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbf800bd80 con 0x559f2e5e7cb0 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc05600640 1 -- 172.21.15.28:0/643780872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800f640 con 0x559f2e5e7cb0 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 -- 172.21.15.28:0/643780872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 msgr2=0x559f2e629210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8001d10 tx=0x7fdbf800b5b0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 -- 172.21.15.28:0/643780872 shutdown_connections 2024-09-10T10:51:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e6b94b0 0x559f2e5e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e62d2d0 0x559f2e6142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/643780872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e5e7cb0 0x559f2e629210 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 -- 172.21.15.28:0/643780872 >> 172.21.15.28:0/643780872 conn(0x559f2e611b90 msgr2=0x559f2e62d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 -- 172.21.15.28:0/643780872 shutdown_connections 2024-09-10T10:51:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.304+0000 7fdc0a153f40 1 -- 172.21.15.28:0/643780872 wait complete. 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.305+0000 7fdc0a153f40 1 Processor -- start 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.305+0000 7fdc0a153f40 1 -- start start 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e5e7cb0 0x559f2e6b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e62d2d0 0x559f2e62a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559f2e62bc60 con 0x559f2e6b94b0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f2e6b2ef0 con 0x559f2e62d2d0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc0a153f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559f2e62de60 con 0x559f2e5e7cb0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48094/0 (socket says 172.21.15.28:48094) 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 learned_addr learned my addr 172.21.15.28:0/462774147 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e5e7cb0 0x559f2e6b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.306+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e62d2d0 0x559f2e62a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e5e7cb0 msgr2=0x559f2e6b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x559f2e5e7cb0 0x559f2e6b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e62d2d0 msgr2=0x559f2e62a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f2e62d2d0 0x559f2e62a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f2e6b09c0 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc07e00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdc00025ad0 tx=0x7fdc00003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdbffe00640 1 -- 172.21.15.28:0/462774147 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0004f660 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdbffe00640 1 -- 172.21.15.28:0/462774147 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc00062400 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdbffe00640 1 -- 172.21.15.28:0/462774147 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc000627f0 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.307+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f2e6af6c0 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.308+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2e6b3f20 con 0x559f2e6b94b0 2024-09-10T10:51:41.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.309+0000 7fdbffe00640 1 -- 172.21.15.28:0/462774147 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc00062990 con 0x559f2e6b94b0 2024-09-10T10:51:41.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.310+0000 7fdbffe00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 0x7fdbe806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.310+0000 7fdbffe00640 1 -- 172.21.15.28:0/462774147 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6992+0+0 (secure 0 0 0) 0x7fdc000cdeb0 con 0x559f2e6b94b0 2024-09-10T10:51:41.313 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:41.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.311+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 0x559f2e6b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.311+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 0x7fdbe806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.311+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e6b1fd0 con 0x559f2e6b16b0 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.311+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 0x559f2e6b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.312+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 0x7fdbe806cd00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b5f0 tx=0x7fdbf8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.312+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 0x559f2e6b1a90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.314+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v108'226 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.314+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75e0e0 con 0x559f2e6b16b0 2024-09-10T10:51:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.315+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v108'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.315+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75e480 con 0x559f2e6b16b0 2024-09-10T10:51:41.318 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.317+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v108'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.317+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f830 con 0x559f2e6b16b0 2024-09-10T10:51:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.319+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v108'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.319+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75fbd0 con 0x559f2e6b16b0 2024-09-10T10:51:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.321+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v108'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.321+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75e820 con 0x559f2e6b16b0 2024-09-10T10:51:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.322+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v108'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.322+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75ec10 con 0x559f2e6b16b0 2024-09-10T10:51:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.324+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v108'232 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.324+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f000 con 0x559f2e6b16b0 2024-09-10T10:51:41.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.325+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v108'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.326+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f000 con 0x559f2e6b16b0 2024-09-10T10:51:41.328 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:41.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.327+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v108'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.327+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.329+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v108'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.329+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.331+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v108'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.331+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.332+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v108'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.332+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.333+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v108'238 uv213 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.334+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.335+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v108'239 uv239 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.335+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.338 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:41.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.337+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v108'240 uv240 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.337+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.339+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v108'241 uv241 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.339+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.340+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v108'242 uv242 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.341+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75efb0 con 0x559f2e6b16b0 2024-09-10T10:51:41.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.342+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v108'243 uv243 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.342+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f3b0 con 0x559f2e6b16b0 2024-09-10T10:51:41.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.344+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v108'244 uv244 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.344+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f390 con 0x559f2e6b16b0 2024-09-10T10:51:41.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.345+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v108'245 uv219 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.345+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.347+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v108'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.347+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.349 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:41.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.348+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v108'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.349+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.350+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v108'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.350+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.352+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v108'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.352+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.354+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v108'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.354+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.355+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v108'251 uv225 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.355+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.357+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v108'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.357+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.359 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.358+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v108'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.358+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.360+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v108'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.360+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.361+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v108'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.361+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e6b16b0 2024-09-10T10:51:41.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.363+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v108'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e6b16b0 2024-09-10T10:51:41.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.363+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 0x559f2e762870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.363+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e75f270 con 0x559f2e760480 2024-09-10T10:51:41.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.364+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 0x559f2e762870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.364+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 0x559f2e762870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.366+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v108'177 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.366+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e762db0 con 0x559f2e760480 2024-09-10T10:51:41.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.368+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v108'178 uv178 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.368+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e763150 con 0x559f2e760480 2024-09-10T10:51:41.370 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:41.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.369+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v108'179 uv179 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.369+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e763590 con 0x559f2e760480 2024-09-10T10:51:41.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.370+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v108'180 uv180 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.370+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7639f0 con 0x559f2e760480 2024-09-10T10:51:41.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.371+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v108'181 uv181 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.371+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e763de0 con 0x559f2e760480 2024-09-10T10:51:41.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.373+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v108'182 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.373+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7641d0 con 0x559f2e760480 2024-09-10T10:51:41.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.374+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v108'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.374+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7645c0 con 0x559f2e760480 2024-09-10T10:51:41.377 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:41.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.375+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v108'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.375+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7645c0 con 0x559f2e760480 2024-09-10T10:51:41.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.376+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v108'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.376+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.378+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v108'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.378+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.379+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v108'187 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.379+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.381+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v108'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.381+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.383 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:41.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.382+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v108'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.382+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.384+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v108'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.384+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.385+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v108'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.385+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.386+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v108'192 uv172 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.386+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.388+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v108'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.388+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.389+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v108'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.391 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:41.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.389+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.390+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v108'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.390+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.391+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v108'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.391+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e764570 con 0x559f2e760480 2024-09-10T10:51:41.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.393+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v108'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e760480 2024-09-10T10:51:41.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.393+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x559f2e764f10 0x559f2e765310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.393+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e767ac0 con 0x559f2e764f10 2024-09-10T10:51:41.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.393+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x559f2e764f10 0x559f2e765310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.393+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x559f2e764f10 0x559f2e765310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.395+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v108'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.395+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e767ac0 con 0x559f2e764f10 2024-09-10T10:51:41.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.396+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v108'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.396+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e767ac0 con 0x559f2e764f10 2024-09-10T10:51:41.399 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:41.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.398+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v108'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.398+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e768ad0 con 0x559f2e764f10 2024-09-10T10:51:41.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.399+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v108'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.399+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e768ad0 con 0x559f2e764f10 2024-09-10T10:51:41.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.401+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v108'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.401+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e767ac0 con 0x559f2e764f10 2024-09-10T10:51:41.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.402+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v108'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.402+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x559f2e768140 0x559f2e768550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.402+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76af20 con 0x559f2e768140 2024-09-10T10:51:41.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.403+0000 7fdc07e00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x559f2e768140 0x559f2e768550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.403+0000 7fdc07e00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x559f2e768140 0x559f2e768550 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.404+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v108'37 uv36 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.404+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76b2c0 con 0x559f2e768140 2024-09-10T10:51:41.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.405+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v108'38 uv38 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.405+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76b680 con 0x559f2e768140 2024-09-10T10:51:41.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.406+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v108'39 uv39 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.408 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:41.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.407+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76bf40 con 0x559f2e768140 2024-09-10T10:51:41.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.408+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v108'40 uv40 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.408+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c2e0 con 0x559f2e764f10 2024-09-10T10:51:41.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.409+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v108'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.409+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c2e0 con 0x559f2e764f10 2024-09-10T10:51:41.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.410+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v108'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.410+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c2e0 con 0x559f2e764f10 2024-09-10T10:51:41.412 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:41.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.411+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v108'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.411+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76d2f0 con 0x559f2e764f10 2024-09-10T10:51:41.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.412+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v108'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.412+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76d2f0 con 0x559f2e764f10 2024-09-10T10:51:41.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.414+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v108'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.414+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c2e0 con 0x559f2e764f10 2024-09-10T10:51:41.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.415+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v108'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.415+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c2e0 con 0x559f2e768140 2024-09-10T10:51:41.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.416+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v108'83 uv75 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.416+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76c680 con 0x559f2e768140 2024-09-10T10:51:41.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.418+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v108'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.418+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76ca20 con 0x559f2e768140 2024-09-10T10:51:41.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.419+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v108'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.420 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:41.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.419+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76ddd0 con 0x559f2e768140 2024-09-10T10:51:41.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.421+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v108'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.421+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76ddd0 con 0x559f2e768140 2024-09-10T10:51:41.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.422+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v108'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.422+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.424+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v108'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.424+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.425+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v108'89 uv82 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.425+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.427+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v108'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.427+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.428+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v108'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.429 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:41.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.428+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.430+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v108'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.430+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e768140 2024-09-10T10:51:41.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.431+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v108'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.431+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e764f10 2024-09-10T10:51:41.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.432+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v108'37 uv36 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.432+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e764f10 2024-09-10T10:51:41.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.434+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v108'38 uv38 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.434+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e764f10 2024-09-10T10:51:41.436 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:41.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.435+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v108'39 uv39 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.435+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e764f10 2024-09-10T10:51:41.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.436+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v108'40 uv40 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.436+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76cdc0 con 0x559f2e764f10 2024-09-10T10:51:41.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.437+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v108'41 uv41 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e764f10 2024-09-10T10:51:41.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.437+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x559f2e76d440 0x559f2e76f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.437+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76fe00 con 0x559f2e76d440 2024-09-10T10:51:41.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.438+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x559f2e76d440 0x559f2e76f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.438+0000 7fdc07400640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x559f2e76d440 0x559f2e76f830 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.440+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v108'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.440+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e76fec0 con 0x559f2e76d440 2024-09-10T10:51:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.441+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v108'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.441+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.443 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:41.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.442+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v108'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.442+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7712f0 con 0x559f2e76d440 2024-09-10T10:51:41.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.443+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v108'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.443+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7712f0 con 0x559f2e76d440 2024-09-10T10:51:41.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.445+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v108'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.445+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.446+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v108'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.446+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.448+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v108'74 uv68 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.448+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.449+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v108'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.449+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.451 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:41.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.450+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v108'76 uv76 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.450+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.451+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v108'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.451+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.453+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v108'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.453+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.454+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v108'79 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.454+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.455+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v108'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.455+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.457 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:41.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.456+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v108'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.457+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7702e0 con 0x559f2e76d440 2024-09-10T10:51:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.458+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v108'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.458+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7730f0 con 0x559f2e76d440 2024-09-10T10:51:41.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.460+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v108'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.460+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.461+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v108'36 uv34 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.461+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.462+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v108'37 uv37 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.462+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.463+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v108'38 uv38 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.464 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.463+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.465+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v108'39 uv39 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.465+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.466+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v108'45 uv44 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.466+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.467+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v108'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.467+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.470 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.468+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v108'47 uv47 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.468+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7717a0 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.469+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v108'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.469+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7717a0 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.470+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v108'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.470+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e770790 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.471+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v108'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdc00060070 con 0x559f2e768140 2024-09-10T10:51:41.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.471+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559f2e770ca0 0x559f2e771080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.471+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e770ca0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.471+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559f2e770ca0 0x559f2e771080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.472+0000 7fdc06a00640 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559f2e770ca0 0x559f2e771080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.473+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v108'37 uv36 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e770ca0 2024-09-10T10:51:41.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.473+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e770ca0 2024-09-10T10:51:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.474+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v108'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e770ca0 2024-09-10T10:51:41.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.474+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e770ca0 2024-09-10T10:51:41.476 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.475+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v108'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e770ca0 2024-09-10T10:51:41.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.475+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e770ca0 2024-09-10T10:51:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.476+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v108'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e770ca0 2024-09-10T10:51:41.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.476+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e770ca0 2024-09-10T10:51:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.477+0000 7fdc06a00640 1 -- 172.21.15.28:0/462774147 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v108'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fdbf4009a10 con 0x559f2e770ca0 2024-09-10T10:51:41.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.478+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.479+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v108'95 uv86 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.479+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.480+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v108'96 uv96 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.480+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.481+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v108'97 uv97 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.482 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.481+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.482+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v108'98 uv98 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.482+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.484+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v108'99 uv99 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.484+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.485+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v108'100 uv93 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.485+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.486+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v108'101 uv101 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.486+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.488 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:41.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.487+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v108'102 uv102 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.487+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.488+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v108'103 uv103 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.488+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.489+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v108'104 uv104 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.489+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e7715c0 con 0x559f2e76d440 2024-09-10T10:51:41.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.490+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v108'105 uv105 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.491+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x559f2e773130 con 0x559f2e76d440 2024-09-10T10:51:41.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc07400640 1 -- 172.21.15.28:0/462774147 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v108'106 uv106 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fdbf801d730 con 0x559f2e76d440 2024-09-10T10:51:41.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x559f2e764f10 msgr2=0x559f2e765310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x559f2e764f10 0x559f2e765310 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x559f2e76d440 msgr2=0x559f2e76f830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x559f2e76d440 0x559f2e76f830 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559f2e770ca0 msgr2=0x559f2e771080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x559f2e770ca0 0x559f2e771080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x559f2e768140 msgr2=0x559f2e768550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x559f2e768140 0x559f2e768550 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 msgr2=0x559f2e762870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 reap_dead start 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 0x559f2e762870 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc07e00640 1 -- 172.21.15.28:0/462774147 reap_dead start 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 msgr2=0x559f2e6b1a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 0x559f2e6b1a90 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 msgr2=0x7fdbe806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 0x7fdbe806cd00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b5f0 tx=0x7fdbf8009000 comp rx=0 tx=0).stop 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 msgr2=0x559f2e62a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdc00025ad0 tx=0x7fdc00003330 comp rx=0 tx=0).stop 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 shutdown_connections 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x559f2e760480 0x559f2e762870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x559f2e6b16b0 0x559f2e6b1a90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdbe806a840 0x7fdbe806cd00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 --2- 172.21.15.28:0/462774147 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559f2e6b94b0 0x559f2e62a8b0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 >> 172.21.15.28:0/462774147 conn(0x559f2e611b90 msgr2=0x559f2e62d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 shutdown_connections 2024-09-10T10:51:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.492+0000 7fdc0a153f40 1 -- 172.21.15.28:0/462774147 wait complete. 2024-09-10T10:51:41.500 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds_snaptable 2024-09-10T10:51:41.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.617+0000 7fb89d71ff40 1 Processor -- start 2024-09-10T10:51:41.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.617+0000 7fb89d71ff40 1 -- start start 2024-09-10T10:51:41.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.617+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.617+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf132e240 0x55bcf1347e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 0x55bcf13d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcf12a5b30 con 0x55bcf1301610 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcf12a9b70 con 0x55bcf13d9a60 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bcf12fedb0 con 0x55bcf132e240 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48104/0 (socket says 172.21.15.28:48104) 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 -- 172.21.15.28:0/2044381472 learned_addr learned my addr 172.21.15.28:0/2044381472 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89a800640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 0x55bcf13d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899400640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf132e240 0x55bcf1347e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 -- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf132e240 msgr2=0x55bcf1347e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf132e240 0x55bcf1347e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 -- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 msgr2=0x55bcf13d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 0x55bcf13d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 -- 172.21.15.28:0/2044381472 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf13da630 con 0x55bcf1301610 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb89a800640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 0x55bcf13d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.618+0000 7fb899e00640 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b320 tx=0x7fb88c00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ee2c527788c88f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb893e00640 1 -- 172.21.15.28:0/2044381472 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c012020 con 0x55bcf1301610 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb893e00640 1 -- 172.21.15.28:0/2044381472 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88c00d690 con 0x55bcf1301610 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb893e00640 1 -- 172.21.15.28:0/2044381472 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00da60 con 0x55bcf1301610 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2044381472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 msgr2=0x55bcf13460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b320 tx=0x7fb88c00b7e0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2044381472 shutdown_connections 2024-09-10T10:51:41.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf13d9a60 0x55bcf13d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf132e240 0x55bcf1347e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2044381472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf1301610 0x55bcf13460f0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2044381472 >> 172.21.15.28:0/2044381472 conn(0x55bcf132bb90 msgr2=0x55bcf13dca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2044381472 shutdown_connections 2024-09-10T10:51:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.619+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2044381472 wait complete. 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.620+0000 7fb89d71ff40 1 Processor -- start 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.620+0000 7fb89d71ff40 1 -- start start 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf1301610 0x55bcf13c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf132e240 0x55bcf13d4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 0x55bcf13d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcf13da350 con 0x55bcf13d9a60 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bcf13da070 con 0x55bcf132e240 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89d71ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bcf13478e0 con 0x55bcf1301610 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 0x55bcf13d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 0x55bcf13d85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48108/0 (socket says 172.21.15.28:48108) 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb89a800640 1 -- 172.21.15.28:0/2805653815 learned_addr learned my addr 172.21.15.28:0/2805653815 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb899e00640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf1301610 0x55bcf13c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.621+0000 7fb899400640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf132e240 0x55bcf13d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 -- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf1301610 msgr2=0x55bcf13c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bcf1301610 0x55bcf13c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 -- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf132e240 msgr2=0x55bcf13d4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bcf132e240 0x55bcf13d4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 -- 172.21.15.28:0/2805653815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf132aeb0 con 0x55bcf13d9a60 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89a800640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 0x55bcf13d85d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb89400a4a0 tx=0x7fb8940033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb892a00640 1 -- 172.21.15.28:0/2805653815 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894069070 con 0x55bcf13d9a60 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb892a00640 1 -- 172.21.15.28:0/2805653815 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb89404f400 con 0x55bcf13d9a60 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bcf13c9030 con 0x55bcf13d9a60 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.622+0000 7fb892a00640 1 -- 172.21.15.28:0/2805653815 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894005a80 con 0x55bcf13d9a60 2024-09-10T10:51:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.623+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf13c95f0 con 0x55bcf13d9a60 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.624+0000 7fb892a00640 1 -- 172.21.15.28:0/2805653815 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb89406c360 con 0x55bcf13d9a60 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.625+0000 7fb892a00640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87c06a810 0x7fb87c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.625+0000 7fb892a00640 1 -- 172.21.15.28:0/2805653815 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6992+0+0 (secure 0 0 0) 0x7fb8940cd0d0 con 0x55bcf13d9a60 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bcf13caaf0 0x55bcf13c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e108) -- 0x55bcf13ca330 con 0x55bcf13caaf0 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb899e00640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87c06a810 0x7fb87c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb899400640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bcf13caaf0 0x55bcf13c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb899400640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bcf13caaf0 0x55bcf13c9df0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.626+0000 7fb899e00640 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87c06a810 0x7fb87c06ccd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b320 tx=0x7fb88c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb899400640 1 -- 172.21.15.28:0/2805653815 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [delete] v108'42 uv42 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb88800e040 con 0x55bcf13caaf0 2024-09-10T10:51:41.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bcf13caaf0 msgr2=0x55bcf13c9df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bcf13caaf0 0x55bcf13c9df0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87c06a810 msgr2=0x7fb87c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb87c06a810 0x7fb87c06ccd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00b320 tx=0x7fb88c009000 comp rx=0 tx=0).stop 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 msgr2=0x55bcf13d85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 --2- 172.21.15.28:0/2805653815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bcf13d9a60 0x55bcf13d85d0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb89400a4a0 tx=0x7fb8940033a0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89a800640 1 -- 172.21.15.28:0/2805653815 reap_dead start 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 shutdown_connections 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 >> 172.21.15.28:0/2805653815 conn(0x55bcf132bb90 msgr2=0x55bcf13dc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 shutdown_connections 2024-09-10T10:51:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.628+0000 7fb89d71ff40 1 -- 172.21.15.28:0/2805653815 wait complete. 2024-09-10T10:51:41.634 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:41.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.793+0000 7fa815800640 1 Processor -- start 2024-09-10T10:51:41.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.793+0000 7fa815800640 1 -- start start 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa815800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa8100b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa815800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa810072050 0x7fa8100724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa815800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8100b4620 con 0x7fa81015e900 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa815800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa8100b4f60 con 0x7fa810072050 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa815800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8100ce100 con 0x7fa8100b4020 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa80f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa810072050 0x7fa8100724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa80ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa81015e900 0x7fa8100ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:51580/0 (socket says 172.21.15.28:51580) 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa80f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa810072050 0x7fa8100724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48124/0 (socket says 172.21.15.28:48124) 2024-09-10T10:51:41.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa80ea00640 1 -- 172.21.15.28:0/1143588664 learned_addr learned my addr 172.21.15.28:0/1143588664 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.794+0000 7fa807e00640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa8100b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.795+0000 7fa80e000640 1 -- 172.21.15.28:0/1143588664 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783284722 0 0) 0x7fa8100b4620 con 0x7fa81015e900 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.795+0000 7fa80f400640 1 -- 172.21.15.28:0/1143588664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 msgr2=0x7fa8100b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.795+0000 7fa80f400640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa8100b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.795+0000 7fa80f400640 1 -- 172.21.15.28:0/1143588664 >> v1:172.21.15.138:6789/0 conn(0x7fa81015e900 legacy=0x7fa8100ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.795+0000 7fa80f400640 1 -- 172.21.15.28:0/1143588664 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8100ce3a0 con 0x7fa810072050 2024-09-10T10:51:41.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa80f400640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa810072050 0x7fa8100724e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa808009d90 tx=0x7fa80802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96c9574c9e06b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa80e000640 1 -- 172.21.15.28:0/1143588664 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808031600 con 0x7fa810072050 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa80e000640 1 -- 172.21.15.28:0/1143588664 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8080317a0 con 0x7fa810072050 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa80e000640 1 -- 172.21.15.28:0/1143588664 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808031b70 con 0x7fa810072050 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa815800640 1 -- 172.21.15.28:0/1143588664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa810072050 msgr2=0x7fa8100724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.796+0000 7fa815800640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa810072050 0x7fa8100724e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fa808009d90 tx=0x7fa80802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 -- 172.21.15.28:0/1143588664 shutdown_connections 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa810072050 0x7fa8100724e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 --2- 172.21.15.28:0/1143588664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa8100b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 -- 172.21.15.28:0/1143588664 >> 172.21.15.28:0/1143588664 conn(0x7fa81006bdf0 msgr2=0x7fa81006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 -- 172.21.15.28:0/1143588664 shutdown_connections 2024-09-10T10:51:41.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.797+0000 7fa815800640 1 -- 172.21.15.28:0/1143588664 wait complete. 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.798+0000 7fa815800640 1 Processor -- start 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.798+0000 7fa815800640 1 -- start start 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa81015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa81015e900 0x7fa810152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100cee80 con 0x7fa81015e900 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8100ce960 con 0x7fa810072050 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.799+0000 7fa815800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa8100cf000 con 0x7fa8100b4020 2024-09-10T10:51:41.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa81015e900 0x7fa810152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:58312/0 (socket says 172.21.15.28:58312) 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 -- 172.21.15.28:0/305546101 learned_addr learned my addr 172.21.15.28:0/305546101 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa807e00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa81015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 -- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 msgr2=0x7fa81015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa81015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 -- 172.21.15.28:0/305546101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa81015e900 msgr2=0x7fa810152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa81015e900 0x7fa810152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 -- 172.21.15.28:0/305546101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8100af200 con 0x7fa810072050 2024-09-10T10:51:41.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.800+0000 7fa80ea00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c3f0 tx=0x7fa80000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.801+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000074e0 con 0x7fa810072050 2024-09-10T10:51:41.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.801+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8000047e0 con 0x7fa810072050 2024-09-10T10:51:41.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.801+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800004c10 con 0x7fa810072050 2024-09-10T10:51:41.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.801+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8100af470 con 0x7fa810072050 2024-09-10T10:51:41.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.801+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81015b1d0 con 0x7fa810072050 2024-09-10T10:51:41.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.802+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8100af640 con 0x7fa810072050 2024-09-10T10:51:41.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.803+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa800011070 con 0x7fa810072050 2024-09-10T10:51:41.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.804+0000 7fa806a00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 0x7fa7ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:41.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.804+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 6992+0+0 (secure 0 0 0) 0x7fa80008ae80 con 0x7fa810072050 2024-09-10T10:51:41.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.804+0000 7fa807e00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 0x7fa7ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:41.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.805+0000 7fa807e00640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 0x7fa7ec06e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa810156370 tx=0x7fa7f8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:41.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:41.809+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa800058c60 con 0x7fa810072050 2024-09-10T10:51:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.031+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa8100b4a20 con 0x7fa810072050 2024-09-10T10:51:42.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.720+0000 7fa806a00640 1 -- 172.21.15.28:0/305546101 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v134) ==== 162+0+0 (secure 0 0 0) 0x7fa80005cf60 con 0x7fa810072050 2024-09-10T10:51:42.721 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:42.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.723+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 msgr2=0x7fa7ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 0x7fa7ec06e2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa810156370 tx=0x7fa7f8009190 comp rx=0 tx=0).stop 2024-09-10T10:51:42.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 msgr2=0x7fa810147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa80000c3f0 tx=0x7fa80000c8b0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 shutdown_connections 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7ec06be30 0x7fa7ec06e2f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa81015e900 0x7fa810152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa8100b4020 0x7fa81015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 --2- 172.21.15.28:0/305546101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa810072050 0x7fa810147790 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 >> 172.21.15.28:0/305546101 conn(0x7fa81006bdf0 msgr2=0x7fa810158810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 shutdown_connections 2024-09-10T10:51:42.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.724+0000 7fa815800640 1 -- 172.21.15.28:0/305546101 wait complete. 2024-09-10T10:51:42.727 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:42.727+0000 7faef9a00640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-10T10:51:42.728 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:42.727+0000 7faef9a00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-10T10:51:42.728 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:42.728+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:42.729 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:42.728+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:42.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:42.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.904+0000 7fbe08400640 1 Processor -- start 2024-09-10T10:51:42.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.904+0000 7fbe08400640 1 -- start start 2024-09-10T10:51:42.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.905+0000 7fbe08400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe00072320 0x7fbe000cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.905+0000 7fbe08400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 0x7fbe0006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.905+0000 7fbe08400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe00146140 con 0x7fbe00072700 2024-09-10T10:51:42.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.905+0000 7fbe08400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbe000cd740 con 0x7fbe00072320 2024-09-10T10:51:42.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.905+0000 7fbe08400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe000cda20 con 0x7fbe00072de0 2024-09-10T10:51:42.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe00072320 0x7fbe000cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe00072320 0x7fbe000cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48144/0 (socket says 172.21.15.28:48144) 2024-09-10T10:51:42.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 -- 172.21.15.28:0/1658220609 learned_addr learned my addr 172.21.15.28:0/1658220609 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:42.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05000640 1 --2- 172.21.15.28:0/1658220609 >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 0x7fbe0006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 -- 172.21.15.28:0/1658220609 >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 msgr2=0x7fbe0006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 --2- 172.21.15.28:0/1658220609 >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 0x7fbe0006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 -- 172.21.15.28:0/1658220609 >> v1:172.21.15.138:6789/0 conn(0x7fbe00072700 legacy=0x7fbe0006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05a00640 1 -- 172.21.15.28:0/1658220609 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe000cdd00 con 0x7fbe00072320 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.906+0000 7fbe05000640 1 --2- 172.21.15.28:0/1658220609 >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 0x7fbe0006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:42.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.907+0000 7fbe05a00640 1 --2- 172.21.15.28:0/1658220609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072320 0x7fbe000cce60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbe0006f8c0 tx=0x7fbdf402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33ab94a1bcdd68c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:42.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbdffe00640 1 -- 172.21.15.28:0/1658220609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4037070 con 0x7fbe00072320 2024-09-10T10:51:42.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbdffe00640 1 -- 172.21.15.28:0/1658220609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf4032440 con 0x7fbe00072320 2024-09-10T10:51:42.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbdffe00640 1 -- 172.21.15.28:0/1658220609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4032810 con 0x7fbe00072320 2024-09-10T10:51:42.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 -- 172.21.15.28:0/1658220609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072320 msgr2=0x7fbe000cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 --2- 172.21.15.28:0/1658220609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072320 0x7fbe000cce60 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbe0006f8c0 tx=0x7fbdf402a200 comp rx=0 tx=0).stop 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 -- 172.21.15.28:0/1658220609 shutdown_connections 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 --2- 172.21.15.28:0/1658220609 >> v2:172.21.15.138:3300/0 conn(0x7fbe00072de0 0x7fbe0006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 --2- 172.21.15.28:0/1658220609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072320 0x7fbe000cce60 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 -- 172.21.15.28:0/1658220609 >> 172.21.15.28:0/1658220609 conn(0x7fbe00067d70 msgr2=0x7fbe0006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 -- 172.21.15.28:0/1658220609 shutdown_connections 2024-09-10T10:51:42.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.908+0000 7fbe08400640 1 -- 172.21.15.28:0/1658220609 wait complete. 2024-09-10T10:51:42.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.910+0000 7fbe08400640 1 Processor -- start 2024-09-10T10:51:42.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.910+0000 7fbe08400640 1 -- start start 2024-09-10T10:51:42.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe00072320 0x7fbe000b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe00072700 0x7fbe001418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbe000cd520 con 0x7fbe00072de0 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe000cd3a0 con 0x7fbe00072320 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe08400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0006ed00 con 0x7fbe00072700 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe06400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe06400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48148/0 (socket says 172.21.15.28:48148) 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe06400640 1 -- 172.21.15.28:0/1742297195 learned_addr learned my addr 172.21.15.28:0/1742297195 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe05a00640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe00072320 0x7fbe000b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.911+0000 7fbe05000640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe00072700 0x7fbe001418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 -- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe00072700 msgr2=0x7fbe001418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe00072700 0x7fbe001418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 -- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe00072320 msgr2=0x7fbe000b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe00072320 0x7fbe000b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 -- 172.21.15.28:0/1742297195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe000c9110 con 0x7fbe00072de0 2024-09-10T10:51:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe06400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c730 tx=0x7fbdf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8016020 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf8006d50 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8003bb0 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe000c93f0 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.912+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe000c99b0 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.913+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe000cce60 con 0x7fbe00072de0 2024-09-10T10:51:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.914+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdf800a7f0 con 0x7fbe00072de0 2024-09-10T10:51:42.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.915+0000 7fbdfd600640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 0x7fbde006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:42.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.915+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7fbdf808af70 con 0x7fbe00072de0 2024-09-10T10:51:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.916+0000 7fbe05a00640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 0x7fbde006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.916+0000 7fbe05a00640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 0x7fbde006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4034000 tx=0x7fbdf402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:42.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:42.920+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbdf8020020 con 0x7fbe00072de0 2024-09-10T10:51:43.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.142+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe0006e8c0 con 0x7fbe00072de0 2024-09-10T10:51:43.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.142+0000 7fbdfd600640 1 -- 172.21.15.28:0/1742297195 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7fbdf8058cd0 con 0x7fbe00072de0 2024-09-10T10:51:43.143 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:43.143 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":136,"btime":"2024-09-10T10:51:42:728611+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:42.728549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:43.144 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 136 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.144+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 msgr2=0x7fbde006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 0x7fbde006e2f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4034000 tx=0x7fbdf402e000 comp rx=0 tx=0).stop 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 msgr2=0x7fbe0015b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800c730 tx=0x7fbdf800cbf0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 shutdown_connections 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbde006be30 0x7fbde006e2f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe00072de0 0x7fbe0015b990 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe00072700 0x7fbe001418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 --2- 172.21.15.28:0/1742297195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe00072320 0x7fbe000b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 >> 172.21.15.28:0/1742297195 conn(0x7fbe00067d70 msgr2=0x7fbe000ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 shutdown_connections 2024-09-10T10:51:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.145+0000 7fbe08400640 1 -- 172.21.15.28:0/1742297195 wait complete. 2024-09-10T10:51:43.161 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:43.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:43.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.317+0000 7f44f3200640 1 Processor -- start 2024-09-10T10:51:43.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.317+0000 7f44f3200640 1 -- start start 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44ec145e00 0x7f44ec0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 0x7f44ec0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 0x7f44ec06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec0d0e80 con 0x7f44ec145e00 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec1461e0 con 0x7f44ec0d0a10 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.318+0000 7f44f3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0af720 con 0x7f44ec15d460 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44ec145e00 0x7f44ec0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44ec145e00 0x7f44ec0cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48150/0 (socket says 172.21.15.28:48150) 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 -- 172.21.15.28:0/4208529781 learned_addr learned my addr 172.21.15.28:0/4208529781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:43.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44e3400640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 0x7f44ec0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44f1200640 1 --2- 172.21.15.28:0/4208529781 >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 0x7f44ec06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 -- 172.21.15.28:0/4208529781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 msgr2=0x7f44ec0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 0x7f44ec0d04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 -- 172.21.15.28:0/4208529781 >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 msgr2=0x7f44ec06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 --2- 172.21.15.28:0/4208529781 >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 0x7f44ec06bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44ebe00640 1 -- 172.21.15.28:0/4208529781 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec15d8c0 con 0x7f44ec145e00 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44e3400640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 0x7f44ec0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.319+0000 7f44f1200640 1 --2- 172.21.15.28:0/4208529781 >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 0x7f44ec06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.320+0000 7f44ebe00640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec145e00 0x7f44ec0cd8a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f44ec073f70 tx=0x7f44dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa694fa32660c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44eb400640 1 -- 172.21.15.28:0/4208529781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc037070 con 0x7f44ec145e00 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44eb400640 1 -- 172.21.15.28:0/4208529781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44dc032440 con 0x7f44ec145e00 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44eb400640 1 -- 172.21.15.28:0/4208529781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44dc032810 con 0x7f44ec145e00 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 -- 172.21.15.28:0/4208529781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec145e00 msgr2=0x7f44ec0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec145e00 0x7f44ec0cd8a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f44ec073f70 tx=0x7f44dc02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 -- 172.21.15.28:0/4208529781 shutdown_connections 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 --2- 172.21.15.28:0/4208529781 >> v2:172.21.15.138:3300/0 conn(0x7f44ec0d0a10 0x7f44ec06bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec15d460 0x7f44ec0d04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 --2- 172.21.15.28:0/4208529781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec145e00 0x7f44ec0cd8a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 -- 172.21.15.28:0/4208529781 >> 172.21.15.28:0/4208529781 conn(0x7f44ec0b2f40 msgr2=0x7f44ec0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 -- 172.21.15.28:0/4208529781 shutdown_connections 2024-09-10T10:51:43.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.321+0000 7f44f3200640 1 -- 172.21.15.28:0/4208529781 wait complete. 2024-09-10T10:51:43.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.323+0000 7f44f3200640 1 Processor -- start 2024-09-10T10:51:43.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.323+0000 7f44f3200640 1 -- start start 2024-09-10T10:51:43.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec0d0a10 0x7f44ec1f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44ec145e00 0x7f44ec0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec0ce550 con 0x7f44ec15d460 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec0ce270 con 0x7f44ec145e00 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0ce830 con 0x7f44ec0d0a10 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48166/0 (socket says 172.21.15.28:48166) 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f1200640 1 -- 172.21.15.28:0/2351612825 learned_addr learned my addr 172.21.15.28:0/2351612825 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44e3400640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44ec145e00 0x7f44ec0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44ebe00640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec0d0a10 0x7f44ec1f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.324+0000 7f44f1200640 1 -- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec0d0a10 msgr2=0x7f44ec1f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f1200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec0d0a10 0x7f44ec1f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f1200640 1 -- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44ec145e00 msgr2=0x7f44ec0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f1200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44ec145e00 0x7f44ec0b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f1200640 1 -- 172.21.15.28:0/2351612825 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec1f4590 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f1200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f44e4007e10 tx=0x7f44e400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e401a070 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e4003b90 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec0b66e0 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004030 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.325+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec0b6ca0 con 0x7f44ec15d460 2024-09-10T10:51:43.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.326+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec068c60 con 0x7f44ec15d460 2024-09-10T10:51:43.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.327+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44e401d070 con 0x7f44ec15d460 2024-09-10T10:51:43.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.328+0000 7f44e8c00640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.328+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7f44e408b550 con 0x7f44ec15d460 2024-09-10T10:51:43.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.329+0000 7f44ebe00640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.329+0000 7f44ebe00640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 0x7f44c806e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f44dc034000 tx=0x7f44dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.332+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44ec068c60 con 0x7f44ec15d460 2024-09-10T10:51:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.552+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44ec157070 con 0x7f44ec15d460 2024-09-10T10:51:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.553+0000 7f44e8c00640 1 -- 172.21.15.28:0/2351612825 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7f44e4010070 con 0x7f44ec15d460 2024-09-10T10:51:43.554 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:43.554 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":136,"btime":"2024-09-10T10:51:42:728611+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:42.728549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 136 2024-09-10T10:51:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 msgr2=0x7f44c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 0x7f44c806e2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f44dc034000 tx=0x7f44dc02e000 comp rx=0 tx=0).stop 2024-09-10T10:51:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 msgr2=0x7f44ec1f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f44e4007e10 tx=0x7f44e400d4a0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 shutdown_connections 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44c806be30 0x7f44c806e2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44ec15d460 0x7f44ec1f7c80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44ec145e00 0x7f44ec0b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 --2- 172.21.15.28:0/2351612825 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44ec0d0a10 0x7f44ec1f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 >> 172.21.15.28:0/2351612825 conn(0x7f44ec0b2f40 msgr2=0x7f44ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 shutdown_connections 2024-09-10T10:51:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.556+0000 7f44f3200640 1 -- 172.21.15.28:0/2351612825 wait complete. 2024-09-10T10:51:43.572 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:43.573 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:43.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.727+0000 7fe66d600640 1 Processor -- start 2024-09-10T10:51:43.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.727+0000 7fe66d600640 1 -- start start 2024-09-10T10:51:43.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe66d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6680d0dd0 0x7fe6680d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe66d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe66815e660 0x7fe66806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe66d600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6680cf000 con 0x7fe6680d1f90 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe66d600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe6680b37d0 con 0x7fe66815e660 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe66d600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe6680cf330 con 0x7fe6680d0dd0 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe66815e660 0x7fe66806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe66815e660 0x7fe66806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48168/0 (socket says 172.21.15.28:48168) 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 -- 172.21.15.28:0/1994752862 learned_addr learned my addr 172.21.15.28:0/1994752862 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe667e00640 1 --2- 172.21.15.28:0/1994752862 >> v2:172.21.15.138:3300/0 conn(0x7fe6680d0dd0 0x7fe6680d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 -- 172.21.15.28:0/1994752862 >> v2:172.21.15.138:3300/0 conn(0x7fe6680d0dd0 msgr2=0x7fe6680d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 --2- 172.21.15.28:0/1994752862 >> v2:172.21.15.138:3300/0 conn(0x7fe6680d0dd0 0x7fe6680d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.729+0000 7fe65fe00640 1 -- 172.21.15.28:0/1994752862 >> v1:172.21.15.138:6789/0 conn(0x7fe6680d1f90 legacy=0x7fe6680cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.730+0000 7fe65fe00640 1 -- 172.21.15.28:0/1994752862 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6680d16f0 con 0x7fe66815e660 2024-09-10T10:51:43.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.730+0000 7fe65fe00640 1 --2- 172.21.15.28:0/1994752862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe66815e660 0x7fe66806ccd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe6680b6fb0 tx=0x7fe65002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b867ba5066378b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe667400640 1 -- 172.21.15.28:0/1994752862 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650037070 con 0x7fe66815e660 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe667400640 1 -- 172.21.15.28:0/1994752862 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe650032440 con 0x7fe66815e660 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe667400640 1 -- 172.21.15.28:0/1994752862 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650032810 con 0x7fe66815e660 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 -- 172.21.15.28:0/1994752862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe66815e660 msgr2=0x7fe66806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 --2- 172.21.15.28:0/1994752862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe66815e660 0x7fe66806ccd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe6680b6fb0 tx=0x7fe65002a270 comp rx=0 tx=0).stop 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 -- 172.21.15.28:0/1994752862 shutdown_connections 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 --2- 172.21.15.28:0/1994752862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe66815e660 0x7fe66806ccd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 --2- 172.21.15.28:0/1994752862 >> v2:172.21.15.138:3300/0 conn(0x7fe6680d0dd0 0x7fe6680d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 -- 172.21.15.28:0/1994752862 >> 172.21.15.28:0/1994752862 conn(0x7fe66808bfc0 msgr2=0x7fe6680d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.731+0000 7fe66d600640 1 -- 172.21.15.28:0/1994752862 shutdown_connections 2024-09-10T10:51:43.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.732+0000 7fe66d600640 1 -- 172.21.15.28:0/1994752862 wait complete. 2024-09-10T10:51:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.733+0000 7fe66d600640 1 Processor -- start 2024-09-10T10:51:43.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.733+0000 7fe66d600640 1 -- start start 2024-09-10T10:51:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6680d0dd0 0x7fe668161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66815e660 0x7fe66815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6680d2f40 con 0x7fe6680d1f90 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6680d2c60 con 0x7fe66815e660 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe66d600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6680d3220 con 0x7fe6680d0dd0 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48182/0 (socket says 172.21.15.28:48182) 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.734+0000 7fe65fe00640 1 -- 172.21.15.28:0/885837318 learned_addr learned my addr 172.21.15.28:0/885837318 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe66cc00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66815e660 0x7fe66815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe667e00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6680d0dd0 0x7fe668161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 -- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6680d0dd0 msgr2=0x7fe668161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6680d0dd0 0x7fe668161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 -- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66815e660 msgr2=0x7fe66815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66815e660 0x7fe66815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 -- 172.21.15.28:0/885837318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668161860 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe65fe00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe65002a2b0 tx=0x7fe6500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650041020 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6500306c0 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe650037040 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.735+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6681e98a0 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.736+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6681e8dc0 con 0x7fe6680d1f90 2024-09-10T10:51:43.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.737+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6680d2420 con 0x7fe6680d1f90 2024-09-10T10:51:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.737+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe650044020 con 0x7fe6680d1f90 2024-09-10T10:51:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.738+0000 7fe664c00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 0x7fe64406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.738+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7fe6500abbe0 con 0x7fe6680d1f90 2024-09-10T10:51:43.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.739+0000 7fe667e00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 0x7fe64406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:43.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.739+0000 7fe667e00640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 0x7fe64406e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe658006fd0 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:43.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.743+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe65004b030 con 0x7fe6680d1f90 2024-09-10T10:51:43.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.966+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6680b6710 con 0x7fe6680d1f90 2024-09-10T10:51:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.967+0000 7fe664c00640 1 -- 172.21.15.28:0/885837318 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) ==== 95+0+4392 (secure 0 0 0) 0x7fe65007a020 con 0x7fe6680d1f90 2024-09-10T10:51:43.968 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:43.968 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":136,"btime":"2024-09-10T10:51:42:728611+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2888169760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2888169760},{"type":"v1","addr":"172.21.15.28:6835","nonce":2888169760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":122}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:42.728549+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 136 2024-09-10T10:51:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 msgr2=0x7fe64406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 0x7fe64406e320 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe658006fd0 tx=0x7fe658008040 comp rx=0 tx=0).stop 2024-09-10T10:51:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 msgr2=0x7fe6680cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe65002a2b0 tx=0x7fe6500046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 shutdown_connections 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe64406be60 0x7fe64406e320 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe66815e660 0x7fe66815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6680d1f90 0x7fe6680cdc70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 --2- 172.21.15.28:0/885837318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6680d0dd0 0x7fe668161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 >> 172.21.15.28:0/885837318 conn(0x7fe66808bfc0 msgr2=0x7fe6680d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 shutdown_connections 2024-09-10T10:51:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:43.969+0000 7fe66d600640 1 -- 172.21.15.28:0/885837318 wait complete. 2024-09-10T10:51:43.986 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds_snaptable' led to DAMAGED state 2024-09-10T10:51:43.986 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_inotable' 2024-09-10T10:51:43.986 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:43.986 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:44.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:44.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:44.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:44.022 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:44.097 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:44.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:44.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:44.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.293+0000 7fdcfa200640 1 Processor -- start 2024-09-10T10:51:44.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.293+0000 7fdcfa200640 1 -- start start 2024-09-10T10:51:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.294+0000 7fdcfa200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 0x7fdcf40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcfa200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 0x7fdcf406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcfa200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdcf4072d30 0x7fdcf406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcfa200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40728a0 con 0x7fdcf4072d30 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcfa200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf40cdbe0 con 0x7fdcf415b300 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcfa200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40cd900 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdcf4072d30 0x7fdcf406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdcf4072d30 0x7fdcf406f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41652/0 (socket says 172.21.15.28:41652) 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 -- 172.21.15.28:0/2128153650 learned_addr learned my addr 172.21.15.28:0/2128153650 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcebe00640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 0x7fdcf406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 -- 172.21.15.28:0/2128153650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 msgr2=0x7fdcf406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3400640 1 --2- 172.21.15.28:0/2128153650 >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 0x7fdcf40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 0x7fdcf406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 -- 172.21.15.28:0/2128153650 >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 msgr2=0x7fdcf40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 --2- 172.21.15.28:0/2128153650 >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 0x7fdcf40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3e00640 1 -- 172.21.15.28:0/2128153650 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf4072250 con 0x7fdcf4072d30 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcebe00640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 0x7fdcf406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.295+0000 7fdcf3400640 1 --2- 172.21.15.28:0/2128153650 >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 0x7fdcf40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.296+0000 7fdcf3e00640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf4072d30 0x7fdcf406f700 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4070250 tx=0x7fdcec02a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0530446b3456e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcf2a00640 1 -- 172.21.15.28:0/2128153650 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec036070 con 0x7fdcf4072d30 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcf2a00640 1 -- 172.21.15.28:0/2128153650 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcec031600 con 0x7fdcf4072d30 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcf2a00640 1 -- 172.21.15.28:0/2128153650 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0319d0 con 0x7fdcf4072d30 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 -- 172.21.15.28:0/2128153650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf4072d30 msgr2=0x7fdcf406f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf4072d30 0x7fdcf406f700 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4070250 tx=0x7fdcec02a690 comp rx=0 tx=0).stop 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 -- 172.21.15.28:0/2128153650 shutdown_connections 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf4072d30 0x7fdcf406f700 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 --2- 172.21.15.28:0/2128153650 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf40cd3f0 0x7fdcf406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 --2- 172.21.15.28:0/2128153650 >> v2:172.21.15.138:3300/0 conn(0x7fdcf415b300 0x7fdcf40cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 -- 172.21.15.28:0/2128153650 >> 172.21.15.28:0/2128153650 conn(0x7fdcf4067d70 msgr2=0x7fdcf406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 -- 172.21.15.28:0/2128153650 shutdown_connections 2024-09-10T10:51:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.297+0000 7fdcfa200640 1 -- 172.21.15.28:0/2128153650 wait complete. 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.299+0000 7fdcfa200640 1 Processor -- start 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.299+0000 7fdcfa200640 1 -- start start 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf4072d30 0x7fdcf4148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf415b300 0x7fdcf4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf40d0160 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf40cffe0 con 0x7fdcf415b300 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcfa200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf40b3c40 con 0x7fdcf4072d30 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41658/0 (socket says 172.21.15.28:41658) 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcebe00640 1 -- 172.21.15.28:0/261928602 learned_addr learned my addr 172.21.15.28:0/261928602 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcf3400640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf4072d30 0x7fdcf4148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.300+0000 7fdcf3e00640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf415b300 0x7fdcf4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 -- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf4072d30 msgr2=0x7fdcf4148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf4072d30 0x7fdcf4148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 -- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf415b300 msgr2=0x7fdcf4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf415b300 0x7fdcf4147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 -- 172.21.15.28:0/261928602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf414cab0 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcebe00640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c820 tx=0x7fdcdc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004230 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0043f0 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc007a70 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf4149390 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.301+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf4166260 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.302+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf40b2fc0 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.303+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdcdc007c10 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.304+0000 7fdceb400640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 0x7fdcd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.304+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7fdcdc08b500 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.305+0000 7fdcf3400640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 0x7fdcd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.305+0000 7fdcf3400640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 0x7fdcd006e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdce400aac0 tx=0x7fdce4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.308+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdcdc015020 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.528+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdcf4165d90 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.724+0000 7fdceb400640 1 -- 172.21.15.28:0/261928602 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v137) ==== 161+0+0 (secure 0 0 0) 0x7fdcdc0591d0 con 0x7fdcf40cd3f0 2024-09-10T10:51:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 msgr2=0x7fdcd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 0x7fdcd006e2f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdce400aac0 tx=0x7fdce4008040 comp rx=0 tx=0).stop 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 msgr2=0x7fdcf4163090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c820 tx=0x7fdcdc00cce0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 shutdown_connections 2024-09-10T10:51:44.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdcd006be30 0x7fdcd006e2f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf415b300 0x7fdcf4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcf40cd3f0 0x7fdcf4163090 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 --2- 172.21.15.28:0/261928602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcf4072d30 0x7fdcf4148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 >> 172.21.15.28:0/261928602 conn(0x7fdcf4067d70 msgr2=0x7fdcf4153050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 shutdown_connections 2024-09-10T10:51:44.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.727+0000 7fdcfa200640 1 -- 172.21.15.28:0/261928602 wait complete. 2024-09-10T10:51:44.745 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:44.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.901+0000 7f692e000640 1 Processor -- start 2024-09-10T10:51:44.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.901+0000 7f692e000640 1 -- start start 2024-09-10T10:51:44.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.902+0000 7f692e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f69280d4180 0x7f69280d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.902+0000 7f692e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f692815e6a0 0x7f692806d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692e000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69280d4aa0 con 0x7f692806dd60 2024-09-10T10:51:44.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692e000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f692806e170 con 0x7f692815e6a0 2024-09-10T10:51:44.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f69280cefb0 con 0x7f69280d4180 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f692815e6a0 0x7f692806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692d600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f692806dd60 0x7f69280d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40460/0 (socket says 172.21.15.28:40460) 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692d600640 1 -- 172.21.15.28:0/427022652 learned_addr learned my addr 172.21.15.28:0/427022652 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f692cc00640 1 --2- 172.21.15.28:0/427022652 >> v2:172.21.15.138:3300/0 conn(0x7f69280d4180 0x7f69280d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927e00640 1 -- 172.21.15.28:0/427022652 >> v2:172.21.15.138:3300/0 conn(0x7f69280d4180 msgr2=0x7f69280d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927e00640 1 --2- 172.21.15.28:0/427022652 >> v2:172.21.15.138:3300/0 conn(0x7f69280d4180 0x7f69280d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927e00640 1 -- 172.21.15.28:0/427022652 >> v1:172.21.15.28:6789/0 conn(0x7f692806dd60 legacy=0x7f69280d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927400640 1 -- 172.21.15.28:0/427022652 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933824135 0 0) 0x7f69280d4aa0 con 0x7f692806dd60 2024-09-10T10:51:44.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.903+0000 7f6927e00640 1 -- 172.21.15.28:0/427022652 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69280d2c20 con 0x7f692815e6a0 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.904+0000 7f6927e00640 1 --2- 172.21.15.28:0/427022652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f692806d820 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f69280b6ea0 tx=0x7f691402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd1da1477455ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f6927400640 1 -- 172.21.15.28:0/427022652 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914037070 con 0x7f692815e6a0 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f6927400640 1 -- 172.21.15.28:0/427022652 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914032440 con 0x7f692815e6a0 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f6927400640 1 -- 172.21.15.28:0/427022652 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691402ae60 con 0x7f692815e6a0 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 -- 172.21.15.28:0/427022652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 msgr2=0x7f692806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 --2- 172.21.15.28:0/427022652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f692806d820 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f69280b6ea0 tx=0x7f691402a110 comp rx=0 tx=0).stop 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 -- 172.21.15.28:0/427022652 shutdown_connections 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 --2- 172.21.15.28:0/427022652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f692806d820 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 --2- 172.21.15.28:0/427022652 >> v2:172.21.15.138:3300/0 conn(0x7f69280d4180 0x7f69280d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 -- 172.21.15.28:0/427022652 >> 172.21.15.28:0/427022652 conn(0x7f692809d6f0 msgr2=0x7f692808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.905+0000 7f692e000640 1 -- 172.21.15.28:0/427022652 shutdown_connections 2024-09-10T10:51:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.906+0000 7f692e000640 1 -- 172.21.15.28:0/427022652 wait complete. 2024-09-10T10:51:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.907+0000 7f692e000640 1 Processor -- start 2024-09-10T10:51:44.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.907+0000 7f692e000640 1 -- start start 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 0x7f69281509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 0x7f692815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f692806e950 con 0x7f692815e6a0 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f692806e670 con 0x7f69280d4180 2024-09-10T10:51:44.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f69280cebd0 con 0x7f692806dd60 2024-09-10T10:51:44.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41678/0 (socket says 172.21.15.28:41678) 2024-09-10T10:51:44.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.908+0000 7f692d600640 1 -- 172.21.15.28:0/438663758 learned_addr learned my addr 172.21.15.28:0/438663758 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f6927e00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 0x7f692815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692cc00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 0x7f69281509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 -- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 msgr2=0x7f69281509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 0x7f69281509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 -- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 msgr2=0x7f692815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:44.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 0x7f692815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 -- 172.21.15.28:0/438663758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692815a860 con 0x7f692815e6a0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f6927e00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 0x7f692815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692cc00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 0x7f69281509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692d600640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f692000c700 tx=0x7f692000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920018070 con 0x7f692815e6a0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692000ce10 con 0x7f692815e6a0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692000a860 con 0x7f692815e6a0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.909+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692815aae0 con 0x7f692815e6a0 2024-09-10T10:51:44.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.910+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692815b0a0 con 0x7f692815e6a0 2024-09-10T10:51:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.911+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692815b2e0 con 0x7f692815e6a0 2024-09-10T10:51:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.911+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f692001b070 con 0x7f692815e6a0 2024-09-10T10:51:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.912+0000 7f6924c00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 0x7f690806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:44.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.912+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7f692008a9e0 con 0x7f692815e6a0 2024-09-10T10:51:44.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.913+0000 7f692cc00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 0x7f690806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:44.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.914+0000 7f692cc00640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 0x7f690806e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f691c009af0 tx=0x7f691c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:44.916+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f692815a860 con 0x7f692815e6a0 2024-09-10T10:51:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.170+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f692806d820 con 0x7f692815e6a0 2024-09-10T10:51:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.730+0000 7f6924c00640 1 -- 172.21.15.28:0/438663758 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v138) ==== 101+0+0 (secure 0 0 0) 0x7f6920058d90 con 0x7f692815e6a0 2024-09-10T10:51:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 msgr2=0x7f690806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 0x7f690806e2f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f691c009af0 tx=0x7f691c008040 comp rx=0 tx=0).stop 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 msgr2=0x7f6928157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f692000c700 tx=0x7f692000cbc0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 shutdown_connections 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f690806be30 0x7f690806e2f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f692815e6a0 0x7f6928157570 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69280d4180 0x7f692815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 --2- 172.21.15.28:0/438663758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f692806dd60 0x7f69281509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 >> 172.21.15.28:0/438663758 conn(0x7f692809d6f0 msgr2=0x7f69280cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 shutdown_connections 2024-09-10T10:51:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.733+0000 7f692e000640 1 -- 172.21.15.28:0/438663758 wait complete. 2024-09-10T10:51:45.751 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.826+0000 7f08181fdf40 1 Processor -- start 2024-09-10T10:51:45.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.826+0000 7f08181fdf40 1 -- start start 2024-09-10T10:51:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 0x558482e2a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482e293e0 0x558482e2aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558482d9e530 con 0x558482e2c9c0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558482cfbb30 con 0x558482d9d8e0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f08181fdf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558482e2bee0 con 0x558482e293e0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41690/0 (socket says 172.21.15.28:41690) 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 -- 172.21.15.28:0/2521380807 learned_addr learned my addr 172.21.15.28:0/2521380807 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815400640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 0x558482e2a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0814a00640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482e293e0 0x558482e2aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 -- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482e293e0 msgr2=0x558482e2aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482e293e0 0x558482e2aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 -- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 msgr2=0x558482e2a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 0x558482e2a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815e00640 1 -- 172.21.15.28:0/2521380807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558482d9e2a0 con 0x558482e2c9c0 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.827+0000 7f0815400640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 0x558482e2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f0815e00640 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0810001620 tx=0x7f0810001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44feb3f347b6aa9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f080f400640 1 -- 172.21.15.28:0/2521380807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810017070 con 0x558482e2c9c0 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f080f400640 1 -- 172.21.15.28:0/2521380807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081000f710 con 0x558482e2c9c0 2024-09-10T10:51:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f080f400640 1 -- 172.21.15.28:0/2521380807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081000fb00 con 0x558482e2c9c0 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 -- 172.21.15.28:0/2521380807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 msgr2=0x558482d9f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0810001620 tx=0x7f0810001ae0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 -- 172.21.15.28:0/2521380807 shutdown_connections 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482d9f9c0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482e293e0 0x558482e2aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2521380807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482d9d8e0 0x558482e2a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 -- 172.21.15.28:0/2521380807 >> 172.21.15.28:0/2521380807 conn(0x558482da20a0 msgr2=0x558482e29850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 -- 172.21.15.28:0/2521380807 shutdown_connections 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.828+0000 7f08181fdf40 1 -- 172.21.15.28:0/2521380807 wait complete. 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.829+0000 7f08181fdf40 1 Processor -- start 2024-09-10T10:51:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.829+0000 7f08181fdf40 1 -- start start 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482d9d8e0 0x558482e2f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482e293e0 0x558482e18510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558482da0b70 con 0x558482e2c9c0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558482d858b0 con 0x558482e293e0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f08181fdf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558482d55b90 con 0x558482d9d8e0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f0815e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f0815e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41692/0 (socket says 172.21.15.28:41692) 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 learned_addr learned my addr 172.21.15.28:0/2467493534 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482e293e0 0x558482e18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.830+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482d9d8e0 0x558482e2f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482d9d8e0 msgr2=0x558482e2f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558482d9d8e0 0x558482e2f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482e293e0 msgr2=0x558482e18510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558482e293e0 0x558482e18510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558482e20c50 con 0x558482e2c9c0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f0815e00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f081000f530 tx=0x7f0810001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f080e000640 1 -- 172.21.15.28:0/2467493534 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810023070 con 0x558482e2c9c0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f080e000640 1 -- 172.21.15.28:0/2467493534 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0810011640 con 0x558482e2c9c0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f080e000640 1 -- 172.21.15.28:0/2467493534 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810017040 con 0x558482e2c9c0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.831+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558482e2ac80 con 0x558482e2c9c0 2024-09-10T10:51:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.832+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558482e1cca0 con 0x558482e2c9c0 2024-09-10T10:51:45.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.834+0000 7f080e000640 1 -- 172.21.15.28:0/2467493534 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0810026070 con 0x558482e2c9c0 2024-09-10T10:51:45.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.834+0000 7f080e000640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 0x7f07f806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.834+0000 7f080e000640 1 -- 172.21.15.28:0/2467493534 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7f081008ad60 con 0x558482e2c9c0 2024-09-10T10:51:45.836 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.835+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 0x558482e21c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.835+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482d85380 con 0x558482e2fe90 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.835+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 0x7f07f806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.835+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 0x558482e21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.836+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 0x7f07f806ccd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0808057000 tx=0x7f0808052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.836+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 0x558482e21c00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.837+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v109'257 uv231 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.838+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482e22140 con 0x558482e2fe90 2024-09-10T10:51:45.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.839+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v109'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.839+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482e225a0 con 0x558482e2fe90 2024-09-10T10:51:45.841 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.841+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v109'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.841+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecf810 con 0x558482e2fe90 2024-09-10T10:51:45.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.843+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v109'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.843+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecfbb0 con 0x558482e2fe90 2024-09-10T10:51:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.844+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v109'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.844+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ece800 con 0x558482e2fe90 2024-09-10T10:51:45.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.846+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v109'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.846+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecebf0 con 0x558482e2fe90 2024-09-10T10:51:45.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.847+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v109'263 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.848+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ecefe0 con 0x558482e2fe90 2024-09-10T10:51:45.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.849+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v109'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.849+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecefe0 con 0x558482e2fe90 2024-09-10T10:51:45.851 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:45.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.851+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v109'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.851+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.852+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v109'266 uv266 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.852+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.854+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v109'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.854+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.856+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v109'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.856+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.857+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v109'269 uv244 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.857+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.859+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v109'270 uv270 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.859+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.861+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v109'271 uv271 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.862 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:45.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.861+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.862+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v109'272 uv272 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.862+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.864+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v109'273 uv273 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.864+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecef90 con 0x558482e2fe90 2024-09-10T10:51:45.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.865+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v109'274 uv274 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.866+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.867+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v109'275 uv275 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.867+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.869+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v109'276 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.869+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.871+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v109'277 uv277 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.871+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.872+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v109'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.873 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:45.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.872+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed0f60 con 0x558482e2fe90 2024-09-10T10:51:45.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.873+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v109'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.873+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed0f60 con 0x558482e2fe90 2024-09-10T10:51:45.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.875+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v109'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.875+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.876+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v109'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.876+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.878+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v109'282 uv256 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.878+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.879+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v109'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.879+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.881 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:45.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.881+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v109'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.881+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed0f60 con 0x558482e2fe90 2024-09-10T10:51:45.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.882+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v109'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.882+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed0f60 con 0x558482e2fe90 2024-09-10T10:51:45.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.884+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v109'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.884+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ecff50 con 0x558482e2fe90 2024-09-10T10:51:45.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.885+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v109'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482e2fe90 2024-09-10T10:51:45.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.885+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558482ed0460 0x558482ed2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.885+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed2d90 con 0x558482ed0460 2024-09-10T10:51:45.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.886+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558482ed0460 0x558482ed2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.886+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558482ed0460 0x558482ed2850 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.888+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v109'198 uv181 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.888+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ed3130 con 0x558482ed0460 2024-09-10T10:51:45.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.889+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v109'199 uv199 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.889+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed34d0 con 0x558482ed0460 2024-09-10T10:51:45.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.891+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v109'200 uv200 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:45.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.891+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed3870 con 0x558482ed0460 2024-09-10T10:51:45.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.892+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v109'201 uv201 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.892+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed3c10 con 0x558482ed0460 2024-09-10T10:51:45.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.894+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v109'202 uv202 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.894+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed3fb0 con 0x558482ed0460 2024-09-10T10:51:45.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.895+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v109'203 uv186 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.895+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.896+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v109'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.896+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.898+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v109'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:45.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.898+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.899+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v109'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.899+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.900+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v109'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.901+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.902+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v109'208 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.902+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.903+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v109'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.903+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.904+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v109'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:45.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.904+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed8a40 con 0x558482ed0460 2024-09-10T10:51:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.906+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v109'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.906+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed8a40 con 0x558482ed0460 2024-09-10T10:51:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.907+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v109'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.907+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.909+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v109'213 uv197 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.909+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.911+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v109'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.911+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.912+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v109'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.912+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed5360 con 0x558482ed0460 2024-09-10T10:51:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.913+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v109'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.913+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed5360 con 0x558482ed0460 2024-09-10T10:51:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.915+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v109'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.915+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed4350 con 0x558482ed0460 2024-09-10T10:51:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.916+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v109'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482ed0460 2024-09-10T10:51:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.916+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558482ed4860 0x558482ed6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.916+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed7190 con 0x558482ed4860 2024-09-10T10:51:45.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.917+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558482ed4860 0x558482ed6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.917+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558482ed4860 0x558482ed6c50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.919+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v109'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.919+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ed7210 con 0x558482ed4860 2024-09-10T10:51:45.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.921+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v109'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.921+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed7600 con 0x558482ed4860 2024-09-10T10:51:45.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.922+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v109'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:45.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.922+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed8610 con 0x558482ed4860 2024-09-10T10:51:45.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.923+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v109'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.923+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed8610 con 0x558482ed4860 2024-09-10T10:51:45.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.925+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v109'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.925+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ed7600 con 0x558482ed4860 2024-09-10T10:51:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.926+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v109'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.926+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558482ed7c80 0x558482eda070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.926+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482eda660 con 0x558482ed7c80 2024-09-10T10:51:45.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.926+0000 7f0815e00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558482ed7c80 0x558482eda070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.927+0000 7f0815e00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558482ed7c80 0x558482eda070 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.928+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v109'41 uv40 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.928+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edab70 con 0x558482ed7c80 2024-09-10T10:51:45.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.929+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v109'42 uv42 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.929+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb070 con 0x558482ed7c80 2024-09-10T10:51:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:45.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.930+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v109'43 uv43 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.931+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb930 con 0x558482ed7c80 2024-09-10T10:51:45.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.932+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v109'44 uv44 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.932+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edbcd0 con 0x558482ed4860 2024-09-10T10:51:45.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.933+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v109'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.933+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edbcd0 con 0x558482ed4860 2024-09-10T10:51:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.934+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v109'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.934+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edbcd0 con 0x558482ed4860 2024-09-10T10:51:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.935+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v109'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.935+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edcce0 con 0x558482ed4860 2024-09-10T10:51:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.936+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v109'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.936+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edcce0 con 0x558482ed4860 2024-09-10T10:51:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.937+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v109'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.937+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edbcd0 con 0x558482ed4860 2024-09-10T10:51:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.939+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v109'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.939+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edbcd0 con 0x558482ed7c80 2024-09-10T10:51:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.940+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v109'94 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.940+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edc070 con 0x558482ed7c80 2024-09-10T10:51:45.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.941+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v109'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.941+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc410 con 0x558482ed7c80 2024-09-10T10:51:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:45.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.943+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v109'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.943+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edd7c0 con 0x558482ed7c80 2024-09-10T10:51:45.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.944+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v109'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.944+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edd7c0 con 0x558482ed7c80 2024-09-10T10:51:45.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.946+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v109'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.946+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.947+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v109'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.947+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.949+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v109'100 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.949+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.950+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v109'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.950+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.952 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:45.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.951+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v109'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.951+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.953+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v109'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.953+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed7c80 2024-09-10T10:51:45.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.954+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v109'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.954+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed4860 2024-09-10T10:51:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.955+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v109'43 uv43 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:45.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.955+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed4860 2024-09-10T10:51:45.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.956+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [write 0~70] v109'44 uv44 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.956+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edc7b0 con 0x558482ed4860 2024-09-10T10:51:45.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.958+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [omap-set-header] v109'45 uv45 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ed4860 2024-09-10T10:51:45.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.958+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558482edccc0 0x558482edf150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.958+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:83 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edf780 con 0x558482edccc0 2024-09-10T10:51:45.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.958+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558482edccc0 0x558482edf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.958+0000 7f0815400640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558482edccc0 0x558482edf150 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.960+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(83 605.00000000 [create] v109'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.960+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:84 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.961+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(84 605.00000000 [delete] v109'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.961+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:45.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.962+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(85 605.00000000 [create] v109'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.962+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee0b60 con 0x558482edccc0 2024-09-10T10:51:45.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.963+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(86 605.00000000 [setxattr (30)] v109'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.963+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee0b60 con 0x558482edccc0 2024-09-10T10:51:45.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.965+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(87 605.00000000 [setxattr (62)] v109'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.965+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.966+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(88 605.00000000 [omap-set-header] v109'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.966+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.967+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(89 mds0_inotable [create] v109'84 uv78 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.967+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.969+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(90 mds0_inotable [delete] v109'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.969+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfb50 con 0x558482edccc0 2024-09-10T10:51:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:45.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.970+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(91 mds0_inotable [create] v109'86 uv86 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.970+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edff40 con 0x558482edccc0 2024-09-10T10:51:45.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.971+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(92 mds0_inotable [write 0~34] v109'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.971+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfbf0 con 0x558482edccc0 2024-09-10T10:51:45.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.972+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v109'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.972+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfbf0 con 0x558482edccc0 2024-09-10T10:51:45.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.973+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(94 100.00000000 [create] v109'89 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.973+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edfbf0 con 0x558482edccc0 2024-09-10T10:51:45.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.975+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(95 100.00000000 [delete] v109'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.975+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfbf0 con 0x558482edccc0 2024-09-10T10:51:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:45.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.976+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(96 100.00000000 [create] v109'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.976+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edfbf0 con 0x558482edccc0 2024-09-10T10:51:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.977+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(97 100.00000000 [omap-set-header] v109'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.977+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb570 con 0x558482edccc0 2024-09-10T10:51:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.978+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v109'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.978+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb530 con 0x558482ed7c80 2024-09-10T10:51:45.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.979+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v109'40 uv39 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.979+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edb530 con 0x558482ed7c80 2024-09-10T10:51:45.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.981+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v109'41 uv41 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.981+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb530 con 0x558482ed7c80 2024-09-10T10:51:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.982+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v109'42 uv42 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.982+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb530 con 0x558482ed7c80 2024-09-10T10:51:45.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.984+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v109'43 uv43 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.984+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb530 con 0x558482ed7c80 2024-09-10T10:51:45.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.985+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v109'51 uv50 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.985+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482edb510 con 0x558482ed7c80 2024-09-10T10:51:45.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.986+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v109'52 uv52 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.986+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb510 con 0x558482ed7c80 2024-09-10T10:51:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:45.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.987+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v109'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.987+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb4f0 con 0x558482ed7c80 2024-09-10T10:51:45.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.988+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v109'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.988+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb4f0 con 0x558482ed7c80 2024-09-10T10:51:45.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.989+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v109'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.989+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482edb4f0 con 0x558482ed7c80 2024-09-10T10:51:45.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.990+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v109'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0810025030 con 0x558482ed7c80 2024-09-10T10:51:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.991+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558482ee0a40 0x558482ee3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.991+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482ee0a40 2024-09-10T10:51:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.991+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558482ee0a40 0x558482ee3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.991+0000 7f0814a00640 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558482ee0a40 0x558482ee3190 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:45.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.992+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v109'42 uv41 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ee0a40 2024-09-10T10:51:45.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.992+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482ee0a40 2024-09-10T10:51:45.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.994+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v109'43 uv43 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ee0a40 2024-09-10T10:51:45.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.994+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482ee0a40 2024-09-10T10:51:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:45.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.995+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v109'44 uv44 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ee0a40 2024-09-10T10:51:45.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.995+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482ee0a40 2024-09-10T10:51:45.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.996+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v109'45 uv45 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ee0a40 2024-09-10T10:51:45.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.996+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482ee0a40 2024-09-10T10:51:45.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.997+0000 7f0814a00640 1 -- 172.21.15.28:0/2467493534 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v109'46 uv46 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f08040098d0 con 0x558482ee0a40 2024-09-10T10:51:45.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.997+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:45.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.998+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v109'107 uv99 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:45.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.998+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:45.999+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v109'108 uv108 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.000+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.001+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v109'109 uv109 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:46.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.001+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.002+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v109'110 uv110 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.002+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.003+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v109'111 uv111 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.003+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.004+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v109'112 uv106 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.004+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.005+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v109'113 uv113 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.005+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:46.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.007+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v109'114 uv114 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.007+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee46e0 con 0x558482edccc0 2024-09-10T10:51:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.008+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v109'115 uv115 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.008+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee46e0 con 0x558482edccc0 2024-09-10T10:51:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.009+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v109'116 uv116 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.009+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee36d0 con 0x558482edccc0 2024-09-10T10:51:46.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.010+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v109'117 uv117 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.010+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) -- 0x558482ee3b80 con 0x558482edccc0 2024-09-10T10:51:46.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f0815400640 1 -- 172.21.15.28:0/2467493534 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v109'118 uv118 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0808057030 con 0x558482edccc0 2024-09-10T10:51:46.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558482ed4860 msgr2=0x558482ed6c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558482ed4860 0x558482ed6c50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558482edccc0 msgr2=0x558482edf150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558482edccc0 0x558482edf150 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558482ee0a40 msgr2=0x558482ee3190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558482ee0a40 0x558482ee3190 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558482ed7c80 msgr2=0x558482eda070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558482ed7c80 0x558482eda070 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558482ed0460 msgr2=0x558482ed2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 reap_dead start 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558482ed0460 0x558482ed2850 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f0815e00640 1 -- 172.21.15.28:0/2467493534 reap_dead start 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 msgr2=0x558482e21c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.011+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 0x558482e21c00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 msgr2=0x7f07f806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 0x7f07f806ccd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0808057000 tx=0x7f0808052000 comp rx=0 tx=0).stop 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 msgr2=0x558482e18a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f081000f530 tx=0x7f0810001250 comp rx=0 tx=0).stop 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 shutdown_connections 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558482e2fe90 0x558482e21c00 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f07f806a810 0x7f07f806ccd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 --2- 172.21.15.28:0/2467493534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558482e2c9c0 0x558482e18a50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 >> 172.21.15.28:0/2467493534 conn(0x558482da20a0 msgr2=0x558482da2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 shutdown_connections 2024-09-10T10:51:46.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.012+0000 7f08181fdf40 1 -- 172.21.15.28:0/2467493534 wait complete. 2024-09-10T10:51:46.017 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_inotable 2024-09-10T10:51:46.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.132+0000 7fe54c6fcf40 1 Processor -- start 2024-09-10T10:51:46.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.132+0000 7fe54c6fcf40 1 -- start start 2024-09-10T10:51:46.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 0x55e8598fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8599a34b0 0x55e8598cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e859875b30 con 0x55e8598d15a0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e859916dc0 con 0x55e8599a34b0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e8599a6770 con 0x55e8599172d0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe549800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe549800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41694/0 (socket says 172.21.15.28:41694) 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe549800640 1 -- 172.21.15.28:0/1137211630 learned_addr learned my addr 172.21.15.28:0/1137211630 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe54a200640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8599a34b0 0x55e8598cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.133+0000 7fe548e00640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 0x55e8598fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 -- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 msgr2=0x55e8598fa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 0x55e8598fa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 -- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8599a34b0 msgr2=0x55e8598cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8599a34b0 0x55e8598cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 -- 172.21.15.28:0/1137211630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8598fb760 con 0x55e8598d15a0 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe548e00640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 0x55e8598fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:46.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe549800640 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bd83f5714f28b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe543400640 1 -- 172.21.15.28:0/1137211630 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c063070 con 0x55e8598d15a0 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe543400640 1 -- 172.21.15.28:0/1137211630 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c055920 con 0x55e8598d15a0 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.134+0000 7fe543400640 1 -- 172.21.15.28:0/1137211630 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c055cf0 con 0x55e8598d15a0 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/1137211630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 msgr2=0x55e859a34130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe53c000c00 tx=0x7fe53c051000 comp rx=0 tx=0).stop 2024-09-10T10:51:46.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/1137211630 shutdown_connections 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8599a34b0 0x55e8598cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599172d0 0x55e8598fa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/1137211630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8598d15a0 0x55e859a34130 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/1137211630 >> 172.21.15.28:0/1137211630 conn(0x55e8598fbb90 msgr2=0x55e8598fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/1137211630 shutdown_connections 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.135+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/1137211630 wait complete. 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.136+0000 7fe54c6fcf40 1 Processor -- start 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.136+0000 7fe54c6fcf40 1 -- start start 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.136+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8598d15a0 0x55e859a35250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 0x55e85999e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54c6fcf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599a34b0 0x55e85999ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e8599a6ca0 con 0x55e8599172d0 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8599a69c0 con 0x55e8598d15a0 2024-09-10T10:51:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54c6fcf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e8598fec00 con 0x55e8599a34b0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 0x55e85999e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 0x55e85999e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41696/0 (socket says 172.21.15.28:41696) 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 -- 172.21.15.28:0/3390615280 learned_addr learned my addr 172.21.15.28:0/3390615280 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe54a200640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599a34b0 0x55e85999ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe549800640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8598d15a0 0x55e859a35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 -- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599a34b0 msgr2=0x55e85999ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e8599a34b0 0x55e85999ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 -- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8598d15a0 msgr2=0x55e859a35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8598d15a0 0x55e859a35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.137+0000 7fe548e00640 1 -- 172.21.15.28:0/3390615280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8599a0cf0 con 0x55e8599172d0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe548e00640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 0x55e85999e690 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ed0 tx=0x7fe538005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe542000640 1 -- 172.21.15.28:0/3390615280 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538014400 con 0x55e8599172d0 2024-09-10T10:51:46.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe542000640 1 -- 172.21.15.28:0/3390615280 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538019070 con 0x55e8599172d0 2024-09-10T10:51:46.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe542000640 1 -- 172.21.15.28:0/3390615280 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5380147a0 con 0x55e8599172d0 2024-09-10T10:51:46.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e8598f9550 con 0x55e8599172d0 2024-09-10T10:51:46.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.138+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8599a1290 con 0x55e8599172d0 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.140+0000 7fe542000640 1 -- 172.21.15.28:0/3390615280 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe538013070 con 0x55e8599172d0 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.140+0000 7fe542000640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe52c06a810 0x7fe52c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.141+0000 7fe542000640 1 -- 172.21.15.28:0/3390615280 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7fe538089d60 con 0x55e8599172d0 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.141+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e859a35790 0x55e85999f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.141+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e109) -- 0x55e8598cf430 con 0x55e859a35790 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.141+0000 7fe549800640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe52c06a810 0x7fe52c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.141+0000 7fe54a200640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e859a35790 0x55e85999f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.142+0000 7fe54a200640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e859a35790 0x55e85999f7e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.142+0000 7fe549800640 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe52c06a810 0x7fe52c06ccd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00a440 tx=0x7fe53c053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54a200640 1 -- 172.21.15.28:0/3390615280 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [delete] v109'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe544001650 con 0x55e859a35790 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e859a35790 msgr2=0x55e85999f7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e859a35790 0x55e85999f7e0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe52c06a810 msgr2=0x7fe52c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe52c06a810 0x7fe52c06ccd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00a440 tx=0x7fe53c053000 comp rx=0 tx=0).stop 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 msgr2=0x55e85999e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54c6fcf40 1 --2- 172.21.15.28:0/3390615280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e8599172d0 0x55e85999e690 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe538009ed0 tx=0x7fe538005f60 comp rx=0 tx=0).stop 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.143+0000 7fe54a200640 1 -- 172.21.15.28:0/3390615280 reap_dead start 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.144+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 shutdown_connections 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.144+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 >> 172.21.15.28:0/3390615280 conn(0x55e8598fbb90 msgr2=0x55e8598fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.144+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 shutdown_connections 2024-09-10T10:51:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.144+0000 7fe54c6fcf40 1 -- 172.21.15.28:0/3390615280 wait complete. 2024-09-10T10:51:46.149 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:46.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.307+0000 7fc8eb200640 1 Processor -- start 2024-09-10T10:51:46.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.307+0000 7fc8eb200640 1 -- start start 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8eb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8ec0d02d0 0x7fc8ec06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8eb200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ec0d3140 con 0x7fc8ec0d0a90 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8eb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8ec163170 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8eb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8ec0d12c0 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8e9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc8ec0d0a90 0x7fc8ec162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40480/0 (socket says 172.21.15.28:40480) 2024-09-10T10:51:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.308+0000 7fc8e9e00640 1 -- 172.21.15.28:0/3901835023 learned_addr learned my addr 172.21.15.28:0/3901835023 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:46.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e9400640 1 --2- 172.21.15.28:0/3901835023 >> v2:172.21.15.138:3300/0 conn(0x7fc8ec0d02d0 0x7fc8ec06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021720048 0 0) 0x7fc8dc001110 con 0x7fc8ec0d0a90 2024-09-10T10:51:46.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8d0003520 con 0x7fc8ec0d0a90 2024-09-10T10:51:46.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e9400640 1 -- 172.21.15.28:0/3901835023 >> v1:172.21.15.138:6789/0 conn(0x7fc8ec0d0e70 legacy=0x7fc8ec162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2927672964 0 0) 0x7fc8d0003520 con 0x7fc8ec0d0a90 2024-09-10T10:51:46.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e9400640 1 -- 172.21.15.28:0/3901835023 >> v1:172.21.15.28:6789/0 conn(0x7fc8ec0d0a90 legacy=0x7fc8ec162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.309+0000 7fc8e9400640 1 -- 172.21.15.28:0/3901835023 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec06dd20 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.310+0000 7fc8e9400640 1 --2- 172.21.15.28:0/3901835023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d02d0 0x7fc8ec06e9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80097a0 tx=0x7fc8d802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d330b3f2f82cc65c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8037070 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8d802abc0 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8e8a00640 1 -- 172.21.15.28:0/3901835023 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc001840 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 -- 172.21.15.28:0/3901835023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d02d0 msgr2=0x7fc8ec06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3901835023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d02d0 0x7fc8ec06e9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80097a0 tx=0x7fc8d802a130 comp rx=0 tx=0).stop 2024-09-10T10:51:46.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 -- 172.21.15.28:0/3901835023 shutdown_connections 2024-09-10T10:51:46.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3901835023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d02d0 0x7fc8ec06e9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 -- 172.21.15.28:0/3901835023 >> 172.21.15.28:0/3901835023 conn(0x7fc8ec08bfc0 msgr2=0x7fc8ec0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 -- 172.21.15.28:0/3901835023 shutdown_connections 2024-09-10T10:51:46.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.311+0000 7fc8eb200640 1 -- 172.21.15.28:0/3901835023 wait complete. 2024-09-10T10:51:46.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.313+0000 7fc8eb200640 1 Processor -- start 2024-09-10T10:51:46.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.313+0000 7fc8eb200640 1 -- start start 2024-09-10T10:51:46.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8ec0d02d0 0x7fc8ec15f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d0a90 0x7fc8ec0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d17d0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d14f0 con 0x7fc8ec0d0a90 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8eb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d5180 con 0x7fc8ec0d02d0 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8ea800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8ea800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41708/0 (socket says 172.21.15.28:41708) 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8ea800640 1 -- 172.21.15.28:0/3099798250 learned_addr learned my addr 172.21.15.28:0/3099798250 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8e9e00640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8ec0d02d0 0x7fc8ec15f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.314+0000 7fc8e9400640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d0a90 0x7fc8ec0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 -- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8ec0d02d0 msgr2=0x7fc8ec15f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8ec0d02d0 0x7fc8ec15f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 -- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d0a90 msgr2=0x7fc8ec0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d0a90 0x7fc8ec0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 -- 172.21.15.28:0/3099798250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1f4330 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8ea800640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c6b0 tx=0x7fc8e400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4003b70 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e4003d30 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4005310 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec0cc640 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.315+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1f48a0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.316+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec06e9c0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.317+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc8e40054b0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.318+0000 7fc8e2000640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 0x7fc8cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.318+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 7078+0+0 (secure 0 0 0) 0x7fc8e408b0e0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.319+0000 7fc8e9e00640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 0x7fc8cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.319+0000 7fc8e9e00640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 0x7fc8cc06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005880 tx=0x7fc8dc001270 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.322+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8e4058ed0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.542+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc8ec06e6a0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.675+0000 7fc8e2000640 1 -- 172.21.15.28:0/3099798250 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v139) ==== 162+0+0 (secure 0 0 0) 0x7fc8e405d1d0 con 0x7fc8ec0d0e70 2024-09-10T10:51:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 msgr2=0x7fc8cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 0x7fc8cc06e2f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005880 tx=0x7fc8dc001270 comp rx=0 tx=0).stop 2024-09-10T10:51:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 msgr2=0x7fc8ec15ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c6b0 tx=0x7fc8e400cb70 comp rx=0 tx=0).stop 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 shutdown_connections 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc8cc06be30 0x7fc8cc06e2f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc8ec0d0e70 0x7fc8ec15ef70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8ec0d0a90 0x7fc8ec0cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 --2- 172.21.15.28:0/3099798250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8ec0d02d0 0x7fc8ec15f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 >> 172.21.15.28:0/3099798250 conn(0x7fc8ec08bfc0 msgr2=0x7fc8ec0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 shutdown_connections 2024-09-10T10:51:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.678+0000 7fc8eb200640 1 -- 172.21.15.28:0/3099798250 wait complete. 2024-09-10T10:51:46.682 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:46.681+0000 7f4dd3600640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-10T10:51:46.682 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:51:46.681+0000 7f4dd3600640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-10T10:51:46.682 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:46.682+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:46.683 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:46.682+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:46.698 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:46.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.855+0000 7f25ab200640 1 Processor -- start 2024-09-10T10:51:46.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.855+0000 7f25ab200640 1 -- start start 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25ab200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 0x7f25a40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25ab200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a40cf000 con 0x7f25a415e660 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25ab200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a40d16f0 con 0x7f25a40d1f90 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25ab200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25a406d3e0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25a9400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f25a415e660 0x7f25a406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40492/0 (socket says 172.21.15.28:40492) 2024-09-10T10:51:46.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25a9400640 1 -- 172.21.15.28:0/3030879502 learned_addr learned my addr 172.21.15.28:0/3030879502 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:46.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.856+0000 7f25a9e00640 1 --2- 172.21.15.28:0/3030879502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 0x7f25a40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 477922811 0 0) 0x7f25a40cf000 con 0x7f25a415e660 2024-09-10T10:51:46.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f258c003520 con 0x7f25a415e660 2024-09-10T10:51:46.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85715422 0 0) 0x7f25a40d16f0 con 0x7f25a40d1f90 2024-09-10T10:51:46.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25a40cf000 con 0x7f25a40d1f90 2024-09-10T10:51:46.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a9e00640 1 -- 172.21.15.28:0/3030879502 >> v1:172.21.15.138:6789/0 conn(0x7f25a40d1f90 legacy=0x7f25a40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a9e00640 1 -- 172.21.15.28:0/3030879502 >> v1:172.21.15.28:6789/0 conn(0x7f25a415e660 legacy=0x7f25a406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.857+0000 7f25a9e00640 1 -- 172.21.15.28:0/3030879502 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a40d29a0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.858+0000 7f25a9e00640 1 --2- 172.21.15.28:0/3030879502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 0x7f25a40d11b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25980097d0 tx=0x7f259802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9850182303b02b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.858+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598037070 con 0x7f25a40d0dd0 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.858+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2598032440 con 0x7f25a40d0dd0 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.858+0000 7f25a8a00640 1 -- 172.21.15.28:0/3030879502 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598032810 con 0x7f25a40d0dd0 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 -- 172.21.15.28:0/3030879502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 msgr2=0x7f25a40d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 --2- 172.21.15.28:0/3030879502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 0x7f25a40d11b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25980097d0 tx=0x7f259802a210 comp rx=0 tx=0).stop 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 -- 172.21.15.28:0/3030879502 shutdown_connections 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 --2- 172.21.15.28:0/3030879502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d0dd0 0x7f25a40d11b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 -- 172.21.15.28:0/3030879502 >> 172.21.15.28:0/3030879502 conn(0x7f25a408bfc0 msgr2=0x7f25a40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:46.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 -- 172.21.15.28:0/3030879502 shutdown_connections 2024-09-10T10:51:46.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.859+0000 7f25ab200640 1 -- 172.21.15.28:0/3030879502 wait complete. 2024-09-10T10:51:46.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.861+0000 7f25ab200640 1 Processor -- start 2024-09-10T10:51:46.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.861+0000 7f25ab200640 1 -- start start 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d1f90 0x7f25a40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 0x7f25a41e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f25a40cf330 con 0x7f25a40d0dd0 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25a40d2f40 con 0x7f25a415e660 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25ab200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25a40d3220 con 0x7f25a40d1f90 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41710/0 (socket says 172.21.15.28:41710) 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 -- 172.21.15.28:0/2955401122 learned_addr learned my addr 172.21.15.28:0/2955401122 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9400640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d1f90 0x7f25a40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25aa800640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 0x7f25a41e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 -- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d1f90 msgr2=0x7f25a40ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d1f90 0x7f25a40ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 -- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 msgr2=0x7f25a41e41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 0x7f25a41e41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25a9e00640 1 -- 172.21.15.28:0/2955401122 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a41e49e0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.862+0000 7f25aa800640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 0x7f25a41e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f25a9e00640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2598035040 tx=0x7f25980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598041020 con 0x7f25a40d0dd0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259803d720 con 0x7f25a40d0dd0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a41e4cc0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598037040 con 0x7f25a40d0dd0 2024-09-10T10:51:46.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.863+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a41e46f0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.864+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a40d2440 con 0x7f25a40d0dd0 2024-09-10T10:51:46.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.865+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2598044020 con 0x7f25a40d0dd0 2024-09-10T10:51:46.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.866+0000 7f259e000640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:46.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.866+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7f25980abbf0 con 0x7f25a40d0dd0 2024-09-10T10:51:46.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.866+0000 7f25a9400640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:46.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.867+0000 7f25a9400640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f25a41f2050 tx=0x7f2594009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:46.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:46.871+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f259804b030 con 0x7f25a40d0dd0 2024-09-10T10:51:47.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.092+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25a40d1b20 con 0x7f25a40d0dd0 2024-09-10T10:51:47.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.093+0000 7f259e000640 1 -- 172.21.15.28:0/2955401122 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+3566 (secure 0 0 0) 0x7f2598079f60 con 0x7f25a40d0dd0 2024-09-10T10:51:47.094 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:47.094 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":140,"btime":"2024-09-10T10:51:46:682646+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:46.682576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:47.094 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 140 2024-09-10T10:51:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 msgr2=0x7f258806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f25a41f2050 tx=0x7f2594009040 comp rx=0 tx=0).stop 2024-09-10T10:51:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 msgr2=0x7f25a4153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2598035040 tx=0x7f25980046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 shutdown_connections 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a415e660 0x7f25a41e41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25a40d1f90 0x7f25a40ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 --2- 172.21.15.28:0/2955401122 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25a40d0dd0 0x7f25a4153ae0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 >> 172.21.15.28:0/2955401122 conn(0x7f25a408bfc0 msgr2=0x7f25a40b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 shutdown_connections 2024-09-10T10:51:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.095+0000 7f25ab200640 1 -- 172.21.15.28:0/2955401122 wait complete. 2024-09-10T10:51:47.112 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:47.112 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.268+0000 7fc607400640 1 Processor -- start 2024-09-10T10:51:47.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.268+0000 7fc607400640 1 -- start start 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072320 0x7fc6000cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc600072de0 0x7fc60006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc600072700 0x7fc60006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc600146140 con 0x7fc600072700 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc6000cd740 con 0x7fc600072de0 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.269+0000 7fc607400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000cda20 con 0x7fc600072320 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc600072700 0x7fc60006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc600072700 0x7fc60006e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41722/0 (socket says 172.21.15.28:41722) 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 -- 172.21.15.28:0/661273937 learned_addr learned my addr 172.21.15.28:0/661273937 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc604a00640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072320 0x7fc6000cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc5ffe00640 1 --2- 172.21.15.28:0/661273937 >> v2:172.21.15.138:3300/0 conn(0x7fc600072de0 0x7fc60006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 -- 172.21.15.28:0/661273937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072320 msgr2=0x7fc6000cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072320 0x7fc6000cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 -- 172.21.15.28:0/661273937 >> v2:172.21.15.138:3300/0 conn(0x7fc600072de0 msgr2=0x7fc60006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 --2- 172.21.15.28:0/661273937 >> v2:172.21.15.138:3300/0 conn(0x7fc600072de0 0x7fc60006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.270+0000 7fc605400640 1 -- 172.21.15.28:0/661273937 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000cdd00 con 0x7fc600072700 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.271+0000 7fc605400640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072700 0x7fc60006e5f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc6000701f0 tx=0x7fc5f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78853dcd05258820 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc5ff400640 1 -- 172.21.15.28:0/661273937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8036070 con 0x7fc600072700 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc5ff400640 1 -- 172.21.15.28:0/661273937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f8031600 con 0x7fc600072700 2024-09-10T10:51:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc5ff400640 1 -- 172.21.15.28:0/661273937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80319d0 con 0x7fc600072700 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 -- 172.21.15.28:0/661273937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072700 msgr2=0x7fc60006e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072700 0x7fc60006e5f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fc6000701f0 tx=0x7fc5f802a610 comp rx=0 tx=0).stop 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 -- 172.21.15.28:0/661273937 shutdown_connections 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072700 0x7fc60006e5f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 --2- 172.21.15.28:0/661273937 >> v2:172.21.15.138:3300/0 conn(0x7fc600072de0 0x7fc60006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 --2- 172.21.15.28:0/661273937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072320 0x7fc6000cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 -- 172.21.15.28:0/661273937 >> 172.21.15.28:0/661273937 conn(0x7fc600067d70 msgr2=0x7fc60006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:47.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 -- 172.21.15.28:0/661273937 shutdown_connections 2024-09-10T10:51:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.272+0000 7fc607400640 1 -- 172.21.15.28:0/661273937 wait complete. 2024-09-10T10:51:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.274+0000 7fc607400640 1 Processor -- start 2024-09-10T10:51:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.274+0000 7fc607400640 1 -- start start 2024-09-10T10:51:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc600072700 0x7fc60015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072de0 0x7fc6001e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000cd520 con 0x7fc600072320 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000cd3a0 con 0x7fc600072700 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc607400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc60006eb30 con 0x7fc600072de0 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc604a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc604a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41726/0 (socket says 172.21.15.28:41726) 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.275+0000 7fc604a00640 1 -- 172.21.15.28:0/1486984626 learned_addr learned my addr 172.21.15.28:0/1486984626 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc5ffe00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc600072700 0x7fc60015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc605400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072de0 0x7fc6001e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 -- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072de0 msgr2=0x7fc6001e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072de0 0x7fc6001e1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 -- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc600072700 msgr2=0x7fc60015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc600072700 0x7fc60015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 -- 172.21.15.28:0/1486984626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6001e2720 con 0x7fc600072320 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc604a00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400e890 tx=0x7fc5f400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.276+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4017070 con 0x7fc600072320 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.277+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5f400c940 con 0x7fc600072320 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.277+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400cd10 con 0x7fc600072320 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.277+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6001e2a00 con 0x7fc600072320 2024-09-10T10:51:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.277+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001e2fc0 con 0x7fc600072320 2024-09-10T10:51:47.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.278+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc600144e00 con 0x7fc600072320 2024-09-10T10:51:47.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.279+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5f401a070 con 0x7fc600072320 2024-09-10T10:51:47.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.279+0000 7fc5fcc00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.280+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7fc5f408b700 con 0x7fc600072320 2024-09-10T10:51:47.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.280+0000 7fc5ffe00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.281+0000 7fc5ffe00640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 0x7fc5e006e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc6001e5620 tx=0x7fc5ec009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.284+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc600144e00 con 0x7fc600072320 2024-09-10T10:51:47.480 INFO:tasks.ceph.mds.c.smithi028.stderr: -47> 2024-09-10T10:51:42.727+0000 7faef9a00640 -1 mds.0.snaptable: load_2 could not read table: -2 2024-09-10T10:51:47.481 INFO:tasks.ceph.mds.c.smithi028.stderr: -45> 2024-09-10T10:51:42.727+0000 7faef9a00640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds_snaptable' -2 ((2) No such file or directory) 2024-09-10T10:51:47.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.505+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6001e2410 con 0x7fc600072320 2024-09-10T10:51:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.506+0000 7fc5fcc00640 1 -- 172.21.15.28:0/1486984626 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) ==== 95+0+3566 (secure 0 0 0) 0x7fc5f4059410 con 0x7fc600072320 2024-09-10T10:51:47.507 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:47.507 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":140,"btime":"2024-09-10T10:51:46:682646+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:46.682576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:47.507 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 140 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 msgr2=0x7fc5e006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 0x7fc5e006e2f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc6001e5620 tx=0x7fc5ec009190 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 msgr2=0x7fc6001e6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400e890 tx=0x7fc5f400ed50 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 shutdown_connections 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5e006be30 0x7fc5e006e2f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc600072de0 0x7fc6001e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc600072700 0x7fc60015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 --2- 172.21.15.28:0/1486984626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc600072320 0x7fc6001e6b70 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.508+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 >> 172.21.15.28:0/1486984626 conn(0x7fc600067d70 msgr2=0x7fc60006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.509+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 shutdown_connections 2024-09-10T10:51:47.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.509+0000 7fc607400640 1 -- 172.21.15.28:0/1486984626 wait complete. 2024-09-10T10:51:47.520 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:51:47.527 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:47.527 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:47.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.681+0000 7f394be00640 1 Processor -- start 2024-09-10T10:51:47.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.682+0000 7f394be00640 1 -- start start 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f394c160bc0 0x7f394c0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f394c0d02c0 con 0x7f394c0ce770 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f394c06f940 con 0x7f394c160bc0 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f394c071d40 con 0x7f394c09b140 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f3942000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f394c0ce770 0x7f394c09aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40520/0 (socket says 172.21.15.28:40520) 2024-09-10T10:51:47.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f3942000640 1 -- 172.21.15.28:0/491190438 learned_addr learned my addr 172.21.15.28:0/491190438 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394aa00640 1 --2- 172.21.15.28:0/491190438 >> v2:172.21.15.138:3300/0 conn(0x7f394c160bc0 0x7f394c0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394b400640 1 --2- 172.21.15.28:0/491190438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620822357 0 0) 0x7f394c0d02c0 con 0x7f394c0ce770 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.683+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3930003520 con 0x7f394c0ce770 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.684+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1881838694 0 0) 0x7f3930003520 con 0x7f394c0ce770 2024-09-10T10:51:47.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.684+0000 7f394b400640 1 -- 172.21.15.28:0/491190438 >> v2:172.21.15.138:3300/0 conn(0x7f394c160bc0 msgr2=0x7f394c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:47.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.684+0000 7f394b400640 1 --2- 172.21.15.28:0/491190438 >> v2:172.21.15.138:3300/0 conn(0x7f394c160bc0 0x7f394c0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.684+0000 7f394b400640 1 -- 172.21.15.28:0/491190438 >> v1:172.21.15.28:6789/0 conn(0x7f394c0ce770 legacy=0x7f394c09aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.684+0000 7f394b400640 1 -- 172.21.15.28:0/491190438 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c0d0020 con 0x7f394c09b140 2024-09-10T10:51:47.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.685+0000 7f394b400640 1 --2- 172.21.15.28:0/491190438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0cd550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3944009d90 tx=0x7f394402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd05e4c327c6fa18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.685+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3934001670 con 0x7f394c09b140 2024-09-10T10:51:47.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.685+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3944007540 con 0x7f394c09b140 2024-09-10T10:51:47.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.685+0000 7f394a000640 1 -- 172.21.15.28:0/491190438 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440078f0 con 0x7f394c09b140 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 -- 172.21.15.28:0/491190438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 msgr2=0x7f394c0cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 --2- 172.21.15.28:0/491190438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0cd550 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3944009d90 tx=0x7f394402a5f0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 -- 172.21.15.28:0/491190438 shutdown_connections 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 --2- 172.21.15.28:0/491190438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0cd550 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 --2- 172.21.15.28:0/491190438 >> v2:172.21.15.138:3300/0 conn(0x7f394c160bc0 0x7f394c0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 -- 172.21.15.28:0/491190438 >> 172.21.15.28:0/491190438 conn(0x7f394c0b3c90 msgr2=0x7f394c0b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 -- 172.21.15.28:0/491190438 shutdown_connections 2024-09-10T10:51:47.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.686+0000 7f394be00640 1 -- 172.21.15.28:0/491190438 wait complete. 2024-09-10T10:51:47.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.687+0000 7f394be00640 1 Processor -- start 2024-09-10T10:51:47.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.688+0000 7f394be00640 1 -- start start 2024-09-10T10:51:47.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.688+0000 7f394be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.688+0000 7f394be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394c0ce770 0x7f394c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.688+0000 7f394be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f394c0d0cc0 con 0x7f394c160bc0 2024-09-10T10:51:47.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f394c0b48c0 con 0x7f394c0ce770 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f394c0d0e40 con 0x7f394c09b140 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41744/0 (socket says 172.21.15.28:41744) 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 -- 172.21.15.28:0/3405125201 learned_addr learned my addr 172.21.15.28:0/3405125201 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394aa00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f3942000640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394c0ce770 0x7f394c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 -- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 msgr2=0x7f394c0c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 -- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394c0ce770 msgr2=0x7f394c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394c0ce770 0x7f394c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.689+0000 7f394b400640 1 -- 172.21.15.28:0/3405125201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c0c69e0 con 0x7f394c160bc0 2024-09-10T10:51:47.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f394b400640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f394402abe0 tx=0x7f3944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944009450 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3944004240 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c0c6c80 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440045f0 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.690+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c0c7240 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.691+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c0711b0 con 0x7f394c160bc0 2024-09-10T10:51:47.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.692+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f394403b410 con 0x7f394c160bc0 2024-09-10T10:51:47.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.693+0000 7f3943400640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 0x7f392806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:47.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.694+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7f39440ac6e0 con 0x7f394c160bc0 2024-09-10T10:51:47.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.694+0000 7f394aa00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 0x7f392806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:47.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.695+0000 7f394aa00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 0x7f392806e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f393c009ad0 tx=0x7f393c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:47.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.698+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f394403f030 con 0x7f394c160bc0 2024-09-10T10:51:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.922+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f394c0d0560 con 0x7f394c160bc0 2024-09-10T10:51:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.923+0000 7f3943400640 1 -- 172.21.15.28:0/3405125201 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) ==== 95+0+4392 (secure 0 0 0) 0x7f394407aa60 con 0x7f394c160bc0 2024-09-10T10:51:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:47.924 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":141,"btime":"2024-09-10T10:51:47:732746+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1369429206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1369429206},{"type":"v1","addr":"172.21.15.28:6834","nonce":1369429206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":141}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:46.682576+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 141 2024-09-10T10:51:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.925+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 msgr2=0x7f392806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.925+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 0x7f392806e2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f393c009ad0 tx=0x7f393c008040 comp rx=0 tx=0).stop 2024-09-10T10:51:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.925+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 msgr2=0x7f394c0c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.925+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f394402abe0 tx=0x7f3944004a40 comp rx=0 tx=0).stop 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 shutdown_connections 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f392806be30 0x7f392806e2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f394c160bc0 0x7f394c0c5b00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f394c0ce770 0x7f394c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 --2- 172.21.15.28:0/3405125201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f394c09b140 0x7f394c0c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 >> 172.21.15.28:0/3405125201 conn(0x7f394c0b3c90 msgr2=0x7f394c0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 shutdown_connections 2024-09-10T10:51:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:47.926+0000 7f394be00640 1 -- 172.21.15.28:0/3405125201 wait complete. 2024-09-10T10:51:47.943 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_inotable' led to DAMAGED state 2024-09-10T10:51:47.943 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000' 2024-09-10T10:51:47.943 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:47.943 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:47.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:47.979 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:47.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:47.979 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:48.053 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:48.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:48.054 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:48.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.253+0000 7fae8f000640 1 Processor -- start 2024-09-10T10:51:48.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.253+0000 7fae8f000640 1 -- start start 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae880cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8f000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fae88072de0 0x7fae8806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8f000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae88146140 con 0x7fae88072700 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8f000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fae880cd740 con 0x7fae88072de0 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fae880cda20 con 0x7fae88072320 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fae88072700 0x7fae8806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40524/0 (socket says 172.21.15.28:40524) 2024-09-10T10:51:48.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae8d000640 1 -- 172.21.15.28:0/2642012940 learned_addr learned my addr 172.21.15.28:0/2642012940 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae7fe00640 1 --2- 172.21.15.28:0/2642012940 >> v2:172.21.15.138:3300/0 conn(0x7fae88072de0 0x7fae8806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae87e00640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae880cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae87400640 1 -- 172.21.15.28:0/2642012940 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22417130 0 0) 0x7fae88146140 con 0x7fae88072700 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.255+0000 7fae87400640 1 -- 172.21.15.28:0/2642012940 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae6c003520 con 0x7fae88072700 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.256+0000 7fae7fe00640 1 -- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 msgr2=0x7fae880cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.256+0000 7fae7fe00640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae880cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.256+0000 7fae7fe00640 1 -- 172.21.15.28:0/2642012940 >> v1:172.21.15.28:6789/0 conn(0x7fae88072700 legacy=0x7fae8806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.256+0000 7fae7fe00640 1 -- 172.21.15.28:0/2642012940 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae880cdd00 con 0x7fae88072de0 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.256+0000 7fae87e00640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae880cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.257+0000 7fae7fe00640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae8806e0b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301f45eee9d438bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.257+0000 7fae87400640 1 -- 172.21.15.28:0/2642012940 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70037070 con 0x7fae88072de0 2024-09-10T10:51:48.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.257+0000 7fae87400640 1 -- 172.21.15.28:0/2642012940 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7002ae50 con 0x7fae88072de0 2024-09-10T10:51:48.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.257+0000 7fae87400640 1 -- 172.21.15.28:0/2642012940 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70032640 con 0x7fae88072de0 2024-09-10T10:51:48.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 -- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 msgr2=0x7fae8806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae8806e0b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae700097a0 tx=0x7fae7002a100 comp rx=0 tx=0).stop 2024-09-10T10:51:48.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 -- 172.21.15.28:0/2642012940 shutdown_connections 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae8806e0b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 --2- 172.21.15.28:0/2642012940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae880cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 -- 172.21.15.28:0/2642012940 >> 172.21.15.28:0/2642012940 conn(0x7fae88067d70 msgr2=0x7fae8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 -- 172.21.15.28:0/2642012940 shutdown_connections 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.258+0000 7fae8f000640 1 -- 172.21.15.28:0/2642012940 wait complete. 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.259+0000 7fae8f000640 1 Processor -- start 2024-09-10T10:51:48.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 -- start start 2024-09-10T10:51:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae881e3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae880c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fae880cd520 con 0x7fae88072700 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae880cd3a0 con 0x7fae88072de0 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.260+0000 7fae8f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fae8806ed00 con 0x7fae88072320 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41750/0 (socket says 172.21.15.28:41750) 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 -- 172.21.15.28:0/3136375056 learned_addr learned my addr 172.21.15.28:0/3136375056 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae87e00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae881e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae8d000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae880c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 -- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 msgr2=0x7fae881e3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae881e3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 -- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 msgr2=0x7fae880c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae880c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.261+0000 7fae7fe00640 1 -- 172.21.15.28:0/3136375056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae880c7f00 con 0x7fae88072700 2024-09-10T10:51:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae7fe00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fae70000c00 tx=0x7fae700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70040020 con 0x7fae88072700 2024-09-10T10:51:48.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae70038040 con 0x7fae88072700 2024-09-10T10:51:48.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70037070 con 0x7fae88072700 2024-09-10T10:51:48.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae881e3ac0 con 0x7fae88072700 2024-09-10T10:51:48.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.262+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae880c81a0 con 0x7fae88072700 2024-09-10T10:51:48.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.263+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae880cce60 con 0x7fae88072700 2024-09-10T10:51:48.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.264+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fae70043020 con 0x7fae88072700 2024-09-10T10:51:48.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.265+0000 7fae84c00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 0x7fae6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.265+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7fae700ab1f0 con 0x7fae88072700 2024-09-10T10:51:48.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.265+0000 7fae87e00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 0x7fae6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.266+0000 7fae87e00640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 0x7fae6406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fae78009560 tx=0x7fae78009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.269+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fae7004a030 con 0x7fae88072700 2024-09-10T10:51:48.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.490+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fae880c8370 con 0x7fae88072700 2024-09-10T10:51:48.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.744+0000 7fae84c00640 1 -- 172.21.15.28:0/3136375056 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v142) ==== 161+0+0 (secure 0 0 0) 0x7fae70003e20 con 0x7fae88072700 2024-09-10T10:51:48.745 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 msgr2=0x7fae6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 0x7fae6406e2f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fae78009560 tx=0x7fae78009190 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 msgr2=0x7fae88134a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fae70000c00 tx=0x7fae700046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 shutdown_connections 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fae6406be30 0x7fae6406e2f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae88072de0 0x7fae880c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae88072700 0x7fae88134a20 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.746+0000 7fae8f000640 1 --2- 172.21.15.28:0/3136375056 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fae88072320 0x7fae881e3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.747+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 >> 172.21.15.28:0/3136375056 conn(0x7fae88067d70 msgr2=0x7fae881449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.747+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 shutdown_connections 2024-09-10T10:51:48.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.747+0000 7fae8f000640 1 -- 172.21.15.28:0/3136375056 wait complete. 2024-09-10T10:51:48.764 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.921+0000 7fb3bb200640 1 Processor -- start 2024-09-10T10:51:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.922+0000 7fb3bb200640 1 -- start start 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3bb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b4072dc0 0x7fb3b40b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3bb200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb3b40b3c90 0x7fb3b406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3bb200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b40b44c0 con 0x7fb3b40afd60 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3bb200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40b0140 con 0x7fb3b40b3c90 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3bb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b415ead0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3b3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb3b40afd60 0x7fb3b40726b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40534/0 (socket says 172.21.15.28:40534) 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3b3e00640 1 -- 172.21.15.28:0/425372517 learned_addr learned my addr 172.21.15.28:0/425372517 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3b9200640 1 --2- 172.21.15.28:0/425372517 >> v2:172.21.15.28:3300/0 conn(0x7fb3b40b3c90 0x7fb3b406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.923+0000 7fb3ab400640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b4072dc0 0x7fb3b40b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b3400640 1 -- 172.21.15.28:0/425372517 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315301087 0 0) 0x7fb3b40b44c0 con 0x7fb3b40afd60 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b3400640 1 -- 172.21.15.28:0/425372517 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb398003520 con 0x7fb3b40afd60 2024-09-10T10:51:48.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b9200640 1 -- 172.21.15.28:0/425372517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b4072dc0 msgr2=0x7fb3b40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:48.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b9200640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b4072dc0 0x7fb3b40b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b9200640 1 -- 172.21.15.28:0/425372517 >> v1:172.21.15.28:6789/0 conn(0x7fb3b40afd60 legacy=0x7fb3b40726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.924+0000 7fb3b9200640 1 -- 172.21.15.28:0/425372517 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b415edb0 con 0x7fb3b40b3c90 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3b9200640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40b3c90 0x7fb3b406f5a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009d90 tx=0x7fb3ac02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817a937ba6e9248e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3b3400640 1 -- 172.21.15.28:0/425372517 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007740 con 0x7fb3b40b3c90 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3b3400640 1 -- 172.21.15.28:0/425372517 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ac0078e0 con 0x7fb3b40b3c90 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3b3400640 1 -- 172.21.15.28:0/425372517 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007cb0 con 0x7fb3b40b3c90 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 -- 172.21.15.28:0/425372517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40b3c90 msgr2=0x7fb3b406f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40b3c90 0x7fb3b406f5a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009d90 tx=0x7fb3ac02a660 comp rx=0 tx=0).stop 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 -- 172.21.15.28:0/425372517 shutdown_connections 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40b3c90 0x7fb3b406f5a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 --2- 172.21.15.28:0/425372517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b4072dc0 0x7fb3b40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.925+0000 7fb3bb200640 1 -- 172.21.15.28:0/425372517 >> 172.21.15.28:0/425372517 conn(0x7fb3b406c030 msgr2=0x7fb3b406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.926+0000 7fb3bb200640 1 -- 172.21.15.28:0/425372517 shutdown_connections 2024-09-10T10:51:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.926+0000 7fb3bb200640 1 -- 172.21.15.28:0/425372517 wait complete. 2024-09-10T10:51:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.927+0000 7fb3bb200640 1 Processor -- start 2024-09-10T10:51:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.927+0000 7fb3bb200640 1 -- start start 2024-09-10T10:51:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40afd60 0x7fb3b4146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b40b3c90 0x7fb3b41570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b415f890 con 0x7fb3b4072dc0 2024-09-10T10:51:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b415f370 con 0x7fb3b40b3c90 2024-09-10T10:51:48.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3bb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40b4140 con 0x7fb3b40afd60 2024-09-10T10:51:48.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41758/0 (socket says 172.21.15.28:41758) 2024-09-10T10:51:48.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.928+0000 7fb3b3e00640 1 -- 172.21.15.28:0/439469711 learned_addr learned my addr 172.21.15.28:0/439469711 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 -- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40afd60 msgr2=0x7fb3b4146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40afd60 0x7fb3b4146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 -- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b40b3c90 msgr2=0x7fb3b41570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b40b3c90 0x7fb3b41570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 -- 172.21.15.28:0/439469711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b41f85d0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c390 tx=0x7fb3a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3b4072dc0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4005ce0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b4155e20 con 0x7fb3b4072dc0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40072f0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.929+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41563e0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.930+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b40b3750 con 0x7fb3b4072dc0 2024-09-10T10:51:48.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.931+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x7fb3b4072dc0 2024-09-10T10:51:48.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.932+0000 7fb3b0c00640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:48.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.932+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7fb3a408a8d0 con 0x7fb3b4072dc0 2024-09-10T10:51:48.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.933+0000 7fb3ab400640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:48.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.934+0000 7fb3ab400640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 0x7fb39006e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006a90 tx=0x7fb39c006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:48.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:48.936+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3b41f85d0 con 0x7fb3b4072dc0 2024-09-10T10:51:49.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.189+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb3b406d250 con 0x7fb3b4072dc0 2024-09-10T10:51:49.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.753+0000 7fb3b0c00640 1 -- 172.21.15.28:0/439469711 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v143) ==== 101+0+0 (secure 0 0 0) 0x7fb3a4058c30 con 0x7fb3b4072dc0 2024-09-10T10:51:49.754 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 msgr2=0x7fb39006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 0x7fb39006e2f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb39c006a90 tx=0x7fb39c006a20 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 msgr2=0x7fb3b41f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c390 tx=0x7fb3a400c850 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 shutdown_connections 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39006be30 0x7fb39006e2f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b40b3c90 0x7fb3b41570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40afd60 0x7fb3b4146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 --2- 172.21.15.28:0/439469711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b4072dc0 0x7fb3b41f8090 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.755+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 >> 172.21.15.28:0/439469711 conn(0x7fb3b406c030 msgr2=0x7fb3b4160ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.756+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 shutdown_connections 2024-09-10T10:51:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.756+0000 7fb3bb200640 1 -- 172.21.15.28:0/439469711 wait complete. 2024-09-10T10:51:49.773 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.849+0000 7f90d9561f40 1 Processor -- start 2024-09-10T10:51:49.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.849+0000 7f90d9561f40 1 -- start start 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f444a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f451f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5602f43f0b30 con 0x5602f451e3e0 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602f43f4b70 con 0x5602f45219c0 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d9561f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5602f4520ee0 con 0x5602f4448470 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41760/0 (socket says 172.21.15.28:41760) 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d5e00640 1 -- 172.21.15.28:0/2053870783 learned_addr learned my addr 172.21.15.28:0/2053870783 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d6800640 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f444a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.850+0000 7f90d7200640 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f451f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 -- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 msgr2=0x5602f444a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f444a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 -- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 msgr2=0x5602f451f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f451f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 -- 172.21.15.28:0/2053870783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f4497860 con 0x5602f451e3e0 2024-09-10T10:51:49.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f90c400a520 tx=0x7f90c404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a740198ba59c7d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d4a00640 1 -- 172.21.15.28:0/2053870783 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4059070 con 0x5602f451e3e0 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d4a00640 1 -- 172.21.15.28:0/2053870783 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c4065070 con 0x5602f451e3e0 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.851+0000 7f90d4a00640 1 -- 172.21.15.28:0/2053870783 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c400a300 con 0x5602f451e3e0 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 -- 172.21.15.28:0/2053870783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 msgr2=0x5602f4514b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f90c400a520 tx=0x7f90c404e000 comp rx=0 tx=0).stop 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 -- 172.21.15.28:0/2053870783 shutdown_connections 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f451f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f4514b50 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2053870783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f444a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 -- 172.21.15.28:0/2053870783 >> 172.21.15.28:0/2053870783 conn(0x5602f44970a0 msgr2=0x5602f451e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 -- 172.21.15.28:0/2053870783 shutdown_connections 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.852+0000 7f90d9561f40 1 -- 172.21.15.28:0/2053870783 wait complete. 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.853+0000 7f90d9561f40 1 Processor -- start 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.853+0000 7f90d9561f40 1 -- start start 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f451f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f4479c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5602f451f740 con 0x5602f451e3e0 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602f451f480 con 0x5602f45219c0 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d9561f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5602f447adb0 con 0x5602f4448470 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41772/0 (socket says 172.21.15.28:41772) 2024-09-10T10:51:49.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 learned_addr learned my addr 172.21.15.28:0/2581378953 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f451f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.854+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f4479c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 msgr2=0x5602f451f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602f4448470 0x5602f451f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 msgr2=0x5602f4479c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f4479c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602f45247c0 con 0x5602f451e3e0 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602f45219c0 0x5602f4479c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f90c40457c0 tx=0x7f90c4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90cf400640 1 -- 172.21.15.28:0/2581378953 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4059040 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90cf400640 1 -- 172.21.15.28:0/2581378953 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c405a040 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.855+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602f4524aa0 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.856+0000 7f90cf400640 1 -- 172.21.15.28:0/2581378953 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4054a30 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.856+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602f4477f50 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.858+0000 7f90cf400640 1 -- 172.21.15.28:0/2581378953 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f90c4054bd0 con 0x5602f451e3e0 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.858+0000 7f90cf400640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 0x7f90b806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.858+0000 7f90cf400640 1 -- 172.21.15.28:0/2581378953 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7f90c40cf540 con 0x5602f451e3e0 2024-09-10T10:51:49.860 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:49.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.859+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 0x5602f448ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.859+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f4490480 con 0x5602f4524ea0 2024-09-10T10:51:49.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.859+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 0x7f90b806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.859+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 0x5602f448ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.860+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 0x7f90b806ccd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f90c80017e0 tx=0x7f90c8001340 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.860+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 0x5602f448ff40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.862+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v110'288 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.862+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f4490850 con 0x5602f4524ea0 2024-09-10T10:51:49.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.863+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v110'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.863+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3430 con 0x5602f4524ea0 2024-09-10T10:51:49.866 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:49.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.865+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v110'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.865+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c47e0 con 0x5602f4524ea0 2024-09-10T10:51:49.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.867+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v110'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.867+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c4b80 con 0x5602f4524ea0 2024-09-10T10:51:49.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.869+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v110'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.869+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c37d0 con 0x5602f4524ea0 2024-09-10T10:51:49.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.871+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v110'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.871+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3b70 con 0x5602f4524ea0 2024-09-10T10:51:49.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.872+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v110'294 uv268 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.872+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.874+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v110'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.874+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.876 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.876+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v110'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.876+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.877+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v110'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.878+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.879+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v110'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.879+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.881+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v110'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.881+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.883+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v110'300 uv275 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.883+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.884+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v110'301 uv301 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.884+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.886 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.886+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v110'302 uv302 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.886+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.887+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v110'303 uv303 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.887+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.889+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v110'304 uv304 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.889+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c3f10 con 0x5602f4524ea0 2024-09-10T10:51:49.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.891+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v110'305 uv305 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.891+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.893+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v110'306 uv306 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.893+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.894+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v110'307 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.894+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.896+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v110'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.896+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.898 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.898+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v110'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.898+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.899+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v110'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.899+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.901+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v110'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.901+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.902+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v110'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.902+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.904+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v110'313 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.904+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9e80 con 0x5602f4524ea0 2024-09-10T10:51:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.905+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v110'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.906+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c41d0 con 0x5602f4524ea0 2024-09-10T10:51:49.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:49.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.907+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v110'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.907+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c41d0 con 0x5602f4524ea0 2024-09-10T10:51:49.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.909+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v110'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.909+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c41d0 con 0x5602f4524ea0 2024-09-10T10:51:49.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.910+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v110'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.910+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c41d0 con 0x5602f4524ea0 2024-09-10T10:51:49.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.912+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v110'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f4524ea0 2024-09-10T10:51:49.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.912+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 0x5602f45c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.912+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c7850 con 0x5602f45c4f20 2024-09-10T10:51:49.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.913+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 0x5602f45c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.913+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 0x5602f45c7310 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.915+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v110'219 uv202 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.915+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c7bf0 con 0x5602f45c4f20 2024-09-10T10:51:49.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.916+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v110'220 uv220 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.916+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c7f90 con 0x5602f45c4f20 2024-09-10T10:51:49.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.918+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v110'221 uv221 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.919 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:49.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.918+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c8330 con 0x5602f45c4f20 2024-09-10T10:51:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.919+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v110'222 uv222 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.919+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c86d0 con 0x5602f45c4f20 2024-09-10T10:51:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.921+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v110'223 uv223 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.921+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c8a70 con 0x5602f45c4f20 2024-09-10T10:51:49.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.922+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v110'224 uv207 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.922+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c8e10 con 0x5602f45c4f20 2024-09-10T10:51:49.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.924+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v110'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.924+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.925+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v110'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.927 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:49.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.926+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.927+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v110'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.927+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.929+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v110'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.929+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.930+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v110'229 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.931+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.932+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v110'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.932+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.934 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.933+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v110'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.934+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd8a0 con 0x5602f45c4f20 2024-09-10T10:51:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.936+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v110'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.936+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd8a0 con 0x5602f45c4f20 2024-09-10T10:51:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.937+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v110'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.937+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.938+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v110'234 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.938+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.940+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v110'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.940+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45c91b0 con 0x5602f45c4f20 2024-09-10T10:51:49.941 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.941+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v110'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.941+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45ca1c0 con 0x5602f45c4f20 2024-09-10T10:51:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.942+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v110'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.942+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45ca1c0 con 0x5602f45c4f20 2024-09-10T10:51:49.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.944+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v110'238 uv238 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.944+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45b9f00 con 0x5602f45c4f20 2024-09-10T10:51:49.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.945+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v110'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45c4f20 2024-09-10T10:51:49.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.945+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5602f45c9b70 0x5602f45cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.945+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cc4a0 con 0x5602f45c9b70 2024-09-10T10:51:49.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.946+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5602f45c9b70 0x5602f45cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.946+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5602f45c9b70 0x5602f45cbf60 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.948+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v110'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.948+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cc840 con 0x5602f45c9b70 2024-09-10T10:51:49.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.949+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v110'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.949+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45ccc60 con 0x5602f45c9b70 2024-09-10T10:51:49.953 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:49.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.952+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v110'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.952+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45ce510 con 0x5602f45c9b70 2024-09-10T10:51:49.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.953+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v110'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.953+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45ce8b0 con 0x5602f45c9b70 2024-09-10T10:51:49.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.955+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v110'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.955+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd500 con 0x5602f45c9b70 2024-09-10T10:51:49.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.956+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v110'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.956+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5602f45cde00 0x5602f45d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.957+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d11a0 con 0x5602f45cde00 2024-09-10T10:51:49.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.957+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5602f45cde00 0x5602f45d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.957+0000 7f90d7200640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5602f45cde00 0x5602f45d0c60 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.959+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v110'45 uv44 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.959+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d11a0 con 0x5602f45cde00 2024-09-10T10:51:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.960+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v110'46 uv46 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.960+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1590 con 0x5602f45cde00 2024-09-10T10:51:49.962 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:49.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.961+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v110'47 uv47 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.961+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1590 con 0x5602f45cde00 2024-09-10T10:51:49.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.962+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v110'48 uv48 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.962+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1590 con 0x5602f45c9b70 2024-09-10T10:51:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.963+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v110'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.963+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.964+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v110'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.964+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.965+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v110'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.966 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.965+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d2940 con 0x5602f45c9b70 2024-09-10T10:51:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.966+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v110'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.966+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d2940 con 0x5602f45c9b70 2024-09-10T10:51:49.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.968+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v110'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.968+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.969+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v110'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.969+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.970+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v110'105 uv99 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.970+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.971+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v110'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.971+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.973 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:49.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.972+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v110'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.973+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d2940 con 0x5602f45cde00 2024-09-10T10:51:49.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.974+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v110'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.974+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d2940 con 0x5602f45cde00 2024-09-10T10:51:49.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.975+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v110'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.975+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.977+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v110'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.977+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.978+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v110'111 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.978+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.979+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v110'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.979+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.981 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:49.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.980+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v110'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.980+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.981+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v110'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.981+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45cde00 2024-09-10T10:51:49.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.983+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v110'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:49.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.983+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.984+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v110'46 uv45 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.984+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.985+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v110'47 uv47 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.985+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.987 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:49.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.986+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v110'48 uv48 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.986+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.988+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v110'49 uv49 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.988+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d1930 con 0x5602f45c9b70 2024-09-10T10:51:49.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.989+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v110'50 uv50 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45c9b70 2024-09-10T10:51:49.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.989+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5602f45d1fb0 0x5602f45d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:49.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.989+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d4970 con 0x5602f45d1fb0 2024-09-10T10:51:49.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.989+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5602f45d1fb0 0x5602f45d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:49.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.990+0000 7f90d6800640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5602f45d1fb0 0x5602f45d43a0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:49.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.991+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v110'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.991+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d4d40 con 0x5602f45d1fb0 2024-09-10T10:51:49.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.993+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v110'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.993+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d4d40 con 0x5602f45d1fb0 2024-09-10T10:51:49.995 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.994+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v110'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.994+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5d50 con 0x5602f45d1fb0 2024-09-10T10:51:49.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.995+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v110'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.995+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5d50 con 0x5602f45d1fb0 2024-09-10T10:51:49.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.997+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v110'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.997+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d4d40 con 0x5602f45d1fb0 2024-09-10T10:51:49.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.998+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v110'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:49.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.998+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d4d40 con 0x5602f45d1fb0 2024-09-10T10:51:50.000 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:50.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.999+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v110'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:49.999+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.001+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [write 0~34] v110'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.001+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.002+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [omap-set-header] v110'97 uv97 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.002+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.003+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 100.00000000 [create] v110'98 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.004+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.005+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 100.00000000 [delete] v110'99 uv99 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.005+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.007 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.006+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v110'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.006+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45cd0f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.007+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v110'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.007+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d52d0 con 0x5602f45d1fb0 2024-09-10T10:51:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.009+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v110'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.009+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.010+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v110'44 uv43 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.010+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.011+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v110'45 uv45 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.011+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.013+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v110'46 uv46 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.014 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:50.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.013+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.014+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v110'47 uv47 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.014+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.015+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v110'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.015+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5290 con 0x5602f45cde00 2024-09-10T10:51:50.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.016+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v110'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.016+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5270 con 0x5602f45cde00 2024-09-10T10:51:50.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.017+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v110'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.018 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:50.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.018+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5270 con 0x5602f45cde00 2024-09-10T10:51:50.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.019+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v110'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.019+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5270 con 0x5602f45cde00 2024-09-10T10:51:50.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.020+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v110'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.020+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d5270 con 0x5602f45cde00 2024-09-10T10:51:50.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.021+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v110'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90d0001770 con 0x5602f45cde00 2024-09-10T10:51:50.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.021+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5602f45d5250 0x5602f45d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.021+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d5250 2024-09-10T10:51:50.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.021+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5602f45d5250 0x5602f45d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.022+0000 7f90d5e00640 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5602f45d5250 0x5602f45d7eb0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.023+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v110'47 uv46 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45d5250 2024-09-10T10:51:50.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.023+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d5250 2024-09-10T10:51:50.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.024+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v110'48 uv48 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45d5250 2024-09-10T10:51:50.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.024+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d5250 2024-09-10T10:51:50.026 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:50.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.025+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v110'49 uv49 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45d5250 2024-09-10T10:51:50.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.025+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d5250 2024-09-10T10:51:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.026+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v110'50 uv50 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45d5250 2024-09-10T10:51:50.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.026+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d5250 2024-09-10T10:51:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.027+0000 7f90d5e00640 1 -- 172.21.15.28:0/2581378953 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v110'51 uv51 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f90c40998a0 con 0x5602f45d5250 2024-09-10T10:51:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.027+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.029+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(114 100.00000000.inode [create] v110'119 uv111 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.029+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.030+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(115 100.00000000.inode [delete] v110'120 uv120 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.030+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.032 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.031+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v110'121 uv121 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.031+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.032+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v110'122 uv122 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.032+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.033+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v110'123 uv123 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.033+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.034+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 1.00000000 [create] v110'124 uv118 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.034+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.036+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 1.00000000 [delete] v110'125 uv125 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.036+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.037 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:50.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.037+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v110'126 uv126 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.037+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d9400 con 0x5602f45d1fb0 2024-09-10T10:51:50.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.038+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v110'127 uv127 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.038+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d9400 con 0x5602f45d1fb0 2024-09-10T10:51:50.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.039+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v110'128 uv128 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.039+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d83f0 con 0x5602f45d1fb0 2024-09-10T10:51:50.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.040+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v110'129 uv129 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.040+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x5602f45d8cb0 con 0x5602f45d1fb0 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d6800640 1 -- 172.21.15.28:0/2581378953 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v110'130 uv130 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f90c8010ae0 con 0x5602f45d1fb0 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5602f45c9b70 msgr2=0x5602f45cbf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5602f45c9b70 0x5602f45cbf60 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5602f45d1fb0 msgr2=0x5602f45d43a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5602f45d1fb0 0x5602f45d43a0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5602f45d5250 msgr2=0x5602f45d7eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5602f45d5250 0x5602f45d7eb0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5602f45cde00 msgr2=0x5602f45d0c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5602f45cde00 0x5602f45d0c60 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 msgr2=0x5602f45c7310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 reap_dead start 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 0x5602f45c7310 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d7200640 1 -- 172.21.15.28:0/2581378953 reap_dead start 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 msgr2=0x5602f448ff40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.041+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 0x5602f448ff40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 msgr2=0x7f90b806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 0x7f90b806ccd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f90c80017e0 tx=0x7f90c8001340 comp rx=0 tx=0).stop 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 msgr2=0x5602f45b6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f90c40457c0 tx=0x7f90c4002fb0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 shutdown_connections 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5602f45c4f20 0x5602f45c7310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5602f4524ea0 0x5602f448ff40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f90b806a810 0x7f90b806ccd0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 --2- 172.21.15.28:0/2581378953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602f451e3e0 0x5602f45b6e70 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 >> 172.21.15.28:0/2581378953 conn(0x5602f44970a0 msgr2=0x5602f451e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 shutdown_connections 2024-09-10T10:51:50.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.042+0000 7f90d9561f40 1 -- 172.21.15.28:0/2581378953 wait complete. 2024-09-10T10:51:50.047 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000 2024-09-10T10:51:50.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.162+0000 7f4cc182df40 1 Processor -- start 2024-09-10T10:51:50.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.162+0000 7f4cc182df40 1 -- start start 2024-09-10T10:51:50.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b796b7ff0 0x561b79739be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797469c0 0x561b796b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561b796b8530 con 0x561b797433e0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b79615b30 con 0x561b797469c0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cc182df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561b79745ee0 con 0x561b796b7ff0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cbe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41788/0 (socket says 172.21.15.28:41788) 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cbe000640 1 -- 172.21.15.28:0/3710223119 learned_addr learned my addr 172.21.15.28:0/3710223119 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cbea00640 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b796b7ff0 0x561b79739be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.163+0000 7f4cbf400640 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797469c0 0x561b796b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 -- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b796b7ff0 msgr2=0x561b79739be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b796b7ff0 0x561b79739be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 -- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797469c0 msgr2=0x561b796b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797469c0 0x561b796b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 -- 172.21.15.28:0/3710223119 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b79745550 con 0x561b797433e0 2024-09-10T10:51:50.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbe000640 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00a430 tx=0x7f4cac00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96782bd9fd2fa0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbcc00640 1 -- 172.21.15.28:0/3710223119 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00b920 con 0x561b797433e0 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbcc00640 1 -- 172.21.15.28:0/3710223119 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cac00f030 con 0x561b797433e0 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cbcc00640 1 -- 172.21.15.28:0/3710223119 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00bd20 con 0x561b797433e0 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cc182df40 1 -- 172.21.15.28:0/3710223119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 msgr2=0x561b796b99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.164+0000 7f4cc182df40 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00a430 tx=0x7f4cac00a8f0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 -- 172.21.15.28:0/3710223119 shutdown_connections 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797469c0 0x561b796b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b797433e0 0x561b796b99c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 --2- 172.21.15.28:0/3710223119 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b796b7ff0 0x561b79739be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 -- 172.21.15.28:0/3710223119 >> 172.21.15.28:0/3710223119 conn(0x561b796bc0a0 msgr2=0x561b79743af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 -- 172.21.15.28:0/3710223119 shutdown_connections 2024-09-10T10:51:50.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 -- 172.21.15.28:0/3710223119 wait complete. 2024-09-10T10:51:50.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.165+0000 7f4cc182df40 1 Processor -- start 2024-09-10T10:51:50.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 -- start start 2024-09-10T10:51:50.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 0x561b79743ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797433e0 0x561b7973b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b797469c0 0x561b7973bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561b79745990 con 0x561b796b7ff0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561b796bca10 con 0x561b797433e0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.166+0000 7f4cc182df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561b7973a380 con 0x561b797469c0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 0x561b79743ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 0x561b79743ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41796/0 (socket says 172.21.15.28:41796) 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 -- 172.21.15.28:0/2009609972 learned_addr learned my addr 172.21.15.28:0/2009609972 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbf400640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b797469c0 0x561b7973bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbe000640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797433e0 0x561b7973b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 -- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b797469c0 msgr2=0x561b7973bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561b797469c0 0x561b7973bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 -- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797433e0 msgr2=0x561b7973b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561b797433e0 0x561b7973b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 -- 172.21.15.28:0/2009609972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b79740ec0 con 0x561b796b7ff0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cbea00640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 0x561b79743ce0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009fa0 tx=0x7f4cb0006810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.167+0000 7f4cb7400640 1 -- 172.21.15.28:0/2009609972 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb001a070 con 0x561b796b7ff0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.168+0000 7f4cb7400640 1 -- 172.21.15.28:0/2009609972 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cb0015680 con 0x561b796b7ff0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.168+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561b79740350 con 0x561b796b7ff0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.168+0000 7f4cb7400640 1 -- 172.21.15.28:0/2009609972 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0015a50 con 0x561b796b7ff0 2024-09-10T10:51:50.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.168+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b79740be0 con 0x561b796b7ff0 2024-09-10T10:51:50.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.169+0000 7f4cb7400640 1 -- 172.21.15.28:0/2009609972 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4cb0015bf0 con 0x561b796b7ff0 2024-09-10T10:51:50.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.170+0000 7f4cb7400640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ca006a840 0x7f4ca006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.170+0000 7f4cb7400640 1 -- 172.21.15.28:0/2009609972 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7f4cb0089b30 con 0x561b796b7ff0 2024-09-10T10:51:50.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.170+0000 7f4cc182df40 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561b79744220 0x561b797dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.171+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e110) -- 0x561b796b9d80 con 0x561b79744220 2024-09-10T10:51:50.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.171+0000 7f4cbe000640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ca006a840 0x7f4ca006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.171+0000 7f4cbf400640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561b79744220 0x561b797dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.171+0000 7f4cbf400640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561b79744220 0x561b797dfac0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.171+0000 7f4cbe000640 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ca006a840 0x7f4ca006cd00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x561b79742af0 tx=0x7f4cac016040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cbf400640 1 -- 172.21.15.28:0/2009609972 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [delete] v110'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4cb804e980 con 0x561b79744220 2024-09-10T10:51:50.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561b79744220 msgr2=0x561b797dfac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561b79744220 0x561b797dfac0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ca006a840 msgr2=0x7f4ca006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ca006a840 0x7f4ca006cd00 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x561b79742af0 tx=0x7f4cac016040 comp rx=0 tx=0).stop 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 msgr2=0x561b79743ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 --2- 172.21.15.28:0/2009609972 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561b796b7ff0 0x561b79743ce0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009fa0 tx=0x7f4cb0006810 comp rx=0 tx=0).stop 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cbf400640 1 -- 172.21.15.28:0/2009609972 reap_dead start 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 shutdown_connections 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 >> 172.21.15.28:0/2009609972 conn(0x561b796bc0a0 msgr2=0x561b79739be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 shutdown_connections 2024-09-10T10:51:50.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.173+0000 7f4cc182df40 1 -- 172.21.15.28:0/2009609972 wait complete. 2024-09-10T10:51:50.178 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:51:50.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.336+0000 7fe7ac200640 1 Processor -- start 2024-09-10T10:51:50.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.336+0000 7fe7ac200640 1 -- start start 2024-09-10T10:51:50.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7ac200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe7a415e410 0x7fe7a40b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7ac200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe7a406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7ac200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a40ae240 con 0x7fe7a40b40c0 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7ac200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a406c5c0 con 0x7fe7a415e410 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7ac200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a40ae520 con 0x7fe7a406bc10 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7a8e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe7a40b40c0 0x7fe7a40aff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40554/0 (socket says 172.21.15.28:40554) 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7a8e00640 1 -- 172.21.15.28:0/3314576460 learned_addr learned my addr 172.21.15.28:0/3314576460 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7aa200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe7a406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.337+0000 7fe7a9800640 1 --2- 172.21.15.28:0/3314576460 >> v2:172.21.15.138:3300/0 conn(0x7fe7a415e410 0x7fe7a40b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1835978750 0 0) 0x7fe7a40ae240 con 0x7fe7a40b40c0 2024-09-10T10:51:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe78c003520 con 0x7fe7a40b40c0 2024-09-10T10:51:50.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2330018097 0 0) 0x7fe78c003520 con 0x7fe7a40b40c0 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 msgr2=0x7fe7a406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe7a406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 >> v2:172.21.15.138:3300/0 conn(0x7fe7a415e410 msgr2=0x7fe7a40b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 --2- 172.21.15.28:0/3314576460 >> v2:172.21.15.138:3300/0 conn(0x7fe7a415e410 0x7fe7a40b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a40ae800 con 0x7fe7a40b40c0 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fe790001670 con 0x7fe7a40b40c0 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 >> v1:172.21.15.28:6789/0 conn(0x7fe7a40b40c0 legacy=0x7fe7a40aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe78c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c003f30 0x7fe78c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c004d40 0x7fe78c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a406c5c0 con 0x7fe78c004d40 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a40ae520 con 0x7fe78c003f30 2024-09-10T10:51:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe78c00ba10 con 0x7fe7a406bc10 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a9800640 1 --2- 172.21.15.28:0/3314576460 >> v2:172.21.15.138:3300/0 conn(0x7fe7a415e410 0x7fe7a40b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7aa200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c004d40 0x7fe78c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.338+0000 7fe7a8e00640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c003f30 0x7fe78c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7a9800640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe78c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7aa200640 1 -- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 msgr2=0x7fe78c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7aa200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe78c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7aa200640 1 -- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c003f30 msgr2=0x7fe78c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7aa200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c003f30 0x7fe78c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7aa200640 1 -- 172.21.15.28:0/3314576460 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78c003520 con 0x7fe78c004d40 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.339+0000 7fe7a9800640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a406bc10 0x7fe78c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7aa200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c004d40 0x7fe78c0051c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e170 tx=0x7fe78c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4219fa686763a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c034020 con 0x7fe78c004d40 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c0305f0 con 0x7fe78c004d40 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7a3e00640 1 -- 172.21.15.28:0/3314576460 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c009c10 con 0x7fe78c004d40 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7ac200640 1 -- 172.21.15.28:0/3314576460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c004d40 msgr2=0x7fe78c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7ac200640 1 --2- 172.21.15.28:0/3314576460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe78c004d40 0x7fe78c0051c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e170 tx=0x7fe78c005830 comp rx=0 tx=0).stop 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.340+0000 7fe7aa200640 1 -- 172.21.15.28:0/3314576460 reap_dead start 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.341+0000 7fe7ac200640 1 -- 172.21.15.28:0/3314576460 shutdown_connections 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.341+0000 7fe7ac200640 1 -- 172.21.15.28:0/3314576460 >> 172.21.15.28:0/3314576460 conn(0x7fe7a409aa30 msgr2=0x7fe7a40b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.341+0000 7fe7ac200640 1 -- 172.21.15.28:0/3314576460 shutdown_connections 2024-09-10T10:51:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.341+0000 7fe7ac200640 1 -- 172.21.15.28:0/3314576460 wait complete. 2024-09-10T10:51:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.342+0000 7fe7ac200640 1 Processor -- start 2024-09-10T10:51:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.342+0000 7fe7ac200640 1 -- start start 2024-09-10T10:51:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 0x7fe7a41f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a4162560 0x7fe7a4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a415e7f0 con 0x7fe7a409ec70 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a40b4510 con 0x7fe78c004d40 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.343+0000 7fe7ac200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a415e970 con 0x7fe7a4162560 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41814/0 (socket says 172.21.15.28:41814) 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 -- 172.21.15.28:0/2862888148 learned_addr learned my addr 172.21.15.28:0/2862888148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7a8e00640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a4162560 0x7fe7a4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7a9800640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 0x7fe7a41f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 -- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a4162560 msgr2=0x7fe7a4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a4162560 0x7fe7a4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 -- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 msgr2=0x7fe7a41f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 0x7fe7a41f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 -- 172.21.15.28:0/2862888148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a41f86b0 con 0x7fe7a409ec70 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7a9800640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 0x7fe7a41f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.344+0000 7fe7aa200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e170 tx=0x7fe79c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.345+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c034070 con 0x7fe7a409ec70 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.345+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c003990 con 0x7fe7a409ec70 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.345+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c009c10 con 0x7fe7a409ec70 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.345+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a409dbd0 con 0x7fe7a409ec70 2024-09-10T10:51:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.345+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a409e190 con 0x7fe7a409ec70 2024-09-10T10:51:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.346+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a40b5050 con 0x7fe7a409ec70 2024-09-10T10:51:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.347+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe79c037020 con 0x7fe7a409ec70 2024-09-10T10:51:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.348+0000 7fe7a1600640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 0x7fe78406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.348+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7fe79c0ab7d0 con 0x7fe7a409ec70 2024-09-10T10:51:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.348+0000 7fe7a9800640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 0x7fe78406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.349+0000 7fe7a9800640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 0x7fe78406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe798009ce0 tx=0x7fe798008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.352+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7a40b5050 con 0x7fe7a409ec70 2024-09-10T10:51:50.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.575+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe7a40b12d0 con 0x7fe7a409ec70 2024-09-10T10:51:50.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.757+0000 7fe7a1600640 1 -- 172.21.15.28:0/2862888148 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v144) ==== 162+0+0 (secure 0 0 0) 0x7fe79c079b30 con 0x7fe7a409ec70 2024-09-10T10:51:50.759 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:51:50.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 msgr2=0x7fe78406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 0x7fe78406e320 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe798009ce0 tx=0x7fe798008040 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 msgr2=0x7fe7a409ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e170 tx=0x7fe79c009750 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 shutdown_connections 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe78406be60 0x7fe78406e320 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7a409ec70 0x7fe7a409ba30 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7a4162560 0x7fe7a4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 --2- 172.21.15.28:0/2862888148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe78c004d40 0x7fe7a41f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 >> 172.21.15.28:0/2862888148 conn(0x7fe7a409aa30 msgr2=0x7fe7a4160b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 shutdown_connections 2024-09-10T10:51:50.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.760+0000 7fe7ac200640 1 -- 172.21.15.28:0/2862888148 wait complete. 2024-09-10T10:51:50.780 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:50.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.935+0000 7fccd5400640 1 Processor -- start 2024-09-10T10:51:50.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.935+0000 7fccd5400640 1 -- start start 2024-09-10T10:51:50.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.936+0000 7fccd5400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fccd00b40e0 0x7fccd00aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.936+0000 7fccd5400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd00b5470 con 0x7fccd015bef0 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.936+0000 7fccd5400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd00b3640 con 0x7fccd00af1b0 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.936+0000 7fccd5400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fccd00b5190 con 0x7fccd00b40e0 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fccd015bef0 0x7fccd00b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40556/0 (socket says 172.21.15.28:40556) 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccfe00640 1 -- 172.21.15.28:0/3544900970 learned_addr learned my addr 172.21.15.28:0/3544900970 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccf400640 1 --2- 172.21.15.28:0/3544900970 >> v2:172.21.15.28:3300/0 conn(0x7fccd00b40e0 0x7fccd00aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccea00640 1 -- 172.21.15.28:0/3544900970 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840746194 0 0) 0x7fccd00b5470 con 0x7fccd015bef0 2024-09-10T10:51:50.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccea00640 1 -- 172.21.15.28:0/3544900970 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccb4003520 con 0x7fccd015bef0 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccf400640 1 -- 172.21.15.28:0/3544900970 >> v1:172.21.15.138:6789/0 conn(0x7fccd00af1b0 legacy=0x7fccd0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccf400640 1 -- 172.21.15.28:0/3544900970 >> v1:172.21.15.28:6789/0 conn(0x7fccd015bef0 legacy=0x7fccd00b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.937+0000 7fcccf400640 1 -- 172.21.15.28:0/3544900970 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd00b38e0 con 0x7fccd00b40e0 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.938+0000 7fcccf400640 1 --2- 172.21.15.28:0/3544900970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd00b40e0 0x7fccd00aec70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097a0 tx=0x7fccbc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dd0c880f50e80b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.938+0000 7fcccea00640 1 -- 172.21.15.28:0/3544900970 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc037070 con 0x7fccd00b40e0 2024-09-10T10:51:50.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.938+0000 7fcccea00640 1 -- 172.21.15.28:0/3544900970 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccbc032440 con 0x7fccd00b40e0 2024-09-10T10:51:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.938+0000 7fcccea00640 1 -- 172.21.15.28:0/3544900970 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc032810 con 0x7fccd00b40e0 2024-09-10T10:51:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 -- 172.21.15.28:0/3544900970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd00b40e0 msgr2=0x7fccd00aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 --2- 172.21.15.28:0/3544900970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd00b40e0 0x7fccd00aec70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097a0 tx=0x7fccbc02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 -- 172.21.15.28:0/3544900970 shutdown_connections 2024-09-10T10:51:50.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 --2- 172.21.15.28:0/3544900970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd00b40e0 0x7fccd00aec70 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 -- 172.21.15.28:0/3544900970 >> 172.21.15.28:0/3544900970 conn(0x7fccd006c240 msgr2=0x7fccd00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:50.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 -- 172.21.15.28:0/3544900970 shutdown_connections 2024-09-10T10:51:50.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.939+0000 7fccd5400640 1 -- 172.21.15.28:0/3544900970 wait complete. 2024-09-10T10:51:50.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.941+0000 7fccd5400640 1 Processor -- start 2024-09-10T10:51:50.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.941+0000 7fccd5400640 1 -- start start 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccd00af1b0 0x7fccd015f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 0x7fccd014ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fccd00b4e60 con 0x7fccd015bef0 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccd00af640 con 0x7fccd00b40e0 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd5400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccd00b4b10 con 0x7fccd00af1b0 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41828/0 (socket says 172.21.15.28:41828) 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 -- 172.21.15.28:0/113180740 learned_addr learned my addr 172.21.15.28:0/113180740 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fcccf400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 0x7fccd014ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fcccfe00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccd00af1b0 0x7fccd015f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 -- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccd00af1b0 msgr2=0x7fccd015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccd00af1b0 0x7fccd015f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 -- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 msgr2=0x7fccd014ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 0x7fccd014ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.942+0000 7fccd4a00640 1 -- 172.21.15.28:0/113180740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd009c1a0 con 0x7fccd015bef0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fcccf400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 0x7fccd014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccd4a00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c7a0 tx=0x7fccc800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8003b70 con 0x7fccd015bef0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc8003d30 con 0x7fccd015bef0 2024-09-10T10:51:50.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800aa90 con 0x7fccd015bef0 2024-09-10T10:51:50.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd009c480 con 0x7fccd015bef0 2024-09-10T10:51:50.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.943+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd009ca40 con 0x7fccd015bef0 2024-09-10T10:51:50.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.944+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd01592a0 con 0x7fccd015bef0 2024-09-10T10:51:50.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.945+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fccc800ac30 con 0x7fccd015bef0 2024-09-10T10:51:50.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.946+0000 7fccc7e00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 0x7fccb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:50.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.947+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 7164+0+0 (secure 0 0 0) 0x7fccc808b440 con 0x7fccd015bef0 2024-09-10T10:51:50.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.947+0000 7fcccfe00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 0x7fccb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:50.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.948+0000 7fcccfe00640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 0x7fccb006e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fccc00011e0 tx=0x7fccc0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:50.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:50.951+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fccc80597a0 con 0x7fccd015bef0 2024-09-10T10:51:51.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.172+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd00b0940 con 0x7fccd015bef0 2024-09-10T10:51:51.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.172+0000 7fccc7e00640 1 -- 172.21.15.28:0/113180740 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) ==== 95+0+4401 (secure 0 0 0) 0x7fccc80590a0 con 0x7fccd015bef0 2024-09-10T10:51:51.173 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:51.173 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":145,"btime":"2024-09-10T10:51:50:760298+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:50.760289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8100":{"gid":8100,"name":"c","rank":0,"incarnation":145,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/1369429206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1369429206},{"type":"v1","addr":"172.21.15.28:6834","nonce":1369429206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:51.174 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 145 2024-09-10T10:51:51.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 msgr2=0x7fccb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:51.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 0x7fccb006e2f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fccc00011e0 tx=0x7fccc0009040 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 msgr2=0x7fccd01f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c7a0 tx=0x7fccc800cc60 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 shutdown_connections 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fccb006be30 0x7fccb006e2f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccd015bef0 0x7fccd01f4590 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd00b40e0 0x7fccd014ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 --2- 172.21.15.28:0/113180740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccd00af1b0 0x7fccd015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 >> 172.21.15.28:0/113180740 conn(0x7fccd006c240 msgr2=0x7fccd00b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 shutdown_connections 2024-09-10T10:51:51.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:51.174+0000 7fccd5400640 1 -- 172.21.15.28:0/113180740 wait complete. 2024-09-10T10:51:51.191 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:51:51.434 INFO:tasks.ceph.mds.a.smithi028.stderr: -43> 2024-09-10T10:51:46.681+0000 7f4dd3600640 -1 mds.0.inotable: load_2 could not read table: -2 2024-09-10T10:51:51.435 INFO:tasks.ceph.mds.a.smithi028.stderr: -42> 2024-09-10T10:51:46.681+0000 7f4dd3600640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2024-09-10T10:51:51.474 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:51:53.779 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:53.778+0000 7fb10c600640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-10T10:51:53.781 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:53.779+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:51:53.781 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:51:53.780+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:51:56.192 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:56.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.358+0000 7f83ab600640 1 Processor -- start 2024-09-10T10:51:56.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.358+0000 7f83ab600640 1 -- start start 2024-09-10T10:51:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.359+0000 7f83ab600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f83a4072700 0x7f83a415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.359+0000 7f83ab600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a4146140 con 0x7f83a4072de0 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.359+0000 7f83ab600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a40cd740 con 0x7f83a4072320 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.359+0000 7f83ab600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f83a40cda20 con 0x7f83a4072700 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a9600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f83a4072700 0x7f83a415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83a4072de0 0x7f83a415d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41180/0 (socket says 172.21.15.28:41180) 2024-09-10T10:51:56.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a3e00640 1 -- 172.21.15.28:0/253884720 learned_addr learned my addr 172.21.15.28:0/253884720 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a9600640 1 -- 172.21.15.28:0/253884720 >> v1:172.21.15.138:6789/0 conn(0x7f83a4072320 legacy=0x7f83a40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a9600640 1 -- 172.21.15.28:0/253884720 >> v1:172.21.15.28:6789/0 conn(0x7f83a4072de0 legacy=0x7f83a415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.360+0000 7f83a9600640 1 -- 172.21.15.28:0/253884720 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a40cdd00 con 0x7f83a4072700 2024-09-10T10:51:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.361+0000 7f83a9600640 1 --2- 172.21.15.28:0/253884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072700 0x7f83a415d8f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f839c009d90 tx=0x7f839c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c97dcfdba0dcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.361+0000 7f83a3400640 1 -- 172.21.15.28:0/253884720 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c031600 con 0x7f83a4072700 2024-09-10T10:51:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.361+0000 7f83a3400640 1 -- 172.21.15.28:0/253884720 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c0317a0 con 0x7f83a4072700 2024-09-10T10:51:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.361+0000 7f83a3400640 1 -- 172.21.15.28:0/253884720 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c031b70 con 0x7f83a4072700 2024-09-10T10:51:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 -- 172.21.15.28:0/253884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072700 msgr2=0x7f83a415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 --2- 172.21.15.28:0/253884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072700 0x7f83a415d8f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f839c009d90 tx=0x7f839c02a610 comp rx=0 tx=0).stop 2024-09-10T10:51:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 -- 172.21.15.28:0/253884720 shutdown_connections 2024-09-10T10:51:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 --2- 172.21.15.28:0/253884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072700 0x7f83a415d8f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 -- 172.21.15.28:0/253884720 >> 172.21.15.28:0/253884720 conn(0x7f83a4067d70 msgr2=0x7f83a406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 -- 172.21.15.28:0/253884720 shutdown_connections 2024-09-10T10:51:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.362+0000 7f83ab600640 1 -- 172.21.15.28:0/253884720 wait complete. 2024-09-10T10:51:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.364+0000 7f83ab600640 1 Processor -- start 2024-09-10T10:51:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.364+0000 7f83ab600640 1 -- start start 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a4072320 0x7f83a40ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a4072700 0x7f83a40ac150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83a415dfb0 con 0x7f83a4072de0 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83a415de30 con 0x7f83a4072700 2024-09-10T10:51:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83ab600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83a415e130 con 0x7f83a4072320 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52380/0 (socket says 172.21.15.28:52380) 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 -- 172.21.15.28:0/2407400123 learned_addr learned my addr 172.21.15.28:0/2407400123 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a8c00640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a4072320 0x7f83a40ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a3e00640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a4072700 0x7f83a40ac150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 -- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a4072320 msgr2=0x7f83a40ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a4072320 0x7f83a40ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 -- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a4072700 msgr2=0x7f83a40ac150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a4072700 0x7f83a40ac150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.365+0000 7f83a9600640 1 -- 172.21.15.28:0/2407400123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a4142180 con 0x7f83a4072de0 2024-09-10T10:51:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83a9600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f839c009eb0 tx=0x7f839c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c040070 con 0x7f83a4072de0 2024-09-10T10:51:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c030b60 con 0x7f83a4072de0 2024-09-10T10:51:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a4142460 con 0x7f83a4072de0 2024-09-10T10:51:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c03c420 con 0x7f83a4072de0 2024-09-10T10:51:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.366+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a4142a20 con 0x7f83a4072de0 2024-09-10T10:51:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.367+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a40cce60 con 0x7f83a4072de0 2024-09-10T10:51:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.368+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f839c039040 con 0x7f83a4072de0 2024-09-10T10:51:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.369+0000 7f83a0c00640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 0x7f838406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.369+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f839c0ac8f0 con 0x7f83a4072de0 2024-09-10T10:51:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.369+0000 7f83a8c00640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 0x7f838406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.370+0000 7f83a8c00640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 0x7f838406e320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8398007fd0 tx=0x7f8398009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.374+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f839c07ad40 con 0x7f83a4072de0 2024-09-10T10:51:56.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.597+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a40b2f80 con 0x7f83a4072de0 2024-09-10T10:51:56.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.597+0000 7f83a0c00640 1 -- 172.21.15.28:0/2407400123 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f839c07a640 con 0x7f83a4072de0 2024-09-10T10:51:56.599 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:56.599 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":149,"btime":"2024-09-10T10:51:53:780515+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:53.780512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:56.599 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 149 2024-09-10T10:51:56.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 msgr2=0x7f838406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 0x7f838406e320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8398007fd0 tx=0x7f8398009040 comp rx=0 tx=0).stop 2024-09-10T10:51:56.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 msgr2=0x7f83a40ac690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f839c009eb0 tx=0x7f839c004a40 comp rx=0 tx=0).stop 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 shutdown_connections 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f838406be60 0x7f838406e320 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83a4072de0 0x7f83a40ac690 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83a4072700 0x7f83a40ac150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 --2- 172.21.15.28:0/2407400123 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83a4072320 0x7f83a40ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 >> 172.21.15.28:0/2407400123 conn(0x7f83a4067d70 msgr2=0x7f83a40746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 shutdown_connections 2024-09-10T10:51:56.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.600+0000 7f83ab600640 1 -- 172.21.15.28:0/2407400123 wait complete. 2024-09-10T10:51:56.617 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:51:56.617 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:56.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.774+0000 7f34e3200640 1 Processor -- start 2024-09-10T10:51:56.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.774+0000 7f34e3200640 1 -- start start 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 0x7f34dc0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34dc0b25c0 0x7f34dc0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc0b5470 con 0x7f34dc0b25c0 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f34dc0b3640 con 0x7f34dc15bef0 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc0b5190 con 0x7f34dc0af1b0 2024-09-10T10:51:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.775+0000 7f34e1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34dc0b25c0 0x7f34dc0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34dc0b25c0 0x7f34dc0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52386/0 (socket says 172.21.15.28:52386) 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 -- 172.21.15.28:0/1375589481 learned_addr learned my addr 172.21.15.28:0/1375589481 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1e00640 1 --2- 172.21.15.28:0/1375589481 >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 0x7f34dc0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e2800640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 -- 172.21.15.28:0/1375589481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 msgr2=0x7f34dc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 -- 172.21.15.28:0/1375589481 >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 msgr2=0x7f34dc0b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 --2- 172.21.15.28:0/1375589481 >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 0x7f34dc0b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1400640 1 -- 172.21.15.28:0/1375589481 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc0b38e0 con 0x7f34dc0b25c0 2024-09-10T10:51:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e2800640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:56.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.776+0000 7f34e1e00640 1 --2- 172.21.15.28:0/1375589481 >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 0x7f34dc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.777+0000 7f34e1400640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc0aec70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06cbb0 tx=0x7f34cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c110a1fd558726 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.777+0000 7f34e0a00640 1 -- 172.21.15.28:0/1375589481 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc037070 con 0x7f34dc0b25c0 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.777+0000 7f34e0a00640 1 -- 172.21.15.28:0/1375589481 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc032440 con 0x7f34dc0b25c0 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.777+0000 7f34e0a00640 1 -- 172.21.15.28:0/1375589481 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc032810 con 0x7f34dc0b25c0 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 -- 172.21.15.28:0/1375589481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 msgr2=0x7f34dc0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc0aec70 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f34dc06cbb0 tx=0x7f34cc02a140 comp rx=0 tx=0).stop 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 -- 172.21.15.28:0/1375589481 shutdown_connections 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 --2- 172.21.15.28:0/1375589481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc0aec70 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 --2- 172.21.15.28:0/1375589481 >> v2:172.21.15.138:3300/0 conn(0x7f34dc15bef0 0x7f34dc0b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 -- 172.21.15.28:0/1375589481 >> 172.21.15.28:0/1375589481 conn(0x7f34dc06c240 msgr2=0x7f34dc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 -- 172.21.15.28:0/1375589481 shutdown_connections 2024-09-10T10:51:56.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.778+0000 7f34e3200640 1 -- 172.21.15.28:0/1375589481 wait complete. 2024-09-10T10:51:56.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.780+0000 7f34e3200640 1 Processor -- start 2024-09-10T10:51:56.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.780+0000 7f34e3200640 1 -- start start 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 0x7f34dc1e6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0b4e60 con 0x7f34dc0b25c0 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc0af640 con 0x7f34dc15bef0 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34dc0b4b10 con 0x7f34dc0af1b0 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52396/0 (socket says 172.21.15.28:52396) 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 -- 172.21.15.28:0/951443322 learned_addr learned my addr 172.21.15.28:0/951443322 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1e00640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e2800640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 0x7f34dc1e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 -- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 msgr2=0x7f34dc147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 -- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 msgr2=0x7f34dc1e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 0x7f34dc1e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e1400640 1 -- 172.21.15.28:0/951443322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1e57a0 con 0x7f34dc0b25c0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.781+0000 7f34e2800640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 0x7f34dc1e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34e1400640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02a180 tx=0x7f34cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc041020 con 0x7f34dc0b25c0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc0306c0 con 0x7f34dc0b25c0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1e5a80 con 0x7f34dc0b25c0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc037040 con 0x7f34dc0b25c0 2024-09-10T10:51:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.782+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1e6040 con 0x7f34dc0b25c0 2024-09-10T10:51:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.783+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc0b2a30 con 0x7f34dc0b25c0 2024-09-10T10:51:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.784+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34cc044020 con 0x7f34dc0b25c0 2024-09-10T10:51:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.785+0000 7f34d6000640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 0x7f34c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.785+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f34cc0abd20 con 0x7f34dc0b25c0 2024-09-10T10:51:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.786+0000 7f34e1e00640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 0x7f34c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.787+0000 7f34e1e00640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 0x7f34c006e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f34d0006fd0 tx=0x7f34d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:56.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:56.790+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34cc04b030 con 0x7f34dc0b25c0 2024-09-10T10:51:57.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.009+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34dc0b4210 con 0x7f34dc0b25c0 2024-09-10T10:51:57.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.010+0000 7f34d6000640 1 -- 172.21.15.28:0/951443322 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f34cc07a020 con 0x7f34dc0b25c0 2024-09-10T10:51:57.011 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:57.011 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":149,"btime":"2024-09-10T10:51:53:780515+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:53.780512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:57.011 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 149 2024-09-10T10:51:57.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 msgr2=0x7f34c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 0x7f34c006e2f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f34d0006fd0 tx=0x7f34d0008040 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 msgr2=0x7f34dc1487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f34cc02a180 tx=0x7f34cc0046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 shutdown_connections 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c006be30 0x7f34c006e2f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34dc15bef0 0x7f34dc1e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34dc0b25c0 0x7f34dc1487a0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 --2- 172.21.15.28:0/951443322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34dc0af1b0 0x7f34dc147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 >> 172.21.15.28:0/951443322 conn(0x7f34dc06c240 msgr2=0x7f34dc0b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 shutdown_connections 2024-09-10T10:51:57.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.012+0000 7f34e3200640 1 -- 172.21.15.28:0/951443322 wait complete. 2024-09-10T10:51:57.030 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:51:57.030 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:51:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.185+0000 7f136c200640 1 Processor -- start 2024-09-10T10:51:57.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.185+0000 7f136c200640 1 -- start start 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.186+0000 7f136c200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1364147230 0x7f136406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.186+0000 7f136c200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 0x7f13640720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.186+0000 7f136c200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1364072a00 con 0x7f13640725f0 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.186+0000 7f136c200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f13640b34d0 con 0x7f1364147230 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.186+0000 7f136c200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f13640ae9e0 con 0x7f1364147e90 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1364147230 0x7f136406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1364147230 0x7f136406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52400/0 (socket says 172.21.15.28:52400) 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 -- 172.21.15.28:0/538146115 learned_addr learned my addr 172.21.15.28:0/538146115 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 -- 172.21.15.28:0/538146115 >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 msgr2=0x7f13640720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1368e00640 1 --2- 172.21.15.28:0/538146115 >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 0x7f13640720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 --2- 172.21.15.28:0/538146115 >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 0x7f13640720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 -- 172.21.15.28:0/538146115 >> v1:172.21.15.138:6789/0 conn(0x7f13640725f0 legacy=0x7f1364161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1369800640 1 -- 172.21.15.28:0/538146115 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136406f480 con 0x7f1364147230 2024-09-10T10:51:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.187+0000 7f1368e00640 1 --2- 172.21.15.28:0/538146115 >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 0x7f13640720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.188+0000 7f1369800640 1 --2- 172.21.15.28:0/538146115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1364147230 0x7f136406e0b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f136406ccb0 tx=0x7f135802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82e8c57af48b59e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.188+0000 7f1363e00640 1 -- 172.21.15.28:0/538146115 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358037070 con 0x7f1364147230 2024-09-10T10:51:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.188+0000 7f1363e00640 1 -- 172.21.15.28:0/538146115 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358032440 con 0x7f1364147230 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.188+0000 7f1363e00640 1 -- 172.21.15.28:0/538146115 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358032810 con 0x7f1364147230 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 -- 172.21.15.28:0/538146115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1364147230 msgr2=0x7f136406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 --2- 172.21.15.28:0/538146115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1364147230 0x7f136406e0b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f136406ccb0 tx=0x7f135802a230 comp rx=0 tx=0).stop 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 -- 172.21.15.28:0/538146115 shutdown_connections 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 --2- 172.21.15.28:0/538146115 >> v2:172.21.15.138:3300/0 conn(0x7f1364147e90 0x7f13640720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 --2- 172.21.15.28:0/538146115 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1364147230 0x7f136406e0b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 -- 172.21.15.28:0/538146115 >> 172.21.15.28:0/538146115 conn(0x7f1364067d70 msgr2=0x7f136406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 -- 172.21.15.28:0/538146115 shutdown_connections 2024-09-10T10:51:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.189+0000 7f136c200640 1 -- 172.21.15.28:0/538146115 wait complete. 2024-09-10T10:51:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.191+0000 7f136c200640 1 Processor -- start 2024-09-10T10:51:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.191+0000 7f136c200640 1 -- start start 2024-09-10T10:51:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1364147230 0x7f1364130fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 0x7f13641314e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f136406eec0 con 0x7f13640725f0 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f136406ebe0 con 0x7f1364147e90 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136c200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f136406f1a0 con 0x7f1364147230 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f1369800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f1369800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52414/0 (socket says 172.21.15.28:52414) 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f1369800640 1 -- 172.21.15.28:0/3843375579 learned_addr learned my addr 172.21.15.28:0/3843375579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f1368e00640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1364147230 0x7f1364130fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.192+0000 7f136a200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 0x7f13641314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 -- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1364147230 msgr2=0x7f1364130fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1364147230 0x7f1364130fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 -- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 msgr2=0x7f13641314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 0x7f13641314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 -- 172.21.15.28:0/3843375579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640c5a90 con 0x7f13640725f0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f136a200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 0x7f13641314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1369800640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f13580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358037040 con 0x7f13640725f0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358042070 con 0x7f13640725f0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358030bb0 con 0x7f13640725f0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136412ed40 con 0x7f13640725f0 2024-09-10T10:51:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.193+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136412f300 con 0x7f13640725f0 2024-09-10T10:51:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.195+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1364147610 con 0x7f13640725f0 2024-09-10T10:51:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.195+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1358047460 con 0x7f13640725f0 2024-09-10T10:51:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.196+0000 7f1361600640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 0x7f134406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.196+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f13580ac180 con 0x7f13640725f0 2024-09-10T10:51:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.197+0000 7f1368e00640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 0x7f134406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.198+0000 7f1368e00640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 0x7f134406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13641f74c0 tx=0x7f1350008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.201+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1364072a00 con 0x7f13640725f0 2024-09-10T10:51:57.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.419+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1364073020 con 0x7f13640725f0 2024-09-10T10:51:57.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.419+0000 7f1361600640 1 -- 172.21.15.28:0/3843375579 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 149 v149) ==== 95+0+4392 (secure 0 0 0) 0x7f135807a400 con 0x7f13640725f0 2024-09-10T10:51:57.421 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:51:57.421 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":149,"btime":"2024-09-10T10:51:53:780515+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":149,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:51:53.780512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:51:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 149 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.421+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 msgr2=0x7f134406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.421+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 0x7f134406e2f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f13641f74c0 tx=0x7f1350008040 comp rx=0 tx=0).stop 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.421+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 msgr2=0x7f13640c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.421+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f13580046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 shutdown_connections 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f134406be30 0x7f134406e2f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1364147e90 0x7f13641314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1364147230 0x7f1364130fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 --2- 172.21.15.28:0/3843375579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13640725f0 0x7f13640c5440 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 >> 172.21.15.28:0/3843375579 conn(0x7f1364067d70 msgr2=0x7f136406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 shutdown_connections 2024-09-10T10:51:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.422+0000 7f136c200640 1 -- 172.21.15.28:0/3843375579 wait complete. 2024-09-10T10:51:57.439 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000' led to DAMAGED state 2024-09-10T10:51:57.439 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete mds0_sessionmap' 2024-09-10T10:51:57.440 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:51:57.440 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:51:57.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:51:57.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:51:57.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:51:57.476 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:51:57.550 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:51:57.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:51:57.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:51:57.746 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:51:57.744+0000 7fb111600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:57.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.747+0000 7f31ac600640 1 Processor -- start 2024-09-10T10:51:57.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.747+0000 7f31ac600640 1 -- start start 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.748+0000 7f31ac600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31a40b0180 0x7f31a4162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.748+0000 7f31ac600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 0x7f31a4154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.748+0000 7f31ac600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a415eb60 con 0x7f31a40b69d0 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.748+0000 7f31ac600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f31a40b6db0 con 0x7f31a40b0180 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.748+0000 7f31ac600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f31a415ee40 con 0x7f31a40af280 2024-09-10T10:51:57.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31a40b0180 0x7f31a4162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31a40b0180 0x7f31a4162a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52420/0 (socket says 172.21.15.28:52420) 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 -- 172.21.15.28:0/837478714 learned_addr learned my addr 172.21.15.28:0/837478714 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 -- 172.21.15.28:0/837478714 >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 msgr2=0x7f31a4154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31aa600640 1 --2- 172.21.15.28:0/837478714 >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 0x7f31a4154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 --2- 172.21.15.28:0/837478714 >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 0x7f31a4154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 -- 172.21.15.28:0/837478714 >> v1:172.21.15.138:6789/0 conn(0x7f31a40b69d0 legacy=0x7f31a40afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31a9200640 1 -- 172.21.15.28:0/837478714 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a415f120 con 0x7f31a40b0180 2024-09-10T10:51:57.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.749+0000 7f31aa600640 1 --2- 172.21.15.28:0/837478714 >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 0x7f31a4154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:57.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.750+0000 7f31a9200640 1 --2- 172.21.15.28:0/837478714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a4162a50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f31a40b4740 tx=0x7f319002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c932ac6616e1665a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.750+0000 7f31a3e00640 1 -- 172.21.15.28:0/837478714 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190037070 con 0x7f31a40b0180 2024-09-10T10:51:57.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.750+0000 7f31a3e00640 1 -- 172.21.15.28:0/837478714 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319002ae20 con 0x7f31a40b0180 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.750+0000 7f31a3e00640 1 -- 172.21.15.28:0/837478714 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190032660 con 0x7f31a40b0180 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 -- 172.21.15.28:0/837478714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 msgr2=0x7f31a4162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 --2- 172.21.15.28:0/837478714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a4162a50 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f31a40b4740 tx=0x7f319002a110 comp rx=0 tx=0).stop 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 -- 172.21.15.28:0/837478714 shutdown_connections 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 --2- 172.21.15.28:0/837478714 >> v2:172.21.15.138:3300/0 conn(0x7f31a40af280 0x7f31a4154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 --2- 172.21.15.28:0/837478714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a4162a50 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 -- 172.21.15.28:0/837478714 >> 172.21.15.28:0/837478714 conn(0x7f31a406bc10 msgr2=0x7f31a406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 -- 172.21.15.28:0/837478714 shutdown_connections 2024-09-10T10:51:57.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.751+0000 7f31ac600640 1 -- 172.21.15.28:0/837478714 wait complete. 2024-09-10T10:51:57.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.752+0000 7f31ac600640 1 Processor -- start 2024-09-10T10:51:57.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.753+0000 7f31ac600640 1 -- start start 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.753+0000 7f31ac600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a40af280 0x7f31a4150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31ac600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31ac600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 0x7f31a4160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31ac600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4155240 con 0x7f31a40b0180 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31ac600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31a40af6d0 con 0x7f31a40af280 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31ac600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31a41553c0 con 0x7f31a40b69d0 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52426/0 (socket says 172.21.15.28:52426) 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 -- 172.21.15.28:0/3651973966 learned_addr learned my addr 172.21.15.28:0/3651973966 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9c00640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a40af280 0x7f31a4150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31aa600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 0x7f31a4160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 -- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 msgr2=0x7f31a4160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 0x7f31a4160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 -- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a40af280 msgr2=0x7f31a4150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a40af280 0x7f31a4150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31a9200640 1 -- 172.21.15.28:0/3651973966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a41f4ce0 con 0x7f31a40b0180 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.754+0000 7f31aa600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 0x7f31a4160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31a9200640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3190000c00 tx=0x7f31900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190041070 con 0x7f31a40b0180 2024-09-10T10:51:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319003c680 con 0x7f31a40b0180 2024-09-10T10:51:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a41f4fc0 con 0x7f31a40b0180 2024-09-10T10:51:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190037040 con 0x7f31a40b0180 2024-09-10T10:51:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.755+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41f5580 con 0x7f31a40b0180 2024-09-10T10:51:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.756+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a40b3e70 con 0x7f31a40b0180 2024-09-10T10:51:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.757+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3190044070 con 0x7f31a40b0180 2024-09-10T10:51:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.758+0000 7f31a1600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 0x7f318406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.758+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f31900abaf0 con 0x7f31a40b0180 2024-09-10T10:51:57.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.758+0000 7f31a9c00640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 0x7f318406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:57.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.759+0000 7f31a9c00640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 0x7f318406e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3198006fd0 tx=0x7f3198008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:57.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.762+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f31a40b3e70 con 0x7f31a40b0180 2024-09-10T10:51:57.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:57.984+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f31a4155750 con 0x7f31a40b0180 2024-09-10T10:51:58.533 INFO:tasks.ceph.mds.c.smithi028.stderr: -57> 2024-09-10T10:51:53.778+0000 7fb10c600640 -1 log_channel(cluster) log [ERR] : dir 0x100 object missing on disk; some files may be lost (~mds0) 2024-09-10T10:51:58.533 INFO:tasks.ceph.mds.c.smithi028.stderr: -17> 2024-09-10T10:51:57.744+0000 7fb111600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:51:58.575 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:51:58.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.668+0000 7f31a1600640 1 -- 172.21.15.28:0/3651973966 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v150) ==== 161+0+0 (secure 0 0 0) 0x7f319002ae20 con 0x7f31a40b0180 2024-09-10T10:51:58.671 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:51:58.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 msgr2=0x7f318406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 0x7f318406e2f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3198006fd0 tx=0x7f3198008040 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 msgr2=0x7f31a415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f3190000c00 tx=0x7f31900046c0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 shutdown_connections 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f318406be30 0x7f318406e2f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31a40b69d0 0x7f31a4160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31a40b0180 0x7f31a415de30 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.671+0000 7f31ac600640 1 --2- 172.21.15.28:0/3651973966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31a40af280 0x7f31a4150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.672+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 >> 172.21.15.28:0/3651973966 conn(0x7f31a406bc10 msgr2=0x7f31a40b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.672+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 shutdown_connections 2024-09-10T10:51:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.672+0000 7f31ac600640 1 -- 172.21.15.28:0/3651973966 wait complete. 2024-09-10T10:51:58.691 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:51:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.847+0000 7f5cf4e00640 1 Processor -- start 2024-09-10T10:51:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.847+0000 7f5cf4e00640 1 -- start start 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.848+0000 7f5cf4e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.848+0000 7f5cf4e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5cf00d1f90 0x7f5cf00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.848+0000 7f5cf4e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf00cf000 con 0x7f5cf015e660 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.848+0000 7f5cf4e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf006d3e0 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.848+0000 7f5cf4e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5cf00b37d0 con 0x7f5cf00d1f90 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5ceea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5cf015e660 0x7f5cf006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41204/0 (socket says 172.21.15.28:41204) 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5ceea00640 1 -- 172.21.15.28:0/877746195 learned_addr learned my addr 172.21.15.28:0/877746195 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:51:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52468/0 (socket says 172.21.15.28:52468) 2024-09-10T10:51:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cefe00640 1 --2- 172.21.15.28:0/877746195 >> v2:172.21.15.138:3300/0 conn(0x7f5cf00d1f90 0x7f5cf00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 -- 172.21.15.28:0/877746195 >> v2:172.21.15.138:3300/0 conn(0x7f5cf00d1f90 msgr2=0x7f5cf00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 --2- 172.21.15.28:0/877746195 >> v2:172.21.15.138:3300/0 conn(0x7f5cf00d1f90 0x7f5cf00cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 -- 172.21.15.28:0/877746195 >> v1:172.21.15.28:6789/0 conn(0x7f5cf015e660 legacy=0x7f5cf006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.849+0000 7f5cef400640 1 -- 172.21.15.28:0/877746195 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf00d16f0 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.850+0000 7f5cef400640 1 --2- 172.21.15.28:0/877746195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5cf008cb90 tx=0x7f5ce002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da075d78cb0ad04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.850+0000 7f5cee000640 1 -- 172.21.15.28:0/877746195 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0037070 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.850+0000 7f5cee000640 1 -- 172.21.15.28:0/877746195 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce0032440 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.850+0000 7f5cee000640 1 -- 172.21.15.28:0/877746195 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0032810 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 -- 172.21.15.28:0/877746195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf00d0dd0 msgr2=0x7f5cf00d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/877746195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5cf008cb90 tx=0x7f5ce002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 -- 172.21.15.28:0/877746195 shutdown_connections 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/877746195 >> v2:172.21.15.138:3300/0 conn(0x7f5cf00d1f90 0x7f5cf00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/877746195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00d11b0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 -- 172.21.15.28:0/877746195 >> 172.21.15.28:0/877746195 conn(0x7f5cf008bfc0 msgr2=0x7f5cf00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 -- 172.21.15.28:0/877746195 shutdown_connections 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.851+0000 7f5cf4e00640 1 -- 172.21.15.28:0/877746195 wait complete. 2024-09-10T10:51:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.853+0000 7f5cf4e00640 1 Processor -- start 2024-09-10T10:51:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.853+0000 7f5cf4e00640 1 -- start start 2024-09-10T10:51:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00b7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 0x7f5cf01e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00d2c60 con 0x7f5cf015e660 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf00cf330 con 0x7f5cf00d0dd0 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cf4e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf00d2f40 con 0x7f5cf00d1f90 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52472/0 (socket says 172.21.15.28:52472) 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 -- 172.21.15.28:0/1682362897 learned_addr learned my addr 172.21.15.28:0/1682362897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cef400640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00b7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5ceea00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 0x7f5cf01e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 -- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 msgr2=0x7f5cf01e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 0x7f5cf01e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 -- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf00d0dd0 msgr2=0x7f5cf00b7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00b7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5cefe00640 1 -- 172.21.15.28:0/1682362897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf00b8590 con 0x7f5cf015e660 2024-09-10T10:51:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.854+0000 7f5ceea00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 0x7f5cf01e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5cefe00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007e70 tx=0x7f5ce800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8014620 con 0x7f5cf015e660 2024-09-10T10:51:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce8019070 con 0x7f5cf015e660 2024-09-10T10:51:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8014a20 con 0x7f5cf015e660 2024-09-10T10:51:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf00b8870 con 0x7f5cf015e660 2024-09-10T10:51:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.855+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf00b8e30 con 0x7f5cf015e660 2024-09-10T10:51:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.856+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf00d3220 con 0x7f5cf015e660 2024-09-10T10:51:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.857+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ce8014bc0 con 0x7f5cf015e660 2024-09-10T10:51:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.858+0000 7f5ce7400640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 0x7f5cd006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.858+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f5ce808bce0 con 0x7f5cf015e660 2024-09-10T10:51:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.858+0000 7f5cef400640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 0x7f5cd006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:58.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.859+0000 7f5cef400640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 0x7f5cd006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0034000 tx=0x7f5ce002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:58.863+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ce805a070 con 0x7f5cf015e660 2024-09-10T10:51:59.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.114+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f5cf00caec0 con 0x7f5cf015e660 2024-09-10T10:51:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.675+0000 7f5ce7400640 1 -- 172.21.15.28:0/1682362897 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v151) ==== 101+0+0 (secure 0 0 0) 0x7f5ce8059970 con 0x7f5cf015e660 2024-09-10T10:51:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 msgr2=0x7f5cd006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 0x7f5cd006e2f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0034000 tx=0x7f5ce002e000 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 msgr2=0x7f5cf014b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8007e70 tx=0x7f5ce800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 shutdown_connections 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5cd006be30 0x7f5cd006e2f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.678+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cf015e660 0x7f5cf014b5b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.679+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cf00d1f90 0x7f5cf01e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.679+0000 7f5cf4e00640 1 --2- 172.21.15.28:0/1682362897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf00d0dd0 0x7f5cf00b7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.679+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 >> 172.21.15.28:0/1682362897 conn(0x7f5cf008bfc0 msgr2=0x7f5cf00d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.679+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 shutdown_connections 2024-09-10T10:51:59.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.679+0000 7f5cf4e00640 1 -- 172.21.15.28:0/1682362897 wait complete. 2024-09-10T10:51:59.699 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:51:59.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.774+0000 7f0d0140ff40 1 Processor -- start 2024-09-10T10:51:59.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.774+0000 7f0d0140ff40 1 -- start start 2024-09-10T10:51:59.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c67639dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67633f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564c67508b30 con 0x564c676399c0 2024-09-10T10:51:59.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564c6750cb70 con 0x564c676363e0 2024-09-10T10:51:59.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.775+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564c67638ee0 con 0x564c67560470 2024-09-10T10:51:59.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52476/0 (socket says 172.21.15.28:52476) 2024-09-10T10:51:59.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 -- 172.21.15.28:0/4205550852 learned_addr learned my addr 172.21.15.28:0/4205550852 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfda00640 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c67639dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfd000640 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67633f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 -- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 msgr2=0x564c67639dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c67639dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 -- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 msgr2=0x564c67633f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67633f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 -- 172.21.15.28:0/4205550852 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c67633a90 con 0x564c676399c0 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cfe400640 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800a3f0 tx=0x7f0cf8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77fd08c92ad8b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.776+0000 7f0cf7400640 1 -- 172.21.15.28:0/4205550852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8058950 con 0x564c676399c0 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0cf7400640 1 -- 172.21.15.28:0/4205550852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8067070 con 0x564c676399c0 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0cf7400640 1 -- 172.21.15.28:0/4205550852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8058cf0 con 0x564c676399c0 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 -- 172.21.15.28:0/4205550852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 msgr2=0x564c67633550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800a3f0 tx=0x7f0cf8051000 comp rx=0 tx=0).stop 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 -- 172.21.15.28:0/4205550852 shutdown_connections 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c67633550 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67633f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/4205550852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c67639dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 -- 172.21.15.28:0/4205550852 >> 172.21.15.28:0/4205550852 conn(0x564c675af0a0 msgr2=0x564c676382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 -- 172.21.15.28:0/4205550852 shutdown_connections 2024-09-10T10:51:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.777+0000 7f0d0140ff40 1 -- 172.21.15.28:0/4205550852 wait complete. 2024-09-10T10:51:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.778+0000 7f0d0140ff40 1 Processor -- start 2024-09-10T10:51:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.778+0000 7f0d0140ff40 1 -- start start 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c675a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67634f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564c675af520 con 0x564c676399c0 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564c67636850 con 0x564c676363e0 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0d0140ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564c675af800 con 0x564c67560470 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52492/0 (socket says 172.21.15.28:52492) 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 learned_addr learned my addr 172.21.15.28:0/3474913790 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67634f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c675a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 msgr2=0x564c675a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564c67560470 0x564c675a8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 msgr2=0x564c67634f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67634f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6763a880 con 0x564c676399c0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.779+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564c676363e0 0x564c67634f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0cfe400640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8046170 tx=0x7f0cf8003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0cf6000640 1 -- 172.21.15.28:0/3474913790 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8075400 con 0x564c676399c0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0cf6000640 1 -- 172.21.15.28:0/3474913790 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf80755a0 con 0x564c676399c0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0cf6000640 1 -- 172.21.15.28:0/3474913790 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8075950 con 0x564c676399c0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564c6763ab60 con 0x564c676399c0 2024-09-10T10:51:59.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.780+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c6763b120 con 0x564c676399c0 2024-09-10T10:51:59.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.782+0000 7f0cf6000640 1 -- 172.21.15.28:0/3474913790 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0cf804f400 con 0x564c676399c0 2024-09-10T10:51:59.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.783+0000 7f0cf6000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 0x7f0ce006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.783+0000 7f0cf6000640 1 -- 172.21.15.28:0/3474913790 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f0cf80cf860 con 0x564c676399c0 2024-09-10T10:51:59.785 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.783+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 0x7f0ce006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.783+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 0x564c675931b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.783+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676cee50 con 0x564c6763baf0 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.784+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 0x564c675931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.784+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 0x7f0ce006ccd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0003f10 tx=0x7f0cf0003890 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.784+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 0x564c675931b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.786+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v111'319 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.786+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676cf220 con 0x564c6763baf0 2024-09-10T10:51:59.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.788+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v111'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.788+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676cf5f0 con 0x564c6763baf0 2024-09-10T10:51:59.791 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:51:59.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.790+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v111'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.790+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dc3b0 con 0x564c6763baf0 2024-09-10T10:51:59.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.792+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v111'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.792+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dc750 con 0x564c6763baf0 2024-09-10T10:51:59.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.793+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v111'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.793+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db3a0 con 0x564c6763baf0 2024-09-10T10:51:59.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.795+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v111'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.795+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db790 con 0x564c6763baf0 2024-09-10T10:51:59.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.797+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v111'325 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.797+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676db790 con 0x564c6763baf0 2024-09-10T10:51:59.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.798+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v111'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.799+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.800+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v111'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.802 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:51:59.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.800+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.802+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v111'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.802+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.804+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v111'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.804+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.806+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v111'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.806+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.807+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v111'331 uv306 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.807+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.809+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v111'332 uv332 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.809+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.812 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:51:59.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.810+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v111'333 uv333 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.811+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.812+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v111'334 uv334 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.812+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.813+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v111'335 uv335 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.814+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676db740 con 0x564c6763baf0 2024-09-10T10:51:59.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.815+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v111'336 uv336 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.815+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbfb0 con 0x564c6763baf0 2024-09-10T10:51:59.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.817+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v111'337 uv337 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.817+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.819+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v111'338 uv312 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.819+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.820+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v111'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.820+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.824 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:51:59.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.822+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v111'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.822+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.824+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v111'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.824+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.825+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v111'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.825+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.827+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v111'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.827+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.829+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v111'344 uv318 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.829+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.830+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v111'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.830+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.833 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:51:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.831+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v111'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.832+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.833+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v111'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.833+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.835+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v111'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.835+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c6763baf0 2024-09-10T10:51:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.836+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v111'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c6763baf0 2024-09-10T10:51:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.836+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564c676dd000 0x564c676df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.836+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dbeb0 con 0x564c676dd000 2024-09-10T10:51:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.837+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564c676dd000 0x564c676df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.837+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564c676dd000 0x564c676df3f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.839+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v111'241 uv223 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.839+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676df930 con 0x564c676dd000 2024-09-10T10:51:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.841+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v111'242 uv242 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.841+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676dfcd0 con 0x564c676dd000 2024-09-10T10:51:59.844 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:51:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.842+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v111'243 uv243 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.842+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0070 con 0x564c676dd000 2024-09-10T10:51:59.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.844+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v111'244 uv244 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.844+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0410 con 0x564c676dd000 2024-09-10T10:51:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.845+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v111'245 uv245 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.845+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e07b0 con 0x564c676dd000 2024-09-10T10:51:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.846+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v111'246 uv228 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.846+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0b50 con 0x564c676dd000 2024-09-10T10:51:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.847+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v111'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.847+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.850 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:51:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.849+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v111'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.849+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.850+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v111'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.850+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.851+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v111'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.851+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.853+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v111'251 uv233 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.853+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.854+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v111'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.854+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.857 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:51:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.855+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v111'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.855+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.857+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v111'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.857+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.858+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v111'255 uv255 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.858+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0ef0 con 0x564c676dd000 2024-09-10T10:51:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.859+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v111'256 uv239 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.859+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e0f30 con 0x564c676dd000 2024-09-10T10:51:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.860+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v111'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.860+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e1380 con 0x564c676dd000 2024-09-10T10:51:59.863 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:51:59.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.862+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v111'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.862+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e2390 con 0x564c676dd000 2024-09-10T10:51:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.863+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v111'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.863+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e2310 con 0x564c676dd000 2024-09-10T10:51:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.864+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v111'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.864+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e1300 con 0x564c676dd000 2024-09-10T10:51:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.865+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v111'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676dd000 2024-09-10T10:51:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.865+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x564c676e1980 0x564c676e1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.865+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e46e0 con 0x564c676e1980 2024-09-10T10:51:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.866+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x564c676e1980 0x564c676e1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.866+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x564c676e1980 0x564c676e1dc0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.868+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v111'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.868+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e46e0 con 0x564c676e1980 2024-09-10T10:51:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.869+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v111'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.869+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e46e0 con 0x564c676e1980 2024-09-10T10:51:59.872 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:51:59.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.870+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v111'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.871+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e5770 con 0x564c676e1980 2024-09-10T10:51:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.872+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v111'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.872+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e5730 con 0x564c676e1980 2024-09-10T10:51:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.873+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v111'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.874+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e5730 con 0x564c676e1980 2024-09-10T10:51:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.875+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v111'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.875+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564c676e4bf0 0x564c676e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.875+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e7bb0 con 0x564c676e4bf0 2024-09-10T10:51:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.875+0000 7f0cfe400640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564c676e4bf0 0x564c676e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.876+0000 7f0cfe400640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564c676e4bf0 0x564c676e5040 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.877+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v111'49 uv48 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.877+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e7f50 con 0x564c676e4bf0 2024-09-10T10:51:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.878+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v111'50 uv50 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.878+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8390 con 0x564c676e4bf0 2024-09-10T10:51:59.881 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:51:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.879+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v111'51 uv51 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.879+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8c50 con 0x564c676e4bf0 2024-09-10T10:51:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.880+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v111'52 uv52 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.880+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8ff0 con 0x564c676e1980 2024-09-10T10:51:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.881+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v111'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.881+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8ff0 con 0x564c676e1980 2024-09-10T10:51:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.882+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v111'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.882+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8ff0 con 0x564c676e1980 2024-09-10T10:51:59.885 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:51:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.884+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v111'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.884+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ea000 con 0x564c676e1980 2024-09-10T10:51:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.885+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v111'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.885+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ea000 con 0x564c676e1980 2024-09-10T10:51:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.886+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v111'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.886+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8ff0 con 0x564c676e1980 2024-09-10T10:51:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.887+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v111'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.887+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e8ff0 con 0x564c676e4bf0 2024-09-10T10:51:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.889+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v111'116 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.889+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9390 con 0x564c676e4bf0 2024-09-10T10:51:59.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.890+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v111'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.890+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9730 con 0x564c676e4bf0 2024-09-10T10:51:59.893 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:51:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.891+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v111'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.891+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676eaae0 con 0x564c676e4bf0 2024-09-10T10:51:59.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.892+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v111'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.892+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676eaae0 con 0x564c676e4bf0 2024-09-10T10:51:59.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.894+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v111'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.894+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.895+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v111'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.895+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.896+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v111'122 uv115 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.896+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.898+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v111'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.898+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.899+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v111'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.901 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:51:59.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.899+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.901+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v111'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.901+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e4bf0 2024-09-10T10:51:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.902+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v111'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.902+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e1980 2024-09-10T10:51:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.903+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v111'51 uv50 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.903+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e1980 2024-09-10T10:51:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.904+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v111'52 uv52 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.904+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e1980 2024-09-10T10:51:59.907 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:51:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.905+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v111'53 uv53 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.905+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e1980 2024-09-10T10:51:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.907+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v111'54 uv54 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.907+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676e9ad0 con 0x564c676e1980 2024-09-10T10:51:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.908+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v111'55 uv55 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676e1980 2024-09-10T10:51:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.908+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564c676ea150 0x564c676ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.908+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ecac0 con 0x564c676ea150 2024-09-10T10:51:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.908+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564c676ea150 0x564c676ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.909+0000 7f0cfda00640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564c676ea150 0x564c676ec540 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.910+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v111'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.910+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.911+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v111'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.911+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.914 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:51:59.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.913+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v111'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.913+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676edea0 con 0x564c676ea150 2024-09-10T10:51:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.914+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v111'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.914+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676edea0 con 0x564c676ea150 2024-09-10T10:51:59.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.915+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v111'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.915+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.916+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v111'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.916+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.918+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v111'104 uv97 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.918+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.919+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v111'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.919+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.922 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:51:59.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.920+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v111'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.920+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676eced0 con 0x564c676ea150 2024-09-10T10:51:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.921+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v111'107 uv107 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.921+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.923+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v111'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.923+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.924+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v111'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.926 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:51:59.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.924+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ece90 con 0x564c676ea150 2024-09-10T10:51:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.925+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [omap-set-header] v111'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.925+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f00b0 con 0x564c676ea150 2024-09-10T10:51:59.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.926+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [omap-set-vals] v111'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.926+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:99 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.928+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(99 mds0_sessionmap [create] v111'49 uv47 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.928+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:100 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.929+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(100 mds0_sessionmap [delete] v111'50 uv50 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.929+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.932 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:51:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.930+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(101 mds0_sessionmap [create] v111'51 uv51 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.930+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.931+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v111'52 uv52 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.932+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.933+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(103 609.00000000 [create] v111'63 uv62 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.933+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.934+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(104 609.00000000 [delete] v111'64 uv64 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.934+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.937 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:51:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.935+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(105 609.00000000 [create] v111'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.935+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ee760 con 0x564c676e4bf0 2024-09-10T10:51:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.936+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v111'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.936+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ee760 con 0x564c676e4bf0 2024-09-10T10:51:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.937+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v111'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.937+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676ed750 con 0x564c676e4bf0 2024-09-10T10:51:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.939+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(108 609.00000000 [omap-set-header] v111'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cf807c070 con 0x564c676e4bf0 2024-09-10T10:51:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.939+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564c676edc60 0x564c676f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:51:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.939+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676edc60 2024-09-10T10:51:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.939+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564c676edc60 0x564c676f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:51:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.940+0000 7f0cfd000640 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564c676edc60 0x564c676f0050 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:51:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.941+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v111'52 uv51 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676edc60 2024-09-10T10:51:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.941+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676edc60 2024-09-10T10:51:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.942+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v111'53 uv53 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676edc60 2024-09-10T10:51:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.942+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676edc60 2024-09-10T10:51:59.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.943+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v111'54 uv54 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676edc60 2024-09-10T10:51:59.945 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:51:59.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.943+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676edc60 2024-09-10T10:51:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.944+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v111'55 uv55 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676edc60 2024-09-10T10:51:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.944+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676edc60 2024-09-10T10:51:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.946+0000 7f0cfd000640 1 -- 172.21.15.28:0/3474913790 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v111'56 uv56 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f0cec00d3d0 con 0x564c676edc60 2024-09-10T10:51:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.946+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.947+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(114 100.00000000.inode [create] v111'132 uv123 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.947+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.948+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(115 100.00000000.inode [delete] v111'133 uv133 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.948+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.949+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v111'134 uv134 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:51:59.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.949+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.950+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v111'135 uv135 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.950+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.952+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v111'136 uv136 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.952+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.953+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 1.00000000 [create] v111'137 uv130 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.953+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.954+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 1.00000000 [delete] v111'138 uv138 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.954+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.955+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v111'139 uv139 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.957 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:51:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.955+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f15a0 con 0x564c676ea150 2024-09-10T10:51:59.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.956+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v111'140 uv140 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.956+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f15a0 con 0x564c676ea150 2024-09-10T10:51:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.957+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v111'141 uv141 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.957+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0590 con 0x564c676ea150 2024-09-10T10:51:59.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.959+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v111'142 uv142 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.959+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111) -- 0x564c676f0cf0 con 0x564c676ea150 2024-09-10T10:51:59.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0cfda00640 1 -- 172.21.15.28:0/3474913790 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v111'143 uv143 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f0cf001b040 con 0x564c676ea150 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x564c676e1980 msgr2=0x564c676e1dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x564c676e1980 0x564c676e1dc0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564c676ea150 msgr2=0x564c676ec540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564c676ea150 0x564c676ec540 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564c676edc60 msgr2=0x564c676f0050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564c676edc60 0x564c676f0050 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564c676e4bf0 msgr2=0x564c676e5040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564c676e4bf0 0x564c676e5040 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564c676dd000 msgr2=0x564c676df3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 reap_dead start 2024-09-10T10:51:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x564c676dd000 0x564c676df3f0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0cfe400640 1 -- 172.21.15.28:0/3474913790 reap_dead start 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 msgr2=0x564c675931b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 0x564c675931b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 msgr2=0x7f0ce006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 0x7f0ce006ccd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0003f10 tx=0x7f0cf0003890 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 msgr2=0x564c6762f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8046170 tx=0x7f0cf8003330 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 shutdown_connections 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564c6763baf0 0x564c675931b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ce006a810 0x7f0ce006ccd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 --2- 172.21.15.28:0/3474913790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564c676399c0 0x564c6762f830 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:51:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 >> 172.21.15.28:0/3474913790 conn(0x564c675af0a0 msgr2=0x564c67638140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:51:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 shutdown_connections 2024-09-10T10:51:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:51:59.960+0000 7f0d0140ff40 1 -- 172.21.15.28:0/3474913790 wait complete. 2024-09-10T10:51:59.967 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm mds0_sessionmap 2024-09-10T10:52:00.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.080+0000 7f502ff25f40 1 Processor -- start 2024-09-10T10:52:00.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.080+0000 7f502ff25f40 1 -- start start 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab090483e0 0x55ab08f71d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 0x55ab0903ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ab08fbd530 con 0x55ab08fbcff0 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab08f1ab30 con 0x55ab0904b9c0 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502ff25f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ab0904aee0 con 0x55ab090483e0 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502c600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502c600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52496/0 (socket says 172.21.15.28:52496) 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.081+0000 7f502c600640 1 -- 172.21.15.28:0/2177722428 learned_addr learned my addr 172.21.15.28:0/2177722428 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:00.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502d000640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 0x55ab0903ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502bc00640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab090483e0 0x55ab08f71d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 -- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab090483e0 msgr2=0x55ab08f71d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab090483e0 0x55ab08f71d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 -- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 msgr2=0x55ab0903ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 0x55ab0903ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 -- 172.21.15.28:0/2177722428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab09048a80 con 0x55ab08fbcff0 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502d000640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 0x55ab0903ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502c600640 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4b21ce8c06db00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502a800640 1 -- 172.21.15.28:0/2177722428 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c063070 con 0x55ab08fbcff0 2024-09-10T10:52:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502a800640 1 -- 172.21.15.28:0/2177722428 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501c055920 con 0x55ab08fbcff0 2024-09-10T10:52:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.082+0000 7f502a800640 1 -- 172.21.15.28:0/2177722428 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c055cf0 con 0x55ab08fbcff0 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 -- 172.21.15.28:0/2177722428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 msgr2=0x55ab0903f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f501c000c00 tx=0x7f501c051000 comp rx=0 tx=0).stop 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 -- 172.21.15.28:0/2177722428 shutdown_connections 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab0904b9c0 0x55ab0903ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab090483e0 0x55ab08f71d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 --2- 172.21.15.28:0/2177722428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab08fbcff0 0x55ab0903f810 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 -- 172.21.15.28:0/2177722428 >> 172.21.15.28:0/2177722428 conn(0x55ab08fc10a0 msgr2=0x55ab08fc1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 -- 172.21.15.28:0/2177722428 shutdown_connections 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.083+0000 7f502ff25f40 1 -- 172.21.15.28:0/2177722428 wait complete. 2024-09-10T10:52:00.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.084+0000 7f502ff25f40 1 Processor -- start 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.084+0000 7f502ff25f40 1 -- start start 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab08fbcff0 0x55ab0904eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 0x55ab090d9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab0904b9c0 0x55ab090dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ab09042680 con 0x55ab090483e0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab08fa4420 con 0x55ab08fbcff0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502ff25f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ab0904a600 con 0x55ab0904b9c0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 0x55ab090d9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 0x55ab090d9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52504/0 (socket says 172.21.15.28:52504) 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 -- 172.21.15.28:0/467678740 learned_addr learned my addr 172.21.15.28:0/467678740 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502c600640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab08fbcff0 0x55ab0904eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502d000640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab0904b9c0 0x55ab090dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 -- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab0904b9c0 msgr2=0x55ab090dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ab0904b9c0 0x55ab090dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 -- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab08fbcff0 msgr2=0x55ab0904eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab08fbcff0 0x55ab0904eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.085+0000 7f502bc00640 1 -- 172.21.15.28:0/467678740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0903b470 con 0x55ab090483e0 2024-09-10T10:52:00.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f502bc00640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 0x55ab090d9cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5018000f60 tx=0x7f501800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f5029400640 1 -- 172.21.15.28:0/467678740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800f650 con 0x55ab090483e0 2024-09-10T10:52:00.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f5029400640 1 -- 172.21.15.28:0/467678740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501800f7f0 con 0x55ab090483e0 2024-09-10T10:52:00.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f5029400640 1 -- 172.21.15.28:0/467678740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800fbe0 con 0x55ab090483e0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab08fbb840 con 0x55ab090483e0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.086+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0903b170 con 0x55ab090483e0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.088+0000 7f5029400640 1 -- 172.21.15.28:0/467678740 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5018017400 con 0x55ab090483e0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f5029400640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f500c06a810 0x7f500c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f5029400640 1 -- 172.21.15.28:0/467678740 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f501808a6e0 con 0x55ab090483e0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f502c600640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f500c06a810 0x7f500c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f502ff25f40 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ab08fa5dd0 0x55ab090e4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e111) -- 0x55ab0903b6e0 con 0x55ab08fa5dd0 2024-09-10T10:52:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.089+0000 7f502d000640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ab08fa5dd0 0x55ab090e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.090+0000 7f502d000640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ab08fa5dd0 0x55ab090e4840 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.090+0000 7f502c600640 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f500c06a810 0x7f500c06ccd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f501c067000 tx=0x7f501c053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502d000640 1 -- 172.21.15.28:0/467678740 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_sessionmap [delete] v111'53 uv53 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f502400abe0 con 0x55ab08fa5dd0 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ab08fa5dd0 msgr2=0x55ab090e4840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ab08fa5dd0 0x55ab090e4840 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f500c06a810 msgr2=0x7f500c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f500c06a810 0x7f500c06ccd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f501c067000 tx=0x7f501c053000 comp rx=0 tx=0).stop 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 msgr2=0x55ab090d9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 --2- 172.21.15.28:0/467678740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ab090483e0 0x55ab090d9cd0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f5018000f60 tx=0x7f501800b6c0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502d000640 1 -- 172.21.15.28:0/467678740 reap_dead start 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 shutdown_connections 2024-09-10T10:52:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.091+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 >> 172.21.15.28:0/467678740 conn(0x55ab08fc10a0 msgr2=0x55ab08fc1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.092+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 shutdown_connections 2024-09-10T10:52:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.092+0000 7f502ff25f40 1 -- 172.21.15.28:0/467678740 wait complete. 2024-09-10T10:52:00.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:00.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.255+0000 7f18eae00640 1 Processor -- start 2024-09-10T10:52:00.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.255+0000 7f18eae00640 1 -- start start 2024-09-10T10:52:00.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18eae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 0x7f18e4162980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18eae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e40d2600 con 0x7f18e40d3320 2024-09-10T10:52:00.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18eae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e40b3c10 con 0x7f18e40d4c10 2024-09-10T10:52:00.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18eae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18e40d5420 con 0x7f18e40b5f80 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18e8e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f18e40d3320 0x7f18e40ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41234/0 (socket says 172.21.15.28:41234) 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18e8e00640 1 -- 172.21.15.28:0/1995716427 learned_addr learned my addr 172.21.15.28:0/1995716427 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.256+0000 7f18e3e00640 1 --2- 172.21.15.28:0/1995716427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 0x7f18e4162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192118079 0 0) 0x7f18d0001110 con 0x7f18e40d4c10 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18c8003520 con 0x7f18e40d4c10 2024-09-10T10:52:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641970789 0 0) 0x7f18dc001490 con 0x7f18e40d3320 2024-09-10T10:52:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e3e00640 1 -- 172.21.15.28:0/1995716427 >> v1:172.21.15.138:6789/0 conn(0x7f18e40d4c10 legacy=0x7f18e40b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e3e00640 1 -- 172.21.15.28:0/1995716427 >> v1:172.21.15.28:6789/0 conn(0x7f18e40d3320 legacy=0x7f18e40ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.257+0000 7f18e3e00640 1 -- 172.21.15.28:0/1995716427 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e40d2880 con 0x7f18e40b5f80 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.258+0000 7f18e3e00640 1 --2- 172.21.15.28:0/1995716427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 0x7f18e4162980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18d40097d0 tx=0x7f18d402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71392ac27b69a01b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.258+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4037070 con 0x7f18e40b5f80 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.258+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d4032440 con 0x7f18e40b5f80 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.258+0000 7f18e2a00640 1 -- 172.21.15.28:0/1995716427 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4032810 con 0x7f18e40b5f80 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 -- 172.21.15.28:0/1995716427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 msgr2=0x7f18e4162980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 --2- 172.21.15.28:0/1995716427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 0x7f18e4162980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f18d40097d0 tx=0x7f18d402a210 comp rx=0 tx=0).stop 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 -- 172.21.15.28:0/1995716427 shutdown_connections 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 --2- 172.21.15.28:0/1995716427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40b5f80 0x7f18e4162980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 -- 172.21.15.28:0/1995716427 >> 172.21.15.28:0/1995716427 conn(0x7f18e408bfc0 msgr2=0x7f18e40d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 -- 172.21.15.28:0/1995716427 shutdown_connections 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.259+0000 7f18eae00640 1 -- 172.21.15.28:0/1995716427 wait complete. 2024-09-10T10:52:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.261+0000 7f18eae00640 1 Processor -- start 2024-09-10T10:52:00.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.261+0000 7f18eae00640 1 -- start start 2024-09-10T10:52:00.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.261+0000 7f18eae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.261+0000 7f18eae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40d3320 0x7f18e4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18eae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 0x7f18e40b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18eae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18e40d2b20 con 0x7f18e40b5f80 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18eae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18e40d2e90 con 0x7f18e40d4c10 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18eae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18e415f690 con 0x7f18e40d3320 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52520/0 (socket says 172.21.15.28:52520) 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 -- 172.21.15.28:0/2839069731 learned_addr learned my addr 172.21.15.28:0/2839069731 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3400640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40d3320 0x7f18e4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e8e00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 0x7f18e40b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 -- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40d3320 msgr2=0x7f18e4150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40d3320 0x7f18e4150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 -- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 msgr2=0x7f18e40b9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 0x7f18e40b9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e3e00640 1 -- 172.21.15.28:0/2839069731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e40ca4d0 con 0x7f18e40b5f80 2024-09-10T10:52:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.262+0000 7f18e8e00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 0x7f18e40b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:00.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18e3e00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f18d4035040 tx=0x7f18d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4041030 con 0x7f18e40b5f80 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d403d4e0 con 0x7f18e40b5f80 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4037040 con 0x7f18e40b5f80 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e40ca7b0 con 0x7f18e40b5f80 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.263+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e40cad70 con 0x7f18e40b5f80 2024-09-10T10:52:00.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.264+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18e40d37b0 con 0x7f18e40b5f80 2024-09-10T10:52:00.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.265+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f18d4044070 con 0x7f18e40b5f80 2024-09-10T10:52:00.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.266+0000 7f18dbe00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 0x7f18c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.266+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 7250+0+0 (secure 0 0 0) 0x7f18d40abbe0 con 0x7f18e40b5f80 2024-09-10T10:52:00.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.266+0000 7f18e3400640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 0x7f18c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.267+0000 7f18e3400640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 0x7f18c406e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f18e415d3b0 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.270+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18d407a000 con 0x7f18e40b5f80 2024-09-10T10:52:00.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.490+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f18e415fd00 con 0x7f18e40b5f80 2024-09-10T10:52:00.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.679+0000 7f18dbe00640 1 -- 172.21.15.28:0/2839069731 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v152) ==== 162+0+0 (secure 0 0 0) 0x7f18d4079900 con 0x7f18e40b5f80 2024-09-10T10:52:00.681 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 msgr2=0x7f18c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 0x7f18c406e2f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f18e415d3b0 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 msgr2=0x7f18e40c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f18d4035040 tx=0x7f18d40046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 shutdown_connections 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f18c406be30 0x7f18c406e2f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18e40d4c10 0x7f18e40b9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18e40d3320 0x7f18e4150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 --2- 172.21.15.28:0/2839069731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18e40b5f80 0x7f18e40c8110 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.681+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 >> 172.21.15.28:0/2839069731 conn(0x7f18e408bfc0 msgr2=0x7f18e4160ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.682+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 shutdown_connections 2024-09-10T10:52:00.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.682+0000 7f18eae00640 1 -- 172.21.15.28:0/2839069731 wait complete. 2024-09-10T10:52:00.688 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:00.686+0000 7f8f15800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-10T10:52:00.688 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:00.686+0000 7f8f15800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-10T10:52:00.689 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:00.687+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:00.689 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:00.687+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:00.702 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:00.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.856+0000 7ff066c00640 1 Processor -- start 2024-09-10T10:52:00.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.856+0000 7ff066c00640 1 -- start start 2024-09-10T10:52:00.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.857+0000 7ff066c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 0x7ff0600cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.857+0000 7ff066c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff060146140 con 0x7ff06015e1f0 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.857+0000 7ff066c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06006eed0 con 0x7ff060072de0 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.857+0000 7ff066c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0600cd740 con 0x7ff060072320 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff064c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff06015e1f0 0x7ff06006e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41248/0 (socket says 172.21.15.28:41248) 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff064c00640 1 -- 172.21.15.28:0/1143104601 learned_addr learned my addr 172.21.15.28:0/1143104601 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:00.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05fe00640 1 --2- 172.21.15.28:0/1143104601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 0x7ff0600cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1957222171 0 0) 0x7ff060146140 con 0x7ff06015e1f0 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff044003520 con 0x7ff06015e1f0 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05fe00640 1 -- 172.21.15.28:0/1143104601 >> v1:172.21.15.138:6789/0 conn(0x7ff060072de0 legacy=0x7ff06006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1128047079 0 0) 0x7ff06006eed0 con 0x7ff060072de0 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05fe00640 1 -- 172.21.15.28:0/1143104601 >> v1:172.21.15.28:6789/0 conn(0x7ff06015e1f0 legacy=0x7ff06006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.858+0000 7ff05fe00640 1 -- 172.21.15.28:0/1143104601 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0600cda20 con 0x7ff060072320 2024-09-10T10:52:00.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.859+0000 7ff05fe00640 1 --2- 172.21.15.28:0/1143104601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 0x7ff0600cce60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0500097d0 tx=0x7ff05002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=713c02cef0ca793b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050037070 con 0x7ff060072320 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff050032440 con 0x7ff060072320 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff05f400640 1 -- 172.21.15.28:0/1143104601 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050032810 con 0x7ff060072320 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 -- 172.21.15.28:0/1143104601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 msgr2=0x7ff0600cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 --2- 172.21.15.28:0/1143104601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 0x7ff0600cce60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0500097d0 tx=0x7ff05002a180 comp rx=0 tx=0).stop 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 -- 172.21.15.28:0/1143104601 shutdown_connections 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 --2- 172.21.15.28:0/1143104601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072320 0x7ff0600cce60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 -- 172.21.15.28:0/1143104601 >> 172.21.15.28:0/1143104601 conn(0x7ff060067d70 msgr2=0x7ff06006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 -- 172.21.15.28:0/1143104601 shutdown_connections 2024-09-10T10:52:00.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.860+0000 7ff066c00640 1 -- 172.21.15.28:0/1143104601 wait complete. 2024-09-10T10:52:00.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.862+0000 7ff066c00640 1 Processor -- start 2024-09-10T10:52:00.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.862+0000 7ff066c00640 1 -- start start 2024-09-10T10:52:00.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072de0 0x7ff0601f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06015e1f0 0x7ff0600a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff0600cd3a0 con 0x7ff060072320 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff06015c9e0 con 0x7ff06015e1f0 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff066c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0600cd520 con 0x7ff060072de0 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff05fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff05fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52532/0 (socket says 172.21.15.28:52532) 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff05fe00640 1 -- 172.21.15.28:0/3833637730 learned_addr learned my addr 172.21.15.28:0/3833637730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff057e00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072de0 0x7ff0601f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.863+0000 7ff064c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06015e1f0 0x7ff0600a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 -- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072de0 msgr2=0x7ff0601f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072de0 0x7ff0601f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 -- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06015e1f0 msgr2=0x7ff0600a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06015e1f0 0x7ff0600a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:00.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 -- 172.21.15.28:0/3833637730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0600aa5c0 con 0x7ff060072320 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05fe00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff0500097d0 tx=0x7ff0500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050037040 con 0x7ff060072320 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff050004230 con 0x7ff060072320 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050006e10 con 0x7ff060072320 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0600aa8a0 con 0x7ff060072320 2024-09-10T10:52:00.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.864+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0600aae60 con 0x7ff060072320 2024-09-10T10:52:00.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.865+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff060072700 con 0x7ff060072320 2024-09-10T10:52:00.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.866+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff05003a070 con 0x7ff060072320 2024-09-10T10:52:00.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.867+0000 7ff05cc00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 0x7ff03c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:00.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.867+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7ff0500abd50 con 0x7ff060072320 2024-09-10T10:52:00.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.868+0000 7ff057e00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 0x7ff03c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:00.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.868+0000 7ff057e00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 0x7ff03c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0600ccf40 tx=0x7ff048009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:00.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:00.871+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff050041020 con 0x7ff060072320 2024-09-10T10:52:01.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.094+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff060145900 con 0x7ff060072320 2024-09-10T10:52:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.095+0000 7ff05cc00640 1 -- 172.21.15.28:0/3833637730 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7ff050079a40 con 0x7ff060072320 2024-09-10T10:52:01.097 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:01.097 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":154,"btime":"2024-09-10T10:52:00:689405+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:00.689349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 154 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 msgr2=0x7ff03c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 0x7ff03c06e2f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff0600ccf40 tx=0x7ff048009040 comp rx=0 tx=0).stop 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 msgr2=0x7ff0600b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff0500097d0 tx=0x7ff0500046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 shutdown_connections 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff03c06be30 0x7ff03c06e2f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06015e1f0 0x7ff0600a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff060072de0 0x7ff0601f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 --2- 172.21.15.28:0/3833637730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff060072320 0x7ff0600b5250 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 >> 172.21.15.28:0/3833637730 conn(0x7ff060067d70 msgr2=0x7ff0601439e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 shutdown_connections 2024-09-10T10:52:01.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.097+0000 7ff066c00640 1 -- 172.21.15.28:0/3833637730 wait complete. 2024-09-10T10:52:01.115 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:01.115 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:01.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.270+0000 7fc385e00640 1 Processor -- start 2024-09-10T10:52:01.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.270+0000 7fc385e00640 1 -- start start 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.271+0000 7fc385e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc380147360 0x7fc380070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.271+0000 7fc385e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc380070dd0 0x7fc3800d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.271+0000 7fc385e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3800d09e0 0x7fc38006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc385e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc38006cb30 con 0x7fc380147360 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc385e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3800d0e50 con 0x7fc380070dd0 2024-09-10T10:52:01.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc385e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc38006bfb0 con 0x7fc3800d09e0 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc380147360 0x7fc380070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc380147360 0x7fc380070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52534/0 (socket says 172.21.15.28:52534) 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 -- 172.21.15.28:0/3578871467 learned_addr learned my addr 172.21.15.28:0/3578871467 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37f400640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3800d09e0 0x7fc38006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc377e00640 1 --2- 172.21.15.28:0/3578871467 >> v2:172.21.15.138:3300/0 conn(0x7fc380070dd0 0x7fc3800d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 -- 172.21.15.28:0/3578871467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3800d09e0 msgr2=0x7fc38006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3800d09e0 0x7fc38006f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 -- 172.21.15.28:0/3578871467 >> v2:172.21.15.138:3300/0 conn(0x7fc380070dd0 msgr2=0x7fc3800d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 --2- 172.21.15.28:0/3578871467 >> v2:172.21.15.138:3300/0 conn(0x7fc380070dd0 0x7fc3800d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.272+0000 7fc37ea00640 1 -- 172.21.15.28:0/3578871467 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38006c290 con 0x7fc380147360 2024-09-10T10:52:01.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.273+0000 7fc37ea00640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc380147360 0x7fc380070890 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc3800cec70 tx=0x7fc37002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57e3a662e626e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.273+0000 7fc37e000640 1 -- 172.21.15.28:0/3578871467 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370037070 con 0x7fc380147360 2024-09-10T10:52:01.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc37e000640 1 -- 172.21.15.28:0/3578871467 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc370032440 con 0x7fc380147360 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc37e000640 1 -- 172.21.15.28:0/3578871467 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370032810 con 0x7fc380147360 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 -- 172.21.15.28:0/3578871467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc380147360 msgr2=0x7fc380070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc380147360 0x7fc380070890 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc3800cec70 tx=0x7fc37002a140 comp rx=0 tx=0).stop 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 -- 172.21.15.28:0/3578871467 shutdown_connections 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3800d09e0 0x7fc38006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 --2- 172.21.15.28:0/3578871467 >> v2:172.21.15.138:3300/0 conn(0x7fc380070dd0 0x7fc3800d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 --2- 172.21.15.28:0/3578871467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc380147360 0x7fc380070890 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 -- 172.21.15.28:0/3578871467 >> 172.21.15.28:0/3578871467 conn(0x7fc3800b2f40 msgr2=0x7fc3800b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 -- 172.21.15.28:0/3578871467 shutdown_connections 2024-09-10T10:52:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.274+0000 7fc385e00640 1 -- 172.21.15.28:0/3578871467 wait complete. 2024-09-10T10:52:01.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.276+0000 7fc385e00640 1 Processor -- start 2024-09-10T10:52:01.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.276+0000 7fc385e00640 1 -- start start 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc380070dd0 0x7fc38013ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 0x7fc38013d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc38006bc10 con 0x7fc3800d09e0 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc38006c850 con 0x7fc380147360 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc385e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc38006bd90 con 0x7fc380070dd0 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52542/0 (socket says 172.21.15.28:52542) 2024-09-10T10:52:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 -- 172.21.15.28:0/1300274909 learned_addr learned my addr 172.21.15.28:0/1300274909 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc37f400640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 0x7fc38013d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc37ea00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc380070dd0 0x7fc38013ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 -- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc380070dd0 msgr2=0x7fc38013ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc380070dd0 0x7fc38013ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 -- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 msgr2=0x7fc38013d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 0x7fc38013d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.277+0000 7fc377e00640 1 -- 172.21.15.28:0/1300274909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3800b4240 con 0x7fc3800d09e0 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc37f400640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 0x7fc38013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:01.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc377e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc36800d800 tx=0x7fc36800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368018030 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3680096a0 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38013b150 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368009870 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.278+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38013ca00 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.279+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc380067e90 con 0x7fc3800d09e0 2024-09-10T10:52:01.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.280+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc36801b070 con 0x7fc3800d09e0 2024-09-10T10:52:01.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.281+0000 7fc377400640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 0x7fc35c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.281+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7fc36808b850 con 0x7fc3800d09e0 2024-09-10T10:52:01.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.281+0000 7fc37ea00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 0x7fc35c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.282+0000 7fc37ea00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 0x7fc35c06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc370034000 tx=0x7fc37002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.285+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc380067e90 con 0x7fc3800d09e0 2024-09-10T10:52:01.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.507+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3800d04a0 con 0x7fc3800d09e0 2024-09-10T10:52:01.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.508+0000 7fc377400640 1 -- 172.21.15.28:0/1300274909 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7fc368059b00 con 0x7fc3800d09e0 2024-09-10T10:52:01.510 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:01.510 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":154,"btime":"2024-09-10T10:52:00:689405+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:00.689349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:01.511 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 154 2024-09-10T10:52:01.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 msgr2=0x7fc35c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 0x7fc35c06e2f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc370034000 tx=0x7fc37002e000 comp rx=0 tx=0).stop 2024-09-10T10:52:01.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 msgr2=0x7fc38013bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc36800d800 tx=0x7fc36800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 shutdown_connections 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc35c06be30 0x7fc35c06e2f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc380147360 0x7fc38013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3800d09e0 0x7fc38013bdc0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 --2- 172.21.15.28:0/1300274909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc380070dd0 0x7fc38013ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 >> 172.21.15.28:0/1300274909 conn(0x7fc3800b2f40 msgr2=0x7fc3800ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 shutdown_connections 2024-09-10T10:52:01.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.511+0000 7fc385e00640 1 -- 172.21.15.28:0/1300274909 wait complete. 2024-09-10T10:52:01.532 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:01.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:01.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.686+0000 7f6efbe00640 1 Processor -- start 2024-09-10T10:52:01.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.686+0000 7f6efbe00640 1 -- start start 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.687+0000 7f6efbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ef4161c80 0x7f6ef406e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.687+0000 7f6efbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ef4162060 0x7f6ef40b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6efbe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef406e870 con 0x7f6ef40b64b0 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6efbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef406d9f0 con 0x7f6ef4161c80 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6efbe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef406dcd0 con 0x7f6ef4162060 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ef4161c80 0x7f6ef406e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6ef40b64b0 0x7f6ef4162500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41258/0 (socket says 172.21.15.28:41258) 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9e00640 1 -- 172.21.15.28:0/132039154 learned_addr learned my addr 172.21.15.28:0/132039154 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef8a00640 1 --2- 172.21.15.28:0/132039154 >> v2:172.21.15.138:3300/0 conn(0x7f6ef4162060 0x7f6ef40b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9400640 1 -- 172.21.15.28:0/132039154 >> v2:172.21.15.138:3300/0 conn(0x7f6ef4162060 msgr2=0x7f6ef40b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9400640 1 --2- 172.21.15.28:0/132039154 >> v2:172.21.15.138:3300/0 conn(0x7f6ef4162060 0x7f6ef40b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9400640 1 -- 172.21.15.28:0/132039154 >> v1:172.21.15.28:6789/0 conn(0x7f6ef40b64b0 legacy=0x7f6ef4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef9400640 1 -- 172.21.15.28:0/132039154 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef406dfb0 con 0x7f6ef4161c80 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.688+0000 7f6ef3e00640 1 -- 172.21.15.28:0/132039154 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148657393 0 0) 0x7f6eec001490 con 0x7f6ef40b64b0 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.689+0000 7f6ef9400640 1 --2- 172.21.15.28:0/132039154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef406e330 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6ef408e830 tx=0x7f6ee802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78d18cfa27e1389 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6ef3e00640 1 -- 172.21.15.28:0/132039154 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8037070 con 0x7f6ef4161c80 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6ef3e00640 1 -- 172.21.15.28:0/132039154 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee8032440 con 0x7f6ef4161c80 2024-09-10T10:52:01.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6ef3e00640 1 -- 172.21.15.28:0/132039154 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8032810 con 0x7f6ef4161c80 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6efbe00640 1 -- 172.21.15.28:0/132039154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 msgr2=0x7f6ef406e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6efbe00640 1 --2- 172.21.15.28:0/132039154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef406e330 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f6ef408e830 tx=0x7f6ee802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6efbe00640 1 -- 172.21.15.28:0/132039154 shutdown_connections 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.690+0000 7f6efbe00640 1 --2- 172.21.15.28:0/132039154 >> v2:172.21.15.138:3300/0 conn(0x7f6ef4162060 0x7f6ef40b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.691+0000 7f6efbe00640 1 --2- 172.21.15.28:0/132039154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef406e330 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.691+0000 7f6efbe00640 1 -- 172.21.15.28:0/132039154 >> 172.21.15.28:0/132039154 conn(0x7f6ef40d44f0 msgr2=0x7f6ef40d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:01.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.691+0000 7f6efbe00640 1 -- 172.21.15.28:0/132039154 shutdown_connections 2024-09-10T10:52:01.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.691+0000 7f6efbe00640 1 -- 172.21.15.28:0/132039154 wait complete. 2024-09-10T10:52:01.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.692+0000 7f6efbe00640 1 Processor -- start 2024-09-10T10:52:01.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.692+0000 7f6efbe00640 1 -- start start 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef40b64b0 0x7f6ef40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ef4162060 0x7f6ef4155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40d2a10 con 0x7f6ef4161c80 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40d2890 con 0x7f6ef40b64b0 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.693+0000 7f6efbe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef40b6960 con 0x7f6ef4162060 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52570/0 (socket says 172.21.15.28:52570) 2024-09-10T10:52:01.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 -- 172.21.15.28:0/1599712737 learned_addr learned my addr 172.21.15.28:0/1599712737 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef9400640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef40b64b0 0x7f6ef40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef9e00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ef4162060 0x7f6ef4155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 -- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ef4162060 msgr2=0x7f6ef4155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ef4162060 0x7f6ef4155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 -- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef40b64b0 msgr2=0x7f6ef40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef40b64b0 0x7f6ef40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.694+0000 7f6ef8a00640 1 -- 172.21.15.28:0/1599712737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef40b74e0 con 0x7f6ef4161c80 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6ef8a00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000e870 tx=0x7f6ee000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0017070 con 0x7f6ef4161c80 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee000c940 con 0x7f6ef4161c80 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000cd10 con 0x7f6ef4161c80 2024-09-10T10:52:01.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef40b77c0 con 0x7f6ef4161c80 2024-09-10T10:52:01.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.695+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef40b7d80 con 0x7f6ef4161c80 2024-09-10T10:52:01.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.696+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef40ba470 con 0x7f6ef4161c80 2024-09-10T10:52:01.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.697+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ee001a070 con 0x7f6ef4161c80 2024-09-10T10:52:01.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.698+0000 7f6ef1600640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 0x7f6ed406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:01.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.698+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f6ee008b760 con 0x7f6ef4161c80 2024-09-10T10:52:01.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.698+0000 7f6ef9400640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 0x7f6ed406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:01.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.699+0000 7f6ef9400640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 0x7f6ed406e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8034000 tx=0x7f6ee802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:01.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.702+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ee0059330 con 0x7f6ef4161c80 2024-09-10T10:52:01.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.923+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef4160950 con 0x7f6ef4161c80 2024-09-10T10:52:01.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.923+0000 7f6ef1600640 1 -- 172.21.15.28:0/1599712737 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) ==== 95+0+4392 (secure 0 0 0) 0x7f6ee005d630 con 0x7f6ef4161c80 2024-09-10T10:52:01.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:01.926 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":154,"btime":"2024-09-10T10:52:00:689405+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":146}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:00.689349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:01.926 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 154 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 msgr2=0x7f6ed406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 0x7f6ed406e2f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8034000 tx=0x7f6ee802e000 comp rx=0 tx=0).stop 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 msgr2=0x7f6ef40b9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000e870 tx=0x7f6ee000ed30 comp rx=0 tx=0).stop 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 shutdown_connections 2024-09-10T10:52:01.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ed406be30 0x7f6ed406e2f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ef4162060 0x7f6ef4155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef4161c80 0x7f6ef40b9ce0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 --2- 172.21.15.28:0/1599712737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ef40b64b0 0x7f6ef40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 >> 172.21.15.28:0/1599712737 conn(0x7f6ef40d44f0 msgr2=0x7f6ef40d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 shutdown_connections 2024-09-10T10:52:01.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:01.926+0000 7f6efbe00640 1 -- 172.21.15.28:0/1599712737 wait complete. 2024-09-10T10:52:01.944 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete mds0_sessionmap' led to DAMAGED state 2024-09-10T10:52:01.944 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 400.00000000' 2024-09-10T10:52:01.944 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:01.944 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:01.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:01.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:01.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:01.980 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:02.054 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:02.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:02.055 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:02.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.250+0000 7f1f66600640 1 Processor -- start 2024-09-10T10:52:02.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.250+0000 7f1f66600640 1 -- start start 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f66600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f60072320 0x7f1f600cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f66600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f60146140 con 0x7f1f60072de0 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f66600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f6006eed0 con 0x7f1f6015e1f0 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f66600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f600cd740 con 0x7f1f60072320 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f5ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1f60072de0 0x7f1f6006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41264/0 (socket says 172.21.15.28:41264) 2024-09-10T10:52:02.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f5ea00640 1 -- 172.21.15.28:0/2880179992 learned_addr learned my addr 172.21.15.28:0/2880179992 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4287649026 0 0) 0x7f1f60146140 con 0x7f1f60072de0 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f5f400640 1 --2- 172.21.15.28:0/2880179992 >> v2:172.21.15.138:3300/0 conn(0x7f1f60072320 0x7f1f600cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.252+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f44003520 con 0x7f1f60072de0 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3145268109 0 0) 0x7f1f44003520 con 0x7f1f60072de0 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 >> v2:172.21.15.138:3300/0 conn(0x7f1f60072320 msgr2=0x7f1f600cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 --2- 172.21.15.28:0/2880179992 >> v2:172.21.15.138:3300/0 conn(0x7f1f60072320 0x7f1f600cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 >> v1:172.21.15.138:6789/0 conn(0x7f1f6015e1f0 legacy=0x7f1f6006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600cda20 con 0x7f1f60072de0 2024-09-10T10:52:02.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f1f4c001670 con 0x7f1f60072de0 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 >> v1:172.21.15.28:6789/0 conn(0x7f1f60072de0 legacy=0x7f1f6006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44003a00 0x7f1f44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 0x7f1f4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600cd740 con 0x7f1f44005210 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400c160 con 0x7f1f44003a00 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4400c360 con 0x7f1f44004800 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5fe00640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 0x7f1f4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5f400640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44003a00 0x7f1f44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.253+0000 7f1f5ea00640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 -- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f44004800 msgr2=0x7f1f44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:02.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f44004800 0x7f1f44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 -- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44003a00 msgr2=0x7f1f44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:02.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44003a00 0x7f1f44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 -- 172.21.15.28:0/2880179992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44003520 con 0x7f1f44005210 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.254+0000 7f1f5fe00640 1 -- 172.21.15.28:0/2880179992 reap_dead start 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f5fe00640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 0x7f1f4400b9c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1f6015e680 tx=0x7f1f44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23d42d57eeebc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58001a70 con 0x7f1f44005210 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f5802adf0 con 0x7f1f44005210 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f5e000640 1 -- 172.21.15.28:0/2880179992 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58003b90 con 0x7f1f44005210 2024-09-10T10:52:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f66600640 1 -- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 msgr2=0x7f1f4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f66600640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 0x7f1f4400b9c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f1f6015e680 tx=0x7f1f44005830 comp rx=0 tx=0).stop 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f66600640 1 -- 172.21.15.28:0/2880179992 shutdown_connections 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f66600640 1 --2- 172.21.15.28:0/2880179992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f44005210 0x7f1f4400b9c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.255+0000 7f1f66600640 1 -- 172.21.15.28:0/2880179992 >> 172.21.15.28:0/2880179992 conn(0x7f1f60067d70 msgr2=0x7f1f6006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.256+0000 7f1f66600640 1 -- 172.21.15.28:0/2880179992 shutdown_connections 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.256+0000 7f1f66600640 1 -- 172.21.15.28:0/2880179992 wait complete. 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.257+0000 7f1f66600640 1 Processor -- start 2024-09-10T10:52:02.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.257+0000 7f1f66600640 1 -- start start 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44005210 0x7f1f601f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f60140160 0x7f1f60142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60072700 con 0x7f1f600b0f90 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6015c9e0 con 0x7f1f44005210 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f66600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f60072880 con 0x7f1f60140160 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f5ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f5ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52594/0 (socket says 172.21.15.28:52594) 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.258+0000 7f1f5ea00640 1 -- 172.21.15.28:0/2297837619 learned_addr learned my addr 172.21.15.28:0/2297837619 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5f400640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44005210 0x7f1f601f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5fe00640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f60140160 0x7f1f60142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 -- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f60140160 msgr2=0x7f1f60142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f60140160 0x7f1f60142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 -- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44005210 msgr2=0x7f1f601f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44005210 0x7f1f601f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 -- 172.21.15.28:0/2297837619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f600b1550 con 0x7f1f600b0f90 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f5ea00640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0072f0 tx=0x7f1f4c0076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.259+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c01a070 con 0x7f1f600b0f90 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.260+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c007760 con 0x7f1f600b0f90 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.260+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c00fb60 con 0x7f1f600b0f90 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.260+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601e3380 con 0x7f1f600b0f90 2024-09-10T10:52:02.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.260+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601e3940 con 0x7f1f600b0f90 2024-09-10T10:52:02.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.261+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f600cce60 con 0x7f1f600b0f90 2024-09-10T10:52:02.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.262+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f4c00fd00 con 0x7f1f600b0f90 2024-09-10T10:52:02.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.262+0000 7f1f57400640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 0x7f1f4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.262+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f1f4c08abd0 con 0x7f1f600b0f90 2024-09-10T10:52:02.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.263+0000 7f1f5f400640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 0x7f1f4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.264+0000 7f1f5f400640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 0x7f1f4006e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cee70 tx=0x7f1f50009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.267+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f4c0588c0 con 0x7f1f600b0f90 2024-09-10T10:52:02.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.486+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1f6013f090 con 0x7f1f600b0f90 2024-09-10T10:52:02.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.689+0000 7f1f57400640 1 -- 172.21.15.28:0/2297837619 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v155) ==== 161+0+0 (secure 0 0 0) 0x7f1f4c088e20 con 0x7f1f600b0f90 2024-09-10T10:52:02.691 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 msgr2=0x7f1f4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 0x7f1f4006e2f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cee70 tx=0x7f1f50009190 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 msgr2=0x7f1f6013eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0072f0 tx=0x7f1f4c0076f0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 shutdown_connections 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f4006be30 0x7f1f4006e2f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f60140160 0x7f1f60142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f600b0f90 0x7f1f6013eb50 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 --2- 172.21.15.28:0/2297837619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44005210 0x7f1f601f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 >> 172.21.15.28:0/2297837619 conn(0x7f1f60067d70 msgr2=0x7f1f6006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 shutdown_connections 2024-09-10T10:52:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.691+0000 7f1f66600640 1 -- 172.21.15.28:0/2297837619 wait complete. 2024-09-10T10:52:02.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:02.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.865+0000 7f2f5a400640 1 Processor -- start 2024-09-10T10:52:02.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.865+0000 7f2f5a400640 1 -- start start 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.866+0000 7f2f5a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f54072d30 0x7f2f5406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f5a400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f5406f700 0x7f2f5415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f5a400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f540cdbe0 con 0x7f2f5415b300 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f5a400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f5406fb50 con 0x7f2f5406f700 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f5a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f540728a0 con 0x7f2f54072d30 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f53400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2f5415b300 0x7f2f540cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41280/0 (socket says 172.21.15.28:41280) 2024-09-10T10:52:02.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f53400640 1 -- 172.21.15.28:0/1818576772 learned_addr learned my addr 172.21.15.28:0/1818576772 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f53e00640 1 --2- 172.21.15.28:0/1818576772 >> v2:172.21.15.138:3300/0 conn(0x7f2f5406f700 0x7f2f5415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f52a00640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f54072d30 0x7f2f5406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2428814239 0 0) 0x7f2f540cdbe0 con 0x7f2f5415b300 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.867+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f38003520 con 0x7f2f5415b300 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.868+0000 7f2f53e00640 1 -- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f54072d30 msgr2=0x7f2f5406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.868+0000 7f2f53e00640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f54072d30 0x7f2f5406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.868+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 195104270 0 0) 0x7f2f38003520 con 0x7f2f5415b300 2024-09-10T10:52:02.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.868+0000 7f2f53e00640 1 -- 172.21.15.28:0/1818576772 >> v1:172.21.15.28:6789/0 conn(0x7f2f5415b300 legacy=0x7f2f540cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.868+0000 7f2f53e00640 1 -- 172.21.15.28:0/1818576772 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f540cd900 con 0x7f2f5406f700 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.869+0000 7f2f53e00640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f5415b6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009d90 tx=0x7f2f4c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3442df34d38d5d3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.869+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44001670 con 0x7f2f5406f700 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.869+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c007540 con 0x7f2f5406f700 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.869+0000 7f2f52000640 1 -- 172.21.15.28:0/1818576772 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c007910 con 0x7f2f5406f700 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 -- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 msgr2=0x7f2f5415b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f5415b6e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c009d90 tx=0x7f2f4c02a5f0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 -- 172.21.15.28:0/1818576772 shutdown_connections 2024-09-10T10:52:02.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f5415b6e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 --2- 172.21.15.28:0/1818576772 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f54072d30 0x7f2f5406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 -- 172.21.15.28:0/1818576772 >> 172.21.15.28:0/1818576772 conn(0x7f2f54067d70 msgr2=0x7f2f5406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 -- 172.21.15.28:0/1818576772 shutdown_connections 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.870+0000 7f2f5a400640 1 -- 172.21.15.28:0/1818576772 wait complete. 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.871+0000 7f2f5a400640 1 Processor -- start 2024-09-10T10:52:02.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.872+0000 7f2f5a400640 1 -- start start 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.872+0000 7f2f5a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f541561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.872+0000 7f2f5a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f5a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 0x7f2f54153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f5a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f540cd5c0 con 0x7f2f54072d30 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f5a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f54072530 con 0x7f2f5406f700 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f5a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f540cffe0 con 0x7f2f5415b300 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52610/0 (socket says 172.21.15.28:52610) 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 -- 172.21.15.28:0/927294050 learned_addr learned my addr 172.21.15.28:0/927294050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:02.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f53400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f541561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f53e00640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 0x7f2f54153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 -- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 msgr2=0x7f2f54153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 0x7f2f54153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 -- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 msgr2=0x7f2f541561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f541561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 -- 172.21.15.28:0/927294050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f54154480 con 0x7f2f54072d30 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f53e00640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 0x7f2f54153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.873+0000 7f2f52a00640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e9e0 tx=0x7f2f4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.874+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40017070 con 0x7f2f54072d30 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.874+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4000c9d0 con 0x7f2f54072d30 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.874+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5415cc80 con 0x7f2f54072d30 2024-09-10T10:52:02.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.874+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000cdc0 con 0x7f2f54072d30 2024-09-10T10:52:02.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.874+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f541e9620 con 0x7f2f54072d30 2024-09-10T10:52:02.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.875+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f54154e40 con 0x7f2f54072d30 2024-09-10T10:52:02.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.876+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f4001a070 con 0x7f2f54072d30 2024-09-10T10:52:02.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.876+0000 7f2f4b400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:02.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.877+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f2f4008b880 con 0x7f2f54072d30 2024-09-10T10:52:02.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.877+0000 7f2f53400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:02.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.878+0000 7f2f53400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 0x7f2f3406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2f440011e0 tx=0x7f2f440016d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:02.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:02.881+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f400594e0 con 0x7f2f54072d30 2024-09-10T10:52:03.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.135+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f2f54159040 con 0x7f2f54072d30 2024-09-10T10:52:03.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.695+0000 7f2f4b400640 1 -- 172.21.15.28:0/927294050 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v156) ==== 101+0+0 (secure 0 0 0) 0x7f2f4005d7e0 con 0x7f2f54072d30 2024-09-10T10:52:03.697 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 msgr2=0x7f2f3406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 0x7f2f3406e2f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2f440011e0 tx=0x7f2f440016d0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 msgr2=0x7f2f54152dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000e9e0 tx=0x7f2f4000eea0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 shutdown_connections 2024-09-10T10:52:03.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f3406be30 0x7f2f3406e2f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f5415b300 0x7f2f54153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f54072d30 0x7f2f54152dc0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 --2- 172.21.15.28:0/927294050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f5406f700 0x7f2f541561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 >> 172.21.15.28:0/927294050 conn(0x7f2f54067d70 msgr2=0x7f2f540af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 shutdown_connections 2024-09-10T10:52:03.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.698+0000 7f2f5a400640 1 -- 172.21.15.28:0/927294050 wait complete. 2024-09-10T10:52:03.717 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.793+0000 7fc5e291ff40 1 Processor -- start 2024-09-10T10:52:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.793+0000 7fc5e291ff40 1 -- start start 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c80ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8a3e0 0x55b461c015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 0x55b461c84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b461bff530 con 0x55b461c8d9c0 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b461b5cb30 con 0x55b461bfe8e0 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b461c8cee0 con 0x55b461c8a3e0 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5df000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c80ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 0x55b461c84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5df000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c80ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:48048/0 (socket says 172.21.15.28:48048) 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5df000640 1 -- 172.21.15.28:0/1095480443 learned_addr learned my addr 172.21.15.28:0/1095480443 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:03.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5de600640 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8a3e0 0x55b461c015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8a3e0 msgr2=0x55b461c015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8a3e0 0x55b461c015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 msgr2=0x55b461c80ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c80ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.794+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1095480443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b461c039f0 con 0x55b461c8d9c0 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 0x55b461c84ff0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001580 tx=0x7fc5d8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660a3ffcbdd7bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5dd200640 1 -- 172.21.15.28:0/1095480443 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800f710 con 0x55b461c8d9c0 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5dd200640 1 -- 172.21.15.28:0/1095480443 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d800f8d0 con 0x55b461c8d9c0 2024-09-10T10:52:03.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5dd200640 1 -- 172.21.15.28:0/1095480443 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800fcc0 con 0x55b461c8d9c0 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1095480443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 msgr2=0x55b461c84ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.795+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 0x55b461c84ff0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001580 tx=0x7fc5d8001a40 comp rx=0 tx=0).stop 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1095480443 shutdown_connections 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8d9c0 0x55b461c84ff0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8a3e0 0x55b461c015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1095480443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c80ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1095480443 >> 172.21.15.28:0/1095480443 conn(0x55b461c030a0 msgr2=0x55b461bb4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1095480443 shutdown_connections 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1095480443 wait complete. 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 Processor -- start 2024-09-10T10:52:03.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.796+0000 7fc5e291ff40 1 -- start start 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c90520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 0x55b461c00bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8d9c0 0x55b461d1b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b461c8c9a0 con 0x55b461c8a3e0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b461c8ad60 con 0x55b461bfe8e0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5e291ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b461c81370 con 0x55b461c8d9c0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.797+0000 7fc5de600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 0x55b461c00bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 0x55b461c00bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52636/0 (socket says 172.21.15.28:52636) 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 learned_addr learned my addr 172.21.15.28:0/1993931996 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c90520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8d9c0 0x55b461d1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8d9c0 msgr2=0x55b461d1b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b461c8d9c0 0x55b461d1b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 msgr2=0x55b461c90520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b461bfe8e0 0x55b461c90520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b461c8e030 con 0x55b461c8a3e0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.798+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 0x55b461c00bb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009f10 tx=0x7fc5cc008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.799+0000 7fc5d7400640 1 -- 172.21.15.28:0/1993931996 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc014600 con 0x55b461c8a3e0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.799+0000 7fc5d7400640 1 -- 172.21.15.28:0/1993931996 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc019070 con 0x55b461c8a3e0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.799+0000 7fc5d7400640 1 -- 172.21.15.28:0/1993931996 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0149a0 con 0x55b461c8a3e0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.799+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b461c8fa40 con 0x55b461c8a3e0 2024-09-10T10:52:03.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.799+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b461c90000 con 0x55b461c8a3e0 2024-09-10T10:52:03.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.800+0000 7fc5d7400640 1 -- 172.21.15.28:0/1993931996 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5cc013070 con 0x55b461c8a3e0 2024-09-10T10:52:03.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.801+0000 7fc5d7400640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c006a810 0x7fc5c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.801+0000 7fc5d7400640 1 -- 172.21.15.28:0/1993931996 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7fc5cc089fb0 con 0x55b461c8a3e0 2024-09-10T10:52:03.804 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:03.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.802+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c006a810 0x7fc5c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.802+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b461c010f0 0x55b461d26840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.802+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461c7e660 con 0x55b461c010f0 2024-09-10T10:52:03.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.802+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b461c010f0 0x55b461d26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.803+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c006a810 0x7fc5c006ccd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000a5a0 tx=0x7fc5d0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.803+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b461c010f0 0x55b461d26840 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.805+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v112'350 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.805+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d26d80 con 0x55b461c010f0 2024-09-10T10:52:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.807+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v112'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.807+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d27150 con 0x55b461c010f0 2024-09-10T10:52:03.810 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:03.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.809+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v112'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.809+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d30b20 con 0x55b461c010f0 2024-09-10T10:52:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.810+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v112'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.810+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d30ec0 con 0x55b461c010f0 2024-09-10T10:52:03.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.812+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v112'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.812+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d2fb10 con 0x55b461c010f0 2024-09-10T10:52:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.815+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v112'355 uv355 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.815+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d2ff00 con 0x55b461c010f0 2024-09-10T10:52:03.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.816+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v112'356 uv330 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.816+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302f0 con 0x55b461c010f0 2024-09-10T10:52:03.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.818+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v112'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.818+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302f0 con 0x55b461c010f0 2024-09-10T10:52:03.821 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:03.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.819+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v112'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.819+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.821+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v112'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.821+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.823+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v112'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.823+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.825+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v112'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.825+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.826+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v112'362 uv337 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.826+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.829+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v112'363 uv363 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.829+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.832 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:03.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.830+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v112'364 uv364 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.830+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.831+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v112'365 uv365 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.831+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.833+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v112'366 uv366 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.833+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.835+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v112'367 uv367 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.835+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.836+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v112'368 uv368 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.836+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.839+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v112'369 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.839+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.840+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v112'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.840+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.843 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.841+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v112'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.842+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.843+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v112'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.843+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.845+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v112'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.845+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.846+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v112'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.846+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.848+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v112'375 uv349 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.848+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.850+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v112'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.850+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.853 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:03.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.852+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v112'377 uv377 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.852+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.853+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v112'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.853+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.855+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v112'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.855+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461c010f0 2024-09-10T10:52:03.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.857+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v112'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461c010f0 2024-09-10T10:52:03.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.857+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b461d30660 0x55b461d33780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.857+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d302a0 con 0x55b461d30660 2024-09-10T10:52:03.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.857+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b461d30660 0x55b461d33780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.857+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b461d30660 0x55b461d33780 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.859+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v112'262 uv245 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.859+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d33cc0 con 0x55b461d30660 2024-09-10T10:52:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.861+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v112'263 uv263 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.861+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d34060 con 0x55b461d30660 2024-09-10T10:52:03.864 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:03.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.862+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v112'264 uv264 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.862+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d34400 con 0x55b461d30660 2024-09-10T10:52:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.864+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v112'265 uv265 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.864+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d34810 con 0x55b461d30660 2024-09-10T10:52:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.866+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v112'266 uv266 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.866+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d34c00 con 0x55b461d30660 2024-09-10T10:52:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.867+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v112'267 uv250 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.867+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d34fa0 con 0x55b461d30660 2024-09-10T10:52:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.869+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v112'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.869+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.872 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:03.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.870+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v112'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.871+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.872+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v112'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.872+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.874+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v112'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.874+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.876+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v112'272 uv255 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.876+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.877+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v112'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.877+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.880 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:03.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.879+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v112'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.879+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.880+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v112'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.880+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.882+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v112'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.882+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.883+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v112'277 uv261 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.883+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.885+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v112'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.885+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.888 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.886+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v112'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.886+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.888+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v112'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.888+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.889+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v112'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.889+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d35340 con 0x55b461d30660 2024-09-10T10:52:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.891+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v112'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d30660 2024-09-10T10:52:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.891+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b461d35c90 0x55b461d38080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.891+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d385c0 con 0x55b461d35c90 2024-09-10T10:52:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.891+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b461d35c90 0x55b461d38080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.892+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b461d35c90 0x55b461d38080 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.894+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v112'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.894+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d38a20 con 0x55b461d35c90 2024-09-10T10:52:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.896+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v112'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.896+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d38ed0 con 0x55b461d35c90 2024-09-10T10:52:03.899 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:03.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.897+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v112'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.897+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3a780 con 0x55b461d35c90 2024-09-10T10:52:03.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.899+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v112'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.899+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3ab20 con 0x55b461d35c90 2024-09-10T10:52:03.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.900+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v112'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.900+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d39770 con 0x55b461d35c90 2024-09-10T10:52:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.902+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v112'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.902+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b461d3a070 0x55b461d3ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.902+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3d410 con 0x55b461d3a070 2024-09-10T10:52:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.902+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b461d3a070 0x55b461d3ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.903+0000 7fc5dfa00640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b461d3a070 0x55b461d3ced0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.904+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v112'53 uv52 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.904+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3d410 con 0x55b461d3a070 2024-09-10T10:52:03.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.905+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v112'54 uv54 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.905+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3d490 con 0x55b461d3a070 2024-09-10T10:52:03.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.906+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v112'55 uv55 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:03.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.906+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3d8b0 con 0x55b461d3a070 2024-09-10T10:52:03.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.907+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v112'56 uv56 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.907+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3d8b0 con 0x55b461d35c90 2024-09-10T10:52:03.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.908+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v112'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.908+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.909+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v112'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.910+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.912 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:03.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.911+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v112'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.911+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3ec60 con 0x55b461d35c90 2024-09-10T10:52:03.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.912+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v112'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.912+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3ec60 con 0x55b461d35c90 2024-09-10T10:52:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.913+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v112'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.913+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.914+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v112'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.914+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.916+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v112'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.916+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.917+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v112'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.917+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.920 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.919+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v112'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.919+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3ec60 con 0x55b461d3a070 2024-09-10T10:52:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.920+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v112'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.920+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3ec60 con 0x55b461d3a070 2024-09-10T10:52:03.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.922+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v112'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.922+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.923+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v112'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.924+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.925+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v112'133 uv126 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.925+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.926+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v112'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.926+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.927+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v112'135 uv135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.927+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.929+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v112'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.929+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d3a070 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.930+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v112'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.930+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.931+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v112'56 uv55 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.931+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.933+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v112'57 uv57 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.933+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.935 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.934+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v112'58 uv58 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.934+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.935+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v112'59 uv59 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.935+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d3dc50 con 0x55b461d35c90 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.936+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v112'60 uv60 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d35c90 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.936+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b461d3e2d0 0x55b461d406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.936+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d40c90 con 0x55b461d3e2d0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.937+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b461d3e2d0 0x55b461d406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.937+0000 7fc5df000640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b461d3e2d0 0x55b461d406c0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.939+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v112'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.939+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d40d50 con 0x55b461d3e2d0 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.940+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v112'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.940+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.942+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v112'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.942+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d42150 con 0x55b461d3e2d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.943+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v112'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.943+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d42150 con 0x55b461d3e2d0 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.944+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v112'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.944+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.946+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v112'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.946+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.947+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v112'112 uv108 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.947+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.949+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v112'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.949+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.952 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:03.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.950+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v112'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.950+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.952+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v112'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.952+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.953+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v112'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.953+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.954+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v112'117 uv111 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.954+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.956+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v112'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.956+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.959 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:03.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.957+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v112'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.957+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d41140 con 0x55b461d3e2d0 2024-09-10T10:52:03.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.958+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v112'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.959+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d43f50 con 0x55b461d3e2d0 2024-09-10T10:52:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.960+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v112'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.960+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.965 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.963+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v112'54 uv54 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.963+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.966+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [omap-set-header] v112'55 uv55 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.966+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.967+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 609.00000000 [create] v112'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.967+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.968+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 609.00000000 [delete] v112'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.968+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.971 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.969+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v112'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.969+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.970+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [setxattr (30)] v112'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.970+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.971+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [setxattr (62)] v112'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.971+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d415f0 con 0x55b461d3a070 2024-09-10T10:52:03.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.973+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [omap-set-header] v112'74 uv74 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d801b040 con 0x55b461d3a070 2024-09-10T10:52:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.973+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b461d41ef0 0x55b461d44510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.973+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:109 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d41ef0 2024-09-10T10:52:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.973+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b461d41ef0 0x55b461d44510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.974+0000 7fc5de600640 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b461d41ef0 0x55b461d44510 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:03.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.975+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(109 400.00000000 [create] v112'57 uv56 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d41ef0 2024-09-10T10:52:03.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.975+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:110 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d41ef0 2024-09-10T10:52:03.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.976+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(110 400.00000000 [delete] v112'58 uv58 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d41ef0 2024-09-10T10:52:03.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.976+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d41ef0 2024-09-10T10:52:03.979 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:03.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.977+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(111 400.00000000 [create] v112'59 uv59 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d41ef0 2024-09-10T10:52:03.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.977+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d41ef0 2024-09-10T10:52:03.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.978+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(112 400.00000000 [write 0~22] v112'60 uv60 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d41ef0 2024-09-10T10:52:03.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.978+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d41ef0 2024-09-10T10:52:03.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.980+0000 7fc5de600640 1 -- 172.21.15.28:0/1993931996 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(113 400.00000000 [omap-set-header] v112'61 uv61 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5cc054260 con 0x55b461d41ef0 2024-09-10T10:52:03.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.980+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:114 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.981+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(114 100.00000000.inode [create] v112'144 uv136 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.981+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:115 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.982+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(115 100.00000000.inode [delete] v112'145 uv145 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.982+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:03.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.983+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(116 100.00000000.inode [create] v112'146 uv146 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.983+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.984+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v112'147 uv147 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.984+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.986+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v112'148 uv148 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.986+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.987+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(119 1.00000000 [create] v112'149 uv143 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.987+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.988+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(120 1.00000000 [delete] v112'150 uv150 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.988+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.991 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:03.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.989+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(121 1.00000000 [create] v112'151 uv151 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.989+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d45a60 con 0x55b461d3e2d0 2024-09-10T10:52:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.990+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v112'152 uv152 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.990+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d45a60 con 0x55b461d3e2d0 2024-09-10T10:52:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.991+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v112'153 uv153 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.991+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d44a50 con 0x55b461d3e2d0 2024-09-10T10:52:03.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.992+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(124 1.00000000 [omap-set-header] v112'154 uv154 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.992+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55b461d451b0 con 0x55b461d3e2d0 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5df000640 1 -- 172.21.15.28:0/1993931996 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v112'155 uv155 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d0065070 con 0x55b461d3e2d0 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b461d35c90 msgr2=0x55b461d38080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55b461d35c90 0x55b461d38080 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b461d3e2d0 msgr2=0x55b461d406c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55b461d3e2d0 0x55b461d406c0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b461d41ef0 msgr2=0x55b461d44510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55b461d41ef0 0x55b461d44510 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b461d3a070 msgr2=0x55b461d3ced0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55b461d3a070 0x55b461d3ced0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b461d30660 msgr2=0x55b461d33780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55b461d30660 0x55b461d33780 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b461c010f0 msgr2=0x55b461d26840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55b461c010f0 0x55b461d26840 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 reap_dead start 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 reap_dead start 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c006a810 msgr2=0x7fc5c006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.993+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c006a810 0x7fc5c006ccd0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000a5a0 tx=0x7fc5d0052000 comp rx=0 tx=0).stop 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 msgr2=0x55b461c00bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 --2- 172.21.15.28:0/1993931996 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b461c8a3e0 0x55b461c00bb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009f10 tx=0x7fc5cc008400 comp rx=0 tx=0).stop 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 reap_dead start 2024-09-10T10:52:03.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5dfa00640 1 -- 172.21.15.28:0/1993931996 reap_dead start 2024-09-10T10:52:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 shutdown_connections 2024-09-10T10:52:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 >> 172.21.15.28:0/1993931996 conn(0x55b461c030a0 msgr2=0x55b461be7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 shutdown_connections 2024-09-10T10:52:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:03.994+0000 7fc5e291ff40 1 -- 172.21.15.28:0/1993931996 wait complete. 2024-09-10T10:52:04.000 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 400.00000000 2024-09-10T10:52:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.114+0000 7f1c17729f40 1 Processor -- start 2024-09-10T10:52:04.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.114+0000 7f1c17729f40 1 -- start start 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d511ff0 0x55850d4f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 0x55850d5a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55850d512530 con 0x55850d59d3e0 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55850d46fb30 con 0x55850d5a09c0 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c17729f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55850d59fee0 con 0x55850d511ff0 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44552/0 (socket says 172.21.15.28:44552) 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 -- 172.21.15.28:0/2403489618 learned_addr learned my addr 172.21.15.28:0/2403489618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c14a00640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d511ff0 0x55850d4f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c15400640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 0x55850d5a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 -- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d511ff0 msgr2=0x55850d4f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d511ff0 0x55850d4f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 -- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 msgr2=0x55850d5a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 0x55850d5a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c0fe00640 1 -- 172.21.15.28:0/2403489618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850d5110a0 con 0x55850d59d3e0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.115+0000 7f1c15400640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 0x55850d5a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b210 tx=0x7f1c0000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce56f63a25263ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c0ea00640 1 -- 172.21.15.28:0/2403489618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00008070 con 0x55850d59d3e0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c0ea00640 1 -- 172.21.15.28:0/2403489618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0000d500 con 0x55850d59d3e0 2024-09-10T10:52:04.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c0ea00640 1 -- 172.21.15.28:0/2403489618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000d8b0 con 0x55850d59d3e0 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 -- 172.21.15.28:0/2403489618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 msgr2=0x55850d4fae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000b210 tx=0x7f1c0000b6d0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 -- 172.21.15.28:0/2403489618 shutdown_connections 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d5a09c0 0x55850d5a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d59d3e0 0x55850d4fae40 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 --2- 172.21.15.28:0/2403489618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d511ff0 0x55850d4f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.116+0000 7f1c17729f40 1 -- 172.21.15.28:0/2403489618 >> 172.21.15.28:0/2403489618 conn(0x55850d5160a0 msgr2=0x55850d4c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.117+0000 7f1c17729f40 1 -- 172.21.15.28:0/2403489618 shutdown_connections 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.117+0000 7f1c17729f40 1 -- 172.21.15.28:0/2403489618 wait complete. 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.117+0000 7f1c17729f40 1 Processor -- start 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.117+0000 7f1c17729f40 1 -- start start 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 0x55850d5a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d59d3e0 0x55850d5a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d5a09c0 0x55850d5a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55850d516640 con 0x55850d511ff0 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55850d5169b0 con 0x55850d59d3e0 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.118+0000 7f1c17729f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55850d514cb0 con 0x55850d5a09c0 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 0x55850d5a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 0x55850d5a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44554/0 (socket says 172.21.15.28:44554) 2024-09-10T10:52:04.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 -- 172.21.15.28:0/1481971355 learned_addr learned my addr 172.21.15.28:0/1481971355 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d59d3e0 0x55850d5a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c15400640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d5a09c0 0x55850d5a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 -- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d5a09c0 msgr2=0x55850d5a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55850d5a09c0 0x55850d5a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 -- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d59d3e0 msgr2=0x55850d5a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d59d3e0 0x55850d5a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 -- 172.21.15.28:0/1481971355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850d590f60 con 0x55850d511ff0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55850d59d3e0 0x55850d5a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c14a00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 0x55850d5a0fb0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1c08008870 tx=0x7f1c08008960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.119+0000 7f1c0d600640 1 -- 172.21.15.28:0/1481971355 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c080147e0 con 0x55850d511ff0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.120+0000 7f1c0d600640 1 -- 172.21.15.28:0/1481971355 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0801a070 con 0x55850d511ff0 2024-09-10T10:52:04.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.120+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55850d5a3810 con 0x55850d511ff0 2024-09-10T10:52:04.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.120+0000 7f1c0d600640 1 -- 172.21.15.28:0/1481971355 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08014bc0 con 0x55850d511ff0 2024-09-10T10:52:04.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.120+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850d5a14f0 con 0x55850d511ff0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.121+0000 7f1c0d600640 1 -- 172.21.15.28:0/1481971355 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c08015470 con 0x55850d511ff0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.122+0000 7f1c0d600640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1bf406a840 0x7f1bf406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.122+0000 7f1c0d600640 1 -- 172.21.15.28:0/1481971355 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f1c0808aba0 con 0x55850d511ff0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.123+0000 7f1c17729f40 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55850d6306a0 0x55850d62f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.123+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e112) -- 0x55850d62f620 con 0x55850d6306a0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.123+0000 7f1c15400640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55850d6306a0 0x55850d62f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.123+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1bf406a840 0x7f1bf406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.123+0000 7f1c15400640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55850d6306a0 0x55850d62f0e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.124+0000 7f1c0fe00640 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1bf406a840 0x7f1bf406cd00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55850d59c730 tx=0x7f1c00008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c15400640 1 -- 172.21.15.28:0/1481971355 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [delete] v112'62 uv62 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1c1004f9b0 con 0x55850d6306a0 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55850d6306a0 msgr2=0x55850d62f0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55850d6306a0 0x55850d62f0e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1bf406a840 msgr2=0x7f1bf406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1bf406a840 0x7f1bf406cd00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55850d59c730 tx=0x7f1c00008000 comp rx=0 tx=0).stop 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 msgr2=0x55850d5a0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 --2- 172.21.15.28:0/1481971355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55850d511ff0 0x55850d5a0fb0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f1c08008870 tx=0x7f1c08008960 comp rx=0 tx=0).stop 2024-09-10T10:52:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c15400640 1 -- 172.21.15.28:0/1481971355 reap_dead start 2024-09-10T10:52:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 shutdown_connections 2024-09-10T10:52:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 >> 172.21.15.28:0/1481971355 conn(0x55850d5160a0 msgr2=0x55850d4c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 shutdown_connections 2024-09-10T10:52:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.125+0000 7f1c17729f40 1 -- 172.21.15.28:0/1481971355 wait complete. 2024-09-10T10:52:04.131 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:04.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.290+0000 7fb566400640 1 Processor -- start 2024-09-10T10:52:04.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.290+0000 7fb566400640 1 -- start start 2024-09-10T10:52:04.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.291+0000 7fb566400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.291+0000 7fb566400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 0x7fb5600b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.291+0000 7fb566400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5600b1170 con 0x7fb5600b3640 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.291+0000 7fb566400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb5600ceba0 con 0x7fb5600d2ac0 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.291+0000 7fb566400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb56006dfb0 con 0x7fb560161ac0 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 0x7fb5600b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:47070/0 (socket says 172.21.15.28:47070) 2024-09-10T10:52:04.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 -- 172.21.15.28:0/2185420605 learned_addr learned my addr 172.21.15.28:0/2185420605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 -- 172.21.15.28:0/2185420605 >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 msgr2=0x7fb5600b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 --2- 172.21.15.28:0/2185420605 >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 0x7fb5600b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 -- 172.21.15.28:0/2185420605 >> v1:172.21.15.138:6789/0 conn(0x7fb5600b3640 legacy=0x7fb5600d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565000640 1 -- 172.21.15.28:0/2185420605 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600b1450 con 0x7fb560161ac0 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.292+0000 7fb565a00640 1 --2- 172.21.15.28:0/2185420605 >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 0x7fb5600b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:04.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.293+0000 7fb565000640 1 --2- 172.21.15.28:0/2185420605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb554009870 tx=0x7fb55402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2582a99ab48fbfef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.293+0000 7fb55f400640 1 -- 172.21.15.28:0/2185420605 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554037070 con 0x7fb560161ac0 2024-09-10T10:52:04.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.293+0000 7fb55f400640 1 -- 172.21.15.28:0/2185420605 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55402ae50 con 0x7fb560161ac0 2024-09-10T10:52:04.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.293+0000 7fb55f400640 1 -- 172.21.15.28:0/2185420605 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554032860 con 0x7fb560161ac0 2024-09-10T10:52:04.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 -- 172.21.15.28:0/2185420605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 msgr2=0x7fb56006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 --2- 172.21.15.28:0/2185420605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb554009870 tx=0x7fb55402a300 comp rx=0 tx=0).stop 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 -- 172.21.15.28:0/2185420605 shutdown_connections 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 --2- 172.21.15.28:0/2185420605 >> v2:172.21.15.138:3300/0 conn(0x7fb5600d2ac0 0x7fb5600b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 --2- 172.21.15.28:0/2185420605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb560161ac0 0x7fb56006e6c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 -- 172.21.15.28:0/2185420605 >> 172.21.15.28:0/2185420605 conn(0x7fb56008bfc0 msgr2=0x7fb5600d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 -- 172.21.15.28:0/2185420605 shutdown_connections 2024-09-10T10:52:04.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.294+0000 7fb566400640 1 -- 172.21.15.28:0/2185420605 wait complete. 2024-09-10T10:52:04.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.296+0000 7fb566400640 1 Processor -- start 2024-09-10T10:52:04.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.296+0000 7fb566400640 1 -- start start 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5600d2ac0 0x7fb5601f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560161ac0 0x7fb5601f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb560161ea0 con 0x7fb5600b3640 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5600b1a10 con 0x7fb560161ac0 2024-09-10T10:52:04.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb566400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb560162020 con 0x7fb5600d2ac0 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44562/0 (socket says 172.21.15.28:44562) 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 -- 172.21.15.28:0/2268041903 learned_addr learned my addr 172.21.15.28:0/2268041903 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb55fe00640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5600d2ac0 0x7fb5601f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565a00640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560161ac0 0x7fb5601f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 -- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5600d2ac0 msgr2=0x7fb5601f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5600d2ac0 0x7fb5601f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 -- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560161ac0 msgr2=0x7fb5601f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560161ac0 0x7fb5601f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.297+0000 7fb565000640 1 -- 172.21.15.28:0/2268041903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5600b6360 con 0x7fb5600b3640 2024-09-10T10:52:04.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb565000640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb55402a340 tx=0x7fb5540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554041070 con 0x7fb5600b3640 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55403c680 con 0x7fb5600b3640 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554037040 con 0x7fb5600b3640 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5600b6640 con 0x7fb5600b3640 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.298+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5600c7ff0 con 0x7fb5600b3640 2024-09-10T10:52:04.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.299+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5600b6b60 con 0x7fb5600b3640 2024-09-10T10:52:04.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.300+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb554044070 con 0x7fb5600b3640 2024-09-10T10:52:04.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.301+0000 7fb55cc00640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 0x7fb54006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:04.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.301+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7fb5540abc10 con 0x7fb5600b3640 2024-09-10T10:52:04.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.302+0000 7fb55fe00640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 0x7fb54006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:04.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.302+0000 7fb55fe00640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 0x7fb54006e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb5600b9860 tx=0x7fb54c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:04.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.306+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb554079900 con 0x7fb5600b3640 2024-09-10T10:52:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.528+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb56006e6c0 con 0x7fb5600b3640 2024-09-10T10:52:04.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.699+0000 7fb55cc00640 1 -- 172.21.15.28:0/2268041903 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v157) ==== 162+0+0 (secure 0 0 0) 0x7fb55407dc00 con 0x7fb5600b3640 2024-09-10T10:52:04.702 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:04.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 msgr2=0x7fb54006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 0x7fb54006e2f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb5600b9860 tx=0x7fb54c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:04.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 msgr2=0x7fb5600b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb55402a340 tx=0x7fb5540046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 shutdown_connections 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb54006be30 0x7fb54006e2f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560161ac0 0x7fb5601f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb5600d2ac0 0x7fb5601f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 --2- 172.21.15.28:0/2268041903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5600b3640 0x7fb5600b8dd0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 >> 172.21.15.28:0/2268041903 conn(0x7fb56008bfc0 msgr2=0x7fb5600b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 shutdown_connections 2024-09-10T10:52:04.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:04.702+0000 7fb566400640 1 -- 172.21.15.28:0/2268041903 wait complete. 2024-09-10T10:52:05.125 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:05.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.283+0000 7f9bc8600640 1 Processor -- start 2024-09-10T10:52:05.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.283+0000 7f9bc8600640 1 -- start start 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.284+0000 7f9bc8600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bc0147230 0x7f9bc006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc8600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9bc00727c0 0x7f9bc00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc8600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bc006f480 con 0x7f9bc0147e90 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc8600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc00b34d0 con 0x7f9bc00727c0 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc8600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9bc00ae9e0 con 0x7f9bc0147230 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc5200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9bc0147e90 0x7f9bc00720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60516/0 (socket says 172.21.15.28:60516) 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc5200640 1 -- 172.21.15.28:0/2667165288 learned_addr learned my addr 172.21.15.28:0/2667165288 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2667165288 >> v2:172.21.15.28:3300/0 conn(0x7f9bc00727c0 0x7f9bc00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc5c00640 1 --2- 172.21.15.28:0/2667165288 >> v2:172.21.15.138:3300/0 conn(0x7f9bc0147230 0x7f9bc006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bbfe00640 1 -- 172.21.15.28:0/2667165288 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518230883 0 0) 0x7f9bc006f480 con 0x7f9bc0147e90 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bbfe00640 1 -- 172.21.15.28:0/2667165288 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ba8003520 con 0x7f9bc0147e90 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc6600640 1 -- 172.21.15.28:0/2667165288 >> v2:172.21.15.138:3300/0 conn(0x7f9bc0147230 msgr2=0x7f9bc006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2667165288 >> v2:172.21.15.138:3300/0 conn(0x7f9bc0147230 0x7f9bc006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.285+0000 7f9bc6600640 1 -- 172.21.15.28:0/2667165288 >> v1:172.21.15.28:6789/0 conn(0x7f9bc0147e90 legacy=0x7f9bc00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.286+0000 7f9bc6600640 1 -- 172.21.15.28:0/2667165288 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc006e900 con 0x7f9bc00727c0 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2667165288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc00727c0 0x7f9bc00aee10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0070280 tx=0x7f9bb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc19061ced600da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bbfe00640 1 -- 172.21.15.28:0/2667165288 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007740 con 0x7f9bc00727c0 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bbfe00640 1 -- 172.21.15.28:0/2667165288 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb80078e0 con 0x7f9bc00727c0 2024-09-10T10:52:05.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bbfe00640 1 -- 172.21.15.28:0/2667165288 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8007cb0 con 0x7f9bc00727c0 2024-09-10T10:52:05.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 -- 172.21.15.28:0/2667165288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc00727c0 msgr2=0x7f9bc00aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2667165288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc00727c0 0x7f9bc00aee10 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0070280 tx=0x7f9bb802a680 comp rx=0 tx=0).stop 2024-09-10T10:52:05.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 -- 172.21.15.28:0/2667165288 shutdown_connections 2024-09-10T10:52:05.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2667165288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc00727c0 0x7f9bc00aee10 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2667165288 >> v2:172.21.15.138:3300/0 conn(0x7f9bc0147230 0x7f9bc006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 -- 172.21.15.28:0/2667165288 >> 172.21.15.28:0/2667165288 conn(0x7f9bc0067d70 msgr2=0x7f9bc006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:05.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.287+0000 7f9bc8600640 1 -- 172.21.15.28:0/2667165288 shutdown_connections 2024-09-10T10:52:05.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.288+0000 7f9bc8600640 1 -- 172.21.15.28:0/2667165288 wait complete. 2024-09-10T10:52:05.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.289+0000 7f9bc8600640 1 Processor -- start 2024-09-10T10:52:05.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.289+0000 7f9bc8600640 1 -- start start 2024-09-10T10:52:05.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 0x7f9bc00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc0147230 0x7f9bc00c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc006f1a0 con 0x7f9bc0147e90 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc006eec0 con 0x7f9bc0147230 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc006e5f0 con 0x7f9bc00727c0 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44582/0 (socket says 172.21.15.28:44582) 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.290+0000 7f9bc6600640 1 -- 172.21.15.28:0/2706838989 learned_addr learned my addr 172.21.15.28:0/2706838989 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc5200640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc0147230 0x7f9bc00c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc5c00640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 0x7f9bc00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 -- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 msgr2=0x7f9bc00c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 0x7f9bc00c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 -- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc0147230 msgr2=0x7f9bc00c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc0147230 0x7f9bc00c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 -- 172.21.15.28:0/2706838989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc0131fd0 con 0x7f9bc0147e90 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc5c00640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 0x7f9bc00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bc6600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9bb802ac70 tx=0x7f9bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.291+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8040070 con 0x7f9bc0147e90 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.292+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb8031bc0 con 0x7f9bc0147e90 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.292+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8009450 con 0x7f9bc0147e90 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.292+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc01322b0 con 0x7f9bc0147e90 2024-09-10T10:52:05.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.292+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc0132870 con 0x7f9bc0147e90 2024-09-10T10:52:05.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.293+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc0132a70 con 0x7f9bc0147e90 2024-09-10T10:52:05.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.294+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bb8039040 con 0x7f9bc0147e90 2024-09-10T10:52:05.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.294+0000 7f9bbd600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 0x7f9ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:05.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.295+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f9bb80acd30 con 0x7f9bc0147e90 2024-09-10T10:52:05.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.295+0000 7f9bc5c00640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 0x7f9ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:05.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.296+0000 7f9bc5c00640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 0x7f9ba006e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0134ed0 tx=0x7f9bb4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:05.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.299+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9bc0132a70 con 0x7f9bc0147e90 2024-09-10T10:52:05.442 INFO:tasks.ceph.mds.c.smithi028.stderr: -45> 2024-09-10T10:52:00.686+0000 7f8f15800640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2024-09-10T10:52:05.442 INFO:tasks.ceph.mds.c.smithi028.stderr: -44> 2024-09-10T10:52:00.686+0000 7f8f15800640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2024-09-10T10:52:05.481 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:52:05.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.525+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc0147610 con 0x7f9bc0147e90 2024-09-10T10:52:05.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.526+0000 7f9bbd600640 1 -- 172.21.15.28:0/2706838989 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) ==== 95+0+4401 (secure 0 0 0) 0x7f9bb807fe20 con 0x7f9bc0147e90 2024-09-10T10:52:05.528 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:05.528 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":158,"btime":"2024-09-10T10:52:04:703062+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:04.703052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"a","rank":0,"incarnation":158,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:05.528 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 158 2024-09-10T10:52:05.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 msgr2=0x7f9ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 0x7f9ba006e2f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0134ed0 tx=0x7f9bb4008040 comp rx=0 tx=0).stop 2024-09-10T10:52:05.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 msgr2=0x7f9bc0132d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:05.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9bb802ac70 tx=0x7f9bb8004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 shutdown_connections 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ba006be30 0x7f9ba006e2f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bc0147e90 0x7f9bc0132d90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bc0147230 0x7f9bc00c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 --2- 172.21.15.28:0/2706838989 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bc00727c0 0x7f9bc00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 >> 172.21.15.28:0/2706838989 conn(0x7f9bc0067d70 msgr2=0x7f9bc006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 shutdown_connections 2024-09-10T10:52:05.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:05.528+0000 7f9bc8600640 1 -- 172.21.15.28:0/2706838989 wait complete. 2024-09-10T10:52:05.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 158, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:52:04.703052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8163': {'gid': 8163, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2774580459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2774580459}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2774580459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:52:05.546 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8163:up:replay 2024-09-10T10:52:05.546 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:52:10.213 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:52:10.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:52:10.950 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.115+0000 7fa2c6800640 1 Processor -- start 2024-09-10T10:52:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.115+0000 7fa2c6800640 1 -- start start 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 0x7fa2c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d1f90 0x7fa2c00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00b37d0 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00cf330 con 0x7fa2c015e660 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c6800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00cf000 con 0x7fa2c00d1f90 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c5400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c5400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44626/0 (socket says 172.21.15.28:44626) 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c5400640 1 -- 172.21.15.28:0/2343169712 learned_addr learned my addr 172.21.15.28:0/2343169712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.116+0000 7fa2c4a00640 1 --2- 172.21.15.28:0/2343169712 >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 0x7fa2c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5e00640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d1f90 0x7fa2c00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 -- 172.21.15.28:0/2343169712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d1f90 msgr2=0x7fa2c00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d1f90 0x7fa2c00cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 -- 172.21.15.28:0/2343169712 >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 msgr2=0x7fa2c006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 --2- 172.21.15.28:0/2343169712 >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 0x7fa2c006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 -- 172.21.15.28:0/2343169712 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00d16f0 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c4a00640 1 --2- 172.21.15.28:0/2343169712 >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 0x7fa2c006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2c5400640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00b3980 tx=0x7fa2b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debd886e5f9d7068 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2bfe00640 1 -- 172.21.15.28:0/2343169712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4037070 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2bfe00640 1 -- 172.21.15.28:0/2343169712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b4032440 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.117+0000 7fa2bfe00640 1 -- 172.21.15.28:0/2343169712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4032810 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 -- 172.21.15.28:0/2343169712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d0dd0 msgr2=0x7fa2c00d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00b3980 tx=0x7fa2b402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 -- 172.21.15.28:0/2343169712 shutdown_connections 2024-09-10T10:52:11.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d1f90 0x7fa2c00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 --2- 172.21.15.28:0/2343169712 >> v2:172.21.15.138:3300/0 conn(0x7fa2c015e660 0x7fa2c006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 --2- 172.21.15.28:0/2343169712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d0dd0 0x7fa2c00d11b0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 -- 172.21.15.28:0/2343169712 >> 172.21.15.28:0/2343169712 conn(0x7fa2c008bfc0 msgr2=0x7fa2c00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 -- 172.21.15.28:0/2343169712 shutdown_connections 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.118+0000 7fa2c6800640 1 -- 172.21.15.28:0/2343169712 wait complete. 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.120+0000 7fa2c6800640 1 Processor -- start 2024-09-10T10:52:11.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 -- start start 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d0dd0 0x7fa2c01e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c015e660 0x7fa2c00b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00d2f40 con 0x7fa2c00d1f90 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00d2c60 con 0x7fa2c015e660 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c6800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00d3220 con 0x7fa2c00d0dd0 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44636/0 (socket says 172.21.15.28:44636) 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 -- 172.21.15.28:0/1416848043 learned_addr learned my addr 172.21.15.28:0/1416848043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c5400640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d0dd0 0x7fa2c01e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c5e00640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c015e660 0x7fa2c00b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 -- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d0dd0 msgr2=0x7fa2c01e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d0dd0 0x7fa2c01e7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 -- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c015e660 msgr2=0x7fa2c00b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c015e660 0x7fa2c00b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.121+0000 7fa2c4a00640 1 -- 172.21.15.28:0/1416848043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c00b5750 con 0x7fa2c00d1f90 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2c4a00640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d830 tx=0x7fa2ac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac004030 con 0x7fa2c00d1f90 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ac0041d0 con 0x7fa2c00d1f90 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0096a0 con 0x7fa2c00d1f90 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c00b3ea0 con 0x7fa2c00d1f90 2024-09-10T10:52:11.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.123+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c00b4730 con 0x7fa2c00d1f90 2024-09-10T10:52:11.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.124+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00b7870 con 0x7fa2c00d1f90 2024-09-10T10:52:11.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.125+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa2ac009a50 con 0x7fa2c00d1f90 2024-09-10T10:52:11.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.126+0000 7fa2bd600640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.126+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7fa2ac08b880 con 0x7fa2c00d1f90 2024-09-10T10:52:11.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.126+0000 7fa2c5400640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.126+0000 7fa2c5400640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4034000 tx=0x7fa2b402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.129+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2c00b7870 con 0x7fa2c00d1f90 2024-09-10T10:52:11.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.354+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2c00d2440 con 0x7fa2c00d1f90 2024-09-10T10:52:11.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.354+0000 7fa2bd600640 1 -- 172.21.15.28:0/1416848043 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) ==== 95+0+5234 (secure 0 0 0) 0x7fa2ac059570 con 0x7fa2c00d1f90 2024-09-10T10:52:11.356 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:11.356 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":161,"btime":"2024-09-10T10:52:07:717325+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8304,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1314136636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1314136636},{"type":"v1","addr":"172.21.15.28:6834","nonce":1314136636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:07.717315+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8163":{"gid":8163,"name":"a","rank":0,"incarnation":158,"state":"up:active","state_seq":7,"addr":"172.21.15.28:6835/2774580459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2774580459},{"type":"v1","addr":"172.21.15.28:6835","nonce":2774580459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8163,"qdb_cluster":[8163]},"id":6}]} 2024-09-10T10:52:11.356 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 161 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 msgr2=0x7fa2a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 0x7fa2a006e2f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4034000 tx=0x7fa2b402e000 comp rx=0 tx=0).stop 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 msgr2=0x7fa2c00b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d830 tx=0x7fa2ac00dcf0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 shutdown_connections 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa2a006be30 0x7fa2a006e2f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2c015e660 0x7fa2c00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2c00d1f90 0x7fa2c00b5210 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 --2- 172.21.15.28:0/1416848043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2c00d0dd0 0x7fa2c01e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 >> 172.21.15.28:0/1416848043 conn(0x7fa2c008bfc0 msgr2=0x7fa2c00b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 shutdown_connections 2024-09-10T10:52:11.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.357+0000 7fa2c6800640 1 -- 172.21.15.28:0/1416848043 wait complete. 2024-09-10T10:52:11.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 161, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:52:07.717315+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8163': {'gid': 8163, 'name': 'a', 'rank': 0, 'incarnation': 158, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.28:6835/2774580459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2774580459}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2774580459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8163, 'qdb_cluster': [8163]} 2024-09-10T10:52:11.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:52:11.377 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:52:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.534+0000 7f517b200640 1 Processor -- start 2024-09-10T10:52:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.534+0000 7f517b200640 1 -- start start 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f517c15e6a0 0x7f517c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f517c06dd60 0x7f517c0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517c0d2c20 con 0x7f517c0d4180 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f517c0cefb0 con 0x7f517c06dd60 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f517c06e390 con 0x7f517c15e6a0 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f517c06dd60 0x7f517c0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f517c06dd60 0x7f517c0cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44642/0 (socket says 172.21.15.28:44642) 2024-09-10T10:52:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f517a800640 1 -- 172.21.15.28:0/1222661607 learned_addr learned my addr 172.21.15.28:0/1222661607 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f5179e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f517c0d4180 0x7f517c0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60542/0 (socket says 172.21.15.28:60542) 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.535+0000 7f5179400640 1 --2- 172.21.15.28:0/1222661607 >> v2:172.21.15.138:3300/0 conn(0x7f517c15e6a0 0x7f517c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.536+0000 7f517a800640 1 -- 172.21.15.28:0/1222661607 >> v2:172.21.15.138:3300/0 conn(0x7f517c15e6a0 msgr2=0x7f517c06d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.536+0000 7f517a800640 1 --2- 172.21.15.28:0/1222661607 >> v2:172.21.15.138:3300/0 conn(0x7f517c15e6a0 0x7f517c06d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.536+0000 7f517a800640 1 -- 172.21.15.28:0/1222661607 >> v1:172.21.15.28:6789/0 conn(0x7f517c0d4180 legacy=0x7f517c0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.536+0000 7f5178a00640 1 -- 172.21.15.28:0/1222661607 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2184776059 0 0) 0x7f517c0d2c20 con 0x7f517c0d4180 2024-09-10T10:52:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.536+0000 7f517a800640 1 -- 172.21.15.28:0/1222661607 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c06e670 con 0x7f517c06dd60 2024-09-10T10:52:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f517a800640 1 --2- 172.21.15.28:0/1222661607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cf440 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b5e20 tx=0x7f517402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d06fe4a582ea82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f5178a00640 1 -- 172.21.15.28:0/1222661607 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174036070 con 0x7f517c06dd60 2024-09-10T10:52:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f5178a00640 1 -- 172.21.15.28:0/1222661607 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5174031600 con 0x7f517c06dd60 2024-09-10T10:52:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f5178a00640 1 -- 172.21.15.28:0/1222661607 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740319d0 con 0x7f517c06dd60 2024-09-10T10:52:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f517b200640 1 -- 172.21.15.28:0/1222661607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 msgr2=0x7f517c0cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.537+0000 7f517b200640 1 --2- 172.21.15.28:0/1222661607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cf440 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f517c0b5e20 tx=0x7f517402a6a0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 -- 172.21.15.28:0/1222661607 shutdown_connections 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 --2- 172.21.15.28:0/1222661607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cf440 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 --2- 172.21.15.28:0/1222661607 >> v2:172.21.15.138:3300/0 conn(0x7f517c15e6a0 0x7f517c06d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 -- 172.21.15.28:0/1222661607 >> 172.21.15.28:0/1222661607 conn(0x7f517c09d6f0 msgr2=0x7f517c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 -- 172.21.15.28:0/1222661607 shutdown_connections 2024-09-10T10:52:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.538+0000 7f517b200640 1 -- 172.21.15.28:0/1222661607 wait complete. 2024-09-10T10:52:11.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.539+0000 7f517b200640 1 Processor -- start 2024-09-10T10:52:11.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.539+0000 7f517b200640 1 -- start start 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 0x7f517c158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f517c15e6a0 0x7f517c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0cf980 con 0x7f517c06dd60 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f517c0cebd0 con 0x7f517c0d4180 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.540+0000 7f517b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f517c0cfb00 con 0x7f517c15e6a0 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44658/0 (socket says 172.21.15.28:44658) 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 -- 172.21.15.28:0/1963353221 learned_addr learned my addr 172.21.15.28:0/1963353221 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f517a800640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f517c15e6a0 0x7f517c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179400640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 0x7f517c158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 -- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f517c15e6a0 msgr2=0x7f517c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f517c15e6a0 0x7f517c0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 -- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 msgr2=0x7f517c158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 0x7f517c158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 -- 172.21.15.28:0/1963353221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517c1f87b0 con 0x7f517c06dd60 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179400640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 0x7f517c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.541+0000 7f5179e00640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c4d0 tx=0x7f516c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.542+0000 7f5172000640 1 -- 172.21.15.28:0/1963353221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c018070 con 0x7f517c06dd60 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.542+0000 7f5172000640 1 -- 172.21.15.28:0/1963353221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c007500 con 0x7f517c06dd60 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.542+0000 7f5172000640 1 -- 172.21.15.28:0/1963353221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c004a20 con 0x7f517c06dd60 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.542+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517c0ccfc0 con 0x7f517c06dd60 2024-09-10T10:52:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.542+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f517c0b67f0 con 0x7f517c06dd60 2024-09-10T10:52:11.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.544+0000 7f5172000640 1 -- 172.21.15.28:0/1963353221 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f516c004bc0 con 0x7f517c06dd60 2024-09-10T10:52:11.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.545+0000 7f5172000640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 0x7f515c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.545+0000 7f5172000640 1 -- 172.21.15.28:0/1963353221 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f516c08a800 con 0x7f517c06dd60 2024-09-10T10:52:11.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.545+0000 7f5179400640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 0x7f515c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.545+0000 7f51839bb740 1 Processor -- start 2024-09-10T10:52:11.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.545+0000 7f51839bb740 1 -- start start 2024-09-10T10:52:11.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f5179400640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 0x7f515c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5168006fd0 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c96640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589d0baf3b0 con 0x5589d0c4e660 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589d0bb4410 con 0x5589d0c50ea0 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589d0b5c620 con 0x5589d0c52630 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f5179400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f5179400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44660/0 (socket says 172.21.15.28:44660) 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.546+0000 7f5179400640 1 -- 172.21.15.28:0/1920605266 learned_addr learned my addr 172.21.15.28:0/1920605266 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179e00640 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c96640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f517a800640 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 -- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 msgr2=0x5589d0c4b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 -- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 msgr2=0x5589d0c96640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c96640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 -- 172.21.15.28:0/1920605266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d08c72d0 con 0x5589d0c4e660 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5179400640 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f517c15da00 tx=0x7f5168011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467358ed63da4a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.547+0000 7f5166600640 1 -- 172.21.15.28:0/1920605266 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168025070 con 0x5589d0c4e660 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f5166600640 1 -- 172.21.15.28:0/1920605266 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168015640 con 0x5589d0c4e660 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f5166600640 1 -- 172.21.15.28:0/1920605266 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516800f040 con 0x5589d0c4e660 2024-09-10T10:52:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 -- 172.21.15.28:0/1920605266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 msgr2=0x5589d0c51280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f517c15da00 tx=0x7f5168011000 comp rx=0 tx=0).stop 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 -- 172.21.15.28:0/1920605266 shutdown_connections 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c4e660 0x5589d0c51280 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 --2- 172.21.15.28:0/1920605266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c96640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 -- 172.21.15.28:0/1920605266 >> 172.21.15.28:0/1920605266 conn(0x5589d0c4d8f0 msgr2=0x5589d0c95b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 -- 172.21.15.28:0/1920605266 shutdown_connections 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.548+0000 7f51839bb740 1 -- 172.21.15.28:0/1920605266 wait complete. 2024-09-10T10:52:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.549+0000 7f51839bb740 1 -- start start 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.549+0000 7f51839bb740 1 Processor -- start 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c90940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c74710 0x5589d0c7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589d0b6f780 con 0x5589d0c52630 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589d0bb03d0 con 0x5589d0c50ea0 2024-09-10T10:52:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589d0c4cd90 con 0x5589d0c74710 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f51839bb740 10 client.0 _ll_register_callbacks cb 0x5589d0bad5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.550+0000 7f5179400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44670/0 (socket says 172.21.15.28:44670) 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 -- 172.21.15.28:0/2677768200 learned_addr learned my addr 172.21.15.28:0/2677768200 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f517a800640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c74710 0x5589d0c7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179e00640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c90940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 -- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c74710 msgr2=0x5589d0c7a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c74710 0x5589d0c7a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 -- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 msgr2=0x5589d0c90940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c90940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 -- 172.21.15.28:0/2677768200 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d09916b0 con 0x5589d0c52630 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179e00640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c90940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f5179400640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5168023040 tx=0x7f5168004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce001fa59c61df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f515aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.551+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516802e060 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f51680157f0 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5168015640 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) ==== 24+0+0 (secure 0 0 0) 0x7f5168034050 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 10 client.8325 fetch_fsmap learned FSMap version 161 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=2+}) -- 0x5589d0c90f10 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5168015640 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 161) ==== 3502+0+0 (secure 0 0 0) 0x7f5168037020 con 0x5589d0c52630 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 10 client.8325 fetch_fsmap finished waiting for FSMap version 161 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 10 client.8325 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:52:11.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 20 client.8325 populate_metadata read hostname 'smithi028' 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0c4a830 0x5589d0c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 4 client.8325 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.552+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 --> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589d0bc1b40 con 0x5589d0c4a830 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.553+0000 7f5179e00640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0c4a830 0x5589d0c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.553+0000 7f5179e00640 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0c4a830 0x5589d0c4ac10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415c23840ad019cc server_cookie=bbea2540a2c1d5bc in_seq=0 out_seq=0 2024-09-10T10:52:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.553+0000 7f515aa00640 10 client.8325 ms_handle_connect on v2:172.21.15.28:6833/2774580459 2024-09-10T10:52:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f515aa00640 1 -- 172.21.15.28:0/2677768200 <== mds.0 v2:172.21.15.28:6833/2774580459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589d0bc1b40 con 0x5589d0c4a830 2024-09-10T10:52:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f515aa00640 10 client.8325 handle_command_reply: tid=0 2024-09-10T10:52:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 client.8325 shutdown 2024-09-10T10:52:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 msgr2=0x5589d0c741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5168023040 tx=0x7f5168004730 comp rx=0 tx=0).stop 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 shutdown_connections 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c74710 0x5589d0c7a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0c4a830 0x5589d0c4ac10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c52630 0x5589d0c741d0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 --2- 172.21.15.28:0/2677768200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c50ea0 0x5589d0c90940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f515aa00640 0 client.8325 ms_handle_reset on v2:172.21.15.28:6833/2774580459 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 >> 172.21.15.28:0/2677768200 conn(0x5589d0c4e660 msgr2=0x5589d0c95100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 shutdown_connections 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.554+0000 7f51839bb740 1 -- 172.21.15.28:0/2677768200 wait complete. 2024-09-10T10:52:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.555+0000 7f51839bb740 20 client.8325 trim_cache size 0 max 16384 2024-09-10T10:52:11.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.573+0000 7f51839bb740 1 Processor -- start 2024-09-10T10:52:11.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.573+0000 7f51839bb740 1 -- start start 2024-09-10T10:52:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c92150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f517c200ac0 con 0x5589d0c50ea0 2024-09-10T10:52:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f517c1ffeb0 con 0x5589d0c74710 2024-09-10T10:52:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f517c200440 con 0x5589d0c52630 2024-09-10T10:52:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f5179e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f5179e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44686/0 (socket says 172.21.15.28:44686) 2024-09-10T10:52:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.574+0000 7f5179e00640 1 -- 172.21.15.28:0/505468092 learned_addr learned my addr 172.21.15.28:0/505468092 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f517a800640 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c92150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179400640 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 -- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 msgr2=0x5589d0c4b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 -- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 msgr2=0x5589d0c92150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c92150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 -- 172.21.15.28:0/505468092 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589d0c96340 con 0x5589d0c50ea0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5179e00640 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f516c058340 tx=0x7f516c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51af847b6d88d896 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5165c00640 1 -- 172.21.15.28:0/505468092 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c088740 con 0x5589d0c50ea0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5165c00640 1 -- 172.21.15.28:0/505468092 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c0566b0 con 0x5589d0c50ea0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.575+0000 7f5165c00640 1 -- 172.21.15.28:0/505468092 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c070150 con 0x5589d0c50ea0 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 -- 172.21.15.28:0/505468092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 msgr2=0x5589d0c4ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f516c058340 tx=0x7f516c05c040 comp rx=0 tx=0).stop 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 -- 172.21.15.28:0/505468092 shutdown_connections 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c92150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c4b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 --2- 172.21.15.28:0/505468092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c50ea0 0x5589d0c4ac10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 -- 172.21.15.28:0/505468092 >> 172.21.15.28:0/505468092 conn(0x5589d0c4e660 msgr2=0x5589d0c95a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 -- 172.21.15.28:0/505468092 shutdown_connections 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.576+0000 7f51839bb740 1 -- 172.21.15.28:0/505468092 wait complete. 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.577+0000 7f51839bb740 1 -- start start 2024-09-10T10:52:11.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.577+0000 7f51839bb740 1 Processor -- start 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c78880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c78dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589d0c7aa40 con 0x5589d0c90700 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589d0c79b90 con 0x5589d0c74710 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589d0c71e90 con 0x5589d0c52630 2024-09-10T10:52:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f51839bb740 10 client.0 _ll_register_callbacks cb 0x7f5168023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44690/0 (socket says 172.21.15.28:44690) 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 -- 172.21.15.28:0/386407603 learned_addr learned my addr 172.21.15.28:0/386407603 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f5179400640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c78dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f5179e00640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c78880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 -- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 msgr2=0x5589d0c78880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c78880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 -- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 msgr2=0x5589d0c78dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c78dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.578+0000 7f517a800640 1 -- 172.21.15.28:0/386407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589d0c77c40 con 0x5589d0c90700 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f5179e00640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f5179400640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c78dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f517a800640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5174009f40 tx=0x7f5174004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13b4e8ab458ea3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f515aa00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5174036040 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f5174004240 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f516c0566b0 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=161) ==== 24+0+0 (secure 0 0 0) 0x7f5174004440 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f51839bb740 10 client.8331 fetch_fsmap learned FSMap version 161 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.579+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=161,monmap=2+}) -- 0x5589d0c70e10 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c0566b0 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 161) ==== 3502+0+0 (secure 0 0 0) 0x7f5174053080 con 0x5589d0c90700 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 10 client.8331 fetch_fsmap finished waiting for FSMap version 161 2024-09-10T10:52:11.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 10 client.8331 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 20 client.8331 populate_metadata read hostname 'smithi028' 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0d138e0 0x5589d0d13cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 4 client.8331 mds_command: new command op to 8163 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 --> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] -- command(tid 0: {"prefix": "status"}) -- 0x5589d0c7b450 con 0x5589d0d138e0 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f5179e00640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0d138e0 0x5589d0d13cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f515aa00640 10 client.8331 ms_handle_connect on v2:172.21.15.28:6833/2774580459 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.580+0000 7f5179e00640 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0d138e0 0x5589d0d13cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e3afafa8b7c7539 server_cookie=2cd435f30cf3a286 in_seq=0 out_seq=0 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f515aa00640 1 -- 172.21.15.28:0/386407603 <== mds.0 v2:172.21.15.28:6833/2774580459 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5589d0c7b450 con 0x5589d0d138e0 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f515aa00640 10 client.8331 handle_command_reply: tid=0 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 client.8331 shutdown 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 msgr2=0x5589d0cb28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5174009f40 tx=0x7f5174004ab0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 shutdown_connections 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:6833/2774580459,v1:172.21.15.28:6835/2774580459] conn(0x5589d0d138e0 0x5589d0d13cc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589d0c90700 0x5589d0cb28a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589d0c74710 0x5589d0c78dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 --2- 172.21.15.28:0/386407603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589d0c52630 0x5589d0c78880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f515aa00640 0 client.8331 ms_handle_reset on v2:172.21.15.28:6833/2774580459 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 >> 172.21.15.28:0/386407603 conn(0x5589d0c4e660 msgr2=0x5589d0c95060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 shutdown_connections 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 1 -- 172.21.15.28:0/386407603 wait complete. 2024-09-10T10:52:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.581+0000 7f51839bb740 20 client.8331 trim_cache size 0 max 16384 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 8163, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8767558910000002, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 161, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 112, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 112, 2024-09-10T10:52:11.591 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 20.104422558 2024-09-10T10:52:11.592 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:52:11.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 msgr2=0x7f515c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 0x7f515c06e320 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5168006fd0 tx=0x7f5168008040 comp rx=0 tx=0).stop 2024-09-10T10:52:11.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 msgr2=0x7f517c0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c4d0 tx=0x7f516c00c990 comp rx=0 tx=0).stop 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 shutdown_connections 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f515c06be60 0x7f515c06e320 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f517c15e6a0 0x7f517c0b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f517c0d4180 0x7f517c158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 --2- 172.21.15.28:0/1963353221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f517c06dd60 0x7f517c0cdaa0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 >> 172.21.15.28:0/1963353221 conn(0x7f517c09d6f0 msgr2=0x7f517c0b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 shutdown_connections 2024-09-10T10:52:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.582+0000 7f517b200640 1 -- 172.21.15.28:0/1963353221 wait complete. 2024-09-10T10:52:11.598 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:52:11.598 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Delete 400.00000000', proceeding to ls 2024-09-10T10:52:11.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:52:11.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.753+0000 7ff2fde00640 1 Processor -- start 2024-09-10T10:52:11.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.754+0000 7ff2fde00640 1 -- start start 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2fde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2f8162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2fde00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f815eb60 con 0x7ff2f80b69d0 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2fde00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2f815ee40 con 0x7ff2f80af280 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2fde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f80b6db0 con 0x7ff2f80b0180 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2f6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff2f80b69d0 0x7ff2f80afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:60546/0 (socket says 172.21.15.28:60546) 2024-09-10T10:52:11.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2f6a00640 1 -- 172.21.15.28:0/18863110 learned_addr learned my addr 172.21.15.28:0/18863110 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2ee000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2f8162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.755+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228224043 0 0) 0x7ff2f815eb60 con 0x7ff2f80b69d0 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2dc003520 con 0x7ff2f80b69d0 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 349739260 0 0) 0x7ff2dc003520 con 0x7ff2f80b69d0 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 msgr2=0x7ff2f8162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2f8162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 >> v1:172.21.15.138:6789/0 conn(0x7ff2f80af280 legacy=0x7ff2f8154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f815f120 con 0x7ff2f80b69d0 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7ff2e8001670 con 0x7ff2f80b69d0 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 >> v1:172.21.15.28:6789/0 conn(0x7ff2f80b69d0 legacy=0x7ff2f80afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2dc0039e0 0x7ff2dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc004800 0x7ff2dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2dc005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f80b6db0 con 0x7ff2dc0039e0 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc00bb80 con 0x7ff2dc004800 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc00bd80 con 0x7ff2f80b0180 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.756+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2dc0039e0 0x7ff2dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2ee000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc004800 0x7ff2dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f7400640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2dc005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f6a00640 1 -- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 msgr2=0x7ff2dc005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2f80b0180 0x7ff2dc005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f6a00640 1 -- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc004800 msgr2=0x7ff2dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc004800 0x7ff2dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2f6a00640 1 -- 172.21.15.28:0/18863110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f815f120 con 0x7ff2dc0039e0 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.757+0000 7ff2ee000640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc004800 0x7ff2dc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.758+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2dc0039e0 0x7ff2dc003e20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8001050 tx=0x7ff2dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900ca8f49e20537a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.758+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8036070 con 0x7ff2dc0039e0 2024-09-10T10:52:11.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.758+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2e8005af0 con 0x7ff2dc0039e0 2024-09-10T10:52:11.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.758+0000 7ff2f6000640 1 -- 172.21.15.28:0/18863110 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8007dd0 con 0x7ff2dc0039e0 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 -- 172.21.15.28:0/18863110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2dc0039e0 msgr2=0x7ff2dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 --2- 172.21.15.28:0/18863110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2dc0039e0 0x7ff2dc003e20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8001050 tx=0x7ff2dc005830 comp rx=0 tx=0).stop 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2f7400640 1 -- 172.21.15.28:0/18863110 reap_dead start 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 -- 172.21.15.28:0/18863110 shutdown_connections 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 -- 172.21.15.28:0/18863110 >> 172.21.15.28:0/18863110 conn(0x7ff2f806bc10 msgr2=0x7ff2f806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 -- 172.21.15.28:0/18863110 shutdown_connections 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.759+0000 7ff2fde00640 1 -- 172.21.15.28:0/18863110 wait complete. 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.760+0000 7ff2fde00640 1 Processor -- start 2024-09-10T10:52:11.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.760+0000 7ff2fde00640 1 -- start start 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc0039e0 0x7ff2f814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81487f0 0x7ff2f8147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f806f870 con 0x7ff2f80cdef0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f80af6f0 con 0x7ff2f81487f0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.761+0000 7ff2fde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f806f9f0 con 0x7ff2dc0039e0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44716/0 (socket says 172.21.15.28:44716) 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 -- 172.21.15.28:0/3158808061 learned_addr learned my addr 172.21.15.28:0/3158808061 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2ee000640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81487f0 0x7ff2f8147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc0039e0 0x7ff2f814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 -- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc0039e0 msgr2=0x7ff2f814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc0039e0 0x7ff2f814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 -- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81487f0 msgr2=0x7ff2f8147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81487f0 0x7ff2f8147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 -- 172.21.15.28:0/3158808061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f815dc60 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.762+0000 7ff2f7400640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8067c20 tx=0x7ff2f000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.763+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003970 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.763+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f0018070 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.763+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003d50 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.763+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f814b7d0 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.763+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f8149270 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.764+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f8068f80 con 0x7ff2f80cdef0 2024-09-10T10:52:11.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.765+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2f00050d0 con 0x7ff2f80cdef0 2024-09-10T10:52:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.766+0000 7ff2efe00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 0x7ff2d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.766+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7ff2f008b080 con 0x7ff2f80cdef0 2024-09-10T10:52:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.766+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 0x7ff2d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:11.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.767+0000 7ff2f6a00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 0x7ff2d406e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8033000 tx=0x7ff2e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:11.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.770+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff2f8068f80 con 0x7ff2f80cdef0 2024-09-10T10:52:11.773 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:52:11.772+0000 7f8ca8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:11.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.994+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff2f80cd4c0 con 0x7ff2f80cdef0 2024-09-10T10:52:11.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.994+0000 7ff2efe00640 1 -- 172.21.15.28:0/3158808061 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v161) ==== 53+0+73 (secure 0 0 0) 0x7ff2f814b7d0 con 0x7ff2f80cdef0 2024-09-10T10:52:11.995 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 msgr2=0x7ff2d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 0x7ff2d406e320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8033000 tx=0x7ff2e802e000 comp rx=0 tx=0).stop 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 msgr2=0x7ff2f80ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8067c20 tx=0x7ff2f000eb90 comp rx=0 tx=0).stop 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 shutdown_connections 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2d406be60 0x7ff2d406e320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2f80cdef0 0x7ff2f80ce360 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2f81487f0 0x7ff2f8147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 --2- 172.21.15.28:0/3158808061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc0039e0 0x7ff2f814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 >> 172.21.15.28:0/3158808061 conn(0x7ff2f806bc10 msgr2=0x7ff2f80aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 shutdown_connections 2024-09-10T10:52:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:11.997+0000 7ff2fde00640 1 -- 172.21.15.28:0/3158808061 wait complete. 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:52:12.015 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:12.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.016 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:52:12.051 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:52:12.052 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:12.053 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:52:12.054 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:12.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.055 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:52:12.145 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:52:12.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.149 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:52:12.186 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:52:12.186 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:52:12.186 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:52:12.186 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:52:12.186 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:52:12.186 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:52:12.187 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:52:12.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:52:12.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:52:12.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:52:12.187 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:52:12.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:52:12.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:52:12.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.384 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:52:12.419 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:52:12.420 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:52:12.420 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:12.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.420 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:52:12.516 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:52:12.516 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:12.551 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:12.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:52:12.551 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:12.625 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:52:12.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:12.757 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:12.757 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:12.757 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:12.772 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:12.772 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:12.772 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:12.772 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:52:12.773 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:52:12.794 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:12.794 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:12.794 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:12.794 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:52:12.795 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:52:12.795 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:52:12.795 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:52:12.795 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:52:12.796 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:52:12.796 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:52:12.796 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:52:12.796 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:52:12.796 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:52:12.799 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:52:12.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.799 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:12.854 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:52:12.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:52:12.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:52:12.945 INFO:teuthology.orchestra.run.smithi028.stdout:client6995 2024-09-10T10:52:12.945 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:12.945 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:12.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:12.980 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:52:13.025 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:52:13.058 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:52:13.059 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:52:13.059 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:13.059 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:52:13.059 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:52:13.060 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 400.00000000' did not caused DAMAGED state 2024-09-10T10:52:13.060 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 100.00000000.inode' 2024-09-10T10:52:13.060 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:13.060 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:13.136 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:13.136 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:13.211 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:52:13.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.407+0000 7f43cc200640 1 Processor -- start 2024-09-10T10:52:13.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.407+0000 7f43cc200640 1 -- start start 2024-09-10T10:52:13.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.408+0000 7f43cc200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.408+0000 7f43cc200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f43c415e1f0 0x7f43c406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.408+0000 7f43cc200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c4146140 con 0x7f43c4072de0 2024-09-10T10:52:13.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.408+0000 7f43cc200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f43c40cd740 con 0x7f43c415e1f0 2024-09-10T10:52:13.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43cc200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43c40cda20 con 0x7f43c4072320 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43c9800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43c9800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:47200/0 (socket says 172.21.15.28:47200) 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43c9800640 1 -- 172.21.15.28:0/3905371425 learned_addr learned my addr 172.21.15.28:0/3905371425 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43ca200640 1 --2- 172.21.15.28:0/3905371425 >> v2:172.21.15.138:3300/0 conn(0x7f43c415e1f0 0x7f43c406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.409+0000 7f43c8e00640 1 --1- 172.21.15.28:0/3905371425 >> v1:172.21.15.138:6789/0 conn(0x7f43c4072de0 0x7f43c406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:41120/0 (socket says 172.21.15.28:41120) 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.410+0000 7f43c9800640 1 -- 172.21.15.28:0/3905371425 >> v2:172.21.15.138:3300/0 conn(0x7f43c415e1f0 msgr2=0x7f43c406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.410+0000 7f43c9800640 1 --2- 172.21.15.28:0/3905371425 >> v2:172.21.15.138:3300/0 conn(0x7f43c415e1f0 0x7f43c406e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.410+0000 7f43c9800640 1 -- 172.21.15.28:0/3905371425 >> v1:172.21.15.138:6789/0 conn(0x7f43c4072de0 legacy=0x7f43c406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.410+0000 7f43c3e00640 1 -- 172.21.15.28:0/3905371425 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2115253202 0 0) 0x7f43c4146140 con 0x7f43c4072de0 2024-09-10T10:52:13.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.410+0000 7f43c9800640 1 -- 172.21.15.28:0/3905371425 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c40cdd00 con 0x7f43c4072320 2024-09-10T10:52:13.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.411+0000 7f43c9800640 1 --2- 172.21.15.28:0/3905371425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009e50 tx=0x7f43b802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa80f202e52150f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.411+0000 7f43c3e00640 1 -- 172.21.15.28:0/3905371425 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8036070 con 0x7f43c4072320 2024-09-10T10:52:13.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.411+0000 7f43c3e00640 1 -- 172.21.15.28:0/3905371425 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8031600 con 0x7f43c4072320 2024-09-10T10:52:13.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.411+0000 7f43c3e00640 1 -- 172.21.15.28:0/3905371425 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b80319d0 con 0x7f43c4072320 2024-09-10T10:52:13.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 -- 172.21.15.28:0/3905371425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 msgr2=0x7f43c40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 --2- 172.21.15.28:0/3905371425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009e50 tx=0x7f43b802a680 comp rx=0 tx=0).stop 2024-09-10T10:52:13.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 -- 172.21.15.28:0/3905371425 shutdown_connections 2024-09-10T10:52:13.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 --2- 172.21.15.28:0/3905371425 >> v2:172.21.15.138:3300/0 conn(0x7f43c415e1f0 0x7f43c406e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 --2- 172.21.15.28:0/3905371425 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c4072320 0x7f43c40cce60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 -- 172.21.15.28:0/3905371425 >> 172.21.15.28:0/3905371425 conn(0x7f43c4067d70 msgr2=0x7f43c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 -- 172.21.15.28:0/3905371425 shutdown_connections 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.412+0000 7f43cc200640 1 -- 172.21.15.28:0/3905371425 wait complete. 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.413+0000 7f43cc200640 1 Processor -- start 2024-09-10T10:52:13.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.414+0000 7f43cc200640 1 -- start start 2024-09-10T10:52:13.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.414+0000 7f43cc200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4072320 0x7f43c4141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43cc200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43cc200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c415e1f0 0x7f43c41e8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43cc200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43c40cd520 con 0x7f43c4072de0 2024-09-10T10:52:13.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43cc200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43c40cd3a0 con 0x7f43c4072320 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43cc200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43c406ed00 con 0x7f43c415e1f0 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44728/0 (socket says 172.21.15.28:44728) 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 -- 172.21.15.28:0/1217070257 learned_addr learned my addr 172.21.15.28:0/1217070257 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c9800640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4072320 0x7f43c4141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43ca200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c415e1f0 0x7f43c41e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 -- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c415e1f0 msgr2=0x7f43c41e8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c415e1f0 0x7f43c41e8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 -- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4072320 msgr2=0x7f43c4141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4072320 0x7f43c4141eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.415+0000 7f43c8e00640 1 -- 172.21.15.28:0/1217070257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c40cc720 con 0x7f43c4072de0 2024-09-10T10:52:13.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43c8e00640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c4e0 tx=0x7f43b000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0018070 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b0007500 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c40cc010 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004a20 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.416+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c40cb770 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.417+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4072700 con 0x7f43c4072de0 2024-09-10T10:52:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.418+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f43b001b070 con 0x7f43c4072de0 2024-09-10T10:52:13.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.419+0000 7f43c1600640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 0x7f43a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.419+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f43b008a720 con 0x7f43c4072de0 2024-09-10T10:52:13.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.420+0000 7f43c9800640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 0x7f43a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.420+0000 7f43c9800640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 0x7f43a406e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f43b8033000 tx=0x7f43b8009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.423+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43b0058570 con 0x7f43c4072de0 2024-09-10T10:52:13.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.619+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43c4144820 con 0x7f43c4072de0 2024-09-10T10:52:13.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.619+0000 7f43c1600640 1 -- 172.21.15.28:0/1217070257 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 24 entries v112) ==== 81+0+1456 (secure 0 0 0) 0x7f43b005c870 con 0x7f43c4072de0 2024-09-10T10:52:13.620 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:52:13.621 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:52:13.622 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:52:13.623 INFO:teuthology.orchestra.run.smithi028.stderr:listed 24 entries 2024-09-10T10:52:13.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 msgr2=0x7f43a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 0x7f43a406e2f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f43b8033000 tx=0x7f43b8009000 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 msgr2=0x7f43c41e3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c4e0 tx=0x7f43b000c9a0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 shutdown_connections 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f43a406be30 0x7f43a406e2f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43c415e1f0 0x7f43c41e8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c4072de0 0x7f43c41e3dd0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 --2- 172.21.15.28:0/1217070257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43c4072320 0x7f43c4141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 >> 172.21.15.28:0/1217070257 conn(0x7f43c4067d70 msgr2=0x7f43c4142a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 shutdown_connections 2024-09-10T10:52:13.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.621+0000 7f43cc200640 1 -- 172.21.15.28:0/1217070257 wait complete. 2024-09-10T10:52:13.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:52:13.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:13.638 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:52:13.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:13.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:13.719 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:13.755 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.952+0000 7f1897600640 1 Processor -- start 2024-09-10T10:52:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.952+0000 7f1897600640 1 -- start start 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1897600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1890072d30 0x7f189006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1897600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189006fae0 con 0x7f189006f700 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1897600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18900cdbe0 con 0x7f189015b300 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1897600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18900728a0 con 0x7f1890072d30 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1895600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f189006f700 0x7f189015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58718/0 (socket says 172.21.15.28:58718) 2024-09-10T10:52:13.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f1895600640 1 -- 172.21.15.28:0/1238380882 learned_addr learned my addr 172.21.15.28:0/1238380882 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f188fe00640 1 --2- 172.21.15.28:0/1238380882 >> v2:172.21.15.138:3300/0 conn(0x7f1890072d30 0x7f189006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3011647700 0 0) 0x7f189006fae0 con 0x7f189006f700 2024-09-10T10:52:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.954+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1878003520 con 0x7f189006f700 2024-09-10T10:52:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3929103245 0 0) 0x7f18900cdbe0 con 0x7f189015b300 2024-09-10T10:52:13.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f189006fae0 con 0x7f189015b300 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504735306 0 0) 0x7f1878003520 con 0x7f189006f700 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 >> v2:172.21.15.138:3300/0 conn(0x7f1890072d30 msgr2=0x7f189006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 --2- 172.21.15.28:0/1238380882 >> v2:172.21.15.138:3300/0 conn(0x7f1890072d30 0x7f189006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 >> v1:172.21.15.138:6789/0 conn(0x7f189015b300 legacy=0x7f18900cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900cd900 con 0x7f189006f700 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f18880019f0 con 0x7f189006f700 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 >> v1:172.21.15.28:6789/0 conn(0x7f189006f700 legacy=0x7f189015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1878003b40 0x7f1878003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18900728a0 con 0x7f1878004800 2024-09-10T10:52:13.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f187800c0f0 con 0x7f18780051c0 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f187800c2f0 con 0x7f1878003b40 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f188fe00640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f1895600640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.955+0000 7f1894c00640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1878003b40 0x7f1878003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f188fe00640 1 -- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1878003b40 msgr2=0x7f1878003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f188fe00640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1878003b40 0x7f1878003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f188fe00640 1 -- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18780051c0 msgr2=0x7f187800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f188fe00640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18780051c0 0x7f187800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f188fe00640 1 -- 172.21.15.28:0/1238380882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1878003520 con 0x7f1878004800 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.956+0000 7f1895600640 1 -- 172.21.15.28:0/1238380882 reap_dead start 2024-09-10T10:52:13.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f188fe00640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1878004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f187c009ac0 tx=0x7f1878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfadade9484d6d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c036070 con 0x7f1878004800 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187c031400 con 0x7f1878004800 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f188f400640 1 -- 172.21.15.28:0/1238380882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c009690 con 0x7f1878004800 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 -- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 msgr2=0x7f1878004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1878004c80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f187c009ac0 tx=0x7f1878005830 comp rx=0 tx=0).stop 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 -- 172.21.15.28:0/1238380882 shutdown_connections 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 --2- 172.21.15.28:0/1238380882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1878004c80 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 -- 172.21.15.28:0/1238380882 >> 172.21.15.28:0/1238380882 conn(0x7f1890067d70 msgr2=0x7f189006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.957+0000 7f1897600640 1 -- 172.21.15.28:0/1238380882 shutdown_connections 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.958+0000 7f1897600640 1 -- 172.21.15.28:0/1238380882 wait complete. 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.959+0000 7f1897600640 1 Processor -- start 2024-09-10T10:52:13.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.959+0000 7f1897600640 1 -- start start 2024-09-10T10:52:13.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 0x7f1890152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 0x7f18900ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18900cffe0 con 0x7f1878004800 2024-09-10T10:52:13.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1890072530 con 0x7f1890154ee0 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1897600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18900d0160 con 0x7f1890153330 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1894c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1894c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43342/0 (socket says 172.21.15.28:43342) 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.960+0000 7f1894c00640 1 -- 172.21.15.28:0/1787927841 learned_addr learned my addr 172.21.15.28:0/1787927841 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1895600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 0x7f18900ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f188fe00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 0x7f1890152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 -- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 msgr2=0x7f18900ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 0x7f18900ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 -- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 msgr2=0x7f1890152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 0x7f1890152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 -- 172.21.15.28:0/1787927841 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18900caf00 con 0x7f1878004800 2024-09-10T10:52:13.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f188fe00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 0x7f1890152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1895600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 0x7f18900ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1894c00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f188400f720 tx=0x7f188400fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840050d0 con 0x7f1878004800 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1884018070 con 0x7f1878004800 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188400d6b0 con 0x7f1878004800 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.961+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1890150c30 con 0x7f1878004800 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.962+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18901511f0 con 0x7f1878004800 2024-09-10T10:52:13.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.962+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1890153740 con 0x7f1878004800 2024-09-10T10:52:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.963+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f188400d850 con 0x7f1878004800 2024-09-10T10:52:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.964+0000 7f188cc00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 0x7f187006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.964+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 7336+0+0 (secure 0 0 0) 0x7f188408b2e0 con 0x7f1878004800 2024-09-10T10:52:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.965+0000 7f188fe00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 0x7f187006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:13.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.966+0000 7f188fe00640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 0x7f187006e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f187c033000 tx=0x7f187c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:13.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:13.969+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1890153740 con 0x7f1878004800 2024-09-10T10:52:14.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.189+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f189006d090 con 0x7f1878004800 2024-09-10T10:52:14.191 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:14.190+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:14.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.192+0000 7f188cc00640 1 -- 172.21.15.28:0/1787927841 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v162) ==== 161+0+0 (secure 0 0 0) 0x7f1884059620 con 0x7f1878004800 2024-09-10T10:52:14.193 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 msgr2=0x7f187006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 0x7f187006e320 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f187c033000 tx=0x7f187c008030 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 msgr2=0x7f1890148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f188400f720 tx=0x7f188400fbe0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 shutdown_connections 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f187006be60 0x7f187006e320 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1890153330 0x7f18900ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1890154ee0 0x7f1890152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 --2- 172.21.15.28:0/1787927841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1878004800 0x7f1890148d80 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 >> 172.21.15.28:0/1787927841 conn(0x7f1890067d70 msgr2=0x7f18900b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:14.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.196+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 shutdown_connections 2024-09-10T10:52:14.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.197+0000 7f1897600640 1 -- 172.21.15.28:0/1787927841 wait complete. 2024-09-10T10:52:14.202 INFO:tasks.ceph.mds.a.smithi028.stderr: -528> 2024-09-10T10:52:11.772+0000 7f8ca8600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:14.218 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:14.247 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:52:14.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.375+0000 7f8357000640 1 Processor -- start 2024-09-10T10:52:14.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.375+0000 7f8357000640 1 -- start start 2024-09-10T10:52:14.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8357000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8350072320 0x7f83500cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8357000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f835015e1f0 0x7f835006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8357000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8350146140 con 0x7f8350072de0 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8357000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f83500cd740 con 0x7f835015e1f0 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8357000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83500cda20 con 0x7f8350072320 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f835015e1f0 0x7f835006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f835015e1f0 0x7f835006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43350/0 (socket says 172.21.15.28:43350) 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 -- 172.21.15.28:0/1317805142 learned_addr learned my addr 172.21.15.28:0/1317805142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f834fe00640 1 --2- 172.21.15.28:0/1317805142 >> v2:172.21.15.138:3300/0 conn(0x7f8350072320 0x7f83500cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 -- 172.21.15.28:0/1317805142 >> v2:172.21.15.138:3300/0 conn(0x7f8350072320 msgr2=0x7f83500cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 --2- 172.21.15.28:0/1317805142 >> v2:172.21.15.138:3300/0 conn(0x7f8350072320 0x7f83500cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 -- 172.21.15.28:0/1317805142 >> v1:172.21.15.138:6789/0 conn(0x7f8350072de0 legacy=0x7f835006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.377+0000 7f8355000640 1 -- 172.21.15.28:0/1317805142 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83500cdd00 con 0x7f835015e1f0 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.378+0000 7f8355000640 1 --2- 172.21.15.28:0/1317805142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f835006e7c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f83500b31e0 tx=0x7f834802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d101328b0739240e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f834f400640 1 -- 172.21.15.28:0/1317805142 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348036070 con 0x7f835015e1f0 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f834f400640 1 -- 172.21.15.28:0/1317805142 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348031600 con 0x7f835015e1f0 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f834f400640 1 -- 172.21.15.28:0/1317805142 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83480319d0 con 0x7f835015e1f0 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 -- 172.21.15.28:0/1317805142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 msgr2=0x7f835006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 --2- 172.21.15.28:0/1317805142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f835006e7c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f83500b31e0 tx=0x7f834802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 -- 172.21.15.28:0/1317805142 shutdown_connections 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 --2- 172.21.15.28:0/1317805142 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f835006e7c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 --2- 172.21.15.28:0/1317805142 >> v2:172.21.15.138:3300/0 conn(0x7f8350072320 0x7f83500cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 -- 172.21.15.28:0/1317805142 >> 172.21.15.28:0/1317805142 conn(0x7f8350067d70 msgr2=0x7f835006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.379+0000 7f8357000640 1 -- 172.21.15.28:0/1317805142 shutdown_connections 2024-09-10T10:52:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.380+0000 7f8357000640 1 -- 172.21.15.28:0/1317805142 wait complete. 2024-09-10T10:52:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.381+0000 7f8357000640 1 Processor -- start 2024-09-10T10:52:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.381+0000 7f8357000640 1 -- start start 2024-09-10T10:52:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8350072320 0x7f83501e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8350072de0 0x7f83501e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83500cd520 con 0x7f835015e1f0 2024-09-10T10:52:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83500cd3a0 con 0x7f8350072320 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.382+0000 7f8357000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f835006ed00 con 0x7f8350072de0 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43356/0 (socket says 172.21.15.28:43356) 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 -- 172.21.15.28:0/1239316437 learned_addr learned my addr 172.21.15.28:0/1239316437 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8347400640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8350072de0 0x7f83501e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f834fe00640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8350072320 0x7f83501e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 -- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8350072de0 msgr2=0x7f83501e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8350072de0 0x7f83501e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 -- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8350072320 msgr2=0x7f83501e8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8350072320 0x7f83501e8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 -- 172.21.15.28:0/1239316437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83501e5990 con 0x7f835015e1f0 2024-09-10T10:52:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.383+0000 7f8355000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8348009f40 tx=0x7f8348004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.384+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348036050 con 0x7f835015e1f0 2024-09-10T10:52:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.384+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8348042070 con 0x7f835015e1f0 2024-09-10T10:52:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.384+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83501e3ee0 con 0x7f835015e1f0 2024-09-10T10:52:14.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.384+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348009510 con 0x7f835015e1f0 2024-09-10T10:52:14.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.384+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83501e4720 con 0x7f835015e1f0 2024-09-10T10:52:14.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.385+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8350072700 con 0x7f835015e1f0 2024-09-10T10:52:14.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.386+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8348007bd0 con 0x7f835015e1f0 2024-09-10T10:52:14.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.387+0000 7f834cc00640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 0x7f832c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:14.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.387+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7422+0+0 (secure 0 0 0) 0x7f83480ac370 con 0x7f835015e1f0 2024-09-10T10:52:14.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.388+0000 7f834fe00640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 0x7f832c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:14.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.388+0000 7f834fe00640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 0x7f832c06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8340006fd0 tx=0x7f8340008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:14.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.392+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83501e3ee0 con 0x7f835015e1f0 2024-09-10T10:52:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:14.648+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8350144dc0 con 0x7f835015e1f0 2024-09-10T10:52:15.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.197+0000 7f834cc00640 1 -- 172.21.15.28:0/1239316437 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v163) ==== 107+0+0 (secure 0 0 0) 0x7f8348079ef0 con 0x7f835015e1f0 2024-09-10T10:52:15.199 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:52:15.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 msgr2=0x7f832c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 0x7f832c06e2f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8340006fd0 tx=0x7f8340008040 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 msgr2=0x7f83501e3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f8348009f40 tx=0x7f8348004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 shutdown_connections 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f832c06be30 0x7f832c06e2f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f835015e1f0 0x7f83501e3850 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8350072de0 0x7f83501e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 --2- 172.21.15.28:0/1239316437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8350072320 0x7f83501e8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 >> 172.21.15.28:0/1239316437 conn(0x7f8350067d70 msgr2=0x7f83500b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 shutdown_connections 2024-09-10T10:52:15.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.200+0000 7f8357000640 1 -- 172.21.15.28:0/1239316437 wait complete. 2024-09-10T10:52:15.219 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.338+0000 7fc5e4bdaf40 1 Processor -- start 2024-09-10T10:52:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.338+0000 7fc5e4bdaf40 1 -- start start 2024-09-10T10:52:15.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb15f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e63e0 0x55c6eb1e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb0b8b30 con 0x55c6eb1e99c0 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb0bcb70 con 0x55c6eb110470 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c6eb1e8ee0 con 0x55c6eb1e63e0 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb15f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43364/0 (socket says 172.21.15.28:43364) 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e1400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e63e0 0x55c6eb1e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.339+0000 7fc5e2800640 1 -- 172.21.15.28:0/178534820 learned_addr learned my addr 172.21.15.28:0/178534820 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 -- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e63e0 msgr2=0x55c6eb1e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e63e0 0x55c6eb1e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 -- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 msgr2=0x55c6eb15f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb15f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 -- 172.21.15.28:0/178534820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6eb112c20 con 0x55c6eb1e99c0 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5e2800640 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4d818d35b9232 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5dbe00640 1 -- 172.21.15.28:0/178534820 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00f710 con 0x55c6eb1e99c0 2024-09-10T10:52:15.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5dbe00640 1 -- 172.21.15.28:0/178534820 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5dc00f8b0 con 0x55c6eb1e99c0 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.340+0000 7fc5dbe00640 1 -- 172.21.15.28:0/178534820 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00fc80 con 0x55c6eb1e99c0 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/178534820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 msgr2=0x55c6eb141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc001a40 comp rx=0 tx=0).stop 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/178534820 shutdown_connections 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e99c0 0x55c6eb141ea0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e63e0 0x55c6eb1e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/178534820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb15f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/178534820 >> 172.21.15.28:0/178534820 conn(0x55c6eb15f0a0 msgr2=0x55c6eb1e83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/178534820 shutdown_connections 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.341+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/178534820 wait complete. 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.342+0000 7fc5e4bdaf40 1 Processor -- start 2024-09-10T10:52:15.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.342+0000 7fc5e4bdaf40 1 -- start start 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb1436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e99c0 0x55c6eb1df830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb1e5c20 con 0x55c6eb1e63e0 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6eb1ec2a0 con 0x55c6eb110470 2024-09-10T10:52:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e4bdaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c6eb1e5940 con 0x55c6eb1e99c0 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43378/0 (socket says 172.21.15.28:43378) 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 learned_addr learned my addr 172.21.15.28:0/1359526892 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e2800640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e99c0 0x55c6eb1df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.343+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb1436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e99c0 msgr2=0x55c6eb1df830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c6eb1e99c0 0x55c6eb1df830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 msgr2=0x55c6eb1436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6eb110470 0x55c6eb1436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6eb1446e0 con 0x55c6eb1e63e0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000a520 tx=0x7fc5d0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5daa00640 1 -- 172.21.15.28:0/1359526892 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0065030 con 0x55c6eb1e63e0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5daa00640 1 -- 172.21.15.28:0/1359526892 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d005a040 con 0x55c6eb1e63e0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c6eb282290 con 0x55c6eb1e63e0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.344+0000 7fc5daa00640 1 -- 172.21.15.28:0/1359526892 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0059070 con 0x55c6eb1e63e0 2024-09-10T10:52:15.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.345+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6eb282850 con 0x55c6eb1e63e0 2024-09-10T10:52:15.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.346+0000 7fc5daa00640 1 -- 172.21.15.28:0/1359526892 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5d0068030 con 0x55c6eb1e63e0 2024-09-10T10:52:15.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.347+0000 7fc5daa00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 0x7fc5c406ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.347+0000 7fc5daa00640 1 -- 172.21.15.28:0/1359526892 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7422+0+0 (secure 0 0 0) 0x7fc5d00cdaa0 con 0x55c6eb1e63e0 2024-09-10T10:52:15.350 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:15.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.348+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 0x55c6eb1e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.348+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 0x7fc5c406ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.348+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb282d70 con 0x55c6eb1e3090 2024-09-10T10:52:15.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.348+0000 7fc5e2800640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 0x55c6eb1e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.348+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 0x7fc5c406ccc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4001ed0 tx=0x7fc5d40016b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.349+0000 7fc5e2800640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 0x55c6eb1e34a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.353+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v113'381 uv355 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.353+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb1e39e0 con 0x55c6eb1e3090 2024-09-10T10:52:15.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.355+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v113'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.355+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb1e3db0 con 0x55c6eb1e3090 2024-09-10T10:52:15.357 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:15.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.357+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v113'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.357+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28c210 con 0x55c6eb1e3090 2024-09-10T10:52:15.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.358+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v113'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.358+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28c5b0 con 0x55c6eb1e3090 2024-09-10T10:52:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.360+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v113'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.360+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28b6b0 con 0x55c6eb1e3090 2024-09-10T10:52:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.362+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v113'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.362+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28baa0 con 0x55c6eb1e3090 2024-09-10T10:52:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.363+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v113'387 uv361 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.363+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28c950 con 0x55c6eb1e3090 2024-09-10T10:52:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.365+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v113'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.365+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.366+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v113'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.368 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:15.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.367+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.368+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v113'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.368+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.370+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v113'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.370+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.372+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v113'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.372+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.373+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v113'393 uv368 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.373+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.375+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v113'394 uv394 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.375+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.376+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v113'395 uv395 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.377 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:15.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.376+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.378+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v113'396 uv396 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.378+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.380+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v113'397 uv397 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.380+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28be40 con 0x55c6eb1e3090 2024-09-10T10:52:15.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.381+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v113'398 uv398 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.382+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.383+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v113'399 uv399 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.383+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.384+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v113'400 uv374 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.385+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.386+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v113'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.386+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.388 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:15.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.387+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v113'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.387+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28dd00 con 0x55c6eb1e3090 2024-09-10T10:52:15.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.389+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v113'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.389+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28dd00 con 0x55c6eb1e3090 2024-09-10T10:52:15.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.390+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v113'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.390+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.392+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v113'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.392+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.393+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v113'406 uv380 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.393+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.395+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v113'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.395+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.396+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v113'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.397 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:15.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.396+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28dd00 con 0x55c6eb1e3090 2024-09-10T10:52:15.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.398+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v113'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.398+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28dd00 con 0x55c6eb1e3090 2024-09-10T10:52:15.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.399+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v113'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.399+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28ccf0 con 0x55c6eb1e3090 2024-09-10T10:52:15.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.401+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v113'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb1e3090 2024-09-10T10:52:15.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.401+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 0x55c6eb28f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.401+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28fb30 con 0x55c6eb28d200 2024-09-10T10:52:15.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.401+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 0x55c6eb28f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.402+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 0x55c6eb28f5f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.403+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v113'287 uv266 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.403+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28fed0 con 0x55c6eb28d200 2024-09-10T10:52:15.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.405+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v113'288 uv288 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.405+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb290270 con 0x55c6eb28d200 2024-09-10T10:52:15.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.406+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v113'289 uv289 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.407 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:15.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.406+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb290610 con 0x55c6eb28d200 2024-09-10T10:52:15.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.408+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v113'290 uv290 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.408+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb290a00 con 0x55c6eb28d200 2024-09-10T10:52:15.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.409+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v113'291 uv291 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.409+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb290da0 con 0x55c6eb28d200 2024-09-10T10:52:15.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.410+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v113'292 uv285 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.410+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.412+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v113'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.412+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.413+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v113'294 uv294 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.414 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:15.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.413+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.414+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v113'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.414+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.416+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v113'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.416+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.417+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v113'297 uv286 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.417+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.418+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v113'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.418+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.420 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:15.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.419+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v113'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.419+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.421+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v113'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.421+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.422+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v113'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.423+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.424+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v113'302 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.424+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.425+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v113'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.425+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.427 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:15.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.427+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v113'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.427+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb292150 con 0x55c6eb28d200 2024-09-10T10:52:15.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.428+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v113'305 uv305 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.428+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb292150 con 0x55c6eb28d200 2024-09-10T10:52:15.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.429+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v113'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.429+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb291140 con 0x55c6eb28d200 2024-09-10T10:52:15.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.430+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v113'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb28d200 2024-09-10T10:52:15.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.430+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c6eb291a10 0x55c6eb291df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.430+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb294560 con 0x55c6eb291a10 2024-09-10T10:52:15.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.431+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c6eb291a10 0x55c6eb291df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.431+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c6eb291a10 0x55c6eb291df0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.435+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v113'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.435+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb294900 con 0x55c6eb291a10 2024-09-10T10:52:15.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.436+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v113'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.436+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb294ca0 con 0x55c6eb291a10 2024-09-10T10:52:15.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.438+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v113'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.439 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:15.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.438+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb28b2c0 con 0x55c6eb291a10 2024-09-10T10:52:15.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.439+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v113'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.439+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2965a0 con 0x55c6eb291a10 2024-09-10T10:52:15.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.440+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v113'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.441+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2954f0 con 0x55c6eb291a10 2024-09-10T10:52:15.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.442+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v113'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.442+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c6eb295f60 0x55c6eb298a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.442+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb298f40 con 0x55c6eb295f60 2024-09-10T10:52:15.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.443+0000 7fc5e2800640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c6eb295f60 0x55c6eb298a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.443+0000 7fc5e2800640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c6eb295f60 0x55c6eb298a00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.444+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v113'59 uv58 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.444+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299310 con 0x55c6eb295f60 2024-09-10T10:52:15.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.446+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v113'60 uv60 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.446+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299310 con 0x55c6eb295f60 2024-09-10T10:52:15.447 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:15.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.447+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v113'61 uv61 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.447+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299330 con 0x55c6eb295f60 2024-09-10T10:52:15.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.448+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v113'62 uv62 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.448+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299330 con 0x55c6eb291a10 2024-09-10T10:52:15.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.449+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v113'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.449+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299790 con 0x55c6eb291a10 2024-09-10T10:52:15.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.450+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v113'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.450+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299790 con 0x55c6eb291a10 2024-09-10T10:52:15.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.451+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v113'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.452 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:15.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.451+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29a7a0 con 0x55c6eb291a10 2024-09-10T10:52:15.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.453+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v113'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.453+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29a7a0 con 0x55c6eb291a10 2024-09-10T10:52:15.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.454+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v113'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.454+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299790 con 0x55c6eb291a10 2024-09-10T10:52:15.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.455+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v113'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.455+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299790 con 0x55c6eb295f60 2024-09-10T10:52:15.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.456+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v113'138 uv132 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.456+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb299790 con 0x55c6eb295f60 2024-09-10T10:52:15.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.458+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v113'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.458+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.460 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:15.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.459+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v113'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.459+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29a6e0 con 0x55c6eb295f60 2024-09-10T10:52:15.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.461+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v113'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.461+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29a6e0 con 0x55c6eb295f60 2024-09-10T10:52:15.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.462+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v113'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.462+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.464+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v113'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.464+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.465+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v113'144 uv137 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.465+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.467+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v113'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.467+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.469 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:15.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.468+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v113'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.468+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.470+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v113'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.470+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb295f60 2024-09-10T10:52:15.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.471+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v113'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.471+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb291a10 2024-09-10T10:52:15.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.472+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v113'61 uv60 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.472+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb291a10 2024-09-10T10:52:15.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.474+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v113'62 uv62 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.474+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb291a10 2024-09-10T10:52:15.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.475+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v113'63 uv63 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.476 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:15.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.475+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb291a10 2024-09-10T10:52:15.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.476+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v113'64 uv64 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.476+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2996d0 con 0x55c6eb291a10 2024-09-10T10:52:15.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.478+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v113'65 uv65 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb291a10 2024-09-10T10:52:15.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.478+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c6eb29a020 0x55c6eb29c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.478+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29c950 con 0x55c6eb29a020 2024-09-10T10:52:15.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.478+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c6eb29a020 0x55c6eb29c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.479+0000 7fc5e1e00640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c6eb29a020 0x55c6eb29c410 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.480+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v113'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.480+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.481+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v113'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.481+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.483 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:15.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.483+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v113'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.483+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29dd30 con 0x55c6eb29a020 2024-09-10T10:52:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.484+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v113'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.484+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29dd30 con 0x55c6eb29a020 2024-09-10T10:52:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.485+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v113'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.485+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.487+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v113'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.487+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.488+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v113'122 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.488+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.489+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v113'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.489+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb29a020 2024-09-10T10:52:15.491 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.491+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v113'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.491+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.492+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v113'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.492+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.493+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v113'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.493+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.495+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v113'127 uv121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.495+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.496+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v113'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.496+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.498 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.497+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v113'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.497+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2950e0 con 0x55c6eb29a020 2024-09-10T10:52:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.499+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v113'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.499+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29fb90 con 0x55c6eb29a020 2024-09-10T10:52:15.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.500+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v113'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.500+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb295f60 2024-09-10T10:52:15.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.502+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v113'57 uv55 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.502+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb295f60 2024-09-10T10:52:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.503+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v113'58 uv58 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.503+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb295f60 2024-09-10T10:52:15.505 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:15.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.504+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v113'59 uv59 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.504+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29cd20 con 0x55c6eb295f60 2024-09-10T10:52:15.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.506+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v113'60 uv60 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.506+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29d1d0 con 0x55c6eb295f60 2024-09-10T10:52:15.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.507+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v113'75 uv74 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.507+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29d1d0 con 0x55c6eb295f60 2024-09-10T10:52:15.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.508+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v113'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.508+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29d1d0 con 0x55c6eb295f60 2024-09-10T10:52:15.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.509+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v113'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.510 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:15.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.509+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29e1e0 con 0x55c6eb295f60 2024-09-10T10:52:15.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.511+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v113'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.511+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29e1e0 con 0x55c6eb295f60 2024-09-10T10:52:15.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.512+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v113'79 uv79 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.512+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb29d1d0 con 0x55c6eb295f60 2024-09-10T10:52:15.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.513+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v113'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5dc00f070 con 0x55c6eb295f60 2024-09-10T10:52:15.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.513+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c6eb29d850 0x55c6eb29fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.513+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29d850 2024-09-10T10:52:15.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.513+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c6eb29d850 0x55c6eb29fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.514+0000 7fc5e1400640 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c6eb29d850 0x55c6eb29fc40 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.515+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v113'64 uv63 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb29d850 2024-09-10T10:52:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.515+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29d850 2024-09-10T10:52:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.516+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v113'65 uv65 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb29d850 2024-09-10T10:52:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.516+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29d850 2024-09-10T10:52:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.517+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v113'66 uv66 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb29d850 2024-09-10T10:52:15.518 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.517+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29d850 2024-09-10T10:52:15.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.518+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v113'67 uv67 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb29d850 2024-09-10T10:52:15.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.518+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29d850 2024-09-10T10:52:15.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.519+0000 7fc5e1400640 1 -- 172.21.15.28:0/1359526892 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v113'68 uv68 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fc5d0071070 con 0x55c6eb29d850 2024-09-10T10:52:15.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.520+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29a020 2024-09-10T10:52:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.521+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v113'157 uv148 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.521+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0250 con 0x55c6eb29a020 2024-09-10T10:52:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.522+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v113'158 uv158 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.522+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0200 con 0x55c6eb29a020 2024-09-10T10:52:15.524 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.523+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v113'159 uv159 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.523+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0200 con 0x55c6eb29a020 2024-09-10T10:52:15.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.524+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v113'160 uv160 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.524+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0200 con 0x55c6eb29a020 2024-09-10T10:52:15.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.526+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v113'161 uv161 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.526+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.527+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v113'162 uv155 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.527+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.528+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v113'163 uv163 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.528+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.530 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:15.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.529+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v113'164 uv164 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.529+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.530+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v113'165 uv165 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.530+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.531+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v113'166 uv166 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.531+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a01d0 con 0x55c6eb29a020 2024-09-10T10:52:15.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.532+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v113'167 uv167 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.532+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x55c6eb2a0b90 con 0x55c6eb29a020 2024-09-10T10:52:15.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e1e00640 1 -- 172.21.15.28:0/1359526892 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v113'168 uv168 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fc5d4009040 con 0x55c6eb29a020 2024-09-10T10:52:15.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c6eb291a10 msgr2=0x55c6eb291df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c6eb291a10 0x55c6eb291df0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c6eb29a020 msgr2=0x55c6eb29c410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c6eb29a020 0x55c6eb29c410 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c6eb29d850 msgr2=0x55c6eb29fc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c6eb29d850 0x55c6eb29fc40 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c6eb295f60 msgr2=0x55c6eb298a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c6eb295f60 0x55c6eb298a00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 msgr2=0x55c6eb28f5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 0x55c6eb28f5f0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 reap_dead start 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e2800640 1 -- 172.21.15.28:0/1359526892 reap_dead start 2024-09-10T10:52:15.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 msgr2=0x55c6eb1e34a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 0x55c6eb1e34a0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 msgr2=0x7fc5c406ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 0x7fc5c406ccc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4001ed0 tx=0x7fc5d40016b0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 msgr2=0x55c6eb143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000a520 tx=0x7fc5d0002fb0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 shutdown_connections 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c6eb28d200 0x55c6eb28f5f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c6eb1e3090 0x55c6eb1e34a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5c406a800 0x7fc5c406ccc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 --2- 172.21.15.28:0/1359526892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c6eb1e63e0 0x55c6eb143fb0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 >> 172.21.15.28:0/1359526892 conn(0x55c6eb15f0a0 msgr2=0x55c6eb1e9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 shutdown_connections 2024-09-10T10:52:15.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.534+0000 7fc5e4bdaf40 1 -- 172.21.15.28:0/1359526892 wait complete. 2024-09-10T10:52:15.540 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 100.00000000.inode 2024-09-10T10:52:15.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.654+0000 7f1286223f40 1 Processor -- start 2024-09-10T10:52:15.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.654+0000 7f1286223f40 1 -- start start 2024-09-10T10:52:15.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44612e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b446129c0 0x557b44610a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b44584530 con 0x557b4460f3e0 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b444e1b30 con 0x557b44583ff0 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1286223f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557b44611ee0 con 0x557b446129c0 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1282a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1282a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43398/0 (socket says 172.21.15.28:43398) 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1282a00640 1 -- 172.21.15.28:0/2252979054 learned_addr learned my addr 172.21.15.28:0/2252979054 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1283e00640 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b446129c0 0x557b44610a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.655+0000 7f1283400640 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44612e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 -- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b446129c0 msgr2=0x557b44610a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b446129c0 0x557b44610a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 -- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 msgr2=0x557b44612e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44612e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 -- 172.21.15.28:0/2252979054 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4456cd80 con 0x557b4460f3e0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1282a00640 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1270001d00 tx=0x7f127000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa4e4cae6c06291 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1281600640 1 -- 172.21.15.28:0/2252979054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127000bd80 con 0x557b4460f3e0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1281600640 1 -- 172.21.15.28:0/2252979054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1270001e50 con 0x557b4460f3e0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.656+0000 7f1281600640 1 -- 172.21.15.28:0/2252979054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127000f640 con 0x557b4460f3e0 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 -- 172.21.15.28:0/2252979054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 msgr2=0x557b44605c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1270001d00 tx=0x7f127000b5b0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 -- 172.21.15.28:0/2252979054 shutdown_connections 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b446129c0 0x557b44610a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b4460f3e0 0x557b44605c70 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 --2- 172.21.15.28:0/2252979054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44612e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 -- 172.21.15.28:0/2252979054 >> 172.21.15.28:0/2252979054 conn(0x557b445880a0 msgr2=0x557b4460f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 -- 172.21.15.28:0/2252979054 shutdown_connections 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.657+0000 7f1286223f40 1 -- 172.21.15.28:0/2252979054 wait complete. 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.658+0000 7f1286223f40 1 Processor -- start 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.658+0000 7f1286223f40 1 -- start start 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.658+0000 7f1286223f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44610020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.658+0000 7f1286223f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b4460f3e0 0x557b446014a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.658+0000 7f1286223f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 0x557b44610a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1286223f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b44606390 con 0x557b446129c0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1286223f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b44615e80 con 0x557b44583ff0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1286223f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557b4456d020 con 0x557b4460f3e0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 0x557b44610a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 0x557b44610a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43404/0 (socket says 172.21.15.28:43404) 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 -- 172.21.15.28:0/1041788396 learned_addr learned my addr 172.21.15.28:0/1041788396 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283400640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44610020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1282a00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b4460f3e0 0x557b446014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 -- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b4460f3e0 msgr2=0x557b446014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b4460f3e0 0x557b446014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 -- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 msgr2=0x557b44610020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b44583ff0 0x557b44610020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.659+0000 7f1283e00640 1 -- 172.21.15.28:0/1041788396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b44610560 con 0x557b446129c0 2024-09-10T10:52:15.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f1283e00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 0x557b44610a50 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f127c045dc0 tx=0x7f127c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f127be00640 1 -- 172.21.15.28:0/1041788396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c064030 con 0x557b446129c0 2024-09-10T10:52:15.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f127be00640 1 -- 172.21.15.28:0/1041788396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127c060440 con 0x557b446129c0 2024-09-10T10:52:15.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f127be00640 1 -- 172.21.15.28:0/1041788396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0605f0 con 0x557b446129c0 2024-09-10T10:52:15.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557b44612e80 con 0x557b446129c0 2024-09-10T10:52:15.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.660+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b44613440 con 0x557b446129c0 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.662+0000 7f127be00640 1 -- 172.21.15.28:0/1041788396 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f127c067030 con 0x557b446129c0 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f127be00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f126406a840 0x7f126406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f127be00640 1 -- 172.21.15.28:0/1041788396 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7422+0+0 (secure 0 0 0) 0x7f127c0ce0f0 con 0x557b446129c0 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f1286223f40 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b44586a20 0x557b44601d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f1283400640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f126406a840 0x7f126406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e113) -- 0x557b44602280 con 0x557b44586a20 2024-09-10T10:52:15.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.663+0000 7f1282a00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b44586a20 0x557b44601d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.664+0000 7f1283400640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f126406a840 0x7f126406cd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1274005fd0 tx=0x7f127400d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.664+0000 7f1282a00640 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b44586a20 0x557b44601d40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1282a00640 1 -- 172.21.15.28:0/1041788396 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [delete] v113'169 uv169 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f127001d450 con 0x557b44586a20 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b44586a20 msgr2=0x557b44601d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b44586a20 0x557b44601d40 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f126406a840 msgr2=0x7f126406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f126406a840 0x7f126406cd00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1274005fd0 tx=0x7f127400d040 comp rx=0 tx=0).stop 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 msgr2=0x557b44610a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1286223f40 1 --2- 172.21.15.28:0/1041788396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b446129c0 0x557b44610a50 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f127c045dc0 tx=0x7f127c003330 comp rx=0 tx=0).stop 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.665+0000 7f1283e00640 1 -- 172.21.15.28:0/1041788396 reap_dead start 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.666+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 shutdown_connections 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.666+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 >> 172.21.15.28:0/1041788396 conn(0x557b445880a0 msgr2=0x557b44605c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.666+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 shutdown_connections 2024-09-10T10:52:15.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.666+0000 7f1286223f40 1 -- 172.21.15.28:0/1041788396 wait complete. 2024-09-10T10:52:15.671 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:15.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.831+0000 7fedecc00640 1 Processor -- start 2024-09-10T10:52:15.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.831+0000 7fedecc00640 1 -- start start 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fedecc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fede8162340 0x7fede806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fedecc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede815ff70 con 0x7fede80cf280 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fedecc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede80d4d10 con 0x7fede8162e30 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fedecc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fede81632a0 con 0x7fede8162340 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fede5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fede8162340 0x7fede806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fede5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fede8162340 0x7fede806d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43410/0 (socket says 172.21.15.28:43410) 2024-09-10T10:52:15.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7fede5e00640 1 -- 172.21.15.28:0/3886593025 learned_addr learned my addr 172.21.15.28:0/3886593025 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.832+0000 7feddfe00640 1 --1- 172.21.15.28:0/3886593025 >> v1:172.21.15.28:6789/0 conn(0x7fede80cf280 0x7fede8162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58760/0 (socket says 172.21.15.28:58760) 2024-09-10T10:52:15.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.833+0000 7fede5e00640 1 -- 172.21.15.28:0/3886593025 >> v1:172.21.15.138:6789/0 conn(0x7fede8162e30 legacy=0x7fede80d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.833+0000 7fede5e00640 1 -- 172.21.15.28:0/3886593025 >> v1:172.21.15.28:6789/0 conn(0x7fede80cf280 legacy=0x7fede8162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.833+0000 7fede5e00640 1 -- 172.21.15.28:0/3886593025 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80d4ff0 con 0x7fede8162340 2024-09-10T10:52:15.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fede5e00640 1 --2- 172.21.15.28:0/3886593025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162340 0x7fede806d740 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009870 tx=0x7fedd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5cc846194e5493 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fede5400640 1 -- 172.21.15.28:0/3886593025 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037070 con 0x7fede8162340 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fede5400640 1 -- 172.21.15.28:0/3886593025 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8032440 con 0x7fede8162340 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fede5400640 1 -- 172.21.15.28:0/3886593025 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8032810 con 0x7fede8162340 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fedecc00640 1 -- 172.21.15.28:0/3886593025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162340 msgr2=0x7fede806d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.834+0000 7fedecc00640 1 --2- 172.21.15.28:0/3886593025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162340 0x7fede806d740 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fedd8009870 tx=0x7fedd802a270 comp rx=0 tx=0).stop 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.835+0000 7fedecc00640 1 -- 172.21.15.28:0/3886593025 shutdown_connections 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.835+0000 7fedecc00640 1 --2- 172.21.15.28:0/3886593025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162340 0x7fede806d740 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.835+0000 7fedecc00640 1 -- 172.21.15.28:0/3886593025 >> 172.21.15.28:0/3886593025 conn(0x7fede808bfc0 msgr2=0x7fede80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.835+0000 7fedecc00640 1 -- 172.21.15.28:0/3886593025 shutdown_connections 2024-09-10T10:52:15.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.835+0000 7fedecc00640 1 -- 172.21.15.28:0/3886593025 wait complete. 2024-09-10T10:52:15.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.836+0000 7fedecc00640 1 Processor -- start 2024-09-10T10:52:15.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.836+0000 7fedecc00640 1 -- start start 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede80cf280 0x7fede8157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8162340 0x7fede8152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fede80d52d0 con 0x7fede8162e30 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fede806de90 con 0x7fede80cf280 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fedecc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fede80cf6c0 con 0x7fede8162340 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.837+0000 7fede6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43412/0 (socket says 172.21.15.28:43412) 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 -- 172.21.15.28:0/1876197511 learned_addr learned my addr 172.21.15.28:0/1876197511 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede5e00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede80cf280 0x7fede8157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7feddfe00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8162340 0x7fede8152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 -- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8162340 msgr2=0x7fede8152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8162340 0x7fede8152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 -- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede80cf280 msgr2=0x7fede8157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede80cf280 0x7fede8157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 -- 172.21.15.28:0/1876197511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede8156720 con 0x7fede8162e30 2024-09-10T10:52:15.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fede6800640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fede000c6b0 tx=0x7fede000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0003b70 con 0x7fede8162e30 2024-09-10T10:52:15.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fede0003d30 con 0x7fede8162e30 2024-09-10T10:52:15.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.838+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0005310 con 0x7fede8162e30 2024-09-10T10:52:15.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.839+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede8156990 con 0x7fede8162e30 2024-09-10T10:52:15.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.839+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede8156f50 con 0x7fede8162e30 2024-09-10T10:52:15.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.840+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede806d740 con 0x7fede8162e30 2024-09-10T10:52:15.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.841+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fede00054b0 con 0x7fede8162e30 2024-09-10T10:52:15.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.841+0000 7fedde000640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 0x7fedc406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:15.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.842+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 7422+0+0 (secure 0 0 0) 0x7fede008b320 con 0x7fede8162e30 2024-09-10T10:52:15.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.842+0000 7fede5e00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 0x7fedc406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:15.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.842+0000 7fede5e00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 0x7fedc406e320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:15.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:15.846+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fede0059610 con 0x7fede8162e30 2024-09-10T10:52:16.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.066+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fede80ba450 con 0x7fede8162e30 2024-09-10T10:52:16.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.683+0000 7fedde000640 1 -- 172.21.15.28:0/1876197511 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v164) ==== 162+0+0 (secure 0 0 0) 0x7fede0058f10 con 0x7fede8162e30 2024-09-10T10:52:16.684 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:16.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 msgr2=0x7fedc406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 0x7fedc406e320 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).stop 2024-09-10T10:52:16.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 msgr2=0x7fede8157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fede000c6b0 tx=0x7fede000cb70 comp rx=0 tx=0).stop 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 shutdown_connections 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fedc406be60 0x7fedc406e320 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fede8162e30 0x7fede8157fd0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8162340 0x7fede8152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 --2- 172.21.15.28:0/1876197511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede80cf280 0x7fede8157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 >> 172.21.15.28:0/1876197511 conn(0x7fede808bfc0 msgr2=0x7fede80d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.687+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 shutdown_connections 2024-09-10T10:52:16.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.688+0000 7fedecc00640 1 -- 172.21.15.28:0/1876197511 wait complete. 2024-09-10T10:52:16.694 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:52:16.693+0000 7f72c3e00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T10:52:16.694 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:52:16.693+0000 7f72c3e00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:52:16.695 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:16.694+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:16.695 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:16.694+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:16.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:16.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.872+0000 7fdfeaa00640 1 Processor -- start 2024-09-10T10:52:16.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.872+0000 7fdfeaa00640 1 -- start start 2024-09-10T10:52:16.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfeaa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 0x7fdfe40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfeaa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdfe415e660 0x7fdfe406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfeaa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe40cf000 con 0x7fdfe40d1f90 2024-09-10T10:52:16.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfeaa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe40b37d0 con 0x7fdfe415e660 2024-09-10T10:52:16.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfeaa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe40cf330 con 0x7fdfe40d0dd0 2024-09-10T10:52:16.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.873+0000 7fdfe8c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdfe415e660 0x7fdfe406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdfe415e660 0x7fdfe406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43416/0 (socket says 172.21.15.28:43416) 2024-09-10T10:52:16.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2591172758 learned_addr learned my addr 172.21.15.28:0/2591172758 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe9600640 1 --2- 172.21.15.28:0/2591172758 >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 0x7fdfe40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2591172758 >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 msgr2=0x7fdfe40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 --2- 172.21.15.28:0/2591172758 >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 0x7fdfe40d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2591172758 >> v1:172.21.15.138:6789/0 conn(0x7fdfe40d1f90 legacy=0x7fdfe40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2591172758 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe40d16f0 con 0x7fdfe415e660 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.874+0000 7fdfe9600640 1 --2- 172.21.15.28:0/2591172758 >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 0x7fdfe40d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:16.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.875+0000 7fdfe8c00640 1 --2- 172.21.15.28:0/2591172758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe415e660 0x7fdfe406ccd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b6fb0 tx=0x7fdfd002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153203433820d368 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.875+0000 7fdfe3e00640 1 -- 172.21.15.28:0/2591172758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0037070 con 0x7fdfe415e660 2024-09-10T10:52:16.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.875+0000 7fdfe3e00640 1 -- 172.21.15.28:0/2591172758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd0032440 con 0x7fdfe415e660 2024-09-10T10:52:16.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.875+0000 7fdfe3e00640 1 -- 172.21.15.28:0/2591172758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0032810 con 0x7fdfe415e660 2024-09-10T10:52:16.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2591172758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe415e660 msgr2=0x7fdfe406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2591172758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe415e660 0x7fdfe406ccd0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b6fb0 tx=0x7fdfd002a230 comp rx=0 tx=0).stop 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2591172758 shutdown_connections 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2591172758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe415e660 0x7fdfe406ccd0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2591172758 >> v2:172.21.15.138:3300/0 conn(0x7fdfe40d0dd0 0x7fdfe40d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2591172758 >> 172.21.15.28:0/2591172758 conn(0x7fdfe408bfc0 msgr2=0x7fdfe40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2591172758 shutdown_connections 2024-09-10T10:52:16.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.876+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2591172758 wait complete. 2024-09-10T10:52:16.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.877+0000 7fdfeaa00640 1 Processor -- start 2024-09-10T10:52:16.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 -- start start 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40d0dd0 0x7fdfe415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfe415e660 0x7fdfe41546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe40d2f40 con 0x7fdfe40d1f90 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe40d2c60 con 0x7fdfe40d0dd0 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.878+0000 7fdfeaa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe40d3220 con 0x7fdfe415e660 2024-09-10T10:52:16.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43430/0 (socket says 172.21.15.28:43430) 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2662605473 learned_addr learned my addr 172.21.15.28:0/2662605473 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe9600640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40d0dd0 0x7fdfe415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfea000640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfe415e660 0x7fdfe41546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfe415e660 msgr2=0x7fdfe41546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfe415e660 0x7fdfe41546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40d0dd0 msgr2=0x7fdfe415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40d0dd0 0x7fdfe415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:16.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.879+0000 7fdfe8c00640 1 -- 172.21.15.28:0/2662605473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe415ece0 con 0x7fdfe40d1f90 2024-09-10T10:52:16.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfe8c00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fdfd002a270 tx=0x7fdfd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0041020 con 0x7fdfe40d1f90 2024-09-10T10:52:16.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd0030990 con 0x7fdfe40d1f90 2024-09-10T10:52:16.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe415efc0 con 0x7fdfe40d1f90 2024-09-10T10:52:16.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0037040 con 0x7fdfe40d1f90 2024-09-10T10:52:16.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.880+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe415f580 con 0x7fdfe40d1f90 2024-09-10T10:52:16.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.881+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe40d2420 con 0x7fdfe40d1f90 2024-09-10T10:52:16.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.882+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdfd0044020 con 0x7fdfe40d1f90 2024-09-10T10:52:16.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.883+0000 7fdfe1600640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 0x7fdfc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:16.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.883+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7fdfd00abeb0 con 0x7fdfe40d1f90 2024-09-10T10:52:16.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.883+0000 7fdfe9600640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 0x7fdfc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:16.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.884+0000 7fdfe9600640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 0x7fdfc406e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800aac0 tx=0x7fdfd8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:16.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:16.887+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdfd004b030 con 0x7fdfe40d1f90 2024-09-10T10:52:17.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.110+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfe40d1b20 con 0x7fdfe40d1f90 2024-09-10T10:52:17.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.111+0000 7fdfe1600640 1 -- 172.21.15.28:0/2662605473 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) ==== 95+0+4392 (secure 0 0 0) 0x7fdfd0079af0 con 0x7fdfe40d1f90 2024-09-10T10:52:17.112 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:17.112 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":165,"btime":"2024-09-10T10:52:16:694896+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8304,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1314136636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1314136636},{"type":"v1","addr":"172.21.15.28:6834","nonce":1314136636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:16.694815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:17.113 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 165 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 msgr2=0x7fdfc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 0x7fdfc406e2f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800aac0 tx=0x7fdfd8008040 comp rx=0 tx=0).stop 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 msgr2=0x7fdfe415b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fdfd002a270 tx=0x7fdfd00046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 shutdown_connections 2024-09-10T10:52:17.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfc406be30 0x7fdfc406e2f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfe415e660 0x7fdfe41546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdfe40d1f90 0x7fdfe415b170 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 --2- 172.21.15.28:0/2662605473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40d0dd0 0x7fdfe415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 >> 172.21.15.28:0/2662605473 conn(0x7fdfe408bfc0 msgr2=0x7fdfe406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 shutdown_connections 2024-09-10T10:52:17.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.114+0000 7fdfeaa00640 1 -- 172.21.15.28:0/2662605473 wait complete. 2024-09-10T10:52:17.130 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:17.131 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:17.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.287+0000 7f075fc00640 1 Processor -- start 2024-09-10T10:52:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.287+0000 7f075fc00640 1 -- start start 2024-09-10T10:52:17.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.288+0000 7f075fc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0758072de0 0x7f075815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.288+0000 7f075fc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0758146140 con 0x7f0758072700 2024-09-10T10:52:17.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f075fc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07580cd740 con 0x7f0758072320 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f075fc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f07580cda20 con 0x7f0758072de0 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f075dc00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0758072700 0x7f075815d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58774/0 (socket says 172.21.15.28:58774) 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f075dc00640 1 -- 172.21.15.28:0/2184721170 learned_addr learned my addr 172.21.15.28:0/2184721170 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f0757e00640 1 --2- 172.21.15.28:0/2184721170 >> v2:172.21.15.138:3300/0 conn(0x7f0758072de0 0x7f075815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f0757400640 1 -- 172.21.15.28:0/2184721170 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1310095592 0 0) 0x7f0758146140 con 0x7f0758072700 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.289+0000 7f0757400640 1 -- 172.21.15.28:0/2184721170 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0740003520 con 0x7f0758072700 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.290+0000 7f0757e00640 1 -- 172.21.15.28:0/2184721170 >> v1:172.21.15.138:6789/0 conn(0x7f0758072320 legacy=0x7f07580cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.290+0000 7f0757e00640 1 -- 172.21.15.28:0/2184721170 >> v1:172.21.15.28:6789/0 conn(0x7f0758072700 legacy=0x7f075815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.290+0000 7f0757e00640 1 -- 172.21.15.28:0/2184721170 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07580cdd00 con 0x7f0758072de0 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.291+0000 7f0757e00640 1 --2- 172.21.15.28:0/2184721170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072de0 0x7f075815d1e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f07440097a0 tx=0x7f074402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9a02bdb7ec1c1ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.291+0000 7f0757400640 1 -- 172.21.15.28:0/2184721170 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744037070 con 0x7f0758072de0 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.291+0000 7f0757400640 1 -- 172.21.15.28:0/2184721170 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0744032440 con 0x7f0758072de0 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.291+0000 7f0757400640 1 -- 172.21.15.28:0/2184721170 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744032810 con 0x7f0758072de0 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 -- 172.21.15.28:0/2184721170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072de0 msgr2=0x7f075815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 --2- 172.21.15.28:0/2184721170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072de0 0x7f075815d1e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f07440097a0 tx=0x7f074402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 -- 172.21.15.28:0/2184721170 shutdown_connections 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 --2- 172.21.15.28:0/2184721170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072de0 0x7f075815d1e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 -- 172.21.15.28:0/2184721170 >> 172.21.15.28:0/2184721170 conn(0x7f0758067d70 msgr2=0x7f075806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:17.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 -- 172.21.15.28:0/2184721170 shutdown_connections 2024-09-10T10:52:17.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.292+0000 7f075fc00640 1 -- 172.21.15.28:0/2184721170 wait complete. 2024-09-10T10:52:17.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.293+0000 7f075fc00640 1 Processor -- start 2024-09-10T10:52:17.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.294+0000 7f075fc00640 1 -- start start 2024-09-10T10:52:17.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.294+0000 7f075fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072320 0x7f07581f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075fc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0758072de0 0x7f07581f5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075fc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f075815dfb0 con 0x7f0758072700 2024-09-10T10:52:17.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f075815de30 con 0x7f0758072320 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075fc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f075815e130 con 0x7f0758072de0 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43438/0 (socket says 172.21.15.28:43438) 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 -- 172.21.15.28:0/3280091117 learned_addr learned my addr 172.21.15.28:0/3280091117 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075d200640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072320 0x7f07581f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f075dc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0758072de0 0x7f07581f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 -- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0758072de0 msgr2=0x7f07581f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0758072de0 0x7f07581f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 -- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072320 msgr2=0x7f07581f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072320 0x7f07581f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.295+0000 7f0757e00640 1 -- 172.21.15.28:0/3280091117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07581f5fa0 con 0x7f0758072700 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f0757e00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f07440098c0 tx=0x7f07440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744037040 con 0x7f0758072700 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0744030500 con 0x7f0758072700 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440308d0 con 0x7f0758072700 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581f6280 con 0x7f0758072700 2024-09-10T10:52:17.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.296+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581f6840 con 0x7f0758072700 2024-09-10T10:52:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.297+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07580cce60 con 0x7f0758072700 2024-09-10T10:52:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.298+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0744044020 con 0x7f0758072700 2024-09-10T10:52:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.299+0000 7f0754c00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 0x7f073806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.299+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7f07440ab780 con 0x7f0758072700 2024-09-10T10:52:17.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.299+0000 7f075d200640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 0x7f073806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.300+0000 7f075d200640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 0x7f073806e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f074c000f60 tx=0x7f074c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.303+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0744079a10 con 0x7f0758072700 2024-09-10T10:52:17.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.525+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0758145610 con 0x7f0758072700 2024-09-10T10:52:17.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.525+0000 7f0754c00640 1 -- 172.21.15.28:0/3280091117 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) ==== 95+0+4392 (secure 0 0 0) 0x7f0744079310 con 0x7f0758072700 2024-09-10T10:52:17.526 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:17.527 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":165,"btime":"2024-09-10T10:52:16:694896+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8304,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1314136636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1314136636},{"type":"v1","addr":"172.21.15.28:6834","nonce":1314136636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:16.694815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:17.527 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 165 2024-09-10T10:52:17.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 msgr2=0x7f073806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 0x7f073806e2f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f074c000f60 tx=0x7f074c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 msgr2=0x7f07581f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f07440098c0 tx=0x7f07440046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 shutdown_connections 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f073806be30 0x7f073806e2f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0758072de0 0x7f07581f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0758072700 0x7f07581f5430 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 --2- 172.21.15.28:0/3280091117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0758072320 0x7f07581f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 >> 172.21.15.28:0/3280091117 conn(0x7f0758067d70 msgr2=0x7f075806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 shutdown_connections 2024-09-10T10:52:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.527+0000 7f075fc00640 1 -- 172.21.15.28:0/3280091117 wait complete. 2024-09-10T10:52:17.546 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:17.546 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:17.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.700+0000 7ffaac000640 1 Processor -- start 2024-09-10T10:52:17.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.700+0000 7ffaac000640 1 -- start start 2024-09-10T10:52:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.701+0000 7ffaac000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147e90 0x7ffaa40720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.701+0000 7ffaac000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffaa40725f0 0x7ffaa40aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.701+0000 7ffaac000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa406f480 con 0x7ffaa4147230 2024-09-10T10:52:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.701+0000 7ffaac000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa40b34d0 con 0x7ffaa40725f0 2024-09-10T10:52:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.701+0000 7ffaac000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa40ae9e0 con 0x7ffaa4147e90 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffaa40725f0 0x7ffaa40aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffaa40725f0 0x7ffaa40aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43448/0 (socket says 172.21.15.28:43448) 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 -- 172.21.15.28:0/3758081812 learned_addr learned my addr 172.21.15.28:0/3758081812 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaa8c00640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147e90 0x7ffaa40720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 -- 172.21.15.28:0/3758081812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147e90 msgr2=0x7ffaa40720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147e90 0x7ffaa40720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 -- 172.21.15.28:0/3758081812 >> v1:172.21.15.138:6789/0 conn(0x7ffaa4147230 legacy=0x7ffaa406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.702+0000 7ffaaa000640 1 -- 172.21.15.28:0/3758081812 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa406e900 con 0x7ffaa40725f0 2024-09-10T10:52:17.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.703+0000 7ffaaa000640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa40725f0 0x7ffaa40aee10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009e50 tx=0x7ffa9c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55706e173b028e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.703+0000 7ffaa3e00640 1 -- 172.21.15.28:0/3758081812 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c036070 con 0x7ffaa40725f0 2024-09-10T10:52:17.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.703+0000 7ffaa3e00640 1 -- 172.21.15.28:0/3758081812 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9c031600 con 0x7ffaa40725f0 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.703+0000 7ffaa3e00640 1 -- 172.21.15.28:0/3758081812 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0319d0 con 0x7ffaa40725f0 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 -- 172.21.15.28:0/3758081812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa40725f0 msgr2=0x7ffaa40aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa40725f0 0x7ffaa40aee10 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009e50 tx=0x7ffa9c02a710 comp rx=0 tx=0).stop 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 -- 172.21.15.28:0/3758081812 shutdown_connections 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa40725f0 0x7ffaa40aee10 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 --2- 172.21.15.28:0/3758081812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147e90 0x7ffaa40720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 -- 172.21.15.28:0/3758081812 >> 172.21.15.28:0/3758081812 conn(0x7ffaa4067d70 msgr2=0x7ffaa406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 -- 172.21.15.28:0/3758081812 shutdown_connections 2024-09-10T10:52:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.704+0000 7ffaac000640 1 -- 172.21.15.28:0/3758081812 wait complete. 2024-09-10T10:52:17.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.706+0000 7ffaac000640 1 Processor -- start 2024-09-10T10:52:17.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.706+0000 7ffaac000640 1 -- start start 2024-09-10T10:52:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaa40725f0 0x7ffaa415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147230 0x7ffaa40c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa406eec0 con 0x7ffaa4147e90 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4072a40 con 0x7ffaa40725f0 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaac000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa406f1a0 con 0x7ffaa4147230 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43452/0 (socket says 172.21.15.28:43452) 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 -- 172.21.15.28:0/2940190730 learned_addr learned my addr 172.21.15.28:0/2940190730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaa9600640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaa40725f0 0x7ffaa415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaa8c00640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147230 0x7ffaa40c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 -- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147230 msgr2=0x7ffaa40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147230 0x7ffaa40c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 -- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaa40725f0 msgr2=0x7ffaa415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaa40725f0 0x7ffaa415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.707+0000 7ffaaa000640 1 -- 172.21.15.28:0/2940190730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa41f4f10 con 0x7ffaa4147e90 2024-09-10T10:52:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaaa000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009f70 tx=0x7ffa9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c03f020 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9c037040 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c036070 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41f51f0 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.708+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa40c57c0 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.709+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa40b52b0 con 0x7ffaa4147e90 2024-09-10T10:52:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.710+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffa9c042020 con 0x7ffaa4147e90 2024-09-10T10:52:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.711+0000 7ffaa1600640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 0x7ffa8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.711+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7ffa9c0aba40 con 0x7ffaa4147e90 2024-09-10T10:52:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.712+0000 7ffaa9600640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 0x7ffa8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.712+0000 7ffaa9600640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 0x7ffa8406e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffa98001780 tx=0x7ffa98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:17.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.716+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa9c049030 con 0x7ffaa4147e90 2024-09-10T10:52:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.935+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaa4147610 con 0x7ffaa4147e90 2024-09-10T10:52:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.936+0000 7ffaa1600640 1 -- 172.21.15.28:0/2940190730 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) ==== 95+0+4392 (secure 0 0 0) 0x7ffa9c079c40 con 0x7ffaa4147e90 2024-09-10T10:52:17.937 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:17.937 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":165,"btime":"2024-09-10T10:52:16:694896+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8304,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1314136636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1314136636},{"type":"v1","addr":"172.21.15.28:6834","nonce":1314136636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":159}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:16.694815+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 165 2024-09-10T10:52:17.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 msgr2=0x7ffa8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 0x7ffa8406e2f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffa98001780 tx=0x7ffa98009040 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 msgr2=0x7ffaa41e36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009f70 tx=0x7ffa9c004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 shutdown_connections 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffa8406be30 0x7ffa8406e2f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffaa4147e90 0x7ffaa41e36c0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffaa4147230 0x7ffaa40c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 --2- 172.21.15.28:0/2940190730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffaa40725f0 0x7ffaa415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 >> 172.21.15.28:0/2940190730 conn(0x7ffaa4067d70 msgr2=0x7ffaa406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 shutdown_connections 2024-09-10T10:52:17.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:17.938+0000 7ffaac000640 1 -- 172.21.15.28:0/2940190730 wait complete. 2024-09-10T10:52:17.955 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 100.00000000.inode' led to DAMAGED state 2024-09-10T10:52:17.955 INFO:tasks.cephfs.test_damage:Applying mutation 'Delete 1.00000000' 2024-09-10T10:52:17.955 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:17.955 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:17.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:17.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:17.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:17.991 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:18.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:18.067 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:18.067 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:18.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.263+0000 7f50a3200640 1 Processor -- start 2024-09-10T10:52:18.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.263+0000 7f50a3200640 1 -- start start 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50a40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a40cf000 con 0x7f50a415e660 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50a40d29a0 con 0x7f50a40d0dd0 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50a406d3e0 con 0x7f50a40d1f90 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a1400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f50a415e660 0x7f50a406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58782/0 (socket says 172.21.15.28:58782) 2024-09-10T10:52:18.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.264+0000 7f50a1400640 1 -- 172.21.15.28:0/1358273069 learned_addr learned my addr 172.21.15.28:0/1358273069 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:18.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a2800640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50a40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3382668597 0 0) 0x7f50a40cf000 con 0x7f50a415e660 2024-09-10T10:52:18.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5088003520 con 0x7f50a415e660 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177256895 0 0) 0x7f50a40d29a0 con 0x7f50a40d0dd0 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50a40cf000 con 0x7f50a40d0dd0 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1131134524 0 0) 0x7f5088003520 con 0x7f50a415e660 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 msgr2=0x7f50a40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50a40cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 >> v1:172.21.15.138:6789/0 conn(0x7f50a40d0dd0 legacy=0x7f50a40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a40d2c60 con 0x7f50a415e660 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f5090001670 con 0x7f50a415e660 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 >> v1:172.21.15.28:6789/0 conn(0x7f50a415e660 legacy=0x7f50a406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088003b70 0x7f5088003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5088004800 0x7f5088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50a406d3e0 con 0x7f5088004800 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f508800baa0 con 0x7f5088003b70 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.265+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f508800bca0 con 0x7f50a40d1f90 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5088004800 0x7f5088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a2800640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1e00640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088003b70 0x7f5088003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 -- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 msgr2=0x7f50880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a40d1f90 0x7f50880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 -- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088003b70 msgr2=0x7f5088003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088003b70 0x7f5088003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.266+0000 7f50a1400640 1 -- 172.21.15.28:0/1358273069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5088003520 con 0x7f5088004800 2024-09-10T10:52:18.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.267+0000 7f50a1400640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5088004800 0x7f5088004c80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f50a415ea60 tx=0x7f5088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2787b347e19823 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.267+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090007c00 con 0x7f5088004800 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.267+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5090005af0 con 0x7f5088004800 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.267+0000 7f50a0a00640 1 -- 172.21.15.28:0/1358273069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090007070 con 0x7f5088004800 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 -- 172.21.15.28:0/1358273069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5088004800 msgr2=0x7f5088004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 --2- 172.21.15.28:0/1358273069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5088004800 0x7f5088004c80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f50a415ea60 tx=0x7f5088005830 comp rx=0 tx=0).stop 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a2800640 1 -- 172.21.15.28:0/1358273069 reap_dead start 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 -- 172.21.15.28:0/1358273069 shutdown_connections 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 -- 172.21.15.28:0/1358273069 >> 172.21.15.28:0/1358273069 conn(0x7f50a408bfc0 msgr2=0x7f50a40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 -- 172.21.15.28:0/1358273069 shutdown_connections 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.268+0000 7f50a3200640 1 -- 172.21.15.28:0/1358273069 wait complete. 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.270+0000 7f50a3200640 1 Processor -- start 2024-09-10T10:52:18.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.270+0000 7f50a3200640 1 -- start start 2024-09-10T10:52:18.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 0x7f50a40c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a41e9190 0x7f50a41e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50a40cf330 con 0x7f50a41e8d10 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50a40d3220 con 0x7f5088004800 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50a40d33a0 con 0x7f50a41e9190 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43478/0 (socket says 172.21.15.28:43478) 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 -- 172.21.15.28:0/2951777217 learned_addr learned my addr 172.21.15.28:0/2951777217 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1e00640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 0x7f50a40c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a2800640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a41e9190 0x7f50a41e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 -- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a41e9190 msgr2=0x7f50a41e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a41e9190 0x7f50a41e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 -- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 msgr2=0x7f50a40c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 0x7f50a40c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1400640 1 -- 172.21.15.28:0/2951777217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a40c7760 con 0x7f50a41e8d10 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.271+0000 7f50a1e00640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 0x7f50a40c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f50a1400640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f509002aca0 tx=0x7f5090001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090042070 con 0x7f50a41e8d10 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50900047e0 con 0x7f50a41e8d10 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090037040 con 0x7f50a41e8d10 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a40cbe40 con 0x7f50a41e8d10 2024-09-10T10:52:18.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.272+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a40cc400 con 0x7f50a41e8d10 2024-09-10T10:52:18.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.273+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a40d2510 con 0x7f50a41e8d10 2024-09-10T10:52:18.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.274+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5090045070 con 0x7f50a41e8d10 2024-09-10T10:52:18.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.275+0000 7f509a000640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 0x7f508406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.275+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7f50900aad20 con 0x7f50a41e8d10 2024-09-10T10:52:18.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.275+0000 7f50a1e00640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 0x7f508406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.276+0000 7f50a1e00640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 0x7f508406e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f5094009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.279+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50a40c7760 con 0x7f50a41e8d10 2024-09-10T10:52:18.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.498+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50a40d1b20 con 0x7f50a41e8d10 2024-09-10T10:52:18.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.749+0000 7f509a000640 1 -- 172.21.15.28:0/2951777217 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v166) ==== 161+0+0 (secure 0 0 0) 0x7f50900788d0 con 0x7f50a41e8d10 2024-09-10T10:52:18.750 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 msgr2=0x7f508406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 0x7f508406e2f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5094000c00 tx=0x7f5094009040 comp rx=0 tx=0).stop 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 msgr2=0x7f50a41e3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f509002aca0 tx=0x7f5090001970 comp rx=0 tx=0).stop 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 shutdown_connections 2024-09-10T10:52:18.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f508406be30 0x7f508406e2f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50a41e9190 0x7f50a41e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50a41e8d10 0x7f50a41e3750 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 --2- 172.21.15.28:0/2951777217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5088004800 0x7f50a40c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 >> 172.21.15.28:0/2951777217 conn(0x7f50a408bfc0 msgr2=0x7f50a406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 shutdown_connections 2024-09-10T10:52:18.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.751+0000 7f50a3200640 1 -- 172.21.15.28:0/2951777217 wait complete. 2024-09-10T10:52:18.769 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:18.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.924+0000 7fc5cc200640 1 Processor -- start 2024-09-10T10:52:18.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.924+0000 7fc5cc200640 1 -- start start 2024-09-10T10:52:18.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.925+0000 7fc5cc200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 0x7fc5c40cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.925+0000 7fc5cc200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc5c4162ad0 0x7fc5c406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5cc200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c40b49a0 con 0x7fc5c4072490 2024-09-10T10:52:18.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5cc200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c40afa20 con 0x7fc5c4162ad0 2024-09-10T10:52:18.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5cc200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c40afce0 con 0x7fc5c40720b0 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5c9800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 0x7fc5c40cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5ca200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc5c4072490 0x7fc5c40b4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:34698/0 (socket says 172.21.15.28:34698) 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5ca200640 1 -- 172.21.15.28:0/2562165756 learned_addr learned my addr 172.21.15.28:0/2562165756 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.926+0000 7fc5c8e00640 1 --2- 172.21.15.28:0/2562165756 >> v2:172.21.15.138:3300/0 conn(0x7fc5c4162ad0 0x7fc5c406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.927+0000 7fc5c9800640 1 -- 172.21.15.28:0/2562165756 >> v2:172.21.15.138:3300/0 conn(0x7fc5c4162ad0 msgr2=0x7fc5c406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.927+0000 7fc5c9800640 1 --2- 172.21.15.28:0/2562165756 >> v2:172.21.15.138:3300/0 conn(0x7fc5c4162ad0 0x7fc5c406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.927+0000 7fc5c9800640 1 -- 172.21.15.28:0/2562165756 >> v1:172.21.15.138:6789/0 conn(0x7fc5c4072490 legacy=0x7fc5c40b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.927+0000 7fc5c3e00640 1 -- 172.21.15.28:0/2562165756 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233856197 0 0) 0x7fc5c40b49a0 con 0x7fc5c4072490 2024-09-10T10:52:18.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.927+0000 7fc5c9800640 1 -- 172.21.15.28:0/2562165756 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c40b34d0 con 0x7fc5c40720b0 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5c9800640 1 --2- 172.21.15.28:0/2562165756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 0x7fc5c40cfca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097a0 tx=0x7fc5b802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167b760e2d19e468 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5c3e00640 1 -- 172.21.15.28:0/2562165756 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8037070 con 0x7fc5c40720b0 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5c3e00640 1 -- 172.21.15.28:0/2562165756 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b8032440 con 0x7fc5c40720b0 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5c3e00640 1 -- 172.21.15.28:0/2562165756 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8032810 con 0x7fc5c40720b0 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5cc200640 1 -- 172.21.15.28:0/2562165756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 msgr2=0x7fc5c40cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.928+0000 7fc5cc200640 1 --2- 172.21.15.28:0/2562165756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 0x7fc5c40cfca0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097a0 tx=0x7fc5b802a180 comp rx=0 tx=0).stop 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 -- 172.21.15.28:0/2562165756 shutdown_connections 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 --2- 172.21.15.28:0/2562165756 >> v2:172.21.15.138:3300/0 conn(0x7fc5c4162ad0 0x7fc5c406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 --2- 172.21.15.28:0/2562165756 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c40720b0 0x7fc5c40cfca0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 -- 172.21.15.28:0/2562165756 >> 172.21.15.28:0/2562165756 conn(0x7fc5c4067d70 msgr2=0x7fc5c406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:18.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 -- 172.21.15.28:0/2562165756 shutdown_connections 2024-09-10T10:52:18.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.929+0000 7fc5cc200640 1 -- 172.21.15.28:0/2562165756 wait complete. 2024-09-10T10:52:18.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.930+0000 7fc5cc200640 1 Processor -- start 2024-09-10T10:52:18.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.930+0000 7fc5cc200640 1 -- start start 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 0x7fc5c415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 0x7fc5c41548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40b3d70 con 0x7fc5c40720b0 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40b3a90 con 0x7fc5c4072490 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.931+0000 7fc5cc200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c4072900 con 0x7fc5c4162ad0 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43488/0 (socket says 172.21.15.28:43488) 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 -- 172.21.15.28:0/526951813 learned_addr learned my addr 172.21.15.28:0/526951813 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:18.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 -- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 msgr2=0x7fc5c41548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c8e00640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 0x7fc5c415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5ca200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 0x7fc5c41548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 0x7fc5c41548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 -- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 msgr2=0x7fc5c415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 0x7fc5c415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 -- 172.21.15.28:0/526951813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c4157a20 con 0x7fc5c40720b0 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c8e00640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 0x7fc5c415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5ca200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 0x7fc5c41548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.932+0000 7fc5c9800640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000c00 tx=0x7fc5b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.933+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8037040 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.933+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b8042070 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.933+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8030940 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.933+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c4157fe0 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.933+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41585a0 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.934+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c4073200 con 0x7fc5c40720b0 2024-09-10T10:52:18.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.935+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5b8030ae0 con 0x7fc5c40720b0 2024-09-10T10:52:18.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.936+0000 7fc5c1600640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 0x7fc5a406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:18.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.936+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7fc5b80abfa0 con 0x7fc5c40720b0 2024-09-10T10:52:18.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.937+0000 7fc5c8e00640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 0x7fc5a406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:18.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.938+0000 7fc5c8e00640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 0x7fc5a406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40af7b0 tx=0x7fc5b0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:18.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:18.941+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc5b8045360 con 0x7fc5c40720b0 2024-09-10T10:52:19.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.192+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fc5c415faa0 con 0x7fc5c40720b0 2024-09-10T10:52:19.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.759+0000 7fc5c1600640 1 -- 172.21.15.28:0/526951813 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v167) ==== 101+0+0 (secure 0 0 0) 0x7fc5b807a1a0 con 0x7fc5c40720b0 2024-09-10T10:52:19.760 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 msgr2=0x7fc5a406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 0x7fc5a406e2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40af7b0 tx=0x7fc5b0008040 comp rx=0 tx=0).stop 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 msgr2=0x7fc5c4157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8000c00 tx=0x7fc5b80046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 shutdown_connections 2024-09-10T10:52:19.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5a406be30 0x7fc5a406e2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5c4162ad0 0x7fc5c41548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5c4072490 0x7fc5c415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 --2- 172.21.15.28:0/526951813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5c40720b0 0x7fc5c4157440 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 >> 172.21.15.28:0/526951813 conn(0x7fc5c4067d70 msgr2=0x7fc5c40cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 shutdown_connections 2024-09-10T10:52:19.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.762+0000 7fc5cc200640 1 -- 172.21.15.28:0/526951813 wait complete. 2024-09-10T10:52:19.785 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:19.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.862+0000 7f9467e5ff40 1 Processor -- start 2024-09-10T10:52:19.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.862+0000 7f9467e5ff40 1 -- start start 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e14530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6d902d0 0x5616f6eadd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5616f6d8fdc0 con 0x5616f6e1c4b0 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616f6ceeb30 con 0x5616f6d902d0 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5616f6d8f1f0 con 0x5616f6d4acb0 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43498/0 (socket says 172.21.15.28:43498) 2024-09-10T10:52:19.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 -- 172.21.15.28:0/1448764925 learned_addr learned my addr 172.21.15.28:0/1448764925 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f945fe00640 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6d902d0 0x5616f6eadd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465000640 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e14530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 -- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 msgr2=0x5616f6e14530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e14530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 -- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6d902d0 msgr2=0x5616f6eadd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6d902d0 0x5616f6eadd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.863+0000 7f9465a00640 1 -- 172.21.15.28:0/1448764925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f6e1f990 con 0x5616f6e1c4b0 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f9465a00640 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f946000a4b0 tx=0x7f9460050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e2e228f77caeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f945ea00640 1 -- 172.21.15.28:0/1448764925 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460067070 con 0x5616f6e1c4b0 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f945ea00640 1 -- 172.21.15.28:0/1448764925 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f946004fb90 con 0x5616f6e1c4b0 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f945ea00640 1 -- 172.21.15.28:0/1448764925 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460062470 con 0x5616f6e1c4b0 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f9467e5ff40 1 -- 172.21.15.28:0/1448764925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 msgr2=0x5616f6d906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.864+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f946000a4b0 tx=0x7f9460050000 comp rx=0 tx=0).stop 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 -- 172.21.15.28:0/1448764925 shutdown_connections 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6d902d0 0x5616f6eadd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/1448764925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e14530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 -- 172.21.15.28:0/1448764925 >> 172.21.15.28:0/1448764925 conn(0x5616f6d74b90 msgr2=0x5616f6d76d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 -- 172.21.15.28:0/1448764925 shutdown_connections 2024-09-10T10:52:19.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 -- 172.21.15.28:0/1448764925 wait complete. 2024-09-10T10:52:19.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.865+0000 7f9467e5ff40 1 Processor -- start 2024-09-10T10:52:19.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 -- start start 2024-09-10T10:52:19.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5616f6e1fc30 con 0x5616f6d902d0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5616f6e1e980 con 0x5616f6e1c4b0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.866+0000 7f9467e5ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5616f6d752a0 con 0x5616f6d4acb0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43514/0 (socket says 172.21.15.28:43514) 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 learned_addr learned my addr 172.21.15.28:0/2542369016 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f9465a00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 msgr2=0x5616f6e18ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5616f6d4acb0 0x5616f6e18ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6e1c4b0 msgr2=0x5616f6d906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5616f6e1c4b0 0x5616f6d906b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f6e1e350 con 0x5616f6d902d0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f94500086e0 tx=0x7f94500087d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.867+0000 7f945d600640 1 -- 172.21.15.28:0/2542369016 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450019020 con 0x5616f6d902d0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.868+0000 7f945d600640 1 -- 172.21.15.28:0/2542369016 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9450015670 con 0x5616f6d902d0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.868+0000 7f945d600640 1 -- 172.21.15.28:0/2542369016 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450015a20 con 0x5616f6d902d0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.868+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5616f6e16e90 con 0x5616f6d902d0 2024-09-10T10:52:19.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.868+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f6e17450 con 0x5616f6d902d0 2024-09-10T10:52:19.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.869+0000 7f945d600640 1 -- 172.21.15.28:0/2542369016 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f945001c020 con 0x5616f6d902d0 2024-09-10T10:52:19.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.870+0000 7f945d600640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 0x7f944406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.870+0000 7f945d600640 1 -- 172.21.15.28:0/2542369016 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7f945008a7e0 con 0x5616f6d902d0 2024-09-10T10:52:19.875 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.871+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5616f6e15fa0 0x5616f6e14fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.871+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6e154f0 con 0x5616f6e15fa0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.871+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 0x7f944406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.871+0000 7f9465a00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5616f6e15fa0 0x5616f6e14fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.872+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 0x7f944406cd00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9458010410 tx=0x7f9458001790 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.872+0000 7f9465a00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5616f6e15fa0 0x5616f6e14fb0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.874+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v114'412 uv386 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.874+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6e158e0 con 0x5616f6e15fa0 2024-09-10T10:52:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.875+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v114'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.875+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1500 con 0x5616f6e15fa0 2024-09-10T10:52:19.877 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:19.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.877+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v114'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.877+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec28b0 con 0x5616f6e15fa0 2024-09-10T10:52:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.878+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v114'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.878+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec2c50 con 0x5616f6e15fa0 2024-09-10T10:52:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.880+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v114'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.880+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec18a0 con 0x5616f6e15fa0 2024-09-10T10:52:19.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.882+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v114'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.882+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c60 con 0x5616f6e15fa0 2024-09-10T10:52:19.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.884+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v114'418 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.884+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c60 con 0x5616f6e15fa0 2024-09-10T10:52:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.886+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v114'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.886+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c60 con 0x5616f6e15fa0 2024-09-10T10:52:19.888 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.887+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v114'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.887+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c60 con 0x5616f6e15fa0 2024-09-10T10:52:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.888+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v114'421 uv421 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.888+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.890+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v114'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.890+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.892+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v114'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.892+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.894+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v114'424 uv399 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.894+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.896+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v114'425 uv425 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.896+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.897+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v114'426 uv426 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.898 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:19.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.897+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.899+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v114'427 uv427 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.899+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.901+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v114'428 uv428 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.901+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1c40 con 0x5616f6e15fa0 2024-09-10T10:52:19.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.902+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v114'429 uv429 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.903+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec2060 con 0x5616f6e15fa0 2024-09-10T10:52:19.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.904+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v114'430 uv430 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.904+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.906+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v114'431 uv405 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.906+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.907+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v114'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.907+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.909+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v114'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.910 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:19.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.909+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.910+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v114'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.910+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.912+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v114'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.912+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.913+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v114'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.914+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.915+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v114'437 uv411 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.915+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.917+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v114'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.917+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.919 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:19.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.918+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v114'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.918+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.920+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v114'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.920+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.922+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v114'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.922+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6e15fa0 2024-09-10T10:52:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.923+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v114'442 uv442 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6e15fa0 2024-09-10T10:52:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.923+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5616f6ec2ff0 0x5616f6ec53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.923+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec1f00 con 0x5616f6ec2ff0 2024-09-10T10:52:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.924+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5616f6ec2ff0 0x5616f6ec53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.924+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5616f6ec2ff0 0x5616f6ec53e0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.926+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v114'308 uv291 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.926+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec5920 con 0x5616f6ec2ff0 2024-09-10T10:52:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.928+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v114'309 uv309 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.928+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec5cc0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.929+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v114'310 uv310 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.930 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.929+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6060 con 0x5616f6ec2ff0 2024-09-10T10:52:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.930+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v114'311 uv311 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.931+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6e1ee00 con 0x5616f6ec2ff0 2024-09-10T10:52:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.932+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v114'312 uv312 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.932+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec68b0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.933+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v114'313 uv296 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.933+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6c50 con 0x5616f6ec2ff0 2024-09-10T10:52:19.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.935+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v114'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.935+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.936+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v114'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.938 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:19.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.936+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.938+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v114'316 uv316 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.938+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.940+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v114'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.940+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.941+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v114'318 uv301 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.941+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.943+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v114'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.943+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.944+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v114'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.946 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:19.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.944+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecb6e0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.946+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v114'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.946+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecb6e0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.947+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v114'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.947+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.949+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v114'323 uv307 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.949+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.950+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v114'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.950+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.952 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:19.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.952+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v114'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.952+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec8000 con 0x5616f6ec2ff0 2024-09-10T10:52:19.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.953+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v114'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.953+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec8000 con 0x5616f6ec2ff0 2024-09-10T10:52:19.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.954+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v114'327 uv327 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.954+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6ff0 con 0x5616f6ec2ff0 2024-09-10T10:52:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.956+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v114'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ec2ff0 2024-09-10T10:52:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.956+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5616f6ec78f0 0x5616f6ec7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.956+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6eca420 con 0x5616f6ec78f0 2024-09-10T10:52:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.956+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5616f6ec78f0 0x5616f6ec7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.957+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5616f6ec78f0 0x5616f6ec7cd0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.959+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v114'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.959+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6eca7c0 con 0x5616f6ec78f0 2024-09-10T10:52:19.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.960+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v114'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.960+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecab60 con 0x5616f6ec78f0 2024-09-10T10:52:19.963 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:19.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.962+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v114'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.962+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ec6500 con 0x5616f6ec78f0 2024-09-10T10:52:19.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.963+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v114'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.963+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecc4a0 con 0x5616f6ec78f0 2024-09-10T10:52:19.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.966+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v114'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.966+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecb3f0 con 0x5616f6ec78f0 2024-09-10T10:52:19.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.969+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v114'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.969+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5616f6ecbe60 0x5616f6ece8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:19.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.969+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecee40 con 0x5616f6ecbe60 2024-09-10T10:52:19.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.969+0000 7f9465a00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5616f6ecbe60 0x5616f6ece8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:19.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.970+0000 7f9465a00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5616f6ecbe60 0x5616f6ece8e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:19.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.971+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v114'63 uv62 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.971+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf230 con 0x5616f6ecbe60 2024-09-10T10:52:19.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.972+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v114'64 uv64 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.972+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf230 con 0x5616f6ecbe60 2024-09-10T10:52:19.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.973+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v114'65 uv65 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.975 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:19.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.973+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf230 con 0x5616f6ecbe60 2024-09-10T10:52:19.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.974+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v114'66 uv66 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.975+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf650 con 0x5616f6ec78f0 2024-09-10T10:52:19.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.976+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v114'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.976+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf9f0 con 0x5616f6ec78f0 2024-09-10T10:52:19.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.977+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v114'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.977+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:19.979 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:19.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.978+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v114'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.978+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed0da0 con 0x5616f6ec78f0 2024-09-10T10:52:19.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.979+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v114'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.979+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed0da0 con 0x5616f6ec78f0 2024-09-10T10:52:19.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.981+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v114'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.981+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:19.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.982+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v114'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.982+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.983+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v114'149 uv143 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.983+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.985+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v114'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.985+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecf230 con 0x5616f6ecbe60 2024-09-10T10:52:19.987 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:19.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.986+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v114'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.986+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed0da0 con 0x5616f6ecbe60 2024-09-10T10:52:19.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.987+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v114'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.987+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed0da0 con 0x5616f6ecbe60 2024-09-10T10:52:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.988+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v114'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.988+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.990+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v114'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.990+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.991+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v114'155 uv148 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.991+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.992+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v114'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.992+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.994 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.994+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v114'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.994+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.995+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v114'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.995+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ecbe60 2024-09-10T10:52:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.997+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v114'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.997+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.998+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v114'66 uv65 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.998+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.999+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v114'67 uv67 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:19.999+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:20.001 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.000+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v114'68 uv68 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.000+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.001+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v114'69 uv69 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.001+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ecfd90 con 0x5616f6ec78f0 2024-09-10T10:52:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.003+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v114'70 uv70 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ec78f0 2024-09-10T10:52:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.003+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5616f6ed02a0 0x5616f6ed26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.003+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed2d00 con 0x5616f6ed02a0 2024-09-10T10:52:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.003+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5616f6ed02a0 0x5616f6ed26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.004+0000 7f9465000640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5616f6ed02a0 0x5616f6ed26d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.005+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v114'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.005+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed2dc0 con 0x5616f6ed02a0 2024-09-10T10:52:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.006+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v114'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.007+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.008+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v114'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.009 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:20.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.008+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed41f0 con 0x5616f6ed02a0 2024-09-10T10:52:20.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.009+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v114'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.009+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed41f0 con 0x5616f6ed02a0 2024-09-10T10:52:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.010+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v114'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.010+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.011+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v114'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.011+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.013+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v114'132 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.013+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.014+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v114'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.014+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.016+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v114'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.017 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.016+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.017+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v114'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.017+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.019+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v114'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.019+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.020+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v114'137 uv131 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.020+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.021+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v114'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.021+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.022+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v114'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.024 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.023+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed31e0 con 0x5616f6ed02a0 2024-09-10T10:52:20.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.024+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v114'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.024+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed5ff0 con 0x5616f6ed02a0 2024-09-10T10:52:20.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.025+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v114'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.025+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.027+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v114'61 uv60 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.027+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.028+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v114'62 uv62 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.028+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.030 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:20.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.029+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v114'63 uv63 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.029+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.030+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v114'64 uv64 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.030+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.032+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v114'81 uv80 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.032+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.033+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v114'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.033+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.035 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.034+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v114'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.034+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed46a0 con 0x5616f6ecbe60 2024-09-10T10:52:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.035+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v114'84 uv84 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.035+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed46a0 con 0x5616f6ecbe60 2024-09-10T10:52:20.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.036+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v114'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.036+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed3690 con 0x5616f6ecbe60 2024-09-10T10:52:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.037+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v114'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9460062470 con 0x5616f6ecbe60 2024-09-10T10:52:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.037+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5616f6ed3ba0 0x5616f6ed5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.037+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6590 con 0x5616f6ed3ba0 2024-09-10T10:52:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.037+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5616f6ed3ba0 0x5616f6ed5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.038+0000 7f945fe00640 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5616f6ed3ba0 0x5616f6ed5f90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.039+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v114'69 uv68 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ed3ba0 2024-09-10T10:52:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.039+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6590 con 0x5616f6ed3ba0 2024-09-10T10:52:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.040+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v114'70 uv70 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ed3ba0 2024-09-10T10:52:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.040+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6610 con 0x5616f6ed3ba0 2024-09-10T10:52:20.042 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.041+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v114'71 uv71 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ed3ba0 2024-09-10T10:52:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.041+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6610 con 0x5616f6ed3ba0 2024-09-10T10:52:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.043+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v114'72 uv72 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ed3ba0 2024-09-10T10:52:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.043+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6610 con 0x5616f6ed3ba0 2024-09-10T10:52:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.044+0000 7f945fe00640 1 -- 172.21.15.28:0/2542369016 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v114'73 uv73 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f94500549e0 con 0x5616f6ed3ba0 2024-09-10T10:52:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.044+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6610 con 0x5616f6ed02a0 2024-09-10T10:52:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.045+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v114'170 uv170 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.046 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:20.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.046+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a50 con 0x5616f6ed02a0 2024-09-10T10:52:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.047+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [write 0~582] v114'171 uv171 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.047+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a50 con 0x5616f6ed02a0 2024-09-10T10:52:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.048+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [omap-set-header] v114'172 uv172 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.048+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a00 con 0x5616f6ed02a0 2024-09-10T10:52:20.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.049+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 1.00000000 [create] v114'173 uv168 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.049+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a00 con 0x5616f6ed02a0 2024-09-10T10:52:20.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.051+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 1.00000000 [delete] v114'174 uv174 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.051+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a00 con 0x5616f6ed02a0 2024-09-10T10:52:20.052 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:20.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.052+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v114'175 uv175 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.052+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed7a10 con 0x5616f6ed02a0 2024-09-10T10:52:20.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.053+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v114'176 uv176 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.053+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed7a10 con 0x5616f6ed02a0 2024-09-10T10:52:20.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.054+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v114'177 uv177 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.054+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6a00 con 0x5616f6ed02a0 2024-09-10T10:52:20.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.055+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v114'178 uv178 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.055+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e114) -- 0x5616f6ed6eb0 con 0x5616f6ed02a0 2024-09-10T10:52:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.056+0000 7f9465000640 1 -- 172.21.15.28:0/2542369016 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v114'179 uv179 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f945801b040 con 0x5616f6ed02a0 2024-09-10T10:52:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5616f6ec78f0 msgr2=0x5616f6ec7cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5616f6ec78f0 0x5616f6ec7cd0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5616f6ed02a0 msgr2=0x5616f6ed26d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5616f6ed02a0 0x5616f6ed26d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5616f6ed3ba0 msgr2=0x5616f6ed5f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5616f6ed3ba0 0x5616f6ed5f90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5616f6ecbe60 msgr2=0x5616f6ece8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5616f6ecbe60 0x5616f6ece8e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5616f6ec2ff0 msgr2=0x5616f6ec53e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5616f6ec2ff0 0x5616f6ec53e0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5616f6e15fa0 msgr2=0x5616f6e14fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5616f6e15fa0 0x5616f6e14fb0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 reap_dead start 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 reap_dead start 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 reap_dead start 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9465a00640 1 -- 172.21.15.28:0/2542369016 reap_dead start 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 msgr2=0x7f944406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 0x7f944406cd00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9458010410 tx=0x7f9458001790 comp rx=0 tx=0).stop 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 msgr2=0x5616f6e13880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f94500086e0 tx=0x7f94500087d0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 shutdown_connections 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f944406a840 0x7f944406cd00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 --2- 172.21.15.28:0/2542369016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5616f6d902d0 0x5616f6e13880 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 >> 172.21.15.28:0/2542369016 conn(0x5616f6d74b90 msgr2=0x5616f6e1aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 shutdown_connections 2024-09-10T10:52:20.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.057+0000 7f9467e5ff40 1 -- 172.21.15.28:0/2542369016 wait complete. 2024-09-10T10:52:20.062 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 1.00000000 2024-09-10T10:52:20.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.177+0000 7fbcf040af40 1 Processor -- start 2024-09-10T10:52:20.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.177+0000 7fbcf040af40 1 -- start start 2024-09-10T10:52:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb56dcb0 0x558bbb63a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb640480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558bbb5b2dc0 con 0x558bbb63f4b0 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bbb511b30 con 0x558bbb56dcb0 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcf040af40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558bbb5b21f0 con 0x558bbb5b32d0 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbced200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbced200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43526/0 (socket says 172.21.15.28:43526) 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbced200640 1 -- 172.21.15.28:0/2225745478 learned_addr learned my addr 172.21.15.28:0/2225745478 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcec800640 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb56dcb0 0x558bbb63a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.178+0000 7fbcebe00640 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb640480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 -- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 msgr2=0x558bbb640480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb640480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 -- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb56dcb0 msgr2=0x558bbb63a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb56dcb0 0x558bbb63a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 -- 172.21.15.28:0/2225745478 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbb5978b0 con 0x558bbb63f4b0 2024-09-10T10:52:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbced200640 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbce400c440 tx=0x7fbce400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a773c090a5c1be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbceaa00640 1 -- 172.21.15.28:0/2225745478 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40115f0 con 0x558bbb63f4b0 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbceaa00640 1 -- 172.21.15.28:0/2225745478 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce4011790 con 0x558bbb63f4b0 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.179+0000 7fbceaa00640 1 -- 172.21.15.28:0/2225745478 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4011b60 con 0x558bbb63f4b0 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 -- 172.21.15.28:0/2225745478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 msgr2=0x558bbb6409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fbce400c440 tx=0x7fbce400c900 comp rx=0 tx=0).stop 2024-09-10T10:52:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 -- 172.21.15.28:0/2225745478 shutdown_connections 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb63f4b0 0x558bbb6409c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb640480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 --2- 172.21.15.28:0/2225745478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb56dcb0 0x558bbb63a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 -- 172.21.15.28:0/2225745478 >> 172.21.15.28:0/2225745478 conn(0x558bbb597b90 msgr2=0x558bbb597f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 -- 172.21.15.28:0/2225745478 shutdown_connections 2024-09-10T10:52:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.180+0000 7fbcf040af40 1 -- 172.21.15.28:0/2225745478 wait complete. 2024-09-10T10:52:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.181+0000 7fbcf040af40 1 Processor -- start 2024-09-10T10:52:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.181+0000 7fbcf040af40 1 -- start start 2024-09-10T10:52:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 0x558bbb63b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb595a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb63f4b0 0x558bbb595f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558bbb63e1a0 con 0x558bbb56dcb0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558bbb597660 con 0x558bbb63f4b0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcf040af40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558bbb59ac20 con 0x558bbb5b32d0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcec800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 0x558bbb63b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcec800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 0x558bbb63b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43528/0 (socket says 172.21.15.28:43528) 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcec800640 1 -- 172.21.15.28:0/644426867 learned_addr learned my addr 172.21.15.28:0/644426867 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbcebe00640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb595a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.182+0000 7fbced200640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb63f4b0 0x558bbb595f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 -- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 msgr2=0x558bbb595a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558bbb5b32d0 0x558bbb595a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 -- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb63f4b0 msgr2=0x558bbb595f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb63f4b0 0x558bbb595f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 -- 172.21.15.28:0/644426867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbb6405b0 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbced200640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558bbb63f4b0 0x558bbb595f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcec800640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 0x558bbb63b030 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c610 tx=0x7fbce000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbce9600640 1 -- 172.21.15.28:0/644426867 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000f650 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbce9600640 1 -- 172.21.15.28:0/644426867 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbce000f7f0 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbce9600640 1 -- 172.21.15.28:0/644426867 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce000fbc0 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558bbb640890 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.183+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbb637740 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.185+0000 7fbce9600640 1 -- 172.21.15.28:0/644426867 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbce0009390 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.186+0000 7fbce9600640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06a810 0x7fbccc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.186+0000 7fbce9600640 1 -- 172.21.15.28:0/644426867 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7fbce008ac00 con 0x558bbb56dcb0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.187+0000 7fbcebe00640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06a810 0x7fbccc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.187+0000 7fbcf040af40 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558bbb642870 0x558bbb56aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.187+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e114) -- 0x558bbb56b530 con 0x558bbb642870 2024-09-10T10:52:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.187+0000 7fbced200640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558bbb642870 0x558bbb56aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.187+0000 7fbcebe00640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06a810 0x7fbccc06ccd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8057000 tx=0x7fbcd8052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.188+0000 7fbced200640 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558bbb642870 0x558bbb56aff0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbced200640 1 -- 172.21.15.28:0/644426867 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [delete] v114'180 uv180 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbce4011b80 con 0x558bbb642870 2024-09-10T10:52:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558bbb642870 msgr2=0x558bbb56aff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558bbb642870 0x558bbb56aff0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06a810 msgr2=0x7fbccc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbccc06a810 0x7fbccc06ccd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8057000 tx=0x7fbcd8052000 comp rx=0 tx=0).stop 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 msgr2=0x558bbb63b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 --2- 172.21.15.28:0/644426867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558bbb56dcb0 0x558bbb63b030 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fbce000c610 tx=0x7fbce000ca10 comp rx=0 tx=0).stop 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbced200640 1 -- 172.21.15.28:0/644426867 reap_dead start 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 shutdown_connections 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 >> 172.21.15.28:0/644426867 conn(0x558bbb597b90 msgr2=0x558bbb597f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 shutdown_connections 2024-09-10T10:52:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.189+0000 7fbcf040af40 1 -- 172.21.15.28:0/644426867 wait complete. 2024-09-10T10:52:20.197 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.355+0000 7efd7e400640 1 Processor -- start 2024-09-10T10:52:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.355+0000 7efd7e400640 1 -- start start 2024-09-10T10:52:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7e400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efd780d2ac0 0x7efd7806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7e400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780b1170 con 0x7efd78161ac0 2024-09-10T10:52:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7e400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780b1450 con 0x7efd780b3640 2024-09-10T10:52:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7e400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efd780ceba0 con 0x7efd780d2ac0 2024-09-10T10:52:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efd78161ac0 0x7efd780d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58806/0 (socket says 172.21.15.28:58806) 2024-09-10T10:52:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.356+0000 7efd7d000640 1 -- 172.21.15.28:0/372139575 learned_addr learned my addr 172.21.15.28:0/372139575 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77e00640 1 --2- 172.21.15.28:0/372139575 >> v2:172.21.15.138:3300/0 conn(0x7efd780d2ac0 0x7efd7806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77400640 1 -- 172.21.15.28:0/372139575 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240613989 0 0) 0x7efd780b1170 con 0x7efd78161ac0 2024-09-10T10:52:20.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77400640 1 -- 172.21.15.28:0/372139575 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd60003520 con 0x7efd78161ac0 2024-09-10T10:52:20.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77e00640 1 -- 172.21.15.28:0/372139575 >> v1:172.21.15.138:6789/0 conn(0x7efd780b3640 legacy=0x7efd780b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77e00640 1 -- 172.21.15.28:0/372139575 >> v1:172.21.15.28:6789/0 conn(0x7efd78161ac0 legacy=0x7efd780d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.357+0000 7efd77e00640 1 -- 172.21.15.28:0/372139575 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780b1730 con 0x7efd780d2ac0 2024-09-10T10:52:20.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.358+0000 7efd77e00640 1 --2- 172.21.15.28:0/372139575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd780d2ac0 0x7efd7806e6c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ba2efbe1df943 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.358+0000 7efd77400640 1 -- 172.21.15.28:0/372139575 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64037070 con 0x7efd780d2ac0 2024-09-10T10:52:20.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.358+0000 7efd77400640 1 -- 172.21.15.28:0/372139575 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd64032640 con 0x7efd780d2ac0 2024-09-10T10:52:20.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.358+0000 7efd77400640 1 -- 172.21.15.28:0/372139575 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd640329f0 con 0x7efd780d2ac0 2024-09-10T10:52:20.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 -- 172.21.15.28:0/372139575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd780d2ac0 msgr2=0x7efd7806e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 --2- 172.21.15.28:0/372139575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd780d2ac0 0x7efd7806e6c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd6402a2c0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 -- 172.21.15.28:0/372139575 shutdown_connections 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 --2- 172.21.15.28:0/372139575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd780d2ac0 0x7efd7806e6c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 -- 172.21.15.28:0/372139575 >> 172.21.15.28:0/372139575 conn(0x7efd7808bfc0 msgr2=0x7efd780d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 -- 172.21.15.28:0/372139575 shutdown_connections 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.359+0000 7efd7e400640 1 -- 172.21.15.28:0/372139575 wait complete. 2024-09-10T10:52:20.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.361+0000 7efd7e400640 1 Processor -- start 2024-09-10T10:52:20.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.361+0000 7efd7e400640 1 -- start start 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 0x7efd780cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 0x7efd781f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efd78161ea0 con 0x7efd780d2ac0 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd7806dfb0 con 0x7efd78161ac0 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7e400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd78162020 con 0x7efd780b3640 2024-09-10T10:52:20.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd77e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43542/0 (socket says 172.21.15.28:43542) 2024-09-10T10:52:20.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd77e00640 1 -- 172.21.15.28:0/2097697215 learned_addr learned my addr 172.21.15.28:0/2097697215 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:20.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7da00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 0x7efd781f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.362+0000 7efd7d000640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 0x7efd780cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 -- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 msgr2=0x7efd780cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 0x7efd780cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 -- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 msgr2=0x7efd781f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 0x7efd781f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 -- 172.21.15.28:0/2097697215 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780ce5f0 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd7da00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 0x7efd781f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd7d000640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 0x7efd780cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd77e00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64040020 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd64038040 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd64037070 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd780ce310 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.363+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd780b5610 con 0x7efd780d2ac0 2024-09-10T10:52:20.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.364+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7806e6c0 con 0x7efd780d2ac0 2024-09-10T10:52:20.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.365+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd64043020 con 0x7efd780d2ac0 2024-09-10T10:52:20.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.367+0000 7efd74c00640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 0x7efd5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.367+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7efd640ab940 con 0x7efd780d2ac0 2024-09-10T10:52:20.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.367+0000 7efd7d000640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 0x7efd5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.368+0000 7efd7d000640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 0x7efd5806e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0011e0 tx=0x7efd6c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.371+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd6404a030 con 0x7efd780d2ac0 2024-09-10T10:52:20.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.595+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7efd780b2fa0 con 0x7efd780d2ac0 2024-09-10T10:52:20.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.765+0000 7efd74c00640 1 -- 172.21.15.28:0/2097697215 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v168) ==== 162+0+0 (secure 0 0 0) 0x7efd64079bd0 con 0x7efd780d2ac0 2024-09-10T10:52:20.767 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 msgr2=0x7efd5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 0x7efd5806e2f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0011e0 tx=0x7efd6c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 msgr2=0x7efd780cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7efd640097a0 tx=0x7efd640046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 shutdown_connections 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efd5806be30 0x7efd5806e2f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd78161ac0 0x7efd781f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd780d2ac0 0x7efd780cdbe0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 --2- 172.21.15.28:0/2097697215 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd780b3640 0x7efd780cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 >> 172.21.15.28:0/2097697215 conn(0x7efd7808bfc0 msgr2=0x7efd780ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 shutdown_connections 2024-09-10T10:52:20.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.768+0000 7efd7e400640 1 -- 172.21.15.28:0/2097697215 wait complete. 2024-09-10T10:52:20.787 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.943+0000 7f9f1e000640 1 Processor -- start 2024-09-10T10:52:20.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.943+0000 7f9f1e000640 1 -- start start 2024-09-10T10:52:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.944+0000 7f9f1e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f18072320 0x7f9f180cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.944+0000 7f9f1e000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f18146140 con 0x7f9f1815e1f0 2024-09-10T10:52:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.944+0000 7f9f1e000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1806eed0 con 0x7f9f18072de0 2024-09-10T10:52:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.944+0000 7f9f1e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f180cd740 con 0x7f9f18072320 2024-09-10T10:52:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9f1815e1f0 0x7f9f1806e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58822/0 (socket says 172.21.15.28:58822) 2024-09-10T10:52:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17e00640 1 -- 172.21.15.28:0/3341748239 learned_addr learned my addr 172.21.15.28:0/3341748239 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:20.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17400640 1 --2- 172.21.15.28:0/3341748239 >> v2:172.21.15.138:3300/0 conn(0x7f9f18072320 0x7f9f180cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1732564128 0 0) 0x7f9f18146140 con 0x7f9f1815e1f0 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9efc003520 con 0x7f9f1815e1f0 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2295506063 0 0) 0x7f9f1806eed0 con 0x7f9f18072de0 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17400640 1 -- 172.21.15.28:0/3341748239 >> v1:172.21.15.138:6789/0 conn(0x7f9f18072de0 legacy=0x7f9f1806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17400640 1 -- 172.21.15.28:0/3341748239 >> v1:172.21.15.28:6789/0 conn(0x7f9f1815e1f0 legacy=0x7f9f1806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.945+0000 7f9f17400640 1 -- 172.21.15.28:0/3341748239 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f180cda20 con 0x7f9f18072320 2024-09-10T10:52:20.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.946+0000 7f9f17400640 1 --2- 172.21.15.28:0/3341748239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f18072320 0x7f9f180cce60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097a0 tx=0x7f9f0802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b5bf8ac58ff7c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.947+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08037070 con 0x7f9f18072320 2024-09-10T10:52:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.947+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f08032440 con 0x7f9f18072320 2024-09-10T10:52:20.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.947+0000 7f9f16000640 1 -- 172.21.15.28:0/3341748239 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08032810 con 0x7f9f18072320 2024-09-10T10:52:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 -- 172.21.15.28:0/3341748239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f18072320 msgr2=0x7f9f180cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 --2- 172.21.15.28:0/3341748239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f18072320 0x7f9f180cce60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097a0 tx=0x7f9f0802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 -- 172.21.15.28:0/3341748239 shutdown_connections 2024-09-10T10:52:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 --2- 172.21.15.28:0/3341748239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f18072320 0x7f9f180cce60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 -- 172.21.15.28:0/3341748239 >> 172.21.15.28:0/3341748239 conn(0x7f9f18067d70 msgr2=0x7f9f1806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 -- 172.21.15.28:0/3341748239 shutdown_connections 2024-09-10T10:52:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.948+0000 7f9f1e000640 1 -- 172.21.15.28:0/3341748239 wait complete. 2024-09-10T10:52:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.949+0000 7f9f1e000640 1 Processor -- start 2024-09-10T10:52:20.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.950+0000 7f9f1e000640 1 -- start start 2024-09-10T10:52:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.950+0000 7f9f1e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f1e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f18072de0 0x7f9f1813dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f1e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1815e1f0 0x7f9f1813e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f1e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180cd3a0 con 0x7f9f18072320 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f1e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1815c9e0 con 0x7f9f1815e1f0 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f1e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f180cd520 con 0x7f9f18072de0 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43556/0 (socket says 172.21.15.28:43556) 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 -- 172.21.15.28:0/1197144697 learned_addr learned my addr 172.21.15.28:0/1197144697 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17e00640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1815e1f0 0x7f9f1813e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f16a00640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f18072de0 0x7f9f1813dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 -- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f18072de0 msgr2=0x7f9f1813dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f18072de0 0x7f9f1813dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 -- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1815e1f0 msgr2=0x7f9f1813e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1815e1f0 0x7f9f1813e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.951+0000 7f9f17400640 1 -- 172.21.15.28:0/1197144697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f18140dc0 con 0x7f9f18072320 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f17400640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098c0 tx=0x7f9f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08037050 con 0x7f9f18072320 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f08038040 con 0x7f9f18072320 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f181410a0 con 0x7f9f18072320 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004430 con 0x7f9f18072320 2024-09-10T10:52:20.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.952+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f18141660 con 0x7f9f18072320 2024-09-10T10:52:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.953+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f18072700 con 0x7f9f18072320 2024-09-10T10:52:20.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.954+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f0803c6d0 con 0x7f9f18072320 2024-09-10T10:52:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.955+0000 7f9f0f400640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.955+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 7508+0+0 (secure 0 0 0) 0x7f9f080ac120 con 0x7f9f18072320 2024-09-10T10:52:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.955+0000 7f9f16a00640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:20.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.956+0000 7f9f16a00640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 0x7f9ef806e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f180b2c00 tx=0x7f9f04009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:20.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:20.959+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f0807a440 con 0x7f9f18072320 2024-09-10T10:52:21.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.179+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1815c5e0 con 0x7f9f18072320 2024-09-10T10:52:21.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.179+0000 7f9f0f400640 1 -- 172.21.15.28:0/1197144697 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) ==== 95+0+4401 (secure 0 0 0) 0x7f9f08079d40 con 0x7f9f18072320 2024-09-10T10:52:21.180 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:21.181 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":169,"btime":"2024-09-10T10:52:20:768171+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:20.768159+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8304":{"gid":8304,"name":"c","rank":0,"incarnation":169,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/1314136636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1314136636},{"type":"v1","addr":"172.21.15.28:6834","nonce":1314136636}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:21.181 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 169 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 msgr2=0x7f9ef806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 0x7f9ef806e2f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f180b2c00 tx=0x7f9f04009040 comp rx=0 tx=0).stop 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 msgr2=0x7f9f1815b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9f080098c0 tx=0x7f9f080046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 shutdown_connections 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9ef806be30 0x7f9ef806e2f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1815e1f0 0x7f9f1813e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f18072de0 0x7f9f1813dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 --2- 172.21.15.28:0/1197144697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f18072320 0x7f9f1815b690 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:21.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 >> 172.21.15.28:0/1197144697 conn(0x7f9f18067d70 msgr2=0x7f9f180b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:21.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 shutdown_connections 2024-09-10T10:52:21.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:21.182+0000 7f9f1e000640 1 -- 172.21.15.28:0/1197144697 wait complete. 2024-09-10T10:52:21.198 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:52:21.446 INFO:tasks.ceph.mds.a.smithi028.stderr: -45> 2024-09-10T10:52:16.693+0000 7f72c3e00640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T10:52:21.446 INFO:tasks.ceph.mds.a.smithi028.stderr: -36> 2024-09-10T10:52:16.693+0000 7f72c3e00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:52:21.486 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:52:23.789 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:23.788+0000 7f3053600640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-10T10:52:23.789 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:23.788+0000 7f3053600640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T10:52:23.790 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:23.789+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:23.790 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:23.789+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:25.717 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:25.716+0000 7f3058600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:26.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:26.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.366+0000 7f8a29200640 1 Processor -- start 2024-09-10T10:52:26.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.366+0000 7f8a29200640 1 -- start start 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a29200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8a24072d30 0x7f8a2406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a29200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a2415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a29200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a240cdbe0 con 0x7f8a2415b300 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a29200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8a240728a0 con 0x7f8a24072d30 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a29200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2406fb30 con 0x7f8a2406f700 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a21e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8a2415b300 0x7f8a240cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41728/0 (socket says 172.21.15.28:41728) 2024-09-10T10:52:26.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.367+0000 7f8a21e00640 1 -- 172.21.15.28:0/1958165101 learned_addr learned my addr 172.21.15.28:0/1958165101 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:26.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a19400640 1 --2- 172.21.15.28:0/1958165101 >> v2:172.21.15.138:3300/0 conn(0x7f8a24072d30 0x7f8a2406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a22800640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a2415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734919540 0 0) 0x7f8a240cdbe0 con 0x7f8a2415b300 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a08005730 con 0x7f8a2415b300 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1997578144 0 0) 0x7f8a08005730 con 0x7f8a2415b300 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a19400640 1 -- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 msgr2=0x7f8a2415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a19400640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a2415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a19400640 1 -- 172.21.15.28:0/1958165101 >> v1:172.21.15.28:6789/0 conn(0x7f8a2415b300 legacy=0x7f8a240cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a19400640 1 -- 172.21.15.28:0/1958165101 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240cd900 con 0x7f8a24072d30 2024-09-10T10:52:26.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.368+0000 7f8a22800640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a2415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:26.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.369+0000 7f8a19400640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a2406f1c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0097a0 tx=0x7f8a0c02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d126f37c63de7e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.369+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c037070 con 0x7f8a24072d30 2024-09-10T10:52:26.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.369+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c02ab10 con 0x7f8a24072d30 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.369+0000 7f8a21400640 1 -- 172.21.15.28:0/1958165101 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14001690 con 0x7f8a24072d30 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 -- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 msgr2=0x7f8a2406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a2406f1c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0097a0 tx=0x7f8a0c02a080 comp rx=0 tx=0).stop 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 -- 172.21.15.28:0/1958165101 shutdown_connections 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a2415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 --2- 172.21.15.28:0/1958165101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a2406f1c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 -- 172.21.15.28:0/1958165101 >> 172.21.15.28:0/1958165101 conn(0x7f8a24067d70 msgr2=0x7f8a2406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 -- 172.21.15.28:0/1958165101 shutdown_connections 2024-09-10T10:52:26.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.370+0000 7f8a29200640 1 -- 172.21.15.28:0/1958165101 wait complete. 2024-09-10T10:52:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.372+0000 7f8a29200640 1 Processor -- start 2024-09-10T10:52:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.372+0000 7f8a29200640 1 -- start start 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a24149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a240a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240cd5c0 con 0x7f8a2415b300 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a24072530 con 0x7f8a24072d30 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a29200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8a240cffe0 con 0x7f8a2406f700 2024-09-10T10:52:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a22800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a21e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a24149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a19400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a240a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a22800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37606/0 (socket says 172.21.15.28:37606) 2024-09-10T10:52:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.373+0000 7f8a22800640 1 -- 172.21.15.28:0/1413880922 learned_addr learned my addr 172.21.15.28:0/1413880922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 -- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 msgr2=0x7f8a24149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a24149ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 -- 172.21.15.28:0/1413880922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 msgr2=0x7f8a240ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 -- 172.21.15.28:0/1413880922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240a9ee0 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a21e00640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a24149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a22800640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a19400640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a240a9490 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c031000 tx=0x7f8a0c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c041020 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0c03d620 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c037040 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a240aa1c0 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.374+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a24159480 con 0x7f8a24072d30 2024-09-10T10:52:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.375+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a240aa390 con 0x7f8a24072d30 2024-09-10T10:52:26.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.376+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8a0c044020 con 0x7f8a24072d30 2024-09-10T10:52:26.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.377+0000 7f8a1aa00640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 0x7f8a0006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.377+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f8a0c0abb00 con 0x7f8a24072d30 2024-09-10T10:52:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.378+0000 7f8a21e00640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 0x7f8a0006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.378+0000 7f8a21e00640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 0x7f8a0006e320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000f60 tx=0x7f8a14000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.381+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8a0c04b030 con 0x7f8a24072d30 2024-09-10T10:52:26.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.602+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a2415be40 con 0x7f8a24072d30 2024-09-10T10:52:26.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.603+0000 7f8a1aa00640 1 -- 172.21.15.28:0/1413880922 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) ==== 95+0+4389 (secure 0 0 0) 0x7f8a0c079800 con 0x7f8a24072d30 2024-09-10T10:52:26.604 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:26.604 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":173,"btime":"2024-09-10T10:52:23:789844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:23.789841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:26.605 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 173 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.605+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 msgr2=0x7f8a0006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.605+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 0x7f8a0006e320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000f60 tx=0x7f8a14000ed0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.605+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 msgr2=0x7f8a240a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.605+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a240a9490 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c031000 tx=0x7f8a0c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 shutdown_connections 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8a0006be60 0x7f8a0006e320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a2415b300 0x7f8a240ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a24072d30 0x7f8a240a9490 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 --2- 172.21.15.28:0/1413880922 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a2406f700 0x7f8a24149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 >> 172.21.15.28:0/1413880922 conn(0x7f8a24067d70 msgr2=0x7f8a2415d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:26.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 shutdown_connections 2024-09-10T10:52:26.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.606+0000 7f8a29200640 1 -- 172.21.15.28:0/1413880922 wait complete. 2024-09-10T10:52:26.623 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:52:26.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.780+0000 7fc623e00640 1 Processor -- start 2024-09-10T10:52:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.780+0000 7fc623e00640 1 -- start start 2024-09-10T10:52:26.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.781+0000 7fc623e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 0x7fc61c1690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.781+0000 7fc623e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c06f480 con 0x7fc61c147e90 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.781+0000 7fc623e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc61c06e900 con 0x7fc61c147230 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.781+0000 7fc623e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c0b34d0 con 0x7fc61c0727c0 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc620a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc61c147e90 0x7fc61c0720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41740/0 (socket says 172.21.15.28:41740) 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc620a00640 1 -- 172.21.15.28:0/374011737 learned_addr learned my addr 172.21.15.28:0/374011737 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc621e00640 1 --2- 172.21.15.28:0/374011737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 0x7fc61c1690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc61be00640 1 -- 172.21.15.28:0/374011737 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646607351 0 0) 0x7fc61c06f480 con 0x7fc61c147e90 2024-09-10T10:52:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc61be00640 1 -- 172.21.15.28:0/374011737 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc604003520 con 0x7fc61c147e90 2024-09-10T10:52:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc621e00640 1 -- 172.21.15.28:0/374011737 >> v1:172.21.15.138:6789/0 conn(0x7fc61c147230 legacy=0x7fc61c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc621e00640 1 -- 172.21.15.28:0/374011737 >> v1:172.21.15.28:6789/0 conn(0x7fc61c147e90 legacy=0x7fc61c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.782+0000 7fc621e00640 1 -- 172.21.15.28:0/374011737 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c06ebe0 con 0x7fc61c0727c0 2024-09-10T10:52:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.783+0000 7fc621e00640 1 --2- 172.21.15.28:0/374011737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 0x7fc61c1690f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc614009dc0 tx=0x7fc61402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d5bf7e48cae044 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc61be00640 1 -- 172.21.15.28:0/374011737 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614007740 con 0x7fc61c0727c0 2024-09-10T10:52:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc61be00640 1 -- 172.21.15.28:0/374011737 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6140078e0 con 0x7fc61c0727c0 2024-09-10T10:52:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc61be00640 1 -- 172.21.15.28:0/374011737 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614007cb0 con 0x7fc61c0727c0 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 -- 172.21.15.28:0/374011737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 msgr2=0x7fc61c1690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 --2- 172.21.15.28:0/374011737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 0x7fc61c1690f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc614009dc0 tx=0x7fc61402a6d0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 -- 172.21.15.28:0/374011737 shutdown_connections 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 --2- 172.21.15.28:0/374011737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c0727c0 0x7fc61c1690f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 -- 172.21.15.28:0/374011737 >> 172.21.15.28:0/374011737 conn(0x7fc61c067d70 msgr2=0x7fc61c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 -- 172.21.15.28:0/374011737 shutdown_connections 2024-09-10T10:52:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.784+0000 7fc623e00640 1 -- 172.21.15.28:0/374011737 wait complete. 2024-09-10T10:52:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.786+0000 7fc623e00640 1 Processor -- start 2024-09-10T10:52:26.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.786+0000 7fc623e00640 1 -- start start 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c147230 0x7fc61c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc61c147e90 0x7fc61c13ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c06f1a0 con 0x7fc61c0727c0 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c0afe20 con 0x7fc61c147e90 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc623e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c147610 con 0x7fc61c147230 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc621400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc621400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37610/0 (socket says 172.21.15.28:37610) 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc621400640 1 -- 172.21.15.28:0/2759524464 learned_addr learned my addr 172.21.15.28:0/2759524464 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc621e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc61c147e90 0x7fc61c13ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.787+0000 7fc620a00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c147230 0x7fc61c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 -- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c147230 msgr2=0x7fc61c0b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c147230 0x7fc61c0b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 -- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc61c147e90 msgr2=0x7fc61c13ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc61c147e90 0x7fc61c13ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 -- 172.21.15.28:0/2759524464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c0b11e0 con 0x7fc61c0727c0 2024-09-10T10:52:26.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc621400640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c400 tx=0x7fc61000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100052d0 con 0x7fc61c0727c0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc610005490 con 0x7fc61c0727c0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610007260 con 0x7fc61c0727c0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.788+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c0b14c0 con 0x7fc61c0727c0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.789+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c0b1a80 con 0x7fc61c0727c0 2024-09-10T10:52:26.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.789+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c06e0b0 con 0x7fc61c0727c0 2024-09-10T10:52:26.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.791+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc610007400 con 0x7fc61c0727c0 2024-09-10T10:52:26.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.791+0000 7fc619600640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:26.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.791+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7fc61008abd0 con 0x7fc61c0727c0 2024-09-10T10:52:26.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.792+0000 7fc620a00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:26.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.792+0000 7fc620a00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc61c1f7680 tx=0x7fc608007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:26.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:26.796+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc6100586a0 con 0x7fc61c0727c0 2024-09-10T10:52:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.020+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc61c0b5090 con 0x7fc61c0727c0 2024-09-10T10:52:27.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.020+0000 7fc619600640 1 -- 172.21.15.28:0/2759524464 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) ==== 95+0+4389 (secure 0 0 0) 0x7fc61005c9a0 con 0x7fc61c0727c0 2024-09-10T10:52:27.022 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:27.022 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":173,"btime":"2024-09-10T10:52:23:789844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:23.789841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:27.022 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 173 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 msgr2=0x7fc5fc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc61c1f7680 tx=0x7fc608007d30 comp rx=0 tx=0).stop 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 msgr2=0x7fc61c13d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c400 tx=0x7fc61000c8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 shutdown_connections 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc5fc06be30 0x7fc5fc06e2f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc61c147e90 0x7fc61c13ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc61c147230 0x7fc61c0b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 --2- 172.21.15.28:0/2759524464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc61c0727c0 0x7fc61c13d8c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 >> 172.21.15.28:0/2759524464 conn(0x7fc61c067d70 msgr2=0x7fc61c0b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:27.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 shutdown_connections 2024-09-10T10:52:27.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.023+0000 7fc623e00640 1 -- 172.21.15.28:0/2759524464 wait complete. 2024-09-10T10:52:27.040 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:27.041 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:27.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.197+0000 7f8be1600640 1 Processor -- start 2024-09-10T10:52:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.197+0000 7f8be1600640 1 -- start start 2024-09-10T10:52:27.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be1600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8bdc15e6a0 0x7f8bdc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be1600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc0d2c20 con 0x7f8bdc06dd60 2024-09-10T10:52:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be1600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc06e390 con 0x7f8bdc0d4180 2024-09-10T10:52:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be1600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0cefb0 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be0c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8bdc06dd60 0x7f8bdc0cf440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41744/0 (socket says 172.21.15.28:41744) 2024-09-10T10:52:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8be0c00640 1 -- 172.21.15.28:0/1416942524 learned_addr learned my addr 172.21.15.28:0/1416942524 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8bd3400640 1 --2- 172.21.15.28:0/1416942524 >> v2:172.21.15.138:3300/0 conn(0x7f8bdc15e6a0 0x7f8bdc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094947313 0 0) 0x7f8bdc0d2c20 con 0x7f8bdc06dd60 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.199+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bc0003520 con 0x7f8bdc06dd60 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.200+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525716625 0 0) 0x7f8bdc06e390 con 0x7f8bdc0d4180 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.200+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bdc0d2c20 con 0x7f8bdc0d4180 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.200+0000 7f8bd3400640 1 -- 172.21.15.28:0/1416942524 >> v1:172.21.15.138:6789/0 conn(0x7f8bdc0d4180 legacy=0x7f8bdc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.200+0000 7f8bd3400640 1 -- 172.21.15.28:0/1416942524 >> v1:172.21.15.28:6789/0 conn(0x7f8bdc06dd60 legacy=0x7f8bdc0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.200+0000 7f8bd3400640 1 -- 172.21.15.28:0/1416942524 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc06e670 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.201+0000 7f8bd3400640 1 --2- 172.21.15.28:0/1416942524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc15e6a0 0x7f8bdc06d820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40097a0 tx=0x7f8bc402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09575bc6675ee40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.201+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4037070 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.201+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4032440 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.201+0000 7f8bdb400640 1 -- 172.21.15.28:0/1416942524 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40327f0 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 -- 172.21.15.28:0/1416942524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc15e6a0 msgr2=0x7f8bdc06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 --2- 172.21.15.28:0/1416942524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc15e6a0 0x7f8bdc06d820 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40097a0 tx=0x7f8bc402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 -- 172.21.15.28:0/1416942524 shutdown_connections 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 --2- 172.21.15.28:0/1416942524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc15e6a0 0x7f8bdc06d820 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 -- 172.21.15.28:0/1416942524 >> 172.21.15.28:0/1416942524 conn(0x7f8bdc09d6f0 msgr2=0x7f8bdc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 -- 172.21.15.28:0/1416942524 shutdown_connections 2024-09-10T10:52:27.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.202+0000 7f8be1600640 1 -- 172.21.15.28:0/1416942524 wait complete. 2024-09-10T10:52:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.203+0000 7f8be1600640 1 Processor -- start 2024-09-10T10:52:27.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.203+0000 7f8be1600640 1 -- start start 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 0x7f8bdc1f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bdc15e6a0 0x7f8bdc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0cfb50 con 0x7f8bdc0d4180 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc0cebd0 con 0x7f8bdc06dd60 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8be1600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc0cfcd0 con 0x7f8bdc15e6a0 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.204+0000 7f8bd3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37612/0 (socket says 172.21.15.28:37612) 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 -- 172.21.15.28:0/3341472560 learned_addr learned my addr 172.21.15.28:0/3341472560 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8be0c00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bdc15e6a0 0x7f8bdc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bdbe00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 0x7f8bdc1f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 -- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bdc15e6a0 msgr2=0x7f8bdc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bdc15e6a0 0x7f8bdc1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 -- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 msgr2=0x7f8bdc1f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 0x7f8bdc1f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 -- 172.21.15.28:0/3341472560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc1f4d50 con 0x7f8bdc0d4180 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bdbe00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 0x7f8bdc1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:27.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd3400640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4035040 tx=0x7f8bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.205+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4037040 con 0x7f8bdc0d4180 2024-09-10T10:52:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.206+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4042070 con 0x7f8bdc0d4180 2024-09-10T10:52:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.206+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4032830 con 0x7f8bdc0d4180 2024-09-10T10:52:27.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.206+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc0b3b50 con 0x7f8bdc0d4180 2024-09-10T10:52:27.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.206+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc0b4110 con 0x7f8bdc0d4180 2024-09-10T10:52:27.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.207+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc06d180 con 0x7f8bdc0d4180 2024-09-10T10:52:27.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.208+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8bc40329d0 con 0x7f8bdc0d4180 2024-09-10T10:52:27.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.209+0000 7f8bd8c00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 0x7f8bb806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.209+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f8bc40abf90 con 0x7f8bdc0d4180 2024-09-10T10:52:27.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.209+0000 7f8bdbe00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 0x7f8bb806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.210+0000 7f8bdbe00640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 0x7f8bb806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0016b0 tx=0x7f8bcc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.213+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8bdc06d180 con 0x7f8bdc0d4180 2024-09-10T10:52:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.436+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bdc06d820 con 0x7f8bdc0d4180 2024-09-10T10:52:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.436+0000 7f8bd8c00640 1 -- 172.21.15.28:0/3341472560 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) ==== 95+0+4389 (secure 0 0 0) 0x7f8bc407a2f0 con 0x7f8bdc0d4180 2024-09-10T10:52:27.437 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:27.438 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":173,"btime":"2024-09-10T10:52:23:789844+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:23.789841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":115,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 173 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 msgr2=0x7f8bb806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 0x7f8bb806e2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0016b0 tx=0x7f8bcc009040 comp rx=0 tx=0).stop 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 msgr2=0x7f8bdc157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4035040 tx=0x7f8bc40046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 shutdown_connections 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8bb806be30 0x7f8bb806e2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8bdc15e6a0 0x7f8bdc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8bdc0d4180 0x7f8bdc157ca0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 --2- 172.21.15.28:0/3341472560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bdc06dd60 0x7f8bdc1f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 >> 172.21.15.28:0/3341472560 conn(0x7f8bdc09d6f0 msgr2=0x7f8bdc0b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 shutdown_connections 2024-09-10T10:52:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.439+0000 7f8be1600640 1 -- 172.21.15.28:0/3341472560 wait complete. 2024-09-10T10:52:27.457 INFO:tasks.cephfs.test_damage:Result: Mutation 'Delete 1.00000000' led to DAMAGED state 2024-09-10T10:52:27.457 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 1.00000000.inode' 2024-09-10T10:52:27.457 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:27.457 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:27.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:27.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:27.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:27.493 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:27.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:27.568 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:27.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.765+0000 7f7f99a00640 1 Processor -- start 2024-09-10T10:52:27.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.765+0000 7f7f99a00640 1 -- start start 2024-09-10T10:52:27.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.766+0000 7f7f99a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7f940d0dd0 0x7f7f940d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.766+0000 7f7f99a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415e660 0x7f7f9406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.766+0000 7f7f99a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f940cf000 con 0x7f7f940d1f90 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.766+0000 7f7f99a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7f940b37d0 con 0x7f7f940d0dd0 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.766+0000 7f7f99a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f940cf330 con 0x7f7f9415e660 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7f940d0dd0 0x7f7f940d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7f940d0dd0 0x7f7f940d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37620/0 (socket says 172.21.15.28:37620) 2024-09-10T10:52:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 -- 172.21.15.28:0/2854351461 learned_addr learned my addr 172.21.15.28:0/2854351461 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 -- 172.21.15.28:0/2854351461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415e660 msgr2=0x7f7f9406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 --2- 172.21.15.28:0/2854351461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415e660 0x7f7f9406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 -- 172.21.15.28:0/2854351461 >> v1:172.21.15.138:6789/0 conn(0x7f7f940d1f90 legacy=0x7f7f940cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.767+0000 7f7f93e00640 1 -- 172.21.15.28:0/2854351461 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f940d16f0 con 0x7f7f940d0dd0 2024-09-10T10:52:27.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.768+0000 7f7f93e00640 1 --2- 172.21.15.28:0/2854351461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d0dd0 0x7f7f940d11b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009870 tx=0x7f7f8402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77adc37d03453c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.768+0000 7f7f93400640 1 -- 172.21.15.28:0/2854351461 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84037070 con 0x7f7f940d0dd0 2024-09-10T10:52:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.768+0000 7f7f93400640 1 -- 172.21.15.28:0/2854351461 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f84032440 con 0x7f7f940d0dd0 2024-09-10T10:52:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.768+0000 7f7f93400640 1 -- 172.21.15.28:0/2854351461 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84032810 con 0x7f7f940d0dd0 2024-09-10T10:52:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 -- 172.21.15.28:0/2854351461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d0dd0 msgr2=0x7f7f940d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 --2- 172.21.15.28:0/2854351461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d0dd0 0x7f7f940d11b0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f7f84009870 tx=0x7f7f8402a230 comp rx=0 tx=0).stop 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 -- 172.21.15.28:0/2854351461 shutdown_connections 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 --2- 172.21.15.28:0/2854351461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f9415e660 0x7f7f9406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 --2- 172.21.15.28:0/2854351461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d0dd0 0x7f7f940d11b0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 -- 172.21.15.28:0/2854351461 >> 172.21.15.28:0/2854351461 conn(0x7f7f9408bfc0 msgr2=0x7f7f940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 -- 172.21.15.28:0/2854351461 shutdown_connections 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.769+0000 7f7f99a00640 1 -- 172.21.15.28:0/2854351461 wait complete. 2024-09-10T10:52:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.771+0000 7f7f99a00640 1 Processor -- start 2024-09-10T10:52:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.771+0000 7f7f99a00640 1 -- start start 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 0x7f7f94150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9415e660 0x7f7f940b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7f940d2f40 con 0x7f7f940d1f90 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f940d2c60 con 0x7f7f9415e660 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7f940d3220 con 0x7f7f940d0dd0 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37630/0 (socket says 172.21.15.28:37630) 2024-09-10T10:52:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 -- 172.21.15.28:0/75398750 learned_addr learned my addr 172.21.15.28:0/75398750 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f93e00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 0x7f7f94150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f99000640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9415e660 0x7f7f940b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 -- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 msgr2=0x7f7f94150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 0x7f7f94150920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 -- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9415e660 msgr2=0x7f7f940b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9415e660 0x7f7f940b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f8b400640 1 -- 172.21.15.28:0/75398750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f94155130 con 0x7f7f940d1f90 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.772+0000 7f7f93e00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 0x7f7f94150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f8b400640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b390 tx=0x7f7f7c00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c015020 con 0x7f7f940d1f90 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7c011830 con 0x7f7f940d1f90 2024-09-10T10:52:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c011c20 con 0x7f7f940d1f90 2024-09-10T10:52:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f94155410 con 0x7f7f940d1f90 2024-09-10T10:52:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.773+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f940b8730 con 0x7f7f940d1f90 2024-09-10T10:52:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.774+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f94155800 con 0x7f7f940d1f90 2024-09-10T10:52:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.775+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7f7c018020 con 0x7f7f940d1f90 2024-09-10T10:52:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.776+0000 7f7f90c00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 0x7f7f7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.776+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f7f7c08b480 con 0x7f7f940d1f90 2024-09-10T10:52:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.776+0000 7f7f93e00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 0x7f7f7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.777+0000 7f7f93e00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 0x7f7f7006e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f84034000 tx=0x7f7f8402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:27.780+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f7c01f020 con 0x7f7f940d1f90 2024-09-10T10:52:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.004+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7f940d2440 con 0x7f7f940d1f90 2024-09-10T10:52:28.543 INFO:tasks.ceph.mds.c.smithi028.stderr: -50> 2024-09-10T10:52:23.788+0000 7f3053600640 -1 log_channel(cluster) log [ERR] : dir 0x1 object missing on disk; some files may be lost () 2024-09-10T10:52:28.543 INFO:tasks.ceph.mds.c.smithi028.stderr: -48> 2024-09-10T10:52:23.788+0000 7f3053600640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T10:52:28.544 INFO:tasks.ceph.mds.c.smithi028.stderr: -27> 2024-09-10T10:52:25.716+0000 7f3058600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:28.585 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:52:28.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.809+0000 7f7f90c00640 1 -- 172.21.15.28:0/75398750 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v174) ==== 161+0+0 (secure 0 0 0) 0x7f7f7c0172c0 con 0x7f7f940d1f90 2024-09-10T10:52:28.810 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 msgr2=0x7f7f7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 0x7f7f7006e2f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7f84034000 tx=0x7f7f8402e000 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 msgr2=0x7f7f94154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c00b390 tx=0x7f7f7c00b850 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 shutdown_connections 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7f7006be30 0x7f7f7006e2f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f9415e660 0x7f7f940b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f940d1f90 0x7f7f94154750 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 --2- 172.21.15.28:0/75398750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7f940d0dd0 0x7f7f94150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.812+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 >> 172.21.15.28:0/75398750 conn(0x7f7f9408bfc0 msgr2=0x7f7f940b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.813+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 shutdown_connections 2024-09-10T10:52:28.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.813+0000 7f7f99a00640 1 -- 172.21.15.28:0/75398750 wait complete. 2024-09-10T10:52:28.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.989+0000 7f4858a00640 1 Processor -- start 2024-09-10T10:52:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.989+0000 7f4858a00640 1 -- start start 2024-09-10T10:52:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f4858a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f485415e660 0x7f485406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f4858a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48540d1f90 0x7f48540cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f4858a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48540cf000 con 0x7f48540d0dd0 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f4858a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f48540b37d0 con 0x7f485415e660 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f4858a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f48540cf330 con 0x7f48540d1f90 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f484be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f485415e660 0x7f485406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f484be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f485415e660 0x7f485406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37636/0 (socket says 172.21.15.28:37636) 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.990+0000 7f484be00640 1 -- 172.21.15.28:0/1683456545 learned_addr learned my addr 172.21.15.28:0/1683456545 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.991+0000 7f4853e00640 1 --2- 172.21.15.28:0/1683456545 >> v2:172.21.15.138:3300/0 conn(0x7f48540d1f90 0x7f48540cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.991+0000 7f484be00640 1 -- 172.21.15.28:0/1683456545 >> v2:172.21.15.138:3300/0 conn(0x7f48540d1f90 msgr2=0x7f48540cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.991+0000 7f484be00640 1 --2- 172.21.15.28:0/1683456545 >> v2:172.21.15.138:3300/0 conn(0x7f48540d1f90 0x7f48540cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.991+0000 7f484be00640 1 -- 172.21.15.28:0/1683456545 >> v1:172.21.15.138:6789/0 conn(0x7f48540d0dd0 legacy=0x7f48540d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.991+0000 7f484be00640 1 -- 172.21.15.28:0/1683456545 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48540d29a0 con 0x7f485415e660 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f484be00640 1 --2- 172.21.15.28:0/1683456545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f485415e660 0x7f485406ccd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f48540cb420 tx=0x7f483c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21198ddea4fc8311 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f4852a00640 1 -- 172.21.15.28:0/1683456545 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c037070 con 0x7f485415e660 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f4852a00640 1 -- 172.21.15.28:0/1683456545 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c032440 con 0x7f485415e660 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f4852a00640 1 -- 172.21.15.28:0/1683456545 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c032810 con 0x7f485415e660 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f4858a00640 1 -- 172.21.15.28:0/1683456545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f485415e660 msgr2=0x7f485406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.992+0000 7f4858a00640 1 --2- 172.21.15.28:0/1683456545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f485415e660 0x7f485406ccd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f48540cb420 tx=0x7f483c02a270 comp rx=0 tx=0).stop 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 -- 172.21.15.28:0/1683456545 shutdown_connections 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 --2- 172.21.15.28:0/1683456545 >> v2:172.21.15.138:3300/0 conn(0x7f48540d1f90 0x7f48540cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 --2- 172.21.15.28:0/1683456545 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f485415e660 0x7f485406ccd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 -- 172.21.15.28:0/1683456545 >> 172.21.15.28:0/1683456545 conn(0x7f485408bfc0 msgr2=0x7f48540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 -- 172.21.15.28:0/1683456545 shutdown_connections 2024-09-10T10:52:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.993+0000 7f4858a00640 1 -- 172.21.15.28:0/1683456545 wait complete. 2024-09-10T10:52:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.994+0000 7f4858a00640 1 Processor -- start 2024-09-10T10:52:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.994+0000 7f4858a00640 1 -- start start 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48540d0dd0 0x7f48541599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485415e660 0x7f4854157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48540d3220 con 0x7f48540d1f90 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48540d2f40 con 0x7f485415e660 2024-09-10T10:52:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.995+0000 7f4858a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48540d33a0 con 0x7f48540d0dd0 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37650/0 (socket says 172.21.15.28:37650) 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 -- 172.21.15.28:0/1234484872 learned_addr learned my addr 172.21.15.28:0/1234484872 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f4853e00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485415e660 0x7f4854157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f4853400640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48540d0dd0 0x7f48541599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 -- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48540d0dd0 msgr2=0x7f48541599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48540d0dd0 0x7f48541599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 -- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485415e660 msgr2=0x7f4854157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485415e660 0x7f4854157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 -- 172.21.15.28:0/1234484872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48540b87f0 con 0x7f48540d1f90 2024-09-10T10:52:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.996+0000 7f484be00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f483c02a2b0 tx=0x7f483c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:28.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.997+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c041020 con 0x7f48540d1f90 2024-09-10T10:52:28.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.997+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c0306c0 con 0x7f48540d1f90 2024-09-10T10:52:28.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.997+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c037040 con 0x7f48540d1f90 2024-09-10T10:52:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.997+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48540b8ad0 con 0x7f48540d1f90 2024-09-10T10:52:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.997+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48540b9090 con 0x7f48540d1f90 2024-09-10T10:52:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.998+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48540d2440 con 0x7f48540d1f90 2024-09-10T10:52:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:28.999+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f483c044020 con 0x7f48540d1f90 2024-09-10T10:52:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.000+0000 7f484b400640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 0x7f483006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.000+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f483c0abda0 con 0x7f48540d1f90 2024-09-10T10:52:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.000+0000 7f4853400640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 0x7f483006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.001+0000 7f4853400640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 0x7f483006e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4844000f60 tx=0x7f4844009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.004+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f483c04b030 con 0x7f48540d1f90 2024-09-10T10:52:29.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.257+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f48540d1b20 con 0x7f48540d1f90 2024-09-10T10:52:29.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.811+0000 7f484b400640 1 -- 172.21.15.28:0/1234484872 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v175) ==== 101+0+0 (secure 0 0 0) 0x7f483c07a080 con 0x7f48540d1f90 2024-09-10T10:52:29.812 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:29.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 msgr2=0x7f483006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:29.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 0x7f483006e2f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4844000f60 tx=0x7f4844009040 comp rx=0 tx=0).stop 2024-09-10T10:52:29.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 msgr2=0x7f485415b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f483c02a2b0 tx=0x7f483c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 shutdown_connections 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f483006be30 0x7f483006e2f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f485415e660 0x7f4854157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48540d1f90 0x7f485415b230 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 --2- 172.21.15.28:0/1234484872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48540d0dd0 0x7f48541599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 >> 172.21.15.28:0/1234484872 conn(0x7f485408bfc0 msgr2=0x7f48540b4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 shutdown_connections 2024-09-10T10:52:29.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.814+0000 7f4858a00640 1 -- 172.21.15.28:0/1234484872 wait complete. 2024-09-10T10:52:29.835 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.913+0000 7f96ed95ef40 1 Processor -- start 2024-09-10T10:52:29.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.913+0000 7f96ed95ef40 1 -- start start 2024-09-10T10:52:29.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67f2bff0 0x557b67fb8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fb73e0 0x557b6804c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b67f2c530 con 0x557b67fba9c0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b67e89b30 con 0x557b67fb73e0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557b67fb9ee0 con 0x557b67f2bff0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96eb600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96eb600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37660/0 (socket says 172.21.15.28:37660) 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96eb600640 1 -- 172.21.15.28:0/2261216343 learned_addr learned my addr 172.21.15.28:0/2261216343 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96ea200640 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fb73e0 0x557b6804c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.914+0000 7f96eac00640 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67f2bff0 0x557b67fb8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 -- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67f2bff0 msgr2=0x557b67fb8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67f2bff0 0x557b67fb8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 -- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fb73e0 msgr2=0x557b6804c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fb73e0 0x557b6804c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 -- 172.21.15.28:0/2261216343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b67fb1e00 con 0x557b67fba9c0 2024-09-10T10:52:29.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96eb600640 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f96e4025ad0 tx=0x7f96e4050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef556a69e450048 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96e8e00640 1 -- 172.21.15.28:0/2261216343 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e404f9b0 con 0x557b67fba9c0 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96e8e00640 1 -- 172.21.15.28:0/2261216343 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96e404fb50 con 0x557b67fba9c0 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.915+0000 7f96e8e00640 1 -- 172.21.15.28:0/2261216343 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4057400 con 0x557b67fba9c0 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 -- 172.21.15.28:0/2261216343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 msgr2=0x557b6804c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f96e4025ad0 tx=0x7f96e4050000 comp rx=0 tx=0).stop 2024-09-10T10:52:29.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 -- 172.21.15.28:0/2261216343 shutdown_connections 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67fba9c0 0x557b6804c6d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fb73e0 0x557b6804c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/2261216343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67f2bff0 0x557b67fb8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 -- 172.21.15.28:0/2261216343 >> 172.21.15.28:0/2261216343 conn(0x557b67f300a0 msgr2=0x557b67fb7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 -- 172.21.15.28:0/2261216343 shutdown_connections 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 -- 172.21.15.28:0/2261216343 wait complete. 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.916+0000 7f96ed95ef40 1 Processor -- start 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.917+0000 7f96ed95ef40 1 -- start start 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.917+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.917+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67fb73e0 0x557b67fad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96ed95ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fba9c0 0x557b67fb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557b67f2e3d0 con 0x557b67f2bff0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557b67fb99e0 con 0x557b67fba9c0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96ed95ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557b67fae630 con 0x557b67fb73e0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37668/0 (socket says 172.21.15.28:37668) 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 learned_addr learned my addr 172.21.15.28:0/3922841152 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67fb73e0 0x557b67fad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eb600640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fba9c0 0x557b67fb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67fb73e0 msgr2=0x557b67fad9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557b67fb73e0 0x557b67fad9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fba9c0 msgr2=0x557b67fb0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557b67fba9c0 0x557b67fb0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:29.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.918+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b67ee1420 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009f70 tx=0x7f96dc008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96e3400640 1 -- 172.21.15.28:0/3922841152 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0143f0 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96e3400640 1 -- 172.21.15.28:0/3922841152 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96dc019070 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557b67fb6710 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96e3400640 1 -- 172.21.15.28:0/3922841152 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0147b0 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.919+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b67fb2880 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.921+0000 7f96e3400640 1 -- 172.21.15.28:0/3922841152 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96dc014950 con 0x557b67f2bff0 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.922+0000 7f96e3400640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 0x7f96cc06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.922+0000 7f96e3400640 1 -- 172.21.15.28:0/3922841152 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f96dc089fe0 con 0x557b67f2bff0 2024-09-10T10:52:29.924 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:29.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.922+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x557b67f2b000 0x557b67fbcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.922+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 0x7f96cc06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.922+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b67fbd2d0 con 0x557b67f2b000 2024-09-10T10:52:29.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.923+0000 7f96eb600640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x557b67f2b000 0x557b67fbcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.923+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 0x7f96cc06ccd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x557b67fbe140 tx=0x7f96d8001830 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.923+0000 7f96eb600640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x557b67f2b000 0x557b67fbcd90 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.925+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v115'443 uv417 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.925+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b67f2a070 con 0x557b67f2b000 2024-09-10T10:52:29.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.927+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v115'444 uv444 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.927+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805c390 con 0x557b67f2b000 2024-09-10T10:52:29.930 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:29.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.929+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v115'445 uv445 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.929+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d740 con 0x557b67f2b000 2024-09-10T10:52:29.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.931+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v115'446 uv446 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.931+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805dae0 con 0x557b67f2b000 2024-09-10T10:52:29.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.933+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v115'447 uv447 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.933+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805c730 con 0x557b67f2b000 2024-09-10T10:52:29.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.935+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v115'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.935+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805caf0 con 0x557b67f2b000 2024-09-10T10:52:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.937+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v115'449 uv423 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.937+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6805cee0 con 0x557b67f2b000 2024-09-10T10:52:29.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.938+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v115'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.938+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805cee0 con 0x557b67f2b000 2024-09-10T10:52:29.941 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.940+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v115'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.940+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.942+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v115'452 uv452 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.942+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.944+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v115'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.944+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.945+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v115'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.945+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.947+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v115'455 uv430 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.947+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.948+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v115'456 uv456 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.948+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:29.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.950+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v115'457 uv457 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.950+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.952+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v115'458 uv458 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.952+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.954+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v115'459 uv459 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.954+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805d360 con 0x557b67f2b000 2024-09-10T10:52:29.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.956+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v115'460 uv460 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.956+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.958+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v115'461 uv461 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.958+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.960+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v115'462 uv436 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.960+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.961+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v115'463 uv463 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.961+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.963 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.963+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v115'464 uv464 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.963+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.964+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v115'465 uv465 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.964+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.966+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v115'466 uv466 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.966+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.968+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v115'467 uv467 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.968+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.969+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v115'468 uv442 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.969+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.971+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v115'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.971+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.973+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v115'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.974 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:29.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.973+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.974+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v115'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.974+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.976+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v115'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.976+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b67f2b000 2024-09-10T10:52:29.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.977+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v115'473 uv473 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b67f2b000 2024-09-10T10:52:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.977+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557b6805e390 0x557b68060780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.977+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6805ce90 con 0x557b6805e390 2024-09-10T10:52:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.978+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557b6805e390 0x557b68060780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:29.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.978+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557b6805e390 0x557b68060780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.980+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v115'330 uv312 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.980+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b68060cc0 con 0x557b6805e390 2024-09-10T10:52:29.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.981+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v115'331 uv331 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.981+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68061060 con 0x557b6805e390 2024-09-10T10:52:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.982+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v115'332 uv332 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.984 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.983+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68061400 con 0x557b6805e390 2024-09-10T10:52:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.984+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v115'333 uv333 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.984+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b680617a0 con 0x557b6805e390 2024-09-10T10:52:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.985+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v115'334 uv334 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.985+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68061b40 con 0x557b6805e390 2024-09-10T10:52:29.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.986+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v115'335 uv317 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.987+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b68061ee0 con 0x557b6805e390 2024-09-10T10:52:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.988+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v115'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.988+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.990 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.989+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v115'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.989+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.990+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v115'338 uv338 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.990+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.992+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v115'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.992+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.993+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v115'340 uv322 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.993+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.994+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v115'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.995+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:29.997 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:29.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.996+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v115'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.996+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68066970 con 0x557b6805e390 2024-09-10T10:52:29.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.997+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v115'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:29.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.997+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68066970 con 0x557b6805e390 2024-09-10T10:52:30.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.999+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v115'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:29.999+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:30.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.000+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v115'345 uv328 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.000+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:30.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.001+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v115'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.001+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:30.003 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:30.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.003+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v115'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.003+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68063290 con 0x557b6805e390 2024-09-10T10:52:30.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.004+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v115'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.004+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68063290 con 0x557b6805e390 2024-09-10T10:52:30.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.005+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v115'349 uv349 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.005+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68062280 con 0x557b6805e390 2024-09-10T10:52:30.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.006+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v115'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6805e390 2024-09-10T10:52:30.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.006+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x557b68062900 0x557b68064cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.006+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68065270 con 0x557b68062900 2024-09-10T10:52:30.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.007+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x557b68062900 0x557b68064cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.007+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x557b68062900 0x557b68064cf0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.009+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.009+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b67fb1460 con 0x557b68062900 2024-09-10T10:52:30.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.010+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v115'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.010+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68065c00 con 0x557b68062900 2024-09-10T10:52:30.012 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:30.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.011+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v115'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.011+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b680674e0 con 0x557b68062900 2024-09-10T10:52:30.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.013+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v115'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.013+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68067880 con 0x557b68062900 2024-09-10T10:52:30.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.014+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v115'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.014+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b680664d0 con 0x557b68062900 2024-09-10T10:52:30.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.015+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v115'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.015+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557b68066dd0 0x557b68069c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.015+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a170 con 0x557b68066dd0 2024-09-10T10:52:30.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.016+0000 7f96eb600640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557b68066dd0 0x557b68069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.016+0000 7f96eb600640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557b68066dd0 0x557b68069c30 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.018+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v115'67 uv66 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.018+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a170 con 0x557b68066dd0 2024-09-10T10:52:30.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.019+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v115'68 uv68 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.019+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a1f0 con 0x557b68066dd0 2024-09-10T10:52:30.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.020+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v115'69 uv69 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.021 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:30.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.020+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a610 con 0x557b68066dd0 2024-09-10T10:52:30.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.021+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v115'70 uv70 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.021+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a610 con 0x557b68062900 2024-09-10T10:52:30.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.022+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.022+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.023+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v115'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.023+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.025 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:30.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.024+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v115'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.024+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806b9c0 con 0x557b68062900 2024-09-10T10:52:30.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.025+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v115'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.026+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806b9c0 con 0x557b68062900 2024-09-10T10:52:30.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.027+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v115'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.027+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.028+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v115'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.028+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.029+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v115'160 uv154 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.029+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.030+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v115'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.030+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.032 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:30.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.032+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v115'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.032+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806b9c0 con 0x557b68066dd0 2024-09-10T10:52:30.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.033+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v115'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.033+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806b9c0 con 0x557b68066dd0 2024-09-10T10:52:30.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.034+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v115'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.034+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.036+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v115'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.036+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.037+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v115'166 uv159 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.037+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.039+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v115'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.039+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.041 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:30.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.040+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v115'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.040+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.041+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v115'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.041+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68066dd0 2024-09-10T10:52:30.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.042+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v115'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.042+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.044+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v115'71 uv70 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.044+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.045+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v115'72 uv72 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.045+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:30.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.046+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v115'73 uv73 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.046+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.047+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v115'74 uv74 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.047+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806a9b0 con 0x557b68062900 2024-09-10T10:52:30.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.049+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v115'75 uv75 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b68062900 2024-09-10T10:52:30.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.049+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b6806aec0 0x557b6806d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.049+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806d910 con 0x557b6806aec0 2024-09-10T10:52:30.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.049+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b6806aec0 0x557b6806d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.050+0000 7f96eac00640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b6806aec0 0x557b6806d2b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.051+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.051+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.053+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v115'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.053+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.054+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v115'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.055 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:30.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.054+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806ecf0 con 0x557b6806aec0 2024-09-10T10:52:30.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.055+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v115'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.055+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806f0e0 con 0x557b6806aec0 2024-09-10T10:52:30.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.057+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v115'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.057+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.058+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v115'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.058+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.059+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v115'142 uv136 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.060+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.061+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v115'143 uv143 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.061+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806dce0 con 0x557b6806aec0 2024-09-10T10:52:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.062+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v115'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.063 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:30.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.062+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.063+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v115'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.063+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.065+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v115'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.065+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.066+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v115'147 uv141 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.066+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.067+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v115'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.067+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.069+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v115'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.070 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:30.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.069+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e0a0 con 0x557b6806aec0 2024-09-10T10:52:30.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.070+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v115'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.070+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b68070eb0 con 0x557b6806aec0 2024-09-10T10:52:30.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.071+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v115'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.071+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.073+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v115'66 uv64 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.073+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.074+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v115'67 uv67 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.074+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.075+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v115'68 uv68 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.076 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:30.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.076+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.077+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v115'69 uv69 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.077+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.078+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v115'87 uv86 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.078+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.079+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v115'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.079+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.081 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:30.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.080+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v115'89 uv89 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.080+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806f560 con 0x557b68066dd0 2024-09-10T10:52:30.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.081+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v115'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.081+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806f560 con 0x557b68066dd0 2024-09-10T10:52:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.083+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v115'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.083+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b68066dd0 2024-09-10T10:52:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.084+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v115'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96e4060070 con 0x557b68066dd0 2024-09-10T10:52:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.084+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x557b6806ee50 0x557b6806f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.084+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806ee50 2024-09-10T10:52:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.084+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x557b6806ee50 0x557b6806f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.085+0000 7f96ea200640 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x557b6806ee50 0x557b6806f230 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.086+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v115'74 uv73 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b6806ee50 2024-09-10T10:52:30.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.086+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806ee50 2024-09-10T10:52:30.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.087+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v115'75 uv75 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b6806ee50 2024-09-10T10:52:30.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.087+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806ee50 2024-09-10T10:52:30.089 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:30.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.088+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v115'76 uv76 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b6806ee50 2024-09-10T10:52:30.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.088+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806ee50 2024-09-10T10:52:30.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.089+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v115'77 uv77 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b6806ee50 2024-09-10T10:52:30.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.089+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806ee50 2024-09-10T10:52:30.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.091+0000 7f96ea200640 1 -- 172.21.15.28:0/3922841152 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v115'78 uv78 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f96d8011650 con 0x557b6806ee50 2024-09-10T10:52:30.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.091+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.092+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v115'182 uv172 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.092+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.093+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v115'183 uv183 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.093+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.095 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:30.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.094+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v115'184 uv184 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.094+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.096+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v115'185 uv185 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.096+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.097+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v115'186 uv186 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.097+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.099 INFO:teuthology.orchestra.run.smithi028.stdout:Write #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.098+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v115'187 uv187 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.098+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.099+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [setxattr (30)] v115'188 uv188 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.099+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.100+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [setxattr (30)] v115'189 uv189 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.100+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806e550 con 0x557b6806aec0 2024-09-10T10:52:30.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.102+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [omap-set-header] v115'190 uv190 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.102+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x557b6806fdd0 con 0x557b6806aec0 2024-09-10T10:52:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96eac00640 1 -- 172.21.15.28:0/3922841152 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [omap-set-vals] v115'191 uv191 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f96dc054240 con 0x557b6806aec0 2024-09-10T10:52:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x557b68062900 msgr2=0x557b68064cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x557b68062900 0x557b68064cf0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b6806aec0 msgr2=0x557b6806d2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x557b6806aec0 0x557b6806d2b0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x557b6806ee50 msgr2=0x557b6806f230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x557b6806ee50 0x557b6806f230 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557b68066dd0 msgr2=0x557b68069c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x557b68066dd0 0x557b68069c30 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557b6805e390 msgr2=0x557b68060780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557b6805e390 0x557b68060780 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 reap_dead start 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x557b67f2b000 msgr2=0x557b67fbcd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x557b67f2b000 0x557b67fbcd90 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 reap_dead start 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96eb600640 1 -- 172.21.15.28:0/3922841152 reap_dead start 2024-09-10T10:52:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 msgr2=0x7f96cc06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 0x7f96cc06ccd0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x557b67fbe140 tx=0x7f96d8001830 comp rx=0 tx=0).stop 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 msgr2=0x557b67fb2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f96dc009f70 tx=0x7f96dc008400 comp rx=0 tx=0).stop 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 shutdown_connections 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96cc06a810 0x7f96cc06ccd0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 --2- 172.21.15.28:0/3922841152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b67f2bff0 0x557b67fb2340 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 >> 172.21.15.28:0/3922841152 conn(0x557b67f300a0 msgr2=0x557b67fb8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 shutdown_connections 2024-09-10T10:52:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.103+0000 7f96ed95ef40 1 -- 172.21.15.28:0/3922841152 wait complete. 2024-09-10T10:52:30.109 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 1.00000000.inode - 2024-09-10T10:52:30.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.225+0000 7f38c6afaf40 1 Processor -- start 2024-09-10T10:52:30.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.225+0000 7f38c6afaf40 1 -- start start 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 0x561d697b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561d697c9530 con 0x561d6977e470 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d69726b30 con 0x561d698543e0 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561d69856ee0 con 0x561d698579c0 2024-09-10T10:52:30.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37674/0 (socket says 172.21.15.28:37674) 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.226+0000 7f38c3c00640 1 -- 172.21.15.28:0/3445399472 learned_addr learned my addr 172.21.15.28:0/3445399472 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3200640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c4600640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 0x561d697b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 -- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 msgr2=0x561d697b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 0x561d697b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 -- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 msgr2=0x561d6984ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 -- 172.21.15.28:0/3445399472 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d69780930 con 0x561d6977e470 2024-09-10T10:52:30.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3200640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c4600640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 0x561d697b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c3c00640 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001d80 tx=0x7f38b4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67383c819d6dc46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c1e00640 1 -- 172.21.15.28:0/3445399472 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400bd70 con 0x561d6977e470 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c1e00640 1 -- 172.21.15.28:0/3445399472 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b400f400 con 0x561d6977e470 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.227+0000 7f38c1e00640 1 -- 172.21.15.28:0/3445399472 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400f7f0 con 0x561d6977e470 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 -- 172.21.15.28:0/3445399472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 msgr2=0x561d6984e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f38b4001d80 tx=0x7f38b4001e70 comp rx=0 tx=0).stop 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 -- 172.21.15.28:0/3445399472 shutdown_connections 2024-09-10T10:52:30.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698579c0 0x561d697b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/3445399472 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d6977e470 0x561d6984e110 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 -- 172.21.15.28:0/3445399472 >> 172.21.15.28:0/3445399472 conn(0x561d697cd0a0 msgr2=0x561d697cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:30.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 -- 172.21.15.28:0/3445399472 shutdown_connections 2024-09-10T10:52:30.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.228+0000 7f38c6afaf40 1 -- 172.21.15.28:0/3445399472 wait complete. 2024-09-10T10:52:30.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.229+0000 7f38c6afaf40 1 Processor -- start 2024-09-10T10:52:30.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.229+0000 7f38c6afaf40 1 -- start start 2024-09-10T10:52:30.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d6977e470 0x561d697cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 0x561d6984b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561d69780c10 con 0x561d698579c0 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561d697afea0 con 0x561d6977e470 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c6afaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561d69856900 con 0x561d698543e0 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 0x561d6984b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 0x561d6984b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37682/0 (socket says 172.21.15.28:37682) 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 -- 172.21.15.28:0/2332843246 learned_addr learned my addr 172.21.15.28:0/2332843246 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:30.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c3c00640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d6977e470 0x561d697cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c3200640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 -- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698543e0 msgr2=0x561d6984ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 -- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d6977e470 msgr2=0x561d697cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d6977e470 0x561d697cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c4600640 1 -- 172.21.15.28:0/2332843246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d697cb560 con 0x561d698579c0 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c3200640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561d698543e0 0x561d6984ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.230+0000 7f38c3c00640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561d6977e470 0x561d697cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c4600640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 0x561d6984b320 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f38bc008220 tx=0x7f38bc0086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c0a00640 1 -- 172.21.15.28:0/2332843246 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc016020 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c0a00640 1 -- 172.21.15.28:0/2332843246 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38bc012610 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d69841d30 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c0a00640 1 -- 172.21.15.28:0/2332843246 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc00e040 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.231+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d69846a30 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.233+0000 7f38c0a00640 1 -- 172.21.15.28:0/2332843246 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38bc019020 con 0x561d698579c0 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c0a00640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f38a406a810 0x7f38a406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c0a00640 1 -- 172.21.15.28:0/2332843246 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7f38bc08a090 con 0x561d698579c0 2024-09-10T10:52:30.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c3c00640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f38a406a810 0x7f38a406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561d69840220 0x561d69847740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115) -- 0x561d69847c80 con 0x561d69840220 2024-09-10T10:52:30.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.234+0000 7f38c3200640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561d69840220 0x561d69847740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.235+0000 7f38c3c00640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f38a406a810 0x7f38a406ccd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f38b4003040 tx=0x7f38b4009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.235+0000 7f38c3200640 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561d69840220 0x561d69847740 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c3200640 1 -- 172.21.15.28:0/2332843246 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [writefull 0~80] v115'351 uv351 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f38b0057030 con 0x561d69840220 2024-09-10T10:52:30.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561d69840220 msgr2=0x561d69847740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561d69840220 0x561d69847740 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f38a406a810 msgr2=0x7f38a406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f38a406a810 0x7f38a406ccd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f38b4003040 tx=0x7f38b4009000 comp rx=0 tx=0).stop 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 msgr2=0x561d6984b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 --2- 172.21.15.28:0/2332843246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561d698579c0 0x561d6984b320 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f38bc008220 tx=0x7f38bc0086e0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c4600640 1 -- 172.21.15.28:0/2332843246 reap_dead start 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 shutdown_connections 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 >> 172.21.15.28:0/2332843246 conn(0x561d697cd0a0 msgr2=0x561d697b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 shutdown_connections 2024-09-10T10:52:30.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.237+0000 7f38c6afaf40 1 -- 172.21.15.28:0/2332843246 wait complete. 2024-09-10T10:52:30.242 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:30.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.441+0000 7fd488600640 1 Processor -- start 2024-09-10T10:52:30.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.441+0000 7fd488600640 1 -- start start 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd488600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd480072760 0x7fd4800b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd488600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4800b4620 con 0x7fd4800b4020 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd488600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4800ce100 con 0x7fd48015e900 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd488600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4800b4f60 con 0x7fd480072760 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd485200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd4800b4020 0x7fd480072050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41784/0 (socket says 172.21.15.28:41784) 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd485200640 1 -- 172.21.15.28:0/2779501956 learned_addr learned my addr 172.21.15.28:0/2779501956 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:30.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd486600640 1 --2- 172.21.15.28:0/2779501956 >> v2:172.21.15.138:3300/0 conn(0x7fd480072760 0x7fd4800b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84967585 0 0) 0x7fd4800b4620 con 0x7fd4800b4020 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.443+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd468003520 con 0x7fd4800b4020 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219431054 0 0) 0x7fd4800ce100 con 0x7fd48015e900 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4800b4620 con 0x7fd48015e900 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd486600640 1 -- 172.21.15.28:0/2779501956 >> v1:172.21.15.138:6789/0 conn(0x7fd48015e900 legacy=0x7fd4800ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd486600640 1 -- 172.21.15.28:0/2779501956 >> v1:172.21.15.28:6789/0 conn(0x7fd4800b4020 legacy=0x7fd480072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598908687 0 0) 0x7fd468003520 con 0x7fd4800b4020 2024-09-10T10:52:30.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.444+0000 7fd486600640 1 -- 172.21.15.28:0/2779501956 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4800ce3a0 con 0x7fd480072760 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.445+0000 7fd486600640 1 --2- 172.21.15.28:0/2779501956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd480072760 0x7fd4800b4a20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd478009d90 tx=0x7fd47802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c249cb75ee385c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.445+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c001670 con 0x7fd480072760 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.445+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd478007540 con 0x7fd480072760 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.445+0000 7fd47fe00640 1 -- 172.21.15.28:0/2779501956 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478007910 con 0x7fd480072760 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 -- 172.21.15.28:0/2779501956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd480072760 msgr2=0x7fd4800b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 --2- 172.21.15.28:0/2779501956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd480072760 0x7fd4800b4a20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd478009d90 tx=0x7fd47802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 -- 172.21.15.28:0/2779501956 shutdown_connections 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 --2- 172.21.15.28:0/2779501956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd480072760 0x7fd4800b4a20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 -- 172.21.15.28:0/2779501956 >> 172.21.15.28:0/2779501956 conn(0x7fd48006bdf0 msgr2=0x7fd48006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:30.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 -- 172.21.15.28:0/2779501956 shutdown_connections 2024-09-10T10:52:30.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.446+0000 7fd488600640 1 -- 172.21.15.28:0/2779501956 wait complete. 2024-09-10T10:52:30.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.447+0000 7fd488600640 1 Processor -- start 2024-09-10T10:52:30.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.447+0000 7fd488600640 1 -- start start 2024-09-10T10:52:30.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 0x7fd4800c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 0x7fd4800c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800cee80 con 0x7fd480072760 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4800ce960 con 0x7fd48015e900 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd488600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4800cf000 con 0x7fd4800b4020 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.448+0000 7fd485c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37690/0 (socket says 172.21.15.28:37690) 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 -- 172.21.15.28:0/1489165072 learned_addr learned my addr 172.21.15.28:0/1489165072 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485200640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 0x7fd4800c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd486600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 0x7fd4800c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 -- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 msgr2=0x7fd4800c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 0x7fd4800c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 -- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 msgr2=0x7fd4800c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 0x7fd4800c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 -- 172.21.15.28:0/1489165072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48009d160 con 0x7fd480072760 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485200640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 0x7fd4800c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd486600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 0x7fd4800c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd485c00640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c430 tx=0x7fd47400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.449+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474017070 con 0x7fd480072760 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.450+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4740052f0 con 0x7fd480072760 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.450+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48009d440 con 0x7fd480072760 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.450+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd474003830 con 0x7fd480072760 2024-09-10T10:52:30.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.450+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48009da00 con 0x7fd480072760 2024-09-10T10:52:30.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.451+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4800b4a20 con 0x7fd480072760 2024-09-10T10:52:30.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.452+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd47401a070 con 0x7fd480072760 2024-09-10T10:52:30.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.453+0000 7fd47d600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 0x7fd46006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:30.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.453+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 7594+0+0 (secure 0 0 0) 0x7fd47408aae0 con 0x7fd480072760 2024-09-10T10:52:30.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.453+0000 7fd485200640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 0x7fd46006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:30.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.454+0000 7fd485200640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 0x7fd46006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd46c001340 tx=0x7fd46c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:30.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.457+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd474058cc0 con 0x7fd480072760 2024-09-10T10:52:30.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.677+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd4800b1ab0 con 0x7fd480072760 2024-09-10T10:52:30.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.817+0000 7fd47d600640 1 -- 172.21.15.28:0/1489165072 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v176) ==== 162+0+0 (secure 0 0 0) 0x7fd4740585c0 con 0x7fd480072760 2024-09-10T10:52:30.821 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:30.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 msgr2=0x7fd46006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 0x7fd46006e2f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd46c001340 tx=0x7fd46c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 msgr2=0x7fd48015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c430 tx=0x7fd47400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 shutdown_connections 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd46006be30 0x7fd46006e2f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48015e900 0x7fd4800c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4800b4020 0x7fd4800c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 --2- 172.21.15.28:0/1489165072 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd480072760 0x7fd48015b720 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 >> 172.21.15.28:0/1489165072 conn(0x7fd48006bdf0 msgr2=0x7fd4800b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 shutdown_connections 2024-09-10T10:52:30.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.820+0000 7fd488600640 1 -- 172.21.15.28:0/1489165072 wait complete. 2024-09-10T10:52:30.832 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:30.831+0000 7fee85200640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-10T10:52:30.832 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:30.831+0000 7fee85200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:52:30.832 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:30.832+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:30.832 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:30.832+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:30.842 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:30.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.996+0000 7f2f77400640 1 Processor -- start 2024-09-10T10:52:30.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.997+0000 7f2f77400640 1 -- start start 2024-09-10T10:52:30.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f77400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 0x7f2f700cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f77400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f7015e1f0 0x7f2f7006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f77400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f70146140 con 0x7f2f70072de0 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f77400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2f700cd740 con 0x7f2f7015e1f0 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f77400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f700cda20 con 0x7f2f70072320 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f75400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f7015e1f0 0x7f2f7006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f75400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f7015e1f0 0x7f2f7006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37706/0 (socket says 172.21.15.28:37706) 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f75400640 1 -- 172.21.15.28:0/1751831059 learned_addr learned my addr 172.21.15.28:0/1751831059 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.998+0000 7f2f74a00640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 0x7f2f700cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.999+0000 7f2f75400640 1 -- 172.21.15.28:0/1751831059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 msgr2=0x7f2f700cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:31.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.999+0000 7f2f75400640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 0x7f2f700cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.999+0000 7f2f75400640 1 -- 172.21.15.28:0/1751831059 >> v1:172.21.15.138:6789/0 conn(0x7f2f70072de0 legacy=0x7f2f7006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.999+0000 7f2f75400640 1 -- 172.21.15.28:0/1751831059 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f700cdd00 con 0x7f2f7015e1f0 2024-09-10T10:52:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:30.999+0000 7f2f74a00640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 0x7f2f700cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f75400640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f7006e7c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009e50 tx=0x7f2f6802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8c09fe3f9993fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f6fe00640 1 -- 172.21.15.28:0/1751831059 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68036070 con 0x7f2f7015e1f0 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f6fe00640 1 -- 172.21.15.28:0/1751831059 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f68031600 con 0x7f2f7015e1f0 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f6fe00640 1 -- 172.21.15.28:0/1751831059 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680319d0 con 0x7f2f7015e1f0 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 -- 172.21.15.28:0/1751831059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 msgr2=0x7f2f7006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f7006e7c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009e50 tx=0x7f2f6802a640 comp rx=0 tx=0).stop 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 -- 172.21.15.28:0/1751831059 shutdown_connections 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f7006e7c0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 --2- 172.21.15.28:0/1751831059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072320 0x7f2f700cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.000+0000 7f2f77400640 1 -- 172.21.15.28:0/1751831059 >> 172.21.15.28:0/1751831059 conn(0x7f2f70067d70 msgr2=0x7f2f7006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.001+0000 7f2f77400640 1 -- 172.21.15.28:0/1751831059 shutdown_connections 2024-09-10T10:52:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.001+0000 7f2f77400640 1 -- 172.21.15.28:0/1751831059 wait complete. 2024-09-10T10:52:31.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.002+0000 7f2f77400640 1 Processor -- start 2024-09-10T10:52:31.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.002+0000 7f2f77400640 1 -- start start 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 0x7f2f7015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072de0 0x7f2f700b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2f700cd520 con 0x7f2f7015e1f0 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f700cd3a0 con 0x7f2f70072320 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f77400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7006ed00 con 0x7f2f70072de0 2024-09-10T10:52:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.003+0000 7f2f75400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37720/0 (socket says 172.21.15.28:37720) 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 -- 172.21.15.28:0/452921571 learned_addr learned my addr 172.21.15.28:0/452921571 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f74a00640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 0x7f2f7015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f67e00640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072de0 0x7f2f700b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 -- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072de0 msgr2=0x7f2f700b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072de0 0x7f2f700b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 -- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 msgr2=0x7f2f7015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 0x7f2f7015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 -- 172.21.15.28:0/452921571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f700b17a0 con 0x7f2f7015e1f0 2024-09-10T10:52:31.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f74a00640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 0x7f2f7015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f75400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009e50 tx=0x7f2f68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.004+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68036050 con 0x7f2f7015e1f0 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.005+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f68042070 con 0x7f2f7015e1f0 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.005+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68009510 con 0x7f2f7015e1f0 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.005+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f700a9630 con 0x7f2f7015e1f0 2024-09-10T10:52:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.005+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f700a9bf0 con 0x7f2f7015e1f0 2024-09-10T10:52:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.006+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f70072700 con 0x7f2f7015e1f0 2024-09-10T10:52:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.007+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2f68007bd0 con 0x7f2f7015e1f0 2024-09-10T10:52:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.007+0000 7f2f6d600640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 0x7f2f4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.008+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f2f680ac580 con 0x7f2f7015e1f0 2024-09-10T10:52:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.008+0000 7f2f74a00640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 0x7f2f4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.009+0000 7f2f74a00640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 0x7f2f4c06e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000a990 tx=0x7f2f60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.012+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f6802ae20 con 0x7f2f7015e1f0 2024-09-10T10:52:31.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.231+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f700ceb80 con 0x7f2f7015e1f0 2024-09-10T10:52:31.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.232+0000 7f2f6d600640 1 -- 172.21.15.28:0/452921571 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) ==== 95+0+4389 (secure 0 0 0) 0x7f2f6807a790 con 0x7f2f7015e1f0 2024-09-10T10:52:31.233 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:31.233 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":178,"btime":"2024-09-10T10:52:30:832650+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:30.832593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:31.234 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 178 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.234+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 msgr2=0x7f2f4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.234+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 0x7f2f4c06e2f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000a990 tx=0x7f2f60008040 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.234+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 msgr2=0x7f2f700ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.234+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f2f68009e50 tx=0x7f2f68004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 shutdown_connections 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2f4c06be30 0x7f2f4c06e2f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f7015e1f0 0x7f2f700ae170 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f70072de0 0x7f2f700b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 --2- 172.21.15.28:0/452921571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f70072320 0x7f2f7015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 >> 172.21.15.28:0/452921571 conn(0x7f2f70067d70 msgr2=0x7f2f700b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:31.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 shutdown_connections 2024-09-10T10:52:31.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.235+0000 7f2f77400640 1 -- 172.21.15.28:0/452921571 wait complete. 2024-09-10T10:52:31.251 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:31.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:31.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.406+0000 7f44a7a00640 1 Processor -- start 2024-09-10T10:52:31.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.406+0000 7f44a7a00640 1 -- start start 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.407+0000 7f44a7a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 0x7f44a006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.407+0000 7f44a7a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44a0147e90 0x7f44a00720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.407+0000 7f44a7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f44a7a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00b34d0 con 0x7f44a0147e90 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f44a7a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44a00ae9e0 con 0x7f44a0147230 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f44a7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44a0072a20 con 0x7f44a00725f0 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44a0147e90 0x7f44a00720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44a0147e90 0x7f44a00720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37730/0 (socket says 172.21.15.28:37730) 2024-09-10T10:52:31.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 -- 172.21.15.28:0/776422332 learned_addr learned my addr 172.21.15.28:0/776422332 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:31.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 -- 172.21.15.28:0/776422332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 msgr2=0x7f44a0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:31.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 --2- 172.21.15.28:0/776422332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 -- 172.21.15.28:0/776422332 >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 msgr2=0x7f44a006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f44a5000640 1 --2- 172.21.15.28:0/776422332 >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 0x7f44a006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 --2- 172.21.15.28:0/776422332 >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 0x7f44a006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f449fe00640 1 -- 172.21.15.28:0/776422332 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a006f480 con 0x7f44a0147e90 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.408+0000 7f44a5000640 1 --2- 172.21.15.28:0/776422332 >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 0x7f44a006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.409+0000 7f449fe00640 1 --2- 172.21.15.28:0/776422332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147e90 0x7f44a00720b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f44a0070280 tx=0x7f448c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495ca8149ecc6546 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f449f400640 1 -- 172.21.15.28:0/776422332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c037070 con 0x7f44a0147e90 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f449f400640 1 -- 172.21.15.28:0/776422332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c032440 con 0x7f44a0147e90 2024-09-10T10:52:31.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f449f400640 1 -- 172.21.15.28:0/776422332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c032810 con 0x7f44a0147e90 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 -- 172.21.15.28:0/776422332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147e90 msgr2=0x7f44a00720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 --2- 172.21.15.28:0/776422332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147e90 0x7f44a00720b0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f44a0070280 tx=0x7f448c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 -- 172.21.15.28:0/776422332 shutdown_connections 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 --2- 172.21.15.28:0/776422332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a0161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 --2- 172.21.15.28:0/776422332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147e90 0x7f44a00720b0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 --2- 172.21.15.28:0/776422332 >> v2:172.21.15.138:3300/0 conn(0x7f44a0147230 0x7f44a006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 -- 172.21.15.28:0/776422332 >> 172.21.15.28:0/776422332 conn(0x7f44a0067d70 msgr2=0x7f44a006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:31.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 -- 172.21.15.28:0/776422332 shutdown_connections 2024-09-10T10:52:31.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.410+0000 7f44a7a00640 1 -- 172.21.15.28:0/776422332 wait complete. 2024-09-10T10:52:31.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.412+0000 7f44a7a00640 1 Processor -- start 2024-09-10T10:52:31.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.412+0000 7f44a7a00640 1 -- start start 2024-09-10T10:52:31.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 0x7f44a013eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44a006eec0 con 0x7f44a0147230 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44a006ebe0 con 0x7f44a0147e90 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44a006f1a0 con 0x7f44a00725f0 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f449fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f449fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37740/0 (socket says 172.21.15.28:37740) 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f449fe00640 1 -- 172.21.15.28:0/3333436755 learned_addr learned my addr 172.21.15.28:0/3333436755 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a5000640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.413+0000 7f44a5a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 0x7f44a013eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 -- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 msgr2=0x7f44a013d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a013d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 -- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 msgr2=0x7f44a013eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 0x7f44a013eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 -- 172.21.15.28:0/3333436755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a013f620 con 0x7f44a0147230 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f44a5a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 0x7f44a013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449fe00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f448c02a220 tx=0x7f448c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c037040 con 0x7f44a0147230 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c042070 con 0x7f44a0147230 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c030bb0 con 0x7f44a0147230 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a013f900 con 0x7f44a0147230 2024-09-10T10:52:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.414+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a013fec0 con 0x7f44a0147230 2024-09-10T10:52:31.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.415+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a0146d70 con 0x7f44a0147230 2024-09-10T10:52:31.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.416+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f448c047460 con 0x7f44a0147230 2024-09-10T10:52:31.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.417+0000 7f449cc00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 0x7f448006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.417+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f448c0ac230 con 0x7f44a0147230 2024-09-10T10:52:31.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.418+0000 7f44a5000640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 0x7f448006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.418+0000 7f44a5000640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 0x7f448006e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f449400a990 tx=0x7f4494008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.421+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f448c045360 con 0x7f44a0147230 2024-09-10T10:52:31.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.643+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44a0147610 con 0x7f44a0147230 2024-09-10T10:52:31.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.643+0000 7f449cc00640 1 -- 172.21.15.28:0/3333436755 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) ==== 95+0+4389 (secure 0 0 0) 0x7f448c07a440 con 0x7f44a0147230 2024-09-10T10:52:31.645 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:31.645 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":178,"btime":"2024-09-10T10:52:30:832650+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:30.832593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:31.645 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 178 2024-09-10T10:52:31.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 msgr2=0x7f448006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 0x7f448006e2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f449400a990 tx=0x7f4494008040 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 msgr2=0x7f44a015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f448c02a220 tx=0x7f448c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 shutdown_connections 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f448006be30 0x7f448006e2f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0147e90 0x7f44a013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44a0147230 0x7f44a015ddf0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 --2- 172.21.15.28:0/3333436755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44a00725f0 0x7f44a013d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 >> 172.21.15.28:0/3333436755 conn(0x7f44a0067d70 msgr2=0x7f44a0145040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 shutdown_connections 2024-09-10T10:52:31.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.646+0000 7f44a7a00640 1 -- 172.21.15.28:0/3333436755 wait complete. 2024-09-10T10:52:31.662 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:31.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:31.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.817+0000 7ff2ea200640 1 Processor -- start 2024-09-10T10:52:31.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.817+0000 7ff2ea200640 1 -- start start 2024-09-10T10:52:31.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.818+0000 7ff2ea200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff2e406f8d0 0x7ff2e4165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.818+0000 7ff2ea200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e40cdbe0 con 0x7ff2e4072d30 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.818+0000 7ff2ea200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2e40cd900 con 0x7ff2e415b300 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.818+0000 7ff2ea200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff2e40728a0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.818+0000 7ff2e3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff2e406f8d0 0x7ff2e4165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff2e4072d30 0x7ff2e406f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41792/0 (socket says 172.21.15.28:41792) 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e2a00640 1 -- 172.21.15.28:0/554636599 learned_addr learned my addr 172.21.15.28:0/554636599 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:31.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e3e00640 1 -- 172.21.15.28:0/554636599 >> v1:172.21.15.138:6789/0 conn(0x7ff2e415b300 legacy=0x7ff2e40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e3e00640 1 -- 172.21.15.28:0/554636599 >> v1:172.21.15.28:6789/0 conn(0x7ff2e4072d30 legacy=0x7ff2e406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e3e00640 1 -- 172.21.15.28:0/554636599 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e4072250 con 0x7ff2e406f8d0 2024-09-10T10:52:31.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.819+0000 7ff2e2000640 1 -- 172.21.15.28:0/554636599 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4275541689 0 0) 0x7ff2e40cdbe0 con 0x7ff2e4072d30 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.820+0000 7ff2e3e00640 1 --2- 172.21.15.28:0/554636599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e4165de0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009d90 tx=0x7ff2dc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcb35ed9957c79b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.820+0000 7ff2e2000640 1 -- 172.21.15.28:0/554636599 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc007740 con 0x7ff2e406f8d0 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.820+0000 7ff2e2000640 1 -- 172.21.15.28:0/554636599 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2dc0078e0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.820+0000 7ff2e2000640 1 -- 172.21.15.28:0/554636599 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc007cb0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 -- 172.21.15.28:0/554636599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 msgr2=0x7ff2e4165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 --2- 172.21.15.28:0/554636599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e4165de0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009d90 tx=0x7ff2dc02a660 comp rx=0 tx=0).stop 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 -- 172.21.15.28:0/554636599 shutdown_connections 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 --2- 172.21.15.28:0/554636599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e4165de0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 -- 172.21.15.28:0/554636599 >> 172.21.15.28:0/554636599 conn(0x7ff2e4067d70 msgr2=0x7ff2e406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:31.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 -- 172.21.15.28:0/554636599 shutdown_connections 2024-09-10T10:52:31.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.821+0000 7ff2ea200640 1 -- 172.21.15.28:0/554636599 wait complete. 2024-09-10T10:52:31.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.823+0000 7ff2ea200640 1 Processor -- start 2024-09-10T10:52:31.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.823+0000 7ff2ea200640 1 -- start start 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 0x7ff2e41f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 0x7ff2e4154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40cffe0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e40cd5c0 con 0x7ff2e415b300 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2ea200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2e40d0160 con 0x7ff2e4072d30 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2e3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2e3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37746/0 (socket says 172.21.15.28:37746) 2024-09-10T10:52:31.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2e3400640 1 -- 172.21.15.28:0/3631259596 learned_addr learned my addr 172.21.15.28:0/3631259596 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2e3e00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 0x7ff2e4154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.824+0000 7ff2e2a00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 0x7ff2e41f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 -- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 msgr2=0x7ff2e41f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 0x7ff2e41f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 -- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 msgr2=0x7ff2e4154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 0x7ff2e4154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 -- 172.21.15.28:0/3631259596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e41f7b90 con 0x7ff2e406f8d0 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3e00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 0x7ff2e4154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e2a00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 0x7ff2e41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:31.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.825+0000 7ff2e3400640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c400 tx=0x7ff2d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.826+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4017070 con 0x7ff2e406f8d0 2024-09-10T10:52:31.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.826+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d40052d0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.826+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4003830 con 0x7ff2e406f8d0 2024-09-10T10:52:31.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.826+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e41f7e70 con 0x7ff2e406f8d0 2024-09-10T10:52:31.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.826+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41f8430 con 0x7ff2e406f8d0 2024-09-10T10:52:31.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.827+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2e406f1c0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.828+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff2d401a070 con 0x7ff2e406f8d0 2024-09-10T10:52:31.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.829+0000 7ff2db400640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 0x7ff2c406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:31.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.829+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7ff2d408a8d0 con 0x7ff2e406f8d0 2024-09-10T10:52:31.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.829+0000 7ff2e2a00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 0x7ff2c406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:31.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.830+0000 7ff2e2a00640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 0x7ff2c406e320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2e41f8a80 tx=0x7ff2d0007c50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:31.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:31.833+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff2d4058b60 con 0x7ff2e406f8d0 2024-09-10T10:52:32.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.054+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2e4158bb0 con 0x7ff2e406f8d0 2024-09-10T10:52:32.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.054+0000 7ff2db400640 1 -- 172.21.15.28:0/3631259596 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) ==== 95+0+4389 (secure 0 0 0) 0x7ff2d4058460 con 0x7ff2e406f8d0 2024-09-10T10:52:32.055 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:32.056 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":178,"btime":"2024-09-10T10:52:30:832650+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/205773431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":205773431},{"type":"v1","addr":"172.21.15.28:6835","nonce":205773431}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":170}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:30.832593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":116,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:32.056 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 178 2024-09-10T10:52:32.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 msgr2=0x7ff2c406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 0x7ff2c406e320 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff2e41f8a80 tx=0x7ff2d0007c50 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 msgr2=0x7ff2e41e4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400c400 tx=0x7ff2d400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 shutdown_connections 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff2c406be60 0x7ff2c406e320 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2e415b300 0x7ff2e4154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2e4072d30 0x7ff2e41f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 --2- 172.21.15.28:0/3631259596 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2e406f8d0 0x7ff2e41e4c10 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 >> 172.21.15.28:0/3631259596 conn(0x7ff2e4067d70 msgr2=0x7ff2e40b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 shutdown_connections 2024-09-10T10:52:32.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.056+0000 7ff2ea200640 1 -- 172.21.15.28:0/3631259596 wait complete. 2024-09-10T10:52:32.074 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 1.00000000.inode' led to DAMAGED state 2024-09-10T10:52:32.074 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000000' 2024-09-10T10:52:32.074 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:32.074 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:32.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:32.109 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:32.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:32.109 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:32.184 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:32.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:32.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:32.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.381+0000 7ff161400640 1 Processor -- start 2024-09-10T10:52:32.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.381+0000 7ff161400640 1 -- start start 2024-09-10T10:52:32.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.382+0000 7ff161400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.382+0000 7ff161400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 0x7ff15c0d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.382+0000 7ff161400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15c0d2700 con 0x7ff15c15e760 2024-09-10T10:52:32.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.382+0000 7ff161400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff15c0cedb0 con 0x7ff15c0d4fd0 2024-09-10T10:52:32.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.382+0000 7ff161400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c0cf090 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:54118/0 (socket says 172.21.15.28:54118) 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 -- 172.21.15.28:0/316806092 learned_addr learned my addr 172.21.15.28:0/316806092 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff15b200640 1 --2- 172.21.15.28:0/316806092 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 0x7ff15c0d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 -- 172.21.15.28:0/316806092 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 msgr2=0x7ff15c0d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 --2- 172.21.15.28:0/316806092 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 0x7ff15c0d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 -- 172.21.15.28:0/316806092 >> v1:172.21.15.138:6789/0 conn(0x7ff15c15e760 legacy=0x7ff15c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff159e00640 1 -- 172.21.15.28:0/316806092 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c0b3c80 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.383+0000 7ff15b200640 1 --2- 172.21.15.28:0/316806092 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 0x7ff15c0d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:32.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.384+0000 7ff159e00640 1 --2- 172.21.15.28:0/316806092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff148009850 tx=0x7ff14802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f43d7c0c1048283 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff159400640 1 -- 172.21.15.28:0/316806092 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148004030 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff159400640 1 -- 172.21.15.28:0/316806092 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1480041d0 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff159400640 1 -- 172.21.15.28:0/316806092 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148003610 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 -- 172.21.15.28:0/316806092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 msgr2=0x7ff15c06cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 --2- 172.21.15.28:0/316806092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff148009850 tx=0x7ff14802a230 comp rx=0 tx=0).stop 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 -- 172.21.15.28:0/316806092 shutdown_connections 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 --2- 172.21.15.28:0/316806092 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0d4fd0 0x7ff15c0d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 --2- 172.21.15.28:0/316806092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0cfcc0 0x7ff15c06cdd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 -- 172.21.15.28:0/316806092 >> 172.21.15.28:0/316806092 conn(0x7ff15c08bfc0 msgr2=0x7ff15c0d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 -- 172.21.15.28:0/316806092 shutdown_connections 2024-09-10T10:52:32.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.385+0000 7ff161400640 1 -- 172.21.15.28:0/316806092 wait complete. 2024-09-10T10:52:32.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.387+0000 7ff161400640 1 Processor -- start 2024-09-10T10:52:32.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.387+0000 7ff161400640 1 -- start start 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c0cfcc0 0x7ff15c1f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0d4fd0 0x7ff15c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c0d33e0 con 0x7ff15c15e760 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff15c0b38a0 con 0x7ff15c0cfcc0 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff161400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff15c0d3560 con 0x7ff15c0d4fd0 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff15b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff15b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37750/0 (socket says 172.21.15.28:37750) 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff15b200640 1 -- 172.21.15.28:0/898853854 learned_addr learned my addr 172.21.15.28:0/898853854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff159e00640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0d4fd0 0x7ff15c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.388+0000 7ff15a800640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c0cfcc0 0x7ff15c1f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 -- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0d4fd0 msgr2=0x7ff15c0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0d4fd0 0x7ff15c0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 -- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c0cfcc0 msgr2=0x7ff15c1f4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c0cfcc0 0x7ff15c1f4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 -- 172.21.15.28:0/898853854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15c0b8030 con 0x7ff15c15e760 2024-09-10T10:52:32.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff15b200640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff15400efc0 tx=0x7ff15400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154018030 con 0x7ff15c15e760 2024-09-10T10:52:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15400ccd0 con 0x7ff15c15e760 2024-09-10T10:52:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154003bb0 con 0x7ff15c15e760 2024-09-10T10:52:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.389+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15c0b8370 con 0x7ff15c15e760 2024-09-10T10:52:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.390+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15c0b8bb0 con 0x7ff15c15e760 2024-09-10T10:52:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.391+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15c0d3890 con 0x7ff15c15e760 2024-09-10T10:52:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.392+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff154005d10 con 0x7ff15c15e760 2024-09-10T10:52:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.392+0000 7ff152a00640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.392+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7ff15408be60 con 0x7ff15c15e760 2024-09-10T10:52:32.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.393+0000 7ff15a800640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:32.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.393+0000 7ff15a800640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0016e0 tx=0x7ff14c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:32.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.397+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff154022070 con 0x7ff15c15e760 2024-09-10T10:52:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.614+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff15c0b6d20 con 0x7ff15c15e760 2024-09-10T10:52:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.828+0000 7ff152a00640 1 -- 172.21.15.28:0/898853854 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v179) ==== 161+0+0 (secure 0 0 0) 0x7ff15c0b8370 con 0x7ff15c15e760 2024-09-10T10:52:32.829 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 msgr2=0x7ff13c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 0x7ff13c06e2f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0016e0 tx=0x7ff14c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 msgr2=0x7ff15c155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff15400efc0 tx=0x7ff15400c490 comp rx=0 tx=0).stop 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 shutdown_connections 2024-09-10T10:52:32.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff13c06be30 0x7ff13c06e2f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff15c15e760 0x7ff15c155340 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff15c0d4fd0 0x7ff15c0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 --2- 172.21.15.28:0/898853854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c0cfcc0 0x7ff15c1f4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 >> 172.21.15.28:0/898853854 conn(0x7ff15c08bfc0 msgr2=0x7ff15c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 shutdown_connections 2024-09-10T10:52:32.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:32.831+0000 7ff161400640 1 -- 172.21.15.28:0/898853854 wait complete. 2024-09-10T10:52:32.848 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:33.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.004+0000 7f4c4ee00640 1 Processor -- start 2024-09-10T10:52:33.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.005+0000 7f4c4ee00640 1 -- start start 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c4ee00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c4806f8d0 0x7f4c48165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c4ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c480cdbe0 con 0x7f4c48072d30 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c4ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c480cd900 con 0x7f4c4815b300 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c4ee00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4c480728a0 con 0x7f4c4806f8d0 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c3f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4c48072d30 0x7f4c4806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:41800/0 (socket says 172.21.15.28:41800) 2024-09-10T10:52:33.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c3f400640 1 -- 172.21.15.28:0/3984414308 learned_addr learned my addr 172.21.15.28:0/3984414308 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:33.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.006+0000 7f4c4ce00640 1 --2- 172.21.15.28:0/3984414308 >> v2:172.21.15.138:3300/0 conn(0x7f4c4806f8d0 0x7f4c48165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847211257 0 0) 0x7f4c480cdbe0 con 0x7f4c48072d30 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c2c003520 con 0x7f4c48072d30 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228577217 0 0) 0x7f4c480cd900 con 0x7f4c4815b300 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c480cdbe0 con 0x7f4c4815b300 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c4ce00640 1 -- 172.21.15.28:0/3984414308 >> v1:172.21.15.138:6789/0 conn(0x7f4c4815b300 legacy=0x7f4c480cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c4ce00640 1 -- 172.21.15.28:0/3984414308 >> v1:172.21.15.28:6789/0 conn(0x7f4c48072d30 legacy=0x7f4c4806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4154463711 0 0) 0x7f4c2c003520 con 0x7f4c48072d30 2024-09-10T10:52:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.007+0000 7f4c4ce00640 1 -- 172.21.15.28:0/3984414308 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c48072250 con 0x7f4c4806f8d0 2024-09-10T10:52:33.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.008+0000 7f4c4ce00640 1 --2- 172.21.15.28:0/3984414308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4806f8d0 0x7f4c48165de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c4002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed40ad122d4602b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.008+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c30001670 con 0x7f4c4806f8d0 2024-09-10T10:52:33.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.008+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40007540 con 0x7f4c4806f8d0 2024-09-10T10:52:33.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.008+0000 7f4c47400640 1 -- 172.21.15.28:0/3984414308 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007910 con 0x7f4c4806f8d0 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 -- 172.21.15.28:0/3984414308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4806f8d0 msgr2=0x7f4c48165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/3984414308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4806f8d0 0x7f4c48165de0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c4002a5f0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 -- 172.21.15.28:0/3984414308 shutdown_connections 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/3984414308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4806f8d0 0x7f4c48165de0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 -- 172.21.15.28:0/3984414308 >> 172.21.15.28:0/3984414308 conn(0x7f4c48067d70 msgr2=0x7f4c4806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 -- 172.21.15.28:0/3984414308 shutdown_connections 2024-09-10T10:52:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.009+0000 7f4c4ee00640 1 -- 172.21.15.28:0/3984414308 wait complete. 2024-09-10T10:52:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.011+0000 7f4c4ee00640 1 Processor -- start 2024-09-10T10:52:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.011+0000 7f4c4ee00640 1 -- start start 2024-09-10T10:52:33.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c4806f8d0 0x7f4c481f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48072d30 0x7f4c481532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c480cffe0 con 0x7f4c4815b300 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c480cd5c0 con 0x7f4c48072d30 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c480d0160 con 0x7f4c4806f8d0 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37764/0 (socket says 172.21.15.28:37764) 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c4ce00640 1 -- 172.21.15.28:0/2002354715 learned_addr learned my addr 172.21.15.28:0/2002354715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c3f400640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48072d30 0x7f4c481532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.012+0000 7f4c47e00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c4806f8d0 0x7f4c481f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 -- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c4806f8d0 msgr2=0x7f4c481f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c4806f8d0 0x7f4c481f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 -- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48072d30 msgr2=0x7f4c481532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48072d30 0x7f4c481532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 -- 172.21.15.28:0/2002354715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c48153800 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ce00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40040070 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40031bc0 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c48153aa0 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007e90 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.013+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c48154060 con 0x7f4c4815b300 2024-09-10T10:52:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.015+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c480b2580 con 0x7f4c4815b300 2024-09-10T10:52:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.015+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c40039040 con 0x7f4c4815b300 2024-09-10T10:52:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.016+0000 7f4c44c00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 0x7f4c2406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.016+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f4c400ac920 con 0x7f4c4815b300 2024-09-10T10:52:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.017+0000 7f4c47e00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 0x7f4c2406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.018+0000 7f4c47e00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 0x7f4c2406e320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4c380011e0 tx=0x7f4c38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.021+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c480b2580 con 0x7f4c4815b300 2024-09-10T10:52:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.280+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f4c480b2130 con 0x7f4c4815b300 2024-09-10T10:52:33.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.830+0000 7f4c44c00640 1 -- 172.21.15.28:0/2002354715 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v180) ==== 101+0+0 (secure 0 0 0) 0x7f4c4007a450 con 0x7f4c4815b300 2024-09-10T10:52:33.831 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.832+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 msgr2=0x7f4c2406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.832+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 0x7f4c2406e320 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4c380011e0 tx=0x7f4c38009040 comp rx=0 tx=0).stop 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.832+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 msgr2=0x7f4c48152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.832+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009d90 tx=0x7f4c40004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 shutdown_connections 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c2406be60 0x7f4c2406e320 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c4815b300 0x7f4c48152140 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c48072d30 0x7f4c481532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 --2- 172.21.15.28:0/2002354715 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c4806f8d0 0x7f4c481f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 >> 172.21.15.28:0/2002354715 conn(0x7f4c48067d70 msgr2=0x7f4c480b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 shutdown_connections 2024-09-10T10:52:33.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.833+0000 7f4c4ee00640 1 -- 172.21.15.28:0/2002354715 wait complete. 2024-09-10T10:52:33.852 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:33.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.929+0000 7f5e6ad1ff40 1 Processor -- start 2024-09-10T10:52:33.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.929+0000 7f5e6ad1ff40 1 -- start start 2024-09-10T10:52:33.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6ccf8e0 0x555dc6c879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5e9c0 0x555dc6d52460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555dc6cd0530 con 0x555dc6d5b3e0 2024-09-10T10:52:33.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dc6c2db30 con 0x555dc6d5e9c0 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555dc6d5dee0 con 0x555dc6ccf8e0 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48000/0 (socket says 172.21.15.28:48000) 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 -- 172.21.15.28:0/2860358649 learned_addr learned my addr 172.21.15.28:0/2860358649 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e67400640 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6ccf8e0 0x555dc6c879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e67e00640 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5e9c0 0x555dc6d52460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 -- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6ccf8e0 msgr2=0x555dc6c879f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6ccf8e0 0x555dc6c879f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 -- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5e9c0 msgr2=0x555dc6d52460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5e9c0 0x555dc6d52460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.930+0000 7f5e66a00640 1 -- 172.21.15.28:0/2860358649 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc6d5d960 con 0x555dc6d5b3e0 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e66a00640 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009750 tx=0x7f5e54005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca3907bfa3aaaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e65600640 1 -- 172.21.15.28:0/2860358649 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54019070 con 0x555dc6d5b3e0 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e65600640 1 -- 172.21.15.28:0/2860358649 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e54008890 con 0x555dc6d5b3e0 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e65600640 1 -- 172.21.15.28:0/2860358649 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54008c80 con 0x555dc6d5b3e0 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/2860358649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 msgr2=0x555dc6cd19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.931+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009750 tx=0x7f5e54005a00 comp rx=0 tx=0).stop 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/2860358649 shutdown_connections 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5e9c0 0x555dc6d52460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/2860358649 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6ccf8e0 0x555dc6c879f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/2860358649 >> 172.21.15.28:0/2860358649 conn(0x555dc6cd40a0 msgr2=0x555dc6cd4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:33.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/2860358649 shutdown_connections 2024-09-10T10:52:33.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/2860358649 wait complete. 2024-09-10T10:52:33.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.932+0000 7f5e6ad1ff40 1 Processor -- start 2024-09-10T10:52:33.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 -- start start 2024-09-10T10:52:33.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6d5e9c0 0x555dc6d5c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555dc6c851a0 con 0x555dc6ccf8e0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555dc6d529a0 con 0x555dc6d5b3e0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.933+0000 7f5e6ad1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555dc6c85480 con 0x555dc6d5e9c0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48012/0 (socket says 172.21.15.28:48012) 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 learned_addr learned my addr 172.21.15.28:0/32087140 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6d5e9c0 0x555dc6d5c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6d5e9c0 msgr2=0x555dc6d5c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6d5e9c0 0x555dc6d5c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5b3e0 msgr2=0x555dc6cd19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dc6d4cba0 con 0x555dc6ccf8e0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555dc6d5b3e0 0x555dc6cd19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555dc6d5e9c0 0x555dc6d5c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800a5a0 tx=0x7f5e58002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.934+0000 7f5e5fe00640 1 -- 172.21.15.28:0/32087140 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58055070 con 0x555dc6ccf8e0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.935+0000 7f5e5fe00640 1 -- 172.21.15.28:0/32087140 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e58051930 con 0x555dc6ccf8e0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.935+0000 7f5e5fe00640 1 -- 172.21.15.28:0/32087140 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58051d20 con 0x555dc6ccf8e0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.935+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555dc6d4ce80 con 0x555dc6ccf8e0 2024-09-10T10:52:33.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.935+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dc6d4d440 con 0x555dc6ccf8e0 2024-09-10T10:52:33.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.936+0000 7f5e5fe00640 1 -- 172.21.15.28:0/32087140 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e58058070 con 0x555dc6ccf8e0 2024-09-10T10:52:33.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.937+0000 7f5e5fe00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 0x7f5e4806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.937+0000 7f5e5fe00640 1 -- 172.21.15.28:0/32087140 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f5e580cde70 con 0x555dc6ccf8e0 2024-09-10T10:52:33.940 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:33.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.938+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 0x7f5e4806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.938+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 0x555dc6d614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.938+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6d4ec50 con 0x555dc6d610f0 2024-09-10T10:52:33.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.938+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 0x555dc6d614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.938+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 0x7f5e4806ccd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009750 tx=0x7f5e54008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.939+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 0x555dc6d614d0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.941+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v116'474 uv448 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.941+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6d61a10 con 0x555dc6d610f0 2024-09-10T10:52:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.943+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v116'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.943+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6d61db0 con 0x555dc6d610f0 2024-09-10T10:52:33.945 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.945+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v116'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.945+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e015b0 con 0x555dc6d610f0 2024-09-10T10:52:33.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.946+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v116'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.946+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01950 con 0x555dc6d610f0 2024-09-10T10:52:33.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.948+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v116'478 uv478 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.948+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e005a0 con 0x555dc6d610f0 2024-09-10T10:52:33.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.950+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v116'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.950+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00990 con 0x555dc6d610f0 2024-09-10T10:52:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.951+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v116'480 uv454 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.951+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.953+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v116'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.953+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.956 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:33.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.955+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v116'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.955+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.956+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v116'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.957+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.958+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v116'484 uv484 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.958+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.960+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v116'485 uv485 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.960+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.962+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v116'486 uv461 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.962+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.963+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v116'487 uv487 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.963+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.965+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v116'488 uv488 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.965+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.967+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v116'489 uv489 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.967+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.968+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v116'490 uv490 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.968+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e00d30 con 0x555dc6d610f0 2024-09-10T10:52:33.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.970+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v116'491 uv491 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.970+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.972+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v116'492 uv492 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.972+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.974+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v116'493 uv467 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.974+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.975+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v116'494 uv494 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.975+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.977 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:33.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.977+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v116'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.977+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e02d00 con 0x555dc6d610f0 2024-09-10T10:52:33.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.978+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v116'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.978+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e02d00 con 0x555dc6d610f0 2024-09-10T10:52:33.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.980+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v116'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.980+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.982+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v116'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.982+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.983+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v116'499 uv473 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.983+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.985+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v116'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.985+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.987 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:33.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.986+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v116'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.986+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e02d00 con 0x555dc6d610f0 2024-09-10T10:52:33.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.988+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v116'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.988+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e02d00 con 0x555dc6d610f0 2024-09-10T10:52:33.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.990+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v116'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.990+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e01cf0 con 0x555dc6d610f0 2024-09-10T10:52:33.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.992+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v116'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6d610f0 2024-09-10T10:52:33.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.992+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x555dc6e02200 0x555dc6e045f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:33.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.992+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e04b30 con 0x555dc6e02200 2024-09-10T10:52:33.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.992+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x555dc6e02200 0x555dc6e045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:33.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.993+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x555dc6e02200 0x555dc6e045f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.994+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v116'352 uv351 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.994+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e04ed0 con 0x555dc6e02200 2024-09-10T10:52:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.996+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v116'353 uv353 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:33.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.996+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e05270 con 0x555dc6e02200 2024-09-10T10:52:33.998 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:33.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.997+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v116'354 uv354 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:33.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.997+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e05610 con 0x555dc6e02200 2024-09-10T10:52:33.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.998+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v116'355 uv355 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:33.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:33.998+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e059b0 con 0x555dc6e02200 2024-09-10T10:52:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.000+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v116'356 uv356 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.000+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e05d50 con 0x555dc6e02200 2024-09-10T10:52:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.001+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v116'357 uv339 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.001+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.002+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v116'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.002+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.004 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.004+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v116'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.004+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.005+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v116'360 uv360 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.005+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.006+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v116'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.007+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.008+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v116'362 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.008+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.009+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v116'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.009+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.012 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.011+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v116'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.011+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0a7e0 con 0x555dc6e02200 2024-09-10T10:52:34.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.013+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v116'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.013+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0a7e0 con 0x555dc6e02200 2024-09-10T10:52:34.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.014+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v116'366 uv366 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.014+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.015+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v116'367 uv350 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.015+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.016+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v116'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.016+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.018 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:34.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.018+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v116'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.018+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e07100 con 0x555dc6e02200 2024-09-10T10:52:34.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.019+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v116'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.019+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e07100 con 0x555dc6e02200 2024-09-10T10:52:34.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.020+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v116'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.020+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e060f0 con 0x555dc6e02200 2024-09-10T10:52:34.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.021+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v116'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e02200 2024-09-10T10:52:34.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.021+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x555dc6e06a10 0x555dc6e08e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.021+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e09340 con 0x555dc6e06a10 2024-09-10T10:52:34.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.021+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x555dc6e06a10 0x555dc6e08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.022+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x555dc6e06a10 0x555dc6e08e00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.024+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v116'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.024+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e096e0 con 0x555dc6e06a10 2024-09-10T10:52:34.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.025+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v116'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.025+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e09b00 con 0x555dc6e06a10 2024-09-10T10:52:34.027 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:34.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.026+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v116'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.026+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0b3b0 con 0x555dc6e06a10 2024-09-10T10:52:34.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.029+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v116'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.029+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0b750 con 0x555dc6e06a10 2024-09-10T10:52:34.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.031+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v116'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.031+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0a3a0 con 0x555dc6e06a10 2024-09-10T10:52:34.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.035+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v116'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.035+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555dc6e0aca0 0x555dc6e0db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.035+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0a3a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.035+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555dc6e0aca0 0x555dc6e0db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.035+0000 7f5e67e00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555dc6e0aca0 0x555dc6e0db00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.037+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v116'71 uv70 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.037+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0a3a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.038+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v116'72 uv72 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.038+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.040 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:34.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.039+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v116'73 uv73 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.039+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.040+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v116'74 uv74 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.040+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.042+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v116'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.042+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.043+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v116'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.043+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.045 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:34.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.044+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v116'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.044+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0f050 con 0x555dc6e06a10 2024-09-10T10:52:34.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.045+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v116'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.045+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0f050 con 0x555dc6e06a10 2024-09-10T10:52:34.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.046+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v116'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.046+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.047+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v116'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.047+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.049+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v116'171 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.049+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.050+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v116'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.050+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.061 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.051+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v116'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.052+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0f050 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.053+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v116'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.053+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0f050 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.054+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v116'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.054+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.055+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v116'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.055+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.056+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v116'177 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.056+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.057+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v116'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.058+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.063 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.059+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v116'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.059+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.060+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v116'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.060+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e0aca0 2024-09-10T10:52:34.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.062+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v116'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.062+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.063+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v116'76 uv75 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.063+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.064+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v116'77 uv77 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.064+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.066 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:34.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.065+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v116'78 uv78 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.065+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.066+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v116'79 uv79 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.067+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e0e040 con 0x555dc6e06a10 2024-09-10T10:52:34.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.068+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v116'80 uv80 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e06a10 2024-09-10T10:52:34.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.068+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555dc6e0e940 0x555dc6e10d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.068+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11270 con 0x555dc6e0e940 2024-09-10T10:52:34.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.068+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555dc6e0e940 0x555dc6e10d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.068+0000 7f5e67400640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555dc6e0e940 0x555dc6e10d30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.070+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v116'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.070+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e112b0 con 0x555dc6e0e940 2024-09-10T10:52:34.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.072+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v116'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.072+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.074 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:34.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.073+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v116'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.073+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e126e0 con 0x555dc6e0e940 2024-09-10T10:52:34.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.074+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v116'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.074+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e126e0 con 0x555dc6e0e940 2024-09-10T10:52:34.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.076+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v116'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.076+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.077+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v116'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.077+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.079+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v116'152 uv146 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.079+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.080+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v116'153 uv153 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.080+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.081+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v116'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.082 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:34.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.081+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.083+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v116'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.083+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.084+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v116'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.084+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.085+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v116'157 uv151 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.085+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.087+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v116'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.087+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.089 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:34.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.088+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v116'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.088+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e116d0 con 0x555dc6e0e940 2024-09-10T10:52:34.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.089+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v116'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.089+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e144e0 con 0x555dc6e0e940 2024-09-10T10:52:34.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.091+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v116'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.091+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.092+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v116'70 uv69 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.092+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.093+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v116'71 uv71 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.093+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.095 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:34.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.095+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v116'72 uv72 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.095+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.096+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v116'73 uv73 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.096+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.097+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v116'93 uv92 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.097+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.098+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v116'94 uv94 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.098+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.100 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:34.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.100+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v116'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.100+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b90 con 0x555dc6e0aca0 2024-09-10T10:52:34.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.101+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v116'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.101+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b90 con 0x555dc6e0aca0 2024-09-10T10:52:34.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.102+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v116'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.102+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e11b80 con 0x555dc6e0aca0 2024-09-10T10:52:34.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.103+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v116'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e600026a0 con 0x555dc6e0aca0 2024-09-10T10:52:34.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.103+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555dc6e12200 0x555dc6e125e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.103+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e12200 2024-09-10T10:52:34.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.103+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555dc6e12200 0x555dc6e125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.104+0000 7f5e66a00640 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555dc6e12200 0x555dc6e125e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.105+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v116'79 uv78 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e12200 2024-09-10T10:52:34.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.105+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e12200 2024-09-10T10:52:34.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.106+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v116'80 uv80 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e12200 2024-09-10T10:52:34.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.106+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e12200 2024-09-10T10:52:34.109 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:34.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.108+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v116'81 uv81 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e12200 2024-09-10T10:52:34.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.108+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e12200 2024-09-10T10:52:34.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.109+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v116'82 uv82 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e12200 2024-09-10T10:52:34.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.109+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e12200 2024-09-10T10:52:34.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.110+0000 7f5e66a00640 1 -- 172.21.15.28:0/32087140 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v116'83 uv83 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e5401c040 con 0x555dc6e12200 2024-09-10T10:52:34.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.110+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.111+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v116'192 uv186 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.111+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.112+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v116'193 uv193 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.112+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.114 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:34.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.114+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v116'194 uv194 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.114+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.115+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v116'195 uv195 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.115+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.116+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v116'196 uv196 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.116+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.117+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v116'197 uv191 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.117+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.118+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v116'198 uv198 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.118+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.120 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:34.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.120+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v116'199 uv199 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.120+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.121+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v116'200 uv200 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.121+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.122+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v116'201 uv201 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.122+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e12b20 con 0x555dc6e0e940 2024-09-10T10:52:34.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.123+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v116'202 uv202 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.123+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x555dc6e144f0 con 0x555dc6e0e940 2024-09-10T10:52:34.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.124+0000 7f5e67400640 1 -- 172.21.15.28:0/32087140 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v116'203 uv203 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e5806b020 con 0x555dc6e0e940 2024-09-10T10:52:34.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x555dc6e06a10 msgr2=0x555dc6e08e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x555dc6e06a10 0x555dc6e08e00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555dc6e0e940 msgr2=0x555dc6e10d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555dc6e0e940 0x555dc6e10d30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555dc6e12200 msgr2=0x555dc6e125e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x555dc6e12200 0x555dc6e125e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555dc6e0aca0 msgr2=0x555dc6e0db00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555dc6e0aca0 0x555dc6e0db00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x555dc6e02200 msgr2=0x555dc6e045f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 reap_dead start 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x555dc6e02200 0x555dc6e045f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e67e00640 1 -- 172.21.15.28:0/32087140 reap_dead start 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 msgr2=0x555dc6d614d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 0x555dc6d614d0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 msgr2=0x7f5e4806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 0x7f5e4806ccd0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009750 tx=0x7f5e54008000 comp rx=0 tx=0).stop 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 msgr2=0x555dc6d3e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800a5a0 tx=0x7f5e58002fb0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 shutdown_connections 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x555dc6d610f0 0x555dc6d614d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e4806a810 0x7f5e4806ccd0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 --2- 172.21.15.28:0/32087140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555dc6ccf8e0 0x555dc6d3e3b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 >> 172.21.15.28:0/32087140 conn(0x555dc6cd40a0 msgr2=0x555dc6cd4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 shutdown_connections 2024-09-10T10:52:34.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.125+0000 7f5e6ad1ff40 1 -- 172.21.15.28:0/32087140 wait complete. 2024-09-10T10:52:34.130 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000000 - 2024-09-10T10:52:34.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.246+0000 7f1daebd9f40 1 Processor -- start 2024-09-10T10:52:34.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.246+0000 7f1daebd9f40 1 -- start start 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f51380f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 0x55f51380bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f51380edc0 con 0x55f51389b4b0 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f51376db30 con 0x55f51380f2d0 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f51380e1f0 con 0x55f5137c9cb0 2024-09-10T10:52:34.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dac800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dac800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48034/0 (socket says 172.21.15.28:48034) 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dac800640 1 -- 172.21.15.28:0/2525243264 learned_addr learned my addr 172.21.15.28:0/2525243264 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dabe00640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f51380f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dab400640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 0x55f51380bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dac800640 1 -- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 msgr2=0x55f51380f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.247+0000 7f1dac800640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f51380f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dac800640 1 -- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 msgr2=0x55f51380bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dac800640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 0x55f51380bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dac800640 1 -- 172.21.15.28:0/2525243264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f513895140 con 0x55f51389b4b0 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dabe00640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f51380f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:34.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dab400640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 0x55f51380bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:34.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1dac800640 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1da40027a0 tx=0x7f1da4002c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24671900c7992448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1daa000640 1 -- 172.21.15.28:0/2525243264 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4016020 con 0x55f51389b4b0 2024-09-10T10:52:34.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1daa000640 1 -- 172.21.15.28:0/2525243264 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da400d9c0 con 0x55f51389b4b0 2024-09-10T10:52:34.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.248+0000 7f1daa000640 1 -- 172.21.15.28:0/2525243264 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da400ddb0 con 0x55f51389b4b0 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 -- 172.21.15.28:0/2525243264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 msgr2=0x55f51389c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f1da40027a0 tx=0x7f1da4002c60 comp rx=0 tx=0).stop 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 -- 172.21.15.28:0/2525243264 shutdown_connections 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51389b4b0 0x55f51389c240 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51380f2d0 0x55f51380bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/2525243264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f51380f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 -- 172.21.15.28:0/2525243264 >> 172.21.15.28:0/2525243264 conn(0x55f5137f3b90 msgr2=0x55f5137f3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 -- 172.21.15.28:0/2525243264 shutdown_connections 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 -- 172.21.15.28:0/2525243264 wait complete. 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.249+0000 7f1daebd9f40 1 Processor -- start 2024-09-10T10:52:34.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 -- start start 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f513897e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 0x55f51388e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51389b4b0 0x55f5137c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f513895400 con 0x55f51380f2d0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f513895880 con 0x55f51389b4b0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.250+0000 7f1daebd9f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f5137f6340 con 0x55f5137c9cb0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 0x55f51388e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 0x55f51388e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48044/0 (socket says 172.21.15.28:48044) 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 -- 172.21.15.28:0/1297824684 learned_addr learned my addr 172.21.15.28:0/1297824684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dabe00640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f513897e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dac800640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51389b4b0 0x55f5137c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 -- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 msgr2=0x55f513897e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f513897e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 -- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51389b4b0 msgr2=0x55f5137c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f51389b4b0 0x55f5137c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 -- 172.21.15.28:0/1297824684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5137c7b80 con 0x55f51380f2d0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dabe00640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f5137c9cb0 0x55f513897e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.251+0000 7f1dab400640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 0x55f51388e810 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1d98045b80 tx=0x7f1d98002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.252+0000 7f1da8c00640 1 -- 172.21.15.28:0/1297824684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98055040 con 0x55f51380f2d0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.252+0000 7f1da8c00640 1 -- 172.21.15.28:0/1297824684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d98051890 con 0x55f51380f2d0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.252+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f51389a3b0 con 0x55f51380f2d0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.252+0000 7f1da8c00640 1 -- 172.21.15.28:0/1297824684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98051c40 con 0x55f51380f2d0 2024-09-10T10:52:34.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.252+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51388ed50 con 0x55f51380f2d0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.254+0000 7f1da8c00640 1 -- 172.21.15.28:0/1297824684 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1d98058070 con 0x55f51380f2d0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1da8c00640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8c06a840 0x7f1d8c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1da8c00640 1 -- 172.21.15.28:0/1297824684 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f1d980cddf0 con 0x55f51380f2d0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1dabe00640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8c06a840 0x7f1d8c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f513894b30 0x55f513892ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116) -- 0x55f5137c98e0 con 0x55f513894b30 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.255+0000 7f1dac800640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f513894b30 0x55f513892ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.256+0000 7f1dabe00640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8c06a840 0x7f1d8c06cd00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c001630 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.256+0000 7f1dac800640 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f513894b30 0x55f513892ad0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1dac800640 1 -- 172.21.15.28:0/1297824684 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [writefull 0~80] v116'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1da4012470 con 0x55f513894b30 2024-09-10T10:52:34.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f513894b30 msgr2=0x55f513892ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f513894b30 0x55f513892ad0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8c06a840 msgr2=0x7f1d8c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1d8c06a840 0x7f1d8c06cd00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c001630 comp rx=0 tx=0).stop 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 msgr2=0x55f51388e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 --2- 172.21.15.28:0/1297824684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f51380f2d0 0x55f51388e810 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1d98045b80 tx=0x7f1d98002fb0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1dac800640 1 -- 172.21.15.28:0/1297824684 reap_dead start 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 shutdown_connections 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 >> 172.21.15.28:0/1297824684 conn(0x55f5137f3b90 msgr2=0x55f5137f3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 shutdown_connections 2024-09-10T10:52:34.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.258+0000 7f1daebd9f40 1 -- 172.21.15.28:0/1297824684 wait complete. 2024-09-10T10:52:34.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.463+0000 7f2682e00640 1 Processor -- start 2024-09-10T10:52:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.463+0000 7f2682e00640 1 -- start start 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f267c15e900 0x7f267c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c0b4a20 0x7f267c0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f267c072050 0x7f267c0724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0b4620 con 0x7f267c072050 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0ce100 con 0x7f267c15e900 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2682e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0ce3a0 con 0x7f267c0b4a20 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.464+0000 7f2680e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f267c072050 0x7f267c0724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f267c072050 0x7f267c0724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48046/0 (socket says 172.21.15.28:48046) 2024-09-10T10:52:34.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 -- 172.21.15.28:0/1849374337 learned_addr learned my addr 172.21.15.28:0/1849374337 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f267b400640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c0b4a20 0x7f267c0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f267be00640 1 --2- 172.21.15.28:0/1849374337 >> v2:172.21.15.138:3300/0 conn(0x7f267c15e900 0x7f267c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 -- 172.21.15.28:0/1849374337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c0b4a20 msgr2=0x7f267c0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c0b4a20 0x7f267c0ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 -- 172.21.15.28:0/1849374337 >> v2:172.21.15.138:3300/0 conn(0x7f267c15e900 msgr2=0x7f267c0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 --2- 172.21.15.28:0/1849374337 >> v2:172.21.15.138:3300/0 conn(0x7f267c15e900 0x7f267c0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.465+0000 7f2680e00640 1 -- 172.21.15.28:0/1849374337 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0ce680 con 0x7f267c072050 2024-09-10T10:52:34.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.466+0000 7f2680e00640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c072050 0x7f267c0724e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f267c0d0d60 tx=0x7f267402c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75f50e2c7f453e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.466+0000 7f267aa00640 1 -- 172.21.15.28:0/1849374337 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674039070 con 0x7f267c072050 2024-09-10T10:52:34.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.466+0000 7f267aa00640 1 -- 172.21.15.28:0/1849374337 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674009600 con 0x7f267c072050 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.466+0000 7f267aa00640 1 -- 172.21.15.28:0/1849374337 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740099f0 con 0x7f267c072050 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 -- 172.21.15.28:0/1849374337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c072050 msgr2=0x7f267c0724e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c072050 0x7f267c0724e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f267c0d0d60 tx=0x7f267402c810 comp rx=0 tx=0).stop 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 -- 172.21.15.28:0/1849374337 shutdown_connections 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c072050 0x7f267c0724e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 --2- 172.21.15.28:0/1849374337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c0b4a20 0x7f267c0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 --2- 172.21.15.28:0/1849374337 >> v2:172.21.15.138:3300/0 conn(0x7f267c15e900 0x7f267c0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 -- 172.21.15.28:0/1849374337 >> 172.21.15.28:0/1849374337 conn(0x7f267c06bdf0 msgr2=0x7f267c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 -- 172.21.15.28:0/1849374337 shutdown_connections 2024-09-10T10:52:34.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.467+0000 7f2682e00640 1 -- 172.21.15.28:0/1849374337 wait complete. 2024-09-10T10:52:34.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.469+0000 7f2682e00640 1 Processor -- start 2024-09-10T10:52:34.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.469+0000 7f2682e00640 1 -- start start 2024-09-10T10:52:34.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 0x7f267c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 0x7f267c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c15e900 0x7f267c06f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0b4e00 con 0x7f267c0b4a20 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0ad560 con 0x7f267c072050 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2682e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0cee80 con 0x7f267c15e900 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 0x7f267c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 0x7f267c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 0x7f267c156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48054/0 (socket says 172.21.15.28:48054) 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267b400640 1 -- 172.21.15.28:0/2699172887 learned_addr learned my addr 172.21.15.28:0/2699172887 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f2680e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c15e900 0x7f267c06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 -- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c15e900 msgr2=0x7f267c06f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c15e900 0x7f267c06f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 -- 172.21.15.28:0/2699172887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 msgr2=0x7f267c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 0x7f267c156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.470+0000 7f267be00640 1 -- 172.21.15.28:0/2699172887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c154b10 con 0x7f267c072050 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f267be00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 0x7f267c1545d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e8a0 tx=0x7f266c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c017070 con 0x7f267c072050 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266c00c940 con 0x7f267c072050 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c09a710 con 0x7f267c072050 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c009e60 con 0x7f267c072050 2024-09-10T10:52:34.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.471+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c09acd0 con 0x7f267c072050 2024-09-10T10:52:34.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.473+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c0b3590 con 0x7f267c072050 2024-09-10T10:52:34.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.473+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f266c01a070 con 0x7f267c072050 2024-09-10T10:52:34.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.474+0000 7f2673e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 0x7f265c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:34.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.474+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 7680+0+0 (secure 0 0 0) 0x7f266c08b4d0 con 0x7f267c072050 2024-09-10T10:52:34.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.475+0000 7f267b400640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 0x7f265c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:34.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.475+0000 7f267b400640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 0x7f265c06e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f267c153c00 tx=0x7f2668009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:34.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.479+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f267c154b10 con 0x7f267c072050 2024-09-10T10:52:34.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.700+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f267c0ad020 con 0x7f267c072050 2024-09-10T10:52:34.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.845+0000 7f2673e00640 1 -- 172.21.15.28:0/2699172887 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v181) ==== 162+0+0 (secure 0 0 0) 0x7f266c058f70 con 0x7f267c072050 2024-09-10T10:52:34.846 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:34.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 msgr2=0x7f265c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 0x7f265c06e2f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f267c153c00 tx=0x7f2668009190 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 msgr2=0x7f267c1545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 0x7f267c1545d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f266c00e8a0 tx=0x7f266c00ed60 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 shutdown_connections 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f265c06be30 0x7f265c06e2f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f267c15e900 0x7f267c06f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f267c0b4a20 0x7f267c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 --2- 172.21.15.28:0/2699172887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f267c072050 0x7f267c1545d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 >> 172.21.15.28:0/2699172887 conn(0x7f267c06bdf0 msgr2=0x7f267c0b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.847+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 shutdown_connections 2024-09-10T10:52:34.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:34.848+0000 7f2682e00640 1 -- 172.21.15.28:0/2699172887 wait complete. 2024-09-10T10:52:34.853 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:52:34.852+0000 7f5a35000640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-10T10:52:34.853 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:34.853+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:34.853 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:34.853+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:34.866 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:35.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.022+0000 7fef51400640 1 Processor -- start 2024-09-10T10:52:35.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.022+0000 7fef51400640 1 -- start start 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 0x7fef4c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c0b37d0 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c0cf330 con 0x7fef4c0d1f90 2024-09-10T10:52:35.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef51400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0cf000 con 0x7fef4c15e660 2024-09-10T10:52:35.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef4be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef4be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48064/0 (socket says 172.21.15.28:48064) 2024-09-10T10:52:35.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.023+0000 7fef4be00640 1 -- 172.21.15.28:0/643442804 learned_addr learned my addr 172.21.15.28:0/643442804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:35.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef50a00640 1 --2- 172.21.15.28:0/643442804 >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 0x7fef4c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4b400640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4be00640 1 -- 172.21.15.28:0/643442804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 msgr2=0x7fef4c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4be00640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4be00640 1 -- 172.21.15.28:0/643442804 >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 msgr2=0x7fef4c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4be00640 1 --2- 172.21.15.28:0/643442804 >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 0x7fef4c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef4be00640 1 -- 172.21.15.28:0/643442804 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c0d16f0 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.024+0000 7fef50a00640 1 --2- 172.21.15.28:0/643442804 >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 0x7fef4c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:35.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.025+0000 7fef4be00640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0cb420 tx=0x7fef3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59848079ae0cf3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.025+0000 7fef4aa00640 1 -- 172.21.15.28:0/643442804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c037070 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.025+0000 7fef4aa00640 1 -- 172.21.15.28:0/643442804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c032440 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.025+0000 7fef4aa00640 1 -- 172.21.15.28:0/643442804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c032810 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 -- 172.21.15.28:0/643442804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d0dd0 msgr2=0x7fef4c0d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0cb420 tx=0x7fef3c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 -- 172.21.15.28:0/643442804 shutdown_connections 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 --2- 172.21.15.28:0/643442804 >> v2:172.21.15.138:3300/0 conn(0x7fef4c0d1f90 0x7fef4c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 --2- 172.21.15.28:0/643442804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c0d11b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 -- 172.21.15.28:0/643442804 >> 172.21.15.28:0/643442804 conn(0x7fef4c08bfc0 msgr2=0x7fef4c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 -- 172.21.15.28:0/643442804 shutdown_connections 2024-09-10T10:52:35.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.026+0000 7fef51400640 1 -- 172.21.15.28:0/643442804 wait complete. 2024-09-10T10:52:35.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.027+0000 7fef51400640 1 Processor -- start 2024-09-10T10:52:35.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.028+0000 7fef51400640 1 -- start start 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.028+0000 7fef51400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.028+0000 7fef51400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef51400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef51400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0d2f40 con 0x7fef4c0d1f90 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef51400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0d2c60 con 0x7fef4c0d0dd0 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef51400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0d3220 con 0x7fef4c15e660 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48068/0 (socket says 172.21.15.28:48068) 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 -- 172.21.15.28:0/3022854742 learned_addr learned my addr 172.21.15.28:0/3022854742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef50a00640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4be00640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 -- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 msgr2=0x7fef4c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 -- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef4c0d0dd0 msgr2=0x7fef4c157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.029+0000 7fef4b400640 1 -- 172.21.15.28:0/3022854742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c1e4fa0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef4b400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d5d0 tx=0x7fef3800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38004240 con 0x7fef4c0d1f90 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef380043e0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38003830 con 0x7fef4c0d1f90 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1e5280 con 0x7fef4c0d1f90 2024-09-10T10:52:35.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.030+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1e5840 con 0x7fef4c0d1f90 2024-09-10T10:52:35.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.031+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c0d2440 con 0x7fef4c0d1f90 2024-09-10T10:52:35.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.032+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fef380039d0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.033+0000 7fef43e00640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 0x7fef2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.033+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7fef3808b820 con 0x7fef4c0d1f90 2024-09-10T10:52:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.034+0000 7fef4be00640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 0x7fef2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.034+0000 7fef4be00640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 0x7fef2c06e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.037+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fef380592e0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.262+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef4c0ce7f0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.262+0000 7fef43e00640 1 -- 172.21.15.28:0/3022854742 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) ==== 95+0+3566 (secure 0 0 0) 0x7fef3805d5e0 con 0x7fef4c0d1f90 2024-09-10T10:52:35.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:35.263 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":183,"btime":"2024-09-10T10:52:34:853648+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:34.853596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:35.264 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 183 2024-09-10T10:52:35.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 msgr2=0x7fef2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 0x7fef2c06e2f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fef3c034000 tx=0x7fef3c02e000 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 msgr2=0x7fef4c1e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d5d0 tx=0x7fef3800da90 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 shutdown_connections 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fef2c06be30 0x7fef2c06e2f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef4c15e660 0x7fef4c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef4c0d1f90 0x7fef4c1e6040 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 --2- 172.21.15.28:0/3022854742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef4c0d0dd0 0x7fef4c157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.264+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 >> 172.21.15.28:0/3022854742 conn(0x7fef4c08bfc0 msgr2=0x7fef4c06db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.265+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 shutdown_connections 2024-09-10T10:52:35.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.265+0000 7fef51400640 1 -- 172.21.15.28:0/3022854742 wait complete. 2024-09-10T10:52:35.282 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:35.282 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:35.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.436+0000 7fbaa7c00640 1 Processor -- start 2024-09-10T10:52:35.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.436+0000 7fbaa7c00640 1 -- start start 2024-09-10T10:52:35.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.437+0000 7fbaa7c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbaa00afd70 0x7fbaa0160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.437+0000 7fbaa7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00b5f80 0x7fbaa00b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa7c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaa00b68e0 con 0x7fbaa006e7d0 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa7c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbaa0160d60 con 0x7fbaa00afd70 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa006fdb0 con 0x7fbaa00b5f80 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbaa00afd70 0x7fbaa0160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbaa006e7d0 0x7fbaa006ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47178/0 (socket says 172.21.15.28:47178) 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbaa00afd70 0x7fbaa0160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48078/0 (socket says 172.21.15.28:48078) 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5c00640 1 -- 172.21.15.28:0/1438991218 learned_addr learned my addr 172.21.15.28:0/1438991218 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fba9fe00640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00b5f80 0x7fbaa00b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5200640 1 -- 172.21.15.28:0/1438991218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00b5f80 msgr2=0x7fbaa00b63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5200640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00b5f80 0x7fbaa00b63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fbaa5200640 1 -- 172.21.15.28:0/1438991218 >> v1:172.21.15.28:6789/0 conn(0x7fbaa006e7d0 legacy=0x7fbaa006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.438+0000 7fba9f400640 1 -- 172.21.15.28:0/1438991218 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2870099182 0 0) 0x7fbaa00b68e0 con 0x7fbaa006e7d0 2024-09-10T10:52:35.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.439+0000 7fbaa5200640 1 -- 172.21.15.28:0/1438991218 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa0070090 con 0x7fbaa00afd70 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.439+0000 7fbaa5200640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa00afd70 0x7fbaa0160820 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fba940097a0 tx=0x7fba9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5500285fa6b93d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fba9f400640 1 -- 172.21.15.28:0/1438991218 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94037070 con 0x7fbaa00afd70 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fba9f400640 1 -- 172.21.15.28:0/1438991218 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba94032440 con 0x7fbaa00afd70 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fba9f400640 1 -- 172.21.15.28:0/1438991218 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94032810 con 0x7fbaa00afd70 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 -- 172.21.15.28:0/1438991218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa00afd70 msgr2=0x7fbaa0160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa00afd70 0x7fbaa0160820 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fba940097a0 tx=0x7fba9402a180 comp rx=0 tx=0).stop 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 -- 172.21.15.28:0/1438991218 shutdown_connections 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00b5f80 0x7fbaa00b63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/1438991218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa00afd70 0x7fbaa0160820 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 -- 172.21.15.28:0/1438991218 >> 172.21.15.28:0/1438991218 conn(0x7fbaa006bc10 msgr2=0x7fbaa006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.440+0000 7fbaa7c00640 1 -- 172.21.15.28:0/1438991218 shutdown_connections 2024-09-10T10:52:35.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.441+0000 7fbaa7c00640 1 -- 172.21.15.28:0/1438991218 wait complete. 2024-09-10T10:52:35.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.442+0000 7fbaa7c00640 1 Processor -- start 2024-09-10T10:52:35.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.442+0000 7fbaa7c00640 1 -- start start 2024-09-10T10:52:35.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00afd70 0x7fbaa0144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 0x7fbaa0145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa006fa80 con 0x7fbaa006e7d0 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa0070650 con 0x7fbaa00b5f80 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbaa00b0150 con 0x7fbaa00afd70 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48084/0 (socket says 172.21.15.28:48084) 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fbaa5200640 1 -- 172.21.15.28:0/2961554687 learned_addr learned my addr 172.21.15.28:0/2961554687 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.443+0000 7fba9fe00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00afd70 0x7fbaa0144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 0x7fbaa0145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 -- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00afd70 msgr2=0x7fbaa0144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00afd70 0x7fbaa0144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 -- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 msgr2=0x7fbaa0145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 0x7fbaa0145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 -- 172.21.15.28:0/2961554687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa015d190 con 0x7fbaa006e7d0 2024-09-10T10:52:35.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 0x7fbaa0145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa5200640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fba94035040 tx=0x7fba940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94037040 con 0x7fbaa006e7d0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba94030770 con 0x7fbaa006e7d0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba94030b80 con 0x7fbaa006e7d0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.444+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa015d470 con 0x7fbaa006e7d0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.445+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa0145bd0 con 0x7fbaa006e7d0 2024-09-10T10:52:35.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.446+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa015f9f0 con 0x7fbaa006e7d0 2024-09-10T10:52:35.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.447+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba94044020 con 0x7fbaa006e7d0 2024-09-10T10:52:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.448+0000 7fba9cc00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 0x7fba8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.448+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7fba940aba60 con 0x7fbaa006e7d0 2024-09-10T10:52:35.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.448+0000 7fba9fe00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 0x7fba8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.449+0000 7fba9fe00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 0x7fba8006e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00b4e20 tx=0x7fba8c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.452+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbaa015f9f0 con 0x7fbaa006e7d0 2024-09-10T10:52:35.585 INFO:tasks.ceph.mds.c.smithi028.stderr: -35> 2024-09-10T10:52:30.831+0000 7fee85200640 -1 mds.0.cache.ino(0x1) Corrupt inode 0x1: End of buffer [buffer:2] 2024-09-10T10:52:35.585 INFO:tasks.ceph.mds.c.smithi028.stderr: -33> 2024-09-10T10:52:30.831+0000 7fee85200640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:52:35.624 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:52:35.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.676+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaa0160060 con 0x7fbaa006e7d0 2024-09-10T10:52:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.677+0000 7fba9cc00640 1 -- 172.21.15.28:0/2961554687 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) ==== 95+0+3566 (secure 0 0 0) 0x7fba94032e20 con 0x7fbaa006e7d0 2024-09-10T10:52:35.679 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:35.679 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":183,"btime":"2024-09-10T10:52:34:853648+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:34.853596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 183 2024-09-10T10:52:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 msgr2=0x7fba8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 0x7fba8006e320 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00b4e20 tx=0x7fba8c008040 comp rx=0 tx=0).stop 2024-09-10T10:52:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 msgr2=0x7fbaa00b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fba94035040 tx=0x7fba940046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 shutdown_connections 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba8006be60 0x7fba8006e320 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa00b5f80 0x7fbaa0145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbaa00afd70 0x7fbaa0144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 --2- 172.21.15.28:0/2961554687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbaa006e7d0 0x7fbaa00b3fe0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 >> 172.21.15.28:0/2961554687 conn(0x7fbaa006bc10 msgr2=0x7fbaa00b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 shutdown_connections 2024-09-10T10:52:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.679+0000 7fbaa7c00640 1 -- 172.21.15.28:0/2961554687 wait complete. 2024-09-10T10:52:35.697 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:35.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:35.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.852+0000 7f0115a00640 1 Processor -- start 2024-09-10T10:52:35.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.852+0000 7f0115a00640 1 -- start start 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0115a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0110072700 0x7f011015d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0115a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0110146140 con 0x7f0110072de0 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0115a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01100cd740 con 0x7f0110072320 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0115a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f01100cda20 con 0x7f0110072700 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0106000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0110072de0 0x7f011015d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47184/0 (socket says 172.21.15.28:47184) 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f0106000640 1 -- 172.21.15.28:0/2350867083 learned_addr learned my addr 172.21.15.28:0/2350867083 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f010f400640 1 --2- 172.21.15.28:0/2350867083 >> v2:172.21.15.138:3300/0 conn(0x7f0110072700 0x7f011015d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.854+0000 7f010e000640 1 -- 172.21.15.28:0/2350867083 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520483750 0 0) 0x7f0110146140 con 0x7f0110072de0 2024-09-10T10:52:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.855+0000 7f010e000640 1 -- 172.21.15.28:0/2350867083 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00f4003520 con 0x7f0110072de0 2024-09-10T10:52:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.855+0000 7f010f400640 1 -- 172.21.15.28:0/2350867083 >> v1:172.21.15.138:6789/0 conn(0x7f0110072320 legacy=0x7f01100cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.855+0000 7f010f400640 1 -- 172.21.15.28:0/2350867083 >> v1:172.21.15.28:6789/0 conn(0x7f0110072de0 legacy=0x7f011015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.855+0000 7f010f400640 1 -- 172.21.15.28:0/2350867083 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01100cdd00 con 0x7f0110072700 2024-09-10T10:52:35.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.856+0000 7f010f400640 1 --2- 172.21.15.28:0/2350867083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072700 0x7f011015d8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d90 tx=0x7f010802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7926a22cca29ac86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.858+0000 7f010e000640 1 -- 172.21.15.28:0/2350867083 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108007740 con 0x7f0110072700 2024-09-10T10:52:35.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.858+0000 7f010e000640 1 -- 172.21.15.28:0/2350867083 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01080078e0 con 0x7f0110072700 2024-09-10T10:52:35.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.858+0000 7f010e000640 1 -- 172.21.15.28:0/2350867083 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0108007cb0 con 0x7f0110072700 2024-09-10T10:52:35.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 -- 172.21.15.28:0/2350867083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072700 msgr2=0x7f011015d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 --2- 172.21.15.28:0/2350867083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072700 0x7f011015d8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0108009d90 tx=0x7f010802a660 comp rx=0 tx=0).stop 2024-09-10T10:52:35.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 -- 172.21.15.28:0/2350867083 shutdown_connections 2024-09-10T10:52:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 --2- 172.21.15.28:0/2350867083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072700 0x7f011015d8f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 -- 172.21.15.28:0/2350867083 >> 172.21.15.28:0/2350867083 conn(0x7f0110067d70 msgr2=0x7f011006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 -- 172.21.15.28:0/2350867083 shutdown_connections 2024-09-10T10:52:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.859+0000 7f0115a00640 1 -- 172.21.15.28:0/2350867083 wait complete. 2024-09-10T10:52:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.861+0000 7f0115a00640 1 Processor -- start 2024-09-10T10:52:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.861+0000 7f0115a00640 1 -- start start 2024-09-10T10:52:35.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0110072320 0x7f01101f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072de0 0x7f011013ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f011015dfb0 con 0x7f0110072700 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f011015de30 con 0x7f0110072de0 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0115a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f011015e130 con 0x7f0110072320 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0106000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f010ea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0110072320 0x7f01101f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0106000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48112/0 (socket says 172.21.15.28:48112) 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f0106000640 1 -- 172.21.15.28:0/2478799558 learned_addr learned my addr 172.21.15.28:0/2478799558 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.862+0000 7f010f400640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072de0 0x7f011013ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 -- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0110072320 msgr2=0x7f01101f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0110072320 0x7f01101f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 -- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072de0 msgr2=0x7f011013ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072de0 0x7f011013ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 -- 172.21.15.28:0/2478799558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01101f19c0 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0106000640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c410 tx=0x7f00f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8017070 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f8005ce0 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80072f0 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.863+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011013d900 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.864+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101f1f20 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.865+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01100cce60 con 0x7f0110072700 2024-09-10T10:52:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.865+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00f801a070 con 0x7f0110072700 2024-09-10T10:52:35.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.866+0000 7f0107400640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 0x7f00ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:35.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.866+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7f00f808aa80 con 0x7f0110072700 2024-09-10T10:52:35.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.867+0000 7f010ea00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 0x7f00ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:35.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.867+0000 7f010ea00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 0x7f00ec06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0100001780 tx=0x7f0100009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:35.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:35.871+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00f808ce20 con 0x7f0110072700 2024-09-10T10:52:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.092+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f011006d090 con 0x7f0110072700 2024-09-10T10:52:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.092+0000 7f0107400640 1 -- 172.21.15.28:0/2478799558 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 184 v184) ==== 95+0+4392 (secure 0 0 0) 0x7f01101f19c0 con 0x7f0110072700 2024-09-10T10:52:36.094 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:36.095 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":184,"btime":"2024-09-10T10:52:35:855196+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":183,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:34.853596+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 184 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.094+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 msgr2=0x7f00ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.094+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 0x7f00ec06e2f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0100001780 tx=0x7f0100009040 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.094+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 msgr2=0x7f011013c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.094+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c410 tx=0x7f00f800c8d0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 shutdown_connections 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f00ec06be30 0x7f00ec06e2f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0110072de0 0x7f011013ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0110072700 0x7f011013c070 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 --2- 172.21.15.28:0/2478799558 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0110072320 0x7f01101f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 >> 172.21.15.28:0/2478799558 conn(0x7f0110067d70 msgr2=0x7f011015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:36.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 shutdown_connections 2024-09-10T10:52:36.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.095+0000 7f0115a00640 1 -- 172.21.15.28:0/2478799558 wait complete. 2024-09-10T10:52:36.112 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000000' led to DAMAGED state 2024-09-10T10:52:36.112 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 200.00000001' 2024-09-10T10:52:36.112 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:36.112 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:36.148 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:36.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:36.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:36.148 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:36.223 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:36.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:36.224 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:36.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.419+0000 7efcd5200640 1 Processor -- start 2024-09-10T10:52:36.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.419+0000 7efcd5200640 1 -- start start 2024-09-10T10:52:36.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcd5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efcd0147230 0x7efcd006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcd5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcd00727c0 0x7efcd00aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcd5200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd006f480 con 0x7efcd0147e90 2024-09-10T10:52:36.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcd5200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efcd00b34d0 con 0x7efcd00727c0 2024-09-10T10:52:36.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcd5200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efcd00ae9e0 con 0x7efcd0147230 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcd00727c0 0x7efcd00aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcd00727c0 0x7efcd00aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48126/0 (socket says 172.21.15.28:48126) 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 -- 172.21.15.28:0/2395972050 learned_addr learned my addr 172.21.15.28:0/2395972050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efccde00640 1 --2- 172.21.15.28:0/2395972050 >> v2:172.21.15.138:3300/0 conn(0x7efcd0147230 0x7efcd006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 -- 172.21.15.28:0/2395972050 >> v2:172.21.15.138:3300/0 conn(0x7efcd0147230 msgr2=0x7efcd006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 --2- 172.21.15.28:0/2395972050 >> v2:172.21.15.138:3300/0 conn(0x7efcd0147230 0x7efcd006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 -- 172.21.15.28:0/2395972050 >> v1:172.21.15.138:6789/0 conn(0x7efcd0147e90 legacy=0x7efcd00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.421+0000 7efcce800640 1 -- 172.21.15.28:0/2395972050 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd006e900 con 0x7efcd00727c0 2024-09-10T10:52:36.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.422+0000 7efcce800640 1 --2- 172.21.15.28:0/2395972050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd00727c0 0x7efcd00aee10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efcd0070280 tx=0x7efcc802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c0786ea548d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:36.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efccd400640 1 -- 172.21.15.28:0/2395972050 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8036070 con 0x7efcd00727c0 2024-09-10T10:52:36.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efccd400640 1 -- 172.21.15.28:0/2395972050 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc8031600 con 0x7efcd00727c0 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efccd400640 1 -- 172.21.15.28:0/2395972050 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80319d0 con 0x7efcd00727c0 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 -- 172.21.15.28:0/2395972050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd00727c0 msgr2=0x7efcd00aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 --2- 172.21.15.28:0/2395972050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd00727c0 0x7efcd00aee10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efcd0070280 tx=0x7efcc802a730 comp rx=0 tx=0).stop 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 -- 172.21.15.28:0/2395972050 shutdown_connections 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 --2- 172.21.15.28:0/2395972050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd00727c0 0x7efcd00aee10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 --2- 172.21.15.28:0/2395972050 >> v2:172.21.15.138:3300/0 conn(0x7efcd0147230 0x7efcd006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 -- 172.21.15.28:0/2395972050 >> 172.21.15.28:0/2395972050 conn(0x7efcd0067d70 msgr2=0x7efcd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 -- 172.21.15.28:0/2395972050 shutdown_connections 2024-09-10T10:52:36.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.423+0000 7efcd5200640 1 -- 172.21.15.28:0/2395972050 wait complete. 2024-09-10T10:52:36.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.425+0000 7efcd5200640 1 Processor -- start 2024-09-10T10:52:36.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.425+0000 7efcd5200640 1 -- start start 2024-09-10T10:52:36.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcd00727c0 0x7efcd015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 0x7efcd01f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efcd006f1a0 con 0x7efcd0147e90 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcd006eec0 con 0x7efcd0147230 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcd5200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efcd006e5f0 con 0x7efcd00727c0 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48128/0 (socket says 172.21.15.28:48128) 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcce800640 1 -- 172.21.15.28:0/1023199569 learned_addr learned my addr 172.21.15.28:0/1023199569 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efccde00640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcd00727c0 0x7efcd015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.426+0000 7efcc7e00640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 0x7efcd01f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 -- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcd00727c0 msgr2=0x7efcd015dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcd00727c0 0x7efcd015dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 -- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 msgr2=0x7efcd01f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 0x7efcd01f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 -- 172.21.15.28:0/1023199569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd01f4ce0 con 0x7efcd0147e90 2024-09-10T10:52:36.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcc7e00640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 0x7efcd01f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcce800640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f40 tx=0x7efcc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8036050 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc8042070 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8009510 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd01f4fc0 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.427+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd015ea50 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.428+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd0147610 con 0x7efcd0147e90 2024-09-10T10:52:36.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.429+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcc8007bf0 con 0x7efcd0147e90 2024-09-10T10:52:36.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.430+0000 7efcc6000640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 0x7efcac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:36.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.430+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7efcc80ac4b0 con 0x7efcd0147e90 2024-09-10T10:52:36.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.431+0000 7efccde00640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 0x7efcac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:36.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.431+0000 7efccde00640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 0x7efcac06e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:36.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.434+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efcc807a5b0 con 0x7efcd0147e90 2024-09-10T10:52:36.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.654+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efcd00b2f50 con 0x7efcd0147e90 2024-09-10T10:52:36.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.860+0000 7efcc6000640 1 -- 172.21.15.28:0/1023199569 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v185) ==== 161+0+0 (secure 0 0 0) 0x7efcc8079eb0 con 0x7efcd0147e90 2024-09-10T10:52:36.862 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 msgr2=0x7efcac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 0x7efcac06e2f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0008040 comp rx=0 tx=0).stop 2024-09-10T10:52:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 msgr2=0x7efcd01f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7efcc8009f40 tx=0x7efcc8004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 shutdown_connections 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcac06be30 0x7efcac06e2f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcd0147e90 0x7efcd01f4340 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcd0147230 0x7efcd01f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 --2- 172.21.15.28:0/1023199569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcd00727c0 0x7efcd015dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 >> 172.21.15.28:0/1023199569 conn(0x7efcd0067d70 msgr2=0x7efcd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 shutdown_connections 2024-09-10T10:52:36.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:36.863+0000 7efcd5200640 1 -- 172.21.15.28:0/1023199569 wait complete. 2024-09-10T10:52:36.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.037+0000 7f11d0c00640 1 Processor -- start 2024-09-10T10:52:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.037+0000 7f11d0c00640 1 -- start start 2024-09-10T10:52:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11d0c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11cc072320 0x7f11cc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11d0c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11cc15e1f0 0x7f11cc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11d0c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc146140 con 0x7f11cc072de0 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11d0c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0cd740 con 0x7f11cc072320 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11d0c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0cda20 con 0x7f11cc15e1f0 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11c9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11cc072320 0x7f11cc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11c9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11cc072320 0x7f11cc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48132/0 (socket says 172.21.15.28:48132) 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.038+0000 7f11c9e00640 1 -- 172.21.15.28:0/209871055 learned_addr learned my addr 172.21.15.28:0/209871055 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.039+0000 7f11c9e00640 1 -- 172.21.15.28:0/209871055 >> v2:172.21.15.138:3300/0 conn(0x7f11cc15e1f0 msgr2=0x7f11cc06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.039+0000 7f11c9e00640 1 --2- 172.21.15.28:0/209871055 >> v2:172.21.15.138:3300/0 conn(0x7f11cc15e1f0 0x7f11cc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.039+0000 7f11c9e00640 1 -- 172.21.15.28:0/209871055 >> v1:172.21.15.138:6789/0 conn(0x7f11cc072de0 legacy=0x7f11cc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.039+0000 7f11c9e00640 1 -- 172.21.15.28:0/209871055 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0cdd00 con 0x7f11cc072320 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11c9e00640 1 --2- 172.21.15.28:0/209871055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072320 0x7f11cc0cce60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0cfce0 tx=0x7f11bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12465d1d4a3ae31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11c9400640 1 -- 172.21.15.28:0/209871055 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11cc072320 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11c9400640 1 -- 172.21.15.28:0/209871055 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc032440 con 0x7f11cc072320 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11c9400640 1 -- 172.21.15.28:0/209871055 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc032810 con 0x7f11cc072320 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11d0c00640 1 -- 172.21.15.28:0/209871055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072320 msgr2=0x7f11cc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.040+0000 7f11d0c00640 1 --2- 172.21.15.28:0/209871055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072320 0x7f11cc0cce60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0cfce0 tx=0x7f11bc02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 -- 172.21.15.28:0/209871055 shutdown_connections 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 --2- 172.21.15.28:0/209871055 >> v2:172.21.15.138:3300/0 conn(0x7f11cc15e1f0 0x7f11cc06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 --2- 172.21.15.28:0/209871055 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072320 0x7f11cc0cce60 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 -- 172.21.15.28:0/209871055 >> 172.21.15.28:0/209871055 conn(0x7f11cc067d70 msgr2=0x7f11cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 -- 172.21.15.28:0/209871055 shutdown_connections 2024-09-10T10:52:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.041+0000 7f11d0c00640 1 -- 172.21.15.28:0/209871055 wait complete. 2024-09-10T10:52:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.042+0000 7f11d0c00640 1 Processor -- start 2024-09-10T10:52:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.042+0000 7f11d0c00640 1 -- start start 2024-09-10T10:52:37.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 0x7f11cc1e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 0x7f11cc15bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0cd520 con 0x7f11cc072de0 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0cd3a0 con 0x7f11cc15e1f0 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11d0c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc06ed00 con 0x7f11cc072320 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11c1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.043+0000 7f11c1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48140/0 (socket says 172.21.15.28:48140) 2024-09-10T10:52:37.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 -- 172.21.15.28:0/761626406 learned_addr learned my addr 172.21.15.28:0/761626406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:37.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c9e00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 0x7f11cc1e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:37.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 -- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 msgr2=0x7f11cc1e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:37.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11ca800640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 0x7f11cc15bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 0x7f11cc1e7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 -- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 msgr2=0x7f11cc15bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 0x7f11cc15bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 -- 172.21.15.28:0/761626406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0cb870 con 0x7f11cc072de0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c9e00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 0x7f11cc1e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11ca800640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 0x7f11cc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c1400640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c410 tx=0x7f11b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.044+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4017070 con 0x7f11cc072de0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.045+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11b40035f0 con 0x7f11cc072de0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.045+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4003a20 con 0x7f11cc072de0 2024-09-10T10:52:37.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.045+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc0cbb50 con 0x7f11cc072de0 2024-09-10T10:52:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.045+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc0cc110 con 0x7f11cc072de0 2024-09-10T10:52:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.046+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc072700 con 0x7f11cc072de0 2024-09-10T10:52:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.047+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f11b4003bc0 con 0x7f11cc072de0 2024-09-10T10:52:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.048+0000 7f11c2a00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 0x7f11a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.048+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7f11b408a9a0 con 0x7f11cc072de0 2024-09-10T10:52:37.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.048+0000 7f11c9e00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 0x7f11a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:37.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.049+0000 7f11c9e00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 0x7f11a806e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:37.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.052+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f11b4058b30 con 0x7f11cc072de0 2024-09-10T10:52:37.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.304+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f11cc0cea50 con 0x7f11cc072de0 2024-09-10T10:52:37.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.863+0000 7f11c2a00640 1 -- 172.21.15.28:0/761626406 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v186) ==== 101+0+0 (secure 0 0 0) 0x7f11b4058430 con 0x7f11cc072de0 2024-09-10T10:52:38.173 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:38.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 msgr2=0x7f11a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 0x7f11a806e2f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f11bc034000 tx=0x7f11bc02e000 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 msgr2=0x7f11cc15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c410 tx=0x7f11b400c8d0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 shutdown_connections 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f11a806be30 0x7f11a806e2f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11cc15e1f0 0x7f11cc15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11cc072de0 0x7f11cc15b990 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 --2- 172.21.15.28:0/761626406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11cc072320 0x7f11cc1e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.865+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 >> 172.21.15.28:0/761626406 conn(0x7f11cc067d70 msgr2=0x7f11cc0cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.866+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 shutdown_connections 2024-09-10T10:52:38.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:37.866+0000 7f11d0c00640 1 -- 172.21.15.28:0/761626406 wait complete. 2024-09-10T10:52:38.176 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:38.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.258+0000 7f3c1105df40 1 Processor -- start 2024-09-10T10:52:38.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.258+0000 7f3c1105df40 1 -- start start 2024-09-10T10:52:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a3839b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a3831360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5654a3705b30 con 0x5654a37a8030 2024-09-10T10:52:38.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654a3709b70 con 0x7f3bf4002ca0 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c1105df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654a3835ee0 con 0x5654a38349a0 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48152/0 (socket says 172.21.15.28:48152) 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 -- 172.21.15.28:0/4266263454 learned_addr learned my addr 172.21.15.28:0/4266263454 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0e200640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a3839b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a3831360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 -- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 msgr2=0x5654a3831360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:38.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a3831360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 -- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 msgr2=0x5654a3839b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a3839b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0d800640 1 -- 172.21.15.28:0/4266263454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a3832b10 con 0x5654a37a8030 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.259+0000 7f3c0e200640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a3839b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c0d800640 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc046a20 tx=0x7f3bfc050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ae37816cc45948 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c07e00640 1 -- 172.21.15.28:0/4266263454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc0014f0 con 0x5654a37a8030 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c07e00640 1 -- 172.21.15.28:0/4266263454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc0016b0 con 0x5654a37a8030 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c07e00640 1 -- 172.21.15.28:0/4266263454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc04db80 con 0x5654a37a8030 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c1105df40 1 -- 172.21.15.28:0/4266263454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 msgr2=0x5654a3834460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c1105df40 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc046a20 tx=0x7f3bfc050000 comp rx=0 tx=0).stop 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.260+0000 7f3c1105df40 1 -- 172.21.15.28:0/4266263454 shutdown_connections 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a3831360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a37a8030 0x5654a3834460 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 --2- 172.21.15.28:0/4266263454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a3839b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 -- 172.21.15.28:0/4266263454 >> 172.21.15.28:0/4266263454 conn(0x5654a37ac0a0 msgr2=0x5654a37ac480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 -- 172.21.15.28:0/4266263454 shutdown_connections 2024-09-10T10:52:38.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 -- 172.21.15.28:0/4266263454 wait complete. 2024-09-10T10:52:38.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.261+0000 7f3c1105df40 1 Processor -- start 2024-09-10T10:52:38.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 -- start start 2024-09-10T10:52:38.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a382c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a37a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5654a378d5c0 con 0x5654a382cf10 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654a37aac40 con 0x7f3bf4002ca0 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.262+0000 7f3c1105df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654a3834e20 con 0x5654a38349a0 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48170/0 (socket says 172.21.15.28:48170) 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 learned_addr learned my addr 172.21.15.28:0/991599181 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a382c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a37a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 msgr2=0x5654a382c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654a38349a0 0x5654a382c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 msgr2=0x5654a37a3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bf4002ca0 0x5654a37a3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.263+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a378b710 con 0x5654a382cf10 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x5654a3764f00 tx=0x7f3c08012cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c06a00640 1 -- 172.21.15.28:0/991599181 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0800a3b0 con 0x5654a382cf10 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c06a00640 1 -- 172.21.15.28:0/991599181 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c08012e40 con 0x5654a382cf10 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654a37a6190 con 0x5654a382cf10 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c06a00640 1 -- 172.21.15.28:0/991599181 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08010c00 con 0x5654a382cf10 2024-09-10T10:52:38.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.264+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a3835b60 con 0x5654a382cf10 2024-09-10T10:52:38.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.266+0000 7f3c06a00640 1 -- 172.21.15.28:0/991599181 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c0801c020 con 0x5654a382cf10 2024-09-10T10:52:38.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.267+0000 7f3c06a00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 0x7f3bf006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.267+0000 7f3c06a00640 1 -- 172.21.15.28:0/991599181 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7f3c0808ab90 con 0x5654a382cf10 2024-09-10T10:52:38.270 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.267+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 0x5654a37a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.267+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a375d000 con 0x5654a37ac580 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.267+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 0x7f3bf006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.268+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 0x5654a37a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.268+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 0x7f3bf006ccd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5654a374a740 tx=0x7f3c000018a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.268+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 0x5654a37a58d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.270+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v117'505 uv479 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.270+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a37a0e10 con 0x5654a37ac580 2024-09-10T10:52:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.272+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v117'506 uv506 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.272+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d8b10 con 0x5654a37ac580 2024-09-10T10:52:38.276 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.274+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v117'507 uv507 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.274+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d9ec0 con 0x5654a37ac580 2024-09-10T10:52:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.276+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v117'508 uv508 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.276+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da260 con 0x5654a37ac580 2024-09-10T10:52:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.277+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v117'509 uv509 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.278+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d8eb0 con 0x5654a37ac580 2024-09-10T10:52:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.279+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v117'510 uv510 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.279+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d9250 con 0x5654a37ac580 2024-09-10T10:52:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.281+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v117'511 uv485 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.281+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.283+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v117'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.283+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.287 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:38.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.286+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v117'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.286+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.287+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v117'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.287+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.290+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v117'515 uv515 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.290+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.291+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v117'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.291+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.292+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v117'517 uv492 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.292+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.294+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v117'518 uv518 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.294+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.298 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:38.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.296+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v117'519 uv519 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.296+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.298+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v117'520 uv520 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.298+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.299+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v117'521 uv521 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.300+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38d95f0 con 0x5654a37ac580 2024-09-10T10:52:38.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.302+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v117'522 uv522 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.302+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.303+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v117'523 uv523 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.303+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.305+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v117'524 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.305+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.306+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v117'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.306+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.308+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v117'526 uv526 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.310 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:38.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.308+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38db610 con 0x5654a37ac580 2024-09-10T10:52:38.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.309+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v117'527 uv527 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.309+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38db610 con 0x5654a37ac580 2024-09-10T10:52:38.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.312+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v117'528 uv528 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.312+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.313+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v117'529 uv529 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.313+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.315+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v117'530 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.315+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.316+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v117'531 uv531 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.317+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.320 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:38.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.318+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v117'532 uv532 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.318+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38db610 con 0x5654a37ac580 2024-09-10T10:52:38.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.320+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v117'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.320+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38db610 con 0x5654a37ac580 2024-09-10T10:52:38.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.321+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v117'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.321+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38da600 con 0x5654a37ac580 2024-09-10T10:52:38.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.323+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v117'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a37ac580 2024-09-10T10:52:38.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.323+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654a38dab10 0x5654a38dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.323+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dd440 con 0x5654a38dab10 2024-09-10T10:52:38.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.323+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654a38dab10 0x5654a38dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.324+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654a38dab10 0x5654a38dcf00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.326+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v117'374 uv356 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.326+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dd7e0 con 0x5654a38dab10 2024-09-10T10:52:38.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.328+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v117'375 uv375 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.328+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ddb80 con 0x5654a38dab10 2024-09-10T10:52:38.331 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:38.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.329+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v117'376 uv376 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.329+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ddf20 con 0x5654a38dab10 2024-09-10T10:52:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.330+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v117'377 uv377 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.330+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38de2c0 con 0x5654a38dab10 2024-09-10T10:52:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.332+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v117'378 uv378 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.332+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38de660 con 0x5654a38dab10 2024-09-10T10:52:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.333+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v117'379 uv373 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.333+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.335+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v117'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.335+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.338 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:38.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.337+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v117'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.337+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.338+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v117'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.338+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.340+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v117'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.340+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.341+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v117'384 uv366 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.341+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.343+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v117'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.343+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.346 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.345+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v117'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.345+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e30f0 con 0x5654a38dab10 2024-09-10T10:52:38.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.346+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v117'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.346+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e30f0 con 0x5654a38dab10 2024-09-10T10:52:38.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.348+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v117'388 uv388 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.348+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.350+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v117'389 uv372 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.350+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.351+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v117'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.351+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.353+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v117'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.355 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:38.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.353+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dfa10 con 0x5654a38dab10 2024-09-10T10:52:38.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.355+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v117'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.355+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dfa10 con 0x5654a38dab10 2024-09-10T10:52:38.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.357+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v117'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.357+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38dea00 con 0x5654a38dab10 2024-09-10T10:52:38.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.358+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v117'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00020400 con 0x5654a38dab10 2024-09-10T10:52:38.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.358+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654a38df080 0x5654a38e1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.358+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e19f0 con 0x5654a38df080 2024-09-10T10:52:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.358+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654a38df080 0x5654a38e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.359+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654a38df080 0x5654a38e1470 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.361+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v117'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.361+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e1ab0 con 0x5654a38df080 2024-09-10T10:52:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.362+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v117'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.362+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e1ed0 con 0x5654a38df080 2024-09-10T10:52:38.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.365+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v117'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.367 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:38.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.365+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e2ee0 con 0x5654a38df080 2024-09-10T10:52:38.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.366+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v117'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.366+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e2ee0 con 0x5654a38df080 2024-09-10T10:52:38.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.367+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v117'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.367+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e1ed0 con 0x5654a38df080 2024-09-10T10:52:38.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.369+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v117'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.369+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654a38e2550 0x5654a38e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.369+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e4fc0 con 0x5654a38e2550 2024-09-10T10:52:38.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.369+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654a38e2550 0x5654a38e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.370+0000 7f3c0ec00640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654a38e2550 0x5654a38e4940 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.371+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v117'75 uv74 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3c08010da0 con 0x5654a38e2550 2024-09-10T10:52:38.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.371+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e54d0 con 0x5654a38e2550 2024-09-10T10:52:38.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.372+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v117'76 uv76 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.372+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e59d0 con 0x5654a38e2550 2024-09-10T10:52:38.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.373+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v117'77 uv77 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.375 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:38.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.374+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6290 con 0x5654a38e2550 2024-09-10T10:52:38.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.375+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v117'78 uv78 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.375+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6630 con 0x5654a38df080 2024-09-10T10:52:38.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.376+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v117'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.376+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6630 con 0x5654a38df080 2024-09-10T10:52:38.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.377+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v117'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.377+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6630 con 0x5654a38df080 2024-09-10T10:52:38.380 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:38.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.378+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v117'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.379+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e7640 con 0x5654a38df080 2024-09-10T10:52:38.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.380+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v117'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.380+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e7640 con 0x5654a38df080 2024-09-10T10:52:38.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.381+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v117'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.381+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6630 con 0x5654a38df080 2024-09-10T10:52:38.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.382+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v117'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.382+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6630 con 0x5654a38e2550 2024-09-10T10:52:38.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.383+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v117'182 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.383+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e69d0 con 0x5654a38e2550 2024-09-10T10:52:38.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.385+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v117'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.385+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.389 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:38.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.386+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v117'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.386+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e7d80 con 0x5654a38e2550 2024-09-10T10:52:38.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.388+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v117'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.388+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e7d80 con 0x5654a38e2550 2024-09-10T10:52:38.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.389+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v117'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.389+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.390+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v117'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.391+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.392+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v117'188 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.392+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.393+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v117'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.393+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.397 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:38.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.395+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v117'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.395+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.396+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v117'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.396+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38e2550 2024-09-10T10:52:38.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.398+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v117'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.398+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38df080 2024-09-10T10:52:38.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.399+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v117'81 uv80 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.399+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38df080 2024-09-10T10:52:38.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.400+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v117'82 uv82 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.400+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38df080 2024-09-10T10:52:38.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.401+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v117'83 uv83 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.403 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:38.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.401+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38df080 2024-09-10T10:52:38.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.402+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v117'84 uv84 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.403+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e6d70 con 0x5654a38df080 2024-09-10T10:52:38.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.404+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v117'85 uv85 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38df080 2024-09-10T10:52:38.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.404+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654a38e73f0 0x5654a38e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.404+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e9d60 con 0x5654a38e73f0 2024-09-10T10:52:38.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.404+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654a38e73f0 0x5654a38e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.405+0000 7f3c0e200640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654a38e73f0 0x5654a38e97e0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.406+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v117'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.406+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38e9e20 con 0x5654a38e73f0 2024-09-10T10:52:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.408+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v117'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.408+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.413 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.409+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v117'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.409+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38eb250 con 0x5654a38e73f0 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.410+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v117'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.410+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38eb250 con 0x5654a38e73f0 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.412+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v117'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.412+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.413+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v117'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.413+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.415+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v117'162 uv156 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.415+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.416+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v117'163 uv163 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.416+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.419 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:38.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.417+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v117'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.418+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.419+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v117'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.419+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.420+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v117'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.420+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.421+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v117'167 uv161 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.421+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.422+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v117'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.422+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.425 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.424+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v117'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.424+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.425+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v117'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.425+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ed050 con 0x5654a38e73f0 2024-09-10T10:52:38.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.426+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v117'171 uv171 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.426+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.428+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v117'74 uv73 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.428+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.429+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v117'75 uv75 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.429+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.430+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v117'76 uv76 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.432 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.430+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.432+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v117'77 uv77 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.432+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.433+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v117'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.433+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.434+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v117'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.434+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.435+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v117'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.437 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.435+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38eb700 con 0x5654a38e2550 2024-09-10T10:52:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.436+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v117'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.436+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38eb700 con 0x5654a38e2550 2024-09-10T10:52:38.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.438+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v117'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.438+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e2550 2024-09-10T10:52:38.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.439+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v117'104 uv104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3c08054cc0 con 0x5654a38e2550 2024-09-10T10:52:38.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.439+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654a38eaff0 0x5654a38eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.439+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38eaff0 2024-09-10T10:52:38.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.439+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654a38eaff0 0x5654a38eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.440+0000 7f3c0d800640 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654a38eaff0 0x5654a38eb3d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.441+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v117'84 uv83 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38eaff0 2024-09-10T10:52:38.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.441+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38eaff0 2024-09-10T10:52:38.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.442+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v117'85 uv85 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38eaff0 2024-09-10T10:52:38.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.442+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38eaff0 2024-09-10T10:52:38.445 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:38.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.443+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v117'86 uv86 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38eaff0 2024-09-10T10:52:38.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.443+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38eaff0 2024-09-10T10:52:38.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.444+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v117'87 uv87 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38eaff0 2024-09-10T10:52:38.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.444+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38eaff0 2024-09-10T10:52:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.445+0000 7f3c0d800640 1 -- 172.21.15.28:0/991599181 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v117'88 uv88 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3bfc05b030 con 0x5654a38eaff0 2024-09-10T10:52:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.445+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e73f0 2024-09-10T10:52:38.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.446+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v117'204 uv196 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.446+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e73f0 2024-09-10T10:52:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.448+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v117'205 uv205 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.448+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e73f0 2024-09-10T10:52:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.449+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v117'206 uv206 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.451 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.449+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ebbf0 con 0x5654a38e73f0 2024-09-10T10:52:38.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.450+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v117'207 uv207 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.450+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ebbf0 con 0x5654a38e73f0 2024-09-10T10:52:38.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.451+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v117'208 uv208 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.451+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ebb70 con 0x5654a38e73f0 2024-09-10T10:52:38.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.453+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v117'209 uv203 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.453+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e73f0 2024-09-10T10:52:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.454+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v117'210 uv210 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.454+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea6f0 con 0x5654a38e73f0 2024-09-10T10:52:38.456 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.455+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v117'211 uv211 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.455+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ecb10 con 0x5654a38e73f0 2024-09-10T10:52:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.456+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v117'212 uv212 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.456+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ecb10 con 0x5654a38e73f0 2024-09-10T10:52:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.457+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v117'213 uv213 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.457+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ea240 con 0x5654a38e73f0 2024-09-10T10:52:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.458+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v117'214 uv214 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.458+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x5654a38ebb00 con 0x5654a38e73f0 2024-09-10T10:52:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.459+0000 7f3c0e200640 1 -- 172.21.15.28:0/991599181 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v117'215 uv215 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3c00019040 con 0x5654a38e73f0 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654a38df080 msgr2=0x5654a38e1470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654a38df080 0x5654a38e1470 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654a38e73f0 msgr2=0x5654a38e97e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654a38e73f0 0x5654a38e97e0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654a38eaff0 msgr2=0x5654a38eb3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654a38eaff0 0x5654a38eb3d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654a38e2550 msgr2=0x5654a38e4940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654a38e2550 0x5654a38e4940 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654a38dab10 msgr2=0x5654a38dcf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654a38dab10 0x5654a38dcf00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 reap_dead start 2024-09-10T10:52:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c0ec00640 1 -- 172.21.15.28:0/991599181 reap_dead start 2024-09-10T10:52:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 msgr2=0x5654a37a58d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 0x5654a37a58d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 msgr2=0x7f3bf006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 0x7f3bf006ccd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x5654a374a740 tx=0x7f3c000018a0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 msgr2=0x5654a37a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x5654a3764f00 tx=0x7f3c08012cb0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 shutdown_connections 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654a37ac580 0x5654a37a58d0 crc :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bf006a810 0x7f3bf006ccd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 --2- 172.21.15.28:0/991599181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654a382cf10 0x5654a37a3ab0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 >> 172.21.15.28:0/991599181 conn(0x5654a37ac0a0 msgr2=0x5654a378baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 shutdown_connections 2024-09-10T10:52:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.460+0000 7f3c1105df40 1 -- 172.21.15.28:0/991599181 wait complete. 2024-09-10T10:52:38.467 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 200.00000001 - 2024-09-10T10:52:38.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.541+0000 7f69958d2f40 1 Processor -- start 2024-09-10T10:52:38.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.541+0000 7f69958d2f40 1 -- start start 2024-09-10T10:52:38.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e847c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d49c0 0x56019e8cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56019e846530 con 0x56019e8d13e0 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56019e7a3b30 con 0x56019e8d49c0 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f69958d2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56019e8d3ee0 con 0x56019e845ff0 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f6992000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f6992000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48196/0 (socket says 172.21.15.28:48196) 2024-09-10T10:52:38.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f6992000640 1 -- 172.21.15.28:0/1464760555 learned_addr learned my addr 172.21.15.28:0/1464760555 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f6992a00640 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e847c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.542+0000 7f6993400640 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d49c0 0x56019e8cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 -- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 msgr2=0x56019e847c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e847c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 -- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d49c0 msgr2=0x56019e8cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d49c0 0x56019e8cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 -- 172.21.15.28:0/1464760555 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019e7fdd60 con 0x56019e8d13e0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6992000640 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f698000b240 tx=0x7f698000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9e5705a24da821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6990c00640 1 -- 172.21.15.28:0/1464760555 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980008070 con 0x56019e8d13e0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6990c00640 1 -- 172.21.15.28:0/1464760555 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698000d500 con 0x56019e8d13e0 2024-09-10T10:52:38.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.543+0000 7f6990c00640 1 -- 172.21.15.28:0/1464760555 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000d8d0 con 0x56019e8d13e0 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 -- 172.21.15.28:0/1464760555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 msgr2=0x56019e8ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f698000b240 tx=0x7f698000b700 comp rx=0 tx=0).stop 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 -- 172.21.15.28:0/1464760555 shutdown_connections 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d49c0 0x56019e8cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d13e0 0x56019e8ce890 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1464760555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e847c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 -- 172.21.15.28:0/1464760555 >> 172.21.15.28:0/1464760555 conn(0x56019e84a0a0 msgr2=0x56019e7fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 -- 172.21.15.28:0/1464760555 shutdown_connections 2024-09-10T10:52:38.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.544+0000 7f69958d2f40 1 -- 172.21.15.28:0/1464760555 wait complete. 2024-09-10T10:52:38.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.545+0000 7f69958d2f40 1 Processor -- start 2024-09-10T10:52:38.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.545+0000 7f69958d2f40 1 -- start start 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e8cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d13e0 0x56019e8c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 0x56019e8d2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56019e82b010 con 0x56019e8d49c0 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56019e7fdb10 con 0x56019e8d13e0 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f69958d2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56019e848bb0 con 0x56019e845ff0 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 0x56019e8d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 0x56019e8d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48208/0 (socket says 172.21.15.28:48208) 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 -- 172.21.15.28:0/1166722438 learned_addr learned my addr 172.21.15.28:0/1166722438 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6992a00640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e8cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6992000640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d13e0 0x56019e8c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 -- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 msgr2=0x56019e8cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56019e845ff0 0x56019e8cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 -- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d13e0 msgr2=0x56019e8c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56019e8d13e0 0x56019e8c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.546+0000 7f6993400640 1 -- 172.21.15.28:0/1166722438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56019e8cb490 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f6993400640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 0x56019e8d2230 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f698c005fd0 tx=0x7f698c0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f698b400640 1 -- 172.21.15.28:0/1166722438 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c069070 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f698b400640 1 -- 172.21.15.28:0/1166722438 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f698c04f440 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f698b400640 1 -- 172.21.15.28:0/1166722438 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c005b40 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56019e8cb770 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.547+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56019e8cd270 con 0x56019e8d49c0 2024-09-10T10:52:38.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.549+0000 7f698b400640 1 -- 172.21.15.28:0/1166722438 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f698c057400 con 0x56019e8d49c0 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.549+0000 7f698b400640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f697406a840 0x7f697406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.550+0000 7f698b400640 1 -- 172.21.15.28:0/1166722438 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7f698c0ce130 con 0x56019e8d49c0 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.550+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56019e8d7960 0x56019e8d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.550+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x56019e8d5810 con 0x56019e8d7960 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.550+0000 7f6992a00640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f697406a840 0x7f697406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.550+0000 7f6992000640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56019e8d7960 0x56019e8d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.551+0000 7f6992a00640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f697406a840 0x7f697406cd00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f69840097c0 tx=0x7f6984005b30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.551+0000 7f6992000640 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56019e8d7960 0x56019e8d52d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.552+0000 7f6992000640 1 -- 172.21.15.28:0/1166722438 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [writefull 0~80] v117'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f698000d7b0 con 0x56019e8d7960 2024-09-10T10:52:38.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56019e8d7960 msgr2=0x56019e8d52d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56019e8d7960 0x56019e8d52d0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f697406a840 msgr2=0x7f697406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f697406a840 0x7f697406cd00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f69840097c0 tx=0x7f6984005b30 comp rx=0 tx=0).stop 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 msgr2=0x56019e8d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 --2- 172.21.15.28:0/1166722438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56019e8d49c0 0x56019e8d2230 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f698c005fd0 tx=0x7f698c0033a0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f6993400640 1 -- 172.21.15.28:0/1166722438 reap_dead start 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 shutdown_connections 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 >> 172.21.15.28:0/1166722438 conn(0x56019e84a0a0 msgr2=0x56019e82cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 shutdown_connections 2024-09-10T10:52:38.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.553+0000 7f69958d2f40 1 -- 172.21.15.28:0/1166722438 wait complete. 2024-09-10T10:52:38.559 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:38.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.758+0000 7fe7bcc00640 1 Processor -- start 2024-09-10T10:52:38.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.758+0000 7fe7bcc00640 1 -- start start 2024-09-10T10:52:38.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7bcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7bcc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7bcc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b80b1170 con 0x7fe7b80b3640 2024-09-10T10:52:38.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7bcc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80ceba0 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7bcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b806dfb0 con 0x7fe7b8161ac0 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48220/0 (socket says 172.21.15.28:48220) 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7b7e00640 1 -- 172.21.15.28:0/1451431486 learned_addr learned my addr 172.21.15.28:0/1451431486 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.759+0000 7fe7b7400640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.760+0000 7fe7b7e00640 1 -- 172.21.15.28:0/1451431486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 msgr2=0x7fe7b806e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.760+0000 7fe7b7e00640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b806e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.760+0000 7fe7b7e00640 1 -- 172.21.15.28:0/1451431486 >> v1:172.21.15.138:6789/0 conn(0x7fe7b80b3640 legacy=0x7fe7b80d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.760+0000 7fe7b7e00640 1 -- 172.21.15.28:0/1451431486 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80b1450 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.760+0000 7fe7b6a00640 1 -- 172.21.15.28:0/1451431486 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602801171 0 0) 0x7fe7b80b1170 con 0x7fe7b80b3640 2024-09-10T10:52:38.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.761+0000 7fe7b7e00640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009e80 tx=0x7fe7b002a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265e7f537279334b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.761+0000 7fe7b6a00640 1 -- 172.21.15.28:0/1451431486 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0036070 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.761+0000 7fe7b6a00640 1 -- 172.21.15.28:0/1451431486 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b0031600 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.761+0000 7fe7b6a00640 1 -- 172.21.15.28:0/1451431486 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00319d0 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 -- 172.21.15.28:0/1451431486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80d2ac0 msgr2=0x7fe7b80b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009e80 tx=0x7fe7b002a7e0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 -- 172.21.15.28:0/1451431486 shutdown_connections 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b80b1f30 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/1451431486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b806e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 -- 172.21.15.28:0/1451431486 >> 172.21.15.28:0/1451431486 conn(0x7fe7b808bfc0 msgr2=0x7fe7b80d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:38.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 -- 172.21.15.28:0/1451431486 shutdown_connections 2024-09-10T10:52:38.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.762+0000 7fe7bcc00640 1 -- 172.21.15.28:0/1451431486 wait complete. 2024-09-10T10:52:38.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.763+0000 7fe7bcc00640 1 Processor -- start 2024-09-10T10:52:38.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 -- start start 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b81f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b81f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8161ea0 con 0x7fe7b80b3640 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b80b1a10 con 0x7fe7b80d2ac0 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.764+0000 7fe7bcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b8162020 con 0x7fe7b8161ac0 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48224/0 (socket says 172.21.15.28:48224) 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 -- 172.21.15.28:0/3501511277 learned_addr learned my addr 172.21.15.28:0/3501511277 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7afe00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b81f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7e00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b81f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 -- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 msgr2=0x7fe7b81f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b81f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 -- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 msgr2=0x7fe7b81f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b81f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 -- 172.21.15.28:0/3501511277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80cca90 con 0x7fe7b80b3640 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7afe00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b81f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7e00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b81f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.765+0000 7fe7b7400640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e870 tx=0x7fe7a800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.766+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8017070 con 0x7fe7b80b3640 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.766+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a800c940 con 0x7fe7b80b3640 2024-09-10T10:52:38.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.766+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800cd30 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.766+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b80ccd70 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.766+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81f1b30 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.767+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b80c93d0 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.768+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe7a801a070 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.769+0000 7fe7af400640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 0x7fe79406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.769+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 7766+0+0 (secure 0 0 0) 0x7fe7a808b8e0 con 0x7fe7b80b3640 2024-09-10T10:52:38.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.769+0000 7fe7afe00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 0x7fe79406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:38.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.770+0000 7fe7afe00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 0x7fe79406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80cd510 tx=0x7fe7a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:38.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:38.773+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7b80c93d0 con 0x7fe7b80b3640 2024-09-10T10:52:39.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.002+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe7b806e6c0 con 0x7fe7b80b3640 2024-09-10T10:52:39.606 INFO:tasks.ceph.mds.a.smithi028.stderr: -36> 2024-09-10T10:52:34.852+0000 7f5a35000640 -1 log_channel(cluster) log [ERR] : Error recovering journal 0x200: (22) Invalid argument 2024-09-10T10:52:39.649 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:52:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.883+0000 7fe7af400640 1 -- 172.21.15.28:0/3501511277 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v187) ==== 162+0+0 (secure 0 0 0) 0x7fe7a8059a80 con 0x7fe7b80b3640 2024-09-10T10:52:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 msgr2=0x7fe79406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 0x7fe79406e2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80cd510 tx=0x7fe7a0009040 comp rx=0 tx=0).stop 2024-09-10T10:52:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 msgr2=0x7fe7b80cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe7a800e870 tx=0x7fe7a800ed30 comp rx=0 tx=0).stop 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 shutdown_connections 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe79406be30 0x7fe79406e2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe7b8161ac0 0x7fe7b81f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b80d2ac0 0x7fe7b81f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 --2- 172.21.15.28:0/3501511277 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7b80b3640 0x7fe7b80cb530 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 >> 172.21.15.28:0/3501511277 conn(0x7fe7b808bfc0 msgr2=0x7fe7b80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 shutdown_connections 2024-09-10T10:52:39.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:39.887+0000 7fe7bcc00640 1 -- 172.21.15.28:0/3501511277 wait complete. 2024-09-10T10:52:39.904 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:52:39.902+0000 7fc452800640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:52:39.905 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:39.903+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:39.905 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:39.903+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:39.914 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:40.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.071+0000 7f7376000640 1 Processor -- start 2024-09-10T10:52:40.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.071+0000 7f7376000640 1 -- start start 2024-09-10T10:52:40.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f7376000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7370072de0 0x7f737015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f7376000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7370072700 0x7f737015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f7376000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7370146140 con 0x7f7370072320 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f7376000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f73700cd740 con 0x7f7370072de0 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f7376000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f73700cda20 con 0x7f7370072700 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.072+0000 7f736ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7370072de0 0x7f737015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7370072de0 0x7f737015d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48260/0 (socket says 172.21.15.28:48260) 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 -- 172.21.15.28:0/3308734093 learned_addr learned my addr 172.21.15.28:0/3308734093 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736f400640 1 --1- 172.21.15.28:0/3308734093 >> v1:172.21.15.28:6789/0 conn(0x7f7370072320 0x7f73700cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47214/0 (socket says 172.21.15.28:47214) 2024-09-10T10:52:40.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736fe00640 1 --2- 172.21.15.28:0/3308734093 >> v2:172.21.15.138:3300/0 conn(0x7f7370072700 0x7f737015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 -- 172.21.15.28:0/3308734093 >> v2:172.21.15.138:3300/0 conn(0x7f7370072700 msgr2=0x7f737015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:40.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 --2- 172.21.15.28:0/3308734093 >> v2:172.21.15.138:3300/0 conn(0x7f7370072700 0x7f737015d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736e000640 1 -- 172.21.15.28:0/3308734093 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291358805 0 0) 0x7f7370146140 con 0x7f7370072320 2024-09-10T10:52:40.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 -- 172.21.15.28:0/3308734093 >> v1:172.21.15.28:6789/0 conn(0x7f7370072320 legacy=0x7f73700cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.073+0000 7f736ea00640 1 -- 172.21.15.28:0/3308734093 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73700cdd00 con 0x7f7370072de0 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.074+0000 7f736ea00640 1 --2- 172.21.15.28:0/3308734093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072de0 0x7f737015d1e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f73700cfce0 tx=0x7f735c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527d2af8e52d3230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.074+0000 7f736e000640 1 -- 172.21.15.28:0/3308734093 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c037070 con 0x7f7370072de0 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.074+0000 7f736e000640 1 -- 172.21.15.28:0/3308734093 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735c032440 con 0x7f7370072de0 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.074+0000 7f736e000640 1 -- 172.21.15.28:0/3308734093 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c032810 con 0x7f7370072de0 2024-09-10T10:52:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 -- 172.21.15.28:0/3308734093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072de0 msgr2=0x7f737015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 --2- 172.21.15.28:0/3308734093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072de0 0x7f737015d1e0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f73700cfce0 tx=0x7f735c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 -- 172.21.15.28:0/3308734093 shutdown_connections 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 --2- 172.21.15.28:0/3308734093 >> v2:172.21.15.138:3300/0 conn(0x7f7370072700 0x7f737015d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 --2- 172.21.15.28:0/3308734093 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072de0 0x7f737015d1e0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 -- 172.21.15.28:0/3308734093 >> 172.21.15.28:0/3308734093 conn(0x7f7370067d70 msgr2=0x7f737006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 -- 172.21.15.28:0/3308734093 shutdown_connections 2024-09-10T10:52:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.075+0000 7f7376000640 1 -- 172.21.15.28:0/3308734093 wait complete. 2024-09-10T10:52:40.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.076+0000 7f7376000640 1 Processor -- start 2024-09-10T10:52:40.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.077+0000 7f7376000640 1 -- start start 2024-09-10T10:52:40.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.077+0000 7f7376000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 0x7f73700ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.077+0000 7f7376000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f7376000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 0x7f73700c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f7376000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f737015dde0 con 0x7f7370072700 2024-09-10T10:52:40.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f7376000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f737015dc60 con 0x7f7370072de0 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f7376000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f737015df60 con 0x7f7370072320 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48264/0 (socket says 172.21.15.28:48264) 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 -- 172.21.15.28:0/187469934 learned_addr learned my addr 172.21.15.28:0/187469934 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736fe00640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 0x7f73700c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736f400640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 0x7f73700ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 -- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 msgr2=0x7f73700ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 0x7f73700ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 -- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 msgr2=0x7f73700c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 0x7f73700c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 -- 172.21.15.28:0/187469934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73701f7f90 con 0x7f7370072700 2024-09-10T10:52:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736fe00640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 0x7f73700c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736f400640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 0x7f73700ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.078+0000 7f736ea00640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f735c000c00 tx=0x7f735c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.079+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c037040 con 0x7f7370072700 2024-09-10T10:52:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.079+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735c004230 con 0x7f7370072700 2024-09-10T10:52:40.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.079+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c003610 con 0x7f7370072700 2024-09-10T10:52:40.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.079+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7370137510 con 0x7f7370072700 2024-09-10T10:52:40.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.079+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7370137ad0 con 0x7f7370072700 2024-09-10T10:52:40.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.080+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7370137310 con 0x7f7370072700 2024-09-10T10:52:40.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.081+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f735c0037b0 con 0x7f7370072700 2024-09-10T10:52:40.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.082+0000 7f7367400640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 0x7f735006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.082+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f735c0ac4d0 con 0x7f7370072700 2024-09-10T10:52:40.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.082+0000 7f736f400640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 0x7f735006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.083+0000 7f736f400640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 0x7f735006e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7360000f60 tx=0x7f7360009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.086+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f735c07a580 con 0x7f7370072700 2024-09-10T10:52:40.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.307+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f737015e320 con 0x7f7370072700 2024-09-10T10:52:40.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.307+0000 7f7367400640 1 -- 172.21.15.28:0/187469934 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) ==== 95+0+4392 (secure 0 0 0) 0x7f735c07d140 con 0x7f7370072700 2024-09-10T10:52:40.309 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:40.309 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":189,"btime":"2024-09-10T10:52:39:905015+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:39.904950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:40.310 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 189 2024-09-10T10:52:40.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 msgr2=0x7f735006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 0x7f735006e2f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7360000f60 tx=0x7f7360009040 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 msgr2=0x7f737013aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f735c000c00 tx=0x7f735c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 shutdown_connections 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f735006be30 0x7f735006e2f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7370072de0 0x7f73700c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7370072700 0x7f737013aa20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 --2- 172.21.15.28:0/187469934 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7370072320 0x7f73700ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 >> 172.21.15.28:0/187469934 conn(0x7f7370067d70 msgr2=0x7f73701e3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.309+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 shutdown_connections 2024-09-10T10:52:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.310+0000 7f7376000640 1 -- 172.21.15.28:0/187469934 wait complete. 2024-09-10T10:52:40.327 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:40.327 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:40.348 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:52:40.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:52:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.482+0000 7f7a93e00640 1 Processor -- start 2024-09-10T10:52:40.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.482+0000 7f7a93e00640 1 -- start start 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.483+0000 7f7a93e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.483+0000 7f7a93e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.483+0000 7f7a93e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a940cf000 con 0x7f7a940d0dd0 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.483+0000 7f7a93e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a940b37d0 con 0x7f7a940d1f90 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.483+0000 7f7a93e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a940cf330 con 0x7f7a9415e660 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a92000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a93400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a92000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:54988/0 (socket says 172.21.15.28:54988) 2024-09-10T10:52:40.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a92000640 1 -- 172.21.15.28:0/2557626142 learned_addr learned my addr 172.21.15.28:0/2557626142 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a93400640 1 -- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 msgr2=0x7f7a9406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a93400640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a93400640 1 -- 172.21.15.28:0/2557626142 >> v1:172.21.15.138:6789/0 conn(0x7f7a940d0dd0 legacy=0x7f7a940d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a93400640 1 -- 172.21.15.28:0/2557626142 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a940d29a0 con 0x7f7a940d1f90 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.484+0000 7f7a92000640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:40.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.485+0000 7f7a93400640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940cf7c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009d90 tx=0x7f7a8c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146f9c720551e591 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.485+0000 7f7a91600640 1 -- 172.21.15.28:0/2557626142 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c007740 con 0x7f7a940d1f90 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a91600640 1 -- 172.21.15.28:0/2557626142 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c0078e0 con 0x7f7a940d1f90 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a91600640 1 -- 172.21.15.28:0/2557626142 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c007cb0 con 0x7f7a940d1f90 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 -- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 msgr2=0x7f7a940cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940cf7c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c009d90 tx=0x7f7a8c02a660 comp rx=0 tx=0).stop 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 -- 172.21.15.28:0/2557626142 shutdown_connections 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940cf7c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 --2- 172.21.15.28:0/2557626142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a9415e660 0x7f7a9406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 -- 172.21.15.28:0/2557626142 >> 172.21.15.28:0/2557626142 conn(0x7f7a9408bfc0 msgr2=0x7f7a940d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 -- 172.21.15.28:0/2557626142 shutdown_connections 2024-09-10T10:52:40.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.486+0000 7f7a93e00640 1 -- 172.21.15.28:0/2557626142 wait complete. 2024-09-10T10:52:40.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.488+0000 7f7a93e00640 1 Processor -- start 2024-09-10T10:52:40.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.488+0000 7f7a93e00640 1 -- start start 2024-09-10T10:52:40.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a94155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 0x7f7a940b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7a940d3220 con 0x7f7a9415e660 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a940d2f40 con 0x7f7a940d1f90 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a940d33a0 con 0x7f7a940d0dd0 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 0x7f7a940b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 0x7f7a940b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48274/0 (socket says 172.21.15.28:48274) 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a93400640 1 -- 172.21.15.28:0/3360506018 learned_addr learned my addr 172.21.15.28:0/3360506018 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92a00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a94155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 -- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 msgr2=0x7f7a94155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a94155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 -- 172.21.15.28:0/3360506018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 msgr2=0x7f7a940b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 0x7f7a940b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.489+0000 7f7a92000640 1 -- 172.21.15.28:0/3360506018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a94156040 con 0x7f7a940d1f90 2024-09-10T10:52:40.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a92000640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940b5fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7a9415ea60 tx=0x7f7a8000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80017070 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8000c940 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000cd30 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a940b47c0 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.490+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a940b4d80 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.491+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a940cbf10 con 0x7f7a940d1f90 2024-09-10T10:52:40.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.492+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a8001a070 con 0x7f7a940d1f90 2024-09-10T10:52:40.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.493+0000 7f7a8aa00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.493+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f7a8008b4a0 con 0x7f7a940d1f90 2024-09-10T10:52:40.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.494+0000 7f7a92a00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.494+0000 7f7a92a00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7a84001780 tx=0x7f7a84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.497+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a940cbf10 con 0x7f7a940d1f90 2024-09-10T10:52:40.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.720+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a940d2440 con 0x7f7a940d1f90 2024-09-10T10:52:40.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.721+0000 7f7a8aa00640 1 -- 172.21.15.28:0/3360506018 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) ==== 95+0+4392 (secure 0 0 0) 0x7f7a80058ef0 con 0x7f7a940d1f90 2024-09-10T10:52:40.724 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:40.724 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":189,"btime":"2024-09-10T10:52:39:905015+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:39.904950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 189 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 msgr2=0x7f7a7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7a84001780 tx=0x7f7a84009040 comp rx=0 tx=0).stop 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 msgr2=0x7f7a940b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940b5fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7a9415ea60 tx=0x7f7a8000ecd0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 shutdown_connections 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a7406be30 0x7f7a7406e2f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a9415e660 0x7f7a940b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a940d1f90 0x7f7a940b5fc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 --2- 172.21.15.28:0/3360506018 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a940d0dd0 0x7f7a94155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 >> 172.21.15.28:0/3360506018 conn(0x7f7a9408bfc0 msgr2=0x7f7a940b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:40.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 shutdown_connections 2024-09-10T10:52:40.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.724+0000 7f7a93e00640 1 -- 172.21.15.28:0/3360506018 wait complete. 2024-09-10T10:52:40.742 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:40.743 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.939+0000 7fa05ce00640 1 Processor -- start 2024-09-10T10:52:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.939+0000 7fa05ce00640 1 -- start start 2024-09-10T10:52:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa05ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 0x7fa058162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa05ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 0x7fa058154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa05ce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa05815eb60 con 0x7fa0580af280 2024-09-10T10:52:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa05ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa0580b6db0 con 0x7fa0580b69d0 2024-09-10T10:52:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa05ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa05815ee40 con 0x7fa0580b0180 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa04d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa0580af280 0x7fa0580afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47220/0 (socket says 172.21.15.28:47220) 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa04d400640 1 -- 172.21.15.28:0/4197925665 learned_addr learned my addr 172.21.15.28:0/4197925665 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa056800640 1 --2- 172.21.15.28:0/4197925665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 0x7fa058154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa055e00640 1 --2- 172.21.15.28:0/4197925665 >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 0x7fa058162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa055400640 1 -- 172.21.15.28:0/4197925665 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2380223092 0 0) 0x7fa05815eb60 con 0x7fa0580af280 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.941+0000 7fa055400640 1 -- 172.21.15.28:0/4197925665 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa03c003520 con 0x7fa0580af280 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.942+0000 7fa056800640 1 -- 172.21.15.28:0/4197925665 >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 msgr2=0x7fa058162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.942+0000 7fa056800640 1 --2- 172.21.15.28:0/4197925665 >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 0x7fa058162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.942+0000 7fa056800640 1 -- 172.21.15.28:0/4197925665 >> v1:172.21.15.28:6789/0 conn(0x7fa0580af280 legacy=0x7fa0580afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.942+0000 7fa056800640 1 -- 172.21.15.28:0/4197925665 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05815f120 con 0x7fa0580b0180 2024-09-10T10:52:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.942+0000 7fa055e00640 1 --2- 172.21.15.28:0/4197925665 >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 0x7fa058162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa056800640 1 --2- 172.21.15.28:0/4197925665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 0x7fa058154d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa050009d90 tx=0x7fa05002a540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38fa03566827557 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa055400640 1 -- 172.21.15.28:0/4197925665 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050007740 con 0x7fa0580b0180 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa055400640 1 -- 172.21.15.28:0/4197925665 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0500078e0 con 0x7fa0580b0180 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa055400640 1 -- 172.21.15.28:0/4197925665 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050007c90 con 0x7fa0580b0180 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa05ce00640 1 -- 172.21.15.28:0/4197925665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 msgr2=0x7fa058154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.943+0000 7fa05ce00640 1 --2- 172.21.15.28:0/4197925665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 0x7fa058154d00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa050009d90 tx=0x7fa05002a540 comp rx=0 tx=0).stop 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 -- 172.21.15.28:0/4197925665 shutdown_connections 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 --2- 172.21.15.28:0/4197925665 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b0180 0x7fa058154d00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 --2- 172.21.15.28:0/4197925665 >> v2:172.21.15.138:3300/0 conn(0x7fa0580b69d0 0x7fa058162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 -- 172.21.15.28:0/4197925665 >> 172.21.15.28:0/4197925665 conn(0x7fa05806bc10 msgr2=0x7fa05806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 -- 172.21.15.28:0/4197925665 shutdown_connections 2024-09-10T10:52:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.944+0000 7fa05ce00640 1 -- 172.21.15.28:0/4197925665 wait complete. 2024-09-10T10:52:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.945+0000 7fa05ce00640 1 Processor -- start 2024-09-10T10:52:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.946+0000 7fa05ce00640 1 -- start start 2024-09-10T10:52:40.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.946+0000 7fa05ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0580af280 0x7fa058150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.946+0000 7fa05ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.946+0000 7fa05ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 0x7fa058152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa05ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa058155240 con 0x7fa0580b0180 2024-09-10T10:52:40.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa05ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa0580af710 con 0x7fa0580af280 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa05ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa0581553c0 con 0x7fa0580b69d0 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48288/0 (socket says 172.21.15.28:48288) 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 -- 172.21.15.28:0/2832626050 learned_addr learned my addr 172.21.15.28:0/2832626050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa055e00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0580af280 0x7fa058150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa056800640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 0x7fa058152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 -- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 msgr2=0x7fa058152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 0x7fa058152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 -- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0580af280 msgr2=0x7fa058150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0580af280 0x7fa058150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa04d400640 1 -- 172.21.15.28:0/2832626050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0580cd7f0 con 0x7fa0580b0180 2024-09-10T10:52:40.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.947+0000 7fa056800640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 0x7fa058152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:40.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa04d400640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c460 tx=0x7fa04000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040017070 con 0x7fa0580b0180 2024-09-10T10:52:40.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa040005ce0 con 0x7fa0580b0180 2024-09-10T10:52:40.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040007290 con 0x7fa0580b0180 2024-09-10T10:52:40.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0580cdad0 con 0x7fa0580b0180 2024-09-10T10:52:40.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.948+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0580ce090 con 0x7fa0580b0180 2024-09-10T10:52:40.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.949+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0580ad8a0 con 0x7fa0580b0180 2024-09-10T10:52:40.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.950+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa04001a070 con 0x7fa0580b0180 2024-09-10T10:52:40.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.951+0000 7fa04ea00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 0x7fa03406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:40.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.952+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fa04008aa10 con 0x7fa0580b0180 2024-09-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.952+0000 7fa055e00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 0x7fa03406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:40.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.953+0000 7fa055e00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 0x7fa03406e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa048009560 tx=0x7fa048009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:40.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:40.956+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa0580ad8a0 con 0x7fa0580b0180 2024-09-10T10:52:41.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.177+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa058070af0 con 0x7fa0580b0180 2024-09-10T10:52:41.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.178+0000 7fa04ea00640 1 -- 172.21.15.28:0/2832626050 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) ==== 95+0+4392 (secure 0 0 0) 0x7fa058070af0 con 0x7fa0580b0180 2024-09-10T10:52:41.180 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:41.180 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":189,"btime":"2024-09-10T10:52:39:905015+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8589,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":184}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:39.904950+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:41.180 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 189 2024-09-10T10:52:41.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 msgr2=0x7fa03406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 0x7fa03406e2f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa048009560 tx=0x7fa048009190 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 msgr2=0x7fa058152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c460 tx=0x7fa04000c920 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 shutdown_connections 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa03406be30 0x7fa03406e2f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa0580b69d0 0x7fa058152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0580b0180 0x7fa058152620 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 --2- 172.21.15.28:0/2832626050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0580af280 0x7fa058150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.180+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 >> 172.21.15.28:0/2832626050 conn(0x7fa05806bc10 msgr2=0x7fa0581547b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.181+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 shutdown_connections 2024-09-10T10:52:41.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.181+0000 7fa05ce00640 1 -- 172.21.15.28:0/2832626050 wait complete. 2024-09-10T10:52:41.198 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 200.00000001' led to DAMAGED state 2024-09-10T10:52:41.198 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 500.00000000' 2024-09-10T10:52:41.198 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:41.198 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:41.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:41.234 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:41.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:41.234 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:41.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:41.309 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.505+0000 7f7ac1c00640 1 Processor -- start 2024-09-10T10:52:41.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.505+0000 7f7ac1c00640 1 -- start start 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7ac1c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7abc06fdc0 0x7f7abc0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7ac1c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15b2f0 0x7f7abc164f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7ac1c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc072620 con 0x7f7abc06e680 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7ac1c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc15b780 con 0x7f7abc06fdc0 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7ac1c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc06ebf0 con 0x7f7abc15b2f0 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7abbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7abc06fdc0 0x7f7abc0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7abbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7abc06fdc0 0x7f7abc0b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48292/0 (socket says 172.21.15.28:48292) 2024-09-10T10:52:41.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.506+0000 7f7abbe00640 1 -- 172.21.15.28:0/3773365235 learned_addr learned my addr 172.21.15.28:0/3773365235 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.507+0000 7f7ac1200640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15b2f0 0x7f7abc164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.507+0000 7f7abbe00640 1 -- 172.21.15.28:0/3773365235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15b2f0 msgr2=0x7f7abc164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.507+0000 7f7abbe00640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15b2f0 0x7f7abc164f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.507+0000 7f7abbe00640 1 -- 172.21.15.28:0/3773365235 >> v1:172.21.15.138:6789/0 conn(0x7f7abc06e680 legacy=0x7f7abc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.507+0000 7f7abbe00640 1 -- 172.21.15.28:0/3773365235 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc06eed0 con 0x7f7abc06fdc0 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.508+0000 7f7abbe00640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc0b3b30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009870 tx=0x7f7aac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5e870a706ed0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.508+0000 7f7abaa00640 1 -- 172.21.15.28:0/3773365235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac037070 con 0x7f7abc06fdc0 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.508+0000 7f7abaa00640 1 -- 172.21.15.28:0/3773365235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac032440 con 0x7f7abc06fdc0 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.508+0000 7f7abaa00640 1 -- 172.21.15.28:0/3773365235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac032810 con 0x7f7abc06fdc0 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 -- 172.21.15.28:0/3773365235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 msgr2=0x7f7abc0b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc0b3b30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009870 tx=0x7f7aac02a270 comp rx=0 tx=0).stop 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 -- 172.21.15.28:0/3773365235 shutdown_connections 2024-09-10T10:52:41.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc15b2f0 0x7f7abc164f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/3773365235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc0b3b30 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 -- 172.21.15.28:0/3773365235 >> 172.21.15.28:0/3773365235 conn(0x7f7abc06c240 msgr2=0x7f7abc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 -- 172.21.15.28:0/3773365235 shutdown_connections 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.509+0000 7f7ac1c00640 1 -- 172.21.15.28:0/3773365235 wait complete. 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.511+0000 7f7ac1c00640 1 Processor -- start 2024-09-10T10:52:41.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.511+0000 7f7ac1c00640 1 -- start start 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.511+0000 7f7ac1c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06e680 0x7f7abc14bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.511+0000 7f7ac1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7ac1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15b2f0 0x7f7abc1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7ac1c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc0b4070 con 0x7f7abc06fdc0 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7ac1c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc06f490 con 0x7f7abc15b2f0 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7ac1c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc0b41f0 con 0x7f7abc06e680 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48306/0 (socket says 172.21.15.28:48306) 2024-09-10T10:52:41.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 -- 172.21.15.28:0/2425133916 learned_addr learned my addr 172.21.15.28:0/2425133916 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7ac1200640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15b2f0 0x7f7abc1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abbe00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06e680 0x7f7abc14bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 -- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06e680 msgr2=0x7f7abc14bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06e680 0x7f7abc14bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 -- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15b2f0 msgr2=0x7f7abc1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15b2f0 0x7f7abc1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.512+0000 7f7abb400640 1 -- 172.21.15.28:0/2425133916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc14c0a0 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7abb400640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c430 tx=0x7f7aa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80052d0 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa8005490 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc157170 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8007260 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.513+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc09b150 con 0x7f7abc06fdc0 2024-09-10T10:52:41.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.514+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc073c10 con 0x7f7abc06fdc0 2024-09-10T10:52:41.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.515+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7aa8007400 con 0x7f7abc06fdc0 2024-09-10T10:52:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.516+0000 7f7ab3e00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.516+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f7aa808ab40 con 0x7f7abc06fdc0 2024-09-10T10:52:41.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.516+0000 7f7abbe00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:41.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.517+0000 7f7abbe00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7aac034000 tx=0x7f7aac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:41.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.520+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7aa8058630 con 0x7f7abc06fdc0 2024-09-10T10:52:41.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.743+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7abc15a5c0 con 0x7f7abc06fdc0 2024-09-10T10:52:41.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.908+0000 7f7ab3e00640 1 -- 172.21.15.28:0/2425133916 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v190) ==== 161+0+0 (secure 0 0 0) 0x7f7aa805c930 con 0x7f7abc06fdc0 2024-09-10T10:52:41.910 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 msgr2=0x7f7a9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7aac034000 tx=0x7f7aac02e000 comp rx=0 tx=0).stop 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 msgr2=0x7f7abc154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c430 tx=0x7f7aa800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 shutdown_connections 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a9c06be30 0x7f7a9c06e2f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7abc15b2f0 0x7f7abc1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7abc06fdc0 0x7f7abc154390 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 --2- 172.21.15.28:0/2425133916 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7abc06e680 0x7f7abc14bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:41.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 >> 172.21.15.28:0/2425133916 conn(0x7f7abc06c240 msgr2=0x7f7abc0ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:41.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 shutdown_connections 2024-09-10T10:52:41.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:41.911+0000 7f7ac1c00640 1 -- 172.21.15.28:0/2425133916 wait complete. 2024-09-10T10:52:41.929 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:42.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.084+0000 7fb25a400640 1 Processor -- start 2024-09-10T10:52:42.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.084+0000 7fb25a400640 1 -- start start 2024-09-10T10:52:42.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.085+0000 7fb25a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb25a400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb254072de0 0x7fb25406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb25a400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb254146140 con 0x7fb254072700 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb25a400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb2540cd740 con 0x7fb254072de0 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb25a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb2540cda20 con 0x7fb254072320 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb254072de0 0x7fb25406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb254072de0 0x7fb25406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48310/0 (socket says 172.21.15.28:48310) 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 -- 172.21.15.28:0/4124223188 learned_addr learned my addr 172.21.15.28:0/4124223188 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb253400640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 -- 172.21.15.28:0/4124223188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 msgr2=0x7fb2540cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 -- 172.21.15.28:0/4124223188 >> v1:172.21.15.138:6789/0 conn(0x7fb254072700 legacy=0x7fb25406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb24be00640 1 -- 172.21.15.28:0/4124223188 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2540cdd00 con 0x7fb254072de0 2024-09-10T10:52:42.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.086+0000 7fb253400640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:42.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.087+0000 7fb24be00640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb25406e0b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009870 tx=0x7fb23c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba3fbb26733771a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:42.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb252a00640 1 -- 172.21.15.28:0/4124223188 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c037070 con 0x7fb254072de0 2024-09-10T10:52:42.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb252a00640 1 -- 172.21.15.28:0/4124223188 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c032440 con 0x7fb254072de0 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb252a00640 1 -- 172.21.15.28:0/4124223188 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c032810 con 0x7fb254072de0 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 -- 172.21.15.28:0/4124223188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 msgr2=0x7fb25406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb25406e0b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009870 tx=0x7fb23c02a180 comp rx=0 tx=0).stop 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 -- 172.21.15.28:0/4124223188 shutdown_connections 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb25406e0b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 --2- 172.21.15.28:0/4124223188 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 -- 172.21.15.28:0/4124223188 >> 172.21.15.28:0/4124223188 conn(0x7fb254067d70 msgr2=0x7fb25406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 -- 172.21.15.28:0/4124223188 shutdown_connections 2024-09-10T10:52:42.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.088+0000 7fb25a400640 1 -- 172.21.15.28:0/4124223188 wait complete. 2024-09-10T10:52:42.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.090+0000 7fb25a400640 1 Processor -- start 2024-09-10T10:52:42.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.090+0000 7fb25a400640 1 -- start start 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb254072700 0x7fb254134860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540cd520 con 0x7fb254072de0 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2540cd3a0 con 0x7fb254072700 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb25a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb25406ed00 con 0x7fb254072320 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb253e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb253e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48326/0 (socket says 172.21.15.28:48326) 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb253e00640 1 -- 172.21.15.28:0/3181561029 learned_addr learned my addr 172.21.15.28:0/3181561029 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:42.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb24be00640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb254072700 0x7fb254134860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.091+0000 7fb253400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 -- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 msgr2=0x7fb2540b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 -- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb254072700 msgr2=0x7fb254134860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb254072700 0x7fb254134860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 -- 172.21.15.28:0/3181561029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2540ca2c0 con 0x7fb254072de0 2024-09-10T10:52:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb253e00640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00c7b0 tx=0x7fb24c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c004b60 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c004d00 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00aa90 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.092+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2540ca5a0 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.093+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2540cab60 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.094+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2540cce60 con 0x7fb254072de0 2024-09-10T10:52:42.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.095+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb24c00ac30 con 0x7fb254072de0 2024-09-10T10:52:42.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.095+0000 7fb24b400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 0x7fb23006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:42.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.096+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fb24c08b890 con 0x7fb254072de0 2024-09-10T10:52:42.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.096+0000 7fb253400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 0x7fb23006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:42.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.097+0000 7fb253400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 0x7fb23006e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb24400ab60 tx=0x7fb244008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:42.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.100+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb24c020070 con 0x7fb254072de0 2024-09-10T10:52:42.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.351+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fb2540b1c70 con 0x7fb254072de0 2024-09-10T10:52:42.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.919+0000 7fb24b400640 1 -- 172.21.15.28:0/3181561029 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v191) ==== 101+0+0 (secure 0 0 0) 0x7fb24c059380 con 0x7fb254072de0 2024-09-10T10:52:42.921 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 msgr2=0x7fb23006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 0x7fb23006e2f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb24400ab60 tx=0x7fb244008040 comp rx=0 tx=0).stop 2024-09-10T10:52:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 msgr2=0x7fb2540ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00c7b0 tx=0x7fb24c00cc70 comp rx=0 tx=0).stop 2024-09-10T10:52:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 shutdown_connections 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb23006be30 0x7fb23006e2f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb254072de0 0x7fb2540ac550 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb254072700 0x7fb254134860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 --2- 172.21.15.28:0/3181561029 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb254072320 0x7fb2540b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 >> 172.21.15.28:0/3181561029 conn(0x7fb254067d70 msgr2=0x7fb2540746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 shutdown_connections 2024-09-10T10:52:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:42.923+0000 7fb25a400640 1 -- 172.21.15.28:0/3181561029 wait complete. 2024-09-10T10:52:42.944 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:43.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.020+0000 7fbaf9a16f40 1 Processor -- start 2024-09-10T10:52:43.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.020+0000 7fbaf9a16f40 1 -- start start 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 0x556cc12bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc13499c0 0x556cc1344d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556cc12bb530 con 0x556cc13463e0 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cc12bb2b0 con 0x556cc12ba8e0 2024-09-10T10:52:43.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556cc1348ee0 con 0x556cc13499c0 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48352/0 (socket says 172.21.15.28:48352) 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf6200640 1 -- 172.21.15.28:0/845684441 learned_addr learned my addr 172.21.15.28:0/845684441 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.021+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 0x556cc12bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf7600640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc13499c0 0x556cc1344d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 -- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc13499c0 msgr2=0x556cc1344d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc13499c0 0x556cc1344d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 -- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 msgr2=0x556cc12bd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 0x556cc12bd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 -- 172.21.15.28:0/845684441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc119cb30 con 0x556cc13463e0 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 0x556cc12bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:43.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf6200640 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fbae400a770 tx=0x7fbae400a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652036973740bc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf4e00640 1 -- 172.21.15.28:0/845684441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4018070 con 0x556cc13463e0 2024-09-10T10:52:43.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf4e00640 1 -- 172.21.15.28:0/845684441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4007600 con 0x556cc13463e0 2024-09-10T10:52:43.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.022+0000 7fbaf4e00640 1 -- 172.21.15.28:0/845684441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40079b0 con 0x556cc13463e0 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/845684441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 msgr2=0x556cc13447c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fbae400a770 tx=0x7fbae400a860 comp rx=0 tx=0).stop 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/845684441 shutdown_connections 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc13499c0 0x556cc1344d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13463e0 0x556cc13447c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/845684441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc12ba8e0 0x556cc12bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/845684441 >> 172.21.15.28:0/845684441 conn(0x556cc12bf0a0 msgr2=0x556cc1270070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/845684441 shutdown_connections 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.023+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/845684441 wait complete. 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 Processor -- start 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 -- start start 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc12ba8e0 0x556cc12a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc13463e0 0x556cc1343740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556cc1345cd0 con 0x556cc13499c0 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556cc13459f0 con 0x556cc13463e0 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.024+0000 7fbaf9a16f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556cc13481a0 con 0x556cc12ba8e0 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48362/0 (socket says 172.21.15.28:48362) 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 learned_addr learned my addr 172.21.15.28:0/2636882971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc13463e0 0x556cc1343740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc12ba8e0 0x556cc12a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc12ba8e0 msgr2=0x556cc12a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556cc12ba8e0 0x556cc12a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc13463e0 msgr2=0x556cc1343740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556cc13463e0 0x556cc1343740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.025+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc13426e0 con 0x556cc13499c0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaf7600640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000a4b0 tx=0x7fbaf0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaef400640 1 -- 172.21.15.28:0/2636882971 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0054040 con 0x556cc13499c0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaef400640 1 -- 172.21.15.28:0/2636882971 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaf0059450 con 0x556cc13499c0 2024-09-10T10:52:43.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaef400640 1 -- 172.21.15.28:0/2636882971 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0005b40 con 0x556cc13499c0 2024-09-10T10:52:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556cc1342980 con 0x556cc13499c0 2024-09-10T10:52:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.026+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc1342f40 con 0x556cc13499c0 2024-09-10T10:52:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.028+0000 7fbaef400640 1 -- 172.21.15.28:0/2636882971 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbaf0057070 con 0x556cc13499c0 2024-09-10T10:52:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaef400640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 0x7fbad806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaef400640 1 -- 172.21.15.28:0/2636882971 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fbaf00ce330 con 0x556cc13499c0 2024-09-10T10:52:43.032 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:43.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 0x556cc134ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc133dac0 con 0x556cc1344cc0 2024-09-10T10:52:43.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 0x7fbad806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.029+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 0x556cc134ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.030+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 0x7fbad806cd00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009720 tx=0x7fbae8005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.030+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 0x556cc134ac00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.032+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v118'536 uv510 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.032+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc134b140 con 0x556cc1344cc0 2024-09-10T10:52:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.034+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v118'537 uv537 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.034+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb3a0 con 0x556cc1344cc0 2024-09-10T10:52:43.037 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:43.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.036+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v118'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.036+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ec750 con 0x556cc1344cc0 2024-09-10T10:52:43.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.037+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v118'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.037+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ecaf0 con 0x556cc1344cc0 2024-09-10T10:52:43.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.040+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v118'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.040+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.042+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v118'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.042+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.044+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v118'542 uv516 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.044+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.045+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v118'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.045+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.048 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.047+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v118'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.047+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.048+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v118'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.048+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ece90 con 0x556cc1344cc0 2024-09-10T10:52:43.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.050+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v118'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.050+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.052+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v118'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.052+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.054+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v118'548 uv523 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.054+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.055+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v118'549 uv549 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.055+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.058 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.056+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v118'550 uv550 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.056+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.058+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v118'551 uv551 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.058+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.059+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v118'552 uv552 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.059+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13eb740 con 0x556cc1344cc0 2024-09-10T10:52:43.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.061+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v118'553 uv553 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.061+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.063+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v118'554 uv554 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.063+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.064+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v118'555 uv529 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.064+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.066+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v118'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.066+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.069 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:43.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.067+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v118'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.068+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.069+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v118'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.069+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.071+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v118'559 uv559 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.071+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.072+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v118'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.072+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.075+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v118'561 uv535 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.075+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.076+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v118'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.076+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.079 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:43.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.078+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v118'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.078+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.079+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v118'564 uv564 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.079+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.080+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v118'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.080+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ebb00 con 0x556cc1344cc0 2024-09-10T10:52:43.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.083+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v118'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc1344cc0 2024-09-10T10:52:43.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.083+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 0x556cc13eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.083+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ef3e0 con 0x556cc13ec010 2024-09-10T10:52:43.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.083+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 0x556cc13eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.083+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 0x556cc13eeea0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.085+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v118'396 uv378 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fbae80164a0 con 0x556cc13ec010 2024-09-10T10:52:43.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.085+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ef780 con 0x556cc13ec010 2024-09-10T10:52:43.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.086+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v118'397 uv397 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.086+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13efb20 con 0x556cc13ec010 2024-09-10T10:52:43.090 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:43.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.088+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v118'398 uv398 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.088+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13efec0 con 0x556cc13ec010 2024-09-10T10:52:43.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.090+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v118'399 uv399 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.090+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0260 con 0x556cc13ec010 2024-09-10T10:52:43.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.091+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v118'400 uv400 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.091+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0600 con 0x556cc13ec010 2024-09-10T10:52:43.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.092+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v118'401 uv383 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.092+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f09a0 con 0x556cc13ec010 2024-09-10T10:52:43.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.094+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v118'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.094+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.095+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v118'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.097 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:43.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.095+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.097+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v118'404 uv404 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.097+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.098+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v118'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.098+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.100+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v118'406 uv395 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.100+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.101+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v118'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.101+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.102+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v118'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.104 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:43.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.102+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f5430 con 0x556cc13ec010 2024-09-10T10:52:43.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.104+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v118'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.104+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f5430 con 0x556cc13ec010 2024-09-10T10:52:43.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.106+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v118'410 uv410 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.106+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.107+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v118'411 uv394 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.107+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.108+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v118'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.108+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.111 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:43.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.110+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v118'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.110+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f1d50 con 0x556cc13ec010 2024-09-10T10:52:43.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.112+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v118'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.112+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f1d50 con 0x556cc13ec010 2024-09-10T10:52:43.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.113+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v118'415 uv415 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.113+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f0d40 con 0x556cc13ec010 2024-09-10T10:52:43.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.115+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v118'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13ec010 2024-09-10T10:52:43.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.115+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x556cc13f1640 0x556cc13f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.115+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f3f70 con 0x556cc13f1640 2024-09-10T10:52:43.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.115+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x556cc13f1640 0x556cc13f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.116+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x556cc13f1640 0x556cc13f3a30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.117+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v118'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.117+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f3f70 con 0x556cc13f1640 2024-09-10T10:52:43.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.119+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v118'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.119+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f4360 con 0x556cc13f1640 2024-09-10T10:52:43.122 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:43.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.120+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v118'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.120+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f5370 con 0x556cc13f1640 2024-09-10T10:52:43.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.122+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v118'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.122+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f5370 con 0x556cc13f1640 2024-09-10T10:52:43.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.123+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v118'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.123+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f4360 con 0x556cc13f1640 2024-09-10T10:52:43.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.124+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v118'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.124+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x556cc13f49e0 0x556cc13f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.125+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f73a0 con 0x556cc13f49e0 2024-09-10T10:52:43.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.125+0000 7fbaf7600640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x556cc13f49e0 0x556cc13f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.125+0000 7fbaf7600640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x556cc13f49e0 0x556cc13f6dd0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.126+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v118'79 uv78 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.126+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f78b0 con 0x556cc13f49e0 2024-09-10T10:52:43.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.128+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v118'80 uv80 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.128+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f7db0 con 0x556cc13f49e0 2024-09-10T10:52:43.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.129+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v118'81 uv81 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.131 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:43.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.129+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8670 con 0x556cc13f49e0 2024-09-10T10:52:43.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.130+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v118'82 uv82 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.130+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8a10 con 0x556cc13f1640 2024-09-10T10:52:43.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.131+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v118'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.131+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8a10 con 0x556cc13f1640 2024-09-10T10:52:43.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.133+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v118'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.133+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8a10 con 0x556cc13f1640 2024-09-10T10:52:43.136 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:43.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.134+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v118'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.134+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f9a20 con 0x556cc13f1640 2024-09-10T10:52:43.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.135+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v118'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.135+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f9a20 con 0x556cc13f1640 2024-09-10T10:52:43.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.136+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v118'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.136+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8a10 con 0x556cc13f1640 2024-09-10T10:52:43.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.137+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v118'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.138+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8a10 con 0x556cc13f49e0 2024-09-10T10:52:43.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.139+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v118'193 uv187 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.139+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f8db0 con 0x556cc13f49e0 2024-09-10T10:52:43.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.141+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v118'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.141+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f9150 con 0x556cc13f49e0 2024-09-10T10:52:43.144 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:43.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.142+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v118'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.142+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fa500 con 0x556cc13f49e0 2024-09-10T10:52:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.144+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v118'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.144+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fa500 con 0x556cc13f49e0 2024-09-10T10:52:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.145+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v118'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.145+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.146+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v118'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.146+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.147+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v118'199 uv192 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.147+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.149+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v118'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.149+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.150+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v118'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.152 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:43.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.150+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.152+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v118'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.152+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f49e0 2024-09-10T10:52:43.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.153+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v118'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.153+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f1640 2024-09-10T10:52:43.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.154+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v118'86 uv85 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.154+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f1640 2024-09-10T10:52:43.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.155+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v118'87 uv87 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.155+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f1640 2024-09-10T10:52:43.158 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:43.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.157+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v118'88 uv88 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.157+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f1640 2024-09-10T10:52:43.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.158+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v118'89 uv89 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.158+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13f94f0 con 0x556cc13f1640 2024-09-10T10:52:43.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.159+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v118'90 uv90 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13f1640 2024-09-10T10:52:43.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.159+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x556cc13f9b70 0x556cc13fbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.159+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc4e0 con 0x556cc13f9b70 2024-09-10T10:52:43.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.160+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x556cc13f9b70 0x556cc13fbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.160+0000 7fbaf6c00640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x556cc13f9b70 0x556cc13fbf60 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.161+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v118'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.161+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.163+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v118'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.163+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.166 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:43.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.164+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v118'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.164+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fd8c0 con 0x556cc13f9b70 2024-09-10T10:52:43.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.165+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v118'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.165+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fd8c0 con 0x556cc13f9b70 2024-09-10T10:52:43.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.166+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v118'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.166+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.167+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v118'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.168+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.169+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v118'172 uv166 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.169+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.170+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v118'173 uv173 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.170+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.173 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:43.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.172+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v118'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.172+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.173+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v118'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.173+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.174+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v118'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.174+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.175+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v118'177 uv171 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.175+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.177+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v118'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.177+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.180 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.178+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v118'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.178+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fc8b0 con 0x556cc13f9b70 2024-09-10T10:52:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.179+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v118'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.179+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ff690 con 0x556cc13f9b70 2024-09-10T10:52:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.181+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v118'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.181+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.183+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v118'78 uv77 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.183+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.185+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v118'79 uv79 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.185+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.186+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v118'80 uv80 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.188 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:43.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.186+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.187+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v118'81 uv81 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.187+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.188+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v118'105 uv104 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.188+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.190+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v118'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.190+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.193 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.191+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v118'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.191+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fdd00 con 0x556cc13f49e0 2024-09-10T10:52:43.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.192+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v118'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.192+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fdd00 con 0x556cc13f49e0 2024-09-10T10:52:43.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.193+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v118'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.193+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13fccf0 con 0x556cc13f49e0 2024-09-10T10:52:43.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.194+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v118'110 uv110 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbaf0056030 con 0x556cc13f49e0 2024-09-10T10:52:43.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.194+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556cc13fd370 0x556cc13ff760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.194+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ffd50 con 0x556cc13fd370 2024-09-10T10:52:43.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.194+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556cc13fd370 0x556cc13ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.195+0000 7fbaf6200640 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556cc13fd370 0x556cc13ff760 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.196+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v118'89 uv88 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13fd370 2024-09-10T10:52:43.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.196+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ffd50 con 0x556cc13fd370 2024-09-10T10:52:43.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.197+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v118'90 uv90 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13fd370 2024-09-10T10:52:43.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.197+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc13ffd50 con 0x556cc13fd370 2024-09-10T10:52:43.200 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:43.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.199+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v118'91 uv91 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13fd370 2024-09-10T10:52:43.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.199+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13fd370 2024-09-10T10:52:43.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.200+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v118'92 uv92 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13fd370 2024-09-10T10:52:43.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.200+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13fd370 2024-09-10T10:52:43.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.201+0000 7fbaf6200640 1 -- 172.21.15.28:0/2636882971 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v118'93 uv93 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fbae40079d0 con 0x556cc13fd370 2024-09-10T10:52:43.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.201+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.202+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v118'216 uv208 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.203+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.204+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v118'217 uv217 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.204+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.207 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:43.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.205+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v118'218 uv218 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.205+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.206+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v118'219 uv219 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.206+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.207+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v118'220 uv220 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.207+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.209+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v118'221 uv215 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.209+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.210+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v118'222 uv222 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.210+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.212 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:43.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.211+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v118'223 uv223 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.211+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1401150 con 0x556cc13f9b70 2024-09-10T10:52:43.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.212+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v118'224 uv224 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.212+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1401150 con 0x556cc13f9b70 2024-09-10T10:52:43.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.213+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v118'225 uv225 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.213+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc1400140 con 0x556cc13f9b70 2024-09-10T10:52:43.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.214+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v118'226 uv226 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.214+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x556cc14008a0 con 0x556cc13f9b70 2024-09-10T10:52:43.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf6c00640 1 -- 172.21.15.28:0/2636882971 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v118'227 uv227 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fbae80168e0 con 0x556cc13f9b70 2024-09-10T10:52:43.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x556cc13f1640 msgr2=0x556cc13f3a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x556cc13f1640 0x556cc13f3a30 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x556cc13f9b70 msgr2=0x556cc13fbf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x556cc13f9b70 0x556cc13fbf60 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556cc13fd370 msgr2=0x556cc13ff760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x556cc13fd370 0x556cc13ff760 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x556cc13f49e0 msgr2=0x556cc13f6dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x556cc13f49e0 0x556cc13f6dd0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 msgr2=0x556cc13eeea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 0x556cc13eeea0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 reap_dead start 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf7600640 1 -- 172.21.15.28:0/2636882971 reap_dead start 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 msgr2=0x556cc134ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 0x556cc134ac00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 msgr2=0x7fbad806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 0x7fbad806cd00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009720 tx=0x7fbae8005ac0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 msgr2=0x556cc1344780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000a4b0 tx=0x7fbaf0003330 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 shutdown_connections 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556cc13ec010 0x556cc13eeea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x556cc1344cc0 0x556cc134ac00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbad806a840 0x7fbad806cd00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 --2- 172.21.15.28:0/2636882971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556cc13499c0 0x556cc1344780 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 >> 172.21.15.28:0/2636882971 conn(0x556cc12bf0a0 msgr2=0x556cc12bd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 shutdown_connections 2024-09-10T10:52:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.216+0000 7fbaf9a16f40 1 -- 172.21.15.28:0/2636882971 wait complete. 2024-09-10T10:52:43.223 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 500.00000000 - 2024-09-10T10:52:43.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.337+0000 7fca7d398f40 1 Processor -- start 2024-09-10T10:52:43.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.337+0000 7fca7d398f40 1 -- start start 2024-09-10T10:52:43.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fea3e0 0x55ba57fe0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57fed9c0 0x55ba57f47160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ba57f5f530 con 0x55ba57f14470 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba57ebcb30 con 0x55ba57fea3e0 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.338+0000 7fca7d398f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ba57fecee0 con 0x55ba57fed9c0 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48366/0 (socket says 172.21.15.28:48366) 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 -- 172.21.15.28:0/2938843549 learned_addr learned my addr 172.21.15.28:0/2938843549 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca73e00640 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fea3e0 0x55ba57fe0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7b000640 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57fed9c0 0x55ba57f47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 -- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57fed9c0 msgr2=0x55ba57f47160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57fed9c0 0x55ba57f47160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 -- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fea3e0 msgr2=0x55ba57fe0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fea3e0 0x55ba57fe0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 -- 172.21.15.28:0/2938843549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba57f16930 con 0x55ba57f14470 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.339+0000 7fca7a600640 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fca6c001d80 tx=0x7fca6c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cb924bf0b79f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca79200640 1 -- 172.21.15.28:0/2938843549 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00bd70 con 0x55ba57f14470 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca79200640 1 -- 172.21.15.28:0/2938843549 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6c00f400 con 0x55ba57f14470 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca79200640 1 -- 172.21.15.28:0/2938843549 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00f7f0 con 0x55ba57f14470 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 -- 172.21.15.28:0/2938843549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 msgr2=0x55ba57fe4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fca6c001d80 tx=0x7fca6c001e70 comp rx=0 tx=0).stop 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 -- 172.21.15.28:0/2938843549 shutdown_connections 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57fed9c0 0x55ba57f47160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fea3e0 0x55ba57fe0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 --2- 172.21.15.28:0/2938843549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57f14470 0x55ba57fe4110 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 -- 172.21.15.28:0/2938843549 >> 172.21.15.28:0/2938843549 conn(0x55ba57f630a0 msgr2=0x55ba57f63480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 -- 172.21.15.28:0/2938843549 shutdown_connections 2024-09-10T10:52:43.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.340+0000 7fca7d398f40 1 -- 172.21.15.28:0/2938843549 wait complete. 2024-09-10T10:52:43.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.341+0000 7fca7d398f40 1 Processor -- start 2024-09-10T10:52:43.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.341+0000 7fca7d398f40 1 -- start start 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57f14470 0x55ba57f60c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 0x55ba57fd64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fed9c0 0x55ba57fda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ba57f16c10 con 0x55ba57fea3e0 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba57f45ea0 con 0x55ba57fed9c0 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7d398f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ba57fec900 con 0x55ba57f14470 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 0x55ba57fd64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 0x55ba57fd64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48380/0 (socket says 172.21.15.28:48380) 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca73e00640 1 -- 172.21.15.28:0/454685161 learned_addr learned my addr 172.21.15.28:0/454685161 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.342+0000 7fca7a600640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57f14470 0x55ba57f60c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca7b000640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fed9c0 0x55ba57fda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 -- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57f14470 msgr2=0x55ba57f60c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ba57f14470 0x55ba57f60c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 -- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fed9c0 msgr2=0x55ba57fda9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba57fed9c0 0x55ba57fda9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 -- 172.21.15.28:0/454685161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba57ff0210 con 0x55ba57fea3e0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca73e00640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 0x55ba57fd64d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fca64025d10 tx=0x7fca64002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca72a00640 1 -- 172.21.15.28:0/454685161 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64055040 con 0x55ba57fea3e0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca72a00640 1 -- 172.21.15.28:0/454685161 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca64051ab0 con 0x55ba57fea3e0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca72a00640 1 -- 172.21.15.28:0/454685161 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64051e60 con 0x55ba57fea3e0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.343+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba57ff04f0 con 0x55ba57fea3e0 2024-09-10T10:52:43.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.344+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba57ff0ab0 con 0x55ba57fea3e0 2024-09-10T10:52:43.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.345+0000 7fca72a00640 1 -- 172.21.15.28:0/454685161 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca64058070 con 0x55ba57fea3e0 2024-09-10T10:52:43.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.346+0000 7fca72a00640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca5806a810 0x7fca5806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.346+0000 7fca72a00640 1 -- 172.21.15.28:0/454685161 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fca640cdf90 con 0x55ba57fea3e0 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.347+0000 7fca7a600640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca5806a810 0x7fca5806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.347+0000 7fca7d398f40 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ba57fd6a10 0x55ba580865f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.347+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x55ba57f61150 con 0x55ba57fd6a10 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.347+0000 7fca7b000640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ba57fd6a10 0x55ba580865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.347+0000 7fca7a600640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca5806a810 0x7fca5806ccd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca6c003040 tx=0x7fca6c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.348+0000 7fca7b000640 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ba57fd6a10 0x55ba580865f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7b000640 1 -- 172.21.15.28:0/454685161 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [writefull 0~80] v118'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fca7400d960 con 0x55ba57fd6a10 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ba57fd6a10 msgr2=0x55ba580865f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ba57fd6a10 0x55ba580865f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca5806a810 msgr2=0x7fca5806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca5806a810 0x7fca5806ccd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fca6c003040 tx=0x7fca6c009000 comp rx=0 tx=0).stop 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 msgr2=0x55ba57fd64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 --2- 172.21.15.28:0/454685161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ba57fea3e0 0x55ba57fd64d0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fca64025d10 tx=0x7fca64002fb0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7b000640 1 -- 172.21.15.28:0/454685161 reap_dead start 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 shutdown_connections 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 >> 172.21.15.28:0/454685161 conn(0x55ba57f630a0 msgr2=0x55ba57f63480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.349+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 shutdown_connections 2024-09-10T10:52:43.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.350+0000 7fca7d398f40 1 -- 172.21.15.28:0/454685161 wait complete. 2024-09-10T10:52:43.356 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.552+0000 7f025e000640 1 Processor -- start 2024-09-10T10:52:43.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.553+0000 7f025e000640 1 -- start start 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f025e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02580af650 0x7f0258162e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f025e000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 0x7f02580ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f025e000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0258163550 con 0x7f025815cfc0 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f025e000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f02580afac0 con 0x7f02580af650 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f025e000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02580cd2a0 con 0x7f02580cc930 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02580af650 0x7f0258162e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02580af650 0x7f0258162e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48392/0 (socket says 172.21.15.28:48392) 2024-09-10T10:52:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 -- 172.21.15.28:0/207430987 learned_addr learned my addr 172.21.15.28:0/207430987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f0257e00640 1 --2- 172.21.15.28:0/207430987 >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 0x7f02580ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 -- 172.21.15.28:0/207430987 >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 msgr2=0x7f02580ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 --2- 172.21.15.28:0/207430987 >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 0x7f02580ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 -- 172.21.15.28:0/207430987 >> v1:172.21.15.138:6789/0 conn(0x7f025815cfc0 legacy=0x7f02580b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f024fe00640 1 -- 172.21.15.28:0/207430987 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0258163830 con 0x7f02580af650 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.554+0000 7f0257e00640 1 --2- 172.21.15.28:0/207430987 >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 0x7f02580ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.555+0000 7f024fe00640 1 --2- 172.21.15.28:0/207430987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f0258162e60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0258071f70 tx=0x7f024002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e824289f9b411cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f0256a00640 1 -- 172.21.15.28:0/207430987 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240037070 con 0x7f02580af650 2024-09-10T10:52:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f0256a00640 1 -- 172.21.15.28:0/207430987 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0240032440 con 0x7f02580af650 2024-09-10T10:52:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f0256a00640 1 -- 172.21.15.28:0/207430987 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240032810 con 0x7f02580af650 2024-09-10T10:52:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 -- 172.21.15.28:0/207430987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 msgr2=0x7f0258162e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 --2- 172.21.15.28:0/207430987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f0258162e60 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0258071f70 tx=0x7f024002a270 comp rx=0 tx=0).stop 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 -- 172.21.15.28:0/207430987 shutdown_connections 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 --2- 172.21.15.28:0/207430987 >> v2:172.21.15.138:3300/0 conn(0x7f02580cc930 0x7f02580ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 --2- 172.21.15.28:0/207430987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f0258162e60 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 -- 172.21.15.28:0/207430987 >> 172.21.15.28:0/207430987 conn(0x7f025806bdf0 msgr2=0x7f025806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 -- 172.21.15.28:0/207430987 shutdown_connections 2024-09-10T10:52:43.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.556+0000 7f025e000640 1 -- 172.21.15.28:0/207430987 wait complete. 2024-09-10T10:52:43.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.558+0000 7f025e000640 1 Processor -- start 2024-09-10T10:52:43.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.558+0000 7f025e000640 1 -- start start 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 0x7f0258156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 0x7f02581ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02580b3e10 con 0x7f02580af650 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02580b31e0 con 0x7f025815cfc0 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f025e000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0258072e40 con 0x7f02580cc930 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f0257400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f0257400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48406/0 (socket says 172.21.15.28:48406) 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.559+0000 7f0257400640 1 -- 172.21.15.28:0/4267784746 learned_addr learned my addr 172.21.15.28:0/4267784746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257e00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 0x7f02581ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f024fe00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 0x7f0258156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 -- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 msgr2=0x7f0258156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 0x7f0258156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 -- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 msgr2=0x7f02581ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 0x7f02581ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 -- 172.21.15.28:0/4267784746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02580b0620 con 0x7f02580af650 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257e00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 0x7f02581ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f024fe00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 0x7f0258156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f0257400640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f024800c3f0 tx=0x7f024800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480052d0 con 0x7f02580af650 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0248005490 con 0x7f02580af650 2024-09-10T10:52:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248007260 con 0x7f02580af650 2024-09-10T10:52:43.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02580b0900 con 0x7f02580af650 2024-09-10T10:52:43.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.560+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02580b0ec0 con 0x7f02580af650 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.562+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02580ccd60 con 0x7f02580af650 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.563+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0248007400 con 0x7f02580af650 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.563+0000 7f024f400640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.564+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f024808abc0 con 0x7f02580af650 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.564+0000 7f024fe00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:43.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.564+0000 7f024fe00640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 0x7f023406e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0240034000 tx=0x7f024002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:43.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.568+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0248058d00 con 0x7f02580af650 2024-09-10T10:52:43.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.789+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f02580d0900 con 0x7f02580af650 2024-09-10T10:52:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.920+0000 7f024f400640 1 -- 172.21.15.28:0/4267784746 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v192) ==== 162+0+0 (secure 0 0 0) 0x7f0248058600 con 0x7f02580af650 2024-09-10T10:52:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 msgr2=0x7f023406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 0x7f023406e2f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0240034000 tx=0x7f024002e000 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 msgr2=0x7f02580ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f024800c3f0 tx=0x7f024800c8b0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 shutdown_connections 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f023406be30 0x7f023406e2f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f025815cfc0 0x7f02581ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02580cc930 0x7f0258156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.922+0000 7f025e000640 1 --2- 172.21.15.28:0/4267784746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02580af650 0x7f02580ad020 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.923+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 >> 172.21.15.28:0/4267784746 conn(0x7f025806bdf0 msgr2=0x7f02580ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.923+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 shutdown_connections 2024-09-10T10:52:43.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:43.923+0000 7f025e000640 1 -- 172.21.15.28:0/4267784746 wait complete. 2024-09-10T10:52:43.930 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:43.928+0000 7ffbba800640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-10T10:52:43.930 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:43.928+0000 7ffbba800640 -1 mds.0.193 unhandled write error (22) Invalid argument, force readonly... 2024-09-10T10:52:43.944 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:44.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.098+0000 7f9f21800640 1 Processor -- start 2024-09-10T10:52:44.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.098+0000 7f9f21800640 1 -- start start 2024-09-10T10:52:44.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.099+0000 7f9f21800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f1c162060 0x7f9f1c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.099+0000 7f9f21800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.099+0000 7f9f21800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f1c06d9f0 con 0x7f9f1c161c80 2024-09-10T10:52:44.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.099+0000 7f9f21800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f1c06dcd0 con 0x7f9f1c162060 2024-09-10T10:52:44.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.099+0000 7f9f21800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1c06dfb0 con 0x7f9f1c0b64b0 2024-09-10T10:52:44.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.100+0000 7f9f1aa00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9f1c161c80 0x7f9f1c06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:47360/0 (socket says 172.21.15.28:47360) 2024-09-10T10:52:44.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.100+0000 7f9f1aa00640 1 -- 172.21.15.28:0/3564720007 learned_addr learned my addr 172.21.15.28:0/3564720007 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:44.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.100+0000 7f9f1a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f1c162060 0x7f9f1c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.100+0000 7f9f1b400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.100+0000 7f9f19600640 1 -- 172.21.15.28:0/3564720007 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3130261277 0 0) 0x7f9f1c06d9f0 con 0x7f9f1c161c80 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f19600640 1 -- 172.21.15.28:0/3564720007 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f00003520 con 0x7f9f1c161c80 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f1a000640 1 -- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 msgr2=0x7f9f1c0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f1a000640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f1a000640 1 -- 172.21.15.28:0/3564720007 >> v1:172.21.15.138:6789/0 conn(0x7f9f1c161c80 legacy=0x7f9f1c06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f1a000640 1 -- 172.21.15.28:0/3564720007 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c0d01f0 con 0x7f9f1c162060 2024-09-10T10:52:44.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.101+0000 7f9f1b400640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.102+0000 7f9f1a000640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b5f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097a0 tx=0x7f9f0802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509c5e4ad174fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.102+0000 7f9f19600640 1 -- 172.21.15.28:0/3564720007 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08037070 con 0x7f9f1c162060 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.102+0000 7f9f19600640 1 -- 172.21.15.28:0/3564720007 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f08032440 con 0x7f9f1c162060 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.102+0000 7f9f19600640 1 -- 172.21.15.28:0/3564720007 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f080327f0 con 0x7f9f1c162060 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 -- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c162060 msgr2=0x7f9f1c0b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b5f70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9f080097a0 tx=0x7f9f0802a180 comp rx=0 tx=0).stop 2024-09-10T10:52:44.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 -- 172.21.15.28:0/3564720007 shutdown_connections 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 --2- 172.21.15.28:0/3564720007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b5f70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 -- 172.21.15.28:0/3564720007 >> 172.21.15.28:0/3564720007 conn(0x7f9f1c0d44f0 msgr2=0x7f9f1c0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 -- 172.21.15.28:0/3564720007 shutdown_connections 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.103+0000 7f9f21800640 1 -- 172.21.15.28:0/3564720007 wait complete. 2024-09-10T10:52:44.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.104+0000 7f9f21800640 1 Processor -- start 2024-09-10T10:52:44.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.104+0000 7f9f21800640 1 -- start start 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c161c80 0x7f9f1c15a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c0cf360 con 0x7f9f1c162060 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1c0cf1e0 con 0x7f9f1c161c80 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.105+0000 7f9f21800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f1c0cf4e0 con 0x7f9f1c0b64b0 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44936/0 (socket says 172.21.15.28:44936) 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 -- 172.21.15.28:0/2341264567 learned_addr learned my addr 172.21.15.28:0/2341264567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1aa00640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1a000640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c161c80 0x7f9f1c15a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 -- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 msgr2=0x7f9f1c158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 -- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c161c80 msgr2=0x7f9f1c15a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c161c80 0x7f9f1c15a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 -- 172.21.15.28:0/2341264567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1c0cb5c0 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.106+0000 7f9f1b400640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400efc0 tx=0x7f9f14009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.107+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14018030 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.107+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1400cde0 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.107+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14003bb0 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.107+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1c0cb880 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.107+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1c0cbe40 con 0x7f9f1c162060 2024-09-10T10:52:44.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.108+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c0b5f70 con 0x7f9f1c162060 2024-09-10T10:52:44.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.109+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f14005d10 con 0x7f9f1c162060 2024-09-10T10:52:44.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.110+0000 7f9f12a00640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 0x7f9efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.110+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f9f1408bfe0 con 0x7f9f1c162060 2024-09-10T10:52:44.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.110+0000 7f9f1aa00640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 0x7f9efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.111+0000 7f9f1aa00640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 0x7f9efc06e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005880 tx=0x7f9f0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.114+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f14022070 con 0x7f9f1c162060 2024-09-10T10:52:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.335+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f1c0ba470 con 0x7f9f1c162060 2024-09-10T10:52:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.335+0000 7f9f12a00640 1 -- 172.21.15.28:0/2341264567 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) ==== 95+0+4401 (secure 0 0 0) 0x7f9f14010040 con 0x7f9f1c162060 2024-09-10T10:52:44.337 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:44.338 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":193,"btime":"2024-09-10T10:52:43:923728+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:43.923710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":193,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 193 2024-09-10T10:52:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 msgr2=0x7f9efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 0x7f9efc06e2f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c005880 tx=0x7f9f0c009040 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 msgr2=0x7f9f1c0b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400efc0 tx=0x7f9f14009f90 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 shutdown_connections 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9efc06be30 0x7f9efc06e2f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9f1c162060 0x7f9f1c0b8f00 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f1c161c80 0x7f9f1c15a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 --2- 172.21.15.28:0/2341264567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f1c0b64b0 0x7f9f1c158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 >> 172.21.15.28:0/2341264567 conn(0x7f9f1c0d44f0 msgr2=0x7f9f1c0b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 shutdown_connections 2024-09-10T10:52:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.337+0000 7f9f21800640 1 -- 172.21.15.28:0/2341264567 wait complete. 2024-09-10T10:52:44.657 INFO:tasks.ceph.mds.a.smithi028.stderr: -26> 2024-09-10T10:52:39.902+0000 7fc452800640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:52:44.696 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:52:44.758 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:44.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.913+0000 7f5adb200640 1 Processor -- start 2024-09-10T10:52:44.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.913+0000 7f5adb200640 1 -- start start 2024-09-10T10:52:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5adb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 0x7f5adc0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5adb200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5adc0af1b0 0x7f5adc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5adb200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5adc0b5470 con 0x7f5adc15bef0 2024-09-10T10:52:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5adb200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5adc0b3640 con 0x7f5adc0af1b0 2024-09-10T10:52:44.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5adb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc0b5190 con 0x7f5adc0b40e0 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 0x7f5adc0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5ad9e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5adc15bef0 0x7f5adc0b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:47388/0 (socket says 172.21.15.28:47388) 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5ad9e00640 1 -- 172.21.15.28:0/2910730737 learned_addr learned my addr 172.21.15.28:0/2910730737 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.914+0000 7f5ada800640 1 --2- 172.21.15.28:0/2910730737 >> v2:172.21.15.138:3300/0 conn(0x7f5adc0af1b0 0x7f5adc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.915+0000 7f5ad3e00640 1 -- 172.21.15.28:0/2910730737 >> v2:172.21.15.138:3300/0 conn(0x7f5adc0af1b0 msgr2=0x7f5adc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.915+0000 7f5ad3e00640 1 --2- 172.21.15.28:0/2910730737 >> v2:172.21.15.138:3300/0 conn(0x7f5adc0af1b0 0x7f5adc071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.915+0000 7f5ad3e00640 1 -- 172.21.15.28:0/2910730737 >> v1:172.21.15.138:6789/0 conn(0x7f5adc15bef0 legacy=0x7f5adc0b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.915+0000 7f5ad3e00640 1 -- 172.21.15.28:0/2910730737 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc0b38e0 con 0x7f5adc0b40e0 2024-09-10T10:52:44.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.915+0000 7f5ad9400640 1 -- 172.21.15.28:0/2910730737 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176625974 0 0) 0x7f5adc0b5470 con 0x7f5adc15bef0 2024-09-10T10:52:44.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.916+0000 7f5ad3e00640 1 --2- 172.21.15.28:0/2910730737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 0x7f5adc0aec70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77829d63e24d6dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5ad9400640 1 -- 172.21.15.28:0/2910730737 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037070 con 0x7f5adc0b40e0 2024-09-10T10:52:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5ad9400640 1 -- 172.21.15.28:0/2910730737 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac402ae50 con 0x7f5adc0b40e0 2024-09-10T10:52:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5ad9400640 1 -- 172.21.15.28:0/2910730737 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4032660 con 0x7f5adc0b40e0 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 -- 172.21.15.28:0/2910730737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 msgr2=0x7f5adc0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 --2- 172.21.15.28:0/2910730737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 0x7f5adc0aec70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40097a0 tx=0x7f5ac402a100 comp rx=0 tx=0).stop 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 -- 172.21.15.28:0/2910730737 shutdown_connections 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 --2- 172.21.15.28:0/2910730737 >> v2:172.21.15.138:3300/0 conn(0x7f5adc0af1b0 0x7f5adc071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 --2- 172.21.15.28:0/2910730737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc0b40e0 0x7f5adc0aec70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 -- 172.21.15.28:0/2910730737 >> 172.21.15.28:0/2910730737 conn(0x7f5adc06c240 msgr2=0x7f5adc0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 -- 172.21.15.28:0/2910730737 shutdown_connections 2024-09-10T10:52:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.917+0000 7f5adb200640 1 -- 172.21.15.28:0/2910730737 wait complete. 2024-09-10T10:52:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.919+0000 7f5adb200640 1 Processor -- start 2024-09-10T10:52:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.919+0000 7f5adb200640 1 -- start start 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5adc0af1b0 0x7f5adc09d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 0x7f5adc06f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0b4e60 con 0x7f5adc0b40e0 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc0af640 con 0x7f5adc0af1b0 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.920+0000 7f5adb200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc0b4b10 con 0x7f5adc15bef0 2024-09-10T10:52:44.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44948/0 (socket says 172.21.15.28:44948) 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 -- 172.21.15.28:0/1761451116 learned_addr learned my addr 172.21.15.28:0/1761451116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad9e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5adc0af1b0 0x7f5adc09d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ada800640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 0x7f5adc06f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 -- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 msgr2=0x7f5adc06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 0x7f5adc06f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 -- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5adc0af1b0 msgr2=0x7f5adc09d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5adc0af1b0 0x7f5adc09d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 -- 172.21.15.28:0/1761451116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc06fc20 con 0x7f5adc0b40e0 2024-09-10T10:52:44.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ada800640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 0x7f5adc06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.921+0000 7f5ad3e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009890 tx=0x7f5ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.922+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4040020 con 0x7f5adc0b40e0 2024-09-10T10:52:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.922+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac403c630 con 0x7f5adc0b40e0 2024-09-10T10:52:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.922+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037040 con 0x7f5adc0b40e0 2024-09-10T10:52:44.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.922+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5adc09e040 con 0x7f5adc0b40e0 2024-09-10T10:52:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.922+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5adc09cde0 con 0x7f5adc0b40e0 2024-09-10T10:52:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.923+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5adc15c2d0 con 0x7f5adc0b40e0 2024-09-10T10:52:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.927+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ac402ad10 con 0x7f5adc0b40e0 2024-09-10T10:52:44.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.928+0000 7f5ad2000640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:44.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.928+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f5ac40ac250 con 0x7f5adc0b40e0 2024-09-10T10:52:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.929+0000 7f5ad9e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.929+0000 7f5ad9e00640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 0x7f5ab806e320 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000f60 tx=0x7f5acc007b90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:44.931+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ac407a300 con 0x7f5adc0b40e0 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.152+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5adc09bd30 con 0x7f5adc0b40e0 2024-09-10T10:52:45.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.153+0000 7f5ad2000640 1 -- 172.21.15.28:0/1761451116 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) ==== 95+0+5230 (secure 0 0 0) 0x7f5ac4079c00 con 0x7f5adc0b40e0 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:45.157 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":194,"btime":"2024-09-10T10:52:44:927098+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:43.935635+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":193,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 194 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 msgr2=0x7f5ab806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 0x7f5ab806e320 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5acc000f60 tx=0x7f5acc007b90 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 msgr2=0x7f5adc14cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009890 tx=0x7f5ac40046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 shutdown_connections 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ab806be60 0x7f5ab806e320 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5adc15bef0 0x7f5adc06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5adc0b40e0 0x7f5adc14cce0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 --2- 172.21.15.28:0/1761451116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5adc0af1b0 0x7f5adc09d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.155+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 >> 172.21.15.28:0/1761451116 conn(0x7f5adc06c240 msgr2=0x7f5adc0afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.156+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 shutdown_connections 2024-09-10T10:52:45.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:45.156+0000 7f5adb200640 1 -- 172.21.15.28:0/1761451116 wait complete. 2024-09-10T10:52:45.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 194, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:52:43.935635+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 193, 'state': 'up:reconnect', 'state_seq': 4, 'addr': '172.21.15.28:6834/2552808349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 2552808349}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 2552808349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:52:45.173 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8589:up:reconnect 2024-09-10T10:52:45.174 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:52:50.174 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:50.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.340+0000 7f9ba1000640 1 Processor -- start 2024-09-10T10:52:50.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.340+0000 7f9ba1000640 1 -- start start 2024-09-10T10:52:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.341+0000 7f9ba1000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 0x7f9b9c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.341+0000 7f9ba1000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.341+0000 7f9ba1000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9c146140 con 0x7f9b9c072de0 2024-09-10T10:52:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.341+0000 7f9ba1000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c0cd740 con 0x7f9b9c072320 2024-09-10T10:52:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.341+0000 7f9ba1000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c0cda20 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b9a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b99e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 0x7f9b9c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b9a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:60962/0 (socket says 172.21.15.28:60962) 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b99400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9b9c072de0 0x7f9b9c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:47398/0 (socket says 172.21.15.28:47398) 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b9a800640 1 -- 172.21.15.28:0/2710593490 learned_addr learned my addr 172.21.15.28:0/2710593490 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b9a800640 1 -- 172.21.15.28:0/2710593490 >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 msgr2=0x7f9b9c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.342+0000 7f9b9a800640 1 --2- 172.21.15.28:0/2710593490 >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 0x7f9b9c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.343+0000 7f9b9a800640 1 -- 172.21.15.28:0/2710593490 >> v1:172.21.15.138:6789/0 conn(0x7f9b9c072de0 legacy=0x7f9b9c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.343+0000 7f9b9a800640 1 -- 172.21.15.28:0/2710593490 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0cdd00 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.343+0000 7f9b99e00640 1 --2- 172.21.15.28:0/2710593490 >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 0x7f9b9c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:50.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9b9a800640 1 --2- 172.21.15.28:0/2710593490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009e50 tx=0x7f9b9402a6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906f21049e07387f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9b98a00640 1 -- 172.21.15.28:0/2710593490 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94036070 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9b98a00640 1 -- 172.21.15.28:0/2710593490 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94031600 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9b98a00640 1 -- 172.21.15.28:0/2710593490 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940319d0 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9ba1000640 1 -- 172.21.15.28:0/2710593490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 msgr2=0x7f9b9c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.344+0000 7f9ba1000640 1 --2- 172.21.15.28:0/2710593490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9b94009e50 tx=0x7f9b9402a6a0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 -- 172.21.15.28:0/2710593490 shutdown_connections 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 --2- 172.21.15.28:0/2710593490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c15e1f0 0x7f9b9c06e7c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 --2- 172.21.15.28:0/2710593490 >> v2:172.21.15.138:3300/0 conn(0x7f9b9c072320 0x7f9b9c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 -- 172.21.15.28:0/2710593490 >> 172.21.15.28:0/2710593490 conn(0x7f9b9c067d70 msgr2=0x7f9b9c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 -- 172.21.15.28:0/2710593490 shutdown_connections 2024-09-10T10:52:50.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.345+0000 7f9ba1000640 1 -- 172.21.15.28:0/2710593490 wait complete. 2024-09-10T10:52:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.346+0000 7f9ba1000640 1 Processor -- start 2024-09-10T10:52:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.346+0000 7f9ba1000640 1 -- start start 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c072de0 0x7f9b9c0ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9c15e1f0 0x7f9b9c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0cd520 con 0x7f9b9c072320 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0cd3a0 con 0x7f9b9c15e1f0 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.347+0000 7f9ba1000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c06ed00 con 0x7f9b9c072de0 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44964/0 (socket says 172.21.15.28:44964) 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 -- 172.21.15.28:0/181233897 learned_addr learned my addr 172.21.15.28:0/181233897 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99400640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c072de0 0x7f9b9c0ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b9a800640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9c15e1f0 0x7f9b9c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 -- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c072de0 msgr2=0x7f9b9c0ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c072de0 0x7f9b9c0ca5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 -- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9c15e1f0 msgr2=0x7f9b9c0cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9c15e1f0 0x7f9b9c0cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 -- 172.21.15.28:0/181233897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c0c83e0 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.348+0000 7f9b99e00640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00ebe0 tx=0x7f9b8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.349+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c017070 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.349+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00cc30 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.349+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c013470 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.349+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c0c86c0 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.349+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c0c8c80 con 0x7f9b9c072320 2024-09-10T10:52:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.350+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c072700 con 0x7f9b9c072320 2024-09-10T10:52:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.351+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b8c01a070 con 0x7f9b9c072320 2024-09-10T10:52:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.352+0000 7f9b92000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 0x7f9b7c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.352+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f9b8c08ba40 con 0x7f9b9c072320 2024-09-10T10:52:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.352+0000 7f9b99400640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 0x7f9b7c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.353+0000 7f9b99400640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 0x7f9b7c06e320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0c7930 tx=0x7f9b88007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.356+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9b8c059410 con 0x7f9b9c072320 2024-09-10T10:52:50.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.579+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9b9c0ccc20 con 0x7f9b9c072320 2024-09-10T10:52:50.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.579+0000 7f9b92000640 1 -- 172.21.15.28:0/181233897 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5234 (secure 0 0 0) 0x7f9b8c05d710 con 0x7f9b9c072320 2024-09-10T10:52:50.580 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:50.581 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":196,"btime":"2024-09-10T10:52:46:943950+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:46.943946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":6,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8589,"qdb_cluster":[8589]},"id":6}]} 2024-09-10T10:52:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 196 2024-09-10T10:52:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 msgr2=0x7f9b7c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 0x7f9b7c06e320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0c7930 tx=0x7f9b88007e80 comp rx=0 tx=0).stop 2024-09-10T10:52:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 msgr2=0x7f9b9c1359a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:50.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00ebe0 tx=0x7f9b8c00c520 comp rx=0 tx=0).stop 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 shutdown_connections 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b7c06be60 0x7f9b7c06e320 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b9c15e1f0 0x7f9b9c0cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b9c072de0 0x7f9b9c0ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 --2- 172.21.15.28:0/181233897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b9c072320 0x7f9b9c1359a0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 >> 172.21.15.28:0/181233897 conn(0x7f9b9c067d70 msgr2=0x7f9b9c144110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 shutdown_connections 2024-09-10T10:52:50.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:50.581+0000 7f9ba1000640 1 -- 172.21.15.28:0/181233897 wait complete. 2024-09-10T10:52:50.865 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:50.864+0000 7ffbbe400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:51.002 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.159+0000 7fa70b200640 1 Processor -- start 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.159+0000 7fa70b200640 1 -- start start 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7040d0180 0x7fa704073a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7040d0560 0x7fa704071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa704072210 0x7fa7040acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa7040b3d80 con 0x7fa704072210 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa704073f80 con 0x7fa7040d0180 2024-09-10T10:52:51.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7040729a0 con 0x7fa7040d0560 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa704072210 0x7fa7040acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa704072210 0x7fa7040acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44978/0 (socket says 172.21.15.28:44978) 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.160+0000 7fa70a800640 1 -- 172.21.15.28:0/1606121213 learned_addr learned my addr 172.21.15.28:0/1606121213 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.161+0000 7fa709400640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7040d0560 0x7fa704071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.161+0000 7fa709e00640 1 --2- 172.21.15.28:0/1606121213 >> v2:172.21.15.138:3300/0 conn(0x7fa7040d0180 0x7fa704073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.180+0000 7fa70a800640 1 -- 172.21.15.28:0/1606121213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7040d0560 msgr2=0x7fa704071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.180+0000 7fa70a800640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7040d0560 0x7fa704071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.180+0000 7fa70a800640 1 -- 172.21.15.28:0/1606121213 >> v2:172.21.15.138:3300/0 conn(0x7fa7040d0180 msgr2=0x7fa704073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.180+0000 7fa70a800640 1 --2- 172.21.15.28:0/1606121213 >> v2:172.21.15.138:3300/0 conn(0x7fa7040d0180 0x7fa704073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.180+0000 7fa70a800640 1 -- 172.21.15.28:0/1606121213 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704072c80 con 0x7fa704072210 2024-09-10T10:52:51.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.181+0000 7fa70a800640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa704072210 0x7fa7040acf20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa7040d0d40 tx=0x7fa70000a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7999d440e04cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.181+0000 7fa708a00640 1 -- 172.21.15.28:0/1606121213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700003990 con 0x7fa704072210 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.181+0000 7fa708a00640 1 -- 172.21.15.28:0/1606121213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa700003b30 con 0x7fa704072210 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.181+0000 7fa708a00640 1 -- 172.21.15.28:0/1606121213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa700004030 con 0x7fa704072210 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 -- 172.21.15.28:0/1606121213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa704072210 msgr2=0x7fa7040acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa704072210 0x7fa7040acf20 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa7040d0d40 tx=0x7fa70000a850 comp rx=0 tx=0).stop 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 -- 172.21.15.28:0/1606121213 shutdown_connections 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa704072210 0x7fa7040acf20 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 --2- 172.21.15.28:0/1606121213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7040d0560 0x7fa704071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 --2- 172.21.15.28:0/1606121213 >> v2:172.21.15.138:3300/0 conn(0x7fa7040d0180 0x7fa704073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 -- 172.21.15.28:0/1606121213 >> 172.21.15.28:0/1606121213 conn(0x7fa70406bdf0 msgr2=0x7fa70406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 -- 172.21.15.28:0/1606121213 shutdown_connections 2024-09-10T10:52:51.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.182+0000 7fa70b200640 1 -- 172.21.15.28:0/1606121213 wait complete. 2024-09-10T10:52:51.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.184+0000 7fa70b200640 1 Processor -- start 2024-09-10T10:52:51.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.184+0000 7fa70b200640 1 -- start start 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa704072210 0x7fa704165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 0x7fa704164a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa704073520 con 0x7fa7040d0180 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa704073240 con 0x7fa7040d0560 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa704072660 con 0x7fa704072210 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa709400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa709400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44994/0 (socket says 172.21.15.28:44994) 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa709400640 1 -- 172.21.15.28:0/117398806 learned_addr learned my addr 172.21.15.28:0/117398806 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa70a800640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 0x7fa704164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.185+0000 7fa709e00640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa704072210 0x7fa704165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 -- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa704072210 msgr2=0x7fa704165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa704072210 0x7fa704165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 -- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 msgr2=0x7fa704164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 0x7fa704164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 -- 172.21.15.28:0/117398806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704200960 con 0x7fa7040d0180 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa70a800640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 0x7fa704164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:51.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.186+0000 7fa709400640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4007d20 tx=0x7fa6f4013510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.187+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4013e30 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.187+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6f400e0d0 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.187+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70409b770 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.187+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400e4a0 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.187+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70409c010 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.188+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7040d0980 con 0x7fa7040d0180 2024-09-10T10:52:51.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.189+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6f4020020 con 0x7fa7040d0180 2024-09-10T10:52:51.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.190+0000 7fa6fe000640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 0x7fa6e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.190+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fa6f408fa90 con 0x7fa7040d0180 2024-09-10T10:52:51.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.190+0000 7fa709e00640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 0x7fa6e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.191+0000 7fa709e00640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 0x7fa6e806e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009b40 tx=0x7fa6f8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.194+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa6f4015040 con 0x7fa7040d0180 2024-09-10T10:52:51.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.415+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa70408b670 con 0x7fa7040d0180 2024-09-10T10:52:51.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.416+0000 7fa6fe000640 1 -- 172.21.15.28:0/117398806 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5234 (secure 0 0 0) 0x7fa6f405d580 con 0x7fa7040d0180 2024-09-10T10:52:51.417 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:51.417 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":196,"btime":"2024-09-10T10:52:46:943950+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:46.943946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":6,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8589,"qdb_cluster":[8589]},"id":6}]} 2024-09-10T10:52:51.417 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 196 2024-09-10T10:52:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 msgr2=0x7fa6e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 0x7fa6e806e2f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009b40 tx=0x7fa6f8009190 comp rx=0 tx=0).stop 2024-09-10T10:52:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 msgr2=0x7fa704166260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4007d20 tx=0x7fa6f4013510 comp rx=0 tx=0).stop 2024-09-10T10:52:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 shutdown_connections 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6e806be30 0x7fa6e806e2f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7040d0560 0x7fa704164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7040d0180 0x7fa704166260 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 --2- 172.21.15.28:0/117398806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa704072210 0x7fa704165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 >> 172.21.15.28:0/117398806 conn(0x7fa70406bdf0 msgr2=0x7fa70406c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 shutdown_connections 2024-09-10T10:52:51.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.418+0000 7fa70b200640 1 -- 172.21.15.28:0/117398806 wait complete. 2024-09-10T10:52:51.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 196, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:52:46.943946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 193, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.28:6834/2552808349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 2552808349}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 2552808349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8589, 'qdb_cluster': [8589]} 2024-09-10T10:52:51.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:52:51.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:52:51.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.591+0000 7fe4e5200640 1 Processor -- start 2024-09-10T10:52:51.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.592+0000 7fe4e5200640 1 -- start start 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe4e0072de0 0x7fe4e006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4e0072700 0x7fe4e006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e0146140 con 0x7fe4e0072700 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e00cd740 con 0x7fe4e0072de0 2024-09-10T10:52:51.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4e5200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e00cda20 con 0x7fe4e0072320 2024-09-10T10:52:51.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4de800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4e0072700 0x7fe4e006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4de800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4e0072700 0x7fe4e006e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45000/0 (socket says 172.21.15.28:45000) 2024-09-10T10:52:51.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4de800640 1 -- 172.21.15.28:0/4277598060 learned_addr learned my addr 172.21.15.28:0/4277598060 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.593+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/4277598060 >> v2:172.21.15.138:3300/0 conn(0x7fe4e0072de0 0x7fe4e006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.594+0000 7fe4de800640 1 -- 172.21.15.28:0/4277598060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 msgr2=0x7fe4e00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.594+0000 7fe4de800640 1 --2- 172.21.15.28:0/4277598060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e00cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.594+0000 7fe4de800640 1 -- 172.21.15.28:0/4277598060 >> v2:172.21.15.138:3300/0 conn(0x7fe4e0072de0 msgr2=0x7fe4e006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.594+0000 7fe4de800640 1 --2- 172.21.15.28:0/4277598060 >> v2:172.21.15.138:3300/0 conn(0x7fe4e0072de0 0x7fe4e006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.594+0000 7fe4de800640 1 -- 172.21.15.28:0/4277598060 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e00cdd00 con 0x7fe4e0072700 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4de800640 1 --2- 172.21.15.28:0/4277598060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072700 0x7fe4e006e5f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00701f0 tx=0x7fe4d802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd427062b4b9f4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4dd400640 1 -- 172.21.15.28:0/4277598060 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8036070 con 0x7fe4e0072700 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4dd400640 1 -- 172.21.15.28:0/4277598060 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8031600 con 0x7fe4e0072700 2024-09-10T10:52:51.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4dd400640 1 -- 172.21.15.28:0/4277598060 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80319d0 con 0x7fe4e0072700 2024-09-10T10:52:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4e5200640 1 -- 172.21.15.28:0/4277598060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072700 msgr2=0x7fe4e006e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.595+0000 7fe4e5200640 1 --2- 172.21.15.28:0/4277598060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072700 0x7fe4e006e5f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00701f0 tx=0x7fe4d802a610 comp rx=0 tx=0).stop 2024-09-10T10:52:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 -- 172.21.15.28:0/4277598060 shutdown_connections 2024-09-10T10:52:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 --2- 172.21.15.28:0/4277598060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072700 0x7fe4e006e5f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 --2- 172.21.15.28:0/4277598060 >> v2:172.21.15.138:3300/0 conn(0x7fe4e0072de0 0x7fe4e006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 --2- 172.21.15.28:0/4277598060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 -- 172.21.15.28:0/4277598060 >> 172.21.15.28:0/4277598060 conn(0x7fe4e0067d70 msgr2=0x7fe4e006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 -- 172.21.15.28:0/4277598060 shutdown_connections 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.596+0000 7fe4e5200640 1 -- 172.21.15.28:0/4277598060 wait complete. 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.597+0000 7fe4e5200640 1 Processor -- start 2024-09-10T10:52:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 -- start start 2024-09-10T10:52:51.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e01e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 0x7fe4e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e00cd520 con 0x7fe4e0072de0 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e00cd3a0 con 0x7fe4e0072700 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.598+0000 7fe4e5200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e006eb30 con 0x7fe4e0072320 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45012/0 (socket says 172.21.15.28:45012) 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 -- 172.21.15.28:0/172514045 learned_addr learned my addr 172.21.15.28:0/172514045 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4dde00640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e01e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 0x7fe4e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 -- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 msgr2=0x7fe4e01e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e01e6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 -- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 msgr2=0x7fe4e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 0x7fe4e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4de800640 1 -- 172.21.15.28:0/172514045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01e2720 con 0x7fe4e0072de0 2024-09-10T10:52:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.599+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 0x7fe4e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4de800640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009f40 tx=0x7fe4d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4d6000640 1 -- 172.21.15.28:0/172514045 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8036050 con 0x7fe4e0072de0 2024-09-10T10:52:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4d6000640 1 -- 172.21.15.28:0/172514045 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8042070 con 0x7fe4e0072de0 2024-09-10T10:52:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4d6000640 1 -- 172.21.15.28:0/172514045 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8009510 con 0x7fe4e0072de0 2024-09-10T10:52:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e01e2a00 con 0x7fe4e0072de0 2024-09-10T10:52:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.600+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01e2fc0 con 0x7fe4e0072de0 2024-09-10T10:52:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.602+0000 7fe4d6000640 1 -- 172.21.15.28:0/172514045 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe4d8007bf0 con 0x7fe4e0072de0 2024-09-10T10:52:51.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.603+0000 7fe4d6000640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 0x7fe4bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.603+0000 7fe4d6000640 1 -- 172.21.15.28:0/172514045 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fe4d80ac440 con 0x7fe4e0072de0 2024-09-10T10:52:51.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.603+0000 7fe4dde00640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 0x7fe4bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.603+0000 7fe4e7fff740 1 Processor -- start 2024-09-10T10:52:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 -- start start 2024-09-10T10:52:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4dde00640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 0x7fe4bc06e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009ac0 tx=0x7fe4d0009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276788490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5632767404b0 0x5632767430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5632766a13b0 con 0x563276744480 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632766a6410 con 0x563276742cf0 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.604+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56327664e620 con 0x5632767404b0 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45028/0 (socket says 172.21.15.28:45028) 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 -- 172.21.15.28:0/185054360 learned_addr learned my addr 172.21.15.28:0/185054360 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5632767404b0 0x5632767430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4dde00640 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276788490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 -- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5632767404b0 msgr2=0x5632767430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5632767404b0 0x5632767430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 -- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 msgr2=0x563276788490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276788490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 -- 172.21.15.28:0/185054360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632763b92d0 con 0x563276744480 2024-09-10T10:52:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.605+0000 7fe4de800640 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8034040 tx=0x7fe4d8079930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9ebd392cb7de8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4cea00640 1 -- 172.21.15.28:0/185054360 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80a8070 con 0x563276744480 2024-09-10T10:52:51.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4cea00640 1 -- 172.21.15.28:0/185054360 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d807d070 con 0x563276744480 2024-09-10T10:52:51.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4cea00640 1 -- 172.21.15.28:0/185054360 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d807a4f0 con 0x563276744480 2024-09-10T10:52:51.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 -- 172.21.15.28:0/185054360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 msgr2=0x56327673d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8034040 tx=0x7fe4d8079930 comp rx=0 tx=0).stop 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 -- 172.21.15.28:0/185054360 shutdown_connections 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x56327673d570 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5632767404b0 0x5632767430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/185054360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276788490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 -- 172.21.15.28:0/185054360 >> 172.21.15.28:0/185054360 conn(0x56327673f740 msgr2=0x563276787960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 -- 172.21.15.28:0/185054360 shutdown_connections 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.606+0000 7fe4e7fff740 1 -- 172.21.15.28:0/185054360 wait complete. 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.607+0000 7fe4e7fff740 1 -- start start 2024-09-10T10:52:51.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.607+0000 7fe4e7fff740 1 Processor -- start 2024-09-10T10:52:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276744480 0x5632767613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276762b50 0x5632767410d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563276661780 con 0x563276742cf0 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632766a23d0 con 0x563276762b50 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5632766a6410 con 0x563276744480 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.608+0000 7fe4e7fff740 10 client.0 _ll_register_callbacks cb 0x56327669f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45036/0 (socket says 172.21.15.28:45036) 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 -- 172.21.15.28:0/3865867905 learned_addr learned my addr 172.21.15.28:0/3865867905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276744480 0x5632767613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4de800640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276762b50 0x5632767410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 -- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276744480 msgr2=0x5632767613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276744480 0x5632767613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 -- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276762b50 msgr2=0x5632767410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276762b50 0x5632767410d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 -- 172.21.15.28:0/3865867905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632764836b0 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4dde00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0004610 tx=0x7fe4d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2baaeb09175e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4c3400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.609+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0027020 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fe4d0023450 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d807d070 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) ==== 24+0+0 (secure 0 0 0) 0x7fe4d0023650 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 10 client.8724 fetch_fsmap learned FSMap version 196 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=2+}) -- 0x563276762630 con 0x563276742cf0 2024-09-10T10:52:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d807d070 con 0x563276742cf0 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 196) ==== 3502+0+0 (secure 0 0 0) 0x7fe4d00237f0 con 0x563276742cf0 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 10 client.8724 fetch_fsmap finished waiting for FSMap version 196 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 10 client.8724 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 20 client.8724 populate_metadata read hostname 'smithi028' 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327673c960 0x56327673cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 4 client.8724 mds_command: new command op to 8589 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 --> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632766b3b40 con 0x56327673c960 2024-09-10T10:52:51.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.610+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327673c960 0x56327673cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.611+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327673c960 0x56327673cd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01901e11334660b server_cookie=44fb1076ea00b31b in_seq=0 out_seq=0 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.611+0000 7fe4c3400640 10 client.8724 ms_handle_connect on v2:172.21.15.28:6832/2552808349 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4c3400640 1 -- 172.21.15.28:0/3865867905 <== mds.0 v2:172.21.15.28:6832/2552808349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632766b3b40 con 0x56327673c960 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4c3400640 10 client.8724 handle_command_reply: tid=0 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 client.8724 shutdown 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 msgr2=0x563276763640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0004610 tx=0x7fe4d00046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 shutdown_connections 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327673c960 0x56327673cd40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276762b50 0x5632767410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276744480 0x5632767613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/3865867905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276742cf0 0x563276763640 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4c3400640 0 client.8724 ms_handle_reset on v2:172.21.15.28:6832/2552808349 2024-09-10T10:52:51.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 >> 172.21.15.28:0/3865867905 conn(0x5632767404b0 msgr2=0x5632767859e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 shutdown_connections 2024-09-10T10:52:51.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.612+0000 7fe4e7fff740 1 -- 172.21.15.28:0/3865867905 wait complete. 2024-09-10T10:52:51.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.613+0000 7fe4e7fff740 20 client.8724 trim_cache size 0 max 16384 2024-09-10T10:52:51.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.631+0000 7fe4e7fff740 1 Processor -- start 2024-09-10T10:52:51.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.631+0000 7fe4e7fff740 1 -- start start 2024-09-10T10:52:51.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276760b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276762b50 0x563276740f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e02012f0 con 0x563276744480 2024-09-10T10:52:51.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e02005d0 con 0x563276742cf0 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e0200c20 con 0x563276762b50 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45048/0 (socket says 172.21.15.28:45048) 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.632+0000 7fe4d7e00640 1 -- 172.21.15.28:0/1404770280 learned_addr learned my addr 172.21.15.28:0/1404770280 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4de800640 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276762b50 0x563276740f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4dde00640 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276760b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 -- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276762b50 msgr2=0x563276740f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276762b50 0x563276740f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 -- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 msgr2=0x563276760b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276760b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 -- 172.21.15.28:0/1404770280 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563276786c50 con 0x563276744480 2024-09-10T10:52:51.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00b5140 tx=0x7fe4c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43348af55089ddb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4ce000640 1 -- 172.21.15.28:0/1404770280 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004020 con 0x563276744480 2024-09-10T10:52:51.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4ce000640 1 -- 172.21.15.28:0/1404770280 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80041c0 con 0x563276744480 2024-09-10T10:52:51.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.633+0000 7fe4ce000640 1 -- 172.21.15.28:0/1404770280 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8003610 con 0x563276744480 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 -- 172.21.15.28:0/1404770280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 msgr2=0x563276740a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00b5140 tx=0x7fe4c8008040 comp rx=0 tx=0).stop 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 -- 172.21.15.28:0/1404770280 shutdown_connections 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276762b50 0x563276740f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276744480 0x563276740a50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/1404770280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x563276760b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 -- 172.21.15.28:0/1404770280 >> 172.21.15.28:0/1404770280 conn(0x5632767404b0 msgr2=0x563276788a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 -- 172.21.15.28:0/1404770280 shutdown_connections 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.634+0000 7fe4e7fff740 1 -- 172.21.15.28:0/1404770280 wait complete. 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.635+0000 7fe4e7fff740 1 -- start start 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.635+0000 7fe4e7fff740 1 Processor -- start 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x5632767840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276784610 0x563276809820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563276761a10 con 0x563276762b50 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5632766a13b0 con 0x563276742cf0 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56327673e160 con 0x563276784610 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4e7fff740 10 client.0 _ll_register_callbacks cb 0x7fe4d0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45054/0 (socket says 172.21.15.28:45054) 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 -- 172.21.15.28:0/587984716 learned_addr learned my addr 172.21.15.28:0/587984716 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4dde00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x5632767840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4de800640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276784610 0x563276809820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 -- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276784610 msgr2=0x563276809820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276784610 0x563276809820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 -- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 msgr2=0x5632767840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x5632767840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.636+0000 7fe4d7e00640 1 -- 172.21.15.28:0/587984716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632767974c0 con 0x563276762b50 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4d7e00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009bd0 tx=0x7fe4d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d620ec3a3e69a48b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4c3400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c802d030 con 0x563276762b50 2024-09-10T10:52:51.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fe4c801e020 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c80041c0 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=196) ==== 24+0+0 (secure 0 0 0) 0x7fe4c80094e0 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4e7fff740 10 client.8730 fetch_fsmap learned FSMap version 196 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.637+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=196,monmap=2+}) -- 0x563276787dd0 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c80041c0 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 196) ==== 3502+0+0 (secure 0 0 0) 0x7fe4c8004370 con 0x563276762b50 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 10 client.8730 fetch_fsmap finished waiting for FSMap version 196 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 10 client.8730 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 20 client.8730 populate_metadata read hostname 'smithi028' 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327679e610 0x56327679e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 4 client.8730 mds_command: new command op to 8589 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 --> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] -- command(tid 0: {"prefix": "status"}) -- 0x5632766ade90 con 0x56327679e610 2024-09-10T10:52:51.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.638+0000 7fe4dde00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327679e610 0x56327679e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4dde00640 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327679e610 0x56327679e9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47c056d22a4256c server_cookie=d224b49087058b50 in_seq=0 out_seq=0 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4c3400640 10 client.8730 ms_handle_connect on v2:172.21.15.28:6832/2552808349 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4c3400640 1 -- 172.21.15.28:0/587984716 <== mds.0 v2:172.21.15.28:6832/2552808349 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5632766ade90 con 0x56327679e610 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4c3400640 10 client.8730 handle_command_reply: tid=0 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 client.8730 shutdown 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 msgr2=0x5632768092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009bd0 tx=0x7fe4d0014000 comp rx=0 tx=0).stop 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 shutdown_connections 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:6832/2552808349,v1:172.21.15.28:6834/2552808349] conn(0x56327679e610 0x56327679e9f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563276784610 0x563276809820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563276762b50 0x5632768092e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 --2- 172.21.15.28:0/587984716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563276742cf0 0x5632767840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4c3400640 0 client.8730 ms_handle_reset on v2:172.21.15.28:6832/2552808349 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 >> 172.21.15.28:0/587984716 conn(0x5632767404b0 msgr2=0x563276740a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 shutdown_connections 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.639+0000 7fe4e7fff740 1 -- 172.21.15.28:0/587984716 wait complete. 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e7fff740 20 client.8730 trim_cache size 0 max 16384 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:52:51.645 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 8589, 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 7.7141289070000001, 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 196, 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 118, 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 118, 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 16.012525847999999 2024-09-10T10:52:51.646 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 msgr2=0x7fe4bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 0x7fe4bc06e2f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009ac0 tx=0x7fe4d0009190 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 msgr2=0x7fe4e01e1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009f40 tx=0x7fe4d8004a40 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 shutdown_connections 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe4bc06be30 0x7fe4bc06e2f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4e0072de0 0x7fe4e01e1ed0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e0072700 0x7fe4e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 --2- 172.21.15.28:0/172514045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe4e0072320 0x7fe4e01e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 >> 172.21.15.28:0/172514045 conn(0x7fe4e0067d70 msgr2=0x7fe4e006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 shutdown_connections 2024-09-10T10:52:51.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.640+0000 7fe4e5200640 1 -- 172.21.15.28:0/172514045 wait complete. 2024-09-10T10:52:51.657 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:52:51.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.813+0000 7f6792600640 1 Processor -- start 2024-09-10T10:52:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.813+0000 7f6792600640 1 -- start start 2024-09-10T10:52:51.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.814+0000 7f6792600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.814+0000 7f6792600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f678c06bc10 0x7f678c06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.814+0000 7f6792600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c0d0980 con 0x7f678c0d1b70 2024-09-10T10:52:51.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.814+0000 7f6792600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f678c15e9b0 con 0x7f678c06bc10 2024-09-10T10:52:51.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.814+0000 7f6792600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f678c0d0c60 con 0x7f678c15e5d0 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:32836/0 (socket says 172.21.15.28:32836) 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 -- 172.21.15.28:0/2796784302 learned_addr learned my addr 172.21.15.28:0/2796784302 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f678c06bc10 0x7f678c06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f678be00640 1 --1- 172.21.15.28:0/2796784302 >> v1:172.21.15.138:6789/0 conn(0x7f678c0d1b70 0x7f678c073780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:47406/0 (socket says 172.21.15.28:47406) 2024-09-10T10:52:51.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 -- 172.21.15.28:0/2796784302 >> v2:172.21.15.138:3300/0 conn(0x7f678c06bc10 msgr2=0x7f678c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:51.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 --2- 172.21.15.28:0/2796784302 >> v2:172.21.15.138:3300/0 conn(0x7f678c06bc10 0x7f678c06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 -- 172.21.15.28:0/2796784302 >> v1:172.21.15.138:6789/0 conn(0x7f678c0d1b70 legacy=0x7f678c073780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.815+0000 7f6791200640 1 -- 172.21.15.28:0/2796784302 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c072f80 con 0x7f678c15e5d0 2024-09-10T10:52:51.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.816+0000 7f6791200640 1 --2- 172.21.15.28:0/2796784302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6780009870 tx=0x7f678002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7a6612dd9d995b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.816+0000 7f678b400640 1 -- 172.21.15.28:0/2796784302 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780037070 con 0x7f678c15e5d0 2024-09-10T10:52:51.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f678b400640 1 -- 172.21.15.28:0/2796784302 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6780032440 con 0x7f678c15e5d0 2024-09-10T10:52:51.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f678b400640 1 -- 172.21.15.28:0/2796784302 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780032810 con 0x7f678c15e5d0 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 -- 172.21.15.28:0/2796784302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 msgr2=0x7f678c0d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 --2- 172.21.15.28:0/2796784302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6780009870 tx=0x7f678002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 -- 172.21.15.28:0/2796784302 shutdown_connections 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 --2- 172.21.15.28:0/2796784302 >> v2:172.21.15.138:3300/0 conn(0x7f678c06bc10 0x7f678c06c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 --2- 172.21.15.28:0/2796784302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0d0310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 -- 172.21.15.28:0/2796784302 >> 172.21.15.28:0/2796784302 conn(0x7f678c0d1080 msgr2=0x7f678c0d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 -- 172.21.15.28:0/2796784302 shutdown_connections 2024-09-10T10:52:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.817+0000 7f6792600640 1 -- 172.21.15.28:0/2796784302 wait complete. 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.819+0000 7f6792600640 1 Processor -- start 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.819+0000 7f6792600640 1 -- start start 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c06bc10 0x7f678c0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f678c06cb60 con 0x7f678c0d1b70 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f678c06c9e0 con 0x7f678c06bc10 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6792600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f678c162030 con 0x7f678c15e5d0 2024-09-10T10:52:51.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f678be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f678be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45062/0 (socket says 172.21.15.28:45062) 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f678be00640 1 -- 172.21.15.28:0/2242502507 learned_addr learned my addr 172.21.15.28:0/2242502507 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6791c00640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.820+0000 7f6791200640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c06bc10 0x7f678c0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 -- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 msgr2=0x7f678c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 -- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c06bc10 msgr2=0x7f678c0b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c06bc10 0x7f678c0b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 -- 172.21.15.28:0/2242502507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c160e60 con 0x7f678c0d1b70 2024-09-10T10:52:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f678be00640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f677800c440 tx=0x7f6778001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67780074e0 con 0x7f678c0d1b70 2024-09-10T10:52:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6778007680 con 0x7f678c0d1b70 2024-09-10T10:52:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c161140 con 0x7f678c0d1b70 2024-09-10T10:52:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.821+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778004a20 con 0x7f678c0d1b70 2024-09-10T10:52:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.822+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c155db0 con 0x7f678c0d1b70 2024-09-10T10:52:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.823+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c0d0310 con 0x7f678c0d1b70 2024-09-10T10:52:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.824+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f677801a020 con 0x7f678c0d1b70 2024-09-10T10:52:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.824+0000 7f6788c00640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.825+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f677808ad70 con 0x7f678c0d1b70 2024-09-10T10:52:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.825+0000 7f6791200640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.826+0000 7f6791200640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6780034000 tx=0x7f678002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:51.829+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6778058870 con 0x7f678c0d1b70 2024-09-10T10:52:52.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.048+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f678c073ec0 con 0x7f678c0d1b70 2024-09-10T10:52:52.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.049+0000 7f6788c00640 1 -- 172.21.15.28:0/2242502507 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) ==== 95+0+5234 (secure 0 0 0) 0x7f677805cb70 con 0x7f678c0d1b70 2024-09-10T10:52:52.050 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:52.050 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":196,"btime":"2024-09-10T10:52:46:943950+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:46.943946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":193,"state":"up:active","state_seq":6,"addr":"172.21.15.28:6834/2552808349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2552808349},{"type":"v1","addr":"172.21.15.28:6834","nonce":2552808349}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8589,"qdb_cluster":[8589]},"id":6}]} 2024-09-10T10:52:52.050 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 196 2024-09-10T10:52:52.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 msgr2=0x7f676c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6780034000 tx=0x7f678002e000 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 msgr2=0x7f678c155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f677800c440 tx=0x7f6778001e70 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 shutdown_connections 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c15e5d0 0x7f678c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c0d1b70 0x7f678c155870 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 --2- 172.21.15.28:0/2242502507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c06bc10 0x7f678c0b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 >> 172.21.15.28:0/2242502507 conn(0x7f678c0d1080 msgr2=0x7f678c0d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 shutdown_connections 2024-09-10T10:52:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.051+0000 7f6792600640 1 -- 172.21.15.28:0/2242502507 wait complete. 2024-09-10T10:52:52.068 INFO:tasks.cephfs.test_damage:Mutation 'Corrupt 500.00000000' did not prevent MDS startup, attempting ls... 2024-09-10T10:52:52.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:52:52.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.225+0000 7f07b7c00640 1 Processor -- start 2024-09-10T10:52:52.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.225+0000 7f07b7c00640 1 -- start start 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b7c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 0x7f07b006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b7c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b00d2700 con 0x7f07b00b3650 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b7c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f07b00cedb0 con 0x7f07b015e760 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07b00cf090 con 0x7f07b00d4fd0 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07afe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 0x7f07b006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07afe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:32848/0 (socket says 172.21.15.28:32848) 2024-09-10T10:52:52.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07afe00640 1 -- 172.21.15.28:0/1267222406 learned_addr learned my addr 172.21.15.28:0/1267222406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:52.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.227+0000 7f07b5c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f07b00b3650 0x7f07b00d2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:47418/0 (socket says 172.21.15.28:47418) 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.228+0000 7f07afe00640 1 -- 172.21.15.28:0/1267222406 >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 msgr2=0x7f07b006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.228+0000 7f07afe00640 1 --2- 172.21.15.28:0/1267222406 >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 0x7f07b006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.228+0000 7f07afe00640 1 -- 172.21.15.28:0/1267222406 >> v1:172.21.15.138:6789/0 conn(0x7f07b00b3650 legacy=0x7f07b00d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.228+0000 7f07afe00640 1 -- 172.21.15.28:0/1267222406 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00b3c80 con 0x7f07b00d4fd0 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.228+0000 7f07b5200640 1 --2- 172.21.15.28:0/1267222406 >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 0x7f07b006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07afe00640 1 --2- 172.21.15.28:0/1267222406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f079c009850 tx=0x7f079c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9889a22a85f94af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07af400640 1 -- 172.21.15.28:0/1267222406 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c004030 con 0x7f07b00d4fd0 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07af400640 1 -- 172.21.15.28:0/1267222406 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c0041d0 con 0x7f07b00d4fd0 2024-09-10T10:52:52.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07af400640 1 -- 172.21.15.28:0/1267222406 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c003610 con 0x7f07b00d4fd0 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07b7c00640 1 -- 172.21.15.28:0/1267222406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 msgr2=0x7f07b00cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.229+0000 7f07b7c00640 1 --2- 172.21.15.28:0/1267222406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f079c009850 tx=0x7f079c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 -- 172.21.15.28:0/1267222406 shutdown_connections 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 --2- 172.21.15.28:0/1267222406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b00d4fd0 0x7f07b00cf5b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 --2- 172.21.15.28:0/1267222406 >> v2:172.21.15.138:3300/0 conn(0x7f07b015e760 0x7f07b006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 -- 172.21.15.28:0/1267222406 >> 172.21.15.28:0/1267222406 conn(0x7f07b008bfc0 msgr2=0x7f07b00d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:52.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 -- 172.21.15.28:0/1267222406 shutdown_connections 2024-09-10T10:52:52.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.230+0000 7f07b7c00640 1 -- 172.21.15.28:0/1267222406 wait complete. 2024-09-10T10:52:52.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.231+0000 7f07b7c00640 1 Processor -- start 2024-09-10T10:52:52.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.231+0000 7f07b7c00640 1 -- start start 2024-09-10T10:52:52.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 0x7f07b00cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b015e760 0x7f07b00b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07b00cfe80 con 0x7f07b00b3650 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07b00cfd00 con 0x7f07b00d4fd0 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.232+0000 7f07b7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07b00d35b0 con 0x7f07b015e760 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45064/0 (socket says 172.21.15.28:45064) 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 -- 172.21.15.28:0/3913449623 learned_addr learned my addr 172.21.15.28:0/3913449623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07afe00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 0x7f07b00cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b015e760 0x7f07b00b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 -- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b015e760 msgr2=0x7f07b00b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b015e760 0x7f07b00b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 -- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 msgr2=0x7f07b00cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 0x7f07b00cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 -- 172.21.15.28:0/3913449623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b00b7380 con 0x7f07b00b3650 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07afe00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 0x7f07b00cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.233+0000 7f07b5200640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e990 tx=0x7f07a400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.234+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a4017070 con 0x7f07b00b3650 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.234+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a400c9e0 con 0x7f07b00b3650 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.234+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a400cdb0 con 0x7f07b00b3650 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.234+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b00b81c0 con 0x7f07b00b3650 2024-09-10T10:52:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.234+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b00b7920 con 0x7f07b00b3650 2024-09-10T10:52:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.235+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b00b6d20 con 0x7f07b00b3650 2024-09-10T10:52:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.236+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f07a401a070 con 0x7f07b00b3650 2024-09-10T10:52:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.237+0000 7f07acc00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 0x7f079006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.237+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f07a408b520 con 0x7f07b00b3650 2024-09-10T10:52:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.237+0000 7f07afe00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 0x7f079006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:52.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.238+0000 7f07afe00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 0x7f079006e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f079c033000 tx=0x7f079c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.241+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07a4059010 con 0x7f07b00b3650 2024-09-10T10:52:52.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.461+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07b00d4ab0 con 0x7f07b00b3650 2024-09-10T10:52:52.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.462+0000 7f07acc00640 1 -- 172.21.15.28:0/3913449623 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v196) ==== 53+0+73 (secure 0 0 0) 0x7f07a405d310 con 0x7f07b00b3650 2024-09-10T10:52:52.463 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 msgr2=0x7f079006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 0x7f079006e2f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f079c033000 tx=0x7f079c02e000 comp rx=0 tx=0).stop 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 msgr2=0x7f07b0161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f07a400e990 tx=0x7f07a400ee50 comp rx=0 tx=0).stop 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 shutdown_connections 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f079006be30 0x7f079006e2f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07b015e760 0x7f07b00b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07b00d4fd0 0x7f07b00cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 --2- 172.21.15.28:0/3913449623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f07b00b3650 0x7f07b0161980 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 >> 172.21.15.28:0/3913449623 conn(0x7f07b008bfc0 msgr2=0x7f07b015d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 shutdown_connections 2024-09-10T10:52:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52.464+0000 7f07b7c00640 1 -- 172.21.15.28:0/3913449623 wait complete. 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:52:52.481 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:52:52.482 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:52:52.482 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:52.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:52.482 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:52.518 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42132sec preferred_lft 42132sec 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:52:52.519 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:52:52.520 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:52:52.521 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:52:52.521 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:52:52.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:52.521 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:52:52.612 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:52:52.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:52.615 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:52:52.651 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:52:52.652 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:52:52.652 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:52:52.652 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:52:52.652 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:52:52.652 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:52:52.652 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:52:52.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:52:52.652 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:52:52.653 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:52:52.653 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:52:52.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:52:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:52:52.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:52.851 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:52:52.886 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:52:52.887 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:52:52.887 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:52.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:52.887 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:52:52.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:52:52.983 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:53.018 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:52:53.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:52:53.019 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:53.093 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:52:53.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:53.093 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:53.222 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:53.222 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:53.222 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:53.237 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:53.237 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:53.237 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:53.238 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:52:53.238 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:52:53.260 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:52:53.261 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:52:53.261 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:52:53.262 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:52:53.262 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:52:53.262 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:52:53.264 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:52:53.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:53.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:52:53.321 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:52:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:52:53.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:52:53.410 INFO:teuthology.orchestra.run.smithi028.stdout:client7289 2024-09-10T10:52:53.411 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:53.411 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:53.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:53.447 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:52:53.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:53.491 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir foo) 2024-09-10T10:52:53.524 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:52:53.524 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:52:53.524 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:53.524 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:52:53.524 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:52:53.558 INFO:teuthology.orchestra.run.smithi028.stderr:mkdir: cannot create directory ‘foo’: Read-only file system 2024-09-10T10:52:53.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:53.559 INFO:tasks.cephfs.test_damage:mkdir: cannot create directory ‘foo’: Read-only file system 2024-09-10T10:52:53.559 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds_snaptable' 2024-09-10T10:52:53.559 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:53.559 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:53.634 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:53.634 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:53.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:52:53.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.906+0000 7fd4d4600640 1 Processor -- start 2024-09-10T10:52:53.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.906+0000 7fd4d4600640 1 -- start start 2024-09-10T10:52:53.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.907+0000 7fd4d4600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4cc15b300 0x7fd4cc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.907+0000 7fd4d4600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 0x7fd4cc15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.907+0000 7fd4d4600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4cc0cdbe0 con 0x7fd4cc0cd3f0 2024-09-10T10:52:53.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.907+0000 7fd4d4600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc0728a0 con 0x7fd4cc15b300 2024-09-10T10:52:53.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.907+0000 7fd4d4600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd4cc0cd900 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4cc15b300 0x7fd4cc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd4cc15b300 0x7fd4cc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44036/0 (socket says 172.21.15.28:44036) 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3736435062 learned_addr learned my addr 172.21.15.28:0/3736435062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1200640 1 --1- 172.21.15.28:0/3736435062 >> v1:172.21.15.28:6789/0 conn(0x7fd4cc0cd3f0 0x7fd4cc06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43798/0 (socket says 172.21.15.28:43798) 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d2600640 1 --2- 172.21.15.28:0/3736435062 >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 0x7fd4cc15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3736435062 >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 msgr2=0x7fd4cc15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 --2- 172.21.15.28:0/3736435062 >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 0x7fd4cc15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3736435062 >> v1:172.21.15.28:6789/0 conn(0x7fd4cc0cd3f0 legacy=0x7fd4cc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3736435062 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc072250 con 0x7fd4cc15b300 2024-09-10T10:52:53.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.908+0000 7fd4d2600640 1 --2- 172.21.15.28:0/3736435062 >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 0x7fd4cc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:53.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.909+0000 7fd4d1c00640 1 --2- 172.21.15.28:0/3736435062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0cceb0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0b38f0 tx=0x7fd4c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8618a98061dfd417 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.909+0000 7fd4cbe00640 1 -- 172.21.15.28:0/3736435062 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037070 con 0x7fd4cc15b300 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.909+0000 7fd4cbe00640 1 -- 172.21.15.28:0/3736435062 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0032440 con 0x7fd4cc15b300 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4cbe00640 1 -- 172.21.15.28:0/3736435062 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0032810 con 0x7fd4cc15b300 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 -- 172.21.15.28:0/3736435062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc15b300 msgr2=0x7fd4cc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3736435062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0cceb0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0b38f0 tx=0x7fd4c002a270 comp rx=0 tx=0).stop 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 -- 172.21.15.28:0/3736435062 shutdown_connections 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3736435062 >> v2:172.21.15.138:3300/0 conn(0x7fd4cc06f8d0 0x7fd4cc15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3736435062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0cceb0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 -- 172.21.15.28:0/3736435062 >> 172.21.15.28:0/3736435062 conn(0x7fd4cc067d70 msgr2=0x7fd4cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 -- 172.21.15.28:0/3736435062 shutdown_connections 2024-09-10T10:52:53.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.910+0000 7fd4d4600640 1 -- 172.21.15.28:0/3736435062 wait complete. 2024-09-10T10:52:53.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.912+0000 7fd4d4600640 1 Processor -- start 2024-09-10T10:52:53.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.912+0000 7fd4d4600640 1 -- start start 2024-09-10T10:52:53.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 0x7fd4cc0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0d0160 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4cc0cffe0 con 0x7fd4cc15b300 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd4cc072d30 con 0x7fd4cc0cd3f0 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44038/0 (socket says 172.21.15.28:44038) 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3375136507 learned_addr learned my addr 172.21.15.28:0/3375136507 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d2600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1200640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 0x7fd4cc0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 msgr2=0x7fd4cc0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 0x7fd4cc0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4cc15b300 msgr2=0x7fd4cc0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:53.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1c00640 1 -- 172.21.15.28:0/3375136507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4cc0aea40 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.913+0000 7fd4d1200640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 0x7fd4cc0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:53.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4d1c00640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002a2b0 tx=0x7fd4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0037040 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0042070 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4cc0ad9a0 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0030bb0 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.914+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4cc0adf60 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.915+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4cc0cc9a0 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.916+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd4c0047460 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.917+0000 7fd4c9600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 0x7fd4ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:53.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.917+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7fd4c00ac300 con 0x7fd4cc06f8d0 2024-09-10T10:52:53.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.918+0000 7fd4d1200640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 0x7fd4ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:53.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.918+0000 7fd4d1200640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 0x7fd4ac06e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0af6a0 tx=0x7fd4b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:53.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:53.922+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd4c0045360 con 0x7fd4cc06f8d0 2024-09-10T10:52:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.118+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4cc0b0ed0 con 0x7fd4cc06f8d0 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.119+0000 7fd4c9600640 1 -- 172.21.15.28:0/3375136507 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 36 entries v118) ==== 81+0+2184 (secure 0 0 0) 0x7fd4c007a440 con 0x7fd4cc06f8d0 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:52:54.120 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:52:54.121 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:52:54.122 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:52:54.123 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:52:54.123 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:52:54.123 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:52:54.123 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:listed 36 entries 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 msgr2=0x7fd4ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 0x7fd4ac06e2f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0af6a0 tx=0x7fd4b8009040 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 msgr2=0x7fd4cc14e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fd4c002a2b0 tx=0x7fd4c00046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 shutdown_connections 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd4ac06be30 0x7fd4ac06e2f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4cc15b300 0x7fd4cc0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd4cc0cd3f0 0x7fd4cc0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 --2- 172.21.15.28:0/3375136507 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd4cc06f8d0 0x7fd4cc14e3c0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 >> 172.21.15.28:0/3375136507 conn(0x7fd4cc067d70 msgr2=0x7fd4cc0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 shutdown_connections 2024-09-10T10:52:54.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.121+0000 7fd4d4600640 1 -- 172.21.15.28:0/3375136507 wait complete. 2024-09-10T10:52:54.137 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:52:54.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:54.137 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:52:54.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:54.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:54.206 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:54.242 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:54.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.439+0000 7f7090400640 1 Processor -- start 2024-09-10T10:52:54.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.439+0000 7f7090400640 1 -- start start 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 0x7f7088162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70880af280 0x7f70880afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 0x7f7088154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f70880b6db0 con 0x7f70880af280 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f708815eb60 con 0x7f70880b69d0 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f7090400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f708815ee40 con 0x7f70880affb0 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f708d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70880af280 0x7f70880afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f708d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70880af280 0x7f70880afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44042/0 (socket says 172.21.15.28:44042) 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.440+0000 7f708d000640 1 -- 172.21.15.28:0/3172177836 learned_addr learned my addr 172.21.15.28:0/3172177836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:54.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708d000640 1 -- 172.21.15.28:0/3172177836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 msgr2=0x7f7088154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708e400640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 0x7f7088154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708da00640 1 --2- 172.21.15.28:0/3172177836 >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 0x7f7088162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708d000640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 0x7f7088154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708d000640 1 -- 172.21.15.28:0/3172177836 >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 msgr2=0x7f7088162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708d000640 1 --2- 172.21.15.28:0/3172177836 >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 0x7f7088162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708d000640 1 -- 172.21.15.28:0/3172177836 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708815f120 con 0x7f70880af280 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708e400640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 0x7f7088154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.441+0000 7f708da00640 1 --2- 172.21.15.28:0/3172177836 >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 0x7f7088162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:54.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.442+0000 7f708d000640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f70880afa70 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7088070980 tx=0x7f707402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4528e39beaeaeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.442+0000 7f7087e00640 1 -- 172.21.15.28:0/3172177836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074037070 con 0x7f70880af280 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.442+0000 7f7087e00640 1 -- 172.21.15.28:0/3172177836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707402ae00 con 0x7f70880af280 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.442+0000 7f7087e00640 1 -- 172.21.15.28:0/3172177836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074032660 con 0x7f70880af280 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 -- 172.21.15.28:0/3172177836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 msgr2=0x7f70880afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f70880afa70 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f7088070980 tx=0x7f707402a0b0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 -- 172.21.15.28:0/3172177836 shutdown_connections 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880affb0 0x7f7088154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 --2- 172.21.15.28:0/3172177836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f70880afa70 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 --2- 172.21.15.28:0/3172177836 >> v2:172.21.15.138:3300/0 conn(0x7f70880b69d0 0x7f7088162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 -- 172.21.15.28:0/3172177836 >> 172.21.15.28:0/3172177836 conn(0x7f708806bc10 msgr2=0x7f708806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 -- 172.21.15.28:0/3172177836 shutdown_connections 2024-09-10T10:52:54.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.443+0000 7f7090400640 1 -- 172.21.15.28:0/3172177836 wait complete. 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.444+0000 7f7090400640 1 Processor -- start 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 -- start start 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880affb0 0x7f70881f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 0x7f708815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7088155240 con 0x7f70880af280 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70880af710 con 0x7f70880affb0 2024-09-10T10:52:54.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.445+0000 7f7090400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f70881553c0 con 0x7f70880b69d0 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44050/0 (socket says 172.21.15.28:44050) 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 -- 172.21.15.28:0/3214017319 learned_addr learned my addr 172.21.15.28:0/3214017319 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708d000640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880affb0 0x7f70881f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708e400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 0x7f708815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 -- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 msgr2=0x7f708815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 0x7f708815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 -- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880affb0 msgr2=0x7f70881f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880affb0 0x7f70881f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 -- 172.21.15.28:0/3214017319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7088156c80 con 0x7f70880af280 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708e400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 0x7f708815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:54.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.446+0000 7f708da00640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c7f0 tx=0x7f707c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.447+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c015070 con 0x7f70880af280 2024-09-10T10:52:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.447+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f707c007820 con 0x7f70880af280 2024-09-10T10:52:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.447+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7088156f60 con 0x7f70880af280 2024-09-10T10:52:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.447+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c007bd0 con 0x7f70880af280 2024-09-10T10:52:54.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.447+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7088157520 con 0x7f70880af280 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.448+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708806cf30 con 0x7f70880af280 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.449+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f707c018070 con 0x7f70880af280 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.450+0000 7f7085600640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.450+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 7852+0+0 (secure 0 0 0) 0x7f707c08b740 con 0x7f70880af280 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.450+0000 7f708d000640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.451+0000 7f708d000640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 0x7f706806e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.454+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f707c059230 con 0x7f70880af280 2024-09-10T10:52:54.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.673+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f708806cc50 con 0x7f70880af280 2024-09-10T10:52:54.675 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:54.674+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:54.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.676+0000 7f7085600640 1 -- 172.21.15.28:0/3214017319 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v197) ==== 161+0+0 (secure 0 0 0) 0x7f707c05d530 con 0x7f70880af280 2024-09-10T10:52:54.677 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 msgr2=0x7f706806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 0x7f706806e2f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f7074034000 tx=0x7f707402e000 comp rx=0 tx=0).stop 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 msgr2=0x7f7088152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f707c00c7f0 tx=0x7f707c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 shutdown_connections 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f706806be30 0x7f706806e2f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70880b69d0 0x7f708815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880affb0 0x7f70881f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 --2- 172.21.15.28:0/3214017319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70880af280 0x7f7088152f10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 >> 172.21.15.28:0/3214017319 conn(0x7f708806bc10 msgr2=0x7f708806fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 shutdown_connections 2024-09-10T10:52:54.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.680+0000 7f7090400640 1 -- 172.21.15.28:0/3214017319 wait complete. 2024-09-10T10:52:54.682 INFO:tasks.ceph.mds.c.smithi028.stderr: -1806> 2024-09-10T10:52:43.928+0000 7ffbba800640 -1 mds.0.purge_queue operator(): Error -22 loading Journaler 2024-09-10T10:52:54.682 INFO:tasks.ceph.mds.c.smithi028.stderr: -1799> 2024-09-10T10:52:43.928+0000 7ffbba800640 -1 mds.0.193 unhandled write error (22) Invalid argument, force readonly... 2024-09-10T10:52:54.685 INFO:tasks.ceph.mds.c.smithi028.stderr: -618> 2024-09-10T10:52:50.864+0000 7ffbbe400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:52:54.699 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:54.729 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:52:54.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.858+0000 7f3dd3200640 1 Processor -- start 2024-09-10T10:52:54.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.858+0000 7f3dd3200640 1 -- start start 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3dd40d0dd0 0x7f3dd40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd40cf000 con 0x7f3dd40d1f90 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd3200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd40d16f0 con 0x7f3dd415e660 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd406d3e0 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3dd40d0dd0 0x7f3dd40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd2800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3dd40d1f90 0x7f3dd40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43800/0 (socket says 172.21.15.28:43800) 2024-09-10T10:52:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.859+0000 7f3dd2800640 1 -- 172.21.15.28:0/3167653603 learned_addr learned my addr 172.21.15.28:0/3167653603 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.860+0000 7f3dd1e00640 1 -- 172.21.15.28:0/3167653603 >> v1:172.21.15.138:6789/0 conn(0x7f3dd415e660 legacy=0x7f3dd406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.860+0000 7f3dd1e00640 1 -- 172.21.15.28:0/3167653603 >> v1:172.21.15.28:6789/0 conn(0x7f3dd40d1f90 legacy=0x7f3dd40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.860+0000 7f3dd0a00640 1 -- 172.21.15.28:0/3167653603 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479060413 0 0) 0x7f3dd40cf000 con 0x7f3dd40d1f90 2024-09-10T10:52:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.860+0000 7f3dd1e00640 1 -- 172.21.15.28:0/3167653603 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40d29a0 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.861+0000 7f3dd1e00640 1 --2- 172.21.15.28:0/3167653603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd40d11b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40097a0 tx=0x7f3dc402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86543604ebfe81c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.861+0000 7f3dd0a00640 1 -- 172.21.15.28:0/3167653603 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4037070 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.861+0000 7f3dd0a00640 1 -- 172.21.15.28:0/3167653603 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4032440 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.861+0000 7f3dd0a00640 1 -- 172.21.15.28:0/3167653603 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4032810 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 -- 172.21.15.28:0/3167653603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 msgr2=0x7f3dd40d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 --2- 172.21.15.28:0/3167653603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd40d11b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40097a0 tx=0x7f3dc402a230 comp rx=0 tx=0).stop 2024-09-10T10:52:54.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 -- 172.21.15.28:0/3167653603 shutdown_connections 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 --2- 172.21.15.28:0/3167653603 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd40d11b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 -- 172.21.15.28:0/3167653603 >> 172.21.15.28:0/3167653603 conn(0x7f3dd408bfc0 msgr2=0x7f3dd40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 -- 172.21.15.28:0/3167653603 shutdown_connections 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.862+0000 7f3dd3200640 1 -- 172.21.15.28:0/3167653603 wait complete. 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.863+0000 7f3dd3200640 1 Processor -- start 2024-09-10T10:52:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 -- start start 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dd40d1f90 0x7f3dd40cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd415e660 0x7f3dd40cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40cf330 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40d2f40 con 0x7f3dd415e660 2024-09-10T10:52:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.864+0000 7f3dd3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd40d3220 con 0x7f3dd40d1f90 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44084/0 (socket says 172.21.15.28:44084) 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 -- 172.21.15.28:0/4215495459 learned_addr learned my addr 172.21.15.28:0/4215495459 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd2800640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd415e660 0x7f3dd40cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1400640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dd40d1f90 0x7f3dd40cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 -- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dd40d1f90 msgr2=0x7f3dd40cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dd40d1f90 0x7f3dd40cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 -- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd415e660 msgr2=0x7f3dd40cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd415e660 0x7f3dd40cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.865+0000 7f3dd1e00640 1 -- 172.21.15.28:0/4215495459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40cc270 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dd1e00640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40098c0 tx=0x7f3dc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4037040 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc4042070 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4030a70 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd40cc550 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.866+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd40ccb10 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.867+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd40b39f0 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.868+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dc4030c10 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.869+0000 7f3dca000640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 0x7f3db406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.869+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 7938+0+0 (secure 0 0 0) 0x7f3dc40ac0c0 con 0x7f3dd40d0dd0 2024-09-10T10:52:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.869+0000 7f3dd1400640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 0x7f3db406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.870+0000 7f3dd1400640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 0x7f3db406e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3dd41e6b20 tx=0x7f3dc0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:54.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:54.873+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3dd40b39f0 con 0x7f3dd40d0dd0 2024-09-10T10:52:55.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.131+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f3dd40d2420 con 0x7f3dd40d0dd0 2024-09-10T10:52:55.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.697+0000 7f3dca000640 1 -- 172.21.15.28:0/4215495459 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v198) ==== 107+0+0 (secure 0 0 0) 0x7f3dc4079ac0 con 0x7f3dd40d0dd0 2024-09-10T10:52:55.698 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:52:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 msgr2=0x7f3db406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 0x7f3db406e2f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3dd41e6b20 tx=0x7f3dc0009040 comp rx=0 tx=0).stop 2024-09-10T10:52:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 msgr2=0x7f3dd4155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f3dc40098c0 tx=0x7f3dc40046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 shutdown_connections 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3db406be30 0x7f3db406e2f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dd415e660 0x7f3dd40cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3dd40d1f90 0x7f3dd40cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 --2- 172.21.15.28:0/4215495459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dd40d0dd0 0x7f3dd4155da0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 >> 172.21.15.28:0/4215495459 conn(0x7f3dd408bfc0 msgr2=0x7f3dd40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 shutdown_connections 2024-09-10T10:52:55.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.700+0000 7f3dd3200640 1 -- 172.21.15.28:0/4215495459 wait complete. 2024-09-10T10:52:55.720 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.841+0000 7fde4f353f40 1 Processor -- start 2024-09-10T10:52:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.841+0000 7fde4f353f40 1 -- start start 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d054fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055053e0 0x563d0547b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563d0547a530 con 0x563d055089c0 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d053d7b30 con 0x563d05479ff0 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4f353f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563d05507ee0 con 0x563d055053e0 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44092/0 (socket says 172.21.15.28:44092) 2024-09-10T10:52:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 -- 172.21.15.28:0/4027653225 learned_addr learned my addr 172.21.15.28:0/4027653225 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4c600640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d054fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4bc00640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055053e0 0x563d0547b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 -- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055053e0 msgr2=0x563d0547b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055053e0 0x563d0547b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 -- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 msgr2=0x563d054fbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d054fbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4d000640 1 -- 172.21.15.28:0/4027653225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0547cd30 con 0x563d055089c0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.842+0000 7fde4c600640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d054fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4d000640 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fde44045fc0 tx=0x7fde44051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61521d652700800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4a800640 1 -- 172.21.15.28:0/4027653225 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde44052ae0 con 0x563d055089c0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4a800640 1 -- 172.21.15.28:0/4027653225 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde44052c80 con 0x563d055089c0 2024-09-10T10:52:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4a800640 1 -- 172.21.15.28:0/4027653225 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4405b670 con 0x563d055089c0 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4f353f40 1 -- 172.21.15.28:0/4027653225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 msgr2=0x563d0547bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.843+0000 7fde4f353f40 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fde44045fc0 tx=0x7fde44051000 comp rx=0 tx=0).stop 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 -- 172.21.15.28:0/4027653225 shutdown_connections 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055089c0 0x563d0547bf00 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055053e0 0x563d0547b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 --2- 172.21.15.28:0/4027653225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d054fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 -- 172.21.15.28:0/4027653225 >> 172.21.15.28:0/4027653225 conn(0x563d0547e0a0 msgr2=0x563d0547e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 -- 172.21.15.28:0/4027653225 shutdown_connections 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 -- 172.21.15.28:0/4027653225 wait complete. 2024-09-10T10:52:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.844+0000 7fde4f353f40 1 Processor -- start 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 -- start start 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d0547bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055089c0 0x563d055a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563d055070b0 con 0x563d055053e0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563d054fc520 con 0x563d05479ff0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.845+0000 7fde4f353f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563d054fc7c0 con 0x563d055089c0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44102/0 (socket says 172.21.15.28:44102) 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 learned_addr learned my addr 172.21.15.28:0/3219450994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4d000640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055089c0 0x563d055a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d0547bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055089c0 msgr2=0x563d055a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563d055089c0 0x563d055a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 msgr2=0x563d0547bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563d05479ff0 0x563d0547bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d054fb450 con 0x563d055053e0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.846+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fde3800dfd0 tx=0x7fde3800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.847+0000 7fde49400640 1 -- 172.21.15.28:0/3219450994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3800f650 con 0x563d055053e0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.847+0000 7fde49400640 1 -- 172.21.15.28:0/3219450994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3800f7f0 con 0x563d055053e0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.847+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563d05507640 con 0x563d055053e0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.847+0000 7fde49400640 1 -- 172.21.15.28:0/3219450994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3800fbe0 con 0x563d055053e0 2024-09-10T10:52:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.847+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d054f8800 con 0x563d055053e0 2024-09-10T10:52:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.849+0000 7fde49400640 1 -- 172.21.15.28:0/3219450994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fde38017400 con 0x563d055053e0 2024-09-10T10:52:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde49400640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 0x7fde2c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde49400640 1 -- 172.21.15.28:0/3219450994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 7938+0+0 (secure 0 0 0) 0x7fde3808ada0 con 0x563d055053e0 2024-09-10T10:52:55.851 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 0x563d05509db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d054f8cc0 con 0x563d0547c0f0 2024-09-10T10:52:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 0x7fde2c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.850+0000 7fde4d000640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 0x563d05509db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.851+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 0x7fde2c06ccd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c0057d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.851+0000 7fde4d000640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 0x563d05509db0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.853+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v119'567 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde440529e0 con 0x563d0547c0f0 2024-09-10T10:52:55.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.853+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d0550a2f0 con 0x563d0547c0f0 2024-09-10T10:52:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.855+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v119'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.855+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d0550a6c0 con 0x563d0547c0f0 2024-09-10T10:52:55.858 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.857+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v119'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.857+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d0550aa60 con 0x563d0547c0f0 2024-09-10T10:52:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.859+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v119'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.859+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d0550ae40 con 0x563d0547c0f0 2024-09-10T10:52:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.860+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v119'571 uv571 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.860+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aab50 con 0x563d0547c0f0 2024-09-10T10:52:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.862+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v119'572 uv572 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.862+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.864+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v119'573 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.864+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.867+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v119'574 uv574 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.867+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.869 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.868+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v119'575 uv575 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.869+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055abf00 con 0x563d0547c0f0 2024-09-10T10:52:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.871+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v119'576 uv576 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.871+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055abf00 con 0x563d0547c0f0 2024-09-10T10:52:55.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.872+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v119'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.872+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.874+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v119'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.874+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.876+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v119'579 uv554 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.876+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.877+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v119'580 uv580 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.878+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.880 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.879+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v119'581 uv581 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.879+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055abf00 con 0x563d0547c0f0 2024-09-10T10:52:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.881+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v119'582 uv582 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.881+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055abf00 con 0x563d0547c0f0 2024-09-10T10:52:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.883+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v119'583 uv583 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.883+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aaef0 con 0x563d0547c0f0 2024-09-10T10:52:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.884+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v119'584 uv584 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.884+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.886+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v119'585 uv585 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.886+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.888+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v119'586 uv560 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.888+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.889+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v119'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.889+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.892 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:55.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.891+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v119'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.891+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ac670 con 0x563d0547c0f0 2024-09-10T10:52:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.893+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v119'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.893+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ac670 con 0x563d0547c0f0 2024-09-10T10:52:55.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.894+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v119'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.894+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.896+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v119'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.896+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.898+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v119'592 uv566 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.898+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.899+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v119'593 uv593 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.899+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.902 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.901+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v119'594 uv594 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.901+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ac670 con 0x563d0547c0f0 2024-09-10T10:52:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.902+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v119'595 uv595 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.902+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ac670 con 0x563d0547c0f0 2024-09-10T10:52:55.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.904+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v119'596 uv596 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.904+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ab660 con 0x563d0547c0f0 2024-09-10T10:52:55.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.905+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v119'597 uv597 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d0547c0f0 2024-09-10T10:52:55.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.906+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x563d055abb70 0x563d055adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.906+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055ae4a0 con 0x563d055abb70 2024-09-10T10:52:55.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.906+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x563d055abb70 0x563d055adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.907+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x563d055abb70 0x563d055adf60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.908+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v119'421 uv400 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.909+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055ae840 con 0x563d055abb70 2024-09-10T10:52:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.910+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v119'422 uv422 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.910+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aebe0 con 0x563d055abb70 2024-09-10T10:52:55.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.911+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v119'423 uv423 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.912 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:55.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.911+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055aef80 con 0x563d055abb70 2024-09-10T10:52:55.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.913+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v119'424 uv424 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.913+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055af370 con 0x563d055abb70 2024-09-10T10:52:55.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.915+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v119'425 uv425 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.915+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055af710 con 0x563d055abb70 2024-09-10T10:52:55.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.916+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v119'426 uv419 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.916+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055afab0 con 0x563d055abb70 2024-09-10T10:52:55.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.917+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v119'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.918+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afea0 con 0x563d055abb70 2024-09-10T10:52:55.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.919+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v119'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.920 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:55.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.919+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afea0 con 0x563d055abb70 2024-09-10T10:52:55.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.920+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v119'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.921+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.922+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v119'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.922+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.923+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v119'431 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.923+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.925+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v119'432 uv432 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.925+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.927+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v119'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.928 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:55.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.927+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.929+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v119'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.929+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.930+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v119'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.930+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.931+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v119'436 uv416 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.931+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.933+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v119'437 uv437 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.933+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055afe50 con 0x563d055abb70 2024-09-10T10:52:55.934 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:55.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.934+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v119'438 uv438 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.934+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b1390 con 0x563d055abb70 2024-09-10T10:52:55.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.935+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v119'439 uv439 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.935+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b1390 con 0x563d055abb70 2024-09-10T10:52:55.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.936+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v119'440 uv440 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.936+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b0330 con 0x563d055abb70 2024-09-10T10:52:55.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.937+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v119'441 uv441 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055abb70 2024-09-10T10:52:55.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.937+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563d055b0840 0x563d055b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.937+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3170 con 0x563d055b0840 2024-09-10T10:52:55.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.938+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563d055b0840 0x563d055b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.938+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563d055b0840 0x563d055b2c30 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.940+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v119'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.940+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3610 con 0x563d055b0840 2024-09-10T10:52:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.941+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v119'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.941+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3af0 con 0x563d055b0840 2024-09-10T10:52:55.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.942+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v119'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:55.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.942+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b53a0 con 0x563d055b0840 2024-09-10T10:52:55.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.944+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v119'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.944+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b5740 con 0x563d055b0840 2024-09-10T10:52:55.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.945+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v119'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.945+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b4390 con 0x563d055b0840 2024-09-10T10:52:55.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.946+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.946+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563d055b4cb0 0x563d055b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.946+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b8030 con 0x563d055b4cb0 2024-09-10T10:52:55.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.947+0000 7fde4d000640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563d055b4cb0 0x563d055b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.947+0000 7fde4d000640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563d055b4cb0 0x563d055b7af0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.948+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v119'83 uv82 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.948+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b8030 con 0x563d055b4cb0 2024-09-10T10:52:55.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.950+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v119'84 uv84 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.950+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b80b0 con 0x563d055b4cb0 2024-09-10T10:52:55.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.951+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v119'85 uv85 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.952 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:55.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.951+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.952+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v119'86 uv86 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.952+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.953+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v119'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.953+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.954+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v119'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.954+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.955+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v119'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.957 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:55.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.956+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b94b0 con 0x563d055b0840 2024-09-10T10:52:55.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.957+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v119'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.957+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b94b0 con 0x563d055b0840 2024-09-10T10:52:55.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.958+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v119'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.958+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.959+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.959+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.961+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v119'205 uv198 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.961+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.962+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v119'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.962+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.963+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v119'207 uv207 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.964 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:55.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.963+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b94b0 con 0x563d055b4cb0 2024-09-10T10:52:55.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.964+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v119'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.964+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b94b0 con 0x563d055b4cb0 2024-09-10T10:52:55.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.965+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v119'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.966+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.967+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v119'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.967+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.968+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v119'211 uv204 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.968+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.969+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v119'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.969+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.972 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:55.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.971+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v119'213 uv213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.971+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.972+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v119'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.972+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b4cb0 2024-09-10T10:52:55.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.973+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v119'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:55.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.973+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.975+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v119'91 uv90 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.975+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.976+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v119'92 uv92 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.976+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.978 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:55.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.977+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v119'93 uv93 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.977+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.978+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v119'94 uv94 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.978+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b84a0 con 0x563d055b0840 2024-09-10T10:52:55.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.979+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v119'95 uv95 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055b0840 2024-09-10T10:52:55.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.979+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563d055b8b20 0x563d055baf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:55.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.980+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb490 con 0x563d055b8b20 2024-09-10T10:52:55.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.980+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563d055b8b20 0x563d055baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:55.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.980+0000 7fde4c600640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563d055b8b20 0x563d055baf10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:55.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.981+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v119'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.982+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb550 con 0x563d055b8b20 2024-09-10T10:52:55.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.983+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v119'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.983+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb970 con 0x563d055b8b20 2024-09-10T10:52:55.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:55.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.984+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v119'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.984+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bc980 con 0x563d055b8b20 2024-09-10T10:52:55.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.985+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v119'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.985+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bc980 con 0x563d055b8b20 2024-09-10T10:52:55.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.987+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v119'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.987+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb970 con 0x563d055b8b20 2024-09-10T10:52:55.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.988+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.988+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb970 con 0x563d055b8b20 2024-09-10T10:52:55.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.990+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v119'182 uv176 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.990+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb970 con 0x563d055b8b20 2024-09-10T10:52:55.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.991+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v119'183 uv183 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.991+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bb970 con 0x563d055b8b20 2024-09-10T10:52:55.993 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:55.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.992+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v119'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.992+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:55.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.993+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v119'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.993+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:55.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.995+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v119'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.995+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:55.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.996+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v119'187 uv181 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.996+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:55.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.997+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v119'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:55.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.997+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:55.999 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:55.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.998+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v119'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:55.998+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055b3f80 con 0x563d055b8b20 2024-09-10T10:52:56.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.000+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v119'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.000+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055be780 con 0x563d055b8b20 2024-09-10T10:52:56.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.001+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v119'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.001+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.002+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v119'83 uv81 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.002+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.004+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v119'84 uv84 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.004+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.005+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v119'85 uv85 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.006 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:56.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.005+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.006+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v119'86 uv86 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.006+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.007+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v119'111 uv110 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.007+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.009+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v119'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.009+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.011 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:52:56.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.010+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v119'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.010+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bce30 con 0x563d055b4cb0 2024-09-10T10:52:56.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.011+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v119'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.011+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bce30 con 0x563d055b4cb0 2024-09-10T10:52:56.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.012+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v119'115 uv115 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.012+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bbe20 con 0x563d055b4cb0 2024-09-10T10:52:56.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.014+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v119'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde44066440 con 0x563d055b4cb0 2024-09-10T10:52:56.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.014+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x563d055bc330 0x563d055bc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.014+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bcc50 con 0x563d055bc330 2024-09-10T10:52:56.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.014+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x563d055bc330 0x563d055bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.015+0000 7fde4bc00640 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x563d055bc330 0x563d055bc710 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.016+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v119'94 uv93 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055bc330 2024-09-10T10:52:56.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.016+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bcc50 con 0x563d055bc330 2024-09-10T10:52:56.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.017+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v119'95 uv95 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055bc330 2024-09-10T10:52:56.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.017+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bccd0 con 0x563d055bc330 2024-09-10T10:52:56.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.018+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v119'96 uv96 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055bc330 2024-09-10T10:52:56.019 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:52:56.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.018+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bccd0 con 0x563d055bc330 2024-09-10T10:52:56.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.019+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v119'97 uv97 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055bc330 2024-09-10T10:52:56.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.019+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bccd0 con 0x563d055bc330 2024-09-10T10:52:56.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.020+0000 7fde4bc00640 1 -- 172.21.15.28:0/3219450994 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v119'98 uv98 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde38054ea0 con 0x563d055bc330 2024-09-10T10:52:56.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.021+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bccd0 con 0x563d055b8b20 2024-09-10T10:52:56.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.022+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v119'229 uv220 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.022+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bccd0 con 0x563d055b8b20 2024-09-10T10:52:56.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.023+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v119'230 uv230 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.023+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd2c0 con 0x563d055b8b20 2024-09-10T10:52:56.025 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:52:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.024+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v119'231 uv231 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.024+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd2c0 con 0x563d055b8b20 2024-09-10T10:52:56.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.026+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v119'232 uv232 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.026+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd220 con 0x563d055b8b20 2024-09-10T10:52:56.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.027+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v119'233 uv233 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.027+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd220 con 0x563d055b8b20 2024-09-10T10:52:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.028+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v119'234 uv227 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.028+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd220 con 0x563d055b8b20 2024-09-10T10:52:56.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.029+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v119'235 uv235 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.029+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd220 con 0x563d055b8b20 2024-09-10T10:52:56.031 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:52:56.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.030+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v119'236 uv236 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.030+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055be230 con 0x563d055b8b20 2024-09-10T10:52:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.031+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v119'237 uv237 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.031+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055be230 con 0x563d055b8b20 2024-09-10T10:52:56.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.033+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v119'238 uv238 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.033+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055be630 con 0x563d055b8b20 2024-09-10T10:52:56.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.034+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v119'239 uv239 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.034+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x563d055bd220 con 0x563d055b8b20 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4c600640 1 -- 172.21.15.28:0/3219450994 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v119'240 uv240 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fde3c014070 con 0x563d055b8b20 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563d055b0840 msgr2=0x563d055b2c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563d055b0840 0x563d055b2c30 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563d055b8b20 msgr2=0x563d055baf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x563d055b8b20 0x563d055baf10 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x563d055bc330 msgr2=0x563d055bc710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x563d055bc330 0x563d055bc710 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563d055b4cb0 msgr2=0x563d055b7af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563d055b4cb0 0x563d055b7af0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 reap_dead start 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x563d055abb70 msgr2=0x563d055adf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x563d055abb70 0x563d055adf60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 msgr2=0x563d05509db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4d000640 1 -- 172.21.15.28:0/3219450994 reap_dead start 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 0x563d05509db0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 msgr2=0x7fde2c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 0x7fde2c06ccd0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fde3c005fd0 tx=0x7fde3c0057d0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 msgr2=0x563d055a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.035+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fde3800dfd0 tx=0x7fde3800b6c0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 shutdown_connections 2024-09-10T10:52:56.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x563d0547c0f0 0x563d05509db0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fde2c06a810 0x7fde2c06ccd0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 --2- 172.21.15.28:0/3219450994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563d055053e0 0x563d055a1880 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 >> 172.21.15.28:0/3219450994 conn(0x563d0547e0a0 msgr2=0x563d0547e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 shutdown_connections 2024-09-10T10:52:56.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.036+0000 7fde4f353f40 1 -- 172.21.15.28:0/3219450994 wait complete. 2024-09-10T10:52:56.041 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds_snaptable - 2024-09-10T10:52:56.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.158+0000 7f285da11f40 1 Processor -- start 2024-09-10T10:52:56.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.158+0000 7f285da11f40 1 -- start start 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb38683e0 0x55bdb3792070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 0x55bdb385ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37dd530 con 0x55bdb37dcff0 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bdb373ab30 con 0x55bdb386b9c0 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285da11f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bdb386aee0 con 0x55bdb38683e0 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44116/0 (socket says 172.21.15.28:44116) 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.159+0000 7f285ac00640 1 -- 172.21.15.28:0/3190375555 learned_addr learned my addr 172.21.15.28:0/3190375555 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285b600640 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 0x55bdb385ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 -- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb38683e0 msgr2=0x55bdb3792070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb38683e0 0x55bdb3792070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 -- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 msgr2=0x55bdb385ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 0x55bdb385ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 -- 172.21.15.28:0/3190375555 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb37df380 con 0x55bdb37dcff0 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285b600640 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 0x55bdb385ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f285ac00640 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f284c0099d0 tx=0x7f284c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1ac046ba216009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f2858e00640 1 -- 172.21.15.28:0/3190375555 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c008a30 con 0x55bdb37dcff0 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f2858e00640 1 -- 172.21.15.28:0/3190375555 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f284c018070 con 0x55bdb37dcff0 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.160+0000 7f2858e00640 1 -- 172.21.15.28:0/3190375555 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c008dd0 con 0x55bdb37dcff0 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 -- 172.21.15.28:0/3190375555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 msgr2=0x55bdb37c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f284c0099d0 tx=0x7f284c005a00 comp rx=0 tx=0).stop 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 -- 172.21.15.28:0/3190375555 shutdown_connections 2024-09-10T10:52:56.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb386b9c0 0x55bdb385ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb38683e0 0x55bdb3792070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 --2- 172.21.15.28:0/3190375555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb37dcff0 0x55bdb37c5c80 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 -- 172.21.15.28:0/3190375555 >> 172.21.15.28:0/3190375555 conn(0x55bdb37e10a0 msgr2=0x55bdb37e1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 -- 172.21.15.28:0/3190375555 shutdown_connections 2024-09-10T10:52:56.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 -- 172.21.15.28:0/3190375555 wait complete. 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.161+0000 7f285da11f40 1 Processor -- start 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 -- start start 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb37dcff0 0x55bdb386d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 0x55bdb386d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb386b9c0 0x55bdb38695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37dfb70 con 0x55bdb38683e0 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37925b0 con 0x55bdb37dcff0 2024-09-10T10:52:56.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.162+0000 7f285da11f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55bdb37df620 con 0x55bdb386b9c0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 0x55bdb386d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 0x55bdb386d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44130/0 (socket says 172.21.15.28:44130) 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 -- 172.21.15.28:0/1915806644 learned_addr learned my addr 172.21.15.28:0/1915806644 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285b600640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb386b9c0 0x55bdb38695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285ac00640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb37dcff0 0x55bdb386d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 -- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb386b9c0 msgr2=0x55bdb38695e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55bdb386b9c0 0x55bdb38695e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 -- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb37dcff0 msgr2=0x55bdb386d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bdb37dcff0 0x55bdb386d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 -- 172.21.15.28:0/1915806644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb386ebe0 con 0x55bdb38683e0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.163+0000 7f285a200640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 0x55bdb386d8f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f284800ba50 tx=0x7f284800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.164+0000 7f2853400640 1 -- 172.21.15.28:0/1915806644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848012070 con 0x55bdb38683e0 2024-09-10T10:52:56.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.164+0000 7f2853400640 1 -- 172.21.15.28:0/1915806644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2848009d80 con 0x55bdb38683e0 2024-09-10T10:52:56.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.164+0000 7f2853400640 1 -- 172.21.15.28:0/1915806644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284800e6b0 con 0x55bdb38683e0 2024-09-10T10:52:56.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.164+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bdb386c8d0 con 0x55bdb38683e0 2024-09-10T10:52:56.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.164+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb386ce90 con 0x55bdb38683e0 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.166+0000 7f2853400640 1 -- 172.21.15.28:0/1915806644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2848015070 con 0x55bdb38683e0 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.166+0000 7f2853400640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f283c06a840 0x7f283c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.167+0000 7f2853400640 1 -- 172.21.15.28:0/1915806644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 7938+0+0 (secure 0 0 0) 0x7f284808a180 con 0x55bdb38683e0 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.167+0000 7f285da11f40 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bdb386e750 0x55bdb3900cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.167+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e119) -- 0x55bdb386e370 con 0x55bdb386e750 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.167+0000 7f285ac00640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f283c06a840 0x7f283c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.167+0000 7f285b600640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bdb386e750 0x55bdb3900cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.168+0000 7f285ac00640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f283c06a840 0x7f283c06cd00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f284c015040 tx=0x7f284c012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.168+0000 7f285b600640 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bdb386e750 0x55bdb3900cb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.169+0000 7f285b600640 1 -- 172.21.15.28:0/1915806644 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~80] v119'96 uv96 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f285404e040 con 0x55bdb386e750 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.169+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bdb386e750 msgr2=0x55bdb3900cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.169+0000 7f285da11f40 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55bdb386e750 0x55bdb3900cb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f283c06a840 msgr2=0x7f283c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f283c06a840 0x7f283c06cd00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f284c015040 tx=0x7f284c012000 comp rx=0 tx=0).stop 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 msgr2=0x55bdb386d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 --2- 172.21.15.28:0/1915806644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55bdb38683e0 0x55bdb386d8f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f284800ba50 tx=0x7f284800bf10 comp rx=0 tx=0).stop 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285b600640 1 -- 172.21.15.28:0/1915806644 reap_dead start 2024-09-10T10:52:56.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 shutdown_connections 2024-09-10T10:52:56.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 >> 172.21.15.28:0/1915806644 conn(0x55bdb37e10a0 msgr2=0x55bdb37e1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 shutdown_connections 2024-09-10T10:52:56.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.170+0000 7f285da11f40 1 -- 172.21.15.28:0/1915806644 wait complete. 2024-09-10T10:52:56.175 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:52:56.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.373+0000 7f34e7200640 1 Processor -- start 2024-09-10T10:52:56.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.373+0000 7f34e7200640 1 -- start start 2024-09-10T10:52:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.374+0000 7f34e7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34e80af5a0 0x7f34e8071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.374+0000 7f34e7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 0x7f34e80cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.374+0000 7f34e7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34e8073980 con 0x7f34e815d2a0 2024-09-10T10:52:56.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.374+0000 7f34e7200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f34e806ecb0 con 0x7f34e80af5a0 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.374+0000 7f34e7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34e80af980 con 0x7f34e80cc280 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34e80af5a0 0x7f34e8071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34e80af5a0 0x7f34e8071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44146/0 (socket says 172.21.15.28:44146) 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 -- 172.21.15.28:0/2162020690 learned_addr learned my addr 172.21.15.28:0/2162020690 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e6800640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 0x7f34e80cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 -- 172.21.15.28:0/2162020690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 msgr2=0x7f34e80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 0x7f34e80cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 -- 172.21.15.28:0/2162020690 >> v1:172.21.15.138:6789/0 conn(0x7f34e815d2a0 legacy=0x7f34e806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e5e00640 1 -- 172.21.15.28:0/2162020690 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e8073c20 con 0x7f34e80af5a0 2024-09-10T10:52:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.375+0000 7f34e6800640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 0x7f34e80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:56.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.376+0000 7f34e5e00640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80af5a0 0x7f34e8071cf0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009870 tx=0x7f34d802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c0e1fc9d69464b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.376+0000 7f34e5400640 1 -- 172.21.15.28:0/2162020690 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8037070 con 0x7f34e80af5a0 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.376+0000 7f34e5400640 1 -- 172.21.15.28:0/2162020690 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d8032440 con 0x7f34e80af5a0 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.376+0000 7f34e5400640 1 -- 172.21.15.28:0/2162020690 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8032810 con 0x7f34e80af5a0 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 -- 172.21.15.28:0/2162020690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80af5a0 msgr2=0x7f34e8071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80af5a0 0x7f34e8071cf0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009870 tx=0x7f34d802a250 comp rx=0 tx=0).stop 2024-09-10T10:52:56.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 -- 172.21.15.28:0/2162020690 shutdown_connections 2024-09-10T10:52:56.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e80cc280 0x7f34e80cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 --2- 172.21.15.28:0/2162020690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80af5a0 0x7f34e8071cf0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 -- 172.21.15.28:0/2162020690 >> 172.21.15.28:0/2162020690 conn(0x7f34e80b1bd0 msgr2=0x7f34e80b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 -- 172.21.15.28:0/2162020690 shutdown_connections 2024-09-10T10:52:56.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.377+0000 7f34e7200640 1 -- 172.21.15.28:0/2162020690 wait complete. 2024-09-10T10:52:56.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.379+0000 7f34e7200640 1 Processor -- start 2024-09-10T10:52:56.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.379+0000 7f34e7200640 1 -- start start 2024-09-10T10:52:56.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e815d2a0 0x7f34e815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8073f00 con 0x7f34e80cc280 2024-09-10T10:52:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34e806e080 con 0x7f34e80af5a0 2024-09-10T10:52:56.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34e80ccbe0 con 0x7f34e815d2a0 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e815d2a0 0x7f34e815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44150/0 (socket says 172.21.15.28:44150) 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34dfe00640 1 -- 172.21.15.28:0/3754058060 learned_addr learned my addr 172.21.15.28:0/3754058060 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:56.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:43630/0 (socket says 172.21.15.28:43630) 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 -- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e815d2a0 msgr2=0x7f34e815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e815d2a0 0x7f34e815b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 -- 172.21.15.28:0/3754058060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 msgr2=0x7f34e815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34e5e00640 1 -- 172.21.15.28:0/3754058060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e80b3830 con 0x7f34e80af5a0 2024-09-10T10:52:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.380+0000 7f34dfe00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34e5e00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34d8031000 tx=0x7f34d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8041020 con 0x7f34e80af5a0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34d80307a0 con 0x7f34e80af5a0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8037040 con 0x7f34e80af5a0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e80b3b10 con 0x7f34e80af5a0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.381+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e80b40d0 con 0x7f34e80af5a0 2024-09-10T10:52:56.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.382+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e80cc6a0 con 0x7f34e80af5a0 2024-09-10T10:52:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.383+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f34d8044020 con 0x7f34e80af5a0 2024-09-10T10:52:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.384+0000 7f34de000640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 0x7f34c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.384+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 7938+0+0 (secure 0 0 0) 0x7f34d80abe70 con 0x7f34e80af5a0 2024-09-10T10:52:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.384+0000 7f34dfe00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 0x7f34c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.385+0000 7f34dfe00640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 0x7f34c406e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34e80adbe0 tx=0x7f34d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.389+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34d804b030 con 0x7f34e80af5a0 2024-09-10T10:52:56.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.611+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f34e80cbb90 con 0x7f34e80af5a0 2024-09-10T10:52:56.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.692+0000 7f34de000640 1 -- 172.21.15.28:0/3754058060 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v199) ==== 162+0+0 (secure 0 0 0) 0x7f34d80797f0 con 0x7f34e80af5a0 2024-09-10T10:52:56.694 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:52:56.696 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:52:56.695+0000 7f5c10400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:52:56.696 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:56.696+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:52:56.697 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:52:56.696+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:52:56.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 msgr2=0x7f34c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 0x7f34c406e2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f34e80adbe0 tx=0x7f34d0009040 comp rx=0 tx=0).stop 2024-09-10T10:52:56.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 msgr2=0x7f34e8153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34d8031000 tx=0x7f34d80046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 shutdown_connections 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f34c406be30 0x7f34c406e2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34e815d2a0 0x7f34e815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34e80cc280 0x7f34e815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 --2- 172.21.15.28:0/3754058060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34e80af5a0 0x7f34e8153820 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 >> 172.21.15.28:0/3754058060 conn(0x7f34e80b1bd0 msgr2=0x7f34e80b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 shutdown_connections 2024-09-10T10:52:56.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.696+0000 7f34e7200640 1 -- 172.21.15.28:0/3754058060 wait complete. 2024-09-10T10:52:56.714 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:56.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.871+0000 7febc8800640 1 Processor -- start 2024-09-10T10:52:56.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.872+0000 7febc8800640 1 -- start start 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc8800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7febc015b300 0x7febc00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc8800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc00cdbe0 con 0x7febc006f8d0 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc8800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc00cd900 con 0x7febc00cd3f0 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc8800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7febc00728a0 con 0x7febc015b300 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc6800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7febc006f8d0 0x7febc015b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43834/0 (socket says 172.21.15.28:43834) 2024-09-10T10:52:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc6800640 1 -- 172.21.15.28:0/3783623937 learned_addr learned my addr 172.21.15.28:0/3783623937 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:56.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc5e00640 1 --2- 172.21.15.28:0/3783623937 >> v2:172.21.15.138:3300/0 conn(0x7febc015b300 0x7febc00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.873+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548273039 0 0) 0x7febc00cdbe0 con 0x7febc006f8d0 2024-09-10T10:52:56.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feba8003520 con 0x7febc006f8d0 2024-09-10T10:52:56.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3563945085 0 0) 0x7febc00cd900 con 0x7febc00cd3f0 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc00cdbe0 con 0x7febc00cd3f0 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2313182863 0 0) 0x7feba8003520 con 0x7febc006f8d0 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 >> v2:172.21.15.138:3300/0 conn(0x7febc015b300 msgr2=0x7febc00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 --2- 172.21.15.28:0/3783623937 >> v2:172.21.15.138:3300/0 conn(0x7febc015b300 0x7febc00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 >> v1:172.21.15.138:6789/0 conn(0x7febc00cd3f0 legacy=0x7febc006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc0072250 con 0x7febc006f8d0 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7febbc001970 con 0x7febc006f8d0 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 >> v1:172.21.15.28:6789/0 conn(0x7febc006f8d0 legacy=0x7febc015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba8003b90 0x7feba8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7feba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba80051c0 0x7feba800ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7febc00728a0 con 0x7feba8004800 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feba8010510 con 0x7feba80051c0 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7feba80106c0 con 0x7feba8003b90 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc5400640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7feba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5e00640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba8003b90 0x7feba8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.874+0000 7febc6800640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba80051c0 0x7feba800ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5400640 1 -- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba8003b90 msgr2=0x7feba8003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5400640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feba8003b90 0x7feba8003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5400640 1 -- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba80051c0 msgr2=0x7feba800ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5400640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba80051c0 0x7feba800ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc5400640 1 -- 172.21.15.28:0/3783623937 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba8003520 con 0x7feba8004800 2024-09-10T10:52:56.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.875+0000 7febc6800640 1 -- 172.21.15.28:0/3783623937 reap_dead start 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc5400640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7feba8004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7febb00018d0 tx=0x7feba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2ff95b7c7b5a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb00014f0 con 0x7feba8004800 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb00052d0 con 0x7feba8004800 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc4a00640 1 -- 172.21.15.28:0/3783623937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb002ac60 con 0x7feba8004800 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc8800640 1 -- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 msgr2=0x7feba8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.876+0000 7febc8800640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7feba8004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7febb00018d0 tx=0x7feba8005830 comp rx=0 tx=0).stop 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.877+0000 7febc8800640 1 -- 172.21.15.28:0/3783623937 shutdown_connections 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.877+0000 7febc8800640 1 --2- 172.21.15.28:0/3783623937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7feba8004c80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.877+0000 7febc8800640 1 -- 172.21.15.28:0/3783623937 >> 172.21.15.28:0/3783623937 conn(0x7febc0067d70 msgr2=0x7febc006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.877+0000 7febc8800640 1 -- 172.21.15.28:0/3783623937 shutdown_connections 2024-09-10T10:52:56.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.877+0000 7febc8800640 1 -- 172.21.15.28:0/3783623937 wait complete. 2024-09-10T10:52:56.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.878+0000 7febc8800640 1 Processor -- start 2024-09-10T10:52:56.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.878+0000 7febc8800640 1 -- start start 2024-09-10T10:52:56.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febc00af270 0x7febc01f1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc00ab2f0 0x7febc01f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7febc00d0160 con 0x7feba8004800 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febc00cffe0 con 0x7febc00ab2f0 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.879+0000 7febc8800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7febc0072d30 con 0x7febc00af270 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44166/0 (socket says 172.21.15.28:44166) 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 -- 172.21.15.28:0/4282883458 learned_addr learned my addr 172.21.15.28:0/4282883458 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc6800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc00ab2f0 0x7febc01f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5400640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febc00af270 0x7febc01f1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 -- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febc00af270 msgr2=0x7febc01f1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febc00af270 0x7febc01f1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 -- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc00ab2f0 msgr2=0x7febc01f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc00ab2f0 0x7febc01f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 -- 172.21.15.28:0/4282883458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc00aa270 con 0x7feba8004800 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.880+0000 7febc5e00640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7febb4011890 tx=0x7febb4011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.881+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4019070 con 0x7feba8004800 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.881+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb4004240 con 0x7feba8004800 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.881+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb400f820 con 0x7feba8004800 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.881+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc00aa550 con 0x7feba8004800 2024-09-10T10:52:56.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.881+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc00aab10 con 0x7feba8004800 2024-09-10T10:52:56.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.882+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc00cc9a0 con 0x7feba8004800 2024-09-10T10:52:56.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.883+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7febb401c070 con 0x7feba8004800 2024-09-10T10:52:56.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.884+0000 7febba000640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:56.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.884+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7febb408baa0 con 0x7feba8004800 2024-09-10T10:52:56.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.885+0000 7febc5400640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:56.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.885+0000 7febc5400640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 0x7feba406e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7febb0032000 tx=0x7febb0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:56.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:56.889+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7febb40594e0 con 0x7feba8004800 2024-09-10T10:52:57.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.111+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febc00b0cc0 con 0x7feba8004800 2024-09-10T10:52:57.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.111+0000 7febba000640 1 -- 172.21.15.28:0/4282883458 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+4392 (secure 0 0 0) 0x7febb405d7e0 con 0x7feba8004800 2024-09-10T10:52:57.113 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:57.113 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":200,"btime":"2024-09-10T10:52:56:696579+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:56.696523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:57.113 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 200 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 msgr2=0x7feba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 0x7feba406e2f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7febb0032000 tx=0x7febb0009030 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 msgr2=0x7febc00a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7febb4011890 tx=0x7febb4011d50 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 shutdown_connections 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7feba406be30 0x7feba406e2f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febc00ab2f0 0x7febc01f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7febc00af270 0x7febc01f1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 --2- 172.21.15.28:0/4282883458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feba8004800 0x7febc00a9be0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.113+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 >> 172.21.15.28:0/4282883458 conn(0x7febc0067d70 msgr2=0x7febc015def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.114+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 shutdown_connections 2024-09-10T10:52:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.114+0000 7febc8800640 1 -- 172.21.15.28:0/4282883458 wait complete. 2024-09-10T10:52:57.130 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:57.130 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.285+0000 7fed2ee00640 1 Processor -- start 2024-09-10T10:52:57.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.285+0000 7fed2ee00640 1 -- start start 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed2ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed28161c80 0x7fed2806e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed2ee00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2806e870 con 0x7fed28162060 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed2ee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2806d9f0 con 0x7fed280b6680 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed2ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fed2806dcd0 con 0x7fed28161c80 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed27e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed28161c80 0x7fed2806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed27e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed28161c80 0x7fed2806e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44172/0 (socket says 172.21.15.28:44172) 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.287+0000 7fed27e00640 1 -- 172.21.15.28:0/3147501974 learned_addr learned my addr 172.21.15.28:0/3147501974 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:57.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.286+0000 7fed27400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fed28162060 0x7fed280b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43836/0 (socket says 172.21.15.28:43836) 2024-09-10T10:52:57.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.287+0000 7fed27e00640 1 -- 172.21.15.28:0/3147501974 >> v1:172.21.15.138:6789/0 conn(0x7fed280b6680 legacy=0x7fed280cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.287+0000 7fed27e00640 1 -- 172.21.15.28:0/3147501974 >> v1:172.21.15.28:6789/0 conn(0x7fed28162060 legacy=0x7fed280b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.287+0000 7fed27e00640 1 -- 172.21.15.28:0/3147501974 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2806dfb0 con 0x7fed28161c80 2024-09-10T10:52:57.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.287+0000 7fed26a00640 1 -- 172.21.15.28:0/3147501974 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055625531 0 0) 0x7fed14001110 con 0x7fed28162060 2024-09-10T10:52:57.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.288+0000 7fed27e00640 1 --2- 172.21.15.28:0/3147501974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28161c80 0x7fed2806e330 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9978a6b89a334572 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.288+0000 7fed26a00640 1 -- 172.21.15.28:0/3147501974 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18037070 con 0x7fed28161c80 2024-09-10T10:52:57.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.288+0000 7fed26a00640 1 -- 172.21.15.28:0/3147501974 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed18032440 con 0x7fed28161c80 2024-09-10T10:52:57.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.288+0000 7fed26a00640 1 -- 172.21.15.28:0/3147501974 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18032810 con 0x7fed28161c80 2024-09-10T10:52:57.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 -- 172.21.15.28:0/3147501974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28161c80 msgr2=0x7fed2806e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 --2- 172.21.15.28:0/3147501974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28161c80 0x7fed2806e330 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fed18009870 tx=0x7fed1802a2a0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 -- 172.21.15.28:0/3147501974 shutdown_connections 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 --2- 172.21.15.28:0/3147501974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28161c80 0x7fed2806e330 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 -- 172.21.15.28:0/3147501974 >> 172.21.15.28:0/3147501974 conn(0x7fed280d44f0 msgr2=0x7fed280d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 -- 172.21.15.28:0/3147501974 shutdown_connections 2024-09-10T10:52:57.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.289+0000 7fed2ee00640 1 -- 172.21.15.28:0/3147501974 wait complete. 2024-09-10T10:52:57.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.291+0000 7fed2ee00640 1 Processor -- start 2024-09-10T10:52:57.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.291+0000 7fed2ee00640 1 -- start start 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed280b6680 0x7fed28155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 0x7fed280ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed280b6b30 con 0x7fed28162060 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed280d2aa0 con 0x7fed280b6680 2024-09-10T10:52:57.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed280b6cb0 con 0x7fed28161c80 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44180/0 (socket says 172.21.15.28:44180) 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 -- 172.21.15.28:0/94752191 learned_addr learned my addr 172.21.15.28:0/94752191 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed27e00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed280b6680 0x7fed28155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed27400640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 0x7fed280ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 -- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 msgr2=0x7fed280ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 0x7fed280ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 -- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed280b6680 msgr2=0x7fed28155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed280b6680 0x7fed28155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed2ce00640 1 -- 172.21.15.28:0/94752191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed280cd750 con 0x7fed28162060 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.292+0000 7fed27400640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 0x7fed280ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:57.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed2ce00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fed2000b970 tx=0x7fed2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20019070 con 0x7fed28162060 2024-09-10T10:52:57.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2000cb40 con 0x7fed28162060 2024-09-10T10:52:57.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20014440 con 0x7fed28162060 2024-09-10T10:52:57.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed280b7fa0 con 0x7fed28162060 2024-09-10T10:52:57.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.293+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed280b8560 con 0x7fed28162060 2024-09-10T10:52:57.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.294+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed280d0540 con 0x7fed28162060 2024-09-10T10:52:57.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.295+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed2001c070 con 0x7fed28162060 2024-09-10T10:52:57.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.296+0000 7fed1fe00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 0x7fed0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.296+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7fed2008b310 con 0x7fed28162060 2024-09-10T10:52:57.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.296+0000 7fed27e00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 0x7fed0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.297+0000 7fed27e00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 0x7fed0806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fed18034000 tx=0x7fed1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.300+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed280d0540 con 0x7fed28162060 2024-09-10T10:52:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.522+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed280b8730 con 0x7fed28162060 2024-09-10T10:52:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.522+0000 7fed1fe00640 1 -- 172.21.15.28:0/94752191 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+4392 (secure 0 0 0) 0x7fed20058d60 con 0x7fed28162060 2024-09-10T10:52:57.523 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:57.524 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":200,"btime":"2024-09-10T10:52:56:696579+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:56.696523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 200 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 msgr2=0x7fed0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 0x7fed0806e2f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fed18034000 tx=0x7fed1802e000 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 msgr2=0x7fed280b8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fed2000b970 tx=0x7fed2000be30 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 shutdown_connections 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed0806be30 0x7fed0806e2f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed28162060 0x7fed280b8a80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed28161c80 0x7fed280ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 --2- 172.21.15.28:0/94752191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed280b6680 0x7fed28155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.524+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 >> 172.21.15.28:0/94752191 conn(0x7fed280d44f0 msgr2=0x7fed280b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.525+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 shutdown_connections 2024-09-10T10:52:57.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.525+0000 7fed2ee00640 1 -- 172.21.15.28:0/94752191 wait complete. 2024-09-10T10:52:57.541 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:52:57.541 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:52:57.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.697+0000 7f5cc3400640 1 Processor -- start 2024-09-10T10:52:57.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.697+0000 7f5cc3400640 1 -- start start 2024-09-10T10:52:57.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.698+0000 7f5cc3400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5cbc072320 0x7f5cbc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.698+0000 7f5cc3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 0x7f5cbc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc3400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cbc146140 con 0x7f5cbc072700 2024-09-10T10:52:57.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc3400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5cbc0cd740 con 0x7f5cbc072320 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc3400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5cbc0cda20 con 0x7f5cbc072de0 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc1400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5cbc072700 0x7f5cbc06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43848/0 (socket says 172.21.15.28:43848) 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc1400640 1 -- 172.21.15.28:0/2564118077 learned_addr learned my addr 172.21.15.28:0/2564118077 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cbbe00640 1 --2- 172.21.15.28:0/2564118077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 0x7f5cbc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cc0a00640 1 --2- 172.21.15.28:0/2564118077 >> v2:172.21.15.138:3300/0 conn(0x7f5cbc072320 0x7f5cbc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013520583 0 0) 0x7f5cbc146140 con 0x7f5cbc072700 2024-09-10T10:52:57.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.699+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca4003520 con 0x7f5cbc072700 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.700+0000 7f5cbbe00640 1 -- 172.21.15.28:0/2564118077 >> v2:172.21.15.138:3300/0 conn(0x7f5cbc072320 msgr2=0x7f5cbc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.700+0000 7f5cbbe00640 1 --2- 172.21.15.28:0/2564118077 >> v2:172.21.15.138:3300/0 conn(0x7f5cbc072320 0x7f5cbc0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.700+0000 7f5cbbe00640 1 -- 172.21.15.28:0/2564118077 >> v1:172.21.15.28:6789/0 conn(0x7f5cbc072700 legacy=0x7f5cbc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.700+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4104991949 0 0) 0x7f5ca4003520 con 0x7f5cbc072700 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.700+0000 7f5cbbe00640 1 -- 172.21.15.28:0/2564118077 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc0cdd00 con 0x7f5cbc072de0 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.701+0000 7f5cbbe00640 1 --2- 172.21.15.28:0/2564118077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 0x7f5cbc06e0b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80097a0 tx=0x7f5ca802a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f12afe01366bb8cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.701+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8037070 con 0x7f5cbc072de0 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.701+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca802ab10 con 0x7f5cbc072de0 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.701+0000 7f5cbb400640 1 -- 172.21.15.28:0/2564118077 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40019f0 con 0x7f5cbc072de0 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 -- 172.21.15.28:0/2564118077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 msgr2=0x7f5cbc06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 --2- 172.21.15.28:0/2564118077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 0x7f5cbc06e0b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5ca80097a0 tx=0x7f5ca802a080 comp rx=0 tx=0).stop 2024-09-10T10:52:57.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 -- 172.21.15.28:0/2564118077 shutdown_connections 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 --2- 172.21.15.28:0/2564118077 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072de0 0x7f5cbc06e0b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 --2- 172.21.15.28:0/2564118077 >> v2:172.21.15.138:3300/0 conn(0x7f5cbc072320 0x7f5cbc0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 -- 172.21.15.28:0/2564118077 >> 172.21.15.28:0/2564118077 conn(0x7f5cbc067d70 msgr2=0x7f5cbc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 -- 172.21.15.28:0/2564118077 shutdown_connections 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.702+0000 7f5cc3400640 1 -- 172.21.15.28:0/2564118077 wait complete. 2024-09-10T10:52:57.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.703+0000 7f5cc3400640 1 Processor -- start 2024-09-10T10:52:57.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.704+0000 7f5cc3400640 1 -- start start 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.704+0000 7f5cc3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 0x7f5cbc0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.704+0000 7f5cc3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072700 0x7f5cbc1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc3400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0cd520 con 0x7f5cbc072de0 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc3400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cbc0cd3a0 con 0x7f5cbc072320 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc3400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5cbc06ed00 con 0x7f5cbc072700 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44186/0 (socket says 172.21.15.28:44186) 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 -- 172.21.15.28:0/823805190 learned_addr learned my addr 172.21.15.28:0/823805190 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:57.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cbbe00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072700 0x7f5cbc1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc0a00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 0x7f5cbc0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 -- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072700 msgr2=0x7f5cbc1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072700 0x7f5cbc1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 -- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 msgr2=0x7f5cbc0ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 0x7f5cbc0ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 -- 172.21.15.28:0/823805190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cbc0aced0 con 0x7f5cbc072de0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc0a00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 0x7f5cbc0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.705+0000 7f5cc1400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001b60 tx=0x7f5cb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.706+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4017070 con 0x7f5cbc072de0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.706+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb4012400 con 0x7f5cbc072de0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.706+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cbc0b0f70 con 0x7f5cbc072de0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.706+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40127b0 con 0x7f5cbc072de0 2024-09-10T10:52:57.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.706+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cbc0b1530 con 0x7f5cbc072de0 2024-09-10T10:52:57.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.707+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cbc0cce60 con 0x7f5cbc072de0 2024-09-10T10:52:57.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.708+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5cb401a070 con 0x7f5cbc072de0 2024-09-10T10:52:57.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.709+0000 7f5cb8c00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 0x7f5c9c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:57.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.709+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7f5cb408b5e0 con 0x7f5cbc072de0 2024-09-10T10:52:57.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.709+0000 7f5cc0a00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 0x7f5c9c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:57.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.710+0000 7f5cc0a00640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 0x7f5c9c06e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009ac0 tx=0x7f5cb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:57.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.713+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5cbc0aced0 con 0x7f5cbc072de0 2024-09-10T10:52:57.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.936+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cbc0ca040 con 0x7f5cbc072de0 2024-09-10T10:52:57.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.936+0000 7f5cb8c00640 1 -- 172.21.15.28:0/823805190 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) ==== 95+0+4392 (secure 0 0 0) 0x7f5cb4059020 con 0x7f5cbc072de0 2024-09-10T10:52:57.938 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:52:57.938 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":200,"btime":"2024-09-10T10:52:56:696579+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8700,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3588871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3588871837},{"type":"v1","addr":"172.21.15.28:6835","nonce":3588871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":194}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:52:56.696523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:52:57.938 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 200 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.938+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 msgr2=0x7f5c9c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.938+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 0x7f5c9c06e2f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0009ac0 tx=0x7f5cb0008040 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.938+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 msgr2=0x7f5cbc1f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.938+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001b60 tx=0x7f5cb400a430 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 shutdown_connections 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5c9c06be30 0x7f5c9c06e2f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5cbc072de0 0x7f5cbc1f4970 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5cbc072700 0x7f5cbc1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 --2- 172.21.15.28:0/823805190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cbc072320 0x7f5cbc0ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 >> 172.21.15.28:0/823805190 conn(0x7f5cbc067d70 msgr2=0x7f5cbc06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:57.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 shutdown_connections 2024-09-10T10:52:57.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:57.939+0000 7f5cc3400640 1 -- 172.21.15.28:0/823805190 wait complete. 2024-09-10T10:52:57.956 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds_snaptable' led to DAMAGED state 2024-09-10T10:52:57.956 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt mds0_inotable' 2024-09-10T10:52:57.956 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:52:57.956 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:52:57.992 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:52:57.992 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:52:57.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:52:57.993 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:52:58.067 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:52:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:52:58.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:52:58.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.264+0000 7fb3baa00640 1 Processor -- start 2024-09-10T10:52:58.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.264+0000 7fb3baa00640 1 -- start start 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3baa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb3b415e760 0x7fb3b406cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 0x7fb3b40d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3baa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b40d2700 con 0x7fb3b40d4fd0 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3baa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40cedb0 con 0x7fb3b415e760 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3baa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40cf090 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb3b415e760 0x7fb3b406cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb3b415e760 0x7fb3b406cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44192/0 (socket says 172.21.15.28:44192) 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.265+0000 7fb3b3e00640 1 -- 172.21.15.28:0/26167387 learned_addr learned my addr 172.21.15.28:0/26167387 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b3e00640 1 -- 172.21.15.28:0/26167387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 msgr2=0x7fb3b40d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b8a00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 0x7fb3b40d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 0x7fb3b40d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b3e00640 1 -- 172.21.15.28:0/26167387 >> v1:172.21.15.138:6789/0 conn(0x7fb3b40d4fd0 legacy=0x7fb3b40cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b3e00640 1 -- 172.21.15.28:0/26167387 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40b3c80 con 0x7fb3b415e760 2024-09-10T10:52:58.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.266+0000 7fb3b8a00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 0x7fb3b40d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:52:58.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.267+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b415e760 0x7fb3b406cdd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb3b408ca50 tx=0x7fb3a402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa384652005fbe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.267+0000 7fb3b2a00640 1 -- 172.21.15.28:0/26167387 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4037070 con 0x7fb3b415e760 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.267+0000 7fb3b2a00640 1 -- 172.21.15.28:0/26167387 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a4032440 con 0x7fb3b415e760 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.267+0000 7fb3b2a00640 1 -- 172.21.15.28:0/26167387 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4032810 con 0x7fb3b415e760 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 -- 172.21.15.28:0/26167387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b415e760 msgr2=0x7fb3b406cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b415e760 0x7fb3b406cdd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb3b408ca50 tx=0x7fb3a402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 -- 172.21.15.28:0/26167387 shutdown_connections 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40cfcc0 0x7fb3b40d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 --2- 172.21.15.28:0/26167387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b415e760 0x7fb3b406cdd0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 -- 172.21.15.28:0/26167387 >> 172.21.15.28:0/26167387 conn(0x7fb3b408bfc0 msgr2=0x7fb3b40d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:58.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 -- 172.21.15.28:0/26167387 shutdown_connections 2024-09-10T10:52:58.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.268+0000 7fb3baa00640 1 -- 172.21.15.28:0/26167387 wait complete. 2024-09-10T10:52:58.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.269+0000 7fb3baa00640 1 Processor -- start 2024-09-10T10:52:58.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 -- start start 2024-09-10T10:52:58.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 0x7fb3b4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b415e760 0x7fb3b40b9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d33e0 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40b38e0 con 0x7fb3b415e760 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.270+0000 7fb3baa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40d3560 con 0x7fb3b40d4fd0 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44200/0 (socket says 172.21.15.28:44200) 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 -- 172.21.15.28:0/1476137263 learned_addr learned my addr 172.21.15.28:0/1476137263 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3400640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 0x7fb3b4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b8a00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b415e760 0x7fb3b40b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 -- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 msgr2=0x7fb3b4155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 0x7fb3b4155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 -- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b415e760 msgr2=0x7fb3b40b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b415e760 0x7fb3b40b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 -- 172.21.15.28:0/1476137263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40ccaf0 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3400640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 0x7fb3b4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.271+0000 7fb3b3e00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.272+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4041070 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.272+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a403c870 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.272+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b40b5140 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.272+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4037040 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.272+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b40ccd90 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.273+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b40d38e0 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.274+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb3a4044070 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.275+0000 7fb3abe00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.275+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7fb3a40ac5a0 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.275+0000 7fb3b3400640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.276+0000 7fb3b3400640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 0x7fb39406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cdf20 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.279+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3b40d2700 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.500+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb3b406e0c0 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.720+0000 7fb3abe00640 1 -- 172.21.15.28:0/1476137263 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v201) ==== 161+0+0 (secure 0 0 0) 0x7fb3a407a750 con 0x7fb3b40cfcc0 2024-09-10T10:52:58.721 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.722+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 msgr2=0x7fb39406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 0x7fb39406e2f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40cdf20 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 msgr2=0x7fb3b40ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a40046c0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 shutdown_connections 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb39406be30 0x7fb39406e2f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3b415e760 0x7fb3b40b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb3b40d4fd0 0x7fb3b4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 --2- 172.21.15.28:0/1476137263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3b40cfcc0 0x7fb3b40ce090 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 >> 172.21.15.28:0/1476137263 conn(0x7fb3b408bfc0 msgr2=0x7fb3b40d3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 shutdown_connections 2024-09-10T10:52:58.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.723+0000 7fb3baa00640 1 -- 172.21.15.28:0/1476137263 wait complete. 2024-09-10T10:52:58.741 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:52:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.898+0000 7f8ebfc00640 1 Processor -- start 2024-09-10T10:52:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.898+0000 7f8ebfc00640 1 -- start start 2024-09-10T10:52:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebfc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8eb815e5d0 0x7f8eb80d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebfc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8eb809b140 0x7f8eb8168560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebfc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb806cec0 con 0x7f8eb806df60 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebfc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb806dad0 con 0x7f8eb809b140 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebfc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb806d1a0 con 0x7f8eb815e5d0 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.899+0000 7f8ebdc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8eb809b140 0x7f8eb8168560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8eb809b140 0x7f8eb8168560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44216/0 (socket says 172.21.15.28:44216) 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 -- 172.21.15.28:0/505935698 learned_addr learned my addr 172.21.15.28:0/505935698 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8eb7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8eb806df60 0x7f8eb809aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43864/0 (socket says 172.21.15.28:43864) 2024-09-10T10:52:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebd200640 1 --2- 172.21.15.28:0/505935698 >> v2:172.21.15.138:3300/0 conn(0x7f8eb815e5d0 0x7f8eb80d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 -- 172.21.15.28:0/505935698 >> v2:172.21.15.138:3300/0 conn(0x7f8eb815e5d0 msgr2=0x7f8eb80d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 --2- 172.21.15.28:0/505935698 >> v2:172.21.15.138:3300/0 conn(0x7f8eb815e5d0 0x7f8eb80d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 -- 172.21.15.28:0/505935698 >> v1:172.21.15.28:6789/0 conn(0x7f8eb806df60 legacy=0x7f8eb809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.900+0000 7f8ebdc00640 1 -- 172.21.15.28:0/505935698 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb806d480 con 0x7f8eb809b140 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.901+0000 7f8ebdc00640 1 --2- 172.21.15.28:0/505935698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb8168560 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009e50 tx=0x7f8eb002a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bbe97884cc8d84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.901+0000 7f8eb7400640 1 -- 172.21.15.28:0/505935698 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0036070 con 0x7f8eb809b140 2024-09-10T10:52:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.901+0000 7f8eb7400640 1 -- 172.21.15.28:0/505935698 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb0031600 con 0x7f8eb809b140 2024-09-10T10:52:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.901+0000 7f8eb7400640 1 -- 172.21.15.28:0/505935698 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00319d0 con 0x7f8eb809b140 2024-09-10T10:52:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 -- 172.21.15.28:0/505935698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 msgr2=0x7f8eb8168560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/505935698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb8168560 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009e50 tx=0x7f8eb002a7f0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 -- 172.21.15.28:0/505935698 shutdown_connections 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/505935698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb8168560 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/505935698 >> v2:172.21.15.138:3300/0 conn(0x7f8eb815e5d0 0x7f8eb80d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 -- 172.21.15.28:0/505935698 >> 172.21.15.28:0/505935698 conn(0x7f8eb80d23a0 msgr2=0x7f8eb80d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 -- 172.21.15.28:0/505935698 shutdown_connections 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.902+0000 7f8ebfc00640 1 -- 172.21.15.28:0/505935698 wait complete. 2024-09-10T10:52:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.904+0000 7f8ebfc00640 1 Processor -- start 2024-09-10T10:52:58.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.904+0000 7f8ebfc00640 1 -- start start 2024-09-10T10:52:58.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb806df60 0x7f8eb8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8eb815e5d0 0x7f8eb80b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb815eb30 con 0x7f8eb809b140 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb815e9b0 con 0x7f8eb806df60 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebfc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb80d1630 con 0x7f8eb815e5d0 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44220/0 (socket says 172.21.15.28:44220) 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 -- 172.21.15.28:0/637397046 learned_addr learned my addr 172.21.15.28:0/637397046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebdc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8eb815e5d0 0x7f8eb80b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8ebd200640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb806df60 0x7f8eb8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 -- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8eb815e5d0 msgr2=0x7f8eb80b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8eb815e5d0 0x7f8eb80b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 -- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb806df60 msgr2=0x7f8eb8156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb806df60 0x7f8eb8156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.905+0000 7f8eb7e00640 1 -- 172.21.15.28:0/637397046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb80c8540 con 0x7f8eb809b140 2024-09-10T10:52:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8eb7e00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c400 tx=0x7f8ea400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea40074e0 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4007680 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4004a00 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb80c8820 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.906+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb80c5e00 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.907+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb80d10f0 con 0x7f8eb809b140 2024-09-10T10:52:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.908+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ea401a020 con 0x7f8eb809b140 2024-09-10T10:52:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.909+0000 7f8eb4c00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.909+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7f8ea408afe0 con 0x7f8eb809b140 2024-09-10T10:52:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.909+0000 7f8ebd200640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:58.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.910+0000 7f8ebd200640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 0x7f8e9806e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:58.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:58.913+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8ea4059070 con 0x7f8eb809b140 2024-09-10T10:52:59.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.170+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8eb815bc40 con 0x7f8eb809b140 2024-09-10T10:52:59.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.727+0000 7f8eb4c00640 1 -- 172.21.15.28:0/637397046 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v202) ==== 101+0+0 (secure 0 0 0) 0x7f8ea4058970 con 0x7f8eb809b140 2024-09-10T10:52:59.728 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 msgr2=0x7f8e9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 0x7f8e9806e2f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8eac006fd0 tx=0x7f8eac008040 comp rx=0 tx=0).stop 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 msgr2=0x7f8eb80c7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400c400 tx=0x7f8ea400c8c0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 shutdown_connections 2024-09-10T10:52:59.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8e9806be30 0x7f8e9806e2f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8eb815e5d0 0x7f8eb80b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8eb809b140 0x7f8eb80c7cf0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 --2- 172.21.15.28:0/637397046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8eb806df60 0x7f8eb8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 >> 172.21.15.28:0/637397046 conn(0x7f8eb80d23a0 msgr2=0x7f8eb80b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 shutdown_connections 2024-09-10T10:52:59.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.730+0000 7f8ebfc00640 1 -- 172.21.15.28:0/637397046 wait complete. 2024-09-10T10:52:59.749 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:52:59.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.829+0000 7f3023423f40 1 Processor -- start 2024-09-10T10:52:59.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.829+0000 7f3023423f40 1 -- start start 2024-09-10T10:52:59.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55650e9c9210 0x55650ea4fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650ea4ff70 0x55650ea5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55650e9b4790 con 0x7f3004002870 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55650e92bb30 con 0x55650e9c9210 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3023423f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55650e9b3e60 con 0x55650ea4ff70 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44230/0 (socket says 172.21.15.28:44230) 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 -- 172.21.15.28:0/128688836 learned_addr learned my addr 172.21.15.28:0/128688836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 -- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650ea4ff70 msgr2=0x55650ea5da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650ea4ff70 0x55650ea5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 -- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55650e9c9210 msgr2=0x55650ea4fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.830+0000 7f3020600640 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55650e9c9210 0x55650ea4fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f3020600640 1 -- 172.21.15.28:0/128688836 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650e9c9bb0 con 0x7f3004002870 2024-09-10T10:52:59.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f3020600640 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f30100097b0 tx=0x7f3010005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a05563bc0ad0448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f301e800640 1 -- 172.21.15.28:0/128688836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010007830 con 0x7f3004002870 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f301e800640 1 -- 172.21.15.28:0/128688836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30100079d0 con 0x7f3004002870 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f301e800640 1 -- 172.21.15.28:0/128688836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010007d80 con 0x7f3004002870 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f3023423f40 1 -- 172.21.15.28:0/128688836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 msgr2=0x55650e988020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.831+0000 7f3023423f40 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f30100097b0 tx=0x7f3010005a00 comp rx=0 tx=0).stop 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 -- 172.21.15.28:0/128688836 shutdown_connections 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650ea4ff70 0x55650ea5da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55650e9c9210 0x55650ea4fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 --2- 172.21.15.28:0/128688836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3004002870 0x55650e988020 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 -- 172.21.15.28:0/128688836 >> 172.21.15.28:0/128688836 conn(0x55650e9b1b90 msgr2=0x55650e9b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:52:59.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 -- 172.21.15.28:0/128688836 shutdown_connections 2024-09-10T10:52:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 -- 172.21.15.28:0/128688836 wait complete. 2024-09-10T10:52:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.832+0000 7f3023423f40 1 Processor -- start 2024-09-10T10:52:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 -- start start 2024-09-10T10:52:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650e9c9210 0x55650ea4d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3004002870 0x55650e9cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55650e9c98d0 con 0x55650ea4ff70 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55650ea503f0 con 0x7f3004002870 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.833+0000 7f3023423f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55650ea5bf10 con 0x55650e9c9210 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44240/0 (socket says 172.21.15.28:44240) 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 learned_addr learned my addr 172.21.15.28:0/1732202659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650e9c9210 0x55650ea4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f3021000640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3004002870 0x55650e9cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650e9c9210 msgr2=0x55650ea4d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55650e9c9210 0x55650ea4d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3004002870 msgr2=0x55650e9cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3004002870 0x55650e9cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55650e9878c0 con 0x55650ea4ff70 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.834+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f300c046120 tx=0x7f300c005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.835+0000 7f301d400640 1 -- 172.21.15.28:0/1732202659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c0504f0 con 0x55650ea4ff70 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.835+0000 7f301d400640 1 -- 172.21.15.28:0/1732202659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c0508a0 con 0x55650ea4ff70 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.835+0000 7f301d400640 1 -- 172.21.15.28:0/1732202659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c050c70 con 0x55650ea4ff70 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.835+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55650e9cb9d0 con 0x55650ea4ff70 2024-09-10T10:52:59.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.835+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55650ea53be0 con 0x55650ea4ff70 2024-09-10T10:52:59.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.837+0000 7f301d400640 1 -- 172.21.15.28:0/1732202659 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f300c064420 con 0x55650ea4ff70 2024-09-10T10:52:59.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.837+0000 7f301d400640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 0x7f300006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.838+0000 7f301d400640 1 -- 172.21.15.28:0/1732202659 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7f300c0cd840 con 0x55650ea4ff70 2024-09-10T10:52:59.839 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.838+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 0x55650ea546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.838+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650ea54c00 con 0x55650ea4d880 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.838+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 0x7f300006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.838+0000 7f3021000640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 0x55650ea546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.839+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 0x7f300006ccd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3010000c00 tx=0x7f3010007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.839+0000 7f3021000640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 0x55650ea546c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.841+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v120'598 uv572 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.841+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650ea54fd0 con 0x55650ea4d880 2024-09-10T10:52:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.843+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v120'599 uv599 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.843+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650ea553a0 con 0x55650ea4d880 2024-09-10T10:52:59.846 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:52:59.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.845+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v120'600 uv600 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.845+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650ea55740 con 0x55650ea4d880 2024-09-10T10:52:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.847+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v120'601 uv601 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.847+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaffec0 con 0x55650ea4d880 2024-09-10T10:52:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.849+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v120'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.849+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eafeeb0 con 0x55650ea4d880 2024-09-10T10:52:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.851+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v120'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.851+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff250 con 0x55650ea4d880 2024-09-10T10:52:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.852+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v120'604 uv578 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.852+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.854+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v120'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.854+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.856 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:52:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.855+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v120'606 uv606 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.856+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.857+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v120'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.857+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.859+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v120'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.859+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.861+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v120'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.861+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.863+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v120'610 uv585 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.863+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.864+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v120'611 uv611 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.864+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.867 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:52:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.866+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v120'612 uv612 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.866+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.868+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v120'613 uv613 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.868+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.870+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v120'614 uv614 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.870+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eaff5f0 con 0x55650ea4d880 2024-09-10T10:52:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.872+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v120'615 uv615 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.872+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.873+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v120'616 uv616 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.874+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.875+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v120'617 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.875+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.877+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v120'618 uv618 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.877+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.879 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:52:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.879+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v120'619 uv619 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.879+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb01270 con 0x55650ea4d880 2024-09-10T10:52:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.880+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v120'620 uv620 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.880+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb01270 con 0x55650ea4d880 2024-09-10T10:52:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.882+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v120'621 uv621 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.882+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.884+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v120'622 uv622 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.884+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.886+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v120'623 uv597 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.886+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.887+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v120'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.887+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.890 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:52:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.889+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v120'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.889+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb01270 con 0x55650ea4d880 2024-09-10T10:52:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.890+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v120'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.890+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb01270 con 0x55650ea4d880 2024-09-10T10:52:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.892+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v120'627 uv627 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.892+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb00260 con 0x55650ea4d880 2024-09-10T10:52:59.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.894+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v120'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650ea4d880 2024-09-10T10:52:59.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.894+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 0x55650eb02b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.894+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb030a0 con 0x55650eb00770 2024-09-10T10:52:59.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.894+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 0x55650eb02b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.895+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 0x55650eb02b60 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.897+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v120'442 uv425 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.897+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb03440 con 0x55650eb00770 2024-09-10T10:52:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.898+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v120'443 uv443 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.899+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb037e0 con 0x55650eb00770 2024-09-10T10:52:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.900+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v120'444 uv444 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.901 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:52:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.900+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb03b80 con 0x55650eb00770 2024-09-10T10:52:59.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.901+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v120'445 uv445 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.902+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb03fb0 con 0x55650eb00770 2024-09-10T10:52:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.903+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v120'446 uv446 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.903+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb043a0 con 0x55650eb00770 2024-09-10T10:52:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.905+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v120'447 uv430 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.905+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04790 con 0x55650eb00770 2024-09-10T10:52:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.906+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v120'448 uv448 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.906+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b80 con 0x55650eb00770 2024-09-10T10:52:59.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:52:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.908+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v120'449 uv449 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.908+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b80 con 0x55650eb00770 2024-09-10T10:52:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.909+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v120'450 uv450 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.909+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.911+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v120'451 uv451 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.911+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.912+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v120'452 uv435 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.912+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.913+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v120'453 uv453 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.913+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.915 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:52:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.915+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v120'454 uv454 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.915+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.916+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v120'455 uv455 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.916+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.918+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v120'456 uv456 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.918+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.919+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v120'457 uv441 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.919+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.920+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v120'458 uv458 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.920+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.922 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:52:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.922+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v120'459 uv459 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.922+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.923+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v120'460 uv460 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.924+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.925+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v120'461 uv461 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.925+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb04b30 con 0x55650eb00770 2024-09-10T10:52:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.926+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v120'462 uv462 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb00770 2024-09-10T10:52:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.926+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55650eb05500 0x55650eb058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.926+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08000 con 0x55650eb05500 2024-09-10T10:52:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.927+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55650eb05500 0x55650eb058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.927+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55650eb05500 0x55650eb058e0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.929+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.929+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb083a0 con 0x55650eb05500 2024-09-10T10:52:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.930+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v120'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.930+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08740 con 0x55650eb05500 2024-09-10T10:52:59.932 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:52:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.932+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v120'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.932+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb09ff0 con 0x55650eb05500 2024-09-10T10:52:59.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.933+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v120'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.933+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0a390 con 0x55650eb05500 2024-09-10T10:52:59.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.934+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.934+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08fe0 con 0x55650eb05500 2024-09-10T10:52:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.935+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.935+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55650eb09900 0x55650eb0c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.936+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0cc80 con 0x55650eb09900 2024-09-10T10:52:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.936+0000 7f3021000640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55650eb09900 0x55650eb0c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.936+0000 7f3021000640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55650eb09900 0x55650eb0c740 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.937+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v120'87 uv86 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.937+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0cc80 con 0x55650eb09900 2024-09-10T10:52:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.939+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v120'88 uv88 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.939+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0cd00 con 0x55650eb09900 2024-09-10T10:52:59.941 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:52:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.940+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v120'89 uv89 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.940+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d0f0 con 0x55650eb09900 2024-09-10T10:52:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.941+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v120'90 uv90 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.941+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d0f0 con 0x55650eb05500 2024-09-10T10:52:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.942+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.942+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.943+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v120'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.943+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.945 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:52:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.944+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v120'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.945+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0e4a0 con 0x55650eb05500 2024-09-10T10:52:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.946+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v120'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.946+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0e4a0 con 0x55650eb05500 2024-09-10T10:52:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.947+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.947+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.948+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.948+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.950+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v120'216 uv210 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.950+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.951+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v120'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.951+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.953 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:52:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.952+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v120'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.952+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0e4a0 con 0x55650eb09900 2024-09-10T10:52:59.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.953+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v120'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.954+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0e4a0 con 0x55650eb09900 2024-09-10T10:52:59.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.955+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v120'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.955+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.956+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v120'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.956+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.957+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v120'222 uv215 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.957+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.958+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v120'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.958+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.960 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:52:59.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.960+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v120'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.960+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.962+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v120'225 uv225 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.962+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb09900 2024-09-10T10:52:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.963+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v120'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.963+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.964+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v120'97 uv96 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.964+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.965+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v120'98 uv98 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.965+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.967 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:52:59.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.966+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v120'99 uv99 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.966+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.968+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v120'100 uv100 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.968+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb0d490 con 0x55650eb05500 2024-09-10T10:52:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.969+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v120'101 uv101 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb05500 2024-09-10T10:52:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.969+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55650eb0d9a0 0x55650eb0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:52:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.969+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb103c0 con 0x55650eb0d9a0 2024-09-10T10:52:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.969+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55650eb0d9a0 0x55650eb0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:52:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.970+0000 7f3020600640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55650eb0d9a0 0x55650eb0fd90 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:52:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.971+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.971+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.973+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v120'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.973+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.975 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:52:59.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.974+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v120'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.974+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb117a0 con 0x55650eb0d9a0 2024-09-10T10:52:59.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.975+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v120'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.975+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb117a0 con 0x55650eb0d9a0 2024-09-10T10:52:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.976+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.976+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.978+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.978+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.979+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v120'192 uv186 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.979+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.981+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v120'193 uv193 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.981+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb0d9a0 2024-09-10T10:52:59.983 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:52:59.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.982+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v120'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.982+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bf0 con 0x55650eb0d9a0 2024-09-10T10:52:59.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.984+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v120'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.984+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bf0 con 0x55650eb0d9a0 2024-09-10T10:52:59.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.985+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v120'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.985+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bf0 con 0x55650eb0d9a0 2024-09-10T10:52:59.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.986+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v120'197 uv191 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.986+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bf0 con 0x55650eb0d9a0 2024-09-10T10:52:59.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.988+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v120'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.988+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bd0 con 0x55650eb0d9a0 2024-09-10T10:52:59.989 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:52:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.989+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v120'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.989+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb08bd0 con 0x55650eb0d9a0 2024-09-10T10:52:59.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.990+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v120'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.990+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb13620 con 0x55650eb0d9a0 2024-09-10T10:52:59.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.992+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v120'201 uv201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:52:59.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.992+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:52:59.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.993+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v120'87 uv86 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.993+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:52:59.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.995+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v120'88 uv88 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.995+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:52:59.997 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:52:59.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.996+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v120'89 uv89 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:52:59.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.996+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:53:00.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.998+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v120'90 uv90 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.998+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:53:00.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.999+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v120'117 uv116 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:52:59.999+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10790 con 0x55650eb09900 2024-09-10T10:53:00.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.000+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v120'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.000+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10c40 con 0x55650eb09900 2024-09-10T10:53:00.002 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:00.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.001+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v120'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.001+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11c50 con 0x55650eb09900 2024-09-10T10:53:00.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.002+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v120'120 uv120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.002+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11c50 con 0x55650eb09900 2024-09-10T10:53:00.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.003+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v120'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.004+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb10c40 con 0x55650eb09900 2024-09-10T10:53:00.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.005+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v120'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f30180112c0 con 0x55650eb09900 2024-09-10T10:53:00.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.005+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55650eb112c0 0x55650eb116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.005+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb112c0 2024-09-10T10:53:00.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.005+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55650eb112c0 0x55650eb116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.005+0000 7f301fc00640 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55650eb112c0 0x55650eb116a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.006+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v120'99 uv98 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb112c0 2024-09-10T10:53:00.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.007+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb112c0 2024-09-10T10:53:00.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.008+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v120'100 uv100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb112c0 2024-09-10T10:53:00.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.008+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb112c0 2024-09-10T10:53:00.010 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:00.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.009+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v120'101 uv101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb112c0 2024-09-10T10:53:00.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.009+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb112c0 2024-09-10T10:53:00.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.010+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v120'102 uv102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb112c0 2024-09-10T10:53:00.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.010+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb112c0 2024-09-10T10:53:00.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.011+0000 7f301fc00640 1 -- 172.21.15.28:0/1732202659 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v120'103 uv103 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f300c01a2e0 con 0x55650eb112c0 2024-09-10T10:53:00.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.011+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb0d9a0 2024-09-10T10:53:00.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.013+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v120'241 uv233 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.013+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb0d9a0 2024-09-10T10:53:00.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.014+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v120'242 uv242 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.014+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb11be0 con 0x55650eb0d9a0 2024-09-10T10:53:00.016 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:00.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.015+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v120'243 uv243 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.015+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb121e0 con 0x55650eb0d9a0 2024-09-10T10:53:00.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.016+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v120'244 uv244 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.016+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb121e0 con 0x55650eb0d9a0 2024-09-10T10:53:00.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.017+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v120'245 uv245 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.018+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb121a0 con 0x55650eb0d9a0 2024-09-10T10:53:00.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.019+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v120'246 uv240 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.019+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e120) -- 0x55650eb12160 con 0x55650eb0d9a0 2024-09-10T10:53:00.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.020+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v120'247 uv247 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.020+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb120f0 con 0x55650eb0d9a0 2024-09-10T10:53:00.022 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:00.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.021+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v120'248 uv248 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.021+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb120c0 con 0x55650eb0d9a0 2024-09-10T10:53:00.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.022+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v120'249 uv249 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.022+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb120c0 con 0x55650eb0d9a0 2024-09-10T10:53:00.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.023+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v120'250 uv250 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.023+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb120c0 con 0x55650eb0d9a0 2024-09-10T10:53:00.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.024+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v120'251 uv251 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.024+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55650eb12820 con 0x55650eb0d9a0 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3020600640 1 -- 172.21.15.28:0/1732202659 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v120'252 uv252 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3010020520 con 0x55650eb0d9a0 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55650eb05500 msgr2=0x55650eb058e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55650eb05500 0x55650eb058e0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55650eb0d9a0 msgr2=0x55650eb0fd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55650eb0d9a0 0x55650eb0fd90 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55650eb112c0 msgr2=0x55650eb116a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55650eb112c0 0x55650eb116a0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55650eb09900 msgr2=0x55650eb0c740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55650eb09900 0x55650eb0c740 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 msgr2=0x55650eb02b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 reap_dead start 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 0x55650eb02b60 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3021000640 1 -- 172.21.15.28:0/1732202659 reap_dead start 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 msgr2=0x55650ea546c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.025+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 0x55650ea546c0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 msgr2=0x7f300006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 0x7f300006ccd0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f3010000c00 tx=0x7f3010007000 comp rx=0 tx=0).stop 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 msgr2=0x55650e9cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f300c046120 tx=0x7f300c005310 comp rx=0 tx=0).stop 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 shutdown_connections 2024-09-10T10:53:00.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55650eb00770 0x55650eb02b60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55650ea4d880 0x55650ea546c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f300006a810 0x7f300006ccd0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 --2- 172.21.15.28:0/1732202659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55650ea4ff70 0x55650e9cab80 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 >> 172.21.15.28:0/1732202659 conn(0x55650e9b1b90 msgr2=0x55650e9b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 shutdown_connections 2024-09-10T10:53:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.026+0000 7f3023423f40 1 -- 172.21.15.28:0/1732202659 wait complete. 2024-09-10T10:53:00.031 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-10T10:53:00.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.147+0000 7efc53a48f40 1 Processor -- start 2024-09-10T10:53:00.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.147+0000 7efc53a48f40 1 -- start start 2024-09-10T10:53:00.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7addcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7adb4b0 0x55c3d7a05e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c3d7a09f00 con 0x55c3d7a095a0 2024-09-10T10:53:00.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3d7a4edc0 con 0x55c3d7a4f2d0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc53a48f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c3d7a4e1f0 con 0x55c3d7adb4b0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc50c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc50c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44252/0 (socket says 172.21.15.28:44252) 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.148+0000 7efc50c00640 1 -- 172.21.15.28:0/515393464 learned_addr learned my addr 172.21.15.28:0/515393464 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 -- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7adb4b0 msgr2=0x55c3d7a05e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7adb4b0 0x55c3d7a05e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 -- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 msgr2=0x55c3d7addcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc4be00640 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7addcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7addcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 -- 172.21.15.28:0/515393464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d7931b30 con 0x55c3d7a095a0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc50c00640 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efc44000c00 tx=0x7efc44051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c0bd498c96bc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc4aa00640 1 -- 172.21.15.28:0/515393464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44052ad0 con 0x55c3d7a095a0 2024-09-10T10:53:00.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc4aa00640 1 -- 172.21.15.28:0/515393464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44052c70 con 0x55c3d7a095a0 2024-09-10T10:53:00.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.149+0000 7efc4aa00640 1 -- 172.21.15.28:0/515393464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4405d640 con 0x55c3d7a095a0 2024-09-10T10:53:00.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 -- 172.21.15.28:0/515393464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 msgr2=0x55c3d7a4f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efc44000c00 tx=0x7efc44051000 comp rx=0 tx=0).stop 2024-09-10T10:53:00.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 -- 172.21.15.28:0/515393464 shutdown_connections 2024-09-10T10:53:00.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7adb4b0 0x55c3d7a05e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7addcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 --2- 172.21.15.28:0/515393464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7a095a0 0x55c3d7a4f930 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 -- 172.21.15.28:0/515393464 >> 172.21.15.28:0/515393464 conn(0x55c3d7a33b90 msgr2=0x55c3d7a35e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 -- 172.21.15.28:0/515393464 shutdown_connections 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.150+0000 7efc53a48f40 1 -- 172.21.15.28:0/515393464 wait complete. 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.151+0000 7efc53a48f40 1 Processor -- start 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.151+0000 7efc53a48f40 1 -- start start 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7a095a0 0x55c3d7ad7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7ace570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7adb4b0 0x55c3d7a32b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c3d7a09f00 con 0x55c3d7adb4b0 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3d7ada500 con 0x55c3d7a4f2d0 2024-09-10T10:53:00.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc53a48f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c3d7ad9380 con 0x55c3d7a095a0 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7adb4b0 0x55c3d7a32b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc4be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7ace570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc50c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7a095a0 0x55c3d7ad7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc4be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7ace570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:43766/0 (socket says 172.21.15.28:43766) 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc4be00640 1 -- 172.21.15.28:0/1152827135 learned_addr learned my addr 172.21.15.28:0/1152827135 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 -- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7a095a0 msgr2=0x55c3d7ad7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c3d7a095a0 0x55c3d7ad7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 -- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 msgr2=0x55c3d7ace570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7ace570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc51600640 1 -- 172.21.15.28:0/1152827135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3d7a044f0 con 0x55c3d7adb4b0 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.152+0000 7efc4be00640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3d7a4f2d0 0x55c3d7ace570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:00.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc51600640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7adb4b0 0x55c3d7a32b80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efc4c005f40 tx=0x7efc4c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc49600640 1 -- 172.21.15.28:0/1152827135 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c013430 con 0x55c3d7adb4b0 2024-09-10T10:53:00.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc49600640 1 -- 172.21.15.28:0/1152827135 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c013610 con 0x55c3d7adb4b0 2024-09-10T10:53:00.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc49600640 1 -- 172.21.15.28:0/1152827135 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c013800 con 0x55c3d7adb4b0 2024-09-10T10:53:00.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3d7addef0 con 0x55c3d7adb4b0 2024-09-10T10:53:00.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.153+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3d7ad95b0 con 0x55c3d7adb4b0 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.155+0000 7efc49600640 1 -- 172.21.15.28:0/1152827135 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc4c01a020 con 0x55c3d7adb4b0 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.156+0000 7efc49600640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efc3006a810 0x7efc3006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.156+0000 7efc49600640 1 -- 172.21.15.28:0/1152827135 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7efc4c08a060 con 0x55c3d7adb4b0 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.156+0000 7efc53a48f40 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c3d7ad4f00 0x55c3d7ad52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.156+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x55c3d7ad5820 con 0x55c3d7ad4f00 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.156+0000 7efc50c00640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efc3006a810 0x7efc3006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.157+0000 7efc4be00640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c3d7ad4f00 0x55c3d7ad52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.157+0000 7efc50c00640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efc3006a810 0x7efc3006ccd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efc4400a4f0 tx=0x7efc44053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.157+0000 7efc4be00640 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c3d7ad4f00 0x55c3d7ad52e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc4be00640 1 -- 172.21.15.28:0/1152827135 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~80] v120'202 uv202 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7efc3c00b220 con 0x55c3d7ad4f00 2024-09-10T10:53:00.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c3d7ad4f00 msgr2=0x55c3d7ad52e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c3d7ad4f00 0x55c3d7ad52e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efc3006a810 msgr2=0x7efc3006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efc3006a810 0x7efc3006ccd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7efc4400a4f0 tx=0x7efc44053000 comp rx=0 tx=0).stop 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7adb4b0 msgr2=0x55c3d7a32b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 --2- 172.21.15.28:0/1152827135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c3d7adb4b0 0x55c3d7a32b80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efc4c005f40 tx=0x7efc4c00a570 comp rx=0 tx=0).stop 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc51600640 1 -- 172.21.15.28:0/1152827135 reap_dead start 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 shutdown_connections 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 >> 172.21.15.28:0/1152827135 conn(0x55c3d7a33b90 msgr2=0x55c3d7adc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 shutdown_connections 2024-09-10T10:53:00.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.159+0000 7efc53a48f40 1 -- 172.21.15.28:0/1152827135 wait complete. 2024-09-10T10:53:00.164 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:00.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.362+0000 7fb9ca600640 1 Processor -- start 2024-09-10T10:53:00.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.362+0000 7fb9ca600640 1 -- start start 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9ca600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb9c40b4020 0x7fb9c40b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c4072050 0x7fb9c40724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9ca600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9c40b4620 con 0x7fb9c415e900 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9ca600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb9c40b4f60 con 0x7fb9c40b4020 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9ca600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c4072a00 con 0x7fb9c4072050 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb9c40b4020 0x7fb9c40b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb9c40b4020 0x7fb9c40b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44264/0 (socket says 172.21.15.28:44264) 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 -- 172.21.15.28:0/2713167917 learned_addr learned my addr 172.21.15.28:0/2713167917 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:00.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9c3e00640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c4072050 0x7fb9c40724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 -- 172.21.15.28:0/2713167917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c4072050 msgr2=0x7fb9c40724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c4072050 0x7fb9c40724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 -- 172.21.15.28:0/2713167917 >> v1:172.21.15.28:6789/0 conn(0x7fb9c415e900 legacy=0x7fb9c40ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9bbe00640 1 -- 172.21.15.28:0/2713167917 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c40ce100 con 0x7fb9c40b4020 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.364+0000 7fb9c2a00640 1 -- 172.21.15.28:0/2713167917 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1965060497 0 0) 0x7fb9c40b4620 con 0x7fb9c415e900 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.365+0000 7fb9bbe00640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c40b4a20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0f2aff5365899e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9c2a00640 1 -- 172.21.15.28:0/2713167917 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037070 con 0x7fb9c40b4020 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9c2a00640 1 -- 172.21.15.28:0/2713167917 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac032440 con 0x7fb9c40b4020 2024-09-10T10:53:00.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9c2a00640 1 -- 172.21.15.28:0/2713167917 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac032810 con 0x7fb9c40b4020 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 -- 172.21.15.28:0/2713167917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 msgr2=0x7fb9c40b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c40b4a20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 -- 172.21.15.28:0/2713167917 shutdown_connections 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c4072050 0x7fb9c40724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 --2- 172.21.15.28:0/2713167917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c40b4a20 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 -- 172.21.15.28:0/2713167917 >> 172.21.15.28:0/2713167917 conn(0x7fb9c406bdf0 msgr2=0x7fb9c406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 -- 172.21.15.28:0/2713167917 shutdown_connections 2024-09-10T10:53:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.366+0000 7fb9ca600640 1 -- 172.21.15.28:0/2713167917 wait complete. 2024-09-10T10:53:00.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.368+0000 7fb9ca600640 1 Processor -- start 2024-09-10T10:53:00.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.368+0000 7fb9ca600640 1 -- start start 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c4072050 0x7fb9c4151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c415e900 0x7fb9c4153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40ce960 con 0x7fb9c40b4020 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c40ce680 con 0x7fb9c4072050 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9ca600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c40cee80 con 0x7fb9c415e900 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.369+0000 7fb9bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44272/0 (socket says 172.21.15.28:44272) 2024-09-10T10:53:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 -- 172.21.15.28:0/1631942278 learned_addr learned my addr 172.21.15.28:0/1631942278 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 -- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c415e900 msgr2=0x7fb9c4153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c415e900 0x7fb9c4153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 -- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c4072050 msgr2=0x7fb9c4151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c4072050 0x7fb9c4151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 -- 172.21.15.28:0/1631942278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c4153340 con 0x7fb9c40b4020 2024-09-10T10:53:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bbe00640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac037040 con 0x7fb9c40b4020 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9ac004250 con 0x7fb9c40b4020 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac003610 con 0x7fb9c40b4020 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.370+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c40af200 con 0x7fb9c40b4020 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.371+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c4153b60 con 0x7fb9c40b4020 2024-09-10T10:53:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.372+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c40724c0 con 0x7fb9c40b4020 2024-09-10T10:53:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.373+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb9ac02ad10 con 0x7fb9c40b4020 2024-09-10T10:53:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.374+0000 7fb9bb400640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 0x7fb9a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.374+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8024+0+0 (secure 0 0 0) 0x7fb9ac0ac2c0 con 0x7fb9c40b4020 2024-09-10T10:53:00.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.374+0000 7fb9c3400640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 0x7fb9a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.375+0000 7fb9c3400640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 0x7fb9a006e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb9c41f7810 tx=0x7fb9b4007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.378+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb9ac07a480 con 0x7fb9c40b4020 2024-09-10T10:53:00.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.600+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb9c40d05d0 con 0x7fb9c40b4020 2024-09-10T10:53:00.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.734+0000 7fb9bb400640 1 -- 172.21.15.28:0/1631942278 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v203) ==== 162+0+0 (secure 0 0 0) 0x7fb9ac079d80 con 0x7fb9c40b4020 2024-09-10T10:53:00.735 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 msgr2=0x7fb9a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 0x7fb9a006e2f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb9c41f7810 tx=0x7fb9b4007930 comp rx=0 tx=0).stop 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 msgr2=0x7fb9c4153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac009870 tx=0x7fb9ac0046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 shutdown_connections 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb9a006be30 0x7fb9a006e2f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9c415e900 0x7fb9c4153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9c40b4020 0x7fb9c4153620 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 --2- 172.21.15.28:0/1631942278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9c4072050 0x7fb9c4151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 >> 172.21.15.28:0/1631942278 conn(0x7fb9c406bdf0 msgr2=0x7fb9c40ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 shutdown_connections 2024-09-10T10:53:00.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.737+0000 7fb9ca600640 1 -- 172.21.15.28:0/1631942278 wait complete. 2024-09-10T10:53:00.745 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:00.744+0000 7f64f2a00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:53:00.746 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:00.745+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:00.746 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:00.745+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:00.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:00.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.916+0000 7f1817400640 1 Processor -- start 2024-09-10T10:53:00.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.916+0000 7f1817400640 1 -- start start 2024-09-10T10:53:00.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.917+0000 7f1817400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1810072320 0x7f18100cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.917+0000 7f1817400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f181015e1f0 0x7f181006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.917+0000 7f1817400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1810146140 con 0x7f1810072de0 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.917+0000 7f1817400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f18100cd740 con 0x7f1810072320 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.917+0000 7f1817400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18100cda20 con 0x7f181015e1f0 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1810072320 0x7f18100cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f180fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1810072de0 0x7f181006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43896/0 (socket says 172.21.15.28:43896) 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1810072320 0x7f18100cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44276/0 (socket says 172.21.15.28:44276) 2024-09-10T10:53:00.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f180fe00640 1 -- 172.21.15.28:0/4225229291 learned_addr learned my addr 172.21.15.28:0/4225229291 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:00.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1815400640 1 --2- 172.21.15.28:0/4225229291 >> v2:172.21.15.138:3300/0 conn(0x7f181015e1f0 0x7f181006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 -- 172.21.15.28:0/4225229291 >> v2:172.21.15.138:3300/0 conn(0x7f181015e1f0 msgr2=0x7f181006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:00.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 --2- 172.21.15.28:0/4225229291 >> v2:172.21.15.138:3300/0 conn(0x7f181015e1f0 0x7f181006e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 -- 172.21.15.28:0/4225229291 >> v1:172.21.15.28:6789/0 conn(0x7f1810072de0 legacy=0x7f181006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f1814a00640 1 -- 172.21.15.28:0/4225229291 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18100cdd00 con 0x7f1810072320 2024-09-10T10:53:00.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.918+0000 7f180f400640 1 -- 172.21.15.28:0/4225229291 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522815222 0 0) 0x7f1810146140 con 0x7f1810072de0 2024-09-10T10:53:00.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.919+0000 7f1814a00640 1 --2- 172.21.15.28:0/4225229291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100cce60 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f18100cfce0 tx=0x7f180402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c75d0fbc438059 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.919+0000 7f180f400640 1 -- 172.21.15.28:0/4225229291 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804037070 con 0x7f1810072320 2024-09-10T10:53:00.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.919+0000 7f180f400640 1 -- 172.21.15.28:0/4225229291 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804032440 con 0x7f1810072320 2024-09-10T10:53:00.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.919+0000 7f180f400640 1 -- 172.21.15.28:0/4225229291 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180402ae60 con 0x7f1810072320 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 -- 172.21.15.28:0/4225229291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 msgr2=0x7f18100cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 --2- 172.21.15.28:0/4225229291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100cce60 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f18100cfce0 tx=0x7f180402a110 comp rx=0 tx=0).stop 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 -- 172.21.15.28:0/4225229291 shutdown_connections 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 --2- 172.21.15.28:0/4225229291 >> v2:172.21.15.138:3300/0 conn(0x7f181015e1f0 0x7f181006e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 --2- 172.21.15.28:0/4225229291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100cce60 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 -- 172.21.15.28:0/4225229291 >> 172.21.15.28:0/4225229291 conn(0x7f1810067d70 msgr2=0x7f181006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 -- 172.21.15.28:0/4225229291 shutdown_connections 2024-09-10T10:53:00.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.920+0000 7f1817400640 1 -- 172.21.15.28:0/4225229291 wait complete. 2024-09-10T10:53:00.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.922+0000 7f1817400640 1 Processor -- start 2024-09-10T10:53:00.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.922+0000 7f1817400640 1 -- start start 2024-09-10T10:53:00.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1810072de0 0x7f1810143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181015e1f0 0x7f18100adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f18100cd520 con 0x7f1810072320 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18100cd3a0 con 0x7f1810072de0 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1817400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f181006ed00 con 0x7f181015e1f0 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44290/0 (socket says 172.21.15.28:44290) 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 -- 172.21.15.28:0/1613888762 learned_addr learned my addr 172.21.15.28:0/1613888762 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1815400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181015e1f0 0x7f18100adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f180fe00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1810072de0 0x7f1810143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 -- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181015e1f0 msgr2=0x7f18100adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181015e1f0 0x7f18100adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 -- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1810072de0 msgr2=0x7f1810143f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:00.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1810072de0 0x7f1810143f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:00.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.923+0000 7f1814a00640 1 -- 172.21.15.28:0/1613888762 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18100b00d0 con 0x7f1810072320 2024-09-10T10:53:00.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f1814a00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1804000c00 tx=0x7f18040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804037050 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804038040 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804006e10 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18100ae540 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.924+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18100ab4d0 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.925+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810072700 con 0x7f1810072320 2024-09-10T10:53:00.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.926+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1804030980 con 0x7f1810072320 2024-09-10T10:53:00.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.927+0000 7f180cc00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 0x7f17f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:00.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.927+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f18040ac260 con 0x7f1810072320 2024-09-10T10:53:00.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.927+0000 7f180fe00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 0x7f17f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:00.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.928+0000 7f180fe00640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 0x7f17f006e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f181006ec10 tx=0x7f17fc007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:00.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:00.931+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18100b00d0 con 0x7f1810072320 2024-09-10T10:53:01.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.153+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18101e4930 con 0x7f1810072320 2024-09-10T10:53:01.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.154+0000 7f180cc00640 1 -- 172.21.15.28:0/1613888762 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+3566 (secure 0 0 0) 0x7f18040ab090 con 0x7f1810072320 2024-09-10T10:53:01.155 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:01.155 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":205,"btime":"2024-09-10T10:53:00:745882+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:00.745809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:01.155 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 205 2024-09-10T10:53:01.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 msgr2=0x7f17f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 0x7f17f006e2f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f181006ec10 tx=0x7f17fc007d50 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 msgr2=0x7f18100afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1804000c00 tx=0x7f18040046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 shutdown_connections 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f17f006be30 0x7f17f006e2f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f181015e1f0 0x7f18100adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1810072de0 0x7f1810143f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.156+0000 7f1817400640 1 --2- 172.21.15.28:0/1613888762 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1810072320 0x7f18100afb90 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.157+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 >> 172.21.15.28:0/1613888762 conn(0x7f1810067d70 msgr2=0x7f18100ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.157+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 shutdown_connections 2024-09-10T10:53:01.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.157+0000 7f1817400640 1 -- 172.21.15.28:0/1613888762 wait complete. 2024-09-10T10:53:01.173 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:01.173 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:01.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.329+0000 7f000f800640 1 Processor -- start 2024-09-10T10:53:01.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.330+0000 7f000f800640 1 -- start start 2024-09-10T10:53:01.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0008072320 0x7f00080cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f000806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0008072700 0x7f000806e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0008146140 con 0x7f0008072700 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00080cd740 con 0x7f0008072320 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00080cda20 con 0x7f0008072de0 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0008072700 0x7f000806e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0008072700 0x7f000806e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44306/0 (socket says 172.21.15.28:44306) 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000d800640 1 -- 172.21.15.28:0/2209567293 learned_addr learned my addr 172.21.15.28:0/2209567293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f000ce00640 1 --2- 172.21.15.28:0/2209567293 >> v2:172.21.15.138:3300/0 conn(0x7f0008072320 0x7f00080cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.331+0000 7f0007e00640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f000806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.332+0000 7f000d800640 1 -- 172.21.15.28:0/2209567293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 msgr2=0x7f000806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:01.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.332+0000 7f000d800640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f000806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.332+0000 7f000d800640 1 -- 172.21.15.28:0/2209567293 >> v2:172.21.15.138:3300/0 conn(0x7f0008072320 msgr2=0x7f00080cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:01.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.332+0000 7f000d800640 1 --2- 172.21.15.28:0/2209567293 >> v2:172.21.15.138:3300/0 conn(0x7f0008072320 0x7f00080cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.332+0000 7f000d800640 1 -- 172.21.15.28:0/2209567293 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00080cdd00 con 0x7f0008072700 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f000d800640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072700 0x7f000806e5f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f00080b31e0 tx=0x7f000002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868c5e8d38f7d150 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f0007400640 1 -- 172.21.15.28:0/2209567293 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0000036070 con 0x7f0008072700 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f0007400640 1 -- 172.21.15.28:0/2209567293 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0000031600 con 0x7f0008072700 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f0007400640 1 -- 172.21.15.28:0/2209567293 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00000319d0 con 0x7f0008072700 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f000f800640 1 -- 172.21.15.28:0/2209567293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072700 msgr2=0x7f000806e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.333+0000 7f000f800640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072700 0x7f000806e5f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f00080b31e0 tx=0x7f000002a610 comp rx=0 tx=0).stop 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 -- 172.21.15.28:0/2209567293 shutdown_connections 2024-09-10T10:53:01.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072700 0x7f000806e5f0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 --2- 172.21.15.28:0/2209567293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f000806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 --2- 172.21.15.28:0/2209567293 >> v2:172.21.15.138:3300/0 conn(0x7f0008072320 0x7f00080cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 -- 172.21.15.28:0/2209567293 >> 172.21.15.28:0/2209567293 conn(0x7f0008067d70 msgr2=0x7f000806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 -- 172.21.15.28:0/2209567293 shutdown_connections 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.334+0000 7f000f800640 1 -- 172.21.15.28:0/2209567293 wait complete. 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.335+0000 7f000f800640 1 Processor -- start 2024-09-10T10:53:01.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 -- start start 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 0x7f00081e28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f00081e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00080cd520 con 0x7f0008072320 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00080cd3a0 con 0x7f0008072700 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.336+0000 7f000f800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f000806eb30 con 0x7f0008072de0 2024-09-10T10:53:01.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44320/0 (socket says 172.21.15.28:44320) 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 -- 172.21.15.28:0/234405891 learned_addr learned my addr 172.21.15.28:0/234405891 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f0007e00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 0x7f00081e28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000d800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f00081e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 -- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 msgr2=0x7f00081e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f00081e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 -- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 msgr2=0x7f00081e28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 0x7f00081e28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 -- 172.21.15.28:0/234405891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00081f42d0 con 0x7f0008072320 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f0007e00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 0x7f00081e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000d800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f00081e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:01.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.337+0000 7f000ce00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e8c0 tx=0x7efffc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.338+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc017070 con 0x7f0008072320 2024-09-10T10:53:01.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.338+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efffc00c940 con 0x7f0008072320 2024-09-10T10:53:01.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.338+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00081f45b0 con 0x7f0008072320 2024-09-10T10:53:01.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.338+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc00cd10 con 0x7f0008072320 2024-09-10T10:53:01.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.338+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00081f4b70 con 0x7f0008072320 2024-09-10T10:53:01.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.339+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00080cce60 con 0x7f0008072320 2024-09-10T10:53:01.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.340+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efffc01a070 con 0x7f0008072320 2024-09-10T10:53:01.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.341+0000 7f0004c00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 0x7effe806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.341+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7efffc08bbf0 con 0x7f0008072320 2024-09-10T10:53:01.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.341+0000 7f0007e00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 0x7effe806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.342+0000 7f0007e00640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 0x7effe806e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f00080c67b0 tx=0x7efff4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.345+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efffc059ba0 con 0x7f0008072320 2024-09-10T10:53:01.448 INFO:tasks.ceph.mds.c.smithi028.stderr: -33> 2024-09-10T10:52:56.695+0000 7f5c10400640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': Decoder at 'virtual void SnapServer::decode_server_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=5 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:53:01.487 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:01.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.569+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00081f4f70 con 0x7f0008072320 2024-09-10T10:53:01.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.570+0000 7f0004c00640 1 -- 172.21.15.28:0/234405891 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) ==== 95+0+3566 (secure 0 0 0) 0x7efffc0594a0 con 0x7f0008072320 2024-09-10T10:53:01.571 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:01.571 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":205,"btime":"2024-09-10T10:53:00:745882+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:00.745809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:01.571 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 205 2024-09-10T10:53:01.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 msgr2=0x7effe806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 0x7effe806e2f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f00080c67b0 tx=0x7efff4008040 comp rx=0 tx=0).stop 2024-09-10T10:53:01.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 msgr2=0x7f00081f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efffc00e8c0 tx=0x7efffc00ed80 comp rx=0 tx=0).stop 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 shutdown_connections 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7effe806be30 0x7effe806e2f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0008072de0 0x7f00081e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0008072700 0x7f00081e28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 --2- 172.21.15.28:0/234405891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0008072320 0x7f00081f3bf0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 >> 172.21.15.28:0/234405891 conn(0x7f0008067d70 msgr2=0x7f00080cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 shutdown_connections 2024-09-10T10:53:01.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.572+0000 7f000f800640 1 -- 172.21.15.28:0/234405891 wait complete. 2024-09-10T10:53:01.589 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:01.589 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:01.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.746+0000 7f58c2200640 1 Processor -- start 2024-09-10T10:53:01.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.746+0000 7f58c2200640 1 -- start start 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58c2200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f58bc072320 0x7f58bc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58c2200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc15e1f0 0x7f58bc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58c2200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58bc146140 con 0x7f58bc072de0 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58c2200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f58bc0cd740 con 0x7f58bc072320 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58c2200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc0cda20 con 0x7f58bc15e1f0 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58baa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f58bc072de0 0x7f58bc06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43908/0 (socket says 172.21.15.28:43908) 2024-09-10T10:53:01.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58baa00640 1 -- 172.21.15.28:0/2445978944 learned_addr learned my addr 172.21.15.28:0/2445978944 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58bb400640 1 --2- 172.21.15.28:0/2445978944 >> v2:172.21.15.138:3300/0 conn(0x7f58bc072320 0x7f58bc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.748+0000 7f58bbe00640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc15e1f0 0x7f58bc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.749+0000 7f58bb400640 1 -- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc15e1f0 msgr2=0x7f58bc06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.749+0000 7f58bb400640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc15e1f0 0x7f58bc06e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.749+0000 7f58bb400640 1 -- 172.21.15.28:0/2445978944 >> v1:172.21.15.28:6789/0 conn(0x7f58bc072de0 legacy=0x7f58bc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.749+0000 7f58bb400640 1 -- 172.21.15.28:0/2445978944 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc0cdd00 con 0x7f58bc072320 2024-09-10T10:53:01.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.750+0000 7f58bb400640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072320 0x7f58bc0cce60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0097a0 tx=0x7f58ac02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2475a45ebd1e8783 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58ba000640 1 -- 172.21.15.28:0/2445978944 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac037070 con 0x7f58bc072320 2024-09-10T10:53:01.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58ba000640 1 -- 172.21.15.28:0/2445978944 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58ac02ae40 con 0x7f58bc072320 2024-09-10T10:53:01.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58ba000640 1 -- 172.21.15.28:0/2445978944 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58ac032660 con 0x7f58bc072320 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 -- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072320 msgr2=0x7f58bc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072320 0x7f58bc0cce60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0097a0 tx=0x7f58ac02a0f0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 -- 172.21.15.28:0/2445978944 shutdown_connections 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc15e1f0 0x7f58bc06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 --2- 172.21.15.28:0/2445978944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072320 0x7f58bc0cce60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.751+0000 7f58c2200640 1 -- 172.21.15.28:0/2445978944 >> 172.21.15.28:0/2445978944 conn(0x7f58bc067d70 msgr2=0x7f58bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.752+0000 7f58c2200640 1 -- 172.21.15.28:0/2445978944 shutdown_connections 2024-09-10T10:53:01.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.752+0000 7f58c2200640 1 -- 172.21.15.28:0/2445978944 wait complete. 2024-09-10T10:53:01.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.753+0000 7f58c2200640 1 Processor -- start 2024-09-10T10:53:01.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.754+0000 7f58c2200640 1 -- start start 2024-09-10T10:53:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc072320 0x7f58bc0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 0x7f58bc1402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0cd520 con 0x7f58bc15e1f0 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc0cd3a0 con 0x7f58bc072de0 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58c2200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc06ed00 con 0x7f58bc072320 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44338/0 (socket says 172.21.15.28:44338) 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58bbe00640 1 -- 172.21.15.28:0/1662474216 learned_addr learned my addr 172.21.15.28:0/1662474216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58bb400640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc072320 0x7f58bc0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.755+0000 7f58baa00640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 0x7f58bc1402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 -- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc072320 msgr2=0x7f58bc0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc072320 0x7f58bc0ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 -- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 msgr2=0x7f58bc1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 0x7f58bc1402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 -- 172.21.15.28:0/1662474216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc0b53e0 con 0x7f58bc15e1f0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58baa00640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 0x7f58bc1402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58bbe00640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f58b400efc0 tx=0x7f58b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4018020 con 0x7f58bc15e1f0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58b4014480 con 0x7f58bc15e1f0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58bc0c9740 con 0x7f58bc15e1f0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4009e60 con 0x7f58bc15e1f0 2024-09-10T10:53:01.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.756+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58bc140800 con 0x7f58bc15e1f0 2024-09-10T10:53:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.757+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58bc0c9530 con 0x7f58bc15e1f0 2024-09-10T10:53:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.758+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58b4005d10 con 0x7f58bc15e1f0 2024-09-10T10:53:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.759+0000 7f58b3400640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 0x7f589c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:01.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.759+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f58b408c510 con 0x7f58bc15e1f0 2024-09-10T10:53:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.760+0000 7f58bb400640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 0x7f589c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:01.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.760+0000 7f58bb400640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 0x7f589c06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f58ac034000 tx=0x7f58ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:01.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.764+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58b4022030 con 0x7f58bc15e1f0 2024-09-10T10:53:01.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.986+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58bc0008d0 con 0x7f58bc15e1f0 2024-09-10T10:53:01.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.987+0000 7f58b3400640 1 -- 172.21.15.28:0/1662474216 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) ==== 95+0+4392 (secure 0 0 0) 0x7f58b4010040 con 0x7f58bc15e1f0 2024-09-10T10:53:01.988 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:01.988 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":206,"btime":"2024-09-10T10:53:01:747618+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8871,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/4027758898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":4027758898},{"type":"v1","addr":"172.21.15.28:6834","nonce":4027758898}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":206}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:00.745809+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":121,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:01.988 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 206 2024-09-10T10:53:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 msgr2=0x7f589c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 0x7f589c06e2f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f58ac034000 tx=0x7f58ac02e000 comp rx=0 tx=0).stop 2024-09-10T10:53:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 msgr2=0x7f58bc1e77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f58b400efc0 tx=0x7f58b400c490 comp rx=0 tx=0).stop 2024-09-10T10:53:01.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 shutdown_connections 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f589c06be30 0x7f589c06e2f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58bc15e1f0 0x7f58bc1e77d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58bc072de0 0x7f58bc1402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 --2- 172.21.15.28:0/1662474216 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f58bc072320 0x7f58bc0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 >> 172.21.15.28:0/1662474216 conn(0x7f58bc067d70 msgr2=0x7f58bc1441e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 shutdown_connections 2024-09-10T10:53:01.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:01.989+0000 7f58c2200640 1 -- 172.21.15.28:0/1662474216 wait complete. 2024-09-10T10:53:02.006 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt mds0_inotable' led to DAMAGED state 2024-09-10T10:53:02.007 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 400.00000000' 2024-09-10T10:53:02.007 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:02.007 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:02.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:02.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:02.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:02.043 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:02.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:02.118 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:02.118 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:02.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.315+0000 7f9719600640 1 Processor -- start 2024-09-10T10:53:02.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.315+0000 7f9719600640 1 -- start start 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.316+0000 7f9719600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f97140720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.316+0000 7f9719600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f97140725f0 0x7f97140aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.316+0000 7f9719600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f971406f480 con 0x7f9714147230 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.316+0000 7f9719600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f97140b34d0 con 0x7f97140725f0 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.316+0000 7f9719600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97140ae9e0 con 0x7f9714147e90 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9712a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9714147230 0x7f971406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43924/0 (socket says 172.21.15.28:43924) 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9712a00640 1 -- 172.21.15.28:0/3467737084 learned_addr learned my addr 172.21.15.28:0/3467737084 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:02.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9713400640 1 --2- 172.21.15.28:0/3467737084 >> v2:172.21.15.28:3300/0 conn(0x7f97140725f0 0x7f97140aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f970be00640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f97140720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9712000640 1 -- 172.21.15.28:0/3467737084 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091412520 0 0) 0x7f971406f480 con 0x7f9714147230 2024-09-10T10:53:02.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9713400640 1 -- 172.21.15.28:0/3467737084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 msgr2=0x7f97140720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9713400640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f97140720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9713400640 1 -- 172.21.15.28:0/3467737084 >> v1:172.21.15.28:6789/0 conn(0x7f9714147230 legacy=0x7f971406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.317+0000 7f9713400640 1 -- 172.21.15.28:0/3467737084 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971406e900 con 0x7f97140725f0 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.318+0000 7f9713400640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97140725f0 0x7f97140aee10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fc58e8a8737284 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9712000640 1 -- 172.21.15.28:0/3467737084 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c007740 con 0x7f97140725f0 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9712000640 1 -- 172.21.15.28:0/3467737084 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f970c0078e0 con 0x7f97140725f0 2024-09-10T10:53:02.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9712000640 1 -- 172.21.15.28:0/3467737084 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c007cb0 con 0x7f97140725f0 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 -- 172.21.15.28:0/3467737084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97140725f0 msgr2=0x7f97140aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97140725f0 0x7f97140aee10 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f970c009d90 tx=0x7f970c02a660 comp rx=0 tx=0).stop 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 -- 172.21.15.28:0/3467737084 shutdown_connections 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97140725f0 0x7f97140aee10 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 --2- 172.21.15.28:0/3467737084 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f97140720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 -- 172.21.15.28:0/3467737084 >> 172.21.15.28:0/3467737084 conn(0x7f9714067d70 msgr2=0x7f971406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 -- 172.21.15.28:0/3467737084 shutdown_connections 2024-09-10T10:53:02.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.319+0000 7f9719600640 1 -- 172.21.15.28:0/3467737084 wait complete. 2024-09-10T10:53:02.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.321+0000 7f9719600640 1 Processor -- start 2024-09-10T10:53:02.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.321+0000 7f9719600640 1 -- start start 2024-09-10T10:53:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97140725f0 0x7f9714133110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f971415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f971406eec0 con 0x7f9714147230 2024-09-10T10:53:02.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9714072a40 con 0x7f97140725f0 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9719600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f971406f1a0 con 0x7f9714147e90 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f970be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f970be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44344/0 (socket says 172.21.15.28:44344) 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f970be00640 1 -- 172.21.15.28:0/2860075543 learned_addr learned my addr 172.21.15.28:0/2860075543 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.322+0000 7f9712a00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97140725f0 0x7f9714133110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f9713400640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f971415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 -- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 msgr2=0x7f971415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f971415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 -- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97140725f0 msgr2=0x7f9714133110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97140725f0 0x7f9714133110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 -- 172.21.15.28:0/2860075543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9714133780 con 0x7f9714147230 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970be00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00d5f0 tx=0x7f96fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0125f0 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96fc019070 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0129d0 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9714133a60 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.323+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9714134020 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.324+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97140b4900 con 0x7f9714147230 2024-09-10T10:53:02.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.325+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96fc012b70 con 0x7f9714147230 2024-09-10T10:53:02.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.326+0000 7f970aa00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 0x7f96f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.326+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f96fc08bb20 con 0x7f9714147230 2024-09-10T10:53:02.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.327+0000 7f9712a00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 0x7f96f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.327+0000 7f9712a00640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 0x7f96f006e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9704000f60 tx=0x7f9704007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.331+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97140b4900 con 0x7f9714147230 2024-09-10T10:53:02.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.553+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9714146d70 con 0x7f9714147230 2024-09-10T10:53:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.754+0000 7f970aa00640 1 -- 172.21.15.28:0/2860075543 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v207) ==== 161+0+0 (secure 0 0 0) 0x7f96fc059b60 con 0x7f9714147230 2024-09-10T10:53:02.756 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 msgr2=0x7f96f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 0x7f96f006e2f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f9704000f60 tx=0x7f9704007d50 comp rx=0 tx=0).stop 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 msgr2=0x7f97141f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00d5f0 tx=0x7f96fc00dab0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 shutdown_connections 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f96f006be30 0x7f96f006e2f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9714147e90 0x7f971415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9714147230 0x7f97141f4480 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 --2- 172.21.15.28:0/2860075543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97140725f0 0x7f9714133110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 >> 172.21.15.28:0/2860075543 conn(0x7f9714067d70 msgr2=0x7f9714142b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:02.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 shutdown_connections 2024-09-10T10:53:02.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.758+0000 7f9719600640 1 -- 172.21.15.28:0/2860075543 wait complete. 2024-09-10T10:53:02.776 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.933+0000 7f41c1a00640 1 Processor -- start 2024-09-10T10:53:02.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.933+0000 7f41c1a00640 1 -- start start 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.934+0000 7f41c1a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 0x7f41bc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41c1a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc146140 con 0x7f41bc072de0 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41c1a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41bc06eed0 con 0x7f41bc15e1f0 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41c1a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc0cd740 con 0x7f41bc072320 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41b3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f41bc072de0 0x7f41bc06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:43934/0 (socket says 172.21.15.28:43934) 2024-09-10T10:53:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41b3e00640 1 -- 172.21.15.28:0/3541898033 learned_addr learned my addr 172.21.15.28:0/3541898033 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:02.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41baa00640 1 --2- 172.21.15.28:0/3541898033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 0x7f41bc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013818704 0 0) 0x7f41bc146140 con 0x7f41bc072de0 2024-09-10T10:53:02.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.935+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41a0003520 con 0x7f41bc072de0 2024-09-10T10:53:02.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.936+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 693057893 0 0) 0x7f41bc06eed0 con 0x7f41bc15e1f0 2024-09-10T10:53:02.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.936+0000 7f41baa00640 1 -- 172.21.15.28:0/3541898033 >> v1:172.21.15.138:6789/0 conn(0x7f41bc15e1f0 legacy=0x7f41bc06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.936+0000 7f41baa00640 1 -- 172.21.15.28:0/3541898033 >> v1:172.21.15.28:6789/0 conn(0x7f41bc072de0 legacy=0x7f41bc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.936+0000 7f41baa00640 1 -- 172.21.15.28:0/3541898033 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc0cda20 con 0x7f41bc072320 2024-09-10T10:53:02.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41baa00640 1 --2- 172.21.15.28:0/3541898033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 0x7f41bc0cce60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0097d0 tx=0x7f41ac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e764e43583f51774 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac037070 con 0x7f41bc072320 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac032440 con 0x7f41bc072320 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41ba000640 1 -- 172.21.15.28:0/3541898033 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0327f0 con 0x7f41bc072320 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41c1a00640 1 -- 172.21.15.28:0/3541898033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 msgr2=0x7f41bc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.937+0000 7f41c1a00640 1 --2- 172.21.15.28:0/3541898033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 0x7f41bc0cce60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0097d0 tx=0x7f41ac02a180 comp rx=0 tx=0).stop 2024-09-10T10:53:02.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.938+0000 7f41c1a00640 1 -- 172.21.15.28:0/3541898033 shutdown_connections 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.938+0000 7f41c1a00640 1 --2- 172.21.15.28:0/3541898033 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072320 0x7f41bc0cce60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.938+0000 7f41c1a00640 1 -- 172.21.15.28:0/3541898033 >> 172.21.15.28:0/3541898033 conn(0x7f41bc067d70 msgr2=0x7f41bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.938+0000 7f41c1a00640 1 -- 172.21.15.28:0/3541898033 shutdown_connections 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.938+0000 7f41c1a00640 1 -- 172.21.15.28:0/3541898033 wait complete. 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.939+0000 7f41c1a00640 1 Processor -- start 2024-09-10T10:53:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.939+0000 7f41c1a00640 1 -- start start 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41bc072320 0x7f41bc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072de0 0x7f41bc13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0cd3a0 con 0x7f41bc15e1f0 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc15c9e0 con 0x7f41bc072320 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.940+0000 7f41c1a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc0cd520 con 0x7f41bc072de0 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44348/0 (socket says 172.21.15.28:44348) 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 -- 172.21.15.28:0/156421857 learned_addr learned my addr 172.21.15.28:0/156421857 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41baa00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41bc072320 0x7f41bc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41b3e00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072de0 0x7f41bc13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 -- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072de0 msgr2=0x7f41bc13eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072de0 0x7f41bc13eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 -- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41bc072320 msgr2=0x7f41bc0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41bc072320 0x7f41bc0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 -- 172.21.15.28:0/156421857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc13fa90 con 0x7f41bc15e1f0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.941+0000 7f41bb400640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c780 tx=0x7f41b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.942+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4003da0 con 0x7f41bc15e1f0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.942+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b400ce30 con 0x7f41bc15e1f0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.942+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4005290 con 0x7f41bc15e1f0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.942+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc13fd70 con 0x7f41bc15e1f0 2024-09-10T10:53:02.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.942+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc0b3a50 con 0x7f41bc15e1f0 2024-09-10T10:53:02.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.943+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc072700 con 0x7f41bc15e1f0 2024-09-10T10:53:02.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.944+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41b4005430 con 0x7f41bc15e1f0 2024-09-10T10:53:02.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.944+0000 7f41b2a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 0x7f419806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:02.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.945+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f41b408b5f0 con 0x7f41bc15e1f0 2024-09-10T10:53:02.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.945+0000 7f41baa00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 0x7f419806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:02.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.946+0000 7f41baa00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 0x7f419806e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41ac034000 tx=0x7f41ac02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:02.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:02.949+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41b4058f50 con 0x7f41bc15e1f0 2024-09-10T10:53:03.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.208+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f41bc15c600 con 0x7f41bc15e1f0 2024-09-10T10:53:03.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.761+0000 7f41b2a00640 1 -- 172.21.15.28:0/156421857 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v208) ==== 101+0+0 (secure 0 0 0) 0x7f41b405d250 con 0x7f41bc15e1f0 2024-09-10T10:53:03.762 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 msgr2=0x7f419806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 0x7f419806e2f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41ac034000 tx=0x7f41ac02e000 comp rx=0 tx=0).stop 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 msgr2=0x7f41bc13ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c780 tx=0x7f41b400cc40 comp rx=0 tx=0).stop 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 shutdown_connections 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f419806be30 0x7f419806e2f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41bc15e1f0 0x7f41bc13ce80 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41bc072de0 0x7f41bc13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 --2- 172.21.15.28:0/156421857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41bc072320 0x7f41bc0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 >> 172.21.15.28:0/156421857 conn(0x7f41bc067d70 msgr2=0x7f41bc0b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:03.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 shutdown_connections 2024-09-10T10:53:03.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.764+0000 7f41c1a00640 1 -- 172.21.15.28:0/156421857 wait complete. 2024-09-10T10:53:03.782 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:03.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.858+0000 7f7ac3161f40 1 Processor -- start 2024-09-10T10:53:03.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.858+0000 7f7ac3161f40 1 -- start start 2024-09-10T10:53:03.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa734f470 0x55ffa741b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74253e0 0x55ffa739b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ffa72f7b30 con 0x55ffa74289c0 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffa72fbb70 con 0x55ffa74253e0 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.859+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ffa7427ee0 con 0x55ffa734f470 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58708/0 (socket says 172.21.15.28:58708) 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1494287233 learned_addr learned my addr 172.21.15.28:0/1494287233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa734f470 0x55ffa741b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74253e0 0x55ffa739b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa734f470 msgr2=0x55ffa741b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa734f470 0x55ffa741b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74253e0 msgr2=0x55ffa739b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74253e0 0x55ffa739b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1494287233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa741c200 con 0x55ffa74289c0 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.860+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8057040 tx=0x7f7ab804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e669b84f33e37251 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7abe600640 1 -- 172.21.15.28:0/1494287233 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab804d7a0 con 0x55ffa74289c0 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7abe600640 1 -- 172.21.15.28:0/1494287233 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab8065070 con 0x55ffa74289c0 2024-09-10T10:53:03.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7abe600640 1 -- 172.21.15.28:0/1494287233 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab804dba0 con 0x55ffa74289c0 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1494287233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 msgr2=0x55ffa739bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8057040 tx=0x7f7ab804f000 comp rx=0 tx=0).stop 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1494287233 shutdown_connections 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa74289c0 0x55ffa739bf00 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74253e0 0x55ffa739b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1494287233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa734f470 0x55ffa741b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1494287233 >> 172.21.15.28:0/1494287233 conn(0x55ffa739e0a0 msgr2=0x55ffa739e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1494287233 shutdown_connections 2024-09-10T10:53:03.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.861+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1494287233 wait complete. 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.862+0000 7f7ac3161f40 1 Processor -- start 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.862+0000 7f7ac3161f40 1 -- start start 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa74253e0 0x55ffa7397050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74289c0 0x55ffa7397590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ffa741c4e0 con 0x55ffa734f470 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffa7382d40 con 0x55ffa74289c0 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac3161f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ffa7380f40 con 0x55ffa74253e0 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac0400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac0400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58716/0 (socket says 172.21.15.28:58716) 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.863+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 learned_addr learned my addr 172.21.15.28:0/1356520417 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa74253e0 0x55ffa7397050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa74253e0 msgr2=0x55ffa7397050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74289c0 0x55ffa7397590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa74253e0 0x55ffa7397050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74289c0 msgr2=0x55ffa7397590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74289c0 0x55ffa7397590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa741a580 con 0x55ffa734f470 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ffa74253e0 0x55ffa7397050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa74289c0 0x55ffa7397590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c2d0 tx=0x7f7ab000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7abd200640 1 -- 172.21.15.28:0/1356520417 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0010400 con 0x55ffa734f470 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7abd200640 1 -- 172.21.15.28:0/1356520417 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab0015020 con 0x55ffa734f470 2024-09-10T10:53:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ffa741abc0 con 0x55ffa734f470 2024-09-10T10:53:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7abd200640 1 -- 172.21.15.28:0/1356520417 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00107c0 con 0x55ffa734f470 2024-09-10T10:53:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.864+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa7426b10 con 0x55ffa734f470 2024-09-10T10:53:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.866+0000 7f7abd200640 1 -- 172.21.15.28:0/1356520417 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ab0010960 con 0x55ffa734f470 2024-09-10T10:53:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.867+0000 7f7abd200640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 0x7f7aa006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.867+0000 7f7abd200640 1 -- 172.21.15.28:0/1356520417 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f7ab008acb0 con 0x55ffa734f470 2024-09-10T10:53:03.870 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:03.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.867+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 0x7f7aa006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.867+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 0x55ffa74c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.867+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa7351c00 con 0x55ffa73823d0 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.868+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 0x55ffa74c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.868+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 0x7f7aa006cd00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55ffa739c770 tx=0x7f7aac001790 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.869+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 0x55ffa74c9710 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.870+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v121'629 uv603 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.871+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa7397fe0 con 0x55ffa73823d0 2024-09-10T10:53:03.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.872+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v121'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.872+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74c9c50 con 0x55ffa73823d0 2024-09-10T10:53:03.875 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:03.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.875+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v121'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.875+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb000 con 0x55ffa73823d0 2024-09-10T10:53:03.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.876+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v121'632 uv632 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.876+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb3a0 con 0x55ffa73823d0 2024-09-10T10:53:03.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.878+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v121'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.878+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74c9ff0 con 0x55ffa73823d0 2024-09-10T10:53:03.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.879+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v121'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.880+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca400 con 0x55ffa73823d0 2024-09-10T10:53:03.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.881+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v121'635 uv609 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.881+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.883+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v121'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.883+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.884+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v121'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.886 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:03.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.885+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.886+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v121'638 uv638 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.886+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.888+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v121'639 uv639 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.888+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.890+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v121'640 uv640 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.890+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.891+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v121'641 uv616 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.891+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.893+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v121'642 uv642 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.893+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.896 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:03.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.895+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v121'643 uv643 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.895+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.897+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v121'644 uv644 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.897+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.898+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v121'645 uv645 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.898+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ca7a0 con 0x55ffa73823d0 2024-09-10T10:53:03.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.900+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v121'646 uv646 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.900+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.901+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v121'647 uv647 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.901+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.903+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v121'648 uv622 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.903+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.905+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v121'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.905+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.907 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:03.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.907+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v121'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.907+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cc750 con 0x55ffa73823d0 2024-09-10T10:53:03.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.908+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v121'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.908+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cc750 con 0x55ffa73823d0 2024-09-10T10:53:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.910+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v121'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.910+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.912+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v121'653 uv653 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.912+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.913+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v121'654 uv628 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.913+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.915+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v121'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.915+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.917 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.917+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v121'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.917+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cc750 con 0x55ffa73823d0 2024-09-10T10:53:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.918+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v121'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.918+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cc750 con 0x55ffa73823d0 2024-09-10T10:53:03.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.919+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v121'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.919+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cb740 con 0x55ffa73823d0 2024-09-10T10:53:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.921+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v121'659 uv659 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa73823d0 2024-09-10T10:53:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.921+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ffa74cbc50 0x55ffa74ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.921+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ce580 con 0x55ffa74cbc50 2024-09-10T10:53:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.921+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ffa74cbc50 0x55ffa74ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.922+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ffa74cbc50 0x55ffa74ce040 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.924+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v121'463 uv446 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.924+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74ce920 con 0x55ffa74cbc50 2024-09-10T10:53:03.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.925+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v121'464 uv464 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.925+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cecc0 con 0x55ffa74cbc50 2024-09-10T10:53:03.927 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.927+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v121'465 uv465 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.927+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa73968e0 con 0x55ffa74cbc50 2024-09-10T10:53:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.928+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v121'466 uv466 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.928+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cf560 con 0x55ffa74cbc50 2024-09-10T10:53:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.930+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v121'467 uv467 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.930+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cf900 con 0x55ffa74cbc50 2024-09-10T10:53:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.931+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v121'468 uv451 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.931+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74cfcf0 con 0x55ffa74cbc50 2024-09-10T10:53:03.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.932+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v121'469 uv469 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.932+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d00e0 con 0x55ffa74cbc50 2024-09-10T10:53:03.935 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.934+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v121'470 uv470 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.934+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d00e0 con 0x55ffa74cbc50 2024-09-10T10:53:03.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.936+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v121'471 uv471 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.936+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.938+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v121'472 uv472 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.938+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.940+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v121'473 uv456 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.940+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.941+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v121'474 uv474 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.941+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:03.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.942+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v121'475 uv475 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.943+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d4780 con 0x55ffa74cbc50 2024-09-10T10:53:03.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.944+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v121'476 uv476 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.944+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d4780 con 0x55ffa74cbc50 2024-09-10T10:53:03.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.946+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v121'477 uv477 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.946+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.947+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v121'478 uv462 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.947+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.948+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v121'479 uv479 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.948+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.950 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:03.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.950+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v121'480 uv480 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.950+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d10a0 con 0x55ffa74cbc50 2024-09-10T10:53:03.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.951+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v121'481 uv481 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.951+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d10a0 con 0x55ffa74cbc50 2024-09-10T10:53:03.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.952+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v121'482 uv482 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.952+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d0090 con 0x55ffa74cbc50 2024-09-10T10:53:03.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.954+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v121'483 uv483 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74cbc50 2024-09-10T10:53:03.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.954+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ffa74d0960 0x55ffa74d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.954+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d3470 con 0x55ffa74d0960 2024-09-10T10:53:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.954+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ffa74d0960 0x55ffa74d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.954+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ffa74d0960 0x55ffa74d0d60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.956+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v121'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.956+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d3810 con 0x55ffa74d0960 2024-09-10T10:53:03.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.957+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v121'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.957+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d3bb0 con 0x55ffa74d0960 2024-09-10T10:53:03.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.958+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v121'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.959 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:03.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.959+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d5460 con 0x55ffa74d0960 2024-09-10T10:53:03.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.960+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.960+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d5800 con 0x55ffa74d0960 2024-09-10T10:53:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.961+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.961+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d4450 con 0x55ffa74d0960 2024-09-10T10:53:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.963+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.963+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ffa74d4d50 0x55ffa74d7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.963+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d80f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.963+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ffa74d4d50 0x55ffa74d7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.964+0000 7f7ac0e00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ffa74d4d50 0x55ffa74d7bb0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.965+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v121'91 uv90 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.965+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d80f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.966+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v121'92 uv92 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.966+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d8110 con 0x55ffa74d4d50 2024-09-10T10:53:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.967+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v121'93 uv93 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.968 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.967+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d8550 con 0x55ffa74d4d50 2024-09-10T10:53:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.969+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v121'94 uv94 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.969+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d8550 con 0x55ffa74d0960 2024-09-10T10:53:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.970+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v121'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.970+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.971+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v121'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.971+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.973 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.972+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v121'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.972+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d9900 con 0x55ffa74d0960 2024-09-10T10:53:03.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.973+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.973+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d9900 con 0x55ffa74d0960 2024-09-10T10:53:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.975+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.975+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.976+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.976+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.977+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v121'227 uv221 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.977+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.979+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v121'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.979+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.981 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:03.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.980+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v121'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.980+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d9900 con 0x55ffa74d4d50 2024-09-10T10:53:03.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.981+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v121'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.981+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d9900 con 0x55ffa74d4d50 2024-09-10T10:53:03.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.983+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v121'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.983+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.984+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v121'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.984+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.985+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v121'233 uv226 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.985+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.986+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v121'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.986+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.989 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.988+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v121'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.988+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.989+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v121'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.989+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d4d50 2024-09-10T10:53:03.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.990+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v121'237 uv237 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:03.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.990+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.991+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v121'102 uv101 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.991+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.993+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v121'103 uv103 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.993+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.995 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:03.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.994+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v121'104 uv104 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.994+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d8930 con 0x55ffa74d0960 2024-09-10T10:53:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.995+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v121'105 uv105 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.995+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74d88f0 con 0x55ffa74d0960 2024-09-10T10:53:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.997+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v121'106 uv106 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74d0960 2024-09-10T10:53:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.997+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ffa74d9280 0x55ffa74db670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.997+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbbb0 con 0x55ffa74d9280 2024-09-10T10:53:03.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.997+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ffa74d9280 0x55ffa74db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:03.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.997+0000 7f7ac0400640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ffa74d9280 0x55ffa74db670 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.999+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v121'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:03.999+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbbb0 con 0x55ffa74d9280 2024-09-10T10:53:04.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.000+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v121'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.000+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.002 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:04.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.001+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v121'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.001+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dcfe0 con 0x55ffa74d9280 2024-09-10T10:53:04.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.002+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.002+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dcfe0 con 0x55ffa74d9280 2024-09-10T10:53:04.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.003+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.004+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.005+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.005+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.006+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v121'203 uv202 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.006+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.007+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v121'204 uv204 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.007+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.009 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:04.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.009+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v121'205 uv205 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.009+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.010+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v121'206 uv206 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.010+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.011+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v121'207 uv207 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.011+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.012+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v121'208 uv201 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.012+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.014+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v121'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.014+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.015 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.015+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v121'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.015+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dbfd0 con 0x55ffa74d9280 2024-09-10T10:53:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.016+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v121'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.016+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dede0 con 0x55ffa74d9280 2024-09-10T10:53:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.018+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v121'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.018+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.020+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v121'91 uv90 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.020+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.022+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v121'92 uv92 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.022+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.023+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v121'93 uv93 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.024 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:04.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.023+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.024+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v121'94 uv94 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.024+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.025+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v121'123 uv122 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.025+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.027+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v121'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.027+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.028 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:04.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.028+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v121'125 uv125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.028+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd490 con 0x55ffa74d4d50 2024-09-10T10:53:04.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.029+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v121'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.029+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd490 con 0x55ffa74d4d50 2024-09-10T10:53:04.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.030+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v121'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.030+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dc480 con 0x55ffa74d4d50 2024-09-10T10:53:04.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.031+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v121'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7ab804d920 con 0x55ffa74d4d50 2024-09-10T10:53:04.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.031+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ffa74dc990 0x55ffa74dcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.031+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74dc990 2024-09-10T10:53:04.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.031+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ffa74dc990 0x55ffa74dcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.032+0000 7f7abfa00640 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ffa74dc990 0x55ffa74dcd70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.033+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v121'104 uv103 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74dc990 2024-09-10T10:53:04.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.033+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74dc990 2024-09-10T10:53:04.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.034+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v121'105 uv105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74dc990 2024-09-10T10:53:04.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.034+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74dc990 2024-09-10T10:53:04.036 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:04.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.035+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v121'106 uv106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74dc990 2024-09-10T10:53:04.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.035+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74dc990 2024-09-10T10:53:04.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.037+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v121'107 uv107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74dc990 2024-09-10T10:53:04.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.037+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74dc990 2024-09-10T10:53:04.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.038+0000 7f7abfa00640 1 -- 172.21.15.28:0/1356520417 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v121'108 uv108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7aac00e040 con 0x55ffa74dc990 2024-09-10T10:53:04.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.038+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.039+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v121'253 uv245 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.039+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.040+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v121'254 uv254 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.040+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.042 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:04.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.041+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v121'255 uv255 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.041+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd8f0 con 0x55ffa74d9280 2024-09-10T10:53:04.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.043+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v121'256 uv256 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.043+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd8f0 con 0x55ffa74d9280 2024-09-10T10:53:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.044+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v121'257 uv257 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.044+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd8a0 con 0x55ffa74d9280 2024-09-10T10:53:04.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.045+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v121'258 uv252 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.045+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd8a0 con 0x55ffa74d9280 2024-09-10T10:53:04.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.047+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v121'259 uv259 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.047+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.049 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:04.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.048+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v121'260 uv260 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.048+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.049+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v121'261 uv261 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.049+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.050+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v121'262 uv262 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.050+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dd2b0 con 0x55ffa74d9280 2024-09-10T10:53:04.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.051+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v121'263 uv263 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.051+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55ffa74dede0 con 0x55ffa74d9280 2024-09-10T10:53:04.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac0400640 1 -- 172.21.15.28:0/1356520417 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v121'264 uv264 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7ab0021070 con 0x55ffa74d9280 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ffa74d0960 msgr2=0x55ffa74d0d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55ffa74d0960 0x55ffa74d0d60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ffa74d9280 msgr2=0x55ffa74db670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ffa74d9280 0x55ffa74db670 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ffa74dc990 msgr2=0x55ffa74dcd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ffa74dc990 0x55ffa74dcd70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ffa74d4d50 msgr2=0x55ffa74d7bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55ffa74d4d50 0x55ffa74d7bb0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ffa74cbc50 msgr2=0x55ffa74ce040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55ffa74cbc50 0x55ffa74ce040 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 reap_dead start 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac0e00640 1 -- 172.21.15.28:0/1356520417 reap_dead start 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 msgr2=0x55ffa74c9710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 0x55ffa74c9710 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 msgr2=0x7f7aa006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 0x7f7aa006cd00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55ffa739c770 tx=0x7f7aac001790 comp rx=0 tx=0).stop 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 msgr2=0x55ffa7381e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c2d0 tx=0x7f7ab000c790 comp rx=0 tx=0).stop 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 shutdown_connections 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55ffa73823d0 0x55ffa74c9710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7aa006a840 0x7f7aa006cd00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 --2- 172.21.15.28:0/1356520417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ffa734f470 0x55ffa7381e90 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 >> 172.21.15.28:0/1356520417 conn(0x55ffa739e0a0 msgr2=0x55ffa739e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 shutdown_connections 2024-09-10T10:53:04.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.053+0000 7f7ac3161f40 1 -- 172.21.15.28:0/1356520417 wait complete. 2024-09-10T10:53:04.059 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 400.00000000 - 2024-09-10T10:53:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.174+0000 7fb39fb68f40 1 Processor -- start 2024-09-10T10:53:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.174+0000 7fb39fb68f40 1 -- start start 2024-09-10T10:53:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1019503e0 0x55f1018c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019539c0 0x55f1019e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f1018c5530 con 0x55f1018c48e0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f101822b30 con 0x55f1019539c0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.175+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f101952ee0 con 0x55f1019503e0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58742/0 (socket says 172.21.15.28:58742) 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 -- 172.21.15.28:0/3838342293 learned_addr learned my addr 172.21.15.28:0/3838342293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb397e00640 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1019503e0 0x55f1018c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39d800640 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019539c0 0x55f1019e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 -- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1019503e0 msgr2=0x55f1018c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1019503e0 0x55f1018c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 -- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019539c0 msgr2=0x55f1019e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:04.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019539c0 0x55f1019e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 -- 172.21.15.28:0/3838342293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f10187a5b0 con 0x55f1018c48e0 2024-09-10T10:53:04.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.176+0000 7fb39ce00640 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098b0 tx=0x7fb390005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2e2665e0fe0f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb396a00640 1 -- 172.21.15.28:0/3838342293 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390008930 con 0x55f1018c48e0 2024-09-10T10:53:04.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb396a00640 1 -- 172.21.15.28:0/3838342293 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb390018070 con 0x55f1018c48e0 2024-09-10T10:53:04.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb396a00640 1 -- 172.21.15.28:0/3838342293 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390008cd0 con 0x55f1018c48e0 2024-09-10T10:53:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 -- 172.21.15.28:0/3838342293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 msgr2=0x55f10187a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb3900098b0 tx=0x7fb390005a00 comp rx=0 tx=0).stop 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 -- 172.21.15.28:0/3838342293 shutdown_connections 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019539c0 0x55f1019e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1019503e0 0x55f1018c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/3838342293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1018c48e0 0x55f10187a070 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 -- 172.21.15.28:0/3838342293 >> 172.21.15.28:0/3838342293 conn(0x55f1018c90a0 msgr2=0x55f1018c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 -- 172.21.15.28:0/3838342293 shutdown_connections 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.177+0000 7fb39fb68f40 1 -- 172.21.15.28:0/3838342293 wait complete. 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.178+0000 7fb39fb68f40 1 Processor -- start 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.178+0000 7fb39fb68f40 1 -- start start 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1018c48e0 0x55f1019458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019503e0 0x55f101946150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 0x55f101946690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f101947590 con 0x55f1019539c0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f101951dc0 con 0x55f1019503e0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39fb68f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f101952010 con 0x55f1018c48e0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 0x55f101946690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 0x55f101946690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58752/0 (socket says 172.21.15.28:58752) 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39d800640 1 -- 172.21.15.28:0/1520097107 learned_addr learned my addr 172.21.15.28:0/1520097107 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb397e00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019503e0 0x55f101946150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.179+0000 7fb39ce00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1018c48e0 0x55f1019458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 -- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1018c48e0 msgr2=0x55f1019458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f1018c48e0 0x55f1019458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 -- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019503e0 msgr2=0x55f101946150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1019503e0 0x55f101946150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 -- 172.21.15.28:0/1520097107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f101945df0 con 0x55f1019539c0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39d800640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 0x55f101946690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb398006590 tx=0x7fb39800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb395600640 1 -- 172.21.15.28:0/1520097107 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980125f0 con 0x55f1019539c0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb395600640 1 -- 172.21.15.28:0/1520097107 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb398018070 con 0x55f1019539c0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb395600640 1 -- 172.21.15.28:0/1520097107 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980129d0 con 0x55f1019539c0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f1018c28e0 con 0x55f1019539c0 2024-09-10T10:53:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.180+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1018c1eb0 con 0x55f1019539c0 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.182+0000 7fb395600640 1 -- 172.21.15.28:0/1520097107 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb398012b70 con 0x55f1019539c0 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.183+0000 7fb395600640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb37c06a810 0x7fb37c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.183+0000 7fb395600640 1 -- 172.21.15.28:0/1520097107 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7fb39808a3d0 con 0x55f1019539c0 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.183+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f10194f430 0x55f1019564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.183+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121) -- 0x55f10187caa0 con 0x55f10194f430 2024-09-10T10:53:04.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.183+0000 7fb39ce00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb37c06a810 0x7fb37c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.184+0000 7fb397e00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f10194f430 0x55f1019564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.184+0000 7fb39ce00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb37c06a810 0x7fb37c06ccd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb390000c00 tx=0x7fb390012000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.184+0000 7fb397e00640 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f10194f430 0x55f1019564e0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb397e00640 1 -- 172.21.15.28:0/1520097107 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [writefull 0~80] v121'109 uv109 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb388057030 con 0x55f10194f430 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f10194f430 msgr2=0x55f1019564e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f10194f430 0x55f1019564e0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb37c06a810 msgr2=0x7fb37c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb37c06a810 0x7fb37c06ccd0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb390000c00 tx=0x7fb390012000 comp rx=0 tx=0).stop 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 msgr2=0x55f101946690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 --2- 172.21.15.28:0/1520097107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f1019539c0 0x55f101946690 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb398006590 tx=0x7fb39800d820 comp rx=0 tx=0).stop 2024-09-10T10:53:04.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39d800640 1 -- 172.21.15.28:0/1520097107 reap_dead start 2024-09-10T10:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 shutdown_connections 2024-09-10T10:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 >> 172.21.15.28:0/1520097107 conn(0x55f1018c90a0 msgr2=0x55f1018c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 shutdown_connections 2024-09-10T10:53:04.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.186+0000 7fb39fb68f40 1 -- 172.21.15.28:0/1520097107 wait complete. 2024-09-10T10:53:04.193 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:04.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.391+0000 7f6ae8600640 1 Processor -- start 2024-09-10T10:53:04.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.391+0000 7f6ae8600640 1 -- start start 2024-09-10T10:53:04.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.392+0000 7f6ae8600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ae0156740 0x7f6ae00b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.392+0000 7f6ae8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 0x7f6ae00cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.392+0000 7f6ae8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae00b30e0 con 0x7f6ae00cfcd0 2024-09-10T10:53:04.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.392+0000 7f6ae8600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae006edc0 con 0x7f6ae0156740 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.392+0000 7f6ae8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae00b33c0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ae0156740 0x7f6ae00b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 0x7f6ae00cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae6600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6ae00cfcd0 0x7f6ae015fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:55436/0 (socket says 172.21.15.28:55436) 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae6600640 1 -- 172.21.15.28:0/1216229079 learned_addr learned my addr 172.21.15.28:0/1216229079 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5c00640 1 -- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 msgr2=0x7f6ae00cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5c00640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 0x7f6ae00cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5c00640 1 -- 172.21.15.28:0/1216229079 >> v1:172.21.15.138:6789/0 conn(0x7f6ae00cfcd0 legacy=0x7f6ae015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5c00640 1 -- 172.21.15.28:0/1216229079 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae00b36a0 con 0x7f6ae0156740 2024-09-10T10:53:04.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6adfe00640 1 -- 172.21.15.28:0/1216229079 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722711637 0 0) 0x7f6ae00b30e0 con 0x7f6ae00cfcd0 2024-09-10T10:53:04.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.393+0000 7f6ae5200640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 0x7f6ae00cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.394+0000 7f6ae5c00640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae0156740 0x7f6ae00b45b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40097a0 tx=0x7f6ad402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d73eba7751986d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.394+0000 7f6adfe00640 1 -- 172.21.15.28:0/1216229079 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4037070 con 0x7f6ae0156740 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.394+0000 7f6adfe00640 1 -- 172.21.15.28:0/1216229079 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4032440 con 0x7f6ae0156740 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.394+0000 7f6adfe00640 1 -- 172.21.15.28:0/1216229079 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4032810 con 0x7f6ae0156740 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 -- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae0156740 msgr2=0x7f6ae00b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae0156740 0x7f6ae00b45b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40097a0 tx=0x7f6ad402a180 comp rx=0 tx=0).stop 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 -- 172.21.15.28:0/1216229079 shutdown_connections 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae00b4af0 0x7f6ae00cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 --2- 172.21.15.28:0/1216229079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae0156740 0x7f6ae00b45b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 -- 172.21.15.28:0/1216229079 >> 172.21.15.28:0/1216229079 conn(0x7f6ae006bdf0 msgr2=0x7f6ae006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 -- 172.21.15.28:0/1216229079 shutdown_connections 2024-09-10T10:53:04.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.395+0000 7f6ae8600640 1 -- 172.21.15.28:0/1216229079 wait complete. 2024-09-10T10:53:04.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.397+0000 7f6ae8600640 1 Processor -- start 2024-09-10T10:53:04.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.397+0000 7f6ae8600640 1 -- start start 2024-09-10T10:53:04.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae00cfcd0 0x7f6ae0140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae0156740 0x7f6ae01407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae00d0890 con 0x7f6ae00b4af0 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae00b3c60 con 0x7f6ae00cfcd0 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae8600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae00d0a10 con 0x7f6ae0156740 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58766/0 (socket says 172.21.15.28:58766) 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 -- 172.21.15.28:0/606821950 learned_addr learned my addr 172.21.15.28:0/606821950 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:04.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae6600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae0156740 0x7f6ae01407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5200640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae00cfcd0 0x7f6ae0140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 -- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae0156740 msgr2=0x7f6ae01407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae0156740 0x7f6ae01407b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 -- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae00cfcd0 msgr2=0x7f6ae0140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae00cfcd0 0x7f6ae0140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.398+0000 7f6ae5c00640 1 -- 172.21.15.28:0/606821950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae015faf0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6ae5c00640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098c0 tx=0x7f6ad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4037040 con 0x7f6ae00b4af0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4030500 con 0x7f6ae00b4af0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40308d0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae015fdd0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.399+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae0160390 con 0x7f6ae00b4af0 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.400+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae0156b20 con 0x7f6ae00b4af0 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.401+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ad4044020 con 0x7f6ae00b4af0 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.402+0000 7f6add600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 0x7f6ac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.402+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8110+0+0 (secure 0 0 0) 0x7f6ad40ab960 con 0x7f6ae00b4af0 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.402+0000 7f6ae5200640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 0x7f6ac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.403+0000 7f6ae5200640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 0x7f6ac006e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009560 tx=0x7f6acc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.406+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ad40792c0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.630+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6ae0140cf0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.764+0000 7f6add600640 1 -- 172.21.15.28:0/606821950 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v209) ==== 162+0+0 (secure 0 0 0) 0x7f6ad407d5c0 con 0x7f6ae00b4af0 2024-09-10T10:53:04.765 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 msgr2=0x7f6ac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 0x7f6ac006e2f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009560 tx=0x7f6acc009190 comp rx=0 tx=0).stop 2024-09-10T10:53:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 msgr2=0x7f6ae015dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40098c0 tx=0x7f6ad40046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 shutdown_connections 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ac006be30 0x7f6ac006e2f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ae0156740 0x7f6ae01407b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae00cfcd0 0x7f6ae0140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 --2- 172.21.15.28:0/606821950 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ae00b4af0 0x7f6ae015dbd0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 >> 172.21.15.28:0/606821950 conn(0x7f6ae006bdf0 msgr2=0x7f6ae015aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 shutdown_connections 2024-09-10T10:53:04.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.767+0000 7f6ae8600640 1 -- 172.21.15.28:0/606821950 wait complete. 2024-09-10T10:53:04.775 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:04.774+0000 7fbf42200640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T10:53:04.775 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:04.775+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:04.775 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:04.775+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:04.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:04.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.940+0000 7f5511c00640 1 Processor -- start 2024-09-10T10:53:04.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.940+0000 7f5511c00640 1 -- start start 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.941+0000 7f5511c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f550c072320 0x7f550c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.941+0000 7f5511c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f550c15e1f0 0x7f550c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.941+0000 7f5511c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f550c146140 con 0x7f550c072de0 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.941+0000 7f5511c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f550c0cd740 con 0x7f550c15e1f0 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.941+0000 7f5511c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f550c0cda20 con 0x7f550c072320 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550a000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f550c072de0 0x7f550c06e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47186/0 (socket says 172.21.15.28:47186) 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550a000640 1 -- 172.21.15.28:0/1474925713 learned_addr learned my addr 172.21.15.28:0/1474925713 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550b400640 1 --2- 172.21.15.28:0/1474925713 >> v2:172.21.15.28:3300/0 conn(0x7f550c15e1f0 0x7f550c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550aa00640 1 --2- 172.21.15.28:0/1474925713 >> v2:172.21.15.138:3300/0 conn(0x7f550c072320 0x7f550c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550b400640 1 -- 172.21.15.28:0/1474925713 >> v2:172.21.15.138:3300/0 conn(0x7f550c072320 msgr2=0x7f550c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:04.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550b400640 1 --2- 172.21.15.28:0/1474925713 >> v2:172.21.15.138:3300/0 conn(0x7f550c072320 0x7f550c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550b400640 1 -- 172.21.15.28:0/1474925713 >> v1:172.21.15.28:6789/0 conn(0x7f550c072de0 legacy=0x7f550c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f550b400640 1 -- 172.21.15.28:0/1474925713 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c0cdd00 con 0x7f550c15e1f0 2024-09-10T10:53:04.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.942+0000 7f5509600640 1 -- 172.21.15.28:0/1474925713 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1274231506 0 0) 0x7f550c146140 con 0x7f550c072de0 2024-09-10T10:53:04.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.943+0000 7f550b400640 1 --2- 172.21.15.28:0/1474925713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c15e1f0 0x7f550c06e7c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f550c0b31e0 tx=0x7f550402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7f82c950ccc1ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5509600640 1 -- 172.21.15.28:0/1474925713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504007740 con 0x7f550c15e1f0 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5509600640 1 -- 172.21.15.28:0/1474925713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55040078e0 con 0x7f550c15e1f0 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5509600640 1 -- 172.21.15.28:0/1474925713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5504007cb0 con 0x7f550c15e1f0 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 -- 172.21.15.28:0/1474925713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c15e1f0 msgr2=0x7f550c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 --2- 172.21.15.28:0/1474925713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c15e1f0 0x7f550c06e7c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f550c0b31e0 tx=0x7f550402a5f0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 -- 172.21.15.28:0/1474925713 shutdown_connections 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 --2- 172.21.15.28:0/1474925713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c15e1f0 0x7f550c06e7c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 --2- 172.21.15.28:0/1474925713 >> v2:172.21.15.138:3300/0 conn(0x7f550c072320 0x7f550c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 -- 172.21.15.28:0/1474925713 >> 172.21.15.28:0/1474925713 conn(0x7f550c067d70 msgr2=0x7f550c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 -- 172.21.15.28:0/1474925713 shutdown_connections 2024-09-10T10:53:04.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.944+0000 7f5511c00640 1 -- 172.21.15.28:0/1474925713 wait complete. 2024-09-10T10:53:04.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.946+0000 7f5511c00640 1 Processor -- start 2024-09-10T10:53:04.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.946+0000 7f5511c00640 1 -- start start 2024-09-10T10:53:04.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f550c072de0 0x7f550c0aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 0x7f550c0ab440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f550c0cd520 con 0x7f550c072320 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f550c0cd3a0 con 0x7f550c15e1f0 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f5511c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f550c06ed00 con 0x7f550c072de0 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f550aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f550aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58772/0 (socket says 172.21.15.28:58772) 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f550aa00640 1 -- 172.21.15.28:0/2782852888 learned_addr learned my addr 172.21.15.28:0/2782852888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f550a000640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f550c072de0 0x7f550c0aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.947+0000 7f550b400640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 0x7f550c0ab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 -- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f550c072de0 msgr2=0x7f550c0aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f550c072de0 0x7f550c0aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 -- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 msgr2=0x7f550c0ab440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:04.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 0x7f550c0ab440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 -- 172.21.15.28:0/2782852888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550c0b3770 con 0x7f550c072320 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550b400640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 0x7f550c0ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f550aa00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e9b0 tx=0x7f54fc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc017070 con 0x7f550c072320 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54fc00c9a0 con 0x7f550c072320 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00cd90 con 0x7f550c072320 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f550c0b3a50 con 0x7f550c072320 2024-09-10T10:53:04.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.948+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f550c0b4010 con 0x7f550c072320 2024-09-10T10:53:04.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.949+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f550c072700 con 0x7f550c072320 2024-09-10T10:53:04.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.950+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f54fc01a070 con 0x7f550c072320 2024-09-10T10:53:04.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.951+0000 7f5502a00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 0x7f54ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:04.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.951+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7f54fc08bc20 con 0x7f550c072320 2024-09-10T10:53:04.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.952+0000 7f550a000640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 0x7f54ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:04.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.953+0000 7f550a000640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 0x7f54ec06e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f550c0cbe70 tx=0x7f54f8007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:04.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:04.956+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54fc059510 con 0x7f550c072320 2024-09-10T10:53:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.179+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f550c0ccaa0 con 0x7f550c072320 2024-09-10T10:53:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.179+0000 7f5502a00640 1 -- 172.21.15.28:0/2782852888 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) ==== 95+0+3566 (secure 0 0 0) 0x7f54fc05d810 con 0x7f550c072320 2024-09-10T10:53:05.181 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:05.181 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":211,"btime":"2024-09-10T10:53:04:775788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:04.775734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 211 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 msgr2=0x7f54ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 0x7f54ec06e2f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f550c0cbe70 tx=0x7f54f8007d50 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 msgr2=0x7f550c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00e9b0 tx=0x7f54fc00ee70 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 shutdown_connections 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f54ec06be30 0x7f54ec06e2f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f550c15e1f0 0x7f550c0ab440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f550c072de0 0x7f550c0aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 --2- 172.21.15.28:0/2782852888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f550c072320 0x7f550c0caea0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.181+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 >> 172.21.15.28:0/2782852888 conn(0x7f550c067d70 msgr2=0x7f550c0c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.182+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 shutdown_connections 2024-09-10T10:53:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.182+0000 7f5511c00640 1 -- 172.21.15.28:0/2782852888 wait complete. 2024-09-10T10:53:05.199 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:05.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:05.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.354+0000 7efe34c00640 1 Processor -- start 2024-09-10T10:53:05.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.354+0000 7efe34c00640 1 -- start start 2024-09-10T10:53:05.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.355+0000 7efe34c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.355+0000 7efe34c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe30146140 con 0x7efe30072700 2024-09-10T10:53:05.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.355+0000 7efe34c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe300cd740 con 0x7efe30072320 2024-09-10T10:53:05.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.355+0000 7efe34c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe300cda20 con 0x7efe30072de0 2024-09-10T10:53:05.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2e800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efe30072700 0x7efe3015d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47194/0 (socket says 172.21.15.28:47194) 2024-09-10T10:53:05.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2e800640 1 -- 172.21.15.28:0/3842809261 learned_addr learned my addr 172.21.15.28:0/3842809261 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2d400640 1 --2- 172.21.15.28:0/3842809261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2308973978 0 0) 0x7efe30146140 con 0x7efe30072700 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe14003520 con 0x7efe30072700 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2d400640 1 -- 172.21.15.28:0/3842809261 >> v1:172.21.15.138:6789/0 conn(0x7efe30072320 legacy=0x7efe300cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3700879445 0 0) 0x7efe14003520 con 0x7efe30072700 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2d400640 1 -- 172.21.15.28:0/3842809261 >> v1:172.21.15.28:6789/0 conn(0x7efe30072700 legacy=0x7efe3015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.356+0000 7efe2d400640 1 -- 172.21.15.28:0/3842809261 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe300cdd00 con 0x7efe30072de0 2024-09-10T10:53:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe2d400640 1 --2- 172.21.15.28:0/3842809261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015d1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009750 tx=0x7efe1c029fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de453a222cc5e9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c037070 con 0x7efe30072de0 2024-09-10T10:53:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c02aa40 con 0x7efe30072de0 2024-09-10T10:53:05.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe2ca00640 1 -- 172.21.15.28:0/3842809261 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe280019f0 con 0x7efe30072de0 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe34c00640 1 -- 172.21.15.28:0/3842809261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 msgr2=0x7efe3015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.358+0000 7efe34c00640 1 --2- 172.21.15.28:0/3842809261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015d1e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009750 tx=0x7efe1c029fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.359+0000 7efe34c00640 1 -- 172.21.15.28:0/3842809261 shutdown_connections 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.359+0000 7efe34c00640 1 --2- 172.21.15.28:0/3842809261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015d1e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.359+0000 7efe34c00640 1 -- 172.21.15.28:0/3842809261 >> 172.21.15.28:0/3842809261 conn(0x7efe30067d70 msgr2=0x7efe3006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:05.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.359+0000 7efe34c00640 1 -- 172.21.15.28:0/3842809261 shutdown_connections 2024-09-10T10:53:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.359+0000 7efe34c00640 1 -- 172.21.15.28:0/3842809261 wait complete. 2024-09-10T10:53:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.360+0000 7efe34c00640 1 Processor -- start 2024-09-10T10:53:05.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.360+0000 7efe34c00640 1 -- start start 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe30072320 0x7efe301e90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015dfb0 con 0x7efe30072700 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015de30 con 0x7efe30072320 2024-09-10T10:53:05.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.361+0000 7efe34c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe3015e130 con 0x7efe30072de0 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58782/0 (socket says 172.21.15.28:58782) 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 -- 172.21.15.28:0/3655612849 learned_addr learned my addr 172.21.15.28:0/3655612849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2e800640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2de00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe30072320 0x7efe301e90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 -- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 msgr2=0x7efe3015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 -- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe30072320 msgr2=0x7efe301e90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe30072320 0x7efe301e90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.362+0000 7efe2d400640 1 -- 172.21.15.28:0/3655612849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301e4260 con 0x7efe30072700 2024-09-10T10:53:05.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe2d400640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7efe1c035040 tx=0x7efe1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c042020 con 0x7efe30072700 2024-09-10T10:53:05.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe1c038040 con 0x7efe30072700 2024-09-10T10:53:05.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301e8ae0 con 0x7efe30072700 2024-09-10T10:53:05.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c037070 con 0x7efe30072700 2024-09-10T10:53:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.363+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301e3240 con 0x7efe30072700 2024-09-10T10:53:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.364+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300cce60 con 0x7efe30072700 2024-09-10T10:53:05.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.365+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe1c045020 con 0x7efe30072700 2024-09-10T10:53:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.367+0000 7efe26000640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 0x7efe1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.367+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7efe1c0ab820 con 0x7efe30072700 2024-09-10T10:53:05.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.367+0000 7efe2de00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 0x7efe1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.368+0000 7efe2de00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 0x7efe1006e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efe301e46a0 tx=0x7efe20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.371+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe1c04c070 con 0x7efe30072700 2024-09-10T10:53:05.499 INFO:tasks.ceph.mds.a.smithi028.stderr: -38> 2024-09-10T10:53:00.744+0000 7f64f2a00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': Decoder at 'virtual void InoTable::decode_state(ceph::buffer::v15_2_0::list::const_iterator&)' v=2 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:53:05.541 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:05.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.601+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe3015c330 con 0x7efe30072700 2024-09-10T10:53:05.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.601+0000 7efe26000640 1 -- 172.21.15.28:0/3655612849 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 211 v211) ==== 95+0+3566 (secure 0 0 0) 0x7efe1c0791a0 con 0x7efe30072700 2024-09-10T10:53:05.603 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:05.603 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":211,"btime":"2024-09-10T10:53:04:775788+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:04.775734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:05.603 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 211 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 msgr2=0x7efe1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 0x7efe1006e2f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efe301e46a0 tx=0x7efe20009040 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 msgr2=0x7efe301e3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7efe1c035040 tx=0x7efe1c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 shutdown_connections 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe1006be30 0x7efe1006e2f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe30072de0 0x7efe3015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe30072700 0x7efe301e3d20 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 --2- 172.21.15.28:0/3655612849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe30072320 0x7efe301e90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 >> 172.21.15.28:0/3655612849 conn(0x7efe30067d70 msgr2=0x7efe300b40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:05.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.603+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 shutdown_connections 2024-09-10T10:53:05.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.604+0000 7efe34c00640 1 -- 172.21.15.28:0/3655612849 wait complete. 2024-09-10T10:53:05.620 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:05.620 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:05.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.777+0000 7fa577800640 1 Processor -- start 2024-09-10T10:53:05.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.777+0000 7fa577800640 1 -- start start 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa577800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 0x7fa570169830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa577800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57006d9f0 con 0x7fa570162060 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa577800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57006dcd0 con 0x7fa570161c80 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa577800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa57006dfb0 con 0x7fa5700b6680 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa56fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa570162060 0x7fa5700b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47210/0 (socket says 172.21.15.28:47210) 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa56fe00640 1 -- 172.21.15.28:0/3571472011 learned_addr learned my addr 172.21.15.28:0/3571472011 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:05.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.779+0000 7fa575800640 1 --2- 172.21.15.28:0/3571472011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 0x7fa570169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.780+0000 7fa575800640 1 -- 172.21.15.28:0/3571472011 >> v1:172.21.15.138:6789/0 conn(0x7fa570161c80 legacy=0x7fa57006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.780+0000 7fa575800640 1 -- 172.21.15.28:0/3571472011 >> v1:172.21.15.28:6789/0 conn(0x7fa570162060 legacy=0x7fa5700b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.780+0000 7fa575800640 1 -- 172.21.15.28:0/3571472011 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5700d01f0 con 0x7fa5700b6680 2024-09-10T10:53:05.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.781+0000 7fa575800640 1 --2- 172.21.15.28:0/3571472011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 0x7fa570169830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa568009dc0 tx=0x7fa56802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30c551f011f834b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.781+0000 7fa56f400640 1 -- 172.21.15.28:0/3571472011 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568007740 con 0x7fa5700b6680 2024-09-10T10:53:05.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.781+0000 7fa56f400640 1 -- 172.21.15.28:0/3571472011 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5680078e0 con 0x7fa5700b6680 2024-09-10T10:53:05.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.781+0000 7fa56f400640 1 -- 172.21.15.28:0/3571472011 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568007cb0 con 0x7fa5700b6680 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 -- 172.21.15.28:0/3571472011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 msgr2=0x7fa570169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 --2- 172.21.15.28:0/3571472011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 0x7fa570169830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa568009dc0 tx=0x7fa56802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 -- 172.21.15.28:0/3571472011 shutdown_connections 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 --2- 172.21.15.28:0/3571472011 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa5700b6680 0x7fa570169830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 -- 172.21.15.28:0/3571472011 >> 172.21.15.28:0/3571472011 conn(0x7fa5700d44f0 msgr2=0x7fa5700d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 -- 172.21.15.28:0/3571472011 shutdown_connections 2024-09-10T10:53:05.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.782+0000 7fa577800640 1 -- 172.21.15.28:0/3571472011 wait complete. 2024-09-10T10:53:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.784+0000 7fa577800640 1 Processor -- start 2024-09-10T10:53:05.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.784+0000 7fa577800640 1 -- start start 2024-09-10T10:53:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.784+0000 7fa577800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5700b6680 0x7fa5700b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa577800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa577800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa570162060 0x7fa5700ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa577800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5700b6cb0 con 0x7fa570161c80 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa577800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5700b6b30 con 0x7fa5700b6680 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa577800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa570169d70 con 0x7fa570162060 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58796/0 (socket says 172.21.15.28:58796) 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 -- 172.21.15.28:0/1202111893 learned_addr learned my addr 172.21.15.28:0/1202111893 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa575800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa570162060 0x7fa5700ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa574e00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5700b6680 0x7fa5700b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 -- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa570162060 msgr2=0x7fa5700ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa570162060 0x7fa5700ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 -- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5700b6680 msgr2=0x7fa5700b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5700b6680 0x7fa5700b85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.785+0000 7fa56fe00640 1 -- 172.21.15.28:0/1202111893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570156e90 con 0x7fa570161c80 2024-09-10T10:53:05.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa56fe00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c500 tx=0x7fa55c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c017070 con 0x7fa570161c80 2024-09-10T10:53:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c005ce0 con 0x7fa570161c80 2024-09-10T10:53:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0072f0 con 0x7fa570161c80 2024-09-10T10:53:05.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5700c98f0 con 0x7fa570161c80 2024-09-10T10:53:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.786+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5700c9eb0 con 0x7fa570161c80 2024-09-10T10:53:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.787+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5700b5f70 con 0x7fa570161c80 2024-09-10T10:53:05.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.788+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa55c01a070 con 0x7fa570161c80 2024-09-10T10:53:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.789+0000 7fa56cc00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 0x7fa55006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.789+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7fa55c08aa80 con 0x7fa570161c80 2024-09-10T10:53:05.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.789+0000 7fa574e00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 0x7fa55006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:05.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.790+0000 7fa574e00640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 0x7fa55006e320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa5701594f0 tx=0x7fa564009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:05.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:05.793+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa55c08ce20 con 0x7fa570161c80 2024-09-10T10:53:06.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.015+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5700ba470 con 0x7fa570161c80 2024-09-10T10:53:06.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.015+0000 7fa56cc00640 1 -- 172.21.15.28:0/1202111893 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 212 v212) ==== 95+0+4392 (secure 0 0 0) 0x7fa55c058400 con 0x7fa570161c80 2024-09-10T10:53:06.016 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:06.017 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":212,"btime":"2024-09-10T10:53:05:777366+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8931,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2836905257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2836905257},{"type":"v1","addr":"172.21.15.28:6835","nonce":2836905257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":212}],"filesystems":[{"mdsmap":{"epoch":211,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:04.775734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:06.017 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 212 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 msgr2=0x7fa55006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 0x7fa55006e320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa5701594f0 tx=0x7fa564009040 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 msgr2=0x7fa570153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00c500 tx=0x7fa55c00c9c0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 shutdown_connections 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa55006be60 0x7fa55006e320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa570162060 0x7fa5700ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa570161c80 0x7fa570153a80 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 --2- 172.21.15.28:0/1202111893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5700b6680 0x7fa5700b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 >> 172.21.15.28:0/1202111893 conn(0x7fa5700d44f0 msgr2=0x7fa5700747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:06.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 shutdown_connections 2024-09-10T10:53:06.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.018+0000 7fa577800640 1 -- 172.21.15.28:0/1202111893 wait complete. 2024-09-10T10:53:06.035 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 400.00000000' led to DAMAGED state 2024-09-10T10:53:06.035 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt 100.00000000.inode' 2024-09-10T10:53:06.035 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:06.035 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:06.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:06.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:06.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:06.071 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:06.145 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:06.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:06.146 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:06.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.342+0000 7f3be6200640 1 Processor -- start 2024-09-10T10:53:06.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.342+0000 7f3be6200640 1 -- start start 2024-09-10T10:53:06.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.343+0000 7f3be6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3be0072320 0x7f3be00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.343+0000 7f3be6200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be0146140 con 0x7f3be0072de0 2024-09-10T10:53:06.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.343+0000 7f3be6200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be006eed0 con 0x7f3be015e1f0 2024-09-10T10:53:06.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.343+0000 7f3be6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3be00cd740 con 0x7f3be0072320 2024-09-10T10:53:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bd7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3be0072de0 0x7f3be006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47212/0 (socket says 172.21.15.28:47212) 2024-09-10T10:53:06.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bd7e00640 1 -- 172.21.15.28:0/4270971721 learned_addr learned my addr 172.21.15.28:0/4270971721 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdf400640 1 --2- 172.21.15.28:0/4270971721 >> v2:172.21.15.138:3300/0 conn(0x7f3be0072320 0x7f3be00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2716086238 0 0) 0x7f3be0146140 con 0x7f3be0072de0 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bc4003520 con 0x7f3be0072de0 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3860787323 0 0) 0x7f3bc4003520 con 0x7f3be0072de0 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 >> v2:172.21.15.138:3300/0 conn(0x7f3be0072320 msgr2=0x7f3be00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 --2- 172.21.15.28:0/4270971721 >> v2:172.21.15.138:3300/0 conn(0x7f3be0072320 0x7f3be00cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 >> v1:172.21.15.138:6789/0 conn(0x7f3be015e1f0 legacy=0x7f3be006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be00cda20 con 0x7f3be0072de0 2024-09-10T10:53:06.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f3bc8001670 con 0x7f3be0072de0 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 >> v1:172.21.15.28:6789/0 conn(0x7f3be0072de0 legacy=0x7f3be006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3bc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4004800 0x7f3bc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.344+0000 7f3bdea00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bc4005210 0x7f3bc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3be00cd740 con 0x7f3bc4003a00 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc400c160 con 0x7f3bc4004800 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc400c360 con 0x7f3bc4005210 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3bc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bd7e00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4004800 0x7f3bc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdfe00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bc4005210 0x7f3bc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 -- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bc4005210 msgr2=0x7f3bc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:06.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3bc4005210 0x7f3bc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 -- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4004800 msgr2=0x7f3bc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4004800 0x7f3bc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdf400640 1 -- 172.21.15.28:0/4270971721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4003520 con 0x7f3bc4003a00 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bd7e00640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bc4004800 0x7f3bc4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.345+0000 7f3bdfe00640 1 -- 172.21.15.28:0/4270971721 reap_dead start 2024-09-10T10:53:06.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.346+0000 7f3bdf400640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3bc4003e20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009a90 tx=0x7f3bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2455ac4457eb4039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.346+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0036070 con 0x7f3bc4003a00 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.346+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd0031400 con 0x7f3bc4003a00 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.346+0000 7f3bdea00640 1 -- 172.21.15.28:0/4270971721 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0009690 con 0x7f3bc4003a00 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 -- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 msgr2=0x7f3bc4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3bc4003e20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0009a90 tx=0x7f3bc4005830 comp rx=0 tx=0).stop 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 -- 172.21.15.28:0/4270971721 shutdown_connections 2024-09-10T10:53:06.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 --2- 172.21.15.28:0/4270971721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3bc4003e20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 -- 172.21.15.28:0/4270971721 >> 172.21.15.28:0/4270971721 conn(0x7f3be0067d70 msgr2=0x7f3be006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 -- 172.21.15.28:0/4270971721 shutdown_connections 2024-09-10T10:53:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.347+0000 7f3be6200640 1 -- 172.21.15.28:0/4270971721 wait complete. 2024-09-10T10:53:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.349+0000 7f3be6200640 1 Processor -- start 2024-09-10T10:53:06.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.349+0000 7f3be6200640 1 -- start start 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3be015bb30 0x7f3be015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 0x7f3be00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0072700 con 0x7f3bc4003a00 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3be015c9e0 con 0x7f3be015bb30 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3be6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3be0072880 con 0x7f3be00b35d0 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58824/0 (socket says 172.21.15.28:58824) 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 -- 172.21.15.28:0/3437112149 learned_addr learned my addr 172.21.15.28:0/3437112149 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bd7e00640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3be015bb30 0x7f3be015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdfe00640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 0x7f3be00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 -- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 msgr2=0x7f3be00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 0x7f3be00b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 -- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3be015bb30 msgr2=0x7f3be015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3be015bb30 0x7f3be015b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdf400640 1 -- 172.21.15.28:0/3437112149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be01417f0 con 0x7f3bc4003a00 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.350+0000 7f3bdfe00640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 0x7f3be00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3bdf400640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f3bd002ab50 tx=0x7f3bd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd003f020 con 0x7f3bc4003a00 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd0037040 con 0x7f3bc4003a00 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0036070 con 0x7f3bc4003a00 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be0141ad0 con 0x7f3bc4003a00 2024-09-10T10:53:06.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.351+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be0142090 con 0x7f3bc4003a00 2024-09-10T10:53:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.352+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be00cce60 con 0x7f3bc4003a00 2024-09-10T10:53:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.353+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bd0042020 con 0x7f3bc4003a00 2024-09-10T10:53:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.354+0000 7f3bd7400640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 0x7f3bbc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.355+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7f3bd00aae50 con 0x7f3bc4003a00 2024-09-10T10:53:06.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.355+0000 7f3bd7e00640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 0x7f3bbc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.356+0000 7f3bd7e00640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 0x7f3bbc06e320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000ff0 tx=0x7f3bc8007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.359+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3bd00787d0 con 0x7f3bc4003a00 2024-09-10T10:53:06.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.581+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3be006eac0 con 0x7f3bc4003a00 2024-09-10T10:53:06.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.784+0000 7f3bd7400640 1 -- 172.21.15.28:0/3437112149 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v213) ==== 161+0+0 (secure 0 0 0) 0x7f3bd007cad0 con 0x7f3bc4003a00 2024-09-10T10:53:06.785 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:06.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 msgr2=0x7f3bbc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 0x7f3bbc06e320 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8000ff0 tx=0x7f3bc8007040 comp rx=0 tx=0).stop 2024-09-10T10:53:06.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 msgr2=0x7f3be01445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f3bd002ab50 tx=0x7f3bd0004730 comp rx=0 tx=0).stop 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 shutdown_connections 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3bbc06be60 0x7f3bbc06e320 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3be00b35d0 0x7f3be00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3be015bb30 0x7f3be015b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 --2- 172.21.15.28:0/3437112149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3bc4003a00 0x7f3be01445b0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 >> 172.21.15.28:0/3437112149 conn(0x7f3be0067d70 msgr2=0x7f3be00c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 shutdown_connections 2024-09-10T10:53:06.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.786+0000 7f3be6200640 1 -- 172.21.15.28:0/3437112149 wait complete. 2024-09-10T10:53:06.804 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:06.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.959+0000 7fbefee00640 1 Processor -- start 2024-09-10T10:53:06.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.959+0000 7fbefee00640 1 -- start start 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.960+0000 7fbefee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef8072320 0x7fbef80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.960+0000 7fbefee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef815e1f0 0x7fbef806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.960+0000 7fbefee00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef8146140 con 0x7fbef8072de0 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.960+0000 7fbefee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbef80cd740 con 0x7fbef8072320 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.960+0000 7fbefee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbef80cda20 con 0x7fbef815e1f0 2024-09-10T10:53:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef8072320 0x7fbef80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbef8072320 0x7fbef80cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58826/0 (socket says 172.21.15.28:58826) 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 -- 172.21.15.28:0/1923636726 learned_addr learned my addr 172.21.15.28:0/1923636726 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbefce00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef815e1f0 0x7fbef806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 -- 172.21.15.28:0/1923636726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef815e1f0 msgr2=0x7fbef806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef815e1f0 0x7fbef806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 -- 172.21.15.28:0/1923636726 >> v1:172.21.15.138:6789/0 conn(0x7fbef8072de0 legacy=0x7fbef806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.961+0000 7fbef7e00640 1 -- 172.21.15.28:0/1923636726 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef80cdd00 con 0x7fbef8072320 2024-09-10T10:53:06.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.962+0000 7fbef7e00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072320 0x7fbef80cce60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeeeabd8b324456 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.962+0000 7fbef7400640 1 -- 172.21.15.28:0/1923636726 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8037070 con 0x7fbef8072320 2024-09-10T10:53:06.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.962+0000 7fbef7400640 1 -- 172.21.15.28:0/1923636726 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee8032440 con 0x7fbef8072320 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.962+0000 7fbef7400640 1 -- 172.21.15.28:0/1923636726 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8032810 con 0x7fbef8072320 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 -- 172.21.15.28:0/1923636726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072320 msgr2=0x7fbef80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072320 0x7fbef80cce60 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009870 tx=0x7fbee802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 -- 172.21.15.28:0/1923636726 shutdown_connections 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef815e1f0 0x7fbef806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 --2- 172.21.15.28:0/1923636726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072320 0x7fbef80cce60 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 -- 172.21.15.28:0/1923636726 >> 172.21.15.28:0/1923636726 conn(0x7fbef8067d70 msgr2=0x7fbef806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 -- 172.21.15.28:0/1923636726 shutdown_connections 2024-09-10T10:53:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.963+0000 7fbefee00640 1 -- 172.21.15.28:0/1923636726 wait complete. 2024-09-10T10:53:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.965+0000 7fbefee00640 1 Processor -- start 2024-09-10T10:53:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.965+0000 7fbefee00640 1 -- start start 2024-09-10T10:53:06.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef8072320 0x7fbef815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef815e1f0 0x7fbef80cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80cd520 con 0x7fbef8072de0 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbef80cd3a0 con 0x7fbef815e1f0 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefee00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbef806ed00 con 0x7fbef8072320 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58832/0 (socket says 172.21.15.28:58832) 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 -- 172.21.15.28:0/1075255319 learned_addr learned my addr 172.21.15.28:0/1075255319 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbef7e00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef8072320 0x7fbef815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbefce00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef815e1f0 0x7fbef80cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 -- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef8072320 msgr2=0x7fbef815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef8072320 0x7fbef815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 -- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef815e1f0 msgr2=0x7fbef80cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef815e1f0 0x7fbef80cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.966+0000 7fbeefe00640 1 -- 172.21.15.28:0/1075255319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef815c390 con 0x7fbef8072de0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbeefe00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c440 tx=0x7fbee000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00052d0 con 0x7fbef8072de0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee0005470 con 0x7fbef8072de0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef8141780 con 0x7fbef8072de0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0012870 con 0x7fbef8072de0 2024-09-10T10:53:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.967+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef8141d40 con 0x7fbef8072de0 2024-09-10T10:53:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.968+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef8072700 con 0x7fbef8072de0 2024-09-10T10:53:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.969+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbee0012a10 con 0x7fbef8072de0 2024-09-10T10:53:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.970+0000 7fbef4c00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 0x7fbed406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.970+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7fbee008b390 con 0x7fbef8072de0 2024-09-10T10:53:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.970+0000 7fbef7e00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 0x7fbed406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.971+0000 7fbef7e00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 0x7fbed406e320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:06.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:06.975+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbee0058c80 con 0x7fbef8072de0 2024-09-10T10:53:07.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.234+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fbef80c5e40 con 0x7fbef8072de0 2024-09-10T10:53:07.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.791+0000 7fbef4c00640 1 -- 172.21.15.28:0/1075255319 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v214) ==== 101+0+0 (secure 0 0 0) 0x7fbee005cf80 con 0x7fbef8072de0 2024-09-10T10:53:07.793 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:07.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 msgr2=0x7fbed406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 0x7fbed406e320 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbee8034000 tx=0x7fbee802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 msgr2=0x7fbef80cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c440 tx=0x7fbee000c900 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 shutdown_connections 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbed406be60 0x7fbed406e320 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbef815e1f0 0x7fbef80cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbef8072de0 0x7fbef80cbe90 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 --2- 172.21.15.28:0/1075255319 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbef8072320 0x7fbef815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.794+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 >> 172.21.15.28:0/1075255319 conn(0x7fbef8067d70 msgr2=0x7fbef80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.795+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 shutdown_connections 2024-09-10T10:53:07.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.795+0000 7fbefee00640 1 -- 172.21.15.28:0/1075255319 wait complete. 2024-09-10T10:53:07.813 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.891+0000 7fa7e3972f40 1 Processor -- start 2024-09-10T10:53:07.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.891+0000 7fa7e3972f40 1 -- start start 2024-09-10T10:53:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619423e0 0x559361940760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 0x559361940ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5593618b7530 con 0x5593618b6ff0 2024-09-10T10:53:07.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559361814b30 con 0x5593619423e0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559361944ee0 con 0x5593619459c0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58846/0 (socket says 172.21.15.28:58846) 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 -- 172.21.15.28:0/3239236352 learned_addr learned my addr 172.21.15.28:0/3239236352 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619423e0 0x559361940760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e1600640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 0x559361940ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 -- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 msgr2=0x559361940ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 0x559361940ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 -- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619423e0 msgr2=0x559361940760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619423e0 0x559361940760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e0c00640 1 -- 172.21.15.28:0/3239236352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936189e1f0 con 0x5593618b6ff0 2024-09-10T10:53:07.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.892+0000 7fa7e1600640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 0x559361940ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.893+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4000c00 tx=0x7fa7d4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94db3d6237cb8a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.893+0000 7fa7daa00640 1 -- 172.21.15.28:0/3239236352 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4063070 con 0x5593618b6ff0 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.893+0000 7fa7daa00640 1 -- 172.21.15.28:0/3239236352 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d40559d0 con 0x5593618b6ff0 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.893+0000 7fa7daa00640 1 -- 172.21.15.28:0/3239236352 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4055da0 con 0x5593618b6ff0 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 -- 172.21.15.28:0/3239236352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 msgr2=0x55936186bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4000c00 tx=0x7fa7d4051000 comp rx=0 tx=0).stop 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 -- 172.21.15.28:0/3239236352 shutdown_connections 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619459c0 0x559361940ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619423e0 0x559361940760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/3239236352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x55936186bd40 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 -- 172.21.15.28:0/3239236352 >> 172.21.15.28:0/3239236352 conn(0x5593618bb0a0 msgr2=0x5593618bb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 -- 172.21.15.28:0/3239236352 shutdown_connections 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 -- 172.21.15.28:0/3239236352 wait complete. 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.894+0000 7fa7e3972f40 1 Processor -- start 2024-09-10T10:53:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.895+0000 7fa7e3972f40 1 -- start start 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.895+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619423e0 0x5593619dec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e3972f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619459c0 0x5593619db120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x559361948c90 con 0x5593618b6ff0 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55936189e5a0 con 0x5593619459c0 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e3972f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x559361943610 con 0x5593619423e0 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58858/0 (socket says 172.21.15.28:58858) 2024-09-10T10:53:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 learned_addr learned my addr 172.21.15.28:0/326434905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619423e0 msgr2=0x5593619dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5593619423e0 0x5593619dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619459c0 msgr2=0x5593619db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593619459c0 0x5593619db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559361943d90 con 0x5593618b6ff0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.896+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4045ef0 tx=0x7fa7d4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.897+0000 7fa7d9600640 1 -- 172.21.15.28:0/326434905 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4076070 con 0x5593618b6ff0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.897+0000 7fa7d9600640 1 -- 172.21.15.28:0/326434905 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d4055e00 con 0x5593618b6ff0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.897+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593619df390 con 0x5593618b6ff0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.897+0000 7fa7d9600640 1 -- 172.21.15.28:0/326434905 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d406d070 con 0x5593618b6ff0 2024-09-10T10:53:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.897+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593618b61f0 con 0x5593618b6ff0 2024-09-10T10:53:07.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.899+0000 7fa7d9600640 1 -- 172.21.15.28:0/326434905 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7d4079070 con 0x5593618b6ff0 2024-09-10T10:53:07.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7d9600640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 0x7fa7c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7d9600640 1 -- 172.21.15.28:0/326434905 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7fa7d40cfcb0 con 0x5593618b6ff0 2024-09-10T10:53:07.902 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:07.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 0x5593619e7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x559361939870 con 0x5593618b57a0 2024-09-10T10:53:07.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 0x7fa7c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.900+0000 7fa7e1600640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 0x5593619e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.901+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 0x7fa7c006ccd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55936189f350 tx=0x7fa7cc006450 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.901+0000 7fa7e1600640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 0x5593619e7870 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.903+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v122'660 uv634 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.903+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x55936192aec0 con 0x5593618b57a0 2024-09-10T10:53:07.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.905+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v122'661 uv661 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.906+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e7db0 con 0x5593618b57a0 2024-09-10T10:53:07.908 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.907+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v122'662 uv662 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.908+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e9160 con 0x5593618b57a0 2024-09-10T10:53:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.909+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v122'663 uv663 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.909+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e9500 con 0x5593618b57a0 2024-09-10T10:53:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.911+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v122'664 uv664 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.911+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8150 con 0x5593618b57a0 2024-09-10T10:53:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.913+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v122'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.913+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8540 con 0x5593618b57a0 2024-09-10T10:53:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.916+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v122'666 uv640 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.916+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8930 con 0x5593618b57a0 2024-09-10T10:53:07.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.918+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v122'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.918+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8930 con 0x5593618b57a0 2024-09-10T10:53:07.920 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:07.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.919+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v122'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.919+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.921+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v122'669 uv669 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.921+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.923+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v122'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.923+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.924+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v122'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.925+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.926+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v122'672 uv647 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.926+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.928+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v122'673 uv673 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.928+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.930+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v122'674 uv674 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.931 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:07.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.930+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.932+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v122'675 uv675 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.932+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.933+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v122'676 uv676 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.933+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e88e0 con 0x5593618b57a0 2024-09-10T10:53:07.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.936+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v122'677 uv677 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.936+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8cc0 con 0x5593618b57a0 2024-09-10T10:53:07.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.937+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v122'678 uv678 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.937+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8cc0 con 0x5593618b57a0 2024-09-10T10:53:07.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.939+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v122'679 uv653 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.939+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8cc0 con 0x5593618b57a0 2024-09-10T10:53:07.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.941+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v122'680 uv680 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.941+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.943 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:07.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.942+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v122'681 uv681 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.942+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.944+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v122'682 uv682 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.944+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.946+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v122'683 uv683 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.946+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.947+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v122'684 uv684 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.948+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.949+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v122'685 uv659 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.949+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.951+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v122'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.951+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.953+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v122'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.954 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:07.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.953+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.955+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v122'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.955+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.956+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v122'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.956+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619e8ba0 con 0x5593618b57a0 2024-09-10T10:53:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.958+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v122'690 uv690 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593618b57a0 2024-09-10T10:53:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.958+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5593619e98a0 0x5593619ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.958+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619ec1d0 con 0x5593619e98a0 2024-09-10T10:53:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.958+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5593619e98a0 0x5593619ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.959+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5593619e98a0 0x5593619ebc90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.961+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v122'484 uv467 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.961+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619ec570 con 0x5593619e98a0 2024-09-10T10:53:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.962+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v122'485 uv485 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.962+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619ec910 con 0x5593619e98a0 2024-09-10T10:53:07.964 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:07.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.964+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v122'486 uv486 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.964+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619eccb0 con 0x5593619e98a0 2024-09-10T10:53:07.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.965+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v122'487 uv487 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.965+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619ed050 con 0x5593619e98a0 2024-09-10T10:53:07.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.967+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v122'488 uv488 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.967+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619ed3f0 con 0x5593619e98a0 2024-09-10T10:53:07.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.969+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v122'489 uv472 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.969+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619ed790 con 0x5593619e98a0 2024-09-10T10:53:07.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.970+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v122'490 uv490 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.970+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.972 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:07.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.972+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v122'491 uv491 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.972+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.973+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v122'492 uv492 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.973+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.975+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v122'493 uv493 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.975+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.977+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v122'494 uv477 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.977+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.978+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v122'495 uv495 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.978+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.981 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.980+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v122'496 uv496 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.980+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f2220 con 0x5593619e98a0 2024-09-10T10:53:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.982+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v122'497 uv497 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.982+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f2220 con 0x5593619e98a0 2024-09-10T10:53:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.983+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v122'498 uv498 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.983+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.985+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v122'499 uv483 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.985+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.986+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v122'500 uv500 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.986+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.989 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.988+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v122'501 uv501 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.988+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619eeb40 con 0x5593619e98a0 2024-09-10T10:53:07.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.989+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v122'502 uv502 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.989+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619eeb40 con 0x5593619e98a0 2024-09-10T10:53:07.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.990+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v122'503 uv503 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.990+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619edb30 con 0x5593619e98a0 2024-09-10T10:53:07.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.992+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v122'504 uv504 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619e98a0 2024-09-10T10:53:07.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.992+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5593619ee040 0x5593619f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:07.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.992+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f0970 con 0x5593619ee040 2024-09-10T10:53:07.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.992+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5593619ee040 0x5593619f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:07.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.993+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5593619ee040 0x5593619f0430 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.994+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v122'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:07.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.994+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f0d10 con 0x5593619ee040 2024-09-10T10:53:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.996+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v122'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:07.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.996+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f1170 con 0x5593619ee040 2024-09-10T10:53:07.997 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.997+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:07.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.997+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619db760 con 0x5593619ee040 2024-09-10T10:53:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.998+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:07.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.998+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f2b00 con 0x5593619ee040 2024-09-10T10:53:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:07.999+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.000+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f1a10 con 0x5593619ee040 2024-09-10T10:53:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.001+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.001+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5593619f24d0 0x5593619f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.001+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f54c0 con 0x5593619f24d0 2024-09-10T10:53:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.001+0000 7fa7e1600640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5593619f24d0 0x5593619f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.001+0000 7fa7e1600640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5593619f24d0 0x5593619f4f20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.003+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v122'95 uv94 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.003+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f58b0 con 0x5593619f24d0 2024-09-10T10:53:08.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.004+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v122'96 uv96 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.004+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f58b0 con 0x5593619f24d0 2024-09-10T10:53:08.006 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.005+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v122'97 uv97 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.005+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f58b0 con 0x5593619f24d0 2024-09-10T10:53:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.006+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v122'98 uv98 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.006+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f5850 con 0x5593619ee040 2024-09-10T10:53:08.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.008+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v122'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.008+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f5cd0 con 0x5593619ee040 2024-09-10T10:53:08.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.009+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v122'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.009+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619ee040 2024-09-10T10:53:08.011 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.010+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.010+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f7140 con 0x5593619ee040 2024-09-10T10:53:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.011+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.011+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f7100 con 0x5593619ee040 2024-09-10T10:53:08.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.012+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.012+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619ee040 2024-09-10T10:53:08.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.013+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.013+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.015+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v122'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.015+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.016+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v122'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.016+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.018 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.018+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v122'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.018+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.019+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v122'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.019+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.021+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v122'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.021+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.022+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v122'243 uv243 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.022+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.024+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v122'244 uv237 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.024+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.025+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v122'245 uv245 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.025+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.027 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:08.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.027+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v122'246 uv246 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.027+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.028+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v122'247 uv247 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.028+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619f24d0 2024-09-10T10:53:08.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.030+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v122'248 uv248 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.030+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619ee040 2024-09-10T10:53:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.031+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v122'107 uv106 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.031+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f60f0 con 0x5593619ee040 2024-09-10T10:53:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.032+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v122'108 uv108 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.032+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f6130 con 0x5593619ee040 2024-09-10T10:53:08.034 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.033+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v122'109 uv109 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.033+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f65b0 con 0x5593619ee040 2024-09-10T10:53:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.034+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v122'110 uv110 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.034+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f65b0 con 0x5593619ee040 2024-09-10T10:53:08.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.035+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v122'111 uv111 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619ee040 2024-09-10T10:53:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.036+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5593619f6ee0 0x5593619f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.036+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9810 con 0x5593619f6ee0 2024-09-10T10:53:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.036+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5593619f6ee0 0x5593619f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.036+0000 7fa7e0c00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5593619f6ee0 0x5593619f92d0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.038+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v122'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.038+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9850 con 0x5593619f6ee0 2024-09-10T10:53:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.039+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v122'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.039+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.041 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:08.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.040+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.040+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fac80 con 0x5593619f6ee0 2024-09-10T10:53:08.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.042+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.042+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fac80 con 0x5593619f6ee0 2024-09-10T10:53:08.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.043+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.043+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.044+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.044+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.046+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v122'213 uv207 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.046+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.047+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v122'214 uv214 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.047+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.049 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.048+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v122'215 uv215 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.048+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.050+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v122'216 uv216 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.050+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.051+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v122'217 uv217 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.051+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.053+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v122'218 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.053+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.054+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v122'219 uv219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.054+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.055 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.055+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v122'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.055+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619f9c70 con 0x5593619f6ee0 2024-09-10T10:53:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.056+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v122'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.056+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fca80 con 0x5593619f6ee0 2024-09-10T10:53:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.058+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v122'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.058+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.059+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v122'95 uv94 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.059+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.061+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v122'96 uv96 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.061+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.063 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:08.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.062+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v122'97 uv97 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.062+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.064+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v122'98 uv98 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.064+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.065+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v122'129 uv128 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.065+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.066+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v122'130 uv130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.066+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.069 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:08.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.067+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v122'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.067+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.068+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v122'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.068+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.070+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v122'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.070+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f24d0 2024-09-10T10:53:08.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.071+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v122'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7dc002820 con 0x5593619f24d0 2024-09-10T10:53:08.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.071+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5593619faa20 0x5593619fd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.071+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619faa20 2024-09-10T10:53:08.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.071+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5593619faa20 0x5593619fd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.072+0000 7fa7dbe00640 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5593619faa20 0x5593619fd100 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.073+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v122'110 uv109 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619faa20 2024-09-10T10:53:08.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.073+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619faa20 2024-09-10T10:53:08.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.074+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v122'111 uv111 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619faa20 2024-09-10T10:53:08.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.074+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619faa20 2024-09-10T10:53:08.076 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:08.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.075+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v122'112 uv112 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619faa20 2024-09-10T10:53:08.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.075+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619faa20 2024-09-10T10:53:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.077+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v122'113 uv113 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619faa20 2024-09-10T10:53:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.077+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619faa20 2024-09-10T10:53:08.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.078+0000 7fa7dbe00640 1 -- 172.21.15.28:0/326434905 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v122'114 uv114 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fa7cc00f890 con 0x5593619faa20 2024-09-10T10:53:08.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.078+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.079+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v122'265 uv257 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.079+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.080+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v122'266 uv266 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.080+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.082 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.081+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v122'267 uv267 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.082+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fd640 con 0x5593619f6ee0 2024-09-10T10:53:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.083+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v122'268 uv268 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.083+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fd640 con 0x5593619f6ee0 2024-09-10T10:53:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.084+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v122'269 uv269 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.084+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fd640 con 0x5593619f6ee0 2024-09-10T10:53:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.085+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v122'270 uv264 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.085+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.086+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v122'271 uv271 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.087+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.088 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:08.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.088+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v122'272 uv272 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.088+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.089+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v122'273 uv273 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.089+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.090+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v122'274 uv274 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.090+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fa120 con 0x5593619f6ee0 2024-09-10T10:53:08.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.091+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v122'275 uv275 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.091+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x5593619fdaf0 con 0x5593619f6ee0 2024-09-10T10:53:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e0c00640 1 -- 172.21.15.28:0/326434905 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v122'276 uv276 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fa7d4078030 con 0x5593619f6ee0 2024-09-10T10:53:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5593619ee040 msgr2=0x5593619f0430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5593619ee040 0x5593619f0430 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5593619f6ee0 msgr2=0x5593619f92d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5593619f6ee0 0x5593619f92d0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5593619faa20 msgr2=0x5593619fd100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5593619faa20 0x5593619fd100 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5593619f24d0 msgr2=0x5593619f4f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5593619f24d0 0x5593619f4f20 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 reap_dead start 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5593619e98a0 msgr2=0x5593619ebc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5593619e98a0 0x5593619ebc90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 msgr2=0x5593619e7870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 0x5593619e7870 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e1600640 1 -- 172.21.15.28:0/326434905 reap_dead start 2024-09-10T10:53:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 msgr2=0x7fa7c006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 0x7fa7c006ccd0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55936189f350 tx=0x7fa7cc006450 comp rx=0 tx=0).stop 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 msgr2=0x5593619de6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4045ef0 tx=0x7fa7d4002fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 shutdown_connections 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5593618b57a0 0x5593619e7870 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7c006a810 0x7fa7c006ccd0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 --2- 172.21.15.28:0/326434905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5593618b6ff0 0x5593619de6e0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 >> 172.21.15.28:0/326434905 conn(0x5593618bb0a0 msgr2=0x5593618b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 shutdown_connections 2024-09-10T10:53:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.093+0000 7fa7e3972f40 1 -- 172.21.15.28:0/326434905 wait complete. 2024-09-10T10:53:08.099 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put 100.00000000.inode - 2024-09-10T10:53:08.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.216+0000 7fec36b1df40 1 Processor -- start 2024-09-10T10:53:08.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.216+0000 7fec36b1df40 1 -- start start 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f53e0 0x555d01252930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f89c0 0x555d0124e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555d011c7b30 con 0x555d0121f470 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d011cbb70 con 0x555d012f89c0 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec36b1df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555d012f7ee0 con 0x555d012f53e0 2024-09-10T10:53:08.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58886/0 (socket says 172.21.15.28:58886) 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.217+0000 7fec33200640 1 -- 172.21.15.28:0/3715579178 learned_addr learned my addr 172.21.15.28:0/3715579178 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 -- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f53e0 msgr2=0x555d01252930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f53e0 0x555d01252930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 -- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f89c0 msgr2=0x555d0124e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f89c0 0x555d0124e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 -- 172.21.15.28:0/3715579178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0124f6e0 con 0x555d0121f470 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec33200640 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fec24001e00 tx=0x7fec2400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20b21ae69776fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec31400640 1 -- 172.21.15.28:0/3715579178 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2400be40 con 0x555d0121f470 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec31400640 1 -- 172.21.15.28:0/3715579178 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2400f400 con 0x555d0121f470 2024-09-10T10:53:08.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.218+0000 7fec31400640 1 -- 172.21.15.28:0/3715579178 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2400f7d0 con 0x555d0121f470 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 -- 172.21.15.28:0/3715579178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 msgr2=0x555d012eba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fec24001e00 tx=0x7fec2400b670 comp rx=0 tx=0).stop 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 -- 172.21.15.28:0/3715579178 shutdown_connections 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f89c0 0x555d0124e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f53e0 0x555d01252930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 --2- 172.21.15.28:0/3715579178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d012eba80 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 -- 172.21.15.28:0/3715579178 >> 172.21.15.28:0/3715579178 conn(0x555d0126e0a0 msgr2=0x555d0120af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 -- 172.21.15.28:0/3715579178 shutdown_connections 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.219+0000 7fec36b1df40 1 -- 172.21.15.28:0/3715579178 wait complete. 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.220+0000 7fec36b1df40 1 Processor -- start 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.220+0000 7fec36b1df40 1 -- start start 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d0126bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f53e0 0x555d01264310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f89c0 0x555d012e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555d0124e6b0 con 0x555d0121f470 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555d0126a0d0 con 0x555d012f53e0 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec36b1df40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555d012ebfc0 con 0x555d012f89c0 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d0126bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d0126bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58890/0 (socket says 172.21.15.28:58890) 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 -- 172.21.15.28:0/1287989582 learned_addr learned my addr 172.21.15.28:0/1287989582 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec32800640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f53e0 0x555d01264310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33c00640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f89c0 0x555d012e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 -- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f89c0 msgr2=0x555d012e93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f89c0 0x555d012e93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 -- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f53e0 msgr2=0x555d01264310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555d012f53e0 0x555d01264310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33200640 1 -- 172.21.15.28:0/1287989582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d01251e80 con 0x555d0121f470 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.221+0000 7fec33c00640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555d012f89c0 0x555d012e93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec33200640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d0126bbb0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fec24000c00 tx=0x7fec24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec2be00640 1 -- 172.21.15.28:0/1287989582 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec24020070 con 0x555d0121f470 2024-09-10T10:53:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec2be00640 1 -- 172.21.15.28:0/1287989582 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec2401c440 con 0x555d0121f470 2024-09-10T10:53:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec2be00640 1 -- 172.21.15.28:0/1287989582 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2401c810 con 0x555d0121f470 2024-09-10T10:53:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d0124eac0 con 0x555d0121f470 2024-09-10T10:53:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.222+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0124eda0 con 0x555d0121f470 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.224+0000 7fec2be00640 1 -- 172.21.15.28:0/1287989582 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec24023070 con 0x555d0121f470 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec2be00640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a810 0x7fec1406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec2be00640 1 -- 172.21.15.28:0/1287989582 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7fec2408bff0 con 0x555d0121f470 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec32800640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a810 0x7fec1406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec36b1df40 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555d012ec1a0 0x555d012e20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [writefull 0~80 in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) -- 0x555d0126c0f0 con 0x555d012ec1a0 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.225+0000 7fec33c00640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555d012ec1a0 0x555d012e20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.226+0000 7fec32800640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a810 0x7fec1406ccd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x555d0126c510 tx=0x7fec200017a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.226+0000 7fec33c00640 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555d012ec1a0 0x555d012e20c0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec33c00640 1 -- 172.21.15.28:0/1287989582 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [writefull 0~80] v122'277 uv277 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fec2c04d5b0 con 0x555d012ec1a0 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555d012ec1a0 msgr2=0x555d012e20c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x555d012ec1a0 0x555d012e20c0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a810 msgr2=0x7fec1406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec1406a810 0x7fec1406ccd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x555d0126c510 tx=0x7fec200017a0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 msgr2=0x555d0126bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 --2- 172.21.15.28:0/1287989582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555d0121f470 0x555d0126bbb0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fec24000c00 tx=0x7fec24000ed0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec33c00640 1 -- 172.21.15.28:0/1287989582 reap_dead start 2024-09-10T10:53:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 shutdown_connections 2024-09-10T10:53:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 >> 172.21.15.28:0/1287989582 conn(0x555d0126e0a0 msgr2=0x555d0120af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 shutdown_connections 2024-09-10T10:53:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.228+0000 7fec36b1df40 1 -- 172.21.15.28:0/1287989582 wait complete. 2024-09-10T10:53:08.234 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:08.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.432+0000 7f057f000640 1 Processor -- start 2024-09-10T10:53:08.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.432+0000 7f057f000640 1 -- start start 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 0x7f057806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057f000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f057815ff70 con 0x7f05780cf280 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057f000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05780d4d10 con 0x7f0578162340 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f057806de90 con 0x7f0578162e30 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f05780cf280 0x7f05780d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47218/0 (socket says 172.21.15.28:47218) 2024-09-10T10:53:08.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.433+0000 7f057d000640 1 -- 172.21.15.28:0/2185712888 learned_addr learned my addr 172.21.15.28:0/2185712888 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:08.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f056f400640 1 --2- 172.21.15.28:0/2185712888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 0x7f057806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354729161 0 0) 0x7f057815ff70 con 0x7f05780cf280 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f055c003520 con 0x7f05780cf280 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3291417630 0 0) 0x7f05780d4d10 con 0x7f0578162340 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f057815ff70 con 0x7f0578162340 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f056f400640 1 -- 172.21.15.28:0/2185712888 >> v1:172.21.15.138:6789/0 conn(0x7f0578162340 legacy=0x7f0578162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f056f400640 1 -- 172.21.15.28:0/2185712888 >> v1:172.21.15.28:6789/0 conn(0x7f05780cf280 legacy=0x7f05780d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 307170178 0 0) 0x7f055c003520 con 0x7f05780cf280 2024-09-10T10:53:08.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.434+0000 7f056f400640 1 -- 172.21.15.28:0/2185712888 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05780d4ff0 con 0x7f0578162e30 2024-09-10T10:53:08.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.435+0000 7f056f400640 1 --2- 172.21.15.28:0/2185712888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 0x7f057806d740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05600097c0 tx=0x7f056002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1c3de8a7dc9b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.435+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0560037070 con 0x7f0578162e30 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056002ab90 con 0x7f0578162e30 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f0577400640 1 -- 172.21.15.28:0/2185712888 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570001670 con 0x7f0578162e30 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 -- 172.21.15.28:0/2185712888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 msgr2=0x7f057806d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 --2- 172.21.15.28:0/2185712888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 0x7f057806d740 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05600097c0 tx=0x7f056002a100 comp rx=0 tx=0).stop 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 -- 172.21.15.28:0/2185712888 shutdown_connections 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 --2- 172.21.15.28:0/2185712888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162e30 0x7f057806d740 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 -- 172.21.15.28:0/2185712888 >> 172.21.15.28:0/2185712888 conn(0x7f057808bfc0 msgr2=0x7f05780d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 -- 172.21.15.28:0/2185712888 shutdown_connections 2024-09-10T10:53:08.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.436+0000 7f057f000640 1 -- 172.21.15.28:0/2185712888 wait complete. 2024-09-10T10:53:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.438+0000 7f057f000640 1 Processor -- start 2024-09-10T10:53:08.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.438+0000 7f057f000640 1 -- start start 2024-09-10T10:53:08.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 0x7f05780ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162340 0x7f05780cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05780cf710 con 0x7f0578162e30 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0578163270 con 0x7f05780cf280 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057f000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05780d52d0 con 0x7f0578162340 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f0577e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 0x7f05780ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58892/0 (socket says 172.21.15.28:58892) 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f057d000640 1 -- 172.21.15.28:0/825926653 learned_addr learned my addr 172.21.15.28:0/825926653 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.439+0000 7f056f400640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162340 0x7f05780cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 -- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162340 msgr2=0x7f05780cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162340 0x7f05780cb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 -- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 msgr2=0x7f05780ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 0x7f05780ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 -- 172.21.15.28:0/825926653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05781e9660 con 0x7f0578162e30 2024-09-10T10:53:08.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f0577e00640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 0x7f05780ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057d000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f057000c560 tx=0x7f057000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570017070 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0570005d00 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05700072b0 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05781e9940 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.440+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05781e9f00 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.442+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057806d740 con 0x7f0578162e30 2024-09-10T10:53:08.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.442+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f057001a070 con 0x7f0578162e30 2024-09-10T10:53:08.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.443+0000 7f0574c00640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 0x7f055406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:08.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.443+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8196+0+0 (secure 0 0 0) 0x7f057008ac00 con 0x7f0578162e30 2024-09-10T10:53:08.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.444+0000 7f0577e00640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 0x7f055406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:08.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.444+0000 7f0577e00640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 0x7f055406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05780a23a0 tx=0x7f0568009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:08.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.447+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0570091080 con 0x7f0578162e30 2024-09-10T10:53:08.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.671+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f05780ba450 con 0x7f0578162e30 2024-09-10T10:53:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.795+0000 7f0574c00640 1 -- 172.21.15.28:0/825926653 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v215) ==== 162+0+0 (secure 0 0 0) 0x7f05700585e0 con 0x7f0578162e30 2024-09-10T10:53:08.796 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:08.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 msgr2=0x7f055406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 0x7f055406e2f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f05780a23a0 tx=0x7f0568009040 comp rx=0 tx=0).stop 2024-09-10T10:53:08.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 msgr2=0x7f05781ea420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:08.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f057000c560 tx=0x7f057000ca20 comp rx=0 tx=0).stop 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 shutdown_connections 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f055406be30 0x7f055406e2f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0578162e30 0x7f05781ea420 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0578162340 0x7f05780cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 --2- 172.21.15.28:0/825926653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05780cf280 0x7f05780ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.797+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 >> 172.21.15.28:0/825926653 conn(0x7f057808bfc0 msgr2=0x7f057815d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.798+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 shutdown_connections 2024-09-10T10:53:08.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:08.798+0000 7f057f000640 1 -- 172.21.15.28:0/825926653 wait complete. 2024-09-10T10:53:08.852 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:08.807+0000 7f94d7600640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-10T10:53:08.852 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:08.807+0000 7f94d7600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:53:08.852 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:08.807+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:08.852 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:08.808+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:08.853 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.008+0000 7fa75a400640 1 Processor -- start 2024-09-10T10:53:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.008+0000 7fa75a400640 1 -- start start 2024-09-10T10:53:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa75a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 0x7fa7540cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa75a400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa754146140 con 0x7fa75415e1f0 2024-09-10T10:53:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa75a400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa75406eed0 con 0x7fa754072de0 2024-09-10T10:53:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa75a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7540cd740 con 0x7fa754072320 2024-09-10T10:53:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa753e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa75415e1f0 0x7fa75406e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47230/0 (socket says 172.21.15.28:47230) 2024-09-10T10:53:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.009+0000 7fa753e00640 1 -- 172.21.15.28:0/1020896677 learned_addr learned my addr 172.21.15.28:0/1020896677 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa753400640 1 --2- 172.21.15.28:0/1020896677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 0x7fa7540cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4083864858 0 0) 0x7fa754146140 con 0x7fa75415e1f0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa738003520 con 0x7fa75415e1f0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207386421 0 0) 0x7fa75406eed0 con 0x7fa754072de0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa754146140 con 0x7fa754072de0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa753400640 1 -- 172.21.15.28:0/1020896677 >> v1:172.21.15.138:6789/0 conn(0x7fa754072de0 legacy=0x7fa75406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa753400640 1 -- 172.21.15.28:0/1020896677 >> v1:172.21.15.28:6789/0 conn(0x7fa75415e1f0 legacy=0x7fa75406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2476335953 0 0) 0x7fa738003520 con 0x7fa75415e1f0 2024-09-10T10:53:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.010+0000 7fa753400640 1 -- 172.21.15.28:0/1020896677 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7540cda20 con 0x7fa754072320 2024-09-10T10:53:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.011+0000 7fa753400640 1 --2- 172.21.15.28:0/1020896677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 0x7fa7540cce60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097d0 tx=0x7fa74402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb99bcba6607d1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa744037070 con 0x7fa754072320 2024-09-10T10:53:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74402ab90 con 0x7fa754072320 2024-09-10T10:53:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa752a00640 1 -- 172.21.15.28:0/1020896677 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0019f0 con 0x7fa754072320 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 -- 172.21.15.28:0/1020896677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 msgr2=0x7fa7540cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 --2- 172.21.15.28:0/1020896677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 0x7fa7540cce60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa7440097d0 tx=0x7fa74402a100 comp rx=0 tx=0).stop 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 -- 172.21.15.28:0/1020896677 shutdown_connections 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 --2- 172.21.15.28:0/1020896677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072320 0x7fa7540cce60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 -- 172.21.15.28:0/1020896677 >> 172.21.15.28:0/1020896677 conn(0x7fa754067d70 msgr2=0x7fa75406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.012+0000 7fa75a400640 1 -- 172.21.15.28:0/1020896677 shutdown_connections 2024-09-10T10:53:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.013+0000 7fa75a400640 1 -- 172.21.15.28:0/1020896677 wait complete. 2024-09-10T10:53:09.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.014+0000 7fa75a400640 1 Processor -- start 2024-09-10T10:53:09.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.014+0000 7fa75a400640 1 -- start start 2024-09-10T10:53:09.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754072320 0x7fa7540b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072de0 0x7fa754142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7540cd3a0 con 0x7fa75415e1f0 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa75415c9e0 con 0x7fa754072320 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa75a400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7540cd520 con 0x7fa754072de0 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa753e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa753e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58906/0 (socket says 172.21.15.28:58906) 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.015+0000 7fa753e00640 1 -- 172.21.15.28:0/1060102091 learned_addr learned my addr 172.21.15.28:0/1060102091 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754072320 0x7fa7540b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa74be00640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072de0 0x7fa754142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 -- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072de0 msgr2=0x7fa754142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072de0 0x7fa754142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 -- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754072320 msgr2=0x7fa7540b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754072320 0x7fa7540b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 -- 172.21.15.28:0/1060102091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7541f5300 con 0x7fa75415e1f0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa753e00640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa74c002230 tx=0x7fa74c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00ad40 con 0x7fa75415e1f0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c012400 con 0x7fa75415e1f0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.016+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c012810 con 0x7fa75415e1f0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.017+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa754133810 con 0x7fa75415e1f0 2024-09-10T10:53:09.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.017+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa754133dd0 con 0x7fa75415e1f0 2024-09-10T10:53:09.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.018+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa754072700 con 0x7fa75415e1f0 2024-09-10T10:53:09.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.019+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa74c0129b0 con 0x7fa75415e1f0 2024-09-10T10:53:09.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.019+0000 7fa74b400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 0x7fa73006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.020+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7fa74c08c070 con 0x7fa75415e1f0 2024-09-10T10:53:09.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.021+0000 7fa753400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 0x7fa73006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.022+0000 7fa753400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 0x7fa73006e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa744034000 tx=0x7fa74402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.024+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa74c059f60 con 0x7fa75415e1f0 2024-09-10T10:53:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.246+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7541e6d60 con 0x7fa75415e1f0 2024-09-10T10:53:09.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.247+0000 7fa74b400640 1 -- 172.21.15.28:0/1060102091 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) ==== 95+0+3566 (secure 0 0 0) 0x7fa74c059860 con 0x7fa75415e1f0 2024-09-10T10:53:09.248 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:09.248 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":217,"btime":"2024-09-10T10:53:08:808515+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:08.808449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:09.248 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 217 2024-09-10T10:53:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 msgr2=0x7fa73006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 0x7fa73006e2f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa744034000 tx=0x7fa74402e000 comp rx=0 tx=0).stop 2024-09-10T10:53:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 msgr2=0x7fa754142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa74c002230 tx=0x7fa74c00a430 comp rx=0 tx=0).stop 2024-09-10T10:53:09.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 shutdown_connections 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa73006be30 0x7fa73006e2f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa75415e1f0 0x7fa754142d80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa754072de0 0x7fa754142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 --2- 172.21.15.28:0/1060102091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa754072320 0x7fa7540b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 >> 172.21.15.28:0/1060102091 conn(0x7fa754067d70 msgr2=0x7fa7540ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 shutdown_connections 2024-09-10T10:53:09.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.249+0000 7fa75a400640 1 -- 172.21.15.28:0/1060102091 wait complete. 2024-09-10T10:53:09.266 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:09.266 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:09.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.420+0000 7f3f93c00640 1 Processor -- start 2024-09-10T10:53:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.420+0000 7f3f93c00640 1 -- start start 2024-09-10T10:53:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.421+0000 7f3f93c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c15b300 0x7f3f8c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.421+0000 7f3f93c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.421+0000 7f3f93c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8c0cdbe0 con 0x7f3f8c072d30 2024-09-10T10:53:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f93c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c0728a0 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f93c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c0cd900 con 0x7f3f8c15b300 2024-09-10T10:53:09.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58916/0 (socket says 172.21.15.28:58916) 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 -- 172.21.15.28:0/1844115217 learned_addr learned my addr 172.21.15.28:0/1844115217 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f91200640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c15b300 0x7f3f8c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 -- 172.21.15.28:0/1844115217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c15b300 msgr2=0x7f3f8c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c15b300 0x7f3f8c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 -- 172.21.15.28:0/1844115217 >> v1:172.21.15.138:6789/0 conn(0x7f3f8c072d30 legacy=0x7f3f8c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.422+0000 7f3f8be00640 1 -- 172.21.15.28:0/1844115217 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c072250 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.423+0000 7f3f8be00640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800bb10 tx=0x7f3f7802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5edf6905bf905ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f8b400640 1 -- 172.21.15.28:0/1844115217 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78036020 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f8b400640 1 -- 172.21.15.28:0/1844115217 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78032820 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f8b400640 1 -- 172.21.15.28:0/1844115217 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78032bf0 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 -- 172.21.15.28:0/1844115217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c0cd3f0 msgr2=0x7f3f8c06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800bb10 tx=0x7f3f7802c920 comp rx=0 tx=0).stop 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 -- 172.21.15.28:0/1844115217 shutdown_connections 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c0cd3f0 0x7f3f8c06f1c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 --2- 172.21.15.28:0/1844115217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c15b300 0x7f3f8c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 -- 172.21.15.28:0/1844115217 >> 172.21.15.28:0/1844115217 conn(0x7f3f8c067d70 msgr2=0x7f3f8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 -- 172.21.15.28:0/1844115217 shutdown_connections 2024-09-10T10:53:09.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.424+0000 7f3f93c00640 1 -- 172.21.15.28:0/1844115217 wait complete. 2024-09-10T10:53:09.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.426+0000 7f3f93c00640 1 Processor -- start 2024-09-10T10:53:09.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.426+0000 7f3f93c00640 1 -- start start 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c0cd3f0 0x7f3f8c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8c15b300 0x7f3f8c1efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0d0160 con 0x7f3f8c072d30 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c0cffe0 con 0x7f3f8c15b300 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f93c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c06f700 con 0x7f3f8c0cd3f0 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f91200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f91200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58920/0 (socket says 172.21.15.28:58920) 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f91200640 1 -- 172.21.15.28:0/2946280533 learned_addr learned my addr 172.21.15.28:0/2946280533 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f8be00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c0cd3f0 0x7f3f8c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.427+0000 7f3f91c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8c15b300 0x7f3f8c1efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 -- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c0cd3f0 msgr2=0x7f3f8c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c0cd3f0 0x7f3f8c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 -- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8c15b300 msgr2=0x7f3f8c1efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8c15b300 0x7f3f8c1efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 -- 172.21.15.28:0/2946280533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c1f4aa0 con 0x7f3f8c072d30 2024-09-10T10:53:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f91200640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e870 tx=0x7f3f8000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80017070 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8000c940 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c155210 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000cd30 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.428+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c153bc0 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.430+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c0cc290 con 0x7f3f8c072d30 2024-09-10T10:53:09.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.430+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f8001a070 con 0x7f3f8c072d30 2024-09-10T10:53:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.432+0000 7f3f88c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 0x7f3f6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.432+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7f3f8008b690 con 0x7f3f8c072d30 2024-09-10T10:53:09.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.432+0000 7f3f8be00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 0x7f3f6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.433+0000 7f3f8be00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 0x7f3f6c06e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3f78037000 tx=0x7f3f7803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.436+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3f8c1f4aa0 con 0x7f3f8c072d30 2024-09-10T10:53:09.527 INFO:tasks.ceph.mds.c.smithi028.stderr: -42> 2024-09-10T10:53:04.774+0000 7fbf42200640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T10:53:09.567 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.660+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8c15c570 con 0x7f3f8c072d30 2024-09-10T10:53:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.661+0000 7f3f88c00640 1 -- 172.21.15.28:0/2946280533 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) ==== 95+0+3566 (secure 0 0 0) 0x7f3f80059020 con 0x7f3f8c072d30 2024-09-10T10:53:09.662 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:09.662 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":217,"btime":"2024-09-10T10:53:08:808515+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:08.808449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 217 2024-09-10T10:53:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 msgr2=0x7f3f6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 0x7f3f6c06e2f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3f78037000 tx=0x7f3f7803d040 comp rx=0 tx=0).stop 2024-09-10T10:53:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 msgr2=0x7f3f8c1f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e870 tx=0x7f3f8000ed30 comp rx=0 tx=0).stop 2024-09-10T10:53:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 shutdown_connections 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f6c06be30 0x7f3f6c06e2f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f8c15b300 0x7f3f8c1efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f8c0cd3f0 0x7f3f8c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 --2- 172.21.15.28:0/2946280533 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f8c072d30 0x7f3f8c1f4560 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 >> 172.21.15.28:0/2946280533 conn(0x7f3f8c067d70 msgr2=0x7f3f8c15cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 shutdown_connections 2024-09-10T10:53:09.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.663+0000 7f3f93c00640 1 -- 172.21.15.28:0/2946280533 wait complete. 2024-09-10T10:53:09.680 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:09.680 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:09.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.837+0000 7fcd93c00640 1 Processor -- start 2024-09-10T10:53:09.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.837+0000 7fcd93c00640 1 -- start start 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.838+0000 7fcd93c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd8c072320 0x7fcd8c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.838+0000 7fcd93c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 0x7fcd8c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.838+0000 7fcd93c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd8c146140 con 0x7fcd8c072700 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.838+0000 7fcd93c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fcd8c0cd740 con 0x7fcd8c072320 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.838+0000 7fcd93c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c0cda20 con 0x7fcd8c072de0 2024-09-10T10:53:09.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd8c072320 0x7fcd8c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd8c072320 0x7fcd8c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58946/0 (socket says 172.21.15.28:58946) 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 -- 172.21.15.28:0/4210171918 learned_addr learned my addr 172.21.15.28:0/4210171918 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd83e00640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 0x7fcd8c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 -- 172.21.15.28:0/4210171918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 msgr2=0x7fcd8c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 0x7fcd8c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 -- 172.21.15.28:0/4210171918 >> v1:172.21.15.138:6789/0 conn(0x7fcd8c072700 legacy=0x7fcd8c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd91200640 1 -- 172.21.15.28:0/4210171918 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0cdd00 con 0x7fcd8c072320 2024-09-10T10:53:09.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.839+0000 7fcd83e00640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 0x7fcd8c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.840+0000 7fcd91200640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072320 0x7fcd8c0cce60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009870 tx=0x7fcd7c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3cf86e88c11a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.840+0000 7fcd8be00640 1 -- 172.21.15.28:0/4210171918 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c037070 con 0x7fcd8c072320 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd8be00640 1 -- 172.21.15.28:0/4210171918 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7c032440 con 0x7fcd8c072320 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd8be00640 1 -- 172.21.15.28:0/4210171918 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c032810 con 0x7fcd8c072320 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 -- 172.21.15.28:0/4210171918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072320 msgr2=0x7fcd8c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072320 0x7fcd8c0cce60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009870 tx=0x7fcd7c02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 -- 172.21.15.28:0/4210171918 shutdown_connections 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072de0 0x7fcd8c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 --2- 172.21.15.28:0/4210171918 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072320 0x7fcd8c0cce60 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 -- 172.21.15.28:0/4210171918 >> 172.21.15.28:0/4210171918 conn(0x7fcd8c067d70 msgr2=0x7fcd8c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 -- 172.21.15.28:0/4210171918 shutdown_connections 2024-09-10T10:53:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.841+0000 7fcd93c00640 1 -- 172.21.15.28:0/4210171918 wait complete. 2024-09-10T10:53:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.843+0000 7fcd93c00640 1 Processor -- start 2024-09-10T10:53:09.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.843+0000 7fcd93c00640 1 -- start start 2024-09-10T10:53:09.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 0x7fcd8c0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8c072700 0x7fcd8c13b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 0x7fcd8c13bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0cd520 con 0x7fcd8c072de0 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8c0cd3a0 con 0x7fcd8c072700 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd93c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd8c06ed00 con 0x7fcd8c072320 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd91c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 0x7fcd8c13bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd91200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 0x7fcd8c0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd91200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 0x7fcd8c0ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:42088/0 (socket says 172.21.15.28:42088) 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd91200640 1 -- 172.21.15.28:0/708787828 learned_addr learned my addr 172.21.15.28:0/708787828 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.844+0000 7fcd83e00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8c072700 0x7fcd8c13b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 -- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 msgr2=0x7fcd8c0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 0x7fcd8c0ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 -- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8c072700 msgr2=0x7fcd8c13b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:09.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8c072700 0x7fcd8c13b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 -- 172.21.15.28:0/708787828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8c0b1100 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd91c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 0x7fcd8c13bee0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c6c0 tx=0x7fcd8400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84003b70 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd84003d10 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd840052f0 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8c0b13e0 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.845+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd8c0a98a0 con 0x7fcd8c072de0 2024-09-10T10:53:09.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.846+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd8c0cce60 con 0x7fcd8c072de0 2024-09-10T10:53:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.847+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd84005490 con 0x7fcd8c072de0 2024-09-10T10:53:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.848+0000 7fcd89600640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 0x7fcd6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.848+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7fcd8408b550 con 0x7fcd8c072de0 2024-09-10T10:53:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.849+0000 7fcd91200640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 0x7fcd6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:09.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.850+0000 7fcd91200640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 0x7fcd6806e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0090b0 tx=0x7fcd7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:09.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:09.853+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd84058e80 con 0x7fcd8c072de0 2024-09-10T10:53:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.073+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd8c0b15b0 con 0x7fcd8c072de0 2024-09-10T10:53:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.073+0000 7fcd89600640 1 -- 172.21.15.28:0/708787828 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) ==== 95+0+4392 (secure 0 0 0) 0x7fcd8405d180 con 0x7fcd8c072de0 2024-09-10T10:53:10.075 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:10.075 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":218,"btime":"2024-09-10T10:53:09:810173+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2095575543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2095575543},{"type":"v1","addr":"172.21.15.28:6834","nonce":2095575543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:08.808449+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 218 2024-09-10T10:53:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.075+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 msgr2=0x7fcd6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.075+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 0x7fcd6806e2f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0090b0 tx=0x7fcd7c02e000 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.075+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 msgr2=0x7fcd8c13bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.075+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 0x7fcd8c13bee0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400c6c0 tx=0x7fcd8400cb80 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 shutdown_connections 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd6806be30 0x7fcd6806e2f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd8c072de0 0x7fcd8c13bee0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd8c072700 0x7fcd8c13b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 --2- 172.21.15.28:0/708787828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd8c072320 0x7fcd8c0ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 >> 172.21.15.28:0/708787828 conn(0x7fcd8c067d70 msgr2=0x7fcd8c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 shutdown_connections 2024-09-10T10:53:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.076+0000 7fcd93c00640 1 -- 172.21.15.28:0/708787828 wait complete. 2024-09-10T10:53:10.092 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt 100.00000000.inode' led to DAMAGED state 2024-09-10T10:53:10.092 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 1.00000000.inode' 2024-09-10T10:53:10.092 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:10.092 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:10.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:10.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:10.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:10.128 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:10.202 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:10.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:10.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:10.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.401+0000 7fd082e00640 1 Processor -- start 2024-09-10T10:53:10.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.401+0000 7fd082e00640 1 -- start start 2024-09-10T10:53:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd082e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd07c0d0dd0 0x7fd07c0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd082e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd082e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07c0cf000 con 0x7fd07c15e660 2024-09-10T10:53:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd082e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd07c06d3e0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd082e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c0b37d0 con 0x7fd07c0d1f90 2024-09-10T10:53:10.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd081000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd07c15e660 0x7fd07c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47256/0 (socket says 172.21.15.28:47256) 2024-09-10T10:53:10.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd081000640 1 -- 172.21.15.28:0/4060514838 learned_addr learned my addr 172.21.15.28:0/4060514838 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:10.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.402+0000 7fd081a00640 1 --2- 172.21.15.28:0/4060514838 >> v2:172.21.15.138:3300/0 conn(0x7fd07c0d0dd0 0x7fd07c0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd082400640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd07be00640 1 -- 172.21.15.28:0/4060514838 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1012602230 0 0) 0x7fd07c0cf000 con 0x7fd07c15e660 2024-09-10T10:53:10.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd07be00640 1 -- 172.21.15.28:0/4060514838 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd064003520 con 0x7fd07c15e660 2024-09-10T10:53:10.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd081a00640 1 -- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 msgr2=0x7fd07c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:10.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd081a00640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c0cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd081a00640 1 -- 172.21.15.28:0/4060514838 >> v1:172.21.15.28:6789/0 conn(0x7fd07c15e660 legacy=0x7fd07c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.403+0000 7fd081a00640 1 -- 172.21.15.28:0/4060514838 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c0d16f0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.404+0000 7fd081a00640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0d11b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097a0 tx=0x7fd07002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9960987f43f7bc3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.404+0000 7fd07be00640 1 -- 172.21.15.28:0/4060514838 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070037070 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.404+0000 7fd07be00640 1 -- 172.21.15.28:0/4060514838 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070032440 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.404+0000 7fd07be00640 1 -- 172.21.15.28:0/4060514838 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0700327f0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 -- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c0d0dd0 msgr2=0x7fd07c0d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0d11b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd0700097a0 tx=0x7fd07002a180 comp rx=0 tx=0).stop 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 -- 172.21.15.28:0/4060514838 shutdown_connections 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c0cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 --2- 172.21.15.28:0/4060514838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0d11b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 -- 172.21.15.28:0/4060514838 >> 172.21.15.28:0/4060514838 conn(0x7fd07c08bfc0 msgr2=0x7fd07c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 -- 172.21.15.28:0/4060514838 shutdown_connections 2024-09-10T10:53:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.405+0000 7fd082e00640 1 -- 172.21.15.28:0/4060514838 wait complete. 2024-09-10T10:53:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.407+0000 7fd082e00640 1 Processor -- start 2024-09-10T10:53:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.407+0000 7fd082e00640 1 -- start start 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 0x7fd07c1f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c0d2c60 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c0cf330 con 0x7fd07c15e660 2024-09-10T10:53:10.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c0d2f40 con 0x7fd07c0d1f90 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58962/0 (socket says 172.21.15.28:58962) 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 -- 172.21.15.28:0/382009816 learned_addr learned my addr 172.21.15.28:0/382009816 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081000640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082400640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 0x7fd07c1f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 -- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 msgr2=0x7fd07c150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 -- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 msgr2=0x7fd07c1f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 0x7fd07c1f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081a00640 1 -- 172.21.15.28:0/382009816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c153ec0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd082400640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 0x7fd07c1f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.408+0000 7fd081000640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd081a00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd07002a1c0 tx=0x7fd0700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070037040 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd070042070 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07003b930 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c150fc0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.409+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c14e010 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.410+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07c0d1b20 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.411+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd07003bad0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.412+0000 7fd079600640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.412+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7fd0700ac210 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.412+0000 7fd081000640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.413+0000 7fd081000640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 0x7fd05c06e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0b4c10 tx=0x7fd068009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:10.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.416+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd07007a070 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.466 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:53:10.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:53:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.640+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd07c0b64a0 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.816+0000 7fd079600640 1 -- 172.21.15.28:0/382009816 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v219) ==== 161+0+0 (secure 0 0 0) 0x7fd070079970 con 0x7fd07c0d0dd0 2024-09-10T10:53:10.818 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 msgr2=0x7fd05c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 0x7fd05c06e2f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0b4c10 tx=0x7fd068009040 comp rx=0 tx=0).stop 2024-09-10T10:53:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 msgr2=0x7fd07c0cdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fd07002a1c0 tx=0x7fd0700046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 shutdown_connections 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd05c06be30 0x7fd05c06e2f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd07c15e660 0x7fd07c1f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd07c0d1f90 0x7fd07c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 --2- 172.21.15.28:0/382009816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd07c0d0dd0 0x7fd07c0cdff0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 >> 172.21.15.28:0/382009816 conn(0x7fd07c08bfc0 msgr2=0x7fd07c1e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 shutdown_connections 2024-09-10T10:53:10.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.819+0000 7fd082e00640 1 -- 172.21.15.28:0/382009816 wait complete. 2024-09-10T10:53:10.836 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.991+0000 7f2072400640 1 Processor -- start 2024-09-10T10:53:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.991+0000 7f2072400640 1 -- start start 2024-09-10T10:53:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.992+0000 7f2072400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f206c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.992+0000 7f2072400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206c146140 con 0x7f206c072700 2024-09-10T10:53:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.992+0000 7f2072400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f206c0cd740 con 0x7f206c072320 2024-09-10T10:53:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.992+0000 7f2072400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f206c0cda20 con 0x7f206c072de0 2024-09-10T10:53:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f206c072700 0x7f206c15d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47260/0 (socket says 172.21.15.28:47260) 2024-09-10T10:53:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206be00640 1 -- 172.21.15.28:0/1983645261 learned_addr learned my addr 172.21.15.28:0/1983645261 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206aa00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f206c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 916425187 0 0) 0x7f206c146140 con 0x7f206c072700 2024-09-10T10:53:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2050003520 con 0x7f206c072700 2024-09-10T10:53:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3606251280 0 0) 0x7f206c0cd740 con 0x7f206c072320 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f206c146140 con 0x7f206c072320 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1192638881 0 0) 0x7f2050003520 con 0x7f206c072700 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 msgr2=0x7f206c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f206c15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 >> v1:172.21.15.138:6789/0 conn(0x7f206c072320 legacy=0x7f206c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c0cdd00 con 0x7f206c072700 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f20640019f0 con 0x7f206c072700 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 >> v1:172.21.15.28:6789/0 conn(0x7f206c072700 legacy=0x7f206c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2050003b70 0x7f2050003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f2050004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f20500051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:10.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f206c0cda20 con 0x7f2050004800 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f205000bb00 con 0x7f2050003b70 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.993+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f205000bd00 con 0x7f206c072de0 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f2050004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206be00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f20500051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206b400640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2050003b70 0x7f2050003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 -- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 msgr2=0x7f20500051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c072de0 0x7f20500051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 -- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2050003b70 msgr2=0x7f2050003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2050003b70 0x7f2050003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:10.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.994+0000 7f206aa00640 1 -- 172.21.15.28:0/1983645261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2050003520 con 0x7f2050004800 2024-09-10T10:53:10.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.995+0000 7f206aa00640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f2050004c80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2058009ac0 tx=0x7f2050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e495260e65b2c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.995+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058036070 con 0x7f2050004800 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.995+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2058031400 con 0x7f2050004800 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.995+0000 7f206a000640 1 -- 172.21.15.28:0/1983645261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058009690 con 0x7f2050004800 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 -- 172.21.15.28:0/1983645261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 msgr2=0x7f2050004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 --2- 172.21.15.28:0/1983645261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f2050004c80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2058009ac0 tx=0x7f2050005830 comp rx=0 tx=0).stop 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f206be00640 1 -- 172.21.15.28:0/1983645261 reap_dead start 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 -- 172.21.15.28:0/1983645261 shutdown_connections 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 -- 172.21.15.28:0/1983645261 >> 172.21.15.28:0/1983645261 conn(0x7f206c067d70 msgr2=0x7f206c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 -- 172.21.15.28:0/1983645261 shutdown_connections 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.996+0000 7f2072400640 1 -- 172.21.15.28:0/1983645261 wait complete. 2024-09-10T10:53:10.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.998+0000 7f2072400640 1 Processor -- start 2024-09-10T10:53:11.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.998+0000 7f2072400640 1 -- start start 2024-09-10T10:53:11.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c142d70 0x7f206c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f206c1e4a40 0x7f206c1e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f206c15d360 con 0x7f2050004800 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f206c15d1e0 con 0x7f206c1e4a40 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f2072400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f206c15d4e0 con 0x7f206c142d70 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58976/0 (socket says 172.21.15.28:58976) 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 -- 172.21.15.28:0/2725840779 learned_addr learned my addr 172.21.15.28:0/2725840779 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 -- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c142d70 msgr2=0x7f206c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c142d70 0x7f206c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 -- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f206c1e4a40 msgr2=0x7f206c1e4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f206c1e4a40 0x7f206c1e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:10.999+0000 7f206b400640 1 -- 172.21.15.28:0/2725840779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206c143830 con 0x7f2050004800 2024-09-10T10:53:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f206b400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e7b0 tx=0x7f205c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c00ee60 con 0x7f2050004800 2024-09-10T10:53:11.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205c018070 con 0x7f2050004800 2024-09-10T10:53:11.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c0035f0 con 0x7f2050004800 2024-09-10T10:53:11.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206c143b10 con 0x7f2050004800 2024-09-10T10:53:11.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.000+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f206c1e2880 con 0x7f2050004800 2024-09-10T10:53:11.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.001+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f206c0cce60 con 0x7f2050004800 2024-09-10T10:53:11.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.002+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f205c003790 con 0x7f2050004800 2024-09-10T10:53:11.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.003+0000 7f2063400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.003+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7f205c08b630 con 0x7f2050004800 2024-09-10T10:53:11.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.003+0000 7f206aa00640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.004+0000 7f206aa00640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 0x7f204c06e320 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2058033000 tx=0x7f205802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.007+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f205c059520 con 0x7f2050004800 2024-09-10T10:53:11.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.267+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f206c144b70 con 0x7f2050004800 2024-09-10T10:53:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.823+0000 7f2063400640 1 -- 172.21.15.28:0/2725840779 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v220) ==== 101+0+0 (secure 0 0 0) 0x7f205c058e20 con 0x7f2050004800 2024-09-10T10:53:11.824 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 msgr2=0x7f204c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 0x7f204c06e320 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2058033000 tx=0x7f205802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 msgr2=0x7f206c134a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f205c00e7b0 tx=0x7f205c00ec70 comp rx=0 tx=0).stop 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 shutdown_connections 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f204c06be60 0x7f204c06e320 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f206c1e4a40 0x7f206c1e4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f206c142d70 0x7f206c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 --2- 172.21.15.28:0/2725840779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2050004800 0x7f206c134a00 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 >> 172.21.15.28:0/2725840779 conn(0x7f206c067d70 msgr2=0x7f206c0cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 shutdown_connections 2024-09-10T10:53:11.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.826+0000 7f2072400640 1 -- 172.21.15.28:0/2725840779 wait complete. 2024-09-10T10:53:11.846 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.923+0000 7ff883305f40 1 Processor -- start 2024-09-10T10:53:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.923+0000 7ff883305f40 1 -- start start 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeaaf5a0 0x5589eeadbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 0x5589eeb83c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589eeaf4dc0 con 0x5589eeaf52d0 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589eea53b30 con 0x5589eeb814b0 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff883305f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589eeaf41f0 con 0x5589eeaaf5a0 2024-09-10T10:53:11.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff87fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff87fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58986/0 (socket says 172.21.15.28:58986) 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff87fa00640 1 -- 172.21.15.28:0/1459196725 learned_addr learned my addr 172.21.15.28:0/1459196725 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff880400640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeaaf5a0 0x5589eeadbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff880e00640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 0x5589eeb83c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff87fa00640 1 -- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeaaf5a0 msgr2=0x5589eeadbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.924+0000 7ff87fa00640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeaaf5a0 0x5589eeadbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87fa00640 1 -- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 msgr2=0x5589eeb83c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87fa00640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 0x5589eeb83c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87fa00640 1 -- 172.21.15.28:0/1459196725 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589eeb7c840 con 0x5589eeaf52d0 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff880e00640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 0x5589eeb83c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87fa00640 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001d10 tx=0x7ff86c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce25ad16de836cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87e600640 1 -- 172.21.15.28:0/1459196725 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c001e60 con 0x5589eeaf52d0 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87e600640 1 -- 172.21.15.28:0/1459196725 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c00bd80 con 0x5589eeaf52d0 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.925+0000 7ff87e600640 1 -- 172.21.15.28:0/1459196725 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00f640 con 0x5589eeaf52d0 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 -- 172.21.15.28:0/1459196725 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 msgr2=0x5589eeb836e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001d10 tx=0x7ff86c00b5b0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 -- 172.21.15.28:0/1459196725 shutdown_connections 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeb814b0 0x5589eeb83c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaf52d0 0x5589eeb836e0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 --2- 172.21.15.28:0/1459196725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeaaf5a0 0x5589eeadbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 -- 172.21.15.28:0/1459196725 >> 172.21.15.28:0/1459196725 conn(0x5589eead9b90 msgr2=0x5589eead9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 -- 172.21.15.28:0/1459196725 shutdown_connections 2024-09-10T10:53:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 -- 172.21.15.28:0/1459196725 wait complete. 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.926+0000 7ff883305f40 1 Processor -- start 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 -- start start 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeaf52d0 0x5589eeb7a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeb814b0 0x5589eeb836e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589eeaece00 con 0x5589eeaaf5a0 2024-09-10T10:53:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589eeb80310 con 0x5589eeaf52d0 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.927+0000 7ff883305f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589eeb830d0 con 0x5589eeb814b0 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:58998/0 (socket says 172.21.15.28:58998) 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 learned_addr learned my addr 172.21.15.28:0/12735371 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880e00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeb814b0 0x5589eeb836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeaf52d0 0x5589eeb7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeb814b0 msgr2=0x5589eeb836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5589eeb814b0 0x5589eeb836e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeaf52d0 msgr2=0x5589eeb7a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589eeaf52d0 0x5589eeb7a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589eead9620 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.928+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff870025d20 tx=0x7ff870002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.929+0000 7ff87d200640 1 -- 172.21.15.28:0/12735371 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870055070 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.929+0000 7ff87d200640 1 -- 172.21.15.28:0/12735371 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff870051770 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.929+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589eeadc080 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.929+0000 7ff87d200640 1 -- 172.21.15.28:0/12735371 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870051b60 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.929+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589eead92b0 con 0x5589eeaaf5a0 2024-09-10T10:53:11.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.931+0000 7ff87d200640 1 -- 172.21.15.28:0/12735371 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff870058070 con 0x5589eeaaf5a0 2024-09-10T10:53:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff87d200640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 0x7ff86006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff87d200640 1 -- 172.21.15.28:0/12735371 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7ff8700cdec0 con 0x5589eeaaf5a0 2024-09-10T10:53:11.935 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 0x5589eeaee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 0x7ff86006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eeaeee90 con 0x5589eeb7ab70 2024-09-10T10:53:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.932+0000 7ff880e00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 0x5589eeaee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.933+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 0x7ff86006ccd0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x5589eeab0080 tx=0x7ff86c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.933+0000 7ff880e00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 0x5589eeaee950 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.935+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v123'691 uv665 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.935+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eeaef2c0 con 0x5589eeb7ab70 2024-09-10T10:53:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.937+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v123'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.937+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec24880 con 0x5589eeb7ab70 2024-09-10T10:53:11.940 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.939+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v123'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.939+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec25c30 con 0x5589eeb7ab70 2024-09-10T10:53:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.941+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v123'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.941+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec25fd0 con 0x5589eeb7ab70 2024-09-10T10:53:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.943+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v123'695 uv695 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.943+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec24c20 con 0x5589eeb7ab70 2024-09-10T10:53:11.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.945+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v123'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.945+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec25010 con 0x5589eeb7ab70 2024-09-10T10:53:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.947+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v123'697 uv671 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.947+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253e0 con 0x5589eeb7ab70 2024-09-10T10:53:11.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.948+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v123'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.948+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253e0 con 0x5589eeb7ab70 2024-09-10T10:53:11.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:11.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.950+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v123'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.950+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.953+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v123'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.953+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.954+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v123'701 uv701 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.954+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.956+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v123'702 uv702 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.956+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.958+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v123'703 uv678 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.958+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.960+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v123'704 uv704 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.960+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.962 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:11.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.961+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v123'705 uv705 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.961+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.964+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v123'706 uv706 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.964+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.965+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v123'707 uv707 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.965+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec253b0 con 0x5589eeb7ab70 2024-09-10T10:53:11.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.968+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v123'708 uv708 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.968+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.970+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v123'709 uv709 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.970+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.971+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v123'710 uv684 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.971+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.973+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v123'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.973+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:11.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.975+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v123'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.975+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec27380 con 0x5589eeb7ab70 2024-09-10T10:53:11.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.976+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v123'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.977+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec27380 con 0x5589eeb7ab70 2024-09-10T10:53:11.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.978+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v123'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.978+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.981+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v123'715 uv715 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.981+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.982+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v123'716 uv690 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.982+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.983+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v123'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.983+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.986 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:11.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.985+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v123'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.985+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec27380 con 0x5589eeb7ab70 2024-09-10T10:53:11.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.987+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v123'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.987+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec27380 con 0x5589eeb7ab70 2024-09-10T10:53:11.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.988+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v123'720 uv720 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.988+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec26370 con 0x5589eeb7ab70 2024-09-10T10:53:11.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.990+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v123'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eeb7ab70 2024-09-10T10:53:11.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.990+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 0x5589eec28c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:11.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.990+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec291b0 con 0x5589eec26880 2024-09-10T10:53:11.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.990+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 0x5589eec28c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:11.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.991+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 0x5589eec28c70 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:11.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.993+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v123'505 uv488 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:11.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.993+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec29550 con 0x5589eec26880 2024-09-10T10:53:11.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.994+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v123'506 uv506 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:11.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.994+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec298f0 con 0x5589eec26880 2024-09-10T10:53:11.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.995+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v123'507 uv507 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:11.996 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.995+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec29c90 con 0x5589eec26880 2024-09-10T10:53:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.997+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v123'508 uv508 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.997+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a030 con 0x5589eec26880 2024-09-10T10:53:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.998+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v123'509 uv509 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:11.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:11.998+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a3d0 con 0x5589eec26880 2024-09-10T10:53:12.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.000+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v123'510 uv493 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.000+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.001+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v123'511 uv511 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.001+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.003+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v123'512 uv512 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.004 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:12.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.003+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.004+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v123'513 uv513 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.004+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.006+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v123'514 uv514 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.006+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.007+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v123'515 uv498 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.007+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.008+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v123'516 uv516 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.008+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.011 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:12.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.010+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v123'517 uv517 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.010+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2ee60 con 0x5589eec26880 2024-09-10T10:53:12.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.011+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v123'518 uv518 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.011+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2ee60 con 0x5589eec26880 2024-09-10T10:53:12.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.013+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v123'519 uv519 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.013+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.014+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v123'520 uv504 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.014+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.016+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v123'521 uv521 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.016+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.018 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:12.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.017+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v123'522 uv522 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.017+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2b780 con 0x5589eec26880 2024-09-10T10:53:12.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.019+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v123'523 uv523 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.019+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2b780 con 0x5589eec26880 2024-09-10T10:53:12.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.020+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v123'524 uv524 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.020+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2a770 con 0x5589eec26880 2024-09-10T10:53:12.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.021+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v123'525 uv525 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec26880 2024-09-10T10:53:12.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.021+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5589eec2adf0 0x5589eec2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.021+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2d720 con 0x5589eec2adf0 2024-09-10T10:53:12.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.022+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5589eec2adf0 0x5589eec2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.022+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5589eec2adf0 0x5589eec2d1e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.024+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.024+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2dac0 con 0x5589eec2adf0 2024-09-10T10:53:12.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.025+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.025+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2df70 con 0x5589eec2adf0 2024-09-10T10:53:12.027 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:12.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.027+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.027+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2f820 con 0x5589eec2adf0 2024-09-10T10:53:12.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.028+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.028+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2fbc0 con 0x5589eec2adf0 2024-09-10T10:53:12.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.029+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.029+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec2e810 con 0x5589eec2adf0 2024-09-10T10:53:12.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.031+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v123'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.031+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5589eec2f130 0x5589eec31f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.031+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec324b0 con 0x5589eec2f130 2024-09-10T10:53:12.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.031+0000 7ff880e00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5589eec2f130 0x5589eec31f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.031+0000 7ff880e00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5589eec2f130 0x5589eec31f70 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.033+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v123'99 uv98 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.033+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec324b0 con 0x5589eec2f130 2024-09-10T10:53:12.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.034+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v123'100 uv100 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.034+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec324b0 con 0x5589eec2f130 2024-09-10T10:53:12.036 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:12.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.035+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v123'101 uv101 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.035+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec328a0 con 0x5589eec2f130 2024-09-10T10:53:12.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.036+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v123'102 uv102 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.036+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec328a0 con 0x5589eec2adf0 2024-09-10T10:53:12.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.037+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.037+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.039+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.039+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.041 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:12.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.040+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.040+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec33c50 con 0x5589eec2adf0 2024-09-10T10:53:12.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.041+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.041+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec33c50 con 0x5589eec2adf0 2024-09-10T10:53:12.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.042+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.042+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.043+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v123'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.044+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.045+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v123'249 uv243 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.045+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.046+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v123'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.046+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.048 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:12.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.048+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v123'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.048+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec33c50 con 0x5589eec2f130 2024-09-10T10:53:12.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.050+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v123'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.050+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec33c50 con 0x5589eec2f130 2024-09-10T10:53:12.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.051+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v123'253 uv253 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.051+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.052+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v123'254 uv254 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.052+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.054+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v123'255 uv248 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.054+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.055+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v123'256 uv256 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.055+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.057 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:12.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.056+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v123'257 uv257 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.056+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.058+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v123'258 uv258 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.058+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2f130 2024-09-10T10:53:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.059+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v123'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.059+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.061+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v123'112 uv111 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.061+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.062+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v123'113 uv113 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.062+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.064 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.063+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v123'114 uv114 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.063+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.064+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v123'115 uv115 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.065+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec32c40 con 0x5589eec2adf0 2024-09-10T10:53:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.066+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v123'116 uv116 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec2adf0 2024-09-10T10:53:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.066+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5589eec332c0 0x5589eec356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.066+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec35cc0 con 0x5589eec332c0 2024-09-10T10:53:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.066+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5589eec332c0 0x5589eec356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.066+0000 7ff880400640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5589eec332c0 0x5589eec356b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.068+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.068+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec35d80 con 0x5589eec332c0 2024-09-10T10:53:12.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.069+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.069+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.071 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:12.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.070+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.070+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec371b0 con 0x5589eec332c0 2024-09-10T10:53:12.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.072+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.072+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec371b0 con 0x5589eec332c0 2024-09-10T10:53:12.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.073+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.073+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.074+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v123'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.074+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.076+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v123'223 uv217 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.076+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.077+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v123'224 uv224 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.077+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.080 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.079+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v123'225 uv225 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.079+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.080+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v123'226 uv226 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.080+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.082+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v123'227 uv227 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.082+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.084+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v123'228 uv222 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.084+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.085+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v123'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.085+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.087+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v123'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.088 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:12.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.087+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec361a0 con 0x5589eec332c0 2024-09-10T10:53:12.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.088+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v123'231 uv231 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.088+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f90 con 0x5589eec332c0 2024-09-10T10:53:12.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.090+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v123'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.090+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.092+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v123'99 uv98 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.092+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.093+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v123'100 uv100 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.093+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.095 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:12.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.094+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v123'101 uv101 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.094+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.096+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v123'102 uv102 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.096+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.097+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v123'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.097+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.098+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v123'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.098+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.100+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v123'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.101 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:12.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.100+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f50 con 0x5589eec2f130 2024-09-10T10:53:12.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.101+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v123'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.101+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f30 con 0x5589eec2f130 2024-09-10T10:53:12.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.102+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v123'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.102+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38f10 con 0x5589eec2f130 2024-09-10T10:53:12.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.103+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v123'140 uv140 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff878011040 con 0x5589eec2f130 2024-09-10T10:53:12.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.103+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5589eec38f10 0x5589eec39650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.103+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec38f10 2024-09-10T10:53:12.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.104+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5589eec38f10 0x5589eec39650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.104+0000 7ff87fa00640 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5589eec38f10 0x5589eec39650 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.105+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v123'115 uv114 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec38f10 2024-09-10T10:53:12.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.105+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec38f10 2024-09-10T10:53:12.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.106+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v123'116 uv116 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec38f10 2024-09-10T10:53:12.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.106+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec38f10 2024-09-10T10:53:12.108 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:12.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.107+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v123'117 uv117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec38f10 2024-09-10T10:53:12.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.107+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec38f10 2024-09-10T10:53:12.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.109+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v123'118 uv118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec38f10 2024-09-10T10:53:12.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.109+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec38f10 2024-09-10T10:53:12.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.110+0000 7ff87fa00640 1 -- 172.21.15.28:0/12735371 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v123'119 uv119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff86c00f760 con 0x5589eec38f10 2024-09-10T10:53:12.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.110+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.111+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v123'278 uv277 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.111+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.113+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v123'279 uv279 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.113+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.114 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:12.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.114+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v123'280 uv280 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.114+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.115+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v123'281 uv281 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.115+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.116+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v123'282 uv282 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.116+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.118+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v123'283 uv276 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.118+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.119+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v123'284 uv284 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.119+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.121 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:12.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.120+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v123'285 uv285 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.120+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.121+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v123'286 uv286 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.121+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.123+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v123'287 uv287 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.123+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec38ab0 con 0x5589eec332c0 2024-09-10T10:53:12.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.124+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v123'288 uv288 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.124+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5589eec3a060 con 0x5589eec332c0 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff880400640 1 -- 172.21.15.28:0/12735371 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v123'289 uv289 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff87006b020 con 0x5589eec332c0 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5589eec2adf0 msgr2=0x5589eec2d1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5589eec2adf0 0x5589eec2d1e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5589eec332c0 msgr2=0x5589eec356b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5589eec332c0 0x5589eec356b0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5589eec38f10 msgr2=0x5589eec39650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5589eec38f10 0x5589eec39650 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5589eec2f130 msgr2=0x5589eec31f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5589eec2f130 0x5589eec31f70 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 msgr2=0x5589eec28c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 0x5589eec28c70 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 reap_dead start 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff880e00640 1 -- 172.21.15.28:0/12735371 reap_dead start 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 msgr2=0x5589eeaee950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 0x5589eeaee950 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 msgr2=0x7ff86006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 0x7ff86006ccd0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x5589eeab0080 tx=0x7ff86c009000 comp rx=0 tx=0).stop 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 msgr2=0x5589eeb7a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.125+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff870025d20 tx=0x7ff870002fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 shutdown_connections 2024-09-10T10:53:12.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5589eec26880 0x5589eec28c70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5589eeb7ab70 0x5589eeaee950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff86006a810 0x7ff86006ccd0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 --2- 172.21.15.28:0/12735371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5589eeaaf5a0 0x5589eeb7a0d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 >> 172.21.15.28:0/12735371 conn(0x5589eead9b90 msgr2=0x5589eead9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 shutdown_connections 2024-09-10T10:53:12.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.126+0000 7ff883305f40 1 -- 172.21.15.28:0/12735371 wait complete. 2024-09-10T10:53:12.131 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 1.00000000.inode 0 2024-09-10T10:53:12.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.250+0000 7fb0f5748f40 1 Processor -- start 2024-09-10T10:53:12.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 -- start start 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3b7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3b5edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557ad3b76530 con 0x557ad3c049c0 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ad3ad3b30 con 0x557ad3c013e0 2024-09-10T10:53:12.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.251+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557ad3c03ee0 con 0x557ad3b758e0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59026/0 (socket says 172.21.15.28:59026) 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 -- 172.21.15.28:0/1986516614 learned_addr learned my addr 172.21.15.28:0/1986516614 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f2000640 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3b5edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3b7a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 -- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 msgr2=0x557ad3b7a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3b7a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 -- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 msgr2=0x557ad3b5edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3b5edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 -- 172.21.15.28:0/1986516614 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad3b74090 con 0x557ad3c049c0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.252+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec001660 tx=0x7fb0ec001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209c14ac4bb2a277 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f0c00640 1 -- 172.21.15.28:0/1986516614 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003650 con 0x557ad3c049c0 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f0c00640 1 -- 172.21.15.28:0/1986516614 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec003830 con 0x557ad3c049c0 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f0c00640 1 -- 172.21.15.28:0/1986516614 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec003ca0 con 0x557ad3c049c0 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1986516614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 msgr2=0x557ad3c01850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec001660 tx=0x7fb0ec001b20 comp rx=0 tx=0).stop 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1986516614 shutdown_connections 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c01850 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3b5edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1986516614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3b7a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1986516614 >> 172.21.15.28:0/1986516614 conn(0x557ad3b7a0a0 msgr2=0x557ad3b16f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1986516614 shutdown_connections 2024-09-10T10:53:12.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.253+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1986516614 wait complete. 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.254+0000 7fb0f5748f40 1 Processor -- start 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.254+0000 7fb0f5748f40 1 -- start start 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3c01a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3c9daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c9dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x557ad3b74740 con 0x557ad3c049c0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ad3b74330 con 0x557ad3c013e0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f5748f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557ad3b5f300 con 0x557ad3b758e0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c9dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c9dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59034/0 (socket says 172.21.15.28:59034) 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f3400640 1 -- 172.21.15.28:0/1885531877 learned_addr learned my addr 172.21.15.28:0/1885531877 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3c01a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.255+0000 7fb0f2000640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3c9daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 -- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 msgr2=0x557ad3c01a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x557ad3b758e0 0x557ad3c01a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 -- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 msgr2=0x557ad3c9daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ad3c013e0 0x557ad3c9daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 -- 172.21.15.28:0/1885531877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ad3b5ea10 con 0x557ad3c049c0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f3400640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c9dfe0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec001660 tx=0x7fb0ec001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0eb400640 1 -- 172.21.15.28:0/1885531877 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec024070 con 0x557ad3c049c0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0eb400640 1 -- 172.21.15.28:0/1885531877 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec00a040 con 0x557ad3c049c0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0eb400640 1 -- 172.21.15.28:0/1885531877 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec013a70 con 0x557ad3c049c0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ad3bf3220 con 0x557ad3c049c0 2024-09-10T10:53:12.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.256+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ad3bf34d0 con 0x557ad3c049c0 2024-09-10T10:53:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.258+0000 7fb0eb400640 1 -- 172.21.15.28:0/1885531877 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb0ec010070 con 0x557ad3c049c0 2024-09-10T10:53:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.259+0000 7fb0eb400640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406a810 0x7fb0d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.259+0000 7fb0eb400640 1 -- 172.21.15.28:0/1885531877 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7fb0ec08b610 con 0x557ad3c049c0 2024-09-10T10:53:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.259+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557ad3c01f80 0x557ad3bf6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.259+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x557ad3bf3d20 con 0x557ad3c01f80 2024-09-10T10:53:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.259+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406a810 0x7fb0d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.260+0000 7fb0f2000640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557ad3c01f80 0x557ad3bf6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.260+0000 7fb0f2a00640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406a810 0x7fb0d406ccd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x557ad3c032b0 tx=0x7fb0e400c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.260+0000 7fb0f2000640 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557ad3c01f80 0x557ad3bf6ac0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f2000640 1 -- 172.21.15.28:0/1885531877 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 1.00000000.inode [truncate 0] v123'526 uv526 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb0e0057030 con 0x557ad3c01f80 2024-09-10T10:53:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557ad3c01f80 msgr2=0x557ad3bf6ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x557ad3c01f80 0x557ad3bf6ac0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406a810 msgr2=0x7fb0d406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb0d406a810 0x7fb0d406ccd0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x557ad3c032b0 tx=0x7fb0e400c040 comp rx=0 tx=0).stop 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 msgr2=0x557ad3c9dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 --2- 172.21.15.28:0/1885531877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557ad3c049c0 0x557ad3c9dfe0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec001660 tx=0x7fb0ec001250 comp rx=0 tx=0).stop 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f3400640 1 -- 172.21.15.28:0/1885531877 reap_dead start 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 shutdown_connections 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 >> 172.21.15.28:0/1885531877 conn(0x557ad3b7a0a0 msgr2=0x557ad3b16f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 shutdown_connections 2024-09-10T10:53:12.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.262+0000 7fb0f5748f40 1 -- 172.21.15.28:0/1885531877 wait complete. 2024-09-10T10:53:12.268 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:12.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.425+0000 7f12f0e00640 1 Processor -- start 2024-09-10T10:53:12.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.425+0000 7f12f0e00640 1 -- start start 2024-09-10T10:53:12.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 0x7f12ec0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12ec072050 0x7f12ec072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f12ec0b4620 con 0x7f12ec072050 2024-09-10T10:53:12.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f12ec0ce100 con 0x7f12ec0b4a20 2024-09-10T10:53:12.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.426+0000 7f12f0e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12ec0ce3a0 con 0x7f12ec15e900 2024-09-10T10:53:12.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12ec072050 0x7f12ec072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f12ec072050 0x7f12ec072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59042/0 (socket says 172.21.15.28:59042) 2024-09-10T10:53:12.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 -- 172.21.15.28:0/4077668203 learned_addr learned my addr 172.21.15.28:0/4077668203 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12e9e00640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12e1400640 1 --2- 172.21.15.28:0/4077668203 >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 0x7f12ec0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 -- 172.21.15.28:0/4077668203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 msgr2=0x7f12ec0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec0b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 -- 172.21.15.28:0/4077668203 >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 msgr2=0x7f12ec0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 --2- 172.21.15.28:0/4077668203 >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 0x7f12ec0ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12ea800640 1 -- 172.21.15.28:0/4077668203 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec0ce680 con 0x7f12ec072050 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12e9e00640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:12.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.427+0000 7f12e1400640 1 --2- 172.21.15.28:0/4077668203 >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 0x7f12ec0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:12.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.428+0000 7f12ea800640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec072050 0x7f12ec072500 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0cfe80 tx=0x7f12e402a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd4a7d7c75e8943 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.428+0000 7f12e9400640 1 -- 172.21.15.28:0/4077668203 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4036070 con 0x7f12ec072050 2024-09-10T10:53:12.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.428+0000 7f12e9400640 1 -- 172.21.15.28:0/4077668203 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e4031600 con 0x7f12ec072050 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.428+0000 7f12e9400640 1 -- 172.21.15.28:0/4077668203 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40319d0 con 0x7f12ec072050 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 -- 172.21.15.28:0/4077668203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec072050 msgr2=0x7f12ec072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec072050 0x7f12ec072500 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0cfe80 tx=0x7f12e402a600 comp rx=0 tx=0).stop 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 -- 172.21.15.28:0/4077668203 shutdown_connections 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec072050 0x7f12ec072500 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 --2- 172.21.15.28:0/4077668203 >> v2:172.21.15.138:3300/0 conn(0x7f12ec0b4a20 0x7f12ec0ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 --2- 172.21.15.28:0/4077668203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 -- 172.21.15.28:0/4077668203 >> 172.21.15.28:0/4077668203 conn(0x7f12ec06bdf0 msgr2=0x7f12ec06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 -- 172.21.15.28:0/4077668203 shutdown_connections 2024-09-10T10:53:12.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.429+0000 7f12f0e00640 1 -- 172.21.15.28:0/4077668203 wait complete. 2024-09-10T10:53:12.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.431+0000 7f12f0e00640 1 Processor -- start 2024-09-10T10:53:12.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.431+0000 7f12f0e00640 1 -- start start 2024-09-10T10:53:12.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12ec072050 0x7f12ec1f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec0ad560 con 0x7f12ec0b4a20 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12ec0b4e20 con 0x7f12ec072050 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12f0e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f12ec0cee80 con 0x7f12ec15e900 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59046/0 (socket says 172.21.15.28:59046) 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 -- 172.21.15.28:0/2288017535 learned_addr learned my addr 172.21.15.28:0/2288017535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e9e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12ec072050 0x7f12ec1f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12ea800640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 -- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 msgr2=0x7f12ec155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 -- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12ec072050 msgr2=0x7f12ec1f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12ec072050 0x7f12ec1f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.432+0000 7f12e1400640 1 -- 172.21.15.28:0/2288017535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ec155ed0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12e1400640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f12d400e810 tx=0x7f12d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4017070 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d400c6d0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400f040 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ec149150 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.433+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ec149710 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.434+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ec0ad020 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.435+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f12d401a070 con 0x7f12ec0b4a20 2024-09-10T10:53:12.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.436+0000 7f12e2a00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 0x7f12c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:12.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.436+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8282+0+0 (secure 0 0 0) 0x7f12d408b4d0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.436+0000 7f12e9e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 0x7f12c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:12.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.437+0000 7f12e9e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 0x7f12c806e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12dc006fd0 tx=0x7f12dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:12.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.440+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12d40593c0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.660+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f12ec0d05d0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.828+0000 7f12e2a00640 1 -- 172.21.15.28:0/2288017535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v221) ==== 162+0+0 (secure 0 0 0) 0x7f12d4058cc0 con 0x7f12ec0b4a20 2024-09-10T10:53:12.830 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.830+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 msgr2=0x7f12c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.830+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 0x7f12c806e2f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12dc006fd0 tx=0x7f12dc008040 comp rx=0 tx=0).stop 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.830+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 msgr2=0x7f12ec155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.830+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f12d400e810 tx=0x7f12d400ecd0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 shutdown_connections 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f12c806be30 0x7f12c806e2f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f12ec15e900 0x7f12ec155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f12ec0b4a20 0x7f12ec155450 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 --2- 172.21.15.28:0/2288017535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12ec072050 0x7f12ec1f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:12.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 >> 172.21.15.28:0/2288017535 conn(0x7f12ec06bdf0 msgr2=0x7f12ec06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:12.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 shutdown_connections 2024-09-10T10:53:12.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:12.831+0000 7f12f0e00640 1 -- 172.21.15.28:0/2288017535 wait complete. 2024-09-10T10:53:12.842 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:12.840+0000 7f3563000640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T10:53:12.842 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:12.840+0000 7f3563000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:53:12.843 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:12.841+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:12.843 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:12.841+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:12.852 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:13.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.005+0000 7f97f7e00640 1 Processor -- start 2024-09-10T10:53:13.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.005+0000 7f97f7e00640 1 -- start start 2024-09-10T10:53:13.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072320 0x7f97f00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97f0072de0 0x7f97f006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f0146140 con 0x7f97f0072700 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00cd740 con 0x7f97f0072de0 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00cda20 con 0x7f97f0072320 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f97f0072700 0x7f97f006e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47268/0 (socket says 172.21.15.28:47268) 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f5e00640 1 -- 172.21.15.28:0/3057487587 learned_addr learned my addr 172.21.15.28:0/3057487587 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f4a00640 1 --2- 172.21.15.28:0/3057487587 >> v2:172.21.15.138:3300/0 conn(0x7f97f0072de0 0x7f97f006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97f5400640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072320 0x7f97f00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97efe00640 1 -- 172.21.15.28:0/3057487587 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 877447722 0 0) 0x7f97f0146140 con 0x7f97f0072700 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.007+0000 7f97efe00640 1 -- 172.21.15.28:0/3057487587 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97d8003520 con 0x7f97f0072700 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.008+0000 7f97f4a00640 1 -- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072320 msgr2=0x7f97f00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.008+0000 7f97f4a00640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072320 0x7f97f00cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.008+0000 7f97f4a00640 1 -- 172.21.15.28:0/3057487587 >> v1:172.21.15.28:6789/0 conn(0x7f97f0072700 legacy=0x7f97f006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.008+0000 7f97f4a00640 1 -- 172.21.15.28:0/3057487587 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00cdd00 con 0x7f97f0072de0 2024-09-10T10:53:13.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.009+0000 7f97f4a00640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f006e0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0097a0 tx=0x7f97dc02a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d945cacf84f83dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.009+0000 7f97efe00640 1 -- 172.21.15.28:0/3057487587 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc037070 con 0x7f97f0072de0 2024-09-10T10:53:13.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.009+0000 7f97efe00640 1 -- 172.21.15.28:0/3057487587 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc02ae50 con 0x7f97f0072de0 2024-09-10T10:53:13.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.009+0000 7f97efe00640 1 -- 172.21.15.28:0/3057487587 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc032660 con 0x7f97f0072de0 2024-09-10T10:53:13.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 -- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 msgr2=0x7f97f006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f006e0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0097a0 tx=0x7f97dc02a100 comp rx=0 tx=0).stop 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 -- 172.21.15.28:0/3057487587 shutdown_connections 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f006e0b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 --2- 172.21.15.28:0/3057487587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072320 0x7f97f00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 -- 172.21.15.28:0/3057487587 >> 172.21.15.28:0/3057487587 conn(0x7f97f0067d70 msgr2=0x7f97f006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 -- 172.21.15.28:0/3057487587 shutdown_connections 2024-09-10T10:53:13.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.010+0000 7f97f7e00640 1 -- 172.21.15.28:0/3057487587 wait complete. 2024-09-10T10:53:13.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.011+0000 7f97f7e00640 1 Processor -- start 2024-09-10T10:53:13.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 -- start start 2024-09-10T10:53:13.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 0x7f97f00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f00cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00cd520 con 0x7f97f0072320 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.012+0000 7f97f7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97f00cd3a0 con 0x7f97f0072de0 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97f006ed00 con 0x7f97f0072700 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59060/0 (socket says 172.21.15.28:59060) 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 -- 172.21.15.28:0/2705890383 learned_addr learned my addr 172.21.15.28:0/2705890383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f4a00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 0x7f97f00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f00cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 -- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 msgr2=0x7f97f00caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 0x7f97f00caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 -- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 msgr2=0x7f97f00cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f00cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5400640 1 -- 172.21.15.28:0/2705890383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00b03b0 con 0x7f97f0072320 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f5e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f00cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.013+0000 7f97f4a00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 0x7f97f00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:13.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97f5400640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e9b0 tx=0x7f97e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4017070 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e400c9a0 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f00b0690 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e400cd90 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.014+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f00b0c50 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.015+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f00cce60 con 0x7f97f0072320 2024-09-10T10:53:13.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.016+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f97e401a070 con 0x7f97f0072320 2024-09-10T10:53:13.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.017+0000 7f97ed600640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 0x7f97d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.017+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f97e408bd50 con 0x7f97f0072320 2024-09-10T10:53:13.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.017+0000 7f97f4a00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 0x7f97d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.018+0000 7f97f4a00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 0x7f97d006e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.021+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f97e4059c10 con 0x7f97f0072320 2024-09-10T10:53:13.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.239+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f006d090 con 0x7f97f0072320 2024-09-10T10:53:13.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.239+0000 7f97ed600640 1 -- 172.21.15.28:0/2705890383 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) ==== 95+0+3566 (secure 0 0 0) 0x7f97e4059510 con 0x7f97f0072320 2024-09-10T10:53:13.242 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:13.242 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":223,"btime":"2024-09-10T10:53:12:843085+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:12.843026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:13.242 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 223 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 msgr2=0x7f97d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 0x7f97d006e2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f97dc034000 tx=0x7f97dc02e000 comp rx=0 tx=0).stop 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 msgr2=0x7f97f015bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f97e400e9b0 tx=0x7f97e400ee70 comp rx=0 tx=0).stop 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 shutdown_connections 2024-09-10T10:53:13.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f97d006be30 0x7f97d006e2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97f0072de0 0x7f97f00cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97f0072700 0x7f97f00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 --2- 172.21.15.28:0/2705890383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97f0072320 0x7f97f015bc70 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 >> 172.21.15.28:0/2705890383 conn(0x7f97f0067d70 msgr2=0x7f97f01449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 shutdown_connections 2024-09-10T10:53:13.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.242+0000 7f97f7e00640 1 -- 172.21.15.28:0/2705890383 wait complete. 2024-09-10T10:53:13.260 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:13.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:13.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.416+0000 7f3222000640 1 Processor -- start 2024-09-10T10:53:13.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.416+0000 7f3222000640 1 -- start start 2024-09-10T10:53:13.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072320 0x7f321c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f321c072de0 0x7f321c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f321c072700 0x7f321c06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f321c146140 con 0x7f321c072de0 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f321c0cd740 con 0x7f321c072700 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3222000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f321c0cda20 con 0x7f321c072320 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f321c072de0 0x7f321c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f321c072de0 0x7f321c06e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59062/0 (socket says 172.21.15.28:59062) 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 -- 172.21.15.28:0/1194193401 learned_addr learned my addr 172.21.15.28:0/1194193401 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f321b400640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072320 0x7f321c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f321be00640 1 --2- 172.21.15.28:0/1194193401 >> v2:172.21.15.138:3300/0 conn(0x7f321c072700 0x7f321c06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 -- 172.21.15.28:0/1194193401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072320 msgr2=0x7f321c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072320 0x7f321c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 -- 172.21.15.28:0/1194193401 >> v2:172.21.15.138:3300/0 conn(0x7f321c072700 msgr2=0x7f321c06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 --2- 172.21.15.28:0/1194193401 >> v2:172.21.15.138:3300/0 conn(0x7f321c072700 0x7f321c06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.418+0000 7f3213e00640 1 -- 172.21.15.28:0/1194193401 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c0cdd00 con 0x7f321c072de0 2024-09-10T10:53:13.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.419+0000 7f3213e00640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c06e0b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f321c06f4e0 tx=0x7f320402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb55f0f9013c050 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f321aa00640 1 -- 172.21.15.28:0/1194193401 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204037070 con 0x7f321c072de0 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f321aa00640 1 -- 172.21.15.28:0/1194193401 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3204032440 con 0x7f321c072de0 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f321aa00640 1 -- 172.21.15.28:0/1194193401 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204032810 con 0x7f321c072de0 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 -- 172.21.15.28:0/1194193401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 msgr2=0x7f321c06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c06e0b0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f321c06f4e0 tx=0x7f320402a140 comp rx=0 tx=0).stop 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 -- 172.21.15.28:0/1194193401 shutdown_connections 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 --2- 172.21.15.28:0/1194193401 >> v2:172.21.15.138:3300/0 conn(0x7f321c072700 0x7f321c06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c06e0b0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 --2- 172.21.15.28:0/1194193401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072320 0x7f321c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 -- 172.21.15.28:0/1194193401 >> 172.21.15.28:0/1194193401 conn(0x7f321c067d70 msgr2=0x7f321c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 -- 172.21.15.28:0/1194193401 shutdown_connections 2024-09-10T10:53:13.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.420+0000 7f3222000640 1 -- 172.21.15.28:0/1194193401 wait complete. 2024-09-10T10:53:13.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.422+0000 7f3222000640 1 Processor -- start 2024-09-10T10:53:13.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.422+0000 7f3222000640 1 -- start start 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 0x7f321c15bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 0x7f321c137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cd520 con 0x7f321c072de0 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f321c0cd3a0 con 0x7f321c072320 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3222000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f321c06eb30 con 0x7f321c072700 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f321be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f321be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:59078/0 (socket says 172.21.15.28:59078) 2024-09-10T10:53:13.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f321be00640 1 -- 172.21.15.28:0/3801437318 learned_addr learned my addr 172.21.15.28:0/3801437318 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f321b400640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 0x7f321c15bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.423+0000 7f3213e00640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 0x7f321c137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 -- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 msgr2=0x7f321c137dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 0x7f321c137dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 -- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 msgr2=0x7f321c15bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 0x7f321c15bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 -- 172.21.15.28:0/3801437318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321c138860 con 0x7f321c072de0 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321b400640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 0x7f321c15bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3213e00640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 0x7f321c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f321be00640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3214007e70 tx=0x7f321400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214014620 con 0x7f321c072de0 2024-09-10T10:53:13.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214019070 con 0x7f321c072de0 2024-09-10T10:53:13.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214014a20 con 0x7f321c072de0 2024-09-10T10:53:13.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321c138b40 con 0x7f321c072de0 2024-09-10T10:53:13.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.424+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321c139100 con 0x7f321c072de0 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.425+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321c0cce60 con 0x7f321c072de0 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.426+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3214014bc0 con 0x7f321c072de0 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.427+0000 7f3213400640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 0x7f31f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.427+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f321408c260 con 0x7f321c072de0 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.428+0000 7f321b400640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 0x7f31f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.429+0000 7f321b400640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 0x7f31f806e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f320c006fd0 tx=0x7f320c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.432+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f321405a160 con 0x7f321c072de0 2024-09-10T10:53:13.560 INFO:tasks.ceph.mds.a.smithi028.stderr: -46> 2024-09-10T10:53:08.807+0000 7f94d7600640 -1 mds.0.cache.ino(0x100) Corrupt inode 0x100: End of buffer [buffer:2] 2024-09-10T10:53:13.561 INFO:tasks.ceph.mds.a.smithi028.stderr: -36> 2024-09-10T10:53:08.807+0000 7f94d7600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:53:13.599 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:13.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.654+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321c0ce920 con 0x7f321c072de0 2024-09-10T10:53:13.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.654+0000 7f3213400640 1 -- 172.21.15.28:0/3801437318 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) ==== 95+0+3566 (secure 0 0 0) 0x7f3214059a60 con 0x7f321c072de0 2024-09-10T10:53:13.657 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:13.657 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":223,"btime":"2024-09-10T10:53:12:843085+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:12.843026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:13.657 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 223 2024-09-10T10:53:13.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 msgr2=0x7f31f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 0x7f31f806e2f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f320c006fd0 tx=0x7f320c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 msgr2=0x7f321c139e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3214007e70 tx=0x7f321400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 shutdown_connections 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31f806be30 0x7f31f806e2f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f321c072de0 0x7f321c139e50 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f321c072700 0x7f321c137dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 --2- 172.21.15.28:0/3801437318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f321c072320 0x7f321c15bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 >> 172.21.15.28:0/3801437318 conn(0x7f321c067d70 msgr2=0x7f321c0c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 shutdown_connections 2024-09-10T10:53:13.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.656+0000 7f3222000640 1 -- 172.21.15.28:0/3801437318 wait complete. 2024-09-10T10:53:13.675 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:13.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:13.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.829+0000 7fca70c00640 1 Processor -- start 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.829+0000 7fca70c00640 1 -- start start 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.830+0000 7fca70c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 0x7fca6c0d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.830+0000 7fca70c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca6c09b690 0x7fca6c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.830+0000 7fca70c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca6c06e020 con 0x7fca6c06d4c0 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.830+0000 7fca70c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c15eb20 con 0x7fca6c09b690 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.830+0000 7fca70c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c06d260 con 0x7fca6c15e740 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca6c09b690 0x7fca6c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca6c09b690 0x7fca6c06cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56430/0 (socket says 172.21.15.28:56430) 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 -- 172.21.15.28:0/3016590446 learned_addr learned my addr 172.21.15.28:0/3016590446 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:13.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69e00640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 0x7fca6c0d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 -- 172.21.15.28:0/3016590446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 msgr2=0x7fca6c0d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 0x7fca6c0d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 -- 172.21.15.28:0/3016590446 >> v1:172.21.15.28:6789/0 conn(0x7fca6c06d4c0 legacy=0x7fca6c06d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69400640 1 -- 172.21.15.28:0/3016590446 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c0d0780 con 0x7fca6c09b690 2024-09-10T10:53:13.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.831+0000 7fca69e00640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 0x7fca6c0d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.832+0000 7fca69400640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c09b690 0x7fca6c06cd20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fca580098a0 tx=0x7fca5802a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e1f32b1ed33fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.833+0000 7fca68a00640 1 -- 172.21.15.28:0/3016590446 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58037070 con 0x7fca6c09b690 2024-09-10T10:53:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.833+0000 7fca68a00640 1 -- 172.21.15.28:0/3016590446 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca58032640 con 0x7fca6c09b690 2024-09-10T10:53:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.833+0000 7fca68a00640 1 -- 172.21.15.28:0/3016590446 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58032a10 con 0x7fca6c09b690 2024-09-10T10:53:13.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.833+0000 7fca70c00640 1 -- 172.21.15.28:0/3016590446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c09b690 msgr2=0x7fca6c06cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.833+0000 7fca70c00640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c09b690 0x7fca6c06cd20 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fca580098a0 tx=0x7fca5802a320 comp rx=0 tx=0).stop 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 -- 172.21.15.28:0/3016590446 shutdown_connections 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c09b690 0x7fca6c06cd20 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 --2- 172.21.15.28:0/3016590446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15e740 0x7fca6c0d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 -- 172.21.15.28:0/3016590446 >> 172.21.15.28:0/3016590446 conn(0x7fca6c0d23a0 msgr2=0x7fca6c0d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 -- 172.21.15.28:0/3016590446 shutdown_connections 2024-09-10T10:53:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.834+0000 7fca70c00640 1 -- 172.21.15.28:0/3016590446 wait complete. 2024-09-10T10:53:13.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.835+0000 7fca70c00640 1 Processor -- start 2024-09-10T10:53:13.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.835+0000 7fca70c00640 1 -- start start 2024-09-10T10:53:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c09b690 0x7fca6c0b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 0x7fca6c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0d1020 con 0x7fca6c15e740 2024-09-10T10:53:13.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0d0d40 con 0x7fca6c06d4c0 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.836+0000 7fca70c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c09ba90 con 0x7fca6c09b690 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 0x7fca6c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca69e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca69e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:34434/0 (socket says 172.21.15.28:34434) 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca69e00640 1 -- 172.21.15.28:0/3416757574 learned_addr learned my addr 172.21.15.28:0/3416757574 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca69400640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c09b690 0x7fca6c0b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 -- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c09b690 msgr2=0x7fca6c0b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c09b690 0x7fca6c0b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 -- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 msgr2=0x7fca6c1f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca6a800640 1 -- 172.21.15.28:0/3416757574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c0b5920 con 0x7fca6c15e740 2024-09-10T10:53:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.837+0000 7fca69e00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca6a800640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 0x7fca6c15a9a0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c700 tx=0x7fca6400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64003b70 con 0x7fca6c15e740 2024-09-10T10:53:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca64003d10 con 0x7fca6c15e740 2024-09-10T10:53:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64005310 con 0x7fca6c15e740 2024-09-10T10:53:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c0b5c00 con 0x7fca6c15e740 2024-09-10T10:53:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.838+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0b61c0 con 0x7fca6c15e740 2024-09-10T10:53:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.839+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c06cd20 con 0x7fca6c15e740 2024-09-10T10:53:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.840+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fca640054b0 con 0x7fca6c15e740 2024-09-10T10:53:13.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.841+0000 7fca62000640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:13.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.841+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7fca6408b590 con 0x7fca6c15e740 2024-09-10T10:53:13.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.842+0000 7fca69e00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:13.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.843+0000 7fca69e00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006d40 tx=0x7fca5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:13.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:13.846+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca64058e00 con 0x7fca6c15e740 2024-09-10T10:53:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.064+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca6c0d1cb0 con 0x7fca6c15e740 2024-09-10T10:53:14.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.064+0000 7fca62000640 1 -- 172.21.15.28:0/3416757574 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 224 v224) ==== 95+0+4392 (secure 0 0 0) 0x7fca6405d100 con 0x7fca6c15e740 2024-09-10T10:53:14.066 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:14.067 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":224,"btime":"2024-09-10T10:53:13:847505+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9054,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1093900039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1093900039},{"type":"v1","addr":"172.21.15.28:6835","nonce":1093900039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":224}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:12.843026+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:14.067 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 224 2024-09-10T10:53:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 msgr2=0x7fca4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006d40 tx=0x7fca5c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 msgr2=0x7fca6c15a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 0x7fca6c15a9a0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fca6400c700 tx=0x7fca6400cbc0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 shutdown_connections 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fca4c06be30 0x7fca4c06e2f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca6c15e740 0x7fca6c15a9a0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c09b690 0x7fca6c0b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 --2- 172.21.15.28:0/3416757574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c06d4c0 0x7fca6c1f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 >> 172.21.15.28:0/3416757574 conn(0x7fca6c0d23a0 msgr2=0x7fca6c0b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 shutdown_connections 2024-09-10T10:53:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.067+0000 7fca70c00640 1 -- 172.21.15.28:0/3416757574 wait complete. 2024-09-10T10:53:14.085 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 1.00000000.inode' led to DAMAGED state 2024-09-10T10:53:14.085 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000000' 2024-09-10T10:53:14.085 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:14.085 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:14.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:14.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:14.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:14.121 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:14.197 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:14.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:14.198 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.392+0000 7f4b85800640 1 Processor -- start 2024-09-10T10:53:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.392+0000 7f4b85800640 1 -- start start 2024-09-10T10:53:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4b800d0dd0 0x7f4b800d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b8015e660 0x7f4b8006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4b800b37d0 con 0x7f4b800d0dd0 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b800cf330 con 0x7f4b8015e660 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b85800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b800cf000 con 0x7f4b800d1f90 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b7fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4b800d0dd0 0x7f4b800d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b7fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4b800d0dd0 0x7f4b800d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56446/0 (socket says 172.21.15.28:56446) 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.393+0000 7f4b7fe00640 1 -- 172.21.15.28:0/1472060941 learned_addr learned my addr 172.21.15.28:0/1472060941 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b84e00640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7f400640 1 --2- 172.21.15.28:0/1472060941 >> v2:172.21.15.138:3300/0 conn(0x7f4b8015e660 0x7f4b8006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7fe00640 1 -- 172.21.15.28:0/1472060941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 msgr2=0x7f4b800cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7fe00640 1 -- 172.21.15.28:0/1472060941 >> v2:172.21.15.138:3300/0 conn(0x7f4b8015e660 msgr2=0x7f4b8006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/1472060941 >> v2:172.21.15.138:3300/0 conn(0x7f4b8015e660 0x7f4b8006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.394+0000 7f4b7fe00640 1 -- 172.21.15.28:0/1472060941 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b800d16f0 con 0x7f4b800d0dd0 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.395+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b800d11b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b3980 tx=0x7f4b7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba228ce57c4d88f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.395+0000 7f4b7ea00640 1 -- 172.21.15.28:0/1472060941 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70037070 con 0x7f4b800d0dd0 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.395+0000 7f4b7ea00640 1 -- 172.21.15.28:0/1472060941 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70032440 con 0x7f4b800d0dd0 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.395+0000 7f4b7ea00640 1 -- 172.21.15.28:0/1472060941 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70032810 con 0x7f4b800d0dd0 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 -- 172.21.15.28:0/1472060941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 msgr2=0x7f4b800d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b800d11b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b3980 tx=0x7f4b7002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 -- 172.21.15.28:0/1472060941 shutdown_connections 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 --2- 172.21.15.28:0/1472060941 >> v2:172.21.15.138:3300/0 conn(0x7f4b8015e660 0x7f4b8006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 --2- 172.21.15.28:0/1472060941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b800d11b0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 -- 172.21.15.28:0/1472060941 >> 172.21.15.28:0/1472060941 conn(0x7f4b8008bfc0 msgr2=0x7f4b800d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 -- 172.21.15.28:0/1472060941 shutdown_connections 2024-09-10T10:53:14.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.396+0000 7f4b85800640 1 -- 172.21.15.28:0/1472060941 wait complete. 2024-09-10T10:53:14.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.397+0000 7f4b85800640 1 Processor -- start 2024-09-10T10:53:14.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.398+0000 7f4b85800640 1 -- start start 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.398+0000 7f4b85800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b85800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b85800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 0x7f4b800b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b85800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800d2f40 con 0x7f4b800d0dd0 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b85800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b800d2c60 con 0x7f4b8015e660 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b85800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b800d3220 con 0x7f4b800d1f90 2024-09-10T10:53:14.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56460/0 (socket says 172.21.15.28:56460) 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 -- 172.21.15.28:0/126649098 learned_addr learned my addr 172.21.15.28:0/126649098 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7f400640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b84e00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 0x7f4b800b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 -- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 msgr2=0x7f4b800b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 -- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 msgr2=0x7f4b800b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 0x7f4b800b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b7fe00640 1 -- 172.21.15.28:0/126649098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b800b5750 con 0x7f4b800d0dd0 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.399+0000 7f4b84e00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 0x7f4b800b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b7fe00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000c00 tx=0x7f4b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:14.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70041020 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b700306c0 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70037040 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b800b3ea0 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.400+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b800b4450 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.401+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b800b7870 con 0x7f4b800d0dd0 2024-09-10T10:53:14.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.402+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b70044020 con 0x7f4b800d0dd0 2024-09-10T10:53:14.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.403+0000 7f4b77e00640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 0x7f4b6006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:14.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.403+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f4b700abff0 con 0x7f4b800d0dd0 2024-09-10T10:53:14.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.404+0000 7f4b7f400640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 0x7f4b6006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:14.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.404+0000 7f4b7f400640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 0x7f4b6006e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4b801e6310 tx=0x7f4b6c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:14.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.408+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b800b7870 con 0x7f4b800d0dd0 2024-09-10T10:53:14.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.627+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b800d1b20 con 0x7f4b800d0dd0 2024-09-10T10:53:14.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.855+0000 7f4b77e00640 1 -- 172.21.15.28:0/126649098 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v225) ==== 161+0+0 (secure 0 0 0) 0x7f4b7004b030 con 0x7f4b800d0dd0 2024-09-10T10:53:14.858 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 msgr2=0x7f4b6006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 0x7f4b6006e2f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4b801e6310 tx=0x7f4b6c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 msgr2=0x7f4b801e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:14.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4b70000c00 tx=0x7f4b700046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 shutdown_connections 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b6006be30 0x7f4b6006e2f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b8015e660 0x7f4b800b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b800d1f90 0x7f4b800b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 --2- 172.21.15.28:0/126649098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b800d0dd0 0x7f4b801e7700 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 >> 172.21.15.28:0/126649098 conn(0x7f4b8008bfc0 msgr2=0x7f4b800b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 shutdown_connections 2024-09-10T10:53:14.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:14.859+0000 7f4b85800640 1 -- 172.21.15.28:0/126649098 wait complete. 2024-09-10T10:53:14.878 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.035+0000 7f746a000640 1 Processor -- start 2024-09-10T10:53:15.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.035+0000 7f746a000640 1 -- start start 2024-09-10T10:53:15.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7464072320 0x7f74640cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7464072de0 0x7f746406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072700 0x7f746406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7464146140 con 0x7f7464072320 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f74640cd740 con 0x7f7464072de0 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.036+0000 7f746a000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f74640cda20 con 0x7f7464072700 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7464072320 0x7f74640cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7464072320 0x7f74640cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56462/0 (socket says 172.21.15.28:56462) 2024-09-10T10:53:15.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 -- 172.21.15.28:0/165517118 learned_addr learned my addr 172.21.15.28:0/165517118 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 -- 172.21.15.28:0/165517118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072700 msgr2=0x7f746406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 --2- 172.21.15.28:0/165517118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072700 0x7f746406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 -- 172.21.15.28:0/165517118 >> v2:172.21.15.138:3300/0 conn(0x7f7464072de0 msgr2=0x7f746406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 --2- 172.21.15.28:0/165517118 >> v2:172.21.15.138:3300/0 conn(0x7f7464072de0 0x7f746406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.037+0000 7f7463400640 1 -- 172.21.15.28:0/165517118 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74640cdd00 con 0x7f7464072320 2024-09-10T10:53:15.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.038+0000 7f7463400640 1 --2- 172.21.15.28:0/165517118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072320 0x7f74640cce60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f746406f8c0 tx=0x7f745402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cfa06421192b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.038+0000 7f7462000640 1 -- 172.21.15.28:0/165517118 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454037070 con 0x7f7464072320 2024-09-10T10:53:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.038+0000 7f7462000640 1 -- 172.21.15.28:0/165517118 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7454032440 con 0x7f7464072320 2024-09-10T10:53:15.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.038+0000 7f7462000640 1 -- 172.21.15.28:0/165517118 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454032810 con 0x7f7464072320 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 -- 172.21.15.28:0/165517118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072320 msgr2=0x7f74640cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 --2- 172.21.15.28:0/165517118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072320 0x7f74640cce60 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f746406f8c0 tx=0x7f745402a140 comp rx=0 tx=0).stop 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 -- 172.21.15.28:0/165517118 shutdown_connections 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 --2- 172.21.15.28:0/165517118 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072700 0x7f746406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 --2- 172.21.15.28:0/165517118 >> v2:172.21.15.138:3300/0 conn(0x7f7464072de0 0x7f746406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 --2- 172.21.15.28:0/165517118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072320 0x7f74640cce60 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 -- 172.21.15.28:0/165517118 >> 172.21.15.28:0/165517118 conn(0x7f7464067d70 msgr2=0x7f746406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 -- 172.21.15.28:0/165517118 shutdown_connections 2024-09-10T10:53:15.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.039+0000 7f746a000640 1 -- 172.21.15.28:0/165517118 wait complete. 2024-09-10T10:53:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.041+0000 7f746a000640 1 Processor -- start 2024-09-10T10:53:15.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.041+0000 7f746a000640 1 -- start start 2024-09-10T10:53:15.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7464072320 0x7f74640ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 0x7f74640af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74640cd520 con 0x7f7464072700 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74640cd3a0 con 0x7f7464072320 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f746a000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f746406eb30 con 0x7f7464072de0 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56466/0 (socket says 172.21.15.28:56466) 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 -- 172.21.15.28:0/4015442813 learned_addr learned my addr 172.21.15.28:0/4015442813 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7463400640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7464072320 0x7f74640ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7463e00640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 0x7f74640af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 -- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 msgr2=0x7f74640af610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 0x7f74640af610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 -- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7464072320 msgr2=0x7f74640ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7464072320 0x7f74640ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7462a00640 1 -- 172.21.15.28:0/4015442813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f746413f110 con 0x7f7464072700 2024-09-10T10:53:15.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.042+0000 7f7463e00640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 0x7f74640af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:15.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f7462a00640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f745000b410 tx=0x7f745000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450015020 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7450011850 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450011c40 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74640c9490 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.043+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74640c9a50 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.044+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74640cce60 con 0x7f7464072700 2024-09-10T10:53:15.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.045+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7450018020 con 0x7f7464072700 2024-09-10T10:53:15.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.046+0000 7f745b400640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 0x7f744406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.046+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f745008b6c0 con 0x7f7464072700 2024-09-10T10:53:15.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.047+0000 7f7463400640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 0x7f744406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.047+0000 7f7463400640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 0x7f744406e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7454034000 tx=0x7f745402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.050+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f745001f020 con 0x7f7464072700 2024-09-10T10:53:15.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.309+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f74640c75c0 con 0x7f7464072700 2024-09-10T10:53:15.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.861+0000 7f745b400640 1 -- 172.21.15.28:0/4015442813 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v226) ==== 101+0+0 (secure 0 0 0) 0x7f7450059610 con 0x7f7464072700 2024-09-10T10:53:15.863 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 msgr2=0x7f744406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 0x7f744406e320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7454034000 tx=0x7f745402e000 comp rx=0 tx=0).stop 2024-09-10T10:53:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 msgr2=0x7f74640af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f745000b410 tx=0x7f745000b8d0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 shutdown_connections 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f744406be60 0x7f744406e320 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7464072de0 0x7f74640af610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7464072700 0x7f74640af0d0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 --2- 172.21.15.28:0/4015442813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7464072320 0x7f74640ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 >> 172.21.15.28:0/4015442813 conn(0x7f7464067d70 msgr2=0x7f74640ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 shutdown_connections 2024-09-10T10:53:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.865+0000 7f746a000640 1 -- 172.21.15.28:0/4015442813 wait complete. 2024-09-10T10:53:15.887 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.964+0000 7f988a190f40 1 Processor -- start 2024-09-10T10:53:15.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.964+0000 7f988a190f40 1 -- start start 2024-09-10T10:53:15.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 0x564466671850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 0x5644665e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5644665e6530 con 0x5644666713e0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644665e6290 con 0x5644666749c0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f988a190f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564466673ee0 con 0x5644665e58e0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f9886a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f9886a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56478/0 (socket says 172.21.15.28:56478) 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f9886a00640 1 -- 172.21.15.28:0/595224358 learned_addr learned my addr 172.21.15.28:0/595224358 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f9887e00640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 0x5644665e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.965+0000 7f9887400640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 0x564466671850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 -- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 msgr2=0x564466671850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 0x564466671850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 -- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 msgr2=0x5644665e8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 0x5644665e8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 -- 172.21.15.28:0/595224358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644664c7b30 con 0x5644666713e0 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9887e00640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 0x5644665e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:15.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9887400640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 0x564466671850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9886a00640 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9874055040 tx=0x7f9874051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80daf8d7cca48482 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9885600640 1 -- 172.21.15.28:0/595224358 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874056050 con 0x5644666713e0 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9885600640 1 -- 172.21.15.28:0/595224358 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9874052ad0 con 0x5644666713e0 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.966+0000 7f9885600640 1 -- 172.21.15.28:0/595224358 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874065480 con 0x5644666713e0 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 -- 172.21.15.28:0/595224358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 msgr2=0x564466667f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9874055040 tx=0x7f9874051000 comp rx=0 tx=0).stop 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 -- 172.21.15.28:0/595224358 shutdown_connections 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644666749c0 0x5644665e8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466667f10 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 --2- 172.21.15.28:0/595224358 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644665e58e0 0x564466671850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 -- 172.21.15.28:0/595224358 >> 172.21.15.28:0/595224358 conn(0x5644665ea0a0 msgr2=0x56446659b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 -- 172.21.15.28:0/595224358 shutdown_connections 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.967+0000 7f988a190f40 1 -- 172.21.15.28:0/595224358 wait complete. 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.968+0000 7f988a190f40 1 Processor -- start 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.968+0000 7f988a190f40 1 -- start start 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644665e58e0 0x564466667f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644666749c0 0x564466666420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5644665e6530 con 0x5644666713e0 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5644666731d0 con 0x5644665e58e0 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f988a190f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564466705250 con 0x5644666749c0 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f9886a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f9886a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56488/0 (socket says 172.21.15.28:56488) 2024-09-10T10:53:15.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 learned_addr learned my addr 172.21.15.28:0/2730074221 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644665e58e0 0x564466667f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.969+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644666749c0 0x564466666420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644666749c0 msgr2=0x564466666420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644666749c0 0x564466666420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644665e58e0 msgr2=0x564466667f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644665e58e0 0x564466667f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564466664a90 con 0x5644666713e0 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5644665e58e0 0x564466667f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5644666749c0 0x564466666420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:15.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9874045fd0 tx=0x7f9874002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f987fe00640 1 -- 172.21.15.28:0/2730074221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874065480 con 0x5644666713e0 2024-09-10T10:53:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f987fe00640 1 -- 172.21.15.28:0/2730074221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9874065620 con 0x5644666713e0 2024-09-10T10:53:15.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564466664d70 con 0x5644666713e0 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f987fe00640 1 -- 172.21.15.28:0/2730074221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98740659d0 con 0x5644666713e0 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.970+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564466702f70 con 0x5644666713e0 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.972+0000 7f987fe00640 1 -- 172.21.15.28:0/2730074221 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9874077020 con 0x5644666713e0 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.973+0000 7f987fe00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 0x7f986806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.973+0000 7f987fe00640 1 -- 172.21.15.28:0/2730074221 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f98740cfa40 con 0x5644666713e0 2024-09-10T10:53:15.976 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.973+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 0x564466666e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.973+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466667670 con 0x5644666701a0 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.973+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 0x7f986806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.974+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 0x564466666e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.974+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 0x7f986806ccd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9878005630 tx=0x7f9878005540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.975+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 0x564466666e70 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:15.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.977+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v124'722 uv696 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.977+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466704cb0 con 0x5644666701a0 2024-09-10T10:53:15.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.979+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v124'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.979+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667160f0 con 0x5644666701a0 2024-09-10T10:53:15.983 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:15.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.981+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v124'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.981+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667174a0 con 0x5644666701a0 2024-09-10T10:53:15.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.983+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v124'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.983+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717840 con 0x5644666701a0 2024-09-10T10:53:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.986+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v124'726 uv726 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.986+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716490 con 0x5644666701a0 2024-09-10T10:53:15.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.988+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v124'727 uv727 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.988+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716830 con 0x5644666701a0 2024-09-10T10:53:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.989+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v124'728 uv702 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.989+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466716c20 con 0x5644666701a0 2024-09-10T10:53:15.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.991+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v124'729 uv729 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.991+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716c20 con 0x5644666701a0 2024-09-10T10:53:15.995 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.993+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v124'730 uv730 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.993+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.996+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v124'731 uv731 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.996+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.997+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v124'732 uv732 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.997+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.999+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v124'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:15.999+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.000+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v124'734 uv709 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.000+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.002+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v124'735 uv735 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.002+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.005 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:16.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.004+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v124'736 uv736 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.004+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.005+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v124'737 uv737 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.005+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.007+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v124'738 uv738 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.007+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466716bd0 con 0x5644666701a0 2024-09-10T10:53:16.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.009+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v124'739 uv739 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.009+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.011+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v124'740 uv740 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.011+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.012+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v124'741 uv715 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.012+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.014+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v124'742 uv742 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.014+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.017 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:16.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.016+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v124'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.016+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466718bf0 con 0x5644666701a0 2024-09-10T10:53:16.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.017+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v124'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.018+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466718bf0 con 0x5644666701a0 2024-09-10T10:53:16.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.019+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v124'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.019+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.021+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v124'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.021+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.024+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v124'747 uv721 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.024+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.025+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v124'748 uv748 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.025+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.029 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:16.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.027+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v124'749 uv749 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.027+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466718bf0 con 0x5644666701a0 2024-09-10T10:53:16.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.029+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v124'750 uv750 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.029+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466718bf0 con 0x5644666701a0 2024-09-10T10:53:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.030+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v124'751 uv751 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.030+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466717be0 con 0x5644666701a0 2024-09-10T10:53:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.032+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v124'752 uv752 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x5644666701a0 2024-09-10T10:53:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.032+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 0x56446671a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.032+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671aa20 con 0x5644667180f0 2024-09-10T10:53:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.033+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 0x56446671a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.034+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 0x56446671a4e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.036+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v124'527 uv526 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.036+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446671adc0 con 0x5644667180f0 2024-09-10T10:53:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.037+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v124'528 uv528 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.037+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671b160 con 0x5644667180f0 2024-09-10T10:53:16.041 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:16.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.039+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v124'529 uv529 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.039+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671b500 con 0x5644667180f0 2024-09-10T10:53:16.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.041+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v124'530 uv530 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.041+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671b8a0 con 0x5644667180f0 2024-09-10T10:53:16.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.042+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v124'531 uv531 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.042+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671bc40 con 0x5644667180f0 2024-09-10T10:53:16.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.044+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v124'532 uv514 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.044+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446671bfe0 con 0x5644667180f0 2024-09-10T10:53:16.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.046+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v124'533 uv533 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.046+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.050 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:16.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.048+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v124'534 uv534 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.048+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.049+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v124'535 uv535 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.050+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.051+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v124'536 uv536 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.051+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.053+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v124'537 uv519 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.053+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.055+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v124'538 uv538 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.055+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.056+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v124'539 uv539 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.058 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:16.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.056+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466720a70 con 0x5644667180f0 2024-09-10T10:53:16.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.058+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v124'540 uv540 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.058+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466720a70 con 0x5644667180f0 2024-09-10T10:53:16.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.060+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v124'541 uv541 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.060+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.062+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v124'542 uv525 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.062+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.063+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v124'543 uv543 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.063+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.065+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v124'544 uv544 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.067 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:16.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.065+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671d390 con 0x5644667180f0 2024-09-10T10:53:16.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.067+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v124'545 uv545 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.067+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671d390 con 0x5644667180f0 2024-09-10T10:53:16.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.068+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v124'546 uv546 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.068+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671c380 con 0x5644667180f0 2024-09-10T10:53:16.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.069+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v124'547 uv547 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x5644667180f0 2024-09-10T10:53:16.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.069+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56446671c890 0x56446671ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.069+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671f310 con 0x56446671c890 2024-09-10T10:53:16.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.070+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56446671c890 0x56446671ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.070+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56446671c890 0x56446671ec80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.072+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v124'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.072+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446671f7b0 con 0x56446671c890 2024-09-10T10:53:16.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.073+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.073+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446671fc50 con 0x56446671c890 2024-09-10T10:53:16.076 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:16.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.074+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.075+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466721500 con 0x56446671c890 2024-09-10T10:53:16.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.076+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.076+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667218a0 con 0x56446671c890 2024-09-10T10:53:16.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.077+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v124'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.077+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667204f0 con 0x56446671c890 2024-09-10T10:53:16.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.078+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v124'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.079+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564466720df0 0x564466723c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.079+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466724190 con 0x564466720df0 2024-09-10T10:53:16.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.079+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564466720df0 0x564466723c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.079+0000 7f9887e00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564466720df0 0x564466723c50 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.080+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v124'103 uv102 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.081+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466724190 con 0x564466720df0 2024-09-10T10:53:16.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.082+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v124'104 uv104 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.082+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466724210 con 0x564466720df0 2024-09-10T10:53:16.085 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:16.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.083+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v124'105 uv105 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.083+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466724600 con 0x564466720df0 2024-09-10T10:53:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.084+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v124'106 uv106 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.084+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466724600 con 0x56446671c890 2024-09-10T10:53:16.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.085+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v124'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.086+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.087+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.087+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.089 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:16.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.088+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.088+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667259b0 con 0x56446671c890 2024-09-10T10:53:16.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.089+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.089+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667259b0 con 0x56446671c890 2024-09-10T10:53:16.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.090+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v124'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.090+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.091+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v124'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.091+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.093+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v124'260 uv254 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.093+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.094+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v124'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.094+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.097 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:16.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.096+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v124'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.096+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667259b0 con 0x564466720df0 2024-09-10T10:53:16.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.097+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v124'263 uv263 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.097+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667259b0 con 0x564466720df0 2024-09-10T10:53:16.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.098+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v124'264 uv264 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.098+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.100+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v124'265 uv265 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.100+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.101+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v124'266 uv259 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.101+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.102+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v124'267 uv267 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.102+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.105 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:16.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.103+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v124'268 uv268 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.104+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.105+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v124'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.105+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x564466720df0 2024-09-10T10:53:16.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.106+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v124'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.106+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.108+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v124'117 uv116 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.108+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.109+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v124'118 uv118 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.109+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.112 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.110+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v124'119 uv119 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.110+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.111+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v124'120 uv120 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.111+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667249a0 con 0x56446671c890 2024-09-10T10:53:16.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.112+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v124'121 uv121 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9874050050 con 0x56446671c890 2024-09-10T10:53:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.113+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564466724eb0 0x5644667272a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.113+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727900 con 0x564466724eb0 2024-09-10T10:53:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.113+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564466724eb0 0x5644667272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.113+0000 7f9887400640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564466724eb0 0x5644667272a0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.115+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v124'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.115+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667279c0 con 0x564466724eb0 2024-09-10T10:53:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.116+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.116+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.118+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.120 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:16.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.118+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466728df0 con 0x564466724eb0 2024-09-10T10:53:16.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.119+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.119+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466728df0 con 0x564466724eb0 2024-09-10T10:53:16.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.120+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v124'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.120+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.121+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v124'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.122+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.123+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v124'233 uv227 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.123+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.124+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v124'234 uv234 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.124+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.127 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:16.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.125+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v124'235 uv235 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.126+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.127+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v124'236 uv236 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.127+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.128+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v124'237 uv237 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.128+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.129+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v124'238 uv232 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.130+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.131+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v124'239 uv239 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.131+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.134 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:16.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.132+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v124'240 uv240 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.132+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x564466727de0 con 0x564466724eb0 2024-09-10T10:53:16.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.134+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v124'241 uv241 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.134+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672abb0 con 0x564466724eb0 2024-09-10T10:53:16.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.135+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v124'242 uv242 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.135+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672ab70 con 0x564466720df0 2024-09-10T10:53:16.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.137+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v124'103 uv102 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.137+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446672ab70 con 0x564466720df0 2024-09-10T10:53:16.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.138+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v124'104 uv104 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.138+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672ab70 con 0x564466720df0 2024-09-10T10:53:16.142 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:16.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.140+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v124'105 uv105 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.140+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672ab70 con 0x564466720df0 2024-09-10T10:53:16.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.141+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v124'106 uv106 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.141+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667281d0 con 0x564466720df0 2024-09-10T10:53:16.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.142+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v124'141 uv140 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.142+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x5644667281d0 con 0x564466720df0 2024-09-10T10:53:16.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.144+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v124'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.144+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667281d0 con 0x564466720df0 2024-09-10T10:53:16.147 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:16.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.145+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v124'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.145+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667291e0 con 0x564466720df0 2024-09-10T10:53:16.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.146+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v124'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.146+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667291e0 con 0x564466720df0 2024-09-10T10:53:16.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.147+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v124'145 uv145 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.147+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x5644667281d0 con 0x564466720df0 2024-09-10T10:53:16.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.148+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v124'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9880011040 con 0x564466720df0 2024-09-10T10:53:16.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.148+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564466728850 0x56446672ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.148+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b270 con 0x564466728850 2024-09-10T10:53:16.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.149+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564466728850 0x56446672ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.149+0000 7f9886a00640 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564466728850 0x56446672ac40 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.150+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v124'120 uv119 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x564466728850 2024-09-10T10:53:16.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.150+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446672b270 con 0x564466728850 2024-09-10T10:53:16.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.152+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v124'121 uv121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x564466728850 2024-09-10T10:53:16.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.152+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b270 con 0x564466728850 2024-09-10T10:53:16.154 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:16.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.153+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v124'122 uv122 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x564466728850 2024-09-10T10:53:16.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.153+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b660 con 0x564466728850 2024-09-10T10:53:16.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.154+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v124'123 uv123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x564466728850 2024-09-10T10:53:16.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.154+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b660 con 0x564466728850 2024-09-10T10:53:16.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.155+0000 7f9886a00640 1 -- 172.21.15.28:0/2730074221 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v124'124 uv124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9874050050 con 0x564466728850 2024-09-10T10:53:16.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.155+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.156+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v124'290 uv282 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.156+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.158+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v124'291 uv291 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.158+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.161 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:16.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.159+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v124'292 uv292 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.159+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.160+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v124'293 uv293 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.160+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.161+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v124'294 uv294 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.161+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.162+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v124'295 uv289 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.162+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.164+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v124'296 uv296 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.164+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b320 con 0x564466724eb0 2024-09-10T10:53:16.166 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:16.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.165+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v124'297 uv297 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.165+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672c6f0 con 0x564466724eb0 2024-09-10T10:53:16.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.166+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v124'298 uv298 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.166+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672c6f0 con 0x564466724eb0 2024-09-10T10:53:16.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.167+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v124'299 uv299 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.167+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672b6e0 con 0x564466724eb0 2024-09-10T10:53:16.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.168+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v124'300 uv300 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.168+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56446672be40 con 0x564466724eb0 2024-09-10T10:53:16.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f9887400640 1 -- 172.21.15.28:0/2730074221 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v124'301 uv301 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f9878018040 con 0x564466724eb0 2024-09-10T10:53:16.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56446671c890 msgr2=0x56446671ec80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56446671c890 0x56446671ec80 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564466724eb0 msgr2=0x5644667272a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x564466724eb0 0x5644667272a0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564466728850 msgr2=0x56446672ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x564466728850 0x56446672ac40 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564466720df0 msgr2=0x564466723c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x564466720df0 0x564466723c50 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 msgr2=0x56446671a4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 0x56446671a4e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 reap_dead start 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.169+0000 7f9887e00640 1 -- 172.21.15.28:0/2730074221 reap_dead start 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 msgr2=0x564466666e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 0x564466666e70 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 msgr2=0x7f986806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 0x7f986806ccd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9878005630 tx=0x7f9878005540 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 msgr2=0x564466668450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9874045fd0 tx=0x7f9874002fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 shutdown_connections 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5644667180f0 0x56446671a4e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5644666701a0 0x564466666e70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f986806a810 0x7f986806ccd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 --2- 172.21.15.28:0/2730074221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5644666713e0 0x564466668450 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 >> 172.21.15.28:0/2730074221 conn(0x5644665ea0a0 msgr2=0x56446659b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 shutdown_connections 2024-09-10T10:53:16.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.170+0000 7f988a190f40 1 -- 172.21.15.28:0/2730074221 wait complete. 2024-09-10T10:53:16.176 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000000 0 2024-09-10T10:53:16.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 Processor -- start 2024-09-10T10:53:16.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 -- start start 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72963e0 0x556ae7294550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae72999c0 0x556ae720c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ae7168b30 con 0x556ae71c0470 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556ae716cb70 con 0x556ae72999c0 2024-09-10T10:53:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.292+0000 7fce92c62f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556ae7298ee0 con 0x556ae72963e0 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56496/0 (socket says 172.21.15.28:56496) 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 -- 172.21.15.28:0/1337716968 learned_addr learned my addr 172.21.15.28:0/1337716968 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8f400640 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72963e0 0x556ae7294550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce90800640 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae72999c0 0x556ae720c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 -- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72963e0 msgr2=0x556ae7294550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72963e0 0x556ae7294550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 -- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae72999c0 msgr2=0x556ae720c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae72999c0 0x556ae720c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 -- 172.21.15.28:0/1337716968 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae7294fa0 con 0x556ae71c0470 2024-09-10T10:53:16.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.293+0000 7fce8fe00640 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fce800010e0 tx=0x7fce8000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247fb50de6ce312 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce8e000640 1 -- 172.21.15.28:0/1337716968 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80012020 con 0x556ae71c0470 2024-09-10T10:53:16.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce8e000640 1 -- 172.21.15.28:0/1337716968 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8000e670 con 0x556ae71c0470 2024-09-10T10:53:16.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce8e000640 1 -- 172.21.15.28:0/1337716968 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8000ea60 con 0x556ae71c0470 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 -- 172.21.15.28:0/1337716968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 msgr2=0x556ae71ef8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fce800010e0 tx=0x7fce8000b6c0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 -- 172.21.15.28:0/1337716968 shutdown_connections 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae72999c0 0x556ae720c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72963e0 0x556ae7294550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 --2- 172.21.15.28:0/1337716968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae71c0470 0x556ae71ef8d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 -- 172.21.15.28:0/1337716968 >> 172.21.15.28:0/1337716968 conn(0x556ae720f0a0 msgr2=0x556ae720f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 -- 172.21.15.28:0/1337716968 shutdown_connections 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.294+0000 7fce92c62f40 1 -- 172.21.15.28:0/1337716968 wait complete. 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.295+0000 7fce92c62f40 1 Processor -- start 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.295+0000 7fce92c62f40 1 -- start start 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae71c0470 0x556ae72927b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 0x556ae7297ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72999c0 0x556ae7298430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556ae7295280 con 0x556ae72963e0 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556ae7295560 con 0x556ae71c0470 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce92c62f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x556ae7297af0 con 0x556ae72999c0 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce8f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 0x556ae7297ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce8f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 0x556ae7297ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56500/0 (socket says 172.21.15.28:56500) 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce8f400640 1 -- 172.21.15.28:0/2887478578 learned_addr learned my addr 172.21.15.28:0/2887478578 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce8fe00640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae71c0470 0x556ae72927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.296+0000 7fce90800640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72999c0 0x556ae7298430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 -- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72999c0 msgr2=0x556ae7298430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x556ae72999c0 0x556ae7298430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 -- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae71c0470 msgr2=0x556ae72927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ae71c0470 0x556ae72927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 -- 172.21.15.28:0/2887478578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae7292cf0 con 0x556ae72963e0 2024-09-10T10:53:16.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8f400640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 0x556ae7297ef0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fce7c003e80 tx=0x7fce7c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8cc00640 1 -- 172.21.15.28:0/2887478578 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c011460 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8cc00640 1 -- 172.21.15.28:0/2887478578 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce7c016070 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce8cc00640 1 -- 172.21.15.28:0/2887478578 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c011840 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.297+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ae7294900 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.298+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ae7295810 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.299+0000 7fce8cc00640 1 -- 172.21.15.28:0/2887478578 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fce7c0119e0 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.300+0000 7fce8cc00640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce7006a840 0x7fce7006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.300+0000 7fce8cc00640 1 -- 172.21.15.28:0/2887478578 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7fce7c08a420 con 0x556ae72963e0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce92c62f40 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556ae71f21a0 0x556ae71f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce8fe00640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce7006a840 0x7fce7006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:292cf221:::200.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x556ae71f2ac0 con 0x556ae71f21a0 2024-09-10T10:53:16.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce90800640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556ae71f21a0 0x556ae71f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce8fe00640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce7006a840 0x7fce7006cd00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce80011040 tx=0x7fce80009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.301+0000 7fce90800640 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556ae71f21a0 0x556ae71f2580 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.303+0000 7fce90800640 1 -- 172.21.15.28:0/2887478578 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000000 [truncate 0] v124'548 uv548 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fce88058950 con 0x556ae71f21a0 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.303+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556ae71f21a0 msgr2=0x556ae71f2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.303+0000 7fce92c62f40 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x556ae71f21a0 0x556ae71f2580 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce7006a840 msgr2=0x7fce7006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fce7006a840 0x7fce7006cd00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce80011040 tx=0x7fce80009000 comp rx=0 tx=0).stop 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 msgr2=0x556ae7297ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 --2- 172.21.15.28:0/2887478578 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x556ae72963e0 0x556ae7297ef0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fce7c003e80 tx=0x7fce7c003eb0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce90800640 1 -- 172.21.15.28:0/2887478578 reap_dead start 2024-09-10T10:53:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 shutdown_connections 2024-09-10T10:53:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 >> 172.21.15.28:0/2887478578 conn(0x556ae720f0a0 msgr2=0x556ae720f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 shutdown_connections 2024-09-10T10:53:16.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.304+0000 7fce92c62f40 1 -- 172.21.15.28:0/2887478578 wait complete. 2024-09-10T10:53:16.310 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:16.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.467+0000 7f149b600640 1 Processor -- start 2024-09-10T10:53:16.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.467+0000 7f149b600640 1 -- start start 2024-09-10T10:53:16.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.468+0000 7f149b600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1494162340 0x7f149406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.468+0000 7f149b600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.468+0000 7f149b600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149415ff70 con 0x7f14940cf280 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.468+0000 7f149b600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f149406de90 con 0x7f1494162340 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.468+0000 7f149b600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f14940d4d10 con 0x7f1494162e30 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1498c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1494162340 0x7f149406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1499600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1493e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f14940cf280 0x7f1494162720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:45750/0 (socket says 172.21.15.28:45750) 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1493e00640 1 -- 172.21.15.28:0/410167395 learned_addr learned my addr 172.21.15.28:0/410167395 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1493400640 1 -- 172.21.15.28:0/410167395 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972803235 0 0) 0x7f149415ff70 con 0x7f14940cf280 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1493400640 1 -- 172.21.15.28:0/410167395 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f147c003520 con 0x7f14940cf280 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1498c00640 1 -- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 msgr2=0x7f14940d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1498c00640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1498c00640 1 -- 172.21.15.28:0/410167395 >> v1:172.21.15.138:6789/0 conn(0x7f14940cf280 legacy=0x7f1494162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1498c00640 1 -- 172.21.15.28:0/410167395 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940d4ff0 con 0x7f1494162340 2024-09-10T10:53:16.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.469+0000 7f1499600640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.470+0000 7f1498c00640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f149406d740 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f14880097a0 tx=0x7f148802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46035284f03137dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f1493400640 1 -- 172.21.15.28:0/410167395 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488037070 con 0x7f1494162340 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f1493400640 1 -- 172.21.15.28:0/410167395 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148802ae50 con 0x7f1494162340 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f1493400640 1 -- 172.21.15.28:0/410167395 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488032640 con 0x7f1494162340 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 -- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 msgr2=0x7f149406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f149406d740 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f14880097a0 tx=0x7f148802a100 comp rx=0 tx=0).stop 2024-09-10T10:53:16.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 -- 172.21.15.28:0/410167395 shutdown_connections 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 --2- 172.21.15.28:0/410167395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f149406d740 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 -- 172.21.15.28:0/410167395 >> 172.21.15.28:0/410167395 conn(0x7f149408bfc0 msgr2=0x7f14940d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 -- 172.21.15.28:0/410167395 shutdown_connections 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.471+0000 7f149b600640 1 -- 172.21.15.28:0/410167395 wait complete. 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.473+0000 7f149b600640 1 Processor -- start 2024-09-10T10:53:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.473+0000 7f149b600640 1 -- start start 2024-09-10T10:53:16.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f14940b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f14940cf6c0 con 0x7f14940cf280 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f14940d52d0 con 0x7f1494162340 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f149b600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f14940d2ed0 con 0x7f1494162e30 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f1498c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f1498c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56510/0 (socket says 172.21.15.28:56510) 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.474+0000 7f1498c00640 1 -- 172.21.15.28:0/2355401385 learned_addr learned my addr 172.21.15.28:0/2355401385 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:16.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1493e00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f14940b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 -- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 msgr2=0x7f14940ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1499600640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 -- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 msgr2=0x7f14940b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f14940b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 -- 172.21.15.28:0/2355401385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940b48a0 con 0x7f14940cf280 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1493e00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f14940b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1499600640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:16.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1498c00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f1488035040 tx=0x7f14880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488041020 con 0x7f14940cf280 2024-09-10T10:53:16.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148803c670 con 0x7f14940cf280 2024-09-10T10:53:16.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488037040 con 0x7f14940cf280 2024-09-10T10:53:16.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.475+0000 7f149b600640 1 -- 172.21.15.28:0/2355401385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14940b4b80 con 0x7f14940cf280 2024-09-10T10:53:16.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.476+0000 7f149b600640 1 -- 172.21.15.28:0/2355401385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14940b5140 con 0x7f14940cf280 2024-09-10T10:53:16.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.477+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14940d2990 con 0x7f14940cf280 2024-09-10T10:53:16.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.478+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1488044020 con 0x7f14940cf280 2024-09-10T10:53:16.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.478+0000 7f1490c00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 0x7f147406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:16.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.478+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8368+0+0 (secure 0 0 0) 0x7f14880abff0 con 0x7f14940cf280 2024-09-10T10:53:16.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.479+0000 7f1493e00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 0x7f147406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:16.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.480+0000 7f1493e00640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 0x7f147406e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f149415f7f0 tx=0x7f1480009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:16.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.483+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f148804b030 con 0x7f14940cf280 2024-09-10T10:53:16.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.706+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f149408e640 con 0x7f14940cf280 2024-09-10T10:53:16.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.865+0000 7f1490c00640 1 -- 172.21.15.28:0/2355401385 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v227) ==== 162+0+0 (secure 0 0 0) 0x7f148807a010 con 0x7f14940cf280 2024-09-10T10:53:16.867 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 msgr2=0x7f147406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 0x7f147406e320 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f149415f7f0 tx=0x7f1480009040 comp rx=0 tx=0).stop 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 msgr2=0x7f14940cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f1488035040 tx=0x7f14880046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 shutdown_connections 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f147406be60 0x7f147406e320 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1494162e30 0x7f14940ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494162340 0x7f14940b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 --2- 172.21.15.28:0/2355401385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f14940cf280 0x7f14940cd9e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:16.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.867+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 >> 172.21.15.28:0/2355401385 conn(0x7f149408bfc0 msgr2=0x7f149406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:16.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.868+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 shutdown_connections 2024-09-10T10:53:16.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:16.868+0000 7f1486000640 1 -- 172.21.15.28:0/2355401385 wait complete. 2024-09-10T10:53:16.877 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:16.875+0000 7fadedc00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:53:16.878 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:16.876+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:16.878 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:16.876+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:16.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:17.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.042+0000 7f6bcc800640 1 Processor -- start 2024-09-10T10:53:17.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.042+0000 7f6bcc800640 1 -- start start 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bcc800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc4072700 0x7f6bc415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bcc800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc4146140 con 0x7f6bc4072320 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bcc800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc40cd740 con 0x7f6bc4072de0 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bcc800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc40cda20 con 0x7f6bc4072700 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bc9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6bc4072320 0x7f6bc40cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54226/0 (socket says 172.21.15.28:54226) 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bc9e00640 1 -- 172.21.15.28:0/3925884203 learned_addr learned my addr 172.21.15.28:0/3925884203 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:17.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bca800640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc4072700 0x7f6bc415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337761914 0 0) 0x7f6bc4146140 con 0x7f6bc4072320 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.044+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bac003520 con 0x7f6bc4072320 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525285147 0 0) 0x7f6bc40cd740 con 0x7f6bc4072de0 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bc4146140 con 0x7f6bc4072de0 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 957560188 0 0) 0x7f6bac003520 con 0x7f6bc4072320 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc4072700 msgr2=0x7f6bc415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc4072700 0x7f6bc415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 >> v1:172.21.15.138:6789/0 conn(0x7f6bc4072de0 legacy=0x7f6bc415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc40cdd00 con 0x7f6bc4072320 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f6bb8001670 con 0x7f6bc4072320 2024-09-10T10:53:17.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 >> v1:172.21.15.28:6789/0 conn(0x7f6bc4072320 legacy=0x7f6bc40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bac003b70 0x7f6bac003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bac004800 0x7f6bac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac0051c0 0x7f6bac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc40cda20 con 0x7f6bac0051c0 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bac00c090 con 0x7f6bac004800 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bac00c290 con 0x7f6bac003b70 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bca800640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc4072700 0x7f6bc415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bac003b70 0x7f6bac003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bc9400640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bac004800 0x7f6bac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.045+0000 7f6bca800640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac0051c0 0x7f6bac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bc9e00640 1 -- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bac004800 msgr2=0x7f6bac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bac004800 0x7f6bac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bc9e00640 1 -- 172.21.15.28:0/3925884203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac0051c0 msgr2=0x7f6bac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac0051c0 0x7f6bac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bc9e00640 1 -- 172.21.15.28:0/3925884203 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bac003520 con 0x7f6bac003b70 2024-09-10T10:53:17.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.046+0000 7f6bca800640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac0051c0 0x7f6bac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:17.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.047+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bac003b70 0x7f6bac003f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6f63585609c573 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.047+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007bc0 con 0x7f6bac003b70 2024-09-10T10:53:17.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.047+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8005af0 con 0x7f6bac003b70 2024-09-10T10:53:17.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.047+0000 7f6bc8a00640 1 -- 172.21.15.28:0/3925884203 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007070 con 0x7f6bac003b70 2024-09-10T10:53:17.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 -- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bac003b70 msgr2=0x7f6bac003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 --2- 172.21.15.28:0/3925884203 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bac003b70 0x7f6bac003f70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8007fd0 tx=0x7f6bac005830 comp rx=0 tx=0).stop 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bca800640 1 -- 172.21.15.28:0/3925884203 reap_dead start 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 -- 172.21.15.28:0/3925884203 shutdown_connections 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 -- 172.21.15.28:0/3925884203 >> 172.21.15.28:0/3925884203 conn(0x7f6bc4067d70 msgr2=0x7f6bc406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 -- 172.21.15.28:0/3925884203 shutdown_connections 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.048+0000 7f6bcc800640 1 -- 172.21.15.28:0/3925884203 wait complete. 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.049+0000 7f6bcc800640 1 Processor -- start 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 -- start start 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc41f49c0 0x7f6bc40ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc40a9fd0 0x7f6bc40aa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc415da70 con 0x7f6bac003b70 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc415d8f0 con 0x7f6bc40a9fd0 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.050+0000 7f6bcc800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc415dbf0 con 0x7f6bc41f49c0 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56524/0 (socket says 172.21.15.28:56524) 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 -- 172.21.15.28:0/1233888911 learned_addr learned my addr 172.21.15.28:0/1233888911 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9400640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc41f49c0 0x7f6bc40ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bca800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc40a9fd0 0x7f6bc40aa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 -- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc41f49c0 msgr2=0x7f6bc40ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc41f49c0 0x7f6bc40ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 -- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc40a9fd0 msgr2=0x7f6bc40aa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc40a9fd0 0x7f6bc40aa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 -- 172.21.15.28:0/1233888911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc40a9530 con 0x7f6bac003b70 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.051+0000 7f6bc9e00640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c1d0 tx=0x7f6bc40cded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.052+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8001670 con 0x7f6bac003b70 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.052+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb8042070 con 0x7f6bac003b70 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.052+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007270 con 0x7f6bac003b70 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.052+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc40ad180 con 0x7f6bac003b70 2024-09-10T10:53:17.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.052+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc40b2680 con 0x7f6bac003b70 2024-09-10T10:53:17.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.053+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc415dfd0 con 0x7f6bac003b70 2024-09-10T10:53:17.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.054+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6bb8007410 con 0x7f6bac003b70 2024-09-10T10:53:17.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.054+0000 7f6bbe000640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 0x7f6ba806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.055+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f6bb80ab960 con 0x7f6bac003b70 2024-09-10T10:53:17.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.055+0000 7f6bc9400640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 0x7f6ba806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.056+0000 7f6bc9400640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 0x7f6ba806e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40b51f0 tx=0x7f6bb4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.059+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6bb8079830 con 0x7f6bac003b70 2024-09-10T10:53:17.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.283+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bc40cedc0 con 0x7f6bac003b70 2024-09-10T10:53:17.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.284+0000 7f6bbe000640 1 -- 172.21.15.28:0/1233888911 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) ==== 95+0+3566 (secure 0 0 0) 0x7f6bb8079130 con 0x7f6bac003b70 2024-09-10T10:53:17.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:17.286 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":229,"btime":"2024-09-10T10:53:16:878133+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":229,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:16.878080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:17.286 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 229 2024-09-10T10:53:17.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 msgr2=0x7f6ba806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 0x7f6ba806e2f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40b51f0 tx=0x7f6bb4009040 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 msgr2=0x7f6bc40b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c1d0 tx=0x7f6bc40cded0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 shutdown_connections 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ba806be30 0x7f6ba806e2f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bc40a9fd0 0x7f6bc40aa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bc41f49c0 0x7f6bc40ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 --2- 172.21.15.28:0/1233888911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bac003b70 0x7f6bc40b2140 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 >> 172.21.15.28:0/1233888911 conn(0x7f6bc4067d70 msgr2=0x7f6bc415d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 shutdown_connections 2024-09-10T10:53:17.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.286+0000 7f6bcc800640 1 -- 172.21.15.28:0/1233888911 wait complete. 2024-09-10T10:53:17.305 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:17.305 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:17.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.459+0000 7f2e25e00640 1 Processor -- start 2024-09-10T10:53:17.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.459+0000 7f2e25e00640 1 -- start start 2024-09-10T10:53:17.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.460+0000 7f2e25e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e2015b300 0x7f2e200cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.460+0000 7f2e25e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e2006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.460+0000 7f2e25e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e200cdbe0 con 0x7f2e20072d30 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.460+0000 7f2e25e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2e200728a0 con 0x7f2e2015b300 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.460+0000 7f2e25e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200cd900 con 0x7f2e200cd3f0 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e2015b300 0x7f2e200cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2e2015b300 0x7f2e200cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56532/0 (socket says 172.21.15.28:56532) 2024-09-10T10:53:17.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 -- 172.21.15.28:0/4188623459 learned_addr learned my addr 172.21.15.28:0/4188623459 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:17.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 -- 172.21.15.28:0/4188623459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 msgr2=0x7f2e2006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:17.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 --2- 172.21.15.28:0/4188623459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e2006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 -- 172.21.15.28:0/4188623459 >> v1:172.21.15.138:6789/0 conn(0x7f2e20072d30 legacy=0x7f2e200b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.461+0000 7f2e1ea00640 1 -- 172.21.15.28:0/4188623459 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e20072250 con 0x7f2e2015b300 2024-09-10T10:53:17.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.462+0000 7f2e1ea00640 1 --2- 172.21.15.28:0/4188623459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e2015b300 0x7f2e200cceb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf49b60603b0f90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e1e000640 1 -- 172.21.15.28:0/4188623459 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10037070 con 0x7f2e2015b300 2024-09-10T10:53:17.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e1e000640 1 -- 172.21.15.28:0/4188623459 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e10032440 con 0x7f2e2015b300 2024-09-10T10:53:17.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e1e000640 1 -- 172.21.15.28:0/4188623459 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10032810 con 0x7f2e2015b300 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 -- 172.21.15.28:0/4188623459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e2015b300 msgr2=0x7f2e200cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 --2- 172.21.15.28:0/4188623459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e2015b300 0x7f2e200cceb0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e1002a270 comp rx=0 tx=0).stop 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 -- 172.21.15.28:0/4188623459 shutdown_connections 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 --2- 172.21.15.28:0/4188623459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e2006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 --2- 172.21.15.28:0/4188623459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e2015b300 0x7f2e200cceb0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 -- 172.21.15.28:0/4188623459 >> 172.21.15.28:0/4188623459 conn(0x7f2e20067d70 msgr2=0x7f2e2006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 -- 172.21.15.28:0/4188623459 shutdown_connections 2024-09-10T10:53:17.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.463+0000 7f2e25e00640 1 -- 172.21.15.28:0/4188623459 wait complete. 2024-09-10T10:53:17.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.465+0000 7f2e25e00640 1 Processor -- start 2024-09-10T10:53:17.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.465+0000 7f2e25e00640 1 -- start start 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e201e5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e2015b300 0x7f2e20143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200d0160 con 0x7f2e20072d30 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200cffe0 con 0x7f2e2015b300 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e25e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2006f700 con 0x7f2e200cd3f0 2024-09-10T10:53:17.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56546/0 (socket says 172.21.15.28:56546) 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 -- 172.21.15.28:0/1460644294 learned_addr learned my addr 172.21.15.28:0/1460644294 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e17e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e201e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1f400640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e2015b300 0x7f2e20143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 -- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 msgr2=0x7f2e201e5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e201e5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 -- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e2015b300 msgr2=0x7f2e20143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e2015b300 0x7f2e20143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.466+0000 7f2e1ea00640 1 -- 172.21.15.28:0/1460644294 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2015e5d0 con 0x7f2e20072d30 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e17e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e201e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e1ea00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10041070 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1003c870 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e10037040 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2015e2f0 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.467+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2015cbc0 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.468+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2006f1c0 con 0x7f2e20072d30 2024-09-10T10:53:17.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.469+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2e10044070 con 0x7f2e20072d30 2024-09-10T10:53:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.470+0000 7f2e17400640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.470+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f2e100ac530 con 0x7f2e20072d30 2024-09-10T10:53:17.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.470+0000 7f2e17e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.471+0000 7f2e17e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2e201eaff0 tx=0x7f2e08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.474+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e1007a400 con 0x7f2e20072d30 2024-09-10T10:53:17.595 INFO:tasks.ceph.mds.c.smithi028.stderr: -38> 2024-09-10T10:53:12.840+0000 7f3563000640 -1 mds.0.cache.ino(0x1) No data while reading inode 0x1 2024-09-10T10:53:17.595 INFO:tasks.ceph.mds.c.smithi028.stderr: -36> 2024-09-10T10:53:12.840+0000 7f3563000640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:53:17.634 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:17.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.697+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e200c9e40 con 0x7f2e20072d30 2024-09-10T10:53:17.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.698+0000 7f2e17400640 1 -- 172.21.15.28:0/1460644294 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 229 v229) ==== 95+0+3566 (secure 0 0 0) 0x7f2e10079d00 con 0x7f2e20072d30 2024-09-10T10:53:17.700 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:17.700 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":229,"btime":"2024-09-10T10:53:16:878133+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":229,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:16.878080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:17.700 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 229 2024-09-10T10:53:17.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 msgr2=0x7f2dfc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2e201eaff0 tx=0x7f2e08008040 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 msgr2=0x7f2e2015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f2e10009870 tx=0x7f2e100046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 shutdown_connections 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dfc06be30 0x7f2dfc06e2f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e2015b300 0x7f2e20143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e200cd3f0 0x7f2e201e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 --2- 172.21.15.28:0/1460644294 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2e20072d30 0x7f2e2015d3c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 >> 172.21.15.28:0/1460644294 conn(0x7f2e20067d70 msgr2=0x7f2e2006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 shutdown_connections 2024-09-10T10:53:17.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.700+0000 7f2e25e00640 1 -- 172.21.15.28:0/1460644294 wait complete. 2024-09-10T10:53:17.718 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:17.718 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:17.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.874+0000 7fed95400640 1 Processor -- start 2024-09-10T10:53:17.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.874+0000 7fed95400640 1 -- start start 2024-09-10T10:53:17.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed95400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed9015e6a0 0x7fed9006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed95400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 0x7fed900cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed95400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed900d2c20 con 0x7fed900d4180 2024-09-10T10:53:17.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed95400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fed900cefb0 con 0x7fed9015e6a0 2024-09-10T10:53:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed95400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed9006e390 con 0x7fed9006dd60 2024-09-10T10:53:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed8fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fed900d4180 0x7fed900d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54238/0 (socket says 172.21.15.28:54238) 2024-09-10T10:53:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed8fe00640 1 -- 172.21.15.28:0/1562758619 learned_addr learned my addr 172.21.15.28:0/1562758619 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.875+0000 7fed87e00640 1 --2- 172.21.15.28:0/1562758619 >> v2:172.21.15.28:3300/0 conn(0x7fed9015e6a0 0x7fed9006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed94a00640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 0x7fed900cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed87e00640 1 -- 172.21.15.28:0/1562758619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 msgr2=0x7fed900cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed87e00640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 0x7fed900cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed87e00640 1 -- 172.21.15.28:0/1562758619 >> v1:172.21.15.28:6789/0 conn(0x7fed900d4180 legacy=0x7fed900d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed87e00640 1 -- 172.21.15.28:0/1562758619 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9006e670 con 0x7fed9015e6a0 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed8f400640 1 -- 172.21.15.28:0/1562758619 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876952525 0 0) 0x7fed900d2c20 con 0x7fed900d4180 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.876+0000 7fed94a00640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 0x7fed900cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:17.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.877+0000 7fed87e00640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9015e6a0 0x7fed9006d820 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fed780097a0 tx=0x7fed7802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1b2a441a7cac6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.877+0000 7fed8f400640 1 -- 172.21.15.28:0/1562758619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78037070 con 0x7fed9015e6a0 2024-09-10T10:53:17.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.877+0000 7fed8f400640 1 -- 172.21.15.28:0/1562758619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7802ae50 con 0x7fed9015e6a0 2024-09-10T10:53:17.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.877+0000 7fed8f400640 1 -- 172.21.15.28:0/1562758619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78032660 con 0x7fed9015e6a0 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 -- 172.21.15.28:0/1562758619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9015e6a0 msgr2=0x7fed9006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9015e6a0 0x7fed9006d820 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fed780097a0 tx=0x7fed7802a100 comp rx=0 tx=0).stop 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 -- 172.21.15.28:0/1562758619 shutdown_connections 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9006dd60 0x7fed900cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 --2- 172.21.15.28:0/1562758619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9015e6a0 0x7fed9006d820 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 -- 172.21.15.28:0/1562758619 >> 172.21.15.28:0/1562758619 conn(0x7fed9009d6f0 msgr2=0x7fed9008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 -- 172.21.15.28:0/1562758619 shutdown_connections 2024-09-10T10:53:17.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.878+0000 7fed95400640 1 -- 172.21.15.28:0/1562758619 wait complete. 2024-09-10T10:53:17.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.880+0000 7fed95400640 1 Processor -- start 2024-09-10T10:53:17.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.880+0000 7fed95400640 1 -- start start 2024-09-10T10:53:17.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed900d4180 0x7fed901f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9015e6a0 0x7fed900b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed900cf980 con 0x7fed9006dd60 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed900cebd0 con 0x7fed900d4180 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed95400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed900cfb00 con 0x7fed9015e6a0 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56588/0 (socket says 172.21.15.28:56588) 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed8fe00640 1 -- 172.21.15.28:0/2671653622 learned_addr learned my addr 172.21.15.28:0/2671653622 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed94a00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9015e6a0 0x7fed900b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.881+0000 7fed87e00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed900d4180 0x7fed901f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 -- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9015e6a0 msgr2=0x7fed900b8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9015e6a0 0x7fed900b8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 -- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed900d4180 msgr2=0x7fed901f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed900d4180 0x7fed901f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 -- 172.21.15.28:0/2671653622 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed900b8990 con 0x7fed9006dd60 2024-09-10T10:53:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.882+0000 7fed8fe00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c4d0 tx=0x7fed8000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.883+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80018070 con 0x7fed9006dd60 2024-09-10T10:53:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.883+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed80007500 con 0x7fed9006dd60 2024-09-10T10:53:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.883+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed80004a20 con 0x7fed9006dd60 2024-09-10T10:53:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.883+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed900b8c70 con 0x7fed9006dd60 2024-09-10T10:53:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.883+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed900b9230 con 0x7fed9006dd60 2024-09-10T10:53:17.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.884+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed900d5130 con 0x7fed9006dd60 2024-09-10T10:53:17.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.885+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed8001b070 con 0x7fed9006dd60 2024-09-10T10:53:17.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.886+0000 7fed8cc00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 0x7fed6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:17.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.886+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7fed8008aca0 con 0x7fed9006dd60 2024-09-10T10:53:17.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.886+0000 7fed87e00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 0x7fed6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:17.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.887+0000 7fed87e00640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 0x7fed6c06e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed7802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:17.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:17.891+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed900d5130 con 0x7fed9006dd60 2024-09-10T10:53:18.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.112+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed900b4bc0 con 0x7fed9006dd60 2024-09-10T10:53:18.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.113+0000 7fed8cc00640 1 -- 172.21.15.28:0/2671653622 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 230 v230) ==== 95+0+4392 (secure 0 0 0) 0x7fed80058490 con 0x7fed9006dd60 2024-09-10T10:53:18.115 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:18.115 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":230,"btime":"2024-09-10T10:53:17:879740+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9111,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2349436086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2349436086},{"type":"v1","addr":"172.21.15.28:6834","nonce":2349436086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":230}],"filesystems":[{"mdsmap":{"epoch":229,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:16.878080+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:18.116 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 230 2024-09-10T10:53:18.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 msgr2=0x7fed6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 0x7fed6c06e2f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed78034000 tx=0x7fed7802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 msgr2=0x7fed901f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fed8000c4d0 tx=0x7fed8000c990 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 shutdown_connections 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed6c06be30 0x7fed6c06e2f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed9015e6a0 0x7fed900b8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed900d4180 0x7fed901f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 --2- 172.21.15.28:0/2671653622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9006dd60 0x7fed901f4270 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 >> 172.21.15.28:0/2671653622 conn(0x7fed9009d6f0 msgr2=0x7fed900b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 shutdown_connections 2024-09-10T10:53:18.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.115+0000 7fed95400640 1 -- 172.21.15.28:0/2671653622 wait complete. 2024-09-10T10:53:18.133 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000000' led to DAMAGED state 2024-09-10T10:53:18.133 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 200.00000001' 2024-09-10T10:53:18.133 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:18.133 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:18.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:18.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:18.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:18.170 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:18.245 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:18.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:18.246 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:18.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.441+0000 7f57cde00640 1 Processor -- start 2024-09-10T10:53:18.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.441+0000 7f57cde00640 1 -- start start 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.442+0000 7f57cde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57c80cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.442+0000 7f57cde00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57c806f8d0 0x7f57c815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.442+0000 7f57cde00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c80cdbe0 con 0x7f57c80cd3f0 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.442+0000 7f57cde00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57c80728a0 con 0x7f57c806f8d0 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.442+0000 7f57cde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57c80cd900 con 0x7f57c815b300 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c6000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f57c80cd3f0 0x7f57c806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54250/0 (socket says 172.21.15.28:54250) 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c6000640 1 -- 172.21.15.28:0/29571502 learned_addr learned my addr 172.21.15.28:0/29571502 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c6a00640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57c80cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c7400640 1 --2- 172.21.15.28:0/29571502 >> v2:172.21.15.138:3300/0 conn(0x7f57c806f8d0 0x7f57c815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695293475 0 0) 0x7f57c80cdbe0 con 0x7f57c80cd3f0 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.443+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57ac005730 con 0x7f57c80cd3f0 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1919376557 0 0) 0x7f57ac005730 con 0x7f57c80cd3f0 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 msgr2=0x7f57c80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57c80cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 >> v2:172.21.15.138:3300/0 conn(0x7f57c806f8d0 msgr2=0x7f57c815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 --2- 172.21.15.28:0/29571502 >> v2:172.21.15.138:3300/0 conn(0x7f57c806f8d0 0x7f57c815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c8072250 con 0x7f57c80cd3f0 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f57b4001670 con 0x7f57c80cd3f0 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c7400640 1 --2- 172.21.15.28:0/29571502 >> v2:172.21.15.138:3300/0 conn(0x7f57c806f8d0 0x7f57c815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:18.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 >> v1:172.21.15.28:6789/0 conn(0x7f57c80cd3f0 legacy=0x7f57c806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57ac005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57ac006140 0x7f57ac006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ac006f50 0x7f57ac0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80728a0 con 0x7f57ac006140 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80cd900 con 0x7f57ac006f50 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57ac00dbd0 con 0x7f57c815b300 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57ac006140 0x7f57ac006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c7400640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ac006f50 0x7f57ac0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6a00640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57ac005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 -- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 msgr2=0x7f57ac005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57c815b300 0x7f57ac005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 -- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ac006f50 msgr2=0x7f57ac0073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:18.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57ac006f50 0x7f57ac0073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.444+0000 7f57c6000640 1 -- 172.21.15.28:0/29571502 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ac005730 con 0x7f57ac006140 2024-09-10T10:53:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.445+0000 7f57c6000640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57ac006140 0x7f57ac006a10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001050 tx=0x7f57ac007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e1dce5af53f919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4007d70 con 0x7f57ac006140 2024-09-10T10:53:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b40058d0 con 0x7f57ac006140 2024-09-10T10:53:18.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57c5600640 1 -- 172.21.15.28:0/29571502 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005ce0 con 0x7f57ac006140 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 -- 172.21.15.28:0/29571502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57ac006140 msgr2=0x7f57ac006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 --2- 172.21.15.28:0/29571502 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57ac006140 0x7f57ac006a10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001050 tx=0x7f57ac007a40 comp rx=0 tx=0).stop 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57c7400640 1 -- 172.21.15.28:0/29571502 reap_dead start 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 -- 172.21.15.28:0/29571502 shutdown_connections 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 -- 172.21.15.28:0/29571502 >> 172.21.15.28:0/29571502 conn(0x7f57c8067d70 msgr2=0x7f57c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 -- 172.21.15.28:0/29571502 shutdown_connections 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.446+0000 7f57cde00640 1 -- 172.21.15.28:0/29571502 wait complete. 2024-09-10T10:53:18.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.448+0000 7f57cde00640 1 Processor -- start 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.448+0000 7f57cde00640 1 -- start start 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57ac006140 0x7f57c8150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c81502c0 0x7f57c8156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80d0160 con 0x7f57c8151210 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57c80cffe0 con 0x7f57c81502c0 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57cde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57c8072d30 con 0x7f57ac006140 2024-09-10T10:53:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57c6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57c6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56598/0 (socket says 172.21.15.28:56598) 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.449+0000 7f57c6000640 1 -- 172.21.15.28:0/3309048426 learned_addr learned my addr 172.21.15.28:0/3309048426 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 -- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57ac006140 msgr2=0x7f57c8150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57ac006140 0x7f57c8150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 -- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c81502c0 msgr2=0x7f57c8156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c81502c0 0x7f57c8156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 -- 172.21.15.28:0/3309048426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c8155250 con 0x7f57c8151210 2024-09-10T10:53:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57c6000640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f57b402adf0 tx=0x7f57b4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4043070 con 0x7f57c8151210 2024-09-10T10:53:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b4007d50 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c814eb30 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.450+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4038040 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.451+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c814f2a0 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.452+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c815c120 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.452+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f57b4046070 con 0x7f57c8151210 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.453+0000 7f57bf400640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 0x7f57a806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.453+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f57b40ab100 con 0x7f57c8151210 2024-09-10T10:53:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.454+0000 7f57c6a00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 0x7f57a806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.455+0000 7f57c6a00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 0x7f57a806e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009830 tx=0x7f57b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:18.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.457+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57b4078980 con 0x7f57c8151210 2024-09-10T10:53:18.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.681+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f57c815be40 con 0x7f57c8151210 2024-09-10T10:53:18.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.882+0000 7f57bf400640 1 -- 172.21.15.28:0/3309048426 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v231) ==== 161+0+0 (secure 0 0 0) 0x7f57b407cc80 con 0x7f57c8151210 2024-09-10T10:53:18.885 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 msgr2=0x7f57a806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 0x7f57a806e320 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009830 tx=0x7f57b8008040 comp rx=0 tx=0).stop 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 msgr2=0x7f57c814fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f57b402adf0 tx=0x7f57b4000fe0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 shutdown_connections 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f57a806be60 0x7f57a806e320 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c81502c0 0x7f57c8156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57c8151210 0x7f57c814fd80 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 --2- 172.21.15.28:0/3309048426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57ac006140 0x7f57c8150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:18.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.884+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 >> 172.21.15.28:0/3309048426 conn(0x7f57c8067d70 msgr2=0x7f57c806ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:18.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.885+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 shutdown_connections 2024-09-10T10:53:18.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:18.885+0000 7f57cde00640 1 -- 172.21.15.28:0/3309048426 wait complete. 2024-09-10T10:53:18.903 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:19.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.057+0000 7f44bda00640 1 Processor -- start 2024-09-10T10:53:19.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.057+0000 7f44bda00640 1 -- start start 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.058+0000 7f44bda00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44b8072de0 0x7f44b815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.058+0000 7f44bda00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072700 0x7f44b815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.058+0000 7f44bda00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44b8146140 con 0x7f44b8072320 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.058+0000 7f44bda00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44b80cd740 con 0x7f44b8072de0 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.058+0000 7f44bda00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44b80cda20 con 0x7f44b8072700 2024-09-10T10:53:19.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f44b8072320 0x7f44b80cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54260/0 (socket says 172.21.15.28:54260) 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6a00640 1 -- 172.21.15.28:0/330493287 learned_addr learned my addr 172.21.15.28:0/330493287 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6000640 1 --2- 172.21.15.28:0/330493287 >> v2:172.21.15.138:3300/0 conn(0x7f44b8072de0 0x7f44b815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971448006 0 0) 0x7f44b8146140 con 0x7f44b8072320 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f449c003520 con 0x7f44b8072320 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b7400640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072700 0x7f44b815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6000640 1 -- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072700 msgr2=0x7f44b815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6000640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072700 0x7f44b815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6000640 1 -- 172.21.15.28:0/330493287 >> v1:172.21.15.28:6789/0 conn(0x7f44b8072320 legacy=0x7f44b80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1099902765 0 0) 0x7f449c003520 con 0x7f44b8072320 2024-09-10T10:53:19.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.059+0000 7f44b6000640 1 -- 172.21.15.28:0/330493287 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b80cdd00 con 0x7f44b8072de0 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.060+0000 7f44b6000640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072de0 0x7f44b815d1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097a0 tx=0x7f44a402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b2a4a4d7e66fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4037070 con 0x7f44b8072de0 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a402ab10 con 0x7f44b8072de0 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44b5600640 1 -- 172.21.15.28:0/330493287 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8001670 con 0x7f44b8072de0 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 -- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072de0 msgr2=0x7f44b815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072de0 0x7f44b815d1e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097a0 tx=0x7f44a402a080 comp rx=0 tx=0).stop 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 -- 172.21.15.28:0/330493287 shutdown_connections 2024-09-10T10:53:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072700 0x7f44b815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 --2- 172.21.15.28:0/330493287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072de0 0x7f44b815d1e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 -- 172.21.15.28:0/330493287 >> 172.21.15.28:0/330493287 conn(0x7f44b8067d70 msgr2=0x7f44b806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 -- 172.21.15.28:0/330493287 shutdown_connections 2024-09-10T10:53:19.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.061+0000 7f44bda00640 1 -- 172.21.15.28:0/330493287 wait complete. 2024-09-10T10:53:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.063+0000 7f44bda00640 1 Processor -- start 2024-09-10T10:53:19.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.063+0000 7f44bda00640 1 -- start start 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072700 0x7f44b80cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 0x7f44b80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44b815dde0 con 0x7f44b8072320 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44b815dc60 con 0x7f44b8072700 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44bda00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44b815df60 con 0x7f44b8072de0 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44b6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44b6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56606/0 (socket says 172.21.15.28:56606) 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44b6a00640 1 -- 172.21.15.28:0/2829724890 learned_addr learned my addr 172.21.15.28:0/2829724890 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44b6000640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072700 0x7f44b80cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.064+0000 7f44b7400640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 0x7f44b80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 -- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 msgr2=0x7f44b80cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 0x7f44b80cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 -- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072700 msgr2=0x7f44b80cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072700 0x7f44b80cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 -- 172.21.15.28:0/2829724890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b81f83c0 con 0x7f44b8072320 2024-09-10T10:53:19.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b7400640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 0x7f44b80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44b6a00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f44a80011e0 tx=0x7f44a800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8017070 con 0x7f44b8072320 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a8005ad0 con 0x7f44b8072320 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b80b37e0 con 0x7f44b8072320 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8007070 con 0x7f44b8072320 2024-09-10T10:53:19.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.065+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b80b3da0 con 0x7f44b8072320 2024-09-10T10:53:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.066+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b80b35d0 con 0x7f44b8072320 2024-09-10T10:53:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.067+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44a801a070 con 0x7f44b8072320 2024-09-10T10:53:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.069+0000 7f44aea00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.069+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f44a808ae50 con 0x7f44b8072320 2024-09-10T10:53:19.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.069+0000 7f44b6000640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.070+0000 7f44b6000640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 0x7f449806e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f44a4034000 tx=0x7f44a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:19.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.073+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44a8058640 con 0x7f44b8072320 2024-09-10T10:53:19.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.329+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f44b81e78b0 con 0x7f44b8072320 2024-09-10T10:53:19.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.892+0000 7f44aea00640 1 -- 172.21.15.28:0/2829724890 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v232) ==== 101+0+0 (secure 0 0 0) 0x7f44a805c940 con 0x7f44b8072320 2024-09-10T10:53:19.894 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:19.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 msgr2=0x7f449806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 0x7f449806e2f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f44a4034000 tx=0x7f44a402e000 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 msgr2=0x7f44b815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f44a80011e0 tx=0x7f44a800cbe0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 shutdown_connections 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f449806be30 0x7f449806e2f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44b8072de0 0x7f44b80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44b8072700 0x7f44b80cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 --2- 172.21.15.28:0/2829724890 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44b8072320 0x7f44b815b950 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.895+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 >> 172.21.15.28:0/2829724890 conn(0x7f44b8067d70 msgr2=0x7f44b81e80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.896+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 shutdown_connections 2024-09-10T10:53:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.896+0000 7f44bda00640 1 -- 172.21.15.28:0/2829724890 wait complete. 2024-09-10T10:53:19.914 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.990+0000 7f6ed9e0bf40 1 Processor -- start 2024-09-10T10:53:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.990+0000 7f6ed9e0bf40 1 -- start start 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56179524f470 0x561795282080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 0x56179531eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56179529a530 con 0x5617953253e0 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5617951f7b30 con 0x5617953289c0 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561795327ee0 con 0x56179524f470 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56622/0 (socket says 172.21.15.28:56622) 2024-09-10T10:53:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 -- 172.21.15.28:0/1993556546 learned_addr learned my addr 172.21.15.28:0/1993556546 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed6400640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56179524f470 0x561795282080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 0x56179531eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 -- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56179524f470 msgr2=0x561795282080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56179524f470 0x561795282080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 -- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 msgr2=0x56179531eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 0x56179531eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed5a00640 1 -- 172.21.15.28:0/1993556546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617952825c0 con 0x5617953253e0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.991+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 0x56179531eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000c00 tx=0x7f6ec4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7df522c2d9df449 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ecfe00640 1 -- 172.21.15.28:0/1993556546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4054640 con 0x5617953253e0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ecfe00640 1 -- 172.21.15.28:0/1993556546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec40547e0 con 0x5617953253e0 2024-09-10T10:53:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ecfe00640 1 -- 172.21.15.28:0/1993556546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4054b90 con 0x5617953253e0 2024-09-10T10:53:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/1993556546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 msgr2=0x56179531e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.992+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4000c00 tx=0x7f6ec4051000 comp rx=0 tx=0).stop 2024-09-10T10:53:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/1993556546 shutdown_connections 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953289c0 0x56179531eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5617953253e0 0x56179531e960 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/1993556546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56179524f470 0x561795282080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/1993556546 >> 172.21.15.28:0/1993556546 conn(0x56179529e0a0 msgr2=0x56179529e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/1993556546 shutdown_connections 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/1993556546 wait complete. 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.993+0000 7f6ed9e0bf40 1 Processor -- start 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 -- start start 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953253e0 0x56179531fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617953289c0 0x56179531a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561795251b40 con 0x56179524f470 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56179531c230 con 0x5617953253e0 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.994+0000 7f6ed9e0bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56179531c780 con 0x5617953289c0 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56630/0 (socket says 172.21.15.28:56630) 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 learned_addr learned my addr 172.21.15.28:0/3598910015 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617953289c0 0x56179531a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953253e0 0x56179531fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617953289c0 msgr2=0x56179531a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5617953289c0 0x56179531a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953253e0 msgr2=0x56179531fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5617953253e0 0x56179531fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617953216e0 con 0x56179524f470 2024-09-10T10:53:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.995+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800df20 tx=0x7f6ec800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.996+0000 7f6ecea00640 1 -- 172.21.15.28:0/3598910015 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8014020 con 0x56179524f470 2024-09-10T10:53:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.996+0000 7f6ecea00640 1 -- 172.21.15.28:0/3598910015 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec800be90 con 0x56179524f470 2024-09-10T10:53:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.996+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617953219c0 con 0x56179524f470 2024-09-10T10:53:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.996+0000 7f6ecea00640 1 -- 172.21.15.28:0/3598910015 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8010a60 con 0x56179524f470 2024-09-10T10:53:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.996+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561795321f80 con 0x56179524f470 2024-09-10T10:53:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.998+0000 7f6ecea00640 1 -- 172.21.15.28:0/3598910015 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ec8017020 con 0x56179524f470 2024-09-10T10:53:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.998+0000 7f6ecea00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 0x7f6eb806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.998+0000 7f6ecea00640 1 -- 172.21.15.28:0/3598910015 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f6ec808aa90 con 0x56179524f470 2024-09-10T10:53:20.002 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.999+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 0x56179532a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.999+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x56179531a980 con 0x561795329e70 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.999+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 0x7f6eb806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:19.999+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 0x56179532a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.000+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 0x7f6eb806ccd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400a520 tx=0x7f6ec4053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.000+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 0x56179532a250 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.002+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v125'753 uv727 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.002+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x561795280c90 con 0x561795329e70 2024-09-10T10:53:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.004+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v125'754 uv754 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.004+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x561795281060 con 0x561795329e70 2024-09-10T10:53:20.008 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:20.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.006+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v125'755 uv755 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.006+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb5b0 con 0x561795329e70 2024-09-10T10:53:20.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.008+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v125'756 uv756 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.008+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb950 con 0x561795329e70 2024-09-10T10:53:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.010+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v125'757 uv757 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.010+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ca5a0 con 0x561795329e70 2024-09-10T10:53:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.011+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v125'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.011+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ca940 con 0x561795329e70 2024-09-10T10:53:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.013+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v125'759 uv733 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.013+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953cace0 con 0x561795329e70 2024-09-10T10:53:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.016+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v125'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.016+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.019 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.017+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v125'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.018+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.020+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v125'762 uv762 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.020+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.021+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v125'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.021+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.023+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v125'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.023+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.025+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v125'765 uv740 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.025+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.026+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v125'766 uv766 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.026+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.030 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:20.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.029+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v125'767 uv767 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.029+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.031+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v125'768 uv768 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.031+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.032+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v125'769 uv769 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.032+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cb080 con 0x561795329e70 2024-09-10T10:53:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.034+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v125'770 uv770 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.034+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.036+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v125'771 uv771 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.036+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.037+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v125'772 uv746 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.037+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.038+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v125'773 uv773 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.038+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.042 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:20.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.041+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v125'774 uv774 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.041+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ccd00 con 0x561795329e70 2024-09-10T10:53:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.043+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v125'775 uv775 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.043+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ccd00 con 0x561795329e70 2024-09-10T10:53:20.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.044+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v125'776 uv776 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.044+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.045+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v125'777 uv777 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.046+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.047+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v125'778 uv752 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.047+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.050+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v125'779 uv779 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.050+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.053 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:20.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.051+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v125'780 uv780 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.051+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ccd00 con 0x561795329e70 2024-09-10T10:53:20.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.053+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v125'781 uv781 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.053+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ccd00 con 0x561795329e70 2024-09-10T10:53:20.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.054+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v125'782 uv782 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.054+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cbcf0 con 0x561795329e70 2024-09-10T10:53:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.057+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v125'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x561795329e70 2024-09-10T10:53:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.057+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 0x5617953ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.057+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953ceb30 con 0x5617953cc200 2024-09-10T10:53:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.057+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 0x5617953ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.057+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 0x5617953ce5f0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.060+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v125'549 uv531 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.060+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953ceed0 con 0x5617953cc200 2024-09-10T10:53:20.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.061+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v125'550 uv550 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.061+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cf2b0 con 0x5617953cc200 2024-09-10T10:53:20.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.063+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v125'551 uv551 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.065 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:20.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.063+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x561795322860 con 0x5617953cc200 2024-09-10T10:53:20.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.064+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v125'552 uv552 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.064+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953cf740 con 0x5617953cc200 2024-09-10T10:53:20.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.066+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v125'553 uv553 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.066+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0090 con 0x5617953cc200 2024-09-10T10:53:20.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.067+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v125'554 uv548 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.067+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0480 con 0x5617953cc200 2024-09-10T10:53:20.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.068+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v125'555 uv555 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.068+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0870 con 0x5617953cc200 2024-09-10T10:53:20.071 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:20.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.070+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v125'556 uv556 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.070+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0870 con 0x5617953cc200 2024-09-10T10:53:20.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.071+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v125'557 uv557 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.071+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.073+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v125'558 uv558 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.073+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.074+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v125'559 uv541 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.074+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.075+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v125'560 uv560 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.075+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.078 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:20.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.076+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v125'561 uv561 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.077+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.078+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v125'562 uv562 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.078+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.079+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v125'563 uv563 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.080+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.081+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v125'564 uv547 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.081+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.082+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v125'565 uv565 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.082+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.085 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:20.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.084+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v125'566 uv566 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.084+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.085+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v125'567 uv567 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.085+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.086+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v125'568 uv568 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.086+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d0820 con 0x5617953cc200 2024-09-10T10:53:20.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.088+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v125'569 uv569 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953cc200 2024-09-10T10:53:20.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.088+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5617953d1140 0x5617953d1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.088+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d3d70 con 0x5617953d1140 2024-09-10T10:53:20.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.088+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5617953d1140 0x5617953d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.089+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5617953d1140 0x5617953d1520 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.090+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v125'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.090+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d4110 con 0x5617953d1140 2024-09-10T10:53:20.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.091+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.091+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d44b0 con 0x5617953d1140 2024-09-10T10:53:20.094 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:20.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.092+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.092+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d5860 con 0x5617953d1140 2024-09-10T10:53:20.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.094+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.094+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d5ce0 con 0x5617953d1140 2024-09-10T10:53:20.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.095+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v125'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.095+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d4850 con 0x5617953d1140 2024-09-10T10:53:20.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.096+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v125'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.096+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5617953d5310 0x5617953d8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.096+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8660 con 0x5617953d5310 2024-09-10T10:53:20.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.096+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5617953d5310 0x5617953d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.097+0000 7f6ed6e00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5617953d5310 0x5617953d8090 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.098+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v125'107 uv106 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.098+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8a30 con 0x5617953d5310 2024-09-10T10:53:20.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.099+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v125'108 uv108 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.099+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8a30 con 0x5617953d5310 2024-09-10T10:53:20.103 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:20.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.101+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v125'109 uv109 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.101+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8a30 con 0x5617953d5310 2024-09-10T10:53:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.102+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v125'110 uv110 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.102+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8970 con 0x5617953d1140 2024-09-10T10:53:20.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.103+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v125'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.103+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8e50 con 0x5617953d1140 2024-09-10T10:53:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.104+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.104+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8e50 con 0x5617953d1140 2024-09-10T10:53:20.107 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.105+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.105+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d9e60 con 0x5617953d1140 2024-09-10T10:53:20.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.106+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.107+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d9e60 con 0x5617953d1140 2024-09-10T10:53:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.108+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v125'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.108+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8e50 con 0x5617953d1140 2024-09-10T10:53:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.109+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v125'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.109+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8e50 con 0x5617953d5310 2024-09-10T10:53:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.110+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v125'271 uv265 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.110+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8e50 con 0x5617953d5310 2024-09-10T10:53:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.112+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v125'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.112+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8df0 con 0x5617953d5310 2024-09-10T10:53:20.115 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.113+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v125'273 uv273 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.114+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8dc0 con 0x5617953d5310 2024-09-10T10:53:20.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.115+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v125'274 uv274 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.115+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d80 con 0x5617953d5310 2024-09-10T10:53:20.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.116+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v125'275 uv275 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.116+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.118+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v125'276 uv276 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.118+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.120+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v125'277 uv270 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.120+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.121+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v125'278 uv278 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.121+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.125 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:20.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.123+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v125'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.123+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.125+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v125'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.125+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d5310 2024-09-10T10:53:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.126+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v125'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.126+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d60 con 0x5617953d1140 2024-09-10T10:53:20.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.127+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v125'122 uv121 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.127+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d10 con 0x5617953d1140 2024-09-10T10:53:20.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.129+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v125'123 uv123 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.129+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d10 con 0x5617953d1140 2024-09-10T10:53:20.132 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:20.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.130+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v125'124 uv124 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.130+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d10 con 0x5617953d1140 2024-09-10T10:53:20.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.131+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v125'125 uv125 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.131+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953d8d10 con 0x5617953d1140 2024-09-10T10:53:20.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.132+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v125'126 uv126 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953d1140 2024-09-10T10:53:20.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.132+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5617953d97c0 0x5617953dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.132+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc0f0 con 0x5617953d97c0 2024-09-10T10:53:20.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.133+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5617953d97c0 0x5617953dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.133+0000 7f6ed6400640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5617953d97c0 0x5617953dbbb0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.135+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v125'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.135+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc130 con 0x5617953d97c0 2024-09-10T10:53:20.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.136+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.136+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.139 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:20.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.138+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.138+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dd560 con 0x5617953d97c0 2024-09-10T10:53:20.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.139+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.139+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dd560 con 0x5617953d97c0 2024-09-10T10:53:20.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.140+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v125'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.140+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.142+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v125'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.142+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.143+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v125'243 uv237 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.143+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.144+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v125'244 uv244 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.144+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.147 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:20.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.145+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v125'245 uv245 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.145+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.147+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v125'246 uv246 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.147+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.148+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v125'247 uv247 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.148+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.149+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v125'248 uv242 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.149+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.151+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v125'249 uv249 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.151+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.154 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:20.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.152+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v125'250 uv250 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.152+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dc550 con 0x5617953d97c0 2024-09-10T10:53:20.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.154+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v125'251 uv251 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.154+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df360 con 0x5617953d97c0 2024-09-10T10:53:20.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.155+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v125'252 uv252 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.155+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.157+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v125'107 uv106 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.157+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.158+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v125'108 uv108 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.158+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.161 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.159+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v125'109 uv109 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.159+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.161+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v125'110 uv110 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.161+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.162+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v125'147 uv146 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.162+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.163+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v125'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.163+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.166 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:20.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.164+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v125'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.164+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.166+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v125'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.166+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.167+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v125'151 uv151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.167+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953dca00 con 0x5617953d5310 2024-09-10T10:53:20.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.168+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v125'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ed0011040 con 0x5617953d5310 2024-09-10T10:53:20.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.168+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5617953dd080 0x5617953df960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.168+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953dd080 2024-09-10T10:53:20.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.168+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5617953dd080 0x5617953df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.169+0000 7f6ed5a00640 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5617953dd080 0x5617953df960 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.170+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v125'125 uv124 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953dd080 2024-09-10T10:53:20.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.170+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953dd080 2024-09-10T10:53:20.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.171+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v125'126 uv126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953dd080 2024-09-10T10:53:20.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.171+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953dd080 2024-09-10T10:53:20.174 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:20.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.172+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v125'127 uv127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953dd080 2024-09-10T10:53:20.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.173+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953dd080 2024-09-10T10:53:20.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.174+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v125'128 uv128 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953dd080 2024-09-10T10:53:20.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.174+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953dd080 2024-09-10T10:53:20.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.175+0000 7f6ed5a00640 1 -- 172.21.15.28:0/3598910015 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v125'129 uv129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f6ec4054cc0 con 0x5617953dd080 2024-09-10T10:53:20.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.175+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.176+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v125'302 uv294 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.176+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.178+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v125'303 uv303 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.178+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.181 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.179+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v125'304 uv304 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.179+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.181+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v125'305 uv305 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.181+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.182+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v125'306 uv306 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.182+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.183+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v125'307 uv301 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.183+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.184+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v125'308 uv308 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.185+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.187 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.186+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v125'309 uv309 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.186+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.187+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v125'310 uv310 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.187+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.188+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v125'311 uv311 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.188+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953df4b0 con 0x5617953d97c0 2024-09-10T10:53:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.189+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v125'312 uv312 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.189+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5617953e0350 con 0x5617953d97c0 2024-09-10T10:53:20.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed6400640 1 -- 172.21.15.28:0/3598910015 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v125'313 uv313 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f6ec8020070 con 0x5617953d97c0 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5617953d1140 msgr2=0x5617953d1520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5617953d1140 0x5617953d1520 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5617953d97c0 msgr2=0x5617953dbbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5617953d97c0 0x5617953dbbb0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5617953dd080 msgr2=0x5617953df960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5617953dd080 0x5617953df960 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5617953d5310 msgr2=0x5617953d8090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5617953d5310 0x5617953d8090 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 msgr2=0x5617953ce5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 reap_dead start 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 0x5617953ce5f0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed6e00640 1 -- 172.21.15.28:0/3598910015 reap_dead start 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 msgr2=0x56179532a250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 0x56179532a250 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 msgr2=0x7f6eb806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 0x7f6eb806ccd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f6ec400a520 tx=0x7f6ec4053000 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 msgr2=0x56179531b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800df20 tx=0x7f6ec800b6c0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 shutdown_connections 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5617953cc200 0x5617953ce5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561795329e70 0x56179532a250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6eb806a810 0x7f6eb806ccd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 --2- 172.21.15.28:0/3598910015 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56179524f470 0x56179531b690 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 >> 172.21.15.28:0/3598910015 conn(0x56179529e0a0 msgr2=0x56179529c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:20.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 shutdown_connections 2024-09-10T10:53:20.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.191+0000 7f6ed9e0bf40 1 -- 172.21.15.28:0/3598910015 wait complete. 2024-09-10T10:53:20.198 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 200.00000001 0 2024-09-10T10:53:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.314+0000 7f9b868faf40 1 Processor -- start 2024-09-10T10:53:20.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 -- start start 2024-09-10T10:53:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57da480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5634c574d530 con 0x5634c57db9c0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634c56aab30 con 0x5634c574c8e0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.315+0000 7f9b868faf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5634c57daee0 con 0x5634c57d83e0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56658/0 (socket says 172.21.15.28:56658) 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 -- 172.21.15.28:0/910401243 learned_addr learned my addr 172.21.15.28:0/910401243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b83000640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b83a00640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57da480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 -- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 msgr2=0x5634c57cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 -- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 msgr2=0x5634c57da480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:20.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57da480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 -- 172.21.15.28:0/910401243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c57d74f0 con 0x5634c57db9c0 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b83a00640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.316+0000 7f9b84400640 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c001660 tx=0x7f9b7c001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401e59861096bb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b81c00640 1 -- 172.21.15.28:0/910401243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003650 con 0x5634c57db9c0 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b81c00640 1 -- 172.21.15.28:0/910401243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c003830 con 0x5634c57db9c0 2024-09-10T10:53:20.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b81c00640 1 -- 172.21.15.28:0/910401243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003ca0 con 0x5634c57db9c0 2024-09-10T10:53:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 -- 172.21.15.28:0/910401243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 msgr2=0x5634c57d93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c001660 tx=0x7f9b7c001b20 comp rx=0 tx=0).stop 2024-09-10T10:53:20.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 -- 172.21.15.28:0/910401243 shutdown_connections 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 --2- 172.21.15.28:0/910401243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57da480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 -- 172.21.15.28:0/910401243 >> 172.21.15.28:0/910401243 conn(0x5634c57510a0 msgr2=0x5634c574cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 -- 172.21.15.28:0/910401243 shutdown_connections 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.317+0000 7f9b868faf40 1 -- 172.21.15.28:0/910401243 wait complete. 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.318+0000 7f9b868faf40 1 Processor -- start 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.318+0000 7f9b868faf40 1 -- start start 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5634c574f3d0 con 0x5634c57db9c0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5634c57da9c0 con 0x5634c574c8e0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b868faf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5634c5702320 con 0x5634c57d83e0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b84400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b84400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56670/0 (socket says 172.21.15.28:56670) 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b84400640 1 -- 172.21.15.28:0/2694096593 learned_addr learned my addr 172.21.15.28:0/2694096593 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b83a00640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.319+0000 7f9b83000640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 -- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 msgr2=0x5634c57caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5634c57d83e0 0x5634c57caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 -- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 msgr2=0x5634c57cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5634c574c8e0 0x5634c57cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 -- 172.21.15.28:0/2694096593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c57cb450 con 0x5634c57db9c0 2024-09-10T10:53:20.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b84400640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000f80 tx=0x7f9b7c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b7be00640 1 -- 172.21.15.28:0/2694096593 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c025070 con 0x5634c57db9c0 2024-09-10T10:53:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b7be00640 1 -- 172.21.15.28:0/2694096593 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c019040 con 0x5634c57db9c0 2024-09-10T10:53:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5634c57cf220 con 0x5634c57db9c0 2024-09-10T10:53:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.320+0000 7f9b7be00640 1 -- 172.21.15.28:0/2694096593 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c013880 con 0x5634c57db9c0 2024-09-10T10:53:20.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.321+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c57cb6f0 con 0x5634c57db9c0 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.322+0000 7f9b7be00640 1 -- 172.21.15.28:0/2694096593 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9b7c010070 con 0x5634c57db9c0 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.323+0000 7f9b7be00640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b6406a810 0x7f9b6406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.323+0000 7f9b7be00640 1 -- 172.21.15.28:0/2694096593 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f9b7c08b7c0 con 0x5634c57db9c0 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.323+0000 7f9b868faf40 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5634c57d9920 0x5634c57dce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.323+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:1 13.14 13:2e2fa760:::200.00000001:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) -- 0x5634c57cafc0 con 0x5634c57d9920 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.324+0000 7f9b83a00640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b6406a810 0x7f9b6406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.324+0000 7f9b83000640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5634c57d9920 0x5634c57dce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.324+0000 7f9b83a00640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b6406a810 0x7f9b6406ccd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9b74057000 tx=0x7f9b74052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.325+0000 7f9b83000640 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5634c57d9920 0x5634c57dce50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b83000640 1 -- 172.21.15.28:0/2694096593 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(1 200.00000001 [truncate 0] v125'570 uv570 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f9b70009930 con 0x5634c57d9920 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5634c57d9920 msgr2=0x5634c57dce50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5634c57d9920 0x5634c57dce50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b6406a810 msgr2=0x7f9b6406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b6406a810 0x7f9b6406ccd0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9b74057000 tx=0x7f9b74052000 comp rx=0 tx=0).stop 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 msgr2=0x5634c57d93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 --2- 172.21.15.28:0/2694096593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5634c57db9c0 0x5634c57d93e0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000f80 tx=0x7f9b7c001250 comp rx=0 tx=0).stop 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b84400640 1 -- 172.21.15.28:0/2694096593 reap_dead start 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 shutdown_connections 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.326+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 >> 172.21.15.28:0/2694096593 conn(0x5634c57510a0 msgr2=0x5634c574cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.327+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 shutdown_connections 2024-09-10T10:53:20.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.327+0000 7f9b868faf40 1 -- 172.21.15.28:0/2694096593 wait complete. 2024-09-10T10:53:20.333 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:20.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.490+0000 7f8feae00640 1 Processor -- start 2024-09-10T10:53:20.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.490+0000 7f8feae00640 1 -- start start 2024-09-10T10:53:20.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8feae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fe4161ac0 0x7f8fe406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8feae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 0x7f8fe40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8feae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fe40b1170 con 0x7f8fe40b3640 2024-09-10T10:53:20.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8feae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8fe40ceba0 con 0x7f8fe4161ac0 2024-09-10T10:53:20.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8feae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe406dfb0 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8fe9000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8fe40b3640 0x7f8fe40d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54290/0 (socket says 172.21.15.28:54290) 2024-09-10T10:53:20.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8fe9000640 1 -- 172.21.15.28:0/788941474 learned_addr learned my addr 172.21.15.28:0/788941474 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:20.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8fea400640 1 --2- 172.21.15.28:0/788941474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 0x7f8fe40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.491+0000 7f8fe9a00640 1 --2- 172.21.15.28:0/788941474 >> v2:172.21.15.138:3300/0 conn(0x7f8fe4161ac0 0x7f8fe406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.492+0000 7f8fe3e00640 1 -- 172.21.15.28:0/788941474 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014594123 0 0) 0x7f8fe40b1170 con 0x7f8fe40b3640 2024-09-10T10:53:20.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.492+0000 7f8fea400640 1 -- 172.21.15.28:0/788941474 >> v2:172.21.15.138:3300/0 conn(0x7f8fe4161ac0 msgr2=0x7f8fe406e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:20.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.492+0000 7f8fea400640 1 --2- 172.21.15.28:0/788941474 >> v2:172.21.15.138:3300/0 conn(0x7f8fe4161ac0 0x7f8fe406e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.492+0000 7f8fea400640 1 -- 172.21.15.28:0/788941474 >> v1:172.21.15.28:6789/0 conn(0x7f8fe40b3640 legacy=0x7f8fe40d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.492+0000 7f8fea400640 1 -- 172.21.15.28:0/788941474 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe40b1450 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.493+0000 7f8fea400640 1 --2- 172.21.15.28:0/788941474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 0x7f8fe40b1f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d90 tx=0x7f8fdc02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1b7a62ed956e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.493+0000 7f8fe3e00640 1 -- 172.21.15.28:0/788941474 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc007740 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.493+0000 7f8fe3e00640 1 -- 172.21.15.28:0/788941474 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fdc0078e0 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.493+0000 7f8fe3e00640 1 -- 172.21.15.28:0/788941474 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc007cb0 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 -- 172.21.15.28:0/788941474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 msgr2=0x7f8fe40b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 --2- 172.21.15.28:0/788941474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 0x7f8fe40b1f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d90 tx=0x7f8fdc02a6f0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 -- 172.21.15.28:0/788941474 shutdown_connections 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 --2- 172.21.15.28:0/788941474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40d2ac0 0x7f8fe40b1f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 --2- 172.21.15.28:0/788941474 >> v2:172.21.15.138:3300/0 conn(0x7f8fe4161ac0 0x7f8fe406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 -- 172.21.15.28:0/788941474 >> 172.21.15.28:0/788941474 conn(0x7f8fe408bfc0 msgr2=0x7f8fe40d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 -- 172.21.15.28:0/788941474 shutdown_connections 2024-09-10T10:53:20.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.494+0000 7f8feae00640 1 -- 172.21.15.28:0/788941474 wait complete. 2024-09-10T10:53:20.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.496+0000 7f8feae00640 1 Processor -- start 2024-09-10T10:53:20.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.496+0000 7f8feae00640 1 -- start start 2024-09-10T10:53:20.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 0x7f8fe415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fe40d2ac0 0x7f8fe415aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe4161ea0 con 0x7f8fe4161ac0 2024-09-10T10:53:20.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe40b1a10 con 0x7f8fe40d2ac0 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8feae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fe4162020 con 0x7f8fe40b3640 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fea400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fea400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56682/0 (socket says 172.21.15.28:56682) 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fea400640 1 -- 172.21.15.28:0/1882844560 learned_addr learned my addr 172.21.15.28:0/1882844560 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 0x7f8fe415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9000640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fe40d2ac0 0x7f8fe415aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 -- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fe40d2ac0 msgr2=0x7f8fe415aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fe40d2ac0 0x7f8fe415aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 -- 172.21.15.28:0/1882844560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 msgr2=0x7f8fe40cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fe9a00640 1 -- 172.21.15.28:0/1882844560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe415e2e0 con 0x7f8fe40b3640 2024-09-10T10:53:20.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.497+0000 7f8fea400640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8fe9a00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 0x7f8fe415b4c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e8a0 tx=0x7f8fd800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8017070 con 0x7f8fe40b3640 2024-09-10T10:53:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fd800c940 con 0x7f8fe40b3640 2024-09-10T10:53:20.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8009e60 con 0x7f8fe40b3640 2024-09-10T10:53:20.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe415e5c0 con 0x7f8fe40b3640 2024-09-10T10:53:20.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.498+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe40cc000 con 0x7f8fe40b3640 2024-09-10T10:53:20.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.499+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe406e6c0 con 0x7f8fe40b3640 2024-09-10T10:53:20.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.500+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8fd801a070 con 0x7f8fe40b3640 2024-09-10T10:53:20.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.501+0000 7f8fe1600640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:20.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.501+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(125..125 src has 1..125) ==== 8454+0+0 (secure 0 0 0) 0x7f8fd808baf0 con 0x7f8fe40b3640 2024-09-10T10:53:20.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.502+0000 7f8fe9000640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:20.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.502+0000 7f8fe9000640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 0x7f8fc406e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cafd0 tx=0x7f8fd0007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:20.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.507+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8fd80592e0 con 0x7f8fe40b3640 2024-09-10T10:53:20.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.729+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8fe406d740 con 0x7f8fe40b3640 2024-09-10T10:53:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.898+0000 7f8fe1600640 1 -- 172.21.15.28:0/1882844560 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v233) ==== 162+0+0 (secure 0 0 0) 0x7f8fd805d5e0 con 0x7f8fe40b3640 2024-09-10T10:53:20.900 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.900+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 msgr2=0x7f8fc406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.900+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 0x7f8fc406e2f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40cafd0 tx=0x7f8fd0007d50 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.900+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 msgr2=0x7f8fe415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.900+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 0x7f8fe415b4c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e8a0 tx=0x7f8fd800ed60 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 shutdown_connections 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8fc406be30 0x7f8fc406e2f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8fe4161ac0 0x7f8fe40cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fe40d2ac0 0x7f8fe415aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 --2- 172.21.15.28:0/1882844560 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fe40b3640 0x7f8fe415b4c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:20.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 >> 172.21.15.28:0/1882844560 conn(0x7f8fe408bfc0 msgr2=0x7f8fe40d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 shutdown_connections 2024-09-10T10:53:20.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:20.901+0000 7f8feae00640 1 -- 172.21.15.28:0/1882844560 wait complete. 2024-09-10T10:53:20.909 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:20.907+0000 7f2f67400640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:53:20.910 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:20.908+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:20.910 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:20.908+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:20.922 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:21.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.076+0000 7f5b55800640 1 Processor -- start 2024-09-10T10:53:21.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.076+0000 7f5b55800640 1 -- start start 2024-09-10T10:53:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b50072320 0x7f5b500cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b50072de0 0x7f5b5006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072700 0x7f5b5006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5b50146140 con 0x7f5b50072de0 2024-09-10T10:53:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5b500cd740 con 0x7f5b50072320 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b55800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b500cda20 con 0x7f5b50072700 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b50072de0 0x7f5b5006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b50072de0 0x7f5b5006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56698/0 (socket says 172.21.15.28:56698) 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 -- 172.21.15.28:0/2126318192 learned_addr learned my addr 172.21.15.28:0/2126318192 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b4f400640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072700 0x7f5b5006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b4ea00640 1 --2- 172.21.15.28:0/2126318192 >> v2:172.21.15.138:3300/0 conn(0x7f5b50072320 0x7f5b500cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 -- 172.21.15.28:0/2126318192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072700 msgr2=0x7f5b5006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072700 0x7f5b5006e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 -- 172.21.15.28:0/2126318192 >> v2:172.21.15.138:3300/0 conn(0x7f5b50072320 msgr2=0x7f5b500cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 --2- 172.21.15.28:0/2126318192 >> v2:172.21.15.138:3300/0 conn(0x7f5b50072320 0x7f5b500cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.078+0000 7f5b47e00640 1 -- 172.21.15.28:0/2126318192 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b500cdd00 con 0x7f5b50072de0 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.079+0000 7f5b47e00640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b5006e0b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5b500b31e0 tx=0x7f5b3802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de44c6fb71b7abae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b4e000640 1 -- 172.21.15.28:0/2126318192 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38037070 con 0x7f5b50072de0 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b4e000640 1 -- 172.21.15.28:0/2126318192 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b38032440 con 0x7f5b50072de0 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b4e000640 1 -- 172.21.15.28:0/2126318192 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38032810 con 0x7f5b50072de0 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 -- 172.21.15.28:0/2126318192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 msgr2=0x7f5b5006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b5006e0b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5b500b31e0 tx=0x7f5b3802a140 comp rx=0 tx=0).stop 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 -- 172.21.15.28:0/2126318192 shutdown_connections 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072700 0x7f5b5006e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 --2- 172.21.15.28:0/2126318192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b5006e0b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 --2- 172.21.15.28:0/2126318192 >> v2:172.21.15.138:3300/0 conn(0x7f5b50072320 0x7f5b500cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 -- 172.21.15.28:0/2126318192 >> 172.21.15.28:0/2126318192 conn(0x7f5b50067d70 msgr2=0x7f5b5006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 -- 172.21.15.28:0/2126318192 shutdown_connections 2024-09-10T10:53:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.080+0000 7f5b55800640 1 -- 172.21.15.28:0/2126318192 wait complete. 2024-09-10T10:53:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.082+0000 7f5b55800640 1 Processor -- start 2024-09-10T10:53:21.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.082+0000 7f5b55800640 1 -- start start 2024-09-10T10:53:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072320 0x7f5b500cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50072700 0x7f5b5013ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b500cd520 con 0x7f5b50072de0 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b500cd3a0 con 0x7f5b50072700 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b55800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5006eb30 con 0x7f5b50072320 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56714/0 (socket says 172.21.15.28:56714) 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.083+0000 7f5b4f400640 1 -- 172.21.15.28:0/2145910202 learned_addr learned my addr 172.21.15.28:0/2145910202 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4ea00640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072320 0x7f5b500cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b47e00640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50072700 0x7f5b5013ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 -- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072320 msgr2=0x7f5b500cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072320 0x7f5b500cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 -- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50072700 msgr2=0x7f5b5013ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50072700 0x7f5b5013ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 -- 172.21.15.28:0/2145910202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b500c6ee0 con 0x7f5b50072de0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b4f400640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5b48007e30 tx=0x7f5b4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48014620 con 0x7f5b50072de0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b48019070 con 0x7f5b50072de0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48014a00 con 0x7f5b50072de0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.084+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b500c71c0 con 0x7f5b50072de0 2024-09-10T10:53:21.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.085+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b500c7780 con 0x7f5b50072de0 2024-09-10T10:53:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.086+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b500cce60 con 0x7f5b50072de0 2024-09-10T10:53:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.087+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b48014ba0 con 0x7f5b50072de0 2024-09-10T10:53:21.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.088+0000 7f5b46a00640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 0x7f5b2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.088+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f5b4808c2a0 con 0x7f5b50072de0 2024-09-10T10:53:21.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.088+0000 7f5b4ea00640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 0x7f5b2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.089+0000 7f5b4ea00640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 0x7f5b2c06e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006fd0 tx=0x7f5b40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.092+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b4805a120 con 0x7f5b50072de0 2024-09-10T10:53:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.314+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b500c6770 con 0x7f5b50072de0 2024-09-10T10:53:21.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.315+0000 7f5b46a00640 1 -- 172.21.15.28:0/2145910202 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) ==== 95+0+3566 (secure 0 0 0) 0x7f5b48059a20 con 0x7f5b50072de0 2024-09-10T10:53:21.317 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:21.317 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":235,"btime":"2024-09-10T10:53:20:909980+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":235,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:20.909923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 235 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.317+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 msgr2=0x7f5b2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.317+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 0x7f5b2c06e2f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b40006fd0 tx=0x7f5b40008040 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.317+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 msgr2=0x7f5b50143dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.317+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5b48007e30 tx=0x7f5b4800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 shutdown_connections 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b2c06be30 0x7f5b2c06e2f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b50072de0 0x7f5b50143dd0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b50072700 0x7f5b5013ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 --2- 172.21.15.28:0/2145910202 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b50072320 0x7f5b500cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 >> 172.21.15.28:0/2145910202 conn(0x7f5b50067d70 msgr2=0x7f5b500cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 shutdown_connections 2024-09-10T10:53:21.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.318+0000 7f5b55800640 1 -- 172.21.15.28:0/2145910202 wait complete. 2024-09-10T10:53:21.335 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:21.335 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.492+0000 7f5b0ce00640 1 Processor -- start 2024-09-10T10:53:21.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.492+0000 7f5b0ce00640 1 -- start start 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5b0806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b0ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b0809aa30 0x7f5b0809aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b0ce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b0806ddd0 con 0x7f5b081629e0 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b0ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5b0809b3e0 con 0x7f5b0809aa30 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b0ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0806e0b0 con 0x7f5b080b47b0 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b07400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5b081629e0 0x7f5b08070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54296/0 (socket says 172.21.15.28:54296) 2024-09-10T10:53:21.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b07400640 1 -- 172.21.15.28:0/609896251 learned_addr learned my addr 172.21.15.28:0/609896251 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5b07e00640 1 --2- 172.21.15.28:0/609896251 >> v2:172.21.15.138:3300/0 conn(0x7f5b0809aa30 0x7f5b0809aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.493+0000 7f5affe00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5b0806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749823310 0 0) 0x7f5b0806ddd0 con 0x7f5b081629e0 2024-09-10T10:53:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aec003520 con 0x7f5b081629e0 2024-09-10T10:53:21.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2777650312 0 0) 0x7f5aec003520 con 0x7f5b081629e0 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 msgr2=0x7f5b0806e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5b0806e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 >> v2:172.21.15.138:3300/0 conn(0x7f5b0809aa30 msgr2=0x7f5b0809aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 --2- 172.21.15.28:0/609896251 >> v2:172.21.15.138:3300/0 conn(0x7f5b0809aa30 0x7f5b0809aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080d1b60 con 0x7f5b081629e0 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f5af8001670 con 0x7f5b081629e0 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 >> v1:172.21.15.28:6789/0 conn(0x7f5b081629e0 legacy=0x7f5b08070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5aec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5aec003f30 0x7f5aec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aec004d40 0x7f5aec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0809b3e0 con 0x7f5aec003f30 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b0806e0b0 con 0x7f5aec004d40 2024-09-10T10:53:21.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5aec00ba10 con 0x7f5b080b47b0 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5affe00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5aec003f30 0x7f5aec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b07400640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5aec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.494+0000 7f5b07e00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aec004d40 0x7f5aec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5affe00640 1 -- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 msgr2=0x7f5aec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5affe00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b080b47b0 0x7f5aec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5affe00640 1 -- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aec004d40 msgr2=0x7f5aec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5affe00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aec004d40 0x7f5aec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5affe00640 1 -- 172.21.15.28:0/609896251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec003520 con 0x7f5aec003f30 2024-09-10T10:53:21.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.495+0000 7f5b07e00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aec004d40 0x7f5aec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:21.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5affe00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5aec003f30 0x7f5aec004800 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5aec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd2e63300c3656e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0036070 con 0x7f5aec003f30 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af0031400 con 0x7f5aec003f30 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b06a00640 1 -- 172.21.15.28:0/609896251 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0009690 con 0x7f5aec003f30 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b0ce00640 1 -- 172.21.15.28:0/609896251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5aec003f30 msgr2=0x7f5aec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/609896251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5aec003f30 0x7f5aec004800 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5aec005830 comp rx=0 tx=0).stop 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.496+0000 7f5b07e00640 1 -- 172.21.15.28:0/609896251 reap_dead start 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.497+0000 7f5b0ce00640 1 -- 172.21.15.28:0/609896251 shutdown_connections 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.497+0000 7f5b0ce00640 1 -- 172.21.15.28:0/609896251 >> 172.21.15.28:0/609896251 conn(0x7f5b080b3c90 msgr2=0x7f5b080b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.497+0000 7f5b0ce00640 1 -- 172.21.15.28:0/609896251 shutdown_connections 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.497+0000 7f5b0ce00640 1 -- 172.21.15.28:0/609896251 wait complete. 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.498+0000 7f5b0ce00640 1 Processor -- start 2024-09-10T10:53:21.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.498+0000 7f5b0ce00640 1 -- start start 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aec003f30 0x7f5b0814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b08158240 0x7f5b081597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08162f40 con 0x7f5b08158790 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08162dc0 con 0x7f5b08158240 2024-09-10T10:53:21.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.499+0000 7f5b0ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b0806e9d0 con 0x7f5aec003f30 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56726/0 (socket says 172.21.15.28:56726) 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 -- 172.21.15.28:0/192141304 learned_addr learned my addr 172.21.15.28:0/192141304 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5b07400640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aec003f30 0x7f5b0814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5b07e00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b08158240 0x7f5b081597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 -- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aec003f30 msgr2=0x7f5b0814e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aec003f30 0x7f5b0814e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 -- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b08158240 msgr2=0x7f5b081597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b08158240 0x7f5b081597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 -- 172.21.15.28:0/192141304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b080b6bd0 con 0x7f5b08158790 2024-09-10T10:53:21.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.500+0000 7f5affe00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5af0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.501+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0040020 con 0x7f5b08158790 2024-09-10T10:53:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.501+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af0003c60 con 0x7f5b08158790 2024-09-10T10:53:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.501+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0036040 con 0x7f5b08158790 2024-09-10T10:53:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.501+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b080b6e70 con 0x7f5b08158790 2024-09-10T10:53:21.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.501+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b080b7430 con 0x7f5b08158790 2024-09-10T10:53:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.502+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b08160ec0 con 0x7f5b08158790 2024-09-10T10:53:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.503+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5af0043020 con 0x7f5b08158790 2024-09-10T10:53:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.504+0000 7f5aff400640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 0x7f5ae406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.504+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f5af00abe10 con 0x7f5b08158790 2024-09-10T10:53:21.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.504+0000 7f5b07400640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 0x7f5ae406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.505+0000 7f5b07400640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 0x7f5ae406e2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5af8001550 tx=0x7f5af8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.508+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b08160ec0 con 0x7f5b08158790 2024-09-10T10:53:21.630 INFO:tasks.ceph.mds.a.smithi028.stderr: -36> 2024-09-10T10:53:16.875+0000 7fadedc00640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument 2024-09-10T10:53:21.670 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.732+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b0806ec10 con 0x7f5b08158790 2024-09-10T10:53:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.732+0000 7f5aff400640 1 -- 172.21.15.28:0/192141304 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 235 v235) ==== 95+0+3566 (secure 0 0 0) 0x7f5af004a030 con 0x7f5b08158790 2024-09-10T10:53:21.735 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:21.735 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":235,"btime":"2024-09-10T10:53:20:909980+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":235,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:20.909923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:21.735 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 235 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 msgr2=0x7f5ae406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 0x7f5ae406e2f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5af8001550 tx=0x7f5af8009040 comp rx=0 tx=0).stop 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 msgr2=0x7f5b08159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009af0 tx=0x7f5af0004730 comp rx=0 tx=0).stop 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 shutdown_connections 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ae406be30 0x7f5ae406e2f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b08158240 0x7f5b081597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b08158790 0x7f5b08159260 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 --2- 172.21.15.28:0/192141304 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5aec003f30 0x7f5b0814e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 >> 172.21.15.28:0/192141304 conn(0x7f5b080b3c90 msgr2=0x7f5b081610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:21.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 shutdown_connections 2024-09-10T10:53:21.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.735+0000 7f5b0ce00640 1 -- 172.21.15.28:0/192141304 wait complete. 2024-09-10T10:53:21.752 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:21.752 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.906+0000 7fd6a2800640 1 Processor -- start 2024-09-10T10:53:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.906+0000 7fd6a2800640 1 -- start start 2024-09-10T10:53:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.907+0000 7fd6a2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd69c15e740 0x7fd69c0d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd6a2800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd69c06dbd0 0x7fd69c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd6a2800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd69c0d0780 con 0x7fd69c09b690 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd6a2800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c15eb20 con 0x7fd69c15e740 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd6a2800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd69c06d260 con 0x7fd69c06dbd0 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd69c15e740 0x7fd69c0d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd693e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd69c09b690 0x7fd69c06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54310/0 (socket says 172.21.15.28:54310) 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd69c15e740 0x7fd69c0d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56746/0 (socket says 172.21.15.28:56746) 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd693e00640 1 -- 172.21.15.28:0/3896935316 learned_addr learned my addr 172.21.15.28:0/3896935316 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69be00640 1 --2- 172.21.15.28:0/3896935316 >> v2:172.21.15.138:3300/0 conn(0x7fd69c06dbd0 0x7fd69c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 -- 172.21.15.28:0/3896935316 >> v2:172.21.15.138:3300/0 conn(0x7fd69c06dbd0 msgr2=0x7fd69c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 --2- 172.21.15.28:0/3896935316 >> v2:172.21.15.138:3300/0 conn(0x7fd69c06dbd0 0x7fd69c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 -- 172.21.15.28:0/3896935316 >> v1:172.21.15.28:6789/0 conn(0x7fd69c09b690 legacy=0x7fd69c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69b400640 1 -- 172.21.15.28:0/3896935316 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c0d0a60 con 0x7fd69c15e740 2024-09-10T10:53:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.908+0000 7fd69aa00640 1 -- 172.21.15.28:0/3896935316 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 629845000 0 0) 0x7fd69c0d0780 con 0x7fd69c09b690 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.909+0000 7fd69b400640 1 --2- 172.21.15.28:0/3896935316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c15e740 0x7fd69c0d1540 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd69c06cba0 tx=0x7fd68c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2069863f5e64ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.909+0000 7fd69aa00640 1 -- 172.21.15.28:0/3896935316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037070 con 0x7fd69c15e740 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.910+0000 7fd69aa00640 1 -- 172.21.15.28:0/3896935316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c032440 con 0x7fd69c15e740 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.910+0000 7fd69aa00640 1 -- 172.21.15.28:0/3896935316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c032810 con 0x7fd69c15e740 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.910+0000 7fd6a2800640 1 -- 172.21.15.28:0/3896935316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c15e740 msgr2=0x7fd69c0d1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.910+0000 7fd6a2800640 1 --2- 172.21.15.28:0/3896935316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c15e740 0x7fd69c0d1540 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fd69c06cba0 tx=0x7fd68c02a230 comp rx=0 tx=0).stop 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 -- 172.21.15.28:0/3896935316 shutdown_connections 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 --2- 172.21.15.28:0/3896935316 >> v2:172.21.15.138:3300/0 conn(0x7fd69c06dbd0 0x7fd69c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 --2- 172.21.15.28:0/3896935316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c15e740 0x7fd69c0d1540 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 -- 172.21.15.28:0/3896935316 >> 172.21.15.28:0/3896935316 conn(0x7fd69c0d23a0 msgr2=0x7fd69c0d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 -- 172.21.15.28:0/3896935316 shutdown_connections 2024-09-10T10:53:21.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.911+0000 7fd6a2800640 1 -- 172.21.15.28:0/3896935316 wait complete. 2024-09-10T10:53:21.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.912+0000 7fd6a2800640 1 Processor -- start 2024-09-10T10:53:21.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 -- start start 2024-09-10T10:53:21.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c09b690 0x7fd69c1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c15e740 0x7fd69c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c09bac0 con 0x7fd69c06dbd0 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.913+0000 7fd6a2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd69c0d1020 con 0x7fd69c09b690 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd6a2800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd69c06dfe0 con 0x7fd69c15e740 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56752/0 (socket says 172.21.15.28:56752) 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 -- 172.21.15.28:0/4143995915 learned_addr learned my addr 172.21.15.28:0/4143995915 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd693e00640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c09b690 0x7fd69c1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69be00640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c15e740 0x7fd69c0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 -- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c15e740 msgr2=0x7fd69c0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c15e740 0x7fd69c0cb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 -- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c09b690 msgr2=0x7fd69c1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c09b690 0x7fd69c1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.914+0000 7fd69b400640 1 -- 172.21.15.28:0/4143995915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69c0cb9c0 con 0x7fd69c06dbd0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd69b400640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c037040 con 0x7fd69c06dbd0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd68c042070 con 0x7fd69c06dbd0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68c030bd0 con 0x7fd69c06dbd0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69c157d80 con 0x7fd69c06dbd0 2024-09-10T10:53:21.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.915+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd69c0b5600 con 0x7fd69c06dbd0 2024-09-10T10:53:21.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.917+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd69c0d0780 con 0x7fd69c06dbd0 2024-09-10T10:53:21.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.918+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd68c047460 con 0x7fd69c06dbd0 2024-09-10T10:53:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.919+0000 7fd693400640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 0x7fd67806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.919+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7fd68c0ac3f0 con 0x7fd69c06dbd0 2024-09-10T10:53:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.919+0000 7fd693e00640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 0x7fd67806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.920+0000 7fd693e00640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 0x7fd67806e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0cd470 tx=0x7fd684007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:21.923+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd68c032e20 con 0x7fd69c06dbd0 2024-09-10T10:53:22.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.142+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd69c0d1cb0 con 0x7fd69c06dbd0 2024-09-10T10:53:22.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.143+0000 7fd693400640 1 -- 172.21.15.28:0/4143995915 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 236 v236) ==== 95+0+4392 (secure 0 0 0) 0x7fd68c079c20 con 0x7fd69c06dbd0 2024-09-10T10:53:22.145 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:22.145 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":236,"btime":"2024-09-10T10:53:21:911607+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9174,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2097945693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2097945693},{"type":"v1","addr":"172.21.15.28:6835","nonce":2097945693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":236}],"filesystems":[{"mdsmap":{"epoch":235,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:20.909923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:22.146 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 236 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 msgr2=0x7fd67806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 0x7fd67806e2f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0cd470 tx=0x7fd684007de0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 msgr2=0x7fd69c1576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fd68c000c00 tx=0x7fd68c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 shutdown_connections 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd67806be30 0x7fd67806e2f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd69c15e740 0x7fd69c0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd69c09b690 0x7fd69c1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 --2- 172.21.15.28:0/4143995915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd69c06dbd0 0x7fd69c1576f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 >> 172.21.15.28:0/4143995915 conn(0x7fd69c0d23a0 msgr2=0x7fd69c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 shutdown_connections 2024-09-10T10:53:22.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.145+0000 7fd6a2800640 1 -- 172.21.15.28:0/4143995915 wait complete. 2024-09-10T10:53:22.164 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 200.00000001' led to DAMAGED state 2024-09-10T10:53:22.164 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 500.00000000' 2024-09-10T10:53:22.164 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:22.164 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:22.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:22.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:22.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:22.200 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:22.275 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:22.276 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:22.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.470+0000 7fa4d1c00640 1 Processor -- start 2024-09-10T10:53:22.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.470+0000 7fa4d1c00640 1 -- start start 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 0x7fa4cc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa4cc072d30 0x7fa4cc06f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0728a0 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa4cc0cdbe0 con 0x7fa4cc072d30 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.471+0000 7fa4d1c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0cd900 con 0x7fa4cc15b300 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56762/0 (socket says 172.21.15.28:56762) 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 -- 172.21.15.28:0/202580636 learned_addr learned my addr 172.21.15.28:0/202580636 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4cb400640 1 --2- 172.21.15.28:0/202580636 >> v2:172.21.15.138:3300/0 conn(0x7fa4cc072d30 0x7fa4cc06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4caa00640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 0x7fa4cc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 -- 172.21.15.28:0/202580636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 msgr2=0x7fa4cc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 0x7fa4cc0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 -- 172.21.15.28:0/202580636 >> v2:172.21.15.138:3300/0 conn(0x7fa4cc072d30 msgr2=0x7fa4cc06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 --2- 172.21.15.28:0/202580636 >> v2:172.21.15.138:3300/0 conn(0x7fa4cc072d30 0x7fa4cc06f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4ca000640 1 -- 172.21.15.28:0/202580636 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc072250 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.472+0000 7fa4caa00640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 0x7fa4cc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:22.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.473+0000 7fa4ca000640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc072040 tx=0x7fa4b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4be7b42384227f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:22.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.473+0000 7fa4c9600640 1 -- 172.21.15.28:0/202580636 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8037070 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.473+0000 7fa4c9600640 1 -- 172.21.15.28:0/202580636 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b8032440 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.473+0000 7fa4c9600640 1 -- 172.21.15.28:0/202580636 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8032810 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 -- 172.21.15.28:0/202580636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 msgr2=0x7fa4cc06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc072040 tx=0x7fa4b802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 -- 172.21.15.28:0/202580636 shutdown_connections 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/202580636 >> v2:172.21.15.138:3300/0 conn(0x7fa4cc072d30 0x7fa4cc06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc06f1c0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/202580636 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc15b300 0x7fa4cc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 -- 172.21.15.28:0/202580636 >> 172.21.15.28:0/202580636 conn(0x7fa4cc067d70 msgr2=0x7fa4cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 -- 172.21.15.28:0/202580636 shutdown_connections 2024-09-10T10:53:22.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.474+0000 7fa4d1c00640 1 -- 172.21.15.28:0/202580636 wait complete. 2024-09-10T10:53:22.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.476+0000 7fa4d1c00640 1 Processor -- start 2024-09-10T10:53:22.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.476+0000 7fa4d1c00640 1 -- start start 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc072d30 0x7fa4cc1e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 0x7fa4cc1e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0d0160 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4cc0cffe0 con 0x7fa4cc15b300 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4d1c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4cc0b3c40 con 0x7fa4cc072d30 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56770/0 (socket says 172.21.15.28:56770) 2024-09-10T10:53:22.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 -- 172.21.15.28:0/2628614708 learned_addr learned my addr 172.21.15.28:0/2628614708 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4cb400640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 0x7fa4cc1e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4caa00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc072d30 0x7fa4cc1e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 -- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc072d30 msgr2=0x7fa4cc1e36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc072d30 0x7fa4cc1e36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 -- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 msgr2=0x7fa4cc1e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 0x7fa4cc1e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4ca000640 1 -- 172.21.15.28:0/2628614708 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4cc156340 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.477+0000 7fa4cb400640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 0x7fa4cc1e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4ca000640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa4b802a220 tx=0x7fa4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8037040 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b8042070 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8030880 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4cc156620 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.478+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4cc156be0 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.479+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4cc06fb20 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.480+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4b8030a20 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.481+0000 7fa4c2a00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 0x7fa4ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:22.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.481+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7fa4b80ac270 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.482+0000 7fa4caa00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 0x7fa4ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:22.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.483+0000 7fa4caa00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 0x7fa4ac06e2f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc006fd0 tx=0x7fa4bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:22.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.485+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa4cc06fb20 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.708+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa4cc06eea0 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.917+0000 7fa4c2a00640 1 -- 172.21.15.28:0/2628614708 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v237) ==== 161+0+0 (secure 0 0 0) 0x7fa4b807a0f0 con 0x7fa4cc0cd3f0 2024-09-10T10:53:22.919 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.919+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 msgr2=0x7fa4ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.919+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 0x7fa4ac06e2f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc006fd0 tx=0x7fa4bc008040 comp rx=0 tx=0).stop 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.919+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 msgr2=0x7fa4cc0b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.919+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fa4b802a220 tx=0x7fa4b80046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 shutdown_connections 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa4ac06be30 0x7fa4ac06e2f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4cc15b300 0x7fa4cc1e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4cc0cd3f0 0x7fa4cc0b2820 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 --2- 172.21.15.28:0/2628614708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4cc072d30 0x7fa4cc1e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:22.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 >> 172.21.15.28:0/2628614708 conn(0x7fa4cc067d70 msgr2=0x7fa4cc0c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:22.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 shutdown_connections 2024-09-10T10:53:22.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:22.920+0000 7fa4d1c00640 1 -- 172.21.15.28:0/2628614708 wait complete. 2024-09-10T10:53:22.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.093+0000 7fdba3600640 1 Processor -- start 2024-09-10T10:53:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.094+0000 7fdba3600640 1 -- start start 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba3600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb9c0afd70 0x7fdb9c160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba3600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c160d60 con 0x7fdb9c0ce660 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba3600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c06f5f0 con 0x7fdb9c0b5f80 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba3600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c06fdb0 con 0x7fdb9c0afd70 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba0c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb9c0afd70 0x7fdb9c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba0c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdb9c0afd70 0x7fdb9c160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56778/0 (socket says 172.21.15.28:56778) 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba1600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdb9c0ce660 0x7fdb9c06eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54320/0 (socket says 172.21.15.28:54320) 2024-09-10T10:53:23.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.095+0000 7fdba0c00640 1 -- 172.21.15.28:0/428439708 learned_addr learned my addr 172.21.15.28:0/428439708 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.096+0000 7fdba0c00640 1 -- 172.21.15.28:0/428439708 >> v1:172.21.15.138:6789/0 conn(0x7fdb9c0b5f80 legacy=0x7fdb9c06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.096+0000 7fdba0c00640 1 -- 172.21.15.28:0/428439708 >> v1:172.21.15.28:6789/0 conn(0x7fdb9c0ce660 legacy=0x7fdb9c06eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.096+0000 7fdba0c00640 1 -- 172.21.15.28:0/428439708 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c070090 con 0x7fdb9c0afd70 2024-09-10T10:53:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdba0c00640 1 --2- 172.21.15.28:0/428439708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c160820 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009870 tx=0x7fdb9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8877c0a0ba5b2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdb9b400640 1 -- 172.21.15.28:0/428439708 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90037070 con 0x7fdb9c0afd70 2024-09-10T10:53:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdb9b400640 1 -- 172.21.15.28:0/428439708 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb90032440 con 0x7fdb9c0afd70 2024-09-10T10:53:23.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdb9b400640 1 -- 172.21.15.28:0/428439708 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90032810 con 0x7fdb9c0afd70 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdba3600640 1 -- 172.21.15.28:0/428439708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 msgr2=0x7fdb9c160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.097+0000 7fdba3600640 1 --2- 172.21.15.28:0/428439708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c160820 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009870 tx=0x7fdb9002a2a0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.098+0000 7fdba3600640 1 -- 172.21.15.28:0/428439708 shutdown_connections 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.098+0000 7fdba3600640 1 --2- 172.21.15.28:0/428439708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c160820 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.098+0000 7fdba3600640 1 -- 172.21.15.28:0/428439708 >> 172.21.15.28:0/428439708 conn(0x7fdb9c06bc10 msgr2=0x7fdb9c06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.098+0000 7fdba3600640 1 -- 172.21.15.28:0/428439708 shutdown_connections 2024-09-10T10:53:23.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.098+0000 7fdba3600640 1 -- 172.21.15.28:0/428439708 wait complete. 2024-09-10T10:53:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.099+0000 7fdba3600640 1 Processor -- start 2024-09-10T10:53:23.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.099+0000 7fdba3600640 1 -- start start 2024-09-10T10:53:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb9c0b5f80 0x7fdb9c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 0x7fdb9c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:23.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06fa80 con 0x7fdb9c0afd70 2024-09-10T10:53:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c070650 con 0x7fdb9c0b5f80 2024-09-10T10:53:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba3600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0b0150 con 0x7fdb9c0ce660 2024-09-10T10:53:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.100+0000 7fdba0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:56786/0 (socket says 172.21.15.28:56786) 2024-09-10T10:53:23.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 -- 172.21.15.28:0/2299641511 learned_addr learned my addr 172.21.15.28:0/2299641511 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdb9be00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb9c0b5f80 0x7fdb9c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba1600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 0x7fdb9c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 -- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 msgr2=0x7fdb9c14f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 0x7fdb9c14f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 -- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb9c0b5f80 msgr2=0x7fdb9c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb9c0b5f80 0x7fdb9c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 -- 172.21.15.28:0/2299641511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0aac80 con 0x7fdb9c0afd70 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba1600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 0x7fdb9c14f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.101+0000 7fdba0c00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009870 tx=0x7fdb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.102+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90041070 con 0x7fdb9c0afd70 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.102+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb9003c870 con 0x7fdb9c0afd70 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.102+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90037040 con 0x7fdb9c0afd70 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.102+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c0aaf60 con 0x7fdb9c0afd70 2024-09-10T10:53:23.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.102+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0ab520 con 0x7fdb9c0afd70 2024-09-10T10:53:23.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.103+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c160060 con 0x7fdb9c0afd70 2024-09-10T10:53:23.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.104+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdb90044070 con 0x7fdb9c0afd70 2024-09-10T10:53:23.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.105+0000 7fdb98c00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 0x7fdb7c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:23.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.105+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7fdb900ac520 con 0x7fdb9c0afd70 2024-09-10T10:53:23.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.106+0000 7fdb9be00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 0x7fdb7c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:23.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.109+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb9007a310 con 0x7fdb9c0afd70 2024-09-10T10:53:23.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.110+0000 7fdb9be00640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 0x7fdb7c06e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdb88007fd0 tx=0x7fdb88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:23.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.371+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fdb9c067fc0 con 0x7fdb9c0afd70 2024-09-10T10:53:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.924+0000 7fdb98c00640 1 -- 172.21.15.28:0/2299641511 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v238) ==== 101+0+0 (secure 0 0 0) 0x7fdb90079c10 con 0x7fdb9c0afd70 2024-09-10T10:53:23.926 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:23.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 msgr2=0x7fdb7c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 0x7fdb7c06e2f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdb88007fd0 tx=0x7fdb88009040 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 msgr2=0x7fdb9c14ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009870 tx=0x7fdb900046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 shutdown_connections 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdb7c06be30 0x7fdb7c06e2f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdb9c0ce660 0x7fdb9c14f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb9c0b5f80 0x7fdb9c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 --2- 172.21.15.28:0/2299641511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb9c0afd70 0x7fdb9c14ead0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 >> 172.21.15.28:0/2299641511 conn(0x7fdb9c06bc10 msgr2=0x7fdb9c15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 shutdown_connections 2024-09-10T10:53:23.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:23.926+0000 7fdba3600640 1 -- 172.21.15.28:0/2299641511 wait complete. 2024-09-10T10:53:23.945 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.020+0000 7ff6f7a48f40 1 Processor -- start 2024-09-10T10:53:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.020+0000 7ff6f7a48f40 1 -- start start 2024-09-10T10:53:24.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf887ff0 0x55dcaf88a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf9133e0 0x55dcaf914650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dcaf888530 con 0x55dcaf9169c0 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcaf7e5b30 con 0x55dcaf887ff0 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dcaf915ee0 con 0x55dcaf9133e0 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46210/0 (socket says 172.21.15.28:46210) 2024-09-10T10:53:24.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f5600640 1 -- 172.21.15.28:0/1479506599 learned_addr learned my addr 172.21.15.28:0/1479506599 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf887ff0 0x55dcaf88a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.021+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf9133e0 0x55dcaf914650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 -- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf9133e0 msgr2=0x55dcaf914650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf9133e0 0x55dcaf914650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 -- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf887ff0 msgr2=0x55dcaf88a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf887ff0 0x55dcaf88a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 -- 172.21.15.28:0/1479506599 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaf870bd0 con 0x55dcaf9169c0 2024-09-10T10:53:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0025b60 tx=0x7ff6f0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c44d152030be71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6eea00640 1 -- 172.21.15.28:0/1479506599 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f004eba0 con 0x55dcaf9169c0 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6eea00640 1 -- 172.21.15.28:0/1479506599 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f0066070 con 0x55dcaf9169c0 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.022+0000 7ff6eea00640 1 -- 172.21.15.28:0/1479506599 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0061420 con 0x55dcaf9169c0 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1479506599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 msgr2=0x55dcaf914b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0025b60 tx=0x7ff6f0050000 comp rx=0 tx=0).stop 2024-09-10T10:53:24.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1479506599 shutdown_connections 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914b90 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf9133e0 0x55dcaf914650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1479506599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf887ff0 0x55dcaf88a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1479506599 >> 172.21.15.28:0/1479506599 conn(0x55dcaf88c0a0 msgr2=0x55dcaf88c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1479506599 shutdown_connections 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.023+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1479506599 wait complete. 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.024+0000 7ff6f7a48f40 1 Processor -- start 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.024+0000 7ff6f7a48f40 1 -- start start 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf887ff0 0x55dcaf908c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf9133e0 0x55dcaf83e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dcaf88abf0 con 0x55dcaf9169c0 2024-09-10T10:53:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dcaf83d070 con 0x55dcaf9133e0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f7a48f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dcaf9139e0 con 0x55dcaf887ff0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46224/0 (socket says 172.21.15.28:46224) 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 learned_addr learned my addr 172.21.15.28:0/1870500360 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf887ff0 0x55dcaf908c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf887ff0 msgr2=0x55dcaf908c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dcaf887ff0 0x55dcaf908c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf9133e0 msgr2=0x55dcaf83e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dcaf9133e0 0x55dcaf83e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.025+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaf912520 con 0x55dcaf9169c0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000a190 tx=0x7ff6f0003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6ed600640 1 -- 172.21.15.28:0/1870500360 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f006a420 con 0x55dcaf9169c0 2024-09-10T10:53:24.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6ed600640 1 -- 172.21.15.28:0/1870500360 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f005c040 con 0x55dcaf9169c0 2024-09-10T10:53:24.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6ed600640 1 -- 172.21.15.28:0/1870500360 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f006a800 con 0x55dcaf9169c0 2024-09-10T10:53:24.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dcaf919120 con 0x55dcaf9169c0 2024-09-10T10:53:24.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.026+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcaf86fec0 con 0x55dcaf9169c0 2024-09-10T10:53:24.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.028+0000 7ff6ed600640 1 -- 172.21.15.28:0/1870500360 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff6f006a9a0 con 0x55dcaf9169c0 2024-09-10T10:53:24.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.029+0000 7ff6ed600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 0x7ff6d406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.029+0000 7ff6ed600640 1 -- 172.21.15.28:0/1870500360 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7ff6f00cf770 con 0x55dcaf9169c0 2024-09-10T10:53:24.032 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.029+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dcaf919920 0x55dcaf9b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.029+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b7cd0 con 0x55dcaf919920 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.029+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 0x7ff6d406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.030+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dcaf919920 0x55dcaf9b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.030+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 0x7ff6d406ccd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c730 tx=0x7ff6e800e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.030+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dcaf919920 0x55dcaf9b7790 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.033+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v126'784 uv758 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.033+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8070 con 0x55dcaf919920 2024-09-10T10:53:24.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.035+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v126'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.035+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8410 con 0x55dcaf919920 2024-09-10T10:53:24.038 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.036+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v126'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.036+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b97c0 con 0x55dcaf919920 2024-09-10T10:53:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.038+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v126'787 uv787 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.038+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b9b60 con 0x55dcaf919920 2024-09-10T10:53:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.041+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v126'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.041+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b87b0 con 0x55dcaf919920 2024-09-10T10:53:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.043+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v126'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.043+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8b50 con 0x55dcaf919920 2024-09-10T10:53:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.045+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v126'790 uv764 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.045+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.046+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v126'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.046+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.050 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:24.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.049+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v126'792 uv792 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.049+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.051+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v126'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.051+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.052+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v126'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.052+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.053+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v126'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.054+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.055+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v126'796 uv771 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.055+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.058+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v126'797 uv797 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.058+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.060+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v126'798 uv798 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.062 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:24.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.060+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.061+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v126'799 uv799 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.061+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.062+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v126'800 uv800 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.062+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9b8ef0 con 0x55dcaf919920 2024-09-10T10:53:24.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.064+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v126'801 uv801 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.064+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.067+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v126'802 uv802 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.067+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.067+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v126'803 uv777 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.067+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.069+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v126'804 uv804 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.069+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.072+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v126'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.073 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:24.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.072+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.074+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v126'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.074+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.076+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v126'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.076+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.078+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v126'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.078+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.080+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v126'809 uv783 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.080+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.081+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v126'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.081+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.084 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:24.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.083+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v126'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.083+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.085+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v126'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.085+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.087+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v126'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.087+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf919920 2024-09-10T10:53:24.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.089+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v126'814 uv814 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf919920 2024-09-10T10:53:24.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.089+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dcaf9ba410 0x55dcaf9bc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.089+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf914a40 con 0x55dcaf9ba410 2024-09-10T10:53:24.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.089+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dcaf9ba410 0x55dcaf9bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.089+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dcaf9ba410 0x55dcaf9bc800 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.091+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v126'571 uv553 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.091+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bcd40 con 0x55dcaf9ba410 2024-09-10T10:53:24.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.093+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v126'572 uv572 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.093+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bd0e0 con 0x55dcaf9ba410 2024-09-10T10:53:24.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.094+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v126'573 uv573 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.096 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:24.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.094+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bd480 con 0x55dcaf9ba410 2024-09-10T10:53:24.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.096+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v126'574 uv574 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.096+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bd820 con 0x55dcaf9ba410 2024-09-10T10:53:24.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.097+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v126'575 uv575 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.097+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bdbc0 con 0x55dcaf9ba410 2024-09-10T10:53:24.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.099+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v126'576 uv558 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.099+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bdf60 con 0x55dcaf9ba410 2024-09-10T10:53:24.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.100+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v126'577 uv577 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.100+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.102+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v126'578 uv578 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.104 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:24.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.102+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.104+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v126'579 uv579 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.104+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.106+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v126'580 uv580 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.106+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.107+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v126'581 uv570 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.107+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.109+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v126'582 uv582 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.109+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.111+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v126'583 uv583 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.113 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:24.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.111+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c29f0 con 0x55dcaf9ba410 2024-09-10T10:53:24.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.113+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v126'584 uv584 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.113+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c29f0 con 0x55dcaf9ba410 2024-09-10T10:53:24.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.114+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v126'585 uv585 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.114+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.116+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v126'586 uv569 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.116+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.117+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v126'587 uv587 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.117+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.119+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v126'588 uv588 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.121 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:24.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.119+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bf310 con 0x55dcaf9ba410 2024-09-10T10:53:24.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.120+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v126'589 uv589 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.120+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9bf310 con 0x55dcaf9ba410 2024-09-10T10:53:24.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.122+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v126'590 uv590 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.122+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9be300 con 0x55dcaf9ba410 2024-09-10T10:53:24.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.123+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v126'591 uv591 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9ba410 2024-09-10T10:53:24.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.123+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55dcaf9be810 0x55dcaf9c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.123+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c1190 con 0x55dcaf9be810 2024-09-10T10:53:24.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.124+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55dcaf9be810 0x55dcaf9c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.124+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55dcaf9be810 0x55dcaf9c0c00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.126+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v126'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.126+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c1560 con 0x55dcaf9be810 2024-09-10T10:53:24.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.127+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.127+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c1560 con 0x55dcaf9be810 2024-09-10T10:53:24.130 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:24.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.128+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.128+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c2570 con 0x55dcaf9be810 2024-09-10T10:53:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.130+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v126'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.130+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c2570 con 0x55dcaf9be810 2024-09-10T10:53:24.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.131+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v126'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.131+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c1560 con 0x55dcaf9be810 2024-09-10T10:53:24.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.132+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v126'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.132+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55dcaf9c1e30 0x55dcaf9c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.132+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c4760 con 0x55dcaf9c1e30 2024-09-10T10:53:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.133+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55dcaf9c1e30 0x55dcaf9c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.133+0000 7ff6f5600640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55dcaf9c1e30 0x55dcaf9c4220 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.134+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v126'111 uv110 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.134+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c4bc0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.136+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v126'112 uv112 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.136+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c50b0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.138 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:24.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.137+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v126'113 uv113 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.137+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5970 con 0x55dcaf9c1e30 2024-09-10T10:53:24.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.138+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v126'114 uv114 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.138+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5d10 con 0x55dcaf9be810 2024-09-10T10:53:24.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.139+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v126'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.139+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5d10 con 0x55dcaf9be810 2024-09-10T10:53:24.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.140+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.140+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5d10 con 0x55dcaf9be810 2024-09-10T10:53:24.143 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:24.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.142+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.142+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c6d20 con 0x55dcaf9be810 2024-09-10T10:53:24.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.143+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v126'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.143+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c6d20 con 0x55dcaf9be810 2024-09-10T10:53:24.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.144+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v126'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.144+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5d10 con 0x55dcaf9be810 2024-09-10T10:53:24.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.145+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v126'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.145+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c5d10 con 0x55dcaf9c1e30 2024-09-10T10:53:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.147+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v126'282 uv276 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.147+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c60b0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.148+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v126'283 uv283 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.149+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c6450 con 0x55dcaf9c1e30 2024-09-10T10:53:24.152 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.150+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v126'284 uv284 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.150+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c7800 con 0x55dcaf9c1e30 2024-09-10T10:53:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.151+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v126'285 uv285 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.151+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c7800 con 0x55dcaf9c1e30 2024-09-10T10:53:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.153+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v126'286 uv286 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.153+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.154+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v126'287 uv287 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.154+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.155+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v126'288 uv281 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.155+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.157+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v126'289 uv289 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.157+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.160 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:24.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.158+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v126'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.158+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.160+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v126'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.160+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9c1e30 2024-09-10T10:53:24.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.161+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v126'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.161+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9be810 2024-09-10T10:53:24.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.162+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v126'127 uv126 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.163+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9be810 2024-09-10T10:53:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.164+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v126'128 uv128 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.164+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9be810 2024-09-10T10:53:24.167 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.165+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v126'129 uv129 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.165+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9be810 2024-09-10T10:53:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.166+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v126'130 uv130 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.166+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c67f0 con 0x55dcaf9be810 2024-09-10T10:53:24.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.168+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v126'131 uv131 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9be810 2024-09-10T10:53:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.168+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55dcaf9c6d00 0x55dcaf9c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.168+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c97c0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.168+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55dcaf9c6d00 0x55dcaf9c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.169+0000 7ff6f4c00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55dcaf9c6d00 0x55dcaf9c9190 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.170+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v126'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.170+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9880 con 0x55dcaf9c6d00 2024-09-10T10:53:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.171+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.171+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.174 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.173+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.173+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cacb0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.174+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v126'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.174+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cacb0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.175+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v126'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.175+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.177+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v126'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.177+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.178+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v126'253 uv247 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.178+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.179+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v126'254 uv254 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.180+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.182 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.181+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v126'255 uv255 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.181+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.182+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v126'256 uv256 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.182+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.183+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v126'257 uv257 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.184+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.185+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v126'258 uv252 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.185+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.186+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v126'259 uv259 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.186+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.192 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.187+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v126'260 uv260 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.187+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9c9ca0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.189+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v126'261 uv261 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.189+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ccab0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.190+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v126'262 uv262 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.190+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.193+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v126'111 uv110 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.193+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.194+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v126'112 uv112 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.194+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.197 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.196+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v126'113 uv113 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.196+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.197+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v126'114 uv114 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.197+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.198+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v126'153 uv152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.198+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.199+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v126'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.199+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.202 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.201+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v126'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.201+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb160 con 0x55dcaf9c1e30 2024-09-10T10:53:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.202+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v126'156 uv156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.202+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb160 con 0x55dcaf9c1e30 2024-09-10T10:53:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.203+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v126'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.203+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ca150 con 0x55dcaf9c1e30 2024-09-10T10:53:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.204+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v126'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6f0099560 con 0x55dcaf9c1e30 2024-09-10T10:53:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.204+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55dcaf9ca7d0 0x55dcaf9cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.204+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.205+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55dcaf9ca7d0 0x55dcaf9cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.205+0000 7ff6efe00640 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55dcaf9ca7d0 0x55dcaf9cabb0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.207+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v126'130 uv129 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.207+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.208+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v126'131 uv131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.208+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.211 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.209+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v126'132 uv132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.209+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.210+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v126'133 uv133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.210+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.211+0000 7ff6efe00640 1 -- 172.21.15.28:0/1870500360 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v126'134 uv134 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff6e000c5e0 con 0x55dcaf9ca7d0 2024-09-10T10:53:24.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.211+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb0f0 con 0x55dcaf9c6d00 2024-09-10T10:53:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.213+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v126'314 uv306 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.213+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb110 con 0x55dcaf9c6d00 2024-09-10T10:53:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.214+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v126'315 uv315 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.214+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb600 con 0x55dcaf9c6d00 2024-09-10T10:53:24.217 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.215+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v126'316 uv316 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.215+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.216+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v126'317 uv317 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.216+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.218+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v126'318 uv318 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.218+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.219+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v126'319 uv313 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.219+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.220+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v126'320 uv320 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.220+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.223 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:24.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.221+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v126'321 uv321 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.221+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb130 con 0x55dcaf9c6d00 2024-09-10T10:53:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.222+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v126'322 uv322 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.222+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb110 con 0x55dcaf9c6d00 2024-09-10T10:53:24.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.223+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v126'323 uv323 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.223+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9cb110 con 0x55dcaf9c6d00 2024-09-10T10:53:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.224+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v126'324 uv324 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.224+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x55dcaf9ccb70 con 0x55dcaf9c6d00 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f4c00640 1 -- 172.21.15.28:0/1870500360 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v126'325 uv325 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff6e8008810 con 0x55dcaf9c6d00 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55dcaf9be810 msgr2=0x55dcaf9c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55dcaf9be810 0x55dcaf9c0c00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55dcaf9c6d00 msgr2=0x55dcaf9c9190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55dcaf9c6d00 0x55dcaf9c9190 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55dcaf9ca7d0 msgr2=0x55dcaf9cabb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55dcaf9ca7d0 0x55dcaf9cabb0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55dcaf9c1e30 msgr2=0x55dcaf9c4220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55dcaf9c1e30 0x55dcaf9c4220 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dcaf9ba410 msgr2=0x55dcaf9bc800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55dcaf9ba410 0x55dcaf9bc800 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dcaf919920 msgr2=0x55dcaf9b7790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dcaf919920 0x55dcaf9b7790 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 reap_dead start 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 reap_dead start 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 reap_dead start 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f5600640 1 -- 172.21.15.28:0/1870500360 reap_dead start 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 msgr2=0x7ff6d406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 0x7ff6d406ccd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800c730 tx=0x7ff6e800e040 comp rx=0 tx=0).stop 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 msgr2=0x55dcaf914500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000a190 tx=0x7ff6f0003330 comp rx=0 tx=0).stop 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 shutdown_connections 2024-09-10T10:53:24.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff6d406a810 0x7ff6d406ccd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 --2- 172.21.15.28:0/1870500360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dcaf9169c0 0x55dcaf914500 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 >> 172.21.15.28:0/1870500360 conn(0x55dcaf88c0a0 msgr2=0x55dcaf88c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 shutdown_connections 2024-09-10T10:53:24.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.226+0000 7ff6f7a48f40 1 -- 172.21.15.28:0/1870500360 wait complete. 2024-09-10T10:53:24.234 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 500.00000000 0 2024-09-10T10:53:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 Processor -- start 2024-09-10T10:53:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 -- start start 2024-09-10T10:53:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b047d9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0473c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555b04754530 con 0x555b047df3e0 2024-09-10T10:53:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b046b1b30 con 0x555b047e29c0 2024-09-10T10:53:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.349+0000 7f381675cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555b047e1ee0 con 0x555b04709470 2024-09-10T10:53:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46244/0 (socket says 172.21.15.28:46244) 2024-09-10T10:53:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 -- 172.21.15.28:0/1203266814 learned_addr learned my addr 172.21.15.28:0/1203266814 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 -- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 msgr2=0x555b047d9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b047d9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 -- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 msgr2=0x555b0473c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0473c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 -- 172.21.15.28:0/1203266814 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0470b930 con 0x555b047df3e0 2024-09-10T10:53:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.350+0000 7f3813000640 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3800000c00 tx=0x7f3800051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797259109ea336bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f3811c00640 1 -- 172.21.15.28:0/1203266814 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800055070 con 0x555b047df3e0 2024-09-10T10:53:24.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f3811c00640 1 -- 172.21.15.28:0/1203266814 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38000517d0 con 0x555b047df3e0 2024-09-10T10:53:24.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f3811c00640 1 -- 172.21.15.28:0/1203266814 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800051ba0 con 0x555b047df3e0 2024-09-10T10:53:24.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 -- 172.21.15.28:0/1203266814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 msgr2=0x555b047d5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3800000c00 tx=0x7f3800051000 comp rx=0 tx=0).stop 2024-09-10T10:53:24.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 -- 172.21.15.28:0/1203266814 shutdown_connections 2024-09-10T10:53:24.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0473c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b047d5de0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 --2- 172.21.15.28:0/1203266814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b047d9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 -- 172.21.15.28:0/1203266814 >> 172.21.15.28:0/1203266814 conn(0x555b047580a0 msgr2=0x555b04758480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 -- 172.21.15.28:0/1203266814 shutdown_connections 2024-09-10T10:53:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.351+0000 7f381675cf40 1 -- 172.21.15.28:0/1203266814 wait complete. 2024-09-10T10:53:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.352+0000 7f381675cf40 1 Processor -- start 2024-09-10T10:53:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.352+0000 7f381675cf40 1 -- start start 2024-09-10T10:53:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b04755bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b0487b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0487bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x555b0470bc10 con 0x555b047df3e0 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b0473aea0 con 0x555b047e29c0 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f381675cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x555b047e1900 con 0x555b04709470 2024-09-10T10:53:24.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b0487b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b0487b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46250/0 (socket says 172.21.15.28:46250) 2024-09-10T10:53:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 -- 172.21.15.28:0/885289724 learned_addr learned my addr 172.21.15.28:0/885289724 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 -- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 msgr2=0x555b04755bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813a00640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b04755bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b04755bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 -- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 msgr2=0x555b0487bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3814400640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0487bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0487bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813000640 1 -- 172.21.15.28:0/885289724 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b047d2590 con 0x555b047df3e0 2024-09-10T10:53:24.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3813a00640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x555b04709470 0x555b04755bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.353+0000 7f3814400640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b047e29c0 0x555b0487bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f3813000640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b0487b6d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3800046070 tx=0x7f3800002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f380be00640 1 -- 172.21.15.28:0/885289724 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800055070 con 0x555b047df3e0 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f380be00640 1 -- 172.21.15.28:0/885289724 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800051d10 con 0x555b047df3e0 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f380be00640 1 -- 172.21.15.28:0/885289724 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380006a070 con 0x555b047df3e0 2024-09-10T10:53:24.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555b047e4110 con 0x555b047df3e0 2024-09-10T10:53:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.354+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b047d2890 con 0x555b047df3e0 2024-09-10T10:53:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.356+0000 7f380be00640 1 -- 172.21.15.28:0/885289724 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3800075020 con 0x555b047df3e0 2024-09-10T10:53:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f380be00640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37f406a810 0x7f37f406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f380be00640 1 -- 172.21.15.28:0/885289724 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f38000cf4b0 con 0x555b047df3e0 2024-09-10T10:53:24.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f3813a00640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37f406a810 0x7f37f406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f381675cf40 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555b047560f0 0x555b047e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.15 13:aa448500:::500.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) -- 0x555b047d2de0 con 0x555b047560f0 2024-09-10T10:53:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.357+0000 7f3814400640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555b047560f0 0x555b047e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.358+0000 7f3813a00640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37f406a810 0x7f37f406ccd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3804001e80 tx=0x7f3804001480 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.358+0000 7f3814400640 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555b047560f0 0x555b047e4f90 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f3814400640 1 -- 172.21.15.28:0/885289724 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 500.00000000 [truncate 0] v126'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f380c00c690 con 0x555b047560f0 2024-09-10T10:53:24.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555b047560f0 msgr2=0x555b047e4f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x555b047560f0 0x555b047e4f90 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37f406a810 msgr2=0x7f37f406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f37f406a810 0x7f37f406ccd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3804001e80 tx=0x7f3804001480 comp rx=0 tx=0).stop 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 msgr2=0x555b0487b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 --2- 172.21.15.28:0/885289724 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555b047df3e0 0x555b0487b6d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f3800046070 tx=0x7f3800002fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f3814400640 1 -- 172.21.15.28:0/885289724 reap_dead start 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 shutdown_connections 2024-09-10T10:53:24.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 >> 172.21.15.28:0/885289724 conn(0x555b047580a0 msgr2=0x555b04758480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 shutdown_connections 2024-09-10T10:53:24.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.360+0000 7f381675cf40 1 -- 172.21.15.28:0/885289724 wait complete. 2024-09-10T10:53:24.378 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:24.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.536+0000 7fbd97e00640 1 Processor -- start 2024-09-10T10:53:24.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.536+0000 7fbd97e00640 1 -- start start 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd98161ac0 0x7fbd9806e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd980b1170 con 0x7fbd980d2ac0 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd980b1450 con 0x7fbd980b3640 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd980ceba0 con 0x7fbd98161ac0 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbd980d2ac0 0x7fbd980b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45620/0 (socket says 172.21.15.28:45620) 2024-09-10T10:53:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.537+0000 7fbd97400640 1 -- 172.21.15.28:0/1319265652 learned_addr learned my addr 172.21.15.28:0/1319265652 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96a00640 1 --2- 172.21.15.28:0/1319265652 >> v2:172.21.15.138:3300/0 conn(0x7fbd98161ac0 0x7fbd9806e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2470924769 0 0) 0x7fbd980b1170 con 0x7fbd980d2ac0 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd7c003520 con 0x7fbd980d2ac0 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 149736393 0 0) 0x7fbd7c003520 con 0x7fbd980d2ac0 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 >> v2:172.21.15.138:3300/0 conn(0x7fbd98161ac0 msgr2=0x7fbd9806e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 --2- 172.21.15.28:0/1319265652 >> v2:172.21.15.138:3300/0 conn(0x7fbd98161ac0 0x7fbd9806e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 >> v1:172.21.15.138:6789/0 conn(0x7fbd980b3640 legacy=0x7fbd980d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd980b1730 con 0x7fbd980d2ac0 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fbd90001a10 con 0x7fbd980d2ac0 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 >> v1:172.21.15.28:6789/0 conn(0x7fbd980d2ac0 legacy=0x7fbd980b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7c0043b0 0x7fbd7c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c00bf00 con 0x7fbd7c005080 2024-09-10T10:53:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7c00c0b0 con 0x7fbd7c0039b0 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd7c00c2b0 con 0x7fbd7c0043b0 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd97400640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd96a00640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.538+0000 7fbd8fe00640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7c0043b0 0x7fbd7c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd97400640 1 -- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7c0043b0 msgr2=0x7fbd7c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd97400640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd7c0043b0 0x7fbd7c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd97400640 1 -- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7c0039b0 msgr2=0x7fbd7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd97400640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd7c0039b0 0x7fbd7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.539+0000 7fbd97400640 1 -- 172.21.15.28:0/1319265652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7c003520 con 0x7fbd7c005080 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.540+0000 7fbd97400640 1 -- 172.21.15.28:0/1319265652 reap_dead start 2024-09-10T10:53:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.540+0000 7fbd97400640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd7c00b9c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000a7f0 tx=0x7fbd7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6391d94c6a948b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.540+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000a450 con 0x7fbd7c005080 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.540+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9002f070 con 0x7fbd7c005080 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.540+0000 7fbd96000640 1 -- 172.21.15.28:0/1319265652 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9002bc00 con 0x7fbd7c005080 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 -- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 msgr2=0x7fbd7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd7c00b9c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000a7f0 tx=0x7fbd7c005830 comp rx=0 tx=0).stop 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 -- 172.21.15.28:0/1319265652 shutdown_connections 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1319265652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd7c00b9c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 -- 172.21.15.28:0/1319265652 >> 172.21.15.28:0/1319265652 conn(0x7fbd9808bfc0 msgr2=0x7fbd980d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 -- 172.21.15.28:0/1319265652 shutdown_connections 2024-09-10T10:53:24.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.541+0000 7fbd97e00640 1 -- 172.21.15.28:0/1319265652 wait complete. 2024-09-10T10:53:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 Processor -- start 2024-09-10T10:53:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 -- start start 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd980cd5b0 0x7fbd980cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd980c7e70 0x7fbd980c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd98161ea0 con 0x7fbd7c005080 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9806dfb0 con 0x7fbd980c7e70 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.543+0000 7fbd97e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd98162020 con 0x7fbd980cd5b0 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46260/0 (socket says 172.21.15.28:46260) 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 -- 172.21.15.28:0/1612896213 learned_addr learned my addr 172.21.15.28:0/1612896213 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 -- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd980cd5b0 msgr2=0x7fbd980cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd980cd5b0 0x7fbd980cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 -- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd980c7e70 msgr2=0x7fbd980c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd980c7e70 0x7fbd980c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 -- 172.21.15.28:0/1612896213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd980c9f50 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.544+0000 7fbd96a00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011910 tx=0x7fbd88011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.545+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800faf0 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.545+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8800fc90 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.545+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd88014440 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.545+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd980ca1c0 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.545+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd980ca780 con 0x7fbd7c005080 2024-09-10T10:53:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.546+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9806e6c0 con 0x7fbd7c005080 2024-09-10T10:53:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.548+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd88012040 con 0x7fbd7c005080 2024-09-10T10:53:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.548+0000 7fbd8ea00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 0x7fbd7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.548+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7fbd8808bff0 con 0x7fbd7c005080 2024-09-10T10:53:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.548+0000 7fbd8fe00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 0x7fbd7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.549+0000 7fbd8fe00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 0x7fbd7406e320 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbd980d1200 tx=0x7fbd80009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.552+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd88019030 con 0x7fbd7c005080 2024-09-10T10:53:24.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.772+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbd980b2b40 con 0x7fbd7c005080 2024-09-10T10:53:24.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.931+0000 7fbd8ea00640 1 -- 172.21.15.28:0/1612896213 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v239) ==== 162+0+0 (secure 0 0 0) 0x7fbd88059de0 con 0x7fbd7c005080 2024-09-10T10:53:24.932 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:24.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 msgr2=0x7fbd7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 0x7fbd7406e320 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fbd980d1200 tx=0x7fbd80009040 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 msgr2=0x7fbd980caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011910 tx=0x7fbd88011dd0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 shutdown_connections 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd7406be60 0x7fbd7406e320 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd980c7e70 0x7fbd980c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd980cd5b0 0x7fbd980cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 --2- 172.21.15.28:0/1612896213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd7c005080 0x7fbd980caca0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 >> 172.21.15.28:0/1612896213 conn(0x7fbd9808bfc0 msgr2=0x7fbd980b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 shutdown_connections 2024-09-10T10:53:24.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:24.933+0000 7fbd97e00640 1 -- 172.21.15.28:0/1612896213 wait complete. 2024-09-10T10:53:25.357 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:25.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.514+0000 7fd639200640 1 Processor -- start 2024-09-10T10:53:25.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.514+0000 7fd639200640 1 -- start start 2024-09-10T10:53:25.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.515+0000 7fd639200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6340727c0 0x7fd6341690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.515+0000 7fd639200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63406f480 con 0x7fd634147230 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.515+0000 7fd639200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63406e900 con 0x7fd634147e90 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.515+0000 7fd639200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6340b34d0 con 0x7fd6340727c0 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.515+0000 7fd631e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd634147230 0x7fd63406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45634/0 (socket says 172.21.15.28:45634) 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631e00640 1 -- 172.21.15.28:0/1396798770 learned_addr learned my addr 172.21.15.28:0/1396798770 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd632800640 1 --2- 172.21.15.28:0/1396798770 >> v2:172.21.15.138:3300/0 conn(0x7fd6340727c0 0x7fd6341690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1507052247 0 0) 0x7fd63406f480 con 0x7fd634147230 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd618003520 con 0x7fd634147230 2024-09-10T10:53:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842935207 0 0) 0x7fd63406e900 con 0x7fd634147e90 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd63406f480 con 0x7fd634147e90 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1124871586 0 0) 0x7fd618003520 con 0x7fd634147230 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 >> v2:172.21.15.138:3300/0 conn(0x7fd6340727c0 msgr2=0x7fd6341690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 --2- 172.21.15.28:0/1396798770 >> v2:172.21.15.138:3300/0 conn(0x7fd6340727c0 0x7fd6341690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 >> v1:172.21.15.138:6789/0 conn(0x7fd634147e90 legacy=0x7fd6340720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63406ebe0 con 0x7fd634147230 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fd624001670 con 0x7fd634147230 2024-09-10T10:53:25.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 >> v1:172.21.15.28:6789/0 conn(0x7fd634147230 legacy=0x7fd63406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.516+0000 7fd631400640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 0x7fd618003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631400640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618004800 0x7fd618004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631400640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6180051c0 0x7fd61800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd61800c010 con 0x7fd618003b40 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd61800c1c0 con 0x7fd6180051c0 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd61800c3c0 con 0x7fd618004800 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 0x7fd618003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd632800640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6180051c0 0x7fd61800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd62be00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618004800 0x7fd618004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 -- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618004800 msgr2=0x7fd618004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618004800 0x7fd618004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 -- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6180051c0 msgr2=0x7fd61800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6180051c0 0x7fd61800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd631e00640 1 -- 172.21.15.28:0/1396798770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd618003520 con 0x7fd618003b40 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd632800640 1 -- 172.21.15.28:0/1396798770 reap_dead start 2024-09-10T10:53:25.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.517+0000 7fd62be00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618004800 0x7fd618004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.518+0000 7fd631e00640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 0x7fd618003f40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c210 tx=0x7fd618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7ac6c3ed352931 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.518+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624007ce0 con 0x7fd618003b40 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.518+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd624005ad0 con 0x7fd618003b40 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.518+0000 7fd631400640 1 -- 172.21.15.28:0/1396798770 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624007070 con 0x7fd618003b40 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 -- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 msgr2=0x7fd618003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 0x7fd618003f40 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fd62400c210 tx=0x7fd618005830 comp rx=0 tx=0).stop 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 -- 172.21.15.28:0/1396798770 shutdown_connections 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 --2- 172.21.15.28:0/1396798770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd618003b40 0x7fd618003f40 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 -- 172.21.15.28:0/1396798770 >> 172.21.15.28:0/1396798770 conn(0x7fd634067d70 msgr2=0x7fd63406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 -- 172.21.15.28:0/1396798770 shutdown_connections 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.519+0000 7fd639200640 1 -- 172.21.15.28:0/1396798770 wait complete. 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.521+0000 7fd639200640 1 Processor -- start 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.521+0000 7fd639200640 1 -- start start 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618003b40 0x7fd6340cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 0x7fd634145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd63406f1a0 con 0x7fd6341f4e30 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd634147610 con 0x7fd63415ffc0 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd639200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd634147790 con 0x7fd618003b40 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd62be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd62be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46278/0 (socket says 172.21.15.28:46278) 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd62be00640 1 -- 172.21.15.28:0/817461451 learned_addr learned my addr 172.21.15.28:0/817461451 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd631e00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618003b40 0x7fd6340cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.522+0000 7fd632800640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 0x7fd634145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 -- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618003b40 msgr2=0x7fd6340cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618003b40 0x7fd6340cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 -- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 msgr2=0x7fd634145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 0x7fd634145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 -- 172.21.15.28:0/817461451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6341604d0 con 0x7fd6341f4e30 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd632800640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 0x7fd634145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62be00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00e4d0 tx=0x7fd61c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c0052e0 con 0x7fd6341f4e30 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61c005480 con 0x7fd6341f4e30 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c007020 con 0x7fd6341f4e30 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6341439a0 con 0x7fd6341f4e30 2024-09-10T10:53:25.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.523+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd634143f60 con 0x7fd6341f4e30 2024-09-10T10:53:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.525+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd634072090 con 0x7fd6341f4e30 2024-09-10T10:53:25.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.525+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd61c01a020 con 0x7fd6341f4e30 2024-09-10T10:53:25.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.526+0000 7fd62a000640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 0x7fd61006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:25.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.526+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7fd61c08afa0 con 0x7fd6341f4e30 2024-09-10T10:53:25.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.527+0000 7fd631e00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 0x7fd61006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:25.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.528+0000 7fd631e00640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 0x7fd61006e320 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd624001eb0 tx=0x7fd624009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:25.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.531+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd61c058d90 con 0x7fd6341f4e30 2024-09-10T10:53:25.661 INFO:tasks.ceph.mds.c.smithi028.stderr: -38> 2024-09-10T10:53:20.907+0000 7f2f67400640 -1 log_channel(cluster) log [ERR] : missing journal object 2024-09-10T10:53:25.700 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.751+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6340b2f20 con 0x7fd6341f4e30 2024-09-10T10:53:25.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.751+0000 7fd62a000640 1 -- 172.21.15.28:0/817461451 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 240 v240) ==== 95+0+4401 (secure 0 0 0) 0x7fd61c058690 con 0x7fd6341f4e30 2024-09-10T10:53:25.753 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:25.753 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":240,"btime":"2024-09-10T10:53:24:933494+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":240,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:24.933479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9174":{"gid":9174,"name":"a","rank":0,"incarnation":240,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2097945693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2097945693},{"type":"v1","addr":"172.21.15.28:6835","nonce":2097945693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:25.753 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 240 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 msgr2=0x7fd61006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 0x7fd61006e320 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd624001eb0 tx=0x7fd624009030 comp rx=0 tx=0).stop 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 msgr2=0x7fd63415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00e4d0 tx=0x7fd61c00e990 comp rx=0 tx=0).stop 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 shutdown_connections 2024-09-10T10:53:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd61006be60 0x7fd61006e320 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd63415ffc0 0x7fd634145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6341f4e30 0x7fd63415fa80 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 --2- 172.21.15.28:0/817461451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd618003b40 0x7fd6340cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 >> 172.21.15.28:0/817461451 conn(0x7fd634067d70 msgr2=0x7fd6340b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 shutdown_connections 2024-09-10T10:53:25.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:25.754+0000 7fd639200640 1 -- 172.21.15.28:0/817461451 wait complete. 2024-09-10T10:53:25.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 240, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:53:24.933479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9174}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9174': {'gid': 9174, 'name': 'a', 'rank': 0, 'incarnation': 240, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2097945693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2097945693}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2097945693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:53:25.772 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9174:up:replay 2024-09-10T10:53:25.773 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:53:31.177 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:31.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.344+0000 7ff924a00640 1 Processor -- start 2024-09-10T10:53:31.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.344+0000 7ff924a00640 1 -- start start 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff924a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 0x7ff9200d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff924a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92015e6a0 0x7ff92006d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff924a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9200d4aa0 con 0x7ff92006dd60 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff924a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff92006e170 con 0x7ff92015e6a0 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff924a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff9200cefb0 con 0x7ff9200d4180 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff917e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92015e6a0 0x7ff92006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff917e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92015e6a0 0x7ff92006d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46302/0 (socket says 172.21.15.28:46302) 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.345+0000 7ff917e00640 1 -- 172.21.15.28:0/3334191744 learned_addr learned my addr 172.21.15.28:0/3334191744 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff91f400640 1 --2- 172.21.15.28:0/3334191744 >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 0x7ff9200d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff917e00640 1 -- 172.21.15.28:0/3334191744 >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 msgr2=0x7ff9200d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff917e00640 1 --2- 172.21.15.28:0/3334191744 >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 0x7ff9200d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff917e00640 1 -- 172.21.15.28:0/3334191744 >> v1:172.21.15.138:6789/0 conn(0x7ff92006dd60 legacy=0x7ff9200d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff917e00640 1 -- 172.21.15.28:0/3334191744 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9200d2c20 con 0x7ff92015e6a0 2024-09-10T10:53:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.346+0000 7ff91f400640 1 --2- 172.21.15.28:0/3334191744 >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 0x7ff9200d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:31.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff917e00640 1 --2- 172.21.15.28:0/3334191744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff92006d820 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff9200b6ea0 tx=0x7ff90802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913f6d1bf12d3ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff91ea00640 1 -- 172.21.15.28:0/3334191744 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908037070 con 0x7ff92015e6a0 2024-09-10T10:53:31.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff91ea00640 1 -- 172.21.15.28:0/3334191744 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff908032440 con 0x7ff92015e6a0 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff91ea00640 1 -- 172.21.15.28:0/3334191744 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908032810 con 0x7ff92015e6a0 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff924a00640 1 -- 172.21.15.28:0/3334191744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 msgr2=0x7ff92006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.347+0000 7ff924a00640 1 --2- 172.21.15.28:0/3334191744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff92006d820 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7ff9200b6ea0 tx=0x7ff90802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 -- 172.21.15.28:0/3334191744 shutdown_connections 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 --2- 172.21.15.28:0/3334191744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff92006d820 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 --2- 172.21.15.28:0/3334191744 >> v2:172.21.15.138:3300/0 conn(0x7ff9200d4180 0x7ff9200d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 -- 172.21.15.28:0/3334191744 >> 172.21.15.28:0/3334191744 conn(0x7ff92009d6f0 msgr2=0x7ff92008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 -- 172.21.15.28:0/3334191744 shutdown_connections 2024-09-10T10:53:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.348+0000 7ff924a00640 1 -- 172.21.15.28:0/3334191744 wait complete. 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.349+0000 7ff924a00640 1 Processor -- start 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.349+0000 7ff924a00640 1 -- start start 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 0x7ff920157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 0x7ff920159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff92006e950 con 0x7ff92015e6a0 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff92006e670 con 0x7ff92006dd60 2024-09-10T10:53:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.350+0000 7ff924a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff9200cebd0 con 0x7ff9200d4180 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46314/0 (socket says 172.21.15.28:46314) 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 -- 172.21.15.28:0/969605645 learned_addr learned my addr 172.21.15.28:0/969605645 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91f400640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 0x7ff920157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff917e00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 0x7ff920159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 -- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 msgr2=0x7ff920159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 0x7ff920159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 -- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 msgr2=0x7ff920157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 0x7ff920157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 -- 172.21.15.28:0/969605645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9200cca80 con 0x7ff92015e6a0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff917e00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 0x7ff920159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91f400640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 0x7ff920157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff91fe00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b9a0 tx=0x7ff91800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.351+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918019070 con 0x7ff92015e6a0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.352+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91800cba0 con 0x7ff92015e6a0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.352+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9200ce430 con 0x7ff92015e6a0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.352+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918014440 con 0x7ff92015e6a0 2024-09-10T10:53:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.352+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9200cbfa0 con 0x7ff92015e6a0 2024-09-10T10:53:31.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.353+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9200d4560 con 0x7ff92015e6a0 2024-09-10T10:53:31.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.354+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff91801c070 con 0x7ff92015e6a0 2024-09-10T10:53:31.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.354+0000 7ff917400640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 0x7ff8fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.355+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7ff91808b5f0 con 0x7ff92015e6a0 2024-09-10T10:53:31.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.355+0000 7ff91f400640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 0x7ff8fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.356+0000 7ff91f400640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 0x7ff8fc06e320 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff91000a990 tx=0x7ff910008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.359+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff9180593e0 con 0x7ff92015e6a0 2024-09-10T10:53:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.578+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff920158190 con 0x7ff92015e6a0 2024-09-10T10:53:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.578+0000 7ff917400640 1 -- 172.21.15.28:0/969605645 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 243 v243) ==== 95+0+5234 (secure 0 0 0) 0x7ff918058ce0 con 0x7ff92015e6a0 2024-09-10T10:53:31.579 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:31.580 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":243,"btime":"2024-09-10T10:53:27:947504+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1706206889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1706206889},{"type":"v1","addr":"172.21.15.28:6834","nonce":1706206889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":243,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:27.947497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9174},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9174":{"gid":9174,"name":"a","rank":0,"incarnation":240,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2097945693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2097945693},{"type":"v1","addr":"172.21.15.28:6835","nonce":2097945693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9174,"qdb_cluster":[9174]},"id":6}]} 2024-09-10T10:53:31.580 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 243 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.580+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 msgr2=0x7ff8fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 0x7ff8fc06e320 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff91000a990 tx=0x7ff910008040 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 msgr2=0x7ff9200b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b9a0 tx=0x7ff91800be60 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 shutdown_connections 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff8fc06be60 0x7ff8fc06e320 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff92015e6a0 0x7ff9200b8d00 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff9200d4180 0x7ff920159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 --2- 172.21.15.28:0/969605645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff92006dd60 0x7ff920157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 >> 172.21.15.28:0/969605645 conn(0x7ff92009d6f0 msgr2=0x7ff9200d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 shutdown_connections 2024-09-10T10:53:31.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.581+0000 7ff924a00640 1 -- 172.21.15.28:0/969605645 wait complete. 2024-09-10T10:53:31.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 243, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:53:27.947497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9174}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9174': {'gid': 9174, 'name': 'a', 'rank': 0, 'incarnation': 240, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6835/2097945693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2097945693}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2097945693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9174, 'qdb_cluster': [9174]} 2024-09-10T10:53:31.598 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:53:31.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:53:31.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.756+0000 7f8b8e200640 1 Processor -- start 2024-09-10T10:53:31.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.756+0000 7f8b8e200640 1 -- start start 2024-09-10T10:53:31.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.757+0000 7f8b8e200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b8815b300 0x7f8b880cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.757+0000 7f8b8e200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8b8806f8d0 0x7f8b8815b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.757+0000 7f8b8e200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b880cdbe0 con 0x7f8b880cd3f0 2024-09-10T10:53:31.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b8e200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8b880728a0 con 0x7f8b8815b300 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b8e200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8b880cd900 con 0x7f8b8806f8d0 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b8815b300 0x7f8b880cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b7fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8b880cd3f0 0x7f8b8806f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45642/0 (socket says 172.21.15.28:45642) 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b7fe00640 1 -- 172.21.15.28:0/3477375247 learned_addr learned my addr 172.21.15.28:0/3477375247 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3477375247 >> v2:172.21.15.138:3300/0 conn(0x7f8b8806f8d0 0x7f8b8815b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87400640 1 -- 172.21.15.28:0/3477375247 >> v2:172.21.15.138:3300/0 conn(0x7f8b8806f8d0 msgr2=0x7f8b8815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87400640 1 --2- 172.21.15.28:0/3477375247 >> v2:172.21.15.138:3300/0 conn(0x7f8b8806f8d0 0x7f8b8815b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87400640 1 -- 172.21.15.28:0/3477375247 >> v1:172.21.15.28:6789/0 conn(0x7f8b880cd3f0 legacy=0x7f8b8806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b87400640 1 -- 172.21.15.28:0/3477375247 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b88072250 con 0x7f8b8815b300 2024-09-10T10:53:31.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.758+0000 7f8b86a00640 1 -- 172.21.15.28:0/3477375247 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 959584956 0 0) 0x7f8b880cdbe0 con 0x7f8b880cd3f0 2024-09-10T10:53:31.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.759+0000 7f8b87400640 1 --2- 172.21.15.28:0/3477375247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b880cceb0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8b880b38f0 tx=0x7f8b7802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69f0a684c4f9649 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b86a00640 1 -- 172.21.15.28:0/3477375247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78037070 con 0x7f8b8815b300 2024-09-10T10:53:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b86a00640 1 -- 172.21.15.28:0/3477375247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78032440 con 0x7f8b8815b300 2024-09-10T10:53:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b86a00640 1 -- 172.21.15.28:0/3477375247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78032810 con 0x7f8b8815b300 2024-09-10T10:53:31.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 -- 172.21.15.28:0/3477375247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 msgr2=0x7f8b880cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3477375247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b880cceb0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f8b880b38f0 tx=0x7f8b7802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 -- 172.21.15.28:0/3477375247 shutdown_connections 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3477375247 >> v2:172.21.15.138:3300/0 conn(0x7f8b8806f8d0 0x7f8b8815b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3477375247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b880cceb0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 -- 172.21.15.28:0/3477375247 >> 172.21.15.28:0/3477375247 conn(0x7f8b88067d70 msgr2=0x7f8b8806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 -- 172.21.15.28:0/3477375247 shutdown_connections 2024-09-10T10:53:31.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.760+0000 7f8b8e200640 1 -- 172.21.15.28:0/3477375247 wait complete. 2024-09-10T10:53:31.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.762+0000 7f8b8e200640 1 Processor -- start 2024-09-10T10:53:31.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.762+0000 7f8b8e200640 1 -- start start 2024-09-10T10:53:31.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b8806f8d0 0x7f8b881473c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b880cd3f0 0x7f8b88154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b880d0160 con 0x7f8b8815b300 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b880cffe0 con 0x7f8b8806f8d0 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b8e200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b88072d30 con 0x7f8b880cd3f0 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46328/0 (socket says 172.21.15.28:46328) 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b87e00640 1 -- 172.21.15.28:0/3368088490 learned_addr learned my addr 172.21.15.28:0/3368088490 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b87400640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b8806f8d0 0x7f8b881473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.763+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b880cd3f0 0x7f8b88154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 -- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b880cd3f0 msgr2=0x7f8b88154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b880cd3f0 0x7f8b88154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 -- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b8806f8d0 msgr2=0x7f8b881473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b8806f8d0 0x7f8b881473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 -- 172.21.15.28:0/3368088490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b88148780 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009dd0 tx=0x7f8b8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b7f400640 1 -- 172.21.15.28:0/3368088490 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018020 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b7f400640 1 -- 172.21.15.28:0/3368088490 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80014480 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b7f400640 1 -- 172.21.15.28:0/3368088490 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80014870 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.764+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b88148a60 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.765+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b88147a80 con 0x7f8b8815b300 2024-09-10T10:53:31.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.766+0000 7f8b7f400640 1 -- 172.21.15.28:0/3368088490 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b80005d10 con 0x7f8b8815b300 2024-09-10T10:53:31.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.767+0000 7f8b7f400640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 0x7f8b6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.767+0000 7f8b7f400640 1 -- 172.21.15.28:0/3368088490 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f8b8008c6e0 con 0x7f8b8815b300 2024-09-10T10:53:31.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.768+0000 7f8b87400640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 0x7f8b6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.768+0000 7f8b910c7740 1 Processor -- start 2024-09-10T10:53:31.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.768+0000 7f8b910c7740 1 -- start start 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.768+0000 7f8b87400640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 0x7f8b6406e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b7802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325d100 0x5639a32a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 0x5639a325d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5639a31b73b0 con 0x5639a325e890 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639a31bc410 con 0x5639a325a8c0 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5639a3164620 con 0x5639a325d100 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46330/0 (socket says 172.21.15.28:46330) 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b87e00640 1 -- 172.21.15.28:0/3149824901 learned_addr learned my addr 172.21.15.28:0/3149824901 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b87400640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325d100 0x5639a32a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.769+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 0x5639a325d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 -- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325d100 msgr2=0x5639a32a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325d100 0x5639a32a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 -- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 msgr2=0x5639a325d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 0x5639a325d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 -- 172.21.15.28:0/3149824901 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a2ecf2d0 con 0x5639a325e890 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 0x5639a325d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8b800562e0 tx=0x7f8b80059fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabf4ce57fa90b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b77400640 1 -- 172.21.15.28:0/3149824901 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80088070 con 0x5639a325e890 2024-09-10T10:53:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b77400640 1 -- 172.21.15.28:0/3149824901 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80058180 con 0x5639a325e890 2024-09-10T10:53:31.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.770+0000 7f8b77400640 1 -- 172.21.15.28:0/3149824901 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80062520 con 0x5639a325e890 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 -- 172.21.15.28:0/3149824901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 msgr2=0x5639a3257980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f8b800562e0 tx=0x7f8b80059fc0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 -- 172.21.15.28:0/3149824901 shutdown_connections 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a3257980 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325a8c0 0x5639a325d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3149824901 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325d100 0x5639a32a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 -- 172.21.15.28:0/3149824901 >> 172.21.15.28:0/3149824901 conn(0x5639a3259b50 msgr2=0x5639a32a1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 -- 172.21.15.28:0/3149824901 shutdown_connections 2024-09-10T10:53:31.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.771+0000 7f8b910c7740 1 -- 172.21.15.28:0/3149824901 wait complete. 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.772+0000 7f8b910c7740 1 -- start start 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.772+0000 7f8b910c7740 1 Processor -- start 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a3281640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a32b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5639a3177780 con 0x5639a327b680 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639a31b83d0 con 0x5639a325d100 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5639a3258ff0 con 0x5639a325e890 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b910c7740 10 client.0 _ll_register_callbacks cb 0x5639a31b55d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46334/0 (socket says 172.21.15.28:46334) 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 -- 172.21.15.28:0/3959882120 learned_addr learned my addr 172.21.15.28:0/3959882120 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a32b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87400640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a3281640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 -- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 msgr2=0x5639a32b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a32b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 -- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 msgr2=0x5639a3281640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a3281640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87e00640 1 -- 172.21.15.28:0/3959882120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a2f996b0 con 0x5639a327b680 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.773+0000 7f8b87400640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a3281640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b87e00640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c990 tx=0x7f8b8001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd77c84bb048a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8007a020 con 0x5639a327b680 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f8b80062720 con 0x5639a327b680 2024-09-10T10:53:31.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b80058180 con 0x5639a327b680 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=243) ==== 24+0+0 (secure 0 0 0) 0x7f8b80088040 con 0x5639a327b680 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 10 client.9252 fetch_fsmap learned FSMap version 243 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=243,monmap=2+}) -- 0x5639a3280270 con 0x5639a327b680 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80058180 con 0x5639a327b680 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 243) ==== 3502+0+0 (secure 0 0 0) 0x7f8b8008c0d0 con 0x5639a327b680 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 10 client.9252 fetch_fsmap finished waiting for FSMap version 243 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 10 client.9252 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 20 client.9252 populate_metadata read hostname 'smithi028' 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a327c4a0 0x5639a327c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 4 client.9252 mds_command: new command op to 9174 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.774+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 --> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a31c9b40 con 0x5639a327c4a0 2024-09-10T10:53:31.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.775+0000 7f8b87400640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a327c4a0 0x5639a327c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.775+0000 7f8b87400640 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a327c4a0 0x5639a327c880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ecf3c7b5409ab1 server_cookie=4e43519a9238fd09 in_seq=0 out_seq=0 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.775+0000 7f8b6be00640 10 client.9252 ms_handle_connect on v2:172.21.15.28:6833/2097945693 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.776+0000 7f8b6be00640 1 -- 172.21.15.28:0/3959882120 <== mds.0 v2:172.21.15.28:6833/2097945693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639a31c9b40 con 0x5639a327c4a0 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.776+0000 7f8b6be00640 10 client.9252 handle_command_reply: tid=0 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.776+0000 7f8b910c7740 1 client.9252 shutdown 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.776+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 msgr2=0x5639a327f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.776+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c990 tx=0x7f8b8001f040 comp rx=0 tx=0).stop 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 shutdown_connections 2024-09-10T10:53:31.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a327c4a0 0x5639a327c880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327f880 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a32b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 --2- 172.21.15.28:0/3959882120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a3281640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 >> 172.21.15.28:0/3959882120 conn(0x5639a325a8c0 msgr2=0x5639a32a58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b6be00640 0 client.9252 ms_handle_reset on v2:172.21.15.28:6833/2097945693 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 shutdown_connections 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 1 -- 172.21.15.28:0/3959882120 wait complete. 2024-09-10T10:53:31.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.777+0000 7f8b910c7740 20 client.9252 trim_cache size 0 max 16384 2024-09-10T10:53:31.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.796+0000 7f8b910c7740 1 Processor -- start 2024-09-10T10:53:31.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.796+0000 7f8b910c7740 1 -- start start 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.796+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a32a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.796+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a3282ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.796+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b910c7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b881fd110 con 0x5639a327b680 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b88168320 con 0x5639a325d100 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b88168970 con 0x5639a325e890 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46342/0 (socket says 172.21.15.28:46342) 2024-09-10T10:53:31.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 -- 172.21.15.28:0/2393434835 learned_addr learned my addr 172.21.15.28:0/2393434835 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a3282ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87400640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a32a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 -- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 msgr2=0x5639a3282ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a3282ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 -- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 msgr2=0x5639a32a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a32a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87e00640 1 -- 172.21.15.28:0/2393434835 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a32859c0 con 0x5639a327b680 2024-09-10T10:53:31.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b87400640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a32a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.797+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a3282ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b87e00640 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b80056800 tx=0x7f8b80073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770ec686f9e18fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b76a00640 1 -- 172.21.15.28:0/2393434835 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8007a040 con 0x5639a327b680 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b76a00640 1 -- 172.21.15.28:0/2393434835 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80077070 con 0x5639a327b680 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b76a00640 1 -- 172.21.15.28:0/2393434835 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80062520 con 0x5639a327b680 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 -- 172.21.15.28:0/2393434835 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 msgr2=0x5639a327cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8b80056800 tx=0x7f8b80073000 comp rx=0 tx=0).stop 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 -- 172.21.15.28:0/2393434835 shutdown_connections 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a327b680 0x5639a327cab0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a325e890 0x5639a3282ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2393434835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a32a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 -- 172.21.15.28:0/2393434835 >> 172.21.15.28:0/2393434835 conn(0x5639a325a8c0 msgr2=0x5639a3252530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 -- 172.21.15.28:0/2393434835 shutdown_connections 2024-09-10T10:53:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.798+0000 7f8b910c7740 1 -- 172.21.15.28:0/2393434835 wait complete. 2024-09-10T10:53:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.799+0000 7f8b910c7740 1 -- start start 2024-09-10T10:53:31.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.799+0000 7f8b910c7740 1 Processor -- start 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a331daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a327b680 0x5639a32a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5639a3177780 con 0x5639a325e890 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639a31b83d0 con 0x5639a325d100 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5639a327e2c0 con 0x5639a327b680 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b910c7740 10 client.0 _ll_register_callbacks cb 0x7f8b80086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:31.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46356/0 (socket says 172.21.15.28:46356) 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.800+0000 7f8b7fe00640 1 -- 172.21.15.28:0/2307507007 learned_addr learned my addr 172.21.15.28:0/2307507007 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b87e00640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a327b680 0x5639a32a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b87400640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a331daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 -- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a327b680 msgr2=0x5639a32a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a327b680 0x5639a32a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 -- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 msgr2=0x5639a331daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a331daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 -- 172.21.15.28:0/2307507007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a329e390 con 0x5639a325e890 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b87400640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a331daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b7fe00640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8b700011e0 tx=0x7f8b70007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e636ca7515742319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b6be00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70001a40 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b80077070 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.801+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f8b70001be0 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=243) ==== 24+0+0 (secure 0 0 0) 0x7f8b70026050 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 10 client.9258 fetch_fsmap learned FSMap version 243 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=243,monmap=2+}) -- 0x5639a3321410 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80077070 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 243) ==== 3502+0+0 (secure 0 0 0) 0x7f8b70017460 con 0x5639a325e890 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 10 client.9258 fetch_fsmap finished waiting for FSMap version 243 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 10 client.9258 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 20 client.9258 populate_metadata read hostname 'smithi028' 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a3320d30 0x5639a327e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 4 client.9258 mds_command: new command op to 9174 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 --> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] -- command(tid 0: {"prefix": "status"}) -- 0x5639a31c3e90 con 0x5639a3320d30 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.802+0000 7f8b87400640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a3320d30 0x5639a327e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b87400640 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a3320d30 0x5639a327e440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c736cb3c1fd9d02d server_cookie=382cfa4bc115fae4 in_seq=0 out_seq=0 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b6be00640 10 client.9258 ms_handle_connect on v2:172.21.15.28:6833/2097945693 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b6be00640 1 -- 172.21.15.28:0/2307507007 <== mds.0 v2:172.21.15.28:6833/2097945693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5639a31c3e90 con 0x5639a3320d30 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b6be00640 10 client.9258 handle_command_reply: tid=0 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 client.9258 shutdown 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 msgr2=0x5639a32a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8b700011e0 tx=0x7f8b70007d50 comp rx=0 tx=0).stop 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 shutdown_connections 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:6833/2097945693,v1:172.21.15.28:6835/2097945693] conn(0x5639a3320d30 0x5639a327e440 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5639a327b680 0x5639a32a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639a325e890 0x5639a32a5280 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 --2- 172.21.15.28:0/2307507007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639a325d100 0x5639a331daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 >> 172.21.15.28:0/2307507007 conn(0x5639a325a8c0 msgr2=0x5639a3282ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b6be00640 0 client.9258 ms_handle_reset on v2:172.21.15.28:6833/2097945693 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 shutdown_connections 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.803+0000 7f8b910c7740 1 -- 172.21.15.28:0/2307507007 wait complete. 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b910c7740 20 client.9258 trim_cache size 0 max 16384 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:53:31.809 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 9174, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8682649619999996, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 243, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 126, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 126, 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 10.130258361999999 2024-09-10T10:53:31.810 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 msgr2=0x7f8b6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 0x7f8b6406e2f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b7802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 msgr2=0x7f8b8814c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009dd0 tx=0x7f8b8000c490 comp rx=0 tx=0).stop 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 shutdown_connections 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b6406be30 0x7f8b6406e2f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b8815b300 0x7f8b8814c250 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b880cd3f0 0x7f8b88154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 --2- 172.21.15.28:0/3368088490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b8806f8d0 0x7f8b881473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 >> 172.21.15.28:0/3368088490 conn(0x7f8b88067d70 msgr2=0x7f8b880cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 shutdown_connections 2024-09-10T10:53:31.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.804+0000 7f8b8e200640 1 -- 172.21.15.28:0/3368088490 wait complete. 2024-09-10T10:53:31.820 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:53:31.820 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Truncate 500.00000000', proceeding to ls 2024-09-10T10:53:31.821 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:53:31.921 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:31.920+0000 7f82ff600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:53:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.976+0000 7ff045200640 1 Processor -- start 2024-09-10T10:53:31.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.976+0000 7ff045200640 1 -- start start 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff0400d0dd0 0x7ff0400d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff04006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff0400d1f90 0x7ff0400cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400b37d0 con 0x7ff0400d1f90 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400cf330 con 0x7ff0400d0dd0 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff045200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0400cf000 con 0x7ff04015e660 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff03fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff0400d1f90 0x7ff0400cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff03fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff0400d1f90 0x7ff0400cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46372/0 (socket says 172.21.15.28:46372) 2024-09-10T10:53:31.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.977+0000 7ff03fe00640 1 -- 172.21.15.28:0/2636951685 learned_addr learned my addr 172.21.15.28:0/2636951685 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03f400640 1 --2- 172.21.15.28:0/2636951685 >> v2:172.21.15.138:3300/0 conn(0x7ff0400d0dd0 0x7ff0400d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03ea00640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff04006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03fe00640 1 -- 172.21.15.28:0/2636951685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 msgr2=0x7ff04006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03fe00640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff04006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03fe00640 1 -- 172.21.15.28:0/2636951685 >> v2:172.21.15.138:3300/0 conn(0x7ff0400d0dd0 msgr2=0x7ff0400d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03fe00640 1 --2- 172.21.15.28:0/2636951685 >> v2:172.21.15.138:3300/0 conn(0x7ff0400d0dd0 0x7ff0400d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.978+0000 7ff03fe00640 1 -- 172.21.15.28:0/2636951685 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0400d16f0 con 0x7ff0400d1f90 2024-09-10T10:53:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.979+0000 7ff03fe00640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d1f90 0x7ff0400cf7c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff0400b6fb0 tx=0x7ff03802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a83004c76403b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.979+0000 7ff03e000640 1 -- 172.21.15.28:0/2636951685 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038036070 con 0x7ff0400d1f90 2024-09-10T10:53:31.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.979+0000 7ff03e000640 1 -- 172.21.15.28:0/2636951685 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff038031600 con 0x7ff0400d1f90 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.979+0000 7ff03e000640 1 -- 172.21.15.28:0/2636951685 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380319d0 con 0x7ff0400d1f90 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 -- 172.21.15.28:0/2636951685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d1f90 msgr2=0x7ff0400cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d1f90 0x7ff0400cf7c0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff0400b6fb0 tx=0x7ff03802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 -- 172.21.15.28:0/2636951685 shutdown_connections 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d1f90 0x7ff0400cf7c0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 --2- 172.21.15.28:0/2636951685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff04006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 --2- 172.21.15.28:0/2636951685 >> v2:172.21.15.138:3300/0 conn(0x7ff0400d0dd0 0x7ff0400d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 -- 172.21.15.28:0/2636951685 >> 172.21.15.28:0/2636951685 conn(0x7ff04008bfc0 msgr2=0x7ff0400d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 -- 172.21.15.28:0/2636951685 shutdown_connections 2024-09-10T10:53:31.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.980+0000 7ff045200640 1 -- 172.21.15.28:0/2636951685 wait complete. 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.981+0000 7ff045200640 1 Processor -- start 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 -- start start 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0400d1f90 0x7ff0401e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff0401e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400d2f40 con 0x7ff0400d0dd0 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400d2c60 con 0x7ff0400d1f90 2024-09-10T10:53:31.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.982+0000 7ff045200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff0400d3220 con 0x7ff04015e660 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46374/0 (socket says 172.21.15.28:46374) 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 -- 172.21.15.28:0/2101712738 learned_addr learned my addr 172.21.15.28:0/2101712738 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03fe00640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff0401e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03ea00640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0400d1f90 0x7ff0401e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 -- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 msgr2=0x7ff0401e7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff0401e7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 -- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0400d1f90 msgr2=0x7ff0401e7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0400d1f90 0x7ff0401e7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:31.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 -- 172.21.15.28:0/2101712738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0401e3a60 con 0x7ff0400d0dd0 2024-09-10T10:53:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.983+0000 7ff03f400640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d800 tx=0x7ff03000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.984+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030018020 con 0x7ff0400d0dd0 2024-09-10T10:53:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.984+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0300096a0 con 0x7ff0400d0dd0 2024-09-10T10:53:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.984+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030009a70 con 0x7ff0400d0dd0 2024-09-10T10:53:31.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.984+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0401e62b0 con 0x7ff0400d0dd0 2024-09-10T10:53:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.984+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401483f0 con 0x7ff0400d0dd0 2024-09-10T10:53:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.985+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0401481e0 con 0x7ff0400d0dd0 2024-09-10T10:53:31.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.986+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff03001b020 con 0x7ff0400d0dd0 2024-09-10T10:53:31.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.987+0000 7ff037400640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 0x7ff02006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:31.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.987+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7ff03008bb40 con 0x7ff0400d0dd0 2024-09-10T10:53:31.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.987+0000 7ff03ea00640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 0x7ff02006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:31.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.988+0000 7ff03ea00640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 0x7ff02006e2f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d54e0 tx=0x7ff02c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:31.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:31.991+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff030022030 con 0x7ff0400d0dd0 2024-09-10T10:53:32.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.213+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0400d2440 con 0x7ff0400d0dd0 2024-09-10T10:53:32.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.213+0000 7ff037400640 1 -- 172.21.15.28:0/2101712738 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v243) ==== 53+0+73 (secure 0 0 0) 0x7ff030059a20 con 0x7ff0400d0dd0 2024-09-10T10:53:32.215 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:53:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 msgr2=0x7ff02006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 0x7ff02006e2f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff0400d54e0 tx=0x7ff02c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 msgr2=0x7ff04014a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d800 tx=0x7ff03000dcc0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 shutdown_connections 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff02006be30 0x7ff02006e2f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff04015e660 0x7ff0401e7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0400d1f90 0x7ff0401e7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 --2- 172.21.15.28:0/2101712738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0400d0dd0 0x7ff04014a680 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 >> 172.21.15.28:0/2101712738 conn(0x7ff04008bfc0 msgr2=0x7ff040157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 shutdown_connections 2024-09-10T10:53:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32.216+0000 7ff045200640 1 -- 172.21.15.28:0/2101712738 wait complete. 2024-09-10T10:53:32.232 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:53:32.232 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:53:32.232 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:53:32.232 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:53:32.233 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:53:32.233 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:53:32.233 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:53:32.233 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:53:32.233 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:53:32.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.233 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:53:32.269 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:53:32.269 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:53:32.269 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:53:32.270 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42093sec preferred_lft 42093sec 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:53:32.271 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:53:32.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.273 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:53:32.364 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:53:32.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.367 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:53:32.403 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:53:32.403 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:53:32.403 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:53:32.403 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:53:32.403 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:53:32.404 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:53:32.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:53:32.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:53:32.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.601 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:53:32.636 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:53:32.636 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:53:32.637 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:53:32.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.637 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:53:32.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:53:32.735 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:53:32.770 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:53:32.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:53:32.770 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:53:32.845 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:53:32.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:32.845 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:53:32.975 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:53:32.975 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:53:32.975 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:53:32.992 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:53:32.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:53:32.992 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:53:32.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:53:32.992 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:53:33.013 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:53:33.013 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:53:33.014 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:53:33.015 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:53:33.015 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:53:33.015 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:53:33.015 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:53:33.015 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:53:33.018 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:53:33.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:33.018 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:53:33.074 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:53:33.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:53:33.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:53:33.164 INFO:teuthology.orchestra.run.smithi028.stdout:client9267 2024-09-10T10:53:33.165 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:33.165 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:33.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:33.200 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:53:33.244 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:53:33.278 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:53:33.278 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:53:33.278 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:33.278 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:53:33.278 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:53:33.279 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 500.00000000' did not caused DAMAGED state 2024-09-10T10:53:33.279 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds_snaptable' 2024-09-10T10:53:33.280 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:33.280 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:33.356 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:33.356 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:33.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:53:33.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.626+0000 7f1fd7a00640 1 Processor -- start 2024-09-10T10:53:33.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.627+0000 7f1fd7a00640 1 -- start start 2024-09-10T10:53:33.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd015b300 0x7f1fd00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd7a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd7a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd00cdbe0 con 0x7f1fd0072d30 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd7a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd00728a0 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd00cd900 con 0x7f1fd015b300 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fcfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fcfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46380/0 (socket says 172.21.15.28:46380) 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fcfe00640 1 -- 172.21.15.28:0/68685863 learned_addr learned my addr 172.21.15.28:0/68685863 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd5a00640 1 --1- 172.21.15.28:0/68685863 >> v1:172.21.15.28:6789/0 conn(0x7f1fd0072d30 0x7f1fd00b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45646/0 (socket says 172.21.15.28:45646) 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.628+0000 7f1fd5000640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd015b300 0x7f1fd00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.629+0000 7f1fcfe00640 1 -- 172.21.15.28:0/68685863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd015b300 msgr2=0x7f1fd00cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.629+0000 7f1fcfe00640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd015b300 0x7f1fd00cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.629+0000 7f1fcfe00640 1 -- 172.21.15.28:0/68685863 >> v1:172.21.15.28:6789/0 conn(0x7f1fd0072d30 legacy=0x7f1fd00b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.629+0000 7f1fcfe00640 1 -- 172.21.15.28:0/68685863 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd0072250 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fcfe00640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009870 tx=0x7f1fbc02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a54c8405f25c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fcf400640 1 -- 172.21.15.28:0/68685863 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc037070 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fcf400640 1 -- 172.21.15.28:0/68685863 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fbc032440 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fcf400640 1 -- 172.21.15.28:0/68685863 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc032810 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fd7a00640 1 -- 172.21.15.28:0/68685863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd00cd3f0 msgr2=0x7f1fd006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009870 tx=0x7f1fbc02a250 comp rx=0 tx=0).stop 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.630+0000 7f1fd7a00640 1 -- 172.21.15.28:0/68685863 shutdown_connections 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.631+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd00cd3f0 0x7f1fd006f1c0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.631+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/68685863 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd015b300 0x7f1fd00cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.631+0000 7f1fd7a00640 1 -- 172.21.15.28:0/68685863 >> 172.21.15.28:0/68685863 conn(0x7f1fd0067d70 msgr2=0x7f1fd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.631+0000 7f1fd7a00640 1 -- 172.21.15.28:0/68685863 shutdown_connections 2024-09-10T10:53:33.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.631+0000 7f1fd7a00640 1 -- 172.21.15.28:0/68685863 wait complete. 2024-09-10T10:53:33.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.632+0000 7f1fd7a00640 1 Processor -- start 2024-09-10T10:53:33.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.632+0000 7f1fd7a00640 1 -- start start 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd0072d30 0x7f1fd0155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd00cd3f0 0x7f1fd00c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00d0160 con 0x7f1fd015b300 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00cffe0 con 0x7f1fd0072d30 2024-09-10T10:53:33.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd7a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd006f700 con 0x7f1fd00cd3f0 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.633+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46396/0 (socket says 172.21.15.28:46396) 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 -- 172.21.15.28:0/1470759138 learned_addr learned my addr 172.21.15.28:0/1470759138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fcfe00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd00cd3f0 0x7f1fd00c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5000640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd0072d30 0x7f1fd0155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 -- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd00cd3f0 msgr2=0x7f1fd00c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd00cd3f0 0x7f1fd00c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 -- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd0072d30 msgr2=0x7f1fd0155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd0072d30 0x7f1fd0155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 -- 172.21.15.28:0/1470759138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd00c5c60 con 0x7f1fd015b300 2024-09-10T10:53:33.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.634+0000 7f1fd5a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ebd0 tx=0x7f1fc800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.635+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8019070 con 0x7f1fd015b300 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.635+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc800cd40 con 0x7f1fd015b300 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.635+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8014600 con 0x7f1fd015b300 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.635+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd01f0740 con 0x7f1fd015b300 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.635+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd01f0d00 con 0x7f1fd015b300 2024-09-10T10:53:33.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.636+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd0154210 con 0x7f1fd015b300 2024-09-10T10:53:33.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.637+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fc801c070 con 0x7f1fd015b300 2024-09-10T10:53:33.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.638+0000 7f1fccc00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 0x7f1fb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:33.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.638+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f1fc808d7a0 con 0x7f1fd015b300 2024-09-10T10:53:33.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.639+0000 7f1fd5000640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 0x7f1fb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:33.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.640+0000 7f1fd5000640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 0x7f1fb006e320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00cb850 tx=0x7f1fc4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:33.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.643+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1fc805af40 con 0x7f1fd015b300 2024-09-10T10:53:33.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.839+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1fd015c100 con 0x7f1fd015b300 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.840+0000 7f1fccc00640 1 -- 172.21.15.28:0/1470759138 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 52 entries v126) ==== 81+0+3160 (secure 0 0 0) 0x7f1fc805f240 con 0x7f1fd015b300 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:53:33.841 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:53:33.842 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:53:33.843 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:53:33.844 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:53:33.845 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:53:33.845 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:53:33.845 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:53:33.845 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:53:33.845 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:listed 52 entries 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 msgr2=0x7f1fb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 0x7f1fb006e320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00cb850 tx=0x7f1fc4008040 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 msgr2=0x7f1fd014fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ebd0 tx=0x7f1fc800c430 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 shutdown_connections 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1fb006be60 0x7f1fb006e320 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1fd015b300 0x7f1fd014fe70 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fd00cd3f0 0x7f1fd00c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 --2- 172.21.15.28:0/1470759138 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fd0072d30 0x7f1fd0155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:33.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 >> 172.21.15.28:0/1470759138 conn(0x7f1fd0067d70 msgr2=0x7f1fd015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:33.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 shutdown_connections 2024-09-10T10:53:33.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:33.842+0000 7f1fd7a00640 1 -- 172.21.15.28:0/1470759138 wait complete. 2024-09-10T10:53:33.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:53:33.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:33.859 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:53:33.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:33.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:33.940 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:33.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:34.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.173+0000 7f5959e00640 1 Processor -- start 2024-09-10T10:53:34.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.173+0000 7f5959e00640 1 -- start start 2024-09-10T10:53:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.174+0000 7f5959e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f59540d4180 0x7f59540d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.174+0000 7f5959e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 0x7f595406d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.174+0000 7f5959e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59540d4aa0 con 0x7f595406dd60 2024-09-10T10:53:34.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.174+0000 7f5959e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f595406e170 con 0x7f59540d4180 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.174+0000 7f5959e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f59540cefb0 con 0x7f595415e6a0 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f59540d4180 0x7f59540d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f59540d4180 0x7f59540d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52260/0 (socket says 172.21.15.28:52260) 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 -- 172.21.15.28:0/3970661594 learned_addr learned my addr 172.21.15.28:0/3970661594 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5953e00640 1 --2- 172.21.15.28:0/3970661594 >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 0x7f595406d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 -- 172.21.15.28:0/3970661594 >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 msgr2=0x7f595406d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 --2- 172.21.15.28:0/3970661594 >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 0x7f595406d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 -- 172.21.15.28:0/3970661594 >> v1:172.21.15.138:6789/0 conn(0x7f595406dd60 legacy=0x7f59540d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5958a00640 1 -- 172.21.15.28:0/3970661594 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59540d2c20 con 0x7f59540d4180 2024-09-10T10:53:34.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.175+0000 7f5953e00640 1 --2- 172.21.15.28:0/3970661594 >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 0x7f595406d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.176+0000 7f5958a00640 1 --2- 172.21.15.28:0/3970661594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f59540d4560 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f595408d740 tx=0x7f594802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf60af4a6bca44f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.176+0000 7f5953400640 1 -- 172.21.15.28:0/3970661594 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948037070 con 0x7f59540d4180 2024-09-10T10:53:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.176+0000 7f5953400640 1 -- 172.21.15.28:0/3970661594 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5948032440 con 0x7f59540d4180 2024-09-10T10:53:34.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5953400640 1 -- 172.21.15.28:0/3970661594 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948032810 con 0x7f59540d4180 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 -- 172.21.15.28:0/3970661594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 msgr2=0x7f59540d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 --2- 172.21.15.28:0/3970661594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f59540d4560 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f595408d740 tx=0x7f594802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 -- 172.21.15.28:0/3970661594 shutdown_connections 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 --2- 172.21.15.28:0/3970661594 >> v2:172.21.15.138:3300/0 conn(0x7f595415e6a0 0x7f595406d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 --2- 172.21.15.28:0/3970661594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f59540d4560 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 -- 172.21.15.28:0/3970661594 >> 172.21.15.28:0/3970661594 conn(0x7f595409d6f0 msgr2=0x7f595408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 -- 172.21.15.28:0/3970661594 shutdown_connections 2024-09-10T10:53:34.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.177+0000 7f5959e00640 1 -- 172.21.15.28:0/3970661594 wait complete. 2024-09-10T10:53:34.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.179+0000 7f5959e00640 1 Processor -- start 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.179+0000 7f5959e00640 1 -- start start 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 0x7f59540b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f595415e6a0 0x7f59540cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f595406e950 con 0x7f59540d4180 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f595406e670 con 0x7f595406dd60 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f59540cebd0 con 0x7f595415e6a0 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52266/0 (socket says 172.21.15.28:52266) 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 -- 172.21.15.28:0/1562797695 learned_addr learned my addr 172.21.15.28:0/1562797695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5959400640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f595415e6a0 0x7f59540cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5958a00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 0x7f59540b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 -- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f595415e6a0 msgr2=0x7f59540cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f595415e6a0 0x7f59540cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 -- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 msgr2=0x7f59540b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.180+0000 7f5953e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 0x7f59540b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5953e00640 1 -- 172.21.15.28:0/1562797695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59540b9c00 con 0x7f59540d4180 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5958a00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 0x7f59540b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5953e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f594000c7f0 tx=0x7f594000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940015070 con 0x7f59540d4180 2024-09-10T10:53:34.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5940007820 con 0x7f59540d4180 2024-09-10T10:53:34.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59540b7d60 con 0x7f59540d4180 2024-09-10T10:53:34.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940007bd0 con 0x7f59540d4180 2024-09-10T10:53:34.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.181+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595415cf10 con 0x7f59540d4180 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.182+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59540d4560 con 0x7f59540d4180 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.183+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5940018070 con 0x7f59540d4180 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.184+0000 7f5950c00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 0x7f593406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.184+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8540+0+0 (secure 0 0 0) 0x7f594008bbb0 con 0x7f59540d4180 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.184+0000 7f5958a00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 0x7f593406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.185+0000 7f5958a00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 0x7f593406e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5948034000 tx=0x7f594802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.188+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5940059910 con 0x7f59540d4180 2024-09-10T10:53:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.408+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f59540b41b0 con 0x7f59540d4180 2024-09-10T10:53:34.410 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:34.409+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:34.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.411+0000 7f5950c00640 1 -- 172.21.15.28:0/1562797695 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v244) ==== 161+0+0 (secure 0 0 0) 0x7f5940059210 con 0x7f59540d4180 2024-09-10T10:53:34.412 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 msgr2=0x7f593406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 0x7f593406e2f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5948034000 tx=0x7f594802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 msgr2=0x7f5954150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f594000c7f0 tx=0x7f594000ccb0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 shutdown_connections 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f593406be30 0x7f593406e2f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f595415e6a0 0x7f59540cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59540d4180 0x7f5954150b30 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 --2- 172.21.15.28:0/1562797695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f595406dd60 0x7f59540b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 >> 172.21.15.28:0/1562797695 conn(0x7f595409d6f0 msgr2=0x7f59540b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:34.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 shutdown_connections 2024-09-10T10:53:34.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.415+0000 7f5959e00640 1 -- 172.21.15.28:0/1562797695 wait complete. 2024-09-10T10:53:34.420 INFO:tasks.ceph.mds.a.smithi028.stderr: -556> 2024-09-10T10:53:31.920+0000 7f82ff600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:53:34.433 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:34.464 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:34.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.590+0000 7fa236a00640 1 Processor -- start 2024-09-10T10:53:34.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.590+0000 7fa236a00640 1 -- start start 2024-09-10T10:53:34.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.591+0000 7fa236a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa23006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.591+0000 7fa236a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa23006e870 con 0x7fa2300b6680 2024-09-10T10:53:34.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.591+0000 7fa236a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa23006d9f0 con 0x7fa230162060 2024-09-10T10:53:34.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.591+0000 7fa236a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa23006dcd0 con 0x7fa230161c80 2024-09-10T10:53:34.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa234a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa2300b6680 0x7fa2300cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38548/0 (socket says 172.21.15.28:38548) 2024-09-10T10:53:34.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa234a00640 1 -- 172.21.15.28:0/787502943 learned_addr learned my addr 172.21.15.28:0/787502943 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:34.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22fe00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa23006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2079196123 0 0) 0x7fa228001490 con 0x7fa2300b6680 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa214003520 con 0x7fa2300b6680 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240872454 0 0) 0x7fa23006d9f0 con 0x7fa230162060 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa228001490 con 0x7fa230162060 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893890354 0 0) 0x7fa214003520 con 0x7fa2300b6680 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 msgr2=0x7fa23006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa23006e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 >> v1:172.21.15.138:6789/0 conn(0x7fa230162060 legacy=0x7fa2300b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23006dfb0 con 0x7fa2300b6680 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa228001bc0 con 0x7fa2300b6680 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 >> v1:172.21.15.28:6789/0 conn(0x7fa2300b6680 legacy=0x7fa2300cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.592+0000 7fa22ea00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214003b70 0x7fa214003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22ea00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa214004800 0x7fa214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22ea00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa2140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa23006dcd0 con 0x7fa214004800 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa21400ba70 con 0x7fa214003b70 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa21400bc40 con 0x7fa230161c80 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa214004800 0x7fa214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22fe00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214003b70 0x7fa214003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa234a00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa2140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 -- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 msgr2=0x7fa2140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa230161c80 0x7fa2140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 -- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214003b70 msgr2=0x7fa214003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214003b70 0x7fa214003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.593+0000 7fa22f400640 1 -- 172.21.15.28:0/787502943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa214003520 con 0x7fa214004800 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.594+0000 7fa22f400640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa214004800 0x7fa214004c80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa23008e830 tx=0x7fa214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1c6587742d0f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.594+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0014f0 con 0x7fa214004800 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa21c0052d0 con 0x7fa214004800 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa22ea00640 1 -- 172.21.15.28:0/787502943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c02aba0 con 0x7fa214004800 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 -- 172.21.15.28:0/787502943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa214004800 msgr2=0x7fa214004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 --2- 172.21.15.28:0/787502943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa214004800 0x7fa214004c80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa23008e830 tx=0x7fa214005830 comp rx=0 tx=0).stop 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa234a00640 1 -- 172.21.15.28:0/787502943 reap_dead start 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 -- 172.21.15.28:0/787502943 shutdown_connections 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 -- 172.21.15.28:0/787502943 >> 172.21.15.28:0/787502943 conn(0x7fa2300d44f0 msgr2=0x7fa2300d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 -- 172.21.15.28:0/787502943 shutdown_connections 2024-09-10T10:53:34.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.595+0000 7fa236a00640 1 -- 172.21.15.28:0/787502943 wait complete. 2024-09-10T10:53:34.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.597+0000 7fa236a00640 1 Processor -- start 2024-09-10T10:53:34.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.597+0000 7fa236a00640 1 -- start start 2024-09-10T10:53:34.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214004800 0x7fa2301f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2300b7340 0x7fa230159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300b6cb0 con 0x7fa2301f4bf0 2024-09-10T10:53:34.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300b6b30 con 0x7fa214004800 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa236a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa23006e330 con 0x7fa2300b7340 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52288/0 (socket says 172.21.15.28:52288) 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 -- 172.21.15.28:0/947942141 learned_addr learned my addr 172.21.15.28:0/947942141 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 -- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2300b7340 msgr2=0x7fa230159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2300b7340 0x7fa230159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 -- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214004800 msgr2=0x7fa2301f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214004800 0x7fa2301f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.598+0000 7fa234a00640 1 -- 172.21.15.28:0/947942141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2300ce890 con 0x7fa2301f4bf0 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa234a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa23008e1d0 tx=0x7fa22800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800edc0 con 0x7fa2301f4bf0 2024-09-10T10:53:34.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa228013420 con 0x7fa2301f4bf0 2024-09-10T10:53:34.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228013610 con 0x7fa2301f4bf0 2024-09-10T10:53:34.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2301f81d0 con 0x7fa2301f4bf0 2024-09-10T10:53:34.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.599+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2301f8790 con 0x7fa2301f4bf0 2024-09-10T10:53:34.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.600+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2300b5f70 con 0x7fa2301f4bf0 2024-09-10T10:53:34.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.601+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa22801a020 con 0x7fa2301f4bf0 2024-09-10T10:53:34.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.602+0000 7fa227e00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 0x7fa21006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:34.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.602+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8626+0+0 (secure 0 0 0) 0x7fa22808b570 con 0x7fa2301f4bf0 2024-09-10T10:53:34.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.602+0000 7fa22fe00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 0x7fa21006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:34.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.603+0000 7fa22fe00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 0x7fa21006e320 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2200099e0 tx=0x7fa220006c30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:34.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.606+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa228021020 con 0x7fa2301f4bf0 2024-09-10T10:53:34.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:34.863+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fa2301f8960 con 0x7fa2301f4bf0 2024-09-10T10:53:35.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.417+0000 7fa227e00640 1 -- 172.21.15.28:0/947942141 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v245) ==== 107+0+0 (secure 0 0 0) 0x7fa23006e870 con 0x7fa2301f4bf0 2024-09-10T10:53:35.419 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:53:35.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 msgr2=0x7fa21006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 0x7fa21006e320 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa2200099e0 tx=0x7fa220006c30 comp rx=0 tx=0).stop 2024-09-10T10:53:35.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 msgr2=0x7fa2301f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa23008e1d0 tx=0x7fa22800e530 comp rx=0 tx=0).stop 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 shutdown_connections 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa21006be60 0x7fa21006e320 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2301f4bf0 0x7fa2301f5070 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2300b7340 0x7fa230159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 --2- 172.21.15.28:0/947942141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa214004800 0x7fa2301f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 >> 172.21.15.28:0/947942141 conn(0x7fa2300d44f0 msgr2=0x7fa2300b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 shutdown_connections 2024-09-10T10:53:35.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.420+0000 7fa236a00640 1 -- 172.21.15.28:0/947942141 wait complete. 2024-09-10T10:53:35.439 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 Processor -- start 2024-09-10T10:53:35.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 -- start start 2024-09-10T10:53:35.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5a153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5ab0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cfc5a2e530 con 0x55cfc5abc9c0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cfc598bb30 con 0x55cfc5ab93e0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.560+0000 7f7760f20f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cfc5abbee0 con 0x55cfc5a2dff0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52308/0 (socket says 172.21.15.28:52308) 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 -- 172.21.15.28:0/1003311884 learned_addr learned my addr 172.21.15.28:0/1003311884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775d800640 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5a153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775ce00640 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5ab0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 -- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 msgr2=0x55cfc5a153f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:35.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5a153f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 -- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 msgr2=0x55cfc5ab0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5ab0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 -- 172.21.15.28:0/1003311884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc5a30a80 con 0x55cfc5abc9c0 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.561+0000 7f775e200640 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f775800b5f0 tx=0x7f775800bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261e9a0cadaedfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7757400640 1 -- 172.21.15.28:0/1003311884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758014030 con 0x55cfc5abc9c0 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7757400640 1 -- 172.21.15.28:0/1003311884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7758009640 con 0x55cfc5abc9c0 2024-09-10T10:53:35.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7757400640 1 -- 172.21.15.28:0/1003311884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77580099f0 con 0x55cfc5abc9c0 2024-09-10T10:53:35.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 -- 172.21.15.28:0/1003311884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 msgr2=0x55cfc59e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f775800b5f0 tx=0x7f775800bab0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 -- 172.21.15.28:0/1003311884 shutdown_connections 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc59e2ac0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5ab0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 --2- 172.21.15.28:0/1003311884 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5a153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 -- 172.21.15.28:0/1003311884 >> 172.21.15.28:0/1003311884 conn(0x55cfc5a320a0 msgr2=0x55cfc5a32480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 -- 172.21.15.28:0/1003311884 shutdown_connections 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.562+0000 7f7760f20f40 1 -- 172.21.15.28:0/1003311884 wait complete. 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.563+0000 7f7760f20f40 1 Processor -- start 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.563+0000 7f7760f20f40 1 -- start start 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5ab1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5aba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cfc5a15020 con 0x55cfc5abc9c0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cfc5a30db0 con 0x55cfc5ab93e0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f7760f20f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cfc5ab34d0 con 0x55cfc5a2dff0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f775e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f775e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52314/0 (socket says 172.21.15.28:52314) 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.564+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 learned_addr learned my addr 172.21.15.28:0/521274739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5ab1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5aba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 msgr2=0x55cfc5ab1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cfc5a2dff0 0x55cfc5ab1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 msgr2=0x55cfc5aba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfc5ab93e0 0x55cfc5aba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc5aae9b0 con 0x55cfc5abc9c0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f775e200640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7758007010 tx=0x7f77580033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.565+0000 7f7756000640 1 -- 172.21.15.28:0/521274739 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7758027070 con 0x55cfc5abc9c0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.566+0000 7f7756000640 1 -- 172.21.15.28:0/521274739 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7758010bd0 con 0x55cfc5abc9c0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.566+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfc5aaec90 con 0x55cfc5abc9c0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.566+0000 7f7756000640 1 -- 172.21.15.28:0/521274739 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775801a460 con 0x55cfc5abc9c0 2024-09-10T10:53:35.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.566+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc5aaf250 con 0x55cfc5abc9c0 2024-09-10T10:53:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.567+0000 7f7756000640 1 -- 172.21.15.28:0/521274739 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f775802a070 con 0x55cfc5abc9c0 2024-09-10T10:53:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.568+0000 7f7756000640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 0x7f774006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.568+0000 7f7756000640 1 -- 172.21.15.28:0/521274739 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8626+0+0 (secure 0 0 0) 0x7f775808c220 con 0x55cfc5abc9c0 2024-09-10T10:53:35.570 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.569+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cfc5aae520 0x55cfc5abd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.569+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5abaaf0 con 0x55cfc5aae520 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.569+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 0x7f774006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.569+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cfc5aae520 0x55cfc5abd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.570+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 0x7f774006cd00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7750057000 tx=0x7f7750052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.570+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cfc5aae520 0x55cfc5abd850 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.572+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v127'815 uv789 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.572+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5abdd90 con 0x55cfc5aae520 2024-09-10T10:53:35.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.574+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v127'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.574+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5abe1f0 con 0x55cfc5aae520 2024-09-10T10:53:35.576 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:35.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.576+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v127'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.576+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5abe590 con 0x55cfc5aae520 2024-09-10T10:53:35.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.578+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v127'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.578+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f800 con 0x55cfc5aae520 2024-09-10T10:53:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.579+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v127'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.579+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5e7f0 con 0x55cfc5aae520 2024-09-10T10:53:35.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.581+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v127'820 uv820 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.581+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ebe0 con 0x55cfc5aae520 2024-09-10T10:53:35.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.583+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v127'821 uv795 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.583+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5efd0 con 0x55cfc5aae520 2024-09-10T10:53:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.585+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v127'822 uv822 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.585+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5efd0 con 0x55cfc5aae520 2024-09-10T10:53:35.587 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:35.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.586+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v127'823 uv823 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.587+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.589+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v127'824 uv824 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.589+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.590+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v127'825 uv825 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.590+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.592+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v127'826 uv826 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.592+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.594+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v127'827 uv802 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.594+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.596+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v127'828 uv828 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.596+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.597+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v127'829 uv829 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.598 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:35.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.597+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.599+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v127'830 uv830 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.599+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.601+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v127'831 uv831 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.601+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5ef80 con 0x55cfc5aae520 2024-09-10T10:53:35.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.602+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v127'832 uv832 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.602+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.604+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v127'833 uv833 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.604+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.606+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v127'834 uv808 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.606+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.608+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v127'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.608+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.610+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v127'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.611 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:35.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.610+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.611+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v127'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.611+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.613+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v127'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.613+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.615+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v127'839 uv839 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.615+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.617+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v127'840 uv814 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.617+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.619+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v127'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.619+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.621 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:35.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.620+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v127'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.620+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f360 con 0x55cfc5aae520 2024-09-10T10:53:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.622+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v127'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.622+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f240 con 0x55cfc5aae520 2024-09-10T10:53:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.624+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v127'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.624+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f240 con 0x55cfc5aae520 2024-09-10T10:53:35.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.625+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v127'845 uv845 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5aae520 2024-09-10T10:53:35.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.626+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cfc5b600b0 0x55cfc5b624a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.626+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b5f240 con 0x55cfc5b600b0 2024-09-10T10:53:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.626+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cfc5b600b0 0x55cfc5b624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.627+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cfc5b600b0 0x55cfc5b624a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.629+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v127'596 uv575 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.629+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b629e0 con 0x55cfc5b600b0 2024-09-10T10:53:35.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.630+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v127'597 uv597 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.631+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b62d80 con 0x55cfc5b600b0 2024-09-10T10:53:35.633 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:35.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.632+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v127'598 uv598 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.632+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b63120 con 0x55cfc5b600b0 2024-09-10T10:53:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.633+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v127'599 uv599 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.633+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5ab05d0 con 0x55cfc5b600b0 2024-09-10T10:53:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.635+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v127'600 uv600 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.635+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b639c0 con 0x55cfc5b600b0 2024-09-10T10:53:35.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.636+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v127'601 uv594 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.636+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b63d60 con 0x55cfc5b600b0 2024-09-10T10:53:35.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.638+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v127'602 uv602 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.638+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.640 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:35.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.639+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v127'603 uv603 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.639+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.641+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v127'604 uv604 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.641+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.642+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v127'605 uv605 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.642+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.644+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v127'606 uv595 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.644+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.645+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v127'607 uv607 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.645+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.647 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.647+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v127'608 uv608 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.647+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b687f0 con 0x55cfc5b600b0 2024-09-10T10:53:35.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.649+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v127'609 uv609 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.649+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b687f0 con 0x55cfc5b600b0 2024-09-10T10:53:35.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.650+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v127'610 uv610 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.650+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.652+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v127'611 uv591 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.652+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.653+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v127'612 uv612 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.653+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.656 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:35.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.655+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v127'613 uv613 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.655+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b65110 con 0x55cfc5b600b0 2024-09-10T10:53:35.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.656+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v127'614 uv614 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.657+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b65110 con 0x55cfc5b600b0 2024-09-10T10:53:35.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.658+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v127'615 uv615 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.658+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b64100 con 0x55cfc5b600b0 2024-09-10T10:53:35.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.659+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v127'616 uv616 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b600b0 2024-09-10T10:53:35.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.659+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cfc5b64780 0x55cfc5b66b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.659+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b67180 con 0x55cfc5b64780 2024-09-10T10:53:35.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.660+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cfc5b64780 0x55cfc5b66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.660+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cfc5b64780 0x55cfc5b66b70 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.662+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v127'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.662+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b67550 con 0x55cfc5b64780 2024-09-10T10:53:35.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.663+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.663+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b67550 con 0x55cfc5b64780 2024-09-10T10:53:35.665 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:35.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.664+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v127'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.664+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b68560 con 0x55cfc5b64780 2024-09-10T10:53:35.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.666+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v127'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.666+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b68560 con 0x55cfc5b64780 2024-09-10T10:53:35.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.667+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v127'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.667+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b67550 con 0x55cfc5b64780 2024-09-10T10:53:35.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.668+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v127'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.668+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cfc5b67bd0 0x55cfc5b6a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.669+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6a710 con 0x55cfc5b67bd0 2024-09-10T10:53:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.669+0000 7f775e200640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cfc5b67bd0 0x55cfc5b6a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.669+0000 7f775e200640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cfc5b67bd0 0x55cfc5b6a040 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.671+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v127'116 uv115 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.671+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6ac20 con 0x55cfc5b67bd0 2024-09-10T10:53:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.672+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v127'117 uv117 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.672+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6b120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.674 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.673+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v127'118 uv118 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.673+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6b9e0 con 0x55cfc5b67bd0 2024-09-10T10:53:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.674+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v127'119 uv119 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.674+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6bd80 con 0x55cfc5b64780 2024-09-10T10:53:35.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.675+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v127'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.675+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6bd80 con 0x55cfc5b64780 2024-09-10T10:53:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.676+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.676+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6bd80 con 0x55cfc5b64780 2024-09-10T10:53:35.678 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.678+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v127'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.678+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6cd90 con 0x55cfc5b64780 2024-09-10T10:53:35.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.679+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v127'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.679+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6cd90 con 0x55cfc5b64780 2024-09-10T10:53:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.680+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v127'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.680+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6bd80 con 0x55cfc5b64780 2024-09-10T10:53:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.681+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v127'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.681+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6bd80 con 0x55cfc5b67bd0 2024-09-10T10:53:35.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.683+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v127'294 uv287 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.683+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.684+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v127'295 uv295 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.684+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.686 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:35.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.685+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v127'296 uv296 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.685+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6d130 con 0x55cfc5b67bd0 2024-09-10T10:53:35.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.687+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v127'297 uv297 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.687+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6d130 con 0x55cfc5b67bd0 2024-09-10T10:53:35.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.688+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v127'298 uv298 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.688+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.690+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v127'299 uv299 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.690+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.691+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v127'300 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.691+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.692+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v127'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.692+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.694 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:35.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.693+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v127'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.693+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.695+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v127'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.695+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b67bd0 2024-09-10T10:53:35.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.696+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v127'304 uv304 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.696+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b64780 2024-09-10T10:53:35.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.698+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v127'132 uv131 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.698+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b64780 2024-09-10T10:53:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.699+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v127'133 uv133 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.699+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b64780 2024-09-10T10:53:35.701 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.700+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v127'134 uv134 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.700+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b64780 2024-09-10T10:53:35.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.702+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v127'135 uv135 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.702+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6c120 con 0x55cfc5b64780 2024-09-10T10:53:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.703+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v127'136 uv136 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b64780 2024-09-10T10:53:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.703+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cfc5b6c7a0 0x55cfc5b6eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.703+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f1f0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.703+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cfc5b6c7a0 0x55cfc5b6eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.704+0000 7f775d800640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cfc5b6c7a0 0x55cfc5b6eb90 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.705+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v127'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.705+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f2b0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.707+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.707+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.709 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.708+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v127'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.708+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b706b0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.709+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v127'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.709+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b706b0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.711+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v127'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.711+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.712+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v127'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.712+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.714+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v127'263 uv257 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.714+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.715+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v127'264 uv264 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.715+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.716+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v127'265 uv265 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.717 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:35.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.716+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.718+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v127'266 uv266 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.718+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.719+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v127'267 uv267 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.719+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.720+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v127'268 uv262 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.720+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.722+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v127'269 uv269 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.722+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.724 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:35.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.723+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v127'270 uv270 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.723+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6f6a0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.724+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v127'271 uv271 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.724+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b724b0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.726+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v127'272 uv272 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.726+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.727+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v127'116 uv114 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.727+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.729+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v127'117 uv117 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.729+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.730+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v127'118 uv118 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.731 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:35.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.730+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.731+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v127'119 uv119 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.732+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.733+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v127'159 uv158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.733+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.734+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v127'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.734+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.736 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.735+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v127'161 uv161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.735+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70b60 con 0x55cfc5b67bd0 2024-09-10T10:53:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.736+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v127'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.736+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70b60 con 0x55cfc5b67bd0 2024-09-10T10:53:35.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.738+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v127'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.738+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b6fb50 con 0x55cfc5b67bd0 2024-09-10T10:53:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.739+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v127'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f7758029030 con 0x55cfc5b67bd0 2024-09-10T10:53:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.739+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cfc5b701d0 0x55cfc5b705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.739+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b701d0 2024-09-10T10:53:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.739+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cfc5b701d0 0x55cfc5b705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.740+0000 7f775ce00640 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cfc5b701d0 0x55cfc5b705b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.741+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v127'135 uv134 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b701d0 2024-09-10T10:53:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.741+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b701d0 2024-09-10T10:53:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.742+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v127'136 uv136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b701d0 2024-09-10T10:53:35.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.742+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b701d0 2024-09-10T10:53:35.744 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.743+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v127'137 uv137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b701d0 2024-09-10T10:53:35.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.743+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b701d0 2024-09-10T10:53:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.745+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v127'138 uv138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b701d0 2024-09-10T10:53:35.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.745+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b701d0 2024-09-10T10:53:35.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.746+0000 7f775ce00640 1 -- 172.21.15.28:0/521274739 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v127'139 uv139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f774c010650 con 0x55cfc5b701d0 2024-09-10T10:53:35.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.746+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.747+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v127'327 uv318 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.747+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.749+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v127'328 uv328 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.749+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.751 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.750+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v127'329 uv329 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.750+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b710d0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.751+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v127'330 uv330 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.751+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b710d0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.752+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v127'331 uv331 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.752+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b71050 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.754+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v127'332 uv325 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.754+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b71050 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.755+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v127'333 uv333 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.755+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.757 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.756+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v127'334 uv334 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.756+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.757+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v127'335 uv335 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.757+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.758+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v127'336 uv336 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.758+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b70af0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.760+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v127'337 uv337 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.760+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x55cfc5b71540 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f775d800640 1 -- 172.21.15.28:0/521274739 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v127'338 uv338 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f7750050bf0 con 0x55cfc5b6c7a0 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cfc5b64780 msgr2=0x55cfc5b66b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cfc5b64780 0x55cfc5b66b70 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cfc5b6c7a0 msgr2=0x55cfc5b6eb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cfc5b6c7a0 0x55cfc5b6eb90 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cfc5b701d0 msgr2=0x55cfc5b705b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cfc5b701d0 0x55cfc5b705b0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cfc5b67bd0 msgr2=0x55cfc5b6a040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cfc5b67bd0 0x55cfc5b6a040 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cfc5b600b0 msgr2=0x55cfc5b624a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 reap_dead start 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cfc5b600b0 0x55cfc5b624a0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f775e200640 1 -- 172.21.15.28:0/521274739 reap_dead start 2024-09-10T10:53:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cfc5aae520 msgr2=0x55cfc5abd850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cfc5aae520 0x55cfc5abd850 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 msgr2=0x7f774006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 0x7f774006cd00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7750057000 tx=0x7f7750052000 comp rx=0 tx=0).stop 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 msgr2=0x55cfc5ab0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7758007010 tx=0x7f77580033a0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 shutdown_connections 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f774006a840 0x7f774006cd00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 --2- 172.21.15.28:0/521274739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cfc5abc9c0 0x55cfc5ab0010 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 >> 172.21.15.28:0/521274739 conn(0x55cfc5a320a0 msgr2=0x55cfc5a32480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 shutdown_connections 2024-09-10T10:53:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.761+0000 7f7760f20f40 1 -- 172.21.15.28:0/521274739 wait complete. 2024-09-10T10:53:35.767 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds_snaptable 0 2024-09-10T10:53:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 Processor -- start 2024-09-10T10:53:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 -- start start 2024-09-10T10:53:35.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eed9a78e0 0x563eeda32800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eeda333e0 0x563eeda38990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563eed9a8530 con 0x563eeda369c0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eed9a82d0 con 0x563eeda333e0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.884+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563eeda35ee0 con 0x563eed9a78e0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52328/0 (socket says 172.21.15.28:52328) 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 -- 172.21.15.28:0/2393776921 learned_addr learned my addr 172.21.15.28:0/2393776921 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6400640 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eed9a78e0 0x563eeda32800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b5a00640 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eeda333e0 0x563eeda38990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 -- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eed9a78e0 msgr2=0x563eeda32800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eed9a78e0 0x563eeda32800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 -- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eeda333e0 msgr2=0x563eeda38990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eeda333e0 0x563eeda38990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.885+0000 7f05b6e00640 1 -- 172.21.15.28:0/2393776921 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eed889b30 con 0x563eeda369c0 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b6e00640 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f05b0002800 tx=0x7f05b0002cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b7a5cb0aef33d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05afe00640 1 -- 172.21.15.28:0/2393776921 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0016020 con 0x563eeda369c0 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05afe00640 1 -- 172.21.15.28:0/2393776921 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b000d9c0 con 0x563eeda369c0 2024-09-10T10:53:35.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05afe00640 1 -- 172.21.15.28:0/2393776921 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b000ddf0 con 0x563eeda369c0 2024-09-10T10:53:35.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/2393776921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 msgr2=0x563eeda38ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f05b0002800 tx=0x7f05b0002cc0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/2393776921 shutdown_connections 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda38ed0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eeda333e0 0x563eeda38990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/2393776921 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eed9a78e0 0x563eeda32800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/2393776921 >> 172.21.15.28:0/2393776921 conn(0x563eed9ac0a0 msgr2=0x563eed9aa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/2393776921 shutdown_connections 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.886+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/2393776921 wait complete. 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.887+0000 7f05b9d1ff40 1 Processor -- start 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.887+0000 7f05b9d1ff40 1 -- start start 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eed9a78e0 0x563eeda21230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eeda333e0 0x563eeda39a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563eed991860 con 0x563eeda369c0 2024-09-10T10:53:35.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563eeda37c10 con 0x563eed9a78e0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b9d1ff40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563eeda37eb0 con 0x563eeda333e0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eed9a78e0 0x563eeda21230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b6e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eed9a78e0 0x563eeda21230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:38344/0 (socket says 172.21.15.28:38344) 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.888+0000 7f05b6400640 1 -- 172.21.15.28:0/1553569402 learned_addr learned my addr 172.21.15.28:0/1553569402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b5a00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eeda333e0 0x563eeda39a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 -- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eeda333e0 msgr2=0x563eeda39a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563eeda333e0 0x563eeda39a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 -- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eed9a78e0 msgr2=0x563eeda21230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563eed9a78e0 0x563eeda21230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 -- 172.21.15.28:0/1553569402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eeda24d10 con 0x563eeda369c0 2024-09-10T10:53:35.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05b6e00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda2fcc0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f05b0017000 tx=0x7f05b00033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05aea00640 1 -- 172.21.15.28:0/1553569402 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0016040 con 0x563eeda369c0 2024-09-10T10:53:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05aea00640 1 -- 172.21.15.28:0/1553569402 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05b001b420 con 0x563eeda369c0 2024-09-10T10:53:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.889+0000 7f05aea00640 1 -- 172.21.15.28:0/1553569402 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b001b810 con 0x563eeda369c0 2024-09-10T10:53:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.890+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563eeda24fd0 con 0x563eeda369c0 2024-09-10T10:53:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.890+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eeda25590 con 0x563eeda369c0 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.892+0000 7f05aea00640 1 -- 172.21.15.28:0/1553569402 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f05b0019020 con 0x563eeda369c0 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.892+0000 7f05aea00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f059806a810 0x7f059806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.892+0000 7f05aea00640 1 -- 172.21.15.28:0/1553569402 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8626+0+0 (secure 0 0 0) 0x7f05b008c310 con 0x563eeda369c0 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.893+0000 7f05b6400640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f059806a810 0x7f059806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.893+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563eedacc0e0 0x563eed98f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.893+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) -- 0x563eed98fbc0 con 0x563eedacc0e0 2024-09-10T10:53:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.893+0000 7f05b5a00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563eedacc0e0 0x563eed98f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:35.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.893+0000 7f05b6400640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f059806a810 0x7f059806ccd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f05a8005fd0 tx=0x7f05a800c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.894+0000 7f05b5a00640 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563eedacc0e0 0x563eed98f680 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:35.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b5a00640 1 -- 172.21.15.28:0/1553569402 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(1 mds_snaptable [truncate 0] v127'137 uv137 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f05a4050050 con 0x563eedacc0e0 2024-09-10T10:53:35.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563eedacc0e0 msgr2=0x563eed98f680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x563eedacc0e0 0x563eed98f680 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f059806a810 msgr2=0x7f059806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f059806a810 0x7f059806ccd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f05a8005fd0 tx=0x7f05a800c040 comp rx=0 tx=0).stop 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 msgr2=0x563eeda2fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 --2- 172.21.15.28:0/1553569402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563eeda369c0 0x563eeda2fcc0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f05b0017000 tx=0x7f05b00033a0 comp rx=0 tx=0).stop 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b6e00640 1 -- 172.21.15.28:0/1553569402 reap_dead start 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 shutdown_connections 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 >> 172.21.15.28:0/1553569402 conn(0x563eed9ac0a0 msgr2=0x563eeda32800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 shutdown_connections 2024-09-10T10:53:35.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:35.895+0000 7f05b9d1ff40 1 -- 172.21.15.28:0/1553569402 wait complete. 2024-09-10T10:53:35.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:36.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.058+0000 7fd814400640 1 Processor -- start 2024-09-10T10:53:36.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.058+0000 7fd814400640 1 -- start start 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.059+0000 7fd814400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.059+0000 7fd814400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd80c0cfea0 0x7fd80c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.059+0000 7fd814400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd80c0b30e0 con 0x7fd80c0b4af0 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.059+0000 7fd814400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd80c06edc0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.059+0000 7fd814400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0b33c0 con 0x7fd80c156740 2024-09-10T10:53:36.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd80c0cfea0 0x7fd80c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd80c0cfea0 0x7fd80c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52344/0 (socket says 172.21.15.28:52344) 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 -- 172.21.15.28:0/3213814899 learned_addr learned my addr 172.21.15.28:0/3213814899 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd811000640 1 --1- 172.21.15.28:0/3213814899 >> v1:172.21.15.28:6789/0 conn(0x7fd80c0b4af0 0x7fd80c0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38564/0 (socket says 172.21.15.28:38564) 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd811a00640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 -- 172.21.15.28:0/3213814899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 msgr2=0x7fd80c0b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 -- 172.21.15.28:0/3213814899 >> v1:172.21.15.28:6789/0 conn(0x7fd80c0b4af0 legacy=0x7fd80c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd812400640 1 -- 172.21.15.28:0/3213814899 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0b36a0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd80be00640 1 -- 172.21.15.28:0/3213814899 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 439592519 0 0) 0x7fd80c0b30e0 con 0x7fd80c0b4af0 2024-09-10T10:53:36.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.060+0000 7fd811a00640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.061+0000 7fd812400640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c0ce060 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd804009e50 tx=0x7fd80402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5aca6445d00eb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.061+0000 7fd80be00640 1 -- 172.21.15.28:0/3213814899 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804036070 con 0x7fd80c0cfea0 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.061+0000 7fd80be00640 1 -- 172.21.15.28:0/3213814899 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd804031600 con 0x7fd80c0cfea0 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.061+0000 7fd80be00640 1 -- 172.21.15.28:0/3213814899 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8040319d0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 -- 172.21.15.28:0/3213814899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 msgr2=0x7fd80c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c0ce060 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fd804009e50 tx=0x7fd80402a730 comp rx=0 tx=0).stop 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 -- 172.21.15.28:0/3213814899 shutdown_connections 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c0ce060 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 --2- 172.21.15.28:0/3213814899 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 -- 172.21.15.28:0/3213814899 >> 172.21.15.28:0/3213814899 conn(0x7fd80c06bdf0 msgr2=0x7fd80c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 -- 172.21.15.28:0/3213814899 shutdown_connections 2024-09-10T10:53:36.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.062+0000 7fd814400640 1 -- 172.21.15.28:0/3213814899 wait complete. 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.064+0000 7fd814400640 1 Processor -- start 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.064+0000 7fd814400640 1 -- start start 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 0x7fd80c149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0d0890 con 0x7fd80c0cfea0 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3c60 con 0x7fd80c0b4af0 2024-09-10T10:53:36.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd814400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0d0a10 con 0x7fd80c156740 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52348/0 (socket says 172.21.15.28:52348) 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 -- 172.21.15.28:0/2077211986 learned_addr learned my addr 172.21.15.28:0/2077211986 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd812400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811a00640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 0x7fd80c149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 -- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 msgr2=0x7fd80c0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 -- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 msgr2=0x7fd80c149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 0x7fd80c149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811000640 1 -- 172.21.15.28:0/2077211986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0b18a0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.065+0000 7fd811a00640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 0x7fd80c149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd811000640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c430 tx=0x7fd7f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80074e0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80047e0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8004bd0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c14c1b0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.066+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c14c770 con 0x7fd80c0cfea0 2024-09-10T10:53:36.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.067+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c0cf2c0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.068+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7f8011070 con 0x7fd80c0cfea0 2024-09-10T10:53:36.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.069+0000 7fd809600640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.070+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 8626+0+0 (secure 0 0 0) 0x7fd7f808b5e0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.070+0000 7fd811a00640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.070+0000 7fd811a00640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd80c155270 tx=0x7fd800008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.074+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7f8059400 con 0x7fd80c0cfea0 2024-09-10T10:53:36.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.294+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd80c156bb0 con 0x7fd80c0cfea0 2024-09-10T10:53:36.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.697+0000 7fd809600640 1 -- 172.21.15.28:0/2077211986 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v246) ==== 162+0+0 (secure 0 0 0) 0x7fd7f8058d00 con 0x7fd80c0cfea0 2024-09-10T10:53:36.698 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 msgr2=0x7fd7ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd80c155270 tx=0x7fd800008040 comp rx=0 tx=0).stop 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 msgr2=0x7fd80c14fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fd7f800c430 tx=0x7fd7f800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 shutdown_connections 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd7ec06be30 0x7fd7ec06e2f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c156740 0x7fd80c0b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0cfea0 0x7fd80c14fd50 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 --2- 172.21.15.28:0/2077211986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd80c0b4af0 0x7fd80c149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 >> 172.21.15.28:0/2077211986 conn(0x7fd80c06bdf0 msgr2=0x7fd80c0ad020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 shutdown_connections 2024-09-10T10:53:36.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.700+0000 7fd814400640 1 -- 172.21.15.28:0/2077211986 wait complete. 2024-09-10T10:53:36.704 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:36.703+0000 7f0c03600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-10T10:53:36.707 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:36.707+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:36.707 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:36.707+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:36.721 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:36.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.878+0000 7fb08ea00640 1 Processor -- start 2024-09-10T10:53:36.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.878+0000 7fb08ea00640 1 -- start start 2024-09-10T10:53:36.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.879+0000 7fb08ea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb08815e1f0 0x7fb08806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb088146140 con 0x7fb088072de0 2024-09-10T10:53:36.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ea00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb0880cd740 con 0x7fb08815e1f0 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0880cda20 con 0x7fb088072320 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb07fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb088072de0 0x7fb08806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38576/0 (socket says 172.21.15.28:38576) 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb07fe00640 1 -- 172.21.15.28:0/3656621134 learned_addr learned my addr 172.21.15.28:0/3656621134 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ca00640 1 --2- 172.21.15.28:0/3656621134 >> v2:172.21.15.28:3300/0 conn(0x7fb08815e1f0 0x7fb08806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb087e00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ca00640 1 -- 172.21.15.28:0/3656621134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 msgr2=0x7fb0880cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ca00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ca00640 1 -- 172.21.15.28:0/3656621134 >> v1:172.21.15.28:6789/0 conn(0x7fb088072de0 legacy=0x7fb08806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb08ca00640 1 -- 172.21.15.28:0/3656621134 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0880cdd00 con 0x7fb08815e1f0 2024-09-10T10:53:36.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.880+0000 7fb087400640 1 -- 172.21.15.28:0/3656621134 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903425978 0 0) 0x7fb088146140 con 0x7fb088072de0 2024-09-10T10:53:36.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.881+0000 7fb08ca00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb08815e1f0 0x7fb08806e7c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb080009d90 tx=0x7fb08002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974e7de6671fbee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb087400640 1 -- 172.21.15.28:0/3656621134 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080031600 con 0x7fb08815e1f0 2024-09-10T10:53:36.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb087400640 1 -- 172.21.15.28:0/3656621134 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0800317a0 con 0x7fb08815e1f0 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb087400640 1 -- 172.21.15.28:0/3656621134 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080031b70 con 0x7fb08815e1f0 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 -- 172.21.15.28:0/3656621134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb08815e1f0 msgr2=0x7fb08806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb08815e1f0 0x7fb08806e7c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb080009d90 tx=0x7fb08002a5d0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 -- 172.21.15.28:0/3656621134 shutdown_connections 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb08815e1f0 0x7fb08806e7c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 --2- 172.21.15.28:0/3656621134 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 -- 172.21.15.28:0/3656621134 >> 172.21.15.28:0/3656621134 conn(0x7fb088067d70 msgr2=0x7fb08806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 -- 172.21.15.28:0/3656621134 shutdown_connections 2024-09-10T10:53:36.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.882+0000 7fb08ea00640 1 -- 172.21.15.28:0/3656621134 wait complete. 2024-09-10T10:53:36.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.884+0000 7fb08ea00640 1 Processor -- start 2024-09-10T10:53:36.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.884+0000 7fb08ea00640 1 -- start start 2024-09-10T10:53:36.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb08815e1f0 0x7fb0880c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cd520 con 0x7fb088072de0 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880cd3a0 con 0x7fb08815e1f0 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb08ea00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb08806ed00 con 0x7fb088072320 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb07fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52354/0 (socket says 172.21.15.28:52354) 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb07fe00640 1 -- 172.21.15.28:0/1347610459 learned_addr learned my addr 172.21.15.28:0/1347610459 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.885+0000 7fb087e00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb08ca00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb08815e1f0 0x7fb0880c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 -- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 msgr2=0x7fb0880c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 -- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb08815e1f0 msgr2=0x7fb0880c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:36.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb08815e1f0 0x7fb0880c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:36.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 -- 172.21.15.28:0/1347610459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0880ca0a0 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb07fe00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c410 tx=0x7fb07000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700074e0 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0700047e0 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070004bd0 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.886+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0880cbf80 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.887+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb088134ce0 con 0x7fb088072de0 2024-09-10T10:53:36.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.888+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb088072700 con 0x7fb088072de0 2024-09-10T10:53:36.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.889+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb070011070 con 0x7fb088072de0 2024-09-10T10:53:36.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.890+0000 7fb084c00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 0x7fb06406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:36.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.890+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fb07008b510 con 0x7fb088072de0 2024-09-10T10:53:36.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.890+0000 7fb087e00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 0x7fb06406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:36.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.891+0000 7fb087e00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 0x7fb06406e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb078009aa0 tx=0x7fb078009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:36.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:36.895+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb070058c90 con 0x7fb088072de0 2024-09-10T10:53:37.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.114+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0880b3cc0 con 0x7fb088072de0 2024-09-10T10:53:37.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.115+0000 7fb084c00640 1 -- 172.21.15.28:0/1347610459 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 247 v247) ==== 95+0+4392 (secure 0 0 0) 0x7fb07005cf90 con 0x7fb088072de0 2024-09-10T10:53:37.116 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:37.116 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":247,"btime":"2024-09-10T10:53:36:707644+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1706206889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1706206889},{"type":"v1","addr":"172.21.15.28:6834","nonce":1706206889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:36.707595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:37.116 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 247 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 msgr2=0x7fb06406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 0x7fb06406e2f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb078009aa0 tx=0x7fb078009190 comp rx=0 tx=0).stop 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 msgr2=0x7fb0880cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c410 tx=0x7fb07000c8d0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 shutdown_connections 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb06406be30 0x7fb06406e2f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb08815e1f0 0x7fb0880c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb088072de0 0x7fb0880cba40 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 --2- 172.21.15.28:0/1347610459 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb088072320 0x7fb0880c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 >> 172.21.15.28:0/1347610459 conn(0x7fb088067d70 msgr2=0x7fb088145330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:37.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 shutdown_connections 2024-09-10T10:53:37.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.117+0000 7fb08ea00640 1 -- 172.21.15.28:0/1347610459 wait complete. 2024-09-10T10:53:37.134 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:37.134 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.290+0000 7fcf45c00640 1 Processor -- start 2024-09-10T10:53:37.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.290+0000 7fcf45c00640 1 -- start start 2024-09-10T10:53:37.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf400b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf45c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcf400b64c0 0x7fcf4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf45c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf400d3690 con 0x7fcf4015e5f0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf45c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4006d530 con 0x7fcf400b64c0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf45c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf400d3970 con 0x7fcf400d30f0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf3ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcf4015e5f0 0x7fcf400d4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38590/0 (socket says 172.21.15.28:38590) 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.291+0000 7fcf3ea00640 1 -- 172.21.15.28:0/1976526101 learned_addr learned my addr 172.21.15.28:0/1976526101 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3f400640 1 --2- 172.21.15.28:0/1976526101 >> v2:172.21.15.138:3300/0 conn(0x7fcf400b64c0 0x7fcf4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3e000640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf400b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1253795355 0 0) 0x7fcf400d3690 con 0x7fcf4015e5f0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf24003520 con 0x7fcf4015e5f0 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3f400640 1 -- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 msgr2=0x7fcf400b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3f400640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf400b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3f400640 1 -- 172.21.15.28:0/1976526101 >> v1:172.21.15.28:6789/0 conn(0x7fcf4015e5f0 legacy=0x7fcf400d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2035586150 0 0) 0x7fcf24003520 con 0x7fcf4015e5f0 2024-09-10T10:53:37.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.292+0000 7fcf3f400640 1 -- 172.21.15.28:0/1976526101 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4006cee0 con 0x7fcf400b64c0 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.293+0000 7fcf3f400640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf400b64c0 0x7fcf4006d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009d90 tx=0x7fcf3802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31c990b7895271e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30001670 con 0x7fcf400b64c0 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf38007540 con 0x7fcf400b64c0 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf3d600640 1 -- 172.21.15.28:0/1976526101 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38007910 con 0x7fcf400b64c0 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 -- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf400b64c0 msgr2=0x7fcf4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf400b64c0 0x7fcf4006d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009d90 tx=0x7fcf3802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 -- 172.21.15.28:0/1976526101 shutdown_connections 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf400b64c0 0x7fcf4006d9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 --2- 172.21.15.28:0/1976526101 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf400b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 -- 172.21.15.28:0/1976526101 >> 172.21.15.28:0/1976526101 conn(0x7fcf4008bfc0 msgr2=0x7fcf400d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:37.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 -- 172.21.15.28:0/1976526101 shutdown_connections 2024-09-10T10:53:37.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.294+0000 7fcf45c00640 1 -- 172.21.15.28:0/1976526101 wait complete. 2024-09-10T10:53:37.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.296+0000 7fcf45c00640 1 Processor -- start 2024-09-10T10:53:37.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.296+0000 7fcf45c00640 1 -- start start 2024-09-10T10:53:37.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf401f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 0x7fcf400b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4006df00 con 0x7fcf400b64c0 2024-09-10T10:53:37.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4015e9d0 con 0x7fcf4015e5f0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf45c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4006e080 con 0x7fcf400d30f0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf3ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52358/0 (socket says 172.21.15.28:52358) 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf3ea00640 1 -- 172.21.15.28:0/2375507382 learned_addr learned my addr 172.21.15.28:0/2375507382 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf3f400640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 0x7fcf400b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.297+0000 7fcf3e000640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf401f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 -- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 msgr2=0x7fcf401f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf401f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 -- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 msgr2=0x7fcf400b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 0x7fcf400b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 -- 172.21.15.28:0/2375507382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf401f52d0 con 0x7fcf400b64c0 2024-09-10T10:53:37.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3e000640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf401f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3f400640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 0x7fcf400b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf3ea00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c700 tx=0x7fcf3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30017070 con 0x7fcf400b64c0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf30005ad0 con 0x7fcf400b64c0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf400b7680 con 0x7fcf400b64c0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30007070 con 0x7fcf400b64c0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.298+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf400b7c40 con 0x7fcf400b64c0 2024-09-10T10:53:37.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.299+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4006d9a0 con 0x7fcf400b64c0 2024-09-10T10:53:37.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.300+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf3001a070 con 0x7fcf400b64c0 2024-09-10T10:53:37.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.301+0000 7fcf36a00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 0x7fcf2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.301+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fcf3008af20 con 0x7fcf400b64c0 2024-09-10T10:53:37.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.302+0000 7fcf3e000640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 0x7fcf2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.303+0000 7fcf3e000640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 0x7fcf2006e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf401f89f0 tx=0x7fcf2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.305+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf30058590 con 0x7fcf400b64c0 2024-09-10T10:53:37.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.531+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf400b1e60 con 0x7fcf400b64c0 2024-09-10T10:53:37.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.532+0000 7fcf36a00640 1 -- 172.21.15.28:0/2375507382 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 247 v247) ==== 95+0+4392 (secure 0 0 0) 0x7fcf3005c890 con 0x7fcf400b64c0 2024-09-10T10:53:37.533 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:37.533 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":247,"btime":"2024-09-10T10:53:36:707644+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1706206889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1706206889},{"type":"v1","addr":"172.21.15.28:6834","nonce":1706206889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:36.707595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:37.533 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 247 2024-09-10T10:53:37.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 msgr2=0x7fcf2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 0x7fcf2006e2f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fcf401f89f0 tx=0x7fcf2c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:37.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 msgr2=0x7fcf401ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c700 tx=0x7fcf3000cbc0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 shutdown_connections 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcf2006be30 0x7fcf2006e2f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf4015e5f0 0x7fcf400b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf400d30f0 0x7fcf401f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 --2- 172.21.15.28:0/2375507382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf400b64c0 0x7fcf401ee4a0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 >> 172.21.15.28:0/2375507382 conn(0x7fcf4008bfc0 msgr2=0x7fcf400b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 shutdown_connections 2024-09-10T10:53:37.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.534+0000 7fcf45c00640 1 -- 172.21.15.28:0/2375507382 wait complete. 2024-09-10T10:53:37.551 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:37.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:37.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.707+0000 7fd19a800640 1 Processor -- start 2024-09-10T10:53:37.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.707+0000 7fd19a800640 1 -- start start 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.708+0000 7fd19a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd194161920 0x7fd1940d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.708+0000 7fd19a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d2aa0 0x7fd1940d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.708+0000 7fd19a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1940d1c20 con 0x7fd1940d4ac0 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.708+0000 7fd19a800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd1940cffd0 con 0x7fd194161920 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.708+0000 7fd19a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd194161d00 con 0x7fd1940d2aa0 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd194161920 0x7fd1940d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd194161920 0x7fd1940d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52364/0 (socket says 172.21.15.28:52364) 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 -- 172.21.15.28:0/109980520 learned_addr learned my addr 172.21.15.28:0/109980520 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:37.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199e00640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d2aa0 0x7fd1940d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 -- 172.21.15.28:0/109980520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d2aa0 msgr2=0x7fd1940d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:37.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d2aa0 0x7fd1940d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 -- 172.21.15.28:0/109980520 >> v1:172.21.15.138:6789/0 conn(0x7fd1940d4ac0 legacy=0x7fd1940d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.709+0000 7fd199400640 1 -- 172.21.15.28:0/109980520 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1940cf3c0 con 0x7fd194161920 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.710+0000 7fd199400640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940d16e0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd188009870 tx=0x7fd18802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92becb390d274b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.710+0000 7fd193e00640 1 -- 172.21.15.28:0/109980520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188037070 con 0x7fd194161920 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.710+0000 7fd193e00640 1 -- 172.21.15.28:0/109980520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd188032440 con 0x7fd194161920 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.710+0000 7fd193e00640 1 -- 172.21.15.28:0/109980520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188032810 con 0x7fd194161920 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 -- 172.21.15.28:0/109980520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 msgr2=0x7fd1940d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940d16e0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd188009870 tx=0x7fd18802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 -- 172.21.15.28:0/109980520 shutdown_connections 2024-09-10T10:53:37.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d2aa0 0x7fd1940d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 --2- 172.21.15.28:0/109980520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940d16e0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 -- 172.21.15.28:0/109980520 >> 172.21.15.28:0/109980520 conn(0x7fd19409d770 msgr2=0x7fd19408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:37.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 -- 172.21.15.28:0/109980520 shutdown_connections 2024-09-10T10:53:37.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.711+0000 7fd19a800640 1 -- 172.21.15.28:0/109980520 wait complete. 2024-09-10T10:53:37.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.713+0000 7fd19a800640 1 Processor -- start 2024-09-10T10:53:37.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.713+0000 7fd19a800640 1 -- start start 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1940d2aa0 0x7fd1941e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d4ac0 0x7fd1941585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cfc60 con 0x7fd194161920 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd1940cf980 con 0x7fd1940d2aa0 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd19a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd1940cf070 con 0x7fd1940d4ac0 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52368/0 (socket says 172.21.15.28:52368) 2024-09-10T10:53:37.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 -- 172.21.15.28:0/2178941469 learned_addr learned my addr 172.21.15.28:0/2178941469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199400640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1940d2aa0 0x7fd1941e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd198a00640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d4ac0 0x7fd1941585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 -- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d4ac0 msgr2=0x7fd1941585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d4ac0 0x7fd1941585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 -- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1940d2aa0 msgr2=0x7fd1941e89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1940d2aa0 0x7fd1941e89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.714+0000 7fd199e00640 1 -- 172.21.15.28:0/2178941469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1941e6e70 con 0x7fd194161920 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd199e00640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd18c007e60 tx=0x7fd18c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c014420 con 0x7fd194161920 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18c019070 con 0x7fd194161920 2024-09-10T10:53:37.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c014820 con 0x7fd194161920 2024-09-10T10:53:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1941e7150 con 0x7fd194161920 2024-09-10T10:53:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.715+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1941e7710 con 0x7fd194161920 2024-09-10T10:53:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.716+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1940500d0 con 0x7fd194161920 2024-09-10T10:53:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.717+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd18c0149c0 con 0x7fd194161920 2024-09-10T10:53:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.718+0000 7fd191600640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 0x7fd17406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:37.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.718+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fd18c08c1a0 con 0x7fd194161920 2024-09-10T10:53:37.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.718+0000 7fd199400640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 0x7fd17406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:37.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.719+0000 7fd199400640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 0x7fd17406e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd188034000 tx=0x7fd18802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:37.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.722+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd1940500d0 con 0x7fd194161920 2024-09-10T10:53:37.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.943+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1940d16e0 con 0x7fd194161920 2024-09-10T10:53:37.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.944+0000 7fd191600640 1 -- 172.21.15.28:0/2178941469 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 247 v247) ==== 95+0+4392 (secure 0 0 0) 0x7fd18c059890 con 0x7fd194161920 2024-09-10T10:53:37.945 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:37.945 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":247,"btime":"2024-09-10T10:53:36:707644+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/1706206889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":1706206889},{"type":"v1","addr":"172.21.15.28:6834","nonce":1706206889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":241}],"filesystems":[{"mdsmap":{"epoch":247,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:36.707595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:37.945 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 247 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 msgr2=0x7fd17406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 0x7fd17406e2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd188034000 tx=0x7fd18802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 msgr2=0x7fd1940b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fd18c007e60 tx=0x7fd18c00e4b0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 shutdown_connections 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd17406be30 0x7fd17406e2f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd194161920 0x7fd1940b54c0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd1940d4ac0 0x7fd1941585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 --2- 172.21.15.28:0/2178941469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1940d2aa0 0x7fd1941e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 >> 172.21.15.28:0/2178941469 conn(0x7fd19409d770 msgr2=0x7fd1940cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:37.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.946+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 shutdown_connections 2024-09-10T10:53:37.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:37.947+0000 7fd19a800640 1 -- 172.21.15.28:0/2178941469 wait complete. 2024-09-10T10:53:37.963 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds_snaptable' led to DAMAGED state 2024-09-10T10:53:37.963 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate mds0_inotable' 2024-09-10T10:53:37.963 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:37.964 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:37.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:37.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:37.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:37.999 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:38.073 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:38.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:38.074 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.270+0000 7fc433200640 1 Processor -- start 2024-09-10T10:53:38.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.270+0000 7fc433200640 1 -- start start 2024-09-10T10:53:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 0x7fc42c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 0x7fc42c06f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc42c0728a0 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc42c0cdbe0 con 0x7fc42c072d30 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc433200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0cd900 con 0x7fc42c15b300 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52384/0 (socket says 172.21.15.28:52384) 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 -- 172.21.15.28:0/39264817 learned_addr learned my addr 172.21.15.28:0/39264817 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc42be00640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 0x7fc42c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc431200640 1 --2- 172.21.15.28:0/39264817 >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 0x7fc42c06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 -- 172.21.15.28:0/39264817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 msgr2=0x7fc42c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 0x7fc42c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 -- 172.21.15.28:0/39264817 >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 msgr2=0x7fc42c06f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 --2- 172.21.15.28:0/39264817 >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 0x7fc42c06f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc423400640 1 -- 172.21.15.28:0/39264817 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c072250 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc42be00640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 0x7fc42c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:38.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.272+0000 7fc431200640 1 --2- 172.21.15.28:0/39264817 >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 0x7fc42c06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:38.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.273+0000 7fc423400640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc42c072040 tx=0x7fc41402c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5c189abd39f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc42b400640 1 -- 172.21.15.28:0/39264817 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414038070 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc42b400640 1 -- 172.21.15.28:0/39264817 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414031710 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc42b400640 1 -- 172.21.15.28:0/39264817 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414031ae0 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 -- 172.21.15.28:0/39264817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c0cd3f0 msgr2=0x7fc42c06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc42c072040 tx=0x7fc41402c3e0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 -- 172.21.15.28:0/39264817 shutdown_connections 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 --2- 172.21.15.28:0/39264817 >> v2:172.21.15.138:3300/0 conn(0x7fc42c072d30 0x7fc42c06f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c0cd3f0 0x7fc42c06f1c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 --2- 172.21.15.28:0/39264817 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c15b300 0x7fc42c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 -- 172.21.15.28:0/39264817 >> 172.21.15.28:0/39264817 conn(0x7fc42c067d70 msgr2=0x7fc42c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 -- 172.21.15.28:0/39264817 shutdown_connections 2024-09-10T10:53:38.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.274+0000 7fc433200640 1 -- 172.21.15.28:0/39264817 wait complete. 2024-09-10T10:53:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.276+0000 7fc433200640 1 Processor -- start 2024-09-10T10:53:38.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.276+0000 7fc433200640 1 -- start start 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c0cd3f0 0x7fc42c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc42c15b300 0x7fc42c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0d0160 con 0x7fc42c072d30 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc42c0cffe0 con 0x7fc42c15b300 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc433200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc42c0b3c40 con 0x7fc42c0cd3f0 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc42be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc42be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52394/0 (socket says 172.21.15.28:52394) 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.277+0000 7fc42be00640 1 -- 172.21.15.28:0/1884083677 learned_addr learned my addr 172.21.15.28:0/1884083677 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc423400640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c0cd3f0 0x7fc42c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc431200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc42c15b300 0x7fc42c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 -- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c0cd3f0 msgr2=0x7fc42c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c0cd3f0 0x7fc42c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 -- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc42c15b300 msgr2=0x7fc42c0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc42c15b300 0x7fc42c0cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 -- 172.21.15.28:0/1884083677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42c0ca0d0 con 0x7fc42c072d30 2024-09-10T10:53:38.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc42be00640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c7f0 tx=0x7fc41c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c015070 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c007820 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c007bd0 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42c0c9550 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.278+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc42c0c9b10 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.280+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c15b780 con 0x7fc42c072d30 2024-09-10T10:53:38.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.280+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc41c018070 con 0x7fc42c072d30 2024-09-10T10:53:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.281+0000 7fc428c00640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 0x7fc40806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.281+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fc41c08bbf0 con 0x7fc42c072d30 2024-09-10T10:53:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.282+0000 7fc423400640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 0x7fc40806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.283+0000 7fc423400640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 0x7fc40806e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc414035000 tx=0x7fc414008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.285+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc41c059930 con 0x7fc42c072d30 2024-09-10T10:53:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.506+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc42c1585c0 con 0x7fc42c072d30 2024-09-10T10:53:38.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.709+0000 7fc428c00640 1 -- 172.21.15.28:0/1884083677 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v248) ==== 161+0+0 (secure 0 0 0) 0x7fc41c059230 con 0x7fc42c072d30 2024-09-10T10:53:38.710 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:38.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 msgr2=0x7fc40806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 0x7fc40806e2f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc414035000 tx=0x7fc414008030 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 msgr2=0x7fc42c158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00c7f0 tx=0x7fc41c00ccb0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 shutdown_connections 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc40806be30 0x7fc40806e2f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc42c15b300 0x7fc42c0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc42c0cd3f0 0x7fc42c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 --2- 172.21.15.28:0/1884083677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc42c072d30 0x7fc42c158080 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 >> 172.21.15.28:0/1884083677 conn(0x7fc42c067d70 msgr2=0x7fc42c15d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 shutdown_connections 2024-09-10T10:53:38.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.711+0000 7fc433200640 1 -- 172.21.15.28:0/1884083677 wait complete. 2024-09-10T10:53:38.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.883+0000 7fad84e00640 1 Processor -- start 2024-09-10T10:53:38.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.884+0000 7fad84e00640 1 -- start start 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad84e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad800cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad84e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800b49a0 con 0x7fad800720b0 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad84e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad800afa20 con 0x7fad8006f6f0 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad84e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad800afce0 con 0x7fad80162ad0 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad7de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fad800720b0 0x7fad80072490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38602/0 (socket says 172.21.15.28:38602) 2024-09-10T10:53:38.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad7de00640 1 -- 172.21.15.28:0/3600737602 learned_addr learned my addr 172.21.15.28:0/3600737602 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 955571764 0 0) 0x7fad800b49a0 con 0x7fad800720b0 2024-09-10T10:53:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad64003520 con 0x7fad800720b0 2024-09-10T10:53:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.885+0000 7fad75400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad800cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 268800719 0 0) 0x7fad64003520 con 0x7fad800720b0 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 msgr2=0x7fad800cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad800cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 >> v1:172.21.15.138:6789/0 conn(0x7fad8006f6f0 legacy=0x7fad800b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad800b34d0 con 0x7fad800720b0 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fad70001670 con 0x7fad800720b0 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 >> v1:172.21.15.28:6789/0 conn(0x7fad800720b0 legacy=0x7fad80072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad640039e0 0x7fad64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad64005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad800afce0 con 0x7fad64004800 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad6400bb80 con 0x7fad640039e0 2024-09-10T10:53:38.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad6400bd80 con 0x7fad80162ad0 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad75400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7de00640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad640039e0 0x7fad64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad7e800640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad64005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.886+0000 7fad75400640 1 -- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 msgr2=0x7fad64005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.887+0000 7fad75400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad64005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.887+0000 7fad75400640 1 -- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad640039e0 msgr2=0x7fad64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.887+0000 7fad75400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad640039e0 0x7fad64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.887+0000 7fad75400640 1 -- 172.21.15.28:0/3600737602 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad64003520 con 0x7fad64004800 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.887+0000 7fad7e800640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad80162ad0 0x7fad64005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:38.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad75400640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad64004c80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ba30 tx=0x7fad64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e12d856168986 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68030070 con 0x7fad64004800 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6802ade0 con 0x7fad64004800 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad7d400640 1 -- 172.21.15.28:0/3600737602 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6800b600 con 0x7fad64004800 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad84e00640 1 -- 172.21.15.28:0/3600737602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 msgr2=0x7fad64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad84e00640 1 --2- 172.21.15.28:0/3600737602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad64004c80 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ba30 tx=0x7fad64005830 comp rx=0 tx=0).stop 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad7e800640 1 -- 172.21.15.28:0/3600737602 reap_dead start 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad84e00640 1 -- 172.21.15.28:0/3600737602 shutdown_connections 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad84e00640 1 -- 172.21.15.28:0/3600737602 >> 172.21.15.28:0/3600737602 conn(0x7fad80067d70 msgr2=0x7fad8006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.888+0000 7fad84e00640 1 -- 172.21.15.28:0/3600737602 shutdown_connections 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.889+0000 7fad84e00640 1 -- 172.21.15.28:0/3600737602 wait complete. 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.890+0000 7fad84e00640 1 Processor -- start 2024-09-10T10:53:38.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.890+0000 7fad84e00640 1 -- start start 2024-09-10T10:53:38.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 0x7fad8015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad800a9780 0x7fad800a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad800b3d70 con 0x7fad64004800 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad800b3a90 con 0x7fad801599d0 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.891+0000 7fad84e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad800cfca0 con 0x7fad800a9780 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52414/0 (socket says 172.21.15.28:52414) 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 -- 172.21.15.28:0/688283131 learned_addr learned my addr 172.21.15.28:0/688283131 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad75400640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 0x7fad8015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7e800640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad800a9780 0x7fad800a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 -- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad800a9780 msgr2=0x7fad800a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad800a9780 0x7fad800a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 -- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 msgr2=0x7fad8015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 0x7fad8015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 -- 172.21.15.28:0/688283131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8015b950 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad75400640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 0x7fad8015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad7de00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fad70007290 tx=0x7fad70007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000f960 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70018070 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.892+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000fd60 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.893+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad8015bc30 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.893+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8015c1f0 con 0x7fad64004800 2024-09-10T10:53:38.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.894+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad80072490 con 0x7fad64004800 2024-09-10T10:53:38.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.894+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad7001d460 con 0x7fad64004800 2024-09-10T10:53:38.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.895+0000 7fad76a00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 0x7fad5c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:38.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.895+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fad7008bac0 con 0x7fad64004800 2024-09-10T10:53:38.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.896+0000 7fad75400640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 0x7fad5c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:38.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.897+0000 7fad75400640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 0x7fad5c06e320 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ba00 tx=0x7fad68030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:38.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:38.899+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad70059890 con 0x7fad64004800 2024-09-10T10:53:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.152+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fad8015d840 con 0x7fad64004800 2024-09-10T10:53:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.716+0000 7fad76a00640 1 -- 172.21.15.28:0/688283131 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v249) ==== 101+0+0 (secure 0 0 0) 0x7fad70059190 con 0x7fad64004800 2024-09-10T10:53:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:39.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 msgr2=0x7fad5c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 0x7fad5c06e320 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ba00 tx=0x7fad68030000 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 msgr2=0x7fad801591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fad70007290 tx=0x7fad70007690 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 shutdown_connections 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad5c06be60 0x7fad5c06e320 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad800a9780 0x7fad800a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad801599d0 0x7fad8015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 --2- 172.21.15.28:0/688283131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad64004800 0x7fad801591d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 >> 172.21.15.28:0/688283131 conn(0x7fad80067d70 msgr2=0x7fad801652a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 shutdown_connections 2024-09-10T10:53:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.719+0000 7fad84e00640 1 -- 172.21.15.28:0/688283131 wait complete. 2024-09-10T10:53:39.738 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:39.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 Processor -- start 2024-09-10T10:53:39.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 -- start start 2024-09-10T10:53:39.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de18d3e0 0x5646de104490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5646de102530 con 0x5646de1018e0 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646de05fb30 con 0x5646de18d3e0 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.814+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5646de18fee0 con 0x5646de1909c0 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52420/0 (socket says 172.21.15.28:52420) 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 -- 172.21.15.28:0/112840574 learned_addr learned my addr 172.21.15.28:0/112840574 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924e00640 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 -- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 msgr2=0x5646de18a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 -- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de18d3e0 msgr2=0x5646de104490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:39.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de18d3e0 0x5646de104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 -- 172.21.15.28:0/112840574 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646de1941c0 con 0x5646de1018e0 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924e00640 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.815+0000 7ff924400640 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff91400a7a0 tx=0x7ff91400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccddae98a8a2690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff922600640 1 -- 172.21.15.28:0/112840574 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914019070 con 0x5646de1018e0 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff922600640 1 -- 172.21.15.28:0/112840574 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff914007600 con 0x5646de1018e0 2024-09-10T10:53:39.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff922600640 1 -- 172.21.15.28:0/112840574 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140079d0 con 0x5646de1018e0 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 -- 172.21.15.28:0/112840574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 msgr2=0x5646de18f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff91400a7a0 tx=0x7ff91400a890 comp rx=0 tx=0).stop 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 -- 172.21.15.28:0/112840574 shutdown_connections 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de18d3e0 0x5646de104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/112840574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de1018e0 0x5646de18f620 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 -- 172.21.15.28:0/112840574 >> 172.21.15.28:0/112840574 conn(0x5646de1060a0 msgr2=0x5646de101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 -- 172.21.15.28:0/112840574 shutdown_connections 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.816+0000 7ff9272d2f40 1 -- 172.21.15.28:0/112840574 wait complete. 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.817+0000 7ff9272d2f40 1 Processor -- start 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.817+0000 7ff9272d2f40 1 -- start start 2024-09-10T10:53:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de1018e0 0x5646de21e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5646de193ee0 con 0x5646de18d3e0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5646de0b71a0 con 0x5646de1018e0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff9272d2f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5646de0b7440 con 0x5646de1909c0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52426/0 (socket says 172.21.15.28:52426) 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 learned_addr learned my addr 172.21.15.28:0/1637954171 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de1018e0 0x5646de21e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 msgr2=0x5646de18e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de1018e0 msgr2=0x5646de21e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5646de1018e0 0x5646de21e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.818+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646de185c70 con 0x5646de18d3e0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5646de1909c0 0x5646de18e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff910025d20 tx=0x7ff9100058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff921200640 1 -- 172.21.15.28:0/1637954171 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910055070 con 0x5646de18d3e0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff921200640 1 -- 172.21.15.28:0/1637954171 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910051730 con 0x5646de18d3e0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5646de1826d0 con 0x5646de18d3e0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff921200640 1 -- 172.21.15.28:0/1637954171 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910051ae0 con 0x5646de18d3e0 2024-09-10T10:53:39.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.819+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646de1823f0 con 0x5646de18d3e0 2024-09-10T10:53:39.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.821+0000 7ff921200640 1 -- 172.21.15.28:0/1637954171 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff910058070 con 0x5646de18d3e0 2024-09-10T10:53:39.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.822+0000 7ff921200640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 0x7ff90406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.822+0000 7ff921200640 1 -- 172.21.15.28:0/1637954171 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7ff9100cdfe0 con 0x5646de18d3e0 2024-09-10T10:53:39.824 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:39.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.822+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 0x5646de17efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.822+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de17f500 con 0x5646de184a70 2024-09-10T10:53:39.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.822+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 0x7ff90406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.823+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 0x5646de17efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.823+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 0x7ff90406ccd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff914017040 tx=0x7ff91400c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.824+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 0x5646de17efc0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.826+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v128'846 uv820 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.826+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de17f8a0 con 0x5646de184a70 2024-09-10T10:53:39.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.828+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v128'847 uv847 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.828+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2326b0 con 0x5646de184a70 2024-09-10T10:53:39.830 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:39.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.830+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v128'848 uv848 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.830+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233a60 con 0x5646de184a70 2024-09-10T10:53:39.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.831+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v128'849 uv849 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.832+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233e00 con 0x5646de184a70 2024-09-10T10:53:39.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.833+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v128'850 uv850 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.833+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de232a50 con 0x5646de184a70 2024-09-10T10:53:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.835+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v128'851 uv851 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.835+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de232e70 con 0x5646de184a70 2024-09-10T10:53:39.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.837+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v128'852 uv826 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.837+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de233260 con 0x5646de184a70 2024-09-10T10:53:39.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.838+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v128'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.838+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233260 con 0x5646de184a70 2024-09-10T10:53:39.841 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:39.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.840+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v128'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.841+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.842+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v128'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.842+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.844+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v128'856 uv856 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.844+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.846+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v128'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.846+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.847+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v128'858 uv833 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.847+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.849+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v128'859 uv859 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.849+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.852 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.851+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v128'860 uv860 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.851+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.852+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v128'861 uv861 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.852+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.854+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v128'862 uv862 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.854+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.856+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v128'863 uv863 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.856+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.858+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v128'864 uv864 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.858+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.859+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v128'865 uv839 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.859+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.861+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v128'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.861+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.863 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:39.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.863+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v128'867 uv867 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.863+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.864+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v128'868 uv868 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.864+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.866+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v128'869 uv869 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.866+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.868+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v128'870 uv870 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.868+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.870+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v128'871 uv845 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.870+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.871+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v128'872 uv872 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.871+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.873 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.872+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v128'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.873+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.874+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v128'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.874+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.876+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v128'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.876+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de233210 con 0x5646de184a70 2024-09-10T10:53:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.877+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v128'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de184a70 2024-09-10T10:53:39.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.877+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 0x5646de2366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.877+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de236c00 con 0x5646de233210 2024-09-10T10:53:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.878+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 0x5646de2366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.878+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 0x5646de2366c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.881+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v128'617 uv600 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.881+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de236fa0 con 0x5646de233210 2024-09-10T10:53:39.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.882+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v128'618 uv618 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.882+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de237340 con 0x5646de233210 2024-09-10T10:53:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.884+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v128'619 uv619 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.885 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.884+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2376e0 con 0x5646de233210 2024-09-10T10:53:39.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.885+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v128'620 uv620 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.885+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de237a80 con 0x5646de233210 2024-09-10T10:53:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.887+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v128'621 uv621 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.887+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de237e20 con 0x5646de233210 2024-09-10T10:53:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.889+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v128'622 uv605 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.889+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.890+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v128'623 uv623 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.890+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.891+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v128'624 uv624 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.892 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:39.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.891+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.893+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v128'625 uv625 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.893+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.894+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v128'626 uv626 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.894+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.895+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v128'627 uv610 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.895+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.897+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v128'628 uv628 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.897+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.898+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v128'629 uv629 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.899 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:39.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.898+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23c8b0 con 0x5646de233210 2024-09-10T10:53:39.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.900+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v128'630 uv630 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.900+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23c8b0 con 0x5646de233210 2024-09-10T10:53:39.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.901+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v128'631 uv631 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.901+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.903+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v128'632 uv616 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.903+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.905+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v128'633 uv633 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.905+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.907 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:39.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.906+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v128'634 uv634 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.906+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2391d0 con 0x5646de233210 2024-09-10T10:53:39.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.907+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v128'635 uv635 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.907+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2391d0 con 0x5646de233210 2024-09-10T10:53:39.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.909+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v128'636 uv636 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.909+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2381c0 con 0x5646de233210 2024-09-10T10:53:39.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.910+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v128'637 uv637 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de233210 2024-09-10T10:53:39.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.910+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5646de238840 0x5646de23ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.910+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23b200 con 0x5646de238840 2024-09-10T10:53:39.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.911+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5646de238840 0x5646de23ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.911+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5646de238840 0x5646de23ac30 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.912+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v128'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.913+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23b6a0 con 0x5646de238840 2024-09-10T10:53:39.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.914+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v128'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.914+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bb50 con 0x5646de238840 2024-09-10T10:53:39.916 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:39.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.915+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v128'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.915+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23d400 con 0x5646de238840 2024-09-10T10:53:39.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.916+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v128'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.916+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23d7a0 con 0x5646de238840 2024-09-10T10:53:39.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.917+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v128'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.917+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23c3f0 con 0x5646de238840 2024-09-10T10:53:39.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.918+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.919+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5646de23ccf0 0x5646de23fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.919+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de240090 con 0x5646de23ccf0 2024-09-10T10:53:39.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.919+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5646de23ccf0 0x5646de23fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.919+0000 7ff924e00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5646de23ccf0 0x5646de23fb50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.920+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v128'120 uv119 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.920+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de240090 con 0x5646de23ccf0 2024-09-10T10:53:39.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.922+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v128'121 uv121 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.922+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de240090 con 0x5646de23ccf0 2024-09-10T10:53:39.923 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.923+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v128'122 uv122 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.923+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.924+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v128'123 uv123 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.924+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.925+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v128'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.925+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.926+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v128'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.926+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.927+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v128'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:39.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.927+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2414b0 con 0x5646de238840 2024-09-10T10:53:39.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.928+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v128'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.929+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2414b0 con 0x5646de238840 2024-09-10T10:53:39.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.930+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v128'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.930+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.931+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.931+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.932+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v128'305 uv299 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.932+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.933+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v128'306 uv306 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.934+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.936 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:39.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.935+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v128'307 uv307 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.935+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2414b0 con 0x5646de23ccf0 2024-09-10T10:53:39.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.937+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v128'308 uv308 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.937+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2414b0 con 0x5646de23ccf0 2024-09-10T10:53:39.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.938+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v128'309 uv309 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.938+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.939+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v128'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.939+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.941+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v128'311 uv304 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.941+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.942+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v128'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.942+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.944 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:39.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.943+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v128'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.943+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.945+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v128'314 uv314 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.945+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de23ccf0 2024-09-10T10:53:39.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.946+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v128'315 uv315 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.946+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.947+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v128'138 uv137 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.947+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.949+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v128'139 uv139 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.949+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:39.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.950+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v128'140 uv140 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.950+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.951+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v128'141 uv141 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.951+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2404a0 con 0x5646de238840 2024-09-10T10:53:39.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.952+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v128'142 uv142 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de238840 2024-09-10T10:53:39.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.953+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646de240b20 0x5646de242f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.953+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243450 con 0x5646de240b20 2024-09-10T10:53:39.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.953+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646de240b20 0x5646de242f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.953+0000 7ff924400640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646de240b20 0x5646de242f10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.955+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v128'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.955+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.956+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v128'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.956+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.959 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:39.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.958+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v128'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.958+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de244830 con 0x5646de240b20 2024-09-10T10:53:39.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.959+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v128'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.959+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de244830 con 0x5646de240b20 2024-09-10T10:53:39.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.960+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v128'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.961+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.962+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.962+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.964+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v128'273 uv267 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.964+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.965+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v128'274 uv274 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.965+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243820 con 0x5646de240b20 2024-09-10T10:53:39.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.966+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v128'275 uv275 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.967 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.966+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243c10 con 0x5646de240b20 2024-09-10T10:53:39.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.968+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v128'276 uv276 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.968+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de243c10 con 0x5646de240b20 2024-09-10T10:53:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.969+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v128'277 uv277 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.969+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2438c0 con 0x5646de240b20 2024-09-10T10:53:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.970+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v128'278 uv272 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.971+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de2438c0 con 0x5646de240b20 2024-09-10T10:53:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.972+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v128'279 uv279 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.972+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2438c0 con 0x5646de240b20 2024-09-10T10:53:39.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.973+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v128'280 uv280 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.974 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:39.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.973+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de2438c0 con 0x5646de240b20 2024-09-10T10:53:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.975+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v128'281 uv281 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.975+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de240b20 2024-09-10T10:53:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.976+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v128'282 uv282 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.976+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.977+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v128'120 uv119 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.978+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.979+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v128'121 uv121 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.979+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.981 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:39.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.980+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v128'122 uv122 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.980+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.982+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v128'123 uv123 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.982+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.983+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v128'165 uv164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.983+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.984+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v128'166 uv166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.984+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.986 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:39.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.985+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v128'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.985+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bff0 con 0x5646de23ccf0 2024-09-10T10:53:39.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.986+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v128'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.986+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de23ccf0 2024-09-10T10:53:39.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.988+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v128'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.988+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de23ccf0 2024-09-10T10:53:39.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.989+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v128'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91c0016a0 con 0x5646de23ccf0 2024-09-10T10:53:39.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.989+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5646de244670 0x5646de246d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:39.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.989+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de244670 2024-09-10T10:53:39.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.989+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5646de244670 0x5646de246d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:39.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.990+0000 7ff923a00640 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5646de244670 0x5646de246d80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:39.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.991+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v128'140 uv139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de244670 2024-09-10T10:53:39.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.991+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de244670 2024-09-10T10:53:39.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.992+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v128'141 uv141 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de244670 2024-09-10T10:53:39.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.992+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de244670 2024-09-10T10:53:39.994 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:39.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.993+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v128'142 uv142 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de244670 2024-09-10T10:53:39.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.993+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de244670 2024-09-10T10:53:39.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.994+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v128'143 uv143 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de244670 2024-09-10T10:53:39.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.995+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de244670 2024-09-10T10:53:39.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.996+0000 7ff923a00640 1 -- 172.21.15.28:0/1637954171 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v128'144 uv144 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff91006a020 con 0x5646de244670 2024-09-10T10:53:39.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.996+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.997+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v128'339 uv331 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.997+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.998+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v128'340 uv340 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:39.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:39.998+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.000 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.000+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v128'341 uv341 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.000+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.001+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v128'342 uv342 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.001+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.002+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v128'343 uv343 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.002+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.003+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v128'344 uv338 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.003+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.004+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v128'345 uv345 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.004+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.005+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v128'346 uv346 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.006 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.005+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.007+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v128'347 uv347 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.007+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.008+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v128'348 uv348 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.008+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de23bfd0 con 0x5646de240b20 2024-09-10T10:53:40.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.009+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v128'349 uv349 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.009+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5646de247770 con 0x5646de240b20 2024-09-10T10:53:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff924400640 1 -- 172.21.15.28:0/1637954171 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v128'350 uv350 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff91401c040 con 0x5646de240b20 2024-09-10T10:53:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5646de238840 msgr2=0x5646de23ac30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5646de238840 0x5646de23ac30 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646de240b20 msgr2=0x5646de242f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5646de240b20 0x5646de242f10 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5646de244670 msgr2=0x5646de246d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5646de244670 0x5646de246d80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5646de23ccf0 msgr2=0x5646de23fb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5646de23ccf0 0x5646de23fb50 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 msgr2=0x5646de2366c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 reap_dead start 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 0x5646de2366c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.010+0000 7ff924e00640 1 -- 172.21.15.28:0/1637954171 reap_dead start 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 msgr2=0x5646de17efc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 0x5646de17efc0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 msgr2=0x7ff90406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 0x7ff90406ccd0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff914017040 tx=0x7ff91400c000 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 msgr2=0x5646de18e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7ff910025d20 tx=0x7ff9100058b0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 shutdown_connections 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5646de233210 0x5646de2366c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5646de184a70 0x5646de17efc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff90406a810 0x7ff90406ccd0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 --2- 172.21.15.28:0/1637954171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5646de18d3e0 0x5646de18e2f0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 >> 172.21.15.28:0/1637954171 conn(0x5646de1060a0 msgr2=0x5646de18d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 shutdown_connections 2024-09-10T10:53:40.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.011+0000 7ff9272d2f40 1 -- 172.21.15.28:0/1637954171 wait complete. 2024-09-10T10:53:40.016 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate mds0_inotable 0 2024-09-10T10:53:40.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.134+0000 7f7b117f8f40 1 Processor -- start 2024-09-10T10:53:40.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.134+0000 7f7b117f8f40 1 -- start start 2024-09-10T10:53:40.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568950e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x556895132530 con 0x5568950e6ef0 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55689508fb30 con 0x556895118f60 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568951bfee0 con 0x7f7af40030f0 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.135+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52452/0 (socket says 172.21.15.28:52452) 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 -- 172.21.15.28:0/1680023848 learned_addr learned my addr 172.21.15.28:0/1680023848 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0ea00640 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0e000640 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568950e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 -- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 msgr2=0x5568951b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 -- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 msgr2=0x5568950e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568950e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 -- 172.21.15.28:0/1680023848 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55689511a1c0 con 0x5568950e6ef0 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0f400640 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x556895132460 tx=0x7f7b08005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ffdfb89976e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0cc00640 1 -- 172.21.15.28:0/1680023848 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08013710 con 0x5568950e6ef0 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0cc00640 1 -- 172.21.15.28:0/1680023848 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b080138d0 con 0x5568950e6ef0 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.136+0000 7f7b0cc00640 1 -- 172.21.15.28:0/1680023848 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08013cc0 con 0x5568950e6ef0 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 -- 172.21.15.28:0/1680023848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 msgr2=0x5568951bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x556895132460 tx=0x7f7b08005d80 comp rx=0 tx=0).stop 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 -- 172.21.15.28:0/1680023848 shutdown_connections 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x5568951bb440 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568950e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/1680023848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 -- 172.21.15.28:0/1680023848 >> 172.21.15.28:0/1680023848 conn(0x5568951360a0 msgr2=0x5568950e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 -- 172.21.15.28:0/1680023848 shutdown_connections 2024-09-10T10:53:40.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.137+0000 7f7b117f8f40 1 -- 172.21.15.28:0/1680023848 wait complete. 2024-09-10T10:53:40.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.138+0000 7f7b117f8f40 1 Processor -- start 2024-09-10T10:53:40.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.138+0000 7f7b117f8f40 1 -- start start 2024-09-10T10:53:40.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x556895116b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568951b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568951bf0c0 con 0x5568950e6ef0 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556895119f70 con 0x556895118f60 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b117f8f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568951bf310 con 0x7f7af40030f0 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b0ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x556895116b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b0ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x556895116b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52464/0 (socket says 172.21.15.28:52464) 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b0ea00640 1 -- 172.21.15.28:0/398042367 learned_addr learned my addr 172.21.15.28:0/398042367 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b0f400640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.139+0000 7f7b0e000640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568951b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 -- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 msgr2=0x5568951b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7af40030f0 0x5568951b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 -- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 msgr2=0x5568951b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895118f60 0x5568951b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 -- 172.21.15.28:0/398042367 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568951b5340 con 0x5568950e6ef0 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b0ea00640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x556895116b60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f7b00008a70 tx=0x7f7b00008b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b07400640 1 -- 172.21.15.28:0/398042367 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00013c90 con 0x5568950e6ef0 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b07400640 1 -- 172.21.15.28:0/398042367 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b00013e30 con 0x5568950e6ef0 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b07400640 1 -- 172.21.15.28:0/398042367 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00016660 con 0x5568950e6ef0 2024-09-10T10:53:40.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55689511a760 con 0x5568950e6ef0 2024-09-10T10:53:40.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.140+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568951bbe60 con 0x5568950e6ef0 2024-09-10T10:53:40.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.142+0000 7f7b07400640 1 -- 172.21.15.28:0/398042367 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b00016800 con 0x5568950e6ef0 2024-09-10T10:53:40.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.143+0000 7f7b07400640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7af006a810 0x7f7af006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.143+0000 7f7b07400640 1 -- 172.21.15.28:0/398042367 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7f7b0008b500 con 0x5568950e6ef0 2024-09-10T10:53:40.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.143+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568952564b0 0x5568951b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.143+0000 7f7b0e000640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7af006a810 0x7f7af006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.143+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) -- 0x5568951b8010 con 0x5568952564b0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.144+0000 7f7b0f400640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568952564b0 0x5568951b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.144+0000 7f7b0e000640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7af006a810 0x7f7af006ccd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7afc056000 tx=0x7f7afc050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.144+0000 7f7b0f400640 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568952564b0 0x5568951b7ad0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b0f400640 1 -- 172.21.15.28:0/398042367 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 mds0_inotable [truncate 0] v128'283 uv283 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f7b0800e040 con 0x5568952564b0 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568952564b0 msgr2=0x5568951b7ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568952564b0 0x5568951b7ad0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7af006a810 msgr2=0x7f7af006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7af006a810 0x7f7af006ccd0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7afc056000 tx=0x7f7afc050000 comp rx=0 tx=0).stop 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 msgr2=0x556895116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 --2- 172.21.15.28:0/398042367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568950e6ef0 0x556895116b60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f7b00008a70 tx=0x7f7b00008b60 comp rx=0 tx=0).stop 2024-09-10T10:53:40.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b0f400640 1 -- 172.21.15.28:0/398042367 reap_dead start 2024-09-10T10:53:40.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 shutdown_connections 2024-09-10T10:53:40.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 >> 172.21.15.28:0/398042367 conn(0x5568951360a0 msgr2=0x5568950e8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 shutdown_connections 2024-09-10T10:53:40.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.146+0000 7f7b117f8f40 1 -- 172.21.15.28:0/398042367 wait complete. 2024-09-10T10:53:40.151 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:40.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.312+0000 7fad3cc00640 1 Processor -- start 2024-09-10T10:53:40.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.312+0000 7fad3cc00640 1 -- start start 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad3cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad3815e7f0 0x7fad380b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380af210 0x7fad38070a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad3cc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3806cdc0 con 0x7fad380b7250 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad3cc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fad3806d0a0 con 0x7fad3815e7f0 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad3cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad3806d380 con 0x7fad380af210 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad35e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad3815e7f0 0x7fad380b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad35e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad3815e7f0 0x7fad380b6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52474/0 (socket says 172.21.15.28:52474) 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad35e00640 1 -- 172.21.15.28:0/1606153371 learned_addr learned my addr 172.21.15.28:0/1606153371 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.313+0000 7fad2fe00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380af210 0x7fad38070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.314+0000 7fad35e00640 1 -- 172.21.15.28:0/1606153371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380af210 msgr2=0x7fad38070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.314+0000 7fad35e00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380af210 0x7fad38070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.314+0000 7fad35e00640 1 -- 172.21.15.28:0/1606153371 >> v1:172.21.15.138:6789/0 conn(0x7fad380b7250 legacy=0x7fad3809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.314+0000 7fad35e00640 1 -- 172.21.15.28:0/1606153371 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3806d660 con 0x7fad3815e7f0 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.315+0000 7fad35e00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad380b6d10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fad28009870 tx=0x7fad2802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64714ce703466f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.315+0000 7fad35400640 1 -- 172.21.15.28:0/1606153371 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28037070 con 0x7fad3815e7f0 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.315+0000 7fad35400640 1 -- 172.21.15.28:0/1606153371 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad28032440 con 0x7fad3815e7f0 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.315+0000 7fad35400640 1 -- 172.21.15.28:0/1606153371 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad28032810 con 0x7fad3815e7f0 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 -- 172.21.15.28:0/1606153371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 msgr2=0x7fad380b6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad380b6d10 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fad28009870 tx=0x7fad2802a250 comp rx=0 tx=0).stop 2024-09-10T10:53:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 -- 172.21.15.28:0/1606153371 shutdown_connections 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380af210 0x7fad38070a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1606153371 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad380b6d10 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 -- 172.21.15.28:0/1606153371 >> 172.21.15.28:0/1606153371 conn(0x7fad380b3f30 msgr2=0x7fad380b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 -- 172.21.15.28:0/1606153371 shutdown_connections 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.316+0000 7fad3cc00640 1 -- 172.21.15.28:0/1606153371 wait complete. 2024-09-10T10:53:40.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.317+0000 7fad3cc00640 1 Processor -- start 2024-09-10T10:53:40.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.318+0000 7fad3cc00640 1 -- start start 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.318+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380af210 0x7fad38146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.318+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380b7250 0x7fad38155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad3cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad3cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad38070fb0 con 0x7fad3815e7f0 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad3cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad380b6150 con 0x7fad380af210 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad3cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad38071130 con 0x7fad380b7250 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380b7250 0x7fad38155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52486/0 (socket says 172.21.15.28:52486) 2024-09-10T10:53:40.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad35e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380af210 0x7fad38146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 -- 172.21.15.28:0/1837090237 learned_addr learned my addr 172.21.15.28:0/1837090237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 -- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380b7250 msgr2=0x7fad38155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380b7250 0x7fad38155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 -- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380af210 msgr2=0x7fad38146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380af210 0x7fad38146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.319+0000 7fad36800640 1 -- 172.21.15.28:0/1837090237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad38157030 con 0x7fad3815e7f0 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad36800640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c760 tx=0x7fad3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30003b70 con 0x7fad3815e7f0 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad30003d10 con 0x7fad3815e7f0 2024-09-10T10:53:40.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000aa90 con 0x7fad3815e7f0 2024-09-10T10:53:40.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad38157310 con 0x7fad3815e7f0 2024-09-10T10:53:40.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.320+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad381578d0 con 0x7fad3815e7f0 2024-09-10T10:53:40.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.321+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38070a70 con 0x7fad3815e7f0 2024-09-10T10:53:40.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.322+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fad3000ac30 con 0x7fad3815e7f0 2024-09-10T10:53:40.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.323+0000 7fad2e000640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 0x7fad1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.323+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 8712+0+0 (secure 0 0 0) 0x7fad3008b6f0 con 0x7fad3815e7f0 2024-09-10T10:53:40.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.323+0000 7fad35e00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 0x7fad1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.324+0000 7fad35e00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 0x7fad1406e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fad28034000 tx=0x7fad2802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.327+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fad30058e70 con 0x7fad3815e7f0 2024-09-10T10:53:40.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.549+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fad380b32b0 con 0x7fad3815e7f0 2024-09-10T10:53:40.566 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:53:40.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:53:40.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.720+0000 7fad2e000640 1 -- 172.21.15.28:0/1837090237 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v250) ==== 162+0+0 (secure 0 0 0) 0x7fad3005d170 con 0x7fad3815e7f0 2024-09-10T10:53:40.721 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.722+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 msgr2=0x7fad1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.722+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 0x7fad1406e2f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fad28034000 tx=0x7fad2802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.722+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 msgr2=0x7fad38155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.722+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fad3000c760 tx=0x7fad3000cc20 comp rx=0 tx=0).stop 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 shutdown_connections 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fad1406be30 0x7fad1406e2f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad3815e7f0 0x7fad38155540 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad380b7250 0x7fad38155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 --2- 172.21.15.28:0/1837090237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad380af210 0x7fad38146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 >> 172.21.15.28:0/1837090237 conn(0x7fad380b3f30 msgr2=0x7fad380b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 shutdown_connections 2024-09-10T10:53:40.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.723+0000 7fad3cc00640 1 -- 172.21.15.28:0/1837090237 wait complete. 2024-09-10T10:53:40.729 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:40.728+0000 7f6335e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-10T10:53:40.729 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:40.729+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:40.729 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:40.729+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:40.741 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.898+0000 7f5e1d400640 1 Processor -- start 2024-09-10T10:53:40.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.898+0000 7f5e1d400640 1 -- start start 2024-09-10T10:53:40.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e1d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e180cfcc0 0x7f5e1806cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e1d400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e180d4fd0 0x7f5e180d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e1d400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e180d2700 con 0x7f5e1815e760 2024-09-10T10:53:40.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e1d400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5e180cedb0 con 0x7f5e180cfcc0 2024-09-10T10:53:40.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e1d400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5e180cf090 con 0x7f5e180d4fd0 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e16800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5e1815e760 0x7f5e180cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38620/0 (socket says 172.21.15.28:38620) 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e16800640 1 -- 172.21.15.28:0/2723525824 learned_addr learned my addr 172.21.15.28:0/2723525824 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e15e00640 1 --2- 172.21.15.28:0/2723525824 >> v2:172.21.15.28:3300/0 conn(0x7f5e180cfcc0 0x7f5e1806cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.899+0000 7f5e17200640 1 --2- 172.21.15.28:0/2723525824 >> v2:172.21.15.138:3300/0 conn(0x7f5e180d4fd0 0x7f5e180d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15400640 1 -- 172.21.15.28:0/2723525824 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4100819654 0 0) 0x7f5e08001110 con 0x7f5e1815e760 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15400640 1 -- 172.21.15.28:0/2723525824 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dfc003520 con 0x7f5e1815e760 2024-09-10T10:53:40.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15e00640 1 -- 172.21.15.28:0/2723525824 >> v2:172.21.15.138:3300/0 conn(0x7f5e180d4fd0 msgr2=0x7f5e180d2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:40.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15e00640 1 --2- 172.21.15.28:0/2723525824 >> v2:172.21.15.138:3300/0 conn(0x7f5e180d4fd0 0x7f5e180d2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15e00640 1 -- 172.21.15.28:0/2723525824 >> v1:172.21.15.28:6789/0 conn(0x7f5e1815e760 legacy=0x7f5e180cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.900+0000 7f5e15e00640 1 -- 172.21.15.28:0/2723525824 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e180b3c80 con 0x7f5e180cfcc0 2024-09-10T10:53:40.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.901+0000 7f5e15e00640 1 --2- 172.21.15.28:0/2723525824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e180cfcc0 0x7f5e1806cdd0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5e1806ee50 tx=0x7f5e0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c34a2cb76aaf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.901+0000 7f5e15400640 1 -- 172.21.15.28:0/2723525824 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04037070 con 0x7f5e180cfcc0 2024-09-10T10:53:40.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.901+0000 7f5e15400640 1 -- 172.21.15.28:0/2723525824 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e04032440 con 0x7f5e180cfcc0 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.901+0000 7f5e15400640 1 -- 172.21.15.28:0/2723525824 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04032810 con 0x7f5e180cfcc0 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 -- 172.21.15.28:0/2723525824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e180cfcc0 msgr2=0x7f5e1806cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 --2- 172.21.15.28:0/2723525824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e180cfcc0 0x7f5e1806cdd0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5e1806ee50 tx=0x7f5e0402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 -- 172.21.15.28:0/2723525824 shutdown_connections 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 --2- 172.21.15.28:0/2723525824 >> v2:172.21.15.138:3300/0 conn(0x7f5e180d4fd0 0x7f5e180d2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 --2- 172.21.15.28:0/2723525824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e180cfcc0 0x7f5e1806cdd0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 -- 172.21.15.28:0/2723525824 >> 172.21.15.28:0/2723525824 conn(0x7f5e1808bfc0 msgr2=0x7f5e180d4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:40.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 -- 172.21.15.28:0/2723525824 shutdown_connections 2024-09-10T10:53:40.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.902+0000 7f5e1d400640 1 -- 172.21.15.28:0/2723525824 wait complete. 2024-09-10T10:53:40.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.903+0000 7f5e1d400640 1 Processor -- start 2024-09-10T10:53:40.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.903+0000 7f5e1d400640 1 -- start start 2024-09-10T10:53:40.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 0x7f5e181f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e180d4fd0 0x7f5e1815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5e180d33e0 con 0x7f5e1815e760 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e180b38e0 con 0x7f5e180cfcc0 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.904+0000 7f5e1d400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e180d3560 con 0x7f5e180d4fd0 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52500/0 (socket says 172.21.15.28:52500) 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 -- 172.21.15.28:0/1037658508 learned_addr learned my addr 172.21.15.28:0/1037658508 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e16800640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 0x7f5e181f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 -- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e180d4fd0 msgr2=0x7f5e1815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e180d4fd0 0x7f5e1815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 -- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 msgr2=0x7f5e181f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 0x7f5e181f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 -- 172.21.15.28:0/1037658508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e180cc900 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e16800640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 0x7f5e181f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.905+0000 7f5e17200640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000efc0 tx=0x7f5e1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.906+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10018020 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.906+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e10014480 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.906+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e10014870 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.906+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e180ccbe0 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.906+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e180cd1a0 con 0x7f5e1815e760 2024-09-10T10:53:40.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.907+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1815c1f0 con 0x7f5e1815e760 2024-09-10T10:53:40.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.908+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e10005d10 con 0x7f5e1815e760 2024-09-10T10:53:40.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.909+0000 7f5e0ea00640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 0x7f5df806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:40.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.909+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f5e1008c570 con 0x7f5e1815e760 2024-09-10T10:53:40.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.909+0000 7f5e16800640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 0x7f5df806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:40.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.910+0000 7f5e16800640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 0x7f5df806e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5e080012b0 tx=0x7f5e08009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:40.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:40.913+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e10022070 con 0x7f5e1815e760 2024-09-10T10:53:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.136+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e180d3890 con 0x7f5e1815e760 2024-09-10T10:53:41.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.137+0000 7f5e0ea00640 1 -- 172.21.15.28:0/1037658508 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 252 v252) ==== 95+0+3566 (secure 0 0 0) 0x7f5e10010040 con 0x7f5e1815e760 2024-09-10T10:53:41.138 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:41.138 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":252,"btime":"2024-09-10T10:53:40:729674+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":252,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:40.729593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:41.138 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 252 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.139+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 msgr2=0x7f5df806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.139+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 0x7f5df806e2f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5e080012b0 tx=0x7f5e08009040 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.139+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 msgr2=0x7f5e180b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.139+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000efc0 tx=0x7f5e1000c490 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 shutdown_connections 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5df806be30 0x7f5df806e2f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e1815e760 0x7f5e180b4f90 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e180d4fd0 0x7f5e1815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 --2- 172.21.15.28:0/1037658508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e180cfcc0 0x7f5e181f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 >> 172.21.15.28:0/1037658508 conn(0x7f5e1808bfc0 msgr2=0x7f5e180d0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:41.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 shutdown_connections 2024-09-10T10:53:41.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.140+0000 7f5e1d400640 1 -- 172.21.15.28:0/1037658508 wait complete. 2024-09-10T10:53:41.156 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:41.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:41.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.312+0000 7f5795a00640 1 Processor -- start 2024-09-10T10:53:41.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.312+0000 7f5795a00640 1 -- start start 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f5795a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 0x7f5790169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f5795a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57900d2c20 con 0x7f57900d4180 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f5795a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579006e390 con 0x7f579015e6a0 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f5795a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57900cefb0 con 0x7f57900d0450 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f578fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f57900d4180 0x7f57900d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38624/0 (socket says 172.21.15.28:38624) 2024-09-10T10:53:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f578fe00640 1 -- 172.21.15.28:0/4162980419 learned_addr learned my addr 172.21.15.28:0/4162980419 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f5795000640 1 --2- 172.21.15.28:0/4162980419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 0x7f5790169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3391200030 0 0) 0x7f57900d2c20 con 0x7f57900d4180 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.314+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5774003520 con 0x7f57900d4180 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.315+0000 7f5795000640 1 -- 172.21.15.28:0/4162980419 >> v1:172.21.15.138:6789/0 conn(0x7f579015e6a0 legacy=0x7f579006d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.315+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1888295059 0 0) 0x7f579006e390 con 0x7f579015e6a0 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.315+0000 7f5795000640 1 -- 172.21.15.28:0/4162980419 >> v1:172.21.15.28:6789/0 conn(0x7f57900d4180 legacy=0x7f57900d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.315+0000 7f5795000640 1 -- 172.21.15.28:0/4162980419 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579006e670 con 0x7f57900d0450 2024-09-10T10:53:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f5795000640 1 --2- 172.21.15.28:0/4162980419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 0x7f5790169000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5788009dc0 tx=0x7f578802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2700d0eeb2e592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578802ae60 con 0x7f57900d0450 2024-09-10T10:53:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5788007740 con 0x7f57900d0450 2024-09-10T10:53:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f578f400640 1 -- 172.21.15.28:0/4162980419 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788007af0 con 0x7f57900d0450 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f5795a00640 1 -- 172.21.15.28:0/4162980419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 msgr2=0x7f5790169000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.316+0000 7f5795a00640 1 --2- 172.21.15.28:0/4162980419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 0x7f5790169000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5788009dc0 tx=0x7f578802a640 comp rx=0 tx=0).stop 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.317+0000 7f5795a00640 1 -- 172.21.15.28:0/4162980419 shutdown_connections 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.317+0000 7f5795a00640 1 --2- 172.21.15.28:0/4162980419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57900d0450 0x7f5790169000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.317+0000 7f5795a00640 1 -- 172.21.15.28:0/4162980419 >> 172.21.15.28:0/4162980419 conn(0x7f579009d6f0 msgr2=0x7f579008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.317+0000 7f5795a00640 1 -- 172.21.15.28:0/4162980419 shutdown_connections 2024-09-10T10:53:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.317+0000 7f5795a00640 1 -- 172.21.15.28:0/4162980419 wait complete. 2024-09-10T10:53:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.318+0000 7f5795a00640 1 Processor -- start 2024-09-10T10:53:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.318+0000 7f5795a00640 1 -- start start 2024-09-10T10:53:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57900d0450 0x7f57901f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f579015e6a0 0x7f57901544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f579006df30 con 0x7f57900d4180 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57900cebd0 con 0x7f57900d0450 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5795a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f579006e0b0 con 0x7f579015e6a0 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.319+0000 7f5787e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52504/0 (socket says 172.21.15.28:52504) 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 -- 172.21.15.28:0/3796081366 learned_addr learned my addr 172.21.15.28:0/3796081366 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5795000640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f579015e6a0 0x7f57901544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f578fe00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57900d0450 0x7f57901f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 -- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f579015e6a0 msgr2=0x7f57901544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f579015e6a0 0x7f57901544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 -- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57900d0450 msgr2=0x7f57901f7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57900d0450 0x7f57901f7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 -- 172.21.15.28:0/3796081366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57901f8260 con 0x7f57900d4180 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f5787e00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f577800c410 tx=0x7f577800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.320+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778017070 con 0x7f57900d4180 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.321+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57780035f0 con 0x7f57900d4180 2024-09-10T10:53:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.321+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57780039e0 con 0x7f57900d4180 2024-09-10T10:53:41.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.321+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57900c94d0 con 0x7f57900d4180 2024-09-10T10:53:41.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.321+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57900cdd80 con 0x7f57900d4180 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.322+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57900d4560 con 0x7f57900d4180 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.323+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f577801a070 con 0x7f57900d4180 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.323+0000 7f578cc00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 0x7f576c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.324+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f577808af00 con 0x7f57900d4180 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.324+0000 7f578fe00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 0x7f576c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.325+0000 7f578fe00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 0x7f576c06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5780005880 tx=0x7f5780009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.328+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57901f8260 con 0x7f57900d4180 2024-09-10T10:53:41.456 INFO:tasks.ceph.mds.a.smithi028.stderr: -41> 2024-09-10T10:53:36.703+0000 7f0c03600640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds_snaptable': End of buffer [buffer:2] 2024-09-10T10:53:41.495 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:41.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.550+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57900b9210 con 0x7f57900d4180 2024-09-10T10:53:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.551+0000 7f578cc00640 1 -- 172.21.15.28:0/3796081366 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 252 v252) ==== 95+0+3566 (secure 0 0 0) 0x7f5778058bf0 con 0x7f57900d4180 2024-09-10T10:53:41.552 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:41.552 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":252,"btime":"2024-09-10T10:53:40:729674+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":252,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:40.729593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 252 2024-09-10T10:53:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 msgr2=0x7f576c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 0x7f576c06e320 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5780005880 tx=0x7f5780009040 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 msgr2=0x7f57900b76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f577800c410 tx=0x7f577800c8d0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 shutdown_connections 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576c06be60 0x7f576c06e320 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f579015e6a0 0x7f57901544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f57900d4180 0x7f57900b76e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 --2- 172.21.15.28:0/3796081366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57900d0450 0x7f57901f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 >> 172.21.15.28:0/3796081366 conn(0x7f579009d6f0 msgr2=0x7f57900b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 shutdown_connections 2024-09-10T10:53:41.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.553+0000 7f5795a00640 1 -- 172.21.15.28:0/3796081366 wait complete. 2024-09-10T10:53:41.570 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:41.570 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.725+0000 7f22fcc00640 1 Processor -- start 2024-09-10T10:53:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.725+0000 7f22fcc00640 1 -- start start 2024-09-10T10:53:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.726+0000 7f22fcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22f80d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.726+0000 7f22fcc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22f80d1f90 0x7f22f80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.726+0000 7f22fcc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f80cf000 con 0x7f22f815e660 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.726+0000 7f22fcc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f22f806d3e0 con 0x7f22f80d1f90 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.726+0000 7f22fcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22f80b37d0 con 0x7f22f80d0dd0 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f22f815e660 0x7f22f806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38650/0 (socket says 172.21.15.28:38650) 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6a00640 1 -- 172.21.15.28:0/1058374444 learned_addr learned my addr 172.21.15.28:0/1058374444 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f7e00640 1 --2- 172.21.15.28:0/1058374444 >> v2:172.21.15.138:3300/0 conn(0x7f22f80d1f90 0x7f22f80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f7400640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22f80d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667058959 0 0) 0x7f22f80cf000 con 0x7f22f815e660 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22dc003520 con 0x7f22f815e660 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1456163006 0 0) 0x7f22dc003520 con 0x7f22f815e660 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 msgr2=0x7f22f80d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22f80d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 >> v2:172.21.15.138:3300/0 conn(0x7f22f80d1f90 msgr2=0x7f22f80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 --2- 172.21.15.28:0/1058374444 >> v2:172.21.15.138:3300/0 conn(0x7f22f80d1f90 0x7f22f80cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f80d16f0 con 0x7f22f815e660 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f22e4001670 con 0x7f22f815e660 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.727+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 >> v1:172.21.15.28:6789/0 conn(0x7f22f815e660 legacy=0x7f22f806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc003f80 0x7f22dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22dc004d40 0x7f22dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f22f806d3e0 con 0x7f22dc004d40 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc00b9c0 con 0x7f22dc003f80 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc00bb40 con 0x7f22f80d0dd0 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22dc004d40 0x7f22dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7400640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 -- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 msgr2=0x7f22dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80d0dd0 0x7f22dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 -- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc003f80 msgr2=0x7f22dc004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc003f80 0x7f22dc004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.728+0000 7f22f7e00640 1 -- 172.21.15.28:0/1058374444 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc003520 con 0x7f22dc004d40 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.729+0000 7f22f7e00640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22dc004d40 0x7f22dc0051c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e170 tx=0x7f22dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5c236f36c7c255 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.729+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0034020 con 0x7f22dc004d40 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.729+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f0030430 con 0x7f22dc004d40 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.729+0000 7f22f6000640 1 -- 172.21.15.28:0/1058374444 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0009c10 con 0x7f22dc004d40 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 -- 172.21.15.28:0/1058374444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22dc004d40 msgr2=0x7f22dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 --2- 172.21.15.28:0/1058374444 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22dc004d40 0x7f22dc0051c0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f22f000e170 tx=0x7f22dc005830 comp rx=0 tx=0).stop 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22f7e00640 1 -- 172.21.15.28:0/1058374444 reap_dead start 2024-09-10T10:53:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 -- 172.21.15.28:0/1058374444 shutdown_connections 2024-09-10T10:53:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 -- 172.21.15.28:0/1058374444 >> 172.21.15.28:0/1058374444 conn(0x7f22f808bfc0 msgr2=0x7f22f80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 -- 172.21.15.28:0/1058374444 shutdown_connections 2024-09-10T10:53:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.730+0000 7f22fcc00640 1 -- 172.21.15.28:0/1058374444 wait complete. 2024-09-10T10:53:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.732+0000 7f22fcc00640 1 Processor -- start 2024-09-10T10:53:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.732+0000 7f22fcc00640 1 -- start start 2024-09-10T10:53:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 0x7f22f8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 0x7f22f80b9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f22f80d2c60 con 0x7f22f8154c00 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f22f80cf330 con 0x7f22dc004d40 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22fcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f22f80d2f40 con 0x7f22f80b9670 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.733+0000 7f22f6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 0x7f22f8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 0x7f22f8154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:38576/0 (socket says 172.21.15.28:38576) 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f7400640 1 -- 172.21.15.28:0/2540762296 learned_addr learned my addr 172.21.15.28:0/2540762296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52532/0 (socket says 172.21.15.28:52532) 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f7e00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 0x7f22f80b9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 -- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 msgr2=0x7f22f80b9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 0x7f22f80b9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 -- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 msgr2=0x7f22f8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 0x7f22f8154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 -- 172.21.15.28:0/2540762296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f80b3f30 con 0x7f22f8154c00 2024-09-10T10:53:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f7e00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 0x7f22f80b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.734+0000 7f22f6a00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f22f80b3980 tx=0x7f22e400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.735+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4018070 con 0x7f22f8154c00 2024-09-10T10:53:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.735+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e40072d0 con 0x7f22f8154c00 2024-09-10T10:53:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.735+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e40047e0 con 0x7f22f8154c00 2024-09-10T10:53:41.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.735+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f80b4210 con 0x7f22f8154c00 2024-09-10T10:53:41.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.735+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f80b6ba0 con 0x7f22f8154c00 2024-09-10T10:53:41.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.736+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f80ba030 con 0x7f22f8154c00 2024-09-10T10:53:41.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.737+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f22e401b070 con 0x7f22f8154c00 2024-09-10T10:53:41.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.738+0000 7f22ef400640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 0x7f22d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:41.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.738+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f22e408aea0 con 0x7f22f8154c00 2024-09-10T10:53:41.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.738+0000 7f22f7400640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 0x7f22d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:41.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.739+0000 7f22f7400640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 0x7f22d806e2f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006d40 tx=0x7f22e8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:41.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.743+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f22e4058540 con 0x7f22f8154c00 2024-09-10T10:53:41.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.962+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22f80d3220 con 0x7f22f8154c00 2024-09-10T10:53:41.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.963+0000 7f22ef400640 1 -- 172.21.15.28:0/2540762296 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 253 v253) ==== 95+0+4392 (secure 0 0 0) 0x7f22e405c840 con 0x7f22f8154c00 2024-09-10T10:53:41.964 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:41.964 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":253,"btime":"2024-09-10T10:53:41:731562+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/4204725630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":4204725630},{"type":"v1","addr":"172.21.15.28:6835","nonce":4204725630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":253}],"filesystems":[{"mdsmap":{"epoch":252,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:40.729593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":129,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:41.965 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 253 2024-09-10T10:53:41.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 msgr2=0x7f22d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 0x7f22d806e2f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f22e8006d40 tx=0x7f22e8009190 comp rx=0 tx=0).stop 2024-09-10T10:53:41.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 msgr2=0x7f22f81f8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f22f80b3980 tx=0x7f22e400cb90 comp rx=0 tx=0).stop 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 shutdown_connections 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f22d806be30 0x7f22d806e2f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f22f80b9670 0x7f22f80b9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22f8154c00 0x7f22f81f8190 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 --2- 172.21.15.28:0/2540762296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22dc004d40 0x7f22f8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 >> 172.21.15.28:0/2540762296 conn(0x7f22f808bfc0 msgr2=0x7f22f80cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 shutdown_connections 2024-09-10T10:53:41.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:41.965+0000 7f22fcc00640 1 -- 172.21.15.28:0/2540762296 wait complete. 2024-09-10T10:53:41.983 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate mds0_inotable' led to DAMAGED state 2024-09-10T10:53:41.984 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 400.00000000' 2024-09-10T10:53:41.984 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:41.984 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:42.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:42.020 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:42.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:42.020 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:42.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:42.095 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:42.096 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:42.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.291+0000 7f84d6000640 1 Processor -- start 2024-09-10T10:53:42.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.291+0000 7f84d6000640 1 -- start start 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84d6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d0072320 0x7f84d00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84d6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f84d015e1f0 0x7f84d006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84d6000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d0146140 con 0x7f84d0072de0 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84d6000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f84d00cd740 con 0x7f84d015e1f0 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84d6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84d00cda20 con 0x7f84d0072320 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84cfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f84d015e1f0 0x7f84d006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.292+0000 7f84cfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f84d015e1f0 0x7f84d006e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52540/0 (socket says 172.21.15.28:52540) 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cfe00640 1 -- 172.21.15.28:0/4231671804 learned_addr learned my addr 172.21.15.28:0/4231671804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cf400640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d0072320 0x7f84d00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cfe00640 1 -- 172.21.15.28:0/4231671804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d0072320 msgr2=0x7f84d00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cfe00640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d0072320 0x7f84d00cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cfe00640 1 -- 172.21.15.28:0/4231671804 >> v1:172.21.15.28:6789/0 conn(0x7f84d0072de0 legacy=0x7f84d006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.293+0000 7f84cfe00640 1 -- 172.21.15.28:0/4231671804 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d00cdd00 con 0x7f84d015e1f0 2024-09-10T10:53:42.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.294+0000 7f84cfe00640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d015e1f0 0x7f84d006e7c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009e50 tx=0x7f84c802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a2a45d9d36c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.294+0000 7f84ce000640 1 -- 172.21.15.28:0/4231671804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8036070 con 0x7f84d015e1f0 2024-09-10T10:53:42.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.294+0000 7f84ce000640 1 -- 172.21.15.28:0/4231671804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c8031600 con 0x7f84d015e1f0 2024-09-10T10:53:42.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.294+0000 7f84ce000640 1 -- 172.21.15.28:0/4231671804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c80319d0 con 0x7f84d015e1f0 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 -- 172.21.15.28:0/4231671804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d015e1f0 msgr2=0x7f84d006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d015e1f0 0x7f84d006e7c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009e50 tx=0x7f84c802a680 comp rx=0 tx=0).stop 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 -- 172.21.15.28:0/4231671804 shutdown_connections 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d015e1f0 0x7f84d006e7c0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 --2- 172.21.15.28:0/4231671804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d0072320 0x7f84d00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 -- 172.21.15.28:0/4231671804 >> 172.21.15.28:0/4231671804 conn(0x7f84d0067d70 msgr2=0x7f84d006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:42.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 -- 172.21.15.28:0/4231671804 shutdown_connections 2024-09-10T10:53:42.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.295+0000 7f84d6000640 1 -- 172.21.15.28:0/4231671804 wait complete. 2024-09-10T10:53:42.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.296+0000 7f84d6000640 1 Processor -- start 2024-09-10T10:53:42.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 -- start start 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84d0072320 0x7f84d00aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d015e1f0 0x7f84d0138160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f84d00cd520 con 0x7f84d0072de0 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84d00cd3a0 con 0x7f84d0072320 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.297+0000 7f84d6000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f84d006ed00 con 0x7f84d015e1f0 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52554/0 (socket says 172.21.15.28:52554) 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 -- 172.21.15.28:0/3815356233 learned_addr learned my addr 172.21.15.28:0/3815356233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:42.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cfe00640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d015e1f0 0x7f84d0138160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cf400640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84d0072320 0x7f84d00aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 -- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d015e1f0 msgr2=0x7f84d0138160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d015e1f0 0x7f84d0138160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 -- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84d0072320 msgr2=0x7f84d00aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84d0072320 0x7f84d00aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 -- 172.21.15.28:0/3815356233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d013c3c0 con 0x7f84d0072de0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.298+0000 7f84cea00640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c440 tx=0x7f84bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.299+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0052d0 con 0x7f84d0072de0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.299+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84bc005470 con 0x7f84d0072de0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.299+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc007260 con 0x7f84d0072de0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.299+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d0138cd0 con 0x7f84d0072de0 2024-09-10T10:53:42.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.299+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d00aa4e0 con 0x7f84d0072de0 2024-09-10T10:53:42.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.300+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0072700 con 0x7f84d0072de0 2024-09-10T10:53:42.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.301+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f84bc007400 con 0x7f84d0072de0 2024-09-10T10:53:42.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.302+0000 7f84c7400640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 0x7f84b006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.302+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f84bc08afe0 con 0x7f84d0072de0 2024-09-10T10:53:42.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.302+0000 7f84cf400640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 0x7f84b006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.303+0000 7f84cf400640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 0x7f84b006e2f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006fd0 tx=0x7f84c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.306+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f84bc058d30 con 0x7f84d0072de0 2024-09-10T10:53:42.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.525+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84d00cebc0 con 0x7f84d0072de0 2024-09-10T10:53:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.738+0000 7f84c7400640 1 -- 172.21.15.28:0/3815356233 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v254) ==== 161+0+0 (secure 0 0 0) 0x7f84bc058630 con 0x7f84d0072de0 2024-09-10T10:53:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 msgr2=0x7f84b006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 0x7f84b006e2f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006fd0 tx=0x7f84c0008040 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 msgr2=0x7f84d00cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00c440 tx=0x7f84bc001e70 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 shutdown_connections 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f84b006be30 0x7f84b006e2f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f84d015e1f0 0x7f84d0138160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84d0072de0 0x7f84d00cbfe0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 --2- 172.21.15.28:0/3815356233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84d0072320 0x7f84d00aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 >> 172.21.15.28:0/3815356233 conn(0x7f84d0067d70 msgr2=0x7f84d00b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.740+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 shutdown_connections 2024-09-10T10:53:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.741+0000 7f84d6000640 1 -- 172.21.15.28:0/3815356233 wait complete. 2024-09-10T10:53:42.758 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.913+0000 7f0fd5800640 1 Processor -- start 2024-09-10T10:53:42.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.914+0000 7f0fd5800640 1 -- start start 2024-09-10T10:53:42.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0fd015b300 0x7f0fd00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd00cd3f0 0x7f0fd006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0fd0072d30 0x7f0fd006f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd00728a0 con 0x7f0fd015b300 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0fd00cdbe0 con 0x7f0fd0072d30 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fd5800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd00cd900 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fcea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0fd015b300 0x7f0fd00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fcea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0fd015b300 0x7f0fd00cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52564/0 (socket says 172.21.15.28:52564) 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fcea00640 1 -- 172.21.15.28:0/1767298792 learned_addr learned my addr 172.21.15.28:0/1767298792 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fc7e00640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd00cd3f0 0x7f0fd006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.915+0000 7f0fcf400640 1 --2- 172.21.15.28:0/1767298792 >> v2:172.21.15.138:3300/0 conn(0x7f0fd0072d30 0x7f0fd006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.916+0000 7f0fcea00640 1 -- 172.21.15.28:0/1767298792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd00cd3f0 msgr2=0x7f0fd006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.916+0000 7f0fcea00640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd00cd3f0 0x7f0fd006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.916+0000 7f0fcea00640 1 -- 172.21.15.28:0/1767298792 >> v2:172.21.15.138:3300/0 conn(0x7f0fd0072d30 msgr2=0x7f0fd006f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:42.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.916+0000 7f0fcea00640 1 --2- 172.21.15.28:0/1767298792 >> v2:172.21.15.138:3300/0 conn(0x7f0fd0072d30 0x7f0fd006f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.916+0000 7f0fcea00640 1 -- 172.21.15.28:0/1767298792 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd0072250 con 0x7f0fd015b300 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fcea00640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd015b300 0x7f0fd00cceb0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0072040 tx=0x7f0fc002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83876a39508231ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fce000640 1 -- 172.21.15.28:0/1767298792 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0037070 con 0x7f0fd015b300 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fce000640 1 -- 172.21.15.28:0/1767298792 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc0032440 con 0x7f0fd015b300 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fce000640 1 -- 172.21.15.28:0/1767298792 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0032810 con 0x7f0fd015b300 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fd5800640 1 -- 172.21.15.28:0/1767298792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd015b300 msgr2=0x7f0fd00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.917+0000 7f0fd5800640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd015b300 0x7f0fd00cceb0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0072040 tx=0x7f0fc002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 -- 172.21.15.28:0/1767298792 shutdown_connections 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 --2- 172.21.15.28:0/1767298792 >> v2:172.21.15.138:3300/0 conn(0x7f0fd0072d30 0x7f0fd006f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd00cd3f0 0x7f0fd006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 --2- 172.21.15.28:0/1767298792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd015b300 0x7f0fd00cceb0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 -- 172.21.15.28:0/1767298792 >> 172.21.15.28:0/1767298792 conn(0x7f0fd0067d70 msgr2=0x7f0fd006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:42.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 -- 172.21.15.28:0/1767298792 shutdown_connections 2024-09-10T10:53:42.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.918+0000 7f0fd5800640 1 -- 172.21.15.28:0/1767298792 wait complete. 2024-09-10T10:53:42.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.919+0000 7f0fd5800640 1 Processor -- start 2024-09-10T10:53:42.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.919+0000 7f0fd5800640 1 -- start start 2024-09-10T10:53:42.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd0072d30 0x7f0fd01f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd015b300 0x7f0fd00b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00d0160 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd00cffe0 con 0x7f0fd0072d30 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.920+0000 7f0fd5800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0fd00b3c40 con 0x7f0fd015b300 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52570/0 (socket says 172.21.15.28:52570) 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 -- 172.21.15.28:0/4031335861 learned_addr learned my addr 172.21.15.28:0/4031335861 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fcea00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd0072d30 0x7f0fd01f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fcf400640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd015b300 0x7f0fd00b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 -- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd015b300 msgr2=0x7f0fd00b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd015b300 0x7f0fd00b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 -- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd0072d30 msgr2=0x7f0fd01f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd0072d30 0x7f0fd01f82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 -- 172.21.15.28:0/4031335861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd01f87e0 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.921+0000 7f0fc7e00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d7d0 tx=0x7f0fb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.922+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8004020 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.922+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb80041e0 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.922+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd0157110 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.922+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80043d0 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.922+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd0157690 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.923+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd01580e0 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.924+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fb801b020 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.924+0000 7f0fc6a00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 0x7f0fac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:42.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.925+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f0fb808b740 con 0x7f0fd00cd3f0 2024-09-10T10:53:42.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.925+0000 7f0fcea00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 0x7f0fac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:42.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.926+0000 7f0fcea00640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 0x7f0fac06e2f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0034000 tx=0x7f0fc002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:42.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:42.929+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0fd01f87e0 con 0x7f0fd00cd3f0 2024-09-10T10:53:43.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.184+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f0fd0158a40 con 0x7f0fd00cd3f0 2024-09-10T10:53:43.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.744+0000 7f0fc6a00640 1 -- 172.21.15.28:0/4031335861 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v255) ==== 101+0+0 (secure 0 0 0) 0x7f0fb8058e70 con 0x7f0fd00cd3f0 2024-09-10T10:53:43.745 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 msgr2=0x7f0fac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 0x7f0fac06e2f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0034000 tx=0x7f0fc002e000 comp rx=0 tx=0).stop 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 msgr2=0x7f0fd0155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d7d0 tx=0x7f0fb800dc90 comp rx=0 tx=0).stop 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 shutdown_connections 2024-09-10T10:53:43.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fac06be30 0x7f0fac06e2f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0fd015b300 0x7f0fd00b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fd00cd3f0 0x7f0fd0155200 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 --2- 172.21.15.28:0/4031335861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd0072d30 0x7f0fd01f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 >> 172.21.15.28:0/4031335861 conn(0x7f0fd0067d70 msgr2=0x7f0fd00b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 shutdown_connections 2024-09-10T10:53:43.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.747+0000 7f0fd5800640 1 -- 172.21.15.28:0/4031335861 wait complete. 2024-09-10T10:53:43.765 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:43.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.841+0000 7f5e5ce3cf40 1 Processor -- start 2024-09-10T10:53:43.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 -- start start 2024-09-10T10:53:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502a698e0 0x55e502a51510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502af01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e502a6a530 con 0x55e502af53e0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e5029c7b30 con 0x55e502af89c0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.842+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e502af7ee0 con 0x55e502a698e0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37418/0 (socket says 172.21.15.28:37418) 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 -- 172.21.15.28:0/24953063 learned_addr learned my addr 172.21.15.28:0/24953063 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e5a000640 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502a698e0 0x55e502a51510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502af01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 -- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502a698e0 msgr2=0x55e502a51510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502a698e0 0x55e502a51510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 -- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 msgr2=0x55e502af01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502af01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 -- 172.21.15.28:0/24953063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e502a50fa0 con 0x55e502af53e0 2024-09-10T10:53:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.843+0000 7f5e59600640 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800a770 tx=0x7f5e4800a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852e50623287b67b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e53e00640 1 -- 172.21.15.28:0/24953063 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48019070 con 0x55e502af53e0 2024-09-10T10:53:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e53e00640 1 -- 172.21.15.28:0/24953063 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e48007600 con 0x55e502af53e0 2024-09-10T10:53:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e53e00640 1 -- 172.21.15.28:0/24953063 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480079d0 con 0x55e502af53e0 2024-09-10T10:53:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/24953063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 msgr2=0x55e502a1ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5e4800a770 tx=0x7f5e4800a860 comp rx=0 tx=0).stop 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/24953063 shutdown_connections 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502af01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502af53e0 0x55e502a1ed40 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/24953063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502a698e0 0x55e502a51510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/24953063 >> 172.21.15.28:0/24953063 conn(0x55e502a6e0a0 msgr2=0x55e502af5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/24953063 shutdown_connections 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.844+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/24953063 wait complete. 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.845+0000 7f5e5ce3cf40 1 Processor -- start 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.845+0000 7f5e5ce3cf40 1 -- start start 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502af53e0 0x55e502a62e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502add760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e502af7190 con 0x55e502a698e0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e502aec800 con 0x55e502af89c0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5ce3cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55e502a6e520 con 0x55e502af53e0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37434/0 (socket says 172.21.15.28:37434) 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 learned_addr learned my addr 172.21.15.28:0/971226626 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502af53e0 0x55e502a62e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.846+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502add760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502af53e0 msgr2=0x55e502a62e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55e502af53e0 0x55e502a62e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 msgr2=0x55e502add760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502add760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:43.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e502a67ce0 con 0x55e502a698e0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e502af89c0 0x55e502add760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c000c00 tx=0x7f5e4c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e52a00640 1 -- 172.21.15.28:0/971226626 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c056070 con 0x55e502a698e0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e52a00640 1 -- 172.21.15.28:0/971226626 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4c052910 con 0x55e502a698e0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e52a00640 1 -- 172.21.15.28:0/971226626 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c052d00 con 0x55e502a698e0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.847+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e502a67fc0 con 0x55e502a698e0 2024-09-10T10:53:43.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.848+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e502a67660 con 0x55e502a698e0 2024-09-10T10:53:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.849+0000 7f5e52a00640 1 -- 172.21.15.28:0/971226626 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e4c059020 con 0x55e502a698e0 2024-09-10T10:53:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.850+0000 7f5e52a00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 0x7f5e3c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.850+0000 7f5e52a00640 1 -- 172.21.15.28:0/971226626 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f5e4c0ce650 con 0x55e502a698e0 2024-09-10T10:53:43.852 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.851+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 0x55e502a51680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.851+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 0x7f5e3c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.851+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502addca0 con 0x55e502a63350 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.851+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 0x55e502a51680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.851+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 0x7f5e3c06ccd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e4800c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.852+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 0x55e502a51680 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.854+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v129'877 uv851 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.854+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502a65270 con 0x55e502a63350 2024-09-10T10:53:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.856+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v129'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.856+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502a656a0 con 0x55e502a63350 2024-09-10T10:53:43.859 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.858+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v129'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.858+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9b460 con 0x55e502a63350 2024-09-10T10:53:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.860+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v129'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.860+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9b800 con 0x55e502a63350 2024-09-10T10:53:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.862+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v129'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.862+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9a450 con 0x55e502a63350 2024-09-10T10:53:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.864+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v129'882 uv882 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.864+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9a840 con 0x55e502a63350 2024-09-10T10:53:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.866+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v129'883 uv857 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.866+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ac30 con 0x55e502a63350 2024-09-10T10:53:43.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.868+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v129'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.868+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ac30 con 0x55e502a63350 2024-09-10T10:53:43.870 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:43.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.870+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v129'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.870+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.871+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v129'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.871+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.872+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v129'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.873+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.874+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v129'888 uv888 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.874+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.876+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v129'889 uv864 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.876+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.878+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v129'890 uv890 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.878+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.881 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.880+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v129'891 uv891 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.880+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.882+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v129'892 uv892 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.882+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.883+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v129'893 uv893 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.883+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9abe0 con 0x55e502a63350 2024-09-10T10:53:43.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.886+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v129'894 uv894 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.886+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.887+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v129'895 uv895 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.887+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.888+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v129'896 uv870 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.889+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.891+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v129'897 uv897 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.891+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.893 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:43.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.892+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v129'898 uv898 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.892+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9cbb0 con 0x55e502a63350 2024-09-10T10:53:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.894+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v129'899 uv899 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.894+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9cbb0 con 0x55e502a63350 2024-09-10T10:53:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.896+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v129'900 uv900 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.896+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.898+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v129'901 uv901 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.898+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.899+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v129'902 uv876 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.899+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.901+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v129'903 uv903 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.901+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.903 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:43.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.902+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v129'904 uv904 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.902+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9cbb0 con 0x55e502a63350 2024-09-10T10:53:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.904+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v129'905 uv905 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.904+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9cbb0 con 0x55e502a63350 2024-09-10T10:53:43.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.906+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v129'906 uv906 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.906+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9bba0 con 0x55e502a63350 2024-09-10T10:53:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.907+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v129'907 uv907 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502a63350 2024-09-10T10:53:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.907+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 0x55e502b9e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.907+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9e9e0 con 0x55e502b9c0b0 2024-09-10T10:53:43.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.907+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 0x55e502b9e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.908+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 0x55e502b9e4a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.910+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v129'638 uv621 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.910+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ed80 con 0x55e502b9c0b0 2024-09-10T10:53:43.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.912+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v129'639 uv639 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.912+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9f120 con 0x55e502b9c0b0 2024-09-10T10:53:43.914 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.913+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v129'640 uv640 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.913+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9f4c0 con 0x55e502b9c0b0 2024-09-10T10:53:43.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.915+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v129'641 uv641 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.915+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9f860 con 0x55e502b9c0b0 2024-09-10T10:53:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.916+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v129'642 uv642 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.916+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9fc00 con 0x55e502b9c0b0 2024-09-10T10:53:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.918+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v129'643 uv626 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.918+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9fff0 con 0x55e502b9c0b0 2024-09-10T10:53:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.919+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v129'644 uv644 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.919+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9fff0 con 0x55e502b9c0b0 2024-09-10T10:53:43.921 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.920+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v129'645 uv645 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.921+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.922+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v129'646 uv646 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.922+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.923+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v129'647 uv647 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.923+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.925+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v129'648 uv631 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.925+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.926+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v129'649 uv649 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.926+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.927+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v129'650 uv650 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.928+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.929+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v129'651 uv651 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.929+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.931+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v129'652 uv652 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.931+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.932+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v129'653 uv637 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.932+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.934+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v129'654 uv654 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.934+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502b9ffa0 con 0x55e502b9c0b0 2024-09-10T10:53:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.935+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v129'655 uv655 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.936 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:43.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.935+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba13a0 con 0x55e502b9c0b0 2024-09-10T10:53:43.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.936+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v129'656 uv656 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.936+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba13a0 con 0x55e502b9c0b0 2024-09-10T10:53:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.937+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v129'657 uv657 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.937+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba0070 con 0x55e502b9c0b0 2024-09-10T10:53:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.939+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v129'658 uv658 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502b9c0b0 2024-09-10T10:53:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.939+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55e502ba0970 0x55e502ba2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.939+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba32a0 con 0x55e502ba0970 2024-09-10T10:53:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.939+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55e502ba0970 0x55e502ba2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.940+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55e502ba0970 0x55e502ba2d60 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.942+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v129'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.942+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3680 con 0x55e502ba0970 2024-09-10T10:53:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.943+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v129'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.943+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3b60 con 0x55e502ba0970 2024-09-10T10:53:43.945 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:43.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.944+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v129'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.944+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba5410 con 0x55e502ba0970 2024-09-10T10:53:43.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.945+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v129'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.945+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba57b0 con 0x55e502ba0970 2024-09-10T10:53:43.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.948+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v129'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.948+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba4400 con 0x55e502ba0970 2024-09-10T10:53:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.949+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v129'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.949+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e502ba4d00 0x55e502ba7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.949+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba80a0 con 0x55e502ba4d00 2024-09-10T10:53:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.949+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e502ba4d00 0x55e502ba7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.950+0000 7f5e5aa00640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e502ba4d00 0x55e502ba7b60 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.951+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v129'124 uv123 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.951+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba80a0 con 0x55e502ba4d00 2024-09-10T10:53:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.952+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v129'125 uv125 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.952+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba80a0 con 0x55e502ba4d00 2024-09-10T10:53:43.954 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:43.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.953+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v129'126 uv126 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.953+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.954+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v129'127 uv127 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.954+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.956+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v129'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.956+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.957+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v129'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.957+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.959 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.958+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v129'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.958+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba94a0 con 0x55e502ba0970 2024-09-10T10:53:43.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.959+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v129'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.959+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba94a0 con 0x55e502ba0970 2024-09-10T10:53:43.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.960+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v129'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.960+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.961+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v129'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.961+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.963+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v129'316 uv310 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.963+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.965+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v129'317 uv317 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.965+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.967 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:43.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.966+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v129'318 uv318 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.966+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba94a0 con 0x55e502ba4d00 2024-09-10T10:53:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.967+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v129'319 uv319 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.967+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba94a0 con 0x55e502ba4d00 2024-09-10T10:53:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.969+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v129'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.969+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.971+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v129'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.971+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.972+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v129'322 uv315 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.972+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.973+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v129'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.973+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.974+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v129'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.975+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.976+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v129'325 uv325 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.976+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba4d00 2024-09-10T10:53:43.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.977+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v129'326 uv326 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:43.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.977+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.978+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v129'143 uv142 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.978+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.980+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v129'144 uv144 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.980+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.982 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.981+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v129'145 uv145 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.981+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.982+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v129'146 uv146 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.982+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba8490 con 0x55e502ba0970 2024-09-10T10:53:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.983+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v129'147 uv147 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502ba0970 2024-09-10T10:53:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.983+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e502ba8b10 0x55e502baaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.983+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab480 con 0x55e502ba8b10 2024-09-10T10:53:43.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.983+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e502ba8b10 0x55e502baaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:43.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.984+0000 7f5e5a000640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e502ba8b10 0x55e502baaf00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.985+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v129'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.985+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.987+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v129'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.987+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.989 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:43.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.988+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v129'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.988+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bac860 con 0x55e502ba8b10 2024-09-10T10:53:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.989+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v129'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.989+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bac860 con 0x55e502ba8b10 2024-09-10T10:53:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.991+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v129'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.991+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.992+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v129'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.992+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.993+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v129'284 uv283 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.993+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.995+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v129'285 uv285 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.995+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba8b10 2024-09-10T10:53:43.997 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.996+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v129'286 uv286 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.996+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.997+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v129'287 uv287 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.997+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.999+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v129'288 uv288 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:43.999+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.000+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v129'289 uv282 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.000+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:44.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.001+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v129'290 uv290 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.001+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:44.003 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:44.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.003+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v129'291 uv291 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.003+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502ba3ff0 con 0x55e502ba8b10 2024-09-10T10:53:44.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.004+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v129'292 uv292 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.004+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bae6e0 con 0x55e502ba8b10 2024-09-10T10:53:44.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.005+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v129'293 uv293 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.006+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.007+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v129'124 uv123 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.007+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.008+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v129'125 uv125 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.008+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.010 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:44.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.010+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v129'126 uv126 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.010+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.011+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v129'127 uv127 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.011+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.012+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v129'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.012+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502bab850 con 0x55e502ba4d00 2024-09-10T10:53:44.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.013+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v129'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.013+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502babd00 con 0x55e502ba4d00 2024-09-10T10:53:44.015 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:44.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.014+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v129'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.014+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502babd00 con 0x55e502ba4d00 2024-09-10T10:53:44.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.015+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v129'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.015+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502babd00 con 0x55e502ba4d00 2024-09-10T10:53:44.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.017+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v129'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.017+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502babd00 con 0x55e502ba4d00 2024-09-10T10:53:44.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.018+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v129'176 uv176 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e54003040 con 0x55e502ba4d00 2024-09-10T10:53:44.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.018+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55e502bac600 0x55e502baeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.018+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502bac600 2024-09-10T10:53:44.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.018+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55e502bac600 0x55e502baeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.019+0000 7f5e59600640 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55e502bac600 0x55e502baeda0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.020+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v129'145 uv144 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502bac600 2024-09-10T10:53:44.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.020+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502bac600 2024-09-10T10:53:44.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.021+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v129'146 uv146 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502bac600 2024-09-10T10:53:44.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.021+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502bac600 2024-09-10T10:53:44.023 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:44.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.022+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v129'147 uv147 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502bac600 2024-09-10T10:53:44.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.022+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502bac600 2024-09-10T10:53:44.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.023+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v129'148 uv148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502bac600 2024-09-10T10:53:44.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.023+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502bac600 2024-09-10T10:53:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.024+0000 7f5e59600640 1 -- 172.21.15.28:0/971226626 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v129'149 uv149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f5e48007c80 con 0x55e502bac600 2024-09-10T10:53:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.025+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.026+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v129'351 uv343 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.026+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.027+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v129'352 uv352 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.027+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.029 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:44.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.028+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v129'353 uv353 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.028+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.029+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v129'354 uv354 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.029+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.031+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v129'355 uv355 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.031+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.032+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v129'356 uv350 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.032+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.033+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v129'357 uv357 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.033+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.035 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:44.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.034+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v129'358 uv358 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.034+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bb02f0 con 0x55e502ba8b10 2024-09-10T10:53:44.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.035+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v129'359 uv359 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.035+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bb02f0 con 0x55e502ba8b10 2024-09-10T10:53:44.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.036+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v129'360 uv360 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.036+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502baf2e0 con 0x55e502ba8b10 2024-09-10T10:53:44.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.038+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v129'361 uv361 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.038+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55e502bafa40 con 0x55e502ba8b10 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5a000640 1 -- 172.21.15.28:0/971226626 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v129'362 uv362 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f5e4c06d020 con 0x55e502ba8b10 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55e502ba0970 msgr2=0x55e502ba2d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55e502ba0970 0x55e502ba2d60 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e502ba8b10 msgr2=0x55e502baaf00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55e502ba8b10 0x55e502baaf00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55e502bac600 msgr2=0x55e502baeda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55e502bac600 0x55e502baeda0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e502ba4d00 msgr2=0x55e502ba7b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55e502ba4d00 0x55e502ba7b60 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 msgr2=0x55e502b9e4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 reap_dead start 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 0x55e502b9e4a0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5aa00640 1 -- 172.21.15.28:0/971226626 reap_dead start 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 msgr2=0x55e502a51680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 0x55e502a51680 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 msgr2=0x7f5e3c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 0x7f5e3c06ccd0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5e48000c00 tx=0x7f5e4800c000 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 msgr2=0x55e502a67120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c000c00 tx=0x7f5e4c002fb0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 shutdown_connections 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55e502b9c0b0 0x55e502b9e4a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55e502a63350 0x55e502a51680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e3c06a810 0x7f5e3c06ccd0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 --2- 172.21.15.28:0/971226626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e502a698e0 0x55e502a67120 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 >> 172.21.15.28:0/971226626 conn(0x55e502a6e0a0 msgr2=0x55e502ade310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.039+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 shutdown_connections 2024-09-10T10:53:44.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.040+0000 7f5e5ce3cf40 1 -- 172.21.15.28:0/971226626 wait complete. 2024-09-10T10:53:44.045 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 400.00000000 0 2024-09-10T10:53:44.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.161+0000 7f8ba0fe7f40 1 Processor -- start 2024-09-10T10:53:44.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.161+0000 7f8ba0fe7f40 1 -- start start 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86dfe8e0 0x55ad86e80de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86e8a3e0 0x55ad86e850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ad86dff530 con 0x55ad86e8d9c0 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad86dff290 con 0x55ad86dfe8e0 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ad86e8cee0 con 0x55ad86e8a3e0 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8b9ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8b9ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37444/0 (socket says 172.21.15.28:37444) 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8b9ec00640 1 -- 172.21.15.28:0/3229084461 learned_addr learned my addr 172.21.15.28:0/3229084461 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:44.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8b9d800640 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86e8a3e0 0x55ad86e850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.162+0000 7f8b9e200640 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86dfe8e0 0x55ad86e80de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 -- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86e8a3e0 msgr2=0x55ad86e850f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86e8a3e0 0x55ad86e850f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 -- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86dfe8e0 msgr2=0x55ad86e80de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86dfe8e0 0x55ad86e80de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 -- 172.21.15.28:0/3229084461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad86ce0b30 con 0x55ad86e8d9c0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c460 tx=0x7f8b9800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333b7bacbf7bfedc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b97e00640 1 -- 172.21.15.28:0/3229084461 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980115f0 con 0x55ad86e8d9c0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b97e00640 1 -- 172.21.15.28:0/3229084461 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b98017070 con 0x55ad86e8d9c0 2024-09-10T10:53:44.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.163+0000 7f8b97e00640 1 -- 172.21.15.28:0/3229084461 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980119f0 con 0x55ad86e8d9c0 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/3229084461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 msgr2=0x55ad86e8a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800c460 tx=0x7f8b9800c920 comp rx=0 tx=0).stop 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/3229084461 shutdown_connections 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86e8a3e0 0x55ad86e850f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/3229084461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86dfe8e0 0x55ad86e80de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/3229084461 >> 172.21.15.28:0/3229084461 conn(0x55ad86e030a0 msgr2=0x55ad86e03480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/3229084461 shutdown_connections 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/3229084461 wait complete. 2024-09-10T10:53:44.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.164+0000 7f8ba0fe7f40 1 Processor -- start 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 -- start start 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86dfe8e0 0x55ad86e7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 0x55ad86e79190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ad86e01470 con 0x55ad86e8a3e0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad86e814b0 con 0x55ad86e8d9c0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.165+0000 7f8ba0fe7f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ad86e01750 con 0x55ad86dfe8e0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 0x55ad86e79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 0x55ad86e79190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37448/0 (socket says 172.21.15.28:37448) 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 -- 172.21.15.28:0/2608763940 learned_addr learned my addr 172.21.15.28:0/2608763940 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9e200640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86dfe8e0 0x55ad86e7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 -- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86dfe8e0 msgr2=0x55ad86e7dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86dfe8e0 0x55ad86e7dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 -- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86e8d9c0 msgr2=0x55ad86e8a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9d800640 1 -- 172.21.15.28:0/2608763940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad86e796d0 con 0x55ad86e8a3e0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9e200640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ad86dfe8e0 0x55ad86e7dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.166+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad86e8d9c0 0x55ad86e8a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8b9d800640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 0x55ad86e79190 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e90 tx=0x7f8b8c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8b96a00640 1 -- 172.21.15.28:0/2608763940 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c014400 con 0x55ad86e8a3e0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8b96a00640 1 -- 172.21.15.28:0/2608763940 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c019070 con 0x55ad86e8a3e0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ad86db5e30 con 0x55ad86e8a3e0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8b96a00640 1 -- 172.21.15.28:0/2608763940 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0147c0 con 0x55ad86e8a3e0 2024-09-10T10:53:44.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.167+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad86e7cee0 con 0x55ad86e8a3e0 2024-09-10T10:53:44.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.169+0000 7f8b96a00640 1 -- 172.21.15.28:0/2608763940 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8b8c013070 con 0x55ad86e8a3e0 2024-09-10T10:53:44.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.170+0000 7f8b96a00640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b8006a840 0x7f8b8006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.170+0000 7f8b96a00640 1 -- 172.21.15.28:0/2608763940 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f8b8c08a410 con 0x55ad86e8a3e0 2024-09-10T10:53:44.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.170+0000 7f8b9e200640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b8006a840 0x7f8b8006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.170+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ad86e8b680 0x55ad86f26350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.170+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129) -- 0x55ad86f26890 con 0x55ad86e8b680 2024-09-10T10:53:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.171+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ad86e8b680 0x55ad86f26350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.171+0000 7f8b9e200640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b8006a840 0x7f8b8006cd00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8b90057000 tx=0x7f8b90052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.171+0000 7f8b9ec00640 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ad86e8b680 0x55ad86f26350 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.172+0000 7f8b9ec00640 1 -- 172.21.15.28:0/2608763940 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [truncate 0] v129'150 uv150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f8b98011a10 con 0x55ad86e8b680 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.172+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ad86e8b680 msgr2=0x55ad86f26350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.172+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55ad86e8b680 0x55ad86f26350 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b8006a840 msgr2=0x7f8b8006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8b8006a840 0x7f8b8006cd00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8b90057000 tx=0x7f8b90052000 comp rx=0 tx=0).stop 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 msgr2=0x55ad86e79190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 --2- 172.21.15.28:0/2608763940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ad86e8a3e0 0x55ad86e79190 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009e90 tx=0x7f8b8c009f80 comp rx=0 tx=0).stop 2024-09-10T10:53:44.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8b9ec00640 1 -- 172.21.15.28:0/2608763940 reap_dead start 2024-09-10T10:53:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 shutdown_connections 2024-09-10T10:53:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 >> 172.21.15.28:0/2608763940 conn(0x55ad86e030a0 msgr2=0x55ad86e8be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 shutdown_connections 2024-09-10T10:53:44.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.173+0000 7f8ba0fe7f40 1 -- 172.21.15.28:0/2608763940 wait complete. 2024-09-10T10:53:44.178 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.336+0000 7f9c09600640 1 Processor -- start 2024-09-10T10:53:44.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.336+0000 7f9c09600640 1 -- start start 2024-09-10T10:53:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c09600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9c040cf790 0x7f9c040cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c09600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040b30e0 con 0x7f9c04156740 2024-09-10T10:53:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c09600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c040b33c0 con 0x7f9c040ce060 2024-09-10T10:53:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c09600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9c040d0140 con 0x7f9c040cf790 2024-09-10T10:53:44.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c02a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9c04156740 0x7f9c040b45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47998/0 (socket says 172.21.15.28:47998) 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c02a00640 1 -- 172.21.15.28:0/4240604884 learned_addr learned my addr 172.21.15.28:0/4240604884 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.337+0000 7f9c02000640 1 --2- 172.21.15.28:0/4240604884 >> v2:172.21.15.28:3300/0 conn(0x7f9c040cf790 0x7f9c040cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.338+0000 7f9c01600640 1 -- 172.21.15.28:0/4240604884 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2053266036 0 0) 0x7f9c040b30e0 con 0x7f9c04156740 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.338+0000 7f9c01600640 1 -- 172.21.15.28:0/4240604884 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be8003520 con 0x7f9c04156740 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.338+0000 7f9c02000640 1 -- 172.21.15.28:0/4240604884 >> v1:172.21.15.138:6789/0 conn(0x7f9c040ce060 legacy=0x7f9c040ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.338+0000 7f9c02000640 1 -- 172.21.15.28:0/4240604884 >> v1:172.21.15.28:6789/0 conn(0x7f9c04156740 legacy=0x7f9c040b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.338+0000 7f9c02000640 1 -- 172.21.15.28:0/4240604884 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c040b36a0 con 0x7f9c040cf790 2024-09-10T10:53:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.339+0000 7f9c02000640 1 --2- 172.21.15.28:0/4240604884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040cfc00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b115816e317576a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.339+0000 7f9c01600640 1 -- 172.21.15.28:0/4240604884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0037070 con 0x7f9c040cf790 2024-09-10T10:53:44.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.339+0000 7f9c01600640 1 -- 172.21.15.28:0/4240604884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf0032440 con 0x7f9c040cf790 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.339+0000 7f9c01600640 1 -- 172.21.15.28:0/4240604884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf00327f0 con 0x7f9c040cf790 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 -- 172.21.15.28:0/4240604884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 msgr2=0x7f9c040cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 --2- 172.21.15.28:0/4240604884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040cfc00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf002a230 comp rx=0 tx=0).stop 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 -- 172.21.15.28:0/4240604884 shutdown_connections 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 --2- 172.21.15.28:0/4240604884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040cfc00 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 -- 172.21.15.28:0/4240604884 >> 172.21.15.28:0/4240604884 conn(0x7f9c0406bdf0 msgr2=0x7f9c0406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 -- 172.21.15.28:0/4240604884 shutdown_connections 2024-09-10T10:53:44.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.340+0000 7f9c09600640 1 -- 172.21.15.28:0/4240604884 wait complete. 2024-09-10T10:53:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.341+0000 7f9c09600640 1 Processor -- start 2024-09-10T10:53:44.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 -- start start 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 0x7f9c041570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c04156740 0x7f9c041f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c040b3c60 con 0x7f9c040cf790 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0406edc0 con 0x7f9c04156740 2024-09-10T10:53:44.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.342+0000 7f9c09600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9c040b4cc0 con 0x7f9c040ce060 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37466/0 (socket says 172.21.15.28:37466) 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 -- 172.21.15.28:0/831965764 learned_addr learned my addr 172.21.15.28:0/831965764 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c03400640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c04156740 0x7f9c041f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02a00640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 0x7f9c041570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 -- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 msgr2=0x7f9c041570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 0x7f9c041570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 -- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c04156740 msgr2=0x7f9c041f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c04156740 0x7f9c041f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02000640 1 -- 172.21.15.28:0/831965764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0414d230 con 0x7f9c040cf790 2024-09-10T10:53:44.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.343+0000 7f9c02a00640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 0x7f9c041570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9c02000640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0037050 con 0x7f9c040cf790 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf0043070 con 0x7f9c040cf790 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0032810 con 0x7f9c040cf790 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0414d510 con 0x7f9c040cf790 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.344+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c040729d0 con 0x7f9c040cf790 2024-09-10T10:53:44.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.345+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c040d0920 con 0x7f9c040cf790 2024-09-10T10:53:44.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.346+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9bf00329b0 con 0x7f9c040cf790 2024-09-10T10:53:44.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.347+0000 7f9bfaa00640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 0x7f9be406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.347+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8798+0+0 (secure 0 0 0) 0x7f9bf00ac700 con 0x7f9c040cf790 2024-09-10T10:53:44.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.348+0000 7f9c02a00640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 0x7f9be406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.348+0000 7f9c02a00640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 0x7f9be406e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011e0 tx=0x7f9bf4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.352+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9bf007a490 con 0x7f9c040cf790 2024-09-10T10:53:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.577+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f9c0415a3a0 con 0x7f9c040cf790 2024-09-10T10:53:44.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.750+0000 7f9bfaa00640 1 -- 172.21.15.28:0/831965764 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v256) ==== 162+0+0 (secure 0 0 0) 0x7f9bf0079d90 con 0x7f9c040cf790 2024-09-10T10:53:44.751 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:44.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.752+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 msgr2=0x7f9be406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.752+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 0x7f9be406e2f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40011e0 tx=0x7f9bf4009040 comp rx=0 tx=0).stop 2024-09-10T10:53:44.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.752+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 msgr2=0x7f9c040731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf00046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 shutdown_connections 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9be406be30 0x7f9be406e2f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c04156740 0x7f9c041f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c040cf790 0x7f9c040731d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 --2- 172.21.15.28:0/831965764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9c040ce060 0x7f9c041570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 >> 172.21.15.28:0/831965764 conn(0x7f9c0406bdf0 msgr2=0x7f9c0406ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 shutdown_connections 2024-09-10T10:53:44.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.753+0000 7f9c09600640 1 -- 172.21.15.28:0/831965764 wait complete. 2024-09-10T10:53:44.760 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:53:44.759+0000 7f4393000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T10:53:44.760 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:44.760+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:44.761 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:44.760+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:44.773 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.930+0000 7fa694400640 1 Processor -- start 2024-09-10T10:53:44.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.930+0000 7fa694400640 1 -- start start 2024-09-10T10:53:44.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.931+0000 7fa694400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa68c072de0 0x7fa68c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.931+0000 7fa694400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68c146140 con 0x7fa68c072700 2024-09-10T10:53:44.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.931+0000 7fa694400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68c0cd740 con 0x7fa68c072320 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.931+0000 7fa694400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa68c0cda20 con 0x7fa68c072de0 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.931+0000 7fa691000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa68c072de0 0x7fa68c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.932+0000 7fa692400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa68c072700 0x7fa68c15d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48008/0 (socket says 172.21.15.28:48008) 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.932+0000 7fa692400640 1 -- 172.21.15.28:0/915887981 learned_addr learned my addr 172.21.15.28:0/915887981 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.932+0000 7fa691000640 1 -- 172.21.15.28:0/915887981 >> v1:172.21.15.138:6789/0 conn(0x7fa68c072320 legacy=0x7fa68c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.932+0000 7fa691000640 1 -- 172.21.15.28:0/915887981 >> v1:172.21.15.28:6789/0 conn(0x7fa68c072700 legacy=0x7fa68c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.932+0000 7fa691000640 1 -- 172.21.15.28:0/915887981 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c0cdd00 con 0x7fa68c072de0 2024-09-10T10:53:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.933+0000 7fa691000640 1 --2- 172.21.15.28:0/915887981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072de0 0x7fa68c15d1e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa6780097a0 tx=0x7fa67802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2239971fdcfb7402 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.933+0000 7fa68be00640 1 -- 172.21.15.28:0/915887981 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678037070 con 0x7fa68c072de0 2024-09-10T10:53:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.933+0000 7fa68be00640 1 -- 172.21.15.28:0/915887981 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa678032440 con 0x7fa68c072de0 2024-09-10T10:53:44.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.933+0000 7fa68be00640 1 -- 172.21.15.28:0/915887981 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678032810 con 0x7fa68c072de0 2024-09-10T10:53:44.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 -- 172.21.15.28:0/915887981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072de0 msgr2=0x7fa68c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 --2- 172.21.15.28:0/915887981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072de0 0x7fa68c15d1e0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa6780097a0 tx=0x7fa67802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 -- 172.21.15.28:0/915887981 shutdown_connections 2024-09-10T10:53:44.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 --2- 172.21.15.28:0/915887981 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072de0 0x7fa68c15d1e0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 -- 172.21.15.28:0/915887981 >> 172.21.15.28:0/915887981 conn(0x7fa68c067d70 msgr2=0x7fa68c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:44.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 -- 172.21.15.28:0/915887981 shutdown_connections 2024-09-10T10:53:44.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.934+0000 7fa694400640 1 -- 172.21.15.28:0/915887981 wait complete. 2024-09-10T10:53:44.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.936+0000 7fa694400640 1 Processor -- start 2024-09-10T10:53:44.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.936+0000 7fa694400640 1 -- start start 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.936+0000 7fa694400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa694400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68c072700 0x7fa68c13e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa694400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 0x7fa68c13ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa694400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c15dfb0 con 0x7fa68c072320 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa694400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa68c15de30 con 0x7fa68c072de0 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa694400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa68c15e130 con 0x7fa68c072700 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37484/0 (socket says 172.21.15.28:37484) 2024-09-10T10:53:44.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 -- 172.21.15.28:0/3005594419 learned_addr learned my addr 172.21.15.28:0/3005594419 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:44.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa692400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 0x7fa68c13ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 -- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68c072700 msgr2=0x7fa68c13e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68c072700 0x7fa68c13e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 -- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 msgr2=0x7fa68c13ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 0x7fa68c13ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa691a00640 1 -- 172.21.15.28:0/3005594419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68c0c5eb0 con 0x7fa68c072320 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.937+0000 7fa692400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 0x7fa68c13ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa691a00640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c430 tx=0x7fa68000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6800052d0 con 0x7fa68c072320 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa680005490 con 0x7fa68c072320 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680007260 con 0x7fa68c072320 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68c143350 con 0x7fa68c072320 2024-09-10T10:53:44.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.938+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa68c1f3c80 con 0x7fa68c072320 2024-09-10T10:53:44.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.939+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68c143870 con 0x7fa68c072320 2024-09-10T10:53:44.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.940+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa680007400 con 0x7fa68c072320 2024-09-10T10:53:44.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.941+0000 7fa689600640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 0x7fa66c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:44.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.941+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7fa68008b100 con 0x7fa68c072320 2024-09-10T10:53:44.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.942+0000 7fa691000640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 0x7fa66c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:44.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.942+0000 7fa691000640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 0x7fa66c06e2f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa678034000 tx=0x7fa67802c000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:44.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:44.945+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa68c143870 con 0x7fa68c072320 2024-09-10T10:53:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.165+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa68c0cce60 con 0x7fa68c072320 2024-09-10T10:53:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.165+0000 7fa689600640 1 -- 172.21.15.28:0/3005594419 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) ==== 95+0+3566 (secure 0 0 0) 0x7fa680058d90 con 0x7fa68c072320 2024-09-10T10:53:45.167 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:45.167 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":258,"btime":"2024-09-10T10:53:44:760776+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:44.760711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 258 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 msgr2=0x7fa66c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 0x7fa66c06e2f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa678034000 tx=0x7fa67802c000 comp rx=0 tx=0).stop 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 msgr2=0x7fa68c0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c430 tx=0x7fa68000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 shutdown_connections 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66c06be30 0x7fa66c06e2f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa68c072de0 0x7fa68c13ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68c072700 0x7fa68c13e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 --2- 172.21.15.28:0/3005594419 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa68c072320 0x7fa68c0b1670 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 >> 172.21.15.28:0/3005594419 conn(0x7fa68c067d70 msgr2=0x7fa68c0b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 shutdown_connections 2024-09-10T10:53:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.168+0000 7fa694400640 1 -- 172.21.15.28:0/3005594419 wait complete. 2024-09-10T10:53:45.185 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:45.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:45.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.341+0000 7fa6ba600640 1 Processor -- start 2024-09-10T10:53:45.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.341+0000 7fa6ba600640 1 -- start start 2024-09-10T10:53:45.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6ba600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6b415bef0 0x7fa6b40b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6ba600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40af380 0x7fa6b4071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6ba600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b40b5470 con 0x7fa6b40b25c0 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6ba600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa6b40b3640 con 0x7fa6b415bef0 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6ba600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b40b5190 con 0x7fa6b40af380 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6b9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6b415bef0 0x7fa6b40b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6b9200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6b415bef0 0x7fa6b40b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37492/0 (socket says 172.21.15.28:37492) 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6b9200640 1 -- 172.21.15.28:0/3017045530 learned_addr learned my addr 172.21.15.28:0/3017045530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.342+0000 7fa6b9c00640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40af380 0x7fa6b4071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.343+0000 7fa6b9200640 1 -- 172.21.15.28:0/3017045530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40af380 msgr2=0x7fa6b4071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.343+0000 7fa6b9200640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40af380 0x7fa6b4071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.343+0000 7fa6b9200640 1 -- 172.21.15.28:0/3017045530 >> v1:172.21.15.138:6789/0 conn(0x7fa6b40b25c0 legacy=0x7fa6b40aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.343+0000 7fa6b9200640 1 -- 172.21.15.28:0/3017045530 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b40b38e0 con 0x7fa6b415bef0 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6b9200640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b415bef0 0x7fa6b40b40e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009870 tx=0x7fa6a802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22d8dcfb0bee006 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6b3400640 1 -- 172.21.15.28:0/3017045530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8037070 con 0x7fa6b415bef0 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6b3400640 1 -- 172.21.15.28:0/3017045530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a8032440 con 0x7fa6b415bef0 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6b3400640 1 -- 172.21.15.28:0/3017045530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8032810 con 0x7fa6b415bef0 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6ba600640 1 -- 172.21.15.28:0/3017045530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b415bef0 msgr2=0x7fa6b40b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.344+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b415bef0 0x7fa6b40b40e0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009870 tx=0x7fa6a802a130 comp rx=0 tx=0).stop 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 -- 172.21.15.28:0/3017045530 shutdown_connections 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40af380 0x7fa6b4071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3017045530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b415bef0 0x7fa6b40b40e0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 -- 172.21.15.28:0/3017045530 >> 172.21.15.28:0/3017045530 conn(0x7fa6b406c240 msgr2=0x7fa6b40b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 -- 172.21.15.28:0/3017045530 shutdown_connections 2024-09-10T10:53:45.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.345+0000 7fa6ba600640 1 -- 172.21.15.28:0/3017045530 wait complete. 2024-09-10T10:53:45.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.346+0000 7fa6ba600640 1 Processor -- start 2024-09-10T10:53:45.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.346+0000 7fa6ba600640 1 -- start start 2024-09-10T10:53:45.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 0x7fa6b415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 0x7fa6b4154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40b4b10 con 0x7fa6b40af380 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b40b4e60 con 0x7fa6b415bef0 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.347+0000 7fa6ba600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b40b2a30 con 0x7fa6b40b25c0 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37498/0 (socket says 172.21.15.28:37498) 2024-09-10T10:53:45.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 -- 172.21.15.28:0/3758769629 learned_addr learned my addr 172.21.15.28:0/3758769629 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:45.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 -- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 msgr2=0x7fa6b415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b3e00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 0x7fa6b415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9c00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 0x7fa6b4154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 0x7fa6b415cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 -- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 msgr2=0x7fa6b4154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 0x7fa6b4154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 -- 172.21.15.28:0/3758769629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b415ddb0 con 0x7fa6b40af380 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b3e00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 0x7fa6b415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9c00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 0x7fa6b4154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.348+0000 7fa6b9200640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000c00 tx=0x7fa6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.349+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8037040 con 0x7fa6b40af380 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.349+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a8042070 con 0x7fa6b40af380 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.349+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8030980 con 0x7fa6b40af380 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.349+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b415e090 con 0x7fa6b40af380 2024-09-10T10:53:45.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.349+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b415e650 con 0x7fa6b40af380 2024-09-10T10:53:45.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.350+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b415c2d0 con 0x7fa6b40af380 2024-09-10T10:53:45.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.351+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6a8030b20 con 0x7fa6b40af380 2024-09-10T10:53:45.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.352+0000 7fa6b0c00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 0x7fa69406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.352+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7fa6a80ac380 con 0x7fa6b40af380 2024-09-10T10:53:45.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.352+0000 7fa6b3e00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 0x7fa69406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.353+0000 7fa6b3e00640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 0x7fa69406e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ce6c0 tx=0x7fa6a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.356+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa6a807a100 con 0x7fa6b40af380 2024-09-10T10:53:45.481 INFO:tasks.ceph.mds.c.smithi028.stderr: -45> 2024-09-10T10:53:40.728+0000 7f6335e00640 -1 log_channel(cluster) log [ERR] : error decoding table object 'mds0_inotable': End of buffer [buffer:2] 2024-09-10T10:53:45.521 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:45.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.579+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b4158100 con 0x7fa6b40af380 2024-09-10T10:53:45.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.579+0000 7fa6b0c00640 1 -- 172.21.15.28:0/3758769629 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 258 v258) ==== 95+0+3566 (secure 0 0 0) 0x7fa6a8079a00 con 0x7fa6b40af380 2024-09-10T10:53:45.581 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:45.581 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":258,"btime":"2024-09-10T10:53:44:760776+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:44.760711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:45.581 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 258 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 msgr2=0x7fa69406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 0x7fa69406e2f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40ce6c0 tx=0x7fa6a0009040 comp rx=0 tx=0).stop 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 msgr2=0x7fa6b415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000c00 tx=0x7fa6a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 shutdown_connections 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa69406be30 0x7fa69406e2f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6b415bef0 0x7fa6b4154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6b40b25c0 0x7fa6b415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 --2- 172.21.15.28:0/3758769629 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6b40af380 0x7fa6b415eb70 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.581+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 >> 172.21.15.28:0/3758769629 conn(0x7fa6b406c240 msgr2=0x7fa6b40aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:45.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.582+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 shutdown_connections 2024-09-10T10:53:45.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.582+0000 7fa6ba600640 1 -- 172.21.15.28:0/3758769629 wait complete. 2024-09-10T10:53:45.598 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:45.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:45.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.754+0000 7f4873200640 1 Processor -- start 2024-09-10T10:53:45.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.754+0000 7f4873200640 1 -- start start 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4873200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f487415e660 0x7f487406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4873200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48740d1f90 0x7f48740cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4873200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48740cf000 con 0x7f48740d0dd0 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4873200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f48740b37d0 con 0x7f487415e660 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4873200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f48740cf330 con 0x7f48740d1f90 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.755+0000 7f4871400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f487415e660 0x7f487406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f48740d0dd0 0x7f48740d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48028/0 (socket says 172.21.15.28:48028) 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871e00640 1 -- 172.21.15.28:0/816640499 learned_addr learned my addr 172.21.15.28:0/816640499 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:45.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4872800640 1 --2- 172.21.15.28:0/816640499 >> v2:172.21.15.138:3300/0 conn(0x7f48740d1f90 0x7f48740cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871400640 1 -- 172.21.15.28:0/816640499 >> v2:172.21.15.138:3300/0 conn(0x7f48740d1f90 msgr2=0x7f48740cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871400640 1 --2- 172.21.15.28:0/816640499 >> v2:172.21.15.138:3300/0 conn(0x7f48740d1f90 0x7f48740cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871400640 1 -- 172.21.15.28:0/816640499 >> v1:172.21.15.28:6789/0 conn(0x7f48740d0dd0 legacy=0x7f48740d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4870a00640 1 -- 172.21.15.28:0/816640499 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2003369100 0 0) 0x7f48740cf000 con 0x7f48740d0dd0 2024-09-10T10:53:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.756+0000 7f4871400640 1 -- 172.21.15.28:0/816640499 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740d29a0 con 0x7f487415e660 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.757+0000 7f4871400640 1 --2- 172.21.15.28:0/816640499 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f487415e660 0x7f487406ccd0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f48740cb420 tx=0x7f486002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cca1cc7bf280f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.757+0000 7f4870a00640 1 -- 172.21.15.28:0/816640499 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860037070 con 0x7f487415e660 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.757+0000 7f4870a00640 1 -- 172.21.15.28:0/816640499 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4860032440 con 0x7f487415e660 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.757+0000 7f4870a00640 1 -- 172.21.15.28:0/816640499 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4860032810 con 0x7f487415e660 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 -- 172.21.15.28:0/816640499 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f487415e660 msgr2=0x7f487406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 --2- 172.21.15.28:0/816640499 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f487415e660 0x7f487406ccd0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f48740cb420 tx=0x7f486002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 -- 172.21.15.28:0/816640499 shutdown_connections 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 --2- 172.21.15.28:0/816640499 >> v2:172.21.15.138:3300/0 conn(0x7f48740d1f90 0x7f48740cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 --2- 172.21.15.28:0/816640499 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f487415e660 0x7f487406ccd0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 -- 172.21.15.28:0/816640499 >> 172.21.15.28:0/816640499 conn(0x7f487408bfc0 msgr2=0x7f48740d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 -- 172.21.15.28:0/816640499 shutdown_connections 2024-09-10T10:53:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.758+0000 7f4873200640 1 -- 172.21.15.28:0/816640499 wait complete. 2024-09-10T10:53:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.760+0000 7f4873200640 1 Processor -- start 2024-09-10T10:53:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.760+0000 7f4873200640 1 -- start start 2024-09-10T10:53:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48740d1f90 0x7f48740cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487415e660 0x7f48740b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48740d3220 con 0x7f48740d0dd0 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48740d2f40 con 0x7f487415e660 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4873200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48740d33a0 con 0x7f48740d1f90 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4871e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4871e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37552/0 (socket says 172.21.15.28:37552) 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4871e00640 1 -- 172.21.15.28:0/3406727479 learned_addr learned my addr 172.21.15.28:0/3406727479 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4872800640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487415e660 0x7f48740b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.761+0000 7f4871400640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48740d1f90 0x7f48740cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 -- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48740d1f90 msgr2=0x7f48740cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48740d1f90 0x7f48740cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 -- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487415e660 msgr2=0x7f48740b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487415e660 0x7f48740b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 -- 172.21.15.28:0/3406727479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48741571e0 con 0x7f48740d0dd0 2024-09-10T10:53:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4871e00640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f486400c4d0 tx=0x7f486400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864018070 con 0x7f48740d0dd0 2024-09-10T10:53:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4864007500 con 0x7f48740d0dd0 2024-09-10T10:53:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48740b8d50 con 0x7f48740d0dd0 2024-09-10T10:53:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864004a20 con 0x7f48740d0dd0 2024-09-10T10:53:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.762+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48740b8a70 con 0x7f48740d0dd0 2024-09-10T10:53:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.764+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48740d2440 con 0x7f48740d0dd0 2024-09-10T10:53:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.764+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f486401b070 con 0x7f48740d0dd0 2024-09-10T10:53:45.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.765+0000 7f486a000640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:45.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.766+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7f486408ae40 con 0x7f48740d0dd0 2024-09-10T10:53:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.766+0000 7f4871400640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.767+0000 7f4871400640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 0x7f485406e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4860034000 tx=0x7f486002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:45.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.771+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4864058b60 con 0x7f48740d0dd0 2024-09-10T10:53:45.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.989+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48740d1b20 con 0x7f48740d0dd0 2024-09-10T10:53:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.990+0000 7f486a000640 1 -- 172.21.15.28:0/3406727479 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 259 v259) ==== 95+0+4389 (secure 0 0 0) 0x7f4864058460 con 0x7f48740d0dd0 2024-09-10T10:53:45.992 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":259,"btime":"2024-09-10T10:53:45:762710+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9474,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/208022459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":208022459},{"type":"v1","addr":"172.21.15.28:6834","nonce":208022459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":259}],"filesystems":[{"mdsmap":{"epoch":258,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:44.760711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":130,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:45.992 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 259 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.992+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 msgr2=0x7f485406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.992+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 0x7f485406e2f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4860034000 tx=0x7f486002e000 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.992+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 msgr2=0x7f4874156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.992+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f486400c4d0 tx=0x7f486400c990 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 shutdown_connections 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f485406be30 0x7f485406e2f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487415e660 0x7f48740b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48740d1f90 0x7f48740cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 --2- 172.21.15.28:0/3406727479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48740d0dd0 0x7f4874156ca0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:45.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 >> 172.21.15.28:0/3406727479 conn(0x7f487408bfc0 msgr2=0x7f48740b4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:45.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 shutdown_connections 2024-09-10T10:53:45.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:45.993+0000 7f4873200640 1 -- 172.21.15.28:0/3406727479 wait complete. 2024-09-10T10:53:46.011 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 400.00000000' led to DAMAGED state 2024-09-10T10:53:46.011 INFO:tasks.cephfs.test_damage:Applying mutation 'Truncate 100.00000000.inode' 2024-09-10T10:53:46.011 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:46.011 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:46.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:46.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:46.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:46.047 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:46.121 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:46.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:46.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:46.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.318+0000 7fbd46200640 1 Processor -- start 2024-09-10T10:53:46.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.318+0000 7fbd46200640 1 -- start start 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.319+0000 7fbd46200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072320 0x7fbd400cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.319+0000 7fbd46200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd40072de0 0x7fbd4006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.319+0000 7fbd46200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd40146140 con 0x7fbd40072700 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd46200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd400cd740 con 0x7fbd40072de0 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd46200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd400cda20 con 0x7fbd40072320 2024-09-10T10:53:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd40072de0 0x7fbd4006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd40072de0 0x7fbd4006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37568/0 (socket says 172.21.15.28:37568) 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 -- 172.21.15.28:0/2970607046 learned_addr learned my addr 172.21.15.28:0/2970607046 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3f400640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072320 0x7fbd400cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 -- 172.21.15.28:0/2970607046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072320 msgr2=0x7fbd400cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072320 0x7fbd400cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 -- 172.21.15.28:0/2970607046 >> v1:172.21.15.138:6789/0 conn(0x7fbd40072700 legacy=0x7fbd4006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.320+0000 7fbd3ea00640 1 -- 172.21.15.28:0/2970607046 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd400cdd00 con 0x7fbd40072de0 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.321+0000 7fbd3ea00640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd4006e0b0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009870 tx=0x7fbd2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f069cddb22f05b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd3e000640 1 -- 172.21.15.28:0/2970607046 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c037070 con 0x7fbd40072de0 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd3e000640 1 -- 172.21.15.28:0/2970607046 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2c032440 con 0x7fbd40072de0 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd3e000640 1 -- 172.21.15.28:0/2970607046 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c032810 con 0x7fbd40072de0 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 -- 172.21.15.28:0/2970607046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 msgr2=0x7fbd4006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd4006e0b0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009870 tx=0x7fbd2c02a180 comp rx=0 tx=0).stop 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 -- 172.21.15.28:0/2970607046 shutdown_connections 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd4006e0b0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 --2- 172.21.15.28:0/2970607046 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072320 0x7fbd400cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 -- 172.21.15.28:0/2970607046 >> 172.21.15.28:0/2970607046 conn(0x7fbd40067d70 msgr2=0x7fbd4006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 -- 172.21.15.28:0/2970607046 shutdown_connections 2024-09-10T10:53:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.322+0000 7fbd46200640 1 -- 172.21.15.28:0/2970607046 wait complete. 2024-09-10T10:53:46.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.324+0000 7fbd46200640 1 Processor -- start 2024-09-10T10:53:46.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.324+0000 7fbd46200640 1 -- start start 2024-09-10T10:53:46.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd40072320 0x7fbd4013f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072700 0x7fbd401e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd400cd520 con 0x7fbd40072de0 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd400cd3a0 con 0x7fbd40072320 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd46200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4006ed00 con 0x7fbd40072700 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37584/0 (socket says 172.21.15.28:37584) 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd3fe00640 1 -- 172.21.15.28:0/3428361537 learned_addr learned my addr 172.21.15.28:0/3428361537 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd3f400640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd40072320 0x7fbd4013f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.325+0000 7fbd3ea00640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072700 0x7fbd401e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 -- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072700 msgr2=0x7fbd401e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072700 0x7fbd401e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 -- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd40072320 msgr2=0x7fbd4013f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd40072320 0x7fbd4013f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 -- 172.21.15.28:0/3428361537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd401428d0 con 0x7fbd40072de0 2024-09-10T10:53:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd3fe00640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c780 tx=0x7fbd3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38004b60 con 0x7fbd40072de0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd38004d00 con 0x7fbd40072de0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3800aa90 con 0x7fbd40072de0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd401f75f0 con 0x7fbd40072de0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.326+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd401f7bb0 con 0x7fbd40072de0 2024-09-10T10:53:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.328+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd400cce60 con 0x7fbd40072de0 2024-09-10T10:53:46.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.328+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd3800ac30 con 0x7fbd40072de0 2024-09-10T10:53:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.330+0000 7fbd37400640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 0x7fbd2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.330+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7fbd3808bc00 con 0x7fbd40072de0 2024-09-10T10:53:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.330+0000 7fbd3f400640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 0x7fbd2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.331+0000 7fbd3f400640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 0x7fbd2006e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbd30006fd0 tx=0x7fbd30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.334+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd38020070 con 0x7fbd40072de0 2024-09-10T10:53:46.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.554+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbd4006bf90 con 0x7fbd40072de0 2024-09-10T10:53:46.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.770+0000 7fbd37400640 1 -- 172.21.15.28:0/3428361537 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v260) ==== 161+0+0 (secure 0 0 0) 0x7fbd380592d0 con 0x7fbd40072de0 2024-09-10T10:53:46.771 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.772+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 msgr2=0x7fbd2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.772+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 0x7fbd2006e2f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbd30006fd0 tx=0x7fbd30008040 comp rx=0 tx=0).stop 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.772+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 msgr2=0x7fbd401e9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.772+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c780 tx=0x7fbd3800cc40 comp rx=0 tx=0).stop 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 shutdown_connections 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbd2006be30 0x7fbd2006e2f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd40072de0 0x7fbd401e9810 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd40072700 0x7fbd401e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 --2- 172.21.15.28:0/3428361537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd40072320 0x7fbd4013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 >> 172.21.15.28:0/3428361537 conn(0x7fbd40067d70 msgr2=0x7fbd400cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 shutdown_connections 2024-09-10T10:53:46.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.773+0000 7fbd46200640 1 -- 172.21.15.28:0/3428361537 wait complete. 2024-09-10T10:53:46.790 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:46.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.946+0000 7f0069a00640 1 Processor -- start 2024-09-10T10:53:46.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.946+0000 7f0069a00640 1 -- start start 2024-09-10T10:53:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.947+0000 7f0069a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0064072320 0x7f00640cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.947+0000 7f0069a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 0x7f006406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0069a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0064146140 con 0x7f0064072700 2024-09-10T10:53:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0069a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f00640cd740 con 0x7f0064072320 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0069a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00640cda20 con 0x7f0064072de0 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0064072320 0x7f00640cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0064072320 0x7f00640cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37590/0 (socket says 172.21.15.28:37590) 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 -- 172.21.15.28:0/3615662837 learned_addr learned my addr 172.21.15.28:0/3615662837 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f005a000640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 0x7f006406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 -- 172.21.15.28:0/3615662837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 msgr2=0x7f006406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 0x7f006406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 -- 172.21.15.28:0/3615662837 >> v1:172.21.15.138:6789/0 conn(0x7f0064072700 legacy=0x7f006406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f0062a00640 1 -- 172.21.15.28:0/3615662837 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00640cdd00 con 0x7f0064072320 2024-09-10T10:53:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.948+0000 7f005a000640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 0x7f006406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:46.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.949+0000 7f0062a00640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00640cce60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0054009870 tx=0x7f005402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b47c9af5f32606 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0062000640 1 -- 172.21.15.28:0/3615662837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054037070 con 0x7f0064072320 2024-09-10T10:53:46.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0062000640 1 -- 172.21.15.28:0/3615662837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054032440 con 0x7f0064072320 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0062000640 1 -- 172.21.15.28:0/3615662837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054032810 con 0x7f0064072320 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 -- 172.21.15.28:0/3615662837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 msgr2=0x7f00640cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00640cce60 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0054009870 tx=0x7f005402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 -- 172.21.15.28:0/3615662837 shutdown_connections 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072de0 0x7f006406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 --2- 172.21.15.28:0/3615662837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00640cce60 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 -- 172.21.15.28:0/3615662837 >> 172.21.15.28:0/3615662837 conn(0x7f0064067d70 msgr2=0x7f006406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 -- 172.21.15.28:0/3615662837 shutdown_connections 2024-09-10T10:53:46.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.950+0000 7f0069a00640 1 -- 172.21.15.28:0/3615662837 wait complete. 2024-09-10T10:53:46.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.952+0000 7f0069a00640 1 Processor -- start 2024-09-10T10:53:46.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.952+0000 7f0069a00640 1 -- start start 2024-09-10T10:53:46.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 0x7f00640cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 0x7f00640b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00640cd520 con 0x7f0064072320 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00640cd3a0 con 0x7f0064072de0 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0069a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f006406ed00 con 0x7f0064072700 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0062a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0062a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37600/0 (socket says 172.21.15.28:37600) 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.953+0000 7f0062a00640 1 -- 172.21.15.28:0/3459071434 learned_addr learned my addr 172.21.15.28:0/3459071434 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 -- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 msgr2=0x7f00640cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0063400640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 0x7f00640b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f005a000640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 0x7f00640cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 0x7f00640cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 -- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 msgr2=0x7f00640b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 0x7f00640b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 -- 172.21.15.28:0/3459071434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0064143c80 con 0x7f0064072320 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0063400640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 0x7f00640b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f005a000640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 0x7f00640cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:46.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0062a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f005402a200 tx=0x7f00540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054037040 con 0x7f0064072320 2024-09-10T10:53:46.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054042070 con 0x7f0064072320 2024-09-10T10:53:46.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00641f80a0 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054030bd0 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.954+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641f8660 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.955+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00641f88a0 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.956+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f005402ad10 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.957+0000 7f005b400640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.957+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7f00540ac490 con 0x7f0064072320 2024-09-10T10:53:46.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.958+0000 7f005a000640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:46.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.958+0000 7f005a000640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 0x7f004006e2f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f004c006fd0 tx=0x7f004c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:46.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:46.962+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00641f88a0 con 0x7f0064072320 2024-09-10T10:53:47.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.217+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f00640cce60 con 0x7f0064072320 2024-09-10T10:53:47.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.776+0000 7f005b400640 1 -- 172.21.15.28:0/3459071434 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v261) ==== 101+0+0 (secure 0 0 0) 0x7f005407a180 con 0x7f0064072320 2024-09-10T10:53:47.778 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:47.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 msgr2=0x7f004006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 0x7f004006e2f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f004c006fd0 tx=0x7f004c008040 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 msgr2=0x7f00641f79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f005402a200 tx=0x7f00540046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 shutdown_connections 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f004006be30 0x7f004006e2f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0064072de0 0x7f00640b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0064072700 0x7f00640cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 --2- 172.21.15.28:0/3459071434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0064072320 0x7f00641f79c0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 >> 172.21.15.28:0/3459071434 conn(0x7f0064067d70 msgr2=0x7f00641e82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 shutdown_connections 2024-09-10T10:53:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.779+0000 7f0069a00640 1 -- 172.21.15.28:0/3459071434 wait complete. 2024-09-10T10:53:47.797 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:47.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.872+0000 7eff5d325f40 1 Processor -- start 2024-09-10T10:53:47.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 -- start start 2024-09-10T10:53:47.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e9763e0 0x55c57e8d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9799c0 0x55c57e8d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c57e848b30 con 0x55c57e8a0470 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c57e84cb70 con 0x55c57e9799c0 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.873+0000 7eff5d325f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c57e978ee0 con 0x55c57e9763e0 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37624/0 (socket says 172.21.15.28:37624) 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 -- 172.21.15.28:0/3639079395 learned_addr learned my addr 172.21.15.28:0/3639079395 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff5a600640 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9799c0 0x55c57e8d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59200640 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e9763e0 0x55c57e8d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 -- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e9763e0 msgr2=0x55c57e8d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e9763e0 0x55c57e8d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 -- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9799c0 msgr2=0x55c57e8d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:47.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9799c0 0x55c57e8d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 -- 172.21.15.28:0/3639079395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e977f80 con 0x55c57e8a0470 2024-09-10T10:53:47.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff59c00640 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b2e0 tx=0x7eff4c00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad3e1662d96f84d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.874+0000 7eff53400640 1 -- 172.21.15.28:0/3639079395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00e5c0 con 0x55c57e8a0470 2024-09-10T10:53:47.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff53400640 1 -- 172.21.15.28:0/3639079395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4c006070 con 0x55c57e8a0470 2024-09-10T10:53:47.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff53400640 1 -- 172.21.15.28:0/3639079395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c00e9a0 con 0x55c57e8a0470 2024-09-10T10:53:47.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 -- 172.21.15.28:0/3639079395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 msgr2=0x55c57e971a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b2e0 tx=0x7eff4c00b7a0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 -- 172.21.15.28:0/3639079395 shutdown_connections 2024-09-10T10:53:47.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9799c0 0x55c57e8d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e9763e0 0x55c57e8d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 --2- 172.21.15.28:0/3639079395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e8a0470 0x55c57e971a60 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 -- 172.21.15.28:0/3639079395 >> 172.21.15.28:0/3639079395 conn(0x55c57e8ef0a0 msgr2=0x55c57e8ef480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 -- 172.21.15.28:0/3639079395 shutdown_connections 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.875+0000 7eff5d325f40 1 -- 172.21.15.28:0/3639079395 wait complete. 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.876+0000 7eff5d325f40 1 Processor -- start 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.876+0000 7eff5d325f40 1 -- start start 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e8a0470 0x55c57e967a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9763e0 0x55c57e971a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c57e975940 con 0x55c57e9799c0 2024-09-10T10:53:47.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c57e8a29c0 con 0x55c57e9763e0 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5d325f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c57e977a60 con 0x55c57e8a0470 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37630/0 (socket says 172.21.15.28:37630) 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.877+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 learned_addr learned my addr 172.21.15.28:0/2034276842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e8a0470 0x55c57e967a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9763e0 0x55c57e971a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e8a0470 msgr2=0x55c57e967a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c57e8a0470 0x55c57e967a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9763e0 msgr2=0x55c57e971a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9763e0 0x55c57e971a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c57e9751b0 con 0x55c57e9799c0 2024-09-10T10:53:47.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c57e9763e0 0x55c57e971a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5a600640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7eff540041e0 tx=0x7eff54006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff52000640 1 -- 172.21.15.28:0/2034276842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff54018070 con 0x55c57e9799c0 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff52000640 1 -- 172.21.15.28:0/2034276842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff540030d0 con 0x55c57e9799c0 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff52000640 1 -- 172.21.15.28:0/2034276842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff540014a0 con 0x55c57e9799c0 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.878+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c57e97bd60 con 0x55c57e9799c0 2024-09-10T10:53:47.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.879+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c57e97c320 con 0x55c57e9799c0 2024-09-10T10:53:47.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.880+0000 7eff52000640 1 -- 172.21.15.28:0/2034276842 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7eff54001640 con 0x55c57e9799c0 2024-09-10T10:53:47.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.881+0000 7eff52000640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 0x7eff3c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.881+0000 7eff52000640 1 -- 172.21.15.28:0/2034276842 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7eff54089c50 con 0x55c57e9799c0 2024-09-10T10:53:47.884 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.882+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c57e97cb20 0x55c57e973d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.882+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57e974270 con 0x55c57e97cb20 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.882+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 0x7eff3c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.882+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c57e97cb20 0x55c57e973d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.882+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 0x7eff3c06ccd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b2e0 tx=0x7eff4c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.883+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c57e97cb20 0x55c57e973d30 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.885+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v130'908 uv882 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.885+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57e974650 con 0x55c57e97cb20 2024-09-10T10:53:47.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.887+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v130'909 uv909 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.887+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57e974a80 con 0x55c57e97cb20 2024-09-10T10:53:47.890 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.889+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v130'910 uv910 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.889+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1c810 con 0x55c57e97cb20 2024-09-10T10:53:47.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.891+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v130'911 uv911 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.891+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1cbb0 con 0x55c57e97cb20 2024-09-10T10:53:47.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.892+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v130'912 uv912 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.893+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1b800 con 0x55c57e97cb20 2024-09-10T10:53:47.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.894+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v130'913 uv913 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.895+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.896+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v130'914 uv888 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.896+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.898+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v130'915 uv915 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.899+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.901 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:47.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.900+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v130'916 uv916 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.900+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.903+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v130'917 uv917 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.903+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.904+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v130'918 uv918 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.904+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.906+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v130'919 uv919 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.906+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.908+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v130'920 uv895 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.908+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.910+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v130'921 uv921 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.910+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.912 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:47.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.912+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v130'922 uv922 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.912+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.913+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v130'923 uv923 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.913+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.915+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v130'924 uv924 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.915+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bc10 con 0x55c57e97cb20 2024-09-10T10:53:47.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.917+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v130'925 uv925 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.917+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bfd0 con 0x55c57e97cb20 2024-09-10T10:53:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.919+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v130'926 uv926 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.919+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bfd0 con 0x55c57e97cb20 2024-09-10T10:53:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.922+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v130'927 uv901 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.922+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bfd0 con 0x55c57e97cb20 2024-09-10T10:53:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.923+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v130'928 uv928 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.923+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bfd0 con 0x55c57e97cb20 2024-09-10T10:53:47.925 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.924+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v130'929 uv929 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.924+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bfd0 con 0x55c57e97cb20 2024-09-10T10:53:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.926+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v130'930 uv930 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.926+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1c420 con 0x55c57e97cb20 2024-09-10T10:53:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.927+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v130'931 uv931 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.928+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1c420 con 0x55c57e97cb20 2024-09-10T10:53:47.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.929+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v130'932 uv932 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.929+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.932+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v130'933 uv907 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.932+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.933+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v130'934 uv934 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.933+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.935 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:47.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.934+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v130'935 uv935 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.934+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.936+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v130'936 uv936 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.936+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.938+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v130'937 uv937 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.938+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1bed0 con 0x55c57e97cb20 2024-09-10T10:53:47.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.940+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v130'938 uv938 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57e97cb20 2024-09-10T10:53:47.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.940+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c57ea1cf50 0x55c57ea1f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.940+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1f880 con 0x55c57ea1cf50 2024-09-10T10:53:47.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.940+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c57ea1cf50 0x55c57ea1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.940+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c57ea1cf50 0x55c57ea1f340 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.942+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v130'659 uv642 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.942+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1fc20 con 0x55c57ea1cf50 2024-09-10T10:53:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.944+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v130'660 uv660 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.944+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea1ffc0 con 0x55c57ea1cf50 2024-09-10T10:53:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.945+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v130'661 uv661 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.946 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.945+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea20360 con 0x55c57ea1cf50 2024-09-10T10:53:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.947+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v130'662 uv662 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.947+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea20700 con 0x55c57ea1cf50 2024-09-10T10:53:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.948+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v130'663 uv663 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.948+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea20aa0 con 0x55c57ea1cf50 2024-09-10T10:53:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.949+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v130'664 uv647 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.949+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea20e40 con 0x55c57ea1cf50 2024-09-10T10:53:47.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.951+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v130'665 uv665 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.951+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.953 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:47.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.952+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v130'666 uv666 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.953+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.954+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v130'667 uv667 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.954+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.955+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v130'668 uv668 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.955+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.957+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v130'669 uv652 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.957+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.958+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v130'670 uv670 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.958+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.960 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:47.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.960+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v130'671 uv671 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.960+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.962+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v130'672 uv672 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.962+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.963+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v130'673 uv673 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.963+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.965+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v130'674 uv658 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.965+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.966+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v130'675 uv675 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.966+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.968 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:47.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.967+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v130'676 uv676 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.967+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.969+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v130'677 uv677 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.969+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.970+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v130'678 uv678 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.970+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea211e0 con 0x55c57ea1cf50 2024-09-10T10:53:47.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.971+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v130'679 uv679 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea1cf50 2024-09-10T10:53:47.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.971+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c57ea21c70 0x55c57ea22050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.972+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea24650 con 0x55c57ea21c70 2024-09-10T10:53:47.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.972+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c57ea21c70 0x55c57ea22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.972+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c57ea21c70 0x55c57ea22050 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.974+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v130'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.974+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea24650 con 0x55c57ea21c70 2024-09-10T10:53:47.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.975+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v130'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.975+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea24650 con 0x55c57ea21c70 2024-09-10T10:53:47.977 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:47.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.976+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v130'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.976+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea256a0 con 0x55c57ea21c70 2024-09-10T10:53:47.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.977+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v130'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.977+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea25660 con 0x55c57ea21c70 2024-09-10T10:53:47.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.979+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v130'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.979+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea24650 con 0x55c57ea21c70 2024-09-10T10:53:47.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.980+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v130'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.980+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c57ea24d20 0x55c57ea25160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:47.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.980+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea27ae0 con 0x55c57ea24d20 2024-09-10T10:53:47.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.980+0000 7eff5a600640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c57ea24d20 0x55c57ea25160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:47.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.981+0000 7eff5a600640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c57ea24d20 0x55c57ea25160 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:47.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.982+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v130'128 uv127 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.982+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea27e80 con 0x55c57ea24d20 2024-09-10T10:53:47.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.983+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v130'129 uv129 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.983+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea282a0 con 0x55c57ea24d20 2024-09-10T10:53:47.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:47.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.984+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v130'130 uv130 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.984+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28b60 con 0x55c57ea24d20 2024-09-10T10:53:47.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.986+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v130'131 uv131 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.986+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28f00 con 0x55c57ea21c70 2024-09-10T10:53:47.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.987+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v130'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.987+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28f00 con 0x55c57ea21c70 2024-09-10T10:53:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.988+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v130'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.988+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28f00 con 0x55c57ea21c70 2024-09-10T10:53:47.990 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:47.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.989+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v130'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.989+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea29f10 con 0x55c57ea21c70 2024-09-10T10:53:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.990+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v130'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.990+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea29f10 con 0x55c57ea21c70 2024-09-10T10:53:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.992+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v130'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.992+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28f00 con 0x55c57ea21c70 2024-09-10T10:53:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.993+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v130'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.993+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea28f00 con 0x55c57ea24d20 2024-09-10T10:53:47.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.995+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v130'327 uv321 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.995+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea292a0 con 0x55c57ea24d20 2024-09-10T10:53:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.996+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v130'328 uv328 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.996+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea29640 con 0x55c57ea24d20 2024-09-10T10:53:47.998 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:48.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.997+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v130'329 uv329 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.997+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2a9f0 con 0x55c57ea24d20 2024-09-10T10:53:48.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.999+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v130'330 uv330 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:47.999+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2a9f0 con 0x55c57ea24d20 2024-09-10T10:53:48.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.000+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v130'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.000+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.002+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v130'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.002+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.003+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v130'333 uv326 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.003+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.004+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v130'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.004+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.006 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:48.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.006+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v130'335 uv335 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.006+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.007+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v130'336 uv336 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.007+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea24d20 2024-09-10T10:53:48.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.008+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v130'337 uv337 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.008+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea21c70 2024-09-10T10:53:48.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.009+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v130'148 uv147 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:48.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.009+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea21c70 2024-09-10T10:53:48.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.011+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v130'149 uv149 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:48.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.011+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea21c70 2024-09-10T10:53:48.013 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:48.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.012+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v130'150 uv150 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:48.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.012+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea21c70 2024-09-10T10:53:48.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.013+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v130'151 uv151 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:48.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.013+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea299e0 con 0x55c57ea21c70 2024-09-10T10:53:48.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.014+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v130'152 uv152 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea21c70 2024-09-10T10:53:48.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.014+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c57ea2a060 0x55c57ea2c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.014+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2c9d0 con 0x55c57ea2a060 2024-09-10T10:53:48.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.014+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c57ea2a060 0x55c57ea2c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.015+0000 7eff59c00640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c57ea2a060 0x55c57ea2c450 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.016+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v130'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.016+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.018+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v130'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.018+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.020 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:48.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.019+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v130'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.019+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ddb0 con 0x55c57ea2a060 2024-09-10T10:53:48.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.021+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v130'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.021+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ddb0 con 0x55c57ea2a060 2024-09-10T10:53:48.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.022+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v130'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.022+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.023+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v130'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.023+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.025+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v130'294 uv288 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.025+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.026+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v130'295 uv295 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.026+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2cda0 con 0x55c57ea2a060 2024-09-10T10:53:48.028 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.027+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v130'296 uv296 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.027+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.029+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v130'297 uv297 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.029+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.030+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v130'298 uv298 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.030+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.032+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v130'299 uv293 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.032+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.033+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v130'300 uv300 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.033+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.035 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.034+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v130'301 uv301 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.034+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2ce40 con 0x55c57ea2a060 2024-09-10T10:53:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.035+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v130'302 uv302 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.035+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc80 con 0x55c57ea2a060 2024-09-10T10:53:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.037+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v130'303 uv303 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.037+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc40 con 0x55c57ea24d20 2024-09-10T10:53:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.038+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v130'128 uv127 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.038+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc40 con 0x55c57ea24d20 2024-09-10T10:53:48.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.040+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v130'129 uv129 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.040+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc40 con 0x55c57ea24d20 2024-09-10T10:53:48.042 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.041+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v130'130 uv130 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.041+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc40 con 0x55c57ea24d20 2024-09-10T10:53:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.042+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v130'131 uv131 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.042+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.044+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v130'177 uv176 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.044+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.045+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v130'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.045+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:48.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.046+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v130'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.046+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.047+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v130'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.047+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.049+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v130'181 uv181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.049+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2fc00 con 0x55c57ea24d20 2024-09-10T10:53:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.050+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v130'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff54053990 con 0x55c57ea24d20 2024-09-10T10:53:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.050+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c57ea2fc00 0x55c57ea30340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.050+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2fc00 2024-09-10T10:53:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.050+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c57ea2fc00 0x55c57ea30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.051+0000 7eff59200640 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c57ea2fc00 0x55c57ea30340 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.052+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v130'151 uv150 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea2fc00 2024-09-10T10:53:48.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.052+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2fc00 2024-09-10T10:53:48.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.053+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v130'152 uv152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea2fc00 2024-09-10T10:53:48.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.053+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2fc00 2024-09-10T10:53:48.055 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.054+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v130'153 uv153 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea2fc00 2024-09-10T10:53:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.054+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2fc00 2024-09-10T10:53:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.055+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v130'154 uv154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea2fc00 2024-09-10T10:53:48.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.055+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2fc00 2024-09-10T10:53:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.056+0000 7eff59200640 1 -- 172.21.15.28:0/2034276842 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v130'155 uv155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7eff4804f9d0 con 0x55c57ea2fc00 2024-09-10T10:53:48.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.056+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2a060 2024-09-10T10:53:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.058+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v130'363 uv355 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.058+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2a060 2024-09-10T10:53:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.059+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v130'364 uv364 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.059+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea2f7a0 con 0x55c57ea2a060 2024-09-10T10:53:48.061 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.060+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v130'365 uv365 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.060+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea30970 con 0x55c57ea2a060 2024-09-10T10:53:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.061+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v130'366 uv366 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.062+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea308d0 con 0x55c57ea2a060 2024-09-10T10:53:48.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.063+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v130'367 uv367 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.063+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea308d0 con 0x55c57ea2a060 2024-09-10T10:53:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.064+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v130'368 uv362 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.064+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea308d0 con 0x55c57ea2a060 2024-09-10T10:53:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.065+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v130'369 uv369 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.065+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea308d0 con 0x55c57ea2a060 2024-09-10T10:53:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.066+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v130'370 uv370 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.067 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:48.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.066+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea318e0 con 0x55c57ea2a060 2024-09-10T10:53:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.067+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v130'371 uv371 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.067+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea318e0 con 0x55c57ea2a060 2024-09-10T10:53:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.068+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v130'372 uv372 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.068+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea308d0 con 0x55c57ea2a060 2024-09-10T10:53:48.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.069+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v130'373 uv373 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.069+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55c57ea31030 con 0x55c57ea2a060 2024-09-10T10:53:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.070+0000 7eff59c00640 1 -- 172.21.15.28:0/2034276842 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v130'374 uv374 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7eff4c00ec40 con 0x55c57ea2a060 2024-09-10T10:53:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c57ea21c70 msgr2=0x55c57ea22050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55c57ea21c70 0x55c57ea22050 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c57ea2a060 msgr2=0x55c57ea2c450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55c57ea2a060 0x55c57ea2c450 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c57ea2fc00 msgr2=0x55c57ea30340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55c57ea2fc00 0x55c57ea30340 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c57ea24d20 msgr2=0x55c57ea25160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c57ea24d20 0x55c57ea25160 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c57ea1cf50 msgr2=0x55c57ea1f340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55c57ea1cf50 0x55c57ea1f340 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c57e97cb20 msgr2=0x55c57e973d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55c57e97cb20 0x55c57e973d30 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 reap_dead start 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 reap_dead start 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 reap_dead start 2024-09-10T10:53:48.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5a600640 1 -- 172.21.15.28:0/2034276842 reap_dead start 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 msgr2=0x7eff3c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 0x7eff3c06ccd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b2e0 tx=0x7eff4c009000 comp rx=0 tx=0).stop 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 msgr2=0x55c57e971fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7eff540041e0 tx=0x7eff54006f90 comp rx=0 tx=0).stop 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 shutdown_connections 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7eff3c06a810 0x7eff3c06ccd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 --2- 172.21.15.28:0/2034276842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c57e9799c0 0x55c57e971fa0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 >> 172.21.15.28:0/2034276842 conn(0x55c57e8ef0a0 msgr2=0x55c57e8ef480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 shutdown_connections 2024-09-10T10:53:48.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.071+0000 7eff5d325f40 1 -- 172.21.15.28:0/2034276842 wait complete. 2024-09-10T10:53:48.076 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata truncate 100.00000000.inode 0 2024-09-10T10:53:48.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.193+0000 7efcd55ecf40 1 Processor -- start 2024-09-10T10:53:48.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.194+0000 7efcd55ecf40 1 -- start start 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f3953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55926f266b30 con 0x55926f2be470 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55926f26ab70 con 0x55926f3979c0 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55926f396ee0 con 0x55926f3943e0 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37644/0 (socket says 172.21.15.28:37644) 2024-09-10T10:53:48.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd1e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f3953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 -- 172.21.15.28:0/2848717645 learned_addr learned my addr 172.21.15.28:0/2848717645 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 -- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 msgr2=0x55926f38b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 -- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 msgr2=0x55926f3953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f3953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.195+0000 7efcd2800640 1 -- 172.21.15.28:0/2848717645 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926f394d20 con 0x55926f2be470 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.196+0000 7efcd2800640 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7efcc4001d80 tx=0x7efcc4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be11fe32265b50c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.196+0000 7efcd0a00640 1 -- 172.21.15.28:0/2848717645 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400bd70 con 0x55926f2be470 2024-09-10T10:53:48.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.196+0000 7efcd0a00640 1 -- 172.21.15.28:0/2848717645 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc400f400 con 0x55926f2be470 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.196+0000 7efcd0a00640 1 -- 172.21.15.28:0/2848717645 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400f7f0 con 0x55926f2be470 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 -- 172.21.15.28:0/2848717645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 msgr2=0x55926f2f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7efcc4001d80 tx=0x7efcc4001e70 comp rx=0 tx=0).stop 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 -- 172.21.15.28:0/2848717645 shutdown_connections 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f3953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/2848717645 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f2f1d10 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 -- 172.21.15.28:0/2848717645 >> 172.21.15.28:0/2848717645 conn(0x55926f30d0a0 msgr2=0x55926f2a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 -- 172.21.15.28:0/2848717645 shutdown_connections 2024-09-10T10:53:48.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 -- 172.21.15.28:0/2848717645 wait complete. 2024-09-10T10:53:48.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.197+0000 7efcd55ecf40 1 Processor -- start 2024-09-10T10:53:48.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 -- start start 2024-09-10T10:53:48.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f387fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f38bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55926f3964e0 con 0x55926f2be470 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55926f30b3a0 con 0x55926f3979c0 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.198+0000 7efcd55ecf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55926f30d620 con 0x55926f3943e0 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f387fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f387fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37658/0 (socket says 172.21.15.28:37658) 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 -- 172.21.15.28:0/1511107008 learned_addr learned my addr 172.21.15.28:0/1511107008 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd3200640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f38bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd1e00640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 -- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 msgr2=0x55926f38b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 -- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 msgr2=0x55926f38bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f38bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 -- 172.21.15.28:0/1511107008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926f3959e0 con 0x55926f2be470 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd3200640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55926f3979c0 0x55926f38bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd1e00640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55926f3943e0 0x55926f38b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.199+0000 7efcd2800640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f387fe0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7efcc400ba40 tx=0x7efcc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.200+0000 7efccb400640 1 -- 172.21.15.28:0/1511107008 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4020070 con 0x55926f2be470 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.200+0000 7efccb400640 1 -- 172.21.15.28:0/1511107008 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc4001120 con 0x55926f2be470 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.200+0000 7efccb400640 1 -- 172.21.15.28:0/1511107008 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc401c660 con 0x55926f2be470 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.200+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55926f3989d0 con 0x55926f2be470 2024-09-10T10:53:48.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.200+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55926f398f90 con 0x55926f2be470 2024-09-10T10:53:48.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.202+0000 7efccb400640 1 -- 172.21.15.28:0/1511107008 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efcc4023070 con 0x55926f2be470 2024-09-10T10:53:48.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efccb400640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcb406a810 0x7efcb406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efccb400640 1 -- 172.21.15.28:0/1511107008 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7efcc408bde0 con 0x55926f2be470 2024-09-10T10:53:48.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efcd1e00640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcb406a810 0x7efcb406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55926f2f0ff0 0x55926f38df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:fe07bba1:::100.00000000.inode:head [truncate 0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130) -- 0x55926f38e4c0 con 0x55926f2f0ff0 2024-09-10T10:53:48.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.203+0000 7efcd3200640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55926f2f0ff0 0x55926f38df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.204+0000 7efcd1e00640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcb406a810 0x7efcb406ccd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efcc0057000 tx=0x7efcc0052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.204+0000 7efcd3200640 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55926f2f0ff0 0x55926f38df80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd3200640 1 -- 172.21.15.28:0/1511107008 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000.inode [truncate 0] v130'375 uv375 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7efccc001590 con 0x55926f2f0ff0 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55926f2f0ff0 msgr2=0x55926f38df80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55926f2f0ff0 0x55926f38df80 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcb406a810 msgr2=0x7efcb406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efcb406a810 0x7efcb406ccd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7efcc0057000 tx=0x7efcc0052000 comp rx=0 tx=0).stop 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 msgr2=0x55926f387fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 --2- 172.21.15.28:0/1511107008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55926f2be470 0x55926f387fe0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7efcc400ba40 tx=0x7efcc4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd3200640 1 -- 172.21.15.28:0/1511107008 reap_dead start 2024-09-10T10:53:48.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 shutdown_connections 2024-09-10T10:53:48.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 >> 172.21.15.28:0/1511107008 conn(0x55926f30d0a0 msgr2=0x55926f2a9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 shutdown_connections 2024-09-10T10:53:48.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.206+0000 7efcd55ecf40 1 -- 172.21.15.28:0/1511107008 wait complete. 2024-09-10T10:53:48.212 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:48.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.370+0000 7f578ae00640 1 Processor -- start 2024-09-10T10:53:48.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.370+0000 7f578ae00640 1 -- start start 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.371+0000 7f578ae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 0x7f57840b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.371+0000 7f578ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 0x7f57840724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.371+0000 7f578ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57840b4620 con 0x7f578415e900 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.371+0000 7f578ae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f57840b4f60 con 0x7f57840b4020 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.371+0000 7f578ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5784072a00 con 0x7f5784072050 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5783e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f578415e900 0x7f57840ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:42656/0 (socket says 172.21.15.28:42656) 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5783e00640 1 -- 172.21.15.28:0/4278463732 learned_addr learned my addr 172.21.15.28:0/4278463732 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5788e00640 1 --2- 172.21.15.28:0/4278463732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 0x7f57840724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f577b400640 1 --2- 172.21.15.28:0/4278463732 >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 0x7f57840b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5783400640 1 -- 172.21.15.28:0/4278463732 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383134234 0 0) 0x7f57840b4620 con 0x7f578415e900 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5783400640 1 -- 172.21.15.28:0/4278463732 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5768003520 con 0x7f578415e900 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5788e00640 1 -- 172.21.15.28:0/4278463732 >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 msgr2=0x7f57840b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5788e00640 1 --2- 172.21.15.28:0/4278463732 >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 0x7f57840b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5788e00640 1 -- 172.21.15.28:0/4278463732 >> v1:172.21.15.138:6789/0 conn(0x7f578415e900 legacy=0x7f57840ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f5788e00640 1 -- 172.21.15.28:0/4278463732 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57840ce100 con 0x7f5784072050 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.372+0000 7f577b400640 1 --2- 172.21.15.28:0/4278463732 >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 0x7f57840b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.373+0000 7f5788e00640 1 --2- 172.21.15.28:0/4278463732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 0x7f57840724c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d90 tx=0x7f577c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a40d2d8e2a2ac5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.373+0000 7f5783400640 1 -- 172.21.15.28:0/4278463732 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c031600 con 0x7f5784072050 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f5783400640 1 -- 172.21.15.28:0/4278463732 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c0317a0 con 0x7f5784072050 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f5783400640 1 -- 172.21.15.28:0/4278463732 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c031b70 con 0x7f5784072050 2024-09-10T10:53:48.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 -- 172.21.15.28:0/4278463732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 msgr2=0x7f57840724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 --2- 172.21.15.28:0/4278463732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 0x7f57840724c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f577c009d90 tx=0x7f577c02a5d0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 -- 172.21.15.28:0/4278463732 shutdown_connections 2024-09-10T10:53:48.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 --2- 172.21.15.28:0/4278463732 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5784072050 0x7f57840724c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 --2- 172.21.15.28:0/4278463732 >> v2:172.21.15.138:3300/0 conn(0x7f57840b4020 0x7f57840b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 -- 172.21.15.28:0/4278463732 >> 172.21.15.28:0/4278463732 conn(0x7f578406bdf0 msgr2=0x7f578406c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 -- 172.21.15.28:0/4278463732 shutdown_connections 2024-09-10T10:53:48.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.374+0000 7f578ae00640 1 -- 172.21.15.28:0/4278463732 wait complete. 2024-09-10T10:53:48.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.376+0000 7f578ae00640 1 Processor -- start 2024-09-10T10:53:48.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.376+0000 7f578ae00640 1 -- start start 2024-09-10T10:53:48.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57840b4020 0x7f578409d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f578415e900 0x7f578409b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f57840ce960 con 0x7f5784072050 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57840ce680 con 0x7f578415e900 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f578ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f57840cee80 con 0x7f57840b4020 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37670/0 (socket says 172.21.15.28:37670) 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 -- 172.21.15.28:0/2971139117 learned_addr learned my addr 172.21.15.28:0/2971139117 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5788e00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f578415e900 0x7f578409b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f577b400640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57840b4020 0x7f578409d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 -- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57840b4020 msgr2=0x7f578409d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57840b4020 0x7f578409d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 -- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f578415e900 msgr2=0x7f578409b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.377+0000 7f5783e00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f578415e900 0x7f578409b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f5783e00640 1 -- 172.21.15.28:0/2971139117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578409dc50 con 0x7f5784072050 2024-09-10T10:53:48.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f5783e00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f577400c390 tx=0x7f577400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774017070 con 0x7f5784072050 2024-09-10T10:53:48.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5774005ce0 con 0x7f5784072050 2024-09-10T10:53:48.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57740072f0 con 0x7f5784072050 2024-09-10T10:53:48.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578409df30 con 0x7f5784072050 2024-09-10T10:53:48.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.378+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578414fa90 con 0x7f5784072050 2024-09-10T10:53:48.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.379+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57840724c0 con 0x7f5784072050 2024-09-10T10:53:48.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.380+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f577401a070 con 0x7f5784072050 2024-09-10T10:53:48.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.381+0000 7f5780c00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 0x7f576006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.381+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8884+0+0 (secure 0 0 0) 0x7f577408af10 con 0x7f5784072050 2024-09-10T10:53:48.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.382+0000 7f577b400640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 0x7f576006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.382+0000 7f577b400640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 0x7f576006e2f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f576c009560 tx=0x7f576c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.385+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5774058550 con 0x7f5784072050 2024-09-10T10:53:48.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.610+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f57840d05d0 con 0x7f5784072050 2024-09-10T10:53:48.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.780+0000 7f5780c00640 1 -- 172.21.15.28:0/2971139117 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v262) ==== 162+0+0 (secure 0 0 0) 0x7f577405c850 con 0x7f5784072050 2024-09-10T10:53:48.781 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 msgr2=0x7f576006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 0x7f576006e2f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f576c009560 tx=0x7f576c009190 comp rx=0 tx=0).stop 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 msgr2=0x7f5784155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f577400c390 tx=0x7f577400c850 comp rx=0 tx=0).stop 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 shutdown_connections 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f576006be30 0x7f576006e2f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f578415e900 0x7f578409b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f57840b4020 0x7f578409d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 --2- 172.21.15.28:0/2971139117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5784072050 0x7f5784155510 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 >> 172.21.15.28:0/2971139117 conn(0x7f578406bdf0 msgr2=0x7f57840b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 shutdown_connections 2024-09-10T10:53:48.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.783+0000 7f578ae00640 1 -- 172.21.15.28:0/2971139117 wait complete. 2024-09-10T10:53:48.792 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:48.791+0000 7fee68600640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T10:53:48.792 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:53:48.792+0000 7fee68600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:53:48.793 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:48.792+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:53:48.793 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:53:48.792+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:53:48.803 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:48.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.959+0000 7f4bbfe00640 1 Processor -- start 2024-09-10T10:53:48.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.959+0000 7f4bbfe00640 1 -- start start 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4bc00d1f90 0x7f4bc00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbfe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc00cf000 con 0x7f4bc00d0dd0 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbfe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc00d29a0 con 0x7f4bc015e660 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbfe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc006d3e0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4bc00d0dd0 0x7f4bc00d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48054/0 (socket says 172.21.15.28:48054) 2024-09-10T10:53:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbea00640 1 -- 172.21.15.28:0/800862667 learned_addr learned my addr 172.21.15.28:0/800862667 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.960+0000 7f4bbf400640 1 --2- 172.21.15.28:0/800862667 >> v2:172.21.15.28:3300/0 conn(0x7f4bc00d1f90 0x7f4bc00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.961+0000 7f4bbd600640 1 -- 172.21.15.28:0/800862667 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585659556 0 0) 0x7f4bc00cf000 con 0x7f4bc00d0dd0 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.961+0000 7f4bbd600640 1 -- 172.21.15.28:0/800862667 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ba4003520 con 0x7f4bc00d0dd0 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.961+0000 7f4bbf400640 1 -- 172.21.15.28:0/800862667 >> v1:172.21.15.138:6789/0 conn(0x7f4bc015e660 legacy=0x7f4bc006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.961+0000 7f4bbf400640 1 -- 172.21.15.28:0/800862667 >> v1:172.21.15.28:6789/0 conn(0x7f4bc00d0dd0 legacy=0x7f4bc00d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.961+0000 7f4bbf400640 1 -- 172.21.15.28:0/800862667 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc00d2c60 con 0x7f4bc00d1f90 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.962+0000 7f4bbf400640 1 --2- 172.21.15.28:0/800862667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cf7c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8005077d949c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.962+0000 7f4bbd600640 1 -- 172.21.15.28:0/800862667 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8007740 con 0x7f4bc00d1f90 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.962+0000 7f4bbd600640 1 -- 172.21.15.28:0/800862667 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80078e0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.962+0000 7f4bbd600640 1 -- 172.21.15.28:0/800862667 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8007cb0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 -- 172.21.15.28:0/800862667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 msgr2=0x7f4bc00cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/800862667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cf7c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8009d90 tx=0x7f4bb802a6f0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 -- 172.21.15.28:0/800862667 shutdown_connections 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/800862667 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cf7c0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 -- 172.21.15.28:0/800862667 >> 172.21.15.28:0/800862667 conn(0x7f4bc008bfc0 msgr2=0x7f4bc00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 -- 172.21.15.28:0/800862667 shutdown_connections 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.963+0000 7f4bbfe00640 1 -- 172.21.15.28:0/800862667 wait complete. 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.964+0000 7f4bbfe00640 1 Processor -- start 2024-09-10T10:53:48.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 -- start start 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bc00d0dd0 0x7f4bc015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 0x7f4bc00b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc00cf330 con 0x7f4bc00d1f90 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc00d3220 con 0x7f4bc00d0dd0 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.965+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc00d33a0 con 0x7f4bc015e660 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37678/0 (socket says 172.21.15.28:37678) 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 -- 172.21.15.28:0/961060352 learned_addr learned my addr 172.21.15.28:0/961060352 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbea00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bc00d0dd0 0x7f4bc015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbf400640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 0x7f4bc00b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 -- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 msgr2=0x7f4bc00b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 0x7f4bc00b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 -- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bc00d0dd0 msgr2=0x7f4bc015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bc00d0dd0 0x7f4bc015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 -- 172.21.15.28:0/961060352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc015bce0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbf400640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 0x7f4bc00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.966+0000 7f4bbe000640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c440 tx=0x7f4bac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.967+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac0052d0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.967+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bac005470 con 0x7f4bc00d1f90 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.967+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007260 con 0x7f4bc00d1f90 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.967+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc00b7320 con 0x7f4bc00d1f90 2024-09-10T10:53:48.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.967+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc00b78e0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.968+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc00d24d0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.969+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bac0074c0 con 0x7f4bc00d1f90 2024-09-10T10:53:48.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.970+0000 7f4bb6a00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:48.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.970+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f4bac08af20 con 0x7f4bc00d1f90 2024-09-10T10:53:48.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.970+0000 7f4bbea00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:48.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.971+0000 7f4bbea00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 0x7f4ba006e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00011e0 tx=0x7f4bb0007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:48.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:48.974+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4bac0585a0 con 0x7f4bc00d1f90 2024-09-10T10:53:49.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.195+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bc00d1b20 con 0x7f4bc00d1f90 2024-09-10T10:53:49.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.196+0000 7f4bb6a00640 1 -- 172.21.15.28:0/961060352 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 264 v264) ==== 95+0+3566 (secure 0 0 0) 0x7f4bac05c8a0 con 0x7f4bc00d1f90 2024-09-10T10:53:49.198 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:49.198 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":264,"btime":"2024-09-10T10:53:48:793436+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:48.793344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:49.198 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 264 2024-09-10T10:53:49.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 msgr2=0x7f4ba006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 0x7f4ba006e2f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00011e0 tx=0x7f4bb0007d30 comp rx=0 tx=0).stop 2024-09-10T10:53:49.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 msgr2=0x7f4bc00cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f4bac00c440 tx=0x7f4bac00c900 comp rx=0 tx=0).stop 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 shutdown_connections 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ba006be30 0x7f4ba006e2f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bc015e660 0x7f4bc00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bc00d1f90 0x7f4bc00cab10 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 --2- 172.21.15.28:0/961060352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bc00d0dd0 0x7f4bc015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 >> 172.21.15.28:0/961060352 conn(0x7f4bc008bfc0 msgr2=0x7f4bc00d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 shutdown_connections 2024-09-10T10:53:49.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.198+0000 7f4bbfe00640 1 -- 172.21.15.28:0/961060352 wait complete. 2024-09-10T10:53:49.216 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:49.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:49.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.370+0000 7f9a3cc00640 1 Processor -- start 2024-09-10T10:53:49.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.370+0000 7f9a3cc00640 1 -- start start 2024-09-10T10:53:49.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.372+0000 7f9a3cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a380d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.372+0000 7f9a3cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9a380d1f90 0x7f9a380cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.372+0000 7f9a3cc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a380cf000 con 0x7f9a3815e660 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.372+0000 7f9a3cc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9a3806d3e0 con 0x7f9a380d1f90 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.372+0000 7f9a3cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380b37d0 con 0x7f9a380d0dd0 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9a3815e660 0x7f9a3806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48068/0 (socket says 172.21.15.28:48068) 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36a00640 1 -- 172.21.15.28:0/2014726212 learned_addr learned my addr 172.21.15.28:0/2014726212 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:49.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a37e00640 1 --2- 172.21.15.28:0/2014726212 >> v2:172.21.15.138:3300/0 conn(0x7f9a380d1f90 0x7f9a380cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309909764 0 0) 0x7f9a380cf000 con 0x7f9a3815e660 2024-09-10T10:53:49.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a1c003520 con 0x7f9a3815e660 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3793535132 0 0) 0x7f9a1c003520 con 0x7f9a3815e660 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 msgr2=0x7f9a380d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a37400640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a380d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a380d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 >> v2:172.21.15.138:3300/0 conn(0x7f9a380d1f90 msgr2=0x7f9a380cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 --2- 172.21.15.28:0/2014726212 >> v2:172.21.15.138:3300/0 conn(0x7f9a380d1f90 0x7f9a380cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380d16f0 con 0x7f9a3815e660 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.373+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f9a24001670 con 0x7f9a3815e660 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a37400640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a380d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a37e00640 1 --2- 172.21.15.28:0/2014726212 >> v2:172.21.15.138:3300/0 conn(0x7f9a380d1f90 0x7f9a380cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 >> v1:172.21.15.28:6789/0 conn(0x7f9a3815e660 legacy=0x7f9a3806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a1c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c004d40 0x7f9a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3806d3e0 con 0x7f9a1c003ef0 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1c00ba70 con 0x7f9a1c004d40 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1c00bc40 con 0x7f9a380d0dd0 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a37400640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a1c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a37e00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c004d40 0x7f9a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 -- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 msgr2=0x7f9a1c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a380d0dd0 0x7f9a1c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 -- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c004d40 msgr2=0x7f9a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a1c004d40 0x7f9a1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.374+0000 7f9a36a00640 1 -- 172.21.15.28:0/2014726212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1c003520 con 0x7f9a1c003ef0 2024-09-10T10:53:49.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.375+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9a3815ea60 tx=0x7f9a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd27414e45a2cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24007d70 con 0x7f9a1c003ef0 2024-09-10T10:53:49.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24005ad0 con 0x7f9a1c003ef0 2024-09-10T10:53:49.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a36000640 1 -- 172.21.15.28:0/2014726212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24007070 con 0x7f9a1c003ef0 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2014726212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a1c003ef0 msgr2=0x7f9a1c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2014726212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a1c003ef0 0x7f9a1c004800 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f9a3815ea60 tx=0x7f9a1c005830 comp rx=0 tx=0).stop 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a37e00640 1 -- 172.21.15.28:0/2014726212 reap_dead start 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2014726212 shutdown_connections 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2014726212 >> 172.21.15.28:0/2014726212 conn(0x7f9a3808bfc0 msgr2=0x7f9a380d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.376+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2014726212 shutdown_connections 2024-09-10T10:53:49.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.377+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2014726212 wait complete. 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.378+0000 7f9a3cc00640 1 Processor -- start 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.378+0000 7f9a3cc00640 1 -- start start 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c003ef0 0x7f9a381f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a381f7fc0 0x7f9a3809ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380d2c60 con 0x7f9a3815ca70 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a380cf330 con 0x7f9a381f7fc0 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.379+0000 7f9a3cc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9a380d2f40 con 0x7f9a1c003ef0 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37684/0 (socket says 172.21.15.28:37684) 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 -- 172.21.15.28:0/2617694503 learned_addr learned my addr 172.21.15.28:0/2617694503 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a37400640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c003ef0 0x7f9a381f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a37e00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a381f7fc0 0x7f9a3809ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 -- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c003ef0 msgr2=0x7f9a381f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c003ef0 0x7f9a381f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 -- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a381f7fc0 msgr2=0x7f9a3809ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a381f7fc0 0x7f9a3809ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 -- 172.21.15.28:0/2617694503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a380b3eb0 con 0x7f9a3815ca70 2024-09-10T10:53:49.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.380+0000 7f9a36a00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c300 tx=0x7f9a24001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.381+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24040070 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.381+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a24005c80 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.381+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a381f5ab0 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.381+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a240074a0 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.381+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a380b4430 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.382+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a3815dea0 con 0x7f9a3815ca70 2024-09-10T10:53:49.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.383+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9a24043070 con 0x7f9a3815ca70 2024-09-10T10:53:49.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.384+0000 7f9a2f400640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.384+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f9a240ab870 con 0x7f9a3815ca70 2024-09-10T10:53:49.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.384+0000 7f9a37400640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.385+0000 7f9a37400640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 0x7f9a1806e320 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9a3809ff00 tx=0x7f9a28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.388+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a3815dea0 con 0x7f9a3815ca70 2024-09-10T10:53:49.512 INFO:tasks.ceph.mds.a.smithi028.stderr: -43> 2024-09-10T10:53:44.759+0000 7f4393000640 -1 log_channel(cluster) log [ERR] : failed to read JournalPointer: -22 ((22) Invalid argument) 2024-09-10T10:53:49.551 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:53:49.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.609+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a380d3220 con 0x7f9a3815ca70 2024-09-10T10:53:49.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.609+0000 7f9a2f400640 1 -- 172.21.15.28:0/2617694503 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 264 v264) ==== 95+0+3566 (secure 0 0 0) 0x7f9a24005aa0 con 0x7f9a3815ca70 2024-09-10T10:53:49.611 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:49.611 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":264,"btime":"2024-09-10T10:53:48:793436+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:48.793344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 264 2024-09-10T10:53:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 msgr2=0x7f9a1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 0x7f9a1806e320 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9a3809ff00 tx=0x7f9a28008040 comp rx=0 tx=0).stop 2024-09-10T10:53:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 msgr2=0x7f9a381f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400c300 tx=0x7f9a24001a40 comp rx=0 tx=0).stop 2024-09-10T10:53:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 shutdown_connections 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9a1806be60 0x7f9a1806e320 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a381f7fc0 0x7f9a3809ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a3815ca70 0x7f9a381f7a80 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 --2- 172.21.15.28:0/2617694503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9a1c003ef0 0x7f9a381f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 >> 172.21.15.28:0/2617694503 conn(0x7f9a3808bfc0 msgr2=0x7f9a380d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 shutdown_connections 2024-09-10T10:53:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.612+0000 7f9a3cc00640 1 -- 172.21.15.28:0/2617694503 wait complete. 2024-09-10T10:53:49.629 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:53:49.630 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.785+0000 7f0056200640 1 Processor -- start 2024-09-10T10:53:49.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.786+0000 7f0056200640 1 -- start start 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0056200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005015e660 0x7f005006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0056200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00500d1f90 0x7f00500cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0056200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00500cf000 con 0x7f00500d0dd0 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0056200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00500b37d0 con 0x7f00500d1f90 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0056200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00500cf330 con 0x7f005015e660 2024-09-10T10:53:49.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0054e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f00500d0dd0 0x7f00500d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48090/0 (socket says 172.21.15.28:48090) 2024-09-10T10:53:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0054e00640 1 -- 172.21.15.28:0/443434988 learned_addr learned my addr 172.21.15.28:0/443434988 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f004fe00640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005015e660 0x7f005006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f0055800640 1 --2- 172.21.15.28:0/443434988 >> v2:172.21.15.138:3300/0 conn(0x7f00500d1f90 0x7f00500cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f004f400640 1 -- 172.21.15.28:0/443434988 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 829179928 0 0) 0x7f00500cf000 con 0x7f00500d0dd0 2024-09-10T10:53:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.787+0000 7f004f400640 1 -- 172.21.15.28:0/443434988 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0038003520 con 0x7f00500d0dd0 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.788+0000 7f0055800640 1 -- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005015e660 msgr2=0x7f005006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.788+0000 7f0055800640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005015e660 0x7f005006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.788+0000 7f0055800640 1 -- 172.21.15.28:0/443434988 >> v1:172.21.15.28:6789/0 conn(0x7f00500d0dd0 legacy=0x7f00500d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.788+0000 7f0055800640 1 -- 172.21.15.28:0/443434988 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00500d29a0 con 0x7f00500d1f90 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.789+0000 7f0055800640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00500d1f90 0x7f00500cf7c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0048009d90 tx=0x7f004802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4348bb0aaabdbc26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.789+0000 7f004f400640 1 -- 172.21.15.28:0/443434988 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048007740 con 0x7f00500d1f90 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.789+0000 7f004f400640 1 -- 172.21.15.28:0/443434988 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00480078e0 con 0x7f00500d1f90 2024-09-10T10:53:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.789+0000 7f004f400640 1 -- 172.21.15.28:0/443434988 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048007c90 con 0x7f00500d1f90 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 -- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00500d1f90 msgr2=0x7f00500cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00500d1f90 0x7f00500cf7c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0048009d90 tx=0x7f004802a660 comp rx=0 tx=0).stop 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 -- 172.21.15.28:0/443434988 shutdown_connections 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00500d1f90 0x7f00500cf7c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 --2- 172.21.15.28:0/443434988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005015e660 0x7f005006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 -- 172.21.15.28:0/443434988 >> 172.21.15.28:0/443434988 conn(0x7f005008bfc0 msgr2=0x7f00500d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:49.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 -- 172.21.15.28:0/443434988 shutdown_connections 2024-09-10T10:53:49.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.790+0000 7f0056200640 1 -- 172.21.15.28:0/443434988 wait complete. 2024-09-10T10:53:49.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.791+0000 7f0056200640 1 Processor -- start 2024-09-10T10:53:49.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.792+0000 7f0056200640 1 -- start start 2024-09-10T10:53:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.792+0000 7f0056200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00500d0dd0 0x7f0050155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.792+0000 7f0056200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0056200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005015e660 0x7f00501e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0056200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00500d3220 con 0x7f00500d1f90 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0056200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00500d2f40 con 0x7f005015e660 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0056200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00500d33a0 con 0x7f00500d0dd0 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37706/0 (socket says 172.21.15.28:37706) 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 -- 172.21.15.28:0/3106676043 learned_addr learned my addr 172.21.15.28:0/3106676043 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0054e00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00500d0dd0 0x7f0050155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f0055800640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005015e660 0x7f00501e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 -- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00500d0dd0 msgr2=0x7f0050155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00500d0dd0 0x7f0050155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 -- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005015e660 msgr2=0x7f00501e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005015e660 0x7f00501e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.793+0000 7f004fe00640 1 -- 172.21.15.28:0/3106676043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00500cc410 con 0x7f00500d1f90 2024-09-10T10:53:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f004fe00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e870 tx=0x7f003c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c017070 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c00ee60 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c00cb40 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00500cbe50 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.794+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501e3840 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.796+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00500d2440 con 0x7f00500d1f90 2024-09-10T10:53:49.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.796+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f003c01a070 con 0x7f00500d1f90 2024-09-10T10:53:49.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.797+0000 7f004cc00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 0x7f003006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:49.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.798+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f003c08b940 con 0x7f00500d1f90 2024-09-10T10:53:49.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.798+0000 7f0054e00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 0x7f003006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:49.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.799+0000 7f0054e00640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 0x7f003006e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0044005880 tx=0x7f0044009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:49.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:49.804+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f003c059270 con 0x7f00500d1f90 2024-09-10T10:53:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.026+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00500b1bc0 con 0x7f00500d1f90 2024-09-10T10:53:50.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.027+0000 7f004cc00640 1 -- 172.21.15.28:0/3106676043 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 265 v265) ==== 95+0+4392 (secure 0 0 0) 0x7f003c05d570 con 0x7f00500d1f90 2024-09-10T10:53:50.029 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:50.029 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":265,"btime":"2024-09-10T10:53:49:795044+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9540,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/2147565603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2147565603},{"type":"v1","addr":"172.21.15.28:6835","nonce":2147565603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":265}],"filesystems":[{"mdsmap":{"epoch":264,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:48.793344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:50.030 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 265 2024-09-10T10:53:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 msgr2=0x7f003006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 0x7f003006e2f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0044005880 tx=0x7f0044009040 comp rx=0 tx=0).stop 2024-09-10T10:53:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 msgr2=0x7f00500cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f003c00e870 tx=0x7f003c00ed30 comp rx=0 tx=0).stop 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 shutdown_connections 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f003006be30 0x7f003006e2f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005015e660 0x7f00501e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00500d1f90 0x7f00500cb490 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 --2- 172.21.15.28:0/3106676043 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00500d0dd0 0x7f0050155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 >> 172.21.15.28:0/3106676043 conn(0x7f005008bfc0 msgr2=0x7f005006db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 shutdown_connections 2024-09-10T10:53:50.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.029+0000 7f0056200640 1 -- 172.21.15.28:0/3106676043 wait complete. 2024-09-10T10:53:50.048 INFO:tasks.cephfs.test_damage:Result: Mutation 'Truncate 100.00000000.inode' led to DAMAGED state 2024-09-10T10:53:50.048 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head' 2024-09-10T10:53:50.049 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:53:50.049 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:53:50.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:53:50.084 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:53:50.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:53:50.085 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:53:50.159 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:53:50.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:53:50.160 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:53:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.356+0000 7f06c7e00640 1 Processor -- start 2024-09-10T10:53:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.356+0000 7f06c7e00640 1 -- start start 2024-09-10T10:53:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06c80d0dd0 0x7f06c80d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c7e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 0x7f06c80cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c80cf000 con 0x7f06c815e660 2024-09-10T10:53:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f06c806d3e0 con 0x7f06c80d0dd0 2024-09-10T10:53:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c7e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06c80b37d0 con 0x7f06c80d1f90 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06c80d0dd0 0x7f06c80d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f06c80d0dd0 0x7f06c80d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37710/0 (socket says 172.21.15.28:37710) 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.357+0000 7f06c6000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f06c815e660 0x7f06c806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48106/0 (socket says 172.21.15.28:48106) 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 -- 172.21.15.28:0/3419628018 learned_addr learned my addr 172.21.15.28:0/3419628018 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c7400640 1 --2- 172.21.15.28:0/3419628018 >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 0x7f06c80cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 -- 172.21.15.28:0/3419628018 >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 msgr2=0x7f06c80cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 --2- 172.21.15.28:0/3419628018 >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 0x7f06c80cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 -- 172.21.15.28:0/3419628018 >> v1:172.21.15.28:6789/0 conn(0x7f06c815e660 legacy=0x7f06c806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c6a00640 1 -- 172.21.15.28:0/3419628018 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80d16f0 con 0x7f06c80d0dd0 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c5600640 1 -- 172.21.15.28:0/3419628018 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756102515 0 0) 0x7f06c80cf000 con 0x7f06c815e660 2024-09-10T10:53:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.358+0000 7f06c7400640 1 --2- 172.21.15.28:0/3419628018 >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 0x7f06c80cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.359+0000 7f06c6a00640 1 --2- 172.21.15.28:0/3419628018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c80d0dd0 0x7f06c80d11b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f06c808cb90 tx=0x7f06b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d13c3e0c4f48a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.359+0000 7f06c5600640 1 -- 172.21.15.28:0/3419628018 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8037070 con 0x7f06c80d0dd0 2024-09-10T10:53:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.359+0000 7f06c5600640 1 -- 172.21.15.28:0/3419628018 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b8032440 con 0x7f06c80d0dd0 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.359+0000 7f06c5600640 1 -- 172.21.15.28:0/3419628018 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8032810 con 0x7f06c80d0dd0 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 -- 172.21.15.28:0/3419628018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c80d0dd0 msgr2=0x7f06c80d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 --2- 172.21.15.28:0/3419628018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c80d0dd0 0x7f06c80d11b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f06c808cb90 tx=0x7f06b802a270 comp rx=0 tx=0).stop 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 -- 172.21.15.28:0/3419628018 shutdown_connections 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 --2- 172.21.15.28:0/3419628018 >> v2:172.21.15.138:3300/0 conn(0x7f06c80d1f90 0x7f06c80cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 --2- 172.21.15.28:0/3419628018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c80d0dd0 0x7f06c80d11b0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 -- 172.21.15.28:0/3419628018 >> 172.21.15.28:0/3419628018 conn(0x7f06c808bfc0 msgr2=0x7f06c80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 -- 172.21.15.28:0/3419628018 shutdown_connections 2024-09-10T10:53:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.360+0000 7f06c7e00640 1 -- 172.21.15.28:0/3419628018 wait complete. 2024-09-10T10:53:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.362+0000 7f06c7e00640 1 Processor -- start 2024-09-10T10:53:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.362+0000 7f06c7e00640 1 -- start start 2024-09-10T10:53:50.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 0x7f06c80b8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 0x7f06c80b94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f06c80d2c60 con 0x7f06c815e660 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c80cf330 con 0x7f06c80d0dd0 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06c80d2f40 con 0x7f06c80d1f90 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 0x7f06c80b94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:56720/0 (socket says 172.21.15.28:56720) 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c6a00640 1 -- 172.21.15.28:0/608189503 learned_addr learned my addr 172.21.15.28:0/608189503 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.363+0000 7f06c6000640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 0x7f06c80b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 -- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 msgr2=0x7f06c80b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 0x7f06c80b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 -- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 msgr2=0x7f06c80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 -- 172.21.15.28:0/608189503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c80c8bf0 con 0x7f06c815e660 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c6a00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c6000640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 0x7f06c80b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7400640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 0x7f06c80b94f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f06c0007e90 tx=0x7f06c000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001a070 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c0003990 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0003d80 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c80ce6c0 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.364+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c80cd610 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.365+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c80d3220 con 0x7f06c815e660 2024-09-10T10:53:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.366+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06c001d070 con 0x7f06c815e660 2024-09-10T10:53:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.367+0000 7f06bea00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.367+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f06c008bd80 con 0x7f06c815e660 2024-09-10T10:53:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.368+0000 7f06c6a00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.368+0000 7f06c6a00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 0x7f06a806e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f06b80090b0 tx=0x7f06b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.371+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06c0010070 con 0x7f06c815e660 2024-09-10T10:53:50.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.592+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f06c80cd980 con 0x7f06c815e660 2024-09-10T10:53:50.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.801+0000 7f06bea00640 1 -- 172.21.15.28:0/608189503 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v266) ==== 161+0+0 (secure 0 0 0) 0x7f06c0059370 con 0x7f06c815e660 2024-09-10T10:53:50.804 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:53:50.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 msgr2=0x7f06a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 0x7f06a806e2f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f06b80090b0 tx=0x7f06b802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:50.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 msgr2=0x7f06c80b94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 0x7f06c80b94f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f06c0007e90 tx=0x7f06c000d4a0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 shutdown_connections 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06a806be30 0x7f06a806e2f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f06c815e660 0x7f06c80b94f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06c80d1f90 0x7f06c80b8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 --2- 172.21.15.28:0/608189503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c80d0dd0 0x7f06c80b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 >> 172.21.15.28:0/608189503 conn(0x7f06c808bfc0 msgr2=0x7f06c80b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 shutdown_connections 2024-09-10T10:53:50.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.804+0000 7f06c7e00640 1 -- 172.21.15.28:0/608189503 wait complete. 2024-09-10T10:53:50.823 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:53:50.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.978+0000 7fed36200640 1 Processor -- start 2024-09-10T10:53:50.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.978+0000 7fed36200640 1 -- start start 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed36200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed300b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed36200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed300b3c90 0x7fed3006f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed36200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed300b44c0 con 0x7fed300afd60 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed36200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fed300b0140 con 0x7fed300b3c90 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed36200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed3015ead0 con 0x7fed30072dc0 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.979+0000 7fed2fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed300b3c90 0x7fed3006f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fed300afd60 0x7fed300726b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:48118/0 (socket says 172.21.15.28:48118) 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2f400640 1 -- 172.21.15.28:0/1783061120 learned_addr learned my addr 172.21.15.28:0/1783061120 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2ea00640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed300b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2fe00640 1 -- 172.21.15.28:0/1783061120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 msgr2=0x7fed300b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:50.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2fe00640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed300b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2fe00640 1 -- 172.21.15.28:0/1783061120 >> v1:172.21.15.28:6789/0 conn(0x7fed300afd60 legacy=0x7fed300726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2fe00640 1 -- 172.21.15.28:0/1783061120 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3015edb0 con 0x7fed300b3c90 2024-09-10T10:53:50.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.980+0000 7fed2e000640 1 -- 172.21.15.28:0/1783061120 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149091378 0 0) 0x7fed300b44c0 con 0x7fed300afd60 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.981+0000 7fed2fe00640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed3006f5a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fed28009d90 tx=0x7fed2802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfa529442a04a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.981+0000 7fed2e000640 1 -- 172.21.15.28:0/1783061120 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28007740 con 0x7fed300b3c90 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.981+0000 7fed2e000640 1 -- 172.21.15.28:0/1783061120 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed280078e0 con 0x7fed300b3c90 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.981+0000 7fed2e000640 1 -- 172.21.15.28:0/1783061120 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28007cb0 con 0x7fed300b3c90 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 -- 172.21.15.28:0/1783061120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 msgr2=0x7fed3006f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed3006f5a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fed28009d90 tx=0x7fed2802a660 comp rx=0 tx=0).stop 2024-09-10T10:53:50.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 -- 172.21.15.28:0/1783061120 shutdown_connections 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed3006f5a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 --2- 172.21.15.28:0/1783061120 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed300b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 -- 172.21.15.28:0/1783061120 >> 172.21.15.28:0/1783061120 conn(0x7fed3006c030 msgr2=0x7fed3006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 -- 172.21.15.28:0/1783061120 shutdown_connections 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.982+0000 7fed36200640 1 -- 172.21.15.28:0/1783061120 wait complete. 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.984+0000 7fed36200640 1 Processor -- start 2024-09-10T10:53:50.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.984+0000 7fed36200640 1 -- start start 2024-09-10T10:53:50.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.984+0000 7fed36200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed30143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed36200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 0x7fed30156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed36200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed36200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed3015f890 con 0x7fed300b3c90 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed36200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed3015f370 con 0x7fed300afd60 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed36200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fed300b4140 con 0x7fed30072dc0 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37738/0 (socket says 172.21.15.28:37738) 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 -- 172.21.15.28:0/384418345 learned_addr learned my addr 172.21.15.28:0/384418345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 -- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 msgr2=0x7fed30143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2f400640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed30143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2ea00640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 0x7fed30156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed30143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 -- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 msgr2=0x7fed30156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 0x7fed30156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2fe00640 1 -- 172.21.15.28:0/384418345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed30157e20 con 0x7fed300b3c90 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2f400640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed30143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.985+0000 7fed2ea00640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 0x7fed30156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed2fe00640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fed2802ac50 tx=0x7fed28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28040070 con 0x7fed300b3c90 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28031bc0 con 0x7fed300b3c90 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28045460 con 0x7fed300b3c90 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed30158100 con 0x7fed300b3c90 2024-09-10T10:53:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.986+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed30156850 con 0x7fed300b3c90 2024-09-10T10:53:50.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.987+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed300b3750 con 0x7fed300b3c90 2024-09-10T10:53:50.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.988+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed28039040 con 0x7fed300b3c90 2024-09-10T10:53:50.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.989+0000 7fed27400640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 0x7fed1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:50.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.989+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fed280ad4c0 con 0x7fed300b3c90 2024-09-10T10:53:50.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.989+0000 7fed2f400640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 0x7fed1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:50.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.990+0000 7fed2f400640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 0x7fed1006e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fed20000f60 tx=0x7fed20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:50.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:50.993+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fed2807b220 con 0x7fed300b3c90 2024-09-10T10:53:51.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.246+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fed301554f0 con 0x7fed300b3c90 2024-09-10T10:53:51.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.808+0000 7fed27400640 1 -- 172.21.15.28:0/384418345 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v267) ==== 101+0+0 (secure 0 0 0) 0x7fed28043e40 con 0x7fed300b3c90 2024-09-10T10:53:51.810 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 msgr2=0x7fed1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 0x7fed1006e2f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fed20000f60 tx=0x7fed20009040 comp rx=0 tx=0).stop 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 msgr2=0x7fed300b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fed2802ac50 tx=0x7fed28004a40 comp rx=0 tx=0).stop 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 shutdown_connections 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed1006be30 0x7fed1006e2f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed300b3c90 0x7fed300b0d40 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed300afd60 0x7fed30156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 --2- 172.21.15.28:0/384418345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fed30072dc0 0x7fed30143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 >> 172.21.15.28:0/384418345 conn(0x7fed3006c030 msgr2=0x7fed30161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 shutdown_connections 2024-09-10T10:53:51.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.811+0000 7fed36200640 1 -- 172.21.15.28:0/384418345 wait complete. 2024-09-10T10:53:51.830 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:53:51.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.909+0000 7fecbe71ef40 1 Processor -- start 2024-09-10T10:53:51.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.909+0000 7fecbe71ef40 1 -- start start 2024-09-10T10:53:51.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 0x558b4f14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 0x558b4f0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558b4f022b30 con 0x558b4f1473d0 2024-09-10T10:53:51.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b4f026b70 con 0x558b4f151e30 2024-09-10T10:53:51.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558b4f152ee0 con 0x7feca0002460 2024-09-10T10:53:51.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecba400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecba400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37756/0 (socket says 172.21.15.28:37756) 2024-09-10T10:53:51.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.910+0000 7fecba400640 1 -- 172.21.15.28:0/3210618973 learned_addr learned my addr 172.21.15.28:0/3210618973 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 -- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 msgr2=0x558b4f14aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecbb800640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 0x558b4f0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecbae00640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 0x558b4f14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 0x558b4f14aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 -- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 msgr2=0x558b4f0adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 0x558b4f0adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 -- 172.21.15.28:0/3210618973 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4f152740 con 0x558b4f1473d0 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecbb800640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 0x558b4f0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecbae00640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 0x558b4f14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecba400640 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7feca8009750 tx=0x7feca8005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9461b95be61c0c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecb9000640 1 -- 172.21.15.28:0/3210618973 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8008690 con 0x558b4f1473d0 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecb9000640 1 -- 172.21.15.28:0/3210618973 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feca8008830 con 0x558b4f1473d0 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.911+0000 7fecb9000640 1 -- 172.21.15.28:0/3210618973 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca8008c00 con 0x558b4f1473d0 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3210618973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 msgr2=0x558b4f0ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7feca8009750 tx=0x7feca8005a00 comp rx=0 tx=0).stop 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3210618973 shutdown_connections 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558b4f151e30 0x558b4f0adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f1473d0 0x558b4f0ad600 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3210618973 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7feca0002460 0x558b4f14aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3210618973 >> 172.21.15.28:0/3210618973 conn(0x558b4f0c90a0 msgr2=0x558b4f150850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3210618973 shutdown_connections 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.912+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3210618973 wait complete. 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.913+0000 7fecbe71ef40 1 Processor -- start 2024-09-10T10:53:51.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.913+0000 7fecbe71ef40 1 -- start start 2024-09-10T10:53:51.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 0x558b4f13fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca0002460 0x558b4f14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b4f14f920 0x558b4f14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x558b4f0c6f50 con 0x558b4f151e30 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558b4f1524f0 con 0x7feca0002460 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbe71ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558b4f0c6be0 con 0x558b4f14f920 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 0x558b4f13fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 0x558b4f13fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37762/0 (socket says 172.21.15.28:37762) 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 learned_addr learned my addr 172.21.15.28:0/3514995567 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca0002460 0x558b4f14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.914+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b4f14f920 0x558b4f14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b4f14f920 msgr2=0x558b4f14ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b4f14f920 0x558b4f14ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca0002460 msgr2=0x558b4f14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feca0002460 0x558b4f14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:51.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4f07ac60 con 0x558b4f151e30 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558b4f14f920 0x558b4f14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 0x558b4f13fc50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fecac0259f0 tx=0x7fecac0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecb3400640 1 -- 172.21.15.28:0/3514995567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac052e30 con 0x558b4f151e30 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecb3400640 1 -- 172.21.15.28:0/3514995567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecac052440 con 0x558b4f151e30 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecb3400640 1 -- 172.21.15.28:0/3514995567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac05b630 con 0x558b4f151e30 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558b4f0adcd0 con 0x558b4f151e30 2024-09-10T10:53:51.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.915+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4f143a40 con 0x558b4f151e30 2024-09-10T10:53:51.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.917+0000 7fecb3400640 1 -- 172.21.15.28:0/3514995567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fecac05b7d0 con 0x558b4f151e30 2024-09-10T10:53:51.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.918+0000 7fecb3400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec9c06a840 0x7fec9c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.918+0000 7fecb3400640 1 -- 172.21.15.28:0/3514995567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fecac0ce660 con 0x558b4f151e30 2024-09-10T10:53:51.921 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:53:51.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.918+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558b4f155ec0 0x558b4f1562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.918+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f156820 con 0x558b4f155ec0 2024-09-10T10:53:51.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.919+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec9c06a840 0x7fec9c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.919+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558b4f155ec0 0x558b4f1562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.919+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec9c06a840 0x7fec9c06cd00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x558b4f14e3f0 tx=0x7feca8008000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.920+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558b4f155ec0 0x558b4f1562e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.922+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v131'939 uv913 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.922+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f5b10 con 0x558b4f155ec0 2024-09-10T10:53:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.924+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v131'940 uv940 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.924+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f5eb0 con 0x558b4f155ec0 2024-09-10T10:53:51.927 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:53:51.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.926+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v131'941 uv941 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.926+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f7260 con 0x558b4f155ec0 2024-09-10T10:53:51.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.928+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v131'942 uv942 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.928+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f7600 con 0x558b4f155ec0 2024-09-10T10:53:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.930+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v131'943 uv943 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.930+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6250 con 0x558b4f155ec0 2024-09-10T10:53:51.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.932+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v131'944 uv944 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.932+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f65f0 con 0x558b4f155ec0 2024-09-10T10:53:51.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.934+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v131'945 uv919 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.934+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.935+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v131'946 uv946 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.935+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.939 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:53:51.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.937+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v131'947 uv947 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.937+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.939+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v131'948 uv948 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.939+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.941+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v131'949 uv949 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.941+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.942+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v131'950 uv950 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.943+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.945+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v131'951 uv926 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.945+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.947+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v131'952 uv952 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.947+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.950 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:53:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.949+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v131'953 uv953 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.949+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.950+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v131'954 uv954 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.950+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.952+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v131'955 uv955 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.952+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f6990 con 0x558b4f155ec0 2024-09-10T10:53:51.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.954+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v131'956 uv956 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.954+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.955+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v131'957 uv957 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.955+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.957+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v131'958 uv932 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.957+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.960+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v131'959 uv959 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.960+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.962 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:53:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.960+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v131'960 uv960 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.960+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f89b0 con 0x558b4f155ec0 2024-09-10T10:53:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.963+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v131'961 uv961 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.963+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f89b0 con 0x558b4f155ec0 2024-09-10T10:53:51.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.964+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v131'962 uv962 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.964+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.966+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v131'963 uv963 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.966+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.968+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v131'964 uv938 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.968+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.970+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v131'965 uv965 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.970+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.973 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:53:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.971+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v131'966 uv966 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.971+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f89b0 con 0x558b4f155ec0 2024-09-10T10:53:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.974+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v131'967 uv967 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.974+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f89b0 con 0x558b4f155ec0 2024-09-10T10:53:51.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.975+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v131'968 uv968 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.975+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1f79a0 con 0x558b4f155ec0 2024-09-10T10:53:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.978+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v131'969 uv969 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f155ec0 2024-09-10T10:53:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.978+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558b4f1f7eb0 0x558b4f1fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:51.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.978+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fa7e0 con 0x558b4f1f7eb0 2024-09-10T10:53:51.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.978+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558b4f1f7eb0 0x558b4f1fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:51.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.978+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558b4f1f7eb0 0x558b4f1fa2a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.980+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v131'680 uv663 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.980+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fab80 con 0x558b4f1f7eb0 2024-09-10T10:53:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.982+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v131'681 uv681 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.982+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1faf20 con 0x558b4f1f7eb0 2024-09-10T10:53:51.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:53:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.983+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v131'682 uv682 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.983+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fb2c0 con 0x558b4f1f7eb0 2024-09-10T10:53:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.985+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v131'683 uv683 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.985+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fb660 con 0x558b4f1f7eb0 2024-09-10T10:53:51.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.986+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v131'684 uv684 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.986+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fba00 con 0x558b4f1f7eb0 2024-09-10T10:53:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.988+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v131'685 uv668 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.988+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fbda0 con 0x558b4f1f7eb0 2024-09-10T10:53:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.989+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v131'686 uv686 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.989+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:51.992 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:53:51.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.990+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v131'687 uv687 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.991+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:51.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.992+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v131'688 uv688 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.992+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.994+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v131'689 uv689 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.994+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.995+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v131'690 uv673 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.995+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.997+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v131'691 uv691 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.997+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:52.000 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:53:52.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.998+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v131'692 uv692 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:51.998+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f200830 con 0x558b4f1f7eb0 2024-09-10T10:53:52.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.000+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v131'693 uv693 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.000+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f200830 con 0x558b4f1f7eb0 2024-09-10T10:53:52.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.001+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v131'694 uv694 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.001+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:52.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.003+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v131'695 uv679 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.003+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:52.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.004+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v131'696 uv696 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.004+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:52.008 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:53:52.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.006+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v131'697 uv697 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.006+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fd150 con 0x558b4f1f7eb0 2024-09-10T10:53:52.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.007+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v131'698 uv698 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.007+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fd150 con 0x558b4f1f7eb0 2024-09-10T10:53:52.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.009+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v131'699 uv699 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.009+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fc140 con 0x558b4f1f7eb0 2024-09-10T10:53:52.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.010+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v131'700 uv700 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f1f7eb0 2024-09-10T10:53:52.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.010+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558b4f1fc650 0x558b4f1fea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.010+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1fef80 con 0x558b4f1fc650 2024-09-10T10:53:52.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.011+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558b4f1fc650 0x558b4f1fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.011+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558b4f1fc650 0x558b4f1fea40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.013+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v131'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.013+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1ff320 con 0x558b4f1fc650 2024-09-10T10:53:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.014+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v131'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.014+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1ff6c0 con 0x558b4f1fc650 2024-09-10T10:53:52.017 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:53:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.015+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v131'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.015+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f200a70 con 0x558b4f1fc650 2024-09-10T10:53:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.017+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v131'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.017+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f200e90 con 0x558b4f1fc650 2024-09-10T10:53:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.018+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v131'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.018+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1ffa60 con 0x558b4f1fc650 2024-09-10T10:53:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.019+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v131'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.019+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558b4f200520 0x558b4f203290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.019+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f203840 con 0x558b4f200520 2024-09-10T10:53:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.019+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558b4f200520 0x558b4f203290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.020+0000 7fecbb800640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558b4f200520 0x558b4f203290 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.021+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v131'132 uv131 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.021+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f203c30 con 0x558b4f200520 2024-09-10T10:53:52.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.022+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v131'133 uv133 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.022+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f203c30 con 0x558b4f200520 2024-09-10T10:53:52.025 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:53:52.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.024+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v131'134 uv134 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.024+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f203c30 con 0x558b4f200520 2024-09-10T10:53:52.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.025+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v131'135 uv135 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.025+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f203b70 con 0x558b4f1fc650 2024-09-10T10:53:52.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.026+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v131'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.026+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204050 con 0x558b4f1fc650 2024-09-10T10:53:52.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.027+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v131'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.027+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204050 con 0x558b4f1fc650 2024-09-10T10:53:52.030 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:53:52.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.029+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v131'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.029+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f205060 con 0x558b4f1fc650 2024-09-10T10:53:52.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.030+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v131'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.030+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f205060 con 0x558b4f1fc650 2024-09-10T10:53:52.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.031+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v131'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.031+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204050 con 0x558b4f1fc650 2024-09-10T10:53:52.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.032+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v131'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.032+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204050 con 0x558b4f200520 2024-09-10T10:53:52.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.034+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v131'338 uv332 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.034+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204050 con 0x558b4f200520 2024-09-10T10:53:52.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.036+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v131'339 uv339 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.036+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204000 con 0x558b4f200520 2024-09-10T10:53:52.039 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:53:52.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.037+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v131'340 uv340 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.037+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f205420 con 0x558b4f200520 2024-09-10T10:53:52.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.038+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v131'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.038+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f205420 con 0x558b4f200520 2024-09-10T10:53:52.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.040+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v131'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.040+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204160 con 0x558b4f200520 2024-09-10T10:53:52.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.041+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v131'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.041+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204160 con 0x558b4f200520 2024-09-10T10:53:52.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.042+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v131'344 uv337 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.042+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204110 con 0x558b4f200520 2024-09-10T10:53:52.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.044+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v131'345 uv345 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.044+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204110 con 0x558b4f200520 2024-09-10T10:53:52.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:53:52.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.045+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v131'346 uv346 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.045+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204070 con 0x558b4f200520 2024-09-10T10:53:52.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.047+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v131'347 uv347 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.047+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204070 con 0x558b4f200520 2024-09-10T10:53:52.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.048+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v131'348 uv348 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.048+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204020 con 0x558b4f1fc650 2024-09-10T10:53:52.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.049+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v131'153 uv152 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.049+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204020 con 0x558b4f1fc650 2024-09-10T10:53:52.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.050+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v131'154 uv154 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.050+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204020 con 0x558b4f1fc650 2024-09-10T10:53:52.053 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:53:52.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.051+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v131'155 uv155 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.051+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204060 con 0x558b4f1fc650 2024-09-10T10:53:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.052+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v131'156 uv156 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.052+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f204520 con 0x558b4f1fc650 2024-09-10T10:53:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.054+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v131'157 uv157 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f1fc650 2024-09-10T10:53:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.054+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b4f204ba0 0x558b4f207430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.054+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.054+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b4f204ba0 0x558b4f207430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.054+0000 7fecbae00640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b4f204ba0 0x558b4f207430 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.056+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v131'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.056+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.057+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v131'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.057+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.059+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v131'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.061 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:53:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.059+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f208980 con 0x558b4f204ba0 2024-09-10T10:53:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.060+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v131'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.060+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f208980 con 0x558b4f204ba0 2024-09-10T10:53:52.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.061+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v131'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.061+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.062+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v131'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.062+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.064+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v131'304 uv298 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.064+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.065+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v131'305 uv305 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.065+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207970 con 0x558b4f204ba0 2024-09-10T10:53:52.068 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:53:52.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.066+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v131'306 uv306 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.067+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207d60 con 0x558b4f204ba0 2024-09-10T10:53:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.068+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v131'307 uv307 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.068+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207d60 con 0x558b4f204ba0 2024-09-10T10:53:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.069+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v131'308 uv308 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.069+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207a10 con 0x558b4f204ba0 2024-09-10T10:53:52.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.071+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v131'309 uv303 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.071+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207a10 con 0x558b4f204ba0 2024-09-10T10:53:52.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.072+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v131'310 uv310 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.072+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207a10 con 0x558b4f204ba0 2024-09-10T10:53:52.075 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:53:52.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.073+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v131'311 uv311 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.073+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f207a10 con 0x558b4f204ba0 2024-09-10T10:53:52.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.074+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v131'312 uv312 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.075+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f1465d0 con 0x558b4f204ba0 2024-09-10T10:53:52.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.076+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v131'313 uv313 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.076+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146590 con 0x558b4f200520 2024-09-10T10:53:52.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.077+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v131'132 uv131 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.077+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146590 con 0x558b4f200520 2024-09-10T10:53:52.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.079+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v131'133 uv133 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.079+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146590 con 0x558b4f200520 2024-09-10T10:53:52.082 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:53:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.080+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v131'134 uv134 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.080+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146590 con 0x558b4f200520 2024-09-10T10:53:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.081+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v131'135 uv135 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.081+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.082+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v131'183 uv182 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.082+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.084+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v131'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.084+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.086 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:53:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.085+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v131'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.085+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.086+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v131'186 uv186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.086+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.087+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v131'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.087+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f200520 2024-09-10T10:53:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.088+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v131'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fecb4006140 con 0x558b4f200520 2024-09-10T10:53:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.088+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558b4f208b20 0x558b4f20af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.088+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f208b20 2024-09-10T10:53:52.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.089+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558b4f208b20 0x558b4f20af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.089+0000 7fecba400640 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558b4f208b20 0x558b4f20af10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.090+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v131'156 uv155 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f208b20 2024-09-10T10:53:52.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.090+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f208b20 2024-09-10T10:53:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.091+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v131'157 uv157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f208b20 2024-09-10T10:53:52.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.092+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f208b20 2024-09-10T10:53:52.095 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:53:52.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.093+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v131'158 uv158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f208b20 2024-09-10T10:53:52.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.093+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f208b20 2024-09-10T10:53:52.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.094+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v131'159 uv159 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f208b20 2024-09-10T10:53:52.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.094+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f208b20 2024-09-10T10:53:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.095+0000 7fecba400640 1 -- 172.21.15.28:0/3514995567 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v131'160 uv160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7feca8015610 con 0x558b4f208b20 2024-09-10T10:53:52.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.095+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.097+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v131'376 uv375 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.097+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.098+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v131'377 uv377 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.098+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.099+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v131'378 uv378 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.101 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:53:52.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.099+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.100+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v131'379 uv379 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.100+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.101+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v131'380 uv380 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.101+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.102+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v131'381 uv374 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.102+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.104+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v131'382 uv382 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.104+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f146570 con 0x558b4f204ba0 2024-09-10T10:53:52.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.105+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v131'383 uv383 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.107 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:53:52.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.105+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f20c4b0 con 0x558b4f204ba0 2024-09-10T10:53:52.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.106+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v131'384 uv384 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.106+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f20c4b0 con 0x558b4f204ba0 2024-09-10T10:53:52.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.107+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v131'385 uv385 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.107+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f20b450 con 0x558b4f204ba0 2024-09-10T10:53:52.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.108+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v131'386 uv386 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.108+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x558b4f20b900 con 0x558b4f204ba0 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbae00640 1 -- 172.21.15.28:0/3514995567 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v131'387 uv387 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fecac06f070 con 0x558b4f204ba0 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558b4f1fc650 msgr2=0x558b4f1fea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x558b4f1fc650 0x558b4f1fea40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b4f204ba0 msgr2=0x558b4f207430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x558b4f204ba0 0x558b4f207430 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558b4f208b20 msgr2=0x558b4f20af10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x558b4f208b20 0x558b4f20af10 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558b4f200520 msgr2=0x558b4f203290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x558b4f200520 0x558b4f203290 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558b4f1f7eb0 msgr2=0x558b4f1fa2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x558b4f1f7eb0 0x558b4f1fa2a0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558b4f155ec0 msgr2=0x558b4f1562e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x558b4f155ec0 0x558b4f1562e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 reap_dead start 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.109+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 reap_dead start 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 reap_dead start 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec9c06a840 msgr2=0x7fec9c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec9c06a840 0x7fec9c06cd00 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x558b4f14e3f0 tx=0x7feca8008000 comp rx=0 tx=0).stop 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 msgr2=0x558b4f13fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 --2- 172.21.15.28:0/3514995567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558b4f151e30 0x558b4f13fc50 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fecac0259f0 tx=0x7fecac0012d0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbb800640 1 -- 172.21.15.28:0/3514995567 reap_dead start 2024-09-10T10:53:52.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 shutdown_connections 2024-09-10T10:53:52.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 >> 172.21.15.28:0/3514995567 conn(0x558b4f0c90a0 msgr2=0x558b4f150850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:52.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 shutdown_connections 2024-09-10T10:53:52.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.110+0000 7fecbe71ef40 1 -- 172.21.15.28:0/3514995567 wait complete. 2024-09-10T10:53:52.116 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 sixmegs_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:53:52.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.232+0000 7f9de4b1ef40 1 Processor -- start 2024-09-10T10:53:52.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 -- start start 2024-09-10T10:53:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd817914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dd8166ddc0 con 0x55dd816285a0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd815ccb30 con 0x55dd816fa4b0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.233+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dd8166d1f0 con 0x55dd8166e2d0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37794/0 (socket says 172.21.15.28:37794) 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 -- 172.21.15.28:0/300228590 learned_addr learned my addr 172.21.15.28:0/300228590 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de0a00640 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1e00640 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd817914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 -- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 msgr2=0x55dd816fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 -- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 msgr2=0x55dd817914f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd817914f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 -- 172.21.15.28:0/300228590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd81655710 con 0x55dd816285a0 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9de1400640 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400a4f0 tx=0x7f9dd4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e718bf16773c54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.234+0000 7f9ddb400640 1 -- 172.21.15.28:0/300228590 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4068070 con 0x55dd816285a0 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9ddb400640 1 -- 172.21.15.28:0/300228590 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd40516f0 con 0x55dd816285a0 2024-09-10T10:53:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9ddb400640 1 -- 172.21.15.28:0/300228590 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4055070 con 0x55dd816285a0 2024-09-10T10:53:52.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/300228590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 msgr2=0x55dd81651d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400a4f0 tx=0x7f9dd4051000 comp rx=0 tx=0).stop 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/300228590 shutdown_connections 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd817914f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/300228590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd81651d90 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/300228590 >> 172.21.15.28:0/300228590 conn(0x55dd81652b90 msgr2=0x55dd816289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/300228590 shutdown_connections 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.235+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/300228590 wait complete. 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.236+0000 7f9de4b1ef40 1 Processor -- start 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.236+0000 7f9de4b1ef40 1 -- start start 2024-09-10T10:53:52.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd816eb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd816f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55dd816fc6c0 con 0x55dd816285a0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dd816551e0 con 0x55dd816fa4b0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de4b1ef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55dd81654030 con 0x55dd8166e2d0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd816eb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd816eb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37806/0 (socket says 172.21.15.28:37806) 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de1400640 1 -- 172.21.15.28:0/1982250479 learned_addr learned my addr 172.21.15.28:0/1982250479 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de0a00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.237+0000 7f9de1e00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd816f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 -- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 msgr2=0x55dd816fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55dd8166e2d0 0x55dd816fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 -- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 msgr2=0x55dd816f7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dd816fa4b0 0x55dd816f7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 -- 172.21.15.28:0/1982250479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd816f3220 con 0x55dd816285a0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de1400640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd816eb930 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4000c00 tx=0x7f9dd4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9dda000640 1 -- 172.21.15.28:0/1982250479 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4055070 con 0x55dd816285a0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9dda000640 1 -- 172.21.15.28:0/1982250479 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd4051d40 con 0x55dd816285a0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9dda000640 1 -- 172.21.15.28:0/1982250479 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4068070 con 0x55dd816285a0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dd816f7790 con 0x55dd816285a0 2024-09-10T10:53:52.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.238+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd816eee60 con 0x55dd816285a0 2024-09-10T10:53:52.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.240+0000 7f9dda000640 1 -- 172.21.15.28:0/1982250479 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9dd4073060 con 0x55dd816285a0 2024-09-10T10:53:52.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.241+0000 7f9dda000640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9dc406a840 0x7f9dc406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.241+0000 7f9dda000640 1 -- 172.21.15.28:0/1982250479 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f9dd40cefe0 con 0x55dd816285a0 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.241+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dd816fb8c0 0x55dd817949e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.241+0000 7f9de0a00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9dc406a840 0x7f9dc406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.241+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=104b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) -- 0x55dd816ebe70 con 0x55dd816fb8c0 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.242+0000 7f9de1e00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dd816fb8c0 0x55dd817949e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.242+0000 7f9de0a00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9dc406a840 0x7f9dc406cd00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000c00 tx=0x7f9dd00017e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.242+0000 7f9de1e00640 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dd816fb8c0 0x55dd817949e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de1e00640 1 -- 172.21.15.28:0/1982250479 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v131'970 uv970 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9ddc00f7a0 con 0x55dd816fb8c0 2024-09-10T10:53:52.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dd816fb8c0 msgr2=0x55dd817949e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55dd816fb8c0 0x55dd817949e0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9dc406a840 msgr2=0x7f9dc406cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9dc406a840 0x7f9dc406cd00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000c00 tx=0x7f9dd00017e0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 msgr2=0x55dd816eb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 --2- 172.21.15.28:0/1982250479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dd816285a0 0x55dd816eb930 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4000c00 tx=0x7f9dd4003020 comp rx=0 tx=0).stop 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de1e00640 1 -- 172.21.15.28:0/1982250479 reap_dead start 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 shutdown_connections 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 >> 172.21.15.28:0/1982250479 conn(0x55dd81652b90 msgr2=0x55dd816289c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 shutdown_connections 2024-09-10T10:53:52.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.244+0000 7f9de4b1ef40 1 -- 172.21.15.28:0/1982250479 wait complete. 2024-09-10T10:53:52.250 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:53:52.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.408+0000 7f0da7400640 1 Processor -- start 2024-09-10T10:53:52.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.408+0000 7f0da7400640 1 -- start start 2024-09-10T10:53:52.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0da7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da015eac0 0x7f0da0073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0da7400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 0x7f0da0162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0da7400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da015fae0 con 0x7f0da0158ad0 2024-09-10T10:53:52.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0da7400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0da01630c0 con 0x7f0da015eac0 2024-09-10T10:53:52.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0da7400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0da01622a0 con 0x7f0da0162730 2024-09-10T10:53:52.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0d9fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da015eac0 0x7f0da0073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0d9fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0da015eac0 0x7f0da0073c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37810/0 (socket says 172.21.15.28:37810) 2024-09-10T10:53:52.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.409+0000 7f0d9fe00640 1 -- 172.21.15.28:0/3222052734 learned_addr learned my addr 172.21.15.28:0/3222052734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0da5400640 1 --2- 172.21.15.28:0/3222052734 >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 0x7f0da0162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0d9fe00640 1 -- 172.21.15.28:0/3222052734 >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 msgr2=0x7f0da0162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0d9fe00640 1 --2- 172.21.15.28:0/3222052734 >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 0x7f0da0162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0d9fe00640 1 -- 172.21.15.28:0/3222052734 >> v1:172.21.15.138:6789/0 conn(0x7f0da0158ad0 legacy=0x7f0da00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0d9fe00640 1 -- 172.21.15.28:0/3222052734 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da015f240 con 0x7f0da015eac0 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.410+0000 7f0da5400640 1 --2- 172.21.15.28:0/3222052734 >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 0x7f0da0162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.411+0000 7f0d9fe00640 1 --2- 172.21.15.28:0/3222052734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da015eac0 0x7f0da0073c20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0da0162100 tx=0x7f0d8c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf773b575c020f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.411+0000 7f0d9f400640 1 -- 172.21.15.28:0/3222052734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c037070 con 0x7f0da015eac0 2024-09-10T10:53:52.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.411+0000 7f0d9f400640 1 -- 172.21.15.28:0/3222052734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c032440 con 0x7f0da015eac0 2024-09-10T10:53:52.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.411+0000 7f0d9f400640 1 -- 172.21.15.28:0/3222052734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c032810 con 0x7f0da015eac0 2024-09-10T10:53:52.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 -- 172.21.15.28:0/3222052734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da015eac0 msgr2=0x7f0da0073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 --2- 172.21.15.28:0/3222052734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da015eac0 0x7f0da0073c20 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f0da0162100 tx=0x7f0d8c02a230 comp rx=0 tx=0).stop 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 -- 172.21.15.28:0/3222052734 shutdown_connections 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 --2- 172.21.15.28:0/3222052734 >> v2:172.21.15.138:3300/0 conn(0x7f0da0162730 0x7f0da0162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 --2- 172.21.15.28:0/3222052734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da015eac0 0x7f0da0073c20 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 -- 172.21.15.28:0/3222052734 >> 172.21.15.28:0/3222052734 conn(0x7f0da006bdf0 msgr2=0x7f0da006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 -- 172.21.15.28:0/3222052734 shutdown_connections 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.412+0000 7f0da7400640 1 -- 172.21.15.28:0/3222052734 wait complete. 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.413+0000 7f0da7400640 1 Processor -- start 2024-09-10T10:53:52.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.413+0000 7f0da7400640 1 -- start start 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 0x7f0da01f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 0x7f0da01f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0da015f800 con 0x7f0da0162730 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0074160 con 0x7f0da0158ad0 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.414+0000 7f0da7400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0da00cf550 con 0x7f0da015eac0 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37812/0 (socket says 172.21.15.28:37812) 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 -- 172.21.15.28:0/3489624821 learned_addr learned my addr 172.21.15.28:0/3489624821 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0d9fe00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 0x7f0da01f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da4a00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 0x7f0da01f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 -- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 msgr2=0x7f0da01f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 0x7f0da01f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 -- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 msgr2=0x7f0da01f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 0x7f0da01f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da5400640 1 -- 172.21.15.28:0/3489624821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da01f4ef0 con 0x7f0da0162730 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0da4a00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 0x7f0da01f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.415+0000 7f0d9fe00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 0x7f0da01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0da5400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800cd50 tx=0x7f0d9800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800f680 con 0x7f0da0162730 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9800f840 con 0x7f0da0162730 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800fc30 con 0x7f0da0162730 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da0159fa0 con 0x7f0da0162730 2024-09-10T10:53:52.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.416+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da015a560 con 0x7f0da0162730 2024-09-10T10:53:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.417+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da015ef50 con 0x7f0da0162730 2024-09-10T10:53:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.418+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0d980213f0 con 0x7f0da0162730 2024-09-10T10:53:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.419+0000 7f0d9cc00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 0x7f0d8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.419+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f0d9808c990 con 0x7f0da0162730 2024-09-10T10:53:52.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.419+0000 7f0da4a00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 0x7f0d8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:52.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.420+0000 7f0da4a00640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 0x7f0d8006e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003eb0 tx=0x7f0d94009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:52.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.423+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d98014030 con 0x7f0da0162730 2024-09-10T10:53:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.644+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f0da015b580 con 0x7f0da0162730 2024-09-10T10:53:52.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.812+0000 7f0d9cc00640 1 -- 172.21.15.28:0/3489624821 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v268) ==== 162+0+0 (secure 0 0 0) 0x7f0d9805a5d0 con 0x7f0da0162730 2024-09-10T10:53:52.814 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:53:52.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 msgr2=0x7f0d8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 0x7f0d8006e2f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d94003eb0 tx=0x7f0d94009040 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 msgr2=0x7f0da015b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800cd50 tx=0x7f0d9800a650 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 shutdown_connections 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0d8006be30 0x7f0d8006e2f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0da0162730 0x7f0da015b040 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0da015eac0 0x7f0da01f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 --2- 172.21.15.28:0/3489624821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da0158ad0 0x7f0da01f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 >> 172.21.15.28:0/3489624821 conn(0x7f0da006bdf0 msgr2=0x7f0da006fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 shutdown_connections 2024-09-10T10:53:52.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:52.814+0000 7f0da7400640 1 -- 172.21.15.28:0/3489624821 wait complete. 2024-09-10T10:53:53.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.393+0000 7f1ab1200640 1 Processor -- start 2024-09-10T10:53:53.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.393+0000 7f1ab1200640 1 -- start start 2024-09-10T10:53:53.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.394+0000 7f1ab1200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072de0 0x7f1aac15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.394+0000 7f1ab1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aac072700 0x7f1aac15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.394+0000 7f1ab1200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aac146140 con 0x7f1aac072320 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.394+0000 7f1ab1200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1aac0cd740 con 0x7f1aac072700 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.394+0000 7f1ab1200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac0cda20 con 0x7f1aac072de0 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aac072700 0x7f1aac15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aac072700 0x7f1aac15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37814/0 (socket says 172.21.15.28:37814) 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 -- 172.21.15.28:0/332038064 learned_addr learned my addr 172.21.15.28:0/332038064 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:53.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aa9400640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072de0 0x7f1aac15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 -- 172.21.15.28:0/332038064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072de0 msgr2=0x7f1aac15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:53.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072de0 0x7f1aac15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 -- 172.21.15.28:0/332038064 >> v1:172.21.15.138:6789/0 conn(0x7f1aac072320 legacy=0x7f1aac0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.395+0000 7f1aaa800640 1 -- 172.21.15.28:0/332038064 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac0cdd00 con 0x7f1aac072700 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.396+0000 7f1aaa800640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072700 0x7f1aac15d720 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009e50 tx=0x7f1aa402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8974d088827c797f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.396+0000 7f1aa8a00640 1 -- 172.21.15.28:0/332038064 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4036070 con 0x7f1aac072700 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1aa8a00640 1 -- 172.21.15.28:0/332038064 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4031600 con 0x7f1aac072700 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1aa8a00640 1 -- 172.21.15.28:0/332038064 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa40319d0 con 0x7f1aac072700 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 -- 172.21.15.28:0/332038064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072700 msgr2=0x7f1aac15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072700 0x7f1aac15d720 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009e50 tx=0x7f1aa402a680 comp rx=0 tx=0).stop 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 -- 172.21.15.28:0/332038064 shutdown_connections 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072700 0x7f1aac15d720 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 --2- 172.21.15.28:0/332038064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072de0 0x7f1aac15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 -- 172.21.15.28:0/332038064 >> 172.21.15.28:0/332038064 conn(0x7f1aac067d70 msgr2=0x7f1aac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 -- 172.21.15.28:0/332038064 shutdown_connections 2024-09-10T10:53:53.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.397+0000 7f1ab1200640 1 -- 172.21.15.28:0/332038064 wait complete. 2024-09-10T10:53:53.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.399+0000 7f1ab1200640 1 Processor -- start 2024-09-10T10:53:53.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.399+0000 7f1ab1200640 1 -- start start 2024-09-10T10:53:53.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072320 0x7f1aac13f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 0x7f1aac0b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15dde0 con 0x7f1aac072de0 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15dc60 con 0x7f1aac072700 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1ab1200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac15df60 con 0x7f1aac072320 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37820/0 (socket says 172.21.15.28:37820) 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 -- 172.21.15.28:0/3808952752 learned_addr learned my addr 172.21.15.28:0/3808952752 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aa9e00640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072320 0x7f1aac13f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aa9400640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 0x7f1aac0b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 -- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072320 msgr2=0x7f1aac13f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072320 0x7f1aac13f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.400+0000 7f1aaa800640 1 -- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 msgr2=0x7f1aac0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aaa800640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 0x7f1aac0b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aaa800640 1 -- 172.21.15.28:0/3808952752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac0b1c00 con 0x7f1aac072de0 2024-09-10T10:53:53.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aa9400640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 0x7f1aac0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aaa800640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009f70 tx=0x7f1aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa403f020 con 0x7f1aac072de0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa4037040 con 0x7f1aac072de0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4036070 con 0x7f1aac072de0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac13fa90 con 0x7f1aac072de0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.401+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac0b3a50 con 0x7f1aac072de0 2024-09-10T10:53:53.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.402+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac15e390 con 0x7f1aac072de0 2024-09-10T10:53:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.403+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1aa4042020 con 0x7f1aac072de0 2024-09-10T10:53:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.404+0000 7f1aa2000640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 0x7f1a8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.404+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f1aa40abef0 con 0x7f1aac072de0 2024-09-10T10:53:53.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.405+0000 7f1aa9e00640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 0x7f1a8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:53.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.405+0000 7f1aa9e00640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 0x7f1a8c06e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c001780 tx=0x7f1a9c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:53.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.408+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1aa4049030 con 0x7f1aac072de0 2024-09-10T10:53:53.546 INFO:tasks.ceph.mds.c.smithi028.stderr: -46> 2024-09-10T10:53:48.791+0000 7fee68600640 -1 mds.0.cache.ino(0x100) No data while reading inode 0x100 2024-09-10T10:53:53.546 INFO:tasks.ceph.mds.c.smithi028.stderr: -36> 2024-09-10T10:53:48.792+0000 7fee68600640 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (2) No such file or directory 2024-09-10T10:53:53.589 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:53:53.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.637+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aac145880 con 0x7f1aac072de0 2024-09-10T10:53:53.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.637+0000 7f1aa2000640 1 -- 172.21.15.28:0/3808952752 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 269 v269) ==== 95+0+4401 (secure 0 0 0) 0x7f1aa4079b30 con 0x7f1aac072de0 2024-09-10T10:53:53.640 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:53.640 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":269,"btime":"2024-09-10T10:53:52:815341+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":269,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:52.815331+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9540":{"gid":9540,"name":"a","rank":0,"incarnation":269,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2147565603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2147565603},{"type":"v1","addr":"172.21.15.28:6835","nonce":2147565603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:53:53.640 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 269 2024-09-10T10:53:53.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 msgr2=0x7f1a8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 0x7f1a8c06e2f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c001780 tx=0x7f1a9c009040 comp rx=0 tx=0).stop 2024-09-10T10:53:53.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 msgr2=0x7f1aac0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:53.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009f70 tx=0x7f1aa4004a40 comp rx=0 tx=0).stop 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 shutdown_connections 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a8c06be30 0x7f1a8c06e2f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aac072de0 0x7f1aac0cb2f0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1aac072700 0x7f1aac0b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 --2- 172.21.15.28:0/3808952752 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1aac072320 0x7f1aac13f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 >> 172.21.15.28:0/3808952752 conn(0x7f1aac067d70 msgr2=0x7f1aac15d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 shutdown_connections 2024-09-10T10:53:53.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:53.640+0000 7f1ab1200640 1 -- 172.21.15.28:0/3808952752 wait complete. 2024-09-10T10:53:53.658 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 269, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:53:52.815331+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9540': {'gid': 9540, 'name': 'a', 'rank': 0, 'incarnation': 269, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2147565603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2147565603}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2147565603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:53:53.658 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9540:up:replay 2024-09-10T10:53:53.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:53:59.063 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:53:59.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.229+0000 7fec7d000640 1 Processor -- start 2024-09-10T10:53:59.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.229+0000 7fec7d000640 1 -- start start 2024-09-10T10:53:59.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.230+0000 7fec7d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fec78147e90 0x7fec780720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.230+0000 7fec7d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fec780725f0 0x7fec780aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec7d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec78072a00 con 0x7fec78147230 2024-09-10T10:53:59.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec7d000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fec780b34d0 con 0x7fec78147e90 2024-09-10T10:53:59.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec7d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fec780ae9e0 con 0x7fec780725f0 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fec78147e90 0x7fec780720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fec78147e90 0x7fec780720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54774/0 (socket says 172.21.15.28:54774) 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 -- 172.21.15.28:0/1706813425 learned_addr learned my addr 172.21.15.28:0/1706813425 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec76800640 1 --2- 172.21.15.28:0/1706813425 >> v2:172.21.15.138:3300/0 conn(0x7fec780725f0 0x7fec780aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 -- 172.21.15.28:0/1706813425 >> v2:172.21.15.138:3300/0 conn(0x7fec780725f0 msgr2=0x7fec780aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 --2- 172.21.15.28:0/1706813425 >> v2:172.21.15.138:3300/0 conn(0x7fec780725f0 0x7fec780aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 -- 172.21.15.28:0/1706813425 >> v1:172.21.15.138:6789/0 conn(0x7fec78147230 legacy=0x7fec7806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.231+0000 7fec6fe00640 1 -- 172.21.15.28:0/1706813425 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7806f480 con 0x7fec78147e90 2024-09-10T10:53:59.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.232+0000 7fec6fe00640 1 --2- 172.21.15.28:0/1706813425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147e90 0x7fec780720b0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fec78148320 tx=0x7fec6002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3466629b8a3792a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec75400640 1 -- 172.21.15.28:0/1706813425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60037070 con 0x7fec78147e90 2024-09-10T10:53:59.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec75400640 1 -- 172.21.15.28:0/1706813425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60032440 con 0x7fec78147e90 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec75400640 1 -- 172.21.15.28:0/1706813425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60032810 con 0x7fec78147e90 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 -- 172.21.15.28:0/1706813425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147e90 msgr2=0x7fec780720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 --2- 172.21.15.28:0/1706813425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147e90 0x7fec780720b0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fec78148320 tx=0x7fec6002a230 comp rx=0 tx=0).stop 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 -- 172.21.15.28:0/1706813425 shutdown_connections 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 --2- 172.21.15.28:0/1706813425 >> v2:172.21.15.138:3300/0 conn(0x7fec780725f0 0x7fec780aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 --2- 172.21.15.28:0/1706813425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147e90 0x7fec780720b0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 -- 172.21.15.28:0/1706813425 >> 172.21.15.28:0/1706813425 conn(0x7fec78067d70 msgr2=0x7fec7806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.233+0000 7fec7d000640 1 -- 172.21.15.28:0/1706813425 shutdown_connections 2024-09-10T10:53:59.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.234+0000 7fec7d000640 1 -- 172.21.15.28:0/1706813425 wait complete. 2024-09-10T10:53:59.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.235+0000 7fec7d000640 1 Processor -- start 2024-09-10T10:53:59.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.235+0000 7fec7d000640 1 -- start start 2024-09-10T10:53:59.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec780725f0 0x7fec7813f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec78147e90 0x7fec78130dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fec7806eec0 con 0x7fec78147230 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec7806ebe0 con 0x7fec780725f0 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec7d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fec7806f1a0 con 0x7fec78147e90 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec6fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec6fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54776/0 (socket says 172.21.15.28:54776) 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.236+0000 7fec6fe00640 1 -- 172.21.15.28:0/902689486 learned_addr learned my addr 172.21.15.28:0/902689486 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 -- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec78147e90 msgr2=0x7fec78130dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec78147e90 0x7fec78130dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 -- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec780725f0 msgr2=0x7fec7813f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec780725f0 0x7fec7813f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 -- 172.21.15.28:0/902689486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7813f690 con 0x7fec78147230 2024-09-10T10:53:59.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6fe00640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fec60000c00 tx=0x7fec600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60041020 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec600306c0 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60037040 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec780b21a0 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.237+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7812f260 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.238+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec78147610 con 0x7fec78147230 2024-09-10T10:53:59.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.239+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fec60044020 con 0x7fec78147230 2024-09-10T10:53:59.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.240+0000 7fec6e000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.240+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fec600ac410 con 0x7fec78147230 2024-09-10T10:53:59.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.241+0000 7fec75e00640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.241+0000 7fec75e00640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 0x7fec5406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec780b2580 tx=0x7fec680079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.245+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec6004b030 con 0x7fec78147230 2024-09-10T10:53:59.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.466+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec7812ed30 con 0x7fec78147230 2024-09-10T10:53:59.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.467+0000 7fec6e000640 1 -- 172.21.15.28:0/902689486 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) ==== 95+0+5234 (secure 0 0 0) 0x7fec7813f690 con 0x7fec78147230 2024-09-10T10:53:59.469 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:53:59.469 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":272,"btime":"2024-09-10T10:53:55:827350+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:55.827345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9540":{"gid":9540,"name":"a","rank":0,"incarnation":269,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2147565603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2147565603},{"type":"v1","addr":"172.21.15.28:6835","nonce":2147565603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9540,"qdb_cluster":[9540]},"id":6}]} 2024-09-10T10:53:59.469 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 272 2024-09-10T10:53:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 msgr2=0x7fec5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 0x7fec5406e2f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec780b2580 tx=0x7fec680079b0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 msgr2=0x7fec7813dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fec60000c00 tx=0x7fec600046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 shutdown_connections 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fec5406be30 0x7fec5406e2f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fec78147e90 0x7fec78130dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec78147230 0x7fec7813dba0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 --2- 172.21.15.28:0/902689486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec780725f0 0x7fec7813f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 >> 172.21.15.28:0/902689486 conn(0x7fec78067d70 msgr2=0x7fec780aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 shutdown_connections 2024-09-10T10:53:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.469+0000 7fec7d000640 1 -- 172.21.15.28:0/902689486 wait complete. 2024-09-10T10:53:59.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 272, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:53:55.827345+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9540': {'gid': 9540, 'name': 'a', 'rank': 0, 'incarnation': 269, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6835/2147565603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2147565603}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2147565603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9540, 'qdb_cluster': [9540]} 2024-09-10T10:53:59.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:53:59.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:53:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.643+0000 7fe716800640 1 Processor -- start 2024-09-10T10:53:59.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.643+0000 7fe716800640 1 -- start start 2024-09-10T10:53:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072320 0x7fe7100cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe710072de0 0x7fe71006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe710072700 0x7fe71006e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe710146140 con 0x7fe710072700 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe7100cd740 con 0x7fe710072de0 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe716800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100cda20 con 0x7fe710072320 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe70fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe710072700 0x7fe71006e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe70fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe710072700 0x7fe71006e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54786/0 (socket says 172.21.15.28:54786) 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe70fe00640 1 -- 172.21.15.28:0/4046814274 learned_addr learned my addr 172.21.15.28:0/4046814274 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe70f400640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072320 0x7fe7100cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.645+0000 7fe707e00640 1 --2- 172.21.15.28:0/4046814274 >> v2:172.21.15.138:3300/0 conn(0x7fe710072de0 0x7fe71006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.646+0000 7fe70fe00640 1 -- 172.21.15.28:0/4046814274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072320 msgr2=0x7fe7100cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.646+0000 7fe70fe00640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072320 0x7fe7100cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.646+0000 7fe70fe00640 1 -- 172.21.15.28:0/4046814274 >> v2:172.21.15.138:3300/0 conn(0x7fe710072de0 msgr2=0x7fe71006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.646+0000 7fe70fe00640 1 --2- 172.21.15.28:0/4046814274 >> v2:172.21.15.138:3300/0 conn(0x7fe710072de0 0x7fe71006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.646+0000 7fe70fe00640 1 -- 172.21.15.28:0/4046814274 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100cdd00 con 0x7fe710072700 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe70fe00640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072700 0x7fe71006e5f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe7100701f0 tx=0x7fe70802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31be965668066ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe70ea00640 1 -- 172.21.15.28:0/4046814274 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708036070 con 0x7fe710072700 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe70ea00640 1 -- 172.21.15.28:0/4046814274 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe708031600 con 0x7fe710072700 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe70ea00640 1 -- 172.21.15.28:0/4046814274 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7080319d0 con 0x7fe710072700 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 -- 172.21.15.28:0/4046814274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072700 msgr2=0x7fe71006e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072700 0x7fe71006e5f0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fe7100701f0 tx=0x7fe70802a610 comp rx=0 tx=0).stop 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 -- 172.21.15.28:0/4046814274 shutdown_connections 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072700 0x7fe71006e5f0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 --2- 172.21.15.28:0/4046814274 >> v2:172.21.15.138:3300/0 conn(0x7fe710072de0 0x7fe71006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 --2- 172.21.15.28:0/4046814274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072320 0x7fe7100cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.647+0000 7fe716800640 1 -- 172.21.15.28:0/4046814274 >> 172.21.15.28:0/4046814274 conn(0x7fe710067d70 msgr2=0x7fe71006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.648+0000 7fe716800640 1 -- 172.21.15.28:0/4046814274 shutdown_connections 2024-09-10T10:53:59.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.648+0000 7fe716800640 1 -- 172.21.15.28:0/4046814274 wait complete. 2024-09-10T10:53:59.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.649+0000 7fe716800640 1 Processor -- start 2024-09-10T10:53:59.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.649+0000 7fe716800640 1 -- start start 2024-09-10T10:53:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 0x7fe7101f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 0x7fe7100c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100cd520 con 0x7fe710072320 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100cd3a0 con 0x7fe710072700 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.650+0000 7fe716800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe71006eb30 con 0x7fe710072de0 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54798/0 (socket says 172.21.15.28:54798) 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 -- 172.21.15.28:0/123632640 learned_addr learned my addr 172.21.15.28:0/123632640 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe707e00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 0x7fe7101f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70fe00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 0x7fe7100c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 -- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 msgr2=0x7fe7100c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 0x7fe7100c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 -- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 msgr2=0x7fe7101f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 0x7fe7101f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 -- 172.21.15.28:0/123632640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7100abb50 con 0x7fe710072320 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe707e00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 0x7fe7101f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70fe00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 0x7fe7100c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.651+0000 7fe70f400640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe70000e890 tx=0x7fe70000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.652+0000 7fe707400640 1 -- 172.21.15.28:0/123632640 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700017070 con 0x7fe710072320 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.652+0000 7fe707400640 1 -- 172.21.15.28:0/123632640 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe70000c940 con 0x7fe710072320 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.652+0000 7fe707400640 1 -- 172.21.15.28:0/123632640 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000cd10 con 0x7fe710072320 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.652+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7100abe30 con 0x7fe710072320 2024-09-10T10:53:59.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.652+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7100ac3f0 con 0x7fe710072320 2024-09-10T10:53:59.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.655+0000 7fe707400640 1 -- 172.21.15.28:0/123632640 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe70001a070 con 0x7fe710072320 2024-09-10T10:53:59.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.655+0000 7fe707400640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 0x7fe6ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.656+0000 7fe707400640 1 -- 172.21.15.28:0/123632640 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fe70008bdf0 con 0x7fe710072320 2024-09-10T10:53:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.656+0000 7fe7196d4740 1 Processor -- start 2024-09-10T10:53:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.656+0000 7fe7196d4740 1 -- start start 2024-09-10T10:53:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.656+0000 7fe707e00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 0x7fe6ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe707e00640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 0x7fe6ec06e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe7100adc30 tx=0x7fe6f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x558925423cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589253db4b0 0x558925424200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55892533c3b0 con 0x558925407b50 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558925341410 con 0x5589253db4b0 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5589252e9620 con 0x558925422510 2024-09-10T10:53:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54806/0 (socket says 172.21.15.28:54806) 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70f400640 1 -- 172.21.15.28:0/2048660933 learned_addr learned my addr 172.21.15.28:0/2048660933 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70fe00640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589253db4b0 0x558925424200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe707e00640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x558925423cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70f400640 1 -- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 msgr2=0x558925423cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.657+0000 7fe70f400640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x558925423cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe70f400640 1 -- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589253db4b0 msgr2=0x558925424200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe70f400640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589253db4b0 0x558925424200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe70f400640 1 -- 172.21.15.28:0/2048660933 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589250542d0 con 0x558925407b50 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe707e00640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x558925423cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe70f400640 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe700061ad0 tx=0x7fe700059c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e913089c9cec53f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe6ffe00640 1 -- 172.21.15.28:0/2048660933 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700021040 con 0x558925407b50 2024-09-10T10:53:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe6ffe00640 1 -- 172.21.15.28:0/2048660933 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700062000 con 0x558925407b50 2024-09-10T10:53:59.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.658+0000 7fe6ffe00640 1 -- 172.21.15.28:0/2048660933 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700055a90 con 0x558925407b50 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 -- 172.21.15.28:0/2048660933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 msgr2=0x558925407f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe700061ad0 tx=0x7fe700059c60 comp rx=0 tx=0).stop 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 -- 172.21.15.28:0/2048660933 shutdown_connections 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5589253db4b0 0x558925424200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x558925423cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 --2- 172.21.15.28:0/2048660933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x558925407f30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 -- 172.21.15.28:0/2048660933 >> 172.21.15.28:0/2048660933 conn(0x5589253da740 msgr2=0x558925422960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 -- 172.21.15.28:0/2048660933 shutdown_connections 2024-09-10T10:53:59.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.659+0000 7fe7196d4740 1 -- 172.21.15.28:0/2048660933 wait complete. 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.660+0000 7fe7196d4740 1 -- start start 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.660+0000 7fe7196d4740 1 Processor -- start 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 0x5589253f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589252fc780 con 0x558925422510 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55892533d3d0 con 0x558925407b50 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x558925408910 con 0x55892542b8c0 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe7196d4740 10 client.0 _ll_register_callbacks cb 0x55892533a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54816/0 (socket says 172.21.15.28:54816) 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 -- 172.21.15.28:0/1796652984 learned_addr learned my addr 172.21.15.28:0/1796652984 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe70f400640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe70fe00640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 0x5589253f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 -- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 msgr2=0x5589253f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 0x5589253f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 -- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 msgr2=0x5589253dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe707e00640 1 -- 172.21.15.28:0/1796652984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892511e6b0 con 0x558925422510 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.661+0000 7fe70fe00640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 0x5589253f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe707e00640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80157d0 tx=0x7fe6f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948f0a9223c42821 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe6fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8026070 con 0x558925422510 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe700062000 con 0x558925422510 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fe6f8014960 con 0x558925422510 2024-09-10T10:53:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=272) ==== 24+0+0 (secure 0 0 0) 0x7fe6f8032050 con 0x558925422510 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe7196d4740 10 client.9624 fetch_fsmap learned FSMap version 272 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=272,monmap=2+}) -- 0x5589253dc500 con 0x558925422510 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.662+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe700062000 con 0x558925422510 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 272) ==== 3502+0+0 (secure 0 0 0) 0x7fe6f8014b60 con 0x558925422510 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 10 client.9624 fetch_fsmap finished waiting for FSMap version 272 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 10 client.9624 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 20 client.9624 populate_metadata read hostname 'smithi028' 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55892542bcc0 0x55892543aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 4 client.9624 mds_command: new command op to 9540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 --> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55892534eb40 con 0x55892542bcc0 2024-09-10T10:53:59.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.663+0000 7fe70f400640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55892542bcc0 0x55892543aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.664+0000 7fe70f400640 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55892542bcc0 0x55892543aeb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0d57b3c5bef108 server_cookie=b0126acdd1fd0b58 in_seq=0 out_seq=0 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.664+0000 7fe6fcc00640 10 client.9624 ms_handle_connect on v2:172.21.15.28:6833/2147565603 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.664+0000 7fe6fcc00640 1 -- 172.21.15.28:0/1796652984 <== mds.0 v2:172.21.15.28:6833/2147565603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55892534eb40 con 0x55892542bcc0 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.664+0000 7fe6fcc00640 10 client.9624 handle_command_reply: tid=0 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.664+0000 7fe7196d4740 1 client.9624 shutdown 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 msgr2=0x5589253f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80157d0 tx=0x7fe6f80046c0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 shutdown_connections 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55892542b8c0 0x5589253f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925422510 0x5589253f47e0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55892542bcc0 0x55892543aeb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 --2- 172.21.15.28:0/1796652984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 >> 172.21.15.28:0/1796652984 conn(0x5589253db4b0 msgr2=0x55892542c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe6fcc00640 0 client.9624 ms_handle_reset on v2:172.21.15.28:6833/2147565603 2024-09-10T10:53:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 shutdown_connections 2024-09-10T10:53:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 1 -- 172.21.15.28:0/1796652984 wait complete. 2024-09-10T10:53:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.665+0000 7fe7196d4740 20 client.9624 trim_cache size 0 max 16384 2024-09-10T10:53:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.684+0000 7fe7196d4740 1 Processor -- start 2024-09-10T10:53:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.684+0000 7fe7196d4740 1 -- start start 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x5589254412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7101fda50 con 0x55892542b8c0 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7100b4db0 con 0x558925407b50 2024-09-10T10:53:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe7100b5400 con 0x558925422510 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54832/0 (socket says 172.21.15.28:54832) 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 -- 172.21.15.28:0/3251092783 learned_addr learned my addr 172.21.15.28:0/3251092783 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70f400640 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe707e00640 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x5589254412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 -- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 msgr2=0x5589254412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x5589254412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 -- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 msgr2=0x5589253f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.685+0000 7fe70fe00640 1 -- 172.21.15.28:0/3251092783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589253f41b0 con 0x55892542b8c0 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe70fe00640 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fe708009f10 tx=0x7fe70802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5478953e21258b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe6ff400640 1 -- 172.21.15.28:0/3251092783 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708036040 con 0x55892542b8c0 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe6ff400640 1 -- 172.21.15.28:0/3251092783 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7080317e0 con 0x55892542b8c0 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe6ff400640 1 -- 172.21.15.28:0/3251092783 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708031bd0 con 0x55892542b8c0 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe7196d4740 1 -- 172.21.15.28:0/3251092783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 msgr2=0x558925406fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.686+0000 7fe7196d4740 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fe708009f10 tx=0x7fe70802e000 comp rx=0 tx=0).stop 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 -- 172.21.15.28:0/3251092783 shutdown_connections 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55892542b8c0 0x558925406fe0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x5589254412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 --2- 172.21.15.28:0/3251092783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558925407b50 0x5589253f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 -- 172.21.15.28:0/3251092783 >> 172.21.15.28:0/3251092783 conn(0x5589253db4b0 msgr2=0x5589253f4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 -- 172.21.15.28:0/3251092783 shutdown_connections 2024-09-10T10:53:59.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 -- 172.21.15.28:0/3251092783 wait complete. 2024-09-10T10:53:59.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 -- start start 2024-09-10T10:53:59.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.687+0000 7fe7196d4740 1 Processor -- start 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x55892541f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55892542b8c0 0x5589253dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5589254213b0 con 0x558925407b50 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5589253f26e0 con 0x55892542b8c0 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55892543a920 con 0x558925422510 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.688+0000 7fe7196d4740 10 client.0 _ll_register_callbacks cb 0x7fe6f8011dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54834/0 (socket says 172.21.15.28:54834) 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 -- 172.21.15.28:0/442784785 learned_addr learned my addr 172.21.15.28:0/442784785 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe707e00640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x55892541f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70fe00640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55892542b8c0 0x5589253dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 -- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 msgr2=0x55892541f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x55892541f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 -- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55892542b8c0 msgr2=0x5589253dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55892542b8c0 0x5589253dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 -- 172.21.15.28:0/442784785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892541e9b0 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe70f400640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe7000593b0 tx=0x7fe70008b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfde89f797388b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe6fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.689+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700021040 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fe700055a90 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7080317e0 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=272) ==== 24+0+0 (secure 0 0 0) 0x7fe700062000 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 10 client.9630 fetch_fsmap learned FSMap version 272 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=272,monmap=2+}) -- 0x5589253ddf40 con 0x558925407b50 2024-09-10T10:53:59.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7080317e0 con 0x558925407b50 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 272) ==== 3502+0+0 (secure 0 0 0) 0x7fe7000781f0 con 0x558925407b50 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 10 client.9630 fetch_fsmap finished waiting for FSMap version 272 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 10 client.9630 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 20 client.9630 populate_metadata read hostname 'smithi028' 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x5589253d7600 0x5589253d79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 4 client.9630 mds_command: new command op to 9540 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 --> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] -- command(tid 0: {"prefix": "status"}) -- 0x558925348e90 con 0x5589253d7600 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.690+0000 7fe707e00640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x5589253d7600 0x5589253d79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe707e00640 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x5589253d7600 0x5589253d79e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e1306d90e3608a server_cookie=c08a1c32545d1270 in_seq=0 out_seq=0 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe6fcc00640 10 client.9630 ms_handle_connect on v2:172.21.15.28:6833/2147565603 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe6fcc00640 1 -- 172.21.15.28:0/442784785 <== mds.0 v2:172.21.15.28:6833/2147565603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x558925348e90 con 0x5589253d7600 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe6fcc00640 10 client.9630 handle_command_reply: tid=0 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 client.9630 shutdown 2024-09-10T10:53:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 msgr2=0x5589254a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fe7000593b0 tx=0x7fe70008b070 comp rx=0 tx=0).stop 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 shutdown_connections 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x5589253d7600 0x5589253d79e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55892542b8c0 0x5589253dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x558925422510 0x55892541f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 --2- 172.21.15.28:0/442784785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558925407b50 0x5589254a5430 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe6fcc00640 0 client.9630 ms_handle_reset on v2:172.21.15.28:6833/2147565603 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.691+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 >> 172.21.15.28:0/442784785 conn(0x5589253db4b0 msgr2=0x55892540aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 shutdown_connections 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe7196d4740 1 -- 172.21.15.28:0/442784785 wait complete. 2024-09-10T10:53:59.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe7196d4740 20 client.9630 trim_cache size 0 max 16384 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 9540, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8757563990000001, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 272, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 131, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 131, 2024-09-10T10:53:59.698 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 10.138199064 2024-09-10T10:53:59.699 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:53:59.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 msgr2=0x7fe6ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 0x7fe6ec06e2f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe7100adc30 tx=0x7fe6f8008040 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 msgr2=0x7fe71015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe70000e890 tx=0x7fe70000ed50 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 shutdown_connections 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6ec06be30 0x7fe6ec06e2f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe710072de0 0x7fe7100c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe710072700 0x7fe7101f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 --2- 172.21.15.28:0/123632640 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe710072320 0x7fe71015bc20 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.692+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 >> 172.21.15.28:0/123632640 conn(0x7fe710067d70 msgr2=0x7fe71006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.693+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 shutdown_connections 2024-09-10T10:53:59.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.693+0000 7fe716800640 1 -- 172.21.15.28:0/123632640 wait complete. 2024-09-10T10:53:59.710 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:53:59.710 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 10000000000.00000000:sixmegs_head', proceeding to ls 2024-09-10T10:53:59.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:53:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.864+0000 7f1b6be00640 1 Processor -- start 2024-09-10T10:53:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.864+0000 7f1b6be00640 1 -- start start 2024-09-10T10:53:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.865+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 0x7f1b6c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.865+0000 7f1b6be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b6c0d1f90 0x7f1b6c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.865+0000 7f1b6be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b6c0cf000 con 0x7f1b6c0d0dd0 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.865+0000 7f1b6be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1b6c0b37d0 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.865+0000 7f1b6be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c0cf330 con 0x7f1b6c15e660 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6aa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1b6c0d0dd0 0x7f1b6c0d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42324/0 (socket says 172.21.15.28:42324) 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6aa00640 1 -- 172.21.15.28:0/2501895396 learned_addr learned my addr 172.21.15.28:0/2501895396 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6b400640 1 --2- 172.21.15.28:0/2501895396 >> v2:172.21.15.28:3300/0 conn(0x7f1b6c0d1f90 0x7f1b6c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6a000640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 0x7f1b6c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6b400640 1 -- 172.21.15.28:0/2501895396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 msgr2=0x7f1b6c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:53:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6b400640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 0x7f1b6c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6b400640 1 -- 172.21.15.28:0/2501895396 >> v1:172.21.15.28:6789/0 conn(0x7f1b6c0d0dd0 legacy=0x7f1b6c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6b400640 1 -- 172.21.15.28:0/2501895396 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c0d29a0 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b69600640 1 -- 172.21.15.28:0/2501895396 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307564570 0 0) 0x7f1b6c0cf000 con 0x7f1b6c0d0dd0 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.866+0000 7f1b6a000640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 0x7f1b6c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.867+0000 7f1b6b400640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c0cf7c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f1b640097a0 tx=0x7f1b6402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b0161a16381593 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.867+0000 7f1b69600640 1 -- 172.21.15.28:0/2501895396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64037070 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.867+0000 7f1b69600640 1 -- 172.21.15.28:0/2501895396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64032440 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.867+0000 7f1b69600640 1 -- 172.21.15.28:0/2501895396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b64032810 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 -- 172.21.15.28:0/2501895396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 msgr2=0x7f1b6c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c0cf7c0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f1b640097a0 tx=0x7f1b6402a180 comp rx=0 tx=0).stop 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 -- 172.21.15.28:0/2501895396 shutdown_connections 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c0cf7c0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2501895396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c15e660 0x7f1b6c06ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 -- 172.21.15.28:0/2501895396 >> 172.21.15.28:0/2501895396 conn(0x7f1b6c08bfc0 msgr2=0x7f1b6c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 -- 172.21.15.28:0/2501895396 shutdown_connections 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.868+0000 7f1b6be00640 1 -- 172.21.15.28:0/2501895396 wait complete. 2024-09-10T10:53:59.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.870+0000 7f1b6be00640 1 Processor -- start 2024-09-10T10:53:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.870+0000 7f1b6be00640 1 -- start start 2024-09-10T10:53:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 0x7f1b6c1f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 0x7f1b6c1f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d3220 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d2f40 con 0x7f1b6c15e660 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1b6c0d33a0 con 0x7f1b6c0d0dd0 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54852/0 (socket says 172.21.15.28:54852) 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 -- 172.21.15.28:0/2656001392 learned_addr learned my addr 172.21.15.28:0/2656001392 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6aa00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 0x7f1b6c1f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6b400640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 0x7f1b6c1f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 -- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 msgr2=0x7f1b6c1f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 0x7f1b6c1f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 -- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 msgr2=0x7f1b6c1f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 0x7f1b6c1f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:53:59.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6a000640 1 -- 172.21.15.28:0/2656001392 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c0c9220 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6aa00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 0x7f1b6c1f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.871+0000 7f1b6b400640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 0x7f1b6c1f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b6a000640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800d7d0 tx=0x7f1b5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b580096a0 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b58019070 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c0c94a0 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b58009a80 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.872+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c0c9a60 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.873+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c0d2440 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.874+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1b58009c20 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.875+0000 7f1b63400640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 0x7f1b4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:53:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.875+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f1b5808bfd0 con 0x7f1b6c0d1f90 2024-09-10T10:53:59.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.875+0000 7f1b6aa00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 0x7f1b4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:53:59.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.876+0000 7f1b6aa00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 0x7f1b4c06e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b5c005b70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:53:59.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:53:59.879+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1b6c0c9220 con 0x7f1b6c0d1f90 2024-09-10T10:54:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.101+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b6c15ca40 con 0x7f1b6c0d1f90 2024-09-10T10:54:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.101+0000 7f1b63400640 1 -- 172.21.15.28:0/2656001392 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v272) ==== 53+0+73 (secure 0 0 0) 0x7f1b58059c10 con 0x7f1b6c0d1f90 2024-09-10T10:54:00.103 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 msgr2=0x7f1b4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 0x7f1b4c06e2f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c007fd0 tx=0x7f1b5c005b70 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 msgr2=0x7f1b6c1f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800d7d0 tx=0x7f1b5800dc90 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 shutdown_connections 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1b4c06be30 0x7f1b4c06e2f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b6c15e660 0x7f1b6c1f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b6c0d1f90 0x7f1b6c1f7760 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.103+0000 7f1b6be00640 1 --2- 172.21.15.28:0/2656001392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1b6c0d0dd0 0x7f1b6c1f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.104+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 >> 172.21.15.28:0/2656001392 conn(0x7f1b6c08bfc0 msgr2=0x7f1b6c0b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.104+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 shutdown_connections 2024-09-10T10:54:00.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00.104+0000 7f1b6be00640 1 -- 172.21.15.28:0/2656001392 wait complete. 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:54:00.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:54:00.123 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:54:00.123 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:00.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.123 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:54:00.159 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42065sec preferred_lft 42065sec 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:54:00.160 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:54:00.161 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:00.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.162 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:54:00.252 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:54:00.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.255 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:54:00.291 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:54:00.292 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:54:00.292 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:54:00.292 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:54:00.292 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:54:00.292 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:54:00.292 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:54:00.292 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:54:00.292 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:54:00.293 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:54:00.293 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:54:00.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:54:00.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:00.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.488 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:54:00.523 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:54:00.524 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:54:00.524 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:00.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:54:00.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:00.621 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:00.656 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:00.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:00.656 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:00.732 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:54:00.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.732 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:00.805 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:54:00.803+0000 7fa7c5600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:00.861 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:00.862 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:00.862 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:00.877 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:00.877 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:00.877 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:00.877 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:00.877 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:54:00.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:54:00.903 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:54:00.903 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:54:00.903 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:54:00.903 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:54:00.903 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:54:00.906 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:54:00.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:00.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:01.001 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:54:01.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:54:01.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:01.092 INFO:teuthology.orchestra.run.smithi028.stdout:client7898 2024-09-10T10:54:01.092 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:01.092 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:01.128 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:54:01.128 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:54:01.128 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> import json 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir/sixmegs'"'"') 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> except OSError as e: 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(e.errno) 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> print(json.dumps( 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T10:54:01.129 DEBUG:teuthology.orchestra.run.smithi028:> indent=2)) 2024-09-10T10:54:01.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:54:01.173 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:54:01.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:01 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T10:54:07.275 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T10:54:07.276 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-10T10:54:07.276 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:07.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.443+0000 7ff73c400640 1 Processor -- start 2024-09-10T10:54:07.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.443+0000 7ff73c400640 1 -- start start 2024-09-10T10:54:07.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff73c400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff73415e5f0 0x7ff7340d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff73c400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7340d30f0 0x7ff7340b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff73c400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7340d3690 con 0x7ff7340b64c0 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff73c400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff73406d530 con 0x7ff7340d30f0 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff73c400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7340d3970 con 0x7ff73415e5f0 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.444+0000 7ff739000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7340d30f0 0x7ff7340b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7340d30f0 0x7ff7340b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36864/0 (socket says 172.21.15.28:36864) 2024-09-10T10:54:07.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 -- 172.21.15.28:0/3253024333 learned_addr learned my addr 172.21.15.28:0/3253024333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 -- 172.21.15.28:0/3253024333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff73415e5f0 msgr2=0x7ff7340d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 --2- 172.21.15.28:0/3253024333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff73415e5f0 0x7ff7340d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 -- 172.21.15.28:0/3253024333 >> v1:172.21.15.28:6789/0 conn(0x7ff7340b64c0 legacy=0x7ff7340cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.445+0000 7ff739000640 1 -- 172.21.15.28:0/3253024333 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73406cee0 con 0x7ff7340d30f0 2024-09-10T10:54:07.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.446+0000 7ff739000640 1 --2- 172.21.15.28:0/3253024333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340d30f0 0x7ff7340b5f80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff720009870 tx=0x7ff72002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a95f67e2c7dc20e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.446+0000 7ff733e00640 1 -- 172.21.15.28:0/3253024333 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720037070 con 0x7ff7340d30f0 2024-09-10T10:54:07.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.446+0000 7ff733e00640 1 -- 172.21.15.28:0/3253024333 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff720032440 con 0x7ff7340d30f0 2024-09-10T10:54:07.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.446+0000 7ff733e00640 1 -- 172.21.15.28:0/3253024333 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720032810 con 0x7ff7340d30f0 2024-09-10T10:54:07.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 -- 172.21.15.28:0/3253024333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340d30f0 msgr2=0x7ff7340b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 --2- 172.21.15.28:0/3253024333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340d30f0 0x7ff7340b5f80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7ff720009870 tx=0x7ff72002a250 comp rx=0 tx=0).stop 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 -- 172.21.15.28:0/3253024333 shutdown_connections 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 --2- 172.21.15.28:0/3253024333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340d30f0 0x7ff7340b5f80 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 --2- 172.21.15.28:0/3253024333 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff73415e5f0 0x7ff7340d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 -- 172.21.15.28:0/3253024333 >> 172.21.15.28:0/3253024333 conn(0x7ff73408bfc0 msgr2=0x7ff7340d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 -- 172.21.15.28:0/3253024333 shutdown_connections 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.447+0000 7ff73c400640 1 -- 172.21.15.28:0/3253024333 wait complete. 2024-09-10T10:54:07.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.448+0000 7ff73c400640 1 Processor -- start 2024-09-10T10:54:07.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.449+0000 7ff73c400640 1 -- start start 2024-09-10T10:54:07.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.449+0000 7ff73c400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.449+0000 7ff73c400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7340d30f0 0x7ff7340b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 0x7ff7340b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73c400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7340d5310 con 0x7ff7340b64c0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff73415e9d0 con 0x7ff73415e5f0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73c400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff73406e9d0 con 0x7ff7340d30f0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36878/0 (socket says 172.21.15.28:36878) 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 -- 172.21.15.28:0/3578295205 learned_addr learned my addr 172.21.15.28:0/3578295205 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739000640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7340d30f0 0x7ff7340b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73a400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 0x7ff7340b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 -- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7340d30f0 msgr2=0x7ff7340b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7340d30f0 0x7ff7340b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 -- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 msgr2=0x7ff7340b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 0x7ff7340b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff739a00640 1 -- 172.21.15.28:0/3578295205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7340b35c0 con 0x7ff7340b64c0 2024-09-10T10:54:07.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.450+0000 7ff73a400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 0x7ff7340b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:07.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff739a00640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff72800b410 tx=0x7ff72800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728015020 con 0x7ff7340b64c0 2024-09-10T10:54:07.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728011850 con 0x7ff7340b64c0 2024-09-10T10:54:07.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728011c40 con 0x7ff7340b64c0 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7340b29b0 con 0x7ff7340b64c0 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.451+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7340b2f70 con 0x7ff7340b64c0 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.452+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7340b5f80 con 0x7ff7340b64c0 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.453+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff728018020 con 0x7ff7340b64c0 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.455+0000 7ff731600640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 0x7ff71406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.455+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7ff72808b9f0 con 0x7ff7340b64c0 2024-09-10T10:54:07.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.455+0000 7ff739000640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 0x7ff71406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.456+0000 7ff739000640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 0x7ff71406e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff720034000 tx=0x7ff72002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.460+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff72801f020 con 0x7ff7340b64c0 2024-09-10T10:54:07.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.686+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7340b3310 con 0x7ff7340b64c0 2024-09-10T10:54:07.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.687+0000 7ff731600640 1 -- 172.21.15.28:0/3578295205 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 272 v272) ==== 95+0+5234 (secure 0 0 0) 0x7ff7280596c0 con 0x7ff7340b64c0 2024-09-10T10:54:07.688 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:07.688 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":272,"btime":"2024-09-10T10:53:55:827350+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":272,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:53:55.827345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9540":{"gid":9540,"name":"a","rank":0,"incarnation":269,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2147565603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2147565603},{"type":"v1","addr":"172.21.15.28:6835","nonce":2147565603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9540,"qdb_cluster":[9540]},"id":6}]} 2024-09-10T10:54:07.688 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 272 2024-09-10T10:54:07.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 msgr2=0x7ff71406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 0x7ff71406e2f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff720034000 tx=0x7ff72002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:07.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 msgr2=0x7ff7340cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff72800b410 tx=0x7ff72800b8d0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 shutdown_connections 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff71406be30 0x7ff71406e2f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff73415e5f0 0x7ff7340b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7340d30f0 0x7ff7340b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 --2- 172.21.15.28:0/3578295205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7340b64c0 0x7ff7340cdd40 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 >> 172.21.15.28:0/3578295205 conn(0x7ff73408bfc0 msgr2=0x7ff7340b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 shutdown_connections 2024-09-10T10:54:07.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.689+0000 7ff73c400640 1 -- 172.21.15.28:0/3578295205 wait complete. 2024-09-10T10:54:07.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T10:54:07.715 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:07.714+0000 7fcc62600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T10:54:07.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.864+0000 7efea2000640 1 Processor -- start 2024-09-10T10:54:07.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.864+0000 7efea2000640 1 -- start start 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efea2000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe9c0b1c00 0x7efe9c1696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efea2000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9c0d2ff0 con 0x7efe9c15f9a0 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efea2000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9c15fd80 con 0x7efe9c0b32a0 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efea2000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efe9c0d19a0 con 0x7efe9c0b1c00 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efe9c15f9a0 0x7efe9c06d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33734/0 (socket says 172.21.15.28:33734) 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9b400640 1 -- 172.21.15.28:0/3107227875 learned_addr learned my addr 172.21.15.28:0/3107227875 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:07.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9be00640 1 --2- 172.21.15.28:0/3107227875 >> v2:172.21.15.28:3300/0 conn(0x7efe9c0b1c00 0x7efe9c1696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9aa00640 1 -- 172.21.15.28:0/3107227875 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493712727 0 0) 0x7efe9c0d2ff0 con 0x7efe9c15f9a0 2024-09-10T10:54:07.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9aa00640 1 -- 172.21.15.28:0/3107227875 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe80003520 con 0x7efe9c15f9a0 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9be00640 1 -- 172.21.15.28:0/3107227875 >> v1:172.21.15.138:6789/0 conn(0x7efe9c0b32a0 legacy=0x7efe9c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9be00640 1 -- 172.21.15.28:0/3107227875 >> v1:172.21.15.28:6789/0 conn(0x7efe9c15f9a0 legacy=0x7efe9c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.866+0000 7efe9be00640 1 -- 172.21.15.28:0/3107227875 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c0d1100 con 0x7efe9c0b1c00 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.867+0000 7efe9be00640 1 --2- 172.21.15.28:0/3107227875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c1696e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7efe94009d90 tx=0x7efe9402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b541944073d4a342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efe9aa00640 1 -- 172.21.15.28:0/3107227875 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94007740 con 0x7efe9c0b1c00 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efe9aa00640 1 -- 172.21.15.28:0/3107227875 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe940078e0 con 0x7efe9c0b1c00 2024-09-10T10:54:07.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efe9aa00640 1 -- 172.21.15.28:0/3107227875 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94007c90 con 0x7efe9c0b1c00 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 -- 172.21.15.28:0/3107227875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 msgr2=0x7efe9c1696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 --2- 172.21.15.28:0/3107227875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c1696e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7efe94009d90 tx=0x7efe9402a6f0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 -- 172.21.15.28:0/3107227875 shutdown_connections 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 --2- 172.21.15.28:0/3107227875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c1696e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 -- 172.21.15.28:0/3107227875 >> 172.21.15.28:0/3107227875 conn(0x7efe9c0d2310 msgr2=0x7efe9c0d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 -- 172.21.15.28:0/3107227875 shutdown_connections 2024-09-10T10:54:07.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.868+0000 7efea2000640 1 -- 172.21.15.28:0/3107227875 wait complete. 2024-09-10T10:54:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.870+0000 7efea2000640 1 Processor -- start 2024-09-10T10:54:07.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.870+0000 7efea2000640 1 -- start start 2024-09-10T10:54:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c0b32a0 0x7efe9c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c15f9a0 0x7efe9c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c169c20 con 0x7efe9c0b1c00 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c0d16c0 con 0x7efe9c15f9a0 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efea2000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe9c169da0 con 0x7efe9c0b32a0 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efe9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efe9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36900/0 (socket says 172.21.15.28:36900) 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efe9b400640 1 -- 172.21.15.28:0/619943314 learned_addr learned my addr 172.21.15.28:0/619943314 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efe9be00640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c15f9a0 0x7efe9c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.871+0000 7efe93e00640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c0b32a0 0x7efe9c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 -- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c0b32a0 msgr2=0x7efe9c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c0b32a0 0x7efe9c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 -- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c15f9a0 msgr2=0x7efe9c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c15f9a0 0x7efe9c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 -- 172.21.15.28:0/619943314 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c156a40 con 0x7efe9c0b1c00 2024-09-10T10:54:07.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe9b400640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c410 tx=0x7efe8c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe93400640 1 -- 172.21.15.28:0/619943314 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c017070 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe93400640 1 -- 172.21.15.28:0/619943314 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8c005ce0 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efe93400640 1 -- 172.21.15.28:0/619943314 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0072f0 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c0cb440 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.872+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c0cba00 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.874+0000 7efe93400640 1 -- 172.21.15.28:0/619943314 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe8c01a070 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.875+0000 7efe93400640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 0x7efe7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.875+0000 7efe93400640 1 -- 172.21.15.28:0/619943314 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7efe8c08afb0 con 0x7efe9c0b1c00 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.876+0000 7efea500d740 1 Processor -- start 2024-09-10T10:54:07.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.876+0000 7efe93e00640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 0x7efe7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.876+0000 7efea500d740 1 -- start start 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 0x7efe7806e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0cea70 tx=0x7efe84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da891030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da845e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a4da7af760 con 0x55a4da849050 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4da7576c0 con 0x55a4da84d020 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a4da7ab720 con 0x55a4da84b890 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36904/0 (socket says 172.21.15.28:36904) 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 -- 172.21.15.28:0/3466612061 learned_addr learned my addr 172.21.15.28:0/3466612061 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe9b400640 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da891030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe9be00640 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da845e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 -- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 msgr2=0x55a4da891030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da891030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 -- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 msgr2=0x55a4da845e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.877+0000 7efe93e00640 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da845e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.878+0000 7efe93e00640 1 -- 172.21.15.28:0/3466612061 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4da58e7c0 con 0x55a4da849050 2024-09-10T10:54:07.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.878+0000 7efe93e00640 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7efe840052c0 tx=0x7efe84011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d162fa062c99771 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.878+0000 7efe8b400640 1 -- 172.21.15.28:0/3466612061 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400e4a0 con 0x55a4da849050 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.878+0000 7efe8b400640 1 -- 172.21.15.28:0/3466612061 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8400e640 con 0x55a4da849050 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.878+0000 7efe8b400640 1 -- 172.21.15.28:0/3466612061 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400e9f0 con 0x55a4da849050 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- 172.21.15.28:0/3466612061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 msgr2=0x55a4da84bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7efe840052c0 tx=0x7efe84011000 comp rx=0 tx=0).stop 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- 172.21.15.28:0/3466612061 shutdown_connections 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da845e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da849050 0x55a4da84bc70 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 --2- 172.21.15.28:0/3466612061 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da891030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- 172.21.15.28:0/3466612061 >> 172.21.15.28:0/3466612061 conn(0x55a4da848050 msgr2=0x55a4da890500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- 172.21.15.28:0/3466612061 shutdown_connections 2024-09-10T10:54:07.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- 172.21.15.28:0/3466612061 wait complete. 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.879+0000 7efea500d740 1 -- start start 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.880+0000 7efea500d740 1 Processor -- start 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84d020 0x55a4da86baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da86f690 0x55a4da874520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a4da8474f0 con 0x55a4da84b890 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4da76a8e0 con 0x55a4da86f690 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a4da891860 con 0x55a4da84d020 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efea500d740 10 client.0 _ll_register_callbacks cb 0x55a4da75c3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36906/0 (socket says 172.21.15.28:36906) 2024-09-10T10:54:07.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 -- 172.21.15.28:0/2724264524 learned_addr learned my addr 172.21.15.28:0/2724264524 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe93e00640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84d020 0x55a4da86baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9be00640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da86f690 0x55a4da874520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 -- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84d020 msgr2=0x55a4da86baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84d020 0x55a4da86baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 -- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da86f690 msgr2=0x55a4da874520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da86f690 0x55a4da874520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.881+0000 7efe9b400640 1 -- 172.21.15.28:0/2724264524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4da5eef50 con 0x55a4da84b890 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe9b400640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7efe8c054aa0 tx=0x7efe8c058700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63098ca983bf1354 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c060f90 con 0x55a4da84b890 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7efe8c05db80 con 0x55a4da84b890 2024-09-10T10:54:07.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8400e640 con 0x55a4da84b890 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7efe8c05dd80 con 0x55a4da84b890 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 10 client.9663 fetch_fsmap learned FSMap version 273 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55a4da4820b0 con 0x55a4da84b890 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8400e640 con 0x55a4da84b890 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 273) ==== 3502+0+0 (secure 0 0 0) 0x7efe8c086070 con 0x55a4da84b890 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 10 client.9663 fetch_fsmap finished waiting for FSMap version 273 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 10 client.9663 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 20 client.9663 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da86cdf0 0x55a4da86d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 4 client.9663 mds_command: new command op to 9540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.882+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 --> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4da7bce90 con 0x55a4da86cdf0 2024-09-10T10:54:07.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.883+0000 7efe93e00640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da86cdf0 0x55a4da86d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.883+0000 7efe93e00640 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da86cdf0 0x55a4da86d1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8d01ac66a082a0 server_cookie=bb6f67963038c275 in_seq=0 out_seq=0 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.883+0000 7efe7fe00640 10 client.9663 ms_handle_connect on v2:172.21.15.28:6833/2147565603 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efe7fe00640 1 -- 172.21.15.28:0/2724264524 <== mds.0 v2:172.21.15.28:6833/2147565603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4da7bce90 con 0x55a4da86cdf0 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efe7fe00640 10 client.9663 handle_command_reply: tid=0 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 client.9663 shutdown 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 msgr2=0x55a4da8956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7efe8c054aa0 tx=0x7efe8c058700 comp rx=0 tx=0).stop 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 shutdown_connections 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da86cdf0 0x55a4da86d1d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da86f690 0x55a4da874520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84d020 0x55a4da86baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 --2- 172.21.15.28:0/2724264524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84b890 0x55a4da8956b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efe7fe00640 0 client.9663 ms_handle_reset on v2:172.21.15.28:6833/2147565603 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.884+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 >> 172.21.15.28:0/2724264524 conn(0x55a4da849050 msgr2=0x55a4da88e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.885+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 shutdown_connections 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.885+0000 7efea500d740 1 -- 172.21.15.28:0/2724264524 wait complete. 2024-09-10T10:54:07.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.885+0000 7efea500d740 20 client.9663 trim_cache size 0 max 16384 2024-09-10T10:54:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 Processor -- start 2024-09-10T10:54:07.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 -- start start 2024-09-10T10:54:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 0x55a4da88ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 0x55a4da86d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c201fb0 con 0x55a4da84d020 2024-09-10T10:54:07.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c201290 con 0x55a4da84b890 2024-09-10T10:54:07.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.899+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe9c2018e0 con 0x55a4da86f690 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36910/0 (socket says 172.21.15.28:36910) 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 -- 172.21.15.28:0/2961589427 learned_addr learned my addr 172.21.15.28:0/2961589427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 -- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 msgr2=0x55a4da86d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe9be00640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 0x55a4da86d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe9b400640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 0x55a4da88ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 0x55a4da86d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 -- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 msgr2=0x55a4da88ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 0x55a4da88ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 -- 172.21.15.28:0/2961589427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4da895470 con 0x55a4da84d020 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe9be00640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 0x55a4da86d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:07.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe9b400640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 0x55a4da88ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.900+0000 7efe93e00640 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7efe8400e630 tx=0x7efe84017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28063ebac3e83dc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efe8aa00640 1 -- 172.21.15.28:0/2961589427 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400eb70 con 0x55a4da84d020 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efe8aa00640 1 -- 172.21.15.28:0/2961589427 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8400ed10 con 0x55a4da84d020 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efe8aa00640 1 -- 172.21.15.28:0/2961589427 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84021650 con 0x55a4da84d020 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 -- 172.21.15.28:0/2961589427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 msgr2=0x55a4da86d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7efe8400e630 tx=0x7efe84017070 comp rx=0 tx=0).stop 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 -- 172.21.15.28:0/2961589427 shutdown_connections 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da86f690 0x55a4da86d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da84d020 0x55a4da86d1d0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 --2- 172.21.15.28:0/2961589427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84b890 0x55a4da88ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 -- 172.21.15.28:0/2961589427 >> 172.21.15.28:0/2961589427 conn(0x55a4da849050 msgr2=0x55a4da874500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 -- 172.21.15.28:0/2961589427 shutdown_connections 2024-09-10T10:54:07.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.901+0000 7efea500d740 1 -- 172.21.15.28:0/2961589427 wait complete. 2024-09-10T10:54:07.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.902+0000 7efea500d740 1 -- start start 2024-09-10T10:54:07.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.902+0000 7efea500d740 1 Processor -- start 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da86e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da88c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55a4da86c030 con 0x55a4da86f690 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a4da895bf0 con 0x55a4da84d020 2024-09-10T10:54:07.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55a4da8760d0 con 0x55a4da84b890 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efea500d740 10 client.0 _ll_register_callbacks cb 0x7efe8c058d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efe9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efe9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36920/0 (socket says 172.21.15.28:36920) 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.903+0000 7efe9be00640 1 -- 172.21.15.28:0/1378009368 learned_addr learned my addr 172.21.15.28:0/1378009368 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9b400640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da86e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe93e00640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da88c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 -- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 msgr2=0x55a4da86e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da86e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 -- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 msgr2=0x55a4da88c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da88c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 -- 172.21.15.28:0/1378009368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4da8750a0 con 0x55a4da86f690 2024-09-10T10:54:07.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe9be00640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7efe9402ace0 tx=0x7efe94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c937d065bc35e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94003e20 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7efe94004500 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe8400ed10 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.904+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=273) ==== 24+0+0 (secure 0 0 0) 0x7efe94004700 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 10 client.9669 fetch_fsmap learned FSMap version 273 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=273,monmap=2+}) -- 0x55a4da86bcb0 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8400ed10 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 273) ==== 3502+0+0 (secure 0 0 0) 0x7efe94007540 con 0x55a4da86f690 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 10 client.9669 fetch_fsmap finished waiting for FSMap version 273 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 10 client.9669 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 20 client.9669 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da8a3070 0x55a4da8a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 4 client.9669 mds_command: new command op to 9540 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 --> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55a4da7b71e0 con 0x55a4da8a3070 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.905+0000 7efe9b400640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da8a3070 0x55a4da8a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efe9b400640 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da8a3070 0x55a4da8a3450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376514afbccec6ff server_cookie=481ca40a803a1b1a in_seq=0 out_seq=0 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efe7fe00640 10 client.9669 ms_handle_connect on v2:172.21.15.28:6833/2147565603 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efe7fe00640 1 -- 172.21.15.28:0/1378009368 <== mds.0 v2:172.21.15.28:6833/2147565603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+215 (crc 0 0 0) 0x55a4da7b71e0 con 0x55a4da8a3070 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efe7fe00640 10 client.9669 handle_command_reply: tid=0 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 client.9669 shutdown 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 msgr2=0x55a4da86d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7efe9402ace0 tx=0x7efe94004ab0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 shutdown_connections 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a4da86f690 0x55a4da86d3c0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.28:6833/2147565603,v1:172.21.15.28:6835/2147565603] conn(0x55a4da8a3070 0x55a4da8a3450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a4da84d020 0x55a4da88c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 --2- 172.21.15.28:0/1378009368 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55a4da84b890 0x55a4da86e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efe7fe00640 0 client.9669 ms_handle_reset on v2:172.21.15.28:6833/2147565603 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 >> 172.21.15.28:0/1378009368 conn(0x55a4da849050 msgr2=0x55a4da872730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 shutdown_connections 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 1 -- 172.21.15.28:0/1378009368 wait complete. 2024-09-10T10:54:07.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.906+0000 7efea500d740 20 client.9669 trim_cache size 0 max 16384 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "dentry", 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 1358462900, 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627776, 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "frag": "*", 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "sixmegs", 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_id": "head", 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/subdir/sixmegs" 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:54:07.913 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:54:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 msgr2=0x7efe7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 0x7efe7806e2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0cea70 tx=0x7efe84009040 comp rx=0 tx=0).stop 2024-09-10T10:54:07.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 msgr2=0x7efe9c155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c410 tx=0x7efe8c00c8d0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 shutdown_connections 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7efe7806be30 0x7efe7806e2f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe9c15f9a0 0x7efe9c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe9c0b32a0 0x7efe9c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 --2- 172.21.15.28:0/619943314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe9c0b1c00 0x7efe9c155fc0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 >> 172.21.15.28:0/619943314 conn(0x7efe9c0d2310 msgr2=0x7efe9c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 shutdown_connections 2024-09-10T10:54:07.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:07.907+0000 7efea2000640 1 -- 172.21.15.28:0/619943314 wait complete. 2024-09-10T10:54:07.924 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 100.00000000:stray0_head' 2024-09-10T10:54:07.924 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:07.924 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:07.999 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:07.999 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:08.076 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:54:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.271+0000 7fbb66c00640 1 Processor -- start 2024-09-10T10:54:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.271+0000 7fbb66c00640 1 -- start start 2024-09-10T10:54:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.272+0000 7fbb66c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb60072d30 0x7fbb6006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.272+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb6015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.272+0000 7fbb66c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600cdbe0 con 0x7fbb6015b300 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb66c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbb600728a0 con 0x7fbb60072d30 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb66c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb6006fb30 con 0x7fbb6006f700 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbb6015b300 0x7fbb600cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33744/0 (socket says 172.21.15.28:33744) 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5fe00640 1 -- 172.21.15.28:0/1299959644 learned_addr learned my addr 172.21.15.28:0/1299959644 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520771947 0 0) 0x7fbb600cdbe0 con 0x7fbb6015b300 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb44003520 con 0x7fbb6015b300 2024-09-10T10:54:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb57e00640 1 --2- 172.21.15.28:0/1299959644 >> v2:172.21.15.138:3300/0 conn(0x7fbb60072d30 0x7fbb6006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb64c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb6015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2771198568 0 0) 0x7fbb44003520 con 0x7fbb6015b300 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 msgr2=0x7fbb6015b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb6015b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 >> v2:172.21.15.138:3300/0 conn(0x7fbb60072d30 msgr2=0x7fbb6006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.273+0000 7fbb5f400640 1 --2- 172.21.15.28:0/1299959644 >> v2:172.21.15.138:3300/0 conn(0x7fbb60072d30 0x7fbb6006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600cd900 con 0x7fbb6015b300 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fbb50001690 con 0x7fbb6015b300 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 >> v1:172.21.15.28:6789/0 conn(0x7fbb6015b300 legacy=0x7fbb600cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb440039b0 0x7fbb44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb44004d40 0x7fbb440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600728a0 con 0x7fbb44004d40 2024-09-10T10:54:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb6006fb30 con 0x7fbb440039b0 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb4400bb20 con 0x7fbb6006f700 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb44004d40 0x7fbb440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb57e00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb5fe00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb440039b0 0x7fbb44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 -- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 msgr2=0x7fbb44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb6006f700 0x7fbb44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 -- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb440039b0 msgr2=0x7fbb44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb440039b0 0x7fbb44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.274+0000 7fbb64c00640 1 -- 172.21.15.28:0/1299959644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb44003520 con 0x7fbb44004d40 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.275+0000 7fbb64c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb44004d40 0x7fbb440051c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007360 tx=0x7fbb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7a5621b372baf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58009be0 con 0x7fbb44004d40 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58036600 con 0x7fbb44004d40 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb5f400640 1 -- 172.21.15.28:0/1299959644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb580369d0 con 0x7fbb44004d40 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 -- 172.21.15.28:0/1299959644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb44004d40 msgr2=0x7fbb440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 --2- 172.21.15.28:0/1299959644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb44004d40 0x7fbb440051c0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007360 tx=0x7fbb44005830 comp rx=0 tx=0).stop 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb64c00640 1 -- 172.21.15.28:0/1299959644 reap_dead start 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 -- 172.21.15.28:0/1299959644 shutdown_connections 2024-09-10T10:54:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 -- 172.21.15.28:0/1299959644 >> 172.21.15.28:0/1299959644 conn(0x7fbb60067d70 msgr2=0x7fbb6006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 -- 172.21.15.28:0/1299959644 shutdown_connections 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.276+0000 7fbb66c00640 1 -- 172.21.15.28:0/1299959644 wait complete. 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.278+0000 7fbb66c00640 1 Processor -- start 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.278+0000 7fbb66c00640 1 -- start start 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb44004d40 0x7fbb600b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 0x7fbb6015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600cd5c0 con 0x7fbb60147600 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60072530 con 0x7fbb6014bd40 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb66c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600cffe0 con 0x7fbb44004d40 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb64c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb64c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36938/0 (socket says 172.21.15.28:36938) 2024-09-10T10:54:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb64c00640 1 -- 172.21.15.28:0/2815524713 learned_addr learned my addr 172.21.15.28:0/2815524713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb5fe00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb44004d40 0x7fbb600b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.279+0000 7fbb57e00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 0x7fbb6015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 -- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb44004d40 msgr2=0x7fbb600b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb44004d40 0x7fbb600b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 -- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 msgr2=0x7fbb6015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 0x7fbb6015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 -- 172.21.15.28:0/2815524713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6014c2b0 con 0x7fbb60147600 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb57e00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 0x7fbb6015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb64c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007450 tx=0x7fbb58004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58045070 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58003990 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58009be0 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6014cd40 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.280+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb6014d300 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.282+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb6014d500 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.282+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb5802e860 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.283+0000 7fbb5cc00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 0x7fbb3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.283+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7fbb580ac430 con 0x7fbb60147600 2024-09-10T10:54:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.284+0000 7fbb5fe00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 0x7fbb3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.285+0000 7fbb5fe00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 0x7fbb3c06e320 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbb50001880 tx=0x7fbb50007040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.287+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb6014d500 con 0x7fbb60147600 2024-09-10T10:54:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.484+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb600b4c50 con 0x7fbb60147600 2024-09-10T10:54:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.485+0000 7fbb5cc00640 1 -- 172.21.15.28:0/2815524713 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 62 entries v131) ==== 81+0+3768 (secure 0 0 0) 0x7fbb5803d070 con 0x7fbb60147600 2024-09-10T10:54:08.486 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:08.488 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:08.489 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:08.490 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:08.491 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:08.492 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:08.493 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:08.493 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:08.493 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:08.493 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:08.493 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:listed 62 entries 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 msgr2=0x7fbb3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 0x7fbb3c06e320 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbb50001880 tx=0x7fbb50007040 comp rx=0 tx=0).stop 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 msgr2=0x7fbb6015cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbb58007450 tx=0x7fbb58004ab0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 shutdown_connections 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbb3c06be60 0x7fbb3c06e320 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb60147600 0x7fbb6015cd70 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb6014bd40 0x7fbb6015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 --2- 172.21.15.28:0/2815524713 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb44004d40 0x7fbb600b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 >> 172.21.15.28:0/2815524713 conn(0x7fbb60067d70 msgr2=0x7fbb60155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 shutdown_connections 2024-09-10T10:54:08.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.488+0000 7fbb66c00640 1 -- 172.21.15.28:0/2815524713 wait complete. 2024-09-10T10:54:08.505 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:54:08.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:08.505 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:54:08.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:54:08.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:08.564 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:08.601 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:54:08.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.800+0000 7f77e4e00640 1 Processor -- start 2024-09-10T10:54:08.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.801+0000 7f77e4e00640 1 -- start start 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77e4e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f77e0072320 0x7f77e00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77e4e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e0146140 con 0x7f77e0072de0 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77e4e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e006eed0 con 0x7f77e015e1f0 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77e4e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f77e00cd740 con 0x7f77e0072320 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77dde00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f77e0072320 0x7f77e00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77d7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f77e0072de0 0x7f77e006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33758/0 (socket says 172.21.15.28:33758) 2024-09-10T10:54:08.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77d7e00640 1 -- 172.21.15.28:0/1696146434 learned_addr learned my addr 172.21.15.28:0/1696146434 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77dd400640 1 -- 172.21.15.28:0/1696146434 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507570114 0 0) 0x7f77e0146140 con 0x7f77e0072de0 2024-09-10T10:54:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.802+0000 7f77dd400640 1 -- 172.21.15.28:0/1696146434 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c4003520 con 0x7f77e0072de0 2024-09-10T10:54:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.803+0000 7f77dde00640 1 -- 172.21.15.28:0/1696146434 >> v1:172.21.15.138:6789/0 conn(0x7f77e015e1f0 legacy=0x7f77e006e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.803+0000 7f77dde00640 1 -- 172.21.15.28:0/1696146434 >> v1:172.21.15.28:6789/0 conn(0x7f77e0072de0 legacy=0x7f77e006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.803+0000 7f77dde00640 1 -- 172.21.15.28:0/1696146434 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e00cda20 con 0x7f77e0072320 2024-09-10T10:54:08.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77dde00640 1 --2- 172.21.15.28:0/1696146434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e0072320 0x7f77e00cce60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097a0 tx=0x7f77d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d378a26fdebec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77dd400640 1 -- 172.21.15.28:0/1696146434 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0037070 con 0x7f77e0072320 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77dd400640 1 -- 172.21.15.28:0/1696146434 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d0032440 con 0x7f77e0072320 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77dd400640 1 -- 172.21.15.28:0/1696146434 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00327f0 con 0x7f77e0072320 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77e4e00640 1 -- 172.21.15.28:0/1696146434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e0072320 msgr2=0x7f77e00cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.804+0000 7f77e4e00640 1 --2- 172.21.15.28:0/1696146434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e0072320 0x7f77e00cce60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097a0 tx=0x7f77d002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.805+0000 7f77e4e00640 1 -- 172.21.15.28:0/1696146434 shutdown_connections 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.805+0000 7f77e4e00640 1 --2- 172.21.15.28:0/1696146434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e0072320 0x7f77e00cce60 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.805+0000 7f77e4e00640 1 -- 172.21.15.28:0/1696146434 >> 172.21.15.28:0/1696146434 conn(0x7f77e0067d70 msgr2=0x7f77e006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.805+0000 7f77e4e00640 1 -- 172.21.15.28:0/1696146434 shutdown_connections 2024-09-10T10:54:08.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.805+0000 7f77e4e00640 1 -- 172.21.15.28:0/1696146434 wait complete. 2024-09-10T10:54:08.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.806+0000 7f77e4e00640 1 Processor -- start 2024-09-10T10:54:08.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.806+0000 7f77e4e00640 1 -- start start 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e0072320 0x7f77e00a97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e0072de0 0x7f77e01e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f77e00cd3a0 con 0x7f77e015e1f0 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77e015c9e0 con 0x7f77e0072320 2024-09-10T10:54:08.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.807+0000 7f77e4e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77e00cd520 con 0x7f77e0072de0 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77dde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e0072320 0x7f77e00a97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36958/0 (socket says 172.21.15.28:36958) 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 -- 172.21.15.28:0/2521257804 learned_addr learned my addr 172.21.15.28:0/2521257804 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77d7e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e0072de0 0x7f77e01e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 -- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e0072de0 msgr2=0x7f77e01e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e0072de0 0x7f77e01e6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 -- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e0072320 msgr2=0x7f77e00a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e0072320 0x7f77e00a97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:08.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 -- 172.21.15.28:0/2521257804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e01e73a0 con 0x7f77e015e1f0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.808+0000 7f77de800640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c6b0 tx=0x7f77d800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.809+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8003b70 con 0x7f77e015e1f0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.809+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d8003d10 con 0x7f77e015e1f0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.809+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8005310 con 0x7f77e015e1f0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.809+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e00ab150 con 0x7f77e015e1f0 2024-09-10T10:54:08.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.809+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e01e7900 con 0x7f77e015e1f0 2024-09-10T10:54:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.810+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e0072700 con 0x7f77e015e1f0 2024-09-10T10:54:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.811+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f77d80054b0 con 0x7f77e015e1f0 2024-09-10T10:54:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.811+0000 7f77d6000640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.812+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8970+0+0 (secure 0 0 0) 0x7f77d808b860 con 0x7f77e015e1f0 2024-09-10T10:54:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.812+0000 7f77dde00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:08.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.813+0000 7f77dde00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 0x7f77bc06e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f77d0034000 tx=0x7f77d002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:08.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:08.816+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77e01e73a0 con 0x7f77e015e1f0 2024-09-10T10:54:09.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.040+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f77e01450f0 con 0x7f77e015e1f0 2024-09-10T10:54:09.041 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:09.040+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:09.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.043+0000 7f77d6000640 1 -- 172.21.15.28:0/2521257804 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v274) ==== 161+0+0 (secure 0 0 0) 0x7f77d8058e50 con 0x7f77e015e1f0 2024-09-10T10:54:09.044 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 msgr2=0x7f77bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 0x7f77bc06e2f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f77d0034000 tx=0x7f77d002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 msgr2=0x7f77e015be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c6b0 tx=0x7f77d800cb70 comp rx=0 tx=0).stop 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 shutdown_connections 2024-09-10T10:54:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f77bc06be30 0x7f77bc06e2f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77e015e1f0 0x7f77e015be10 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77e0072de0 0x7f77e01e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 --2- 172.21.15.28:0/2521257804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77e0072320 0x7f77e00a97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 >> 172.21.15.28:0/2521257804 conn(0x7f77e0067d70 msgr2=0x7f77e00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 shutdown_connections 2024-09-10T10:54:09.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.046+0000 7f77e4e00640 1 -- 172.21.15.28:0/2521257804 wait complete. 2024-09-10T10:54:09.051 INFO:tasks.ceph.mds.a.smithi028.stderr: -833> 2024-09-10T10:54:00.803+0000 7fa7c5600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:09.064 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:54:09.094 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.222+0000 7f1ed7200640 1 Processor -- start 2024-09-10T10:54:09.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.222+0000 7f1ed7200640 1 -- start start 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.223+0000 7f1ed7200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ed00d0dd0 0x7f1ed00d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.223+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.223+0000 7f1ed7200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed00cf000 con 0x7f1ed015e660 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.223+0000 7f1ed7200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed006d3e0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.223+0000 7f1ed7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed00b37d0 con 0x7f1ed00d1f90 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1ed00d0dd0 0x7f1ed00d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:33520/0 (socket says 172.21.15.28:33520) 2024-09-10T10:54:09.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed6800640 1 -- 172.21.15.28:0/4121288557 learned_addr learned my addr 172.21.15.28:0/4121288557 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1ed015e660 0x7f1ed006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:55620/0 (socket says 172.21.15.28:55620) 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5e00640 1 -- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 msgr2=0x7f1ed00cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5e00640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5e00640 1 -- 172.21.15.28:0/4121288557 >> v1:172.21.15.138:6789/0 conn(0x7f1ed015e660 legacy=0x7f1ed006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed5e00640 1 -- 172.21.15.28:0/4121288557 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed00d16f0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed4a00640 1 -- 172.21.15.28:0/4121288557 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141878046 0 0) 0x7f1ed00cf000 con 0x7f1ed015e660 2024-09-10T10:54:09.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.224+0000 7f1ed6800640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.225+0000 7f1ed5e00640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00d11b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097a0 tx=0x7f1ec402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51204ad2f9d3ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.225+0000 7f1ed4a00640 1 -- 172.21.15.28:0/4121288557 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4037070 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.225+0000 7f1ed4a00640 1 -- 172.21.15.28:0/4121288557 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec4032440 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.225+0000 7f1ed4a00640 1 -- 172.21.15.28:0/4121288557 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4032810 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 -- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d0dd0 msgr2=0x7f1ed00d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00d11b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097a0 tx=0x7f1ec402a180 comp rx=0 tx=0).stop 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 -- 172.21.15.28:0/4121288557 shutdown_connections 2024-09-10T10:54:09.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed00d1f90 0x7f1ed00cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 --2- 172.21.15.28:0/4121288557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00d11b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 -- 172.21.15.28:0/4121288557 >> 172.21.15.28:0/4121288557 conn(0x7f1ed008bfc0 msgr2=0x7f1ed00d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 -- 172.21.15.28:0/4121288557 shutdown_connections 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.226+0000 7f1ed7200640 1 -- 172.21.15.28:0/4121288557 wait complete. 2024-09-10T10:54:09.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.228+0000 7f1ed7200640 1 Processor -- start 2024-09-10T10:54:09.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.228+0000 7f1ed7200640 1 -- start start 2024-09-10T10:54:09.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 0x7f1ed00ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed015e660 0x7f1ed01f7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed00d2c60 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed00cf330 con 0x7f1ed00d1f90 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed7200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed00d2f40 con 0x7f1ed015e660 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36982/0 (socket says 172.21.15.28:36982) 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 -- 172.21.15.28:0/3610889194 learned_addr learned my addr 172.21.15.28:0/3610889194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed6800640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed015e660 0x7f1ed01f7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5400640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 0x7f1ed00ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 -- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed015e660 msgr2=0x7f1ed01f7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed015e660 0x7f1ed01f7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 -- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 msgr2=0x7f1ed00ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 0x7f1ed00ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5e00640 1 -- 172.21.15.28:0/3610889194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed009e650 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.229+0000 7f1ed5400640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 0x7f1ed00ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1ed5e00640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40098c0 tx=0x7f1ec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:09.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4037040 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec4030500 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40308d0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed00a0010 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.230+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed00a05d0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.231+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed00d3220 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.232+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1ec4044020 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.233+0000 7f1eca000640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 0x7f1eb406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.233+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9056+0+0 (secure 0 0 0) 0x7f1ec40abdd0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.233+0000 7f1ed5400640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 0x7f1eb406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:09.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.234+0000 7f1ed5400640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 0x7f1eb406e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00b2750 tx=0x7f1ec0007c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:09.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.237+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ec40799e0 con 0x7f1ed00d0dd0 2024-09-10T10:54:09.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:09.489+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f1ed00d1b20 con 0x7f1ed00d0dd0 2024-09-10T10:54:10.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.045+0000 7f1eca000640 1 -- 172.21.15.28:0/3610889194 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v275) ==== 107+0+0 (secure 0 0 0) 0x7f1ec40792e0 con 0x7f1ed00d0dd0 2024-09-10T10:54:10.046 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.047+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 msgr2=0x7f1eb406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.047+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 0x7f1eb406e2f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00b2750 tx=0x7f1ec0007c10 comp rx=0 tx=0).stop 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.047+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 msgr2=0x7f1ed00cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.047+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40098c0 tx=0x7f1ec40046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 shutdown_connections 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1eb406be30 0x7f1eb406e2f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ed015e660 0x7f1ed01f7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ed00d1f90 0x7f1ed00ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 --2- 172.21.15.28:0/3610889194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ed00d0dd0 0x7f1ed00cdca0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 >> 172.21.15.28:0/3610889194 conn(0x7f1ed008bfc0 msgr2=0x7f1ed00b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 shutdown_connections 2024-09-10T10:54:10.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.048+0000 7f1ed7200640 1 -- 172.21.15.28:0/3610889194 wait complete. 2024-09-10T10:54:10.068 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:54:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.184+0000 7ff6132f6f40 1 Processor -- start 2024-09-10T10:54:10.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.184+0000 7ff6132f6f40 1 -- start start 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 0x5568b4478750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b445df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568b4476530 con 0x5568b44758e0 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568b4476290 con 0x5568b45049c0 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568b4503ee0 con 0x5568b45013e0 2024-09-10T10:54:10.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff610400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff610400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37000/0 (socket says 172.21.15.28:37000) 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.185+0000 7ff610400640 1 -- 172.21.15.28:0/1926722238 learned_addr learned my addr 172.21.15.28:0/1926722238 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 0x5568b4478750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 -- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 msgr2=0x5568b4478750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 0x5568b4478750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 -- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 msgr2=0x5568b445df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b445df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 -- 172.21.15.28:0/1926722238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b4357b30 con 0x5568b44758e0 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 0x5568b4478750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff610400640 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff600009790 tx=0x7ff600009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ec0146a4e175c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff60e600640 1 -- 172.21.15.28:0/1926722238 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600017070 con 0x5568b44758e0 2024-09-10T10:54:10.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff60e600640 1 -- 172.21.15.28:0/1926722238 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6000090d0 con 0x5568b44758e0 2024-09-10T10:54:10.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.186+0000 7ff60e600640 1 -- 172.21.15.28:0/1926722238 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff600006b60 con 0x5568b44758e0 2024-09-10T10:54:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1926722238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 msgr2=0x5568b4478210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7ff600009790 tx=0x7ff600009c50 comp rx=0 tx=0).stop 2024-09-10T10:54:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1926722238 shutdown_connections 2024-09-10T10:54:10.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b445df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b45013e0 0x5568b4478750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1926722238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b44758e0 0x5568b4478210 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1926722238 >> 172.21.15.28:0/1926722238 conn(0x5568b447a0a0 msgr2=0x5568b442b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1926722238 shutdown_connections 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.187+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1926722238 wait complete. 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.188+0000 7ff6132f6f40 1 Processor -- start 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.188+0000 7ff6132f6f40 1 -- start start 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.188+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b44758e0 0x5568b44f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.188+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 0x5568b459db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.188+0000 7ff6132f6f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b4501850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5568b445f230 con 0x5568b45013e0 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5568b4503120 con 0x5568b45049c0 2024-09-10T10:54:10.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff6132f6f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5568b4479610 con 0x5568b44758e0 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 0x5568b459db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 0x5568b459db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37002/0 (socket says 172.21.15.28:37002) 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 learned_addr learned my addr 172.21.15.28:0/1960899213 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b44758e0 0x5568b44f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b4501850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b44758e0 msgr2=0x5568b44f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5568b44758e0 0x5568b44f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 msgr2=0x5568b4501850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b4501850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568b459e0d0 con 0x5568b45013e0 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.189+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5568b45049c0 0x5568b4501850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:10.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 0x5568b459db90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc045be0 tx=0x7ff5fc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff60d200640 1 -- 172.21.15.28:0/1960899213 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc055040 con 0x5568b45013e0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff60d200640 1 -- 172.21.15.28:0/1960899213 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0519b0 con 0x5568b45013e0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff60d200640 1 -- 172.21.15.28:0/1960899213 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc051d60 con 0x5568b45013e0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568b44f7a90 con 0x5568b45013e0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.190+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568b44f6cd0 con 0x5568b45013e0 2024-09-10T10:54:10.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.192+0000 7ff60d200640 1 -- 172.21.15.28:0/1960899213 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5fc058070 con 0x5568b45013e0 2024-09-10T10:54:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.193+0000 7ff60d200640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff5f006a810 0x7ff5f006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.193+0000 7ff60d200640 1 -- 172.21.15.28:0/1960899213 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9056+0+0 (secure 0 0 0) 0x7ff5fc0ce5c0 con 0x5568b45013e0 2024-09-10T10:54:10.195 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.193+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff5f006a810 0x7ff5f006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.193+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5568b445ea20 0x5568b445d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.193+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b445d980 con 0x5568b445ea20 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.194+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5568b445ea20 0x5568b445d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.194+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff5f006a810 0x7ff5f006ccd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff600015040 tx=0x7ff60000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.194+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5568b445ea20 0x5568b445d440 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.197+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v132'971 uv944 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.197+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b445dd50 con 0x5568b445ea20 2024-09-10T10:54:10.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.199+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v132'972 uv972 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.199+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6310 con 0x5568b445ea20 2024-09-10T10:54:10.201 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:54:10.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.200+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v132'973 uv973 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.200+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a76c0 con 0x5568b445ea20 2024-09-10T10:54:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.202+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v132'974 uv974 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.202+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7a60 con 0x5568b445ea20 2024-09-10T10:54:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.204+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v132'975 uv975 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.204+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a66b0 con 0x5568b445ea20 2024-09-10T10:54:10.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.206+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v132'976 uv976 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.206+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6aa0 con 0x5568b445ea20 2024-09-10T10:54:10.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.208+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v132'977 uv950 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.208+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e90 con 0x5568b445ea20 2024-09-10T10:54:10.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.209+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v132'978 uv978 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.209+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e90 con 0x5568b445ea20 2024-09-10T10:54:10.211 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:54:10.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.211+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v132'979 uv979 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.211+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.213+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v132'980 uv980 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.213+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.214+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v132'981 uv981 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.214+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.216+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v132'982 uv982 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.216+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.218+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v132'983 uv970 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.218+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.220+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v132'984 uv984 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.220+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.222 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:54:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.222+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v132'985 uv985 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.222+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.223+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v132'986 uv986 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.223+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.225+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v132'987 uv987 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.225+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a6e40 con 0x5568b445ea20 2024-09-10T10:54:10.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.227+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v132'988 uv988 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.227+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.228+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v132'989 uv989 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.228+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.230+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v132'990 uv963 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.230+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.232+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v132'991 uv991 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.232+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.235 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:54:10.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.234+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v132'992 uv992 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.234+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a8e10 con 0x5568b445ea20 2024-09-10T10:54:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.236+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v132'993 uv993 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.236+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a8e10 con 0x5568b445ea20 2024-09-10T10:54:10.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.238+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v132'994 uv994 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.238+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.239+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v132'995 uv995 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.239+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.241+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v132'996 uv969 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.241+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.243+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v132'997 uv997 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.243+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.246 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:54:10.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.245+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v132'998 uv998 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.245+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a8e10 con 0x5568b445ea20 2024-09-10T10:54:10.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.247+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v132'999 uv999 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.247+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a8e10 con 0x5568b445ea20 2024-09-10T10:54:10.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.249+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v132'1000 uv1000 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.249+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45a7e00 con 0x5568b445ea20 2024-09-10T10:54:10.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.251+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v132'1001 uv1001 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b445ea20 2024-09-10T10:54:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.251+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568b45a8310 0x5568b45aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.251+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45aac40 con 0x5568b45a8310 2024-09-10T10:54:10.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.251+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568b45a8310 0x5568b45aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.252+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568b45a8310 0x5568b45aa700 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.254+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v132'705 uv684 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.254+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45aafe0 con 0x5568b45a8310 2024-09-10T10:54:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.255+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v132'706 uv706 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.255+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ab380 con 0x5568b45a8310 2024-09-10T10:54:10.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.257+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v132'707 uv707 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.258 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:54:10.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.257+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ab720 con 0x5568b45a8310 2024-09-10T10:54:10.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.258+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v132'708 uv708 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.258+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45abac0 con 0x5568b45a8310 2024-09-10T10:54:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.260+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v132'709 uv709 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.260+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45abe60 con 0x5568b45a8310 2024-09-10T10:54:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.262+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v132'710 uv703 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.262+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.263+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v132'711 uv711 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.263+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.265 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:54:10.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.264+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v132'712 uv712 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.264+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.266+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v132'713 uv713 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.266+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.268+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v132'714 uv714 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.268+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.269+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v132'715 uv704 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.270+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.271+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v132'716 uv716 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.271+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.272+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v132'717 uv717 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.274 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:54:10.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.272+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b08f0 con 0x5568b45a8310 2024-09-10T10:54:10.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.274+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v132'718 uv718 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.274+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b08f0 con 0x5568b45a8310 2024-09-10T10:54:10.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.276+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v132'719 uv719 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.276+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.277+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v132'720 uv700 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.277+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.278+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v132'721 uv721 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.278+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.280+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v132'722 uv722 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.281 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:54:10.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.280+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ad210 con 0x5568b45a8310 2024-09-10T10:54:10.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.281+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v132'723 uv723 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.282+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ad210 con 0x5568b45a8310 2024-09-10T10:54:10.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.283+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v132'724 uv724 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.283+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45ac200 con 0x5568b45a8310 2024-09-10T10:54:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.284+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v132'725 uv725 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45a8310 2024-09-10T10:54:10.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.284+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5568b45ac710 0x5568b45aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.284+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45af040 con 0x5568b45ac710 2024-09-10T10:54:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.285+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5568b45ac710 0x5568b45aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.285+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5568b45ac710 0x5568b45aeb00 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.287+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v132'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.287+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45af3e0 con 0x5568b45ac710 2024-09-10T10:54:10.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.288+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v132'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.288+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45af780 con 0x5568b45ac710 2024-09-10T10:54:10.290 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:54:10.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.289+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v132'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.289+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b4501e70 con 0x5568b45ac710 2024-09-10T10:54:10.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.291+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v132'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.291+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b10f0 con 0x5568b45ac710 2024-09-10T10:54:10.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.292+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v132'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.292+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b0020 con 0x5568b45ac710 2024-09-10T10:54:10.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.293+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v132'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.293+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5568b45b0ae0 0x5568b45b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.293+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3a40 con 0x5568b45b0ae0 2024-09-10T10:54:10.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.293+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5568b45b0ae0 0x5568b45b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.294+0000 7ff610e00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5568b45b0ae0 0x5568b45b34a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.295+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v132'139 uv138 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.295+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3e30 con 0x5568b45b0ae0 2024-09-10T10:54:10.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.296+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v132'140 uv140 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.296+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3e30 con 0x5568b45b0ae0 2024-09-10T10:54:10.298 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:54:10.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.297+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v132'141 uv141 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.298+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3e30 con 0x5568b45b0ae0 2024-09-10T10:54:10.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.299+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v132'142 uv142 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.299+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3d70 con 0x5568b45ac710 2024-09-10T10:54:10.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.300+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v132'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.300+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3d70 con 0x5568b45ac710 2024-09-10T10:54:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.301+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v132'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.301+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3d70 con 0x5568b45ac710 2024-09-10T10:54:10.303 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:54:10.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.302+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v132'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.302+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3d70 con 0x5568b45ac710 2024-09-10T10:54:10.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.304+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v132'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.304+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b3db0 con 0x5568b45ac710 2024-09-10T10:54:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.305+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v132'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.305+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4250 con 0x5568b45ac710 2024-09-10T10:54:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.306+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v132'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.306+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4250 con 0x5568b45b0ae0 2024-09-10T10:54:10.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.308+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v132'349 uv343 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.308+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4250 con 0x5568b45b0ae0 2024-09-10T10:54:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.309+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v132'350 uv350 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.309+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.311+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v132'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.312 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:54:10.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.311+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.312+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v132'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.312+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.314+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v132'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.314+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.315+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v132'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.315+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.317+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v132'355 uv348 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.317+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.318+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v132'356 uv356 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.318+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.319+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v132'357 uv357 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.320 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:54:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.319+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.321+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v132'358 uv358 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.321+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45b0ae0 2024-09-10T10:54:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.322+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v132'359 uv359 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.322+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b4200 con 0x5568b45ac710 2024-09-10T10:54:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.323+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v132'158 uv157 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.323+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b41e0 con 0x5568b45ac710 2024-09-10T10:54:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.325+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v132'159 uv159 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.325+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b41e0 con 0x5568b45ac710 2024-09-10T10:54:10.328 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:54:10.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.326+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v132'160 uv160 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.326+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b41e0 con 0x5568b45ac710 2024-09-10T10:54:10.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.327+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v132'161 uv161 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.327+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b41e0 con 0x5568b45ac710 2024-09-10T10:54:10.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.328+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v132'162 uv162 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45ac710 2024-09-10T10:54:10.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.328+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568b45b4c40 0x5568b45b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.328+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.328+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568b45b4c40 0x5568b45b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.329+0000 7ff610400640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568b45b4c40 0x5568b45b7330 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.330+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v132'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.330+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.332+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v132'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.332+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.333+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v132'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.334 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:54:10.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.333+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b88c0 con 0x5568b45b4c40 2024-09-10T10:54:10.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.334+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v132'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.334+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b8880 con 0x5568b45b4c40 2024-09-10T10:54:10.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.336+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v132'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.336+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.337+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v132'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.337+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.339+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v132'314 uv308 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.339+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.340+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v132'315 uv315 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.340+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7870 con 0x5568b45b4c40 2024-09-10T10:54:10.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.341+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v132'316 uv316 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.342 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:54:10.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.341+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7970 con 0x5568b45b4c40 2024-09-10T10:54:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.343+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v132'317 uv317 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.343+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7930 con 0x5568b45b4c40 2024-09-10T10:54:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.344+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v132'318 uv318 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.344+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7930 con 0x5568b45b4c40 2024-09-10T10:54:10.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.346+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v132'319 uv313 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.346+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7930 con 0x5568b45b4c40 2024-09-10T10:54:10.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.347+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v132'320 uv320 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.347+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7910 con 0x5568b45b4c40 2024-09-10T10:54:10.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.349+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v132'321 uv321 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.350 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:54:10.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.349+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45b7950 con 0x5568b45b4c40 2024-09-10T10:54:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.350+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v132'322 uv322 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.350+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc60 con 0x5568b45b4c40 2024-09-10T10:54:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.352+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v132'323 uv323 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.352+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.353+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v132'137 uv135 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.353+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.354+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v132'138 uv138 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.354+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.356 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:54:10.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.355+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v132'139 uv139 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.355+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.357+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v132'140 uv140 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.357+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.358+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v132'189 uv188 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.358+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.359+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v132'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.359+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.360+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v132'191 uv191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.361 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:54:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.360+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afc20 con 0x5568b45b0ae0 2024-09-10T10:54:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.362+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v132'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.362+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afbe0 con 0x5568b45b0ae0 2024-09-10T10:54:10.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.363+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v132'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.363+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45afbe0 con 0x5568b45b0ae0 2024-09-10T10:54:10.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.364+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v132'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff60800b770 con 0x5568b45b0ae0 2024-09-10T10:54:10.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.364+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5568b45b8740 0x5568b45bae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.364+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b8740 2024-09-10T10:54:10.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.364+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5568b45b8740 0x5568b45bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.365+0000 7ff60fa00640 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5568b45b8740 0x5568b45bae90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.366+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v132'161 uv160 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45b8740 2024-09-10T10:54:10.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.366+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b8740 2024-09-10T10:54:10.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.367+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v132'162 uv162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45b8740 2024-09-10T10:54:10.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.367+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b8740 2024-09-10T10:54:10.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.368+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v132'163 uv163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45b8740 2024-09-10T10:54:10.370 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:54:10.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.368+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b8740 2024-09-10T10:54:10.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.370+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v132'164 uv164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45b8740 2024-09-10T10:54:10.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.370+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b8740 2024-09-10T10:54:10.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.371+0000 7ff60fa00640 1 -- 172.21.15.28:0/1960899213 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v132'165 uv165 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff5fc06c020 con 0x5568b45b8740 2024-09-10T10:54:10.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.371+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.372+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v132'389 uv380 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.372+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.374+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v132'390 uv390 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.374+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.376 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:54:10.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.375+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v132'391 uv391 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.375+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.376+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v132'392 uv392 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.376+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.378+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v132'393 uv393 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.378+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.379+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v132'394 uv387 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.379+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.380+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v132'395 uv395 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.380+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.382 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:54:10.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.381+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v132'396 uv396 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.381+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bc3e0 con 0x5568b45b4c40 2024-09-10T10:54:10.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.382+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v132'397 uv397 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.382+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bc3e0 con 0x5568b45b4c40 2024-09-10T10:54:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.384+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v132'398 uv398 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.384+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb3d0 con 0x5568b45b4c40 2024-09-10T10:54:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.385+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v132'399 uv399 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.385+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x5568b45bb880 con 0x5568b45b4c40 2024-09-10T10:54:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff610400640 1 -- 172.21.15.28:0/1960899213 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v132'400 uv400 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7ff600024040 con 0x5568b45b4c40 2024-09-10T10:54:10.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5568b45ac710 msgr2=0x5568b45aeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5568b45ac710 0x5568b45aeb00 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568b45b4c40 msgr2=0x5568b45b7330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5568b45b4c40 0x5568b45b7330 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5568b45b8740 msgr2=0x5568b45bae90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5568b45b8740 0x5568b45bae90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5568b45b0ae0 msgr2=0x5568b45b34a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5568b45b0ae0 0x5568b45b34a0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568b45a8310 msgr2=0x5568b45aa700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5568b45a8310 0x5568b45aa700 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5568b445ea20 msgr2=0x5568b445d440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5568b445ea20 0x5568b445d440 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 reap_dead start 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.386+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 reap_dead start 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff5f006a810 msgr2=0x7ff5f006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff5f006a810 0x7ff5f006ccd0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff600015040 tx=0x7ff60000f000 comp rx=0 tx=0).stop 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 msgr2=0x5568b459db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 --2- 172.21.15.28:0/1960899213 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5568b45013e0 0x5568b459db90 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc045be0 tx=0x7ff5fc002fb0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 reap_dead start 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff610e00640 1 -- 172.21.15.28:0/1960899213 reap_dead start 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 shutdown_connections 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 >> 172.21.15.28:0/1960899213 conn(0x5568b447a0a0 msgr2=0x5568b442b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 shutdown_connections 2024-09-10T10:54:10.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.387+0000 7ff6132f6f40 1 -- 172.21.15.28:0/1960899213 wait complete. 2024-09-10T10:54:10.392 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 100.00000000 stray0_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:54:10.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.510+0000 7ff0827eef40 1 Processor -- start 2024-09-10T10:54:10.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 -- start start 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x565275ecf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa79c0 0x565275fa5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x565275f19530 con 0x565275fa43e0 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565275e76b30 con 0x565275f18ff0 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.511+0000 7ff0827eef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x565275fa6ee0 con 0x565275fa79c0 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37004/0 (socket says 172.21.15.28:37004) 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x565275ecf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 -- 172.21.15.28:0/4286247888 learned_addr learned my addr 172.21.15.28:0/4286247888 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff080400640 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa79c0 0x565275fa5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 -- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa79c0 msgr2=0x565275fa5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa79c0 0x565275fa5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 -- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 msgr2=0x565275ecf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x565275ecf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 -- 172.21.15.28:0/4286247888 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565275f01e20 con 0x565275fa43e0 2024-09-10T10:54:10.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07fa00640 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x565275ecf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:10.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07f000640 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff06c001e60 tx=0x7ff06c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49561ad3c29f4d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.512+0000 7ff07dc00640 1 -- 172.21.15.28:0/4286247888 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c016070 con 0x565275fa43e0 2024-09-10T10:54:10.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff07dc00640 1 -- 172.21.15.28:0/4286247888 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c00d6d0 con 0x565275fa43e0 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff07dc00640 1 -- 172.21.15.28:0/4286247888 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00dac0 con 0x565275fa43e0 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 -- 172.21.15.28:0/4286247888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 msgr2=0x565275ece550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff06c001e60 tx=0x7ff06c00b650 comp rx=0 tx=0).stop 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 -- 172.21.15.28:0/4286247888 shutdown_connections 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa79c0 0x565275fa5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa43e0 0x565275ece550 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 --2- 172.21.15.28:0/4286247888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x565275ecf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 -- 172.21.15.28:0/4286247888 >> 172.21.15.28:0/4286247888 conn(0x565275f1d0a0 msgr2=0x565275fa4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 -- 172.21.15.28:0/4286247888 shutdown_connections 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.513+0000 7ff0827eef40 1 -- 172.21.15.28:0/4286247888 wait complete. 2024-09-10T10:54:10.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.514+0000 7ff0827eef40 1 Processor -- start 2024-09-10T10:54:10.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.514+0000 7ff0827eef40 1 -- start start 2024-09-10T10:54:10.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x5652760403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa43e0 0x565276040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 0x565275f89ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x565275ecfd90 con 0x565275fa79c0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565275ece0c0 con 0x565275f18ff0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff0827eef40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x565275ecdd50 con 0x565275fa43e0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff080400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 0x565275f89ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff080400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 0x565275f89ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37006/0 (socket says 172.21.15.28:37006) 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff080400640 1 -- 172.21.15.28:0/853838227 learned_addr learned my addr 172.21.15.28:0/853838227 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff07f000640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa43e0 0x565276040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff07fa00640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x5652760403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff080400640 1 -- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa43e0 msgr2=0x565276040920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.515+0000 7ff080400640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x565275fa43e0 0x565276040920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff080400640 1 -- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 msgr2=0x5652760403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff080400640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565275f18ff0 0x5652760403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff080400640 1 -- 172.21.15.28:0/853838227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565275f8a7f0 con 0x565275fa79c0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff080400640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 0x565275f89ac0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff078025ab0 tx=0x7ff078003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff077e00640 1 -- 172.21.15.28:0/853838227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078062410 con 0x565275fa79c0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff077e00640 1 -- 172.21.15.28:0/853838227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff078067070 con 0x565275fa79c0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff077e00640 1 -- 172.21.15.28:0/853838227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0780627f0 con 0x565275fa79c0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565275f8aad0 con 0x565275fa79c0 2024-09-10T10:54:10.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.516+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565275f16dc0 con 0x565275fa79c0 2024-09-10T10:54:10.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.518+0000 7ff077e00640 1 -- 172.21.15.28:0/853838227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff078062990 con 0x565275fa79c0 2024-09-10T10:54:10.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.519+0000 7ff077e00640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff06006a810 0x7ff06006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.519+0000 7ff077e00640 1 -- 172.21.15.28:0/853838227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9056+0+0 (secure 0 0 0) 0x7ff0780ce8c0 con 0x565275fa79c0 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.519+0000 7ff0827eef40 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56527603c6d0 0x565275f8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.519+0000 7ff07fa00640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff06006a810 0x7ff06006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.519+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132) -- 0x56527603cb10 con 0x56527603c6d0 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.520+0000 7ff07f000640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56527603c6d0 0x565275f8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.520+0000 7ff07fa00640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff06006a810 0x7ff06006ccd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff070005400 tx=0x7ff07000c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.520+0000 7ff07f000640 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56527603c6d0 0x565275f8bbc0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.521+0000 7ff07f000640 1 -- 172.21.15.28:0/853838227 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [omap-set-vals] v132'324 uv324 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7ff06c00d880 con 0x56527603c6d0 2024-09-10T10:54:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.521+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56527603c6d0 msgr2=0x565275f8bbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.521+0000 7ff0827eef40 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56527603c6d0 0x565275f8bbc0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff06006a810 msgr2=0x7ff06006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff06006a810 0x7ff06006ccd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7ff070005400 tx=0x7ff07000c040 comp rx=0 tx=0).stop 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 msgr2=0x565275f89ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 --2- 172.21.15.28:0/853838227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x565275fa79c0 0x565275f89ac0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff078025ab0 tx=0x7ff078003330 comp rx=0 tx=0).stop 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff080400640 1 -- 172.21.15.28:0/853838227 reap_dead start 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 shutdown_connections 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 >> 172.21.15.28:0/853838227 conn(0x565275f1d0a0 msgr2=0x565275fa5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 shutdown_connections 2024-09-10T10:54:10.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.522+0000 7ff0827eef40 1 -- 172.21.15.28:0/853838227 wait complete. 2024-09-10T10:54:10.527 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:54:10.676 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:54:10.680 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:54:10.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.685+0000 7fe697600640 1 Processor -- start 2024-09-10T10:54:10.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.685+0000 7fe697600640 1 -- start start 2024-09-10T10:54:10.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe697600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe690156740 0x7fe6900b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe697600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6900b4af0 0x7fe6900cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe697600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6900b30e0 con 0x7fe6900cfcd0 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe697600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe6900d0140 con 0x7fe690156740 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe697600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe69006edc0 con 0x7fe6900b4af0 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe694c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe690156740 0x7fe6900b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe694c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe690156740 0x7fe6900b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37012/0 (socket says 172.21.15.28:37012) 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe694c00640 1 -- 172.21.15.28:0/119849175 learned_addr learned my addr 172.21.15.28:0/119849175 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe695600640 1 --1- 172.21.15.28:0/119849175 >> v1:172.21.15.28:6789/0 conn(0x7fe6900cfcd0 0x7fe69015fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33770/0 (socket says 172.21.15.28:33770) 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.686+0000 7fe68fe00640 1 --2- 172.21.15.28:0/119849175 >> v2:172.21.15.138:3300/0 conn(0x7fe6900b4af0 0x7fe6900cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.687+0000 7fe694c00640 1 -- 172.21.15.28:0/119849175 >> v2:172.21.15.138:3300/0 conn(0x7fe6900b4af0 msgr2=0x7fe6900cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.687+0000 7fe694c00640 1 --2- 172.21.15.28:0/119849175 >> v2:172.21.15.138:3300/0 conn(0x7fe6900b4af0 0x7fe6900cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.687+0000 7fe694c00640 1 -- 172.21.15.28:0/119849175 >> v1:172.21.15.28:6789/0 conn(0x7fe6900cfcd0 legacy=0x7fe69015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.687+0000 7fe694c00640 1 -- 172.21.15.28:0/119849175 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6900b33c0 con 0x7fe690156740 2024-09-10T10:54:10.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.687+0000 7fe68f400640 1 -- 172.21.15.28:0/119849175 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3399423197 0 0) 0x7fe6900b30e0 con 0x7fe6900cfcd0 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.688+0000 7fe694c00640 1 --2- 172.21.15.28:0/119849175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe690156740 0x7fe6900b45b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe69015a370 tx=0x7fe68402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b115566b1b6c231 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.688+0000 7fe68f400640 1 -- 172.21.15.28:0/119849175 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684037070 con 0x7fe690156740 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.688+0000 7fe68f400640 1 -- 172.21.15.28:0/119849175 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684032440 con 0x7fe690156740 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.688+0000 7fe68f400640 1 -- 172.21.15.28:0/119849175 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684032810 con 0x7fe690156740 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 -- 172.21.15.28:0/119849175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe690156740 msgr2=0x7fe6900b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 --2- 172.21.15.28:0/119849175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe690156740 0x7fe6900b45b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fe69015a370 tx=0x7fe68402a270 comp rx=0 tx=0).stop 2024-09-10T10:54:10.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 -- 172.21.15.28:0/119849175 shutdown_connections 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 --2- 172.21.15.28:0/119849175 >> v2:172.21.15.138:3300/0 conn(0x7fe6900b4af0 0x7fe6900cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 --2- 172.21.15.28:0/119849175 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe690156740 0x7fe6900b45b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 -- 172.21.15.28:0/119849175 >> 172.21.15.28:0/119849175 conn(0x7fe69006bdf0 msgr2=0x7fe69006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 -- 172.21.15.28:0/119849175 shutdown_connections 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.689+0000 7fe697600640 1 -- 172.21.15.28:0/119849175 wait complete. 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.691+0000 7fe697600640 1 Processor -- start 2024-09-10T10:54:10.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.691+0000 7fe697600640 1 -- start start 2024-09-10T10:54:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6900cfcd0 0x7fe6900ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe690156740 0x7fe690140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6900b3c60 con 0x7fe6900b4af0 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6900b3980 con 0x7fe6900cfcd0 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe697600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6900d0890 con 0x7fe690156740 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37028/0 (socket says 172.21.15.28:37028) 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 -- 172.21.15.28:0/2007104153 learned_addr learned my addr 172.21.15.28:0/2007104153 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe68fe00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6900cfcd0 0x7fe6900ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe695600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe690156740 0x7fe690140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 -- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe690156740 msgr2=0x7fe690140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe690156740 0x7fe690140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 -- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6900cfcd0 msgr2=0x7fe6900ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6900cfcd0 0x7fe6900ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:10.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.692+0000 7fe694c00640 1 -- 172.21.15.28:0/2007104153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe690160750 con 0x7fe6900b4af0 2024-09-10T10:54:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe694c00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe684000c00 tx=0x7fe6840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684037040 con 0x7fe6900b4af0 2024-09-10T10:54:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe684004030 con 0x7fe6900b4af0 2024-09-10T10:54:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684004400 con 0x7fe6900b4af0 2024-09-10T10:54:10.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69013e0d0 con 0x7fe6900b4af0 2024-09-10T10:54:10.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.693+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69013e690 con 0x7fe6900b4af0 2024-09-10T10:54:10.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.694+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690156b20 con 0x7fe6900b4af0 2024-09-10T10:54:10.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.695+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe68403a070 con 0x7fe6900b4af0 2024-09-10T10:54:10.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.696+0000 7fe68cc00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 0x7fe67006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:10.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.696+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9056+0+0 (secure 0 0 0) 0x7fe6840ac0c0 con 0x7fe6900b4af0 2024-09-10T10:54:10.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.697+0000 7fe68fe00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 0x7fe67006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:10.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.697+0000 7fe68fe00640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 0x7fe67006e2f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe69006c3f0 tx=0x7fe67c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:10.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.700+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe684041020 con 0x7fe6900b4af0 2024-09-10T10:54:10.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:10.927+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe6900b4430 con 0x7fe6900b4af0 2024-09-10T10:54:11.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.051+0000 7fe68cc00640 1 -- 172.21.15.28:0/2007104153 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v276) ==== 162+0+0 (secure 0 0 0) 0x7fe684079680 con 0x7fe6900b4af0 2024-09-10T10:54:11.052 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:54:11.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 msgr2=0x7fe67006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 0x7fe67006e2f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe69006c3f0 tx=0x7fe67c009190 comp rx=0 tx=0).stop 2024-09-10T10:54:11.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 msgr2=0x7fe6900ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe684000c00 tx=0x7fe6840046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 shutdown_connections 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe67006be30 0x7fe67006e2f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe690156740 0x7fe690140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6900cfcd0 0x7fe6900ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 --2- 172.21.15.28:0/2007104153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6900b4af0 0x7fe6900ae120 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 >> 172.21.15.28:0/2007104153 conn(0x7fe69006bdf0 msgr2=0x7fe6900cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 shutdown_connections 2024-09-10T10:54:11.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.053+0000 7fe697600640 1 -- 172.21.15.28:0/2007104153 wait complete. 2024-09-10T10:54:11.073 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:11.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.230+0000 7f601fe00640 1 Processor -- start 2024-09-10T10:54:11.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.230+0000 7f601fe00640 1 -- start start 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.231+0000 7f601fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60200b40e0 0x7f60200aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.231+0000 7f601fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.231+0000 7f601fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60200b5470 con 0x7f602015bef0 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.231+0000 7f601fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f60200b3640 con 0x7f60200b40e0 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.231+0000 7f601fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60200b5190 con 0x7f60200af1b0 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60200b40e0 0x7f60200aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60200b40e0 0x7f60200aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37044/0 (socket says 172.21.15.28:37044) 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 -- 172.21.15.28:0/1507659564 learned_addr learned my addr 172.21.15.28:0/1507659564 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:11.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f601f400640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 -- 172.21.15.28:0/1507659564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 msgr2=0x7f6020071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:11.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 -- 172.21.15.28:0/1507659564 >> v1:172.21.15.138:6789/0 conn(0x7f602015bef0 legacy=0x7f60200b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f6017e00640 1 -- 172.21.15.28:0/1507659564 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60200b38e0 con 0x7f60200b40e0 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.232+0000 7f601e000640 1 -- 172.21.15.28:0/1507659564 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 420909593 0 0) 0x7f60200b5470 con 0x7f602015bef0 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.233+0000 7f6017e00640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f60200aec70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f6008009870 tx=0x7f600802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9173faab64ce98d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601e000640 1 -- 172.21.15.28:0/1507659564 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008037070 con 0x7f60200b40e0 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601e000640 1 -- 172.21.15.28:0/1507659564 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6008032440 con 0x7f60200b40e0 2024-09-10T10:54:11.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601e000640 1 -- 172.21.15.28:0/1507659564 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008032810 con 0x7f60200b40e0 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 -- 172.21.15.28:0/1507659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 msgr2=0x7f60200aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f60200aec70 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f6008009870 tx=0x7f600802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 -- 172.21.15.28:0/1507659564 shutdown_connections 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 --2- 172.21.15.28:0/1507659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f60200aec70 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 -- 172.21.15.28:0/1507659564 >> 172.21.15.28:0/1507659564 conn(0x7f602006c240 msgr2=0x7f60200b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 -- 172.21.15.28:0/1507659564 shutdown_connections 2024-09-10T10:54:11.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.234+0000 7f601fe00640 1 -- 172.21.15.28:0/1507659564 wait complete. 2024-09-10T10:54:11.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.236+0000 7f601fe00640 1 Processor -- start 2024-09-10T10:54:11.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.236+0000 7f601fe00640 1 -- start start 2024-09-10T10:54:11.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 0x7f60201531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f60200b4e60 con 0x7f60200b40e0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60200af640 con 0x7f602015bef0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60200b4b10 con 0x7f60200af1b0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37058/0 (socket says 172.21.15.28:37058) 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 -- 172.21.15.28:0/3948399316 learned_addr learned my addr 172.21.15.28:0/3948399316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601ea00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601f400640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 0x7f60201531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 -- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 msgr2=0x7f6020151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 -- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 msgr2=0x7f60201531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 0x7f60201531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f6017e00640 1 -- 172.21.15.28:0/3948399316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602015e040 con 0x7f60200b40e0 2024-09-10T10:54:11.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.237+0000 7f601f400640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 0x7f60201531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:11.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f6017e00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f60080090b0 tx=0x7f60080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008037040 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6008004230 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008003610 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602009b090 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.238+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602009b650 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.239+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602015c2d0 con 0x7f60200b40e0 2024-09-10T10:54:11.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.240+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f600802ad10 con 0x7f60200b40e0 2024-09-10T10:54:11.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.241+0000 7f6016a00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 0x7f5ffc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:11.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.241+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9056+0+0 (secure 0 0 0) 0x7f60080ac680 con 0x7f60200b40e0 2024-09-10T10:54:11.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.242+0000 7f601ea00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 0x7f5ffc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:11.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.242+0000 7f601ea00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 0x7f5ffc06e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:11.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.245+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f600807a3b0 con 0x7f60200b40e0 2024-09-10T10:54:11.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.465+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6020072b20 con 0x7f60200b40e0 2024-09-10T10:54:11.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.466+0000 7f6016a00640 1 -- 172.21.15.28:0/3948399316 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 277 v277) ==== 95+0+5224 (secure 0 0 0) 0x7f6008079cb0 con 0x7f60200b40e0 2024-09-10T10:54:11.467 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:11.467 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":277,"btime":"2024-09-10T10:54:11:054617+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":277,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:11.054604+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":132,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9690},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9690":{"gid":9690,"name":"a","rank":0,"incarnation":277,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/509398397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":509398397},{"type":"v1","addr":"172.21.15.28:6835","nonce":509398397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:11.467 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 277 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 msgr2=0x7f5ffc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 0x7f5ffc06e2f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6010000f60 tx=0x7f6010009040 comp rx=0 tx=0).stop 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 msgr2=0x7f602015db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f60080090b0 tx=0x7f60080046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 shutdown_connections 2024-09-10T10:54:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ffc06be30 0x7f5ffc06e2f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f602015bef0 0x7f60201531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60200b40e0 0x7f602015db00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 --2- 172.21.15.28:0/3948399316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60200af1b0 0x7f6020151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 >> 172.21.15.28:0/3948399316 conn(0x7f602006c240 msgr2=0x7f60200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 shutdown_connections 2024-09-10T10:54:11.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:11.468+0000 7f601fe00640 1 -- 172.21.15.28:0/3948399316 wait complete. 2024-09-10T10:54:11.485 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:54:14.075 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:54:14.074+0000 7f4268c00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-10T10:54:14.077 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:14.076+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:14.077 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:14.076+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:54:15.055 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:54:15.054+0000 7f426dc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:16.487 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:16.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.654+0000 7f8d2c800640 1 Processor -- start 2024-09-10T10:54:16.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.654+0000 7f8d2c800640 1 -- start start 2024-09-10T10:54:16.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d2c800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d2c800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d24072de0 0x7f8d2406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d2c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d24146140 con 0x7f8d24072700 2024-09-10T10:54:16.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d2c800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8d240cd740 con 0x7f8d24072de0 2024-09-10T10:54:16.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d2c800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d240cda20 con 0x7f8d24072320 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.655+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8d24072de0 0x7f8d2406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50606/0 (socket says 172.21.15.28:50606) 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 -- 172.21.15.28:0/853077251 learned_addr learned my addr 172.21.15.28:0/853077251 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d2a800640 1 --1- 172.21.15.28:0/853077251 >> v1:172.21.15.138:6789/0 conn(0x7f8d24072700 0x7f8d2406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:42214/0 (socket says 172.21.15.28:42214) 2024-09-10T10:54:16.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 -- 172.21.15.28:0/853077251 >> v2:172.21.15.138:3300/0 conn(0x7f8d24072de0 msgr2=0x7f8d2406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:16.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 --2- 172.21.15.28:0/853077251 >> v2:172.21.15.138:3300/0 conn(0x7f8d24072de0 0x7f8d2406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 -- 172.21.15.28:0/853077251 >> v1:172.21.15.138:6789/0 conn(0x7f8d24072700 legacy=0x7f8d2406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.656+0000 7f8d29e00640 1 -- 172.21.15.28:0/853077251 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d240cdd00 con 0x7f8d24072320 2024-09-10T10:54:16.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.658+0000 7f8d29e00640 1 --2- 172.21.15.28:0/853077251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009870 tx=0x7f8d1802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf08db63c0d6680d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:16.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.659+0000 7f8d28a00640 1 -- 172.21.15.28:0/853077251 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18037070 con 0x7f8d24072320 2024-09-10T10:54:16.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.659+0000 7f8d28a00640 1 -- 172.21.15.28:0/853077251 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d18032440 con 0x7f8d24072320 2024-09-10T10:54:16.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.659+0000 7f8d28a00640 1 -- 172.21.15.28:0/853077251 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18032810 con 0x7f8d24072320 2024-09-10T10:54:16.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.659+0000 7f8d2c800640 1 -- 172.21.15.28:0/853077251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 msgr2=0x7f8d240cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.659+0000 7f8d2c800640 1 --2- 172.21.15.28:0/853077251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009870 tx=0x7f8d1802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 -- 172.21.15.28:0/853077251 shutdown_connections 2024-09-10T10:54:16.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 --2- 172.21.15.28:0/853077251 >> v2:172.21.15.138:3300/0 conn(0x7f8d24072de0 0x7f8d2406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 --2- 172.21.15.28:0/853077251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072320 0x7f8d240cce60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 -- 172.21.15.28:0/853077251 >> 172.21.15.28:0/853077251 conn(0x7f8d24067d70 msgr2=0x7f8d2406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:16.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 -- 172.21.15.28:0/853077251 shutdown_connections 2024-09-10T10:54:16.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.660+0000 7f8d2c800640 1 -- 172.21.15.28:0/853077251 wait complete. 2024-09-10T10:54:16.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.661+0000 7f8d2c800640 1 Processor -- start 2024-09-10T10:54:16.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.661+0000 7f8d2c800640 1 -- start start 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 0x7f8d240b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072700 0x7f8d24144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d240cd520 con 0x7f8d24072de0 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d240cd3a0 con 0x7f8d24072320 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.662+0000 7f8d2c800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2406ed00 con 0x7f8d24072700 2024-09-10T10:54:16.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d29e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 0x7f8d240b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45982/0 (socket says 172.21.15.28:45982) 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 -- 172.21.15.28:0/2837001178 learned_addr learned my addr 172.21.15.28:0/2837001178 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d29400640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072700 0x7f8d24144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 -- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072700 msgr2=0x7f8d24144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072700 0x7f8d24144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 -- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 msgr2=0x7f8d240b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 0x7f8d240b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 -- 172.21.15.28:0/2837001178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d241f3e60 con 0x7f8d24072de0 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d29e00640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 0x7f8d240b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:16.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.663+0000 7f8d2a800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c7b0 tx=0x7f8d2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:16.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.664+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20017070 con 0x7f8d24072de0 2024-09-10T10:54:16.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.664+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2000a5e0 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.664+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d241f4140 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.664+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2000a990 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.664+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d241f4700 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.665+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d240cce60 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.666+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d2001a070 con 0x7f8d24072de0 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.667+0000 7f8d1e000640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:16.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.667+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f8d2008b170 con 0x7f8d24072de0 2024-09-10T10:54:16.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.667+0000 7f8d29e00640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:16.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.668+0000 7f8d29e00640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 0x7f8d0806e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8d18034000 tx=0x7f8d1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:16.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.671+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d200586b0 con 0x7f8d24072de0 2024-09-10T10:54:16.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.892+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d2406d090 con 0x7f8d24072de0 2024-09-10T10:54:16.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.892+0000 7f8d1e000640 1 -- 172.21.15.28:0/2837001178 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+4392 (secure 0 0 0) 0x7f8d2005c9b0 con 0x7f8d24072de0 2024-09-10T10:54:16.894 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:16.894 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":281,"btime":"2024-09-10T10:54:14:076796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:14.076791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:16.894 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 281 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 msgr2=0x7f8d0806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 0x7f8d0806e2f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8d18034000 tx=0x7f8d1802e000 comp rx=0 tx=0).stop 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 msgr2=0x7f8d2413e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c7b0 tx=0x7f8d2000cc70 comp rx=0 tx=0).stop 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 shutdown_connections 2024-09-10T10:54:16.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8d0806be30 0x7f8d0806e2f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d24072de0 0x7f8d2413e850 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d24072700 0x7f8d24144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 --2- 172.21.15.28:0/2837001178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d24072320 0x7f8d240b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 >> 172.21.15.28:0/2837001178 conn(0x7f8d24067d70 msgr2=0x7f8d240c7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 shutdown_connections 2024-09-10T10:54:16.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:16.895+0000 7f8d2c800640 1 -- 172.21.15.28:0/2837001178 wait complete. 2024-09-10T10:54:16.912 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:54:16.912 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:17.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.066+0000 7f667a800640 1 Processor -- start 2024-09-10T10:54:17.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.066+0000 7f667a800640 1 -- start start 2024-09-10T10:54:17.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.067+0000 7f667a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6674072700 0x7f667415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.067+0000 7f667a800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6674146140 con 0x7f6674072320 2024-09-10T10:54:17.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.067+0000 7f667a800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66740cd740 con 0x7f6674072de0 2024-09-10T10:54:17.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.067+0000 7f667a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66740cda20 con 0x7f6674072700 2024-09-10T10:54:17.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6673400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6674072320 0x7f66740cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52458/0 (socket says 172.21.15.28:52458) 2024-09-10T10:54:17.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6673400640 1 -- 172.21.15.28:0/4040055376 learned_addr learned my addr 172.21.15.28:0/4040055376 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:17.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6673e00640 1 --2- 172.21.15.28:0/4040055376 >> v2:172.21.15.138:3300/0 conn(0x7f6674072700 0x7f667415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140790925 0 0) 0x7f6674146140 con 0x7f6674072320 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6658003520 con 0x7f6674072320 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527661567 0 0) 0x7f66740cd740 con 0x7f6674072de0 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6674146140 con 0x7f6674072de0 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 688039797 0 0) 0x7f6658003520 con 0x7f6674072320 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 >> v2:172.21.15.138:3300/0 conn(0x7f6674072700 msgr2=0x7f667415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 --2- 172.21.15.28:0/4040055376 >> v2:172.21.15.138:3300/0 conn(0x7f6674072700 0x7f667415d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.068+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 >> v1:172.21.15.138:6789/0 conn(0x7f6674072de0 legacy=0x7f667415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66740cdd00 con 0x7f6674072320 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f6664001670 con 0x7f6674072320 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 >> v1:172.21.15.28:6789/0 conn(0x7f6674072320 legacy=0x7f66740cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6658003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6658005210 0x7f665800fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66740cda20 con 0x7f6658003b90 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6658010550 con 0x7f6658005210 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6658010750 con 0x7f6658004800 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6658003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f666be00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673e00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6658005210 0x7f665800fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 -- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6658004800 msgr2=0x7f6658004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6658004800 0x7f6658004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 -- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6658005210 msgr2=0x7f665800fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:17.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6658005210 0x7f665800fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673400640 1 -- 172.21.15.28:0/4040055376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6658003520 con 0x7f6658003b90 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673e00640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6658005210 0x7f665800fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.069+0000 7f6673e00640 1 -- 172.21.15.28:0/4040055376 reap_dead start 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.070+0000 7f6673400640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6658003f90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6664007fc0 tx=0x7f6658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9433d0c15b02867 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.070+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664007be0 con 0x7f6658003b90 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6664005af0 con 0x7f6658003b90 2024-09-10T10:54:17.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f6672a00640 1 -- 172.21.15.28:0/4040055376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664007070 con 0x7f6658003b90 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 -- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 msgr2=0x7f6658003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6658003f90 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f6664007fc0 tx=0x7f6658005830 comp rx=0 tx=0).stop 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 -- 172.21.15.28:0/4040055376 shutdown_connections 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 --2- 172.21.15.28:0/4040055376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6658003f90 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 -- 172.21.15.28:0/4040055376 >> 172.21.15.28:0/4040055376 conn(0x7f6674067d70 msgr2=0x7f667406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 -- 172.21.15.28:0/4040055376 shutdown_connections 2024-09-10T10:54:17.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.071+0000 7f667a800640 1 -- 172.21.15.28:0/4040055376 wait complete. 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.073+0000 7f667a800640 1 Processor -- start 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.073+0000 7f667a800640 1 -- start start 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 0x7f667413dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 0x7f66740c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66740ccfe0 con 0x7f6658003b90 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66740cce60 con 0x7f667415bda0 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f667a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66740cd160 con 0x7f667413e2c0 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f6673400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f6673400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46002/0 (socket says 172.21.15.28:46002) 2024-09-10T10:54:17.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f6673400640 1 -- 172.21.15.28:0/3820413909 learned_addr learned my addr 172.21.15.28:0/3820413909 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f666be00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 0x7f667413dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.074+0000 7f6673e00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 0x7f66740c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 -- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 msgr2=0x7f66740c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 0x7f66740c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 -- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 msgr2=0x7f667413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 0x7f667413dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 -- 172.21.15.28:0/3820413909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66740c74c0 con 0x7f6658003b90 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f666be00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 0x7f667413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673e00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 0x7f66740c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f6673400640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f666400c1d0 tx=0x7f66740cded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664037040 con 0x7f6658003b90 2024-09-10T10:54:17.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6664041070 con 0x7f6658003b90 2024-09-10T10:54:17.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664001670 con 0x7f6658003b90 2024-09-10T10:54:17.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66740c77a0 con 0x7f6658003b90 2024-09-10T10:54:17.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.075+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66740c7d60 con 0x7f6658003b90 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.076+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667413e6a0 con 0x7f6658003b90 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.077+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66640049e0 con 0x7f6658003b90 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.078+0000 7f666b400640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 0x7f665006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.078+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f66640ab980 con 0x7f6658003b90 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.078+0000 7f666be00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 0x7f665006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.079+0000 7f666be00640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 0x7f665006e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.083+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66640795a0 con 0x7f6658003b90 2024-09-10T10:54:17.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.301+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66741437b0 con 0x7f6658003b90 2024-09-10T10:54:17.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.301+0000 7f666b400640 1 -- 172.21.15.28:0/3820413909 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+4392 (secure 0 0 0) 0x7f6664078ea0 con 0x7f6658003b90 2024-09-10T10:54:17.302 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:17.302 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":281,"btime":"2024-09-10T10:54:14:076796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:14.076791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:17.303 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 281 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 msgr2=0x7f665006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 0x7f665006e2f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f665c000c00 tx=0x7f665c009040 comp rx=0 tx=0).stop 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 msgr2=0x7f6674140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f666400c1d0 tx=0x7f66740cded0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 shutdown_connections 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f665006be30 0x7f665006e2f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f667413e2c0 0x7f66740c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f667415bda0 0x7f667413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 --2- 172.21.15.28:0/3820413909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6658003b90 0x7f6674140e40 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 >> 172.21.15.28:0/3820413909 conn(0x7f6674067d70 msgr2=0x7f6674142870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:17.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 shutdown_connections 2024-09-10T10:54:17.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.304+0000 7f667a800640 1 -- 172.21.15.28:0/3820413909 wait complete. 2024-09-10T10:54:17.321 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:54:17.321 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:17.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.476+0000 7f3a4ce00640 1 Processor -- start 2024-09-10T10:54:17.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.476+0000 7f3a4ce00640 1 -- start start 2024-09-10T10:54:17.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.477+0000 7f3a4ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a48072de0 0x7f3a4815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.477+0000 7f3a4ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.477+0000 7f3a4ce00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a48146140 con 0x7f3a48072320 2024-09-10T10:54:17.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.477+0000 7f3a4ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3a480cd740 con 0x7f3a48072de0 2024-09-10T10:54:17.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.477+0000 7f3a4ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a480cda20 con 0x7f3a48072700 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a3fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3a48072de0 0x7f3a4815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:50648/0 (socket says 172.21.15.28:50648) 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 -- 172.21.15.28:0/1358638283 learned_addr learned my addr 172.21.15.28:0/1358638283 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 -- 172.21.15.28:0/1358638283 >> v2:172.21.15.138:3300/0 conn(0x7f3a48072de0 msgr2=0x7f3a4815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 --2- 172.21.15.28:0/1358638283 >> v2:172.21.15.138:3300/0 conn(0x7f3a48072de0 0x7f3a4815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 -- 172.21.15.28:0/1358638283 >> v1:172.21.15.138:6789/0 conn(0x7f3a48072320 legacy=0x7f3a480cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a46800640 1 -- 172.21.15.28:0/1358638283 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a480cdd00 con 0x7f3a48072700 2024-09-10T10:54:17.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.478+0000 7f3a45400640 1 -- 172.21.15.28:0/1358638283 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2902448240 0 0) 0x7f3a48146140 con 0x7f3a48072320 2024-09-10T10:54:17.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.479+0000 7f3a46800640 1 --2- 172.21.15.28:0/1358638283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009e50 tx=0x7f3a4002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21276a5b40d1f49c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.479+0000 7f3a45400640 1 -- 172.21.15.28:0/1358638283 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40036070 con 0x7f3a48072700 2024-09-10T10:54:17.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.479+0000 7f3a45400640 1 -- 172.21.15.28:0/1358638283 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a40031600 con 0x7f3a48072700 2024-09-10T10:54:17.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.479+0000 7f3a45400640 1 -- 172.21.15.28:0/1358638283 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a400319d0 con 0x7f3a48072700 2024-09-10T10:54:17.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 -- 172.21.15.28:0/1358638283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 msgr2=0x7f3a4815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/1358638283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009e50 tx=0x7f3a4002a680 comp rx=0 tx=0).stop 2024-09-10T10:54:17.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 -- 172.21.15.28:0/1358638283 shutdown_connections 2024-09-10T10:54:17.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/1358638283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815d720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/1358638283 >> v2:172.21.15.138:3300/0 conn(0x7f3a48072de0 0x7f3a4815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 -- 172.21.15.28:0/1358638283 >> 172.21.15.28:0/1358638283 conn(0x7f3a48067d70 msgr2=0x7f3a4806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:17.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 -- 172.21.15.28:0/1358638283 shutdown_connections 2024-09-10T10:54:17.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.480+0000 7f3a4ce00640 1 -- 172.21.15.28:0/1358638283 wait complete. 2024-09-10T10:54:17.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.482+0000 7f3a4ce00640 1 Processor -- start 2024-09-10T10:54:17.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.482+0000 7f3a4ce00640 1 -- start start 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a48072de0 0x7f3a48133710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4815dde0 con 0x7f3a48072320 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4815dc60 con 0x7f3a48072de0 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a4ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4815df60 con 0x7f3a48072700 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a45e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a45e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46018/0 (socket says 172.21.15.28:46018) 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a45e00640 1 -- 172.21.15.28:0/2659963240 learned_addr learned my addr 172.21.15.28:0/2659963240 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.483+0000 7f3a3fe00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 -- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 msgr2=0x7f3a4815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 -- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a48072de0 msgr2=0x7f3a48133710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a48072de0 0x7f3a48133710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 -- 172.21.15.28:0/2659963240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a480ca1b0 con 0x7f3a48072320 2024-09-10T10:54:17.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a45e00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c4b0 tx=0x7f3a3800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38017070 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a38005480 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a38003830 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a480ca490 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.484+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a480caa50 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.485+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4815e370 con 0x7f3a48072320 2024-09-10T10:54:17.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.486+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3a3801a070 con 0x7f3a48072320 2024-09-10T10:54:17.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.487+0000 7f3a3e000640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 0x7f3a2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:17.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.487+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f3a3808b070 con 0x7f3a48072320 2024-09-10T10:54:17.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.488+0000 7f3a3fe00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 0x7f3a2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:17.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.489+0000 7f3a3fe00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 0x7f3a2406e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3a481447b0 tx=0x7f3a30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:17.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.492+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a38058c00 con 0x7f3a48072320 2024-09-10T10:54:17.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.710+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a48144500 con 0x7f3a48072320 2024-09-10T10:54:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.711+0000 7f3a3e000640 1 -- 172.21.15.28:0/2659963240 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 281 v281) ==== 95+0+4392 (secure 0 0 0) 0x7f3a38058500 con 0x7f3a48072320 2024-09-10T10:54:17.712 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:17.712 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":281,"btime":"2024-09-10T10:54:14:076796+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":281,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:14.076791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 281 2024-09-10T10:54:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 msgr2=0x7f3a2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 0x7f3a2406e2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3a481447b0 tx=0x7f3a30008040 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 msgr2=0x7f3a480c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c4b0 tx=0x7f3a3800c970 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 shutdown_connections 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3a2406be30 0x7f3a2406e2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a48072de0 0x7f3a48133710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3a48072700 0x7f3a4815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 --2- 172.21.15.28:0/2659963240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a48072320 0x7f3a480c9960 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 >> 172.21.15.28:0/2659963240 conn(0x7f3a48067d70 msgr2=0x7f3a480b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 shutdown_connections 2024-09-10T10:54:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:17.713+0000 7f3a4ce00640 1 -- 172.21.15.28:0/2659963240 wait complete. 2024-09-10T10:54:17.730 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap key 100.00000000:stray0_head' led to DAMAGED state 2024-09-10T10:54:17.730 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap key 1.00000000:subdir_head' 2024-09-10T10:54:17.730 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:17.730 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:17.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:54:17.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:54:17.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:17.766 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:17.841 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:54:17.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:54:17.842 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:54:18.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.038+0000 7f91d6200640 1 Processor -- start 2024-09-10T10:54:18.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.039+0000 7f91d6200640 1 -- start start 2024-09-10T10:54:18.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d00720b0 0x7f91d00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 0x7f91d006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 0x7f91d00b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f91d00afa20 con 0x7f91d00720b0 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91d00afce0 con 0x7f91d0162ad0 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91d6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91d00b34d0 con 0x7f91d0072490 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91cf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d00720b0 0x7f91d00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91cf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d00720b0 0x7f91d00cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46020/0 (socket says 172.21.15.28:46020) 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91cf400640 1 -- 172.21.15.28:0/3005653737 learned_addr learned my addr 172.21.15.28:0/3005653737 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91cfe00640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 0x7f91d00b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91c7e00640 1 --2- 172.21.15.28:0/3005653737 >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 0x7f91d006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.040+0000 7f91cf400640 1 -- 172.21.15.28:0/3005653737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 msgr2=0x7f91d00b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91cf400640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 0x7f91d00b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91cf400640 1 -- 172.21.15.28:0/3005653737 >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 msgr2=0x7f91d006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:18.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91cf400640 1 --2- 172.21.15.28:0/3005653737 >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 0x7f91d006f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91cf400640 1 -- 172.21.15.28:0/3005653737 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d00b37b0 con 0x7f91d00720b0 2024-09-10T10:54:18.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91cfe00640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 0x7f91d00b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:18.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.041+0000 7f91c7e00640 1 --2- 172.21.15.28:0/3005653737 >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 0x7f91d006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:18.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91cf400640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d00720b0 0x7f91d00cfca0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f91d0072d20 tx=0x7f91c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede35f069d5d9187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91cea00640 1 -- 172.21.15.28:0/3005653737 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0037070 con 0x7f91d00720b0 2024-09-10T10:54:18.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91cea00640 1 -- 172.21.15.28:0/3005653737 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c0032440 con 0x7f91d00720b0 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91cea00640 1 -- 172.21.15.28:0/3005653737 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c0032810 con 0x7f91d00720b0 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 -- 172.21.15.28:0/3005653737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d00720b0 msgr2=0x7f91d00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d00720b0 0x7f91d00cfca0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f91d0072d20 tx=0x7f91c002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 -- 172.21.15.28:0/3005653737 shutdown_connections 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d0072490 0x7f91d00b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 --2- 172.21.15.28:0/3005653737 >> v2:172.21.15.138:3300/0 conn(0x7f91d0162ad0 0x7f91d006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.042+0000 7f91d6200640 1 --2- 172.21.15.28:0/3005653737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d00720b0 0x7f91d00cfca0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.043+0000 7f91d6200640 1 -- 172.21.15.28:0/3005653737 >> 172.21.15.28:0/3005653737 conn(0x7f91d0067d70 msgr2=0x7f91d006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.043+0000 7f91d6200640 1 -- 172.21.15.28:0/3005653737 shutdown_connections 2024-09-10T10:54:18.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.043+0000 7f91d6200640 1 -- 172.21.15.28:0/3005653737 wait complete. 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.044+0000 7f91d6200640 1 Processor -- start 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.044+0000 7f91d6200640 1 -- start start 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 0x7f91d0149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 0x7f91d0163f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0072900 con 0x7f91d0162ad0 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91d00b3d70 con 0x7f91d0072490 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.045+0000 7f91d6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91d00b47d0 con 0x7f91d00720b0 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46036/0 (socket says 172.21.15.28:46036) 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 -- 172.21.15.28:0/3954333479 learned_addr learned my addr 172.21.15.28:0/3954333479 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cf400640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 0x7f91d0149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91c7e00640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 0x7f91d0163f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 -- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 msgr2=0x7f91d0149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 0x7f91d0149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 -- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 msgr2=0x7f91d0163f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 0x7f91d0163f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 -- 172.21.15.28:0/3954333479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d00a9950 con 0x7f91d0162ad0 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cf400640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 0x7f91d0149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:18.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91c7e00640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 0x7f91d0163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.046+0000 7f91cfe00640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007e10 tx=0x7f91c800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.047+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c801a070 con 0x7f91d0162ad0 2024-09-10T10:54:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.047+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c8003b90 con 0x7f91d0162ad0 2024-09-10T10:54:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.047+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8004030 con 0x7f91d0162ad0 2024-09-10T10:54:18.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.047+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d014a8c0 con 0x7f91d0162ad0 2024-09-10T10:54:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.047+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d00a9c30 con 0x7f91d0162ad0 2024-09-10T10:54:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.048+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d00b4290 con 0x7f91d0162ad0 2024-09-10T10:54:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.049+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f91c801d070 con 0x7f91d0162ad0 2024-09-10T10:54:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.050+0000 7f91c7400640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.050+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f91c808bed0 con 0x7f91d0162ad0 2024-09-10T10:54:18.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.050+0000 7f91cf400640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.051+0000 7f91cf400640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f91c0034000 tx=0x7f91c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.054+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f91c8010070 con 0x7f91d0162ad0 2024-09-10T10:54:18.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.278+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91d00cec40 con 0x7f91d0162ad0 2024-09-10T10:54:18.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.707+0000 7f91c7400640 1 -- 172.21.15.28:0/3954333479 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v282) ==== 161+0+0 (secure 0 0 0) 0x7f91c8059af0 con 0x7f91d0162ad0 2024-09-10T10:54:18.709 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 msgr2=0x7f91ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 0x7f91ac06e2f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f91c0034000 tx=0x7f91c002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:18.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 msgr2=0x7f91d0164440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007e10 tx=0x7f91c800d4a0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 shutdown_connections 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f91ac06be30 0x7f91ac06e2f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d0162ad0 0x7f91d0164440 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91d0072490 0x7f91d0163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 --2- 172.21.15.28:0/3954333479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91d00720b0 0x7f91d0149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 >> 172.21.15.28:0/3954333479 conn(0x7f91d0067d70 msgr2=0x7f91d00b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 shutdown_connections 2024-09-10T10:54:18.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.710+0000 7f91d6200640 1 -- 172.21.15.28:0/3954333479 wait complete. 2024-09-10T10:54:18.728 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:54:18.829 INFO:tasks.ceph.mds.a.smithi028.stderr: -48> 2024-09-10T10:54:14.074+0000 7f4268c00640 -1 mds.0.damage notify_dentry Damage to dentries in fragment * of ino 0x100is fatal because it is a system directory for this rank 2024-09-10T10:54:18.829 INFO:tasks.ceph.mds.a.smithi028.stderr: -30> 2024-09-10T10:54:15.054+0000 7f426dc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:18.868 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:54:18.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.886+0000 7f2caa800640 1 Processor -- start 2024-09-10T10:54:18.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.886+0000 7f2caa800640 1 -- start start 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2caa800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e660 0x7f2ca406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2caa800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca40cf000 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2caa800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca40d29a0 con 0x7f2ca40d1f90 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2caa800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca40b37d0 con 0x7f2ca415e660 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2ca9400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2ca40d0dd0 0x7f2ca40d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52472/0 (socket says 172.21.15.28:52472) 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2ca9400640 1 -- 172.21.15.28:0/2333884047 learned_addr learned my addr 172.21.15.28:0/2333884047 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.887+0000 7f2ca8a00640 1 --2- 172.21.15.28:0/2333884047 >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e660 0x7f2ca406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628624781 0 0) 0x7f2ca40cf000 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c8c003520 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2256566981 0 0) 0x7f2c8c003520 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca8a00640 1 -- 172.21.15.28:0/2333884047 >> v1:172.21.15.138:6789/0 conn(0x7f2ca40d1f90 legacy=0x7f2ca40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca8a00640 1 -- 172.21.15.28:0/2333884047 >> v1:172.21.15.28:6789/0 conn(0x7f2ca40d0dd0 legacy=0x7f2ca40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.888+0000 7f2ca8a00640 1 -- 172.21.15.28:0/2333884047 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca40d2c60 con 0x7f2ca415e660 2024-09-10T10:54:18.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.889+0000 7f2ca8a00640 1 --2- 172.21.15.28:0/2333884047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca406ccd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c900097a0 tx=0x7f2c9002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddf52c5c9ea95b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.889+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90037070 con 0x7f2ca415e660 2024-09-10T10:54:18.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.889+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9002ac50 con 0x7f2ca415e660 2024-09-10T10:54:18.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.889+0000 7f2ca3e00640 1 -- 172.21.15.28:0/2333884047 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98001670 con 0x7f2ca415e660 2024-09-10T10:54:18.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 -- 172.21.15.28:0/2333884047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 msgr2=0x7f2ca406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 --2- 172.21.15.28:0/2333884047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca406ccd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2c900097a0 tx=0x7f2c9002a1c0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 -- 172.21.15.28:0/2333884047 shutdown_connections 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 --2- 172.21.15.28:0/2333884047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca406ccd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 -- 172.21.15.28:0/2333884047 >> 172.21.15.28:0/2333884047 conn(0x7f2ca408bfc0 msgr2=0x7f2ca40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 -- 172.21.15.28:0/2333884047 shutdown_connections 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.890+0000 7f2caa800640 1 -- 172.21.15.28:0/2333884047 wait complete. 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.892+0000 7f2caa800640 1 Processor -- start 2024-09-10T10:54:18.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.892+0000 7f2caa800640 1 -- start start 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca40d1f90 0x7f2ca40a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d3220 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40cf330 con 0x7f2ca415e660 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2caa800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca40d33a0 con 0x7f2ca40d1f90 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46064/0 (socket says 172.21.15.28:46064) 2024-09-10T10:54:18.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 -- 172.21.15.28:0/4268391177 learned_addr learned my addr 172.21.15.28:0/4268391177 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:18.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca8a00640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca40d1f90 0x7f2ca40a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9e00640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 -- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca40d1f90 msgr2=0x7f2ca40a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca40d1f90 0x7f2ca40a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 -- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 msgr2=0x7f2ca415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9400640 1 -- 172.21.15.28:0/4268391177 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca415c5d0 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.893+0000 7f2ca9e00640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2ca9400640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2c98001eb0 tx=0x7f2c9800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98017070 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98005ad0 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98007070 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca40a1c30 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.894+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca415f8f0 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.895+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca406d2a0 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.896+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c9801a070 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.897+0000 7f2ca1600640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 0x7f2c8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:18.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.897+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f2c9808b160 con 0x7f2ca40d0dd0 2024-09-10T10:54:18.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.897+0000 7f2ca8a00640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 0x7f2c8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:18.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.898+0000 7f2ca8a00640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 0x7f2c8406e2f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2c90034000 tx=0x7f2c9002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:18.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:18.901+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ca415c5d0 con 0x7f2ca40d0dd0 2024-09-10T10:54:19.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.153+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f2ca40d1b20 con 0x7f2ca40d0dd0 2024-09-10T10:54:19.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.714+0000 7f2ca1600640 1 -- 172.21.15.28:0/4268391177 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v283) ==== 101+0+0 (secure 0 0 0) 0x7f2c98058d80 con 0x7f2ca40d0dd0 2024-09-10T10:54:19.715 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 msgr2=0x7f2c8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 0x7f2c8406e2f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2c90034000 tx=0x7f2c9002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 msgr2=0x7f2ca40b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f2c98001eb0 tx=0x7f2c9800cbe0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 shutdown_connections 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c8406be30 0x7f2c8406e2f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ca415e660 0x7f2ca415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca40d1f90 0x7f2ca40a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 --2- 172.21.15.28:0/4268391177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ca40d0dd0 0x7f2ca40b9de0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 >> 172.21.15.28:0/4268391177 conn(0x7f2ca408bfc0 msgr2=0x7f2ca40b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.717+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 shutdown_connections 2024-09-10T10:54:19.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.718+0000 7f2caa800640 1 -- 172.21.15.28:0/4268391177 wait complete. 2024-09-10T10:54:19.739 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:54:19.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.818+0000 7fb099d1bf40 1 Processor -- start 2024-09-10T10:54:19.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.818+0000 7fb099d1bf40 1 -- start start 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d18ae8e0 0x5654d1897570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193a3e0 0x5654d1939800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5654d18af530 con 0x5654d193d9c0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654d180cb30 con 0x5654d18ae8e0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654d193cee0 con 0x5654d193a3e0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb097000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb097000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46082/0 (socket says 172.21.15.28:46082) 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb097000640 1 -- 172.21.15.28:0/248735014 learned_addr learned my addr 172.21.15.28:0/248735014 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb095c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193a3e0 0x5654d1939800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.819+0000 7fb096600640 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d18ae8e0 0x5654d1897570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 -- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193a3e0 msgr2=0x5654d1939800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193a3e0 0x5654d1939800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 -- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d18ae8e0 msgr2=0x5654d1897570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:19.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d18ae8e0 0x5654d1897570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 -- 172.21.15.28:0/248735014 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d193bae0 con 0x5654d193d9c0 2024-09-10T10:54:19.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb097000640 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb0900015d0 tx=0x7fb090001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2d649cfd9d3714 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb08fe00640 1 -- 172.21.15.28:0/248735014 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090016070 con 0x5654d193d9c0 2024-09-10T10:54:19.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb08fe00640 1 -- 172.21.15.28:0/248735014 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb09000f730 con 0x5654d193d9c0 2024-09-10T10:54:19.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.820+0000 7fb08fe00640 1 -- 172.21.15.28:0/248735014 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09000fb20 con 0x5654d193d9c0 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 -- 172.21.15.28:0/248735014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 msgr2=0x5654d19ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb0900015d0 tx=0x7fb090001a90 comp rx=0 tx=0).stop 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 -- 172.21.15.28:0/248735014 shutdown_connections 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d193d9c0 0x5654d19ce6c0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193a3e0 0x5654d1939800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/248735014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d18ae8e0 0x5654d1897570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 -- 172.21.15.28:0/248735014 >> 172.21.15.28:0/248735014 conn(0x5654d18b30a0 msgr2=0x5654d193a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 -- 172.21.15.28:0/248735014 shutdown_connections 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 -- 172.21.15.28:0/248735014 wait complete. 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.821+0000 7fb099d1bf40 1 Processor -- start 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 -- start start 2024-09-10T10:54:19.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d193a3e0 0x5654d18b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193d9c0 0x5654d19348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5654d193bdc0 con 0x5654d18ae8e0 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5654d193b890 con 0x5654d193a3e0 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.822+0000 7fb099d1bf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5654d1897050 con 0x5654d193d9c0 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46098/0 (socket says 172.21.15.28:46098) 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 learned_addr learned my addr 172.21.15.28:0/3423536044 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:19.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193d9c0 msgr2=0x5654d19348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5654d193d9c0 0x5654d19348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d193a3e0 msgr2=0x5654d18b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:19.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5654d193a3e0 0x5654d18b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:19.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654d193ea70 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.823+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb088002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.824+0000 7fb08ea00640 1 -- 172.21.15.28:0/3423536044 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088055070 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.824+0000 7fb08ea00640 1 -- 172.21.15.28:0/3423536044 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb088051950 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.824+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654d193ed10 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.824+0000 7fb08ea00640 1 -- 172.21.15.28:0/3423536044 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088051d20 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.824+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654d19409d0 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.826+0000 7fb08ea00640 1 -- 172.21.15.28:0/3423536044 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb088058070 con 0x5654d18ae8e0 2024-09-10T10:54:19.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb08ea00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 0x7fb07806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb08ea00640 1 -- 172.21.15.28:0/3423536044 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7fb0880ce530 con 0x5654d18ae8e0 2024-09-10T10:54:19.829 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 0x5654d1935300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d1936de0 con 0x5654d18b10f0 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 0x7fb07806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.827+0000 7fb097000640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 0x5654d1935300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.828+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 0x7fb07806ccd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x5654d1939300 tx=0x7fb084005ea0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.828+0000 7fb097000640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 0x5654d1935300 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.830+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v133'1002 uv976 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.830+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19371c0 con 0x5654d18b10f0 2024-09-10T10:54:19.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.832+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v133'1003 uv1003 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.832+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d1937620 con 0x5654d18b10f0 2024-09-10T10:54:19.835 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:54:19.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.834+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v133'1004 uv1004 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.834+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d1937a10 con 0x5654d18b10f0 2024-09-10T10:54:19.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.836+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v133'1005 uv1005 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.836+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e0be0 con 0x5654d18b10f0 2024-09-10T10:54:19.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.839+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v133'1006 uv1006 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.839+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dfbd0 con 0x5654d18b10f0 2024-09-10T10:54:19.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.840+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v133'1007 uv1007 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.840+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d1937e10 con 0x5654d18b10f0 2024-09-10T10:54:19.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.842+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v133'1008 uv982 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.842+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dffe0 con 0x5654d18b10f0 2024-09-10T10:54:19.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.844+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v133'1009 uv1009 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.844+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff90 con 0x5654d18b10f0 2024-09-10T10:54:19.846 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:54:19.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.846+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v133'1010 uv1010 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.846+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff90 con 0x5654d18b10f0 2024-09-10T10:54:19.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.847+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v133'1011 uv1011 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.847+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.849+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v133'1012 uv1012 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.849+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.851+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v133'1013 uv1013 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.851+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.853+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v133'1014 uv989 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.853+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.854+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v133'1015 uv1015 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.855+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.857 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:54:19.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.856+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v133'1016 uv1016 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.856+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.858+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v133'1017 uv1017 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.858+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.859+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v133'1018 uv1018 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.859+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19dff70 con 0x5654d18b10f0 2024-09-10T10:54:19.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.861+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v133'1019 uv1019 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.861+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e0800 con 0x5654d18b10f0 2024-09-10T10:54:19.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.862+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v133'1020 uv1020 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.862+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.864+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v133'1021 uv995 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.864+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.866+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v133'1022 uv1022 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.866+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.869 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:54:19.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.868+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v133'1023 uv1023 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.868+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.869+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v133'1024 uv1024 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.870+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.871+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v133'1025 uv1025 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.871+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.873+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v133'1026 uv1026 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.873+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.875+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v133'1027 uv1001 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.875+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.876+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v133'1028 uv1028 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.876+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.878 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:54:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.878+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v133'1029 uv1029 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.878+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.879+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v133'1030 uv1030 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.879+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.881+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v133'1031 uv1031 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.881+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d18b10f0 2024-09-10T10:54:19.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.882+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v133'1032 uv1032 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d18b10f0 2024-09-10T10:54:19.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.882+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 0x5654d19e3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.882+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e06e0 con 0x5654d19e1490 2024-09-10T10:54:19.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.883+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 0x5654d19e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.884+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 0x5654d19e3880 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.886+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v133'727 uv709 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.886+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e3dc0 con 0x5654d19e1490 2024-09-10T10:54:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.888+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v133'728 uv728 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.888+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e4160 con 0x5654d19e1490 2024-09-10T10:54:19.890 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:54:19.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.889+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v133'729 uv729 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.889+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e4500 con 0x5654d19e1490 2024-09-10T10:54:19.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.891+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v133'730 uv730 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.891+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e48a0 con 0x5654d19e1490 2024-09-10T10:54:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.892+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v133'731 uv731 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.892+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e4c90 con 0x5654d19e1490 2024-09-10T10:54:19.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.894+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v133'732 uv714 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.894+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.895+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v133'733 uv733 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.895+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.897 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:54:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.897+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v133'734 uv734 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.897+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.898+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v133'735 uv735 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.898+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.900+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v133'736 uv736 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.900+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.901+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v133'737 uv719 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.901+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.903+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v133'738 uv738 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.903+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.904+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v133'739 uv739 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.905 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:54:19.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.904+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.906+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v133'740 uv740 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.906+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.907+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v133'741 uv741 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.907+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.908+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v133'742 uv725 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.908+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.910+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v133'743 uv743 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.910+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.912 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:54:19.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.911+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v133'744 uv744 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.911+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.912+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v133'745 uv745 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.912+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e6040 con 0x5654d19e1490 2024-09-10T10:54:19.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.914+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v133'746 uv746 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.914+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e5030 con 0x5654d19e1490 2024-09-10T10:54:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.915+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v133'747 uv747 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19e1490 2024-09-10T10:54:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.915+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654d19e5950 0x5654d19e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.915+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e8440 con 0x5654d19e5950 2024-09-10T10:54:19.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.915+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654d19e5950 0x5654d19e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.916+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654d19e5950 0x5654d19e5d50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.918+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v133'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.918+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e87e0 con 0x5654d19e5950 2024-09-10T10:54:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.919+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v133'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.919+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e8c00 con 0x5654d19e5950 2024-09-10T10:54:19.921 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:54:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.920+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v133'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.920+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ea4b0 con 0x5654d19e5950 2024-09-10T10:54:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.921+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v133'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.922+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ea850 con 0x5654d19e5950 2024-09-10T10:54:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.923+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v133'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.923+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e94a0 con 0x5654d19e5950 2024-09-10T10:54:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.924+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v133'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.924+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654d19e9da0 0x5654d19ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.924+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed140 con 0x5654d19e9da0 2024-09-10T10:54:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.924+0000 7fb097000640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654d19e9da0 0x5654d19ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.925+0000 7fb097000640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654d19e9da0 0x5654d19ecc00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.926+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v133'143 uv142 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.926+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed140 con 0x5654d19e9da0 2024-09-10T10:54:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.927+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v133'144 uv144 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.927+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed160 con 0x5654d19e9da0 2024-09-10T10:54:19.929 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:54:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.929+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v133'145 uv145 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.929+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.930+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v133'146 uv146 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.930+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.931+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v133'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.931+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.932+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v133'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.932+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.934 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:54:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.933+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v133'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.933+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ee5b0 con 0x5654d19e5950 2024-09-10T10:54:19.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.935+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v133'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.935+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ee5b0 con 0x5654d19e5950 2024-09-10T10:54:19.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.936+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v133'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.936+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.937+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v133'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.937+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.939+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v133'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.939+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.940+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v133'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.940+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.942 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:54:19.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.941+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v133'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.942+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ee5b0 con 0x5654d19e9da0 2024-09-10T10:54:19.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.943+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v133'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.943+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ee5b0 con 0x5654d19e9da0 2024-09-10T10:54:19.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.945+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v133'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.945+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.946+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v133'365 uv365 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.946+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.947+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v133'366 uv359 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.947+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.949+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v133'367 uv367 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.949+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.951 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:54:19.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.950+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v133'368 uv368 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.950+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.952+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v133'369 uv369 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.952+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e9da0 2024-09-10T10:54:19.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.953+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v133'370 uv370 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.953+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.954+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v133'163 uv162 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.954+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.956+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v133'164 uv164 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.956+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.958 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:54:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.957+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v133'165 uv165 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.957+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.958+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v133'166 uv166 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.958+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19ed5a0 con 0x5654d19e5950 2024-09-10T10:54:19.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.959+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v133'167 uv167 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19e5950 2024-09-10T10:54:19.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.959+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654d19edab0 0x5654d19efea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.959+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f04c0 con 0x5654d19edab0 2024-09-10T10:54:19.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.960+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654d19edab0 0x5654d19efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:19.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.960+0000 7fb096600640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654d19edab0 0x5654d19efea0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:19.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.962+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v133'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.962+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.963+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v133'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.963+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.964+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v133'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.965 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:54:19.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.964+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f18a0 con 0x5654d19edab0 2024-09-10T10:54:19.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.966+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v133'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.966+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f18a0 con 0x5654d19edab0 2024-09-10T10:54:19.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.967+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v133'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.967+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.968+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v133'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.968+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.970+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v133'325 uv318 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.970+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.972+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v133'326 uv326 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.972+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0890 con 0x5654d19edab0 2024-09-10T10:54:19.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.973+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v133'327 uv327 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.974 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:54:19.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.973+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.975+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v133'328 uv328 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.975+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.976+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v133'329 uv329 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.976+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.977+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v133'330 uv324 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.977+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.979+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v133'331 uv331 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.979+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.981 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:54:19.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.980+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v133'332 uv332 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.980+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19e9090 con 0x5654d19edab0 2024-09-10T10:54:19.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.982+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v133'333 uv333 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.982+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f36e0 con 0x5654d19edab0 2024-09-10T10:54:19.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.984+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v133'334 uv334 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:19.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.984+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.985+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v133'142 uv140 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.985+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.988+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v133'143 uv143 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.988+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.990 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:54:19.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.990+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v133'144 uv144 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.990+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.991+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v133'145 uv145 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.991+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.992+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v133'195 uv194 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.992+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.994+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v133'196 uv196 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.994+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.996 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:54:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.995+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v133'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.995+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f1d50 con 0x5654d19e9da0 2024-09-10T10:54:19.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.996+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v133'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.996+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f1d50 con 0x5654d19e9da0 2024-09-10T10:54:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.997+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v133'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.997+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0d40 con 0x5654d19e9da0 2024-09-10T10:54:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.998+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v133'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb09000f070 con 0x5654d19e9da0 2024-09-10T10:54:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.998+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654d19f13c0 0x5654d19f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.998+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3da0 con 0x5654d19f13c0 2024-09-10T10:54:19.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.999+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654d19f13c0 0x5654d19f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:19.999+0000 7fb095c00640 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654d19f13c0 0x5654d19f37b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.000+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v133'166 uv165 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19f13c0 2024-09-10T10:54:20.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.000+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3da0 con 0x5654d19f13c0 2024-09-10T10:54:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.001+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v133'167 uv167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19f13c0 2024-09-10T10:54:20.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.001+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3da0 con 0x5654d19f13c0 2024-09-10T10:54:20.003 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:54:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.003+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v133'168 uv168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19f13c0 2024-09-10T10:54:20.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.003+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0910 con 0x5654d19f13c0 2024-09-10T10:54:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.004+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v133'169 uv169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19f13c0 2024-09-10T10:54:20.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.004+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0910 con 0x5654d19f13c0 2024-09-10T10:54:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.005+0000 7fb095c00640 1 -- 172.21.15.28:0/3423536044 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v133'170 uv170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fb084018040 con 0x5654d19f13c0 2024-09-10T10:54:20.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.005+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0910 con 0x5654d19edab0 2024-09-10T10:54:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.006+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v133'402 uv393 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.006+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f0910 con 0x5654d19edab0 2024-09-10T10:54:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.008+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v133'403 uv403 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.008+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.010 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:54:20.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.009+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v133'404 uv404 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.009+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.010+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v133'405 uv405 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.010+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.011+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v133'406 uv406 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.011+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.012+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v133'407 uv400 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.012+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.014+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v133'408 uv408 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.014+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.016 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:54:20.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.015+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v133'409 uv409 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.015+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3dd0 con 0x5654d19edab0 2024-09-10T10:54:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.016+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v133'410 uv410 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.016+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3db0 con 0x5654d19edab0 2024-09-10T10:54:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.017+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v133'411 uv411 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.017+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f3db0 con 0x5654d19edab0 2024-09-10T10:54:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.019+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v133'412 uv412 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.019+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x5654d19f46f0 con 0x5654d19edab0 2024-09-10T10:54:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb096600640 1 -- 172.21.15.28:0/3423536044 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v133'413 uv413 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fb08806b020 con 0x5654d19edab0 2024-09-10T10:54:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654d19e5950 msgr2=0x5654d19e5d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5654d19e5950 0x5654d19e5d50 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654d19edab0 msgr2=0x5654d19efea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5654d19edab0 0x5654d19efea0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654d19f13c0 msgr2=0x5654d19f37b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5654d19f13c0 0x5654d19f37b0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654d19e9da0 msgr2=0x5654d19ecc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5654d19e9da0 0x5654d19ecc00 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 msgr2=0x5654d19e3880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 reap_dead start 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 0x5654d19e3880 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb097000640 1 -- 172.21.15.28:0/3423536044 reap_dead start 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 msgr2=0x5654d1935300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.020+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 0x5654d1935300 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 msgr2=0x7fb07806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 0x7fb07806ccd0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x5654d1939300 tx=0x7fb084005ea0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 msgr2=0x5654d1933140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb088000c00 tx=0x7fb088002fb0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 shutdown_connections 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x5654d19e1490 0x5654d19e3880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5654d18b10f0 0x5654d1935300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806a810 0x7fb07806ccd0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 --2- 172.21.15.28:0/3423536044 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5654d18ae8e0 0x5654d1933140 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 >> 172.21.15.28:0/3423536044 conn(0x5654d18b30a0 msgr2=0x5654d1939800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 shutdown_connections 2024-09-10T10:54:20.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.021+0000 7fb099d1bf40 1 -- 172.21.15.28:0/3423536044 wait complete. 2024-09-10T10:54:20.026 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 1.00000000 subdir_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:54:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.103+0000 7fcd60520f40 1 Processor -- start 2024-09-10T10:54:20.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.103+0000 7fcd60520f40 1 -- start start 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x55763491add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x5576349376e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55763491e790 con 0x5576349c5640 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557634895b30 con 0x5576349c5a20 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd60520f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55763491de60 con 0x55763491d4c0 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46108/0 (socket says 172.21.15.28:46108) 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 -- 172.21.15.28:0/2666378275 learned_addr learned my addr 172.21.15.28:0/2666378275 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd5d600640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x5576349376e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd5cc00640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x55763491add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 -- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 msgr2=0x55763491add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x55763491add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 -- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 msgr2=0x5576349376e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x5576349376e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd57e00640 1 -- 172.21.15.28:0/2666378275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576349c1560 con 0x5576349c5640 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.104+0000 7fcd5cc00640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x55763491add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd57e00640 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800b240 tx=0x7fcd4800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a003ba03a94640d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd56a00640 1 -- 172.21.15.28:0/2666378275 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48008070 con 0x5576349c5640 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd56a00640 1 -- 172.21.15.28:0/2666378275 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4800d500 con 0x5576349c5640 2024-09-10T10:54:20.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd56a00640 1 -- 172.21.15.28:0/2666378275 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800d8f0 con 0x5576349c5640 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 -- 172.21.15.28:0/2666378275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 msgr2=0x55763491b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800b240 tx=0x7fcd4800b700 comp rx=0 tx=0).stop 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 -- 172.21.15.28:0/2666378275 shutdown_connections 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x5576349376e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x55763491b310 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 --2- 172.21.15.28:0/2666378275 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x55763491add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.105+0000 7fcd60520f40 1 -- 172.21.15.28:0/2666378275 >> 172.21.15.28:0/2666378275 conn(0x55763491bb90 msgr2=0x5576348f1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.106+0000 7fcd60520f40 1 -- 172.21.15.28:0/2666378275 shutdown_connections 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.106+0000 7fcd60520f40 1 -- 172.21.15.28:0/2666378275 wait complete. 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.106+0000 7fcd60520f40 1 Processor -- start 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.106+0000 7fcd60520f40 1 -- start start 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x5576349c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x557634a5b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x557634a5b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5576349334b0 con 0x5576349c5640 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557634935fd0 con 0x5576349c5a20 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd60520f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5576349339b0 con 0x55763491d4c0 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x557634a5b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd57e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x557634a5b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46120/0 (socket says 172.21.15.28:46120) 2024-09-10T10:54:20.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.107+0000 7fcd57e00640 1 -- 172.21.15.28:0/614125907 learned_addr learned my addr 172.21.15.28:0/614125907 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd5cc00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x5576349c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 -- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 msgr2=0x5576349c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55763491d4c0 0x5576349c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 -- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 msgr2=0x557634a5b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd5d600640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x557634a5b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x557634a5b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 -- 172.21.15.28:0/614125907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576349c7930 con 0x5576349c5640 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd5d600640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576349c5a20 0x557634a5b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd57e00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x557634a5b2d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000c00 tx=0x7fcd48000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd55600640 1 -- 172.21.15.28:0/614125907 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48021030 con 0x5576349c5640 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.108+0000 7fcd55600640 1 -- 172.21.15.28:0/614125907 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd48001280 con 0x5576349c5640 2024-09-10T10:54:20.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.109+0000 7fcd55600640 1 -- 172.21.15.28:0/614125907 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801c650 con 0x5576349c5640 2024-09-10T10:54:20.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.109+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576349c7c10 con 0x5576349c5640 2024-09-10T10:54:20.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.109+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576349c81d0 con 0x5576349c5640 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.110+0000 7fcd55600640 1 -- 172.21.15.28:0/614125907 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd48024070 con 0x5576349c5640 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.111+0000 7fcd55600640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3c06a840 0x7fcd3c06cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.111+0000 7fcd55600640 1 -- 172.21.15.28:0/614125907 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7fcd4808baf0 con 0x5576349c5640 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.112+0000 7fcd60520f40 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55763499d7f0 0x557634a67350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.112+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=103b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133) -- 0x557634a67890 con 0x55763499d7f0 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.112+0000 7fcd5cc00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3c06a840 0x7fcd3c06cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.112+0000 7fcd5d600640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55763499d7f0 0x557634a67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.113+0000 7fcd5cc00640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3c06a840 0x7fcd3c06cd00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd50057000 tx=0x7fcd50052000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.113+0000 7fcd5d600640 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55763499d7f0 0x557634a67350 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd5d600640 1 -- 172.21.15.28:0/614125907 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [omap-set-vals] v133'414 uv414 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fcd580029b0 con 0x55763499d7f0 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55763499d7f0 msgr2=0x557634a67350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55763499d7f0 0x557634a67350 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3c06a840 msgr2=0x7fcd3c06cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcd3c06a840 0x7fcd3c06cd00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcd50057000 tx=0x7fcd50052000 comp rx=0 tx=0).stop 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 msgr2=0x557634a5b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 --2- 172.21.15.28:0/614125907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5576349c5640 0x557634a5b2d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fcd48000c00 tx=0x7fcd48000ed0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd5d600640 1 -- 172.21.15.28:0/614125907 reap_dead start 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 shutdown_connections 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.114+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 >> 172.21.15.28:0/614125907 conn(0x55763491bb90 msgr2=0x5576348f1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:20.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.115+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 shutdown_connections 2024-09-10T10:54:20.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.115+0000 7fcd60520f40 1 -- 172.21.15.28:0/614125907 wait complete. 2024-09-10T10:54:20.120 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:54:20.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.279+0000 7f64fde00640 1 Processor -- start 2024-09-10T10:54:20.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.280+0000 7f64fde00640 1 -- start start 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64fde00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f64f8156740 0x7f64f80b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64fde00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f80b30e0 con 0x7f64f80cfea0 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64fde00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f80b33c0 con 0x7f64f80b4af0 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64fde00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f64f806edc0 con 0x7f64f8156740 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f64f8156740 0x7f64f80b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f64f8156740 0x7f64f80b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46126/0 (socket says 172.21.15.28:46126) 2024-09-10T10:54:20.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 -- 172.21.15.28:0/4054583532 learned_addr learned my addr 172.21.15.28:0/4054583532 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:20.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 -- 172.21.15.28:0/4054583532 >> v1:172.21.15.138:6789/0 conn(0x7f64f80b4af0 legacy=0x7f64f80cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 -- 172.21.15.28:0/4054583532 >> v1:172.21.15.28:6789/0 conn(0x7f64f80cfea0 legacy=0x7f64f80ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.281+0000 7f64f6a00640 1 -- 172.21.15.28:0/4054583532 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f80b36a0 con 0x7f64f8156740 2024-09-10T10:54:20.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.282+0000 7f64f6a00640 1 --2- 172.21.15.28:0/4054583532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f8156740 0x7f64f80b45b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3728560795db4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64f6000640 1 -- 172.21.15.28:0/4054583532 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8037070 con 0x7f64f8156740 2024-09-10T10:54:20.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64f6000640 1 -- 172.21.15.28:0/4054583532 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e802ae50 con 0x7f64f8156740 2024-09-10T10:54:20.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64f6000640 1 -- 172.21.15.28:0/4054583532 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8032860 con 0x7f64f8156740 2024-09-10T10:54:20.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 -- 172.21.15.28:0/4054583532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f8156740 msgr2=0x7f64f80b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 --2- 172.21.15.28:0/4054583532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f8156740 0x7f64f80b45b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e802a300 comp rx=0 tx=0).stop 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 -- 172.21.15.28:0/4054583532 shutdown_connections 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 --2- 172.21.15.28:0/4054583532 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f8156740 0x7f64f80b45b0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 -- 172.21.15.28:0/4054583532 >> 172.21.15.28:0/4054583532 conn(0x7f64f806bdf0 msgr2=0x7f64f806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 -- 172.21.15.28:0/4054583532 shutdown_connections 2024-09-10T10:54:20.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.283+0000 7f64fde00640 1 -- 172.21.15.28:0/4054583532 wait complete. 2024-09-10T10:54:20.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.285+0000 7f64fde00640 1 Processor -- start 2024-09-10T10:54:20.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.285+0000 7f64fde00640 1 -- start start 2024-09-10T10:54:20.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64f80b4af0 0x7f64f8147f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64f8156740 0x7f64f80730d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80d0890 con 0x7f64f80cfea0 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80b3c60 con 0x7f64f80b4af0 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64fde00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f64f80d0a10 con 0x7f64f8156740 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64ee000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64ee000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46128/0 (socket says 172.21.15.28:46128) 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.286+0000 7f64ee000640 1 -- 172.21.15.28:0/1614644151 learned_addr learned my addr 172.21.15.28:0/1614644151 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64f6a00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64f80b4af0 0x7f64f8147f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64f7400640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64f8156740 0x7f64f80730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 -- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64f8156740 msgr2=0x7f64f80730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64f8156740 0x7f64f80730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 -- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64f80b4af0 msgr2=0x7f64f8147f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64f80b4af0 0x7f64f8147f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 -- 172.21.15.28:0/1614644151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f8155170 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64ee000640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c430 tx=0x7f64e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0017070 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e00052f0 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0003830 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.287+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f8156fe0 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.288+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f8148450 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.288+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f815b290 con 0x7f64f80cfea0 2024-09-10T10:54:20.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.289+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f64e001a070 con 0x7f64f80cfea0 2024-09-10T10:54:20.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.290+0000 7f64efe00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:20.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.290+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f64e008afa0 con 0x7f64f80cfea0 2024-09-10T10:54:20.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.291+0000 7f64f6a00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:20.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.292+0000 7f64f6a00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 0x7f64d406e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f64e8034000 tx=0x7f64e802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:20.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.295+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f64f815b290 con 0x7f64f80cfea0 2024-09-10T10:54:20.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.519+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f64f80af410 con 0x7f64f80cfea0 2024-09-10T10:54:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.722+0000 7f64efe00640 1 -- 172.21.15.28:0/1614644151 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v284) ==== 162+0+0 (secure 0 0 0) 0x7f64f80af410 con 0x7f64f80cfea0 2024-09-10T10:54:20.724 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 msgr2=0x7f64d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 0x7f64d406e2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f64e8034000 tx=0x7f64e802e000 comp rx=0 tx=0).stop 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 msgr2=0x7f64f81547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c430 tx=0x7f64e000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 shutdown_connections 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64d406be30 0x7f64d406e2f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f64f8156740 0x7f64f80730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64f80cfea0 0x7f64f81547d0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 --2- 172.21.15.28:0/1614644151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64f80b4af0 0x7f64f8147f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:20.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 >> 172.21.15.28:0/1614644151 conn(0x7f64f806bdf0 msgr2=0x7f64f814c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:20.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 shutdown_connections 2024-09-10T10:54:20.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:20.725+0000 7f64fde00640 1 -- 172.21.15.28:0/1614644151 wait complete. 2024-09-10T10:54:21.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:21.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.306+0000 7f5b01000640 1 Processor -- start 2024-09-10T10:54:21.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.306+0000 7f5b01000640 1 -- start start 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0b69d0 0x7f5afc162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5afc0af280 0x7f5afc0afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5afc0affb0 0x7f5afc154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5afc0b6db0 con 0x7f5afc0affb0 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5afc15eb60 con 0x7f5afc0af280 2024-09-10T10:54:21.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5b01000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc15ee40 con 0x7f5afc0b69d0 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5afa800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5afc0affb0 0x7f5afc154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5afa800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5afc0affb0 0x7f5afc154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46140/0 (socket says 172.21.15.28:46140) 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.307+0000 7f5afa800640 1 -- 172.21.15.28:0/1841963139 learned_addr learned my addr 172.21.15.28:0/1841963139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5af9e00640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0b69d0 0x7f5afc162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5af3e00640 1 --2- 172.21.15.28:0/1841963139 >> v2:172.21.15.138:3300/0 conn(0x7f5afc0af280 0x7f5afc0afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5afa800640 1 -- 172.21.15.28:0/1841963139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0b69d0 msgr2=0x7f5afc162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:21.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5afa800640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0b69d0 0x7f5afc162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5afa800640 1 -- 172.21.15.28:0/1841963139 >> v2:172.21.15.138:3300/0 conn(0x7f5afc0af280 msgr2=0x7f5afc0afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5afa800640 1 --2- 172.21.15.28:0/1841963139 >> v2:172.21.15.138:3300/0 conn(0x7f5afc0af280 0x7f5afc0afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.308+0000 7f5afa800640 1 -- 172.21.15.28:0/1841963139 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc15f120 con 0x7f5afc0affb0 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5afa800640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0affb0 0x7f5afc154d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009e50 tx=0x7f5af402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1f850ad8347706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5af9400640 1 -- 172.21.15.28:0/1841963139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4036070 con 0x7f5afc0affb0 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5af9400640 1 -- 172.21.15.28:0/1841963139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af4031600 con 0x7f5afc0affb0 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5af9400640 1 -- 172.21.15.28:0/1841963139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af40319d0 con 0x7f5afc0affb0 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5b01000640 1 -- 172.21.15.28:0/1841963139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0affb0 msgr2=0x7f5afc154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.309+0000 7f5b01000640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0affb0 0x7f5afc154d00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009e50 tx=0x7f5af402a5b0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 -- 172.21.15.28:0/1841963139 shutdown_connections 2024-09-10T10:54:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0affb0 0x7f5afc154d00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 --2- 172.21.15.28:0/1841963139 >> v2:172.21.15.138:3300/0 conn(0x7f5afc0af280 0x7f5afc0afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 --2- 172.21.15.28:0/1841963139 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0b69d0 0x7f5afc162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 -- 172.21.15.28:0/1841963139 >> 172.21.15.28:0/1841963139 conn(0x7f5afc06bc10 msgr2=0x7f5afc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 -- 172.21.15.28:0/1841963139 shutdown_connections 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.310+0000 7f5b01000640 1 -- 172.21.15.28:0/1841963139 wait complete. 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.311+0000 7f5b01000640 1 Processor -- start 2024-09-10T10:54:21.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.311+0000 7f5b01000640 1 -- start start 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 0x7f5afc153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 0x7f5afc15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc155240 con 0x7f5afc0af280 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc0af6e0 con 0x7f5afc0b69d0 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5b01000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5afc1553c0 con 0x7f5afc0affb0 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.312+0000 7f5af9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46148/0 (socket says 172.21.15.28:46148) 2024-09-10T10:54:21.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 -- 172.21.15.28:0/397654539 learned_addr learned my addr 172.21.15.28:0/397654539 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af3e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 0x7f5afc153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 -- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 msgr2=0x7f5afc153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5afa800640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 0x7f5afc15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 0x7f5afc153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 -- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 msgr2=0x7f5afc15d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:21.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 0x7f5afc15d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 -- 172.21.15.28:0/397654539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc160180 con 0x7f5afc0af280 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af3e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 0x7f5afc153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5afa800640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 0x7f5afc15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af9e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00d5f0 tx=0x7f5aec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.313+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0125f0 con 0x7f5afc0af280 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.314+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aec019070 con 0x7f5afc0af280 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.314+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0129d0 con 0x7f5afc0af280 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.314+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc160460 con 0x7f5afc0af280 2024-09-10T10:54:21.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.314+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc160a20 con 0x7f5afc0af280 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.315+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc155820 con 0x7f5afc0af280 2024-09-10T10:54:21.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.316+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5aec012b70 con 0x7f5afc0af280 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.317+0000 7f5af2000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 0x7f5ad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.317+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f5aec08c090 con 0x7f5afc0af280 2024-09-10T10:54:21.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.318+0000 7f5af3e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 0x7f5ad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:21.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.318+0000 7f5af3e00640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 0x7f5ad806e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5afc1618b0 tx=0x7f5ae4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:21.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.322+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5aec059c20 con 0x7f5afc0af280 2024-09-10T10:54:21.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.542+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5afc06ee30 con 0x7f5afc0af280 2024-09-10T10:54:21.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.543+0000 7f5af2000640 1 -- 172.21.15.28:0/397654539 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 285 v285) ==== 95+0+5227 (secure 0 0 0) 0x7f5aec059520 con 0x7f5afc0af280 2024-09-10T10:54:21.544 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:21.544 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":285,"btime":"2024-09-10T10:54:20:725200+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":285,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:20.725184+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9747":{"gid":9747,"name":"a","rank":0,"incarnation":285,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/1056314461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1056314461},{"type":"v1","addr":"172.21.15.28:6835","nonce":1056314461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:21.544 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 285 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 msgr2=0x7f5ad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 0x7f5ad806e2f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5afc1618b0 tx=0x7f5ae4008040 comp rx=0 tx=0).stop 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 msgr2=0x7f5afc15fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00d5f0 tx=0x7f5aec00dab0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 shutdown_connections 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5ad806be30 0x7f5ad806e2f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5afc0b69d0 0x7f5afc15d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5afc0affb0 0x7f5afc153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 --2- 172.21.15.28:0/397654539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5afc0af280 0x7f5afc15fa80 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:21.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 >> 172.21.15.28:0/397654539 conn(0x7f5afc06bc10 msgr2=0x7f5afc0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:21.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 shutdown_connections 2024-09-10T10:54:21.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:21.545+0000 7f5b01000640 1 -- 172.21.15.28:0/397654539 wait complete. 2024-09-10T10:54:21.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 285, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:54:20.725184+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 133, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9747': {'gid': 9747, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/1056314461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 1056314461}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 1056314461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:54:21.563 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9747:up:replay 2024-09-10T10:54:21.563 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:54:24.724 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:54:24.723+0000 7fe150800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:26.751 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:26.749+0000 7fcc62600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T10:54:26.967 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.131+0000 7f019a600640 1 Processor -- start 2024-09-10T10:54:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.132+0000 7f019a600640 1 -- start start 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f019a600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0194072d30 0x7f019406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f019a600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f019406fae0 con 0x7f019406f700 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f019a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01940cdbe0 con 0x7f019415b300 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f019a600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f01940728a0 con 0x7f0194072d30 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f0193e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f019406f700 0x7f019415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49488/0 (socket says 172.21.15.28:49488) 2024-09-10T10:54:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.133+0000 7f0193e00640 1 -- 172.21.15.28:0/128002668 learned_addr learned my addr 172.21.15.28:0/128002668 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> v2:172.21.15.138:3300/0 conn(0x7f0194072d30 0x7f019406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625248017 0 0) 0x7f019406fae0 con 0x7f019406f700 2024-09-10T10:54:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0178005730 con 0x7f019406f700 2024-09-10T10:54:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256608057 0 0) 0x7f01940cdbe0 con 0x7f019415b300 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f019406fae0 con 0x7f019415b300 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1659507092 0 0) 0x7f0178005730 con 0x7f019406f700 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 >> v2:172.21.15.138:3300/0 conn(0x7f0194072d30 msgr2=0x7f019406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 --2- 172.21.15.28:0/128002668 >> v2:172.21.15.138:3300/0 conn(0x7f0194072d30 0x7f019406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 >> v1:172.21.15.138:6789/0 conn(0x7f019415b300 legacy=0x7f01940cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01940cd900 con 0x7f019406f700 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f018c001970 con 0x7f019406f700 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> v2:172.21.15.138:3300/0 conn(0x7f0194072d30 0x7f019406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 >> v1:172.21.15.28:6789/0 conn(0x7f019406f700 legacy=0x7f019415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0178005da0 0x7f01780061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 0x7f0178006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178007420 0x7f017800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f01940728a0 con 0x7f0178006a10 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f017800e350 con 0x7f0178005da0 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f017800e550 con 0x7f0178007420 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.134+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 0x7f0178006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f0193400640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0178005da0 0x7f01780061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f0193e00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178007420 0x7f017800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f018be00640 1 -- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178007420 msgr2=0x7f017800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178007420 0x7f017800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f018be00640 1 -- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0178005da0 msgr2=0x7f01780061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0178005da0 0x7f01780061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f018be00640 1 -- 172.21.15.28:0/128002668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0178005730 con 0x7f0178006a10 2024-09-10T10:54:27.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f0193e00640 1 -- 172.21.15.28:0/128002668 reap_dead start 2024-09-10T10:54:27.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.135+0000 7f0193400640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0178005da0 0x7f01780061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.136+0000 7f018be00640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 0x7f0178006e90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f017c009c90 tx=0x7f0178007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c49986e567e1c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.136+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c036070 con 0x7f0178006a10 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.136+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c031400 con 0x7f0178006a10 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.136+0000 7f0192a00640 1 -- 172.21.15.28:0/128002668 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c009860 con 0x7f0178006a10 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 -- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 msgr2=0x7f0178006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 0x7f0178006e90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f017c009c90 tx=0x7f0178007a40 comp rx=0 tx=0).stop 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 -- 172.21.15.28:0/128002668 shutdown_connections 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 --2- 172.21.15.28:0/128002668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0178006a10 0x7f0178006e90 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 -- 172.21.15.28:0/128002668 >> 172.21.15.28:0/128002668 conn(0x7f0194067d70 msgr2=0x7f019406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 -- 172.21.15.28:0/128002668 shutdown_connections 2024-09-10T10:54:27.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.137+0000 7f019a600640 1 -- 172.21.15.28:0/128002668 wait complete. 2024-09-10T10:54:27.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.139+0000 7f019a600640 1 Processor -- start 2024-09-10T10:54:27.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.139+0000 7f019a600640 1 -- start start 2024-09-10T10:54:27.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178006a10 0x7f01941e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01941e6040 0x7f0194145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f01940cffe0 con 0x7f0194157ba0 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0194072530 con 0x7f01941e6040 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f019a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f01940d0160 con 0x7f0178006a10 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34590/0 (socket says 172.21.15.28:34590) 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 -- 172.21.15.28:0/3473531959 learned_addr learned my addr 172.21.15.28:0/3473531959 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f018be00640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01941e6040 0x7f0194145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193400640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178006a10 0x7f01941e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 -- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178006a10 msgr2=0x7f01941e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178006a10 0x7f01941e49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 -- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01941e6040 msgr2=0x7f0194145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01941e6040 0x7f0194145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.140+0000 7f0193e00640 1 -- 172.21.15.28:0/3473531959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0194146380 con 0x7f0194157ba0 2024-09-10T10:54:27.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f0193e00640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f01940ce8c0 tx=0x7f018c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c00ea80 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f018c018070 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c00ee80 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01941e4fa0 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.141+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01941e5560 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.142+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019406bda0 con 0x7f0194157ba0 2024-09-10T10:54:27.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.143+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f018c01d460 con 0x7f0194157ba0 2024-09-10T10:54:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.144+0000 7f018b400640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 0x7f017006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.145+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f018c08bda0 con 0x7f0194157ba0 2024-09-10T10:54:27.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.145+0000 7f0193400640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 0x7f017006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.146+0000 7f0193400640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 0x7f017006e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f01941e6730 tx=0x7f0184009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.149+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f018c0592e0 con 0x7f0194157ba0 2024-09-10T10:54:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.373+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01940aee30 con 0x7f0194157ba0 2024-09-10T10:54:27.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.373+0000 7f018b400640 1 -- 172.21.15.28:0/3473531959 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5234 (secure 0 0 0) 0x7f018c05d5e0 con 0x7f0194157ba0 2024-09-10T10:54:27.375 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:27.376 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":289,"btime":"2024-09-10T10:54:26:750890+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:26.740514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9747":{"gid":9747,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":5,"addr":"172.21.15.28:6835/1056314461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1056314461},{"type":"v1","addr":"172.21.15.28:6835","nonce":1056314461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9747,"qdb_cluster":[9747]},"id":6}]} 2024-09-10T10:54:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 289 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 msgr2=0x7f017006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 0x7f017006e2f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f01941e6730 tx=0x7f0184009040 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 msgr2=0x7f01941ea2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f01940ce8c0 tx=0x7f018c007bc0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 shutdown_connections 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f017006be30 0x7f017006e2f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0194157ba0 0x7f01941ea2d0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01941e6040 0x7f0194145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 --2- 172.21.15.28:0/3473531959 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0178006a10 0x7f01941e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.375+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 >> 172.21.15.28:0/3473531959 conn(0x7f0194067d70 msgr2=0x7f01940af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.376+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 shutdown_connections 2024-09-10T10:54:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.376+0000 7f019a600640 1 -- 172.21.15.28:0/3473531959 wait complete. 2024-09-10T10:54:27.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 289, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:54:26.740514+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 133, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9747}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9747': {'gid': 9747, 'name': 'a', 'rank': 0, 'incarnation': 285, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.28:6835/1056314461', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 1056314461}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 1056314461}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9747, 'qdb_cluster': [9747]} 2024-09-10T10:54:27.394 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:54:27.394 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:54:27.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.549+0000 7f8f45400640 1 Processor -- start 2024-09-10T10:54:27.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.549+0000 7f8f45400640 1 -- start start 2024-09-10T10:54:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f45400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f45400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f400cf000 con 0x7f8f400d0dd0 2024-09-10T10:54:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f45400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f400d29a0 con 0x7f8f4015e660 2024-09-10T10:54:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f45400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f4006d3e0 con 0x7f8f400d1f90 2024-09-10T10:54:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f3fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8f400d0dd0 0x7f8f400d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49490/0 (socket says 172.21.15.28:49490) 2024-09-10T10:54:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.550+0000 7f8f3fe00640 1 -- 172.21.15.28:0/2270012269 learned_addr learned my addr 172.21.15.28:0/2270012269 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f44a00640 1 --2- 172.21.15.28:0/2270012269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511941320 0 0) 0x7f8f400cf000 con 0x7f8f400d0dd0 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f24003520 con 0x7f8f400d0dd0 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263968833 0 0) 0x7f8f400d29a0 con 0x7f8f4015e660 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f400cf000 con 0x7f8f4015e660 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f44a00640 1 -- 172.21.15.28:0/2270012269 >> v1:172.21.15.138:6789/0 conn(0x7f8f4015e660 legacy=0x7f8f4006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f44a00640 1 -- 172.21.15.28:0/2270012269 >> v1:172.21.15.28:6789/0 conn(0x7f8f400d0dd0 legacy=0x7f8f400d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.551+0000 7f8f44a00640 1 -- 172.21.15.28:0/2270012269 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f400d2c60 con 0x7f8f400d1f90 2024-09-10T10:54:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.552+0000 7f8f44a00640 1 --2- 172.21.15.28:0/2270012269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400cf7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009dc0 tx=0x7f8f3802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64eb6c2fa46b43fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007740 con 0x7f8f400d1f90 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f380078e0 con 0x7f8f400d1f90 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f3f400640 1 -- 172.21.15.28:0/2270012269 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38007cb0 con 0x7f8f400d1f90 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f45400640 1 -- 172.21.15.28:0/2270012269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 msgr2=0x7f8f400cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f45400640 1 --2- 172.21.15.28:0/2270012269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400cf7c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009dc0 tx=0x7f8f3802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f45400640 1 -- 172.21.15.28:0/2270012269 shutdown_connections 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f45400640 1 --2- 172.21.15.28:0/2270012269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400cf7c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.553+0000 7f8f45400640 1 -- 172.21.15.28:0/2270012269 >> 172.21.15.28:0/2270012269 conn(0x7f8f4008bfc0 msgr2=0x7f8f400d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.554+0000 7f8f45400640 1 -- 172.21.15.28:0/2270012269 shutdown_connections 2024-09-10T10:54:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.554+0000 7f8f45400640 1 -- 172.21.15.28:0/2270012269 wait complete. 2024-09-10T10:54:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.555+0000 7f8f45400640 1 Processor -- start 2024-09-10T10:54:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.555+0000 7f8f45400640 1 -- start start 2024-09-10T10:54:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4015e660 0x7f8f400b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400cf330 con 0x7f8f400d0dd0 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f400d3220 con 0x7f8f4015e660 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f45400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f400d33a0 con 0x7f8f400d1f90 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34594/0 (socket says 172.21.15.28:34594) 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.556+0000 7f8f3fe00640 1 -- 172.21.15.28:0/796268704 learned_addr learned my addr 172.21.15.28:0/796268704 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f44a00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4015e660 0x7f8f400b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f37e00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 -- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 msgr2=0x7f8f400b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 -- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4015e660 msgr2=0x7f8f400b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4015e660 0x7f8f400b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 -- 172.21.15.28:0/796268704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f400b5c20 con 0x7f8f400d0dd0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c530 tx=0x7f8f3000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3cc00640 1 -- 172.21.15.28:0/796268704 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30017070 con 0x7f8f400d0dd0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3cc00640 1 -- 172.21.15.28:0/796268704 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30005ce0 con 0x7f8f400d0dd0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f400b4640 con 0x7f8f400d0dd0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.557+0000 7f8f3cc00640 1 -- 172.21.15.28:0/796268704 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30007270 con 0x7f8f400d0dd0 2024-09-10T10:54:27.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.558+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f400b4c00 con 0x7f8f400d0dd0 2024-09-10T10:54:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.560+0000 7f8f3cc00640 1 -- 172.21.15.28:0/796268704 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f3001a070 con 0x7f8f400d0dd0 2024-09-10T10:54:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.560+0000 7f8f3cc00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 0x7f8f1c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.561+0000 7f8f3cc00640 1 -- 172.21.15.28:0/796268704 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f8f3008ae90 con 0x7f8f400d0dd0 2024-09-10T10:54:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.561+0000 7f8f37e00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 0x7f8f1c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.561+0000 7f8f4985f740 1 Processor -- start 2024-09-10T10:54:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.561+0000 7f8f4985f740 1 -- start start 2024-09-10T10:54:27.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f37e00640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 0x7f8f1c06e2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f401f7a10 tx=0x7f8f28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874b660 0x55b4e874e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874f630 0x55b4e8748720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4e86ac3b0 con 0x55b4e874dea0 2024-09-10T10:54:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4e86b1410 con 0x55b4e874b660 2024-09-10T10:54:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b4e8659620 con 0x55b4e874f630 2024-09-10T10:54:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34608/0 (socket says 172.21.15.28:34608) 2024-09-10T10:54:27.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.562+0000 7f8f3fe00640 1 -- 172.21.15.28:0/3606077181 learned_addr learned my addr 172.21.15.28:0/3606077181 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 -- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874f630 msgr2=0x55b4e8748720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874f630 0x55b4e8748720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 -- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874b660 msgr2=0x55b4e874e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874b660 0x55b4e874e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 -- 172.21.15.28:0/3606077181 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e83c42d0 con 0x55b4e874dea0 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f8f300548c0 tx=0x7f8f3005c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84d090f31ea5738 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f2fe00640 1 -- 172.21.15.28:0/3606077181 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30056760 con 0x55b4e874dea0 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f2fe00640 1 -- 172.21.15.28:0/3606077181 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30054ad0 con 0x55b4e874dea0 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.563+0000 7f8f2fe00640 1 -- 172.21.15.28:0/3606077181 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30060d00 con 0x55b4e874dea0 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 -- 172.21.15.28:0/3606077181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 msgr2=0x55b4e8793640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f8f300548c0 tx=0x7f8f3005c760 comp rx=0 tx=0).stop 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 -- 172.21.15.28:0/3606077181 shutdown_connections 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874f630 0x55b4e8748720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874b660 0x55b4e874e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 --2- 172.21.15.28:0/3606077181 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e8793640 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 -- 172.21.15.28:0/3606077181 >> 172.21.15.28:0/3606077181 conn(0x55b4e874a8f0 msgr2=0x55b4e8792b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 -- 172.21.15.28:0/3606077181 shutdown_connections 2024-09-10T10:54:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.564+0000 7f8f4985f740 1 -- 172.21.15.28:0/3606077181 wait complete. 2024-09-10T10:54:27.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.565+0000 7f8f4985f740 1 -- start start 2024-09-10T10:54:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.565+0000 7f8f4985f740 1 Processor -- start 2024-09-10T10:54:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e8771590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4e866c780 con 0x55b4e874dea0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4e86ad3d0 con 0x55b4e874f630 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b4e8749d90 con 0x55b4e87643a0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f4985f740 10 client.0 _ll_register_callbacks cb 0x55b4e86aa5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34622/0 (socket says 172.21.15.28:34622) 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f3fe00640 1 -- 172.21.15.28:0/2501431003 learned_addr learned my addr 172.21.15.28:0/2501431003 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f44a00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e8771590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.566+0000 7f8f37e00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 -- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 msgr2=0x55b4e8771590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e8771590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 -- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 msgr2=0x55b4e87639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 -- 172.21.15.28:0/2501431003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e848e6b0 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f44a00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e8771590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f37e00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000cef0 tx=0x7f8f30000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4fbb20a7d1cd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30085060 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f8f30070370 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f30054ad0 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=289) ==== 24+0+0 (secure 0 0 0) 0x7f8f300578b0 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f4985f740 10 client.9807 fetch_fsmap learned FSMap version 289 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.567+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=289,monmap=2+}) -- 0x55b4e878eef0 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30054ad0 con 0x55b4e874dea0 2024-09-10T10:54:27.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 289) ==== 3502+0+0 (secure 0 0 0) 0x7f8f30057a50 con 0x55b4e874dea0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 10 client.9807 fetch_fsmap finished waiting for FSMap version 289 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 10 client.9807 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 20 client.9807 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e87a68b0 0x55b4e878df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 4 client.9807 mds_command: new command op to 9747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 --> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e86beb40 con 0x55b4e87a68b0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.568+0000 7f8f37e00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e87a68b0 0x55b4e878df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.569+0000 7f8f37e00640 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e87a68b0 0x55b4e878df40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f96c699c9379f2a server_cookie=41982388d02a68d6 in_seq=0 out_seq=0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.569+0000 7f8f2cc00640 10 client.9807 ms_handle_connect on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.569+0000 7f8f2cc00640 1 -- 172.21.15.28:0/2501431003 <== mds.0 v2:172.21.15.28:6833/1056314461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4e86beb40 con 0x55b4e87a68b0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.569+0000 7f8f2cc00640 10 client.9807 handle_command_reply: tid=0 2024-09-10T10:54:27.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 client.9807 shutdown 2024-09-10T10:54:27.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 msgr2=0x55b4e87783a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000cef0 tx=0x7f8f30000ed0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 shutdown_connections 2024-09-10T10:54:27.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e87a68b0 0x55b4e878df40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e8771590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 --2- 172.21.15.28:0/2501431003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87783a0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f2cc00640 0 client.9807 ms_handle_reset on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 >> 172.21.15.28:0/2501431003 conn(0x55b4e874b660 msgr2=0x55b4e8792100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 shutdown_connections 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 1 -- 172.21.15.28:0/2501431003 wait complete. 2024-09-10T10:54:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.570+0000 7f8f4985f740 20 client.9807 trim_cache size 0 max 16384 2024-09-10T10:54:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.589+0000 7f8f4985f740 1 Processor -- start 2024-09-10T10:54:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.589+0000 7f8f4985f740 1 -- start start 2024-09-10T10:54:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e878e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40201e80 con 0x55b4e874dea0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40201180 con 0x55b4e874f630 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f402017d0 con 0x55b4e87643a0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34626/0 (socket says 172.21.15.28:34626) 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1366616285 learned_addr learned my addr 172.21.15.28:0/1366616285 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f44a00640 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e878e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f37e00640 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 msgr2=0x55b4e878e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e878e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 msgr2=0x55b4e87914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.590+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1366616285 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e874a690 con 0x55b4e874dea0 2024-09-10T10:54:27.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f8f30061150 tx=0x7f8f3005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35ed79201bd5f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f2f400640 1 -- 172.21.15.28:0/1366616285 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30085040 con 0x55b4e874dea0 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f2f400640 1 -- 172.21.15.28:0/1366616285 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30057a50 con 0x55b4e874dea0 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f2f400640 1 -- 172.21.15.28:0/1366616285 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30070170 con 0x55b4e874dea0 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 -- 172.21.15.28:0/1366616285 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 msgr2=0x55b4e87b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f8f30061150 tx=0x7f8f3005b040 comp rx=0 tx=0).stop 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 -- 172.21.15.28:0/1366616285 shutdown_connections 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e87643a0 0x55b4e878e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e87914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1366616285 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e874dea0 0x55b4e87b7110 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 -- 172.21.15.28:0/1366616285 >> 172.21.15.28:0/1366616285 conn(0x55b4e874b660 msgr2=0x55b4e8770dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.591+0000 7f8f4985f740 1 -- 172.21.15.28:0/1366616285 shutdown_connections 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.592+0000 7f8f4985f740 1 -- 172.21.15.28:0/1366616285 wait complete. 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.592+0000 7f8f4985f740 1 -- start start 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.592+0000 7f8f4985f740 1 Processor -- start 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 0x55b4e8810340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e8796680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 0x55b4e8796bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55b4e866c780 con 0x55b4e87643a0 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b4e8749d90 con 0x55b4e874f630 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55b4e86ad3d0 con 0x55b4e874dea0 2024-09-10T10:54:27.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f4985f740 10 client.0 _ll_register_callbacks cb 0x7f8f3006f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:27.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 0x55b4e8796bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.593+0000 7f8f3fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 0x55b4e8810340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 0x55b4e8796bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34636/0 (socket says 172.21.15.28:34636) 2024-09-10T10:54:27.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f44a00640 1 -- 172.21.15.28:0/1692971179 learned_addr learned my addr 172.21.15.28:0/1692971179 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f37e00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e8796680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 msgr2=0x55b4e8796680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e8796680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 msgr2=0x55b4e8796bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 0x55b4e8796bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 -- 172.21.15.28:0/1692971179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e8810880 con 0x55b4e874dea0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f3fe00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 0x55b4e8810340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f3005e000 tx=0x7f8f30001150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22003981717e4876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3301/0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.594+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f30085040 con 0x55b4e874dea0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f30057a50 con 0x55b4e874dea0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mon.2 v2:172.21.15.138:3301/0 2 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f8f300541b0 con 0x55b4e874dea0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_get_version_reply(handle=1 version=289) ==== 24+0+0 (secure 0 0 0) 0x7f8f300543d0 con 0x55b4e874dea0 2024-09-10T10:54:27.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 10 client.8009 fetch_fsmap learned FSMap version 289 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,fsmap=289,monmap=2+}) -- 0x55b4e8810880 con 0x55b4e874dea0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mon.2 v2:172.21.15.138:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f30057a50 con 0x55b4e874dea0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mon.2 v2:172.21.15.138:3301/0 5 ==== fsmap(e 289) ==== 3502+0+0 (secure 0 0 0) 0x7f8f30077120 con 0x55b4e874dea0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 10 client.8009 fetch_fsmap finished waiting for FSMap version 289 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 10 client.8009 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 20 client.8009 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e88144c0 0x55b4e88148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 4 client.8009 mds_command: new command op to 9747 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 --> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] -- command(tid 0: {"prefix": "status"}) -- 0x55b4e86b8e90 con 0x55b4e88144c0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.595+0000 7f8f37e00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e88144c0 0x55b4e88148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f37e00640 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e88144c0 0x55b4e88148a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7704e36b67cb81e7 server_cookie=9b68e9201ba16fda in_seq=0 out_seq=0 2024-09-10T10:54:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f2cc00640 10 client.8009 ms_handle_connect on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f2cc00640 1 -- 172.21.15.28:0/1692971179 <== mds.0 v2:172.21.15.28:6833/1056314461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55b4e86b8e90 con 0x55b4e88144c0 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f2cc00640 10 client.8009 handle_command_reply: tid=0 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 client.8009 shutdown 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 msgr2=0x55b4e8810340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 0x55b4e8810340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8f3005e000 tx=0x7f8f30001150 comp rx=0 tx=0).stop 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 shutdown_connections 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55b4e88144c0 0x55b4e88148a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55b4e87643a0 0x55b4e8796bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b4e874f630 0x55b4e8796680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.596+0000 7f8f4985f740 1 --2- 172.21.15.28:0/1692971179 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55b4e874dea0 0x55b4e8810340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f2cc00640 0 client.8009 ms_handle_reset on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 >> 172.21.15.28:0/1692971179 conn(0x55b4e874b660 msgr2=0x55b4e8770dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 shutdown_connections 2024-09-10T10:54:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f4985f740 1 -- 172.21.15.28:0/1692971179 wait complete. 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f4985f740 20 client.8009 trim_cache size 0 max 16384 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 9747, 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8664786629999996, 2024-09-10T10:54:27.603 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 289, 2024-09-10T10:54:27.604 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 133, 2024-09-10T10:54:27.604 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 133, 2024-09-10T10:54:27.604 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 8.7267939410000004 2024-09-10T10:54:27.604 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 msgr2=0x7f8f1c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 0x7f8f1c06e2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f401f7a10 tx=0x7f8f28009040 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 msgr2=0x7f8f40155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.597+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f8f3000c530 tx=0x7f8f3000c9f0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 shutdown_connections 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f1c06be30 0x7f8f1c06e2f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f4015e660 0x7f8f400b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f400d1f90 0x7f8f400b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 --2- 172.21.15.28:0/796268704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f400d0dd0 0x7f8f40155b90 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 >> 172.21.15.28:0/796268704 conn(0x7f8f4008bfc0 msgr2=0x7f8f4006e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 shutdown_connections 2024-09-10T10:54:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.598+0000 7f8f45400640 1 -- 172.21.15.28:0/796268704 wait complete. 2024-09-10T10:54:27.615 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:54:27.615 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap key 1.00000000:subdir_head', proceeding to ls 2024-09-10T10:54:27.615 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:54:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.769+0000 7f3eff800640 1 Processor -- start 2024-09-10T10:54:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.770+0000 7f3eff800640 1 -- start start 2024-09-10T10:54:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3eff800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3ef8072de0 0x7f3ef815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3eff800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ef8072700 0x7f3ef815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3eff800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef8146140 con 0x7f3ef8072320 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3eff800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef80cd740 con 0x7f3ef8072700 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3eff800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef80cda20 con 0x7f3ef8072de0 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3efd800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ef8072700 0x7f3ef815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3efd800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ef8072700 0x7f3ef815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34652/0 (socket says 172.21.15.28:34652) 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3efd800640 1 -- 172.21.15.28:0/696362132 learned_addr learned my addr 172.21.15.28:0/696362132 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.771+0000 7f3ef7e00640 1 --2- 172.21.15.28:0/696362132 >> v2:172.21.15.138:3300/0 conn(0x7f3ef8072de0 0x7f3ef815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.772+0000 7f3efd800640 1 -- 172.21.15.28:0/696362132 >> v2:172.21.15.138:3300/0 conn(0x7f3ef8072de0 msgr2=0x7f3ef815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.772+0000 7f3efd800640 1 --2- 172.21.15.28:0/696362132 >> v2:172.21.15.138:3300/0 conn(0x7f3ef8072de0 0x7f3ef815d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.772+0000 7f3efd800640 1 -- 172.21.15.28:0/696362132 >> v1:172.21.15.138:6789/0 conn(0x7f3ef8072320 legacy=0x7f3ef80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.772+0000 7f3efd800640 1 -- 172.21.15.28:0/696362132 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef80cdd00 con 0x7f3ef8072700 2024-09-10T10:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3efd800640 1 --2- 172.21.15.28:0/696362132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072700 0x7f3ef815d720 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806f8c0 tx=0x7f3ef002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e41172f2dea337 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3ef7400640 1 -- 172.21.15.28:0/696362132 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0036070 con 0x7f3ef8072700 2024-09-10T10:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3ef7400640 1 -- 172.21.15.28:0/696362132 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0031600 con 0x7f3ef8072700 2024-09-10T10:54:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3ef7400640 1 -- 172.21.15.28:0/696362132 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00319d0 con 0x7f3ef8072700 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3eff800640 1 -- 172.21.15.28:0/696362132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072700 msgr2=0x7f3ef815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.773+0000 7f3eff800640 1 --2- 172.21.15.28:0/696362132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072700 0x7f3ef815d720 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806f8c0 tx=0x7f3ef002a6a0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 -- 172.21.15.28:0/696362132 shutdown_connections 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 --2- 172.21.15.28:0/696362132 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072700 0x7f3ef815d720 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 --2- 172.21.15.28:0/696362132 >> v2:172.21.15.138:3300/0 conn(0x7f3ef8072de0 0x7f3ef815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 -- 172.21.15.28:0/696362132 >> 172.21.15.28:0/696362132 conn(0x7f3ef8067d70 msgr2=0x7f3ef806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 -- 172.21.15.28:0/696362132 shutdown_connections 2024-09-10T10:54:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.774+0000 7f3eff800640 1 -- 172.21.15.28:0/696362132 wait complete. 2024-09-10T10:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.775+0000 7f3eff800640 1 Processor -- start 2024-09-10T10:54:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.775+0000 7f3eff800640 1 -- start start 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ef8072700 0x7f3ef80adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef8072de0 0x7f3ef8141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef815dde0 con 0x7f3ef8072320 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef815dc60 con 0x7f3ef8072de0 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3eff800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef815df60 con 0x7f3ef8072700 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.776+0000 7f3efce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34654/0 (socket says 172.21.15.28:34654) 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 -- 172.21.15.28:0/1666007445 learned_addr learned my addr 172.21.15.28:0/1666007445 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3ef7e00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ef8072700 0x7f3ef80adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efd800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef8072de0 0x7f3ef8141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 -- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ef8072700 msgr2=0x7f3ef80adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ef8072700 0x7f3ef80adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 -- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef8072de0 msgr2=0x7f3ef8141880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef8072de0 0x7f3ef8141880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 -- 172.21.15.28:0/1666007445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef806eab0 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3efce00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c400 tx=0x7f3eec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.777+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0052d0 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.778+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec005490 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.778+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec007260 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.778+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef806e7d0 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.778+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef80ad310 con 0x7f3ef8072320 2024-09-10T10:54:27.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.779+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef815e320 con 0x7f3ef8072320 2024-09-10T10:54:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.780+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3eec007400 con 0x7f3ef8072320 2024-09-10T10:54:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.781+0000 7f3ef4c00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 0x7f3ed806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:27.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.781+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f3eec08b080 con 0x7f3ef8072320 2024-09-10T10:54:27.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.781+0000 7f3ef7e00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 0x7f3ed806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.782+0000 7f3ef7e00640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 0x7f3ed806e2f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80cbf60 tx=0x7f3ee4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:27.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:27.785+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3eec058650 con 0x7f3ef8072320 2024-09-10T10:54:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.004+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3ef806bd20 con 0x7f3ef8072320 2024-09-10T10:54:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.005+0000 7f3ef4c00640 1 -- 172.21.15.28:0/1666007445 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v289) ==== 53+0+73 (secure 0 0 0) 0x7f3eec05c950 con 0x7f3ef8072320 2024-09-10T10:54:28.007 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:54:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 msgr2=0x7f3ed806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 0x7f3ed806e2f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80cbf60 tx=0x7f3ee4008040 comp rx=0 tx=0).stop 2024-09-10T10:54:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 msgr2=0x7f3ef815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c400 tx=0x7f3eec00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 shutdown_connections 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3ed806be30 0x7f3ed806e2f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ef8072de0 0x7f3ef8141880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3ef8072700 0x7f3ef80adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 --2- 172.21.15.28:0/1666007445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ef8072320 0x7f3ef815bf30 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 >> 172.21.15.28:0/1666007445 conn(0x7f3ef8067d70 msgr2=0x7f3ef80b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 shutdown_connections 2024-09-10T10:54:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28.007+0000 7f3eff800640 1 -- 172.21.15.28:0/1666007445 wait complete. 2024-09-10T10:54:28.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:54:28.024 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:54:28.024 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:54:28.024 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:54:28.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:28.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:54:28.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:54:28.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:54:28.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:28.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.025 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:54:28.061 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42037sec preferred_lft 42037sec 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:28.062 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:28.063 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:54:28.064 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:54:28.064 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:28.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.064 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:54:28.154 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:54:28.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.158 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:54:28.195 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:54:28.195 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:54:28.195 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:54:28.195 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:54:28.196 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:54:28.196 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:54:28.196 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:54:28.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:54:28.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:54:28.196 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:54:28.197 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:54:28.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:54:28.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:28.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.393 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:54:28.428 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:54:28.428 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:54:28.429 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:28.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.429 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:54:28.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:28.526 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:28.561 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:28.561 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:28.561 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:28.636 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:54:28.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.636 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:28.764 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:28.764 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:28.765 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:28.780 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:28.780 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:28.780 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:28.780 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:28.780 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:54:28.802 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:54:28.803 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:54:28.803 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:54:28.803 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:54:28.803 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:54:28.804 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:54:28.806 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:54:28.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:28.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:28.863 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:54:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:54:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:28.951 INFO:teuthology.orchestra.run.smithi028.stdout:client9822 2024-09-10T10:54:28.952 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:28.952 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> import json 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> import sys 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> try: 2024-09-10T10:54:28.988 DEBUG:teuthology.orchestra.run.smithi028:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/./subdir'"'"') 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> except OSError as e: 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> sys.exit(e.errno) 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> print(json.dumps( 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> indent=2)) 2024-09-10T10:54:28.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:54:29.032 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:54:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:29 stdin-killer INFO: command exited with status 5: exiting normally with same code! 2024-09-10T10:54:35.133 DEBUG:teuthology.orchestra.run:got remote process result: 5 2024-09-10T10:54:35.134 INFO:tasks.cephfs.test_damage:Result: EIO on client 2024-09-10T10:54:35.134 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:35.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.302+0000 7f4b57000640 1 Processor -- start 2024-09-10T10:54:35.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.302+0000 7f4b57000640 1 -- start start 2024-09-10T10:54:35.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b57000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4b500b47b0 0x7f4b5006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b57000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 0x7f4b5009ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b57000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b5006ddd0 con 0x7f4b501629e0 2024-09-10T10:54:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b57000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5009b3a0 con 0x7f4b500b47b0 2024-09-10T10:54:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b57000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5006e0b0 con 0x7f4b5009aa30 2024-09-10T10:54:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b55200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4b500b47b0 0x7f4b5006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b55c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4b501629e0 0x7f4b50070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44168/0 (socket says 172.21.15.28:44168) 2024-09-10T10:54:35.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.303+0000 7f4b55c00640 1 -- 172.21.15.28:0/3805255435 learned_addr learned my addr 172.21.15.28:0/3805255435 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b56600640 1 --2- 172.21.15.28:0/3805255435 >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 0x7f4b5009ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b55200640 1 -- 172.21.15.28:0/3805255435 >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 msgr2=0x7f4b5009ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b55200640 1 --2- 172.21.15.28:0/3805255435 >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 0x7f4b5009ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b55200640 1 -- 172.21.15.28:0/3805255435 >> v1:172.21.15.28:6789/0 conn(0x7f4b501629e0 legacy=0x7f4b50070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b55200640 1 -- 172.21.15.28:0/3805255435 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500d1b60 con 0x7f4b500b47b0 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b4fe00640 1 -- 172.21.15.28:0/3805255435 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2533226008 0 0) 0x7f4b5006ddd0 con 0x7f4b501629e0 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.304+0000 7f4b56600640 1 --2- 172.21.15.28:0/3805255435 >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 0x7f4b5009ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.305+0000 7f4b55200640 1 --2- 172.21.15.28:0/3805255435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b500b47b0 0x7f4b5006e9d0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f4b500c9200 tx=0x7f4b3c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828bff416052937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.305+0000 7f4b4fe00640 1 -- 172.21.15.28:0/3805255435 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c037070 con 0x7f4b500b47b0 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.305+0000 7f4b4fe00640 1 -- 172.21.15.28:0/3805255435 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b3c032440 con 0x7f4b500b47b0 2024-09-10T10:54:35.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.305+0000 7f4b4fe00640 1 -- 172.21.15.28:0/3805255435 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c02ae60 con 0x7f4b500b47b0 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 -- 172.21.15.28:0/3805255435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b500b47b0 msgr2=0x7f4b5006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 --2- 172.21.15.28:0/3805255435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b500b47b0 0x7f4b5006e9d0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f4b500c9200 tx=0x7f4b3c02a110 comp rx=0 tx=0).stop 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 -- 172.21.15.28:0/3805255435 shutdown_connections 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 --2- 172.21.15.28:0/3805255435 >> v2:172.21.15.138:3300/0 conn(0x7f4b5009aa30 0x7f4b5009ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 --2- 172.21.15.28:0/3805255435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b500b47b0 0x7f4b5006e9d0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 -- 172.21.15.28:0/3805255435 >> 172.21.15.28:0/3805255435 conn(0x7f4b500b3c90 msgr2=0x7f4b500b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 -- 172.21.15.28:0/3805255435 shutdown_connections 2024-09-10T10:54:35.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.306+0000 7f4b57000640 1 -- 172.21.15.28:0/3805255435 wait complete. 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.308+0000 7f4b57000640 1 Processor -- start 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.308+0000 7f4b57000640 1 -- start start 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 0x7f4b501f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 0x7f4b500c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50162f40 con 0x7f4b501629e0 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50162dc0 con 0x7f4b5009aa30 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b57000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b500d0170 con 0x7f4b500b47b0 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48122/0 (socket says 172.21.15.28:48122) 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 -- 172.21.15.28:0/3668630211 learned_addr learned my addr 172.21.15.28:0/3668630211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b55c00640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 0x7f4b501f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b55200640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 0x7f4b500c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 -- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 msgr2=0x7f4b500c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 0x7f4b500c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 -- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 msgr2=0x7f4b501f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 0x7f4b501f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b56600640 1 -- 172.21.15.28:0/3668630211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b500c8ed0 con 0x7f4b501629e0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b55c00640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 0x7f4b501f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.309+0000 7f4b55200640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 0x7f4b500c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b56600640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cef0 tx=0x7f4b4800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4801a070 con 0x7f4b501629e0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4800d860 con 0x7f4b501629e0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5015c9a0 con 0x7f4b501629e0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800dc50 con 0x7f4b501629e0 2024-09-10T10:54:35.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.310+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b500c8bf0 con 0x7f4b501629e0 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.311+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b5009ae60 con 0x7f4b501629e0 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.312+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b4801d070 con 0x7f4b501629e0 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.313+0000 7f4b4d600640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 0x7f4b3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.313+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f4b4808c4d0 con 0x7f4b501629e0 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.313+0000 7f4b55c00640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 0x7f4b3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.314+0000 7f4b55c00640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 0x7f4b3006e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000f60 tx=0x7f4b44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.317+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b4805a1b0 con 0x7f4b501629e0 2024-09-10T10:54:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.537+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b5009ba50 con 0x7f4b501629e0 2024-09-10T10:54:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.538+0000 7f4b4d600640 1 -- 172.21.15.28:0/3668630211 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 289 v289) ==== 95+0+5234 (secure 0 0 0) 0x7f4b48059ab0 con 0x7f4b501629e0 2024-09-10T10:54:35.540 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:35.540 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":289,"btime":"2024-09-10T10:54:26:750890+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9606,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":270}],"filesystems":[{"mdsmap":{"epoch":289,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:26.740514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9747},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9747":{"gid":9747,"name":"a","rank":0,"incarnation":285,"state":"up:active","state_seq":5,"addr":"172.21.15.28:6835/1056314461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1056314461},{"type":"v1","addr":"172.21.15.28:6835","nonce":1056314461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9747,"qdb_cluster":[9747]},"id":6}]} 2024-09-10T10:54:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 289 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.540+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 msgr2=0x7f4b3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.540+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 0x7f4b3006e2f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000f60 tx=0x7f4b44009040 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 msgr2=0x7f4b500c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cef0 tx=0x7f4b4800a710 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 shutdown_connections 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b3006be30 0x7f4b3006e2f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4b501629e0 0x7f4b500c8570 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b500b47b0 0x7f4b500c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 --2- 172.21.15.28:0/3668630211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b5009aa30 0x7f4b501f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 >> 172.21.15.28:0/3668630211 conn(0x7f4b500b3c90 msgr2=0x7f4b500b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 shutdown_connections 2024-09-10T10:54:35.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.541+0000 7f4b57000640 1 -- 172.21.15.28:0/3668630211 wait complete. 2024-09-10T10:54:35.559 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T10:54:35.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.716+0000 7f740be00640 1 Processor -- start 2024-09-10T10:54:35.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.716+0000 7f740be00640 1 -- start start 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f740c15e550 0x7f740c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f740c0b3400 0x7f740c0d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c0d2920 0x7f740c0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f740c0d32b0 con 0x7f740c0b3400 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f740c0d4a50 con 0x7f740c15e550 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.717+0000 7f740be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f740c0cf180 con 0x7f740c0d2920 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f740c0b3400 0x7f740c0d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f740c15e550 0x7f740c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f740c0b3400 0x7f740c0d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48128/0 (socket says 172.21.15.28:48128) 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 -- 172.21.15.28:0/344225798 learned_addr learned my addr 172.21.15.28:0/344225798 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740b400640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c0d2920 0x7f740c0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 -- 172.21.15.28:0/344225798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c0d2920 msgr2=0x7f740c0d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c0d2920 0x7f740c0d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 -- 172.21.15.28:0/344225798 >> v2:172.21.15.138:3300/0 conn(0x7f740c15e550 msgr2=0x7f740c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 --2- 172.21.15.28:0/344225798 >> v2:172.21.15.138:3300/0 conn(0x7f740c15e550 0x7f740c06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.718+0000 7f740a000640 1 -- 172.21.15.28:0/344225798 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c06d270 con 0x7f740c0b3400 2024-09-10T10:54:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.719+0000 7f740a000640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c0d1f10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f740c0b6f30 tx=0x7f73f802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0c280029fe4ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.719+0000 7f7409600640 1 -- 172.21.15.28:0/344225798 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8037070 con 0x7f740c0b3400 2024-09-10T10:54:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.719+0000 7f7409600640 1 -- 172.21.15.28:0/344225798 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f802ae00 con 0x7f740c0b3400 2024-09-10T10:54:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.719+0000 7f7409600640 1 -- 172.21.15.28:0/344225798 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8032660 con 0x7f740c0b3400 2024-09-10T10:54:35.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 -- 172.21.15.28:0/344225798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 msgr2=0x7f740c0d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c0d1f10 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f740c0b6f30 tx=0x7f73f802a0b0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 -- 172.21.15.28:0/344225798 shutdown_connections 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c0d2920 0x7f740c0d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 --2- 172.21.15.28:0/344225798 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c0d1f10 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 --2- 172.21.15.28:0/344225798 >> v2:172.21.15.138:3300/0 conn(0x7f740c15e550 0x7f740c06cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 -- 172.21.15.28:0/344225798 >> 172.21.15.28:0/344225798 conn(0x7f740c08bfc0 msgr2=0x7f740c0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 -- 172.21.15.28:0/344225798 shutdown_connections 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.720+0000 7f740be00640 1 -- 172.21.15.28:0/344225798 wait complete. 2024-09-10T10:54:35.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.722+0000 7f740be00640 1 Processor -- start 2024-09-10T10:54:35.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.722+0000 7f740be00640 1 -- start start 2024-09-10T10:54:35.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c0d2920 0x7f740c0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c15e550 0x7f740c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f740c0d4f80 con 0x7f740c0b3400 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f740c0cee50 con 0x7f740c0d2920 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f740c0d2450 con 0x7f740c15e550 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48130/0 (socket says 172.21.15.28:48130) 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740aa00640 1 -- 172.21.15.28:0/3637781212 learned_addr learned my addr 172.21.15.28:0/3637781212 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740a000640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c0d2920 0x7f740c0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740b400640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c15e550 0x7f740c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740aa00640 1 -- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c15e550 msgr2=0x7f740c0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.723+0000 7f740aa00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c15e550 0x7f740c0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740aa00640 1 -- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c0d2920 msgr2=0x7f740c0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740aa00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c0d2920 0x7f740c0cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740aa00640 1 -- 172.21.15.28:0/3637781212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740c153ca0 con 0x7f740c0b3400 2024-09-10T10:54:35.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740aa00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00e9e0 tx=0x7f73fc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f7402a00640 1 -- 172.21.15.28:0/3637781212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc017070 con 0x7f740c0b3400 2024-09-10T10:54:35.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f7402a00640 1 -- 172.21.15.28:0/3637781212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc00c9d0 con 0x7f740c0b3400 2024-09-10T10:54:35.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f7402a00640 1 -- 172.21.15.28:0/3637781212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc00cdc0 con 0x7f740c0b3400 2024-09-10T10:54:35.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740c0cbcf0 con 0x7f740c0b3400 2024-09-10T10:54:35.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.724+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740c0cc2b0 con 0x7f740c0b3400 2024-09-10T10:54:35.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.726+0000 7f7402a00640 1 -- 172.21.15.28:0/3637781212 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73fc01a070 con 0x7f740c0b3400 2024-09-10T10:54:35.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.727+0000 7f7402a00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 0x7f73ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.727+0000 7f7402a00640 1 -- 172.21.15.28:0/3637781212 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f73fc08bec0 con 0x7f740c0b3400 2024-09-10T10:54:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.728+0000 7f7414a47740 1 Processor -- start 2024-09-10T10:54:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.728+0000 7f740a000640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 0x7f73ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.728+0000 7f7414a47740 1 -- start start 2024-09-10T10:54:35.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740a000640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 0x7f73ec06e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73f8034000 tx=0x7f73f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431edf220 0x55d431ee1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431edc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d431e45760 con 0x55d431ee1a60 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d431ded6c0 con 0x55d431ee31f0 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d431e41720 con 0x55d431edf220 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48144/0 (socket says 172.21.15.28:48144) 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 -- 172.21.15.28:0/1179807410 learned_addr learned my addr 172.21.15.28:0/1179807410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740a000640 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431edf220 0x55d431ee1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740b400640 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431edc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 -- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431edf220 msgr2=0x55d431ee1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431edf220 0x55d431ee1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 -- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 msgr2=0x55d431edc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431edc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 -- 172.21.15.28:0/1179807410 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d431c247c0 con 0x55d431ee1a60 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.729+0000 7f740aa00640 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0558f0 tx=0x7f73fc059490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d48f58e8e1be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f73f7400640 1 -- 172.21.15.28:0/1179807410 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc021040 con 0x55d431ee1a60 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f73f7400640 1 -- 172.21.15.28:0/1179807410 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc057790 con 0x55d431ee1a60 2024-09-10T10:54:35.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f73f7400640 1 -- 172.21.15.28:0/1179807410 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc071170 con 0x55d431ee1a60 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 -- 172.21.15.28:0/1179807410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 msgr2=0x55d431f27200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f73fc0558f0 tx=0x7f73fc059490 comp rx=0 tx=0).stop 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 -- 172.21.15.28:0/1179807410 shutdown_connections 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431edc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431edf220 0x55d431ee1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 --2- 172.21.15.28:0/1179807410 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f27200 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 -- 172.21.15.28:0/1179807410 >> 172.21.15.28:0/1179807410 conn(0x55d431ede220 msgr2=0x55d431f266d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 -- 172.21.15.28:0/1179807410 shutdown_connections 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.730+0000 7f7414a47740 1 -- 172.21.15.28:0/1179807410 wait complete. 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.731+0000 7f7414a47740 1 -- start start 2024-09-10T10:54:35.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.731+0000 7f7414a47740 1 Processor -- start 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f0bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431f0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d431edd6c0 con 0x55d431f0c330 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d431e008e0 con 0x55d431ee31f0 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d431f27a30 con 0x55d431ee1a60 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f7414a47740 10 client.0 _ll_register_callbacks cb 0x55d431df23c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.732+0000 7f740b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48158/0 (socket says 172.21.15.28:48158) 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 -- 172.21.15.28:0/3398743256 learned_addr learned my addr 172.21.15.28:0/3398743256 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740aa00640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f0bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 -- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 msgr2=0x55d431f0bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f0bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 -- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 msgr2=0x55d431f0a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431f0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 -- 172.21.15.28:0/3398743256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d431c84f50 con 0x55d431f0c330 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f740b400640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7404007d30 tx=0x7f7404013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146c84d8ef59c854 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f73ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.733+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401d020 con 0x55d431f0c330 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f7404019460 con 0x55d431f0c330 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73fc057790 con 0x55d431f0c330 2024-09-10T10:54:35.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=289) ==== 24+0+0 (secure 0 0 0) 0x7f7404028050 con 0x55d431f0c330 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 10 client.9843 fetch_fsmap learned FSMap version 289 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=289,monmap=2+}) -- 0x55d431b180b0 con 0x55d431f0c330 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73fc057790 con 0x55d431f0c330 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 289) ==== 3502+0+0 (secure 0 0 0) 0x7f7404019660 con 0x55d431f0c330 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 10 client.9843 fetch_fsmap finished waiting for FSMap version 289 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 10 client.9843 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 20 client.9843 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f0ab40 0x55d431f04c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 4 client.9843 mds_command: new command op to 9747 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.734+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 --> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d431e52e90 con 0x55d431f0ab40 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.735+0000 7f740aa00640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f0ab40 0x55d431f04c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.735+0000 7f740aa00640 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f0ab40 0x55d431f04c70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3846e32e0abef252 server_cookie=e9cea04251bfdb0a in_seq=0 out_seq=0 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.735+0000 7f73ebe00640 10 client.9843 ms_handle_connect on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f73ebe00640 1 -- 172.21.15.28:0/3398743256 <== mds.0 v2:172.21.15.28:6833/1056314461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d431e52e90 con 0x55d431f0ab40 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f73ebe00640 10 client.9843 handle_command_reply: tid=0 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 client.9843 shutdown 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 msgr2=0x55d431f0a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7404007d30 tx=0x7f7404013600 comp rx=0 tx=0).stop 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 shutdown_connections 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f0ab40 0x55d431f04c70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431f0c330 0x55d431f0a600 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431ee31f0 0x55d431f0a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f73ebe00640 0 client.9843 ms_handle_reset on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 --2- 172.21.15.28:0/3398743256 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f0bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 >> 172.21.15.28:0/3398743256 conn(0x55d431edf220 msgr2=0x55d431f25600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 shutdown_connections 2024-09-10T10:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 1 -- 172.21.15.28:0/3398743256 wait complete. 2024-09-10T10:54:35.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.736+0000 7f7414a47740 20 client.9843 trim_cache size 0 max 16384 2024-09-10T10:54:35.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.751+0000 7f7414a47740 1 Processor -- start 2024-09-10T10:54:35.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.751+0000 7f7414a47740 1 -- start start 2024-09-10T10:54:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f06730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f24750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f740c201eb0 con 0x55d431ee31f0 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f740c2011b0 con 0x55d431f0c330 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f740c201800 con 0x55d431ee1a60 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48162/0 (socket says 172.21.15.28:48162) 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 -- 172.21.15.28:0/1813955092 learned_addr learned my addr 172.21.15.28:0/1813955092 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740aa00640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f06730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740b400640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f24750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 -- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 msgr2=0x55d431f06730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f06730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 -- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 msgr2=0x55d431f24750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f24750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740a000640 1 -- 172.21.15.28:0/1813955092 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d431f27b30 con 0x55d431ee31f0 2024-09-10T10:54:35.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740b400640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.752+0000 7f740aa00640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f06730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:35.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f740a000640 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f73f802a0f0 tx=0x7f73f802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ceafbb0dab915f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f73f6a00640 1 -- 172.21.15.28:0/1813955092 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8037040 con 0x55d431ee31f0 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f73f6a00640 1 -- 172.21.15.28:0/1813955092 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f804f070 con 0x55d431ee31f0 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f73f6a00640 1 -- 172.21.15.28:0/1813955092 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8032cd0 con 0x55d431ee31f0 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f7414a47740 1 -- 172.21.15.28:0/1813955092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 msgr2=0x55d431f4b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.753+0000 7f7414a47740 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f73f802a0f0 tx=0x7f73f802c000 comp rx=0 tx=0).stop 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 -- 172.21.15.28:0/1813955092 shutdown_connections 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f24750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee31f0 0x55d431f4b300 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 --2- 172.21.15.28:0/1813955092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee1a60 0x55d431f06730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 -- 172.21.15.28:0/1813955092 >> 172.21.15.28:0/1813955092 conn(0x55d431edf220 msgr2=0x55d431ed72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 -- 172.21.15.28:0/1813955092 shutdown_connections 2024-09-10T10:54:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 -- 172.21.15.28:0/1813955092 wait complete. 2024-09-10T10:54:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 -- start start 2024-09-10T10:54:35.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.754+0000 7f7414a47740 1 Processor -- start 2024-09-10T10:54:35.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.755+0000 7f7414a47740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.755+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee31f0 0x55d431fa5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.755+0000 7f7414a47740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f2a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f7414a47740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d431f02110 con 0x55d431ee1a60 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d431f2ba40 con 0x55d431f0c330 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f7414a47740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d431f3a5e0 con 0x55d431ee31f0 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f7414a47740 10 client.0 _ll_register_callbacks cb 0x7f74040079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48178/0 (socket says 172.21.15.28:48178) 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 -- 172.21.15.28:0/350187379 learned_addr learned my addr 172.21.15.28:0/350187379 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740b400640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f2a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740a000640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee31f0 0x55d431fa5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 -- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee31f0 msgr2=0x55d431fa5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee31f0 0x55d431fa5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 -- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 msgr2=0x55d431f2a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f2a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 -- 172.21.15.28:0/350187379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d431f238f0 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f740aa00640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f73fc059420 tx=0x7f73fc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584407ead3b8ec11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.756+0000 7f73ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73fc021040 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f73fc055b00 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f804f070 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=289) ==== 24+0+0 (secure 0 0 0) 0x7f73fc078050 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 10 client.9849 fetch_fsmap learned FSMap version 289 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=289,monmap=2+}) -- 0x55d431fa59b0 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f804f070 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 289) ==== 3502+0+0 (secure 0 0 0) 0x7f73fc0781f0 con 0x55d431ee1a60 2024-09-10T10:54:35.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 10 client.9849 fetch_fsmap finished waiting for FSMap version 289 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 10 client.9849 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 20 client.9849 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f09cf0 0x55d431f0a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 4 client.9849 mds_command: new command op to 9747 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 --> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d431e4d1e0 con 0x55d431f09cf0 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.757+0000 7f740a000640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f09cf0 0x55d431f0a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f740a000640 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f09cf0 0x55d431f0a0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e9098d9d4062b35 server_cookie=bca43bcf921788d1 in_seq=0 out_seq=0 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f73ebe00640 10 client.9849 ms_handle_connect on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f73ebe00640 1 -- 172.21.15.28:0/350187379 <== mds.0 v2:172.21.15.28:6833/1056314461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+194 (crc 0 0 0) 0x55d431e4d1e0 con 0x55d431f09cf0 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f73ebe00640 10 client.9849 handle_command_reply: tid=0 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f7414a47740 1 client.9849 shutdown 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 msgr2=0x55d431f49bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f73fc059420 tx=0x7f73fc000bc0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.758+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 shutdown_connections 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:6833/1056314461,v1:172.21.15.28:6835/1056314461] conn(0x55d431f09cf0 0x55d431f0a0d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d431f0c330 0x55d431f2a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d431ee31f0 0x55d431fa5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 --2- 172.21.15.28:0/350187379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d431ee1a60 0x55d431f49bd0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f73ebe00640 0 client.9849 ms_handle_reset on v2:172.21.15.28:6833/1056314461 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 >> 172.21.15.28:0/350187379 conn(0x55d431edf220 msgr2=0x55d431f227c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 shutdown_connections 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 1 -- 172.21.15.28:0/350187379 wait complete. 2024-09-10T10:54:35.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f7414a47740 20 client.9849 trim_cache size 0 max 16384 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "dentry", 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 3592133532, 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1, 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "frag": "*", 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "subdir", 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "snap_id": "head", 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/subdir" 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:54:35.766 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:54:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 msgr2=0x7f73ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 0x7f73ec06e320 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f73f8034000 tx=0x7f73f802e000 comp rx=0 tx=0).stop 2024-09-10T10:54:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 msgr2=0x7f740c153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.759+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00e9e0 tx=0x7f73fc00eea0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 shutdown_connections 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f73ec06be60 0x7f73ec06e320 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f740c15e550 0x7f740c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f740c0d2920 0x7f740c0cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 --2- 172.21.15.28:0/3637781212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f740c0b3400 0x7f740c153760 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 >> 172.21.15.28:0/3637781212 conn(0x7f740c08bfc0 msgr2=0x7f740c0b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 shutdown_connections 2024-09-10T10:54:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:35.760+0000 7f740be00640 1 -- 172.21.15.28:0/3637781212 wait complete. 2024-09-10T10:54:35.777 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 10000000000.00000000' 2024-09-10T10:54:35.777 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:35.777 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:35.812 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:35.813 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:35.888 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:54:36.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.082+0000 7f66ff200640 1 Processor -- start 2024-09-10T10:54:36.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.083+0000 7f66ff200640 1 -- start start 2024-09-10T10:54:36.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ff200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ff200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66f8072de0 0x7f66f806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ff200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66f8146140 con 0x7f66f8072700 2024-09-10T10:54:36.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ff200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66f80cd740 con 0x7f66f8072de0 2024-09-10T10:54:36.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ff200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66f80cda20 con 0x7f66f8072320 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66f7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66fd200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f66f8072700 0x7f66f806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:60952/0 (socket says 172.21.15.28:60952) 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66fd200640 1 -- 172.21.15.28:0/1130402378 learned_addr learned my addr 172.21.15.28:0/1130402378 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66ef400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66f8072de0 0x7f66f806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.084+0000 7f66f7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:54354/0 (socket says 172.21.15.28:54354) 2024-09-10T10:54:36.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66f7400640 1 -- 172.21.15.28:0/1130402378 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 412936557 0 0) 0x7f66f8146140 con 0x7f66f8072700 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66f7400640 1 -- 172.21.15.28:0/1130402378 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66dc003520 con 0x7f66f8072700 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66ef400640 1 -- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 msgr2=0x7f66f80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66ef400640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66ef400640 1 -- 172.21.15.28:0/1130402378 >> v1:172.21.15.138:6789/0 conn(0x7f66f8072700 legacy=0x7f66f806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66ef400640 1 -- 172.21.15.28:0/1130402378 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f80cdd00 con 0x7f66f8072de0 2024-09-10T10:54:36.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.085+0000 7f66f7e00640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:36.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.086+0000 7f66ef400640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f806e0b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f66e00097a0 tx=0x7f66e002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adbd8a49b6905526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.086+0000 7f66f7400640 1 -- 172.21.15.28:0/1130402378 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0037070 con 0x7f66f8072de0 2024-09-10T10:54:36.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.086+0000 7f66f7400640 1 -- 172.21.15.28:0/1130402378 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e002ae50 con 0x7f66f8072de0 2024-09-10T10:54:36.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.086+0000 7f66f7400640 1 -- 172.21.15.28:0/1130402378 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0032640 con 0x7f66f8072de0 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 -- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 msgr2=0x7f66f806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f806e0b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f66e00097a0 tx=0x7f66e002a100 comp rx=0 tx=0).stop 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 -- 172.21.15.28:0/1130402378 shutdown_connections 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f806e0b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 --2- 172.21.15.28:0/1130402378 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f80cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 -- 172.21.15.28:0/1130402378 >> 172.21.15.28:0/1130402378 conn(0x7f66f8067d70 msgr2=0x7f66f806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 -- 172.21.15.28:0/1130402378 shutdown_connections 2024-09-10T10:54:36.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.087+0000 7f66ff200640 1 -- 172.21.15.28:0/1130402378 wait complete. 2024-09-10T10:54:36.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.089+0000 7f66ff200640 1 Processor -- start 2024-09-10T10:54:36.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.089+0000 7f66ff200640 1 -- start start 2024-09-10T10:54:36.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f8142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f81f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80cd520 con 0x7f66f8072700 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80cd3a0 con 0x7f66f8072de0 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ff200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66f806ed00 con 0x7f66f8072320 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ef400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ef400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48182/0 (socket says 172.21.15.28:48182) 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66ef400640 1 -- 172.21.15.28:0/3184006406 learned_addr learned my addr 172.21.15.28:0/3184006406 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:36.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66f7e00640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f8142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.090+0000 7f66fd200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f81f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 -- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 msgr2=0x7f66f8142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f8142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 -- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 msgr2=0x7f66f81f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f81f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 -- 172.21.15.28:0/3184006406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f80aa1f0 con 0x7f66f8072700 2024-09-10T10:54:36.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66fd200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f81f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ef400640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0041070 con 0x7f66f8072700 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e003c460 con 0x7f66f8072700 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0037040 con 0x7f66f8072700 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f81f5ad0 con 0x7f66f8072700 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.091+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f80abd60 con 0x7f66f8072700 2024-09-10T10:54:36.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.092+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f80cce60 con 0x7f66f8072700 2024-09-10T10:54:36.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.093+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66e0044070 con 0x7f66f8072700 2024-09-10T10:54:36.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.094+0000 7f66f4c00640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.094+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7f66e00ac100 con 0x7f66f8072700 2024-09-10T10:54:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.095+0000 7f66f7e00640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.096+0000 7f66f7e00640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 0x7f66d406e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009560 tx=0x7f66e8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.098+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66e00796d0 con 0x7f66f8072700 2024-09-10T10:54:36.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.296+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66f81f5ff0 con 0x7f66f8072700 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.297+0000 7f66f4c00640 1 -- 172.21.15.28:0/3184006406 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 66 entries v133) ==== 81+0+4010 (secure 0 0 0) 0x7f66e007d9d0 con 0x7f66f8072700 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:36.301 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:36.303 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:36.304 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:listed 66 entries 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 msgr2=0x7f66d406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 0x7f66d406e2f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009560 tx=0x7f66e8009190 comp rx=0 tx=0).stop 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 msgr2=0x7f66f80a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e00046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 shutdown_connections 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f66d406be30 0x7f66d406e2f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66f8072de0 0x7f66f81f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f8072700 0x7f66f80a9cb0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 --2- 172.21.15.28:0/3184006406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66f8072320 0x7f66f8142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 >> 172.21.15.28:0/3184006406 conn(0x7f66f8067d70 msgr2=0x7f66f80c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 shutdown_connections 2024-09-10T10:54:36.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.299+0000 7f66ff200640 1 -- 172.21.15.28:0/3184006406 wait complete. 2024-09-10T10:54:36.317 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:54:36.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:36.318 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:54:36.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:54:36.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:36.377 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:36.414 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:54:36.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.608+0000 7fb1b7c00640 1 Processor -- start 2024-09-10T10:54:36.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.608+0000 7fb1b7c00640 1 -- start start 2024-09-10T10:54:36.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.609+0000 7fb1b7c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1b015b300 0x7fb1b00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.609+0000 7fb1b7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 0x7fb1b006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.609+0000 7fb1b7c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b00cdbe0 con 0x7fb1b0072d30 2024-09-10T10:54:36.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.609+0000 7fb1b7c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b00728a0 con 0x7fb1b015b300 2024-09-10T10:54:36.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.609+0000 7fb1b7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b00cd900 con 0x7fb1b00cd3f0 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1afe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 0x7fb1b006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1b5200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1b015b300 0x7fb1b00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1b5c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb1b0072d30 0x7fb1b00b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:60960/0 (socket says 172.21.15.28:60960) 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1b5c00640 1 -- 172.21.15.28:0/16909963 learned_addr learned my addr 172.21.15.28:0/16909963 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1af400640 1 -- 172.21.15.28:0/16909963 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1759166151 0 0) 0x7fb1b00cdbe0 con 0x7fb1b0072d30 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1b5200640 1 -- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 msgr2=0x7fb1b006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.610+0000 7fb1b5200640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 0x7fb1b006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.611+0000 7fb1b5200640 1 -- 172.21.15.28:0/16909963 >> v1:172.21.15.138:6789/0 conn(0x7fb1b0072d30 legacy=0x7fb1b00b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.611+0000 7fb1b5200640 1 -- 172.21.15.28:0/16909963 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b0072250 con 0x7fb1b015b300 2024-09-10T10:54:36.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.611+0000 7fb1afe00640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 0x7fb1b006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:54:36.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1b5200640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b015b300 0x7fb1b00cceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097a0 tx=0x7fb1a402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b238246cbee11282 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1af400640 1 -- 172.21.15.28:0/16909963 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4037070 con 0x7fb1b015b300 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1af400640 1 -- 172.21.15.28:0/16909963 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a4032440 con 0x7fb1b015b300 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1af400640 1 -- 172.21.15.28:0/16909963 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4032810 con 0x7fb1b015b300 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1b7c00640 1 -- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b015b300 msgr2=0x7fb1b00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.612+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b015b300 0x7fb1b00cceb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097a0 tx=0x7fb1a402a180 comp rx=0 tx=0).stop 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 -- 172.21.15.28:0/16909963 shutdown_connections 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b00cd3f0 0x7fb1b006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/16909963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b015b300 0x7fb1b00cceb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 -- 172.21.15.28:0/16909963 >> 172.21.15.28:0/16909963 conn(0x7fb1b0067d70 msgr2=0x7fb1b006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 -- 172.21.15.28:0/16909963 shutdown_connections 2024-09-10T10:54:36.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.613+0000 7fb1b7c00640 1 -- 172.21.15.28:0/16909963 wait complete. 2024-09-10T10:54:36.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.614+0000 7fb1b7c00640 1 Processor -- start 2024-09-10T10:54:36.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.614+0000 7fb1b7c00640 1 -- start start 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b0072d30 0x7fb1b01f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b00cd3f0 0x7fb1b01e4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b00d0160 con 0x7fb1b015b300 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b00cffe0 con 0x7fb1b00cd3f0 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.615+0000 7fb1b7c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b006f700 con 0x7fb1b0072d30 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48196/0 (socket says 172.21.15.28:48196) 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 -- 172.21.15.28:0/516412248 learned_addr learned my addr 172.21.15.28:0/516412248 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5200640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b0072d30 0x7fb1b01f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1afe00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b00cd3f0 0x7fb1b01e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 -- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b0072d30 msgr2=0x7fb1b01f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b0072d30 0x7fb1b01f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 -- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b00cd3f0 msgr2=0x7fb1b01e4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b00cd3f0 0x7fb1b01e4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 -- 172.21.15.28:0/516412248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b01571e0 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.616+0000 7fb1b5c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c7d0 tx=0x7fb1a800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.617+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8018070 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.617+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a800a5e0 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.617+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a800aa10 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.617+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b01574c0 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.617+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b0157a80 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.618+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b00ca6e0 con 0x7fb1b015b300 2024-09-10T10:54:36.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.619+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb1a801b070 con 0x7fb1b015b300 2024-09-10T10:54:36.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.620+0000 7fb1acc00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 0x7fb19006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:36.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.620+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9142+0+0 (secure 0 0 0) 0x7fb1a808b150 con 0x7fb1b015b300 2024-09-10T10:54:36.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.620+0000 7fb1b5200640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 0x7fb19006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:36.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.621+0000 7fb1b5200640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 0x7fb19006e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:36.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.625+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb1a8058720 con 0x7fb1b015b300 2024-09-10T10:54:36.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.847+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1b01ea8c0 con 0x7fb1b015b300 2024-09-10T10:54:36.850 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:36.848+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:36.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.850+0000 7fb1acc00640 1 -- 172.21.15.28:0/516412248 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v290) ==== 161+0+0 (secure 0 0 0) 0x7fb1a805ca20 con 0x7fb1b015b300 2024-09-10T10:54:36.852 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 msgr2=0x7fb19006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 0x7fb19006e2f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4034000 tx=0x7fb1a402e000 comp rx=0 tx=0).stop 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 msgr2=0x7fb1b015cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c7d0 tx=0x7fb1a800cc90 comp rx=0 tx=0).stop 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 shutdown_connections 2024-09-10T10:54:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb19006be30 0x7fb19006e2f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb1b015b300 0x7fb1b015cbc0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1b00cd3f0 0x7fb1b01e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 --2- 172.21.15.28:0/516412248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb1b0072d30 0x7fb1b01f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 >> 172.21.15.28:0/516412248 conn(0x7fb1b0067d70 msgr2=0x7fb1b00c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 shutdown_connections 2024-09-10T10:54:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:36.853+0000 7fb1b7c00640 1 -- 172.21.15.28:0/516412248 wait complete. 2024-09-10T10:54:36.860 INFO:tasks.ceph.mds.a.smithi028.stderr: -843> 2024-09-10T10:54:24.723+0000 7fe150800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:36.873 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:54:36.905 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:54:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.031+0000 7f0859c00640 1 Processor -- start 2024-09-10T10:54:37.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.031+0000 7f0859c00640 1 -- start start 2024-09-10T10:54:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0859c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 0x7f085406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0859c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f08540d1f90 0x7f08540cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0859c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08540cf000 con 0x7f08540d0dd0 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0859c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f08540b37d0 con 0x7f08540d1f90 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0859c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f08540cf330 con 0x7f085415e660 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0853e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f08540d0dd0 0x7f08540d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44170/0 (socket says 172.21.15.28:44170) 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.032+0000 7f0853e00640 1 -- 172.21.15.28:0/4245177862 learned_addr learned my addr 172.21.15.28:0/4245177862 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0859200640 1 --2- 172.21.15.28:0/4245177862 >> v2:172.21.15.28:3300/0 conn(0x7f08540d1f90 0x7f08540cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0853400640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 0x7f085406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0852a00640 1 -- 172.21.15.28:0/4245177862 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398036005 0 0) 0x7f08540cf000 con 0x7f08540d0dd0 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0859200640 1 -- 172.21.15.28:0/4245177862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 msgr2=0x7f085406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0859200640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 0x7f085406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0859200640 1 -- 172.21.15.28:0/4245177862 >> v1:172.21.15.28:6789/0 conn(0x7f08540d0dd0 legacy=0x7f08540d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0859200640 1 -- 172.21.15.28:0/4245177862 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540d29a0 con 0x7f08540d1f90 2024-09-10T10:54:37.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.033+0000 7f0853400640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 0x7f085406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.034+0000 7f0859200640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08540d1f90 0x7f08540cf7c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d90 tx=0x7f084c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e6d0a9ff80a8b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.034+0000 7f0852a00640 1 -- 172.21.15.28:0/4245177862 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007740 con 0x7f08540d1f90 2024-09-10T10:54:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.034+0000 7f0852a00640 1 -- 172.21.15.28:0/4245177862 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c0078e0 con 0x7f08540d1f90 2024-09-10T10:54:37.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.034+0000 7f0852a00640 1 -- 172.21.15.28:0/4245177862 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007cb0 con 0x7f08540d1f90 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 -- 172.21.15.28:0/4245177862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08540d1f90 msgr2=0x7f08540cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08540d1f90 0x7f08540cf7c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d90 tx=0x7f084c02a660 comp rx=0 tx=0).stop 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 -- 172.21.15.28:0/4245177862 shutdown_connections 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f08540d1f90 0x7f08540cf7c0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 --2- 172.21.15.28:0/4245177862 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f085415e660 0x7f085406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 -- 172.21.15.28:0/4245177862 >> 172.21.15.28:0/4245177862 conn(0x7f085408bfc0 msgr2=0x7f08540d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 -- 172.21.15.28:0/4245177862 shutdown_connections 2024-09-10T10:54:37.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.035+0000 7f0859c00640 1 -- 172.21.15.28:0/4245177862 wait complete. 2024-09-10T10:54:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.037+0000 7f0859c00640 1 Processor -- start 2024-09-10T10:54:37.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.037+0000 7f0859c00640 1 -- start start 2024-09-10T10:54:37.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08540d0dd0 0x7f08541f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08540d1f90 0x7f08541f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d3220 con 0x7f085415e660 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08540d2f40 con 0x7f08540d1f90 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f08540d33a0 con 0x7f08540d0dd0 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48224/0 (socket says 172.21.15.28:48224) 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 -- 172.21.15.28:0/4133726357 learned_addr learned my addr 172.21.15.28:0/4133726357 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0853e00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08540d0dd0 0x7f08541f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0853400640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08540d1f90 0x7f08541f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 -- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08540d0dd0 msgr2=0x7f08541f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08540d0dd0 0x7f08541f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 -- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08540d1f90 msgr2=0x7f08541f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08540d1f90 0x7f08541f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.038+0000 7f0859200640 1 -- 172.21.15.28:0/4133726357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540c9220 con 0x7f085415e660 2024-09-10T10:54:37.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f0859200640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f084c009eb0 tx=0x7f084c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c040070 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c031bc0 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c045460 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08540c94a0 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.039+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08540c9a60 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.040+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540d2440 con 0x7f085415e660 2024-09-10T10:54:37.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.041+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f084c039040 con 0x7f085415e660 2024-09-10T10:54:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.042+0000 7f084be00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 0x7f083406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.042+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f084c0ad560 con 0x7f085415e660 2024-09-10T10:54:37.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.043+0000 7f0853e00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 0x7f083406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:37.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.044+0000 7f0853e00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 0x7f083406e2f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0844000f60 tx=0x7f0844007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:37.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.047+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08540c9220 con 0x7f085415e660 2024-09-10T10:54:37.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.300+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f08540d1b20 con 0x7f085415e660 2024-09-10T10:54:37.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.852+0000 7f084be00640 1 -- 172.21.15.28:0/4133726357 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v291) ==== 107+0+0 (secure 0 0 0) 0x7f084c07aa50 con 0x7f085415e660 2024-09-10T10:54:37.854 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 msgr2=0x7f083406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 0x7f083406e2f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0844000f60 tx=0x7f0844007d50 comp rx=0 tx=0).stop 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 msgr2=0x7f08541f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f084c009eb0 tx=0x7f084c004a40 comp rx=0 tx=0).stop 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 shutdown_connections 2024-09-10T10:54:37.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f083406be30 0x7f083406e2f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f085415e660 0x7f08541f7ca0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08540d1f90 0x7f08541f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 --2- 172.21.15.28:0/4133726357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f08540d0dd0 0x7f08541f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 >> 172.21.15.28:0/4133726357 conn(0x7f085408bfc0 msgr2=0x7f08540b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 shutdown_connections 2024-09-10T10:54:37.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.855+0000 7f0859c00640 1 -- 172.21.15.28:0/4133726357 wait complete. 2024-09-10T10:54:37.876 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:54:37.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.994+0000 7f1a9ab55f40 1 Processor -- start 2024-09-10T10:54:37.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 -- start start 2024-09-10T10:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891da90ff0 0x55891db1b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:37.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 0x55891db214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55891da91530 con 0x55891db1f9c0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55891d9eeb30 con 0x55891da90ff0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.995+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55891db1eee0 con 0x55891db1c3e0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48236/0 (socket says 172.21.15.28:48236) 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 -- 172.21.15.28:0/12401259 learned_addr learned my addr 172.21.15.28:0/12401259 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a97e00640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891da90ff0 0x55891db1b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a97400640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 0x55891db214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 -- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 msgr2=0x55891db214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:38.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 0x55891db214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 -- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891da90ff0 msgr2=0x55891db1b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891da90ff0 0x55891db1b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 -- 172.21.15.28:0/12401259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891db1bbb0 con 0x55891db1f9c0 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a97400640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 0x55891db214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.996+0000 7f1a98800640 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1a900460e0 tx=0x7f1a90051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5b6f70f07660a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a96000640 1 -- 172.21.15.28:0/12401259 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90067070 con 0x55891db1f9c0 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a96000640 1 -- 172.21.15.28:0/12401259 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a90058b50 con 0x55891db1f9c0 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a96000640 1 -- 172.21.15.28:0/12401259 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900585c0 con 0x55891db1f9c0 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/12401259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 msgr2=0x55891db219f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f1a900460e0 tx=0x7f1a90051000 comp rx=0 tx=0).stop 2024-09-10T10:54:38.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/12401259 shutdown_connections 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891db1f9c0 0x55891db219f0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1c3e0 0x55891db214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/12401259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891da90ff0 0x55891db1b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/12401259 >> 172.21.15.28:0/12401259 conn(0x55891da950a0 msgr2=0x55891db1c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/12401259 shutdown_connections 2024-09-10T10:54:38.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.997+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/12401259 wait complete. 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.998+0000 7f1a9ab55f40 1 Processor -- start 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.998+0000 7f1a9ab55f40 1 -- start start 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891db1c3e0 0x55891db16500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1f9c0 0x55891db1a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55891da93a70 con 0x55891da90ff0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55891db21020 con 0x55891db1c3e0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a9ab55f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55891db1d880 con 0x55891db1f9c0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a97e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48240/0 (socket says 172.21.15.28:48240) 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:37.999+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 learned_addr learned my addr 172.21.15.28:0/3075901664 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891db1c3e0 0x55891db16500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a98800640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1f9c0 0x55891db1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1f9c0 msgr2=0x55891db1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55891db1f9c0 0x55891db1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891db1c3e0 msgr2=0x55891db16500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55891db1c3e0 0x55891db16500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891db188a0 con 0x55891da90ff0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.000+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1a88008400 tx=0x7f1a880084f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.001+0000 7f1a94c00640 1 -- 172.21.15.28:0/3075901664 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88015660 con 0x55891da90ff0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.001+0000 7f1a94c00640 1 -- 172.21.15.28:0/3075901664 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8801a070 con 0x55891da90ff0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.001+0000 7f1a94c00640 1 -- 172.21.15.28:0/3075901664 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88015a60 con 0x55891da90ff0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.001+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55891db1b880 con 0x55891da90ff0 2024-09-10T10:54:38.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.001+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891db203a0 con 0x55891da90ff0 2024-09-10T10:54:38.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.003+0000 7f1a94c00640 1 -- 172.21.15.28:0/3075901664 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1a88013030 con 0x55891da90ff0 2024-09-10T10:54:38.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.003+0000 7f1a94c00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 0x7f1a7806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.003+0000 7f1a94c00640 1 -- 172.21.15.28:0/3075901664 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f1a8808a8c0 con 0x55891da90ff0 2024-09-10T10:54:38.007 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:54:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.004+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 0x55891dbb8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.004+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891db15470 con 0x55891db16a40 2024-09-10T10:54:38.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.004+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 0x7f1a7806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.004+0000 7f1a98800640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 0x55891dbb8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.005+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 0x7f1a7806ccd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55891db1e040 tx=0x7f1a84001710 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.005+0000 7f1a98800640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 0x55891dbb8ed0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.008+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v134'1033 uv1007 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.008+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891db1aa90 con 0x55891db16a40 2024-09-10T10:54:38.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.010+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v134'1034 uv1034 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.010+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc1c90 con 0x55891db16a40 2024-09-10T10:54:38.013 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:54:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.012+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v134'1035 uv1035 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.013+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc3040 con 0x55891db16a40 2024-09-10T10:54:38.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.015+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v134'1036 uv1036 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.015+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc33e0 con 0x55891db16a40 2024-09-10T10:54:38.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.016+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v134'1037 uv1037 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.016+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc2030 con 0x55891db16a40 2024-09-10T10:54:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.019+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v134'1038 uv1038 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.019+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.021+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v134'1039 uv1013 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.021+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.023+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v134'1040 uv1040 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.023+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.026 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:54:38.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.025+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v134'1041 uv1041 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.025+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.027+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v134'1042 uv1042 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.027+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.029+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v134'1043 uv1043 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.029+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.031+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v134'1044 uv1044 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.031+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.033+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v134'1045 uv1020 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.033+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.035+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v134'1046 uv1046 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.035+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.038 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:54:38.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.037+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v134'1047 uv1047 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.037+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.039+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v134'1048 uv1048 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.039+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.040+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v134'1049 uv1049 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.040+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.042+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v134'1050 uv1050 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.042+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.044+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v134'1051 uv1051 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.044+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.046+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v134'1052 uv1026 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.046+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.047+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v134'1053 uv1053 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.047+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.051 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:54:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.049+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v134'1054 uv1054 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.049+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.051+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v134'1055 uv1055 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.051+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.052+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v134'1056 uv1056 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.052+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.054+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v134'1057 uv1057 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.054+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.057+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v134'1058 uv1032 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.057+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.059+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v134'1059 uv1059 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.059+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.061 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:54:38.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.061+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v134'1060 uv1060 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.061+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.062+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v134'1061 uv1061 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.062+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.064+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v134'1062 uv1062 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.064+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891db16a40 2024-09-10T10:54:38.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.066+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v134'1063 uv1063 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9004e040 con 0x55891db16a40 2024-09-10T10:54:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.066+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 0x55891dbc5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.066+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc23d0 con 0x55891dbc3780 2024-09-10T10:54:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.066+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 0x55891dbc5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.067+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 0x55891dbc5b70 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.069+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v134'752 uv731 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.069+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc60b0 con 0x55891dbc3780 2024-09-10T10:54:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.070+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v134'753 uv753 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.070+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc6450 con 0x55891dbc3780 2024-09-10T10:54:38.074 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:54:38.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.072+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v134'754 uv754 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.072+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc67f0 con 0x55891dbc3780 2024-09-10T10:54:38.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.075+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v134'755 uv755 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.075+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc6b90 con 0x55891dbc3780 2024-09-10T10:54:38.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.076+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v134'756 uv756 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.076+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc6f30 con 0x55891dbc3780 2024-09-10T10:54:38.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.078+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v134'757 uv750 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.078+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.080+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v134'758 uv758 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.080+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.083 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:54:38.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.081+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v134'759 uv759 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.081+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.083+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v134'760 uv760 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.083+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.084+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v134'761 uv761 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.084+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.086+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v134'762 uv751 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.086+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.087+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v134'763 uv763 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.088+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.091 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:54:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.089+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v134'764 uv764 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.089+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcb9c0 con 0x55891dbc3780 2024-09-10T10:54:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.091+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v134'765 uv765 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.091+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcb9c0 con 0x55891dbc3780 2024-09-10T10:54:38.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.093+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v134'766 uv766 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.093+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.094+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v134'767 uv747 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.094+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.096+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v134'768 uv768 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.096+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.097+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v134'769 uv769 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.099 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:54:38.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.097+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc82e0 con 0x55891dbc3780 2024-09-10T10:54:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.099+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v134'770 uv770 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.099+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc82e0 con 0x55891dbc3780 2024-09-10T10:54:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.101+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v134'771 uv771 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.101+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbc72d0 con 0x55891dbc3780 2024-09-10T10:54:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.102+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v134'772 uv772 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbc3780 2024-09-10T10:54:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.102+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55891dbc77e0 0x55891dbc9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.102+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbca110 con 0x55891dbc77e0 2024-09-10T10:54:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.103+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55891dbc77e0 0x55891dbc9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.103+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55891dbc77e0 0x55891dbc9bd0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.105+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v134'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.105+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbca4b0 con 0x55891dbc77e0 2024-09-10T10:54:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.106+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v134'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.106+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbca850 con 0x55891dbc77e0 2024-09-10T10:54:38.110 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:54:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.108+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v134'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.108+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcbc00 con 0x55891dbc77e0 2024-09-10T10:54:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.109+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v134'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.109+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcc010 con 0x55891dbc77e0 2024-09-10T10:54:38.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.111+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v134'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.111+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbb9490 con 0x55891dbc77e0 2024-09-10T10:54:38.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.112+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v134'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.112+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55891dbcb720 0x55891dbce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.112+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcea20 con 0x55891dbcb720 2024-09-10T10:54:38.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.112+0000 7f1a98800640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55891dbcb720 0x55891dbce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.113+0000 7f1a98800640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55891dbcb720 0x55891dbce410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.114+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v134'148 uv147 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.114+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcee60 con 0x55891dbcb720 2024-09-10T10:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.115+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v134'149 uv149 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.116+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcee60 con 0x55891dbcb720 2024-09-10T10:54:38.118 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:54:38.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.117+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v134'150 uv150 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.117+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcee60 con 0x55891dbcb720 2024-09-10T10:54:38.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.118+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v134'151 uv151 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.118+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcee60 con 0x55891dbc77e0 2024-09-10T10:54:38.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.119+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v134'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.119+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf250 con 0x55891dbc77e0 2024-09-10T10:54:38.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.120+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v134'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.120+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf250 con 0x55891dbc77e0 2024-09-10T10:54:38.123 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:54:38.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.121+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v134'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.121+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd0260 con 0x55891dbc77e0 2024-09-10T10:54:38.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.123+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v134'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.123+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd0260 con 0x55891dbc77e0 2024-09-10T10:54:38.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.124+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v134'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.124+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf250 con 0x55891dbc77e0 2024-09-10T10:54:38.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.125+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v134'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.125+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf250 con 0x55891dbcb720 2024-09-10T10:54:38.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.127+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v134'371 uv365 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.127+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf250 con 0x55891dbcb720 2024-09-10T10:54:38.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.128+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v134'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.128+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.131 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:54:38.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.129+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v134'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.129+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd0210 con 0x55891dbcb720 2024-09-10T10:54:38.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.131+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v134'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.131+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd0210 con 0x55891dbcb720 2024-09-10T10:54:38.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.132+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v134'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.132+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.134+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v134'376 uv376 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.134+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.135+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v134'377 uv370 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.135+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.136+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v134'378 uv378 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.136+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.139 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:54:38.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.138+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v134'379 uv379 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.138+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.139+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v134'380 uv380 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.139+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbcb720 2024-09-10T10:54:38.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.141+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v134'381 uv381 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.141+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbc77e0 2024-09-10T10:54:38.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.142+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v134'168 uv167 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.142+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbc77e0 2024-09-10T10:54:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.143+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v134'169 uv169 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.143+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbc77e0 2024-09-10T10:54:38.146 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:54:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.144+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v134'170 uv170 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.144+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbc77e0 2024-09-10T10:54:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.145+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v134'171 uv171 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.145+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbcf200 con 0x55891dbc77e0 2024-09-10T10:54:38.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.147+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v134'172 uv172 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbc77e0 2024-09-10T10:54:38.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.147+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55891dbcfb40 0x55891dbcff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.147+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2650 con 0x55891dbcfb40 2024-09-10T10:54:38.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.147+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55891dbcfb40 0x55891dbcff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.147+0000 7f1a97e00640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55891dbcfb40 0x55891dbcff60 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.149+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v134'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.149+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2650 con 0x55891dbcfb40 2024-09-10T10:54:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.150+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v134'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.150+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2650 con 0x55891dbcfb40 2024-09-10T10:54:38.153 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:54:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.151+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v134'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.151+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd3a50 con 0x55891dbcfb40 2024-09-10T10:54:38.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.153+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v134'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.153+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd3a50 con 0x55891dbcfb40 2024-09-10T10:54:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.154+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v134'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.154+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.156+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v134'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.156+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.157+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v134'335 uv329 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.157+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.159+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v134'336 uv336 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.159+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.162 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:54:38.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.160+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v134'337 uv337 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.160+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.162+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v134'338 uv338 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.162+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.163+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v134'339 uv339 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.163+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.164+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v134'340 uv334 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.164+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.165+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v134'341 uv341 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.165+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.167+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v134'342 uv342 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.169 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:54:38.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.167+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2a40 con 0x55891dbcfb40 2024-09-10T10:54:38.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.168+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v134'343 uv343 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.168+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd5850 con 0x55891dbcfb40 2024-09-10T10:54:38.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.170+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v134'344 uv344 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.170+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.171+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v134'147 uv145 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.171+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.172+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v134'148 uv148 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.172+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.175 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:54:38.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.174+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v134'149 uv149 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.174+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.175+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v134'150 uv150 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.175+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.177+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v134'201 uv200 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.177+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.178+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v134'202 uv202 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.178+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.181 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:54:38.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.179+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v134'203 uv203 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.179+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.180+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v134'204 uv204 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.180+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.181+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v134'205 uv205 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.181+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcb720 2024-09-10T10:54:38.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.182+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v134'206 uv206 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a9006a040 con 0x55891dbcb720 2024-09-10T10:54:38.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.182+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55891dbd37f0 0x55891dbd5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.182+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbd37f0 2024-09-10T10:54:38.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.183+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55891dbd37f0 0x55891dbd5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.183+0000 7f1a97400640 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55891dbd37f0 0x55891dbd5d90 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.184+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v134'171 uv170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbd37f0 2024-09-10T10:54:38.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.184+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbd37f0 2024-09-10T10:54:38.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.186+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v134'172 uv172 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbd37f0 2024-09-10T10:54:38.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.186+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbd37f0 2024-09-10T10:54:38.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.187+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v134'173 uv173 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbd37f0 2024-09-10T10:54:38.189 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:54:38.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.187+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbd37f0 2024-09-10T10:54:38.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.188+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v134'174 uv174 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbd37f0 2024-09-10T10:54:38.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.188+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbd37f0 2024-09-10T10:54:38.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.189+0000 7f1a97400640 1 -- 172.21.15.28:0/3075901664 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v134'175 uv175 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f1a8400da70 con 0x55891dbd37f0 2024-09-10T10:54:38.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.189+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.191+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v134'416 uv406 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.191+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.192+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v134'417 uv417 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.192+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.193+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v134'418 uv418 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.195 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:54:38.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.193+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.194+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v134'419 uv419 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.194+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.196+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v134'420 uv420 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.196+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.197+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v134'421 uv414 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.197+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.198+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v134'422 uv422 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.198+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.201 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:54:38.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.200+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v134'423 uv423 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.200+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.201+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v134'424 uv424 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.201+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.202+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v134'425 uv425 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.202+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd2ef0 con 0x55891dbcfb40 2024-09-10T10:54:38.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.203+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v134'426 uv426 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.203+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x55891dbd6780 con 0x55891dbcfb40 2024-09-10T10:54:38.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.204+0000 7f1a97e00640 1 -- 172.21.15.28:0/3075901664 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v134'427 uv427 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f1a88054570 con 0x55891dbcfb40 2024-09-10T10:54:38.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55891dbc77e0 msgr2=0x55891dbc9bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55891dbc77e0 0x55891dbc9bd0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55891dbcfb40 msgr2=0x55891dbcff60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55891dbcfb40 0x55891dbcff60 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55891dbd37f0 msgr2=0x55891dbd5d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55891dbd37f0 0x55891dbd5d90 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55891dbcb720 msgr2=0x55891dbce410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55891dbcb720 0x55891dbce410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 msgr2=0x55891dbc5b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 reap_dead start 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 0x55891dbc5b70 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a98800640 1 -- 172.21.15.28:0/3075901664 reap_dead start 2024-09-10T10:54:38.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 msgr2=0x55891dbb8ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 0x55891dbb8ed0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 msgr2=0x7f1a7806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 0x7f1a7806ccd0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55891db1e040 tx=0x7f1a84001710 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 msgr2=0x55891da89e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f1a88008400 tx=0x7f1a880084f0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 shutdown_connections 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55891dbc3780 0x55891dbc5b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55891db16a40 0x55891dbb8ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1a7806a810 0x7f1a7806ccd0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 --2- 172.21.15.28:0/3075901664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55891da90ff0 0x55891da89e10 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 >> 172.21.15.28:0/3075901664 conn(0x55891da950a0 msgr2=0x55891db1c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 shutdown_connections 2024-09-10T10:54:38.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.205+0000 7f1a9ab55f40 1 -- 172.21.15.28:0/3075901664 wait complete. 2024-09-10T10:54:38.211 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 10000000000.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:54:38.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.328+0000 7fd827f46f40 1 Processor -- start 2024-09-10T10:54:38.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.328+0000 7fd827f46f40 1 -- start start 2024-09-10T10:54:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 0x5640933fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x56409332fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5640932d8b30 con 0x7fd80c0022d0 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5640932dcb70 con 0x56409332f9b0 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564093408ee0 con 0x5640933fd590 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48262/0 (socket says 172.21.15.28:48262) 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 -- 172.21.15.28:0/2832048751 learned_addr learned my addr 172.21.15.28:0/2832048751 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825c00640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x56409332fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd81fe00640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 0x5640933fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 -- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 msgr2=0x5640933fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 0x5640933fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 -- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 msgr2=0x56409332fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x56409332fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 -- 172.21.15.28:0/2832048751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640933308e0 con 0x7fd80c0022d0 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd81fe00640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 0x5640933fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825c00640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x56409332fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd825200640 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd818000c00 tx=0x7fd818051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabd63b044c49d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd81ea00640 1 -- 172.21.15.28:0/2832048751 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818050880 con 0x7fd80c0022d0 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd81ea00640 1 -- 172.21.15.28:0/2832048751 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd818050a40 con 0x7fd80c0022d0 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd81ea00640 1 -- 172.21.15.28:0/2832048751 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818050e30 con 0x7fd80c0022d0 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- 172.21.15.28:0/2832048751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 msgr2=0x564093363030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd818000c00 tx=0x7fd818051000 comp rx=0 tx=0).stop 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- 172.21.15.28:0/2832048751 shutdown_connections 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x56409332fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5640933fd590 0x5640933fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 --2- 172.21.15.28:0/2832048751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd80c0022d0 0x564093363030 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- 172.21.15.28:0/2832048751 >> 172.21.15.28:0/2832048751 conn(0x56409337f0a0 msgr2=0x56409332f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- 172.21.15.28:0/2832048751 shutdown_connections 2024-09-10T10:54:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.330+0000 7fd827f46f40 1 -- 172.21.15.28:0/2832048751 wait complete. 2024-09-10T10:54:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.331+0000 7fd827f46f40 1 Processor -- start 2024-09-10T10:54:38.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.331+0000 7fd827f46f40 1 -- start start 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x5640933f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 0x56409340ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c0022d0 0x564093362fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564093362ba0 con 0x5640933fd590 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564093330600 con 0x56409332f9b0 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd827f46f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564093362550 con 0x7fd80c0022d0 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 0x56409340ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.332+0000 7fd81fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 0x56409340ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48264/0 (socket says 172.21.15.28:48264) 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 -- 172.21.15.28:0/3516110623 learned_addr learned my addr 172.21.15.28:0/3516110623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd825c00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c0022d0 0x564093362fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd825200640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x5640933f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 -- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c0022d0 msgr2=0x564093362fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd80c0022d0 0x564093362fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 -- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 msgr2=0x5640933f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56409332f9b0 0x5640933f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 -- 172.21.15.28:0/3516110623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56409340d170 con 0x5640933fd590 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81fe00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 0x56409340ab60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fd8100017c0 tx=0x7fd8100108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.333+0000 7fd81d600640 1 -- 172.21.15.28:0/3516110623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81000e8b0 con 0x5640933fd590 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.334+0000 7fd81d600640 1 -- 172.21.15.28:0/3516110623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd81000ea50 con 0x5640933fd590 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.334+0000 7fd81d600640 1 -- 172.21.15.28:0/3516110623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81000ee00 con 0x5640933fd590 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.334+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56409340c8d0 con 0x5640933fd590 2024-09-10T10:54:38.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.334+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56409340ce90 con 0x5640933fd590 2024-09-10T10:54:38.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.336+0000 7fd81d600640 1 -- 172.21.15.28:0/3516110623 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd810022440 con 0x5640933fd590 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.336+0000 7fd81d600640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd80406a810 0x7fd80406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.336+0000 7fd81d600640 1 -- 172.21.15.28:0/3516110623 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7fd81008b7a0 con 0x5640933fd590 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd825200640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd80406a810 0x7fd80406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd827f46f40 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564093406dd0 0x5640933ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e134) -- 0x5640933fff60 con 0x564093406dd0 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd825c00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564093406dd0 0x5640933ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd825200640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd80406a810 0x7fd80406ccd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd81805a000 tx=0x7fd818053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.338+0000 7fd825c00640 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564093406dd0 0x5640933ffa20 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.339+0000 7fd825c00640 1 -- 172.21.15.28:0/3516110623 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-header] v134'1064 uv1064 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fd8200016b0 con 0x564093406dd0 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564093406dd0 msgr2=0x5640933ffa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564093406dd0 0x5640933ffa20 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd80406a810 msgr2=0x7fd80406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd80406a810 0x7fd80406ccd0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd81805a000 tx=0x7fd818053000 comp rx=0 tx=0).stop 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 msgr2=0x56409340ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 --2- 172.21.15.28:0/3516110623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5640933fd590 0x56409340ab60 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fd8100017c0 tx=0x7fd8100108f0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd825c00640 1 -- 172.21.15.28:0/3516110623 reap_dead start 2024-09-10T10:54:38.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 shutdown_connections 2024-09-10T10:54:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 >> 172.21.15.28:0/3516110623 conn(0x56409337f0a0 msgr2=0x5640934076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 shutdown_connections 2024-09-10T10:54:38.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.340+0000 7fd827f46f40 1 -- 172.21.15.28:0/3516110623 wait complete. 2024-09-10T10:54:38.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:54:38.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.504+0000 7f96b3600640 1 Processor -- start 2024-09-10T10:54:38.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.504+0000 7f96b3600640 1 -- start start 2024-09-10T10:54:38.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b3600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0b5f80 0x7f96ac162980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b3600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96ac0d4c10 0x7f96ac0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b3600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96ac0d2600 con 0x7f96ac0d3320 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b3600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f96ac0d5420 con 0x7f96ac0d4c10 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b3600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac0d2e90 con 0x7f96ac0b5f80 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b1600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f96ac0d3320 0x7f96ac0ceb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44184/0 (socket says 172.21.15.28:44184) 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96abe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96ac0d4c10 0x7f96ac0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.505+0000 7f96b1600640 1 -- 172.21.15.28:0/1892109939 learned_addr learned my addr 172.21.15.28:0/1892109939 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96b0c00640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0b5f80 0x7f96ac162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96abe00640 1 -- 172.21.15.28:0/1892109939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0b5f80 msgr2=0x7f96ac162980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:38.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96abe00640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0b5f80 0x7f96ac162980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96abe00640 1 -- 172.21.15.28:0/1892109939 >> v1:172.21.15.28:6789/0 conn(0x7f96ac0d3320 legacy=0x7f96ac0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96abe00640 1 -- 172.21.15.28:0/1892109939 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0b3c10 con 0x7f96ac0d4c10 2024-09-10T10:54:38.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.506+0000 7f96ab400640 1 -- 172.21.15.28:0/1892109939 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209744470 0 0) 0x7f96a4001490 con 0x7f96ac0d3320 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.507+0000 7f96abe00640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0d4c10 0x7f96ac0b6360 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f96980097a0 tx=0x7f969802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe57014f8b0971e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.507+0000 7f96ab400640 1 -- 172.21.15.28:0/1892109939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698037070 con 0x7f96ac0d4c10 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.507+0000 7f96ab400640 1 -- 172.21.15.28:0/1892109939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9698032440 con 0x7f96ac0d4c10 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.507+0000 7f96ab400640 1 -- 172.21.15.28:0/1892109939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698032810 con 0x7f96ac0d4c10 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 -- 172.21.15.28:0/1892109939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0d4c10 msgr2=0x7f96ac0b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0d4c10 0x7f96ac0b6360 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f96980097a0 tx=0x7f969802a180 comp rx=0 tx=0).stop 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 -- 172.21.15.28:0/1892109939 shutdown_connections 2024-09-10T10:54:38.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0d4c10 0x7f96ac0b6360 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 --2- 172.21.15.28:0/1892109939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0b5f80 0x7f96ac162980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 -- 172.21.15.28:0/1892109939 >> 172.21.15.28:0/1892109939 conn(0x7f96ac08bfc0 msgr2=0x7f96ac0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 -- 172.21.15.28:0/1892109939 shutdown_connections 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.508+0000 7f96b3600640 1 -- 172.21.15.28:0/1892109939 wait complete. 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.509+0000 7f96b3600640 1 Processor -- start 2024-09-10T10:54:38.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 -- start start 2024-09-10T10:54:38.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0d3320 0x7f96ac0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ac0d4c10 0x7f96ac15b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0d5140 con 0x7f96ac0b5f80 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac0d2880 con 0x7f96ac0d4c10 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.510+0000 7f96b3600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac0d2b20 con 0x7f96ac0d3320 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48286/0 (socket says 172.21.15.28:48286) 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 -- 172.21.15.28:0/3859634601 learned_addr learned my addr 172.21.15.28:0/3859634601 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96abe00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0d3320 0x7f96ac0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b1600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ac0d4c10 0x7f96ac15b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 -- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0d3320 msgr2=0x7f96ac0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0d3320 0x7f96ac0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 -- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ac0d4c10 msgr2=0x7f96ac15b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ac0d4c10 0x7f96ac15b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 -- 172.21.15.28:0/3859634601 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0b3470 con 0x7f96ac0b5f80 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.511+0000 7f96b0c00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f96a000e9b0 tx=0x7f96a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.512+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96ac0b5f80 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.512+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a000c9a0 con 0x7f96ac0b5f80 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.512+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a000cd90 con 0x7f96ac0b5f80 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.512+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac0b7580 con 0x7f96ac0b5f80 2024-09-10T10:54:38.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.512+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac0b20a0 con 0x7f96ac0b5f80 2024-09-10T10:54:38.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.513+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ac0b6360 con 0x7f96ac0b5f80 2024-09-10T10:54:38.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.514+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f96a001a070 con 0x7f96ac0b5f80 2024-09-10T10:54:38.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.515+0000 7f96a8c00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 0x7f968c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:38.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.515+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f96a008bfc0 con 0x7f96ac0b5f80 2024-09-10T10:54:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.515+0000 7f96abe00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 0x7f968c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:38.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.516+0000 7f96abe00640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 0x7f968c06e320 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9698034000 tx=0x7f969802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:38.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.519+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f96a0059b90 con 0x7f96ac0b5f80 2024-09-10T10:54:38.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.741+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f96ac0c7bb0 con 0x7f96ac0b5f80 2024-09-10T10:54:38.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.858+0000 7f96a8c00640 1 -- 172.21.15.28:0/3859634601 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v292) ==== 162+0+0 (secure 0 0 0) 0x7f96a0059490 con 0x7f96ac0b5f80 2024-09-10T10:54:38.860 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 msgr2=0x7f968c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 0x7f968c06e320 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9698034000 tx=0x7f969802e000 comp rx=0 tx=0).stop 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 msgr2=0x7f96ac1f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f96a000e9b0 tx=0x7f96a000ee70 comp rx=0 tx=0).stop 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 shutdown_connections 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f968c06be60 0x7f968c06e320 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96ac0d4c10 0x7f96ac15b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f96ac0d3320 0x7f96ac0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 --2- 172.21.15.28:0/3859634601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96ac0b5f80 0x7f96ac1f4380 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 >> 172.21.15.28:0/3859634601 conn(0x7f96ac08bfc0 msgr2=0x7f96ac0d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 shutdown_connections 2024-09-10T10:54:38.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:38.861+0000 7f96b3600640 1 -- 172.21.15.28:0/3859634601 wait complete. 2024-09-10T10:54:39.284 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:39.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.440+0000 7f19c2600640 1 Processor -- start 2024-09-10T10:54:39.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.440+0000 7f19c2600640 1 -- start start 2024-09-10T10:54:39.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19c2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19bc0b5f80 0x7f19bc0b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19c2600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc06fdb0 con 0x7f19bc0cd650 2024-09-10T10:54:39.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19c2600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19bc070090 con 0x7f19bc0afd70 2024-09-10T10:54:39.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19c2600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19bc0b6910 con 0x7f19bc0b5f80 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19bbe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f19bc0cd650 0x7f19bc0cdab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44190/0 (socket says 172.21.15.28:44190) 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19bbe00640 1 -- 172.21.15.28:0/988493299 learned_addr learned my addr 172.21.15.28:0/988493299 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.441+0000 7f19baa00640 1 --2- 172.21.15.28:0/988493299 >> v2:172.21.15.138:3300/0 conn(0x7f19bc0b5f80 0x7f19bc0b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807150315 0 0) 0x7f19bc06fdb0 con 0x7f19bc0cd650 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19a0003520 con 0x7f19bc0cd650 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185732721 0 0) 0x7f19bc070090 con 0x7f19bc0afd70 2024-09-10T10:54:39.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19bc06fdb0 con 0x7f19bc0afd70 2024-09-10T10:54:39.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19baa00640 1 -- 172.21.15.28:0/988493299 >> v1:172.21.15.138:6789/0 conn(0x7f19bc0afd70 legacy=0x7f19bc160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19baa00640 1 -- 172.21.15.28:0/988493299 >> v1:172.21.15.28:6789/0 conn(0x7f19bc0cd650 legacy=0x7f19bc0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.442+0000 7f19baa00640 1 -- 172.21.15.28:0/988493299 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc070370 con 0x7f19bc0b5f80 2024-09-10T10:54:39.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.443+0000 7f19baa00640 1 --2- 172.21.15.28:0/988493299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0b5f80 0x7f19bc0b63d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f19a80097a0 tx=0x7f19a802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffe695bd774b19e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:39.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.443+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8037070 con 0x7f19bc0b5f80 2024-09-10T10:54:39.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.443+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a8032440 con 0x7f19bc0b5f80 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.443+0000 7f19ba000640 1 -- 172.21.15.28:0/988493299 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8032810 con 0x7f19bc0b5f80 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 -- 172.21.15.28:0/988493299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0b5f80 msgr2=0x7f19bc0b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 --2- 172.21.15.28:0/988493299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0b5f80 0x7f19bc0b63d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f19a80097a0 tx=0x7f19a802a230 comp rx=0 tx=0).stop 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 -- 172.21.15.28:0/988493299 shutdown_connections 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 --2- 172.21.15.28:0/988493299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0b5f80 0x7f19bc0b63d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 -- 172.21.15.28:0/988493299 >> 172.21.15.28:0/988493299 conn(0x7f19bc06bc10 msgr2=0x7f19bc06c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 -- 172.21.15.28:0/988493299 shutdown_connections 2024-09-10T10:54:39.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.444+0000 7f19c2600640 1 -- 172.21.15.28:0/988493299 wait complete. 2024-09-10T10:54:39.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.446+0000 7f19c2600640 1 Processor -- start 2024-09-10T10:54:39.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.446+0000 7f19c2600640 1 -- start start 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19bc0afd70 0x7f19bc1f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0cd650 0x7f19bc1e5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc0b0150 con 0x7f19bc0b5f80 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc06fa80 con 0x7f19bc0cd650 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19c2600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc160060 con 0x7f19bc0afd70 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19baa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19baa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48294/0 (socket says 172.21.15.28:48294) 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19baa00640 1 -- 172.21.15.28:0/1981957748 learned_addr learned my addr 172.21.15.28:0/1981957748 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19bb400640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19bc0afd70 0x7f19bc1f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.447+0000 7f19bbe00640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0cd650 0x7f19bc1e5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 -- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19bc0afd70 msgr2=0x7f19bc1f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19bc0afd70 0x7f19bc1f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 -- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0cd650 msgr2=0x7f19bc1e5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0cd650 0x7f19bc1e5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 -- 172.21.15.28:0/1981957748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc1f53f0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19baa00640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f19a8035040 tx=0x7f19a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8041030 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a803d4e0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19bc1500c0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.448+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a8037040 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.449+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19bc150680 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.450+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19bc0b63d0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.451+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f19a8044070 con 0x7f19bc0b5f80 2024-09-10T10:54:39.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.452+0000 7f19b3400640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 0x7f199c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:39.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.452+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f19a80ac2f0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.452+0000 7f19bb400640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 0x7f199c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:39.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.453+0000 7f19bb400640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 0x7f199c06e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0cd350 tx=0x7f19ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:39.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.457+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f19a8044e40 con 0x7f19bc0b5f80 2024-09-10T10:54:39.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.678+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19bc067fc0 con 0x7f19bc0b5f80 2024-09-10T10:54:39.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.679+0000 7f19b3400640 1 -- 172.21.15.28:0/1981957748 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 293 v293) ==== 95+0+5224 (secure 0 0 0) 0x7f19a8079e40 con 0x7f19bc0b5f80 2024-09-10T10:54:39.680 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:39.680 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":293,"btime":"2024-09-10T10:54:38:862333+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":293,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:38.862320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9606":{"gid":9606,"name":"c","rank":0,"incarnation":293,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:39.680 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 293 2024-09-10T10:54:39.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 msgr2=0x7f199c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 0x7f199c06e2f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0cd350 tx=0x7f19ac009040 comp rx=0 tx=0).stop 2024-09-10T10:54:39.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 msgr2=0x7f19bc1f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:39.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f19a8035040 tx=0x7f19a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 shutdown_connections 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f199c06be30 0x7f199c06e2f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19bc0cd650 0x7f19bc1e5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0b5f80 0x7f19bc1f4eb0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 --2- 172.21.15.28:0/1981957748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19bc0afd70 0x7f19bc1f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 >> 172.21.15.28:0/1981957748 conn(0x7f19bc06bc10 msgr2=0x7f19bc06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 shutdown_connections 2024-09-10T10:54:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:39.681+0000 7f19c2600640 1 -- 172.21.15.28:0/1981957748 wait complete. 2024-09-10T10:54:39.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 293, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:54:38.862320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 134, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9606': {'gid': 9606, 'name': 'c', 'rank': 0, 'incarnation': 293, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6834/2572217302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 2572217302}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 2572217302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:54:39.698 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9606:up:replay 2024-09-10T10:54:39.698 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:54:40.768 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:54:40.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:54:43.828 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:54:43.827+0000 7f77b7c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:45.103 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.269+0000 7f0709800640 1 Processor -- start 2024-09-10T10:54:45.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.269+0000 7f0709800640 1 -- start start 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0709800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f070415e660 0x7f070406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0709800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d1f90 0x7f07040cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0709800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07040cf000 con 0x7f07040d0dd0 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0709800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f07040b37d0 con 0x7f070415e660 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0709800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07040cf330 con 0x7f07040d1f90 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.270+0000 7f0703400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f070415e660 0x7f070406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f070415e660 0x7f070406ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49190/0 (socket says 172.21.15.28:49190) 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 -- 172.21.15.28:0/1275437146 learned_addr learned my addr 172.21.15.28:0/1275437146 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0708e00640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d1f90 0x7f07040cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 -- 172.21.15.28:0/1275437146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d1f90 msgr2=0x7f07040cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d1f90 0x7f07040cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 -- 172.21.15.28:0/1275437146 >> v1:172.21.15.138:6789/0 conn(0x7f07040d0dd0 legacy=0x7f07040d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.271+0000 7f0703400640 1 -- 172.21.15.28:0/1275437146 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07040d29a0 con 0x7f070415e660 2024-09-10T10:54:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.272+0000 7f0703400640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f070406ccd0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009870 tx=0x7f06f002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608ddd3a01384c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.272+0000 7f0702a00640 1 -- 172.21.15.28:0/1275437146 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0037070 con 0x7f070415e660 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.272+0000 7f0702a00640 1 -- 172.21.15.28:0/1275437146 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f0032440 con 0x7f070415e660 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.272+0000 7f0702a00640 1 -- 172.21.15.28:0/1275437146 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f0032810 con 0x7f070415e660 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 -- 172.21.15.28:0/1275437146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 msgr2=0x7f070406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f070406ccd0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009870 tx=0x7f06f002a250 comp rx=0 tx=0).stop 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 -- 172.21.15.28:0/1275437146 shutdown_connections 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d1f90 0x7f07040cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 --2- 172.21.15.28:0/1275437146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f070406ccd0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 -- 172.21.15.28:0/1275437146 >> 172.21.15.28:0/1275437146 conn(0x7f070408bfc0 msgr2=0x7f07040d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 -- 172.21.15.28:0/1275437146 shutdown_connections 2024-09-10T10:54:45.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.273+0000 7f0709800640 1 -- 172.21.15.28:0/1275437146 wait complete. 2024-09-10T10:54:45.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.275+0000 7f0709800640 1 Processor -- start 2024-09-10T10:54:45.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.275+0000 7f0709800640 1 -- start start 2024-09-10T10:54:45.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.275+0000 7f0709800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d0dd0 0x7f07041e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0709800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07040d1f90 0x7f0704153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0709800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0709800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07040d3220 con 0x7f070415e660 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0709800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07040d2f40 con 0x7f07040d1f90 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0709800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f07040d33a0 con 0x7f07040d0dd0 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49200/0 (socket says 172.21.15.28:49200) 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 -- 172.21.15.28:0/807513271 learned_addr learned my addr 172.21.15.28:0/807513271 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0703400640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07040d1f90 0x7f0704153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0703e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d0dd0 0x7f07041e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 -- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d0dd0 msgr2=0x7f07041e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d0dd0 0x7f07041e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 -- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07040d1f90 msgr2=0x7f0704153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07040d1f90 0x7f0704153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.276+0000 7f0708e00640 1 -- 172.21.15.28:0/807513271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07041e4ff0 con 0x7f070415e660 2024-09-10T10:54:45.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f0708e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc018020 con 0x7f070415e660 2024-09-10T10:54:45.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc014480 con 0x7f070415e660 2024-09-10T10:54:45.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc014870 con 0x7f070415e660 2024-09-10T10:54:45.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07040b8840 con 0x7f070415e660 2024-09-10T10:54:45.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.277+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07040b8e00 con 0x7f070415e660 2024-09-10T10:54:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.278+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07040d2440 con 0x7f070415e660 2024-09-10T10:54:45.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.279+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f06fc005d10 con 0x7f070415e660 2024-09-10T10:54:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.280+0000 7f06fbe00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 0x7f06e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.280+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f06fc08c870 con 0x7f070415e660 2024-09-10T10:54:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.280+0000 7f0703e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 0x7f06e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.281+0000 7f0703e00640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 0x7f06e406e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f06f4001640 tx=0x7f06f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.284+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06fc022070 con 0x7f070415e660 2024-09-10T10:54:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.506+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07040d1b20 con 0x7f070415e660 2024-09-10T10:54:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.506+0000 7f06fbe00640 1 -- 172.21.15.28:0/807513271 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 296 v296) ==== 95+0+5232 (secure 0 0 0) 0x7f06fc010040 con 0x7f070415e660 2024-09-10T10:54:45.507 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:45.508 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":296,"btime":"2024-09-10T10:54:41:872892+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":296,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:41.872883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":134,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9606},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9606":{"gid":9606,"name":"c","rank":0,"incarnation":293,"state":"up:active","state_seq":15,"addr":"172.21.15.28:6834/2572217302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2572217302},{"type":"v1","addr":"172.21.15.28:6834","nonce":2572217302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9606,"qdb_cluster":[9606]},"id":6}]} 2024-09-10T10:54:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 296 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 msgr2=0x7f06e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 0x7f06e406e2f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f06f4001640 tx=0x7f06f4009040 comp rx=0 tx=0).stop 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 msgr2=0x7f0704155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c490 comp rx=0 tx=0).stop 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 shutdown_connections 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f06e406be30 0x7f06e406e2f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f070415e660 0x7f0704155320 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07040d1f90 0x7f0704153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 --2- 172.21.15.28:0/807513271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f07040d0dd0 0x7f07041e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 >> 172.21.15.28:0/807513271 conn(0x7f070408bfc0 msgr2=0x7f0704157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 shutdown_connections 2024-09-10T10:54:45.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.509+0000 7f0709800640 1 -- 172.21.15.28:0/807513271 wait complete. 2024-09-10T10:54:45.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 296, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:54:41.872883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 134, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9606}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9606': {'gid': 9606, 'name': 'c', 'rank': 0, 'incarnation': 293, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.28:6834/2572217302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 2572217302}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 2572217302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9606, 'qdb_cluster': [9606]} 2024-09-10T10:54:45.525 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:54:45.526 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:54:45.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.682+0000 7fedb2800640 1 Processor -- start 2024-09-10T10:54:45.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.682+0000 7fedb2800640 1 -- start start 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fedac15e6a0 0x7fedac06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac06dd60 0x7fedac0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb2800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedac0d2c20 con 0x7fedac0d4180 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb2800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fedac0cefb0 con 0x7fedac15e6a0 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb2800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fedac06e390 con 0x7fedac06dd60 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb1400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fedac0d4180 0x7fedac0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59286/0 (socket says 172.21.15.28:59286) 2024-09-10T10:54:45.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb1400640 1 -- 172.21.15.28:0/3122080121 learned_addr learned my addr 172.21.15.28:0/3122080121 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:45.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.683+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3122080121 >> v2:172.21.15.28:3300/0 conn(0x7fedac15e6a0 0x7fedac06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.684+0000 7fedb1e00640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac06dd60 0x7fedac0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.684+0000 7fedb0a00640 1 -- 172.21.15.28:0/3122080121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac06dd60 msgr2=0x7fedac0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.684+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac06dd60 0x7fedac0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.684+0000 7fedb0a00640 1 -- 172.21.15.28:0/3122080121 >> v1:172.21.15.28:6789/0 conn(0x7fedac0d4180 legacy=0x7fedac0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.684+0000 7fedb0a00640 1 -- 172.21.15.28:0/3122080121 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac06e670 con 0x7fedac15e6a0 2024-09-10T10:54:45.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac15e6a0 0x7fedac06d820 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e07a047d56275 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedabe00640 1 -- 172.21.15.28:0/3122080121 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98037070 con 0x7fedac15e6a0 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedabe00640 1 -- 172.21.15.28:0/3122080121 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9802ae40 con 0x7fedac15e6a0 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedabe00640 1 -- 172.21.15.28:0/3122080121 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98032660 con 0x7fedac15e6a0 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedb2800640 1 -- 172.21.15.28:0/3122080121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac15e6a0 msgr2=0x7fedac06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.685+0000 7fedb2800640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac15e6a0 0x7fedac06d820 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802a0f0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 -- 172.21.15.28:0/3122080121 shutdown_connections 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac06dd60 0x7fedac0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 --2- 172.21.15.28:0/3122080121 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac15e6a0 0x7fedac06d820 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 -- 172.21.15.28:0/3122080121 >> 172.21.15.28:0/3122080121 conn(0x7fedac09d6f0 msgr2=0x7fedac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 -- 172.21.15.28:0/3122080121 shutdown_connections 2024-09-10T10:54:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.686+0000 7fedb2800640 1 -- 172.21.15.28:0/3122080121 wait complete. 2024-09-10T10:54:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.687+0000 7fedb2800640 1 Processor -- start 2024-09-10T10:54:45.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.687+0000 7fedb2800640 1 -- start start 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 0x7fedac1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac15e6a0 0x7fedac0b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0cf980 con 0x7fedac0d4180 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedac0cebd0 con 0x7fedac06dd60 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb2800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fedac0cfb00 con 0x7fedac15e6a0 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.688+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49222/0 (socket says 172.21.15.28:49222) 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 -- 172.21.15.28:0/958463389 learned_addr learned my addr 172.21.15.28:0/958463389 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb1400640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 0x7fedac1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb1e00640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac15e6a0 0x7fedac0b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 -- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac15e6a0 msgr2=0x7fedac0b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac15e6a0 0x7fedac0b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 -- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 msgr2=0x7fedac1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 0x7fedac1576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 -- 172.21.15.28:0/958463389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedac0b4bf0 con 0x7fedac0d4180 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb1400640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 0x7fedac1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:45.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7fedb0a00640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.689+0000 7feda9600640 1 -- 172.21.15.28:0/958463389 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98041070 con 0x7fedac0d4180 2024-09-10T10:54:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.690+0000 7feda9600640 1 -- 172.21.15.28:0/958463389 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9803c670 con 0x7fedac0d4180 2024-09-10T10:54:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.690+0000 7feda9600640 1 -- 172.21.15.28:0/958463389 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98037040 con 0x7fedac0d4180 2024-09-10T10:54:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.690+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedac0cc020 con 0x7fedac0d4180 2024-09-10T10:54:45.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.690+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedac0cc5e0 con 0x7fedac0d4180 2024-09-10T10:54:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.692+0000 7feda9600640 1 -- 172.21.15.28:0/958463389 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fed98044070 con 0x7fedac0d4180 2024-09-10T10:54:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.693+0000 7feda9600640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.693+0000 7feda9600640 1 -- 172.21.15.28:0/958463389 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7fed980ac2c0 con 0x7fedac0d4180 2024-09-10T10:54:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.693+0000 7fedb1400640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.694+0000 7fedb6de5740 1 Processor -- start 2024-09-10T10:54:45.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.694+0000 7fedb6de5740 1 -- start start 2024-09-10T10:54:45.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.694+0000 7fedb1400640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 0x7fed8c06e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7feda0000f60 tx=0x7feda0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f1660 0x55f4a74f4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f5630 0x55f4a74ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f4a74523b0 con 0x55f4a74f3ea0 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7457410 con 0x55f4a74f5630 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f4a73ff620 con 0x55f4a74f1660 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49224/0 (socket says 172.21.15.28:49224) 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 -- 172.21.15.28:0/1936267083 learned_addr learned my addr 172.21.15.28:0/1936267083 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1e00640 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f5630 0x55f4a74ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb0a00640 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f1660 0x55f4a74f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 -- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f1660 msgr2=0x55f4a74f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f1660 0x55f4a74f4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 -- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f5630 msgr2=0x55f4a74ee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f5630 0x55f4a74ee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.695+0000 7fedb1400640 1 -- 172.21.15.28:0/1936267083 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a716a2d0 con 0x55f4a74f3ea0 2024-09-10T10:54:45.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fedb1400640 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7feda0005910 tx=0x7feda0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66aeefd2334b7a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fed9dc00640 1 -- 172.21.15.28:0/1936267083 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000e7e0 con 0x55f4a74f3ea0 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fed9dc00640 1 -- 172.21.15.28:0/1936267083 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda000e980 con 0x55f4a74f3ea0 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fed9dc00640 1 -- 172.21.15.28:0/1936267083 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000ed30 con 0x55f4a74f3ea0 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fedb6de5740 1 -- 172.21.15.28:0/1936267083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 msgr2=0x55f4a7539640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.696+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7feda0005910 tx=0x7feda0011000 comp rx=0 tx=0).stop 2024-09-10T10:54:45.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 -- 172.21.15.28:0/1936267083 shutdown_connections 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f5630 0x55f4a74ee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f1660 0x55f4a74f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1936267083 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f3ea0 0x55f4a7539640 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 -- 172.21.15.28:0/1936267083 >> 172.21.15.28:0/1936267083 conn(0x55f4a74f08f0 msgr2=0x55f4a7538b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 -- 172.21.15.28:0/1936267083 shutdown_connections 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 -- 172.21.15.28:0/1936267083 wait complete. 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.697+0000 7fedb6de5740 1 -- start start 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.698+0000 7fedb6de5740 1 Processor -- start 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f3ea0 0x55f4a75361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a751cd20 0x55f4a7519e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7412780 con 0x55f4a74f5630 2024-09-10T10:54:45.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a74533d0 con 0x55f4a751cd20 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f4a74efd90 con 0x55f4a74f3ea0 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb6de5740 10 client.0 _ll_register_callbacks cb 0x55f4a74505d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49234/0 (socket says 172.21.15.28:49234) 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 -- 172.21.15.28:0/1085393945 learned_addr learned my addr 172.21.15.28:0/1085393945 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 -- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f3ea0 msgr2=0x55f4a75361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f3ea0 0x55f4a75361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 -- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a751cd20 msgr2=0x55f4a7519e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a751cd20 0x55f4a7519e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 -- 172.21.15.28:0/1085393945 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a72346b0 con 0x55f4a74f5630 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fedb0a00640 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fed980a99e0 tx=0x7fed98079170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596b62c94c990d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.699+0000 7fed92000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9804b040 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7fed980820b0 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda000e980 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=296) ==== 24+0+0 (secure 0 0 0) 0x7fed98079810 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 10 client.9921 fetch_fsmap learned FSMap version 296 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=296,monmap=2+}) -- 0x55f4a751c240 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda000e980 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 296) ==== 3502+0+0 (secure 0 0 0) 0x7fed98078c60 con 0x55f4a74f5630 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 10 client.9921 fetch_fsmap finished waiting for FSMap version 296 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 10 client.9921 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 20 client.9921 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:45.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7549880 0x55f4a74edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 4 client.9921 mds_command: new command op to 9606 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.700+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 --> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a7464b40 con 0x55f4a7549880 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.701+0000 7fedb1400640 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7549880 0x55f4a74edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.701+0000 7fedb1400640 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7549880 0x55f4a74edc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ea1dc05ae82a4d server_cookie=a5d7266508d8fc4 in_seq=0 out_seq=0 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.701+0000 7fed92000640 10 client.9921 ms_handle_connect on v2:172.21.15.28:6832/2572217302 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fed92000640 1 -- 172.21.15.28:0/1085393945 <== mds.0 v2:172.21.15.28:6832/2572217302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7feda002ce80 con 0x55f4a7549880 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fed92000640 10 client.9921 handle_command_reply: tid=0 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 client.9921 shutdown 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 msgr2=0x55f4a74ef030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fed980a99e0 tx=0x7fed98079170 comp rx=0 tx=0).stop 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 shutdown_connections 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7549880 0x55f4a74edc30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a751cd20 0x55f4a7519e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a74ef030 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fed92000640 0 client.9921 ms_handle_reset on v2:172.21.15.28:6832/2572217302 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1085393945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f3ea0 0x55f4a75361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 >> 172.21.15.28:0/1085393945 conn(0x55f4a74f1660 msgr2=0x55f4a7538070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 shutdown_connections 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.702+0000 7fedb6de5740 1 -- 172.21.15.28:0/1085393945 wait complete. 2024-09-10T10:54:45.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.703+0000 7fedb6de5740 20 client.9921 trim_cache size 0 max 16384 2024-09-10T10:54:45.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 Processor -- start 2024-09-10T10:54:45.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 -- start start 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a74ee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f5630 0x55f4a74f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedac201e70 con 0x55f4a751cd20 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedac201170 con 0x55f4a74f3ea0 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.722+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fedac2017c0 con 0x55f4a74f5630 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49246/0 (socket says 172.21.15.28:49246) 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 -- 172.21.15.28:0/1108126200 learned_addr learned my addr 172.21.15.28:0/1108126200 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1400640 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a74ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb0a00640 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f5630 0x55f4a74f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 -- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f5630 msgr2=0x55f4a74f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f5630 0x55f4a74f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 -- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 msgr2=0x55f4a74ee150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a74ee150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.723+0000 7fedb1e00640 1 -- 172.21.15.28:0/1108126200 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a750aa90 con 0x55f4a751cd20 2024-09-10T10:54:45.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fedb1e00640 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7feda4003990 tx=0x7feda4003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491686d5a7838bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fed9d200640 1 -- 172.21.15.28:0/1108126200 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda401b020 con 0x55f4a751cd20 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fed9d200640 1 -- 172.21.15.28:0/1108126200 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda40047f0 con 0x55f4a751cd20 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fed9d200640 1 -- 172.21.15.28:0/1108126200 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda40176b0 con 0x55f4a751cd20 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fedb6de5740 1 -- 172.21.15.28:0/1108126200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 msgr2=0x55f4a750a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.724+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7feda4003990 tx=0x7feda4003e50 comp rx=0 tx=0).stop 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 -- 172.21.15.28:0/1108126200 shutdown_connections 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a751cd20 0x55f4a750a550 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a74f5630 0x55f4a74f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 --2- 172.21.15.28:0/1108126200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a74ee150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 -- 172.21.15.28:0/1108126200 >> 172.21.15.28:0/1108126200 conn(0x55f4a74f1660 msgr2=0x55f4a7535cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 -- 172.21.15.28:0/1108126200 shutdown_connections 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 -- 172.21.15.28:0/1108126200 wait complete. 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 -- start start 2024-09-10T10:54:45.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.725+0000 7fedb6de5740 1 Processor -- start 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a75b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a751cd20 0x55f4a7513a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7549700 con 0x55f4a74f5630 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f4a751c520 con 0x55f4a74f3ea0 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f4a7549d50 con 0x55f4a751cd20 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.726+0000 7fedb6de5740 10 client.0 _ll_register_callbacks cb 0x7fed98032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:54:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49258/0 (socket says 172.21.15.28:49258) 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 -- 172.21.15.28:0/3876816416 learned_addr learned my addr 172.21.15.28:0/3876816416 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb1400640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a75b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb1e00640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a751cd20 0x55f4a7513a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 -- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a751cd20 msgr2=0x55f4a7513a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a751cd20 0x55f4a7513a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 -- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 msgr2=0x55f4a75b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a75b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 -- 172.21.15.28:0/3876816416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a750a5f0 con 0x55f4a74f5630 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fedb0a00640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fed980a9a30 tx=0x7fed98047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db361b380d9ca90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.727+0000 7fed92000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9804b040 con 0x55f4a74f5630 2024-09-10T10:54:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7fed980786d0 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda40047f0 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=296) ==== 24+0+0 (secure 0 0 0) 0x7fed98006e10 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 10 client.9927 fetch_fsmap learned FSMap version 296 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=296,monmap=2+}) -- 0x55f4a7514310 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda40047f0 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 296) ==== 3502+0+0 (secure 0 0 0) 0x7fed980788d0 con 0x55f4a74f5630 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 10 client.9927 fetch_fsmap finished waiting for FSMap version 296 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 10 client.9927 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 20 client.9927 populate_metadata read hostname 'smithi028' 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7537030 0x55f4a75172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 4 client.9927 mds_command: new command op to 9606 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:54:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 --> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] -- command(tid 0: {"prefix": "status"}) -- 0x55f4a745ee90 con 0x55f4a7537030 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.728+0000 7fedb1400640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7537030 0x55f4a75172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb1400640 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7537030 0x55f4a75172c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0cd4e8fea8fcc4b server_cookie=77cab60cdbc309e1 in_seq=0 out_seq=0 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fed92000640 10 client.9927 ms_handle_connect on v2:172.21.15.28:6832/2572217302 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fed92000640 1 -- 172.21.15.28:0/3876816416 <== mds.0 v2:172.21.15.28:6832/2572217302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55f4a745ee90 con 0x55f4a7537030 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fed92000640 10 client.9927 handle_command_reply: tid=0 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 client.9927 shutdown 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 msgr2=0x55f4a751dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fed980a9a30 tx=0x7fed98047000 comp rx=0 tx=0).stop 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 shutdown_connections 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:6832/2572217302,v1:172.21.15.28:6834/2572217302] conn(0x55f4a7537030 0x55f4a75172c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f4a751cd20 0x55f4a7513a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f4a74f5630 0x55f4a751dac0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 --2- 172.21.15.28:0/3876816416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f4a74f3ea0 0x55f4a75b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fed92000640 0 client.9927 ms_handle_reset on v2:172.21.15.28:6832/2572217302 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.729+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 >> 172.21.15.28:0/3876816416 conn(0x55f4a74f1660 msgr2=0x55f4a74f2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 shutdown_connections 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb6de5740 1 -- 172.21.15.28:0/3876816416 wait complete. 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb6de5740 20 client.9927 trim_cache size 0 max 16384 2024-09-10T10:54:45.735 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 9606, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8657603119999999, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 296, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 134, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 134, 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 52.137707126999999 2024-09-10T10:54:45.736 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:54:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 msgr2=0x7fed8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 0x7fed8c06e2f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7feda0000f60 tx=0x7feda0009040 comp rx=0 tx=0).stop 2024-09-10T10:54:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 msgr2=0x7fedac0b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fed98000c00 tx=0x7fed980046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 shutdown_connections 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fed8c06be30 0x7fed8c06e2f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fedac15e6a0 0x7fedac0b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedac0d4180 0x7fedac0b3b50 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 --2- 172.21.15.28:0/958463389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedac06dd60 0x7fedac1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.730+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 >> 172.21.15.28:0/958463389 conn(0x7fedac09d6f0 msgr2=0x7fedac0b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.731+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 shutdown_connections 2024-09-10T10:54:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.731+0000 7fedb2800640 1 -- 172.21.15.28:0/958463389 wait complete. 2024-09-10T10:54:45.747 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:54:45.747 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on 10000000000.00000000', proceeding to ls 2024-09-10T10:54:45.747 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:54:45.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.903+0000 7ff10ca00640 1 Processor -- start 2024-09-10T10:54:45.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.903+0000 7ff10ca00640 1 -- start start 2024-09-10T10:54:45.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff10ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff108072320 0x7ff1080cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff10ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff108072de0 0x7ff10806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff10ca00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff108146140 con 0x7ff108072700 2024-09-10T10:54:45.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff10ca00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff1080cd740 con 0x7ff108072de0 2024-09-10T10:54:45.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff10ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff1080cda20 con 0x7ff108072320 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff106800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff108072700 0x7ff10806e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59292/0 (socket says 172.21.15.28:59292) 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff106800640 1 -- 172.21.15.28:0/3447389453 learned_addr learned my addr 172.21.15.28:0/3447389453 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/3447389453 >> v2:172.21.15.28:3300/0 conn(0x7ff108072de0 0x7ff10806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff105e00640 1 --2- 172.21.15.28:0/3447389453 >> v2:172.21.15.138:3300/0 conn(0x7ff108072320 0x7ff1080cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.905+0000 7ff105400640 1 -- 172.21.15.28:0/3447389453 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795177532 0 0) 0x7ff108146140 con 0x7ff108072700 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.906+0000 7ff105400640 1 -- 172.21.15.28:0/3447389453 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ec003520 con 0x7ff108072700 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.906+0000 7ff0ffe00640 1 -- 172.21.15.28:0/3447389453 >> v2:172.21.15.138:3300/0 conn(0x7ff108072320 msgr2=0x7ff1080cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.906+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/3447389453 >> v2:172.21.15.138:3300/0 conn(0x7ff108072320 0x7ff1080cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.906+0000 7ff0ffe00640 1 -- 172.21.15.28:0/3447389453 >> v1:172.21.15.28:6789/0 conn(0x7ff108072700 legacy=0x7ff10806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.906+0000 7ff0ffe00640 1 -- 172.21.15.28:0/3447389453 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080cdd00 con 0x7ff108072de0 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/3447389453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072de0 0x7ff10806e0b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff1080b31e0 tx=0x7ff0f002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1788068fe8adef6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff105400640 1 -- 172.21.15.28:0/3447389453 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0037070 con 0x7ff108072de0 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff105400640 1 -- 172.21.15.28:0/3447389453 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f0032440 con 0x7ff108072de0 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff105400640 1 -- 172.21.15.28:0/3447389453 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f002ae60 con 0x7ff108072de0 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff10ca00640 1 -- 172.21.15.28:0/3447389453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072de0 msgr2=0x7ff10806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.907+0000 7ff10ca00640 1 --2- 172.21.15.28:0/3447389453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072de0 0x7ff10806e0b0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff1080b31e0 tx=0x7ff0f002a110 comp rx=0 tx=0).stop 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 -- 172.21.15.28:0/3447389453 shutdown_connections 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 --2- 172.21.15.28:0/3447389453 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072de0 0x7ff10806e0b0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 --2- 172.21.15.28:0/3447389453 >> v2:172.21.15.138:3300/0 conn(0x7ff108072320 0x7ff1080cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 -- 172.21.15.28:0/3447389453 >> 172.21.15.28:0/3447389453 conn(0x7ff108067d70 msgr2=0x7ff10806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:45.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 -- 172.21.15.28:0/3447389453 shutdown_connections 2024-09-10T10:54:45.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.908+0000 7ff10ca00640 1 -- 172.21.15.28:0/3447389453 wait complete. 2024-09-10T10:54:45.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.909+0000 7ff10ca00640 1 Processor -- start 2024-09-10T10:54:45.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.909+0000 7ff10ca00640 1 -- start start 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 0x7ff1081e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff108072de0 0x7ff1081e3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080cd520 con 0x7ff108072320 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080cd3a0 con 0x7ff108072700 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.910+0000 7ff10ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff10806ed00 con 0x7ff108072de0 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49280/0 (socket says 172.21.15.28:49280) 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 -- 172.21.15.28:0/59249589 learned_addr learned my addr 172.21.15.28:0/59249589 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 -- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff108072de0 msgr2=0x7ff1081e3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff108072de0 0x7ff1081e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 -- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 msgr2=0x7ff1081e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 0x7ff1081e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 0x7ff1081e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 -- 172.21.15.28:0/59249589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080ca740 con 0x7ff108072320 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 0x7ff1081e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:45.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.911+0000 7ff105e00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9b0 tx=0x7ff0f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.912+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8017070 con 0x7ff108072320 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.912+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f800c9a0 con 0x7ff108072320 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.912+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800cd90 con 0x7ff108072320 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.912+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1080ca9e0 con 0x7ff108072320 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.912+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1080af9e0 con 0x7ff108072320 2024-09-10T10:54:45.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.913+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1080cadd0 con 0x7ff108072320 2024-09-10T10:54:45.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.914+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0f801a070 con 0x7ff108072320 2024-09-10T10:54:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.915+0000 7ff0fe000640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 0x7ff0e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.915+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7ff0f808bc30 con 0x7ff108072320 2024-09-10T10:54:45.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.916+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 0x7ff0e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:45.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.916+0000 7ff0ffe00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 0x7ff0e406e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0034000 tx=0x7ff0f002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:45.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:45.920+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0f8059800 con 0x7ff108072320 2024-09-10T10:54:46.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.139+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff1080cfa70 con 0x7ff108072320 2024-09-10T10:54:46.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.140+0000 7ff0fe000640 1 -- 172.21.15.28:0/59249589 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v296) ==== 53+0+73 (secure 0 0 0) 0x7ff1080cfa70 con 0x7ff108072320 2024-09-10T10:54:46.141 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:54:46.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 msgr2=0x7ff0e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 0x7ff0e406e2f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0034000 tx=0x7ff0f002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 msgr2=0x7ff108141e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e9b0 tx=0x7ff0f800ee70 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 shutdown_connections 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff0e406be30 0x7ff0e406e2f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff108072de0 0x7ff1081e3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff108072700 0x7ff1081e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 --2- 172.21.15.28:0/59249589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff108072320 0x7ff108141e20 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 >> 172.21.15.28:0/59249589 conn(0x7ff108067d70 msgr2=0x7ff1080ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 shutdown_connections 2024-09-10T10:54:46.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46.142+0000 7ff10ca00640 1 -- 172.21.15.28:0/59249589 wait complete. 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:54:46.159 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:46.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.160 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:54:46.195 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:54:46.195 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:54:46.196 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42019sec preferred_lft 42019sec 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:54:46.197 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:54:46.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.199 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:54:46.288 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:54:46.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.291 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:54:46.327 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:54:46.328 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:54:46.328 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:54:46.328 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:54:46.328 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:54:46.328 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:54:46.328 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:54:46.328 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:54:46.329 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:54:46.329 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:54:46.329 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:54:46.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:54:46.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:46.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.525 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:54:46.560 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:54:46.560 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:54:46.560 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:46.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.561 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:54:46.658 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:46.658 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:46.694 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:54:46.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:54:46.694 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:46.769 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:54:46.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.769 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:46.900 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:46.900 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:46.900 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:46.915 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:46.915 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:46.915 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:46.915 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:46.916 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:54:46.938 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:54:46.939 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:54:46.939 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:54:46.939 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:54:46.940 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:54:46.940 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:54:46.940 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:54:46.942 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:54:46.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:46.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:54:46.998 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:54:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:54:47.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:54:47.088 INFO:teuthology.orchestra.run.smithi028.stdout:client8099 2024-09-10T10:54:47.089 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:47.089 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:47.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:47.125 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:54:47.169 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:54:47.202 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:54:47.201+0000 7f77b2c00640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:54:47.202 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:54:47.202 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:54:47.202 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:47.203 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:54:47.203 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 10000000000.00000000' did not caused DAMAGED state 2024-09-10T10:54:47.204 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 600.00000000' 2024-09-10T10:54:47.204 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:47.204 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:47.279 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:47.280 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:47.355 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:54:47.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.551+0000 7f665fe00640 1 Processor -- start 2024-09-10T10:54:47.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.551+0000 7f665fe00640 1 -- start start 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66600d0dd0 0x7f66600d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f66600d1f90 0x7f66600cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66600cf000 con 0x7f666015e660 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f666006d3e0 con 0x7f66600d0dd0 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665fe00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66600b37d0 con 0x7f66600d1f90 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665e000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f666015e660 0x7f666006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59300/0 (socket says 172.21.15.28:59300) 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665e000640 1 -- 172.21.15.28:0/3127090700 learned_addr learned my addr 172.21.15.28:0/3127090700 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.552+0000 7f665ea00640 1 --2- 172.21.15.28:0/3127090700 >> v2:172.21.15.28:3300/0 conn(0x7f66600d0dd0 0x7f66600d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665f400640 1 --2- 172.21.15.28:0/3127090700 >> v2:172.21.15.138:3300/0 conn(0x7f66600d1f90 0x7f66600cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665ea00640 1 -- 172.21.15.28:0/3127090700 >> v2:172.21.15.138:3300/0 conn(0x7f66600d1f90 msgr2=0x7f66600cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:47.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665ea00640 1 --2- 172.21.15.28:0/3127090700 >> v2:172.21.15.138:3300/0 conn(0x7f66600d1f90 0x7f66600cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665ea00640 1 -- 172.21.15.28:0/3127090700 >> v1:172.21.15.28:6789/0 conn(0x7f666015e660 legacy=0x7f666006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665ea00640 1 -- 172.21.15.28:0/3127090700 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66600d16f0 con 0x7f66600d0dd0 2024-09-10T10:54:47.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.553+0000 7f665d600640 1 -- 172.21.15.28:0/3127090700 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625069026 0 0) 0x7f66600cf000 con 0x7f666015e660 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.554+0000 7f665ea00640 1 --2- 172.21.15.28:0/3127090700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600d11b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f666008cb90 tx=0x7f665002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74b21e50e166bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.554+0000 7f665d600640 1 -- 172.21.15.28:0/3127090700 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037070 con 0x7f66600d0dd0 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.554+0000 7f665d600640 1 -- 172.21.15.28:0/3127090700 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650032440 con 0x7f66600d0dd0 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.554+0000 7f665d600640 1 -- 172.21.15.28:0/3127090700 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650032810 con 0x7f66600d0dd0 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 -- 172.21.15.28:0/3127090700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 msgr2=0x7f66600d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 --2- 172.21.15.28:0/3127090700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600d11b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f666008cb90 tx=0x7f665002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 -- 172.21.15.28:0/3127090700 shutdown_connections 2024-09-10T10:54:47.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 --2- 172.21.15.28:0/3127090700 >> v2:172.21.15.138:3300/0 conn(0x7f66600d1f90 0x7f66600cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 --2- 172.21.15.28:0/3127090700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600d11b0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 -- 172.21.15.28:0/3127090700 >> 172.21.15.28:0/3127090700 conn(0x7f666008bfc0 msgr2=0x7f66600d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 -- 172.21.15.28:0/3127090700 shutdown_connections 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.555+0000 7f665fe00640 1 -- 172.21.15.28:0/3127090700 wait complete. 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.556+0000 7f665fe00640 1 Processor -- start 2024-09-10T10:54:47.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.557+0000 7f665fe00640 1 -- start start 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.557+0000 7f665fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.557+0000 7f665fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66600d1f90 0x7f66600cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666015e660 0x7f66600c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66600d2c60 con 0x7f66600d0dd0 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66600cf330 con 0x7f666015e660 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f66600d2f40 con 0x7f66600d1f90 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49306/0 (socket says 172.21.15.28:49306) 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 -- 172.21.15.28:0/2900259925 learned_addr learned my addr 172.21.15.28:0/2900259925 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:47.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665e000640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66600d1f90 0x7f66600cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665f400640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666015e660 0x7f66600c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 -- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66600d1f90 msgr2=0x7f66600cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66600d1f90 0x7f66600cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 -- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666015e660 msgr2=0x7f66600c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666015e660 0x7f66600c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.558+0000 7f665ea00640 1 -- 172.21.15.28:0/2900259925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66600c9d20 con 0x7f66600d0dd0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f665ea00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6650035040 tx=0x7f66500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650037050 con 0x7f66600d0dd0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650038040 con 0x7f66600d0dd0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650006e10 con 0x7f66600d0dd0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66600ca860 con 0x7f66600d0dd0 2024-09-10T10:54:47.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.559+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66600ca2a0 con 0x7f66600d0dd0 2024-09-10T10:54:47.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.560+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66600d5300 con 0x7f66600d0dd0 2024-09-10T10:54:47.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.561+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6650030980 con 0x7f66600d0dd0 2024-09-10T10:54:47.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.562+0000 7f6656a00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 0x7f664006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:47.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.562+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f66500ac800 con 0x7f66600d0dd0 2024-09-10T10:54:47.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.563+0000 7f665e000640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 0x7f664006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:47.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.563+0000 7f665e000640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 0x7f664006e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f664c007fd0 tx=0x7f664c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:47.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.567+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66600d5300 con 0x7f66600d0dd0 2024-09-10T10:54:47.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.761+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f66600d3220 con 0x7f66600d0dd0 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.762+0000 7f6656a00640 1 -- 172.21.15.28:0/2900259925 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 68 entries v134) ==== 81+0+4132 (secure 0 0 0) 0x7f6650079e10 con 0x7f66600d0dd0 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1056314461 2024-09-11T10:54:36.849623+0000 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:54:47.763 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1056314461 2024-09-11T10:54:36.849623+0000 2024-09-10T10:54:47.764 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:54:47.765 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:54:47.766 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:47.767 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:47.768 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:54:47.769 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:listed 68 entries 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 msgr2=0x7f664006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 0x7f664006e2f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f664c007fd0 tx=0x7f664c007de0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 msgr2=0x7f66600b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6650035040 tx=0x7f66500046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 shutdown_connections 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f664006be30 0x7f664006e2f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666015e660 0x7f66600c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f66600d1f90 0x7f66600cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 --2- 172.21.15.28:0/2900259925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66600d0dd0 0x7f66600b8f00 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:47.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 >> 172.21.15.28:0/2900259925 conn(0x7f666008bfc0 msgr2=0x7f66600b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:47.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 shutdown_connections 2024-09-10T10:54:47.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:47.764+0000 7f665fe00640 1 -- 172.21.15.28:0/2900259925 wait complete. 2024-09-10T10:54:47.780 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:54:47.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:47.780 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:54:47.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:54:47.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:47.850 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:47.887 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:54:48.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.083+0000 7f8579200640 1 Processor -- start 2024-09-10T10:54:48.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.083+0000 7f8579200640 1 -- start start 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8579200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f857415e660 0x7f857406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8579200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85740cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8579200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85740cf000 con 0x7f85740d0dd0 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8579200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f85740b37d0 con 0x7f857415e660 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8579200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85740cf330 con 0x7f85740d1f90 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8573400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f85740d0dd0 0x7f85740d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59314/0 (socket says 172.21.15.28:59314) 2024-09-10T10:54:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.084+0000 7f8573400640 1 -- 172.21.15.28:0/4238564957 learned_addr learned my addr 172.21.15.28:0/4238564957 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8573e00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85740cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f856be00640 1 --2- 172.21.15.28:0/4238564957 >> v2:172.21.15.138:3300/0 conn(0x7f857415e660 0x7f857406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2743219418 0 0) 0x7f85740cf000 con 0x7f85740d0dd0 2024-09-10T10:54:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8558003520 con 0x7f85740d0dd0 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2748713559 0 0) 0x7f8558003520 con 0x7f85740d0dd0 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 msgr2=0x7f85740cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85740cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 >> v2:172.21.15.138:3300/0 conn(0x7f857415e660 msgr2=0x7f857406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 --2- 172.21.15.28:0/4238564957 >> v2:172.21.15.138:3300/0 conn(0x7f857415e660 0x7f857406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85740d29a0 con 0x7f85740d0dd0 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f8564001670 con 0x7f85740d0dd0 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 >> v1:172.21.15.28:6789/0 conn(0x7f85740d0dd0 legacy=0x7f85740d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85580039b0 0x7f8558003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8558004800 0x7f8558004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f85740cf330 con 0x7f85580039b0 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855800b9c0 con 0x7f8558004800 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f855800bb40 con 0x7f85740d1f90 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.085+0000 7f8573400640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85580039b0 0x7f8558003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f856be00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8558004800 0x7f8558004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573e00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573400640 1 -- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 msgr2=0x7f85580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573400640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573400640 1 -- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8558004800 msgr2=0x7f8558004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573400640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8558004800 0x7f8558004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573400640 1 -- 172.21.15.28:0/4238564957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8558003520 con 0x7f85580039b0 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f856be00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8558004800 0x7f8558004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.086+0000 7f8573e00640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85740d1f90 0x7f85580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8573400640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85580039b0 0x7f8558003dd0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f856400c210 tx=0x7f8558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4501b9bfd4b84d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564007d50 con 0x7f85580039b0 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8564005af0 con 0x7f85580039b0 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8572a00640 1 -- 172.21.15.28:0/4238564957 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564007070 con 0x7f85580039b0 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8579200640 1 -- 172.21.15.28:0/4238564957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85580039b0 msgr2=0x7f8558003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8579200640 1 --2- 172.21.15.28:0/4238564957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85580039b0 0x7f8558003dd0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f856400c210 tx=0x7f8558005830 comp rx=0 tx=0).stop 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.087+0000 7f8573e00640 1 -- 172.21.15.28:0/4238564957 reap_dead start 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.088+0000 7f8579200640 1 -- 172.21.15.28:0/4238564957 shutdown_connections 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.088+0000 7f8579200640 1 -- 172.21.15.28:0/4238564957 >> 172.21.15.28:0/4238564957 conn(0x7f857408bfc0 msgr2=0x7f85740d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.088+0000 7f8579200640 1 -- 172.21.15.28:0/4238564957 shutdown_connections 2024-09-10T10:54:48.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.088+0000 7f8579200640 1 -- 172.21.15.28:0/4238564957 wait complete. 2024-09-10T10:54:48.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.089+0000 7f8579200640 1 Processor -- start 2024-09-10T10:54:48.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.090+0000 7f8579200640 1 -- start start 2024-09-10T10:54:48.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.090+0000 7f8579200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 0x7f857414d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8579200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8579200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741f7c60 0x7f85741f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8579200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f85740d3220 con 0x7f8574160930 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8579200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85740d2f40 con 0x7f85741f7c60 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8579200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85740d33a0 con 0x7f85580039b0 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49324/0 (socket says 172.21.15.28:49324) 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 -- 172.21.15.28:0/4056332482 learned_addr learned my addr 172.21.15.28:0/4056332482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8573e00640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741f7c60 0x7f85741f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8573400640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 0x7f857414d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 -- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 msgr2=0x7f857414d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 0x7f857414d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 -- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741f7c60 msgr2=0x7f85741f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741f7c60 0x7f85741f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f856be00640 1 -- 172.21.15.28:0/4056332482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857414d6b0 con 0x7f8574160930 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.091+0000 7f8573400640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 0x7f857414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f856be00640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f855c011560 tx=0x7f855c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c015870 con 0x7f8574160930 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855c015a10 con 0x7f8574160930 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855c015de0 con 0x7f8574160930 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857414fbf0 con 0x7f8574160930 2024-09-10T10:54:48.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.092+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85741501b0 con 0x7f8574160930 2024-09-10T10:54:48.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.093+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85740d2440 con 0x7f8574160930 2024-09-10T10:54:48.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.094+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f855c01d400 con 0x7f8574160930 2024-09-10T10:54:48.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.095+0000 7f856b400640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 0x7f855006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.096+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9228+0+0 (secure 0 0 0) 0x7f855c08c3c0 con 0x7f8574160930 2024-09-10T10:54:48.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.096+0000 7f8573400640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 0x7f855006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.097+0000 7f8573400640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 0x7f855006e320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8564033000 tx=0x7f8564009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.100+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f855c019030 con 0x7f8574160930 2024-09-10T10:54:48.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.319+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f85740d1b20 con 0x7f8574160930 2024-09-10T10:54:48.321 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:48.320+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:48.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.322+0000 7f856b400640 1 -- 172.21.15.28:0/4056332482 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v297) ==== 161+0+0 (secure 0 0 0) 0x7f855c059f00 con 0x7f8574160930 2024-09-10T10:54:48.323 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:48.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 msgr2=0x7f855006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 0x7f855006e320 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8564033000 tx=0x7f8564009030 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 msgr2=0x7f8574160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f855c011560 tx=0x7f855c011a20 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 shutdown_connections 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f855006be60 0x7f855006e320 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85741f7c60 0x7f85741f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8574160930 0x7f8574160d90 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 --2- 172.21.15.28:0/4056332482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85580039b0 0x7f857414d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 >> 172.21.15.28:0/4056332482 conn(0x7f857408bfc0 msgr2=0x7f85740d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 shutdown_connections 2024-09-10T10:54:48.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.325+0000 7f8579200640 1 -- 172.21.15.28:0/4056332482 wait complete. 2024-09-10T10:54:48.331 INFO:tasks.ceph.mds.c.smithi028.stderr: -620> 2024-09-10T10:54:43.827+0000 7f77b7c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:48.332 INFO:tasks.ceph.mds.c.smithi028.stderr: -128> 2024-09-10T10:54:47.201+0000 7f77b2c00640 -1 mds.0.cache.dir(0x10000000000) Corrupt fnode in dirfrag 0x10000000000: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:54:48.348 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:54:48.378 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:54:48.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.507+0000 7f8b0b000640 1 Processor -- start 2024-09-10T10:54:48.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.507+0000 7f8b0b000640 1 -- start start 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b0b000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b04072de0 0x7f8b0415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b0b000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8b04072700 0x7f8b0415d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b0b000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b04146140 con 0x7f8b04072320 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b0b000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8b040cd740 con 0x7f8b04072de0 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b0b000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8b040cda20 con 0x7f8b04072700 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.508+0000 7f8b03400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b04072de0 0x7f8b0415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8b04072de0 0x7f8b0415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49350/0 (socket says 172.21.15.28:49350) 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 -- 172.21.15.28:0/196689250 learned_addr learned my addr 172.21.15.28:0/196689250 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b09000640 1 --2- 172.21.15.28:0/196689250 >> v2:172.21.15.138:3300/0 conn(0x7f8b04072700 0x7f8b0415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 -- 172.21.15.28:0/196689250 >> v2:172.21.15.138:3300/0 conn(0x7f8b04072700 msgr2=0x7f8b0415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 --2- 172.21.15.28:0/196689250 >> v2:172.21.15.138:3300/0 conn(0x7f8b04072700 0x7f8b0415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 -- 172.21.15.28:0/196689250 >> v1:172.21.15.138:6789/0 conn(0x7f8b04072320 legacy=0x7f8b040cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.509+0000 7f8b03400640 1 -- 172.21.15.28:0/196689250 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b040cdd00 con 0x7f8b04072de0 2024-09-10T10:54:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.510+0000 7f8b03400640 1 --2- 172.21.15.28:0/196689250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072de0 0x7f8b0415d1e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cfce0 tx=0x7f8af002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f42fa812f8dc430 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.510+0000 7f8b02a00640 1 -- 172.21.15.28:0/196689250 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0037070 con 0x7f8b04072de0 2024-09-10T10:54:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.510+0000 7f8b02a00640 1 -- 172.21.15.28:0/196689250 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af0032440 con 0x7f8b04072de0 2024-09-10T10:54:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.510+0000 7f8b02a00640 1 -- 172.21.15.28:0/196689250 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0032810 con 0x7f8b04072de0 2024-09-10T10:54:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 -- 172.21.15.28:0/196689250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072de0 msgr2=0x7f8b0415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 --2- 172.21.15.28:0/196689250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072de0 0x7f8b0415d1e0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cfce0 tx=0x7f8af002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 -- 172.21.15.28:0/196689250 shutdown_connections 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 --2- 172.21.15.28:0/196689250 >> v2:172.21.15.138:3300/0 conn(0x7f8b04072700 0x7f8b0415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 --2- 172.21.15.28:0/196689250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072de0 0x7f8b0415d1e0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 -- 172.21.15.28:0/196689250 >> 172.21.15.28:0/196689250 conn(0x7f8b04067d70 msgr2=0x7f8b0406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 -- 172.21.15.28:0/196689250 shutdown_connections 2024-09-10T10:54:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.511+0000 7f8b0b000640 1 -- 172.21.15.28:0/196689250 wait complete. 2024-09-10T10:54:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.513+0000 7f8b0b000640 1 Processor -- start 2024-09-10T10:54:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.513+0000 7f8b0b000640 1 -- start start 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b04072700 0x7f8b041efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b04072de0 0x7f8b041450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0415dde0 con 0x7f8b04072320 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0415dc60 con 0x7f8b04072700 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b0b000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8b0415df60 con 0x7f8b04072de0 2024-09-10T10:54:48.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49362/0 (socket says 172.21.15.28:49362) 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 -- 172.21.15.28:0/50464009 learned_addr learned my addr 172.21.15.28:0/50464009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03400640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b04072700 0x7f8b041efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b09000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b04072de0 0x7f8b041450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 -- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b04072de0 msgr2=0x7f8b041450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b04072de0 0x7f8b041450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 -- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b04072700 msgr2=0x7f8b041efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b04072700 0x7f8b041efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.514+0000 7f8b03e00640 1 -- 172.21.15.28:0/50464009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b041f03c0 con 0x7f8b04072320 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8b03e00640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c450 tx=0x7f8af400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af40052d0 con 0x7f8b04072320 2024-09-10T10:54:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af4005470 con 0x7f8b04072320 2024-09-10T10:54:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af4012870 con 0x7f8b04072320 2024-09-10T10:54:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b041f06a0 con 0x7f8b04072320 2024-09-10T10:54:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.515+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b041effc0 con 0x7f8b04072320 2024-09-10T10:54:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.516+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0415e320 con 0x7f8b04072320 2024-09-10T10:54:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.517+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8af4012a10 con 0x7f8b04072320 2024-09-10T10:54:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.518+0000 7f8afbe00640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 0x7f8ae406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.518+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9314+0+0 (secure 0 0 0) 0x7f8af408b770 con 0x7f8b04072320 2024-09-10T10:54:48.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.518+0000 7f8b03400640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 0x7f8ae406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:48.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.519+0000 7f8b03400640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 0x7f8ae406e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8af0034000 tx=0x7f8af002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:48.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.522+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8af4058c90 con 0x7f8b04072320 2024-09-10T10:54:48.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:48.781+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f8b041f0bc0 con 0x7f8b04072320 2024-09-10T10:54:49.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.328+0000 7f8afbe00640 1 -- 172.21.15.28:0/50464009 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v298) ==== 107+0+0 (secure 0 0 0) 0x7f8af405cf90 con 0x7f8b04072320 2024-09-10T10:54:49.330 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 msgr2=0x7f8ae406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 0x7f8ae406e2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8af0034000 tx=0x7f8af002e000 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 msgr2=0x7f8b040ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c450 tx=0x7f8af400c910 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 shutdown_connections 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ae406be30 0x7f8ae406e2f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8b04072de0 0x7f8b041450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b04072700 0x7f8b041efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 --2- 172.21.15.28:0/50464009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b04072320 0x7f8b040ca040 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.331+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 >> 172.21.15.28:0/50464009 conn(0x7f8b04067d70 msgr2=0x7f8b040b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.332+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 shutdown_connections 2024-09-10T10:54:49.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.332+0000 7f8b0b000640 1 -- 172.21.15.28:0/50464009 wait complete. 2024-09-10T10:54:49.351 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:54:49.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.469+0000 7fde0068cf40 1 Processor -- start 2024-09-10T10:54:49.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.469+0000 7fde0068cf40 1 -- start start 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d852cb0 0x55f61d922460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d9244b0 0x55f61d9b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f61d7f6b30 con 0x55f61d8982d0 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f61d7fab70 con 0x55f61d852cb0 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.470+0000 7fde0068cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f61d897dc0 con 0x55f61d9244b0 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49382/0 (socket says 172.21.15.28:49382) 2024-09-10T10:54:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 -- 172.21.15.28:0/1872810108 learned_addr learned my addr 172.21.15.28:0/1872810108 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfd800640 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d852cb0 0x55f61d922460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfe200640 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d9244b0 0x55f61d9b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 -- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d9244b0 msgr2=0x55f61d9b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d9244b0 0x55f61d9b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 -- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d852cb0 msgr2=0x55f61d922460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d852cb0 0x55f61d922460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 -- 172.21.15.28:0/1872810108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61d87e200 con 0x55f61d8982d0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.471+0000 7fddfce00640 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fddec00a440 tx=0x7fddec051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71b2067cf79a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fddf7400640 1 -- 172.21.15.28:0/1872810108 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec056050 con 0x55f61d8982d0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fddf7400640 1 -- 172.21.15.28:0/1872810108 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec052d60 con 0x55f61d8982d0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fddf7400640 1 -- 172.21.15.28:0/1872810108 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec065670 con 0x55f61d8982d0 2024-09-10T10:54:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 -- 172.21.15.28:0/1872810108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 msgr2=0x55f61d87c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fddec00a440 tx=0x7fddec051000 comp rx=0 tx=0).stop 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 -- 172.21.15.28:0/1872810108 shutdown_connections 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d9244b0 0x55f61d9b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d87c0c0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 --2- 172.21.15.28:0/1872810108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d852cb0 0x55f61d922460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 -- 172.21.15.28:0/1872810108 >> 172.21.15.28:0/1872810108 conn(0x55f61d87cb90 msgr2=0x55f61d84d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 -- 172.21.15.28:0/1872810108 shutdown_connections 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.472+0000 7fde0068cf40 1 -- 172.21.15.28:0/1872810108 wait complete. 2024-09-10T10:54:49.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.473+0000 7fde0068cf40 1 Processor -- start 2024-09-10T10:54:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.473+0000 7fde0068cf40 1 -- start start 2024-09-10T10:54:49.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d852cb0 0x55f61d87c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d9244b0 0x55f61d922460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55f61d87b1d0 con 0x55f61d8982d0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f61d87dfb0 con 0x55f61d9244b0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fde0068cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f61d896640 con 0x55f61d852cb0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fddfce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fddfce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49394/0 (socket says 172.21.15.28:49394) 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.474+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 learned_addr learned my addr 172.21.15.28:0/3337714291 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d852cb0 0x55f61d87c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfe200640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d9244b0 0x55f61d922460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d852cb0 msgr2=0x55f61d87c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f61d852cb0 0x55f61d87c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d9244b0 msgr2=0x55f61d922460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f61d9244b0 0x55f61d922460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f61d923e10 con 0x55f61d8982d0 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fddec052610 tx=0x7fddec002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddf6000640 1 -- 172.21.15.28:0/3337714291 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec073020 con 0x55f61d8982d0 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddf6000640 1 -- 172.21.15.28:0/3337714291 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddec00a220 con 0x55f61d8982d0 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fddf6000640 1 -- 172.21.15.28:0/3337714291 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddec065710 con 0x55f61d8982d0 2024-09-10T10:54:49.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.475+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f61d920520 con 0x55f61d8982d0 2024-09-10T10:54:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.476+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f61d9207c0 con 0x55f61d8982d0 2024-09-10T10:54:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.477+0000 7fddf6000640 1 -- 172.21.15.28:0/3337714291 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddec076070 con 0x55f61d8982d0 2024-09-10T10:54:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.478+0000 7fddf6000640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 0x7fdde006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.478+0000 7fddf6000640 1 -- 172.21.15.28:0/3337714291 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9314+0+0 (secure 0 0 0) 0x7fddec0cfb90 con 0x55f61d8982d0 2024-09-10T10:54:49.481 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:54:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.479+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f61d924d60 0x55f61d925170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.479+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9256b0 con 0x55f61d924d60 2024-09-10T10:54:49.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.479+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 0x7fdde006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.479+0000 7fddfe200640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f61d924d60 0x55f61d925170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.480+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 0x7fdde006ccd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f61d84f3b0 tx=0x7fddf00015c0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.480+0000 7fddfe200640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f61d924d60 0x55f61d925170 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.483+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v135'1065 uv1038 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.483+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d925a80 con 0x55f61d924d60 2024-09-10T10:54:49.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.485+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v135'1066 uv1066 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.485+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d925e50 con 0x55f61d924d60 2024-09-10T10:54:49.487 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:54:49.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.486+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v135'1067 uv1067 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.486+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ca3b0 con 0x55f61d924d60 2024-09-10T10:54:49.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.488+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v135'1068 uv1068 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.488+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ca750 con 0x55f61d924d60 2024-09-10T10:54:49.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.489+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v135'1069 uv1069 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.489+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c93a0 con 0x55f61d924d60 2024-09-10T10:54:49.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.491+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v135'1070 uv1070 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.491+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9770 con 0x55f61d924d60 2024-09-10T10:54:49.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.493+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v135'1071 uv1044 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.493+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.495+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v135'1072 uv1072 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.495+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.498 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:54:49.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.497+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v135'1073 uv1073 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.497+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.498+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v135'1074 uv1074 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.499+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.500+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v135'1075 uv1075 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.500+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.502+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v135'1076 uv1076 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.502+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.504+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v135'1077 uv1064 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.504+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.506+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v135'1078 uv1078 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.506+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.508 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:54:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.507+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v135'1079 uv1079 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.508+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.509+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v135'1080 uv1080 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.509+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.511+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v135'1081 uv1081 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.511+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9c9b10 con 0x55f61d924d60 2024-09-10T10:54:49.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.512+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v135'1082 uv1082 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.512+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.514+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v135'1083 uv1083 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.514+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.516+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v135'1084 uv1057 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.516+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.518+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v135'1085 uv1085 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.518+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.521 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:54:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.520+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v135'1086 uv1086 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.520+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cbb00 con 0x55f61d924d60 2024-09-10T10:54:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.522+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v135'1087 uv1087 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.522+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cbb00 con 0x55f61d924d60 2024-09-10T10:54:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.524+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v135'1088 uv1088 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.524+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.526+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v135'1089 uv1089 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.526+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.527+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v135'1090 uv1063 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.527+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.529+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v135'1091 uv1091 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.529+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.532 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:54:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.531+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v135'1092 uv1092 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.531+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cbb00 con 0x55f61d924d60 2024-09-10T10:54:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.533+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v135'1093 uv1093 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.533+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cbb00 con 0x55f61d924d60 2024-09-10T10:54:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.534+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v135'1094 uv1094 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.534+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9caaf0 con 0x55f61d924d60 2024-09-10T10:54:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.536+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v135'1095 uv1095 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d924d60 2024-09-10T10:54:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.536+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f61d9cb000 0x55f61d9cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.536+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cd930 con 0x55f61d9cb000 2024-09-10T10:54:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.536+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f61d9cb000 0x55f61d9cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.537+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f61d9cb000 0x55f61d9cd3f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.539+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v135'777 uv756 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.539+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cdcd0 con 0x55f61d9cb000 2024-09-10T10:54:49.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.541+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v135'778 uv778 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.541+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ce070 con 0x55f61d9cb000 2024-09-10T10:54:49.543 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:54:49.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.542+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v135'779 uv779 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.542+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ce410 con 0x55f61d9cb000 2024-09-10T10:54:49.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.544+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v135'780 uv780 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.544+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ce7b0 con 0x55f61d9cb000 2024-09-10T10:54:49.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.545+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v135'781 uv781 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.545+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ceb50 con 0x55f61d9cb000 2024-09-10T10:54:49.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.547+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v135'782 uv775 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.547+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ceef0 con 0x55f61d9cb000 2024-09-10T10:54:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.549+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v135'783 uv783 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.549+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.551 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:54:49.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.550+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v135'784 uv784 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.550+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.552+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v135'785 uv785 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.552+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.553+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v135'786 uv786 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.553+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.555+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v135'787 uv776 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.555+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.556+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v135'788 uv788 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.557+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.559 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:54:49.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.558+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v135'789 uv789 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.558+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d3980 con 0x55f61d9cb000 2024-09-10T10:54:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.560+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v135'790 uv790 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.560+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d3980 con 0x55f61d9cb000 2024-09-10T10:54:49.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.561+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v135'791 uv791 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.561+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.563+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v135'792 uv772 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.563+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.565+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v135'793 uv793 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.565+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.567 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:54:49.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.566+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v135'794 uv794 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.566+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d02a0 con 0x55f61d9cb000 2024-09-10T10:54:49.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.568+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v135'795 uv795 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.568+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d02a0 con 0x55f61d9cb000 2024-09-10T10:54:49.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.569+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v135'796 uv796 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.569+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9cf290 con 0x55f61d9cb000 2024-09-10T10:54:49.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.570+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v135'797 uv797 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9cb000 2024-09-10T10:54:49.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.571+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f61d9cf7a0 0x55f61d9d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.571+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d20d0 con 0x55f61d9cf7a0 2024-09-10T10:54:49.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.571+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f61d9cf7a0 0x55f61d9d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.571+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f61d9cf7a0 0x55f61d9d1b90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.573+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v135'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.573+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2470 con 0x55f61d9cf7a0 2024-09-10T10:54:49.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.574+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v135'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.574+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2920 con 0x55f61d9cf7a0 2024-09-10T10:54:49.576 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:54:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.575+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v135'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.575+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d921100 con 0x55f61d9cf7a0 2024-09-10T10:54:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.577+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v135'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.577+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d42b0 con 0x55f61d9cf7a0 2024-09-10T10:54:49.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.578+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v135'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.578+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d31c0 con 0x55f61d9cf7a0 2024-09-10T10:54:49.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.579+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v135'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.580+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f61d9d3c80 0x55f61d9d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.580+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d6c80 con 0x55f61d9d3c80 2024-09-10T10:54:49.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.580+0000 7fddfe200640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f61d9d3c80 0x55f61d9d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.580+0000 7fddfe200640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f61d9d3c80 0x55f61d9d66b0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.582+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v135'153 uv152 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.582+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7070 con 0x55f61d9d3c80 2024-09-10T10:54:49.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.583+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v135'154 uv154 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.583+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7070 con 0x55f61d9d3c80 2024-09-10T10:54:49.585 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:54:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.584+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v135'155 uv155 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.584+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7070 con 0x55f61d9d3c80 2024-09-10T10:54:49.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.585+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v135'156 uv156 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.585+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d6f90 con 0x55f61d9cf7a0 2024-09-10T10:54:49.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.586+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v135'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.586+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7460 con 0x55f61d9cf7a0 2024-09-10T10:54:49.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.588+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v135'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.588+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.589 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:54:49.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.589+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v135'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.589+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d8810 con 0x55f61d9cf7a0 2024-09-10T10:54:49.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.590+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v135'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.590+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d8810 con 0x55f61d9cf7a0 2024-09-10T10:54:49.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.591+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v135'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.591+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.592+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v135'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.592+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.594+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v135'382 uv376 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.594+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.595+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v135'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.595+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.598 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:54:49.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.597+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v135'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.597+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d8810 con 0x55f61d9d3c80 2024-09-10T10:54:49.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.598+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v135'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.598+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d8810 con 0x55f61d9d3c80 2024-09-10T10:54:49.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.600+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v135'386 uv386 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.600+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.601+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v135'387 uv387 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.601+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.602+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v135'388 uv381 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.603+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.604+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v135'389 uv389 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.604+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.606 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:54:49.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.606+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v135'390 uv390 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.606+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.607+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v135'391 uv391 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.607+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9d3c80 2024-09-10T10:54:49.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.609+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v135'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.609+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.610+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v135'173 uv172 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.610+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.611+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v135'174 uv174 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.611+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.613 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:54:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.612+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v135'175 uv175 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.612+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.613+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v135'176 uv176 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.613+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d7800 con 0x55f61d9cf7a0 2024-09-10T10:54:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.614+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v135'177 uv177 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9cf7a0 2024-09-10T10:54:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.615+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f61d9d8120 0x55f61d9da510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.615+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9daa50 con 0x55f61d9d8120 2024-09-10T10:54:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.615+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f61d9d8120 0x55f61d9da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.615+0000 7fddfd800640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f61d9d8120 0x55f61d9da510 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.617+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v135'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.617+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9daa90 con 0x55f61d9d8120 2024-09-10T10:54:49.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.619+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v135'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.619+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.621 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:54:49.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.620+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v135'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.620+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dbe90 con 0x55f61d9d8120 2024-09-10T10:54:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.622+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v135'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.622+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dbe90 con 0x55f61d9d8120 2024-09-10T10:54:49.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.623+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v135'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.623+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.625+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v135'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.625+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.626+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v135'345 uv339 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.626+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.627+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v135'346 uv346 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.627+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.629 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:54:49.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.629+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v135'347 uv347 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.629+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.630+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v135'348 uv348 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.630+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.631+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v135'349 uv349 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.632+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.633+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v135'350 uv344 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.633+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.634+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v135'351 uv351 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.634+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.637 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:54:49.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.636+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v135'352 uv352 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.636+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9dae80 con 0x55f61d9d8120 2024-09-10T10:54:49.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.637+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v135'353 uv353 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.637+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2e00 con 0x55f61d9d8120 2024-09-10T10:54:49.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.639+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v135'354 uv354 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.639+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.640+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v135'152 uv150 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.640+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.642+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v135'153 uv153 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.642+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.644 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:54:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.643+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v135'154 uv154 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.643+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.645+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v135'155 uv155 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.645+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.646+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v135'207 uv206 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.646+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.647+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v135'208 uv208 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.647+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.649 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:54:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.648+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v135'209 uv209 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.648+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2dc0 con 0x55f61d9d3c80 2024-09-10T10:54:49.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.649+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v135'210 uv210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.649+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2da0 con 0x55f61d9d3c80 2024-09-10T10:54:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.651+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v135'211 uv211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.651+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9d2da0 con 0x55f61d9d3c80 2024-09-10T10:54:49.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.652+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v135'212 uv212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf8003850 con 0x55f61d9d3c80 2024-09-10T10:54:49.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.652+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f61d9dbc30 0x55f61d9de380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.652+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9dbc30 2024-09-10T10:54:49.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.652+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f61d9dbc30 0x55f61d9de380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.653+0000 7fddfce00640 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f61d9dbc30 0x55f61d9de380 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.654+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v135'176 uv175 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9dbc30 2024-09-10T10:54:49.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.654+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9dbc30 2024-09-10T10:54:49.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.655+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v135'177 uv177 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9dbc30 2024-09-10T10:54:49.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.655+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9dbc30 2024-09-10T10:54:49.657 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:54:49.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.656+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v135'178 uv178 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9dbc30 2024-09-10T10:54:49.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.656+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9dbc30 2024-09-10T10:54:49.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.657+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v135'179 uv179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9dbc30 2024-09-10T10:54:49.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.657+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9dbc30 2024-09-10T10:54:49.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.658+0000 7fddfce00640 1 -- 172.21.15.28:0/3337714291 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v135'180 uv180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddec075030 con 0x55f61d9dbc30 2024-09-10T10:54:49.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.658+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.660+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v135'429 uv420 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.660+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.661+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v135'430 uv430 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.661+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.662+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v135'431 uv431 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.664 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:54:49.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.662+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.664+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v135'432 uv432 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.664+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.665+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v135'433 uv433 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.665+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.666+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v135'434 uv427 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.666+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.668+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v135'435 uv435 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.668+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.670 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:54:49.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.669+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v135'436 uv436 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.669+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9df8d0 con 0x55f61d9d8120 2024-09-10T10:54:49.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.670+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v135'437 uv437 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.670+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9df8d0 con 0x55f61d9d8120 2024-09-10T10:54:49.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.671+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v135'438 uv438 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.671+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9de8c0 con 0x55f61d9d8120 2024-09-10T10:54:49.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.673+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v135'439 uv439 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.673+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x55f61d9ded70 con 0x55f61d9d8120 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fddfd800640 1 -- 172.21.15.28:0/3337714291 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v135'440 uv440 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddf001a040 con 0x55f61d9d8120 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f61d9cf7a0 msgr2=0x55f61d9d1b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55f61d9cf7a0 0x55f61d9d1b90 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f61d9d8120 msgr2=0x55f61d9da510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55f61d9d8120 0x55f61d9da510 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f61d9dbc30 msgr2=0x55f61d9de380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55f61d9dbc30 0x55f61d9de380 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f61d9d3c80 msgr2=0x55f61d9d66b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55f61d9d3c80 0x55f61d9d66b0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f61d9cb000 msgr2=0x55f61d9cd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55f61d9cb000 0x55f61d9cd3f0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 reap_dead start 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f61d924d60 msgr2=0x55f61d925170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55f61d924d60 0x55f61d925170 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 reap_dead start 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fddfe200640 1 -- 172.21.15.28:0/3337714291 reap_dead start 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 msgr2=0x7fdde006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 0x7fdde006ccd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x55f61d84f3b0 tx=0x7fddf00015c0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 msgr2=0x55f61d913660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.674+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fddec052610 tx=0x7fddec002fb0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 shutdown_connections 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdde006a810 0x7fdde006ccd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 --2- 172.21.15.28:0/3337714291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f61d8982d0 0x55f61d913660 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 >> 172.21.15.28:0/3337714291 conn(0x55f61d87cb90 msgr2=0x55f61d84d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 shutdown_connections 2024-09-10T10:54:49.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.675+0000 7fde0068cf40 1 -- 172.21.15.28:0/3337714291 wait complete. 2024-09-10T10:54:49.680 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 600.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:54:49.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.758+0000 7f974baeaf40 1 Processor -- start 2024-09-10T10:54:49.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.758+0000 7f974baeaf40 1 -- start start 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 0x564e47f18c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 0x564e47fbcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564e47f30530 con 0x564e47fbe9c0 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e47e8db30 con 0x564e47f2f8e0 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f974baeaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564e47fbdee0 con 0x564e47fbb3e0 2024-09-10T10:54:49.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f9749600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f9749600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49422/0 (socket says 172.21.15.28:49422) 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f9749600640 1 -- 172.21.15.28:0/3301702938 learned_addr learned my addr 172.21.15.28:0/3301702938 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f9743e00640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 0x564e47fbcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.759+0000 7f9748c00640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 0x564e47f18c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 -- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 msgr2=0x564e47fbcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 0x564e47fbcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 -- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 msgr2=0x564e47f18c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 0x564e47f18c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 -- 172.21.15.28:0/3301702938 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e47f32600 con 0x564e47fbe9c0 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9743e00640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 0x564e47fbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:49.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9748c00640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 0x564e47f18c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:49.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9749600640 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9744001680 tx=0x7f9744001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1addaba6b2ea707c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9742a00640 1 -- 172.21.15.28:0/3301702938 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004520 con 0x564e47fbe9c0 2024-09-10T10:54:49.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9742a00640 1 -- 172.21.15.28:0/3301702938 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97440046c0 con 0x564e47fbe9c0 2024-09-10T10:54:49.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.760+0000 7f9742a00640 1 -- 172.21.15.28:0/3301702938 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004af0 con 0x564e47fbe9c0 2024-09-10T10:54:49.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 -- 172.21.15.28:0/3301702938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 msgr2=0x564e47ee5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f9744001680 tx=0x7f9744001b40 comp rx=0 tx=0).stop 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 -- 172.21.15.28:0/3301702938 shutdown_connections 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbe9c0 0x564e47ee5070 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47fbb3e0 0x564e47fbcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 --2- 172.21.15.28:0/3301702938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47f2f8e0 0x564e47f18c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 -- 172.21.15.28:0/3301702938 >> 172.21.15.28:0/3301702938 conn(0x564e47f340a0 msgr2=0x564e47fbb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 -- 172.21.15.28:0/3301702938 shutdown_connections 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 -- 172.21.15.28:0/3301702938 wait complete. 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.761+0000 7f974baeaf40 1 Processor -- start 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 -- start start 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47f2f8e0 0x564e47fb0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 0x564e47fb2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47fbe9c0 0x564e47f319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564e47fbac60 con 0x564e47fbb3e0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564e47f349a0 con 0x564e47fbe9c0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.762+0000 7f974baeaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x564e47fbd120 con 0x564e47f2f8e0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 0x564e47fb2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 0x564e47fb2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49428/0 (socket says 172.21.15.28:49428) 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 -- 172.21.15.28:0/4279852994 learned_addr learned my addr 172.21.15.28:0/4279852994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9748c00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47f2f8e0 0x564e47fb0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9749600640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47fbe9c0 0x564e47f319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 -- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47f2f8e0 msgr2=0x564e47fb0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x564e47f2f8e0 0x564e47fb0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 -- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47fbe9c0 msgr2=0x564e47f319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47fbe9c0 0x564e47f319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 -- 172.21.15.28:0/4279852994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e47fbcf30 con 0x564e47fbb3e0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9749600640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564e47fbe9c0 0x564e47f319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.763+0000 7f9743e00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 0x564e47fb2590 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9734025bb0 tx=0x7f9734002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.764+0000 7f9741600640 1 -- 172.21.15.28:0/4279852994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734068440 con 0x564e47fbb3e0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.764+0000 7f9741600640 1 -- 172.21.15.28:0/4279852994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97340685e0 con 0x564e47fbb3e0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.764+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e47fc21f0 con 0x564e47fbb3e0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.764+0000 7f9741600640 1 -- 172.21.15.28:0/4279852994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734068990 con 0x564e47fbb3e0 2024-09-10T10:54:49.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.764+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e47ee5350 con 0x564e47fbb3e0 2024-09-10T10:54:49.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.766+0000 7f9741600640 1 -- 172.21.15.28:0/4279852994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9734052070 con 0x564e47fbb3e0 2024-09-10T10:54:49.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.766+0000 7f9741600640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f972806a810 0x7f972806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.767+0000 7f9741600640 1 -- 172.21.15.28:0/4279852994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9314+0+0 (secure 0 0 0) 0x7f97340ce7d0 con 0x564e47fbb3e0 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.767+0000 7f974baeaf40 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564e47f2ef00 0x564e47fbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.767+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135) -- 0x564e47fc0330 con 0x564e47f2ef00 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.767+0000 7f9748c00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f972806a810 0x7f972806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.767+0000 7f9749600640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564e47f2ef00 0x564e47fbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.768+0000 7f9748c00640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f972806a810 0x7f972806ccd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f973c005870 tx=0x7f973c00c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.768+0000 7f9749600640 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564e47f2ef00 0x564e47fbfdf0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f9749600640 1 -- 172.21.15.28:0/4279852994 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 600.00000000 [omap-set-header] v135'1096 uv1096 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f974400f030 con 0x564e47f2ef00 2024-09-10T10:54:49.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564e47f2ef00 msgr2=0x564e47fbfdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x564e47f2ef00 0x564e47fbfdf0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f972806a810 msgr2=0x7f972806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f972806a810 0x7f972806ccd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f973c005870 tx=0x7f973c00c040 comp rx=0 tx=0).stop 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 msgr2=0x564e47fb2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 --2- 172.21.15.28:0/4279852994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e47fbb3e0 0x564e47fb2590 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f9734025bb0 tx=0x7f9734002fb0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f9749600640 1 -- 172.21.15.28:0/4279852994 reap_dead start 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 shutdown_connections 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 >> 172.21.15.28:0/4279852994 conn(0x564e47f340a0 msgr2=0x564e47fbb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 shutdown_connections 2024-09-10T10:54:49.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.770+0000 7f974baeaf40 1 -- 172.21.15.28:0/4279852994 wait complete. 2024-09-10T10:54:49.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:54:49.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.933+0000 7f880b200640 1 Processor -- start 2024-09-10T10:54:49.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.933+0000 7f880b200640 1 -- start start 2024-09-10T10:54:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8804161ac0 0x7f880406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f88040d2ac0 0x7f88040b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88040b1170 con 0x7f88040b3640 2024-09-10T10:54:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f88040ceba0 con 0x7f8804161ac0 2024-09-10T10:54:49.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f880406dfb0 con 0x7f88040d2ac0 2024-09-10T10:54:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f8809e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8804161ac0 0x7f880406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f8809400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f88040b3640 0x7f88040d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59348/0 (socket says 172.21.15.28:59348) 2024-09-10T10:54:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f8809400640 1 -- 172.21.15.28:0/1962660097 learned_addr learned my addr 172.21.15.28:0/1962660097 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:49.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.934+0000 7f880a800640 1 --2- 172.21.15.28:0/1962660097 >> v2:172.21.15.138:3300/0 conn(0x7f88040d2ac0 0x7f88040b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.935+0000 7f8809e00640 1 -- 172.21.15.28:0/1962660097 >> v2:172.21.15.138:3300/0 conn(0x7f88040d2ac0 msgr2=0x7f88040b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:49.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.935+0000 7f8809e00640 1 --2- 172.21.15.28:0/1962660097 >> v2:172.21.15.138:3300/0 conn(0x7f88040d2ac0 0x7f88040b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.935+0000 7f8809e00640 1 -- 172.21.15.28:0/1962660097 >> v1:172.21.15.28:6789/0 conn(0x7f88040b3640 legacy=0x7f88040d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.935+0000 7f8809e00640 1 -- 172.21.15.28:0/1962660097 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88040b1450 con 0x7f8804161ac0 2024-09-10T10:54:49.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.935+0000 7f8808a00640 1 -- 172.21.15.28:0/1962660097 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070395510 0 0) 0x7f88040b1170 con 0x7f88040b3640 2024-09-10T10:54:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.936+0000 7f8809e00640 1 --2- 172.21.15.28:0/1962660097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f880406e6c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f880408c4d0 tx=0x7f87f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e65e7f84f44b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.936+0000 7f8808a00640 1 -- 172.21.15.28:0/1962660097 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8037070 con 0x7f8804161ac0 2024-09-10T10:54:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.936+0000 7f8808a00640 1 -- 172.21.15.28:0/1962660097 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f8032440 con 0x7f8804161ac0 2024-09-10T10:54:49.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.936+0000 7f8808a00640 1 -- 172.21.15.28:0/1962660097 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8032810 con 0x7f8804161ac0 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 -- 172.21.15.28:0/1962660097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 msgr2=0x7f880406e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 --2- 172.21.15.28:0/1962660097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f880406e6c0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f880408c4d0 tx=0x7f87f802a230 comp rx=0 tx=0).stop 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 -- 172.21.15.28:0/1962660097 shutdown_connections 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 --2- 172.21.15.28:0/1962660097 >> v2:172.21.15.138:3300/0 conn(0x7f88040d2ac0 0x7f88040b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 --2- 172.21.15.28:0/1962660097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f880406e6c0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 -- 172.21.15.28:0/1962660097 >> 172.21.15.28:0/1962660097 conn(0x7f880408bfc0 msgr2=0x7f88040d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 -- 172.21.15.28:0/1962660097 shutdown_connections 2024-09-10T10:54:49.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.937+0000 7f880b200640 1 -- 172.21.15.28:0/1962660097 wait complete. 2024-09-10T10:54:49.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.938+0000 7f880b200640 1 Processor -- start 2024-09-10T10:54:49.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 -- start start 2024-09-10T10:54:49.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88040b3640 0x7f880415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88040d2ac0 0x7f88040c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8804161ea0 con 0x7f8804161ac0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88040b1a10 con 0x7f88040b3640 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.939+0000 7f880b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8804162020 con 0x7f88040d2ac0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49438/0 (socket says 172.21.15.28:49438) 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 -- 172.21.15.28:0/4007809036 learned_addr learned my addr 172.21.15.28:0/4007809036 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f8809e00640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88040b3640 0x7f880415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f8809400640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88040d2ac0 0x7f88040c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 -- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88040d2ac0 msgr2=0x7f88040c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88040d2ac0 0x7f88040c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 -- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88040b3640 msgr2=0x7f880415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88040b3640 0x7f880415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.940+0000 7f880a800640 1 -- 172.21.15.28:0/4007809036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88040cb5b0 con 0x7f8804161ac0 2024-09-10T10:54:49.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f880a800640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f880000efc0 tx=0x7f880000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800018020 con 0x7f8804161ac0 2024-09-10T10:54:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8800014480 con 0x7f8804161ac0 2024-09-10T10:54:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800014870 con 0x7f8804161ac0 2024-09-10T10:54:49.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88040cb890 con 0x7f8804161ac0 2024-09-10T10:54:49.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.941+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88040cbe50 con 0x7f8804161ac0 2024-09-10T10:54:49.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.942+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880406e6c0 con 0x7f8804161ac0 2024-09-10T10:54:49.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.943+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8800005d10 con 0x7f8804161ac0 2024-09-10T10:54:49.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.944+0000 7f87fe000640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 0x7f87e806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:49.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.944+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9314+0+0 (secure 0 0 0) 0x7f880008c6f0 con 0x7f8804161ac0 2024-09-10T10:54:49.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.944+0000 7f8809e00640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 0x7f87e806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:49.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.945+0000 7f8809e00640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 0x7f87e806e320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:49.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:49.948+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8800022070 con 0x7f8804161ac0 2024-09-10T10:54:50.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.169+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f88040b82d0 con 0x7f8804161ac0 2024-09-10T10:54:50.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.728+0000 7f87fe000640 1 -- 172.21.15.28:0/4007809036 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v299) ==== 162+0+0 (secure 0 0 0) 0x7f8800010040 con 0x7f8804161ac0 2024-09-10T10:54:50.729 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 msgr2=0x7f87e806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 0x7f87e806e320 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87f8034000 tx=0x7f87f802e000 comp rx=0 tx=0).stop 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 msgr2=0x7f88041528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f880000efc0 tx=0x7f880000c490 comp rx=0 tx=0).stop 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 shutdown_connections 2024-09-10T10:54:50.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f87e806be60 0x7f87e806e320 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8804161ac0 0x7f88041528a0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88040d2ac0 0x7f88040c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 --2- 172.21.15.28:0/4007809036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88040b3640 0x7f880415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 >> 172.21.15.28:0/4007809036 conn(0x7f880408bfc0 msgr2=0x7f88040b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 shutdown_connections 2024-09-10T10:54:50.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.731+0000 7f880b200640 1 -- 172.21.15.28:0/4007809036 wait complete. 2024-09-10T10:54:50.753 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.912+0000 7f3e45400640 1 Processor -- start 2024-09-10T10:54:50.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.912+0000 7f3e45400640 1 -- start start 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.913+0000 7f3e45400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3e40072320 0x7f3e400cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.913+0000 7f3e45400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e40072de0 0x7f3e4006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.913+0000 7f3e45400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e40146140 con 0x7f3e40072700 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.913+0000 7f3e45400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3e400cd740 con 0x7f3e40072de0 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.913+0000 7f3e45400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3e400cda20 con 0x7f3e40072320 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e40072de0 0x7f3e4006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e40072de0 0x7f3e4006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49452/0 (socket says 172.21.15.28:49452) 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 -- 172.21.15.28:0/1861583842 learned_addr learned my addr 172.21.15.28:0/1861583842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3e800640 1 --2- 172.21.15.28:0/1861583842 >> v2:172.21.15.138:3300/0 conn(0x7f3e40072320 0x7f3e400cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 -- 172.21.15.28:0/1861583842 >> v2:172.21.15.138:3300/0 conn(0x7f3e40072320 msgr2=0x7f3e400cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 --2- 172.21.15.28:0/1861583842 >> v2:172.21.15.138:3300/0 conn(0x7f3e40072320 0x7f3e400cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 -- 172.21.15.28:0/1861583842 >> v1:172.21.15.138:6789/0 conn(0x7f3e40072700 legacy=0x7f3e4006e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:50.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.914+0000 7f3e3de00640 1 -- 172.21.15.28:0/1861583842 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e400cdd00 con 0x7f3e40072de0 2024-09-10T10:54:50.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.915+0000 7f3e3de00640 1 --2- 172.21.15.28:0/1861583842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072de0 0x7f3e4006e0b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f3e400b31e0 tx=0x7f3e2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4331418720c6518b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:50.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.915+0000 7f3e3d400640 1 -- 172.21.15.28:0/1861583842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c037070 con 0x7f3e40072de0 2024-09-10T10:54:50.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e3d400640 1 -- 172.21.15.28:0/1861583842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c032440 con 0x7f3e40072de0 2024-09-10T10:54:50.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e3d400640 1 -- 172.21.15.28:0/1861583842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c032810 con 0x7f3e40072de0 2024-09-10T10:54:50.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 -- 172.21.15.28:0/1861583842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072de0 msgr2=0x7f3e4006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 --2- 172.21.15.28:0/1861583842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072de0 0x7f3e4006e0b0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f3e400b31e0 tx=0x7f3e2c02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 -- 172.21.15.28:0/1861583842 shutdown_connections 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 --2- 172.21.15.28:0/1861583842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072de0 0x7f3e4006e0b0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 --2- 172.21.15.28:0/1861583842 >> v2:172.21.15.138:3300/0 conn(0x7f3e40072320 0x7f3e400cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 -- 172.21.15.28:0/1861583842 >> 172.21.15.28:0/1861583842 conn(0x7f3e40067d70 msgr2=0x7f3e4006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 -- 172.21.15.28:0/1861583842 shutdown_connections 2024-09-10T10:54:50.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.916+0000 7f3e45400640 1 -- 172.21.15.28:0/1861583842 wait complete. 2024-09-10T10:54:50.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.918+0000 7f3e45400640 1 Processor -- start 2024-09-10T10:54:50.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.918+0000 7f3e45400640 1 -- start start 2024-09-10T10:54:50.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40072700 0x7f3e400ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 0x7f3e400ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400cd520 con 0x7f3e40072320 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e400cd3a0 con 0x7f3e40072700 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e45400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e4006ed00 con 0x7f3e40072de0 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e3e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e3e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49462/0 (socket says 172.21.15.28:49462) 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e3e800640 1 -- 172.21.15.28:0/2467465842 learned_addr learned my addr 172.21.15.28:0/2467465842 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e3de00640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40072700 0x7f3e400ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.919+0000 7f3e3f200640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 0x7f3e400ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 -- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 msgr2=0x7f3e400ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 0x7f3e400ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 -- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40072700 msgr2=0x7f3e400ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40072700 0x7f3e400ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 -- 172.21.15.28:0/2467465842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e401f54f0 con 0x7f3e40072320 2024-09-10T10:54:50.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3f200640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 0x7f3e400ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:50.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e3e800640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9e0 tx=0x7f3e3000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:50.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30017070 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e3000c9d0 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000cdc0 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e401f57d0 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.920+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e401f5d90 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.921+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e400cce60 con 0x7f3e40072320 2024-09-10T10:54:50.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.922+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e3001a070 con 0x7f3e40072320 2024-09-10T10:54:50.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.923+0000 7f3e36a00640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 0x7f3e2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:50.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.923+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 9314+0+0 (secure 0 0 0) 0x7f3e3008bc20 con 0x7f3e40072320 2024-09-10T10:54:50.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.924+0000 7f3e3de00640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 0x7f3e2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.924+0000 7f3e3de00640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 0x7f3e2006e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:50.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:50.928+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e30059140 con 0x7f3e40072320 2024-09-10T10:54:51.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.148+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e4006bdb0 con 0x7f3e40072320 2024-09-10T10:54:51.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.149+0000 7f3e36a00640 1 -- 172.21.15.28:0/2467465842 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 300 v300) ==== 95+0+5221 (secure 0 0 0) 0x7f3e3005d440 con 0x7f3e40072320 2024-09-10T10:54:51.150 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:51.150 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":300,"btime":"2024-09-10T10:54:50:731251+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":300,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:50.731238+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":135,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9957},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9957":{"gid":9957,"name":"c","rank":0,"incarnation":300,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/382198082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":382198082},{"type":"v1","addr":"172.21.15.28:6834","nonce":382198082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:51.150 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 300 2024-09-10T10:54:51.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 msgr2=0x7f3e2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:51.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 0x7f3e2006e2f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c034000 tx=0x7f3e2c02e000 comp rx=0 tx=0).stop 2024-09-10T10:54:51.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 msgr2=0x7f3e400b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:51.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000e9e0 tx=0x7f3e3000eea0 comp rx=0 tx=0).stop 2024-09-10T10:54:51.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 shutdown_connections 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e2006be30 0x7f3e2006e2f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e40072de0 0x7f3e400ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40072700 0x7f3e400ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 --2- 172.21.15.28:0/2467465842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e40072320 0x7f3e400b0bc0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 >> 172.21.15.28:0/2467465842 conn(0x7f3e40067d70 msgr2=0x7f3e400c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 shutdown_connections 2024-09-10T10:54:51.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:51.151+0000 7f3e45400640 1 -- 172.21.15.28:0/2467465842 wait complete. 2024-09-10T10:54:51.168 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:54:53.753 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:54:53.752+0000 7f6a0b000640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:54:53.753 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:54:53.752+0000 7f6a0b000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T10:54:53.753 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:53.753+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:54:53.753 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:54:53.753+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:54:54.338 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:54:54.337+0000 7f6a10000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:56.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:56.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.335+0000 7f0276e00640 1 Processor -- start 2024-09-10T10:54:56.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.335+0000 7f0276e00640 1 -- start start 2024-09-10T10:54:56.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0276e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02700d0dd0 0x7f02700d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0276e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f02700d1f90 0x7f02700cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0276e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02700cf000 con 0x7f027015e660 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0276e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f027006d3e0 con 0x7f02700d0dd0 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0276e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f02700b37d0 con 0x7f02700d1f90 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0275a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f02700d0dd0 0x7f02700d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0275000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f027015e660 0x7f027006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58036/0 (socket says 172.21.15.28:58036) 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.336+0000 7f0275000640 1 -- 172.21.15.28:0/3775267033 learned_addr learned my addr 172.21.15.28:0/3775267033 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f0276400640 1 --2- 172.21.15.28:0/3775267033 >> v2:172.21.15.138:3300/0 conn(0x7f02700d1f90 0x7f02700cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f0275a00640 1 -- 172.21.15.28:0/3775267033 >> v2:172.21.15.138:3300/0 conn(0x7f02700d1f90 msgr2=0x7f02700cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:56.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f0275a00640 1 --2- 172.21.15.28:0/3775267033 >> v2:172.21.15.138:3300/0 conn(0x7f02700d1f90 0x7f02700cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f0275a00640 1 -- 172.21.15.28:0/3775267033 >> v1:172.21.15.28:6789/0 conn(0x7f027015e660 legacy=0x7f027006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f0275a00640 1 -- 172.21.15.28:0/3775267033 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02700d16f0 con 0x7f02700d0dd0 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.337+0000 7f026fe00640 1 -- 172.21.15.28:0/3775267033 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238838459 0 0) 0x7f02700cf000 con 0x7f027015e660 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.338+0000 7f0275a00640 1 --2- 172.21.15.28:0/3775267033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700d11b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f027008cb90 tx=0x7f026402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236121acee50f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.338+0000 7f026fe00640 1 -- 172.21.15.28:0/3775267033 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264037070 con 0x7f02700d0dd0 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.338+0000 7f026fe00640 1 -- 172.21.15.28:0/3775267033 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0264032440 con 0x7f02700d0dd0 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.338+0000 7f026fe00640 1 -- 172.21.15.28:0/3775267033 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264032810 con 0x7f02700d0dd0 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 -- 172.21.15.28:0/3775267033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 msgr2=0x7f02700d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 --2- 172.21.15.28:0/3775267033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700d11b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f027008cb90 tx=0x7f026402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 -- 172.21.15.28:0/3775267033 shutdown_connections 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 --2- 172.21.15.28:0/3775267033 >> v2:172.21.15.138:3300/0 conn(0x7f02700d1f90 0x7f02700cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 --2- 172.21.15.28:0/3775267033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700d11b0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 -- 172.21.15.28:0/3775267033 >> 172.21.15.28:0/3775267033 conn(0x7f027008bfc0 msgr2=0x7f02700d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 -- 172.21.15.28:0/3775267033 shutdown_connections 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.339+0000 7f0276e00640 1 -- 172.21.15.28:0/3775267033 wait complete. 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.341+0000 7f0276e00640 1 Processor -- start 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.341+0000 7f0276e00640 1 -- start start 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02700d1f90 0x7f02700cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 0x7f02700c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f02700d2c60 con 0x7f02700d0dd0 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02700cf330 con 0x7f02700d1f90 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02700d2f40 con 0x7f027015e660 2024-09-10T10:54:56.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48252/0 (socket says 172.21.15.28:48252) 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 -- 172.21.15.28:0/1139441071 learned_addr learned my addr 172.21.15.28:0/1139441071 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275000640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02700d1f90 0x7f02700cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276400640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 0x7f02700c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 -- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 msgr2=0x7f02700c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 0x7f02700c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 -- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02700d1f90 msgr2=0x7f02700cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02700d1f90 0x7f02700cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0275a00640 1 -- 172.21.15.28:0/1139441071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02700c9d20 con 0x7f02700d0dd0 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.342+0000 7f0276400640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 0x7f02700c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f0275a00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f0264035040 tx=0x7f02640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264037050 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0264038040 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264004470 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02700ca860 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.343+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02700ca2a0 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.344+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02700d5300 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.345+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0264003610 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.346+0000 7f026d600640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.346+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f02640ac760 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.347+0000 7f0275000640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.347+0000 7f0275000640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 0x7f025006e2f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f025c007fd0 tx=0x7f025c007de0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.351+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02700d5300 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.575+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02700d3220 con 0x7f02700d0dd0 2024-09-10T10:54:56.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.576+0000 7f026d600640 1 -- 172.21.15.28:0/1139441071 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+4389 (secure 0 0 0) 0x7f026407a1f0 con 0x7f02700d0dd0 2024-09-10T10:54:56.582 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:56.582 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":304,"btime":"2024-09-10T10:54:53:753508+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:53.753505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 304 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 msgr2=0x7f025006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 0x7f025006e2f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f025c007fd0 tx=0x7f025c007de0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 msgr2=0x7f02700b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f0264035040 tx=0x7f02640046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 shutdown_connections 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f025006be30 0x7f025006e2f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f027015e660 0x7f02700c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02700d1f90 0x7f02700cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 --2- 172.21.15.28:0/1139441071 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f02700d0dd0 0x7f02700b8f00 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 >> 172.21.15.28:0/1139441071 conn(0x7f027008bfc0 msgr2=0x7f02700b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:56.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 shutdown_connections 2024-09-10T10:54:56.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.578+0000 7f0276e00640 1 -- 172.21.15.28:0/1139441071 wait complete. 2024-09-10T10:54:56.595 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:54:56.595 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:56.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.751+0000 7fd09fe00640 1 Processor -- start 2024-09-10T10:54:56.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.751+0000 7fd09fe00640 1 -- start start 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd0a0073350 0x7fd0a01629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09fe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a015f410 con 0x7fd0a015e990 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a015f6f0 con 0x7fd0a0073350 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a0161f10 con 0x7fd0a006fc30 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.752+0000 7fd09ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd0a0073350 0x7fd0a01629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd0a0073350 0x7fd0a01629f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48260/0 (socket says 172.21.15.28:48260) 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 -- 172.21.15.28:0/3825201005 learned_addr learned my addr 172.21.15.28:0/3825201005 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:56.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09f400640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 -- 172.21.15.28:0/3825201005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 msgr2=0x7fd0a015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:56.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a015ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 -- 172.21.15.28:0/3825201005 >> v1:172.21.15.138:6789/0 conn(0x7fd0a015e990 legacy=0x7fd0a006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.753+0000 7fd09ea00640 1 -- 172.21.15.28:0/3825201005 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a01621f0 con 0x7fd0a0073350 2024-09-10T10:54:56.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.754+0000 7fd09ea00640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a01629f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd090009e50 tx=0x7fd09002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97a2e1c02ed69b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.754+0000 7fd09e000640 1 -- 172.21.15.28:0/3825201005 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090036070 con 0x7fd0a0073350 2024-09-10T10:54:56.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.754+0000 7fd09e000640 1 -- 172.21.15.28:0/3825201005 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd090031600 con 0x7fd0a0073350 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.754+0000 7fd09e000640 1 -- 172.21.15.28:0/3825201005 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0900319d0 con 0x7fd0a0073350 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 -- 172.21.15.28:0/3825201005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 msgr2=0x7fd0a01629f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a01629f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fd090009e50 tx=0x7fd09002a710 comp rx=0 tx=0).stop 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 -- 172.21.15.28:0/3825201005 shutdown_connections 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3825201005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a01629f0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 -- 172.21.15.28:0/3825201005 >> 172.21.15.28:0/3825201005 conn(0x7fd0a006c240 msgr2=0x7fd0a00b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 -- 172.21.15.28:0/3825201005 shutdown_connections 2024-09-10T10:54:56.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.755+0000 7fd09fe00640 1 -- 172.21.15.28:0/3825201005 wait complete. 2024-09-10T10:54:56.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.757+0000 7fd09fe00640 1 Processor -- start 2024-09-10T10:54:56.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.757+0000 7fd09fe00640 1 -- start start 2024-09-10T10:54:56.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a0155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 0x7fd0a00cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00b3290 con 0x7fd0a0073350 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00b3110 con 0x7fd0a015e990 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09fe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a0071cd0 con 0x7fd0a006fc30 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48266/0 (socket says 172.21.15.28:48266) 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 -- 172.21.15.28:0/3242077705 learned_addr learned my addr 172.21.15.28:0/3242077705 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09ea00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a0155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09f400640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 0x7fd0a00cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 -- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 msgr2=0x7fd0a0155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a0155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 -- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 msgr2=0x7fd0a00cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 0x7fd0a00cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd096000640 1 -- 172.21.15.28:0/3242077705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00afef0 con 0x7fd0a0073350 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.758+0000 7fd09f400640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 0x7fd0a00cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd096000640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c3c0 tx=0x7fd08800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880052d0 con 0x7fd0a0073350 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd088005490 con 0x7fd0a0073350 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a00b01d0 con 0x7fd0a0073350 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088007260 con 0x7fd0a0073350 2024-09-10T10:54:56.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.759+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a00b0a70 con 0x7fd0a0073350 2024-09-10T10:54:56.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.760+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a00b05c0 con 0x7fd0a0073350 2024-09-10T10:54:56.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.761+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd088007400 con 0x7fd0a0073350 2024-09-10T10:54:56.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.762+0000 7fd097400640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 0x7fd07c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:56.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.763+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fd08808b1b0 con 0x7fd0a0073350 2024-09-10T10:54:56.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.763+0000 7fd09ea00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 0x7fd07c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:56.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.764+0000 7fd09ea00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 0x7fd07c06e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd090033000 tx=0x7fd090009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:56.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.768+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd0880585f0 con 0x7fd0a0073350 2024-09-10T10:54:56.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.989+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0a01e80b0 con 0x7fd0a0073350 2024-09-10T10:54:56.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.989+0000 7fd097400640 1 -- 172.21.15.28:0/3242077705 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+4389 (secure 0 0 0) 0x7fd08805c8f0 con 0x7fd0a0073350 2024-09-10T10:54:56.990 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:56.990 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":304,"btime":"2024-09-10T10:54:53:753508+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:53.753505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:56.991 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 304 2024-09-10T10:54:56.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 msgr2=0x7fd07c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 0x7fd07c06e2f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd090033000 tx=0x7fd090009000 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 msgr2=0x7fd0a00c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c3c0 tx=0x7fd08800c880 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 shutdown_connections 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd07c06be30 0x7fd07c06e2f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd0a015e990 0x7fd0a00cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0a0073350 0x7fd0a00c9740 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 --2- 172.21.15.28:0/3242077705 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd0a006fc30 0x7fd0a0155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 >> 172.21.15.28:0/3242077705 conn(0x7fd0a006c240 msgr2=0x7fd0a0162840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 shutdown_connections 2024-09-10T10:54:56.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:56.991+0000 7fd09fe00640 1 -- 172.21.15.28:0/3242077705 wait complete. 2024-09-10T10:54:57.008 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:54:57.009 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:54:57.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.164+0000 7f9bc0c00640 1 Processor -- start 2024-09-10T10:54:57.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.164+0000 7f9bc0c00640 1 -- start start 2024-09-10T10:54:57.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bc0c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9bbc0d1f90 0x7f9bbc0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bc0c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bbc0cf000 con 0x7f9bbc15e660 2024-09-10T10:54:57.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bc0c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bbc0d29a0 con 0x7f9bbc0d0dd0 2024-09-10T10:54:57.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bc0c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9bbc06d3e0 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bb3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9bbc15e660 0x7f9bbc06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58048/0 (socket says 172.21.15.28:58048) 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.165+0000 7f9bb3e00640 1 -- 172.21.15.28:0/2083889181 learned_addr learned my addr 172.21.15.28:0/2083889181 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbbe00640 1 --2- 172.21.15.28:0/2083889181 >> v2:172.21.15.138:3300/0 conn(0x7f9bbc0d1f90 0x7f9bbc0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882622456 0 0) 0x7f9bbc0cf000 con 0x7f9bbc15e660 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ba0003520 con 0x7f9bbc15e660 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3989536992 0 0) 0x7f9bbc0d29a0 con 0x7f9bbc0d0dd0 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bbc0cf000 con 0x7f9bbc0d0dd0 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 819453726 0 0) 0x7f9ba0003520 con 0x7f9bbc15e660 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbbe00640 1 -- 172.21.15.28:0/2083889181 >> v1:172.21.15.138:6789/0 conn(0x7f9bbc0d0dd0 legacy=0x7f9bbc0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbbe00640 1 -- 172.21.15.28:0/2083889181 >> v1:172.21.15.28:6789/0 conn(0x7f9bbc15e660 legacy=0x7f9bbc06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.166+0000 7f9bbbe00640 1 -- 172.21.15.28:0/2083889181 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc0d2c60 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.167+0000 7f9bbbe00640 1 --2- 172.21.15.28:0/2083889181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cf7c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009d90 tx=0x7f9bb402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d69a4e6f00822e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4001670 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb4007540 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bbaa00640 1 -- 172.21.15.28:0/2083889181 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4007910 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 -- 172.21.15.28:0/2083889181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d1f90 msgr2=0x7f9bbc0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/2083889181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cf7c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009d90 tx=0x7f9bb402a680 comp rx=0 tx=0).stop 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 -- 172.21.15.28:0/2083889181 shutdown_connections 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/2083889181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cf7c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 -- 172.21.15.28:0/2083889181 >> 172.21.15.28:0/2083889181 conn(0x7f9bbc08bfc0 msgr2=0x7f9bbc0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 -- 172.21.15.28:0/2083889181 shutdown_connections 2024-09-10T10:54:57.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.168+0000 7f9bc0c00640 1 -- 172.21.15.28:0/2083889181 wait complete. 2024-09-10T10:54:57.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.170+0000 7f9bc0c00640 1 Processor -- start 2024-09-10T10:54:57.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.170+0000 7f9bc0c00640 1 -- start start 2024-09-10T10:54:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d0dd0 0x7f9bbc0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bbc15e660 0x7f9bbc0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc0cf330 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc0d3220 con 0x7f9bbc0d0dd0 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bc0c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9bbc0d33a0 con 0x7f9bbc15e660 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bb3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48274/0 (socket says 172.21.15.28:48274) 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bb3e00640 1 -- 172.21.15.28:0/3638200412 learned_addr learned my addr 172.21.15.28:0/3638200412 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bbb400640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d0dd0 0x7f9bbc0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.171+0000 7f9bbbe00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bbc15e660 0x7f9bbc0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 -- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bbc15e660 msgr2=0x7f9bbc0cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bbc15e660 0x7f9bbc0cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 -- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d0dd0 msgr2=0x7f9bbc0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d0dd0 0x7f9bbc0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 -- 172.21.15.28:0/3638200412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc0ce660 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3e00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001eb0 tx=0x7f9ba400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4005ad0 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba4005c70 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40072b0 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc0cd600 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.172+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc0cdb80 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.173+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bbc0d24d0 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.174+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9ba4007450 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.175+0000 7f9bb3400640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.175+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f9ba408ba80 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.176+0000 7f9bbb400640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.177+0000 7f9bbb400640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 0x7f9b9806e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0011e0 tx=0x7f9bac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.179+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9ba408de20 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.399+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bbc0d1b20 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.400+0000 7f9bb3400640 1 -- 172.21.15.28:0/3638200412 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 304 v304) ==== 95+0+4389 (secure 0 0 0) 0x7f9ba4010300 con 0x7f9bbc0d1f90 2024-09-10T10:54:57.401 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:54:57.401 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":304,"btime":"2024-09-10T10:54:53:753508+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":304,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:54:53.753505+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:54:57.401 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 304 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 msgr2=0x7f9b9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 0x7f9b9806e2f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0011e0 tx=0x7f9bac009040 comp rx=0 tx=0).stop 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 msgr2=0x7f9bbc0cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001eb0 tx=0x7f9ba400cbe0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 shutdown_connections 2024-09-10T10:54:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9b9806be30 0x7f9b9806e2f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9bbc15e660 0x7f9bbc0cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9bbc0d1f90 0x7f9bbc0cb490 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 --2- 172.21.15.28:0/3638200412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9bbc0d0dd0 0x7f9bbc0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 >> 172.21.15.28:0/3638200412 conn(0x7f9bbc08bfc0 msgr2=0x7f9bbc06e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 shutdown_connections 2024-09-10T10:54:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.402+0000 7f9bc0c00640 1 -- 172.21.15.28:0/3638200412 wait complete. 2024-09-10T10:54:57.419 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 600.00000000' led to DAMAGED state 2024-09-10T10:54:57.420 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_openfiles.0' 2024-09-10T10:54:57.420 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:54:57.420 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:54:57.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:54:57.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:54:57.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:54:57.455 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:54:57.530 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:54:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:54:57.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:54:57.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.727+0000 7fdf74a00640 1 Processor -- start 2024-09-10T10:54:57.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.727+0000 7fdf74a00640 1 -- start start 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d0dd0 0x7fdf700d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdf7015e660 0x7fdf7006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf700d1f90 0x7fdf700cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdf700b37d0 con 0x7fdf700d1f90 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdf700cf330 con 0x7fdf7015e660 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf74a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700cf000 con 0x7fdf700d0dd0 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.728+0000 7fdf6f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf700d1f90 0x7fdf700cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf700d1f90 0x7fdf700cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48284/0 (socket says 172.21.15.28:48284) 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 -- 172.21.15.28:0/1156835250 learned_addr learned my addr 172.21.15.28:0/1156835250 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6e800640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d0dd0 0x7fdf700d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6de00640 1 --2- 172.21.15.28:0/1156835250 >> v2:172.21.15.138:3300/0 conn(0x7fdf7015e660 0x7fdf7006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 -- 172.21.15.28:0/1156835250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d0dd0 msgr2=0x7fdf700d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:57.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d0dd0 0x7fdf700d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 -- 172.21.15.28:0/1156835250 >> v2:172.21.15.138:3300/0 conn(0x7fdf7015e660 msgr2=0x7fdf7006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:57.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 --2- 172.21.15.28:0/1156835250 >> v2:172.21.15.138:3300/0 conn(0x7fdf7015e660 0x7fdf7006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.729+0000 7fdf6f200640 1 -- 172.21.15.28:0/1156835250 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700d16f0 con 0x7fdf700d1f90 2024-09-10T10:54:57.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.730+0000 7fdf6f200640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf700d1f90 0x7fdf700cf7c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b3dd0 tx=0x7fdf6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befe4a6355953c5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.730+0000 7fdf6d400640 1 -- 172.21.15.28:0/1156835250 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68036070 con 0x7fdf700d1f90 2024-09-10T10:54:57.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.730+0000 7fdf6d400640 1 -- 172.21.15.28:0/1156835250 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf68031600 con 0x7fdf700d1f90 2024-09-10T10:54:57.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.730+0000 7fdf6d400640 1 -- 172.21.15.28:0/1156835250 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf680319d0 con 0x7fdf700d1f90 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 -- 172.21.15.28:0/1156835250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf700d1f90 msgr2=0x7fdf700cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf700d1f90 0x7fdf700cf7c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fdf700b3dd0 tx=0x7fdf6802a6a0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 -- 172.21.15.28:0/1156835250 shutdown_connections 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf700d1f90 0x7fdf700cf7c0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 --2- 172.21.15.28:0/1156835250 >> v2:172.21.15.138:3300/0 conn(0x7fdf7015e660 0x7fdf7006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 --2- 172.21.15.28:0/1156835250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d0dd0 0x7fdf700d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 -- 172.21.15.28:0/1156835250 >> 172.21.15.28:0/1156835250 conn(0x7fdf7008bfc0 msgr2=0x7fdf700d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 -- 172.21.15.28:0/1156835250 shutdown_connections 2024-09-10T10:54:57.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.731+0000 7fdf74a00640 1 -- 172.21.15.28:0/1156835250 wait complete. 2024-09-10T10:54:57.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.732+0000 7fdf74a00640 1 Processor -- start 2024-09-10T10:54:57.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 -- start start 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf700d0dd0 0x7fdf701e8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 0x7fdf700cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700d2f40 con 0x7fdf7015e660 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700d2c60 con 0x7fdf700d0dd0 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.733+0000 7fdf74a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700d3220 con 0x7fdf700d1f90 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48296/0 (socket says 172.21.15.28:48296) 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 -- 172.21.15.28:0/578080877 learned_addr learned my addr 172.21.15.28:0/578080877 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6de00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 0x7fdf700cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6e800640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf700d0dd0 0x7fdf701e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 -- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 msgr2=0x7fdf700cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 0x7fdf700cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 -- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf700d0dd0 msgr2=0x7fdf701e8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf700d0dd0 0x7fdf701e8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 -- 172.21.15.28:0/578080877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700cc6f0 con 0x7fdf7015e660 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6de00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 0x7fdf700cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.734+0000 7fdf6f200640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fdf68034040 tx=0x7fdf68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.735+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68041020 con 0x7fdf7015e660 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.735+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf680077a0 con 0x7fdf7015e660 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.735+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68036040 con 0x7fdf7015e660 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.735+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf700cc9d0 con 0x7fdf7015e660 2024-09-10T10:54:57.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.735+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf700ccf90 con 0x7fdf7015e660 2024-09-10T10:54:57.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.736+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7004ffb0 con 0x7fdf7015e660 2024-09-10T10:54:57.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.737+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdf68044020 con 0x7fdf7015e660 2024-09-10T10:54:57.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.738+0000 7fdf66a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 0x7fdf5006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:57.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.738+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fdf680ac940 con 0x7fdf7015e660 2024-09-10T10:54:57.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.739+0000 7fdf6e800640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 0x7fdf5006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:57.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.739+0000 7fdf6e800640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 0x7fdf5006e320 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdf60006fd0 tx=0x7fdf60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:57.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.743+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf7004ffb0 con 0x7fdf7015e660 2024-09-10T10:54:57.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:57.963+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdf700d1b20 con 0x7fdf7015e660 2024-09-10T10:54:58.507 INFO:tasks.ceph.mds.c.smithi028.stderr: -49> 2024-09-10T10:54:53.752+0000 7f6a0b000640 -1 mds.0.cache.dir(0x600) Corrupt fnode in dirfrag 0x600: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:54:58.507 INFO:tasks.ceph.mds.c.smithi028.stderr: -46> 2024-09-10T10:54:53.752+0000 7f6a0b000640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x600 is fatal because it is a system directory for this rank 2024-09-10T10:54:58.507 INFO:tasks.ceph.mds.c.smithi028.stderr: -34> 2024-09-10T10:54:54.337+0000 7f6a10000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:54:58.548 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:54:58.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.776+0000 7fdf66a00640 1 -- 172.21.15.28:0/578080877 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v305) ==== 161+0+0 (secure 0 0 0) 0x7fdf6804b030 con 0x7fdf7015e660 2024-09-10T10:54:58.777 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 msgr2=0x7fdf5006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 0x7fdf5006e320 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fdf60006fd0 tx=0x7fdf60008040 comp rx=0 tx=0).stop 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 msgr2=0x7fdf701e9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fdf68034040 tx=0x7fdf68004a40 comp rx=0 tx=0).stop 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 shutdown_connections 2024-09-10T10:54:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdf5006be60 0x7fdf5006e320 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf7015e660 0x7fdf701e9420 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdf700d1f90 0x7fdf700cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 --2- 172.21.15.28:0/578080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf700d0dd0 0x7fdf701e8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 >> 172.21.15.28:0/578080877 conn(0x7fdf7008bfc0 msgr2=0x7fdf700b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 shutdown_connections 2024-09-10T10:54:58.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.779+0000 7fdf74a00640 1 -- 172.21.15.28:0/578080877 wait complete. 2024-09-10T10:54:58.799 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:54:58.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.960+0000 7f82e4600640 1 Processor -- start 2024-09-10T10:54:58.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.960+0000 7f82e4600640 1 -- start start 2024-09-10T10:54:58.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e4600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82dc072de0 0x7f82dc15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 0x7f82dc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e4600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82dc146140 con 0x7f82dc072320 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e4600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f82dc0cd740 con 0x7f82dc072de0 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc0cda20 con 0x7f82dc072700 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82dc072de0 0x7f82dc15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f82dc072de0 0x7f82dc15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48324/0 (socket says 172.21.15.28:48324) 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 -- 172.21.15.28:0/3122843611 learned_addr learned my addr 172.21.15.28:0/3122843611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 -- 172.21.15.28:0/3122843611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 msgr2=0x7f82dc15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e2600640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 0x7f82dc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 0x7f82dc15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.962+0000 7f82e1200640 1 -- 172.21.15.28:0/3122843611 >> v1:172.21.15.138:6789/0 conn(0x7f82dc072320 legacy=0x7f82dc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:58.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.963+0000 7f82e1200640 1 -- 172.21.15.28:0/3122843611 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc0cdd00 con 0x7f82dc072de0 2024-09-10T10:54:58.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.963+0000 7f82e2600640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 0x7f82dc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.963+0000 7f82e1200640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072de0 0x7f82dc15d1e0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009870 tx=0x7f82c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189ea172ca78df26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82dbe00640 1 -- 172.21.15.28:0/3122843611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8037070 con 0x7f82dc072de0 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82dbe00640 1 -- 172.21.15.28:0/3122843611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c8032440 con 0x7f82dc072de0 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82dbe00640 1 -- 172.21.15.28:0/3122843611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8032810 con 0x7f82dc072de0 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 -- 172.21.15.28:0/3122843611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072de0 msgr2=0x7f82dc15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072de0 0x7f82dc15d1e0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009870 tx=0x7f82c802a180 comp rx=0 tx=0).stop 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 -- 172.21.15.28:0/3122843611 shutdown_connections 2024-09-10T10:54:58.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072700 0x7f82dc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 --2- 172.21.15.28:0/3122843611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072de0 0x7f82dc15d1e0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.964+0000 7f82e4600640 1 -- 172.21.15.28:0/3122843611 >> 172.21.15.28:0/3122843611 conn(0x7f82dc067d70 msgr2=0x7f82dc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.965+0000 7f82e4600640 1 -- 172.21.15.28:0/3122843611 shutdown_connections 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.965+0000 7f82e4600640 1 -- 172.21.15.28:0/3122843611 wait complete. 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.966+0000 7f82e4600640 1 Processor -- start 2024-09-10T10:54:58.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.966+0000 7f82e4600640 1 -- start start 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072320 0x7f82dc140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 0x7f82dc1e5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc15dde0 con 0x7f82dc072700 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc15dc60 con 0x7f82dc072de0 2024-09-10T10:54:58.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f82dc15df60 con 0x7f82dc072320 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.967+0000 7f82e1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48340/0 (socket says 172.21.15.28:48340) 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 -- 172.21.15.28:0/3361066941 learned_addr learned my addr 172.21.15.28:0/3361066941 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1c00640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072320 0x7f82dc140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e2600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 0x7f82dc1e5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 -- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072320 msgr2=0x7f82dc140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072320 0x7f82dc140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 -- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 msgr2=0x7f82dc1e5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 0x7f82dc1e5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 -- 172.21.15.28:0/3361066941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc1e5eb0 con 0x7f82dc072700 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e2600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 0x7f82dc1e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:58.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.968+0000 7f82e1200640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009870 tx=0x7f82c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.969+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8041020 con 0x7f82dc072700 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.969+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c8030970 con 0x7f82dc072700 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.969+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8037040 con 0x7f82dc072700 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.969+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc1e6190 con 0x7f82dc072700 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.969+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1e7590 con 0x7f82dc072700 2024-09-10T10:54:58.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.970+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc15e370 con 0x7f82dc072700 2024-09-10T10:54:58.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.971+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f82c8044020 con 0x7f82dc072700 2024-09-10T10:54:58.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.972+0000 7f82d9600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 0x7f82bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:58.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.972+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f82c80ac630 con 0x7f82dc072700 2024-09-10T10:54:58.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.972+0000 7f82e1c00640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 0x7f82bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:58.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.973+0000 7f82e1c00640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 0x7f82bc06e2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82d0004f60 tx=0x7f82d0007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:58.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:58.977+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f82c804b030 con 0x7f82dc072700 2024-09-10T10:54:59.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.233+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7f82dc06e810 con 0x7f82dc072700 2024-09-10T10:54:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.778+0000 7f82d9600640 1 -- 172.21.15.28:0/3361066941 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v306) ==== 101+0+0 (secure 0 0 0) 0x7f82c807a150 con 0x7f82dc072700 2024-09-10T10:54:59.779 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 msgr2=0x7f82bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 0x7f82bc06e2f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82d0004f60 tx=0x7f82d0007e30 comp rx=0 tx=0).stop 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 msgr2=0x7f82dc1e7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009870 tx=0x7f82c80046c0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 shutdown_connections 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f82bc06be30 0x7f82bc06e2f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82dc072de0 0x7f82dc1e5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f82dc072700 0x7f82dc1e7050 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 --2- 172.21.15.28:0/3361066941 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f82dc072320 0x7f82dc140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 >> 172.21.15.28:0/3361066941 conn(0x7f82dc067d70 msgr2=0x7f82dc0b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 shutdown_connections 2024-09-10T10:54:59.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.781+0000 7f82e4600640 1 -- 172.21.15.28:0/3361066941 wait complete. 2024-09-10T10:54:59.801 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:54:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.878+0000 7f4702233f40 1 Processor -- start 2024-09-10T10:54:59.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.878+0000 7f4702233f40 1 -- start start 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07a3e0 0x55764e070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07d9c0 0x55764dfd3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55764df4cb30 con 0x55764dfa4470 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55764df50b70 con 0x55764e07d9c0 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f4702233f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55764e07cee0 con 0x55764e07a3e0 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.879+0000 7f46ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48350/0 (socket says 172.21.15.28:48350) 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 -- 172.21.15.28:0/320102104 learned_addr learned my addr 172.21.15.28:0/320102104 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ffe00640 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07d9c0 0x55764dfd3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46fea00640 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07a3e0 0x55764e070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 -- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07a3e0 msgr2=0x55764e070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07a3e0 0x55764e070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 -- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07d9c0 msgr2=0x55764dfd3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07d9c0 0x55764dfd3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 -- 172.21.15.28:0/320102104 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764df90010 con 0x55764dfa4470 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46ff400640 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001d80 tx=0x7f46f0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d5fa5862f80e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46fd600640 1 -- 172.21.15.28:0/320102104 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000bd10 con 0x55764dfa4470 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46fd600640 1 -- 172.21.15.28:0/320102104 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f000f400 con 0x55764dfa4470 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.880+0000 7f46fd600640 1 -- 172.21.15.28:0/320102104 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000f7f0 con 0x55764dfa4470 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 -- 172.21.15.28:0/320102104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 msgr2=0x55764dff1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:59.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001d80 tx=0x7f46f0001e70 comp rx=0 tx=0).stop 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 -- 172.21.15.28:0/320102104 shutdown_connections 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07d9c0 0x55764dfd3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07a3e0 0x55764e070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 --2- 172.21.15.28:0/320102104 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764dff1860 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 -- 172.21.15.28:0/320102104 >> 172.21.15.28:0/320102104 conn(0x55764dff30a0 msgr2=0x55764dff3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 -- 172.21.15.28:0/320102104 shutdown_connections 2024-09-10T10:54:59.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.881+0000 7f4702233f40 1 -- 172.21.15.28:0/320102104 wait complete. 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.882+0000 7f4702233f40 1 Processor -- start 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.882+0000 7f4702233f40 1 -- start start 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764e06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07a3e0 0x55764dfd3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07d9c0 0x55764dfd3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55764dff13d0 con 0x55764dfa4470 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55764df902f0 con 0x55764e07a3e0 2024-09-10T10:54:59.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f4702233f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55764dfd7ec0 con 0x55764e07d9c0 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764e06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764e06db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48366/0 (socket says 172.21.15.28:48366) 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 learned_addr learned my addr 172.21.15.28:0/1921130504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07a3e0 0x55764dfd3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07d9c0 0x55764dfd3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07d9c0 msgr2=0x55764dfd3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07d9c0 0x55764dfd3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07a3e0 msgr2=0x55764dfd3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:54:59.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55764e07a3e0 0x55764dfd3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55764e0808c0 con 0x55764dfa4470 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.883+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55764e07d9c0 0x55764dfd3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764e06db90 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x55764e074e20 tx=0x7f46f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f46f7e00640 1 -- 172.21.15.28:0/1921130504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f00012d0 con 0x55764dfa4470 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f46f7e00640 1 -- 172.21.15.28:0/1921130504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f0021020 con 0x55764dfa4470 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f46f7e00640 1 -- 172.21.15.28:0/1921130504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f001c440 con 0x55764dfa4470 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55764e07df50 con 0x55764dfa4470 2024-09-10T10:54:59.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.884+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55764e079980 con 0x55764dfa4470 2024-09-10T10:54:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.886+0000 7f46f7e00640 1 -- 172.21.15.28:0/1921130504 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f46f0017070 con 0x55764dfa4470 2024-09-10T10:54:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.887+0000 7f46f7e00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46e006a810 0x7f46e006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.887+0000 7f46f7e00640 1 -- 172.21.15.28:0/1921130504 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f46f008be10 con 0x55764dfa4470 2024-09-10T10:54:59.889 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.887+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55764e070810 0x55764e072140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.887+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46e006a810 0x7f46e006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.887+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e072680 con 0x55764e070810 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.888+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55764e070810 0x55764e072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.888+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46e006a810 0x7f46e006ccd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55764dfd8590 tx=0x7f46ec0017f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.888+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55764e070810 0x55764e072140 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.891+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v136'1097 uv1070 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.891+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e072ab0 con 0x55764e070810 2024-09-10T10:54:59.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.893+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v136'1098 uv1098 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.893+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11f3a0 con 0x55764e070810 2024-09-10T10:54:59.896 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:54:59.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.895+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v136'1099 uv1099 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.895+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120750 con 0x55764e070810 2024-09-10T10:54:59.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.897+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v136'1100 uv1100 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.897+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120af0 con 0x55764e070810 2024-09-10T10:54:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.899+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v136'1101 uv1101 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.899+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11f740 con 0x55764e070810 2024-09-10T10:54:59.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.901+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v136'1102 uv1102 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.901+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fb30 con 0x55764e070810 2024-09-10T10:54:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.903+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v136'1103 uv1076 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.903+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e11ff20 con 0x55764e070810 2024-09-10T10:54:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.905+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v136'1104 uv1104 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.905+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11ff20 con 0x55764e070810 2024-09-10T10:54:59.907 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:54:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.906+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v136'1105 uv1105 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.906+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.908+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v136'1106 uv1106 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.908+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.910+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v136'1107 uv1107 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.910+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.912+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v136'1108 uv1108 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.912+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.914+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v136'1109 uv1083 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.914+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.915+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v136'1110 uv1110 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.915+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.918 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:54:59.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.917+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v136'1111 uv1111 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.917+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.919+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v136'1112 uv1112 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.919+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.921+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v136'1113 uv1113 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.921+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e11fed0 con 0x55764e070810 2024-09-10T10:54:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.922+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v136'1114 uv1114 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.922+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.924+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v136'1115 uv1115 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.924+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.926+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v136'1116 uv1096 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.926+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.928+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v136'1117 uv1117 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.928+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.930 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:54:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.929+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v136'1118 uv1118 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.930+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e121ea0 con 0x55764e070810 2024-09-10T10:54:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.931+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v136'1119 uv1119 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.931+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e121ea0 con 0x55764e070810 2024-09-10T10:54:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.933+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v136'1120 uv1120 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.933+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.935+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v136'1121 uv1121 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.935+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.937+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v136'1122 uv1095 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.937+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.939+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v136'1123 uv1123 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.939+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.941 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:54:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.940+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v136'1124 uv1124 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.940+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e121ea0 con 0x55764e070810 2024-09-10T10:54:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.942+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v136'1125 uv1125 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.942+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e121ea0 con 0x55764e070810 2024-09-10T10:54:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.943+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v136'1126 uv1126 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.943+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e120e90 con 0x55764e070810 2024-09-10T10:54:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.945+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v136'1127 uv1127 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e070810 2024-09-10T10:54:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.945+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55764e1213a0 0x55764e123790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.945+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e123cd0 con 0x55764e1213a0 2024-09-10T10:54:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.946+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55764e1213a0 0x55764e123790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.946+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55764e1213a0 0x55764e123790 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.949+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v136'799 uv781 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.949+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e124070 con 0x55764e1213a0 2024-09-10T10:54:59.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.950+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v136'800 uv800 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.950+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e124410 con 0x55764e1213a0 2024-09-10T10:54:59.952 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:54:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.952+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v136'801 uv801 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.952+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e1247b0 con 0x55764e1213a0 2024-09-10T10:54:59.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.953+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v136'802 uv802 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.953+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e124b50 con 0x55764e1213a0 2024-09-10T10:54:59.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.955+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v136'803 uv803 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.955+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e124ef0 con 0x55764e1213a0 2024-09-10T10:54:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.957+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v136'804 uv786 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.957+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e125290 con 0x55764e1213a0 2024-09-10T10:54:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.958+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v136'805 uv805 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.959+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.961 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:54:59.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.960+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v136'806 uv806 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.960+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.961+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v136'807 uv807 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.961+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.963+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v136'808 uv808 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.963+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.964+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v136'809 uv791 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.964+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.966+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v136'810 uv810 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.966+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.968 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:54:59.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.967+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v136'811 uv811 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.967+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129d20 con 0x55764e1213a0 2024-09-10T10:54:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.969+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v136'812 uv812 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.969+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129d20 con 0x55764e1213a0 2024-09-10T10:54:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.970+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v136'813 uv813 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.971+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.972+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v136'814 uv797 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.972+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.974+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v136'815 uv815 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.974+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.976 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:54:59.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.975+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v136'816 uv816 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.975+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e126640 con 0x55764e1213a0 2024-09-10T10:54:59.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.977+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v136'817 uv817 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.977+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e126640 con 0x55764e1213a0 2024-09-10T10:54:59.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.978+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v136'818 uv818 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.978+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e125630 con 0x55764e1213a0 2024-09-10T10:54:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.979+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v136'819 uv819 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e1213a0 2024-09-10T10:54:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.979+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55764e125cb0 0x55764e1280a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.979+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e1285e0 con 0x55764e125cb0 2024-09-10T10:54:59.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.980+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55764e125cb0 0x55764e1280a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.980+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55764e125cb0 0x55764e1280a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.982+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v136'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.982+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e128980 con 0x55764e125cb0 2024-09-10T10:54:59.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.983+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v136'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.983+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e128d20 con 0x55764e125cb0 2024-09-10T10:54:59.985 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:54:59.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.985+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v136'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.985+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e074230 con 0x55764e125cb0 2024-09-10T10:54:59.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.986+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v136'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.986+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12a5d0 con 0x55764e125cb0 2024-09-10T10:54:59.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.988+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v136'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.988+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129570 con 0x55764e125cb0 2024-09-10T10:54:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.989+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v136'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.989+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55764e12a030 0x55764e12c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:54:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.989+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12cf50 con 0x55764e12a030 2024-09-10T10:54:59.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.989+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55764e12a030 0x55764e12c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:54:59.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.990+0000 7f46ffe00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55764e12a030 0x55764e12c9d0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:54:59.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.991+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v136'157 uv156 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:54:59.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.991+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d340 con 0x55764e12a030 2024-09-10T10:54:59.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.993+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v136'158 uv158 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:54:59.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.993+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d340 con 0x55764e12a030 2024-09-10T10:54:59.994 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:54:59.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.994+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v136'159 uv159 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:54:59.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.994+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d340 con 0x55764e12a030 2024-09-10T10:54:59.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.995+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v136'160 uv160 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:54:59.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.995+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e125cb0 2024-09-10T10:54:59.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.996+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v136'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.996+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e125cb0 2024-09-10T10:54:59.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.997+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v136'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:54:59.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.997+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e125cb0 2024-09-10T10:54:59.999 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:55:00.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.999+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v136'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:54:59.999+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e770 con 0x55764e125cb0 2024-09-10T10:55:00.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.000+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v136'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.000+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e770 con 0x55764e125cb0 2024-09-10T10:55:00.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.001+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v136'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.001+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e125cb0 2024-09-10T10:55:00.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.002+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v136'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.002+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e12a030 2024-09-10T10:55:00.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.004+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v136'393 uv387 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.004+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d760 con 0x55764e12a030 2024-09-10T10:55:00.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.006+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v136'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.006+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12d3c0 con 0x55764e12a030 2024-09-10T10:55:00.008 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:55:00.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.007+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v136'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.007+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e430 con 0x55764e12a030 2024-09-10T10:55:00.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.008+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v136'396 uv396 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.008+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.010+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v136'397 uv397 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.010+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.012+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v136'398 uv398 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.012+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.013+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v136'399 uv392 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.013+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.014+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v136'400 uv400 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.014+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.016 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:55:00.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.016+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v136'401 uv401 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.016+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.017+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v136'402 uv402 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.017+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e12a030 2024-09-10T10:55:00.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.019+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v136'403 uv403 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.019+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e125cb0 2024-09-10T10:55:00.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.020+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v136'178 uv177 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.020+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e125cb0 2024-09-10T10:55:00.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.022+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v136'179 uv179 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.022+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e125cb0 2024-09-10T10:55:00.024 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:55:00.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.023+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v136'180 uv180 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.023+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e125cb0 2024-09-10T10:55:00.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.024+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v136'181 uv181 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.024+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e12e3f0 con 0x55764e125cb0 2024-09-10T10:55:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.026+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v136'182 uv182 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e125cb0 2024-09-10T10:55:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.026+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55764e12d900 0x55764e12dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.026+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130840 con 0x55764e12d900 2024-09-10T10:55:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.026+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55764e12d900 0x55764e12dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.026+0000 7f46ff400640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55764e12d900 0x55764e12dd40 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.028+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v136'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.028+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e130840 con 0x55764e12d900 2024-09-10T10:55:00.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.029+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v136'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.029+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130840 con 0x55764e12d900 2024-09-10T10:55:00.032 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:55:00.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.031+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v136'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.031+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e131c70 con 0x55764e12d900 2024-09-10T10:55:00.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.032+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v136'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.032+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e131c70 con 0x55764e12d900 2024-09-10T10:55:00.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.033+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v136'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.033+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130980 con 0x55764e12d900 2024-09-10T10:55:00.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.035+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v136'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.035+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e1308e0 con 0x55764e12d900 2024-09-10T10:55:00.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.037+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v136'355 uv349 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.037+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e130880 con 0x55764e12d900 2024-09-10T10:55:00.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.038+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v136'356 uv356 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.038+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130880 con 0x55764e12d900 2024-09-10T10:55:00.040 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:55:00.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.039+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v136'357 uv357 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.039+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130d20 con 0x55764e12d900 2024-09-10T10:55:00.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.041+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v136'358 uv358 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.041+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130a10 con 0x55764e12d900 2024-09-10T10:55:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.042+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v136'359 uv359 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.042+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130a10 con 0x55764e12d900 2024-09-10T10:55:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.044+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v136'360 uv354 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.044+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e130960 con 0x55764e12d900 2024-09-10T10:55:00.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.045+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v136'361 uv361 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.045+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130940 con 0x55764e12d900 2024-09-10T10:55:00.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:55:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.046+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v136'362 uv362 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.046+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e130940 con 0x55764e12d900 2024-09-10T10:55:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.048+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v136'363 uv363 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.048+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e1291b0 con 0x55764e12d900 2024-09-10T10:55:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.049+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v136'364 uv364 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.049+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.051+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v136'157 uv155 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.051+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.052+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v136'158 uv158 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.052+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.054 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:55:00.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.053+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v136'159 uv159 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.053+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.054+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v136'160 uv160 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.055+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.056+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v136'213 uv212 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.056+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.057+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v136'214 uv214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.057+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.059 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:55:00.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.058+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v136'215 uv215 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.058+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129170 con 0x55764e12a030 2024-09-10T10:55:00.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.059+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v136'216 uv216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.059+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129130 con 0x55764e12a030 2024-09-10T10:55:00.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.061+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v136'217 uv217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.061+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e129130 con 0x55764e12a030 2024-09-10T10:55:00.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.062+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v136'218 uv218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46f804f970 con 0x55764e12a030 2024-09-10T10:55:00.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.062+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55764e1317b0 0x55764e133f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.062+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e1317b0 2024-09-10T10:55:00.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.062+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55764e1317b0 0x55764e133f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.063+0000 7f46fea00640 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55764e1317b0 0x55764e133f00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.064+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v136'181 uv180 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e1317b0 2024-09-10T10:55:00.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.064+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e1317b0 2024-09-10T10:55:00.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.065+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v136'182 uv182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e1317b0 2024-09-10T10:55:00.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.065+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e1317b0 2024-09-10T10:55:00.067 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:55:00.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.066+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v136'183 uv183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e1317b0 2024-09-10T10:55:00.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.066+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e1317b0 2024-09-10T10:55:00.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.067+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v136'184 uv184 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e1317b0 2024-09-10T10:55:00.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.068+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e1317b0 2024-09-10T10:55:00.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.069+0000 7f46fea00640 1 -- 172.21.15.28:0/1921130504 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v136'185 uv185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f46ec009030 con 0x55764e1317b0 2024-09-10T10:55:00.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.069+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.070+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v136'442 uv433 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.070+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.072+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v136'443 uv443 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.072+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.073 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:55:00.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.073+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v136'444 uv444 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.073+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.074+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v136'445 uv445 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.074+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.075+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v136'446 uv446 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.075+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.076+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v136'447 uv440 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.076+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.077+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v136'448 uv448 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.077+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.079 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:55:00.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.079+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v136'449 uv449 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.079+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e135450 con 0x55764e12d900 2024-09-10T10:55:00.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.080+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v136'450 uv450 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.080+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e135450 con 0x55764e12d900 2024-09-10T10:55:00.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.081+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v136'451 uv451 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.081+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e134440 con 0x55764e12d900 2024-09-10T10:55:00.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.082+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v136'452 uv452 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.082+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55764e1348f0 con 0x55764e12d900 2024-09-10T10:55:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f46ff400640 1 -- 172.21.15.28:0/1921130504 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v136'453 uv453 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f46f00558a0 con 0x55764e12d900 2024-09-10T10:55:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55764e125cb0 msgr2=0x55764e1280a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55764e125cb0 0x55764e1280a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55764e12d900 msgr2=0x55764e12dd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55764e12d900 0x55764e12dd40 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55764e1317b0 msgr2=0x55764e133f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55764e1317b0 0x55764e133f00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55764e12a030 msgr2=0x55764e12c9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55764e12a030 0x55764e12c9d0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 reap_dead start 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55764e1213a0 msgr2=0x55764e123790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55764e1213a0 0x55764e123790 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55764e070810 msgr2=0x55764e072140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55764e070810 0x55764e072140 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46e006a810 msgr2=0x7f46e006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f46e006a810 0x7f46e006ccd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55764dfd8590 tx=0x7f46ec0017f0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 msgr2=0x55764e06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 --2- 172.21.15.28:0/1921130504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55764dfa4470 0x55764e06db90 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x55764e074e20 tx=0x7f46f0000bc0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f46ffe00640 1 -- 172.21.15.28:0/1921130504 reap_dead start 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 shutdown_connections 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 >> 172.21.15.28:0/1921130504 conn(0x55764dff30a0 msgr2=0x55764dff3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 shutdown_connections 2024-09-10T10:55:00.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.084+0000 7f4702233f40 1 -- 172.21.15.28:0/1921130504 wait complete. 2024-09-10T10:55:00.090 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_openfiles.0 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:55:00.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.208+0000 7fe4aa9dcf40 1 Processor -- start 2024-09-10T10:55:00.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.208+0000 7fe4aa9dcf40 1 -- start start 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c0540ff0 0x55c8c0543ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 0x55c8c05c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 0x55c8c05429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c8c0541530 con 0x55c8c05cf9c0 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8c049eb30 con 0x55c8c05cc3e0 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c8c05ceee0 con 0x55c8c0540ff0 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 0x55c8c05429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 0x55c8c05c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a8600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 0x55c8c05429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48374/0 (socket says 172.21.15.28:48374) 2024-09-10T10:55:00.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c0540ff0 0x55c8c0543ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a8600640 1 -- 172.21.15.28:0/4257760806 learned_addr learned my addr 172.21.15.28:0/4257760806 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 -- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c0540ff0 msgr2=0x55c8c0543ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c0540ff0 0x55c8c0543ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 -- 172.21.15.28:0/4257760806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 msgr2=0x55c8c05429c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 0x55c8c05429c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.209+0000 7fe4a7200640 1 -- 172.21.15.28:0/4257760806 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c04f6220 con 0x55c8c05cc3e0 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4a7200640 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 0x55c8c05c30d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55c8c04f49a0 tx=0x7fe49400b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf3aee7f6f1700b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4a5e00640 1 -- 172.21.15.28:0/4257760806 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494012020 con 0x55c8c05cc3e0 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4a5e00640 1 -- 172.21.15.28:0/4257760806 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49400e470 con 0x55c8c05cc3e0 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4a5e00640 1 -- 172.21.15.28:0/4257760806 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49400e8c0 con 0x55c8c05cc3e0 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 msgr2=0x55c8c05c30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.210+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 0x55c8c05c30d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55c8c04f49a0 tx=0x7fe49400b6c0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/4257760806 shutdown_connections 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c05cf9c0 0x55c8c05429c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cc3e0 0x55c8c05c30d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/4257760806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c0540ff0 0x55c8c0543ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/4257760806 >> 172.21.15.28:0/4257760806 conn(0x55c8c05450a0 msgr2=0x55c8c0545480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/4257760806 shutdown_connections 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/4257760806 wait complete. 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 Processor -- start 2024-09-10T10:55:00.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.211+0000 7fe4aa9dcf40 1 -- start start 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 0x55c8c05cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c05cc3e0 0x55c8c05c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cf9c0 0x55c8c05c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55c8c05ce6b0 con 0x55c8c0540ff0 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c8c05cce10 con 0x55c8c05cf9c0 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.212+0000 7fe4aa9dcf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c8c04f6500 con 0x55c8c05cc3e0 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 0x55c8c05cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 0x55c8c05cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48386/0 (socket says 172.21.15.28:48386) 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 -- 172.21.15.28:0/511854605 learned_addr learned my addr 172.21.15.28:0/511854605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7200640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c05cc3e0 0x55c8c05c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a8600640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cf9c0 0x55c8c05c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 -- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c05cc3e0 msgr2=0x55c8c05c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c8c05cc3e0 0x55c8c05c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 -- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cf9c0 msgr2=0x55c8c05c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c8c05cf9c0 0x55c8c05c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 -- 172.21.15.28:0/511854605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8c05c22f0 con 0x55c8c0540ff0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a7c00640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 0x55c8c05cb5e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe498008a10 tx=0x7fe498008b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.213+0000 7fe4a4a00640 1 -- 172.21.15.28:0/511854605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498019070 con 0x55c8c0540ff0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.214+0000 7fe4a4a00640 1 -- 172.21.15.28:0/511854605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe498015860 con 0x55c8c0540ff0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.214+0000 7fe4a4a00640 1 -- 172.21.15.28:0/511854605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498015c10 con 0x55c8c0540ff0 2024-09-10T10:55:00.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.214+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8c05c9520 con 0x55c8c0540ff0 2024-09-10T10:55:00.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.214+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8c05c9ae0 con 0x55c8c0540ff0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.215+0000 7fe4a4a00640 1 -- 172.21.15.28:0/511854605 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe49801c070 con 0x55c8c0540ff0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.216+0000 7fe4a4a00640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe48806a810 0x7fe48806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.216+0000 7fe4a4a00640 1 -- 172.21.15.28:0/511854605 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fe49808b190 con 0x55c8c0540ff0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.217+0000 7fe4a7200640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe48806a810 0x7fe48806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.217+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c8c05cbb20 0x55c8c05c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.217+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136) -- 0x55c8c05c2560 con 0x55c8c05cbb20 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.217+0000 7fe4a8600640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c8c05cbb20 0x55c8c05c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.217+0000 7fe4a7200640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe48806a810 0x7fe48806ccd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe49400bfd0 tx=0x7fe494006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.218+0000 7fe4a8600640 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c8c05cbb20 0x55c8c05c7a30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4a8600640 1 -- 172.21.15.28:0/511854605 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_openfiles.0 [omap-set-header] v136'161 uv161 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fe4a004e040 con 0x55c8c05cbb20 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c8c05cbb20 msgr2=0x55c8c05c7a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55c8c05cbb20 0x55c8c05c7a30 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe48806a810 msgr2=0x7fe48806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe48806a810 0x7fe48806ccd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe49400bfd0 tx=0x7fe494006000 comp rx=0 tx=0).stop 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 msgr2=0x55c8c05cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 --2- 172.21.15.28:0/511854605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c8c0540ff0 0x55c8c05cb5e0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fe498008a10 tx=0x7fe498008b00 comp rx=0 tx=0).stop 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4a8600640 1 -- 172.21.15.28:0/511854605 reap_dead start 2024-09-10T10:55:00.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 shutdown_connections 2024-09-10T10:55:00.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 >> 172.21.15.28:0/511854605 conn(0x55c8c05450a0 msgr2=0x55c8c05cd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 shutdown_connections 2024-09-10T10:55:00.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.219+0000 7fe4aa9dcf40 1 -- 172.21.15.28:0/511854605 wait complete. 2024-09-10T10:55:00.225 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:55:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.383+0000 7fa4b7000640 1 Processor -- start 2024-09-10T10:55:00.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.383+0000 7fa4b7000640 1 -- start start 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 0x7fa4b006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4b00b3640 0x7fa4b00d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b00d28f0 0x7fa4b006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b00ceba0 con 0x7fa4b00b3640 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b006dfb0 con 0x7fa4b0161ac0 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b00d2d20 con 0x7fa4b00d28f0 2024-09-10T10:55:00.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.384+0000 7fa4b5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4b00b3640 0x7fa4b00d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa4b00b3640 0x7fa4b00d23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48402/0 (socket says 172.21.15.28:48402) 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 -- 172.21.15.28:0/2975435163 learned_addr learned my addr 172.21.15.28:0/2975435163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5c00640 1 --2- 172.21.15.28:0/2975435163 >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 0x7fa4b006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b6600640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b00d28f0 0x7fa4b006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 -- 172.21.15.28:0/2975435163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b00d28f0 msgr2=0x7fa4b006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b00d28f0 0x7fa4b006cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 -- 172.21.15.28:0/2975435163 >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 msgr2=0x7fa4b006e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2975435163 >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 0x7fa4b006e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5200640 1 -- 172.21.15.28:0/2975435163 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b00b1170 con 0x7fa4b00b3640 2024-09-10T10:55:00.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.385+0000 7fa4b5c00640 1 --2- 172.21.15.28:0/2975435163 >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 0x7fa4b006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:00.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.386+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00b3640 0x7fa4b00d23b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa4b008cf20 tx=0x7fa49c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31a3020ac365664 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.386+0000 7fa4afe00640 1 -- 172.21.15.28:0/2975435163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c037070 con 0x7fa4b00b3640 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.386+0000 7fa4afe00640 1 -- 172.21.15.28:0/2975435163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49c032440 con 0x7fa4b00b3640 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.386+0000 7fa4afe00640 1 -- 172.21.15.28:0/2975435163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c032810 con 0x7fa4b00b3640 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 -- 172.21.15.28:0/2975435163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00b3640 msgr2=0x7fa4b00d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00b3640 0x7fa4b00d23b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa4b008cf20 tx=0x7fa49c02a270 comp rx=0 tx=0).stop 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 -- 172.21.15.28:0/2975435163 shutdown_connections 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b00d28f0 0x7fa4b006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2975435163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00b3640 0x7fa4b00d23b0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2975435163 >> v2:172.21.15.138:3300/0 conn(0x7fa4b0161ac0 0x7fa4b006e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 -- 172.21.15.28:0/2975435163 >> 172.21.15.28:0/2975435163 conn(0x7fa4b008bfc0 msgr2=0x7fa4b00d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 -- 172.21.15.28:0/2975435163 shutdown_connections 2024-09-10T10:55:00.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.387+0000 7fa4b7000640 1 -- 172.21.15.28:0/2975435163 wait complete. 2024-09-10T10:55:00.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.389+0000 7fa4b7000640 1 Processor -- start 2024-09-10T10:55:00.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.389+0000 7fa4b7000640 1 -- start start 2024-09-10T10:55:00.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b00b3640 0x7fa4b015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b0161ac0 0x7fa4b015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b00b1a10 con 0x7fa4b00d28f0 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b00b1730 con 0x7fa4b00b3640 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b0161ea0 con 0x7fa4b0161ac0 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48418/0 (socket says 172.21.15.28:48418) 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 -- 172.21.15.28:0/2493586642 learned_addr learned my addr 172.21.15.28:0/2493586642 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5c00640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b00b3640 0x7fa4b015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b6600640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b0161ac0 0x7fa4b015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 -- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b0161ac0 msgr2=0x7fa4b015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b0161ac0 0x7fa4b015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 -- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b00b3640 msgr2=0x7fa4b015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b00b3640 0x7fa4b015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.390+0000 7fa4b5200640 1 -- 172.21.15.28:0/2493586642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b015ae10 con 0x7fa4b00d28f0 2024-09-10T10:55:00.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4b5200640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa49c02a2b0 tx=0x7fa49c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c041020 con 0x7fa4b00d28f0 2024-09-10T10:55:00.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa49c0306c0 con 0x7fa4b00d28f0 2024-09-10T10:55:00.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c037040 con 0x7fa4b00d28f0 2024-09-10T10:55:00.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b00cc5c0 con 0x7fa4b00d28f0 2024-09-10T10:55:00.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.391+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b00ccb80 con 0x7fa4b00d28f0 2024-09-10T10:55:00.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.392+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b00d23b0 con 0x7fa4b00d28f0 2024-09-10T10:55:00.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.393+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa49c044020 con 0x7fa4b00d28f0 2024-09-10T10:55:00.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.394+0000 7fa4ad600640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 0x7fa49006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:00.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.394+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fa49c0ac3d0 con 0x7fa4b00d28f0 2024-09-10T10:55:00.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.394+0000 7fa4b5c00640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 0x7fa49006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:00.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.395+0000 7fa4b5c00640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 0x7fa49006e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4006fd0 tx=0x7fa4a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:00.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.398+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa49c04b030 con 0x7fa4b00d28f0 2024-09-10T10:55:00.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.619+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa4b00ce650 con 0x7fa4b00d28f0 2024-09-10T10:55:00.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.789+0000 7fa4ad600640 1 -- 172.21.15.28:0/2493586642 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v307) ==== 162+0+0 (secure 0 0 0) 0x7fa4b015ae10 con 0x7fa4b00d28f0 2024-09-10T10:55:00.791 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:55:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 msgr2=0x7fa49006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 0x7fa49006e2f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4006fd0 tx=0x7fa4a4008040 comp rx=0 tx=0).stop 2024-09-10T10:55:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 msgr2=0x7fa4b00cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fa49c02a2b0 tx=0x7fa49c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 shutdown_connections 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa49006be30 0x7fa49006e2f0 secure :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4006fd0 tx=0x7fa4a4008040 comp rx=0 tx=0).stop 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa4b0161ac0 0x7fa4b015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4b00d28f0 0x7fa4b00cd660 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 --2- 172.21.15.28:0/2493586642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4b00b3640 0x7fa4b015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 >> 172.21.15.28:0/2493586642 conn(0x7fa4b008bfc0 msgr2=0x7fa4b00b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 shutdown_connections 2024-09-10T10:55:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:00.792+0000 7fa4b7000640 1 -- 172.21.15.28:0/2493586642 wait complete. 2024-09-10T10:55:00.799 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:55:00.799+0000 7f3b2ac00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-10T10:55:01.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:01.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.375+0000 7f4442c00640 1 Processor -- start 2024-09-10T10:55:01.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.375+0000 7f4442c00640 1 -- start start 2024-09-10T10:55:01.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4442c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f443c15e660 0x7f443c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:01.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4442c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443c0cf000 con 0x7f443c0d1f90 2024-09-10T10:55:01.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4442c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f443c0d29a0 con 0x7f443c0d0dd0 2024-09-10T10:55:01.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4442c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f443c0b37d0 con 0x7f443c15e660 2024-09-10T10:55:01.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4442200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f443c0d1f90 0x7f443c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58068/0 (socket says 172.21.15.28:58068) 2024-09-10T10:55:01.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.376+0000 7f4440e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f443c15e660 0x7f443c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:01.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.377+0000 7f4442200640 1 -- 172.21.15.28:0/1658178782 learned_addr learned my addr 172.21.15.28:0/1658178782 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.377+0000 7f4440e00640 1 -- 172.21.15.28:0/1658178782 >> v1:172.21.15.138:6789/0 conn(0x7f443c0d0dd0 legacy=0x7f443c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.377+0000 7f4440e00640 1 -- 172.21.15.28:0/1658178782 >> v1:172.21.15.28:6789/0 conn(0x7f443c0d1f90 legacy=0x7f443c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.377+0000 7f4440e00640 1 -- 172.21.15.28:0/1658178782 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c0d2c60 con 0x7f443c15e660 2024-09-10T10:55:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.378+0000 7f4440e00640 1 --2- 172.21.15.28:0/1658178782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c06ccd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f44280097a0 tx=0x7f442802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1b1606e81a4f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.378+0000 7f443be00640 1 -- 172.21.15.28:0/1658178782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428037070 con 0x7f443c15e660 2024-09-10T10:55:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.378+0000 7f443be00640 1 -- 172.21.15.28:0/1658178782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4428032440 con 0x7f443c15e660 2024-09-10T10:55:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.378+0000 7f443be00640 1 -- 172.21.15.28:0/1658178782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428032810 con 0x7f443c15e660 2024-09-10T10:55:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 -- 172.21.15.28:0/1658178782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 msgr2=0x7f443c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 --2- 172.21.15.28:0/1658178782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c06ccd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f44280097a0 tx=0x7f442802a230 comp rx=0 tx=0).stop 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 -- 172.21.15.28:0/1658178782 shutdown_connections 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 --2- 172.21.15.28:0/1658178782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c06ccd0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 -- 172.21.15.28:0/1658178782 >> 172.21.15.28:0/1658178782 conn(0x7f443c08bfc0 msgr2=0x7f443c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 -- 172.21.15.28:0/1658178782 shutdown_connections 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.379+0000 7f4442c00640 1 -- 172.21.15.28:0/1658178782 wait complete. 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.381+0000 7f4442c00640 1 Processor -- start 2024-09-10T10:55:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.381+0000 7f4442c00640 1 -- start start 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 0x7f443c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 0x7f443c15baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f443c0d3220 con 0x7f443c15e660 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f443c0cf330 con 0x7f443c0d1f90 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f443c0d33a0 con 0x7f443c0d0dd0 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48444/0 (socket says 172.21.15.28:48444) 2024-09-10T10:55:01.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 -- 172.21.15.28:0/2926750211 learned_addr learned my addr 172.21.15.28:0/2926750211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4441800640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 0x7f443c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4440e00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 0x7f443c15baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 -- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 msgr2=0x7f443c0b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 0x7f443c0b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 -- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 msgr2=0x7f443c15baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 0x7f443c15baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4442200640 1 -- 172.21.15.28:0/2926750211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c0b9520 con 0x7f443c15e660 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4441800640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 0x7f443c0b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.382+0000 7f4440e00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 0x7f443c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4442200640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f443400cbc0 tx=0x7f443400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:01.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434017070 con 0x7f443c15e660 2024-09-10T10:55:01.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f443400ad40 con 0x7f443c15e660 2024-09-10T10:55:01.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434012600 con 0x7f443c15e660 2024-09-10T10:55:01.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c0b9240 con 0x7f443c15e660 2024-09-10T10:55:01.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.383+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c0b7570 con 0x7f443c15e660 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.384+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c06d2a0 con 0x7f443c15e660 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.385+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f443401a070 con 0x7f443c15e660 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.386+0000 7f4439600640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 0x7f441c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.386+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f443408ba70 con 0x7f443c15e660 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.386+0000 7f4441800640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 0x7f441c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:01.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.387+0000 7f4441800640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 0x7f441c06e2f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4430007fd0 tx=0x7f4430009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:01.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.390+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4434058eb0 con 0x7f443c15e660 2024-09-10T10:55:01.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.608+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f443c0d1b20 con 0x7f443c15e660 2024-09-10T10:55:01.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.609+0000 7f4439600640 1 -- 172.21.15.28:0/2926750211 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 308 v308) ==== 95+0+5227 (secure 0 0 0) 0x7f443405d1b0 con 0x7f443c15e660 2024-09-10T10:55:01.610 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:01.610 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":308,"btime":"2024-09-10T10:55:00:793236+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":308,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:00.793222+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10035":{"gid":10035,"name":"c","rank":0,"incarnation":308,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/3154009439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3154009439},{"type":"v1","addr":"172.21.15.28:6834","nonce":3154009439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:01.611 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 308 2024-09-10T10:55:01.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 msgr2=0x7f441c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 0x7f441c06e2f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4430007fd0 tx=0x7f4430009040 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 msgr2=0x7f443c0b7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f443400cbc0 tx=0x7f443400a430 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 shutdown_connections 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f441c06be30 0x7f441c06e2f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f443c15e660 0x7f443c0b7d70 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f443c0d1f90 0x7f443c15baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 --2- 172.21.15.28:0/2926750211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f443c0d0dd0 0x7f443c0b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 >> 172.21.15.28:0/2926750211 conn(0x7f443c08bfc0 msgr2=0x7f443c15c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 shutdown_connections 2024-09-10T10:55:01.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:01.611+0000 7f4442c00640 1 -- 172.21.15.28:0/2926750211 wait complete. 2024-09-10T10:55:01.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 308, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:55:00.793222+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 136, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10035}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10035': {'gid': 10035, 'name': 'c', 'rank': 0, 'incarnation': 308, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6834/3154009439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 3154009439}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 3154009439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:55:01.628 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10035:up:replay 2024-09-10T10:55:01.628 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:55:07.033 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.200+0000 7fa7dcc00640 1 Processor -- start 2024-09-10T10:55:07.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.200+0000 7fa7dcc00640 1 -- start start 2024-09-10T10:55:07.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.201+0000 7fa7dcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 0x7fa7d806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.201+0000 7fa7dcc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7d8147e90 0x7fa7d80720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7dcc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7d806f480 con 0x7fa7d80725f0 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7dcc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d80b34d0 con 0x7fa7d8147e90 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7dcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d80ae9e0 con 0x7fa7d8147230 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7cd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7d8147e90 0x7fa7d80720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7d6800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa7d80725f0 0x7fa7d8161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53040/0 (socket says 172.21.15.28:53040) 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7d6800640 1 -- 172.21.15.28:0/1663345708 learned_addr learned my addr 172.21.15.28:0/1663345708 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7d5e00640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 0x7fa7d806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7cd400640 1 -- 172.21.15.28:0/1663345708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 msgr2=0x7fa7d806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7cd400640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 0x7fa7d806e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7cd400640 1 -- 172.21.15.28:0/1663345708 >> v1:172.21.15.28:6789/0 conn(0x7fa7d80725f0 legacy=0x7fa7d8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7cd400640 1 -- 172.21.15.28:0/1663345708 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d806e900 con 0x7fa7d8147e90 2024-09-10T10:55:07.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.202+0000 7fa7d5e00640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 0x7fa7d806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.203+0000 7fa7cd400640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147e90 0x7fa7d80720b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00097a0 tx=0x7fa7c002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d561bf2cb4bd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7d5400640 1 -- 172.21.15.28:0/1663345708 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0037070 con 0x7fa7d8147e90 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7d5400640 1 -- 172.21.15.28:0/1663345708 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c0032440 con 0x7fa7d8147e90 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7d5400640 1 -- 172.21.15.28:0/1663345708 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0032810 con 0x7fa7d8147e90 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 -- 172.21.15.28:0/1663345708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147e90 msgr2=0x7fa7d80720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147e90 0x7fa7d80720b0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00097a0 tx=0x7fa7c002a180 comp rx=0 tx=0).stop 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 -- 172.21.15.28:0/1663345708 shutdown_connections 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147e90 0x7fa7d80720b0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/1663345708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147230 0x7fa7d806e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 -- 172.21.15.28:0/1663345708 >> 172.21.15.28:0/1663345708 conn(0x7fa7d8067d70 msgr2=0x7fa7d806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 -- 172.21.15.28:0/1663345708 shutdown_connections 2024-09-10T10:55:07.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.204+0000 7fa7dcc00640 1 -- 172.21.15.28:0/1663345708 wait complete. 2024-09-10T10:55:07.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.206+0000 7fa7dcc00640 1 Processor -- start 2024-09-10T10:55:07.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.206+0000 7fa7dcc00640 1 -- start start 2024-09-10T10:55:07.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d80725f0 0x7fa7d80c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 0x7fa7d8142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d806eec0 con 0x7fa7d8147230 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d8072a40 con 0x7fa7d80725f0 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7dcc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d806f1a0 con 0x7fa7d8147e90 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7cd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7cd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60150/0 (socket says 172.21.15.28:60150) 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7cd400640 1 -- 172.21.15.28:0/174877593 learned_addr learned my addr 172.21.15.28:0/174877593 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7d5e00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d80725f0 0x7fa7d80c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.207+0000 7fa7d6800640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 0x7fa7d8142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 -- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 msgr2=0x7fa7d8142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 0x7fa7d8142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 -- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d80725f0 msgr2=0x7fa7d80c98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d80725f0 0x7fa7d80c98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 -- 172.21.15.28:0/174877593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d81f4eb0 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7d6800640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 0x7fa7d8142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cd400640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0000c00 tx=0x7fa7c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0037040 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7c0042070 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0030bd0 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d80c55e0 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.208+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d80c5ba0 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.209+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d8147610 con 0x7fa7d8147230 2024-09-10T10:55:07.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.210+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7c0047460 con 0x7fa7d8147230 2024-09-10T10:55:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.211+0000 7fa7cea00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 0x7fa7b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.211+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fa7c00ac920 con 0x7fa7d8147230 2024-09-10T10:55:07.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.212+0000 7fa7d5e00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 0x7fa7b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.213+0000 7fa7d5e00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 0x7fa7b406e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8006e80 tx=0x7fa7c8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.216+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7c0032e20 con 0x7fa7d8147230 2024-09-10T10:55:07.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.437+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7d80b1c00 con 0x7fa7d8147230 2024-09-10T10:55:07.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.437+0000 7fa7cea00640 1 -- 172.21.15.28:0/174877593 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 311 v311) ==== 95+0+5236 (secure 0 0 0) 0x7fa7c007a3b0 con 0x7fa7d8147230 2024-09-10T10:55:07.440 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:07.440 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":311,"btime":"2024-09-10T10:55:03:803863+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":311,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:03.803856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":136,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10035},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10035":{"gid":10035,"name":"c","rank":0,"incarnation":308,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6834/3154009439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3154009439},{"type":"v1","addr":"172.21.15.28:6834","nonce":3154009439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10035,"qdb_cluster":[10035]},"id":6}]} 2024-09-10T10:55:07.440 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 311 2024-09-10T10:55:07.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 msgr2=0x7fa7b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 0x7fa7b406e2f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8006e80 tx=0x7fa7c8008040 comp rx=0 tx=0).stop 2024-09-10T10:55:07.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 msgr2=0x7fa7d8142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0000c00 tx=0x7fa7c00046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 shutdown_connections 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7b406be30 0x7fa7b406e2f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7d8147e90 0x7fa7d8142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d8147230 0x7fa7d8142a50 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 --2- 172.21.15.28:0/174877593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7d80725f0 0x7fa7d80c98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 >> 172.21.15.28:0/174877593 conn(0x7fa7d8067d70 msgr2=0x7fa7d80aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 shutdown_connections 2024-09-10T10:55:07.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.440+0000 7fa7dcc00640 1 -- 172.21.15.28:0/174877593 wait complete. 2024-09-10T10:55:07.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 311, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:50:27.593484+0000', 'modified': '2024-09-10T10:55:03.803856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 136, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10035}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10035': {'gid': 10035, 'name': 'c', 'rank': 0, 'incarnation': 308, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6834/3154009439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6832', 'nonce': 3154009439}, {'type': 'v1', 'addr': '172.21.15.28:6834', 'nonce': 3154009439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10035, 'qdb_cluster': [10035]} 2024-09-10T10:55:07.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:55:07.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T10:55:07.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.614+0000 7f0039a00640 1 Processor -- start 2024-09-10T10:55:07.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.614+0000 7f0039a00640 1 -- start start 2024-09-10T10:55:07.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00340d0dd0 0x7f00340d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f003415e660 0x7f003406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00340cf000 con 0x7f00340d1f90 2024-09-10T10:55:07.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f00340b37d0 con 0x7f00340d0dd0 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00340cf330 con 0x7f003415e660 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f00340d1f90 0x7f00340cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53042/0 (socket says 172.21.15.28:53042) 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0033e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00340d0dd0 0x7f00340d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.615+0000 7f0039000640 1 -- 172.21.15.28:0/2651350581 learned_addr learned my addr 172.21.15.28:0/2651350581 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0033400640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f003415e660 0x7f003406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0033e00640 1 -- 172.21.15.28:0/2651350581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f003415e660 msgr2=0x7f003406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0033e00640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f003415e660 0x7f003406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0033e00640 1 -- 172.21.15.28:0/2651350581 >> v1:172.21.15.28:6789/0 conn(0x7f00340d1f90 legacy=0x7f00340cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0033e00640 1 -- 172.21.15.28:0/2651350581 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00340d16f0 con 0x7f00340d0dd0 2024-09-10T10:55:07.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.616+0000 7f0032a00640 1 -- 172.21.15.28:0/2651350581 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368319518 0 0) 0x7f00340cf000 con 0x7f00340d1f90 2024-09-10T10:55:07.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.617+0000 7f0033e00640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340d11b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74bc347c0d71405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.617+0000 7f0032a00640 1 -- 172.21.15.28:0/2651350581 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024037070 con 0x7f00340d0dd0 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.617+0000 7f0032a00640 1 -- 172.21.15.28:0/2651350581 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024032440 con 0x7f00340d0dd0 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.617+0000 7f0032a00640 1 -- 172.21.15.28:0/2651350581 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024032810 con 0x7f00340d0dd0 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 -- 172.21.15.28:0/2651350581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 msgr2=0x7f00340d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340d11b0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402a180 comp rx=0 tx=0).stop 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 -- 172.21.15.28:0/2651350581 shutdown_connections 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f003415e660 0x7f003406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 --2- 172.21.15.28:0/2651350581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340d11b0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 -- 172.21.15.28:0/2651350581 >> 172.21.15.28:0/2651350581 conn(0x7f003408bfc0 msgr2=0x7f00340d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 -- 172.21.15.28:0/2651350581 shutdown_connections 2024-09-10T10:55:07.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.618+0000 7f0039a00640 1 -- 172.21.15.28:0/2651350581 wait complete. 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.620+0000 7f0039a00640 1 Processor -- start 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.620+0000 7f0039a00640 1 -- start start 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00340d1f90 0x7f0034157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003415e660 0x7f00340c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f00340d2f40 con 0x7f00340d0dd0 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00340d2c60 con 0x7f003415e660 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0039a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00340d3220 con 0x7f00340d1f90 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60158/0 (socket says 172.21.15.28:60158) 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 -- 172.21.15.28:0/3133202786 learned_addr learned my addr 172.21.15.28:0/3133202786 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 -- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00340d1f90 msgr2=0x7f0034157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00340d1f90 0x7f0034157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 -- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003415e660 msgr2=0x7f00340c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003415e660 0x7f00340c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.621+0000 7f0033e00640 1 -- 172.21.15.28:0/3133202786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00341e52c0 con 0x7f00340d0dd0 2024-09-10T10:55:07.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f0033e00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f00240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f002be00640 1 -- 172.21.15.28:0/3133202786 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024037040 con 0x7f00340d0dd0 2024-09-10T10:55:07.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f002be00640 1 -- 172.21.15.28:0/3133202786 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024004030 con 0x7f00340d0dd0 2024-09-10T10:55:07.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f002be00640 1 -- 172.21.15.28:0/3133202786 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024006e10 con 0x7f00340d0dd0 2024-09-10T10:55:07.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00341e5560 con 0x7f00340d0dd0 2024-09-10T10:55:07.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.622+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341e5b20 con 0x7f00340d0dd0 2024-09-10T10:55:07.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.625+0000 7f002be00640 1 -- 172.21.15.28:0/3133202786 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0024030770 con 0x7f00340d0dd0 2024-09-10T10:55:07.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.626+0000 7f002be00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 0x7f001406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.626+0000 7f002be00640 1 -- 172.21.15.28:0/3133202786 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f00240acfe0 con 0x7f00340d0dd0 2024-09-10T10:55:07.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.626+0000 7f003db22740 1 Processor -- start 2024-09-10T10:55:07.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.626+0000 7f0033400640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 0x7f001406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.626+0000 7f003db22740 1 -- start start 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f0033400640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 0x7f001406e320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0020006fd0 tx=0x7f0020008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d10640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cc8660 0x562441ccb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562441c293b0 con 0x562441ccc630 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562441c2e410 con 0x562441cc8660 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.627+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562441bd6620 con 0x562441ccaea0 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60164/0 (socket says 172.21.15.28:60164) 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 -- 172.21.15.28:0/2296784482 learned_addr learned my addr 172.21.15.28:0/2296784482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0033e00640 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d10640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0033400640 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cc8660 0x562441ccb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 -- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 msgr2=0x562441d10640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d10640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 -- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cc8660 msgr2=0x562441ccb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cc8660 0x562441ccb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 -- 172.21.15.28:0/2296784482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624419412d0 con 0x562441ccc630 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0039000640 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f002c001d90 tx=0x7f002c00c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d6f7565eb61c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.628+0000 7f0028c00640 1 -- 172.21.15.28:0/2296784482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00c910 con 0x562441ccc630 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f0028c00640 1 -- 172.21.15.28:0/2296784482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00cad0 con 0x562441ccc630 2024-09-10T10:55:07.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f0028c00640 1 -- 172.21.15.28:0/2296784482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0070d0 con 0x562441ccc630 2024-09-10T10:55:07.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 -- 172.21.15.28:0/2296784482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 msgr2=0x562441cc5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f002c001d90 tx=0x7f002c00c0e0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 -- 172.21.15.28:0/2296784482 shutdown_connections 2024-09-10T10:55:07.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441cc5720 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cc8660 0x562441ccb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 --2- 172.21.15.28:0/2296784482 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d10640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 -- 172.21.15.28:0/2296784482 >> 172.21.15.28:0/2296784482 conn(0x562441cc78f0 msgr2=0x562441d0fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 -- 172.21.15.28:0/2296784482 shutdown_connections 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.629+0000 7f003db22740 1 -- 172.21.15.28:0/2296784482 wait complete. 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.630+0000 7f003db22740 1 -- start start 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.630+0000 7f003db22740 1 Processor -- start 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441ce3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 0x562441cf3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562441be9780 con 0x562441cf42e0 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562441c2a3d0 con 0x562441ccc630 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562441cc6d90 con 0x562441ccaea0 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f003db22740 10 client.0 _ll_register_callbacks cb 0x562441c275d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:07.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.631+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60178/0 (socket says 172.21.15.28:60178) 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 -- 172.21.15.28:0/450277241 learned_addr learned my addr 172.21.15.28:0/450277241 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0033400640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 0x562441cf3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0033e00640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441ce3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 -- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 msgr2=0x562441ce3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441ce3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 -- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 msgr2=0x562441cf3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 0x562441cf3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 -- 172.21.15.28:0/450277241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562441a0b6b0 con 0x562441cf42e0 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0033e00640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441ce3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:07.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0033400640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 0x562441cf3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:07.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f0039000640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c5e0 tx=0x7f002c001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8071b5f55129210 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f001d600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:55:07.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.632+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c022030 con 0x562441cf42e0 2024-09-10T10:55:07.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f002c00cad0 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f002c001840 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=311) ==== 24+0+0 (secure 0 0 0) 0x7f002c02c050 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 10 client.10095 fetch_fsmap learned FSMap version 311 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=311,monmap=2+}) -- 0x562441cf4d70 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00cad0 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 311) ==== 3502+0+0 (secure 0 0 0) 0x7f002c007650 con 0x562441cf42e0 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 10 client.10095 fetch_fsmap finished waiting for FSMap version 311 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 10 client.10095 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 20 client.10095 populate_metadata read hostname 'smithi028' 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441cf2ee0 0x562441cf32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 4 client.10095 mds_command: new command op to 10035 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.633+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 --> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562441c3bb40 con 0x562441cf2ee0 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.634+0000 7f0033e00640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441cf2ee0 0x562441cf32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.634+0000 7f0033e00640 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441cf2ee0 0x562441cf32c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc47165acd37bd0 server_cookie=91749d75384ac390 in_seq=0 out_seq=0 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.634+0000 7f001d600640 10 client.10095 ms_handle_connect on v2:172.21.15.28:6832/3154009439 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f001d600640 1 -- 172.21.15.28:0/450277241 <== mds.0 v2:172.21.15.28:6832/3154009439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562441c3bb40 con 0x562441cf2ee0 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f001d600640 10 client.10095 handle_command_reply: tid=0 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 client.10095 shutdown 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 msgr2=0x562441cc5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f002c00c5e0 tx=0x7f002c001400 comp rx=0 tx=0).stop 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 shutdown_connections 2024-09-10T10:55:07.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441cf2ee0 0x562441cf32c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cc5630 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccc630 0x562441cf3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 --2- 172.21.15.28:0/450277241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441ce3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f001d600640 0 client.10095 ms_handle_reset on v2:172.21.15.28:6832/3154009439 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.635+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 >> 172.21.15.28:0/450277241 conn(0x562441cc8660 msgr2=0x562441d0db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.636+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 shutdown_connections 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.636+0000 7f003db22740 1 -- 172.21.15.28:0/450277241 wait complete. 2024-09-10T10:55:07.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.636+0000 7f003db22740 20 client.10095 trim_cache size 0 max 16384 2024-09-10T10:55:07.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.654+0000 7f003db22740 1 Processor -- start 2024-09-10T10:55:07.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.654+0000 7f003db22740 1 -- start start 2024-09-10T10:55:07.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccaea0 0x562441cf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccc630 0x562441cf6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0034202020 con 0x562441cf42e0 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0034201300 con 0x562441ccaea0 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0034201950 con 0x562441ccc630 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f0039000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60194/0 (socket says 172.21.15.28:60194) 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f0039000640 1 -- 172.21.15.28:0/481844440 learned_addr learned my addr 172.21.15.28:0/481844440 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f0033400640 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccc630 0x562441cf6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.655+0000 7f0033e00640 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccaea0 0x562441cf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 -- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccc630 msgr2=0x562441cf6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccc630 0x562441cf6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 -- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccaea0 msgr2=0x562441cf6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccaea0 0x562441cf6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 -- 172.21.15.28:0/481844440 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562441d21090 con 0x562441cf42e0 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f0039000640 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f002c007f80 tx=0x7f002c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41808998b592195e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f001fe00640 1 -- 172.21.15.28:0/481844440 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c022040 con 0x562441cf42e0 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f001fe00640 1 -- 172.21.15.28:0/481844440 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00cad0 con 0x562441cf42e0 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.656+0000 7f001fe00640 1 -- 172.21.15.28:0/481844440 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c019040 con 0x562441cf42e0 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- 172.21.15.28:0/481844440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 msgr2=0x562441cf5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f002c007f80 tx=0x7f002c01d000 comp rx=0 tx=0).stop 2024-09-10T10:55:07.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- 172.21.15.28:0/481844440 shutdown_connections 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441cf42e0 0x562441cf5c10 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccc630 0x562441cf6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 --2- 172.21.15.28:0/481844440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441ccaea0 0x562441cf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- 172.21.15.28:0/481844440 >> 172.21.15.28:0/481844440 conn(0x562441cc8660 msgr2=0x562441cc6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- 172.21.15.28:0/481844440 shutdown_connections 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- 172.21.15.28:0/481844440 wait complete. 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.657+0000 7f003db22740 1 -- start start 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.658+0000 7f003db22740 1 Processor -- start 2024-09-10T10:55:07.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d0d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 0x562441cf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x562441d21280 con 0x562441ccc630 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562441cf5050 con 0x562441cf42e0 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x562441d20b60 con 0x562441ccaea0 2024-09-10T10:55:07.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f003db22740 10 client.0 _ll_register_callbacks cb 0x7f002407a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:07.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60204/0 (socket says 172.21.15.28:60204) 2024-09-10T10:55:07.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 -- 172.21.15.28:0/96310569 learned_addr learned my addr 172.21.15.28:0/96310569 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033e00640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d0d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 -- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 msgr2=0x562441d0d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0039000640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 0x562441cf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d0d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 -- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 msgr2=0x562441cf6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 0x562441cf6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033400640 1 -- 172.21.15.28:0/96310569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562441d88550 con 0x562441ccc630 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0033e00640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d0d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.659+0000 7f0039000640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 0x562441cf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:07.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f0033400640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f003415ea60 tx=0x7f0020006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39048134c96c361b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020026020 con 0x562441ccc630 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f002000dd30 con 0x562441ccc630 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f002c00cad0 con 0x562441ccc630 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=311) ==== 24+0+0 (secure 0 0 0) 0x7f0020031050 con 0x562441ccc630 2024-09-10T10:55:07.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 10 client.10101 fetch_fsmap learned FSMap version 311 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=311,monmap=2+}) -- 0x562441d332a0 con 0x562441ccc630 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c00cad0 con 0x562441ccc630 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 311) ==== 3502+0+0 (secure 0 0 0) 0x7f0020013070 con 0x562441ccc630 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 10 client.10101 fetch_fsmap finished waiting for FSMap version 311 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 10 client.10101 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.660+0000 7f003db22740 20 client.10101 populate_metadata read hostname 'smithi028' 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441d0f500 0x562441d1f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f003db22740 4 client.10101 mds_command: new command op to 10035 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 --> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] -- command(tid 0: {"prefix": "status"}) -- 0x562441c35e90 con 0x562441d0f500 2024-09-10T10:55:07.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f0033e00640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441d0f500 0x562441d1f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f0033e00640 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441d0f500 0x562441d1f1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b5aadfb0f3f8254 server_cookie=54b22061a2109e28 in_seq=0 out_seq=0 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.661+0000 7f001d600640 10 client.10101 ms_handle_connect on v2:172.21.15.28:6832/3154009439 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f001d600640 1 -- 172.21.15.28:0/96310569 <== mds.0 v2:172.21.15.28:6832/3154009439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+335 (crc 0 0 0) 0x562441c35e90 con 0x562441d0f500 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f001d600640 10 client.10101 handle_command_reply: tid=0 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 client.10101 shutdown 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 msgr2=0x562441d88010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f003415ea60 tx=0x7f0020006e10 comp rx=0 tx=0).stop 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 shutdown_connections 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:6832/3154009439,v1:172.21.15.28:6834/3154009439] conn(0x562441d0f500 0x562441d1f1f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562441cf42e0 0x562441cf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x562441ccc630 0x562441d88010 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 --2- 172.21.15.28:0/96310569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x562441ccaea0 0x562441d0d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f001d600640 0 client.10101 ms_handle_reset on v2:172.21.15.28:6832/3154009439 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 >> 172.21.15.28:0/96310569 conn(0x562441cc8660 msgr2=0x562441cc61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 shutdown_connections 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 1 -- 172.21.15.28:0/96310569 wait complete. 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.662+0000 7f003db22740 20 client.10101 trim_cache size 0 max 16384 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:55:07.676 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 10035, 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.8680683550000001, 2024-09-10T10:55:07.677 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 311, 2024-09-10T10:55:07.678 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 136, 2024-09-10T10:55:07.678 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 136, 2024-09-10T10:55:07.678 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 9.1115749390000005 2024-09-10T10:55:07.678 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 msgr2=0x7f001406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 0x7f001406e320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0020006fd0 tx=0x7f0020008040 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 msgr2=0x7f00340b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f00240046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 shutdown_connections 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f001406be60 0x7f001406e320 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003415e660 0x7f00340c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00340d1f90 0x7f0034157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 --2- 172.21.15.28:0/3133202786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00340d0dd0 0x7f00340b6960 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 >> 172.21.15.28:0/3133202786 conn(0x7f003408bfc0 msgr2=0x7f00340b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 shutdown_connections 2024-09-10T10:55:07.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.663+0000 7f0039a00640 1 -- 172.21.15.28:0/3133202786 wait complete. 2024-09-10T10:55:07.681 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:55:07.681 INFO:tasks.cephfs.test_damage:Daemons came up after mutation 'Corrupt omap header on mds0_openfiles.0', proceeding to ls 2024-09-10T10:55:07.681 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:55:07.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.836+0000 7fd744000640 1 Processor -- start 2024-09-10T10:55:07.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.836+0000 7fd744000640 1 -- start start 2024-09-10T10:55:07.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.837+0000 7fd744000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd73c072de0 0x7fd73c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.837+0000 7fd744000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c146140 con 0x7fd73c072700 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.837+0000 7fd744000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0cd740 con 0x7fd73c072320 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.837+0000 7fd744000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0cda20 con 0x7fd73c072de0 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd740c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd73c072de0 0x7fd73c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd742000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd73c072700 0x7fd73c15d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53052/0 (socket says 172.21.15.28:53052) 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd742000640 1 -- 172.21.15.28:0/884707556 learned_addr learned my addr 172.21.15.28:0/884707556 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd740c00640 1 -- 172.21.15.28:0/884707556 >> v1:172.21.15.138:6789/0 conn(0x7fd73c072320 legacy=0x7fd73c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd740c00640 1 -- 172.21.15.28:0/884707556 >> v1:172.21.15.28:6789/0 conn(0x7fd73c072700 legacy=0x7fd73c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.838+0000 7fd740c00640 1 -- 172.21.15.28:0/884707556 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0cdd00 con 0x7fd73c072de0 2024-09-10T10:55:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.839+0000 7fd740c00640 1 --2- 172.21.15.28:0/884707556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072de0 0x7fd73c15d1e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368575160d7ed595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.839+0000 7fd73be00640 1 -- 172.21.15.28:0/884707556 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037070 con 0x7fd73c072de0 2024-09-10T10:55:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.839+0000 7fd73be00640 1 -- 172.21.15.28:0/884707556 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd728032440 con 0x7fd73c072de0 2024-09-10T10:55:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.839+0000 7fd73be00640 1 -- 172.21.15.28:0/884707556 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728032810 con 0x7fd73c072de0 2024-09-10T10:55:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 -- 172.21.15.28:0/884707556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072de0 msgr2=0x7fd73c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 --2- 172.21.15.28:0/884707556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072de0 0x7fd73c15d1e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 -- 172.21.15.28:0/884707556 shutdown_connections 2024-09-10T10:55:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 --2- 172.21.15.28:0/884707556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072de0 0x7fd73c15d1e0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 -- 172.21.15.28:0/884707556 >> 172.21.15.28:0/884707556 conn(0x7fd73c067d70 msgr2=0x7fd73c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 -- 172.21.15.28:0/884707556 shutdown_connections 2024-09-10T10:55:07.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.840+0000 7fd744000640 1 -- 172.21.15.28:0/884707556 wait complete. 2024-09-10T10:55:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.842+0000 7fd744000640 1 Processor -- start 2024-09-10T10:55:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.842+0000 7fd744000640 1 -- start start 2024-09-10T10:55:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd73c072320 0x7fd73c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd73c072de0 0x7fd73c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c15dfb0 con 0x7fd73c072700 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c15de30 con 0x7fd73c072de0 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd744000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c15e130 con 0x7fd73c072320 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd740c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd740c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60224/0 (socket says 172.21.15.28:60224) 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd740c00640 1 -- 172.21.15.28:0/3362273433 learned_addr learned my addr 172.21.15.28:0/3362273433 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd741600640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd73c072320 0x7fd73c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.843+0000 7fd742000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd73c072de0 0x7fd73c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 -- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd73c072320 msgr2=0x7fd73c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd73c072320 0x7fd73c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 -- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd73c072de0 msgr2=0x7fd73c15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd73c072de0 0x7fd73c15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 -- 172.21.15.28:0/3362273433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c144060 con 0x7fd73c072700 2024-09-10T10:55:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd740c00640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd7280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728041070 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72803c870 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037040 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c15a710 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.844+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c15acd0 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.845+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c0cce60 con 0x7fd73c072700 2024-09-10T10:55:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.846+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd728044070 con 0x7fd73c072700 2024-09-10T10:55:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.847+0000 7fd739600640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 0x7fd71c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.847+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fd7280aca50 con 0x7fd73c072700 2024-09-10T10:55:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.848+0000 7fd741600640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 0x7fd71c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.848+0000 7fd741600640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 0x7fd71c06e320 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd730000f60 tx=0x7fd730009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:07.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:07.851+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd72807a040 con 0x7fd73c072700 2024-09-10T10:55:08.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.072+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd73c0b4b40 con 0x7fd73c072700 2024-09-10T10:55:08.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.072+0000 7fd739600640 1 -- 172.21.15.28:0/3362273433 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v311) ==== 53+0+73 (secure 0 0 0) 0x7fd72807e340 con 0x7fd73c072700 2024-09-10T10:55:08.074 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:55:08.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 msgr2=0x7fd71c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 0x7fd71c06e320 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd730000f60 tx=0x7fd730009040 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 msgr2=0x7fd73c143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd7280046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 shutdown_connections 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd71c06be60 0x7fd71c06e320 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd73c072de0 0x7fd73c15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd73c072700 0x7fd73c143700 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 --2- 172.21.15.28:0/3362273433 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd73c072320 0x7fd73c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 >> 172.21.15.28:0/3362273433 conn(0x7fd73c067d70 msgr2=0x7fd73c0b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.074+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 shutdown_connections 2024-09-10T10:55:08.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08.075+0000 7fd744000640 1 -- 172.21.15.28:0/3362273433 wait complete. 2024-09-10T10:55:08.092 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:55:08.092 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:55:08.093 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:55:08.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.093 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:55:08.130 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.131 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.132 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:55:08.133 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:55:08.133 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:55:08.133 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:55:08.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.133 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:55:08.224 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:55:08.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.227 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:55:08.263 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:55:08.263 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:55:08.264 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:55:08.264 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:55:08.264 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:55:08.264 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:55:08.264 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:55:08.264 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:55:08.264 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:55:08.264 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:55:08.265 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:55:08.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:55:08.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:55:08.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.461 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:55:08.498 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:55:08.498 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:55:08.499 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:55:08.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.499 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:55:08.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:55:08.595 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:55:08.630 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:55:08.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:55:08.630 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:55:08.705 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:55:08.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:55:08.786 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:55:08.784+0000 7f3b2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:08.835 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:55:08.835 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:55:08.835 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:55:08.851 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:55:08.851 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:55:08.851 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:55:08.851 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:55:08.851 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:55:08.873 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:55:08.874 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:55:08.875 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:55:08.875 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:55:08.875 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:55:08.875 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:55:08.875 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:55:08.878 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:55:08.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:08.878 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:55:08.974 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:55:09.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:55:09.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:55:09.063 INFO:teuthology.orchestra.run.smithi028.stdout:client8192 2024-09-10T10:55:09.063 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:09.063 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:09.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:09.100 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -R .) 2024-09-10T10:55:09.146 INFO:teuthology.orchestra.run:waiting for 20 2024-09-10T10:55:09.178 INFO:teuthology.orchestra.run.smithi028.stdout:.: 2024-09-10T10:55:09.179 INFO:teuthology.orchestra.run.smithi028.stdout:subdir 2024-09-10T10:55:09.179 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:09.179 INFO:teuthology.orchestra.run.smithi028.stdout:./subdir: 2024-09-10T10:55:09.179 INFO:teuthology.orchestra.run.smithi028.stdout:sixmegs 2024-09-10T10:55:09.180 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_openfiles.0' did not caused DAMAGED state 2024-09-10T10:55:09.180 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 100.00000000' 2024-09-10T10:55:09.180 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:09.180 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:09.255 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:09.255 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:09.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:55:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.527+0000 7f42aae00640 1 Processor -- start 2024-09-10T10:55:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.527+0000 7f42aae00640 1 -- start start 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42a415e660 0x7f42a406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40cf000 con 0x7f42a40d1f90 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42a40d29a0 con 0x7f42a40d0dd0 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f42a40b37d0 con 0x7f42a415e660 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42a9000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42a415e660 0x7f42a406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aa400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f42a40d1f90 0x7f42a40cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53062/0 (socket says 172.21.15.28:53062) 2024-09-10T10:55:09.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.528+0000 7f42aa400640 1 -- 172.21.15.28:0/1706171 learned_addr learned my addr 172.21.15.28:0/1706171 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:09.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.529+0000 7f42a9000640 1 -- 172.21.15.28:0/1706171 >> v1:172.21.15.138:6789/0 conn(0x7f42a40d0dd0 legacy=0x7f42a40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.529+0000 7f42a9000640 1 -- 172.21.15.28:0/1706171 >> v1:172.21.15.28:6789/0 conn(0x7f42a40d1f90 legacy=0x7f42a40cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.529+0000 7f42a9000640 1 -- 172.21.15.28:0/1706171 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a40d2c60 con 0x7f42a415e660 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42a9000640 1 --2- 172.21.15.28:0/1706171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a415e660 0x7f42a406ccd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f42900097a0 tx=0x7f429002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53213e5e0a877d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42a3e00640 1 -- 172.21.15.28:0/1706171 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290037070 con 0x7f42a415e660 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42a3e00640 1 -- 172.21.15.28:0/1706171 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290032440 con 0x7f42a415e660 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42a3e00640 1 -- 172.21.15.28:0/1706171 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290032810 con 0x7f42a415e660 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42aae00640 1 -- 172.21.15.28:0/1706171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a415e660 msgr2=0x7f42a406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.530+0000 7f42aae00640 1 --2- 172.21.15.28:0/1706171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a415e660 0x7f42a406ccd0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f42900097a0 tx=0x7f429002a230 comp rx=0 tx=0).stop 2024-09-10T10:55:09.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.531+0000 7f42aae00640 1 -- 172.21.15.28:0/1706171 shutdown_connections 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.531+0000 7f42aae00640 1 --2- 172.21.15.28:0/1706171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a415e660 0x7f42a406ccd0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.531+0000 7f42aae00640 1 -- 172.21.15.28:0/1706171 >> 172.21.15.28:0/1706171 conn(0x7f42a408bfc0 msgr2=0x7f42a40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.531+0000 7f42aae00640 1 -- 172.21.15.28:0/1706171 shutdown_connections 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.531+0000 7f42aae00640 1 -- 172.21.15.28:0/1706171 wait complete. 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.532+0000 7f42aae00640 1 Processor -- start 2024-09-10T10:55:09.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.532+0000 7f42aae00640 1 -- start start 2024-09-10T10:55:09.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 0x7f42a415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a415e660 0x7f42a41f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40d3220 con 0x7f42a40d1f90 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42a40cf330 con 0x7f42a40d0dd0 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42aae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f42a40d33a0 con 0x7f42a415e660 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.533+0000 7f42a9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60250/0 (socket says 172.21.15.28:60250) 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 -- 172.21.15.28:0/140242786 learned_addr learned my addr 172.21.15.28:0/140242786 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 -- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a415e660 msgr2=0x7f42a41f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a415e660 0x7f42a41f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9a00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 0x7f42a415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 -- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 msgr2=0x7f42a415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 0x7f42a415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 -- 172.21.15.28:0/140242786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42a4161020 con 0x7f42a40d1f90 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9a00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 0x7f42a415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.534+0000 7f42a9000640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4290035040 tx=0x7f42900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.535+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290037040 con 0x7f42a40d1f90 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.535+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290030770 con 0x7f42a40d1f90 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.535+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290030b80 con 0x7f42a40d1f90 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.535+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42a415f620 con 0x7f42a40d1f90 2024-09-10T10:55:09.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.535+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42a415fbe0 con 0x7f42a40d1f90 2024-09-10T10:55:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.536+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42a4161ca0 con 0x7f42a40d1f90 2024-09-10T10:55:09.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.537+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4290044020 con 0x7f42a40d1f90 2024-09-10T10:55:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.538+0000 7f42a1600640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 0x7f428406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.538+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7f42900ac2a0 con 0x7f42a40d1f90 2024-09-10T10:55:09.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.538+0000 7f42a9a00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 0x7f428406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:09.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.539+0000 7f42a9a00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 0x7f428406e340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4298001680 tx=0x7f4298009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:09.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.543+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42a4161ca0 con 0x7f42a40d1f90 2024-09-10T10:55:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.738+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42a406cff0 con 0x7f42a40d1f90 2024-09-10T10:55:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.738+0000 7f42a1600640 1 -- 172.21.15.28:0/140242786 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 72 entries v136) ==== 81+0+4374 (secure 0 0 0) 0x7f42a4161020 con 0x7f42a40d1f90 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/382198082 2024-09-11T10:54:53.753232+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/382198082 2024-09-11T10:54:53.753232+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2572217302 2024-09-11T10:54:48.320770+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2572217302 2024-09-11T10:54:48.320770+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1056314461 2024-09-11T10:54:36.849623+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/509398397 2024-09-11T10:54:14.076481+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:55:09.741 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4294783280 2024-09-11T10:52:00.689161+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2888169760 2024-09-11T10:51:46.682332+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1056314461 2024-09-11T10:54:36.849623+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:55:09.742 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/41755343 2024-09-11T10:51:42.728295+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3452166242 2024-09-11T10:51:28.234007+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2774580459 2024-09-11T10:52:14.190678+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2312444496 2024-09-11T10:52:56.696320+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1408648899 2024-09-11T10:51:00.168974+0000 2024-09-10T10:55:09.743 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3446889436 2024-09-11T10:50:37.902654+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1706206889 2024-09-11T10:53:40.729366+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/786875524 2024-09-11T10:51:40.159679+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/208022459 2024-09-11T10:53:48.793090+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3986730211 2024-09-11T10:52:16.694472+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/1369429206 2024-09-11T10:51:53.780213+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1234809424 2024-09-11T10:51:14.743287+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/1314136636 2024-09-11T10:52:23.789551+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:55:09.744 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2147565603 2024-09-11T10:54:09.041224+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/205773431 2024-09-11T10:52:34.853419+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/702039081 2024-09-11T10:52:39.904721+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/3765569847 2024-09-11T10:52:30.832413+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3186644301 2024-09-11T10:51:06.680487+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3539657083 2024-09-11T10:53:36.707389+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/4027758898 2024-09-11T10:53:04.775511+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2836905257 2024-09-11T10:53:08.808221+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/236564669 2024-09-11T10:51:10.699044+0000 2024-09-10T10:55:09.745 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2552808349 2024-09-11T10:52:54.674913+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/2814091387 2024-09-11T10:50:55.135868+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6832/2095575543 2024-09-11T10:53:12.842835+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/1093900039 2024-09-11T10:53:16.877889+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6834/2349436086 2024-09-11T10:53:20.909728+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/3588871837 2024-09-11T10:53:00.745487+0000 2024-09-10T10:55:09.746 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/2097945693 2024-09-11T10:53:34.409937+0000 2024-09-10T10:55:09.747 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/4204725630 2024-09-11T10:53:44.760451+0000 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:listed 72 entries 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 msgr2=0x7f428406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 0x7f428406e340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4298001680 tx=0x7f4298009040 comp rx=0 tx=0).stop 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 msgr2=0x7f42a415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f4290035040 tx=0x7f42900046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 shutdown_connections 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f428406be80 0x7f428406e340 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f42a415e660 0x7f42a41f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42a40d1f90 0x7f42a415ef80 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 --2- 172.21.15.28:0/140242786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42a40d0dd0 0x7f42a415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 >> 172.21.15.28:0/140242786 conn(0x7f42a408bfc0 msgr2=0x7f42a40b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 shutdown_connections 2024-09-10T10:55:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:09.741+0000 7f42aae00640 1 -- 172.21.15.28:0/140242786 wait complete. 2024-09-10T10:55:09.758 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:55:09.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:09.758 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T10:55:09.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:55:09.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:09.839 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:55:09.875 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:55:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.070+0000 7fd7bec00640 1 Processor -- start 2024-09-10T10:55:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.070+0000 7fd7bec00640 1 -- start start 2024-09-10T10:55:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.071+0000 7fd7bec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7b80b69d0 0x7fd7b8162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.071+0000 7fd7bec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 0x7fd7b8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.071+0000 7fd7bec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b815eb60 con 0x7fd7b80af280 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7bec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b80b6db0 con 0x7fd7b80b69d0 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7bec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b815ee40 con 0x7fd7b80b0180 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7afe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd7b80af280 0x7fd7b80afa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53068/0 (socket says 172.21.15.28:53068) 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7afe00640 1 -- 172.21.15.28:0/649632167 learned_addr learned my addr 172.21.15.28:0/649632167 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:10.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7b80b69d0 0x7fd7b8162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7bcc00640 1 --2- 172.21.15.28:0/649632167 >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 0x7fd7b8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7e00640 1 -- 172.21.15.28:0/649632167 >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 msgr2=0x7fd7b8154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7e00640 1 --2- 172.21.15.28:0/649632167 >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 0x7fd7b8154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7e00640 1 -- 172.21.15.28:0/649632167 >> v1:172.21.15.28:6789/0 conn(0x7fd7b80af280 legacy=0x7fd7b80afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7e00640 1 -- 172.21.15.28:0/649632167 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b815f120 con 0x7fd7b80b69d0 2024-09-10T10:55:10.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7b7400640 1 -- 172.21.15.28:0/649632167 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237039436 0 0) 0x7fd7b815eb60 con 0x7fd7b80af280 2024-09-10T10:55:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.072+0000 7fd7bcc00640 1 --2- 172.21.15.28:0/649632167 >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 0x7fd7b8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:10.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.073+0000 7fd7b7e00640 1 --2- 172.21.15.28:0/649632167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80b69d0 0x7fd7b8162a50 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd7b815c590 tx=0x7fd7a802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc043fb685db751 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7b7400640 1 -- 172.21.15.28:0/649632167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8037070 con 0x7fd7b80b69d0 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7b7400640 1 -- 172.21.15.28:0/649632167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a802add0 con 0x7fd7b80b69d0 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7b7400640 1 -- 172.21.15.28:0/649632167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8032660 con 0x7fd7b80b69d0 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 -- 172.21.15.28:0/649632167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80b69d0 msgr2=0x7fd7b8162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 --2- 172.21.15.28:0/649632167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80b69d0 0x7fd7b8162a50 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd7b815c590 tx=0x7fd7a802a080 comp rx=0 tx=0).stop 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 -- 172.21.15.28:0/649632167 shutdown_connections 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 --2- 172.21.15.28:0/649632167 >> v2:172.21.15.138:3300/0 conn(0x7fd7b80b0180 0x7fd7b8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 --2- 172.21.15.28:0/649632167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80b69d0 0x7fd7b8162a50 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 -- 172.21.15.28:0/649632167 >> 172.21.15.28:0/649632167 conn(0x7fd7b806bc10 msgr2=0x7fd7b806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 -- 172.21.15.28:0/649632167 shutdown_connections 2024-09-10T10:55:10.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.074+0000 7fd7bec00640 1 -- 172.21.15.28:0/649632167 wait complete. 2024-09-10T10:55:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.076+0000 7fd7bec00640 1 Processor -- start 2024-09-10T10:55:10.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.076+0000 7fd7bec00640 1 -- start start 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b80b0180 0x7fd7b815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7b80b69d0 0x7fd7b81540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b8155240 con 0x7fd7b80af280 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b80af710 con 0x7fd7b80b0180 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b81553c0 con 0x7fd7b80b69d0 2024-09-10T10:55:10.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60274/0 (socket says 172.21.15.28:60274) 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7b7e00640 1 -- 172.21.15.28:0/2263114336 learned_addr learned my addr 172.21.15.28:0/2263114336 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7afe00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b80b0180 0x7fd7b815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.077+0000 7fd7bcc00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7b80b69d0 0x7fd7b81540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 -- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7b80b69d0 msgr2=0x7fd7b81540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7b80b69d0 0x7fd7b81540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 -- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b80b0180 msgr2=0x7fd7b815dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b80b0180 0x7fd7b815dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 -- 172.21.15.28:0/2263114336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b8152b30 con 0x7fd7b80af280 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b7e00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8037050 con 0x7fd7b80af280 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a8038040 con 0x7fd7b80af280 2024-09-10T10:55:10.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8004270 con 0x7fd7b80af280 2024-09-10T10:55:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b8152e10 con 0x7fd7b80af280 2024-09-10T10:55:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.078+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b80ce7e0 con 0x7fd7b80af280 2024-09-10T10:55:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.079+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b8155780 con 0x7fd7b80af280 2024-09-10T10:55:10.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.080+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd7a8003610 con 0x7fd7b80af280 2024-09-10T10:55:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.081+0000 7fd7b4c00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 0x7fd79406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.081+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9400+0+0 (secure 0 0 0) 0x7fd7a80ac400 con 0x7fd7b80af280 2024-09-10T10:55:10.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.081+0000 7fd7afe00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 0x7fd79406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.082+0000 7fd7afe00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 0x7fd79406e2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd7b806d240 tx=0x7fd7a0007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.085+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd7a8079e90 con 0x7fd7b80af280 2024-09-10T10:55:10.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.309+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd7b8162350 con 0x7fd7b80af280 2024-09-10T10:55:10.312 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:10.310+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:55:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.312+0000 7fd7b4c00640 1 -- 172.21.15.28:0/2263114336 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v312) ==== 161+0+0 (secure 0 0 0) 0x7fd7a8079790 con 0x7fd7b80af280 2024-09-10T10:55:10.314 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:55:10.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 msgr2=0x7fd79406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 0x7fd79406e2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fd7b806d240 tx=0x7fd7a0007b50 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 msgr2=0x7fd7b8151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8000c00 tx=0x7fd7a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 shutdown_connections 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd79406be30 0x7fd79406e2f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd7b80b69d0 0x7fd7b81540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b80b0180 0x7fd7b815dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 --2- 172.21.15.28:0/2263114336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b80af280 0x7fd7b8151960 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 >> 172.21.15.28:0/2263114336 conn(0x7fd7b806bc10 msgr2=0x7fd7b80afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.315+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 shutdown_connections 2024-09-10T10:55:10.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.316+0000 7fd7bec00640 1 -- 172.21.15.28:0/2263114336 wait complete. 2024-09-10T10:55:10.319 INFO:tasks.ceph.mds.c.smithi028.stderr: -1817> 2024-09-10T10:55:00.799+0000 7f3b2ac00640 -1 mds.0.openfiles _load_finish: corrupted header/values: End of buffer [buffer:2] 2024-09-10T10:55:10.324 INFO:tasks.ceph.mds.c.smithi028.stderr: -539> 2024-09-10T10:55:08.784+0000 7f3b2fc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:10.337 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:55:10.368 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:55:10.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.493+0000 7faa6a800640 1 Processor -- start 2024-09-10T10:55:10.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.493+0000 7faa6a800640 1 -- start start 2024-09-10T10:55:10.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa6a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa640d0dd0 0x7faa640d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa6a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa6406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa6a800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa640cf000 con 0x7faa640d1f90 2024-09-10T10:55:10.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa6a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faa640b37d0 con 0x7faa640d0dd0 2024-09-10T10:55:10.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa6a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa640cf330 con 0x7faa6415e660 2024-09-10T10:55:10.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa69e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faa640d1f90 0x7faa640cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53076/0 (socket says 172.21.15.28:53076) 2024-09-10T10:55:10.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.494+0000 7faa69e00640 1 -- 172.21.15.28:0/1777227298 learned_addr learned my addr 172.21.15.28:0/1777227298 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa69400640 1 --2- 172.21.15.28:0/1777227298 >> v2:172.21.15.138:3300/0 conn(0x7faa640d0dd0 0x7faa640d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa68a00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa6406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2537462927 0 0) 0x7faa640cf000 con 0x7faa640d1f90 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa4c003520 con 0x7faa640d1f90 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244226679 0 0) 0x7faa4c003520 con 0x7faa640d1f90 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 msgr2=0x7faa6406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa6406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 >> v2:172.21.15.138:3300/0 conn(0x7faa640d0dd0 msgr2=0x7faa640d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 --2- 172.21.15.28:0/1777227298 >> v2:172.21.15.138:3300/0 conn(0x7faa640d0dd0 0x7faa640d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa640d16f0 con 0x7faa640d1f90 2024-09-10T10:55:10.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7faa5c0019f0 con 0x7faa640d1f90 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 >> v1:172.21.15.28:6789/0 conn(0x7faa640d1f90 legacy=0x7faa640cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa4c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa4c003f80 0x7faa4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.495+0000 7faa63e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4c004d40 0x7faa4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa640cf330 con 0x7faa4c004d40 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa4c00b9c0 con 0x7faa4c003f80 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa4c00bb40 con 0x7faa6415e660 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4c004d40 0x7faa4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69400640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa4c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa68a00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa4c003f80 0x7faa4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 -- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 msgr2=0x7faa4c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa6415e660 0x7faa4c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 -- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa4c003f80 msgr2=0x7faa4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa4c003f80 0x7faa4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa69e00640 1 -- 172.21.15.28:0/1777227298 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4c003520 con 0x7faa4c004d40 2024-09-10T10:55:10.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.496+0000 7faa68a00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa4c003f80 0x7faa4c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.497+0000 7faa69e00640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4c004d40 0x7faa4c0051c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a7c0 tx=0x7faa4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef7b8ee7741581c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.497+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c00a410 con 0x7faa4c004d40 2024-09-10T10:55:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.497+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c02f070 con 0x7faa4c004d40 2024-09-10T10:55:10.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.497+0000 7faa63e00640 1 -- 172.21.15.28:0/1777227298 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c02bbf0 con 0x7faa4c004d40 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 -- 172.21.15.28:0/1777227298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4c004d40 msgr2=0x7faa4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 --2- 172.21.15.28:0/1777227298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa4c004d40 0x7faa4c0051c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a7c0 tx=0x7faa4c005830 comp rx=0 tx=0).stop 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa69e00640 1 -- 172.21.15.28:0/1777227298 reap_dead start 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 -- 172.21.15.28:0/1777227298 shutdown_connections 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 -- 172.21.15.28:0/1777227298 >> 172.21.15.28:0/1777227298 conn(0x7faa6408bfc0 msgr2=0x7faa640d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 -- 172.21.15.28:0/1777227298 shutdown_connections 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.498+0000 7faa6a800640 1 -- 172.21.15.28:0/1777227298 wait complete. 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.500+0000 7faa6a800640 1 Processor -- start 2024-09-10T10:55:10.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.500+0000 7faa6a800640 1 -- start start 2024-09-10T10:55:10.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa4c004d40 0x7faa641f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa641588a0 0x7faa641f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa640d2f40 con 0x7faa641f4af0 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa640d2c60 con 0x7faa641588a0 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa6a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa640d3220 con 0x7faa4c004d40 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa69e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa69e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60294/0 (socket says 172.21.15.28:60294) 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa69e00640 1 -- 172.21.15.28:0/3881447402 learned_addr learned my addr 172.21.15.28:0/3881447402 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa69400640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa4c004d40 0x7faa641f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.501+0000 7faa68a00640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa641588a0 0x7faa641f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 -- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa4c004d40 msgr2=0x7faa641f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa4c004d40 0x7faa641f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 -- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa641588a0 msgr2=0x7faa641f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa641588a0 0x7faa641f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 -- 172.21.15.28:0/3881447402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa64158d20 con 0x7faa641f4af0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa69e00640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a8e0 tx=0x7faa5c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50009690 con 0x7faa641f4af0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa5c00a3f0 con 0x7faa641f4af0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c001f70 con 0x7faa641f4af0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa641f5070 con 0x7faa641f4af0 2024-09-10T10:55:10.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.502+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa641f7df0 con 0x7faa641f4af0 2024-09-10T10:55:10.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.503+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa640d2440 con 0x7faa641f4af0 2024-09-10T10:55:10.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.504+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faa5c034670 con 0x7faa641f4af0 2024-09-10T10:55:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.505+0000 7faa61600640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 0x7faa4406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.505+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9486+0+0 (secure 0 0 0) 0x7faa5c0abcd0 con 0x7faa641f4af0 2024-09-10T10:55:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.506+0000 7faa69400640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 0x7faa4406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.507+0000 7faa69400640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 0x7faa4406e320 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faa6406c9a0 tx=0x7faa58009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:10.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.510+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faa5c079860 con 0x7faa641f4af0 2024-09-10T10:55:10.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:10.766+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7faa640d1b20 con 0x7faa641f4af0 2024-09-10T10:55:10.871 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:55:10.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:55:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.318+0000 7faa61600640 1 -- 172.21.15.28:0/3881447402 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 nothing to do: rank is not damaged v313) ==== 107+0+0 (secure 0 0 0) 0x7faa5c079160 con 0x7faa641f4af0 2024-09-10T10:55:11.320 INFO:teuthology.orchestra.run.smithi028.stderr:nothing to do: rank is not damaged 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 msgr2=0x7faa4406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 0x7faa4406e320 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7faa6406c9a0 tx=0x7faa58009190 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 msgr2=0x7faa6415cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7faa5c00a8e0 tx=0x7faa5c001360 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 shutdown_connections 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7faa4406be60 0x7faa4406e320 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa641f4af0 0x7faa6415cba0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa641588a0 0x7faa641f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 --2- 172.21.15.28:0/3881447402 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa4c004d40 0x7faa641f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.320+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 >> 172.21.15.28:0/3881447402 conn(0x7faa6408bfc0 msgr2=0x7faa640b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.321+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 shutdown_connections 2024-09-10T10:55:11.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.321+0000 7faa6a800640 1 -- 172.21.15.28:0/3881447402 wait complete. 2024-09-10T10:55:11.340 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:55:11.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.414+0000 7f4ae6afaf40 1 Processor -- start 2024-09-10T10:55:11.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.414+0000 7f4ae6afaf40 1 -- start start 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e10fadcb0 0x561e10fda2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10ff32d0 0x561e11081450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561e10ff2dc0 con 0x561e1107f4b0 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e10f51b30 con 0x561e10ff32d0 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561e10ff21f0 con 0x561e10fadcb0 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60298/0 (socket says 172.21.15.28:60298) 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae4600640 1 -- 172.21.15.28:0/2867705979 learned_addr learned my addr 172.21.15.28:0/2867705979 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e10fadcb0 0x561e10fda2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae3200640 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10ff32d0 0x561e11081450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae4600640 1 -- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e10fadcb0 msgr2=0x561e10fda2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.415+0000 7f4ae4600640 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e10fadcb0 0x561e10fda2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae4600640 1 -- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10ff32d0 msgr2=0x561e11081450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae4600640 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10ff32d0 0x561e11081450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae4600640 1 -- 172.21.15.28:0/2867705979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e10fd56e0 con 0x561e1107f4b0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae4600640 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0027a0 tx=0x7f4adc002c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38b572e13e6723e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae1e00640 1 -- 172.21.15.28:0/2867705979 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc016020 con 0x561e1107f4b0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae1e00640 1 -- 172.21.15.28:0/2867705979 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc00d9c0 con 0x561e1107f4b0 2024-09-10T10:55:11.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.416+0000 7f4ae1e00640 1 -- 172.21.15.28:0/2867705979 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00ddb0 con 0x561e1107f4b0 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/2867705979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 msgr2=0x561e10ff3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0027a0 tx=0x7f4adc002c60 comp rx=0 tx=0).stop 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/2867705979 shutdown_connections 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e1107f4b0 0x561e10ff3930 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10ff32d0 0x561e11081450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/2867705979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e10fadcb0 0x561e10fda2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/2867705979 >> 172.21.15.28:0/2867705979 conn(0x561e10fd7b90 msgr2=0x561e10fd7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/2867705979 shutdown_connections 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/2867705979 wait complete. 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.417+0000 7f4ae6afaf40 1 Processor -- start 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 -- start start 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10fadcb0 0x561e1106ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e1107f4b0 0x561e10fd7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x561e10fd5490 con 0x561e10ff32d0 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561e11082840 con 0x561e10fadcb0 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.418+0000 7f4ae6afaf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x561e1107e5e0 con 0x561e1107f4b0 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60300/0 (socket says 172.21.15.28:60300) 2024-09-10T10:55:11.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 learned_addr learned my addr 172.21.15.28:0/4152548192 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae4600640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e1107f4b0 0x561e10fd7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10fadcb0 0x561e1106ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e1107f4b0 msgr2=0x561e10fd7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x561e1107f4b0 0x561e10fd7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10fadcb0 msgr2=0x561e1106ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10fadcb0 0x561e1106ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1106f3b0 con 0x561e10ff32d0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561e10fadcb0 0x561e1106ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.419+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000a440 tx=0x7f4ad0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.420+0000 7f4ae0a00640 1 -- 172.21.15.28:0/4152548192 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad006c070 con 0x561e10ff32d0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.420+0000 7f4ae0a00640 1 -- 172.21.15.28:0/4152548192 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad005f7c0 con 0x561e10ff32d0 2024-09-10T10:55:11.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.420+0000 7f4ae0a00640 1 -- 172.21.15.28:0/4152548192 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad005fbb0 con 0x561e10ff32d0 2024-09-10T10:55:11.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.420+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e10fd8e80 con 0x561e10ff32d0 2024-09-10T10:55:11.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.420+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e10fee210 con 0x561e10ff32d0 2024-09-10T10:55:11.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.422+0000 7f4ae0a00640 1 -- 172.21.15.28:0/4152548192 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ad0052070 con 0x561e10ff32d0 2024-09-10T10:55:11.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.422+0000 7f4ae0a00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 0x7f4ac406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.423+0000 7f4ae0a00640 1 -- 172.21.15.28:0/4152548192 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9486+0+0 (secure 0 0 0) 0x7f4ad00ce370 con 0x561e10ff32d0 2024-09-10T10:55:11.426 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:55:11.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.423+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 0x561e11119ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.423+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1111a010 con 0x561e11081a00 2024-09-10T10:55:11.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.423+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 0x7f4ac406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.423+0000 7f4ae4600640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 0x561e11119ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.424+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 0x7f4ac406ccd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004170 tx=0x7f4ad4000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.424+0000 7f4ae4600640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 0x561e11119ad0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.426+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v137'1128 uv1102 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.426+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1111a3e0 con 0x561e11081a00 2024-09-10T10:55:11.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.428+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v137'1129 uv1129 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.428+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124150 con 0x561e11081a00 2024-09-10T10:55:11.432 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:55:11.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.430+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v137'1130 uv1130 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.430+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125500 con 0x561e11081a00 2024-09-10T10:55:11.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.432+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v137'1131 uv1131 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.432+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111258a0 con 0x561e11081a00 2024-09-10T10:55:11.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.433+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v137'1132 uv1132 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.434+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111244f0 con 0x561e11081a00 2024-09-10T10:55:11.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.435+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v137'1133 uv1133 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.436+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124900 con 0x561e11081a00 2024-09-10T10:55:11.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.438+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v137'1134 uv1108 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.438+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.440+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v137'1135 uv1135 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.440+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.443 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:55:11.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.442+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v137'1136 uv1136 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.442+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.443+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v137'1137 uv1137 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.443+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.446+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v137'1138 uv1138 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.446+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.447+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v137'1139 uv1139 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.447+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.449+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v137'1140 uv1115 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.449+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.450+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v137'1141 uv1141 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.450+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.454 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:55:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.452+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v137'1142 uv1142 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.452+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11124ca0 con 0x561e11081a00 2024-09-10T10:55:11.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.454+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v137'1143 uv1143 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.454+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125100 con 0x561e11081a00 2024-09-10T10:55:11.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.456+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v137'1144 uv1144 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.456+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125100 con 0x561e11081a00 2024-09-10T10:55:11.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.458+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v137'1145 uv1145 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.458+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.460+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v137'1146 uv1146 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.460+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.461+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v137'1147 uv1121 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.462+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.463+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v137'1148 uv1148 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.463+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.467 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:55:11.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.465+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v137'1149 uv1149 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.465+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11126c50 con 0x561e11081a00 2024-09-10T10:55:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.467+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v137'1150 uv1150 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.467+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11126c50 con 0x561e11081a00 2024-09-10T10:55:11.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.469+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v137'1151 uv1151 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.469+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.471+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v137'1152 uv1152 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.471+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.473+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v137'1153 uv1127 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.473+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.475+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v137'1154 uv1154 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.475+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.478 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:55:11.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.477+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v137'1155 uv1155 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.477+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11126c50 con 0x561e11081a00 2024-09-10T10:55:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.479+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v137'1156 uv1156 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.479+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11126c50 con 0x561e11081a00 2024-09-10T10:55:11.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.480+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v137'1157 uv1157 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.480+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11125c40 con 0x561e11081a00 2024-09-10T10:55:11.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.481+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v137'1158 uv1158 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e11081a00 2024-09-10T10:55:11.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.481+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 0x561e11128540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.481+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11128a80 con 0x561e11126150 2024-09-10T10:55:11.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.482+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 0x561e11128540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.482+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 0x561e11128540 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.484+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v137'824 uv803 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.484+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11128e20 con 0x561e11126150 2024-09-10T10:55:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.486+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v137'825 uv825 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.486+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111291c0 con 0x561e11126150 2024-09-10T10:55:11.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.488+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v137'826 uv826 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.490 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:55:11.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.488+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11129560 con 0x561e11126150 2024-09-10T10:55:11.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.490+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v137'827 uv827 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.490+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11129900 con 0x561e11126150 2024-09-10T10:55:11.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.491+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v137'828 uv828 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.491+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11129ca0 con 0x561e11126150 2024-09-10T10:55:11.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.492+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v137'829 uv822 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.492+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.494+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v137'830 uv830 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.494+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.495+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v137'831 uv831 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.497 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:55:11.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.495+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.496+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v137'832 uv832 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.496+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.498+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v137'833 uv833 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.498+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.499+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v137'834 uv823 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.500+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.501+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v137'835 uv835 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.501+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.502+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v137'836 uv836 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.504 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:55:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.502+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112e730 con 0x561e11126150 2024-09-10T10:55:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.504+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v137'837 uv837 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.504+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112e730 con 0x561e11126150 2024-09-10T10:55:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.505+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v137'838 uv838 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.505+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.507+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v137'839 uv819 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.507+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.508+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v137'840 uv840 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.508+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.509+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v137'841 uv841 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.512 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:55:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.510+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112b050 con 0x561e11126150 2024-09-10T10:55:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.511+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v137'842 uv842 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.511+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112b050 con 0x561e11126150 2024-09-10T10:55:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.512+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v137'843 uv843 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.512+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112a040 con 0x561e11126150 2024-09-10T10:55:11.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.513+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v137'844 uv844 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11126150 2024-09-10T10:55:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.513+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x561e1112a6c0 0x561e1112cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.514+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112cff0 con 0x561e1112a6c0 2024-09-10T10:55:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.514+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x561e1112a6c0 0x561e1112cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.514+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x561e1112a6c0 0x561e1112cab0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.516+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v137'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.516+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112d4e0 con 0x561e1112a6c0 2024-09-10T10:55:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.518+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v137'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.518+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112d9c0 con 0x561e1112a6c0 2024-09-10T10:55:11.521 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:55:11.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.519+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v137'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.519+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112f270 con 0x561e1112a6c0 2024-09-10T10:55:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.520+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v137'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.520+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112f610 con 0x561e1112a6c0 2024-09-10T10:55:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.522+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v137'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.522+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112e260 con 0x561e1112a6c0 2024-09-10T10:55:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.523+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v137'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.523+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x561e1112eb60 0x561e111319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.523+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11131f00 con 0x561e1112eb60 2024-09-10T10:55:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.523+0000 7f4ae4600640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x561e1112eb60 0x561e111319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.524+0000 7f4ae4600640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x561e1112eb60 0x561e111319c0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.525+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v137'164 uv163 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.525+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11131f00 con 0x561e1112eb60 2024-09-10T10:55:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.526+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v137'165 uv165 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.526+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111322f0 con 0x561e1112eb60 2024-09-10T10:55:11.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.527+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v137'166 uv166 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.530 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:55:11.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.528+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111322f0 con 0x561e1112eb60 2024-09-10T10:55:11.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.529+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v137'167 uv167 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.529+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111322f0 con 0x561e1112a6c0 2024-09-10T10:55:11.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.530+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v137'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.530+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.531+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v137'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.531+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.534 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:55:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.532+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v137'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.533+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111336a0 con 0x561e1112a6c0 2024-09-10T10:55:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.534+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v137'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.534+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111336a0 con 0x561e1112a6c0 2024-09-10T10:55:11.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.535+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v137'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.535+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.536+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v137'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.536+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.538+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v137'404 uv398 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.538+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.539+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v137'405 uv405 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.539+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.541+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v137'406 uv406 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.543 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:55:11.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.541+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111336a0 con 0x561e1112eb60 2024-09-10T10:55:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.542+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v137'407 uv407 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.542+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111336a0 con 0x561e1112eb60 2024-09-10T10:55:11.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.543+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v137'408 uv408 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.543+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.545+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v137'409 uv409 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.545+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.546+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v137'410 uv403 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.546+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.547+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v137'411 uv411 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.547+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.549+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v137'412 uv412 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.551 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:55:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.549+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.551+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v137'413 uv413 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.551+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112eb60 2024-09-10T10:55:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.552+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v137'414 uv414 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.552+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.553+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v137'183 uv182 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.553+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.555+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v137'184 uv184 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.555+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.558 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:55:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.556+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v137'185 uv185 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.556+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.557+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v137'186 uv186 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.557+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11132690 con 0x561e1112a6c0 2024-09-10T10:55:11.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.558+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v137'187 uv187 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e1112a6c0 2024-09-10T10:55:11.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.558+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561e11132d10 0x561e11135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.558+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135680 con 0x561e11132d10 2024-09-10T10:55:11.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.559+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561e11132d10 0x561e11135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.559+0000 7f4ae3c00640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561e11132d10 0x561e11135100 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.561+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v137'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.561+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11135a50 con 0x561e11132d10 2024-09-10T10:55:11.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.562+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v137'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.562+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135a50 con 0x561e11132d10 2024-09-10T10:55:11.565 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:55:11.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.563+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v137'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.563+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11136a60 con 0x561e11132d10 2024-09-10T10:55:11.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.565+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v137'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.565+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11136a60 con 0x561e11132d10 2024-09-10T10:55:11.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.566+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v137'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.566+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de30 con 0x561e11132d10 2024-09-10T10:55:11.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.567+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v137'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.567+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de30 con 0x561e11132d10 2024-09-10T10:55:11.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.569+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v137'365 uv359 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.569+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de30 con 0x561e11132d10 2024-09-10T10:55:11.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.570+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v137'366 uv366 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.570+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135ad0 con 0x561e11132d10 2024-09-10T10:55:11.573 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:55:11.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.572+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v137'367 uv367 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.572+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.573+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v137'368 uv368 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.573+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.574+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v137'369 uv369 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.574+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.575+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v137'370 uv364 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.575+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.576+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v137'371 uv371 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.576+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.579 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:55:11.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.578+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v137'372 uv372 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.578+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e1112de50 con 0x561e11132d10 2024-09-10T10:55:11.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.579+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v137'373 uv373 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.579+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111389d0 con 0x561e11132d10 2024-09-10T10:55:11.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.580+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v137'374 uv374 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.580+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135ad0 con 0x561e1112eb60 2024-09-10T10:55:11.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.581+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v137'162 uv160 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.581+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11135ad0 con 0x561e1112eb60 2024-09-10T10:55:11.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.583+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v137'163 uv163 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.583+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135ad0 con 0x561e1112eb60 2024-09-10T10:55:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.584+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v137'164 uv164 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.586 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:55:11.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.584+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.585+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v137'165 uv165 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.585+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.587+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v137'219 uv218 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.587+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.588+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v137'220 uv220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.588+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.589+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v137'221 uv221 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.591 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:55:11.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.589+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.590+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v137'222 uv222 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.590+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.592+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v137'223 uv223 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.592+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e1112eb60 2024-09-10T10:55:11.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.593+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v137'224 uv224 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4adc012470 con 0x561e1112eb60 2024-09-10T10:55:11.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.593+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x561e11136880 0x561e11136c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.593+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11135f80 con 0x561e11136880 2024-09-10T10:55:11.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.593+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x561e11136880 0x561e11136c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.594+0000 7f4ae3200640 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x561e11136880 0x561e11136c60 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.595+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v137'186 uv185 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e11136880 2024-09-10T10:55:11.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.595+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11136880 2024-09-10T10:55:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.596+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v137'187 uv187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e11136880 2024-09-10T10:55:11.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.596+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11136880 2024-09-10T10:55:11.599 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:55:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.597+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v137'188 uv188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e11136880 2024-09-10T10:55:11.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.597+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11136880 2024-09-10T10:55:11.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.599+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v137'189 uv189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e11136880 2024-09-10T10:55:11.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.599+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11136880 2024-09-10T10:55:11.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.600+0000 7f4ae3200640 1 -- 172.21.15.28:0/4152548192 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v137'190 uv190 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f4ad005fd50 con 0x561e11136880 2024-09-10T10:55:11.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.600+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.601+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v137'455 uv446 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.601+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.602+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v137'456 uv456 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.602+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.605 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:55:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.604+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v137'457 uv457 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.604+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.605+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v137'458 uv458 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.605+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.606+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v137'459 uv459 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.606+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.607+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v137'460 uv453 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.607+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.609+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v137'461 uv461 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.609+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.612 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:55:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.610+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v137'462 uv462 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.610+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.611+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v137'463 uv463 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.611+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.612+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v137'464 uv464 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.612+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e111371a0 con 0x561e11132d10 2024-09-10T10:55:11.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.614+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v137'465 uv465 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.614+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x561e11137a40 con 0x561e11132d10 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae3c00640 1 -- 172.21.15.28:0/4152548192 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v137'466 uv466 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f4ad400d870 con 0x561e11132d10 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x561e1112a6c0 msgr2=0x561e1112cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x561e1112a6c0 0x561e1112cab0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561e11132d10 msgr2=0x561e11135100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x561e11132d10 0x561e11135100 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x561e11136880 msgr2=0x561e11136c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x561e11136880 0x561e11136c60 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x561e1112eb60 msgr2=0x561e111319c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x561e1112eb60 0x561e111319c0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 msgr2=0x561e11128540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 reap_dead start 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 0x561e11128540 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae4600640 1 -- 172.21.15.28:0/4152548192 reap_dead start 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 msgr2=0x561e11119ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 0x561e11119ad0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 msgr2=0x7f4ac406ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 0x7f4ac406ccd0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4004170 tx=0x7f4ad4000bc0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 msgr2=0x561e10fd6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.615+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000a440 tx=0x7f4ad0002fb0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 shutdown_connections 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x561e11126150 0x561e11128540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x561e11081a00 0x561e11119ad0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ac406a810 0x7f4ac406ccd0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 --2- 172.21.15.28:0/4152548192 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e10ff32d0 0x561e10fd6ff0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 >> 172.21.15.28:0/4152548192 conn(0x561e10fd7b90 msgr2=0x561e10fad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 shutdown_connections 2024-09-10T10:55:11.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.616+0000 7f4ae6afaf40 1 -- 172.21.15.28:0/4152548192 wait complete. 2024-09-10T10:55:11.623 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 100.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:55:11.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.697+0000 7fe6fa20cf40 1 Processor -- start 2024-09-10T10:55:11.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.697+0000 7fe6fa20cf40 1 -- start start 2024-09-10T10:55:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12ae92d0 0x55ac12b6fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12b754b0 0x55ac12b77590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ac12ae8dc0 con 0x55ac12aa35a0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac12a47b30 con 0x55ac12b754b0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ac12ae81f0 con 0x55ac12ae92d0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6f6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60318/0 (socket says 172.21.15.28:60318) 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.698+0000 7fe6f6800640 1 -- 172.21.15.28:0/937787838 learned_addr learned my addr 172.21.15.28:0/937787838 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f7200640 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12b754b0 0x55ac12b77590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f5e00640 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12ae92d0 0x55ac12b6fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 -- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12ae92d0 msgr2=0x55ac12b6fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12ae92d0 0x55ac12b6fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 -- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12b754b0 msgr2=0x55ac12b77590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12b754b0 0x55ac12b77590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 -- 172.21.15.28:0/937787838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12b74350 con 0x55ac12aa35a0 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f6800640 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800a440 tx=0x7fe6e8051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55056997589a6a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f4a00640 1 -- 172.21.15.28:0/937787838 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8063070 con 0x55ac12aa35a0 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f4a00640 1 -- 172.21.15.28:0/937787838 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e8055b20 con 0x55ac12aa35a0 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.699+0000 7fe6f4a00640 1 -- 172.21.15.28:0/937787838 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8005700 con 0x55ac12aa35a0 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/937787838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 msgr2=0x55ac12ae6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800a440 tx=0x7fe6e8051000 comp rx=0 tx=0).stop 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/937787838 shutdown_connections 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12b754b0 0x55ac12b77590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12ae92d0 0x55ac12b6fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/937787838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12aa35a0 0x55ac12ae6b40 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/937787838 >> 172.21.15.28:0/937787838 conn(0x55ac12acdb90 msgr2=0x55ac12b78630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/937787838 shutdown_connections 2024-09-10T10:55:11.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.700+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/937787838 wait complete. 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.701+0000 7fe6fa20cf40 1 Processor -- start 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.701+0000 7fe6fa20cf40 1 -- start start 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.701+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12aa35a0 0x55ac12b63060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.701+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 0x55ac12b73a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6fa20cf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12b754b0 0x55ac12ae6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55ac12b76ec0 con 0x55ac12ae92d0 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac12b703c0 con 0x55ac12aa35a0 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6fa20cf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55ac12b70630 con 0x55ac12b754b0 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 0x55ac12b73a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 0x55ac12b73a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60326/0 (socket says 172.21.15.28:60326) 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 -- 172.21.15.28:0/1561152555 learned_addr learned my addr 172.21.15.28:0/1561152555 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:11.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f6800640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12aa35a0 0x55ac12b63060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f7200640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12b754b0 0x55ac12ae6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 -- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12b754b0 msgr2=0x55ac12ae6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55ac12b754b0 0x55ac12ae6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 -- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12aa35a0 msgr2=0x55ac12b63060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac12aa35a0 0x55ac12b63060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 -- 172.21.15.28:0/1561152555 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac12b6d7b0 con 0x55ac12ae92d0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.702+0000 7fe6f5e00640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 0x55ac12b73a90 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000f60 tx=0x7fe6e400df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.703+0000 7fe6ef400640 1 -- 172.21.15.28:0/1561152555 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4014020 con 0x55ac12ae92d0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.703+0000 7fe6ef400640 1 -- 172.21.15.28:0/1561152555 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e4010410 con 0x55ac12ae92d0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.703+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac12b6da90 con 0x55ac12ae92d0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.703+0000 7fe6ef400640 1 -- 172.21.15.28:0/1561152555 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4008030 con 0x55ac12ae92d0 2024-09-10T10:55:11.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.703+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac12b6e050 con 0x55ac12ae92d0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.705+0000 7fe6ef400640 1 -- 172.21.15.28:0/1561152555 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe6e4017020 con 0x55ac12ae92d0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.705+0000 7fe6ef400640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6d806a840 0x7fe6d806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.706+0000 7fe6ef400640 1 -- 172.21.15.28:0/1561152555 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9486+0+0 (secure 0 0 0) 0x7fe6e408a780 con 0x55ac12ae92d0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.706+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ac12b6e850 0x55ac12b6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.706+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137) -- 0x55ac12b6d340 con 0x55ac12b6e850 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.706+0000 7fe6f6800640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6d806a840 0x7fe6d806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.706+0000 7fe6f7200640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ac12b6e850 0x55ac12b6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.707+0000 7fe6f7200640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ac12b6e850 0x55ac12b6ec60 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.707+0000 7fe6f6800640 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6d806a840 0x7fe6d806cd00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8064000 tx=0x7fe6e8053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6f7200640 1 -- 172.21.15.28:0/1561152555 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 100.00000000 [omap-set-header] v137'375 uv375 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fe6f000f7a0 con 0x55ac12b6e850 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ac12b6e850 msgr2=0x55ac12b6ec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55ac12b6e850 0x55ac12b6ec60 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6d806a840 msgr2=0x7fe6d806cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe6d806a840 0x7fe6d806cd00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8064000 tx=0x7fe6e8053000 comp rx=0 tx=0).stop 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 msgr2=0x55ac12b73a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 --2- 172.21.15.28:0/1561152555 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55ac12ae92d0 0x55ac12b73a90 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4000f60 tx=0x7fe6e400df90 comp rx=0 tx=0).stop 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6f7200640 1 -- 172.21.15.28:0/1561152555 reap_dead start 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 shutdown_connections 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 >> 172.21.15.28:0/1561152555 conn(0x55ac12acdb90 msgr2=0x55ac12ae6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 shutdown_connections 2024-09-10T10:55:11.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.708+0000 7fe6fa20cf40 1 -- 172.21.15.28:0/1561152555 wait complete. 2024-09-10T10:55:11.753 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:55:11.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.910+0000 7fc3af200640 1 Processor -- start 2024-09-10T10:55:11.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.910+0000 7fc3af200640 1 -- start start 2024-09-10T10:55:11.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3af200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc3a80ca7c0 0x7fc3a80cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3af200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 0x7fc3a80cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3af200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a815eb80 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3af200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a815ee60 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3af200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a815f140 con 0x7fc3a80caba0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ae800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc3a80cc6d0 0x7fc3a80ccb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53102/0 (socket says 172.21.15.28:53102) 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ae800640 1 -- 172.21.15.28:0/1662868665 learned_addr learned my addr 172.21.15.28:0/1662868665 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ade00640 1 --2- 172.21.15.28:0/1662868665 >> v2:172.21.15.28:3300/0 conn(0x7fc3a80ca7c0 0x7fc3a80cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ad400640 1 --2- 172.21.15.28:0/1662868665 >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 0x7fc3a80cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3aca00640 1 -- 172.21.15.28:0/1662868665 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614834744 0 0) 0x7fc3a815eb80 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3aca00640 1 -- 172.21.15.28:0/1662868665 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc390003520 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ade00640 1 -- 172.21.15.28:0/1662868665 >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 msgr2=0x7fc3a80cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.912+0000 7fc3ade00640 1 --2- 172.21.15.28:0/1662868665 >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 0x7fc3a80cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.913+0000 7fc3ade00640 1 -- 172.21.15.28:0/1662868665 >> v1:172.21.15.28:6789/0 conn(0x7fc3a80cc6d0 legacy=0x7fc3a80ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.913+0000 7fc3ade00640 1 -- 172.21.15.28:0/1662868665 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a815f420 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.913+0000 7fc3ad400640 1 --2- 172.21.15.28:0/1662868665 >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 0x7fc3a80cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3ade00640 1 --2- 172.21.15.28:0/1662868665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a80cb2e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80ca500 tx=0x7fc39c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0eff3634f2196b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3aca00640 1 -- 172.21.15.28:0/1662868665 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c037070 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3aca00640 1 -- 172.21.15.28:0/1662868665 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39c032440 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3aca00640 1 -- 172.21.15.28:0/1662868665 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c02ae60 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 -- 172.21.15.28:0/1662868665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80ca7c0 msgr2=0x7fc3a80cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 --2- 172.21.15.28:0/1662868665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a80cb2e0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80ca500 tx=0x7fc39c02a110 comp rx=0 tx=0).stop 2024-09-10T10:55:11.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 -- 172.21.15.28:0/1662868665 shutdown_connections 2024-09-10T10:55:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 --2- 172.21.15.28:0/1662868665 >> v2:172.21.15.138:3300/0 conn(0x7fc3a80caba0 0x7fc3a80cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 --2- 172.21.15.28:0/1662868665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a80cb2e0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.914+0000 7fc3af200640 1 -- 172.21.15.28:0/1662868665 >> 172.21.15.28:0/1662868665 conn(0x7fc3a80b1bd0 msgr2=0x7fc3a80b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.915+0000 7fc3af200640 1 -- 172.21.15.28:0/1662868665 shutdown_connections 2024-09-10T10:55:11.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.915+0000 7fc3af200640 1 -- 172.21.15.28:0/1662868665 wait complete. 2024-09-10T10:55:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.916+0000 7fc3af200640 1 Processor -- start 2024-09-10T10:55:11.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.916+0000 7fc3af200640 1 -- start start 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3a80caba0 0x7fc3a80b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a80728f0 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a8072770 con 0x7fc3a80ca7c0 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3af200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a80cd230 con 0x7fc3a80caba0 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3ae800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3ae800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60342/0 (socket says 172.21.15.28:60342) 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3ae800640 1 -- 172.21.15.28:0/3976533789 learned_addr learned my addr 172.21.15.28:0/3976533789 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.917+0000 7fc3ad400640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3a80caba0 0x7fc3a80b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ade00640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 -- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3a80caba0 msgr2=0x7fc3a80b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3a80caba0 0x7fc3a80b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 -- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3a80ca7c0 msgr2=0x7fc3a8159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a8159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 -- 172.21.15.28:0/3976533789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a80b3580 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3ae800640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400caa0 tx=0x7fc3a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4017070 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a400ac00 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a80b3860 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.918+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4012400 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.919+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a80b3e20 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.920+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a80cd620 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.920+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3a401a070 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.922+0000 7fc3a2000640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 0x7fc38c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.922+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9486+0+0 (secure 0 0 0) 0x7fc3a408bac0 con 0x7fc3a80cc6d0 2024-09-10T10:55:11.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.922+0000 7fc3ade00640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 0x7fc38c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:11.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.923+0000 7fc3ade00640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 0x7fc38c06e2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc39c034000 tx=0x7fc39c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:11.927+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3a4058f40 con 0x7fc3a80cc6d0 2024-09-10T10:55:12.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.147+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fc3a80ce390 con 0x7fc3a80cc6d0 2024-09-10T10:55:12.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.731+0000 7fc3a2000640 1 -- 172.21.15.28:0/3976533789 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v314) ==== 162+0+0 (secure 0 0 0) 0x7fc3a405d240 con 0x7fc3a80cc6d0 2024-09-10T10:55:12.734 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:55:12.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 msgr2=0x7fc38c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 0x7fc38c06e2f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fc39c034000 tx=0x7fc39c02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 msgr2=0x7fc3a81586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400caa0 tx=0x7fc3a400cf60 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 shutdown_connections 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc38c06be30 0x7fc38c06e2f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3a80cc6d0 0x7fc3a81586d0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3a80caba0 0x7fc3a80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 --2- 172.21.15.28:0/3976533789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3a80ca7c0 0x7fc3a8159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.735+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 >> 172.21.15.28:0/3976533789 conn(0x7fc3a80b1bd0 msgr2=0x7fc3a806dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.736+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 shutdown_connections 2024-09-10T10:55:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.736+0000 7fc3af200640 1 -- 172.21.15.28:0/3976533789 wait complete. 2024-09-10T10:55:12.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:12.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.917+0000 7fa3ed800640 1 Processor -- start 2024-09-10T10:55:12.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.917+0000 7fa3ed800640 1 -- start start 2024-09-10T10:55:12.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.918+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3ed800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa3e8072700 0x7fa3e815d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3ed800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e8146140 con 0x7fa3e8072320 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3ed800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e80cd740 con 0x7fa3e8072700 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3ed800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e80cda20 con 0x7fa3e8072de0 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3e6a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3e8072320 0x7fa3e80cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:53118/0 (socket says 172.21.15.28:53118) 2024-09-10T10:55:12.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3e6a00640 1 -- 172.21.15.28:0/2319642784 learned_addr learned my addr 172.21.15.28:0/2319642784 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3de000640 1 --2- 172.21.15.28:0/2319642784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3e7400640 1 --2- 172.21.15.28:0/2319642784 >> v2:172.21.15.138:3300/0 conn(0x7fa3e8072700 0x7fa3e815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657620466 0 0) 0x7fa3e8146140 con 0x7fa3e8072320 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.919+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3cc003520 con 0x7fa3e8072320 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.920+0000 7fa3de000640 1 -- 172.21.15.28:0/2319642784 >> v2:172.21.15.138:3300/0 conn(0x7fa3e8072700 msgr2=0x7fa3e815d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:12.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.920+0000 7fa3de000640 1 --2- 172.21.15.28:0/2319642784 >> v2:172.21.15.138:3300/0 conn(0x7fa3e8072700 0x7fa3e815d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.920+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2834777463 0 0) 0x7fa3cc003520 con 0x7fa3e8072320 2024-09-10T10:55:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.920+0000 7fa3de000640 1 -- 172.21.15.28:0/2319642784 >> v1:172.21.15.28:6789/0 conn(0x7fa3e8072320 legacy=0x7fa3e80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.920+0000 7fa3de000640 1 -- 172.21.15.28:0/2319642784 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e80cdd00 con 0x7fa3e8072de0 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3de000640 1 --2- 172.21.15.28:0/2319642784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e815d1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097a0 tx=0x7fa3d002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6202775b9a48ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0037070 con 0x7fa3e8072de0 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d002ab10 con 0x7fa3e8072de0 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3e6000640 1 -- 172.21.15.28:0/2319642784 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8001670 con 0x7fa3e8072de0 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3ed800640 1 -- 172.21.15.28:0/2319642784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 msgr2=0x7fa3e815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.921+0000 7fa3ed800640 1 --2- 172.21.15.28:0/2319642784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e815d1e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3d00097a0 tx=0x7fa3d002a080 comp rx=0 tx=0).stop 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 -- 172.21.15.28:0/2319642784 shutdown_connections 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 --2- 172.21.15.28:0/2319642784 >> v2:172.21.15.138:3300/0 conn(0x7fa3e8072700 0x7fa3e815d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 --2- 172.21.15.28:0/2319642784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e815d1e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 -- 172.21.15.28:0/2319642784 >> 172.21.15.28:0/2319642784 conn(0x7fa3e8067d70 msgr2=0x7fa3e806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:12.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 -- 172.21.15.28:0/2319642784 shutdown_connections 2024-09-10T10:55:12.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.922+0000 7fa3ed800640 1 -- 172.21.15.28:0/2319642784 wait complete. 2024-09-10T10:55:12.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.923+0000 7fa3ed800640 1 Processor -- start 2024-09-10T10:55:12.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.923+0000 7fa3ed800640 1 -- start start 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e8072320 0x7fa3e815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e815dde0 con 0x7fa3e8072700 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e815dc60 con 0x7fa3e8072320 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.924+0000 7fa3ed800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e815df60 con 0x7fa3e8072de0 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60350/0 (socket says 172.21.15.28:60350) 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 -- 172.21.15.28:0/1868408112 learned_addr learned my addr 172.21.15.28:0/1868408112 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:12.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3e6a00640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e8072320 0x7fa3e815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3e7400640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 -- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 msgr2=0x7fa3e80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e80cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 -- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e8072320 msgr2=0x7fa3e815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e8072320 0x7fa3e815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.925+0000 7fa3de000640 1 -- 172.21.15.28:0/1868408112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e80b4650 con 0x7fa3e8072700 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3de000640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0000c00 tx=0x7fa3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0037050 con 0x7fa3e8072700 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d0043070 con 0x7fa3e8072700 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e80b4930 con 0x7fa3e8072700 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d003ca70 con 0x7fa3e8072700 2024-09-10T10:55:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.926+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e80b4ef0 con 0x7fa3e8072700 2024-09-10T10:55:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.927+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e815e390 con 0x7fa3e8072700 2024-09-10T10:55:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.928+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3d003cc10 con 0x7fa3e8072700 2024-09-10T10:55:12.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.929+0000 7fa3df400640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 0x7fa3c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.929+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9486+0+0 (secure 0 0 0) 0x7fa3d00ac770 con 0x7fa3e8072700 2024-09-10T10:55:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.929+0000 7fa3e6a00640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 0x7fa3c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:12.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.930+0000 7fa3e6a00640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 0x7fa3c406e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005880 tx=0x7fa3d80057f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:12.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:12.933+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3d0079b60 con 0x7fa3e8072700 2024-09-10T10:55:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.153+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3e81451a0 con 0x7fa3e8072700 2024-09-10T10:55:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.154+0000 7fa3df400640 1 -- 172.21.15.28:0/1868408112 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 315 v315) ==== 95+0+5224 (secure 0 0 0) 0x7fa3d007de60 con 0x7fa3e8072700 2024-09-10T10:55:13.156 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:13.156 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":315,"btime":"2024-09-10T10:55:12:736494+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":315,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:12.736470+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10131},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10131":{"gid":10131,"name":"c","rank":0,"incarnation":315,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6834/635113338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":635113338},{"type":"v1","addr":"172.21.15.28:6834","nonce":635113338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 315 2024-09-10T10:55:13.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 msgr2=0x7fa3c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:13.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 0x7fa3c406e2f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8005880 tx=0x7fa3d80057f0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 msgr2=0x7fa3e80cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0000c00 tx=0x7fa3d00046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 shutdown_connections 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa3c406be30 0x7fa3c406e2f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3e8072de0 0x7fa3e80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e8072700 0x7fa3e80cad10 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 --2- 172.21.15.28:0/1868408112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3e8072320 0x7fa3e815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 >> 172.21.15.28:0/1868408112 conn(0x7fa3e8067d70 msgr2=0x7fa3e80c57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 shutdown_connections 2024-09-10T10:55:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:13.156+0000 7fa3ed800640 1 -- 172.21.15.28:0/1868408112 wait complete. 2024-09-10T10:55:13.174 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:55:15.755 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:55:15.754+0000 7f9191e00640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:15.756 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:15.756+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:55:15.756 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:15.756+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:55:16.327 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:55:16.326+0000 7f9196e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:18.175 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:18.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.344+0000 7fc71de00640 1 Processor -- start 2024-09-10T10:55:18.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.344+0000 7fc71de00640 1 -- start start 2024-09-10T10:55:18.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc71815e660 0x7fc71806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71de00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d1f90 0x7fc7180cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71de00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7180cf000 con 0x7fc7180d0dd0 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71de00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc7180b37d0 con 0x7fc71815e660 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71de00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc7180cf330 con 0x7fc7180d1f90 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc717e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc71815e660 0x7fc71806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71ca00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc7180d0dd0 0x7fc7180d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47798/0 (socket says 172.21.15.28:47798) 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.345+0000 7fc71ca00640 1 -- 172.21.15.28:0/1519276010 learned_addr learned my addr 172.21.15.28:0/1519276010 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.346+0000 7fc71d400640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d1f90 0x7fc7180cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.346+0000 7fc717e00640 1 -- 172.21.15.28:0/1519276010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d1f90 msgr2=0x7fc7180cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.346+0000 7fc717e00640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d1f90 0x7fc7180cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.346+0000 7fc717e00640 1 -- 172.21.15.28:0/1519276010 >> v1:172.21.15.28:6789/0 conn(0x7fc7180d0dd0 legacy=0x7fc7180d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.346+0000 7fc717e00640 1 -- 172.21.15.28:0/1519276010 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7180d29a0 con 0x7fc71815e660 2024-09-10T10:55:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.347+0000 7fc717e00640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc71806ccd0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc7040097a0 tx=0x7fc70402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf82fd2e03a3f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.347+0000 7fc717400640 1 -- 172.21.15.28:0/1519276010 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704037070 con 0x7fc71815e660 2024-09-10T10:55:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.347+0000 7fc717400640 1 -- 172.21.15.28:0/1519276010 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704032440 con 0x7fc71815e660 2024-09-10T10:55:18.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.347+0000 7fc717400640 1 -- 172.21.15.28:0/1519276010 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704032810 con 0x7fc71815e660 2024-09-10T10:55:18.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 -- 172.21.15.28:0/1519276010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 msgr2=0x7fc71806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc71806ccd0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc7040097a0 tx=0x7fc70402a180 comp rx=0 tx=0).stop 2024-09-10T10:55:18.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 -- 172.21.15.28:0/1519276010 shutdown_connections 2024-09-10T10:55:18.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d1f90 0x7fc7180cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 --2- 172.21.15.28:0/1519276010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc71806ccd0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 -- 172.21.15.28:0/1519276010 >> 172.21.15.28:0/1519276010 conn(0x7fc71808bfc0 msgr2=0x7fc7180d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 -- 172.21.15.28:0/1519276010 shutdown_connections 2024-09-10T10:55:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.348+0000 7fc71de00640 1 -- 172.21.15.28:0/1519276010 wait complete. 2024-09-10T10:55:18.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.350+0000 7fc71de00640 1 Processor -- start 2024-09-10T10:55:18.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.350+0000 7fc71de00640 1 -- start start 2024-09-10T10:55:18.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d0dd0 0x7fc7181e4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7180d1f90 0x7fc718153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180d3220 con 0x7fc71815e660 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7180d2f40 con 0x7fc7180d1f90 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71de00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc7180d33a0 con 0x7fc7180d0dd0 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35996/0 (socket says 172.21.15.28:35996) 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 -- 172.21.15.28:0/3709941180 learned_addr learned my addr 172.21.15.28:0/3709941180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71ca00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d0dd0 0x7fc7181e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc717e00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7180d1f90 0x7fc718153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 -- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d0dd0 msgr2=0x7fc7181e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d0dd0 0x7fc7181e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 -- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7180d1f90 msgr2=0x7fc718153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7180d1f90 0x7fc718153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.351+0000 7fc71d400640 1 -- 172.21.15.28:0/3709941180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7181e4ff0 con 0x7fc71815e660 2024-09-10T10:55:18.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc71d400640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc710007e40 tx=0x7fc71000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710014620 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc710019070 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7180b8840 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc710014a20 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.352+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7180b8e00 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.353+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7180d2440 con 0x7fc71815e660 2024-09-10T10:55:18.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.354+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc710014bc0 con 0x7fc71815e660 2024-09-10T10:55:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.355+0000 7fc714c00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 0x7fc6f806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.355+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7fc71008c770 con 0x7fc71815e660 2024-09-10T10:55:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.355+0000 7fc71ca00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 0x7fc6f806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.356+0000 7fc71ca00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 0x7fc6f806e2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f60 tx=0x7fc70c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.360+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc71005a0c0 con 0x7fc71815e660 2024-09-10T10:55:18.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.581+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7180d1b20 con 0x7fc71815e660 2024-09-10T10:55:18.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.582+0000 7fc714c00640 1 -- 172.21.15.28:0/3709941180 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+4389 (secure 0 0 0) 0x7fc7100599c0 con 0x7fc71815e660 2024-09-10T10:55:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:18.583 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":319,"btime":"2024-09-10T10:55:15:756591+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":319,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:15.756586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:18.583 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 319 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 msgr2=0x7fc6f806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 0x7fc6f806e2f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000f60 tx=0x7fc70c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 msgr2=0x7fc718155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc710007e40 tx=0x7fc71000e4b0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 shutdown_connections 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc6f806be30 0x7fc6f806e2f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc71815e660 0x7fc718155320 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7180d1f90 0x7fc718153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 --2- 172.21.15.28:0/3709941180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7180d0dd0 0x7fc7181e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 >> 172.21.15.28:0/3709941180 conn(0x7fc71808bfc0 msgr2=0x7fc718157610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:18.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 shutdown_connections 2024-09-10T10:55:18.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.584+0000 7fc71de00640 1 -- 172.21.15.28:0/3709941180 wait complete. 2024-09-10T10:55:18.601 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:55:18.601 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.756+0000 7f3554800640 1 Processor -- start 2024-09-10T10:55:18.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.756+0000 7f3554800640 1 -- start start 2024-09-10T10:55:18.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.757+0000 7f3554800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f354c072de0 0x7f354c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.757+0000 7f3554800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354c146140 con 0x7f354c072700 2024-09-10T10:55:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.757+0000 7f3554800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354c0cd740 con 0x7f354c072320 2024-09-10T10:55:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.757+0000 7f3554800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f354c0cda20 con 0x7f354c072de0 2024-09-10T10:55:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3552800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f354c072700 0x7f354c15d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47806/0 (socket says 172.21.15.28:47806) 2024-09-10T10:55:18.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3552800640 1 -- 172.21.15.28:0/507665059 learned_addr learned my addr 172.21.15.28:0/507665059 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3551400640 1 --2- 172.21.15.28:0/507665059 >> v2:172.21.15.138:3300/0 conn(0x7f354c072de0 0x7f354c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2654707942 0 0) 0x7f354c146140 con 0x7f354c072700 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3534003520 con 0x7f354c072700 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.758+0000 7f3551400640 1 -- 172.21.15.28:0/507665059 >> v1:172.21.15.138:6789/0 conn(0x7f354c072320 legacy=0x7f354c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.759+0000 7f3551400640 1 -- 172.21.15.28:0/507665059 >> v1:172.21.15.28:6789/0 conn(0x7f354c072700 legacy=0x7f354c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.759+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2933588722 0 0) 0x7f3534003520 con 0x7f354c072700 2024-09-10T10:55:18.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.759+0000 7f3551400640 1 -- 172.21.15.28:0/507665059 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c0cdd00 con 0x7f354c072de0 2024-09-10T10:55:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3551400640 1 --2- 172.21.15.28:0/507665059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072de0 0x7f354c15d1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f353c0097a0 tx=0x7f353c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a5eb34f3537bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c037070 con 0x7f354c072de0 2024-09-10T10:55:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c02abc0 con 0x7f354c072de0 2024-09-10T10:55:18.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3550a00640 1 -- 172.21.15.28:0/507665059 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480019f0 con 0x7f354c072de0 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3554800640 1 -- 172.21.15.28:0/507665059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072de0 msgr2=0x7f354c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.760+0000 7f3554800640 1 --2- 172.21.15.28:0/507665059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072de0 0x7f354c15d1e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f353c0097a0 tx=0x7f353c02a130 comp rx=0 tx=0).stop 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.761+0000 7f3554800640 1 -- 172.21.15.28:0/507665059 shutdown_connections 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.761+0000 7f3554800640 1 --2- 172.21.15.28:0/507665059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072de0 0x7f354c15d1e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.761+0000 7f3554800640 1 -- 172.21.15.28:0/507665059 >> 172.21.15.28:0/507665059 conn(0x7f354c067d70 msgr2=0x7f354c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.761+0000 7f3554800640 1 -- 172.21.15.28:0/507665059 shutdown_connections 2024-09-10T10:55:18.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.761+0000 7f3554800640 1 -- 172.21.15.28:0/507665059 wait complete. 2024-09-10T10:55:18.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.762+0000 7f3554800640 1 Processor -- start 2024-09-10T10:55:18.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.763+0000 7f3554800640 1 -- start start 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.763+0000 7f3554800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.763+0000 7f3554800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 0x7f354c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.763+0000 7f3554800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c072de0 0x7f354c142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.763+0000 7f3554800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f354c15dfb0 con 0x7f354c072320 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3554800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f354c15de30 con 0x7f354c072700 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3554800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f354c15e130 con 0x7f354c072de0 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36004/0 (socket says 172.21.15.28:36004) 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 -- 172.21.15.28:0/2065116721 learned_addr learned my addr 172.21.15.28:0/2065116721 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:18.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551400640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 0x7f354c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3552800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c072de0 0x7f354c142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 -- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c072de0 msgr2=0x7f354c142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c072de0 0x7f354c142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 -- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 msgr2=0x7f354c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 0x7f354c0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 -- 172.21.15.28:0/2065116721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354c1431f0 con 0x7f354c072320 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551400640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 0x7f354c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.764+0000 7f3551e00640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f354000c450 tx=0x7f354000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.765+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540016020 con 0x7f354c072320 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.765+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3540012830 con 0x7f354c072320 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.765+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540012be0 con 0x7f354c072320 2024-09-10T10:55:18.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.765+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354c0b3e70 con 0x7f354c072320 2024-09-10T10:55:18.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.765+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354c0b4430 con 0x7f354c072320 2024-09-10T10:55:18.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.766+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f354c0cf470 con 0x7f354c072320 2024-09-10T10:55:18.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.767+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3540019020 con 0x7f354c072320 2024-09-10T10:55:18.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.768+0000 7f3546000640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 0x7f353006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:18.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.768+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7f354008b570 con 0x7f354c072320 2024-09-10T10:55:18.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.768+0000 7f3551400640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 0x7f353006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:18.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.769+0000 7f3551400640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 0x7f353006e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f353c034000 tx=0x7f353c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:18.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.773+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f354c0cf470 con 0x7f354c072320 2024-09-10T10:55:19.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:18.999+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f354c0ce400 con 0x7f354c072320 2024-09-10T10:55:19.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.000+0000 7f3546000640 1 -- 172.21.15.28:0/2065116721 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+4389 (secure 0 0 0) 0x7f3540020020 con 0x7f354c072320 2024-09-10T10:55:19.001 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:19.001 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":319,"btime":"2024-09-10T10:55:15:756591+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":319,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:15.756586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:19.001 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 319 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 msgr2=0x7f353006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 0x7f353006e2f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f353c034000 tx=0x7f353c02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 msgr2=0x7f354c15bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f354000c450 tx=0x7f354000c910 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 shutdown_connections 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f353006be30 0x7f353006e2f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f354c072de0 0x7f354c142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f354c072700 0x7f354c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 --2- 172.21.15.28:0/2065116721 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f354c072320 0x7f354c15bbe0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.002+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 >> 172.21.15.28:0/2065116721 conn(0x7f354c067d70 msgr2=0x7f354c1441a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.003+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 shutdown_connections 2024-09-10T10:55:19.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.003+0000 7f3554800640 1 -- 172.21.15.28:0/2065116721 wait complete. 2024-09-10T10:55:19.019 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:55:19.019 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:19.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.173+0000 7f45b3e00640 1 Processor -- start 2024-09-10T10:55:19.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.174+0000 7f45b3e00640 1 -- start start 2024-09-10T10:55:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 0x7f45ac0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f45ac072de0 0x7f45ac06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072700 0x7f45ac06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f45ac146140 con 0x7f45ac072de0 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f45ac0cd740 con 0x7f45ac072320 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac0cda20 con 0x7f45ac072700 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f45ac072de0 0x7f45ac06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f45ac072de0 0x7f45ac06e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36014/0 (socket says 172.21.15.28:36014) 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b0a00640 1 -- 172.21.15.28:0/2302269019 learned_addr learned my addr 172.21.15.28:0/2302269019 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b1400640 1 --2- 172.21.15.28:0/2302269019 >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 0x7f45ac0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.175+0000 7f45b1e00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072700 0x7f45ac06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b0a00640 1 -- 172.21.15.28:0/2302269019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072700 msgr2=0x7f45ac06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b0a00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072700 0x7f45ac06e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b0a00640 1 -- 172.21.15.28:0/2302269019 >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 msgr2=0x7f45ac0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b0a00640 1 --2- 172.21.15.28:0/2302269019 >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 0x7f45ac0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b0a00640 1 -- 172.21.15.28:0/2302269019 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac0cdd00 con 0x7f45ac072de0 2024-09-10T10:55:19.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.176+0000 7f45b1400640 1 --2- 172.21.15.28:0/2302269019 >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 0x7f45ac0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b0a00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072de0 0x7f45ac06e0b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0b31e0 tx=0x7f459802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ca7269ed14b01e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45abe00640 1 -- 172.21.15.28:0/2302269019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598037070 con 0x7f45ac072de0 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45abe00640 1 -- 172.21.15.28:0/2302269019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598032440 con 0x7f45ac072de0 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45abe00640 1 -- 172.21.15.28:0/2302269019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598032810 con 0x7f45ac072de0 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b3e00640 1 -- 172.21.15.28:0/2302269019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072de0 msgr2=0x7f45ac06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b3e00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072de0 0x7f45ac06e0b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0b31e0 tx=0x7f459802a140 comp rx=0 tx=0).stop 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b3e00640 1 -- 172.21.15.28:0/2302269019 shutdown_connections 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b3e00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072700 0x7f45ac06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.177+0000 7f45b3e00640 1 --2- 172.21.15.28:0/2302269019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072de0 0x7f45ac06e0b0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.178+0000 7f45b3e00640 1 --2- 172.21.15.28:0/2302269019 >> v2:172.21.15.138:3300/0 conn(0x7f45ac072320 0x7f45ac0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.178+0000 7f45b3e00640 1 -- 172.21.15.28:0/2302269019 >> 172.21.15.28:0/2302269019 conn(0x7f45ac067d70 msgr2=0x7f45ac06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.178+0000 7f45b3e00640 1 -- 172.21.15.28:0/2302269019 shutdown_connections 2024-09-10T10:55:19.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.178+0000 7f45b3e00640 1 -- 172.21.15.28:0/2302269019 wait complete. 2024-09-10T10:55:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.179+0000 7f45b3e00640 1 Processor -- start 2024-09-10T10:55:19.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.179+0000 7f45b3e00640 1 -- start start 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072320 0x7f45ac1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac072de0 0x7f45ac142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac0cd520 con 0x7f45ac072700 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac0cd3a0 con 0x7f45ac072de0 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.180+0000 7f45b3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac06eb30 con 0x7f45ac072320 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36024/0 (socket says 172.21.15.28:36024) 2024-09-10T10:55:19.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 -- 172.21.15.28:0/794413108 learned_addr learned my addr 172.21.15.28:0/794413108 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b1400640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072320 0x7f45ac1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b1e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac072de0 0x7f45ac142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 -- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072320 msgr2=0x7f45ac1429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072320 0x7f45ac1429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 -- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac072de0 msgr2=0x7f45ac142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac072de0 0x7f45ac142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.181+0000 7f45b0a00640 1 -- 172.21.15.28:0/794413108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1427b0 con 0x7f45ac072700 2024-09-10T10:55:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45b0a00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f459802a180 tx=0x7f45980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598041020 con 0x7f45ac072700 2024-09-10T10:55:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45980306c0 con 0x7f45ac072700 2024-09-10T10:55:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598037040 con 0x7f45ac072700 2024-09-10T10:55:19.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1e3730 con 0x7f45ac072700 2024-09-10T10:55:19.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.182+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac1e3cf0 con 0x7f45ac072700 2024-09-10T10:55:19.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.183+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac0cce60 con 0x7f45ac072700 2024-09-10T10:55:19.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.184+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4598044020 con 0x7f45ac072700 2024-09-10T10:55:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.185+0000 7f45a9600640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.185+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7f45980ac480 con 0x7f45ac072700 2024-09-10T10:55:19.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.185+0000 7f45b1400640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.186+0000 7f45b1400640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 0x7f458c06e2f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45ac1ea430 tx=0x7f45a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.189+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f459804b030 con 0x7f45ac072700 2024-09-10T10:55:19.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.411+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45ac06d090 con 0x7f45ac072700 2024-09-10T10:55:19.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.412+0000 7f45a9600640 1 -- 172.21.15.28:0/794413108 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 319 v319) ==== 95+0+4389 (secure 0 0 0) 0x7f45980798a0 con 0x7f45ac072700 2024-09-10T10:55:19.413 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:19.413 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":319,"btime":"2024-09-10T10:55:15:756591+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":319,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:15.756586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":138,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:19.413 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 319 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 msgr2=0x7f458c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 0x7f458c06e2f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45ac1ea430 tx=0x7f45a0008040 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 msgr2=0x7f45ac1e47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f459802a180 tx=0x7f45980046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 shutdown_connections 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f458c06be30 0x7f458c06e2f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac072de0 0x7f45ac142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45ac072700 0x7f45ac1e47d0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 --2- 172.21.15.28:0/794413108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f45ac072320 0x7f45ac1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.414+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 >> 172.21.15.28:0/794413108 conn(0x7f45ac067d70 msgr2=0x7f45ac0cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.415+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 shutdown_connections 2024-09-10T10:55:19.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.415+0000 7f45b3e00640 1 -- 172.21.15.28:0/794413108 wait complete. 2024-09-10T10:55:19.431 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 100.00000000' led to DAMAGED state 2024-09-10T10:55:19.431 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on mds0_sessionmap' 2024-09-10T10:55:19.431 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:19.431 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:19.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:19.467 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:55:19.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:19.467 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:55:19.541 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:55:19.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:55:19.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:55:19.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.738+0000 7f2562200640 1 Processor -- start 2024-09-10T10:55:19.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.738+0000 7f2562200640 1 -- start start 2024-09-10T10:55:19.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.739+0000 7f2562200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f255c147230 0x7f255c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f2562200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255c06f480 con 0x7f255c147e90 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f2562200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f255c06e900 con 0x7f255c0727c0 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f2562200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f255c0b34d0 con 0x7f255c147230 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f255b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f255c147230 0x7f255c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f2553e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f255c147e90 0x7f255c0720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47816/0 (socket says 172.21.15.28:47816) 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f2553e00640 1 -- 172.21.15.28:0/2014948600 learned_addr learned my addr 172.21.15.28:0/2014948600 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f255b400640 1 -- 172.21.15.28:0/2014948600 >> v1:172.21.15.138:6789/0 conn(0x7f255c0727c0 legacy=0x7f255c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f255b400640 1 -- 172.21.15.28:0/2014948600 >> v1:172.21.15.28:6789/0 conn(0x7f255c147e90 legacy=0x7f255c0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.740+0000 7f255b400640 1 -- 172.21.15.28:0/2014948600 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c06ebe0 con 0x7f255c147230 2024-09-10T10:55:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.741+0000 7f255b400640 1 --2- 172.21.15.28:0/2014948600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c06e0b0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f254c0097a0 tx=0x7f254c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15d0bcfa1f3ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f255aa00640 1 -- 172.21.15.28:0/2014948600 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c037070 con 0x7f255c147230 2024-09-10T10:55:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f255aa00640 1 -- 172.21.15.28:0/2014948600 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f254c032440 con 0x7f255c147230 2024-09-10T10:55:19.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f255aa00640 1 -- 172.21.15.28:0/2014948600 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c032810 con 0x7f255c147230 2024-09-10T10:55:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 -- 172.21.15.28:0/2014948600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 msgr2=0x7f255c06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 --2- 172.21.15.28:0/2014948600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c06e0b0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f254c0097a0 tx=0x7f254c02a230 comp rx=0 tx=0).stop 2024-09-10T10:55:19.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 -- 172.21.15.28:0/2014948600 shutdown_connections 2024-09-10T10:55:19.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 --2- 172.21.15.28:0/2014948600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c06e0b0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 -- 172.21.15.28:0/2014948600 >> 172.21.15.28:0/2014948600 conn(0x7f255c067d70 msgr2=0x7f255c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:19.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 -- 172.21.15.28:0/2014948600 shutdown_connections 2024-09-10T10:55:19.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.742+0000 7f2562200640 1 -- 172.21.15.28:0/2014948600 wait complete. 2024-09-10T10:55:19.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.744+0000 7f2562200640 1 Processor -- start 2024-09-10T10:55:19.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.744+0000 7f2562200640 1 -- start start 2024-09-10T10:55:19.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f255c0727c0 0x7f255c1428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f255c147e90 0x7f255c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f255c06f1a0 con 0x7f255c147230 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f255c06e5f0 con 0x7f255c0727c0 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2562200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f255c147610 con 0x7f255c147e90 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2553e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2553e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36048/0 (socket says 172.21.15.28:36048) 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f2553e00640 1 -- 172.21.15.28:0/663536443 learned_addr learned my addr 172.21.15.28:0/663536443 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f255b400640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f255c0727c0 0x7f255c1428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.745+0000 7f255be00640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f255c147e90 0x7f255c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 -- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f255c147e90 msgr2=0x7f255c0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f255c147e90 0x7f255c0c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 -- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f255c0727c0 msgr2=0x7f255c1428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f255c0727c0 0x7f255c1428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 -- 172.21.15.28:0/663536443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c142e20 con 0x7f255c147230 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553e00640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f254400c480 tx=0x7f254400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544017070 con 0x7f255c147230 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2544005ce0 con 0x7f255c147230 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440072f0 con 0x7f255c147230 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255c0c82f0 con 0x7f255c147230 2024-09-10T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.746+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255c0c88b0 con 0x7f255c147230 2024-09-10T10:55:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.747+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f255c06e0b0 con 0x7f255c147230 2024-09-10T10:55:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.748+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f254401a070 con 0x7f255c147230 2024-09-10T10:55:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.749+0000 7f2553400640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 0x7f253806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.749+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7f254408b090 con 0x7f255c147230 2024-09-10T10:55:19.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.750+0000 7f255b400640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 0x7f253806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:19.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.750+0000 7f255b400640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 0x7f253806e2f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f254c034000 tx=0x7f254c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:19.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.753+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25440584b0 con 0x7f255c147230 2024-09-10T10:55:19.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:19.978+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f255c0b5400 con 0x7f255c147230 2024-09-10T10:55:20.509 INFO:tasks.ceph.mds.c.smithi028.stderr: -53> 2024-09-10T10:55:15.754+0000 7f9191e00640 -1 mds.0.cache.dir(0x100) Corrupt fnode in dirfrag 0x100: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:20.509 INFO:tasks.ceph.mds.c.smithi028.stderr: -33> 2024-09-10T10:55:16.326+0000 7f9196e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:20.552 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:55:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.774+0000 7f2553400640 1 -- 172.21.15.28:0/663536443 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v320) ==== 161+0+0 (secure 0 0 0) 0x7f254405c7b0 con 0x7f255c147230 2024-09-10T10:55:20.775 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 msgr2=0x7f253806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 0x7f253806e2f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f254c034000 tx=0x7f254c02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 msgr2=0x7f255c0c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f254400c480 tx=0x7f254400c940 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 shutdown_connections 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f253806be30 0x7f253806e2f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f255c147e90 0x7f255c0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.777+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f255c147230 0x7f255c0c66d0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.778+0000 7f2562200640 1 --2- 172.21.15.28:0/663536443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f255c0727c0 0x7f255c1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.778+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 >> 172.21.15.28:0/663536443 conn(0x7f255c067d70 msgr2=0x7f255c0af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.778+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 shutdown_connections 2024-09-10T10:55:20.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.778+0000 7f2562200640 1 -- 172.21.15.28:0/663536443 wait complete. 2024-09-10T10:55:20.798 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:55:20.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.960+0000 7fe00da00640 1 Processor -- start 2024-09-10T10:55:20.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.960+0000 7fe00da00640 1 -- start start 2024-09-10T10:55:20.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe00da00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe00815b300 0x7fe0080cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe00da00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 0x7fe00806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe00da00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0080cdbe0 con 0x7fe008072d30 2024-09-10T10:55:20.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe00da00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe0080728a0 con 0x7fe00815b300 2024-09-10T10:55:20.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe00da00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe0080cd900 con 0x7fe0080cd3f0 2024-09-10T10:55:20.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.961+0000 7fe006a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe00815b300 0x7fe0080cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe007400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe008072d30 0x7fe0080b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47818/0 (socket says 172.21.15.28:47818) 2024-09-10T10:55:20.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe007400640 1 -- 172.21.15.28:0/3820674764 learned_addr learned my addr 172.21.15.28:0/3820674764 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006000640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 0x7fe00806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006a00640 1 -- 172.21.15.28:0/3820674764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 msgr2=0x7fe00806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006a00640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 0x7fe00806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006a00640 1 -- 172.21.15.28:0/3820674764 >> v1:172.21.15.28:6789/0 conn(0x7fe008072d30 legacy=0x7fe0080b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006a00640 1 -- 172.21.15.28:0/3820674764 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe008072250 con 0x7fe00815b300 2024-09-10T10:55:20.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe005600640 1 -- 172.21.15.28:0/3820674764 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049835177 0 0) 0x7fe0080cdbe0 con 0x7fe008072d30 2024-09-10T10:55:20.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.962+0000 7fe006000640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 0x7fe00806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:20.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.963+0000 7fe006a00640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00815b300 0x7fe0080cceb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fdff80097a0 tx=0x7fdff802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1662ad68f76e8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:20.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.963+0000 7fe005600640 1 -- 172.21.15.28:0/3820674764 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8037070 con 0x7fe00815b300 2024-09-10T10:55:20.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.963+0000 7fe005600640 1 -- 172.21.15.28:0/3820674764 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff8032440 con 0x7fe00815b300 2024-09-10T10:55:20.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.963+0000 7fe005600640 1 -- 172.21.15.28:0/3820674764 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8032810 con 0x7fe00815b300 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 -- 172.21.15.28:0/3820674764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00815b300 msgr2=0x7fe0080cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00815b300 0x7fe0080cceb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fdff80097a0 tx=0x7fdff802a180 comp rx=0 tx=0).stop 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 -- 172.21.15.28:0/3820674764 shutdown_connections 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe0080cd3f0 0x7fe00806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 --2- 172.21.15.28:0/3820674764 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00815b300 0x7fe0080cceb0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 -- 172.21.15.28:0/3820674764 >> 172.21.15.28:0/3820674764 conn(0x7fe008067d70 msgr2=0x7fe00806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 -- 172.21.15.28:0/3820674764 shutdown_connections 2024-09-10T10:55:20.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.964+0000 7fe00da00640 1 -- 172.21.15.28:0/3820674764 wait complete. 2024-09-10T10:55:20.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.966+0000 7fe00da00640 1 Processor -- start 2024-09-10T10:55:20.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.966+0000 7fe00da00640 1 -- start start 2024-09-10T10:55:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 0x7fe008153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00815b300 0x7fe008154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe0080d0160 con 0x7fe0080cd3f0 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0080cffe0 con 0x7fe008072d30 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe00da00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe00806f700 con 0x7fe00815b300 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36082/0 (socket says 172.21.15.28:36082) 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 -- 172.21.15.28:0/4219735335 learned_addr learned my addr 172.21.15.28:0/4219735335 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe007400640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00815b300 0x7fe008154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006a00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 0x7fe008153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 -- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00815b300 msgr2=0x7fe008154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00815b300 0x7fe008154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 -- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 msgr2=0x7fe008153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 0x7fe008153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:20.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006000640 1 -- 172.21.15.28:0/4219735335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00814e4b0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.967+0000 7fe006a00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 0x7fe008153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fe006000640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c7f0 tx=0x7fdff400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4004230 con 0x7fe0080cd3f0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdff40043d0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4007a70 con 0x7fe0080cd3f0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081538f0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.968+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe008153eb0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.969+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0080c61a0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.970+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdff4007c10 con 0x7fe0080cd3f0 2024-09-10T10:55:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.971+0000 7fdffea00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.971+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7fdff408bea0 con 0x7fe0080cd3f0 2024-09-10T10:55:20.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.971+0000 7fe006a00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:20.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.972+0000 7fe006a00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 0x7fdfe806e2f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdff8034000 tx=0x7fdff802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:20.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:20.975+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe00814e4b0 con 0x7fe0080cd3f0 2024-09-10T10:55:21.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.231+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fe0080b5110 con 0x7fe0080cd3f0 2024-09-10T10:55:21.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.779+0000 7fdffea00640 1 -- 172.21.15.28:0/4219735335 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v321) ==== 101+0+0 (secure 0 0 0) 0x7fdff4015020 con 0x7fe0080cd3f0 2024-09-10T10:55:21.781 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:55:21.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 msgr2=0x7fdfe806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 0x7fdfe806e2f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fdff8034000 tx=0x7fdff802e000 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 msgr2=0x7fe00814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fdff400c7f0 tx=0x7fdff400ccb0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 shutdown_connections 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fdfe806be30 0x7fdfe806e2f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe00815b300 0x7fe008154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0080cd3f0 0x7fe00814df70 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 --2- 172.21.15.28:0/4219735335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe008072d30 0x7fe008153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.782+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 >> 172.21.15.28:0/4219735335 conn(0x7fe008067d70 msgr2=0x7fe008159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.783+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 shutdown_connections 2024-09-10T10:55:21.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.783+0000 7fe00da00640 1 -- 172.21.15.28:0/4219735335 wait complete. 2024-09-10T10:55:21.801 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:55:21.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.879+0000 7fde0e121f40 1 Processor -- start 2024-09-10T10:55:21.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.879+0000 7fde0e121f40 1 -- start start 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae1b410 0x56145aeaf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 0x56145ae1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56145acecb30 con 0x7fddf0002460 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56145acf0b70 con 0x56145ae190c0 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0e121f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56145ae1cee0 con 0x56145ae1b410 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36094/0 (socket says 172.21.15.28:36094) 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde0a800640 1 -- 172.21.15.28:0/2091009232 learned_addr learned my addr 172.21.15.28:0/2091009232 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.880+0000 7fde09e00640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae1b410 0x56145aeaf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0b200640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 0x56145ae1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 -- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae1b410 msgr2=0x56145aeaf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae1b410 0x56145aeaf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 -- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 msgr2=0x56145ae1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 0x56145ae1ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 -- 172.21.15.28:0/2091009232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145ad91e60 con 0x7fddf0002460 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0b200640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 0x56145ae1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde0a800640 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fddfc003040 tx=0x7fddfc051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d0e19747d3ce6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde08a00640 1 -- 172.21.15.28:0/2091009232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc052830 con 0x7fddf0002460 2024-09-10T10:55:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde08a00640 1 -- 172.21.15.28:0/2091009232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddfc062070 con 0x7fddf0002460 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.881+0000 7fde08a00640 1 -- 172.21.15.28:0/2091009232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc052bd0 con 0x7fddf0002460 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 -- 172.21.15.28:0/2091009232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 msgr2=0x56145aeaeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fddfc003040 tx=0x7fddfc051000 comp rx=0 tx=0).stop 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 -- 172.21.15.28:0/2091009232 shutdown_connections 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145ae190c0 0x56145ae1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae1b410 0x56145aeaf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 --2- 172.21.15.28:0/2091009232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fddf0002460 0x56145aeaeb20 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 -- 172.21.15.28:0/2091009232 >> 172.21.15.28:0/2091009232 conn(0x56145ad930a0 msgr2=0x56145ae1a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 -- 172.21.15.28:0/2091009232 shutdown_connections 2024-09-10T10:55:21.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.882+0000 7fde0e121f40 1 -- 172.21.15.28:0/2091009232 wait complete. 2024-09-10T10:55:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.883+0000 7fde0e121f40 1 Processor -- start 2024-09-10T10:55:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.883+0000 7fde0e121f40 1 -- start start 2024-09-10T10:55:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.883+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae190c0 0x56145ae14420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.883+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0e121f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0002460 0x56145ad76e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0e121f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x56145ae1ba40 con 0x56145ae1b410 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0e121f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56145ad91b80 con 0x7fddf0002460 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0e121f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56145ae1bd20 con 0x56145ae190c0 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36104/0 (socket says 172.21.15.28:36104) 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 learned_addr learned my addr 172.21.15.28:0/1024865758 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae190c0 0x56145ae14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0002460 0x56145ad76e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae190c0 msgr2=0x56145ae14420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56145ae190c0 0x56145ae14420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0002460 msgr2=0x56145ad76e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0002460 0x56145ad76e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145ae12930 con 0x56145ae1b410 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.884+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf0002460 0x56145ad76e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fddf800d5d0 tx=0x7fddf800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde03400640 1 -- 172.21.15.28:0/1024865758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf800b7c0 con 0x56145ae1b410 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde03400640 1 -- 172.21.15.28:0/1024865758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf800b960 con 0x56145ae1b410 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56145ae106b0 con 0x56145ae1b410 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde03400640 1 -- 172.21.15.28:0/1024865758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf800bd50 con 0x56145ae1b410 2024-09-10T10:55:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.885+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145ae13450 con 0x56145ae1b410 2024-09-10T10:55:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.887+0000 7fde03400640 1 -- 172.21.15.28:0/1024865758 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fddf8009050 con 0x56145ae1b410 2024-09-10T10:55:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde03400640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 0x7fddec06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde03400640 1 -- 172.21.15.28:0/1024865758 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7fddf808b160 con 0x56145ae1b410 2024-09-10T10:55:21.890 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 0x56145ae1edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145ad773d0 con 0x56145ae12450 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 0x7fddec06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.888+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 0x56145ae1edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.889+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 0x7fddec06ccd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fddfc066000 tx=0x7fddfc053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.889+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 0x56145ae1edf0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.892+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v138'1159 uv1133 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.892+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145ae1f330 con 0x56145ae12450 2024-09-10T10:55:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.893+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v138'1160 uv1160 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.893+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145ae1f790 con 0x56145ae12450 2024-09-10T10:55:21.896 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:55:21.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.895+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v138'1161 uv1161 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.895+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0d70 con 0x56145ae12450 2024-09-10T10:55:21.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.897+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v138'1162 uv1162 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.897+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec1110 con 0x56145ae12450 2024-09-10T10:55:21.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.899+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v138'1163 uv1163 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.899+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aebfd60 con 0x56145ae12450 2024-09-10T10:55:21.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.900+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v138'1164 uv1164 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.901+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0100 con 0x56145ae12450 2024-09-10T10:55:21.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.903+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v138'1165 uv1139 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.903+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.905+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v138'1166 uv1166 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.905+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.907 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:55:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.906+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v138'1167 uv1167 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.907+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.909+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v138'1168 uv1168 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.909+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.911+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v138'1169 uv1169 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.911+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.913+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v138'1170 uv1170 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.913+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.915+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v138'1171 uv1146 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.915+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.917+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v138'1172 uv1172 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.917+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.919 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:55:21.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.918+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v138'1173 uv1173 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.918+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.920+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v138'1174 uv1174 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.920+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.922+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v138'1175 uv1175 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.922+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec04a0 con 0x56145ae12450 2024-09-10T10:55:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.924+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v138'1176 uv1176 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.924+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec08d0 con 0x56145ae12450 2024-09-10T10:55:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.925+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v138'1177 uv1177 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.925+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.927+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v138'1178 uv1152 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.927+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.929+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v138'1179 uv1179 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.929+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.932 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:55:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.931+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v138'1180 uv1180 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.931+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.933+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v138'1181 uv1181 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.933+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.935+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v138'1182 uv1182 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.935+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.937+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v138'1183 uv1183 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.937+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.938+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v138'1184 uv1158 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.938+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.940+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v138'1185 uv1185 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.940+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.942 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:55:21.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.942+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v138'1186 uv1186 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.942+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0880 con 0x56145ae12450 2024-09-10T10:55:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.943+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v138'1187 uv1187 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.943+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0760 con 0x56145ae12450 2024-09-10T10:55:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.945+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v138'1188 uv1188 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.945+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec0760 con 0x56145ae12450 2024-09-10T10:55:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.947+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v138'1189 uv1189 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145ae12450 2024-09-10T10:55:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.947+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 0x56145aec3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.947+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec42f0 con 0x56145aec19c0 2024-09-10T10:55:21.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.947+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 0x56145aec3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.948+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 0x56145aec3db0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.950+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v138'846 uv828 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.950+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec4690 con 0x56145aec19c0 2024-09-10T10:55:21.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.951+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v138'847 uv847 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.951+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec4a30 con 0x56145aec19c0 2024-09-10T10:55:21.954 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:55:21.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.953+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v138'848 uv848 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.953+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec4dd0 con 0x56145aec19c0 2024-09-10T10:55:21.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.954+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v138'849 uv849 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.954+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec5170 con 0x56145aec19c0 2024-09-10T10:55:21.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.956+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v138'850 uv850 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.956+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec5510 con 0x56145aec19c0 2024-09-10T10:55:21.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.958+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v138'851 uv833 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.958+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.959+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v138'852 uv852 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.959+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.962 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:55:21.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.961+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v138'853 uv853 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.961+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.963+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v138'854 uv854 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.963+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.964+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v138'855 uv855 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.964+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.966+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v138'856 uv838 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.966+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.967+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v138'857 uv857 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.967+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.970 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:55:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.969+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v138'858 uv858 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.969+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec9fa0 con 0x56145aec19c0 2024-09-10T10:55:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.971+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v138'859 uv859 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.971+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec9fa0 con 0x56145aec19c0 2024-09-10T10:55:21.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.972+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v138'860 uv860 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.972+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.974+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v138'861 uv844 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.974+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.975+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v138'862 uv862 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.975+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.977 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:55:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.976+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v138'863 uv863 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.977+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec68c0 con 0x56145aec19c0 2024-09-10T10:55:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.978+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v138'864 uv864 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.978+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec68c0 con 0x56145aec19c0 2024-09-10T10:55:21.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.979+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v138'865 uv865 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.979+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec58b0 con 0x56145aec19c0 2024-09-10T10:55:21.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.981+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v138'866 uv866 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aec19c0 2024-09-10T10:55:21.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.981+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56145aec61b0 0x56145aec85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.981+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec8ae0 con 0x56145aec61b0 2024-09-10T10:55:21.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.981+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56145aec61b0 0x56145aec85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.981+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56145aec61b0 0x56145aec85a0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.983+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v138'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.983+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec8e80 con 0x56145aec61b0 2024-09-10T10:55:21.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.984+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v138'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.984+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec9330 con 0x56145aec61b0 2024-09-10T10:55:21.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.985+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v138'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.986 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:55:21.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.985+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecabe0 con 0x56145aec61b0 2024-09-10T10:55:21.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.986+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v138'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.987+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecaf80 con 0x56145aec61b0 2024-09-10T10:55:21.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.988+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v138'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.988+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec9bd0 con 0x56145aec61b0 2024-09-10T10:55:21.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.989+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v138'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.989+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x56145aeca4d0 0x56145aecd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:21.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.989+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecd870 con 0x56145aeca4d0 2024-09-10T10:55:21.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.989+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x56145aeca4d0 0x56145aecd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:21.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.990+0000 7fde0b200640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x56145aeca4d0 0x56145aecd330 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:21.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.991+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v138'168 uv167 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:21.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.991+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aecd870 con 0x56145aeca4d0 2024-09-10T10:55:21.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.993+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v138'169 uv169 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:21.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.993+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:21.995 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:55:21.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.994+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v138'170 uv170 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:21.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.994+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:21.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.995+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v138'171 uv171 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:21.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.995+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:21.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.996+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v138'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.996+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:21.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.998+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v138'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:21.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.998+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:21.999 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:55:22.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.999+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v138'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:21.999+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecec70 con 0x56145aec61b0 2024-09-10T10:55:22.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.000+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v138'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.000+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecec70 con 0x56145aec61b0 2024-09-10T10:55:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.001+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v138'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.001+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.002+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v138'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.002+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.004+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v138'415 uv409 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.004+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.006+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v138'416 uv416 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.006+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.008 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:55:22.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.008+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v138'417 uv417 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.008+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecec70 con 0x56145aeca4d0 2024-09-10T10:55:22.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.009+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v138'418 uv418 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.009+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecec70 con 0x56145aeca4d0 2024-09-10T10:55:22.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.011+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v138'419 uv419 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.011+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.012+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v138'420 uv420 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.012+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.014+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v138'421 uv414 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.014+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.015+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v138'422 uv422 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.015+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.017 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:55:22.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.017+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v138'423 uv423 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.017+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.018+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v138'424 uv424 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.018+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aeca4d0 2024-09-10T10:55:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.020+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v138'425 uv425 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.020+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.021+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v138'188 uv187 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.021+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.022+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v138'189 uv189 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.022+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.024 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:55:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.023+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v138'190 uv190 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.024+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.025+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v138'191 uv191 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.025+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aecdc60 con 0x56145aec61b0 2024-09-10T10:55:22.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.026+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v138'192 uv192 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aec61b0 2024-09-10T10:55:22.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.026+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56145aece2e0 0x56145aed06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.026+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed0ce0 con 0x56145aece2e0 2024-09-10T10:55:22.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.026+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56145aece2e0 0x56145aed06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.027+0000 7fde0a800640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56145aece2e0 0x56145aed06d0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.029+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v138'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.029+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed0da0 con 0x56145aece2e0 2024-09-10T10:55:22.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.030+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v138'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.030+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed11c0 con 0x56145aece2e0 2024-09-10T10:55:22.032 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:55:22.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.031+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v138'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.031+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed21d0 con 0x56145aece2e0 2024-09-10T10:55:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.033+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v138'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.033+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed21d0 con 0x56145aece2e0 2024-09-10T10:55:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.034+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v138'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.034+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed11c0 con 0x56145aece2e0 2024-09-10T10:55:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.035+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v138'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.035+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed11c0 con 0x56145aece2e0 2024-09-10T10:55:22.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.037+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v138'376 uv369 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.037+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed11c0 con 0x56145aece2e0 2024-09-10T10:55:22.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.038+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v138'377 uv377 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.038+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed11c0 con 0x56145aece2e0 2024-09-10T10:55:22.040 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:55:22.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.040+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v138'378 uv378 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.040+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.041+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v138'379 uv379 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.041+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.042+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v138'380 uv380 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.042+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.044+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v138'381 uv375 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.044+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.045+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v138'382 uv382 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.045+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:55:22.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.046+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v138'383 uv383 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.046+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aec97c0 con 0x56145aece2e0 2024-09-10T10:55:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.048+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v138'384 uv384 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.048+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed3fd0 con 0x56145aece2e0 2024-09-10T10:55:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.049+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v138'385 uv385 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.049+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.050+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v138'167 uv165 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.050+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.052+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v138'168 uv168 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.052+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.053+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v138'169 uv169 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.054 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:55:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.053+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.054+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v138'170 uv170 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.054+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.055+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v138'225 uv224 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.055+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.057+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v138'226 uv226 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.057+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.058+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v138'227 uv227 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.059 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:55:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.058+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2680 con 0x56145aeca4d0 2024-09-10T10:55:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.059+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v138'228 uv228 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.059+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2680 con 0x56145aeca4d0 2024-09-10T10:55:22.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.060+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v138'229 uv229 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.060+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed1670 con 0x56145aeca4d0 2024-09-10T10:55:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.061+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v138'230 uv230 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fde0400a9b0 con 0x56145aeca4d0 2024-09-10T10:55:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.061+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x56145aed1cf0 0x56145aed20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.061+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2630 con 0x56145aed1cf0 2024-09-10T10:55:22.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.062+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x56145aed1cf0 0x56145aed20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.062+0000 7fde09e00640 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x56145aed1cf0 0x56145aed20f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.063+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v138'191 uv190 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aed1cf0 2024-09-10T10:55:22.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.064+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2630 con 0x56145aed1cf0 2024-09-10T10:55:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.065+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v138'192 uv192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aed1cf0 2024-09-10T10:55:22.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.065+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2630 con 0x56145aed1cf0 2024-09-10T10:55:22.067 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:55:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.066+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v138'193 uv193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aed1cf0 2024-09-10T10:55:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.066+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aed1cf0 2024-09-10T10:55:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.067+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v138'194 uv194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aed1cf0 2024-09-10T10:55:22.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.067+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aed1cf0 2024-09-10T10:55:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.068+0000 7fde09e00640 1 -- 172.21.15.28:0/1024865758 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v138'195 uv195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7fddf801e020 con 0x56145aed1cf0 2024-09-10T10:55:22.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.068+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.070+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v138'468 uv459 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.070+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.071+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v138'469 uv469 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.071+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.073 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:55:22.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.072+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v138'470 uv470 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.072+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.073+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v138'471 uv471 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.073+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.075+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v138'472 uv472 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.075+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.076+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v138'473 uv466 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.076+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.077+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v138'474 uv474 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.077+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.079 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:55:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.078+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v138'475 uv475 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.078+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.079+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v138'476 uv476 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.079+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.081+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v138'477 uv477 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.081+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed2a60 con 0x56145aece2e0 2024-09-10T10:55:22.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.082+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v138'478 uv478 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.082+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x56145aed3320 con 0x56145aece2e0 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0a800640 1 -- 172.21.15.28:0/1024865758 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v138'479 uv479 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7fddfc052830 con 0x56145aece2e0 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56145aec61b0 msgr2=0x56145aec85a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x56145aec61b0 0x56145aec85a0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56145aece2e0 msgr2=0x56145aed06d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x56145aece2e0 0x56145aed06d0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x56145aed1cf0 msgr2=0x56145aed20f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x56145aed1cf0 0x56145aed20f0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x56145aeca4d0 msgr2=0x56145aecd330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x56145aeca4d0 0x56145aecd330 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 msgr2=0x56145aec3db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 0x56145aec3db0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 reap_dead start 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0b200640 1 -- 172.21.15.28:0/1024865758 reap_dead start 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 msgr2=0x56145ae1edf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 0x56145ae1edf0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 msgr2=0x7fddec06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 0x7fddec06ccd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fddfc066000 tx=0x7fddfc053000 comp rx=0 tx=0).stop 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 msgr2=0x56145ae12f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fddf800d5d0 tx=0x7fddf800da90 comp rx=0 tx=0).stop 2024-09-10T10:55:22.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 shutdown_connections 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x56145aec19c0 0x56145aec3db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x56145ae12450 0x56145ae1edf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.083+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fddec06a810 0x7fddec06ccd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.084+0000 7fde0e121f40 1 --2- 172.21.15.28:0/1024865758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56145ae1b410 0x56145ae12f10 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.084+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 >> 172.21.15.28:0/1024865758 conn(0x56145ad930a0 msgr2=0x56145ae1ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.084+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 shutdown_connections 2024-09-10T10:55:22.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.084+0000 7fde0e121f40 1 -- 172.21.15.28:0/1024865758 wait complete. 2024-09-10T10:55:22.090 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader mds0_sessionmap deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:55:22.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 Processor -- start 2024-09-10T10:55:22.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 -- start start 2024-09-10T10:55:22.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 0x56397864da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639786573e0 0x5639785b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x563978529b30 con 0x56397865a9c0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56397852db70 con 0x5639786573e0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.206+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x563978659ee0 con 0x563978581470 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36116/0 (socket says 172.21.15.28:36116) 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 -- 172.21.15.28:0/855571186 learned_addr learned my addr 172.21.15.28:0/855571186 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fbfc00640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639786573e0 0x5639785b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc0600640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 0x56397864da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 -- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 msgr2=0x56397864da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 0x56397864da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 -- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639786573e0 msgr2=0x5639785b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639786573e0 0x5639785b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 -- 172.21.15.28:0/855571186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639785b16e0 con 0x56397865a9c0 2024-09-10T10:55:22.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc0600640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 0x56397864da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:22.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.207+0000 7f0fc1000640 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8025b90 tx=0x7f0fb804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5693534a6a5f589 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fbe800640 1 -- 172.21.15.28:0/855571186 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb805a050 con 0x56397865a9c0 2024-09-10T10:55:22.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fbe800640 1 -- 172.21.15.28:0/855571186 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb805b040 con 0x56397865a9c0 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fbe800640 1 -- 172.21.15.28:0/855571186 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8055a10 con 0x56397865a9c0 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 -- 172.21.15.28:0/855571186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 msgr2=0x5639785b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8025b90 tx=0x7f0fb804f000 comp rx=0 tx=0).stop 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 -- 172.21.15.28:0/855571186 shutdown_connections 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56397865a9c0 0x5639785b0990 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5639786573e0 0x5639785b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/855571186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x563978581470 0x56397864da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 -- 172.21.15.28:0/855571186 >> 172.21.15.28:0/855571186 conn(0x5639785d00a0 msgr2=0x56397856cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 -- 172.21.15.28:0/855571186 shutdown_connections 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.208+0000 7f0fc33caf40 1 -- 172.21.15.28:0/855571186 wait complete. 2024-09-10T10:55:22.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.209+0000 7f0fc33caf40 1 Processor -- start 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.209+0000 7f0fc33caf40 1 -- start start 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563978581470 0x563978649f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 0x5639785cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56397865a9c0 0x56397865da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5639785b06b0 con 0x5639786573e0 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5639785cc0d0 con 0x563978581470 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc33caf40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56397864dfc0 con 0x56397865a9c0 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fbfc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 0x5639785cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fbfc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 0x5639785cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36132/0 (socket says 172.21.15.28:36132) 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fbfc00640 1 -- 172.21.15.28:0/267758510 learned_addr learned my addr 172.21.15.28:0/267758510 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc0600640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563978581470 0x563978649f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.210+0000 7f0fc1000640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56397865a9c0 0x56397865da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 -- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56397865a9c0 msgr2=0x56397865da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56397865a9c0 0x56397865da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 -- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563978581470 msgr2=0x563978649f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563978581470 0x563978649f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 -- 172.21.15.28:0/267758510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639785cea40 con 0x5639786573e0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbfc00640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 0x5639785cdbb0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00dfd0 tx=0x7f0fac00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbd400640 1 -- 172.21.15.28:0/267758510 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac014020 con 0x5639786573e0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbd400640 1 -- 172.21.15.28:0/267758510 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fac010620 con 0x5639786573e0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fbd400640 1 -- 172.21.15.28:0/267758510 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fac008030 con 0x5639786573e0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.211+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639785b0990 con 0x5639786573e0 2024-09-10T10:55:22.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.212+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639785afe70 con 0x5639786573e0 2024-09-10T10:55:22.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.213+0000 7f0fbd400640 1 -- 172.21.15.28:0/267758510 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0fac017020 con 0x5639786573e0 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.214+0000 7f0fbd400640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fa006a810 0x7f0fa006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.214+0000 7f0fbd400640 1 -- 172.21.15.28:0/267758510 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7f0fac08a790 con 0x5639786573e0 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563978658c90 0x5639786f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e138) -- 0x5639785ce0f0 con 0x563978658c90 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc0600640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fa006a810 0x7f0fa006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc1000640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563978658c90 0x5639786f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc0600640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fa006a810 0x7f0fa006ccd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0001880 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.215+0000 7f0fc1000640 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563978658c90 0x5639786f3560 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc1000640 1 -- 172.21.15.28:0/267758510 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(1 mds0_sessionmap [omap-set-header] v138'171 uv171 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0fb805d070 con 0x563978658c90 2024-09-10T10:55:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563978658c90 msgr2=0x5639786f3560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x563978658c90 0x5639786f3560 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fa006a810 msgr2=0x7f0fa006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0fa006a810 0x7f0fa006ccd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0000c00 tx=0x7f0fb0001880 comp rx=0 tx=0).stop 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 msgr2=0x5639785cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 --2- 172.21.15.28:0/267758510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5639786573e0 0x5639785cdbb0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f0fac00dfd0 tx=0x7f0fac00b6a0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc1000640 1 -- 172.21.15.28:0/267758510 reap_dead start 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 shutdown_connections 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 >> 172.21.15.28:0/267758510 conn(0x5639785d00a0 msgr2=0x563978659300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 shutdown_connections 2024-09-10T10:55:22.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.217+0000 7f0fc33caf40 1 -- 172.21.15.28:0/267758510 wait complete. 2024-09-10T10:55:22.222 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:55:22.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.381+0000 7fe8e8a00640 1 Processor -- start 2024-09-10T10:55:22.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.381+0000 7fe8e8a00640 1 -- start start 2024-09-10T10:55:22.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e406e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe8e40d2ac0 0x7fe8e40b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e8a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e40b1170 con 0x7fe8e40b3640 2024-09-10T10:55:22.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e8a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40ceba0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e406dfb0 con 0x7fe8e4161ac0 2024-09-10T10:55:22.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe8e40b3640 0x7fe8e40d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47850/0 (socket says 172.21.15.28:47850) 2024-09-10T10:55:22.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e2a00640 1 -- 172.21.15.28:0/1905384806 learned_addr learned my addr 172.21.15.28:0/1905384806 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:22.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.382+0000 7fe8e3e00640 1 --2- 172.21.15.28:0/1905384806 >> v2:172.21.15.28:3300/0 conn(0x7fe8e40d2ac0 0x7fe8e40b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3400640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e406e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3e00640 1 -- 172.21.15.28:0/1905384806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 msgr2=0x7fe8e406e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3e00640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e406e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3e00640 1 -- 172.21.15.28:0/1905384806 >> v1:172.21.15.28:6789/0 conn(0x7fe8e40b3640 legacy=0x7fe8e40d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3e00640 1 -- 172.21.15.28:0/1905384806 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40b1450 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e2000640 1 -- 172.21.15.28:0/1905384806 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217271378 0 0) 0x7fe8e40b1170 con 0x7fe8e40b3640 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.383+0000 7fe8e3400640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.384+0000 7fe8e3e00640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e40b1f30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009dc0 tx=0x7fe8dc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e305269745fae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.384+0000 7fe8e2000640 1 -- 172.21.15.28:0/1905384806 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007740 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.384+0000 7fe8e2000640 1 -- 172.21.15.28:0/1905384806 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8dc0078e0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.384+0000 7fe8e2000640 1 -- 172.21.15.28:0/1905384806 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc007cb0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 -- 172.21.15.28:0/1905384806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 msgr2=0x7fe8e40b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e40b1f30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009dc0 tx=0x7fe8dc02a710 comp rx=0 tx=0).stop 2024-09-10T10:55:22.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 -- 172.21.15.28:0/1905384806 shutdown_connections 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e40b1f30 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/1905384806 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e406e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 -- 172.21.15.28:0/1905384806 >> 172.21.15.28:0/1905384806 conn(0x7fe8e408bfc0 msgr2=0x7fe8e40d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 -- 172.21.15.28:0/1905384806 shutdown_connections 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.385+0000 7fe8e8a00640 1 -- 172.21.15.28:0/1905384806 wait complete. 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.386+0000 7fe8e8a00640 1 Processor -- start 2024-09-10T10:55:22.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 -- start start 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e40b3640 0x7fe8e40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e4161ea0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40b1a10 con 0x7fe8e40b3640 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.387+0000 7fe8e8a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e4162020 con 0x7fe8e4161ac0 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36146/0 (socket says 172.21.15.28:36146) 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 -- 172.21.15.28:0/3574865296 learned_addr learned my addr 172.21.15.28:0/3574865296 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e3400640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e40b3640 0x7fe8e40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e3e00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 -- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 msgr2=0x7fe8e415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 -- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e40b3640 msgr2=0x7fe8e40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e40b3640 0x7fe8e40cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 -- 172.21.15.28:0/3574865296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40cd890 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.388+0000 7fe8e2a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c4b0 tx=0x7fe8d000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.389+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0018070 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.389+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0007500 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.389+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004a20 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.389+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e40cdb60 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.389+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e40b49b0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.390+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e406e6c0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.391+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe8d0004bc0 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.391+0000 7fe8db400640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.392+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9572+0+0 (secure 0 0 0) 0x7fe8d008af90 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.392+0000 7fe8e3400640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.393+0000 7fe8e3400640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 0x7fe8c406e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40cb250 tx=0x7fe8d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.396+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8d0058470 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.619+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe8e40b2c40 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.784+0000 7fe8db400640 1 -- 172.21.15.28:0/3574865296 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v322) ==== 162+0+0 (secure 0 0 0) 0x7fe8d005c770 con 0x7fe8e40d2ac0 2024-09-10T10:55:22.785 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 msgr2=0x7fe8c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 0x7fe8c406e2f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40cb250 tx=0x7fe8d4008040 comp rx=0 tx=0).stop 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 msgr2=0x7fe8e4151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c4b0 tx=0x7fe8d000c970 comp rx=0 tx=0).stop 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 shutdown_connections 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe8c406be30 0x7fe8c406e2f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8e4161ac0 0x7fe8e415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8e40d2ac0 0x7fe8e4151890 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 --2- 172.21.15.28:0/3574865296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e40b3640 0x7fe8e40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 >> 172.21.15.28:0/3574865296 conn(0x7fe8e408bfc0 msgr2=0x7fe8e40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 shutdown_connections 2024-09-10T10:55:22.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.787+0000 7fe8e8a00640 1 -- 172.21.15.28:0/3574865296 wait complete. 2024-09-10T10:55:22.793 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-09-10T10:55:22.792+0000 7f0e3ca00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:22.794 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:22.793+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:55:22.794 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:22.793+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:55:22.808 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:22.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.963+0000 7f1f45800640 1 Processor -- start 2024-09-10T10:55:22.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.963+0000 7f1f45800640 1 -- start start 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f45800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f4015e660 0x7f1f4006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f45800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f400d1f90 0x7f1f400cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f45800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f400cf000 con 0x7f1f400d0dd0 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f45800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1f400b37d0 con 0x7f1f400d1f90 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f45800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f400cf330 con 0x7f1f4015e660 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f44e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f400d1f90 0x7f1f400cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f3fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1f400d0dd0 0x7f1f400d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47856/0 (socket says 172.21.15.28:47856) 2024-09-10T10:55:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f3fe00640 1 -- 172.21.15.28:0/3034981933 learned_addr learned my addr 172.21.15.28:0/3034981933 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.964+0000 7f1f44e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f400d1f90 0x7f1f400cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36160/0 (socket says 172.21.15.28:36160) 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f3f400640 1 --2- 172.21.15.28:0/3034981933 >> v2:172.21.15.138:3300/0 conn(0x7f1f4015e660 0x7f1f4006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f44e00640 1 -- 172.21.15.28:0/3034981933 >> v2:172.21.15.138:3300/0 conn(0x7f1f4015e660 msgr2=0x7f1f4006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f44e00640 1 --2- 172.21.15.28:0/3034981933 >> v2:172.21.15.138:3300/0 conn(0x7f1f4015e660 0x7f1f4006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f44e00640 1 -- 172.21.15.28:0/3034981933 >> v1:172.21.15.28:6789/0 conn(0x7f1f400d0dd0 legacy=0x7f1f400d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f44e00640 1 -- 172.21.15.28:0/3034981933 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f400d29a0 con 0x7f1f400d1f90 2024-09-10T10:55:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.965+0000 7f1f3ea00640 1 -- 172.21.15.28:0/3034981933 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220512240 0 0) 0x7f1f400cf000 con 0x7f1f400d0dd0 2024-09-10T10:55:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.966+0000 7f1f44e00640 1 --2- 172.21.15.28:0/3034981933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d1f90 0x7f1f400cf7c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b3980 tx=0x7f1f3802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d85ecf1712bb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.966+0000 7f1f3ea00640 1 -- 172.21.15.28:0/3034981933 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38007740 con 0x7f1f400d1f90 2024-09-10T10:55:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.966+0000 7f1f3ea00640 1 -- 172.21.15.28:0/3034981933 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f380078e0 con 0x7f1f400d1f90 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.966+0000 7f1f3ea00640 1 -- 172.21.15.28:0/3034981933 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38007cb0 con 0x7f1f400d1f90 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 -- 172.21.15.28:0/3034981933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d1f90 msgr2=0x7f1f400cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 --2- 172.21.15.28:0/3034981933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d1f90 0x7f1f400cf7c0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f1f400b3980 tx=0x7f1f3802a680 comp rx=0 tx=0).stop 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 -- 172.21.15.28:0/3034981933 shutdown_connections 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 --2- 172.21.15.28:0/3034981933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d1f90 0x7f1f400cf7c0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 --2- 172.21.15.28:0/3034981933 >> v2:172.21.15.138:3300/0 conn(0x7f1f4015e660 0x7f1f4006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 -- 172.21.15.28:0/3034981933 >> 172.21.15.28:0/3034981933 conn(0x7f1f4008bfc0 msgr2=0x7f1f400d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 -- 172.21.15.28:0/3034981933 shutdown_connections 2024-09-10T10:55:22.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.967+0000 7f1f45800640 1 -- 172.21.15.28:0/3034981933 wait complete. 2024-09-10T10:55:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.968+0000 7f1f45800640 1 Processor -- start 2024-09-10T10:55:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 -- start start 2024-09-10T10:55:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f400d1f90 0x7f1f401e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f4015e660 0x7f1f4015ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400d3220 con 0x7f1f400d0dd0 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400d2f40 con 0x7f1f400d1f90 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.969+0000 7f1f45800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f400d33a0 con 0x7f1f4015e660 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36166/0 (socket says 172.21.15.28:36166) 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 -- 172.21.15.28:0/2305868699 learned_addr learned my addr 172.21.15.28:0/2305868699 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3f400640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f400d1f90 0x7f1f401e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f44e00640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f4015e660 0x7f1f4015ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 -- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f4015e660 msgr2=0x7f1f4015ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f4015e660 0x7f1f4015ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 -- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f400d1f90 msgr2=0x7f1f401e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f400d1f90 0x7f1f401e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.970+0000 7f1f3fe00640 1 -- 172.21.15.28:0/2305868699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f400cdb10 con 0x7f1f400d0dd0 2024-09-10T10:55:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f3fe00640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c4d0 tx=0x7f1f3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30018070 con 0x7f1f400d0dd0 2024-09-10T10:55:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f30007500 con 0x7f1f400d0dd0 2024-09-10T10:55:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004a20 con 0x7f1f400d0dd0 2024-09-10T10:55:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f400cca70 con 0x7f1f400d0dd0 2024-09-10T10:55:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.971+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f400cd030 con 0x7f1f400d0dd0 2024-09-10T10:55:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.972+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f400d2440 con 0x7f1f400d0dd0 2024-09-10T10:55:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.973+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1f3001b070 con 0x7f1f400d0dd0 2024-09-10T10:55:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.974+0000 7f1f37e00640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.974+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f1f3008b0a0 con 0x7f1f400d0dd0 2024-09-10T10:55:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.974+0000 7f1f3f400640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.975+0000 7f1f3f400640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 0x7f1f2006e2f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1f40156060 tx=0x7f1f2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:22.978+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1f300583e0 con 0x7f1f400d0dd0 2024-09-10T10:55:23.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.201+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f400d1b20 con 0x7f1f400d0dd0 2024-09-10T10:55:23.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.201+0000 7f1f37e00640 1 -- 172.21.15.28:0/2305868699 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 324 v324) ==== 95+0+4389 (secure 0 0 0) 0x7f1f3005c6e0 con 0x7f1f400d0dd0 2024-09-10T10:55:23.202 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:23.203 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":324,"btime":"2024-09-10T10:55:22:794185+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:22.794129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:23.203 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 324 2024-09-10T10:55:23.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 msgr2=0x7f1f2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 0x7f1f2006e2f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1f40156060 tx=0x7f1f2c008040 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 msgr2=0x7f1f40155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f1f3000c4d0 tx=0x7f1f3000c990 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 shutdown_connections 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1f2006be30 0x7f1f2006e2f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f4015e660 0x7f1f4015ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f400d1f90 0x7f1f401e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 --2- 172.21.15.28:0/2305868699 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f400d0dd0 0x7f1f40155b20 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.203+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 >> 172.21.15.28:0/2305868699 conn(0x7f1f4008bfc0 msgr2=0x7f1f400b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:23.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.204+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 shutdown_connections 2024-09-10T10:55:23.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.204+0000 7f1f45800640 1 -- 172.21.15.28:0/2305868699 wait complete. 2024-09-10T10:55:23.221 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:55:23.221 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:23.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.376+0000 7f44e0a00640 1 Processor -- start 2024-09-10T10:55:23.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.377+0000 7f44e0a00640 1 -- start start 2024-09-10T10:55:23.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44dc072320 0x7f44dc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f44dc072de0 0x7f44dc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc06e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f44dc146140 con 0x7f44dc072320 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f44dc0cd740 con 0x7f44dc072de0 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44e0a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44dc0cda20 con 0x7f44dc072700 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44dc072320 0x7f44dc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f44dc072320 0x7f44dc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36170/0 (socket says 172.21.15.28:36170) 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 -- 172.21.15.28:0/3319442021 learned_addr learned my addr 172.21.15.28:0/3319442021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 -- 172.21.15.28:0/3319442021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 msgr2=0x7f44dc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 --2- 172.21.15.28:0/3319442021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc06e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 -- 172.21.15.28:0/3319442021 >> v2:172.21.15.138:3300/0 conn(0x7f44dc072de0 msgr2=0x7f44dc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 --2- 172.21.15.28:0/3319442021 >> v2:172.21.15.138:3300/0 conn(0x7f44dc072de0 0x7f44dc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.378+0000 7f44d9e00640 1 -- 172.21.15.28:0/3319442021 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc0cdd00 con 0x7f44dc072320 2024-09-10T10:55:23.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.379+0000 7f44d9e00640 1 --2- 172.21.15.28:0/3319442021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072320 0x7f44dc0cce60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f44dc06f8c0 tx=0x7f44cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a62b1f421c06f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44d9400640 1 -- 172.21.15.28:0/3319442021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc037070 con 0x7f44dc072320 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44d9400640 1 -- 172.21.15.28:0/3319442021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44cc032440 con 0x7f44dc072320 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44d9400640 1 -- 172.21.15.28:0/3319442021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc032810 con 0x7f44dc072320 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 -- 172.21.15.28:0/3319442021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072320 msgr2=0x7f44dc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 --2- 172.21.15.28:0/3319442021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072320 0x7f44dc0cce60 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f44dc06f8c0 tx=0x7f44cc02a140 comp rx=0 tx=0).stop 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 -- 172.21.15.28:0/3319442021 shutdown_connections 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 --2- 172.21.15.28:0/3319442021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc06e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 --2- 172.21.15.28:0/3319442021 >> v2:172.21.15.138:3300/0 conn(0x7f44dc072de0 0x7f44dc06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 --2- 172.21.15.28:0/3319442021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072320 0x7f44dc0cce60 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 -- 172.21.15.28:0/3319442021 >> 172.21.15.28:0/3319442021 conn(0x7f44dc067d70 msgr2=0x7f44dc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 -- 172.21.15.28:0/3319442021 shutdown_connections 2024-09-10T10:55:23.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.380+0000 7f44e0a00640 1 -- 172.21.15.28:0/3319442021 wait complete. 2024-09-10T10:55:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.382+0000 7f44e0a00640 1 Processor -- start 2024-09-10T10:55:23.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.382+0000 7f44e0a00640 1 -- start start 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44dc072320 0x7f44dc1f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc0cd520 con 0x7f44dc072de0 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44dc0cd3a0 con 0x7f44dc072320 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44e0a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44dc06eb30 con 0x7f44dc072700 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44da800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44da800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36184/0 (socket says 172.21.15.28:36184) 2024-09-10T10:55:23.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44da800640 1 -- 172.21.15.28:0/1494116252 learned_addr learned my addr 172.21.15.28:0/1494116252 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.383+0000 7f44d1400640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44d9e00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44dc072320 0x7f44dc1f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 -- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 msgr2=0x7f44dc13ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc13ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 -- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44dc072320 msgr2=0x7f44dc1f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44dc072320 0x7f44dc1f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 -- 172.21.15.28:0/1494116252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44dc1f1f50 con 0x7f44dc072de0 2024-09-10T10:55:23.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44da800640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b9e0 tx=0x7f44d400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4019070 con 0x7f44dc072de0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d400cb20 con 0x7f44dc072de0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4014440 con 0x7f44dc072de0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.384+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44dc1f2230 con 0x7f44dc072de0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.385+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44dc1f27f0 con 0x7f44dc072de0 2024-09-10T10:55:23.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.386+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44dc0cce60 con 0x7f44dc072de0 2024-09-10T10:55:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.387+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44d401c070 con 0x7f44dc072de0 2024-09-10T10:55:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.387+0000 7f44d2a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 0x7f44b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.388+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f44d408ba40 con 0x7f44dc072de0 2024-09-10T10:55:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.388+0000 7f44d9e00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 0x7f44b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.389+0000 7f44d9e00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 0x7f44b806e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44cc034000 tx=0x7f44cc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.392+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44d4058de0 con 0x7f44dc072de0 2024-09-10T10:55:23.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.613+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44dc0c75c0 con 0x7f44dc072de0 2024-09-10T10:55:23.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.614+0000 7f44d2a00640 1 -- 172.21.15.28:0/1494116252 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 324 v324) ==== 95+0+4389 (secure 0 0 0) 0x7f44d408be20 con 0x7f44dc072de0 2024-09-10T10:55:23.615 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:23.616 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":324,"btime":"2024-09-10T10:55:22:794185+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:22.794129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:23.616 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 324 2024-09-10T10:55:23.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 msgr2=0x7f44b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 0x7f44b806e2f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f44cc034000 tx=0x7f44cc02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 msgr2=0x7f44dc141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f44d400b9e0 tx=0x7f44d400bea0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 shutdown_connections 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f44b806be30 0x7f44b806e2f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f44dc072de0 0x7f44dc141860 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44dc072700 0x7f44dc13ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 --2- 172.21.15.28:0/1494116252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44dc072320 0x7f44dc1f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 >> 172.21.15.28:0/1494116252 conn(0x7f44dc067d70 msgr2=0x7f44dc0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 shutdown_connections 2024-09-10T10:55:23.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.616+0000 7f44e0a00640 1 -- 172.21.15.28:0/1494116252 wait complete. 2024-09-10T10:55:23.633 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:55:23.633 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:23.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.788+0000 7fa7b7000640 1 Processor -- start 2024-09-10T10:55:23.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.788+0000 7fa7b7000640 1 -- start start 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.789+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 0x7fa7b0169070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.789+0000 7fa7b7000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b006e820 con 0x7fa7b00d15f0 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.789+0000 7fa7b7000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7b00cf980 con 0x7fa7b0161a20 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.789+0000 7fa7b7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b00cf760 con 0x7fa7b00cf330 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7a7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa7b00d15f0 0x7fa7b00cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47870/0 (socket says 172.21.15.28:47870) 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7a7400640 1 -- 172.21.15.28:0/2791499414 learned_addr learned my addr 172.21.15.28:0/2791499414 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:23.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7b5000640 1 --2- 172.21.15.28:0/2791499414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 0x7fa7b0169070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1335357337 0 0) 0x7fa7b006e820 con 0x7fa7b00d15f0 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa794003520 con 0x7fa7b00d15f0 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649453115 0 0) 0x7fa7b00cf980 con 0x7fa7b0161a20 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7b006e820 con 0x7fa7b0161a20 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7b5000640 1 -- 172.21.15.28:0/2791499414 >> v1:172.21.15.138:6789/0 conn(0x7fa7b0161a20 legacy=0x7fa7b00d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7b5000640 1 -- 172.21.15.28:0/2791499414 >> v1:172.21.15.28:6789/0 conn(0x7fa7b00d15f0 legacy=0x7fa7b00cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.790+0000 7fa7b5000640 1 -- 172.21.15.28:0/2791499414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b006d050 con 0x7fa7b00cf330 2024-09-10T10:55:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.791+0000 7fa7b5000640 1 --2- 172.21.15.28:0/2791499414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 0x7fa7b0169070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7b006d220 tx=0x7fa7a802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f863b3d2d55d14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8007740 con 0x7fa7b00cf330 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7a80078e0 con 0x7fa7b00cf330 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7af400640 1 -- 172.21.15.28:0/2791499414 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a8007c90 con 0x7fa7b00cf330 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 -- 172.21.15.28:0/2791499414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 msgr2=0x7fa7b0169070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 --2- 172.21.15.28:0/2791499414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 0x7fa7b0169070 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa7b006d220 tx=0x7fa7a802a6c0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 -- 172.21.15.28:0/2791499414 shutdown_connections 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 --2- 172.21.15.28:0/2791499414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b00cf330 0x7fa7b0169070 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 -- 172.21.15.28:0/2791499414 >> 172.21.15.28:0/2791499414 conn(0x7fa7b009d850 msgr2=0x7fa7b008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 -- 172.21.15.28:0/2791499414 shutdown_connections 2024-09-10T10:55:23.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.792+0000 7fa7b7000640 1 -- 172.21.15.28:0/2791499414 wait complete. 2024-09-10T10:55:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.794+0000 7fa7b7000640 1 Processor -- start 2024-09-10T10:55:23.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.794+0000 7fa7b7000640 1 -- start start 2024-09-10T10:55:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b00cf330 0x7fa7b0152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b0161a20 0x7fa7b0158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b00cff00 con 0x7fa7b00d15f0 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b0161e00 con 0x7fa7b00cf330 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7b7000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7b006d2f0 con 0x7fa7b0161a20 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7a7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7a7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36188/0 (socket says 172.21.15.28:36188) 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.795+0000 7fa7a7400640 1 -- 172.21.15.28:0/1216900017 learned_addr learned my addr 172.21.15.28:0/1216900017 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7afe00640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b00cf330 0x7fa7b0152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7b5000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b0161a20 0x7fa7b0158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 -- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b0161a20 msgr2=0x7fa7b0158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b0161a20 0x7fa7b0158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 -- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b00cf330 msgr2=0x7fa7b0152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b00cf330 0x7fa7b0152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 -- 172.21.15.28:0/1216900017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b00b9bd0 con 0x7fa7b00d15f0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7a7400640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c4e0 tx=0x7fa79800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798017070 con 0x7fa7b00d15f0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa798005ce0 con 0x7fa7b00d15f0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b00b9eb0 con 0x7fa7b00d15f0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa798007270 con 0x7fa7b00d15f0 2024-09-10T10:55:23.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.796+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b00b8e10 con 0x7fa7b00d15f0 2024-09-10T10:55:23.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.797+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b00b1040 con 0x7fa7b00d15f0 2024-09-10T10:55:23.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.798+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa79801a070 con 0x7fa7b00d15f0 2024-09-10T10:55:23.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.799+0000 7fa7acc00640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 0x7fa78c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:23.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.799+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7fa79808b090 con 0x7fa7b00d15f0 2024-09-10T10:55:23.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.799+0000 7fa7afe00640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 0x7fa78c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:23.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.800+0000 7fa7afe00640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 0x7fa78c06e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00015b0 tx=0x7fa7a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:23.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:23.804+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa798058460 con 0x7fa7b00d15f0 2024-09-10T10:55:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.024+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7b015ea60 con 0x7fa7b00d15f0 2024-09-10T10:55:24.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.024+0000 7fa7acc00640 1 -- 172.21.15.28:0/1216900017 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 324 v324) ==== 95+0+4389 (secure 0 0 0) 0x7fa79805c760 con 0x7fa7b00d15f0 2024-09-10T10:55:24.025 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:24.026 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":324,"btime":"2024-09-10T10:55:22:794185+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":9022,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":291}],"filesystems":[{"mdsmap":{"epoch":324,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:22.794129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:24.027 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 324 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 msgr2=0x7fa78c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 0x7fa78c06e2f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00015b0 tx=0x7fa7a0009040 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 msgr2=0x7fa7b0157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c4e0 tx=0x7fa79800c9a0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 shutdown_connections 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa78c06be30 0x7fa78c06e2f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7b0161a20 0x7fa7b0158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7b00d15f0 0x7fa7b0157f30 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 --2- 172.21.15.28:0/1216900017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7b00cf330 0x7fa7b0152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.026+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 >> 172.21.15.28:0/1216900017 conn(0x7fa7b009d850 msgr2=0x7fa7b015dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.027+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 shutdown_connections 2024-09-10T10:55:24.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.027+0000 7fa7b7000640 1 -- 172.21.15.28:0/1216900017 wait complete. 2024-09-10T10:55:24.043 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on mds0_sessionmap' led to DAMAGED state 2024-09-10T10:55:24.043 INFO:tasks.cephfs.test_damage:Applying mutation 'Corrupt omap header on 1.00000000' 2024-09-10T10:55:24.043 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:24.043 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:24.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:24.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:55:24.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:55:24.079 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:55:24.153 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T10:55:24.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:55:24.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:55:24.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.350+0000 7f958b800640 1 Processor -- start 2024-09-10T10:55:24.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.350+0000 7f958b800640 1 -- start start 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f958b800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9584072d30 0x7f958406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f958b800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f958406fae0 con 0x7f958406f700 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f958b800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95840cdbe0 con 0x7f958415b300 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f958b800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f95840728a0 con 0x7f9584072d30 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f9589800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f958406f700 0x7f958415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59760/0 (socket says 172.21.15.28:59760) 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f9589800640 1 -- 172.21.15.28:0/2823465967 learned_addr learned my addr 172.21.15.28:0/2823465967 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:24.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f9583e00640 1 --2- 172.21.15.28:0/2823465967 >> v2:172.21.15.138:3300/0 conn(0x7f9584072d30 0x7f958406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f9583400640 1 -- 172.21.15.28:0/2823465967 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919205487 0 0) 0x7f958406fae0 con 0x7f958406f700 2024-09-10T10:55:24.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.352+0000 7f9583400640 1 -- 172.21.15.28:0/2823465967 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f956c005730 con 0x7f958406f700 2024-09-10T10:55:24.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.353+0000 7f9583e00640 1 -- 172.21.15.28:0/2823465967 >> v1:172.21.15.138:6789/0 conn(0x7f958415b300 legacy=0x7f95840cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.353+0000 7f9583e00640 1 -- 172.21.15.28:0/2823465967 >> v1:172.21.15.28:6789/0 conn(0x7f958406f700 legacy=0x7f958415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.353+0000 7f9583e00640 1 -- 172.21.15.28:0/2823465967 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840cd900 con 0x7f9584072d30 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f9583e00640 1 --2- 172.21.15.28:0/2823465967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f958406f1c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=673ae9d57cb18a59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f9583400640 1 -- 172.21.15.28:0/2823465967 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570037070 con 0x7f9584072d30 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f9583400640 1 -- 172.21.15.28:0/2823465967 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9570032440 con 0x7f9584072d30 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f9583400640 1 -- 172.21.15.28:0/2823465967 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9570032810 con 0x7f9584072d30 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f958b800640 1 -- 172.21.15.28:0/2823465967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 msgr2=0x7f958406f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.354+0000 7f958b800640 1 --2- 172.21.15.28:0/2823465967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f958406f1c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f95700097a0 tx=0x7f957002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.355+0000 7f958b800640 1 -- 172.21.15.28:0/2823465967 shutdown_connections 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.355+0000 7f958b800640 1 --2- 172.21.15.28:0/2823465967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f958406f1c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.355+0000 7f958b800640 1 -- 172.21.15.28:0/2823465967 >> 172.21.15.28:0/2823465967 conn(0x7f9584067d70 msgr2=0x7f958406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.355+0000 7f958b800640 1 -- 172.21.15.28:0/2823465967 shutdown_connections 2024-09-10T10:55:24.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.355+0000 7f958b800640 1 -- 172.21.15.28:0/2823465967 wait complete. 2024-09-10T10:55:24.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.356+0000 7f958b800640 1 Processor -- start 2024-09-10T10:55:24.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.356+0000 7f958b800640 1 -- start start 2024-09-10T10:55:24.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958406f700 0x7f9584156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f9584154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f95840cd5c0 con 0x7f958415b300 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9584072530 con 0x7f9584072d30 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.357+0000 7f958b800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95840cffe0 con 0x7f958406f700 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49440/0 (socket says 172.21.15.28:49440) 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 -- 172.21.15.28:0/409171676 learned_addr learned my addr 172.21.15.28:0/409171676 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9588e00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958406f700 0x7f9584156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9583e00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f9584154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 -- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958406f700 msgr2=0x7f9584156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958406f700 0x7f9584156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 -- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 msgr2=0x7f9584154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f9584154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 -- 172.21.15.28:0/409171676 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840b1df0 con 0x7f958415b300 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9583e00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f9584154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9589800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f957c00caa0 tx=0x7f957c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c017070 con 0x7f958415b300 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c00ac00 con 0x7f958415b300 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.358+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9584154910 con 0x7f958415b300 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.359+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c012400 con 0x7f958415b300 2024-09-10T10:55:24.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.359+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95840cb140 con 0x7f958415b300 2024-09-10T10:55:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.360+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95840b2370 con 0x7f958415b300 2024-09-10T10:55:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.360+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f957c01a070 con 0x7f958415b300 2024-09-10T10:55:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.361+0000 7f9580c00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 0x7f956406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.361+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f957c08bc30 con 0x7f958415b300 2024-09-10T10:55:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.362+0000 7f9588e00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 0x7f956406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.363+0000 7f9588e00640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 0x7f956406e320 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f60 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.365+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f957c0595c0 con 0x7f958415b300 2024-09-10T10:55:24.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.586+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95841586d0 con 0x7f958415b300 2024-09-10T10:55:24.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.784+0000 7f9580c00640 1 -- 172.21.15.28:0/409171676 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v325) ==== 161+0+0 (secure 0 0 0) 0x7f957c058ec0 con 0x7f958415b300 2024-09-10T10:55:24.786 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:55:24.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 msgr2=0x7f956406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 0x7f956406e320 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f60 tx=0x7f9578009040 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 msgr2=0x7f95841f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f957c00caa0 tx=0x7f957c00cf60 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 shutdown_connections 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f956406be60 0x7f956406e320 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f958415b300 0x7f95841f72c0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9584072d30 0x7f9584154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 --2- 172.21.15.28:0/409171676 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f958406f700 0x7f9584156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 >> 172.21.15.28:0/409171676 conn(0x7f9584067d70 msgr2=0x7f95840b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 shutdown_connections 2024-09-10T10:55:24.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.787+0000 7f958b800640 1 -- 172.21.15.28:0/409171676 wait complete. 2024-09-10T10:55:24.804 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2024-09-10T10:55:24.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.961+0000 7fee5ae00640 1 Processor -- start 2024-09-10T10:55:24.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.962+0000 7fee5ae00640 1 -- start start 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee5ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee540aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee5ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee540b5470 con 0x7fee5415bef0 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee5ae00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee540b3640 con 0x7fee540af1b0 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee5ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee540b5190 con 0x7fee540b40e0 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee59a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fee5415bef0 0x7fee540b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59766/0 (socket says 172.21.15.28:59766) 2024-09-10T10:55:24.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee59a00640 1 -- 172.21.15.28:0/30142391 learned_addr learned my addr 172.21.15.28:0/30142391 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee59000640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee540aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839554497 0 0) 0x7fee540b5470 con 0x7fee5415bef0 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.963+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee3c003520 con 0x7fee5415bef0 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1959685132 0 0) 0x7fee3c003520 con 0x7fee5415bef0 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 msgr2=0x7fee540aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:24.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee540aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 >> v1:172.21.15.138:6789/0 conn(0x7fee540af1b0 legacy=0x7fee54071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee540b38e0 con 0x7fee5415bef0 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fee48001670 con 0x7fee5415bef0 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 >> v1:172.21.15.28:6789/0 conn(0x7fee5415bef0 legacy=0x7fee540b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c0039e0 0x7fee3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee540b5190 con 0x7fee3c004d40 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c00bb50 con 0x7fee3c0039e0 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c00bd50 con 0x7fee540b40e0 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee5a400640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee59a00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c0039e0 0x7fee3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.964+0000 7fee59000640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.965+0000 7fee5a400640 1 -- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 msgr2=0x7fee3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.965+0000 7fee5a400640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b40e0 0x7fee3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.965+0000 7fee5a400640 1 -- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c0039e0 msgr2=0x7fee3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.965+0000 7fee5a400640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee3c0039e0 0x7fee3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.965+0000 7fee5a400640 1 -- 172.21.15.28:0/30142391 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c003520 con 0x7fee3c004d40 2024-09-10T10:55:24.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee5a400640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee3c0051c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fee4c001e50 tx=0x7fee3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856a8e24178c188 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c001a70 con 0x7fee3c004d40 2024-09-10T10:55:24.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee4c003970 con 0x7fee3c004d40 2024-09-10T10:55:24.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee53e00640 1 -- 172.21.15.28:0/30142391 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c003d60 con 0x7fee3c004d40 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee5ae00640 1 -- 172.21.15.28:0/30142391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 msgr2=0x7fee3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee5ae00640 1 --2- 172.21.15.28:0/30142391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee3c0051c0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fee4c001e50 tx=0x7fee3c005830 comp rx=0 tx=0).stop 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.966+0000 7fee5a400640 1 -- 172.21.15.28:0/30142391 reap_dead start 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.967+0000 7fee5ae00640 1 -- 172.21.15.28:0/30142391 shutdown_connections 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.967+0000 7fee5ae00640 1 -- 172.21.15.28:0/30142391 >> 172.21.15.28:0/30142391 conn(0x7fee5406c240 msgr2=0x7fee540b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.967+0000 7fee5ae00640 1 -- 172.21.15.28:0/30142391 shutdown_connections 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.967+0000 7fee5ae00640 1 -- 172.21.15.28:0/30142391 wait complete. 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.968+0000 7fee5ae00640 1 Processor -- start 2024-09-10T10:55:24.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.969+0000 7fee5ae00640 1 -- start start 2024-09-10T10:55:24.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.969+0000 7fee5ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.969+0000 7fee5ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5415a9b0 0x7fee540b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.969+0000 7fee5ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b1930 0x7fee5415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.969+0000 7fee5ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee540b4e60 con 0x7fee3c004d40 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee5ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee540af640 con 0x7fee5415a9b0 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee5ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee540b4b10 con 0x7fee540b1930 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49454/0 (socket says 172.21.15.28:49454) 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 -- 172.21.15.28:0/2760068527 learned_addr learned my addr 172.21.15.28:0/2760068527 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee5a400640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b1930 0x7fee5415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59000640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5415a9b0 0x7fee540b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 -- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b1930 msgr2=0x7fee5415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b1930 0x7fee5415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 -- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5415a9b0 msgr2=0x7fee540b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5415a9b0 0x7fee540b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 -- 172.21.15.28:0/2760068527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee54143ee0 con 0x7fee3c004d40 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.970+0000 7fee59a00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fee48007290 tx=0x7fee48007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.971+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800f960 con 0x7fee3c004d40 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.971+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48018070 con 0x7fee3c004d40 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.971+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4800fd80 con 0x7fee3c004d40 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.971+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee54159910 con 0x7fee3c004d40 2024-09-10T10:55:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.971+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee54159ed0 con 0x7fee3c004d40 2024-09-10T10:55:24.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.972+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5404ffb0 con 0x7fee3c004d40 2024-09-10T10:55:24.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.973+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee4801d460 con 0x7fee3c004d40 2024-09-10T10:55:24.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.974+0000 7fee51600640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 0x7fee3406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:24.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.974+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7fee4808bf70 con 0x7fee3c004d40 2024-09-10T10:55:24.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.974+0000 7fee59000640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 0x7fee3406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:24.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.975+0000 7fee59000640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 0x7fee3406e320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fee541f3650 tx=0x7fee40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:24.978+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fee5404ffb0 con 0x7fee3c004d40 2024-09-10T10:55:25.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.234+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) -- 0x7fee5415f610 con 0x7fee3c004d40 2024-09-10T10:55:25.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.793+0000 7fee51600640 1 -- 172.21.15.28:0/2760068527 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 6:0 v326) ==== 101+0+0 (secure 0 0 0) 0x7fee48059900 con 0x7fee3c004d40 2024-09-10T10:55:25.794 INFO:teuthology.orchestra.run.smithi028.stderr:repaired: restoring rank 6:0 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.795+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 msgr2=0x7fee3406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.795+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 0x7fee3406e320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fee541f3650 tx=0x7fee40008040 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.795+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 msgr2=0x7fee541439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.795+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fee48007290 tx=0x7fee48007690 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 shutdown_connections 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fee3406be60 0x7fee3406e320 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee540b1930 0x7fee5415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5415a9b0 0x7fee540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 --2- 172.21.15.28:0/2760068527 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee3c004d40 0x7fee541439a0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 >> 172.21.15.28:0/2760068527 conn(0x7fee5406c240 msgr2=0x7fee540acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:25.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 shutdown_connections 2024-09-10T10:55:25.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.796+0000 7fee5ae00640 1 -- 172.21.15.28:0/2760068527 wait complete. 2024-09-10T10:55:25.815 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata import - 2024-09-10T10:55:25.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.892+0000 7f21416b0f40 1 Processor -- start 2024-09-10T10:55:25.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.892+0000 7f21416b0f40 1 -- start start 2024-09-10T10:55:25.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 0x55cad4c0dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 0x55cad4c0e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad4b82530 con 0x55cad4b818e0 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cad4b82290 con 0x55cad4c0d3e0 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f21416b0f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cad4c0fee0 con 0x55cad4c109c0 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49482/0 (socket says 172.21.15.28:49482) 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213e800640 1 -- 172.21.15.28:0/1537855124 learned_addr learned my addr 172.21.15.28:0/1537855124 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213f200640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 0x55cad4c0e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213e800640 1 -- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 msgr2=0x55cad4c0e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.893+0000 7f213de00640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 0x55cad4c0dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213e800640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 0x55cad4c0e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213e800640 1 -- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 msgr2=0x55cad4c0dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213e800640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 0x55cad4c0dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213e800640 1 -- 172.21.15.28:0/1537855124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad4a63b30 con 0x55cad4b818e0 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213de00640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 0x55cad4c0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213f200640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 0x55cad4c0e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:25.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213e800640 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f2130045ae0 tx=0x7f2130051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cd05e66d4a05b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213ca00640 1 -- 172.21.15.28:0/1537855124 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130056070 con 0x55cad4b818e0 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213ca00640 1 -- 172.21.15.28:0/1537855124 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21300529d0 con 0x55cad4b818e0 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.894+0000 7f213ca00640 1 -- 172.21.15.28:0/1537855124 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130052dc0 con 0x55cad4b818e0 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 -- 172.21.15.28:0/1537855124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 msgr2=0x55cad4c11100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f2130045ae0 tx=0x7f2130051000 comp rx=0 tx=0).stop 2024-09-10T10:55:25.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 -- 172.21.15.28:0/1537855124 shutdown_connections 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c109c0 0x55cad4c0e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4c0d3e0 0x55cad4c0dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 --2- 172.21.15.28:0/1537855124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4b818e0 0x55cad4c11100 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 -- 172.21.15.28:0/1537855124 >> 172.21.15.28:0/1537855124 conn(0x55cad4b860a0 msgr2=0x55cad4b37070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 -- 172.21.15.28:0/1537855124 shutdown_connections 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 -- 172.21.15.28:0/1537855124 wait complete. 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.895+0000 7f21416b0f40 1 Processor -- start 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 -- start start 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4b818e0 0x55cad4bfea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c0d3e0 0x55cad4bfef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad4b84e50 con 0x55cad4c109c0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cad4b82530 con 0x55cad4b818e0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.896+0000 7f21416b0f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55cad4c14090 con 0x55cad4c0d3e0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49494/0 (socket says 172.21.15.28:49494) 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 learned_addr learned my addr 172.21.15.28:0/2464547227 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4b818e0 0x55cad4bfea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c0d3e0 0x55cad4bfef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c0d3e0 msgr2=0x55cad4bfef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55cad4c0d3e0 0x55cad4bfef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4b818e0 msgr2=0x55cad4bfea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cad4b818e0 0x55cad4bfea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.897+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad4bffd70 con 0x55cad4c109c0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f213f200640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f213800e490 tx=0x7f213800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f2137400640 1 -- 172.21.15.28:0/2464547227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138017020 con 0x55cad4c109c0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f2137400640 1 -- 172.21.15.28:0/2464547227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2138013870 con 0x55cad4c109c0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f2137400640 1 -- 172.21.15.28:0/2464547227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2138013c60 con 0x55cad4c109c0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cad4c07410 con 0x55cad4c109c0 2024-09-10T10:55:25.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.898+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad4c076b0 con 0x55cad4c109c0 2024-09-10T10:55:25.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.900+0000 7f2137400640 1 -- 172.21.15.28:0/2464547227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f213801a020 con 0x55cad4c109c0 2024-09-10T10:55:25.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f2137400640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 0x7f212006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f2137400640 1 -- 172.21.15.28:0/2464547227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f213808adc0 con 0x55cad4c109c0 2024-09-10T10:55:25.903 INFO:teuthology.orchestra.run.smithi028.stdout:Importing pool 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 0x7f212006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cad4c11f50 0x55cad4c12330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:1 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4c12870 con 0x55cad4c11f50 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.901+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cad4c11f50 0x55cad4c12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.902+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 0x7f212006ccd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2130045ae0 tx=0x7f2130053000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:25.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.902+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cad4c11f50 0x55cad4c12330 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:25.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.905+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(1 601.00000000 [create] v139'1190 uv1164 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.905+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:2 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb2200 con 0x55cad4c11f50 2024-09-10T10:55:25.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.907+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== osd_op_reply(2 601.00000000 [delete] v139'1191 uv1191 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.907+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:3 13.10 13:09401a81:::601.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb25a0 con 0x55cad4c11f50 2024-09-10T10:55:25.910 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::601.00000000:head# 2024-09-10T10:55:25.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.909+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 3 ==== osd_op_reply(3 601.00000000 [create] v139'1192 uv1192 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.909+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:4 13.10 13:09401a81:::601.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb3950 con 0x55cad4c11f50 2024-09-10T10:55:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.911+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 4 ==== osd_op_reply(4 601.00000000 [setxattr (30)] v139'1193 uv1193 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.911+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:5 13.10 13:09401a81:::601.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb3cf0 con 0x55cad4c11f50 2024-09-10T10:55:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.912+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 5 ==== osd_op_reply(5 601.00000000 [setxattr (62)] v139'1194 uv1194 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.913+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:6 13.10 13:09401a81:::601.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb2940 con 0x55cad4c11f50 2024-09-10T10:55:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.915+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 6 ==== osd_op_reply(6 601.00000000 [omap-set-header] v139'1195 uv1195 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.915+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:7 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb2d30 con 0x55cad4c11f50 2024-09-10T10:55:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.917+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 7 ==== osd_op_reply(7 602.00000000 [create] v139'1196 uv1170 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.917+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb3120 con 0x55cad4c11f50 2024-09-10T10:55:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.919+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 8 ==== osd_op_reply(8 602.00000000 [delete] v139'1197 uv1197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.919+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:9 13.10 13:0bd6d154:::602.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb3120 con 0x55cad4c11f50 2024-09-10T10:55:25.922 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::602.00000000:head# 2024-09-10T10:55:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.921+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 9 ==== osd_op_reply(9 602.00000000 [create] v139'1198 uv1198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.921+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:10 13.10 13:0bd6d154:::602.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.923+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 10 ==== osd_op_reply(10 602.00000000 [setxattr (30)] v139'1199 uv1199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.923+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:11 13.10 13:0bd6d154:::602.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.925+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 11 ==== osd_op_reply(11 602.00000000 [setxattr (62)] v139'1200 uv1200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.925+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:12 13.10 13:0bd6d154:::602.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.927+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 12 ==== osd_op_reply(12 602.00000000 [omap-set-header] v139'1201 uv1201 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.927+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:13 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.930+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 13 ==== osd_op_reply(13 10000000000.00000000 [create] v139'1202 uv1177 ondisk = -17 ((17) File exists)) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.930+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:14 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.932+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 14 ==== osd_op_reply(14 10000000000.00000000 [delete] v139'1203 uv1203 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.932+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:15 13.10 13:0cf6ad0f:::10000000000.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.934 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::10000000000.00000000:head# 2024-09-10T10:55:25.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.934+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 15 ==== osd_op_reply(15 10000000000.00000000 [create] v139'1204 uv1204 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.934+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:16 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.935+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 16 ==== osd_op_reply(16 10000000000.00000000 [setxattr (30)] v139'1205 uv1205 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.935+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:17 13.10 13:0cf6ad0f:::10000000000.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.937+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 17 ==== osd_op_reply(17 10000000000.00000000 [setxattr (62)] v139'1206 uv1206 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.937+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:18 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.939+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 18 ==== osd_op_reply(18 10000000000.00000000 [omap-set-header] v139'1207 uv1207 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.939+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:19 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=593b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.941+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 19 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v139'1208 uv1208 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.941+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:20 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.943+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 20 ==== osd_op_reply(20 600.00000000 [create] v139'1209 uv1183 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.943+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:21 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.945+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 21 ==== osd_op_reply(21 600.00000000 [delete] v139'1210 uv1210 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.945+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:22 13.10 13:0d82a743:::600.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.947 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::600.00000000:head# 2024-09-10T10:55:25.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.947+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 22 ==== osd_op_reply(22 600.00000000 [create] v139'1211 uv1211 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.947+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:23 13.10 13:0d82a743:::600.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.948+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 23 ==== osd_op_reply(23 600.00000000 [setxattr (30)] v139'1212 uv1212 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.948+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:24 13.10 13:0d82a743:::600.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.950+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 24 ==== osd_op_reply(24 600.00000000 [setxattr (62)] v139'1213 uv1213 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.950+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:25 13.10 13:0d82a743:::600.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.952+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 25 ==== osd_op_reply(25 600.00000000 [omap-set-header] v139'1214 uv1214 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.952+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:26 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.954+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 26 ==== osd_op_reply(26 603.00000000 [create] v139'1215 uv1189 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.954+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:27 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.956+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 27 ==== osd_op_reply(27 603.00000000 [delete] v139'1216 uv1216 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.956+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:28 13.10 13:0d89b25e:::603.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.958 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::603.00000000:head# 2024-09-10T10:55:25.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.957+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 28 ==== osd_op_reply(28 603.00000000 [create] v139'1217 uv1217 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.958+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:29 13.10 13:0d89b25e:::603.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.959+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 29 ==== osd_op_reply(29 603.00000000 [setxattr (30)] v139'1218 uv1218 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.959+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:30 13.10 13:0d89b25e:::603.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.961+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 30 ==== osd_op_reply(30 603.00000000 [setxattr (62)] v139'1219 uv1219 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.961+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:31 13.10 13:0d89b25e:::603.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4c11f50 2024-09-10T10:55:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.963+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.7 v2:172.21.15.138:6826/4029970444 31 ==== osd_op_reply(31 603.00000000 [omap-set-header] v139'1220 uv1220 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4c11f50 2024-09-10T10:55:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.963+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cad4cb34b0 0x55cad4cb65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.963+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:32 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb30d0 con 0x55cad4cb34b0 2024-09-10T10:55:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.963+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cad4cb34b0 0x55cad4cb65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.964+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cad4cb34b0 0x55cad4cb65b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:25.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.966+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== osd_op_reply(32 1.00000000.inode [create] v139'867 uv850 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.966+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:33 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb6af0 con 0x55cad4cb34b0 2024-09-10T10:55:25.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.968+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== osd_op_reply(33 1.00000000.inode [delete] v139'868 uv868 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.968+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:34 13.14 13:287034c4:::1.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb6e90 con 0x55cad4cb34b0 2024-09-10T10:55:25.970 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000.inode:head# 2024-09-10T10:55:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.969+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 3 ==== osd_op_reply(34 1.00000000.inode [create] v139'869 uv869 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.969+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:35 13.14 13:287034c4:::1.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb7730 con 0x55cad4cb34b0 2024-09-10T10:55:25.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.971+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 4 ==== osd_op_reply(35 1.00000000.inode [write 0~582] v139'870 uv870 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.971+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:36 13.14 13:287034c4:::1.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb7ad0 con 0x55cad4cb34b0 2024-09-10T10:55:25.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.972+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 5 ==== osd_op_reply(36 1.00000000.inode [omap-set-header] v139'871 uv871 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.972+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:37 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb7e70 con 0x55cad4cb34b0 2024-09-10T10:55:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.974+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 6 ==== osd_op_reply(37 200.00000000 [create] v139'872 uv855 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.974+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:38 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb8210 con 0x55cad4cb34b0 2024-09-10T10:55:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.976+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 7 ==== osd_op_reply(38 200.00000000 [delete] v139'873 uv873 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.976+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.978 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000000:head# 2024-09-10T10:55:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.977+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 8 ==== osd_op_reply(39 200.00000000 [create] v139'874 uv874 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.977+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:40 13.14 13:292cf221:::200.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.979+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 9 ==== osd_op_reply(40 200.00000000 [write 0~90] v139'875 uv875 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.979+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:41 13.14 13:292cf221:::200.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.980+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 10 ==== osd_op_reply(41 200.00000000 [omap-set-header] v139'876 uv876 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.980+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:42 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.982+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 11 ==== osd_op_reply(42 200.00000001 [create] v139'877 uv860 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.982+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:43 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.984+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 12 ==== osd_op_reply(43 200.00000001 [delete] v139'878 uv878 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.984+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:44 13.14 13:2e2fa760:::200.00000001:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.986 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::200.00000001:head# 2024-09-10T10:55:25.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.985+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 13 ==== osd_op_reply(44 200.00000001 [create] v139'879 uv879 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.985+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:45 13.14 13:2e2fa760:::200.00000001:head [write 0~18118 in=18118b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.987+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 14 ==== osd_op_reply(45 200.00000001 [write 0~18118] v139'880 uv880 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.987+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:46 13.14 13:2e2fa760:::200.00000001:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.988+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 15 ==== osd_op_reply(46 200.00000001 [omap-set-header] v139'881 uv881 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.988+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:47 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.990+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 16 ==== osd_op_reply(47 606.00000000 [create] v139'882 uv866 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.990+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:48 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.992+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 17 ==== osd_op_reply(48 606.00000000 [delete] v139'883 uv883 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.992+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:49 13.14 13:2f55791f:::606.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.994 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::606.00000000:head# 2024-09-10T10:55:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.993+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 18 ==== osd_op_reply(49 606.00000000 [create] v139'884 uv884 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.993+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:50 13.14 13:2f55791f:::606.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.994+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 19 ==== osd_op_reply(50 606.00000000 [setxattr (30)] v139'885 uv885 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.994+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:51 13.14 13:2f55791f:::606.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.996+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 20 ==== osd_op_reply(51 606.00000000 [setxattr (62)] v139'886 uv886 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.996+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- osd_op(unknown.0.0:52 13.14 13:2f55791f:::606.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cb85b0 con 0x55cad4cb34b0 2024-09-10T10:55:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.997+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.6 v2:172.21.15.138:6818/1283254914 21 ==== osd_op_reply(52 606.00000000 [omap-set-header] v139'887 uv887 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cb34b0 2024-09-10T10:55:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.997+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cad4cb8ed0 0x55cad4cb9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.997+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:53 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbba80 con 0x55cad4cb8ed0 2024-09-10T10:55:25.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.997+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cad4cb8ed0 0x55cad4cb9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:25.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.998+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cad4cb8ed0 0x55cad4cb9310 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.999+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(53 607.00000000 [create] v139'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:25.999+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:54 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbba80 con 0x55cad4cb8ed0 2024-09-10T10:55:26.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.001+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== osd_op_reply(54 607.00000000 [delete] v139'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.001+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:55 13.1c 13:3ed09add:::607.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbba80 con 0x55cad4cb8ed0 2024-09-10T10:55:26.003 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::607.00000000:head# 2024-09-10T10:55:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.002+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 3 ==== osd_op_reply(55 607.00000000 [create] v139'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.002+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:56 13.1c 13:3ed09add:::607.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbca90 con 0x55cad4cb8ed0 2024-09-10T10:55:26.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.003+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 4 ==== osd_op_reply(56 607.00000000 [setxattr (30)] v139'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.003+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:57 13.1c 13:3ed09add:::607.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbca90 con 0x55cad4cb8ed0 2024-09-10T10:55:26.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.005+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 5 ==== osd_op_reply(57 607.00000000 [setxattr (62)] v139'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.005+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:58 13.1c 13:3ed09add:::607.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbba80 con 0x55cad4cb8ed0 2024-09-10T10:55:26.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.006+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 6 ==== osd_op_reply(58 607.00000000 [omap-set-header] v139'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.006+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cad4cbc350 0x55cad4cbc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.006+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:59 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbeeb0 con 0x55cad4cbc350 2024-09-10T10:55:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.007+0000 7f213f200640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cad4cbc350 0x55cad4cbc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.007+0000 7f213f200640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cad4cbc350 0x55cad4cbc7e0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.008+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(59 mds0_openfiles.0 [create] v139'172 uv171 ondisk = -17 ((17) File exists)) ==== 160+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.008+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:60 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbf250 con 0x55cad4cbc350 2024-09-10T10:55:26.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.010+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(60 mds0_openfiles.0 [delete] v139'173 uv173 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.010+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:61 13.6 13:654134d2:::mds0_openfiles.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbf6e0 con 0x55cad4cbc350 2024-09-10T10:55:26.012 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_openfiles.0:head# 2024-09-10T10:55:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.011+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(61 mds0_openfiles.0 [create] v139'174 uv174 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.011+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:62 13.6 13:654134d2:::mds0_openfiles.0:head [omap-set-header in=42b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbffa0 con 0x55cad4cbc350 2024-09-10T10:55:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.012+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(62 mds0_openfiles.0 [omap-set-header] v139'175 uv175 ondisk = 0) ==== 160+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.012+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:63 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0340 con 0x55cad4cb8ed0 2024-09-10T10:55:26.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.013+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 7 ==== osd_op_reply(63 608.00000000 [create] v139'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.013+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:64 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0340 con 0x55cad4cb8ed0 2024-09-10T10:55:26.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.015+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 8 ==== osd_op_reply(64 608.00000000 [delete] v139'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.015+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:65 13.e 13:717a0223:::608.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0340 con 0x55cad4cb8ed0 2024-09-10T10:55:26.016 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::608.00000000:head# 2024-09-10T10:55:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.016+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 9 ==== osd_op_reply(65 608.00000000 [create] v139'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.016+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:66 13.e 13:717a0223:::608.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc1350 con 0x55cad4cb8ed0 2024-09-10T10:55:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.017+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 10 ==== osd_op_reply(66 608.00000000 [setxattr (30)] v139'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.017+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:67 13.e 13:717a0223:::608.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc1350 con 0x55cad4cb8ed0 2024-09-10T10:55:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.018+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 11 ==== osd_op_reply(67 608.00000000 [setxattr (62)] v139'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.018+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:68 13.e 13:717a0223:::608.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0340 con 0x55cad4cb8ed0 2024-09-10T10:55:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.019+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 12 ==== osd_op_reply(68 608.00000000 [omap-set-header] v139'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.019+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:69 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0340 con 0x55cad4cbc350 2024-09-10T10:55:26.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.021+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(69 604.00000000 [create] v139'426 uv420 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.021+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:70 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc06e0 con 0x55cad4cbc350 2024-09-10T10:55:26.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.022+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(70 604.00000000 [delete] v139'427 uv427 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.022+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:71 13.15 13:a93a17c2:::604.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0a80 con 0x55cad4cbc350 2024-09-10T10:55:26.024 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::604.00000000:head# 2024-09-10T10:55:26.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.023+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 7 ==== osd_op_reply(71 604.00000000 [create] v139'428 uv428 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.023+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:72 13.15 13:a93a17c2:::604.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc1e30 con 0x55cad4cbc350 2024-09-10T10:55:26.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.025+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 8 ==== osd_op_reply(72 604.00000000 [setxattr (30)] v139'429 uv429 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.025+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:73 13.15 13:a93a17c2:::604.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc1e30 con 0x55cad4cbc350 2024-09-10T10:55:26.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.026+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 9 ==== osd_op_reply(73 604.00000000 [setxattr (62)] v139'430 uv430 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.026+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:74 13.15 13:a93a17c2:::604.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.027+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 10 ==== osd_op_reply(74 604.00000000 [omap-set-header] v139'431 uv431 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.027+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:75 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.029+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 11 ==== osd_op_reply(75 500.00000000 [create] v139'432 uv425 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.029+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:76 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.030+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 12 ==== osd_op_reply(76 500.00000000 [delete] v139'433 uv433 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.030+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:77 13.15 13:aa448500:::500.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.032 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::500.00000000:head# 2024-09-10T10:55:26.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.031+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 13 ==== osd_op_reply(77 500.00000000 [create] v139'434 uv434 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.031+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:78 13.15 13:aa448500:::500.00000000:head [write 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.033+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 14 ==== osd_op_reply(78 500.00000000 [write 0~90] v139'435 uv435 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.033+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:79 13.15 13:aa448500:::500.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cbc350 2024-09-10T10:55:26.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.034+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 15 ==== osd_op_reply(79 500.00000000 [omap-set-header] v139'436 uv436 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.034+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:80 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cb8ed0 2024-09-10T10:55:26.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.036+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 13 ==== osd_op_reply(80 mds_snaptable [create] v139'193 uv192 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.036+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:81 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cb8ed0 2024-09-10T10:55:26.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.037+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 14 ==== osd_op_reply(81 mds_snaptable [delete] v139'194 uv194 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.037+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:82 13.d 13:b50e409b:::mds_snaptable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cb8ed0 2024-09-10T10:55:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.038+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 15 ==== osd_op_reply(82 mds_snaptable [create] v139'195 uv195 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.039 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds_snaptable:head# 2024-09-10T10:55:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.038+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:83 13.d 13:b50e409b:::mds_snaptable:head [write 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cb8ed0 2024-09-10T10:55:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.039+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 16 ==== osd_op_reply(83 mds_snaptable [write 0~70] v139'196 uv196 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.039+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:84 13.d 13:b50e409b:::mds_snaptable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc0e20 con 0x55cad4cb8ed0 2024-09-10T10:55:26.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.041+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.0 v2:172.21.15.28:6808/1098003867 17 ==== osd_op_reply(84 mds_snaptable [omap-set-header] v139'197 uv197 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f212c009830 con 0x55cad4cb8ed0 2024-09-10T10:55:26.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.041+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cad4cc14a0 0x55cad4cc3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.041+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:85 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc3e10 con 0x55cad4cc14a0 2024-09-10T10:55:26.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.041+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cad4cc14a0 0x55cad4cc3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.041+0000 7f213e800640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cad4cc14a0 0x55cad4cc3890 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.043+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(85 605.00000000 [create] v139'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.043+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:86 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.045+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== osd_op_reply(86 605.00000000 [delete] v139'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.045+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:87 13.1d 13:b871830b:::605.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.046+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 3 ==== osd_op_reply(87 605.00000000 [create] v139'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.047 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::605.00000000:head# 2024-09-10T10:55:26.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.046+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:88 13.1d 13:b871830b:::605.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc51f0 con 0x55cad4cc14a0 2024-09-10T10:55:26.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.047+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 4 ==== osd_op_reply(88 605.00000000 [setxattr (30)] v139'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.047+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc51f0 con 0x55cad4cc14a0 2024-09-10T10:55:26.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.048+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 5 ==== osd_op_reply(89 605.00000000 [setxattr (62)] v139'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.048+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:90 13.1d 13:b871830b:::605.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.050+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 6 ==== osd_op_reply(90 605.00000000 [omap-set-header] v139'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.050+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:91 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.051+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 7 ==== osd_op_reply(91 mds0_inotable [create] v139'386 uv380 ondisk = -17 ((17) File exists)) ==== 157+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.051+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:92 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.053+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 8 ==== osd_op_reply(92 mds0_inotable [delete] v139'387 uv387 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.053+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc41e0 con 0x55cad4cc14a0 2024-09-10T10:55:26.055 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_inotable:head# 2024-09-10T10:55:26.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.054+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 9 ==== osd_op_reply(93 mds0_inotable [create] v139'388 uv388 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.054+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [write 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cbfbb0 con 0x55cad4cc14a0 2024-09-10T10:55:26.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.056+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 10 ==== osd_op_reply(94 mds0_inotable [write 0~34] v139'389 uv389 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.056+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:95 13.13 13:c91d4a1d:::mds0_inotable:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc4690 con 0x55cad4cc14a0 2024-09-10T10:55:26.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.057+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 11 ==== osd_op_reply(95 mds0_inotable [omap-set-header] v139'390 uv390 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.057+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:96 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc4690 con 0x55cad4cc14a0 2024-09-10T10:55:26.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.059+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 12 ==== osd_op_reply(96 100.00000000 [create] v139'391 uv385 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.059+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:97 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc4690 con 0x55cad4cc14a0 2024-09-10T10:55:26.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.060+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 13 ==== osd_op_reply(97 100.00000000 [delete] v139'392 uv392 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.060+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:98 13.13 13:cd5a64a3:::100.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc4690 con 0x55cad4cc14a0 2024-09-10T10:55:26.062 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000:head# 2024-09-10T10:55:26.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.062+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 14 ==== osd_op_reply(98 100.00000000 [create] v139'393 uv393 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.062+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:99 13.13 13:cd5a64a3:::100.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc4690 con 0x55cad4cc14a0 2024-09-10T10:55:26.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.063+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 15 ==== osd_op_reply(99 100.00000000 [omap-set-header] v139'394 uv394 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.063+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:100 13.13 13:cd5a64a3:::100.00000000:head [omap-set-vals in=5194b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc7100 con 0x55cad4cc14a0 2024-09-10T10:55:26.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.065+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 16 ==== osd_op_reply(100 100.00000000 [omap-set-vals] v139'395 uv395 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.065+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:101 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.066+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 16 ==== osd_op_reply(101 mds0_sessionmap [create] v139'172 uv171 ondisk = -17 ((17) File exists)) ==== 159+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.066+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:102 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.067+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 17 ==== osd_op_reply(102 mds0_sessionmap [delete] v139'173 uv173 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.068+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:103 13.b 13:d0630e4c:::mds0_sessionmap:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.069+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 18 ==== osd_op_reply(103 mds0_sessionmap [create] v139'174 uv174 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.070 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::mds0_sessionmap:head# 2024-09-10T10:55:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.069+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:104 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.070+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 19 ==== osd_op_reply(104 mds0_sessionmap [omap-set-header] v139'175 uv175 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.070+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:105 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.071+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 20 ==== osd_op_reply(105 609.00000000 [create] v139'231 uv230 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.071+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:106 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.073+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 21 ==== osd_op_reply(106 609.00000000 [delete] v139'232 uv232 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.073+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:107 13.7 13:e0b41d06:::609.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.075 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::609.00000000:head# 2024-09-10T10:55:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.074+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 22 ==== osd_op_reply(107 609.00000000 [create] v139'233 uv233 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.074+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:108 13.7 13:e0b41d06:::609.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.075+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 23 ==== osd_op_reply(108 609.00000000 [setxattr (30)] v139'234 uv234 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.075+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:109 13.7 13:e0b41d06:::609.00000000:head [setxattr parent (62) in=68b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.076+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 24 ==== osd_op_reply(109 609.00000000 [setxattr (62)] v139'235 uv235 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.076+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:110 13.7 13:e0b41d06:::609.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc70c0 con 0x55cad4cbc350 2024-09-10T10:55:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.078+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 <== osd.3 v2:172.21.15.28:6816/2614769734 25 ==== osd_op_reply(110 609.00000000 [omap-set-header] v139'236 uv236 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f21380548c0 con 0x55cad4cbc350 2024-09-10T10:55:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.078+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cad4cc7080 0x55cad4cc77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.078+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:111 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc7080 2024-09-10T10:55:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.078+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cad4cc7080 0x55cad4cc77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.078+0000 7f213de00640 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cad4cc7080 0x55cad4cc77c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.080+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(111 400.00000000 [create] v139'196 uv195 ondisk = -17 ((17) File exists)) ==== 156+0+0 (crc 0 0 0) 0x7f212c023040 con 0x55cad4cc7080 2024-09-10T10:55:26.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.080+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:112 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc7080 2024-09-10T10:55:26.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.081+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== osd_op_reply(112 400.00000000 [delete] v139'197 uv197 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c023040 con 0x55cad4cc7080 2024-09-10T10:55:26.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.081+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:113 13.f 13:f1f69726:::400.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc7080 2024-09-10T10:55:26.083 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::400.00000000:head# 2024-09-10T10:55:26.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.082+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.2 v2:172.21.15.28:6824/1014290416 3 ==== osd_op_reply(113 400.00000000 [create] v139'198 uv198 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c023040 con 0x55cad4cc7080 2024-09-10T10:55:26.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.082+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:114 13.f 13:f1f69726:::400.00000000:head [write 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc7080 2024-09-10T10:55:26.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.083+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.2 v2:172.21.15.28:6824/1014290416 4 ==== osd_op_reply(114 400.00000000 [write 0~22] v139'199 uv199 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c023040 con 0x55cad4cc7080 2024-09-10T10:55:26.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.083+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:115 13.f 13:f1f69726:::400.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc7080 2024-09-10T10:55:26.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.084+0000 7f213de00640 1 -- 172.21.15.28:0/2464547227 <== osd.2 v2:172.21.15.28:6824/1014290416 5 ==== osd_op_reply(115 400.00000000 [omap-set-header] v139'200 uv200 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x7f212c023040 con 0x55cad4cc7080 2024-09-10T10:55:26.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.085+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:116 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.086+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 17 ==== osd_op_reply(116 100.00000000.inode [create] v139'480 uv472 ondisk = -17 ((17) File exists)) ==== 162+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.086+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:117 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.087+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 18 ==== osd_op_reply(117 100.00000000.inode [delete] v139'481 uv481 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.087+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:118 13.1f 13:fe07bba1:::100.00000000.inode:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.089 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::100.00000000.inode:head# 2024-09-10T10:55:26.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.089+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 19 ==== osd_op_reply(118 100.00000000.inode [create] v139'482 uv482 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.089+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:119 13.1f 13:fe07bba1:::100.00000000.inode:head [write 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc7da0 con 0x55cad4cc14a0 2024-09-10T10:55:26.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.090+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 20 ==== osd_op_reply(119 100.00000000.inode [write 0~582] v139'483 uv483 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.090+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:120 13.1f 13:fe07bba1:::100.00000000.inode:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc7da0 con 0x55cad4cc14a0 2024-09-10T10:55:26.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.091+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 21 ==== osd_op_reply(120 100.00000000.inode [omap-set-header] v139'484 uv484 ondisk = 0) ==== 162+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.091+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:121 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc7d50 con 0x55cad4cc14a0 2024-09-10T10:55:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.092+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 22 ==== osd_op_reply(121 1.00000000 [create] v139'485 uv479 ondisk = -17 ((17) File exists)) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.092+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:122 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.093+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 23 ==== osd_op_reply(122 1.00000000 [delete] v139'486 uv486 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.093+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:123 13.1f 13:ff5b34d6:::1.00000000:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.095 INFO:teuthology.orchestra.run.smithi028.stdout:***Overwrite*** #-9223372036854775808:00000000:::1.00000000:head# 2024-09-10T10:55:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.095+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 24 ==== osd_op_reply(123 1.00000000 [create] v139'487 uv487 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.095+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:124 13.1f 13:ff5b34d6:::1.00000000:head [setxattr layout (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.096+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 25 ==== osd_op_reply(124 1.00000000 [setxattr (30)] v139'488 uv488 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.096+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:125 13.1f 13:ff5b34d6:::1.00000000:head [setxattr parent (30) in=36b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.097+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 26 ==== osd_op_reply(125 1.00000000 [setxattr (30)] v139'489 uv489 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.097+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:126 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc6700 con 0x55cad4cc14a0 2024-09-10T10:55:26.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.098+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 27 ==== osd_op_reply(126 1.00000000 [omap-set-header] v139'490 uv490 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.098+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:127 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=584b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55cad4cc81d0 con 0x55cad4cc14a0 2024-09-10T10:55:26.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.099+0000 7f213e800640 1 -- 172.21.15.28:0/2464547227 <== osd.1 v2:172.21.15.28:6800/2223305111 28 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v139'491 uv491 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f2130050050 con 0x55cad4cc14a0 2024-09-10T10:55:26.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cad4cb8ed0 msgr2=0x55cad4cb9310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x55cad4cb8ed0 0x55cad4cb9310 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cad4cc14a0 msgr2=0x55cad4cc3890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55cad4cc14a0 0x55cad4cc3890 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cad4cc7080 msgr2=0x55cad4cc77c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x55cad4cc7080 0x55cad4cc77c0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cad4cbc350 msgr2=0x55cad4cbc7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x55cad4cbc350 0x55cad4cbc7e0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cad4cb34b0 msgr2=0x55cad4cb65b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x55cad4cb34b0 0x55cad4cb65b0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cad4c11f50 msgr2=0x55cad4c12330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x55cad4c11f50 0x55cad4c12330 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 reap_dead start 2024-09-10T10:55:26.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 reap_dead start 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 reap_dead start 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f213f200640 1 -- 172.21.15.28:0/2464547227 reap_dead start 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 msgr2=0x7f212006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 0x7f212006ccd0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2130045ae0 tx=0x7f2130053000 comp rx=0 tx=0).stop 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 msgr2=0x55cad4bff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f213800e490 tx=0x7f213800e950 comp rx=0 tx=0).stop 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 shutdown_connections 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f212006a810 0x7f212006ccd0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 --2- 172.21.15.28:0/2464547227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55cad4c109c0 0x55cad4bff490 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 >> 172.21.15.28:0/2464547227 conn(0x55cad4b860a0 msgr2=0x55cad4b37070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 shutdown_connections 2024-09-10T10:55:26.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.100+0000 7f21416b0f40 1 -- 172.21.15.28:0/2464547227 wait complete. 2024-09-10T10:55:26.105 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapheader 1.00000000 deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T10:55:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.223+0000 7f3f9eb94f40 1 Processor -- start 2024-09-10T10:55:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.223+0000 7f3f9eb94f40 1 -- start start 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c71a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 0x55d5c71a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5c70ce6f0 con 0x55d5c70ce2a0 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5c7076b30 con 0x55d5c70fefd0 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d5c70feb40 con 0x55d5c7116770 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49502/0 (socket says 172.21.15.28:49502) 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 -- 172.21.15.28:0/3146048865 learned_addr learned my addr 172.21.15.28:0/3146048865 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:26.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9b400640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c71a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9c800640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 0x55d5c71a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 -- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 msgr2=0x55d5c71a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 0x55d5c71a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 -- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 msgr2=0x55d5c71a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.224+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c71a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9be00640 1 -- 172.21.15.28:0/3146048865 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c71a8350 con 0x55d5c70ce2a0 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9c800640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 0x55d5c71a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c001ce0 tx=0x7f3f8c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd7974688bc234c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9a000640 1 -- 172.21.15.28:0/3146048865 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00bd80 con 0x55d5c70ce2a0 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9a000640 1 -- 172.21.15.28:0/3146048865 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f8c00f400 con 0x55d5c70ce2a0 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.225+0000 7f3f9a000640 1 -- 172.21.15.28:0/3146048865 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c00f830 con 0x55d5c70ce2a0 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3146048865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 msgr2=0x55d5c70cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c001ce0 tx=0x7f3f8c00b5b0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3146048865 shutdown_connections 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c7116770 0x55d5c71a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c71a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3146048865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c70ce2a0 0x55d5c70cf250 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3146048865 >> 172.21.15.28:0/3146048865 conn(0x55d5c70cee70 msgr2=0x55d5c70d1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3146048865 shutdown_connections 2024-09-10T10:55:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3146048865 wait complete. 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.226+0000 7f3f9eb94f40 1 Processor -- start 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 -- start start 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c70ce2a0 0x55d5c71a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c70d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 0x55d5c70d1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5c70fa800 con 0x55d5c7116770 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5c70f91f0 con 0x55d5c70fefd0 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.227+0000 7f3f9eb94f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55d5c70f9440 con 0x55d5c70ce2a0 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 0x55d5c70d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 0x55d5c70d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49514/0 (socket says 172.21.15.28:49514) 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 -- 172.21.15.28:0/3701382150 learned_addr learned my addr 172.21.15.28:0/3701382150 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c70ce2a0 0x55d5c71a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9b400640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c70d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 -- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c70ce2a0 msgr2=0x55d5c71a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55d5c70ce2a0 0x55d5c71a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 -- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 msgr2=0x55d5c70d10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5c70fefd0 0x55d5c70d10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 -- 172.21.15.28:0/3701382150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5c7230bd0 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.228+0000 7f3f9c800640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 0x55d5c70d1610 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e460 tx=0x7f3f9400e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.229+0000 7f3f98c00640 1 -- 172.21.15.28:0/3701382150 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94017020 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.229+0000 7f3f98c00640 1 -- 172.21.15.28:0/3701382150 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f94013890 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.229+0000 7f3f98c00640 1 -- 172.21.15.28:0/3701382150 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94013c80 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.229+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d5c7186050 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.229+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5c7230eb0 con 0x55d5c7116770 2024-09-10T10:55:26.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.231+0000 7f3f98c00640 1 -- 172.21.15.28:0/3701382150 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f9401a020 con 0x55d5c7116770 2024-09-10T10:55:26.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f98c00640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f7c06a810 0x7f3f7c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f98c00640 1 -- 172.21.15.28:0/3701382150 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f3f9408b1e0 con 0x55d5c7116770 2024-09-10T10:55:26.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5c7188e40 0x55d5c7232e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f7c06a810 0x7f3f7c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:1 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-header in=80b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e139) -- 0x55d5c7116be0 con 0x55d5c7188e40 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.232+0000 7f3f9b400640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5c7188e40 0x55d5c7232e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.233+0000 7f3f9be00640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f7c06a810 0x7f3f7c06ccd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00b5f0 tx=0x7f3f8c009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.233+0000 7f3f9b400640 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5c7188e40 0x55d5c7232e20 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.234+0000 7f3f9b400640 1 -- 172.21.15.28:0/3701382150 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(1 1.00000000 [omap-set-header] v139'492 uv492 ondisk = 0) ==== 154+0+0 (crc 0 0 0) 0x7f3f88058950 con 0x55d5c7188e40 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.234+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5c7188e40 msgr2=0x55d5c7232e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.234+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x55d5c7188e40 0x55d5c7232e20 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f7c06a810 msgr2=0x7f3f7c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3f7c06a810 0x7f3f7c06ccd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00b5f0 tx=0x7f3f8c009000 comp rx=0 tx=0).stop 2024-09-10T10:55:26.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 msgr2=0x55d5c70d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 --2- 172.21.15.28:0/3701382150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55d5c7116770 0x55d5c70d1610 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e460 tx=0x7f3f9400e920 comp rx=0 tx=0).stop 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9c800640 1 -- 172.21.15.28:0/3701382150 reap_dead start 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 shutdown_connections 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 >> 172.21.15.28:0/3701382150 conn(0x55d5c70cee70 msgr2=0x55d5c71a8520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 shutdown_connections 2024-09-10T10:55:26.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.235+0000 7f3f9eb94f40 1 -- 172.21.15.28:0/3701382150 wait complete. 2024-09-10T10:55:26.240 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:55:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.399+0000 7f6f2b200640 1 Processor -- start 2024-09-10T10:55:26.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.399+0000 7f6f2b200640 1 -- start start 2024-09-10T10:55:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.400+0000 7f6f2b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f2c161ac0 0x7f6f2c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.400+0000 7f6f2b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f2c0b1170 con 0x7f6f2c0d2ac0 2024-09-10T10:55:26.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.400+0000 7f6f2b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f2c0b1450 con 0x7f6f2c0b3640 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.400+0000 7f6f2b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6f2c0ceba0 con 0x7f6f2c161ac0 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f29e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f2c161ac0 0x7f6f2c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f2a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6f2c0d2ac0 0x7f6f2c0b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59780/0 (socket says 172.21.15.28:59780) 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f2a800640 1 -- 172.21.15.28:0/1945901395 learned_addr learned my addr 172.21.15.28:0/1945901395 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f29e00640 1 -- 172.21.15.28:0/1945901395 >> v1:172.21.15.138:6789/0 conn(0x7f6f2c0b3640 legacy=0x7f6f2c0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f29e00640 1 -- 172.21.15.28:0/1945901395 >> v1:172.21.15.28:6789/0 conn(0x7f6f2c0d2ac0 legacy=0x7f6f2c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.401+0000 7f6f29e00640 1 -- 172.21.15.28:0/1945901395 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c0b1730 con 0x7f6f2c161ac0 2024-09-10T10:55:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.402+0000 7f6f29e00640 1 --2- 172.21.15.28:0/1945901395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c161ac0 0x7f6f2c06e6c0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0097a0 tx=0x7f6f1c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e651f5660447dfdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.402+0000 7f6f29400640 1 -- 172.21.15.28:0/1945901395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c037070 con 0x7f6f2c161ac0 2024-09-10T10:55:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.402+0000 7f6f29400640 1 -- 172.21.15.28:0/1945901395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1c032640 con 0x7f6f2c161ac0 2024-09-10T10:55:26.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.402+0000 7f6f29400640 1 -- 172.21.15.28:0/1945901395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c032a10 con 0x7f6f2c161ac0 2024-09-10T10:55:26.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 -- 172.21.15.28:0/1945901395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c161ac0 msgr2=0x7f6f2c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 --2- 172.21.15.28:0/1945901395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c161ac0 0x7f6f2c06e6c0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0097a0 tx=0x7f6f1c02a2c0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 -- 172.21.15.28:0/1945901395 shutdown_connections 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 --2- 172.21.15.28:0/1945901395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c161ac0 0x7f6f2c06e6c0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 -- 172.21.15.28:0/1945901395 >> 172.21.15.28:0/1945901395 conn(0x7f6f2c08bfc0 msgr2=0x7f6f2c0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 -- 172.21.15.28:0/1945901395 shutdown_connections 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.403+0000 7f6f2b200640 1 -- 172.21.15.28:0/1945901395 wait complete. 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.405+0000 7f6f2b200640 1 Processor -- start 2024-09-10T10:55:26.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.405+0000 7f6f2b200640 1 -- start start 2024-09-10T10:55:26.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.405+0000 7f6f2b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 0x7f6f2c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f2c161ac0 0x7f6f2c1e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c161ea0 con 0x7f6f2c0b3640 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c06dfb0 con 0x7f6f2c0d2ac0 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2c162020 con 0x7f6f2c161ac0 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49530/0 (socket says 172.21.15.28:49530) 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 -- 172.21.15.28:0/2613661720 learned_addr learned my addr 172.21.15.28:0/2613661720 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f2a800640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f2c161ac0 0x7f6f2c1e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f21400640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 0x7f6f2c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 -- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f2c161ac0 msgr2=0x7f6f2c1e8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f2c161ac0 0x7f6f2c1e8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 -- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 msgr2=0x7f6f2c15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 0x7f6f2c15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f29e00640 1 -- 172.21.15.28:0/2613661720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c1569c0 con 0x7f6f2c0b3640 2024-09-10T10:55:26.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.406+0000 7f6f21400640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 0x7f6f2c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:26.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f29e00640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0098c0 tx=0x7f6f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c041030 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1c030750 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c037040 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2c156c30 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.407+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2c1571f0 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.408+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2c06e6c0 con 0x7f6f2c0b3640 2024-09-10T10:55:26.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.409+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f1c044070 con 0x7f6f2c0b3640 2024-09-10T10:55:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.410+0000 7f6f22a00640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 0x7f6f0806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.410+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f6f1c0ac5a0 con 0x7f6f2c0b3640 2024-09-10T10:55:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.410+0000 7f6f21400640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 0x7f6f0806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.411+0000 7f6f21400640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 0x7f6f0806e320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c160ff0 tx=0x7f6f14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.414+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f1c079f30 con 0x7f6f2c0b3640 2024-09-10T10:55:26.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.635+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f6f2c161020 con 0x7f6f2c0b3640 2024-09-10T10:55:26.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.717+0000 7f6f22a00640 1 -- 172.21.15.28:0/2613661720 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v327) ==== 162+0+0 (secure 0 0 0) 0x7f6f1c079830 con 0x7f6f2c0b3640 2024-09-10T10:55:26.718 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 msgr2=0x7f6f0806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 0x7f6f0806e320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c160ff0 tx=0x7f6f14009040 comp rx=0 tx=0).stop 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 msgr2=0x7f6f2c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c0098c0 tx=0x7f6f1c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 shutdown_connections 2024-09-10T10:55:26.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6f0806be60 0x7f6f0806e320 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f2c161ac0 0x7f6f2c1e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f2c0d2ac0 0x7f6f2c15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 --2- 172.21.15.28:0/2613661720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f2c0b3640 0x7f6f2c0cdaf0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 >> 172.21.15.28:0/2613661720 conn(0x7f6f2c08bfc0 msgr2=0x7f6f2c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 shutdown_connections 2024-09-10T10:55:26.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.720+0000 7f6f2b200640 1 -- 172.21.15.28:0/2613661720 wait complete. 2024-09-10T10:55:26.739 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:26.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.896+0000 7f8c95000640 1 Processor -- start 2024-09-10T10:55:26.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.896+0000 7f8c95000640 1 -- start start 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c95000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c9015bef0 0x7f8c900b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c95000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900b25c0 0x7f8c900aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c95000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c900b5470 con 0x7f8c900af1b0 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c95000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8c900b3640 con 0x7f8c9015bef0 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c95000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c900b5190 con 0x7f8c900b25c0 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8c900af1b0 0x7f8c90071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59788/0 (socket says 172.21.15.28:59788) 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c9015bef0 0x7f8c900b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8fe00640 1 -- 172.21.15.28:0/4020565292 learned_addr learned my addr 172.21.15.28:0/4020565292 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:26.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c87e00640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900b25c0 0x7f8c900aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8f400640 1 -- 172.21.15.28:0/4020565292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900b25c0 msgr2=0x7f8c900aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8f400640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900b25c0 0x7f8c900aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8f400640 1 -- 172.21.15.28:0/4020565292 >> v1:172.21.15.28:6789/0 conn(0x7f8c900af1b0 legacy=0x7f8c90071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.898+0000 7f8c8f400640 1 -- 172.21.15.28:0/4020565292 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c900b38e0 con 0x7f8c9015bef0 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.899+0000 7f8c8f400640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c9015bef0 0x7f8c900b40e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097a0 tx=0x7f8c8002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ed1a5dad0acff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c8ea00640 1 -- 172.21.15.28:0/4020565292 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80037070 con 0x7f8c9015bef0 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c8ea00640 1 -- 172.21.15.28:0/4020565292 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c8002ae50 con 0x7f8c9015bef0 2024-09-10T10:55:26.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c8ea00640 1 -- 172.21.15.28:0/4020565292 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80032660 con 0x7f8c9015bef0 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 -- 172.21.15.28:0/4020565292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c9015bef0 msgr2=0x7f8c900b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c9015bef0 0x7f8c900b40e0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8c800097a0 tx=0x7f8c8002a100 comp rx=0 tx=0).stop 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 -- 172.21.15.28:0/4020565292 shutdown_connections 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900b25c0 0x7f8c900aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 --2- 172.21.15.28:0/4020565292 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c9015bef0 0x7f8c900b40e0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 -- 172.21.15.28:0/4020565292 >> 172.21.15.28:0/4020565292 conn(0x7f8c9006c240 msgr2=0x7f8c900b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:26.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 -- 172.21.15.28:0/4020565292 shutdown_connections 2024-09-10T10:55:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.900+0000 7f8c95000640 1 -- 172.21.15.28:0/4020565292 wait complete. 2024-09-10T10:55:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.902+0000 7f8c95000640 1 Processor -- start 2024-09-10T10:55:26.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.902+0000 7f8c95000640 1 -- start start 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 0x7f8c90152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9015bef0 0x7f8c901f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900b4e60 con 0x7f8c900b25c0 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c900af640 con 0x7f8c9015bef0 2024-09-10T10:55:26.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c95000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c900b4b10 con 0x7f8c900af1b0 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c87e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49546/0 (socket says 172.21.15.28:49546) 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c87e00640 1 -- 172.21.15.28:0/1996763957 learned_addr learned my addr 172.21.15.28:0/1996763957 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c8fe00640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9015bef0 0x7f8c901f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.903+0000 7f8c8f400640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 0x7f8c90152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 -- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 msgr2=0x7f8c90152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 0x7f8c90152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 -- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9015bef0 msgr2=0x7f8c901f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9015bef0 0x7f8c901f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 -- 172.21.15.28:0/1996763957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c901f47f0 con 0x7f8c900b25c0 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c8f400640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 0x7f8c90152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:26.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87e00640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d800 tx=0x7f8c7800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800b6c0 con 0x7f8c900b25c0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c78019070 con 0x7f8c900b25c0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800ba80 con 0x7f8c900b25c0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c901f4ad0 con 0x7f8c900b25c0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.904+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c9006df50 con 0x7f8c900b25c0 2024-09-10T10:55:26.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.905+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c900b2a00 con 0x7f8c900b25c0 2024-09-10T10:55:26.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.906+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c7800bc20 con 0x7f8c900b25c0 2024-09-10T10:55:26.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.907+0000 7f8c87400640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 0x7f8c6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:26.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.907+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 9658+0+0 (secure 0 0 0) 0x7f8c7808c260 con 0x7f8c900b25c0 2024-09-10T10:55:26.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.908+0000 7f8c8f400640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 0x7f8c6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:26.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.909+0000 7f8c8f400640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 0x7f8c6c06e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8c80034000 tx=0x7f8c8002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:26.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:26.912+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c78059c80 con 0x7f8c900b25c0 2024-09-10T10:55:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.133+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c900b4340 con 0x7f8c900b25c0 2024-09-10T10:55:27.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.134+0000 7f8c87400640 1 -- 172.21.15.28:0/1996763957 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 327 v327) ==== 95+0+4398 (secure 0 0 0) 0x7f8c78059580 con 0x7f8c900b25c0 2024-09-10T10:55:27.135 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:27.135 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":327,"btime":"2024-09-10T10:55:26:716319+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":327,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:26.716307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":139,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9022":{"gid":9022,"name":"a","rank":0,"incarnation":327,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/911593499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":911593499},{"type":"v1","addr":"172.21.15.28:6835","nonce":911593499}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 327 2024-09-10T10:55:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 msgr2=0x7f8c6c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 0x7f8c6c06e2f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f8c80034000 tx=0x7f8c8002e000 comp rx=0 tx=0).stop 2024-09-10T10:55:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 msgr2=0x7f8c9006e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800d800 tx=0x7f8c7800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 shutdown_connections 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8c6c06be30 0x7f8c6c06e2f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c9015bef0 0x7f8c901f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c900b25c0 0x7f8c9006e750 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 --2- 172.21.15.28:0/1996763957 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c900af1b0 0x7f8c90152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 >> 172.21.15.28:0/1996763957 conn(0x7f8c9006c240 msgr2=0x7f8c90158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 shutdown_connections 2024-09-10T10:55:27.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:27.136+0000 7f8c95000640 1 -- 172.21.15.28:0/1996763957 wait complete. 2024-09-10T10:55:27.153 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-09-10T10:55:27.545 INFO:tasks.ceph.mds.c.smithi028.stderr: -45> 2024-09-10T10:55:22.792+0000 7f0e3ca00640 -1 log_channel(cluster) log [ERR] : corrupt sessionmap header: Decoder at 'virtual void SessionMapStore::decode_header(ceph::bufferlist&)' v=1 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:27.585 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:55:29.728 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:55:29.727+0000 7f7e8ee00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:29.728 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:55:29.727+0000 7f7e8ee00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T10:55:29.729 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:29.728+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:55:29.729 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:55:29.728+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2024-09-10T10:55:32.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:32.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.324+0000 7f1ea8400640 1 Processor -- start 2024-09-10T10:55:32.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.324+0000 7f1ea8400640 1 -- start start 2024-09-10T10:55:32.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.325+0000 7f1ea8400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ea0072de0 0x7f1ea015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.325+0000 7f1ea8400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea0146140 con 0x7f1ea0072320 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.325+0000 7f1ea8400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea00cd740 con 0x7f1ea0072700 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.325+0000 7f1ea8400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea00cda20 con 0x7f1ea0072de0 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1ea0072320 0x7f1ea00cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59804/0 (socket says 172.21.15.28:59804) 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5a00640 1 -- 172.21.15.28:0/2008871425 learned_addr learned my addr 172.21.15.28:0/2008871425 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:32.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5000640 1 --2- 172.21.15.28:0/2008871425 >> v2:172.21.15.28:3300/0 conn(0x7f1ea0072de0 0x7f1ea015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1e9fe00640 1 -- 172.21.15.28:0/2008871425 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196290489 0 0) 0x7f1ea0146140 con 0x7f1ea0072320 2024-09-10T10:55:32.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1e9fe00640 1 -- 172.21.15.28:0/2008871425 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e88003520 con 0x7f1ea0072320 2024-09-10T10:55:32.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5000640 1 -- 172.21.15.28:0/2008871425 >> v1:172.21.15.138:6789/0 conn(0x7f1ea0072700 legacy=0x7f1ea015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5000640 1 -- 172.21.15.28:0/2008871425 >> v1:172.21.15.28:6789/0 conn(0x7f1ea0072320 legacy=0x7f1ea00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.326+0000 7f1ea5000640 1 -- 172.21.15.28:0/2008871425 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00cdd00 con 0x7f1ea0072de0 2024-09-10T10:55:32.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.327+0000 7f1ea5000640 1 --2- 172.21.15.28:0/2008871425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea015d1e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7fc63bbd62d2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.327+0000 7f1e9fe00640 1 -- 172.21.15.28:0/2008871425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c037070 con 0x7f1ea0072de0 2024-09-10T10:55:32.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1e9fe00640 1 -- 172.21.15.28:0/2008871425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e8c032440 con 0x7f1ea0072de0 2024-09-10T10:55:32.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1e9fe00640 1 -- 172.21.15.28:0/2008871425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c032810 con 0x7f1ea0072de0 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 -- 172.21.15.28:0/2008871425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 msgr2=0x7f1ea015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 --2- 172.21.15.28:0/2008871425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea015d1e0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0097a0 tx=0x7f1e8c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 -- 172.21.15.28:0/2008871425 shutdown_connections 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 --2- 172.21.15.28:0/2008871425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea015d1e0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 -- 172.21.15.28:0/2008871425 >> 172.21.15.28:0/2008871425 conn(0x7f1ea0067d70 msgr2=0x7f1ea006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 -- 172.21.15.28:0/2008871425 shutdown_connections 2024-09-10T10:55:32.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.328+0000 7f1ea8400640 1 -- 172.21.15.28:0/2008871425 wait complete. 2024-09-10T10:55:32.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.330+0000 7f1ea8400640 1 Processor -- start 2024-09-10T10:55:32.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.330+0000 7f1ea8400640 1 -- start start 2024-09-10T10:55:32.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea0072320 0x7f1ea0142a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ea0072700 0x7f1ea00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea015dfb0 con 0x7f1ea0072de0 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea015de30 con 0x7f1ea0072320 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea8400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea015e130 con 0x7f1ea0072700 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49572/0 (socket says 172.21.15.28:49572) 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea6400640 1 -- 172.21.15.28:0/1722042648 learned_addr learned my addr 172.21.15.28:0/1722042648 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea5a00640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea0072320 0x7f1ea0142a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.331+0000 7f1ea5000640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ea0072700 0x7f1ea00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 -- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ea0072700 msgr2=0x7f1ea00b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ea0072700 0x7f1ea00b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 -- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea0072320 msgr2=0x7f1ea0142a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea0072320 0x7f1ea0142a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 -- 172.21.15.28:0/1722042648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea00b40d0 con 0x7f1ea0072de0 2024-09-10T10:55:32.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea6400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c730 tx=0x7f1e9800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98016020 con 0x7f1ea0072de0 2024-09-10T10:55:32.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e98006d50 con 0x7f1ea0072de0 2024-09-10T10:55:32.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea00b43b0 con 0x7f1ea0072de0 2024-09-10T10:55:32.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e98003bb0 con 0x7f1ea0072de0 2024-09-10T10:55:32.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.332+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea01f4420 con 0x7f1ea0072de0 2024-09-10T10:55:32.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.333+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea00d0130 con 0x7f1ea0072de0 2024-09-10T10:55:32.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.334+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1e9800a7f0 con 0x7f1ea0072de0 2024-09-10T10:55:32.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.335+0000 7f1e9d600640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.335+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f1e9808b930 con 0x7f1ea0072de0 2024-09-10T10:55:32.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.336+0000 7f1ea5a00640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.336+0000 7f1ea5a00640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 0x7f1e8006e2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e940011e0 tx=0x7f1e94007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.340+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1ea00d0130 con 0x7f1ea0072de0 2024-09-10T10:55:32.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.562+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea00cce60 con 0x7f1ea0072de0 2024-09-10T10:55:32.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.563+0000 7f1e9d600640 1 -- 172.21.15.28:0/1722042648 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4390 (secure 0 0 0) 0x7f1e98020020 con 0x7f1ea0072de0 2024-09-10T10:55:32.564 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:32.564 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":331,"btime":"2024-09-10T10:55:29:729465+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:32.564 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 331 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 msgr2=0x7f1e8006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 0x7f1e8006e2f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e940011e0 tx=0x7f1e94007d30 comp rx=0 tx=0).stop 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 msgr2=0x7f1ea01f4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f1e9800c730 tx=0x7f1e9800cbf0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 shutdown_connections 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1e8006be30 0x7f1e8006e2f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ea0072de0 0x7f1ea01f4c30 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ea0072700 0x7f1ea00b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 --2- 172.21.15.28:0/1722042648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ea0072320 0x7f1ea0142a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 >> 172.21.15.28:0/1722042648 conn(0x7f1ea0067d70 msgr2=0x7f1ea015d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:32.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 shutdown_connections 2024-09-10T10:55:32.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.565+0000 7f1ea8400640 1 -- 172.21.15.28:0/1722042648 wait complete. 2024-09-10T10:55:32.582 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-10T10:55:32.582 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:32.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.738+0000 7f40cbe00640 1 Processor -- start 2024-09-10T10:55:32.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.738+0000 7f40cbe00640 1 -- start start 2024-09-10T10:55:32.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40cbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f40cc15e6a0 0x7f40cc06d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc0cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40cbe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40cc0d2c20 con 0x7f40cc0d4180 2024-09-10T10:55:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40cbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f40cc0cefb0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40cbe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc06e390 con 0x7f40cc06dd60 2024-09-10T10:55:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40caa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f40cc0d4180 0x7f40cc0d4560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:59812/0 (socket says 172.21.15.28:59812) 2024-09-10T10:55:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40caa00640 1 -- 172.21.15.28:0/3434852646 learned_addr learned my addr 172.21.15.28:0/3434852646 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:32.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.739+0000 7f40ca000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f40cc15e6a0 0x7f40cc06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40cb400640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc0cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40c9600640 1 -- 172.21.15.28:0/3434852646 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146791021 0 0) 0x7f40cc0d2c20 con 0x7f40cc0d4180 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40ca000640 1 -- 172.21.15.28:0/3434852646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 msgr2=0x7f40cc0cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40ca000640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc0cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40ca000640 1 -- 172.21.15.28:0/3434852646 >> v1:172.21.15.28:6789/0 conn(0x7f40cc0d4180 legacy=0x7f40cc0d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.740+0000 7f40ca000640 1 -- 172.21.15.28:0/3434852646 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc06e670 con 0x7f40cc15e6a0 2024-09-10T10:55:32.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.741+0000 7f40ca000640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc06d820 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f40b80097a0 tx=0x7f40b802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b024da574664a5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.743+0000 7f40c9600640 1 -- 172.21.15.28:0/3434852646 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8037070 con 0x7f40cc15e6a0 2024-09-10T10:55:32.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.743+0000 7f40c9600640 1 -- 172.21.15.28:0/3434852646 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b802ae50 con 0x7f40cc15e6a0 2024-09-10T10:55:32.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.743+0000 7f40c9600640 1 -- 172.21.15.28:0/3434852646 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8032660 con 0x7f40cc15e6a0 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 -- 172.21.15.28:0/3434852646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 msgr2=0x7f40cc06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc06d820 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f40b80097a0 tx=0x7f40b802a100 comp rx=0 tx=0).stop 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 -- 172.21.15.28:0/3434852646 shutdown_connections 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc0cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 --2- 172.21.15.28:0/3434852646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc06d820 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 -- 172.21.15.28:0/3434852646 >> 172.21.15.28:0/3434852646 conn(0x7f40cc09d6f0 msgr2=0x7f40cc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 -- 172.21.15.28:0/3434852646 shutdown_connections 2024-09-10T10:55:32.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.744+0000 7f40cbe00640 1 -- 172.21.15.28:0/3434852646 wait complete. 2024-09-10T10:55:32.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.746+0000 7f40cbe00640 1 Processor -- start 2024-09-10T10:55:32.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.746+0000 7f40cbe00640 1 -- start start 2024-09-10T10:55:32.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40cc0d4180 0x7f40cc15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0cf980 con 0x7f40cc15e6a0 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc0cebd0 con 0x7f40cc0d4180 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cbe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc0cfb00 con 0x7f40cc06dd60 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49594/0 (socket says 172.21.15.28:49594) 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 -- 172.21.15.28:0/2016943658 learned_addr learned my addr 172.21.15.28:0/2016943658 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40caa00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40ca000640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40cc0d4180 0x7f40cc15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 -- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 msgr2=0x7f40cc155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 -- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40cc0d4180 msgr2=0x7f40cc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40cc0d4180 0x7f40cc15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.747+0000 7f40cb400640 1 -- 172.21.15.28:0/2016943658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc1f79f0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40cb400640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f40c4007e40 tx=0x7f40c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4014620 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40c4019070 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40cc1f7cd0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4014a20 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.748+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40cc1f8290 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.749+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40cc0cf420 con 0x7f40cc15e6a0 2024-09-10T10:55:32.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.750+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f40c4014bc0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.751+0000 7f40c2a00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:32.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.751+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f40c408c6a0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.752+0000 7f40caa00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:32.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.752+0000 7f40caa00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 0x7f40ac06e2f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f40bc000f60 tx=0x7f40bc007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:32.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.755+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40c40599b0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.864 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:55:32.863+0000 7f7e93e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:32.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.979+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40cc0b2ce0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.980+0000 7f40c2a00640 1 -- 172.21.15.28:0/2016943658 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4390 (secure 0 0 0) 0x7f40c405dcb0 con 0x7f40cc15e6a0 2024-09-10T10:55:32.981 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:32.981 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":331,"btime":"2024-09-10T10:55:29:729465+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:32.981 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 331 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 msgr2=0x7f40ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 0x7f40ac06e2f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f40bc000f60 tx=0x7f40bc007d50 comp rx=0 tx=0).stop 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 msgr2=0x7f40cc15d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f40c4007e40 tx=0x7f40c400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 shutdown_connections 2024-09-10T10:55:32.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f40ac06be30 0x7f40ac06e2f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f40cc15e6a0 0x7f40cc15d830 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40cc0d4180 0x7f40cc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 --2- 172.21.15.28:0/2016943658 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f40cc06dd60 0x7f40cc155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 >> 172.21.15.28:0/2016943658 conn(0x7f40cc09d6f0 msgr2=0x7f40cc0b79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 shutdown_connections 2024-09-10T10:55:32.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:32.982+0000 7f40cbe00640 1 -- 172.21.15.28:0/2016943658 wait complete. 2024-09-10T10:55:32.999 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T10:55:32.999 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.154+0000 7f6a13400640 1 Processor -- start 2024-09-10T10:55:33.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.154+0000 7f6a13400640 1 -- start start 2024-09-10T10:55:33.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a13400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a13400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c0ae550 0x7f6a0c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a13400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0c06cb30 con 0x7f6a0c147360 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a13400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6a0c0d0e50 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a13400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c06bfb0 con 0x7f6a0c0ae550 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49596/0 (socket says 172.21.15.28:49596) 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 -- 172.21.15.28:0/534480471 learned_addr learned my addr 172.21.15.28:0/534480471 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a11400640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c0ae550 0x7f6a0c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 -- 172.21.15.28:0/534480471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c0ae550 msgr2=0x7f6a0c0ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c0ae550 0x7f6a0c0ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 -- 172.21.15.28:0/534480471 >> v1:172.21.15.138:6789/0 conn(0x7f6a0c147360 legacy=0x7f6a0c070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.156+0000 7f6a0be00640 1 -- 172.21.15.28:0/534480471 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c06c290 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.157+0000 7f6a0be00640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009870 tx=0x7f69f802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f524a6779dcb12a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a0b400640 1 -- 172.21.15.28:0/534480471 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8037070 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a0b400640 1 -- 172.21.15.28:0/534480471 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f8032440 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a0b400640 1 -- 172.21.15.28:0/534480471 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8032810 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 -- 172.21.15.28:0/534480471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0d04a0 msgr2=0x7f6a0c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009870 tx=0x7f69f802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 -- 172.21.15.28:0/534480471 shutdown_connections 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c0ae550 0x7f6a0c0ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 --2- 172.21.15.28:0/534480471 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c0d0910 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 -- 172.21.15.28:0/534480471 >> 172.21.15.28:0/534480471 conn(0x7f6a0c0b2f40 msgr2=0x7f6a0c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 -- 172.21.15.28:0/534480471 shutdown_connections 2024-09-10T10:55:33.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.158+0000 7f6a13400640 1 -- 172.21.15.28:0/534480471 wait complete. 2024-09-10T10:55:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.160+0000 7f6a13400640 1 Processor -- start 2024-09-10T10:55:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.160+0000 7f6a13400640 1 -- start start 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c15c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 0x7f6a0c144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c070fa0 con 0x7f6a0c0ae550 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c06c850 con 0x7f6a0c0d04a0 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a13400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0c06bc10 con 0x7f6a0c147360 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49598/0 (socket says 172.21.15.28:49598) 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a10a00640 1 -- 172.21.15.28:0/349671070 learned_addr learned my addr 172.21.15.28:0/349671070 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a0be00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c15c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.161+0000 7f6a11400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 0x7f6a0c144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 -- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 msgr2=0x7f6a0c144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 0x7f6a0c144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 -- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 msgr2=0x7f6a0c15c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c15c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 -- 172.21.15.28:0/349671070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c141220 con 0x7f6a0c0ae550 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a0be00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c15c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a11400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 0x7f6a0c144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a10a00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c3c0 tx=0x7f6a0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000052d0 con 0x7f6a0c0ae550 2024-09-10T10:55:33.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a00005490 con 0x7f6a0c0ae550 2024-09-10T10:55:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c15f1d0 con 0x7f6a0c0ae550 2024-09-10T10:55:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00007260 con 0x7f6a0c0ae550 2024-09-10T10:55:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.162+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c143680 con 0x7f6a0c0ae550 2024-09-10T10:55:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.163+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c0d0910 con 0x7f6a0c0ae550 2024-09-10T10:55:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.164+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a00007400 con 0x7f6a0c0ae550 2024-09-10T10:55:33.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.165+0000 7f6a08c00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 0x7f69ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.166+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f6a0008b3c0 con 0x7f6a0c0ae550 2024-09-10T10:55:33.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.166+0000 7f6a0be00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 0x7f69ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.167+0000 7f6a0be00640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 0x7f69ec06e320 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f69f8034000 tx=0x7f69f802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.170+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6a00058c90 con 0x7f6a0c0ae550 2024-09-10T10:55:33.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.396+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a0c0aeb00 con 0x7f6a0c0ae550 2024-09-10T10:55:33.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.396+0000 7f6a08c00640 1 -- 172.21.15.28:0/349671070 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4390 (secure 0 0 0) 0x7f6a00058590 con 0x7f6a0c0ae550 2024-09-10T10:55:33.397 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:33.398 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":331,"btime":"2024-09-10T10:55:29:729465+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:33.398 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 331 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 msgr2=0x7f69ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 0x7f69ec06e320 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f69f8034000 tx=0x7f69f802e000 comp rx=0 tx=0).stop 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 msgr2=0x7f6a0c1408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000c3c0 tx=0x7f6a0000c880 comp rx=0 tx=0).stop 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 shutdown_connections 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f69ec06be60 0x7f69ec06e320 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a0c147360 0x7f6a0c144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c0d04a0 0x7f6a0c15c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 --2- 172.21.15.28:0/349671070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a0c0ae550 0x7f6a0c1408f0 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 >> 172.21.15.28:0/349671070 conn(0x7f6a0c0b2f40 msgr2=0x7f6a0c0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:33.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 shutdown_connections 2024-09-10T10:55:33.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.399+0000 7f6a13400640 1 -- 172.21.15.28:0/349671070 wait complete. 2024-09-10T10:55:33.415 INFO:tasks.cephfs.test_damage:Result: Mutation 'Corrupt omap header on 1.00000000' led to DAMAGED state 2024-09-10T10:55:33.415 INFO:tasks.cephfs.test_damage:All 39 mutations had expected outcomes 2024-09-10T10:55:33.415 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:55:33.416 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:55:33.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:55:33.596 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:55:33.596 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:55:33.597 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:33.597 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:33.632 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:33.632 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:55:33.632 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:55:33.633 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:33.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:55:33.670 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:55:33.670 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:33.670 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:33.708 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:33.708 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:55:33.708 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:55:33.708 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:33.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:55:33.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:33.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.903+0000 7f3397800640 1 Processor -- start 2024-09-10T10:55:33.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.903+0000 7f3397800640 1 -- start start 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.904+0000 7f3397800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3390072de0 0x7f339015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f3397800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3390146140 con 0x7f3390072320 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f3397800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900cd740 con 0x7f3390072700 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f3397800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f33900cda20 con 0x7f3390072de0 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f338fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3390072de0 0x7f339015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f3394e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3390072320 0x7f33900cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46084/0 (socket says 172.21.15.28:46084) 2024-09-10T10:55:33.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f3394e00640 1 -- 172.21.15.28:0/1986479440 learned_addr learned my addr 172.21.15.28:0/1986479440 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:33.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f338fe00640 1 -- 172.21.15.28:0/1986479440 >> v1:172.21.15.138:6789/0 conn(0x7f3390072700 legacy=0x7f339015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f338fe00640 1 -- 172.21.15.28:0/1986479440 >> v1:172.21.15.28:6789/0 conn(0x7f3390072320 legacy=0x7f33900cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f338f400640 1 -- 172.21.15.28:0/1986479440 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3754568762 0 0) 0x7f3390146140 con 0x7f3390072320 2024-09-10T10:55:33.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.905+0000 7f338fe00640 1 -- 172.21.15.28:0/1986479440 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900cdd00 con 0x7f3390072de0 2024-09-10T10:55:33.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.906+0000 7f338fe00640 1 --2- 172.21.15.28:0/1986479440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f339015d1e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f337c0097a0 tx=0x7f337c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9153de5fefe86e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f338f400640 1 -- 172.21.15.28:0/1986479440 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c037070 con 0x7f3390072de0 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f338f400640 1 -- 172.21.15.28:0/1986479440 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f337c032440 con 0x7f3390072de0 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f338f400640 1 -- 172.21.15.28:0/1986479440 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c032810 con 0x7f3390072de0 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 -- 172.21.15.28:0/1986479440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 msgr2=0x7f339015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 --2- 172.21.15.28:0/1986479440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f339015d1e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f337c0097a0 tx=0x7f337c02a1a0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 -- 172.21.15.28:0/1986479440 shutdown_connections 2024-09-10T10:55:33.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 --2- 172.21.15.28:0/1986479440 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f339015d1e0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 -- 172.21.15.28:0/1986479440 >> 172.21.15.28:0/1986479440 conn(0x7f3390067d70 msgr2=0x7f339006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:33.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 -- 172.21.15.28:0/1986479440 shutdown_connections 2024-09-10T10:55:33.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.907+0000 7f3397800640 1 -- 172.21.15.28:0/1986479440 wait complete. 2024-09-10T10:55:33.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.909+0000 7f3397800640 1 Processor -- start 2024-09-10T10:55:33.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.909+0000 7f3397800640 1 -- start start 2024-09-10T10:55:33.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390072320 0x7f33901f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390072700 0x7f339015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f339015dfb0 con 0x7f3390072de0 2024-09-10T10:55:33.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f339015de30 con 0x7f3390072700 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3397800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f339015e130 con 0x7f3390072320 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3395800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3395800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43798/0 (socket says 172.21.15.28:43798) 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.910+0000 7f3395800640 1 -- 172.21.15.28:0/930642970 learned_addr learned my addr 172.21.15.28:0/930642970 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f338fe00640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390072700 0x7f339015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3394e00640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390072320 0x7f33901f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 -- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390072320 msgr2=0x7f33901f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390072320 0x7f33901f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 -- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390072700 msgr2=0x7f339015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:33.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390072700 0x7f339015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:33.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 -- 172.21.15.28:0/930642970 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33901f47c0 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3395800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f338800c730 tx=0x7f338800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388003b70 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388003d10 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338800aa90 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.911+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33901f4aa0 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.912+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33900b3ba0 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.912+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33900cce60 con 0x7f3390072de0 2024-09-10T10:55:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.914+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f338800ac30 con 0x7f3390072de0 2024-09-10T10:55:33.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.914+0000 7f338cc00640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 0x7f337006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:33.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.915+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f338808bc50 con 0x7f3390072de0 2024-09-10T10:55:33.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.915+0000 7f3394e00640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 0x7f337006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:33.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.915+0000 7f3394e00640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 0x7f337006e320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3384007f70 tx=0x7f3384007f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:33.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:33.919+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3388058f60 con 0x7f3390072de0 2024-09-10T10:55:34.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.141+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33900b1a70 con 0x7f3390072de0 2024-09-10T10:55:34.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.142+0000 7f338cc00640 1 -- 172.21.15.28:0/930642970 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4390 (secure 0 0 0) 0x7f338805d260 con 0x7f3390072de0 2024-09-10T10:55:34.143 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:34.144 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":331,"btime":"2024-09-10T10:55:29:729465+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:34.144 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 331 2024-09-10T10:55:34.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 msgr2=0x7f337006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 0x7f337006e320 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3384007f70 tx=0x7f3384007f00 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 msgr2=0x7f33900caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f338800c730 tx=0x7f338800cbf0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 shutdown_connections 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f337006be60 0x7f337006e320 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3390072de0 0x7f33900caec0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390072700 0x7f339015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 --2- 172.21.15.28:0/930642970 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390072320 0x7f33901f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 >> 172.21.15.28:0/930642970 conn(0x7f3390067d70 msgr2=0x7f3390144210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.144+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 shutdown_connections 2024-09-10T10:55:34.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.145+0000 7f3397800640 1 -- 172.21.15.28:0/930642970 wait complete. 2024-09-10T10:55:34.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:34.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.317+0000 7f3e30a00640 1 Processor -- start 2024-09-10T10:55:34.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.317+0000 7f3e30a00640 1 -- start start 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e2c15b330 0x7f3e2c073320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3e2c0b42b0 0x7f3e2c0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3e2c06dc70 con 0x7f3e2c15b330 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3e2c06df10 con 0x7f3e2c0b42b0 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.318+0000 7f3e30a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c06e1f0 con 0x7f3e2c15af50 2024-09-10T10:55:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e2c15b330 0x7f3e2c073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e2c15b330 0x7f3e2c073320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43812/0 (socket says 172.21.15.28:43812) 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 -- 172.21.15.28:0/330608671 learned_addr learned my addr 172.21.15.28:0/330608671 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e2b200640 1 --2- 172.21.15.28:0/330608671 >> v2:172.21.15.138:3300/0 conn(0x7f3e2c0b42b0 0x7f3e2c0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e2a800640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 -- 172.21.15.28:0/330608671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 msgr2=0x7f3e2c072de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c072de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 -- 172.21.15.28:0/330608671 >> v2:172.21.15.138:3300/0 conn(0x7f3e2c0b42b0 msgr2=0x7f3e2c0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 --2- 172.21.15.28:0/330608671 >> v2:172.21.15.138:3300/0 conn(0x7f3e2c0b42b0 0x7f3e2c0b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.319+0000 7f3e29e00640 1 -- 172.21.15.28:0/330608671 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c06e4d0 con 0x7f3e2c15b330 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.320+0000 7f3e29e00640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c073320 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0b23d0 tx=0x7f3e1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf1f67c6f14bde5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.320+0000 7f3e29400640 1 -- 172.21.15.28:0/330608671 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18037070 con 0x7f3e2c15b330 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.320+0000 7f3e29400640 1 -- 172.21.15.28:0/330608671 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e18032440 con 0x7f3e2c15b330 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.320+0000 7f3e29400640 1 -- 172.21.15.28:0/330608671 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e18032810 con 0x7f3e2c15b330 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 -- 172.21.15.28:0/330608671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 msgr2=0x7f3e2c073320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c073320 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c0b23d0 tx=0x7f3e1802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 -- 172.21.15.28:0/330608671 shutdown_connections 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 --2- 172.21.15.28:0/330608671 >> v2:172.21.15.138:3300/0 conn(0x7f3e2c0b42b0 0x7f3e2c0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c073320 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 --2- 172.21.15.28:0/330608671 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c072de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 -- 172.21.15.28:0/330608671 >> 172.21.15.28:0/330608671 conn(0x7f3e2c06c240 msgr2=0x7f3e2c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 -- 172.21.15.28:0/330608671 shutdown_connections 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.321+0000 7f3e30a00640 1 -- 172.21.15.28:0/330608671 wait complete. 2024-09-10T10:55:34.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.323+0000 7f3e30a00640 1 Processor -- start 2024-09-10T10:55:34.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.323+0000 7f3e30a00640 1 -- start start 2024-09-10T10:55:34.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e2c0b42b0 0x7f3e2c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c0adbc0 con 0x7f3e2c15b330 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c0b4ca0 con 0x7f3e2c0b42b0 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e30a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c0add40 con 0x7f3e2c15af50 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43814/0 (socket says 172.21.15.28:43814) 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 -- 172.21.15.28:0/128351228 learned_addr learned my addr 172.21.15.28:0/128351228 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2a800640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e2c0b42b0 0x7f3e2c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e29e00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 -- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 msgr2=0x7f3e2c148da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c148da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 -- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e2c0b42b0 msgr2=0x7f3e2c1494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e2c0b42b0 0x7f3e2c1494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.324+0000 7f3e2b200640 1 -- 172.21.15.28:0/128351228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c1492e0 con 0x7f3e2c15b330 2024-09-10T10:55:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e2b200640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3e24007e60 tx=0x7f3e2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24014620 con 0x7f3e2c15b330 2024-09-10T10:55:34.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e24019070 con 0x7f3e2c15b330 2024-09-10T10:55:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24014a20 con 0x7f3e2c15b330 2024-09-10T10:55:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c0cda90 con 0x7f3e2c15b330 2024-09-10T10:55:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.325+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c151650 con 0x7f3e2c15b330 2024-09-10T10:55:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.326+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c0ad470 con 0x7f3e2c15b330 2024-09-10T10:55:34.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.327+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3e24014bc0 con 0x7f3e2c15b330 2024-09-10T10:55:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.328+0000 7f3e22a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 0x7f3e0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.328+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f3e2408c550 con 0x7f3e2c15b330 2024-09-10T10:55:34.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.328+0000 7f3e2a800640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 0x7f3e0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.329+0000 7f3e2a800640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 0x7f3e0c06e2f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006fd0 tx=0x7f3e1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.332+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3e2c0ad470 con 0x7f3e2c15b330 2024-09-10T10:55:34.486 INFO:tasks.ceph.mds.a.smithi028.stderr: -51> 2024-09-10T10:55:29.727+0000 7f7e8ee00640 -1 mds.0.cache.dir(0x1) Corrupt fnode in dirfrag 0x1: Decoder at 'void fnode_t::decode(ceph::buffer::v15_2_0::list::const_iterator&)' v=3 cannot decode v=100 minimal_decoder=101: Malformed input [buffer:3] 2024-09-10T10:55:34.486 INFO:tasks.ceph.mds.a.smithi028.stderr: -48> 2024-09-10T10:55:29.727+0000 7f7e8ee00640 -1 mds.0.damage notify_dirfrag Damage to fragment * of ino 0x1 is fatal because it is a system directory for this rank 2024-09-10T10:55:34.486 INFO:tasks.ceph.mds.a.smithi028.stderr: -20> 2024-09-10T10:55:32.863+0000 7f7e93e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:34.527 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:55:34.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.561+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e2c15ea00 con 0x7f3e2c15b330 2024-09-10T10:55:34.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.562+0000 7f3e22a00640 1 -- 172.21.15.28:0/128351228 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 331 v331) ==== 95+0+4390 (secure 0 0 0) 0x7f3e240598f0 con 0x7f3e2c15b330 2024-09-10T10:55:34.563 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:34.563 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":331,"btime":"2024-09-10T10:55:29:729465+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:34.563 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 331 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 msgr2=0x7f3e0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 0x7f3e0c06e2f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c006fd0 tx=0x7f3e1c008040 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 msgr2=0x7f3e2c1509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f3e24007e60 tx=0x7f3e2400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 shutdown_connections 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3e0c06be30 0x7f3e0c06e2f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c15b330 0x7f3e2c1509b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3e2c15af50 0x7f3e2c148da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 --2- 172.21.15.28:0/128351228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e2c0b42b0 0x7f3e2c1494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 >> 172.21.15.28:0/128351228 conn(0x7f3e2c06c240 msgr2=0x7f3e2c0b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:34.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 shutdown_connections 2024-09-10T10:55:34.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.564+0000 7f3e30a00640 1 -- 172.21.15.28:0/128351228 wait complete. 2024-09-10T10:55:34.582 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:34.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.736+0000 7fd9eca00640 1 Processor -- start 2024-09-10T10:55:34.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.737+0000 7fd9eca00640 1 -- start start 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9eca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd9e8072700 0x7fd9e815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9eca00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e8146140 con 0x7fd9e8072320 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9eca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e80cd740 con 0x7fd9e8072de0 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9eca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e80cda20 con 0x7fd9e8072700 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9e5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd9e8072320 0x7fd9e80cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46088/0 (socket says 172.21.15.28:46088) 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9e5e00640 1 -- 172.21.15.28:0/1262311231 learned_addr learned my addr 172.21.15.28:0/1262311231 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9e6800640 1 --2- 172.21.15.28:0/1262311231 >> v2:172.21.15.138:3300/0 conn(0x7fd9e8072700 0x7fd9e815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.738+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907032430 0 0) 0x7fd9e8146140 con 0x7fd9e8072320 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9cc003520 con 0x7fd9e8072320 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140112174 0 0) 0x7fd9e80cd740 con 0x7fd9e8072de0 2024-09-10T10:55:34.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9e8146140 con 0x7fd9e8072de0 2024-09-10T10:55:34.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e6800640 1 -- 172.21.15.28:0/1262311231 >> v1:172.21.15.138:6789/0 conn(0x7fd9e8072de0 legacy=0x7fd9e815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e6800640 1 -- 172.21.15.28:0/1262311231 >> v1:172.21.15.28:6789/0 conn(0x7fd9e8072320 legacy=0x7fd9e80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.739+0000 7fd9e6800640 1 -- 172.21.15.28:0/1262311231 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e80cdd00 con 0x7fd9e8072700 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.740+0000 7fd9e6800640 1 --2- 172.21.15.28:0/1262311231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072700 0x7fd9e815d8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009d90 tx=0x7fd9e002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d924c90497927ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.740+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0007740 con 0x7fd9e8072700 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.740+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e00078e0 con 0x7fd9e8072700 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.740+0000 7fd9e5400640 1 -- 172.21.15.28:0/1262311231 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0007cb0 con 0x7fd9e8072700 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 -- 172.21.15.28:0/1262311231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072700 msgr2=0x7fd9e815d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1262311231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072700 0x7fd9e815d8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009d90 tx=0x7fd9e002a660 comp rx=0 tx=0).stop 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 -- 172.21.15.28:0/1262311231 shutdown_connections 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1262311231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072700 0x7fd9e815d8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 -- 172.21.15.28:0/1262311231 >> 172.21.15.28:0/1262311231 conn(0x7fd9e8067d70 msgr2=0x7fd9e806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:34.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 -- 172.21.15.28:0/1262311231 shutdown_connections 2024-09-10T10:55:34.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.741+0000 7fd9eca00640 1 -- 172.21.15.28:0/1262311231 wait complete. 2024-09-10T10:55:34.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.742+0000 7fd9eca00640 1 Processor -- start 2024-09-10T10:55:34.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 -- start start 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9e8072700 0x7fd9e81f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072de0 0x7fd9e80acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e815dfb0 con 0x7fd9e8072320 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e815de30 con 0x7fd9e8072de0 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.743+0000 7fd9eca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e815e130 con 0x7fd9e8072700 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43828/0 (socket says 172.21.15.28:43828) 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 -- 172.21.15.28:0/1834188971 learned_addr learned my addr 172.21.15.28:0/1834188971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9dfe00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9e8072700 0x7fd9e81f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e6800640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072de0 0x7fd9e80acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 -- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9e8072700 msgr2=0x7fd9e81f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9e8072700 0x7fd9e81f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 -- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072de0 msgr2=0x7fd9e80acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072de0 0x7fd9e80acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 -- 172.21.15.28:0/1834188971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e80ab700 con 0x7fd9e8072320 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.744+0000 7fd9e5e00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c530 tx=0x7fd9d800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.745+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8017070 con 0x7fd9e8072320 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.745+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d8005ce0 con 0x7fd9e8072320 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.745+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8007270 con 0x7fd9e8072320 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.745+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e80ad080 con 0x7fd9e8072320 2024-09-10T10:55:34.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.745+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e81f4e10 con 0x7fd9e8072320 2024-09-10T10:55:34.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.746+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e81447b0 con 0x7fd9e8072320 2024-09-10T10:55:34.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.747+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9d801a070 con 0x7fd9e8072320 2024-09-10T10:55:34.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.748+0000 7fd9de000640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 0x7fd9c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:34.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.748+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7fd9d808b1c0 con 0x7fd9e8072320 2024-09-10T10:55:34.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.748+0000 7fd9dfe00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 0x7fd9c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:34.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.749+0000 7fd9dfe00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 0x7fd9c406e2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80caa70 tx=0x7fd9d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:34.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.752+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9e81447b0 con 0x7fd9e8072320 2024-09-10T10:55:34.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.942+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9e80cce60 con 0x7fd9e8072320 2024-09-10T10:55:34.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.943+0000 7fd9de000640 1 -- 172.21.15.28:0/1834188971 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+22472 (secure 0 0 0) 0x7fd9d80584d0 con 0x7fd9e8072320 2024-09-10T10:55:34.945 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:34.945 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":140,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:29.729180+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:50:25.399081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T10:50:26.587379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.c","mappings":[{"from":6,"to":4}]},{"pgid":"14.9","mappings":[{"from":7,"to":3}]},{"pgid":"14.f","mappings":[{"from":1,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6833/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6834/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6832/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6834/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6835/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6834/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6832/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6834/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6833/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6832/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6834/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6833/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6834/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6835/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6834/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6833/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6833/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6832/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6835/4204725630":"2024-09-11T10:53:44.760451+0000","172.21.15.28:6832/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6834/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6833/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6833/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6832/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6832/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6832/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6835/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6835/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6835/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6833/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6834/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6834/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6834/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6835/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6835/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6832/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6834/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6832/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6832/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6832/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6833/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6832/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6835/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6834/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6835/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6834/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6835/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6833/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6835/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6833/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6835/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6832/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6833/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6833/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6832/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6835/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6834/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6834/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6833/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6832/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6835/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6834/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6834/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6835/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/4204725630":"2024-09-11T10:53:44.760451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:34.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 msgr2=0x7fd9c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 0x7fd9c406e2f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80caa70 tx=0x7fd9d0009040 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 msgr2=0x7fd9e8141fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800c530 tx=0x7fd9d800c9f0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 shutdown_connections 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9c406be30 0x7fd9c406e2f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9e8072de0 0x7fd9e80acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd9e8072700 0x7fd9e81f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 --2- 172.21.15.28:0/1834188971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9e8072320 0x7fd9e8141fb0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:34.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 >> 172.21.15.28:0/1834188971 conn(0x7fd9e8067d70 msgr2=0x7fd9e806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:34.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.945+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 shutdown_connections 2024-09-10T10:55:34.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:34.946+0000 7fd9eca00640 1 -- 172.21.15.28:0/1834188971 wait complete. 2024-09-10T10:55:34.963 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:55:34.964 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:35.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.119+0000 7f31f9600640 1 Processor -- start 2024-09-10T10:55:35.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.119+0000 7f31f9600640 1 -- start start 2024-09-10T10:55:35.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f9600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 0x7f31f40d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f9600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31f40d1f90 0x7f31f40cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f9600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f40cf000 con 0x7f31f415e660 2024-09-10T10:55:35.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f9600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f31f406d3e0 con 0x7f31f40d1f90 2024-09-10T10:55:35.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f9600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31f40b37d0 con 0x7f31f40d0dd0 2024-09-10T10:55:35.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f3400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f31f415e660 0x7f31f406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46094/0 (socket says 172.21.15.28:46094) 2024-09-10T10:55:35.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f3400640 1 -- 172.21.15.28:0/2256143622 learned_addr learned my addr 172.21.15.28:0/2256143622 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:35.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.120+0000 7f31f8c00640 1 --2- 172.21.15.28:0/2256143622 >> v2:172.21.15.28:3300/0 conn(0x7f31f40d1f90 0x7f31f40cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f3e00640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 0x7f31f40d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f2a00640 1 -- 172.21.15.28:0/2256143622 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2706752084 0 0) 0x7f31f40cf000 con 0x7f31f415e660 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f8c00640 1 -- 172.21.15.28:0/2256143622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 msgr2=0x7f31f40d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f8c00640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 0x7f31f40d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f8c00640 1 -- 172.21.15.28:0/2256143622 >> v1:172.21.15.28:6789/0 conn(0x7f31f415e660 legacy=0x7f31f406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f8c00640 1 -- 172.21.15.28:0/2256143622 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f40d16f0 con 0x7f31f40d1f90 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.121+0000 7f31f3e00640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 0x7f31f40d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:35.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.122+0000 7f31f8c00640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40cf7c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009d90 tx=0x7f31ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16e34ee0ef3c993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.122+0000 7f31f2a00640 1 -- 172.21.15.28:0/2256143622 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007740 con 0x7f31f40d1f90 2024-09-10T10:55:35.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.122+0000 7f31f2a00640 1 -- 172.21.15.28:0/2256143622 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec0078e0 con 0x7f31f40d1f90 2024-09-10T10:55:35.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.122+0000 7f31f2a00640 1 -- 172.21.15.28:0/2256143622 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec007cb0 con 0x7f31f40d1f90 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 -- 172.21.15.28:0/2256143622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 msgr2=0x7f31f40cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40cf7c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009d90 tx=0x7f31ec02a680 comp rx=0 tx=0).stop 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 -- 172.21.15.28:0/2256143622 shutdown_connections 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40cf7c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 --2- 172.21.15.28:0/2256143622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f40d0dd0 0x7f31f40d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 -- 172.21.15.28:0/2256143622 >> 172.21.15.28:0/2256143622 conn(0x7f31f408bfc0 msgr2=0x7f31f40d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 -- 172.21.15.28:0/2256143622 shutdown_connections 2024-09-10T10:55:35.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.123+0000 7f31f9600640 1 -- 172.21.15.28:0/2256143622 wait complete. 2024-09-10T10:55:35.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.124+0000 7f31f9600640 1 Processor -- start 2024-09-10T10:55:35.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 -- start start 2024-09-10T10:55:35.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 0x7f31f415cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f415e660 0x7f31f41e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40d2c60 con 0x7f31f40d1f90 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40cf330 con 0x7f31f40d0dd0 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.125+0000 7f31f9600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31f40d2f40 con 0x7f31f415e660 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43842/0 (socket says 172.21.15.28:43842) 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 -- 172.21.15.28:0/3953742344 learned_addr learned my addr 172.21.15.28:0/3953742344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f8c00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f415e660 0x7f31f41e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3e00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 0x7f31f415cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 -- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f415e660 msgr2=0x7f31f41e3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f415e660 0x7f31f41e3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 -- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 msgr2=0x7f31f415cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 0x7f31f415cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 -- 172.21.15.28:0/3953742344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f409e610 con 0x7f31f40d1f90 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3e00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 0x7f31f415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.126+0000 7f31f3400640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c440 tx=0x7f31e0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.127+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00074e0 con 0x7f31f40d1f90 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.127+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e00047e0 con 0x7f31f40d1f90 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.127+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0004bd0 con 0x7f31f40d1f90 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.127+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f409e330 con 0x7f31f40d1f90 2024-09-10T10:55:35.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.127+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f40b7810 con 0x7f31f40d1f90 2024-09-10T10:55:35.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.128+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f40d3220 con 0x7f31f40d1f90 2024-09-10T10:55:35.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.129+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f31e0011070 con 0x7f31f40d1f90 2024-09-10T10:55:35.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.130+0000 7f31ebe00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 0x7f31d406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.130+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f31e008b940 con 0x7f31f40d1f90 2024-09-10T10:55:35.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.130+0000 7f31f3e00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 0x7f31d406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.131+0000 7f31f3e00640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 0x7f31d406e320 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006fd0 tx=0x7f31e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.134+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f31e00592a0 con 0x7f31f40d1f90 2024-09-10T10:55:35.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.354+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31f40d0150 con 0x7f31f40d1f90 2024-09-10T10:55:35.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.355+0000 7f31ebe00640 1 -- 172.21.15.28:0/3953742344 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 332 v332) ==== 95+0+5217 (secure 0 0 0) 0x7f31e0058ba0 con 0x7f31f40d1f90 2024-09-10T10:55:35.356 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:35.356 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":332,"btime":"2024-09-10T10:55:34:747804+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1966252373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1966252373},{"type":"v1","addr":"172.21.15.28:6835","nonce":1966252373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:35.356 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 332 2024-09-10T10:55:35.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 msgr2=0x7f31d406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 0x7f31d406e320 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31e4006fd0 tx=0x7f31e4008040 comp rx=0 tx=0).stop 2024-09-10T10:55:35.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 msgr2=0x7f31f40b9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c440 tx=0x7f31e0001e70 comp rx=0 tx=0).stop 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 shutdown_connections 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f31d406be60 0x7f31d406e320 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31f415e660 0x7f31f41e3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31f40d1f90 0x7f31f40b9c70 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 --2- 172.21.15.28:0/3953742344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31f40d0dd0 0x7f31f415cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 >> 172.21.15.28:0/3953742344 conn(0x7f31f408bfc0 msgr2=0x7f31f40b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 shutdown_connections 2024-09-10T10:55:35.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.357+0000 7f31f9600640 1 -- 172.21.15.28:0/3953742344 wait complete. 2024-09-10T10:55:35.374 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:35.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.530+0000 7f8f57400640 1 Processor -- start 2024-09-10T10:55:35.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.530+0000 7f8f57400640 1 -- start start 2024-09-10T10:55:35.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.531+0000 7f8f57400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8f50072320 0x7f8f500cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.531+0000 7f8f57400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8f5015e1f0 0x7f8f5006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.531+0000 7f8f57400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f50146140 con 0x7f8f50072de0 2024-09-10T10:55:35.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.531+0000 7f8f57400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500cd740 con 0x7f8f5015e1f0 2024-09-10T10:55:35.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.531+0000 7f8f57400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8f500cda20 con 0x7f8f50072320 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f4fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8f50072de0 0x7f8f5006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46098/0 (socket says 172.21.15.28:46098) 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f4fe00640 1 -- 172.21.15.28:0/2714208193 learned_addr learned my addr 172.21.15.28:0/2714208193 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f55400640 1 --2- 172.21.15.28:0/2714208193 >> v2:172.21.15.28:3300/0 conn(0x7f8f5015e1f0 0x7f8f5006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f54a00640 1 --2- 172.21.15.28:0/2714208193 >> v2:172.21.15.138:3300/0 conn(0x7f8f50072320 0x7f8f500cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f55400640 1 -- 172.21.15.28:0/2714208193 >> v2:172.21.15.138:3300/0 conn(0x7f8f50072320 msgr2=0x7f8f500cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f55400640 1 --2- 172.21.15.28:0/2714208193 >> v2:172.21.15.138:3300/0 conn(0x7f8f50072320 0x7f8f500cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f55400640 1 -- 172.21.15.28:0/2714208193 >> v1:172.21.15.28:6789/0 conn(0x7f8f50072de0 legacy=0x7f8f5006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.532+0000 7f8f55400640 1 -- 172.21.15.28:0/2714208193 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500cdd00 con 0x7f8f5015e1f0 2024-09-10T10:55:35.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.533+0000 7f8f55400640 1 --2- 172.21.15.28:0/2714208193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f5015e1f0 0x7f8f5006e7c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f8f500b31e0 tx=0x7f8f4802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344e9fcb5a81d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.533+0000 7f8f4f400640 1 -- 172.21.15.28:0/2714208193 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48007740 con 0x7f8f5015e1f0 2024-09-10T10:55:35.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f4f400640 1 -- 172.21.15.28:0/2714208193 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f480078e0 con 0x7f8f5015e1f0 2024-09-10T10:55:35.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f4f400640 1 -- 172.21.15.28:0/2714208193 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f48007cb0 con 0x7f8f5015e1f0 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 -- 172.21.15.28:0/2714208193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f5015e1f0 msgr2=0x7f8f5006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 --2- 172.21.15.28:0/2714208193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f5015e1f0 0x7f8f5006e7c0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f8f500b31e0 tx=0x7f8f4802a5f0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 -- 172.21.15.28:0/2714208193 shutdown_connections 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 --2- 172.21.15.28:0/2714208193 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f5015e1f0 0x7f8f5006e7c0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 --2- 172.21.15.28:0/2714208193 >> v2:172.21.15.138:3300/0 conn(0x7f8f50072320 0x7f8f500cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 -- 172.21.15.28:0/2714208193 >> 172.21.15.28:0/2714208193 conn(0x7f8f50067d70 msgr2=0x7f8f5006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 -- 172.21.15.28:0/2714208193 shutdown_connections 2024-09-10T10:55:35.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.534+0000 7f8f57400640 1 -- 172.21.15.28:0/2714208193 wait complete. 2024-09-10T10:55:35.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.536+0000 7f8f57400640 1 Processor -- start 2024-09-10T10:55:35.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.536+0000 7f8f57400640 1 -- start start 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f50072de0 0x7f8f501f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f5015e1f0 0x7f8f500cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500cd520 con 0x7f8f50072320 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f500cd3a0 con 0x7f8f50072de0 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f57400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f5006ed00 con 0x7f8f5015e1f0 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f54a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f54a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43854/0 (socket says 172.21.15.28:43854) 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f54a00640 1 -- 172.21.15.28:0/1379826969 learned_addr learned my addr 172.21.15.28:0/1379826969 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f4fe00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f50072de0 0x7f8f501f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.537+0000 7f8f55400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f5015e1f0 0x7f8f500cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 -- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f5015e1f0 msgr2=0x7f8f500cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f5015e1f0 0x7f8f500cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 -- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f50072de0 msgr2=0x7f8f501f7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f50072de0 0x7f8f501f7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 -- 172.21.15.28:0/1379826969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f500cb870 con 0x7f8f50072320 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f54a00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d830 tx=0x7f8f4400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400b6c0 con 0x7f8f50072320 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f44019070 con 0x7f8f50072320 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f500cbb50 con 0x7f8f50072320 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4400ba80 con 0x7f8f50072320 2024-09-10T10:55:35.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.538+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f500cc110 con 0x7f8f50072320 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.539+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f500cc550 con 0x7f8f50072320 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.540+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f4401e460 con 0x7f8f50072320 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.541+0000 7f8f4cc00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.541+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f8f4408c500 con 0x7f8f50072320 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.541+0000 7f8f4fe00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.542+0000 7f8f4fe00640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 0x7f8f3006e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8f501f7720 tx=0x7f8f3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.545+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f44059ef0 con 0x7f8f50072320 2024-09-10T10:55:35.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.767+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f50072700 con 0x7f8f50072320 2024-09-10T10:55:35.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.767+0000 7f8f4cc00640 1 -- 172.21.15.28:0/1379826969 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 332 v332) ==== 95+0+5217 (secure 0 0 0) 0x7f8f440597f0 con 0x7f8f50072320 2024-09-10T10:55:35.768 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:35.768 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":332,"btime":"2024-09-10T10:55:34:747804+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1966252373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1966252373},{"type":"v1","addr":"172.21.15.28:6835","nonce":1966252373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[{"mdsmap":{"epoch":331,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:50:27.593484+0000","modified":"2024-09-10T10:55:29.729463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T10:55:35.769 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 332 2024-09-10T10:55:35.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 msgr2=0x7f8f3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 0x7f8f3006e2f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8f501f7720 tx=0x7f8f3c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 msgr2=0x7f8f501f77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400d830 tx=0x7f8f4400dcf0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 shutdown_connections 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8f3006be30 0x7f8f3006e2f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f5015e1f0 0x7f8f500cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f50072de0 0x7f8f501f7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 --2- 172.21.15.28:0/1379826969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8f50072320 0x7f8f501f77c0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 >> 172.21.15.28:0/1379826969 conn(0x7f8f50067d70 msgr2=0x7f8f500c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 shutdown_connections 2024-09-10T10:55:35.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.769+0000 7f8f57400640 1 -- 172.21.15.28:0/1379826969 wait complete. 2024-09-10T10:55:35.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:35.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.941+0000 7f2eda200640 1 Processor -- start 2024-09-10T10:55:35.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.942+0000 7f2eda200640 1 -- start start 2024-09-10T10:55:35.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2eda200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed4165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2eda200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed40cdbe0 con 0x7f2ed4072d30 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2eda200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed40cd900 con 0x7f2ed415b300 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2eda200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed40728a0 con 0x7f2ed406f8d0 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2ecbe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2ed4072d30 0x7f2ed406f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46112/0 (socket says 172.21.15.28:46112) 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2ecbe00640 1 -- 172.21.15.28:0/555974580 learned_addr learned my addr 172.21.15.28:0/555974580 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.943+0000 7f2ed3e00640 1 --2- 172.21.15.28:0/555974580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed4165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703847603 0 0) 0x7f2ed40cdbe0 con 0x7f2ed4072d30 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2eb8003520 con 0x7f2ed4072d30 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977889174 0 0) 0x7f2ed40cd900 con 0x7f2ed415b300 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed40cdbe0 con 0x7f2ed415b300 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed3e00640 1 -- 172.21.15.28:0/555974580 >> v1:172.21.15.138:6789/0 conn(0x7f2ed415b300 legacy=0x7f2ed40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed3e00640 1 -- 172.21.15.28:0/555974580 >> v1:172.21.15.28:6789/0 conn(0x7f2ed4072d30 legacy=0x7f2ed406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.944+0000 7f2ed3e00640 1 -- 172.21.15.28:0/555974580 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed4072250 con 0x7f2ed406f8d0 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.945+0000 7f2ed3e00640 1 --2- 172.21.15.28:0/555974580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed4165de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009dc0 tx=0x7f2ecc02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d906d5d9bb6bce1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.945+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc02ae60 con 0x7f2ed406f8d0 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.945+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ecc007740 con 0x7f2ed406f8d0 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.945+0000 7f2ed2a00640 1 -- 172.21.15.28:0/555974580 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc007af0 con 0x7f2ed406f8d0 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 -- 172.21.15.28:0/555974580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 msgr2=0x7f2ed4165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 --2- 172.21.15.28:0/555974580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed4165de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009dc0 tx=0x7f2ecc02a640 comp rx=0 tx=0).stop 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 -- 172.21.15.28:0/555974580 shutdown_connections 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 --2- 172.21.15.28:0/555974580 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed4165de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 -- 172.21.15.28:0/555974580 >> 172.21.15.28:0/555974580 conn(0x7f2ed4067d70 msgr2=0x7f2ed406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 -- 172.21.15.28:0/555974580 shutdown_connections 2024-09-10T10:55:35.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.946+0000 7f2eda200640 1 -- 172.21.15.28:0/555974580 wait complete. 2024-09-10T10:55:35.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.948+0000 7f2eda200640 1 Processor -- start 2024-09-10T10:55:35.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.948+0000 7f2eda200640 1 -- start start 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed41f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed415b300 0x7f2ed414a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed40cffe0 con 0x7f2ed4072d30 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed40cd5c0 con 0x7f2ed415b300 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2eda200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed40d0160 con 0x7f2ed406f8d0 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43868/0 (socket says 172.21.15.28:43868) 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 -- 172.21.15.28:0/4074942467 learned_addr learned my addr 172.21.15.28:0/4074942467 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ed3400640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed41f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ed3e00640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed415b300 0x7f2ed414a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 -- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 msgr2=0x7f2ed41f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed41f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 -- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed415b300 msgr2=0x7f2ed414a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed415b300 0x7f2ed414a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.949+0000 7f2ecbe00640 1 -- 172.21.15.28:0/4074942467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed41511b0 con 0x7f2ed4072d30 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2ecbe00640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c500 tx=0x7f2ebc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc017070 con 0x7f2ed4072d30 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ebc005ce0 con 0x7f2ed4072d30 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0072f0 con 0x7f2ed4072d30 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed4157820 con 0x7f2ed4072d30 2024-09-10T10:55:35.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.950+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed4157de0 con 0x7f2ed4072d30 2024-09-10T10:55:35.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.951+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed406f1c0 con 0x7f2ed4072d30 2024-09-10T10:55:35.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.952+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ebc01a070 con 0x7f2ed4072d30 2024-09-10T10:55:35.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.953+0000 7f2ecb400640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 0x7f2eb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:35.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.953+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f2ebc08b150 con 0x7f2ed4072d30 2024-09-10T10:55:35.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.953+0000 7f2ed3400640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 0x7f2eb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:35.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.954+0000 7f2ed3400640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 0x7f2eb006e2f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40011e0 tx=0x7f2ec4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:35.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:35.957+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ebc0585b0 con 0x7f2ed4072d30 2024-09-10T10:55:36.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.141+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ed406d090 con 0x7f2ed4072d30 2024-09-10T10:55:36.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.143+0000 7f2ecb400640 1 -- 172.21.15.28:0/4074942467 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+22472 (secure 0 0 0) 0x7f2ebc05c8b0 con 0x7f2ed4072d30 2024-09-10T10:55:36.144 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:36.144 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":140,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:29.729180+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:50:25.399081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-10T10:50:26.587379+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.c","mappings":[{"from":6,"to":4}]},{"pgid":"14.9","mappings":[{"from":7,"to":3}]},{"pgid":"14.f","mappings":[{"from":1,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.34","mappings":[{"from":7,"to":0}]},{"pgid":"14.36","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6833/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6834/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6832/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6834/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6835/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6834/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6832/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6834/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6833/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6832/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6834/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6833/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6834/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6835/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6834/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6833/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6833/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6832/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6835/4204725630":"2024-09-11T10:53:44.760451+0000","172.21.15.28:6832/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6834/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6833/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6833/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6832/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6832/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6832/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6835/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6835/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6835/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6833/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6834/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6834/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6834/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6835/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6835/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6832/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6834/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6832/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6832/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6832/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6833/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6832/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6835/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6834/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6835/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6834/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6835/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6833/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6835/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6833/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6835/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6832/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6833/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6833/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6832/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6835/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6834/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6834/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6833/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6832/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6835/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6834/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6834/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6835/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/4204725630":"2024-09-11T10:53:44.760451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:36.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 msgr2=0x7f2eb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 0x7f2eb006e2f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40011e0 tx=0x7f2ec4009040 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 msgr2=0x7f2ed41554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c500 tx=0x7f2ebc00c9c0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 shutdown_connections 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2eb006be30 0x7f2eb006e2f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ed415b300 0x7f2ed414a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ed4072d30 0x7f2ed41554e0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 --2- 172.21.15.28:0/4074942467 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ed406f8d0 0x7f2ed41f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 >> 172.21.15.28:0/4074942467 conn(0x7f2ed4067d70 msgr2=0x7f2ed40b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.145+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 shutdown_connections 2024-09-10T10:55:36.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.146+0000 7f2eda200640 1 -- 172.21.15.28:0/4074942467 wait complete. 2024-09-10T10:55:36.162 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:55:36.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.317+0000 7f41f9400640 1 Processor -- start 2024-09-10T10:55:36.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.317+0000 7f41f9400640 1 -- start start 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f41f415b300 0x7f41f40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f9400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f40cdbe0 con 0x7f41f406f8d0 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f9400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f40cd900 con 0x7f41f40cd3f0 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41f40728a0 con 0x7f41f415b300 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f3200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f41f406f8d0 0x7f41f415b6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46122/0 (socket says 172.21.15.28:46122) 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f3200640 1 -- 172.21.15.28:0/3552087324 learned_addr learned my addr 172.21.15.28:0/3552087324 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f2800640 1 --2- 172.21.15.28:0/3552087324 >> v2:172.21.15.138:3300/0 conn(0x7f41f415b300 0x7f41f40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915805878 0 0) 0x7f41f40cdbe0 con 0x7f41f406f8d0 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.319+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41d8005730 con 0x7f41f406f8d0 2024-09-10T10:55:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.320+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 531737143 0 0) 0x7f41f40cd900 con 0x7f41f40cd3f0 2024-09-10T10:55:36.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.320+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41f40cdbe0 con 0x7f41f40cd3f0 2024-09-10T10:55:36.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.320+0000 7f41f2800640 1 -- 172.21.15.28:0/3552087324 >> v1:172.21.15.138:6789/0 conn(0x7f41f40cd3f0 legacy=0x7f41f406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.320+0000 7f41f2800640 1 -- 172.21.15.28:0/3552087324 >> v1:172.21.15.28:6789/0 conn(0x7f41f406f8d0 legacy=0x7f41f415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.320+0000 7f41f2800640 1 -- 172.21.15.28:0/3552087324 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f4072250 con 0x7f41f415b300 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.321+0000 7f41f2800640 1 --2- 172.21.15.28:0/3552087324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f40cceb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f41e40097a0 tx=0x7f41e402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44349f10a4d7f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.321+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4037070 con 0x7f41f415b300 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.321+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4032440 con 0x7f41f415b300 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.321+0000 7f41f1e00640 1 -- 172.21.15.28:0/3552087324 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4032810 con 0x7f41f415b300 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 -- 172.21.15.28:0/3552087324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 msgr2=0x7f41f40cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 --2- 172.21.15.28:0/3552087324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f40cceb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f41e40097a0 tx=0x7f41e402a230 comp rx=0 tx=0).stop 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 -- 172.21.15.28:0/3552087324 shutdown_connections 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 --2- 172.21.15.28:0/3552087324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f40cceb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 -- 172.21.15.28:0/3552087324 >> 172.21.15.28:0/3552087324 conn(0x7f41f4067d70 msgr2=0x7f41f406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 -- 172.21.15.28:0/3552087324 shutdown_connections 2024-09-10T10:55:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.322+0000 7f41f9400640 1 -- 172.21.15.28:0/3552087324 wait complete. 2024-09-10T10:55:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.323+0000 7f41f9400640 1 Processor -- start 2024-09-10T10:55:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.323+0000 7f41f9400640 1 -- start start 2024-09-10T10:55:36.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f406f8d0 0x7f41f4153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f41e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40d0160 con 0x7f41f40cd3f0 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40cffe0 con 0x7f41f415b300 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.324+0000 7f41f9400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f41f4072d30 con 0x7f41f406f8d0 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43874/0 (socket says 172.21.15.28:43874) 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 -- 172.21.15.28:0/2989569867 learned_addr learned my addr 172.21.15.28:0/2989569867 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41f2800640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f406f8d0 0x7f41f4153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41f3200640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f41e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 -- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f406f8d0 msgr2=0x7f41f4153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f406f8d0 0x7f41f4153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 -- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 msgr2=0x7f41f41e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f41e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 -- 172.21.15.28:0/2989569867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f41e4c50 con 0x7f41f40cd3f0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41f3200640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f41e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.325+0000 7f41ebe00640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c4c0 tx=0x7f41dc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.326+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0052d0 con 0x7f41f40cd3f0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.326+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc005470 con 0x7f41f40cd3f0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.326+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc012870 con 0x7f41f40cd3f0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.326+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f41e8a30 con 0x7f41f40cd3f0 2024-09-10T10:55:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.326+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f41e8ff0 con 0x7f41f40cd3f0 2024-09-10T10:55:36.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.327+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f4075670 con 0x7f41f40cd3f0 2024-09-10T10:55:36.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.328+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41dc012a10 con 0x7f41f40cd3f0 2024-09-10T10:55:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.328+0000 7f41eaa00640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 0x7f41d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.329+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f41dc08bb80 con 0x7f41f40cd3f0 2024-09-10T10:55:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.329+0000 7f41f2800640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 0x7f41d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.330+0000 7f41f2800640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 0x7f41d006e2f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f41e4034000 tx=0x7f41e402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.333+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f41f4075670 con 0x7f41f40cd3f0 2024-09-10T10:55:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.551+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f41f4158780 con 0x7f41f40cd3f0 2024-09-10T10:55:36.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.766+0000 7f41eaa00640 1 -- 172.21.15.28:0/2989569867 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v333) ==== 161+0+0 (secure 0 0 0) 0x7f41dc0594e0 con 0x7f41f40cd3f0 2024-09-10T10:55:36.767 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 msgr2=0x7f41d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 0x7f41d006e2f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f41e4034000 tx=0x7f41e402e000 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 msgr2=0x7f41f41540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c4c0 tx=0x7f41dc00c980 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 shutdown_connections 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f41d006be30 0x7f41d006e2f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41f415b300 0x7f41f41e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41f40cd3f0 0x7f41f41540a0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 --2- 172.21.15.28:0/2989569867 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41f406f8d0 0x7f41f4153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.768+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 >> 172.21.15.28:0/2989569867 conn(0x7f41f4067d70 msgr2=0x7f41f40af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.769+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 shutdown_connections 2024-09-10T10:55:36.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.769+0000 7f41f9400640 1 -- 172.21.15.28:0/2989569867 wait complete. 2024-09-10T10:55:36.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:55:36.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.944+0000 7f7566600640 1 Processor -- start 2024-09-10T10:55:36.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.944+0000 7f7566600640 1 -- start start 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7566600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75600b4800 0x7f75600aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7566600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600d0370 0x7f75600d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7566600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75600af7a0 con 0x7f756015c310 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7566600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f75600afa80 con 0x7f75600b4800 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7566600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7560160130 con 0x7f75600d0370 2024-09-10T10:55:36.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f755f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f756015c310 0x7f7560160930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46138/0 (socket says 172.21.15.28:46138) 2024-09-10T10:55:36.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f755f400640 1 -- 172.21.15.28:0/966153334 learned_addr learned my addr 172.21.15.28:0/966153334 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:36.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.945+0000 7f7557e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75600b4800 0x7f75600aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.946+0000 7f755fe00640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600d0370 0x7f75600d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.946+0000 7f7557e00640 1 -- 172.21.15.28:0/966153334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600d0370 msgr2=0x7f75600d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:36.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.946+0000 7f7557e00640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600d0370 0x7f75600d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.946+0000 7f7557e00640 1 -- 172.21.15.28:0/966153334 >> v1:172.21.15.28:6789/0 conn(0x7f756015c310 legacy=0x7f7560160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.946+0000 7f7557e00640 1 -- 172.21.15.28:0/966153334 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600af470 con 0x7f75600b4800 2024-09-10T10:55:36.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f7557e00640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75600b4800 0x7f75600aef30 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f75480097a0 tx=0x7f754802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49c9e069a266bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f755ea00640 1 -- 172.21.15.28:0/966153334 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548037070 con 0x7f75600b4800 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f755ea00640 1 -- 172.21.15.28:0/966153334 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7548032440 con 0x7f75600b4800 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f755ea00640 1 -- 172.21.15.28:0/966153334 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548032810 con 0x7f75600b4800 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f7566600640 1 -- 172.21.15.28:0/966153334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75600b4800 msgr2=0x7f75600aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.947+0000 7f7566600640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75600b4800 0x7f75600aef30 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f75480097a0 tx=0x7f754802a180 comp rx=0 tx=0).stop 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 -- 172.21.15.28:0/966153334 shutdown_connections 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600d0370 0x7f75600d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 --2- 172.21.15.28:0/966153334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75600b4800 0x7f75600aef30 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 -- 172.21.15.28:0/966153334 >> 172.21.15.28:0/966153334 conn(0x7f756006bdf0 msgr2=0x7f756006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:36.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 -- 172.21.15.28:0/966153334 shutdown_connections 2024-09-10T10:55:36.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.948+0000 7f7566600640 1 -- 172.21.15.28:0/966153334 wait complete. 2024-09-10T10:55:36.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.949+0000 7f7566600640 1 Processor -- start 2024-09-10T10:55:36.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.949+0000 7f7566600640 1 -- start start 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600b4800 0x7f75601563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75600d0370 0x7f75601ea450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f756006e090 con 0x7f756015c310 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f756006df10 con 0x7f75600d0370 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.950+0000 7f7566600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f756006e210 con 0x7f75600b4800 2024-09-10T10:55:36.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43900/0 (socket says 172.21.15.28:43900) 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 -- 172.21.15.28:0/3636212994 learned_addr learned my addr 172.21.15.28:0/3636212994 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755f400640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600b4800 0x7f75601563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f7557e00640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75600d0370 0x7f75601ea450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 -- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600b4800 msgr2=0x7f75601563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600b4800 0x7f75601563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 -- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75600d0370 msgr2=0x7f75601ea450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75600d0370 0x7f75601ea450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 -- 172.21.15.28:0/3636212994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601e93b0 con 0x7f756015c310 2024-09-10T10:55:36.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.951+0000 7f755fe00640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7558007e40 tx=0x7f755800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.952+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558014620 con 0x7f756015c310 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.952+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7558019070 con 0x7f756015c310 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.952+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558014a20 con 0x7f756015c310 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.952+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601e9690 con 0x7f756015c310 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.952+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601e9c50 con 0x7f756015c310 2024-09-10T10:55:36.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.953+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75600aef30 con 0x7f756015c310 2024-09-10T10:55:36.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.954+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7558014bc0 con 0x7f756015c310 2024-09-10T10:55:36.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.955+0000 7f7557400640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 0x7f753c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:36.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.955+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f755808c5e0 con 0x7f756015c310 2024-09-10T10:55:36.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.955+0000 7f755f400640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 0x7f753c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:36.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.956+0000 7f755f400640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 0x7f753c06e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7560146c80 tx=0x7f7550007e80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:36.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:36.959+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7558059980 con 0x7f756015c310 2024-09-10T10:55:37.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.180+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f75600cfea0 con 0x7f756015c310 2024-09-10T10:55:37.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.768+0000 7f7557400640 1 -- 172.21.15.28:0/3636212994 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v334) ==== 104+0+0 (secure 0 0 0) 0x7f755805dc80 con 0x7f756015c310 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.771+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 msgr2=0x7f753c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.771+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 0x7f753c06e2f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7560146c80 tx=0x7f7550007e80 comp rx=0 tx=0).stop 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.771+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 msgr2=0x7f7560146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.771+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f7558007e40 tx=0x7f755800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 shutdown_connections 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f753c06be30 0x7f753c06e2f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f756015c310 0x7f7560146460 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75600d0370 0x7f75601ea450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 --2- 172.21.15.28:0/3636212994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f75600b4800 0x7f75601563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 >> 172.21.15.28:0/3636212994 conn(0x7f756006bdf0 msgr2=0x7f7560073810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:37.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 shutdown_connections 2024-09-10T10:55:37.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.772+0000 7f7566600640 1 -- 172.21.15.28:0/3636212994 wait complete. 2024-09-10T10:55:37.791 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:55:37.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.954+0000 7f5e50a00640 1 Processor -- start 2024-09-10T10:55:37.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.954+0000 7f5e50a00640 1 -- start start 2024-09-10T10:55:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.955+0000 7f5e50a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e4c15b0f0 0x7f5e4c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.955+0000 7f5e50a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5e4c15ecf0 0x7f5e4c06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.955+0000 7f5e50a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4c072070 con 0x7f5e4c0cce40 2024-09-10T10:55:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.955+0000 7f5e50a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c15f260 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.955+0000 7f5e50a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c15f540 con 0x7f5e4c15ecf0 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e49e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e4c15b0f0 0x7f5e4c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e4a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5e4c0cce40 0x7f5e4c072550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46152/0 (socket says 172.21.15.28:46152) 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e4a800640 1 -- 172.21.15.28:0/1482057778 learned_addr learned my addr 172.21.15.28:0/1482057778 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e43e00640 1 --2- 172.21.15.28:0/1482057778 >> v2:172.21.15.138:3300/0 conn(0x7f5e4c15ecf0 0x7f5e4c06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e49e00640 1 -- 172.21.15.28:0/1482057778 >> v2:172.21.15.138:3300/0 conn(0x7f5e4c15ecf0 msgr2=0x7f5e4c06f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e49e00640 1 --2- 172.21.15.28:0/1482057778 >> v2:172.21.15.138:3300/0 conn(0x7f5e4c15ecf0 0x7f5e4c06f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e49e00640 1 -- 172.21.15.28:0/1482057778 >> v1:172.21.15.28:6789/0 conn(0x7f5e4c0cce40 legacy=0x7f5e4c072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.956+0000 7f5e49e00640 1 -- 172.21.15.28:0/1482057778 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c15f820 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.957+0000 7f5e49e00640 1 --2- 172.21.15.28:0/1482057778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c0b3c40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0097a0 tx=0x7f5e3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabc30ba6758e5db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e49400640 1 -- 172.21.15.28:0/1482057778 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037070 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e49400640 1 -- 172.21.15.28:0/1482057778 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c02ae50 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e49400640 1 -- 172.21.15.28:0/1482057778 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c032660 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 -- 172.21.15.28:0/1482057778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c15b0f0 msgr2=0x7f5e4c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1482057778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c0b3c40 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0097a0 tx=0x7f5e3c02a140 comp rx=0 tx=0).stop 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 -- 172.21.15.28:0/1482057778 shutdown_connections 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1482057778 >> v2:172.21.15.138:3300/0 conn(0x7f5e4c15ecf0 0x7f5e4c06f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1482057778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c0b3c40 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 -- 172.21.15.28:0/1482057778 >> 172.21.15.28:0/1482057778 conn(0x7f5e4c067d70 msgr2=0x7f5e4c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:37.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 -- 172.21.15.28:0/1482057778 shutdown_connections 2024-09-10T10:55:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.958+0000 7f5e50a00640 1 -- 172.21.15.28:0/1482057778 wait complete. 2024-09-10T10:55:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.960+0000 7f5e50a00640 1 Processor -- start 2024-09-10T10:55:37.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.960+0000 7f5e50a00640 1 -- start start 2024-09-10T10:55:37.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e4c15ecf0 0x7f5e4c159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c15b4d0 con 0x7f5e4c0cce40 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0b4180 con 0x7f5e4c15b0f0 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e50a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c0afa00 con 0x7f5e4c15ecf0 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e49e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e49e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43920/0 (socket says 172.21.15.28:43920) 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e49e00640 1 -- 172.21.15.28:0/1220002156 learned_addr learned my addr 172.21.15.28:0/1220002156 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e43e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.961+0000 7f5e4a800640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e4c15ecf0 0x7f5e4c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 -- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e4c15ecf0 msgr2=0x7f5e4c159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e4c15ecf0 0x7f5e4c159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 -- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c15b0f0 msgr2=0x7f5e4c159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:37.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:37.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 -- 172.21.15.28:0/1220002156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c159dc0 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e49e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035040 tx=0x7f5e3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c041070 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c03c460 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c037040 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c15a0a0 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.962+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c15a660 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.963+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c1568f0 con 0x7f5e4c0cce40 2024-09-10T10:55:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.964+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e3c03a070 con 0x7f5e4c0cce40 2024-09-10T10:55:37.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.965+0000 7f5e42000640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:37.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.965+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9744+0+0 (secure 0 0 0) 0x7f5e3c0ac2b0 con 0x7f5e4c0cce40 2024-09-10T10:55:37.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.966+0000 7f5e43e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:37.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.966+0000 7f5e43e00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 0x7f5e2806e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c1456f0 tx=0x7f5e34009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:37.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:37.969+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e3c0796e0 con 0x7f5e4c0cce40 2024-09-10T10:55:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.155+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5e4c156610 con 0x7f5e4c0cce40 2024-09-10T10:55:38.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.780+0000 7f5e42000640 1 -- 172.21.15.28:0/1220002156 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) ==== 181+0+0 (secure 0 0 0) 0x7f5e3c07d9e0 con 0x7f5e4c0cce40 2024-09-10T10:55:38.781 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:55:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 msgr2=0x7f5e2806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 0x7f5e2806e2f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c1456f0 tx=0x7f5e34009190 comp rx=0 tx=0).stop 2024-09-10T10:55:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 msgr2=0x7f5e4c13e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035040 tx=0x7f5e3c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 shutdown_connections 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5e2806be30 0x7f5e2806e2f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e4c15ecf0 0x7f5e4c159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c15b0f0 0x7f5e4c159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 --2- 172.21.15.28:0/1220002156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e4c0cce40 0x7f5e4c13e830 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.784+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 >> 172.21.15.28:0/1220002156 conn(0x7f5e4c067d70 msgr2=0x7f5e4c072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:38.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.785+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 shutdown_connections 2024-09-10T10:55:38.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.785+0000 7f5e50a00640 1 -- 172.21.15.28:0/1220002156 wait complete. 2024-09-10T10:55:38.805 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:55:38.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.968+0000 7f6c33200640 1 Processor -- start 2024-09-10T10:55:38.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.968+0000 7f6c33200640 1 -- start start 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.969+0000 7f6c33200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c340b5020 0x7f6c340b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.969+0000 7f6c33200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c340b34d0 con 0x7f6c3415bf50 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.969+0000 7f6c33200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c340b37b0 con 0x7f6c340af310 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.969+0000 7f6c33200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c340b5450 con 0x7f6c340b5020 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6c3415bf50 0x7f6c340aec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46164/0 (socket says 172.21.15.28:46164) 2024-09-10T10:55:38.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31e00640 1 -- 172.21.15.28:0/2206221690 learned_addr learned my addr 172.21.15.28:0/2206221690 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309873162 0 0) 0x7f6c340b34d0 con 0x7f6c3415bf50 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c18003520 con 0x7f6c3415bf50 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c32800640 1 --2- 172.21.15.28:0/2206221690 >> v2:172.21.15.138:3300/0 conn(0x7f6c340b5020 0x7f6c340b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 179162792 0 0) 0x7f6c18003520 con 0x7f6c3415bf50 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 >> v2:172.21.15.138:3300/0 conn(0x7f6c340b5020 msgr2=0x7f6c340b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 --2- 172.21.15.28:0/2206221690 >> v2:172.21.15.138:3300/0 conn(0x7f6c340b5020 0x7f6c340b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 >> v1:172.21.15.138:6789/0 conn(0x7f6c340af310 legacy=0x7f6c340b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.970+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c340b3a90 con 0x7f6c3415bf50 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f6c24001670 con 0x7f6c3415bf50 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 >> v1:172.21.15.28:6789/0 conn(0x7f6c3415bf50 legacy=0x7f6c340aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c180039b0 0x7f6c18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18004800 0x7f6c18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c340b5450 con 0x7f6c180051c0 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1800c110 con 0x7f6c18004800 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1800c310 con 0x7f6c180039b0 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c2be00640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18004800 0x7f6c18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c31e00640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c180039b0 0x7f6c18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 -- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c180039b0 msgr2=0x7f6c18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c180039b0 0x7f6c18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 -- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18004800 msgr2=0x7f6c18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c18004800 0x7f6c18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.971+0000 7f6c32800640 1 -- 172.21.15.28:0/2206221690 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18003520 con 0x7f6c180051c0 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.972+0000 7f6c32800640 1 -- 172.21.15.28:0/2206221690 reap_dead start 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.972+0000 7f6c32800640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c1800b9c0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007360 tx=0x7f6c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d841653bf9b865a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.972+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c009be0 con 0x7f6c180051c0 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c2c003bd0 con 0x7f6c180051c0 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c31400640 1 -- 172.21.15.28:0/2206221690 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c004020 con 0x7f6c180051c0 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 -- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 msgr2=0x7f6c1800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c1800b9c0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c007360 tx=0x7f6c18005830 comp rx=0 tx=0).stop 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 -- 172.21.15.28:0/2206221690 shutdown_connections 2024-09-10T10:55:38.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 --2- 172.21.15.28:0/2206221690 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c1800b9c0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 -- 172.21.15.28:0/2206221690 >> 172.21.15.28:0/2206221690 conn(0x7f6c3406c240 msgr2=0x7f6c340b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:38.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 -- 172.21.15.28:0/2206221690 shutdown_connections 2024-09-10T10:55:38.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.973+0000 7f6c33200640 1 -- 172.21.15.28:0/2206221690 wait complete. 2024-09-10T10:55:38.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.975+0000 7f6c33200640 1 Processor -- start 2024-09-10T10:55:38.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.975+0000 7f6c33200640 1 -- start start 2024-09-10T10:55:38.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 0x7f6c3414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c3415c9c0 0x7f6c341503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c340b3e00 con 0x7f6c180051c0 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c340b3280 con 0x7f6c3415c540 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c33200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c340b4290 con 0x7f6c3415c9c0 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c31e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c31e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43928/0 (socket says 172.21.15.28:43928) 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c31e00640 1 -- 172.21.15.28:0/2995546575 learned_addr learned my addr 172.21.15.28:0/2995546575 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c32800640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c3415c9c0 0x7f6c341503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c2be00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 0x7f6c3414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c31e00640 1 -- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c3415c9c0 msgr2=0x7f6c341503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.976+0000 7f6c31e00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c3415c9c0 0x7f6c341503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c31e00640 1 -- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 msgr2=0x7f6c3414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c31e00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 0x7f6c3414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c31e00640 1 -- 172.21.15.28:0/2995546575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3415cf20 con 0x7f6c180051c0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c2be00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 0x7f6c3414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c31e00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e730 tx=0x7f6c2400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400ee40 con 0x7f6c180051c0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c24018070 con 0x7f6c180051c0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2400ca90 con 0x7f6c180051c0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3415d200 con 0x7f6c180051c0 2024-09-10T10:55:38.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.977+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c3415d7c0 con 0x7f6c180051c0 2024-09-10T10:55:38.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.978+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c340b4620 con 0x7f6c180051c0 2024-09-10T10:55:38.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.979+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6c2400cc30 con 0x7f6c180051c0 2024-09-10T10:55:38.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.980+0000 7f6c2a000640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 0x7f6c1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:38.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.980+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9277+0+0 (secure 0 0 0) 0x7f6c2408b820 con 0x7f6c180051c0 2024-09-10T10:55:38.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.980+0000 7f6c2be00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 0x7f6c1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:38.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.981+0000 7f6c2be00640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 0x7f6c1006e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:38.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:38.984+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c24059330 con 0x7f6c180051c0 2024-09-10T10:55:39.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.166+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c34157840 con 0x7f6c180051c0 2024-09-10T10:55:39.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.781+0000 7f6c2a000640 1 -- 172.21.15.28:0/2995546575 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) ==== 169+0+0 (secure 0 0 0) 0x7f6c24058c30 con 0x7f6c180051c0 2024-09-10T10:55:39.784 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.784+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 msgr2=0x7f6c1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.784+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 0x7f6c1006e320 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c000c00 tx=0x7f6c1c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.784+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 msgr2=0x7f6c3414c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.784+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400e730 tx=0x7f6c2400ebf0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 shutdown_connections 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6c1006be60 0x7f6c1006e320 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c3415c9c0 0x7f6c341503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c3415c540 0x7f6c3414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 --2- 172.21.15.28:0/2995546575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6c180051c0 0x7f6c3414c950 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 >> 172.21.15.28:0/2995546575 conn(0x7f6c3406c240 msgr2=0x7f6c340b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 shutdown_connections 2024-09-10T10:55:39.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.785+0000 7f6c33200640 1 -- 172.21.15.28:0/2995546575 wait complete. 2024-09-10T10:55:39.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:55:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.964+0000 7f116fa00640 1 Processor -- start 2024-09-10T10:55:39.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.964+0000 7f116fa00640 1 -- start start 2024-09-10T10:55:39.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116fa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1168161d00 0x7f11680b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116fa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11680d4300 con 0x7f1168161920 2024-09-10T10:55:39.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11680b49a0 con 0x7f11680b62b0 2024-09-10T10:55:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116fa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f11680b4c40 con 0x7f1168161d00 2024-09-10T10:55:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1168161920 0x7f11680b5ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46178/0 (socket says 172.21.15.28:46178) 2024-09-10T10:55:39.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.965+0000 7f116d000640 1 -- 172.21.15.28:0/3138133276 learned_addr learned my addr 172.21.15.28:0/3138133276 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2015554554 0 0) 0x7f11680d4300 con 0x7f1168161920 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1150003520 con 0x7f1168161920 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167e00640 1 --2- 172.21.15.28:0/3138133276 >> v2:172.21.15.138:3300/0 conn(0x7f1168161d00 0x7f11680b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3779126948 0 0) 0x7f1150003520 con 0x7f1168161920 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 >> v2:172.21.15.138:3300/0 conn(0x7f1168161d00 msgr2=0x7f11680b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 --2- 172.21.15.28:0/3138133276 >> v2:172.21.15.138:3300/0 conn(0x7f1168161d00 0x7f11680b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 >> v1:172.21.15.138:6789/0 conn(0x7f11680b62b0 legacy=0x7f11680d2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11680d4db0 con 0x7f1168161920 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f115c001670 con 0x7f1168161920 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 >> v1:172.21.15.28:6789/0 conn(0x7f1168161920 legacy=0x7f11680b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11500039b0 0x7f1150003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 0x7f1150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11500053c0 0x7f115000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11680b4c40 con 0x7f1150004800 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f115000c250 con 0x7f11500039b0 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.966+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f115000c450 con 0x7f11500053c0 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 0x7f1150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f116d000640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11500039b0 0x7f1150003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f116da00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11500053c0 0x7f115000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 -- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11500053c0 msgr2=0x7f115000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11500053c0 0x7f115000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 -- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11500039b0 msgr2=0x7f1150003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11500039b0 0x7f1150003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f1167e00640 1 -- 172.21.15.28:0/3138133276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1150003520 con 0x7f1150004800 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f116da00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11500053c0 0x7f115000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.967+0000 7f116da00640 1 -- 172.21.15.28:0/3138133276 reap_dead start 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.968+0000 7f1167e00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 0x7f1150004c80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f115400ba30 tx=0x7f1150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba1662193ed5703 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.968+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154030070 con 0x7f1150004800 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.968+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115402ade0 con 0x7f1150004800 2024-09-10T10:55:39.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.968+0000 7f1167400640 1 -- 172.21.15.28:0/3138133276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400b600 con 0x7f1150004800 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 -- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 msgr2=0x7f1150004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 0x7f1150004c80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f115400ba30 tx=0x7f1150005830 comp rx=0 tx=0).stop 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 -- 172.21.15.28:0/3138133276 shutdown_connections 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 --2- 172.21.15.28:0/3138133276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1150004800 0x7f1150004c80 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 -- 172.21.15.28:0/3138133276 >> 172.21.15.28:0/3138133276 conn(0x7f116808bfc0 msgr2=0x7f11680d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:39.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 -- 172.21.15.28:0/3138133276 shutdown_connections 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.969+0000 7f116fa00640 1 -- 172.21.15.28:0/3138133276 wait complete. 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.971+0000 7f116fa00640 1 Processor -- start 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.971+0000 7f116fa00640 1 -- start start 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 0x7f11680ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f116814f820 0x7f11680c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11680b5090 con 0x7f11680b9e00 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11680b68d0 con 0x7f116814f820 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116fa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11680b5210 con 0x7f1150004800 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43954/0 (socket says 172.21.15.28:43954) 2024-09-10T10:55:39.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 -- 172.21.15.28:0/2272236465 learned_addr learned my addr 172.21.15.28:0/2272236465 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116d000640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 0x7f11680ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116da00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f116814f820 0x7f11680c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 -- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 msgr2=0x7f11680ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 0x7f11680ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 -- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f116814f820 msgr2=0x7f11680c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f116814f820 0x7f11680c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f1167e00640 1 -- 172.21.15.28:0/2272236465 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11680b8d60 con 0x7f11680b9e00 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.972+0000 7f116d000640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 0x7f11680ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f1167e00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f1154035d90 tx=0x7f11540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154043030 con 0x7f11680b9e00 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f115402abe0 con 0x7f11680b9e00 2024-09-10T10:55:39.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11680b9040 con 0x7f11680b9e00 2024-09-10T10:55:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400b600 con 0x7f11680b9e00 2024-09-10T10:55:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.973+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11680b9600 con 0x7f11680b9e00 2024-09-10T10:55:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.974+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11680b5ba0 con 0x7f11680b9e00 2024-09-10T10:55:39.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.975+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1154046070 con 0x7f11680b9e00 2024-09-10T10:55:39.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.976+0000 7f1164c00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 0x7f114806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:39.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.976+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7f11540ac5d0 con 0x7f11680b9e00 2024-09-10T10:55:39.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.977+0000 7f116d000640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 0x7f114806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:39.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.977+0000 7f116d000640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 0x7f114806e320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f115c007fd0 tx=0x7f115c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:39.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:39.980+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f115403c3a0 con 0x7f11680b9e00 2024-09-10T10:55:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.163+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f11680b6c20 con 0x7f11680b9e00 2024-09-10T10:55:40.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.167+0000 7f1164c00640 1 -- 172.21.15.28:0/2272236465 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f115407a2a0 con 0x7f11680b9e00 2024-09-10T10:55:40.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 msgr2=0x7f114806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 0x7f114806e320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f115c007fd0 tx=0x7f115c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:40.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 msgr2=0x7f11680b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f1154035d90 tx=0x7f11540046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 shutdown_connections 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f114806be60 0x7f114806e320 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f116814f820 0x7f11680c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11680b9e00 0x7f11680b8510 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 --2- 172.21.15.28:0/2272236465 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1150004800 0x7f11680ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 >> 172.21.15.28:0/2272236465 conn(0x7f116808bfc0 msgr2=0x7f11680746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 shutdown_connections 2024-09-10T10:55:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.169+0000 7f116fa00640 1 -- 172.21.15.28:0/2272236465 wait complete. 2024-09-10T10:55:40.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion' 2024-09-10T10:55:40.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.341+0000 7f38bec00640 1 Processor -- start 2024-09-10T10:55:40.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.341+0000 7f38bec00640 1 -- start start 2024-09-10T10:55:40.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.342+0000 7f38bec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.342+0000 7f38bec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38b80b6690 0x7f38b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.342+0000 7f38bec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38b80d3690 con 0x7f38b80d30f0 2024-09-10T10:55:40.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.342+0000 7f38bec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f38b806d530 con 0x7f38b80b6690 2024-09-10T10:55:40.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.342+0000 7f38bec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38b80d3970 con 0x7f38b815e5f0 2024-09-10T10:55:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:52908/0 (socket says 172.21.15.28:52908) 2024-09-10T10:55:40.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 -- 172.21.15.28:0/1162586881 learned_addr learned my addr 172.21.15.28:0/1162586881 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38bcc00640 1 --2- 172.21.15.28:0/1162586881 >> v2:172.21.15.138:3300/0 conn(0x7f38b80b6690 0x7f38b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 -- 172.21.15.28:0/1162586881 >> v2:172.21.15.138:3300/0 conn(0x7f38b80b6690 msgr2=0x7f38b806d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 --2- 172.21.15.28:0/1162586881 >> v2:172.21.15.138:3300/0 conn(0x7f38b80b6690 0x7f38b806d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 -- 172.21.15.28:0/1162586881 >> v1:172.21.15.138:6789/0 conn(0x7f38b80d30f0 legacy=0x7f38b80b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b6a00640 1 -- 172.21.15.28:0/1162586881 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3130723155 0 0) 0x7f38b80d3690 con 0x7f38b80d30f0 2024-09-10T10:55:40.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.343+0000 7f38b7e00640 1 -- 172.21.15.28:0/1162586881 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b806cee0 con 0x7f38b815e5f0 2024-09-10T10:55:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.344+0000 7f38b7e00640 1 --2- 172.21.15.28:0/1162586881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5149c866c0e1fcc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.344+0000 7f38b6a00640 1 -- 172.21.15.28:0/1162586881 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8037070 con 0x7f38b815e5f0 2024-09-10T10:55:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38b6a00640 1 -- 172.21.15.28:0/1162586881 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a8032440 con 0x7f38b815e5f0 2024-09-10T10:55:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38b6a00640 1 -- 172.21.15.28:0/1162586881 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8032810 con 0x7f38b815e5f0 2024-09-10T10:55:40.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 -- 172.21.15.28:0/1162586881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 msgr2=0x7f38b80d4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 --2- 172.21.15.28:0/1162586881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a802a270 comp rx=0 tx=0).stop 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 -- 172.21.15.28:0/1162586881 shutdown_connections 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 --2- 172.21.15.28:0/1162586881 >> v2:172.21.15.138:3300/0 conn(0x7f38b80b6690 0x7f38b806d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 --2- 172.21.15.28:0/1162586881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80d4dd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 -- 172.21.15.28:0/1162586881 >> 172.21.15.28:0/1162586881 conn(0x7f38b808bfc0 msgr2=0x7f38b80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 -- 172.21.15.28:0/1162586881 shutdown_connections 2024-09-10T10:55:40.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.345+0000 7f38bec00640 1 -- 172.21.15.28:0/1162586881 wait complete. 2024-09-10T10:55:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.347+0000 7f38bec00640 1 Processor -- start 2024-09-10T10:55:40.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.347+0000 7f38bec00640 1 -- start start 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b80d30f0 0x7f38b80b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f38b80d5310 con 0x7f38b80b6690 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38b815e9d0 con 0x7f38b80d30f0 2024-09-10T10:55:40.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bec00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38b806df00 con 0x7f38b815e5f0 2024-09-10T10:55:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43956/0 (socket says 172.21.15.28:43956) 2024-09-10T10:55:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38b7e00640 1 -- 172.21.15.28:0/3239450088 learned_addr learned my addr 172.21.15.28:0/3239450088 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38b7400640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b80d30f0 0x7f38b80b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.348+0000 7f38bcc00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 -- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 msgr2=0x7f38b80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 -- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b80d30f0 msgr2=0x7f38b80b8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b80d30f0 0x7f38b80b8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 -- 172.21.15.28:0/3239450088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b80cd990 con 0x7f38b80b6690 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38b7e00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8037040 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a8004030 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a8004400 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.349+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b80cc330 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.350+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b80cdc30 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.350+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b806d9c0 con 0x7f38b80b6690 2024-09-10T10:55:40.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.351+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f38a803a070 con 0x7f38b80b6690 2024-09-10T10:55:40.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.352+0000 7f38afe00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 0x7f389806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.352+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7f38a80abfa0 con 0x7f38b80b6690 2024-09-10T10:55:40.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.353+0000 7f38b7400640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 0x7f389806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.353+0000 7f38b7400640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 0x7f389806e320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38b80b9030 tx=0x7f38a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.356+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38a8041020 con 0x7f38b80b6690 2024-09-10T10:55:40.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.613+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]} v 0) -- 0x7f38b80d4980 con 0x7f38b80b6690 2024-09-10T10:55:40.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.790+0000 7f38afe00640 1 -- 172.21.15.28:0/3239450088 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_object_deletion"]}]=0 v610) ==== 135+0+0 (secure 0 0 0) 0x7f38a80796a0 con 0x7f38b80b6690 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.792+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 msgr2=0x7f389806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.792+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 0x7f389806e320 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f38b80b9030 tx=0x7f38a4009040 comp rx=0 tx=0).stop 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.792+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 msgr2=0x7f38b8157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 shutdown_connections 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f389806be60 0x7f389806e320 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b815e5f0 0x7f38b80cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b80d30f0 0x7f38b80b8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 --2- 172.21.15.28:0/3239450088 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38b80b6690 0x7f38b8157eb0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 >> 172.21.15.28:0/3239450088 conn(0x7f38b808bfc0 msgr2=0x7f38b80b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 shutdown_connections 2024-09-10T10:55:40.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.793+0000 7f38bec00640 1 -- 172.21.15.28:0/3239450088 wait complete. 2024-09-10T10:55:40.811 INFO:tasks.cephfs_test_runner:That the MDS has a clean 'damaged' response to loss of any single metadata object ... ok 2024-09-10T10:55:40.811 INFO:tasks.cephfs_test_runner:Starting test: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) That errors encountered during opening inos are properly propagated 2024-09-10T10:55:40.811 INFO:tasks.cephfs_test_runner:test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:55:40.812 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:40.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.967+0000 7f1c7ce00640 1 Processor -- start 2024-09-10T10:55:40.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.967+0000 7f1c7ce00640 1 -- start start 2024-09-10T10:55:40.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c7ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c7ce00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c780d1f90 0x7f1c780cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c7ce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c780cf000 con 0x7f1c7815e660 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c7ce00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1c7806d3e0 con 0x7f1c780d1f90 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c7ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c780b37d0 con 0x7f1c780d0dd0 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c76a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1c7815e660 0x7f1c7806ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46180/0 (socket says 172.21.15.28:46180) 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.968+0000 7f1c76a00640 1 -- 172.21.15.28:0/2198906181 learned_addr learned my addr 172.21.15.28:0/2198906181 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77e00640 1 --2- 172.21.15.28:0/2198906181 >> v2:172.21.15.138:3300/0 conn(0x7f1c780d1f90 0x7f1c780cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77400640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c76000640 1 -- 172.21.15.28:0/2198906181 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088622554 0 0) 0x7f1c780cf000 con 0x7f1c7815e660 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c76000640 1 -- 172.21.15.28:0/2198906181 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c5c003520 con 0x7f1c7815e660 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77e00640 1 -- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 msgr2=0x7f1c780d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77e00640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77e00640 1 -- 172.21.15.28:0/2198906181 >> v1:172.21.15.28:6789/0 conn(0x7f1c7815e660 legacy=0x7f1c7806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.969+0000 7f1c77e00640 1 -- 172.21.15.28:0/2198906181 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c780d16f0 con 0x7f1c780d1f90 2024-09-10T10:55:40.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.970+0000 7f1c77e00640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c780d1f90 0x7f1c780cf7c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009d90 tx=0x7f1c7002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9568b8d146db05dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.970+0000 7f1c76000640 1 -- 172.21.15.28:0/2198906181 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007740 con 0x7f1c780d1f90 2024-09-10T10:55:40.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.970+0000 7f1c76000640 1 -- 172.21.15.28:0/2198906181 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c700078e0 con 0x7f1c780d1f90 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.970+0000 7f1c76000640 1 -- 172.21.15.28:0/2198906181 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007cb0 con 0x7f1c780d1f90 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c780d1f90 msgr2=0x7f1c780cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c780d1f90 0x7f1c780cf7c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1c70009d90 tx=0x7f1c7002a660 comp rx=0 tx=0).stop 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2198906181 shutdown_connections 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c780d1f90 0x7f1c780cf7c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2198906181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2198906181 >> 172.21.15.28:0/2198906181 conn(0x7f1c7808bfc0 msgr2=0x7f1c780d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2198906181 shutdown_connections 2024-09-10T10:55:40.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.971+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2198906181 wait complete. 2024-09-10T10:55:40.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.973+0000 7f1c7ce00640 1 Processor -- start 2024-09-10T10:55:40.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.973+0000 7f1c7ce00640 1 -- start start 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7815e660 0x7f1c78150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c780d2c60 con 0x7f1c780d1f90 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c780cf330 con 0x7f1c7815e660 2024-09-10T10:55:40.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c7ce00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c780d2f40 con 0x7f1c780d0dd0 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43964/0 (socket says 172.21.15.28:43964) 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 -- 172.21.15.28:0/2227102074 learned_addr learned my addr 172.21.15.28:0/2227102074 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c77400640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c77e00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7815e660 0x7f1c78150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 -- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 msgr2=0x7f1c780b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780b83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 -- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7815e660 msgr2=0x7f1c78150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7815e660 0x7f1c78150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:40.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.974+0000 7f1c76a00640 1 -- 172.21.15.28:0/2227102074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c7815cc90 con 0x7f1c780d1f90 2024-09-10T10:55:40.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c76a00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c3f0 tx=0x7f1c64001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c64017070 con 0x7f1c780d1f90 2024-09-10T10:55:40.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c64005ce0 con 0x7f1c780d1f90 2024-09-10T10:55:40.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640072f0 con 0x7f1c780d1f90 2024-09-10T10:55:40.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781e6490 con 0x7f1c780d1f90 2024-09-10T10:55:40.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.975+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c780c8ad0 con 0x7f1c780d1f90 2024-09-10T10:55:40.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.976+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c780b6ce0 con 0x7f1c780d1f90 2024-09-10T10:55:40.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.977+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c6401a070 con 0x7f1c780d1f90 2024-09-10T10:55:40.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.978+0000 7f1c6f400640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 0x7f1c5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:40.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.978+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7f1c6408ada0 con 0x7f1c780d1f90 2024-09-10T10:55:40.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.978+0000 7f1c77400640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 0x7f1c5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:40.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.979+0000 7f1c77400640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 0x7f1c5806e2f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ad0 tx=0x7f1c68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:40.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:40.982+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1c780b6ce0 con 0x7f1c780d1f90 2024-09-10T10:55:40.988 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:55:40.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:55:41.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.169+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c780d0bb0 con 0x7f1c780d1f90 2024-09-10T10:55:41.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.170+0000 7f1c6f400640 1 -- 172.21.15.28:0/2227102074 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+18381 (secure 0 0 0) 0x7f1c64058af0 con 0x7f1c780d1f90 2024-09-10T10:55:41.172 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:41.172 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":142,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:39.780637+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6833/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6834/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6832/382198082":"2024-09-11T10:54:53.753232+0000","172.21.15.28:6834/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/2572217302":"2024-09-11T10:54:48.320770+0000","172.21.15.28:6832/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6835/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/509398397":"2024-09-11T10:54:14.076481+0000","172.21.15.28:6833/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6834/3154009439":"2024-09-11T10:55:10.311855+0000","172.21.15.28:6832/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6834/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6833/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6832/4294783280":"2024-09-11T10:52:00.689161+0000","172.21.15.28:6834/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6833/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6834/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6835/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6834/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6833/2888169760":"2024-09-11T10:51:46.682332+0000","172.21.15.28:6833/1056314461":"2024-09-11T10:54:36.849623+0000","172.21.15.28:6832/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6835/4204725630":"2024-09-11T10:53:44.760451+0000","172.21.15.28:6832/41755343":"2024-09-11T10:51:42.728295+0000","172.21.15.28:6834/3452166242":"2024-09-11T10:51:28.234007+0000","172.21.15.28:6833/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6833/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6832/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6832/635113338":"2024-09-11T10:55:15.756291+0000","172.21.15.28:6832/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6835/2774580459":"2024-09-11T10:52:14.190678+0000","172.21.15.28:6835/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6835/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6833/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6834/2312444496":"2024-09-11T10:52:56.696320+0000","172.21.15.28:6834/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6834/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6835/1408648899":"2024-09-11T10:51:00.168974+0000","172.21.15.28:6835/3446889436":"2024-09-11T10:50:37.902654+0000","172.21.15.28:6834/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6832/1706206889":"2024-09-11T10:53:40.729366+0000","172.21.15.28:6834/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6832/786875524":"2024-09-11T10:51:40.159679+0000","172.21.15.28:6832/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6832/208022459":"2024-09-11T10:53:48.793090+0000","172.21.15.28:6833/3986730211":"2024-09-11T10:52:16.694472+0000","172.21.15.28:6832/1369429206":"2024-09-11T10:51:53.780213+0000","172.21.15.28:6835/1234809424":"2024-09-11T10:51:14.743287+0000","172.21.15.28:6834/1314136636":"2024-09-11T10:52:23.789551+0000","172.21.15.28:6835/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6834/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6835/2147565603":"2024-09-11T10:54:09.041224+0000","172.21.15.28:6833/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6835/205773431":"2024-09-11T10:52:34.853419+0000","172.21.15.28:6833/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6835/702039081":"2024-09-11T10:52:39.904721+0000","172.21.15.28:6835/911593499":"2024-09-11T10:55:29.729157+0000","172.21.15.28:6832/3765569847":"2024-09-11T10:52:30.832413+0000","172.21.15.28:6832/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/3186644301":"2024-09-11T10:51:06.680487+0000","172.21.15.28:6833/3539657083":"2024-09-11T10:53:36.707389+0000","172.21.15.28:6833/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6832/4027758898":"2024-09-11T10:53:04.775511+0000","172.21.15.28:6835/2836905257":"2024-09-11T10:53:08.808221+0000","172.21.15.28:6834/236564669":"2024-09-11T10:51:10.699044+0000","172.21.15.28:6834/2552808349":"2024-09-11T10:52:54.674913+0000","172.21.15.28:6833/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6833/2814091387":"2024-09-11T10:50:55.135868+0000","172.21.15.28:6832/2095575543":"2024-09-11T10:53:12.842835+0000","172.21.15.28:6835/1093900039":"2024-09-11T10:53:16.877889+0000","172.21.15.28:6834/2349436086":"2024-09-11T10:53:20.909728+0000","172.21.15.28:6834/918067600":"2024-09-11T10:55:22.793837+0000","172.21.15.28:6835/3588871837":"2024-09-11T10:53:00.745487+0000","172.21.15.28:6835/2097945693":"2024-09-11T10:53:34.409937+0000","172.21.15.28:6833/4204725630":"2024-09-11T10:53:44.760451+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:41.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 msgr2=0x7f1c5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 0x7f1c5806e2f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ad0 tx=0x7f1c68008040 comp rx=0 tx=0).stop 2024-09-10T10:55:41.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 msgr2=0x7f1c781e5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c3f0 tx=0x7f1c64001e70 comp rx=0 tx=0).stop 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 shutdown_connections 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f1c5806be30 0x7f1c5806e2f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c7815e660 0x7f1c78150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c780d1f90 0x7f1c781e5d90 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 --2- 172.21.15.28:0/2227102074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c780d0dd0 0x7f1c780b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 >> 172.21.15.28:0/2227102074 conn(0x7f1c7808bfc0 msgr2=0x7f1c780b65d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 shutdown_connections 2024-09-10T10:55:41.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.172+0000 7f1c7ce00640 1 -- 172.21.15.28:0/2227102074 wait complete. 2024-09-10T10:55:41.190 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T10:43:36.538469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T10:55:41.190 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:55:41.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.347+0000 7f6837800640 1 Processor -- start 2024-09-10T10:55:41.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.347+0000 7f6837800640 1 -- start start 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f68300b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 0x7f68300ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6830072050 0x7f6830072500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f68300b4620 con 0x7f6830072050 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f68300ce100 con 0x7f68300b4a20 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6837800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68300ce3a0 con 0x7f683015e900 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6835800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6830072050 0x7f6830072500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6835800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6830072050 0x7f6830072500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43976/0 (socket says 172.21.15.28:43976) 2024-09-10T10:55:41.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6835800640 1 -- 172.21.15.28:0/271950828 learned_addr learned my addr 172.21.15.28:0/271950828 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6834e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f68300b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f682fe00640 1 --2- 172.21.15.28:0/271950828 >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 0x7f68300ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6835800640 1 -- 172.21.15.28:0/271950828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 msgr2=0x7f68300b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.349+0000 7f6835800640 1 --2- 172.21.15.28:0/271950828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f68300b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.350+0000 7f6835800640 1 -- 172.21.15.28:0/271950828 >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 msgr2=0x7f68300ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.350+0000 7f6835800640 1 --2- 172.21.15.28:0/271950828 >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 0x7f68300ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.350+0000 7f6835800640 1 -- 172.21.15.28:0/271950828 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68300ce680 con 0x7f6830072050 2024-09-10T10:55:41.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.350+0000 7f682fe00640 1 --2- 172.21.15.28:0/271950828 >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 0x7f68300ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6835800640 1 --2- 172.21.15.28:0/271950828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6830072050 0x7f6830072500 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f68300cfe80 tx=0x7f682802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca80371b25461f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f682f400640 1 -- 172.21.15.28:0/271950828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828036070 con 0x7f6830072050 2024-09-10T10:55:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f682f400640 1 -- 172.21.15.28:0/271950828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828031600 con 0x7f6830072050 2024-09-10T10:55:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f682f400640 1 -- 172.21.15.28:0/271950828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68280319d0 con 0x7f6830072050 2024-09-10T10:55:41.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 -- 172.21.15.28:0/271950828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6830072050 msgr2=0x7f6830072500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 --2- 172.21.15.28:0/271950828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6830072050 0x7f6830072500 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f68300cfe80 tx=0x7f682802a610 comp rx=0 tx=0).stop 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 -- 172.21.15.28:0/271950828 shutdown_connections 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 --2- 172.21.15.28:0/271950828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6830072050 0x7f6830072500 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 --2- 172.21.15.28:0/271950828 >> v2:172.21.15.138:3300/0 conn(0x7f68300b4a20 0x7f68300ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 --2- 172.21.15.28:0/271950828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f68300b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.351+0000 7f6837800640 1 -- 172.21.15.28:0/271950828 >> 172.21.15.28:0/271950828 conn(0x7f683006bdf0 msgr2=0x7f683006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.352+0000 7f6837800640 1 -- 172.21.15.28:0/271950828 shutdown_connections 2024-09-10T10:55:41.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.352+0000 7f6837800640 1 -- 172.21.15.28:0/271950828 wait complete. 2024-09-10T10:55:41.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.353+0000 7f6837800640 1 Processor -- start 2024-09-10T10:55:41.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.353+0000 7f6837800640 1 -- start start 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 0x7f6830156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f683006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f68300ad560 con 0x7f68300b4a20 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68300b4e20 con 0x7f6830072050 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f6837800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68300cee80 con 0x7f683015e900 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.354+0000 7f682fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43990/0 (socket says 172.21.15.28:43990) 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 -- 172.21.15.28:0/2748730812 learned_addr learned my addr 172.21.15.28:0/2748730812 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f6835800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f683006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f6834e00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 0x7f6830156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 -- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 msgr2=0x7f683006e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f683006e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 -- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 msgr2=0x7f6830156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 0x7f6830156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 -- 172.21.15.28:0/2748730812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68300cbc80 con 0x7f68300b4a20 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f6835800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f683006e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f6834e00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 0x7f6830156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682fe00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e960 tx=0x7f681c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.355+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c017070 con 0x7f68300b4a20 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.356+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f681c00ca00 con 0x7f68300b4a20 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.356+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00cdd0 con 0x7f68300b4a20 2024-09-10T10:55:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.356+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68300cbf20 con 0x7f68300b4a20 2024-09-10T10:55:41.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.356+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68300cc4e0 con 0x7f68300b4a20 2024-09-10T10:55:41.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.357+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68300ad020 con 0x7f68300b4a20 2024-09-10T10:55:41.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.358+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f681c01a070 con 0x7f68300b4a20 2024-09-10T10:55:41.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.358+0000 7f682cc00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 0x7f681006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.359+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7f681c08b970 con 0x7f68300b4a20 2024-09-10T10:55:41.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.359+0000 7f6834e00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 0x7f681006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.360+0000 7f6834e00640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 0x7f681006e320 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f683006f0f0 tx=0x7f6824008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.363+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f681c0596c0 con 0x7f68300b4a20 2024-09-10T10:55:41.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.547+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f68300b3e00 con 0x7f68300b4a20 2024-09-10T10:55:41.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.548+0000 7f682cc00640 1 -- 172.21.15.28:0/2748730812 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) ==== 93+0+10 (secure 0 0 0) 0x7f681c058fc0 con 0x7f68300b4a20 2024-09-10T10:55:41.550 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-09-10T10:55:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 msgr2=0x7f681006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 0x7f681006e320 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f683006f0f0 tx=0x7f6824008040 comp rx=0 tx=0).stop 2024-09-10T10:55:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 msgr2=0x7f6830156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f681c00e960 tx=0x7f681c00ee20 comp rx=0 tx=0).stop 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 shutdown_connections 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f681006be60 0x7f681006e320 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f683015e900 0x7f683006e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68300b4a20 0x7f6830156e90 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 --2- 172.21.15.28:0/2748730812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6830072050 0x7f6830156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 >> 172.21.15.28:0/2748730812 conn(0x7f683006bdf0 msgr2=0x7f68301e7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 shutdown_connections 2024-09-10T10:55:41.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.550+0000 7f6837800640 1 -- 172.21.15.28:0/2748730812 wait complete. 2024-09-10T10:55:41.568 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-10T10:55:41.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.723+0000 7fa987000640 1 Processor -- start 2024-09-10T10:55:41.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.723+0000 7fa987000640 1 -- start start 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa987000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa98006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa987000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa9800d1f90 0x7fa9800cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa987000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9800cf000 con 0x7fa9800d0dd0 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa987000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa9800b37d0 con 0x7fa9800d1f90 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa987000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9800cf330 con 0x7fa98015e660 2024-09-10T10:55:41.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa985c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa9800d0dd0 0x7fa9800d11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46188/0 (socket says 172.21.15.28:46188) 2024-09-10T10:55:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa985c00640 1 -- 172.21.15.28:0/2093524572 learned_addr learned my addr 172.21.15.28:0/2093524572 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.724+0000 7fa986600640 1 --2- 172.21.15.28:0/2093524572 >> v2:172.21.15.28:3300/0 conn(0x7fa9800d1f90 0x7fa9800cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa985200640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa98006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa986600640 1 -- 172.21.15.28:0/2093524572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 msgr2=0x7fa98006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:41.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa986600640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa98006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa986600640 1 -- 172.21.15.28:0/2093524572 >> v1:172.21.15.28:6789/0 conn(0x7fa9800d0dd0 legacy=0x7fa9800d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa986600640 1 -- 172.21.15.28:0/2093524572 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9800d29a0 con 0x7fa9800d1f90 2024-09-10T10:55:41.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.725+0000 7fa97fe00640 1 -- 172.21.15.28:0/2093524572 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92318366 0 0) 0x7fa9800cf000 con 0x7fa9800d0dd0 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.726+0000 7fa986600640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9800cf7c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fa978009d90 tx=0x7fa97802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b3acd32f62e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.726+0000 7fa97fe00640 1 -- 172.21.15.28:0/2093524572 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978007740 con 0x7fa9800d1f90 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.726+0000 7fa97fe00640 1 -- 172.21.15.28:0/2093524572 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9780078e0 con 0x7fa9800d1f90 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.726+0000 7fa97fe00640 1 -- 172.21.15.28:0/2093524572 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa978007cb0 con 0x7fa9800d1f90 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 -- 172.21.15.28:0/2093524572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 msgr2=0x7fa9800cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9800cf7c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fa978009d90 tx=0x7fa97802a660 comp rx=0 tx=0).stop 2024-09-10T10:55:41.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 -- 172.21.15.28:0/2093524572 shutdown_connections 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9800cf7c0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 --2- 172.21.15.28:0/2093524572 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa98006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 -- 172.21.15.28:0/2093524572 >> 172.21.15.28:0/2093524572 conn(0x7fa98008bfc0 msgr2=0x7fa9800d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 -- 172.21.15.28:0/2093524572 shutdown_connections 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.727+0000 7fa987000640 1 -- 172.21.15.28:0/2093524572 wait complete. 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.728+0000 7fa987000640 1 Processor -- start 2024-09-10T10:55:41.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 -- start start 2024-09-10T10:55:41.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9800d0dd0 0x7fa9801f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa9801f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800d3220 con 0x7fa9800d1f90 2024-09-10T10:55:41.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800d2f40 con 0x7fa9800d0dd0 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.729+0000 7fa987000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa9800d33a0 con 0x7fa98015e660 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44006/0 (socket says 172.21.15.28:44006) 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 -- 172.21.15.28:0/1898810365 learned_addr learned my addr 172.21.15.28:0/1898810365 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa986600640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa9801f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985c00640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9800d0dd0 0x7fa9801f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 -- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 msgr2=0x7fa9801f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa9801f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 -- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9800d0dd0 msgr2=0x7fa9801f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9800d0dd0 0x7fa9801f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:41.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 -- 172.21.15.28:0/1898810365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9800c9220 con 0x7fa9800d1f90 2024-09-10T10:55:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.730+0000 7fa985200640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00d5d0 tx=0x7fa96c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.731+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c004240 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.731+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c0043e0 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.731+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9800c94a0 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.731+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c003830 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.731+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9800c9a60 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.732+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9800d2440 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.733+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa96c0039d0 con 0x7fa9800d1f90 2024-09-10T10:55:41.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.734+0000 7fa97d600640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 0x7fa96006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.734+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7fa96c08bbd0 con 0x7fa9800d1f90 2024-09-10T10:55:41.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.734+0000 7fa985c00640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 0x7fa96006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:41.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.735+0000 7fa985c00640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 0x7fa96006e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa974000f60 tx=0x7fa974007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:41.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.738+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9800c9220 con 0x7fa9800d1f90 2024-09-10T10:55:41.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:41.990+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7fa98015ca40 con 0x7fa9800d1f90 2024-09-10T10:55:42.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.797+0000 7fa97d600640 1 -- 172.21.15.28:0/1898810365 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v612) ==== 138+0+0 (secure 0 0 0) 0x7fa96c0592e0 con 0x7fa9800d1f90 2024-09-10T10:55:42.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 msgr2=0x7fa96006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 0x7fa96006e2f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa974000f60 tx=0x7fa974007d50 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 msgr2=0x7fa9801f7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00d5d0 tx=0x7fa96c00da90 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 shutdown_connections 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa96006be30 0x7fa96006e2f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa98015e660 0x7fa9801f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa9800d1f90 0x7fa9801f7760 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 --2- 172.21.15.28:0/1898810365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9800d0dd0 0x7fa9801f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 >> 172.21.15.28:0/1898810365 conn(0x7fa98008bfc0 msgr2=0x7fa9800b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.800+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 shutdown_connections 2024-09-10T10:55:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.801+0000 7fa987000640 1 -- 172.21.15.28:0/1898810365 wait complete. 2024-09-10T10:55:42.822 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T10:55:42.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.985+0000 7fa690000640 1 Processor -- start 2024-09-10T10:55:42.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.985+0000 7fa690000640 1 -- start start 2024-09-10T10:55:42.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa690000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6880b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa690000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa6880af210 0x7fa688070a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa690000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa68806cdc0 con 0x7fa6880b7250 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa690000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa68806d0a0 con 0x7fa6880af210 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa690000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa68806d380 con 0x7fa68815e7f0 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa68e000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa6880b7250 0x7fa68809aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46194/0 (socket says 172.21.15.28:46194) 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa68e000640 1 -- 172.21.15.28:0/1979996264 learned_addr learned my addr 172.21.15.28:0/1979996264 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.986+0000 7fa68cc00640 1 --2- 172.21.15.28:0/1979996264 >> v2:172.21.15.138:3300/0 conn(0x7fa6880af210 0x7fa688070a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa68d600640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6880b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918611030 0 0) 0x7fa68806cdc0 con 0x7fa6880b7250 2024-09-10T10:55:42.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa670003520 con 0x7fa6880b7250 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1782813940 0 0) 0x7fa670003520 con 0x7fa6880b7250 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 msgr2=0x7fa6880b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6880b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 >> v2:172.21.15.138:3300/0 conn(0x7fa6880af210 msgr2=0x7fa688070a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 --2- 172.21.15.28:0/1979996264 >> v2:172.21.15.138:3300/0 conn(0x7fa6880af210 0x7fa688070a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68806d660 con 0x7fa6880b7250 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa6800019f0 con 0x7fa6880b7250 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 >> v1:172.21.15.28:6789/0 conn(0x7fa6880b7250 legacy=0x7fa68809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700039b0 0x7fa670003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670004800 0x7fa670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa68806d0a0 con 0x7fa6700039b0 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa68806d380 con 0x7fa670004800 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa67000b9c0 con 0x7fa68815e7f0 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.987+0000 7fa68d600640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700039b0 0x7fa670003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68cc00640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670004800 0x7fa670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68e000640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68d600640 1 -- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 msgr2=0x7fa6700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68d600640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68d600640 1 -- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670004800 msgr2=0x7fa670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68d600640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa670004800 0x7fa670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68d600640 1 -- 172.21.15.28:0/1979996264 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa670003520 con 0x7fa6700039b0 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.988+0000 7fa68e000640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa68815e7f0 0x7fa6700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.989+0000 7fa68d600640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700039b0 0x7fa670003dd0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009ac0 tx=0x7fa670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4ea68923f6eb19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.989+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c036070 con 0x7fa6700039b0 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.989+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa67c031400 con 0x7fa6700039b0 2024-09-10T10:55:42.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.989+0000 7fa687e00640 1 -- 172.21.15.28:0/1979996264 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c009690 con 0x7fa6700039b0 2024-09-10T10:55:42.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 -- 172.21.15.28:0/1979996264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700039b0 msgr2=0x7fa670003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 --2- 172.21.15.28:0/1979996264 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700039b0 0x7fa670003dd0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009ac0 tx=0x7fa670005830 comp rx=0 tx=0).stop 2024-09-10T10:55:42.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa68e000640 1 -- 172.21.15.28:0/1979996264 reap_dead start 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 -- 172.21.15.28:0/1979996264 shutdown_connections 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 -- 172.21.15.28:0/1979996264 >> 172.21.15.28:0/1979996264 conn(0x7fa6880b3f30 msgr2=0x7fa6880b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 -- 172.21.15.28:0/1979996264 shutdown_connections 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.990+0000 7fa690000640 1 -- 172.21.15.28:0/1979996264 wait complete. 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.992+0000 7fa690000640 1 Processor -- start 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.992+0000 7fa690000640 1 -- start start 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6700039b0 0x7fa688157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa688158640 0x7fa68815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880b6d10 con 0x7fa688158aa0 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6880b6150 con 0x7fa6700039b0 2024-09-10T10:55:42.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa690000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6880b6e90 con 0x7fa688158640 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44024/0 (socket says 172.21.15.28:44024) 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 -- 172.21.15.28:0/4141047323 learned_addr learned my addr 172.21.15.28:0/4141047323 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68d600640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6700039b0 0x7fa688157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68cc00640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa688158640 0x7fa68815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 -- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa688158640 msgr2=0x7fa68815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa688158640 0x7fa68815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 -- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6700039b0 msgr2=0x7fa688157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6700039b0 0x7fa688157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.993+0000 7fa68e000640 1 -- 172.21.15.28:0/4141047323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68815f690 con 0x7fa688158aa0 2024-09-10T10:55:42.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa68e000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fa688073360 tx=0x7fa68000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa68000ebc0 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa68000ed60 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68815f970 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa680014630 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.994+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa688160bc0 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.995+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa688070a70 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.996+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa6800147d0 con 0x7fa688158aa0 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.997+0000 7fa685600640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 0x7fa66806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:42.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.997+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7fa68008b780 con 0x7fa688158aa0 2024-09-10T10:55:43.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.997+0000 7fa68d600640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 0x7fa66806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:42.998+0000 7fa68d600640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 0x7fa66806e320 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa67c033000 tx=0x7fa67c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:43.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.001+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa680058f10 con 0x7fa688158aa0 2024-09-10T10:55:43.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.184+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6880b0550 con 0x7fa688158aa0 2024-09-10T10:55:43.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.188+0000 7fa685600640 1 -- 172.21.15.28:0/4141047323 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7fa68005d210 con 0x7fa688158aa0 2024-09-10T10:55:43.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 msgr2=0x7fa66806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 0x7fa66806e320 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa67c033000 tx=0x7fa67c008030 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 msgr2=0x7fa6881e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fa688073360 tx=0x7fa68000e530 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 shutdown_connections 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa66806be60 0x7fa66806e320 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa688158aa0 0x7fa6881e8db0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa688158640 0x7fa68815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 --2- 172.21.15.28:0/4141047323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6700039b0 0x7fa688157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 >> 172.21.15.28:0/4141047323 conn(0x7fa6880b3f30 msgr2=0x7fa6880d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 shutdown_connections 2024-09-10T10:55:43.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.190+0000 7fa690000640 1 -- 172.21.15.28:0/4141047323 wait complete. 2024-09-10T10:55:43.208 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:55:43.208 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:55:43.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:55:43.393 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:55:43.393 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:43.393 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:43.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:43.429 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:55:43.429 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:43.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:55:43.466 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:55:43.466 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:43.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:55:43.504 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:55:43.504 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:55:43.542 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:55:43.542 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T10:55:43.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.698+0000 7f0e9aa00640 1 Processor -- start 2024-09-10T10:55:43.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.698+0000 7f0e9aa00640 1 -- start start 2024-09-10T10:55:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 0x7f0e940cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e9aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0e94072de0 0x7f0e9406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e9aa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e94146140 con 0x7f0e94072700 2024-09-10T10:55:43.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e9aa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0e940cd740 con 0x7f0e94072de0 2024-09-10T10:55:43.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e9aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0e940cda20 con 0x7f0e94072320 2024-09-10T10:55:43.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e98a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0e94072700 0x7f0e9406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:46200/0 (socket says 172.21.15.28:46200) 2024-09-10T10:55:43.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e98a00640 1 -- 172.21.15.28:0/1751987804 learned_addr learned my addr 172.21.15.28:0/1751987804 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:43.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1751987804 >> v2:172.21.15.138:3300/0 conn(0x7f0e94072de0 0x7f0e9406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.699+0000 7f0e93e00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 0x7f0e940cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e93400640 1 -- 172.21.15.28:0/1751987804 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1562218982 0 0) 0x7f0e94146140 con 0x7f0e94072700 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e93400640 1 -- 172.21.15.28:0/1751987804 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e78003520 con 0x7f0e94072700 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e8be00640 1 -- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 msgr2=0x7f0e940cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 0x7f0e940cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e8be00640 1 -- 172.21.15.28:0/1751987804 >> v1:172.21.15.28:6789/0 conn(0x7f0e94072700 legacy=0x7f0e9406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e8be00640 1 -- 172.21.15.28:0/1751987804 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e940cdd00 con 0x7f0e94072de0 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.700+0000 7f0e93e00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 0x7f0e940cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:43.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.701+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e9406e0b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0097a0 tx=0x7f0e7c02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb9fa9216fe37b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:43.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.701+0000 7f0e93400640 1 -- 172.21.15.28:0/1751987804 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c037070 con 0x7f0e94072de0 2024-09-10T10:55:43.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.701+0000 7f0e93400640 1 -- 172.21.15.28:0/1751987804 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e7c02ae40 con 0x7f0e94072de0 2024-09-10T10:55:43.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.701+0000 7f0e93400640 1 -- 172.21.15.28:0/1751987804 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c032660 con 0x7f0e94072de0 2024-09-10T10:55:43.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 msgr2=0x7f0e9406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e9406e0b0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0097a0 tx=0x7f0e7c02a0f0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1751987804 shutdown_connections 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e9406e0b0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1751987804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072320 0x7f0e940cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1751987804 >> 172.21.15.28:0/1751987804 conn(0x7f0e94067d70 msgr2=0x7f0e9406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1751987804 shutdown_connections 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.702+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1751987804 wait complete. 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.704+0000 7f0e9aa00640 1 Processor -- start 2024-09-10T10:55:43.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.704+0000 7f0e9aa00640 1 -- start start 2024-09-10T10:55:43.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072700 0x7f0e941efa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e941eff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0e940cd520 con 0x7f0e94072320 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e940cd3a0 con 0x7f0e94072de0 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e9aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9406ed00 con 0x7f0e94072700 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44038/0 (socket says 172.21.15.28:44038) 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 -- 172.21.15.28:0/1169764875 learned_addr learned my addr 172.21.15.28:0/1169764875 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072700 0x7f0e941efa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e98a00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e941eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 -- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072700 msgr2=0x7f0e941efa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072700 0x7f0e941efa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 -- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 msgr2=0x7f0e941eff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e941eff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.705+0000 7f0e93e00640 1 -- 172.21.15.28:0/1169764875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e940c9040 con 0x7f0e94072320 2024-09-10T10:55:43.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e93e00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400ebe0 tx=0x7f0e8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:43.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84017070 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e8400cc30 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e940c9320 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e84013470 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.706+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e940c98e0 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.707+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e940cce60 con 0x7f0e94072320 2024-09-10T10:55:43.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.708+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0e8401a070 con 0x7f0e94072320 2024-09-10T10:55:43.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.709+0000 7f0e90c00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 0x7f0e7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:43.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.709+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7f0e8408bd30 con 0x7f0e94072320 2024-09-10T10:55:43.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.709+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 0x7f0e7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:43.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.710+0000 7f0e8be00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 0x7f0e7006e2f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c034000 tx=0x7f0e7c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:43.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.713+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0e84059440 con 0x7f0e94072320 2024-09-10T10:55:43.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.937+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e940a9490 con 0x7f0e94072320 2024-09-10T10:55:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.938+0000 7f0e90c00640 1 -- 172.21.15.28:0/1169764875 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 334 v334) ==== 95+0+3916 (secure 0 0 0) 0x7f0e8405d740 con 0x7f0e94072320 2024-09-10T10:55:43.940 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:43.940 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":334,"btime":"2024-09-10T10:55:37:767114+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5966,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/3056141947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3056141947},{"type":"v1","addr":"172.21.15.138:6836","nonce":3056141947}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/539085687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":539085687},{"type":"v1","addr":"172.21.15.138:6837","nonce":539085687}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":10329,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/422540071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":422540071},{"type":"v1","addr":"172.21.15.28:6834","nonce":422540071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":328},{"gid":10374,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1966252373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":1966252373},{"type":"v1","addr":"172.21.15.28:6835","nonce":1966252373}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":332}],"filesystems":[]} 2024-09-10T10:55:43.940 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 334 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.940+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 msgr2=0x7f0e7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.940+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 0x7f0e7006e2f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c034000 tx=0x7f0e7c02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.940+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 msgr2=0x7f0e940ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.940+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400ebe0 tx=0x7f0e8400c520 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 shutdown_connections 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0e7006be30 0x7f0e7006e2f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e94072de0 0x7f0e941eff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e94072700 0x7f0e941efa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 --2- 172.21.15.28:0/1169764875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0e94072320 0x7f0e940ca840 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:43.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 >> 172.21.15.28:0/1169764875 conn(0x7f0e94067d70 msgr2=0x7f0e94141ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 shutdown_connections 2024-09-10T10:55:43.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:43.941+0000 7f0e9aa00640 1 -- 172.21.15.28:0/1169764875 wait complete. 2024-09-10T10:55:43.959 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T10:55:43.959 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T10:55:43.959 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:55:43.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:55:43.960 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T10:55:43.960 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T10:55:43.960 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:55:43.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:55:43.960 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T10:55:43.960 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T10:55:43.961 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:55:43.961 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:55:43.961 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T10:55:43.961 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T10:55:43.961 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:55:43.961 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:55:43.973 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:55:43.973 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T10:55:43.975 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:55:43.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T10:55:43.977 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:55:43.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T10:55:43.978 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:55:43.978 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T10:55:44.020 INFO:tasks.ceph.mds.a:Started 2024-09-10T10:55:44.021 DEBUG:teuthology.parallel:result is None 2024-09-10T10:55:44.021 INFO:tasks.ceph.mds.b:Started 2024-09-10T10:55:44.022 DEBUG:teuthology.parallel:result is None 2024-09-10T10:55:44.024 INFO:tasks.ceph.mds.c:Started 2024-09-10T10:55:44.024 INFO:tasks.ceph.mds.d:Started 2024-09-10T10:55:44.024 DEBUG:teuthology.parallel:result is None 2024-09-10T10:55:44.024 DEBUG:teuthology.parallel:result is None 2024-09-10T10:55:44.024 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T10:55:44.151 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-09-10T10:55:44.152 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-09-10T10:55:44.154 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:55:44.154 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-09-10T10:55:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.184+0000 7fbd01600640 1 Processor -- start 2024-09-10T10:55:44.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.184+0000 7fbd01600640 1 -- start start 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbd01600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbcfc072320 0x7fbcfc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbd01600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbcfc072de0 0x7fbcfc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbd01600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcfc146140 con 0x7fbcfc072700 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbd01600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc0cd740 con 0x7fbcfc072320 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbd01600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbcfc0cda20 con 0x7fbcfc072de0 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbcfaa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbcfc072320 0x7fbcfc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbcfaa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbcfc072320 0x7fbcfc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33096/0 (socket says 172.21.15.28:33096) 2024-09-10T10:55:44.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.185+0000 7fbcfaa00640 1 -- 172.21.15.28:0/2219727050 learned_addr learned my addr 172.21.15.28:0/2219727050 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.186+0000 7fbcfa000640 1 --2- 172.21.15.28:0/2219727050 >> v2:172.21.15.138:3300/0 conn(0x7fbcfc072de0 0x7fbcfc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.186+0000 7fbcfaa00640 1 -- 172.21.15.28:0/2219727050 >> v2:172.21.15.138:3300/0 conn(0x7fbcfc072de0 msgr2=0x7fbcfc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.186+0000 7fbcfaa00640 1 --2- 172.21.15.28:0/2219727050 >> v2:172.21.15.138:3300/0 conn(0x7fbcfc072de0 0x7fbcfc06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.186+0000 7fbcfaa00640 1 -- 172.21.15.28:0/2219727050 >> v1:172.21.15.138:6789/0 conn(0x7fbcfc072700 legacy=0x7fbcfc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.186+0000 7fbcfaa00640 1 -- 172.21.15.28:0/2219727050 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0cdd00 con 0x7fbcfc072320 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.187+0000 7fbcfaa00640 1 --2- 172.21.15.28:0/2219727050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072320 0x7fbcfc0cce60 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06f8c0 tx=0x7fbcec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb7637a64ee4645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.187+0000 7fbcf9600640 1 -- 172.21.15.28:0/2219727050 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec037070 con 0x7fbcfc072320 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.187+0000 7fbcf9600640 1 -- 172.21.15.28:0/2219727050 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec032440 con 0x7fbcfc072320 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.187+0000 7fbcf9600640 1 -- 172.21.15.28:0/2219727050 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec032810 con 0x7fbcfc072320 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 -- 172.21.15.28:0/2219727050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072320 msgr2=0x7fbcfc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 --2- 172.21.15.28:0/2219727050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072320 0x7fbcfc0cce60 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc06f8c0 tx=0x7fbcec02a1e0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 -- 172.21.15.28:0/2219727050 shutdown_connections 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 --2- 172.21.15.28:0/2219727050 >> v2:172.21.15.138:3300/0 conn(0x7fbcfc072de0 0x7fbcfc06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 --2- 172.21.15.28:0/2219727050 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072320 0x7fbcfc0cce60 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 -- 172.21.15.28:0/2219727050 >> 172.21.15.28:0/2219727050 conn(0x7fbcfc067d70 msgr2=0x7fbcfc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 -- 172.21.15.28:0/2219727050 shutdown_connections 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.188+0000 7fbd01600640 1 -- 172.21.15.28:0/2219727050 wait complete. 2024-09-10T10:55:44.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.189+0000 7fbd01600640 1 Processor -- start 2024-09-10T10:55:44.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 -- start start 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 0x7fbcfc0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcfc072700 0x7fbcfc137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0cd520 con 0x7fbcfc072de0 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc0cd3a0 con 0x7fbcfc072320 2024-09-10T10:55:44.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.190+0000 7fbd01600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbcfc06ed00 con 0x7fbcfc072700 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33106/0 (socket says 172.21.15.28:33106) 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 0x7fbcfc0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfb400640 1 -- 172.21.15.28:0/3148642251 learned_addr learned my addr 172.21.15.28:0/3148642251 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfa000640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcfc072700 0x7fbcfc137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 -- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcfc072700 msgr2=0x7fbcfc137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcfc072700 0x7fbcfc137a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 -- 172.21.15.28:0/3148642251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 msgr2=0x7fbcfc15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfaa00640 1 -- 172.21.15.28:0/3148642251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0aa970 con 0x7fbcfc072320 2024-09-10T10:55:44.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.191+0000 7fbcfb400640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:44.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbcfaa00640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 0x7fbcfc0affa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbcec031000 tx=0x7fbcec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec041030 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcec030440 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec037040 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc0aac50 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.192+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc0ab210 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.193+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcfc0cce60 con 0x7fbcfc072320 2024-09-10T10:55:44.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.194+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbcec044070 con 0x7fbcfc072320 2024-09-10T10:55:44.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.195+0000 7fbcf2a00640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:44.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.195+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 8689+0+0 (secure 0 0 0) 0x7fbcec0ac140 con 0x7fbcfc072320 2024-09-10T10:55:44.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.195+0000 7fbcfa000640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:44.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.196+0000 7fbcfa000640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc137940 tx=0x7fbce8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:44.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.199+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcec079950 con 0x7fbcfc072320 2024-09-10T10:55:44.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.398+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbcfc0c79d0 con 0x7fbcfc072320 2024-09-10T10:55:44.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.818+0000 7fbcf2a00640 1 -- 172.21.15.28:0/3148642251 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v144) ==== 97+0+0 (secure 0 0 0) 0x7fbcec07dc50 con 0x7fbcfc072320 2024-09-10T10:55:44.820 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.820+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 msgr2=0x7fbcdc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.820+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc137940 tx=0x7fbce8009190 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.820+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 msgr2=0x7fbcfc0affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.820+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 0x7fbcfc0affa0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbcec031000 tx=0x7fbcec0046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.820+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 shutdown_connections 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcdc06be30 0x7fbcdc06e2f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcfc072de0 0x7fbcfc15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbcfc072700 0x7fbcfc137a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 --2- 172.21.15.28:0/3148642251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcfc072320 0x7fbcfc0affa0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 >> 172.21.15.28:0/3148642251 conn(0x7fbcfc067d70 msgr2=0x7fbcfc0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 shutdown_connections 2024-09-10T10:55:44.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.821+0000 7fbd01600640 1 -- 172.21.15.28:0/3148642251 wait complete. 2024-09-10T10:55:44.841 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T10:55:44.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.996+0000 7f3fee200640 1 Processor -- start 2024-09-10T10:55:44.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.996+0000 7f3fee200640 1 -- start start 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fee200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fe80d30f0 0x7f3fe80b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fee200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe80b64c0 0x7f3fe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fee200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fe80d3690 con 0x7f3fe815e5f0 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fee200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe806d530 con 0x7f3fe80d30f0 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fee200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe80d3970 con 0x7f3fe80b64c0 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.997+0000 7f3fe6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fe80d30f0 0x7f3fe80b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f3fe815e5f0 0x7f3fe80d4dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:40770/0 (socket says 172.21.15.28:40770) 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fe80d30f0 0x7f3fe80b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33112/0 (socket says 172.21.15.28:33112) 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe7400640 1 -- 172.21.15.28:0/616958670 learned_addr learned my addr 172.21.15.28:0/616958670 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe80b64c0 0x7f3fe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6a00640 1 -- 172.21.15.28:0/616958670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe80b64c0 msgr2=0x7f3fe806d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6a00640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe80b64c0 0x7f3fe806d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6a00640 1 -- 172.21.15.28:0/616958670 >> v1:172.21.15.138:6789/0 conn(0x7f3fe815e5f0 legacy=0x7f3fe80d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6a00640 1 -- 172.21.15.28:0/616958670 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe806cee0 con 0x7f3fe80d30f0 2024-09-10T10:55:45.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.998+0000 7f3fe6000640 1 -- 172.21.15.28:0/616958670 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408265817 0 0) 0x7f3fe80d3690 con 0x7f3fe815e5f0 2024-09-10T10:55:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.999+0000 7f3fe6a00640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe80b5f80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bca93849239a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.999+0000 7f3fe6000640 1 -- 172.21.15.28:0/616958670 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4037070 con 0x7f3fe80d30f0 2024-09-10T10:55:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.999+0000 7f3fe6000640 1 -- 172.21.15.28:0/616958670 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd4032440 con 0x7f3fe80d30f0 2024-09-10T10:55:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:44.999+0000 7f3fe6000640 1 -- 172.21.15.28:0/616958670 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4032810 con 0x7f3fe80d30f0 2024-09-10T10:55:45.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 -- 172.21.15.28:0/616958670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 msgr2=0x7f3fe80b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe80b5f80 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402a180 comp rx=0 tx=0).stop 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 -- 172.21.15.28:0/616958670 shutdown_connections 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe80b64c0 0x7f3fe806d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 --2- 172.21.15.28:0/616958670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe80b5f80 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 -- 172.21.15.28:0/616958670 >> 172.21.15.28:0/616958670 conn(0x7f3fe808bfc0 msgr2=0x7f3fe80d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 -- 172.21.15.28:0/616958670 shutdown_connections 2024-09-10T10:55:45.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.000+0000 7f3fee200640 1 -- 172.21.15.28:0/616958670 wait complete. 2024-09-10T10:55:45.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.001+0000 7f3fee200640 1 Processor -- start 2024-09-10T10:55:45.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.002+0000 7f3fee200640 1 -- start start 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 0x7f3fe815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 0x7f3fe80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe806df00 con 0x7f3fe80d30f0 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe815e9d0 con 0x7f3fe80b64c0 2024-09-10T10:55:45.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fee200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe806e080 con 0x7f3fe815e5f0 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33114/0 (socket says 172.21.15.28:33114) 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe6a00640 1 -- 172.21.15.28:0/1725503102 learned_addr learned my addr 172.21.15.28:0/1725503102 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe7400640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 0x7f3fe815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 0x7f3fe80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe6a00640 1 -- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 msgr2=0x7f3fe80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.003+0000 7f3fe6a00640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 0x7f3fe80cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe6a00640 1 -- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 msgr2=0x7f3fe815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe6a00640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 0x7f3fe815d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe6a00640 1 -- 172.21.15.28:0/1725503102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe815db20 con 0x7f3fe80d30f0 2024-09-10T10:55:45.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe7400640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 0x7f3fe815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe7e00640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 0x7f3fe80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fe6a00640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4035040 tx=0x7f3fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4037040 con 0x7f3fe80d30f0 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd4030500 con 0x7f3fe80d30f0 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4030910 con 0x7f3fe80d30f0 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe81e6fa0 con 0x7f3fe80d30f0 2024-09-10T10:55:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.004+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe81e7560 con 0x7f3fe80d30f0 2024-09-10T10:55:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.005+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe806d9a0 con 0x7f3fe80d30f0 2024-09-10T10:55:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.006+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3fd4044020 con 0x7f3fe80d30f0 2024-09-10T10:55:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.007+0000 7f3fdf400640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.007+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5249+0+0 (secure 0 0 0) 0x7f3fd40aafa0 con 0x7f3fe80d30f0 2024-09-10T10:55:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.007+0000 7f3fe7400640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.008+0000 7f3fe7400640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000f60 tx=0x7f3fd8007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.011+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3fe815db20 con 0x7f3fe80d30f0 2024-09-10T10:55:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.264+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3fe80d1910 con 0x7f3fe80d30f0 2024-09-10T10:55:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.264+0000 7f3fdf400640 1 -- 172.21.15.28:0/1725503102 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f3fd4079b00 con 0x7f3fe80d30f0 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmLncmJxAAzqviyH4lg1IwhwWtnr5Saw==", 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:55:45.267 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmVzvBOhAAkTWWXmNw1eTmunIl6mAFGQ==", 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.268 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmvNPyMBAAyvV1uv6t9LifrX3BjzCUjQ==", 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.269 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmJRH9CBAAge2JGGvijCLMZ1Dwo/K8wA==", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCjIuBmUFNgORAA8sAiM1GV4h2zv/fjn470jw==", 2024-09-10T10:55:45.270 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCnIuBmt3jjBxAA6LuwDoVp2q4xLaPbNhom7w==", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.271 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCqIuBmoyOvERAANO5BdsiawJewErJBTc02hQ==", 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-09-10T10:55:45.272 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCtIuBmSVQ9HRAAbP4yMeMHfsSqEQjTd0y9wA==", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCzIuBmh11xGhAAJAYnHqlSiYkCKm4MTEQtaw==", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.273 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC2IuBmkM/9IhAAzHHSCKLJUtDlJ9ss9L616A==", 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.274 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC5IuBmPSIZMBAABUh3WQBXnSDImtkIc+GZuA==", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQC9IuBmbiAHDBAAW//SA9nOMwlQLscOijh+Ww==", 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.275 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA==", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.276 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmCx9tHhAAPW+F0T8iEGvuwwfp6r/c8w==", 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.277 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBmUif5FxAAt3exj3CG0hWi4Xlu8Uctrg==", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCiIuBm3i7RJBAAMk5NWjrFontdC963A0teWg==", 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.278 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQCgIuBmKMO7MhAAY3FdAS1j8G2dJ93WI5J3/A==", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-09-10T10:55:45.279 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBm+x6CIBAATZ0pGUg1Dg1BKoazuJfoJw==", 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.280 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQChIuBmUg8DGRAAIrcZEt7UWZVq5f6PN44wyA==", 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-09-10T10:55:45.281 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.266+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 msgr2=0x7f3fc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.266+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8000f60 tx=0x7f3fd8007cc0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 msgr2=0x7f3fe81f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4035040 tx=0x7f3fd40046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 shutdown_connections 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3fc806be30 0x7f3fc806e2f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fe815e5f0 0x7f3fe80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fe80d30f0 0x7f3fe81f4340 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 --2- 172.21.15.28:0/1725503102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe80b64c0 0x7f3fe815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 >> 172.21.15.28:0/1725503102 conn(0x7f3fe808bfc0 msgr2=0x7f3fe80b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 shutdown_connections 2024-09-10T10:55:45.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.267+0000 7f3fee200640 1 -- 172.21.15.28:0/1725503102 wait complete. 2024-09-10T10:55:45.284 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T10:55:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.438+0000 7f9670e00640 1 Processor -- start 2024-09-10T10:55:45.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.438+0000 7f9670e00640 1 -- start start 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9670e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 0x7f966c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9670e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c15e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9670e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f966c06f840 con 0x7f966c0720b0 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9670e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f966c15d280 con 0x7f966c15c8d0 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9670e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f966c15d4b0 con 0x7f966c15e550 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9669e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f966c0720b0 0x7f966c072490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47184/0 (socket says 172.21.15.28:47184) 2024-09-10T10:55:45.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9669e00640 1 -- 172.21.15.28:0/3869789494 learned_addr learned my addr 172.21.15.28:0/3869789494 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f9661400640 1 --2- 172.21.15.28:0/3869789494 >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 0x7f966c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.439+0000 7f966a800640 1 --2- 172.21.15.28:0/3869789494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c15e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f9669400640 1 -- 172.21.15.28:0/3869789494 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013250698 0 0) 0x7f966c06f840 con 0x7f966c0720b0 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f9669400640 1 -- 172.21.15.28:0/3869789494 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9650003520 con 0x7f966c0720b0 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f966a800640 1 -- 172.21.15.28:0/3869789494 >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 msgr2=0x7f966c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f966a800640 1 --2- 172.21.15.28:0/3869789494 >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 0x7f966c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f966a800640 1 -- 172.21.15.28:0/3869789494 >> v1:172.21.15.28:6789/0 conn(0x7f966c0720b0 legacy=0x7f966c072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f966a800640 1 -- 172.21.15.28:0/3869789494 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c15d790 con 0x7f966c15e550 2024-09-10T10:55:45.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.440+0000 7f9661400640 1 --2- 172.21.15.28:0/3869789494 >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 0x7f966c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.441+0000 7f966a800640 1 --2- 172.21.15.28:0/3869789494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c15e9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9664009d90 tx=0x7f966402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f0a2408e9748f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.441+0000 7f9669400640 1 -- 172.21.15.28:0/3869789494 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664007740 con 0x7f966c15e550 2024-09-10T10:55:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.441+0000 7f9669400640 1 -- 172.21.15.28:0/3869789494 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96640078e0 con 0x7f966c15e550 2024-09-10T10:55:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.441+0000 7f9669400640 1 -- 172.21.15.28:0/3869789494 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9664007cb0 con 0x7f966c15e550 2024-09-10T10:55:45.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 -- 172.21.15.28:0/3869789494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 msgr2=0x7f966c15e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 --2- 172.21.15.28:0/3869789494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c15e9e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9664009d90 tx=0x7f966402a660 comp rx=0 tx=0).stop 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 -- 172.21.15.28:0/3869789494 shutdown_connections 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 --2- 172.21.15.28:0/3869789494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c15e9e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 --2- 172.21.15.28:0/3869789494 >> v2:172.21.15.138:3300/0 conn(0x7f966c15c8d0 0x7f966c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 -- 172.21.15.28:0/3869789494 >> 172.21.15.28:0/3869789494 conn(0x7f966c067d70 msgr2=0x7f966c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 -- 172.21.15.28:0/3869789494 shutdown_connections 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.442+0000 7f9670e00640 1 -- 172.21.15.28:0/3869789494 wait complete. 2024-09-10T10:55:45.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.444+0000 7f9670e00640 1 Processor -- start 2024-09-10T10:55:45.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.444+0000 7f9670e00640 1 -- start start 2024-09-10T10:55:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f966c0720b0 0x7f966c0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f966c15e030 con 0x7f966c15c8d0 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f966c15dd50 con 0x7f966c0720b0 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9670e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f966c146110 con 0x7f966c15e550 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33118/0 (socket says 172.21.15.28:33118) 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 -- 172.21.15.28:0/373680712 learned_addr learned my addr 172.21.15.28:0/373680712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9669e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f966c0720b0 0x7f966c0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f966a800640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 -- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 msgr2=0x7f966c0add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c0add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 -- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f966c0720b0 msgr2=0x7f966c0ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f966c0720b0 0x7f966c0ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.445+0000 7f9661400640 1 -- 172.21.15.28:0/373680712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966c0ad540 con 0x7f966c15c8d0 2024-09-10T10:55:45.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9661400640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f965400ebe0 tx=0x7f965400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654017070 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965400cc30 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966c0abf60 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654013470 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.446+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f966c0ac800 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.447+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f966c15cd40 con 0x7f966c15c8d0 2024-09-10T10:55:45.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.448+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f965401a070 con 0x7f966c15c8d0 2024-09-10T10:55:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.449+0000 7f9662a00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.449+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5249+0+0 (secure 0 0 0) 0x7f965408ac70 con 0x7f966c15c8d0 2024-09-10T10:55:45.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.449+0000 7f9669e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.450+0000 7f9669e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 0x7f964806e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f965c005880 tx=0x7f965c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.453+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9654059070 con 0x7f966c15c8d0 2024-09-10T10:55:45.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.641+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f966c145610 con 0x7f966c15c8d0 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.642+0000 7f9662a00640 1 -- 172.21.15.28:0/373680712 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) ==== 84+0+65 (secure 0 0 0) 0x7f96540780e0 con 0x7f966c15c8d0 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-09-10T10:55:45.644 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-09-10T10:55:45.645 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-09-10T10:55:45.645 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-09-10T10:55:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 msgr2=0x7f964806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 0x7f964806e2f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f965c005880 tx=0x7f965c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 msgr2=0x7f966c15bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f965400ebe0 tx=0x7f965400c520 comp rx=0 tx=0).stop 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 shutdown_connections 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f964806be30 0x7f964806e2f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f966c15e550 0x7f966c0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f966c15c8d0 0x7f966c15bdc0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 --2- 172.21.15.28:0/373680712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f966c0720b0 0x7f966c0ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 >> 172.21.15.28:0/373680712 conn(0x7f966c067d70 msgr2=0x7f966c15f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 shutdown_connections 2024-09-10T10:55:45.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.644+0000 7f9670e00640 1 -- 172.21.15.28:0/373680712 wait complete. 2024-09-10T10:55:45.662 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T10:55:45.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T10:55:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.822+0000 7f4c99400640 1 Processor -- start 2024-09-10T10:55:45.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.822+0000 7f4c99400640 1 -- start start 2024-09-10T10:55:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c99400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c94162340 0x7f4c9406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c99400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c940cf280 0x7f4c94162720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c99400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c9415ff70 con 0x7f4c94162c60 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c99400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4c9406de90 con 0x7f4c94162340 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c99400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4c940d4d10 con 0x7f4c940cf280 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c92800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c94162340 0x7f4c9406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c92800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c94162340 0x7f4c9406d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33128/0 (socket says 172.21.15.28:33128) 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c92800640 1 -- 172.21.15.28:0/1207026005 learned_addr learned my addr 172.21.15.28:0/1207026005 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c93200640 1 --1- 172.21.15.28:0/1207026005 >> v1:172.21.15.28:6789/0 conn(0x7f4c94162c60 0x7f4c940d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47192/0 (socket says 172.21.15.28:47192) 2024-09-10T10:55:45.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.823+0000 7f4c91e00640 1 --2- 172.21.15.28:0/1207026005 >> v2:172.21.15.138:3300/0 conn(0x7f4c940cf280 0x7f4c94162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.824+0000 7f4c92800640 1 -- 172.21.15.28:0/1207026005 >> v2:172.21.15.138:3300/0 conn(0x7f4c940cf280 msgr2=0x7f4c94162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.824+0000 7f4c92800640 1 --2- 172.21.15.28:0/1207026005 >> v2:172.21.15.138:3300/0 conn(0x7f4c940cf280 0x7f4c94162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.824+0000 7f4c92800640 1 -- 172.21.15.28:0/1207026005 >> v1:172.21.15.28:6789/0 conn(0x7f4c94162c60 legacy=0x7f4c940d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.824+0000 7f4c92800640 1 -- 172.21.15.28:0/1207026005 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c940d4ff0 con 0x7f4c94162340 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.824+0000 7f4c91400640 1 -- 172.21.15.28:0/1207026005 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1884381217 0 0) 0x7f4c9415ff70 con 0x7f4c94162c60 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.825+0000 7f4c92800640 1 --2- 172.21.15.28:0/1207026005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c94162340 0x7f4c9406d740 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f4c940a23d0 tx=0x7f4c8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42c5e0148f1bd88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.825+0000 7f4c91400640 1 -- 172.21.15.28:0/1207026005 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84037070 con 0x7f4c94162340 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.825+0000 7f4c91400640 1 -- 172.21.15.28:0/1207026005 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c84032440 con 0x7f4c94162340 2024-09-10T10:55:45.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.825+0000 7f4c91400640 1 -- 172.21.15.28:0/1207026005 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84032810 con 0x7f4c94162340 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 -- 172.21.15.28:0/1207026005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c94162340 msgr2=0x7f4c9406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 --2- 172.21.15.28:0/1207026005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c94162340 0x7f4c9406d740 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f4c940a23d0 tx=0x7f4c8402a1e0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 -- 172.21.15.28:0/1207026005 shutdown_connections 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 --2- 172.21.15.28:0/1207026005 >> v2:172.21.15.138:3300/0 conn(0x7f4c940cf280 0x7f4c94162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 --2- 172.21.15.28:0/1207026005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c94162340 0x7f4c9406d740 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 -- 172.21.15.28:0/1207026005 >> 172.21.15.28:0/1207026005 conn(0x7f4c9408bfc0 msgr2=0x7f4c940d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 -- 172.21.15.28:0/1207026005 shutdown_connections 2024-09-10T10:55:45.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.826+0000 7f4c99400640 1 -- 172.21.15.28:0/1207026005 wait complete. 2024-09-10T10:55:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 Processor -- start 2024-09-10T10:55:45.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 -- start start 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c94162340 0x7f4c9415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c94162c60 0x7f4c940b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c940cf6c0 con 0x7f4c940cf280 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.828+0000 7f4c99400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c940d52d0 con 0x7f4c94162340 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c99400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c941630d0 con 0x7f4c94162c60 2024-09-10T10:55:45.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33130/0 (socket says 172.21.15.28:33130) 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 -- 172.21.15.28:0/1251853262 learned_addr learned my addr 172.21.15.28:0/1251853262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c91e00640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c94162340 0x7f4c9415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c93200640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c94162c60 0x7f4c940b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 -- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c94162c60 msgr2=0x7f4c940b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c94162c60 0x7f4c940b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 -- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c94162340 msgr2=0x7f4c9415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c94162340 0x7f4c9415c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:45.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.829+0000 7f4c92800640 1 -- 172.21.15.28:0/1251853262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9414ce30 con 0x7f4c940cf280 2024-09-10T10:55:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c92800640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f4c8402a220 tx=0x7f4c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84037040 con 0x7f4c940cf280 2024-09-10T10:55:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c84042070 con 0x7f4c940cf280 2024-09-10T10:55:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84030880 con 0x7f4c940cf280 2024-09-10T10:55:45.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c940b4bd0 con 0x7f4c940cf280 2024-09-10T10:55:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.830+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c940b5d30 con 0x7f4c940cf280 2024-09-10T10:55:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.831+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9415a020 con 0x7f4c940cf280 2024-09-10T10:55:45.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.832+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c84030a20 con 0x7f4c940cf280 2024-09-10T10:55:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.833+0000 7f4c8aa00640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 0x7f4c7406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.833+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5249+0+0 (secure 0 0 0) 0x7f4c840ab410 con 0x7f4c940cf280 2024-09-10T10:55:45.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.833+0000 7f4c91e00640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 0x7f4c7406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:45.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.834+0000 7f4c91e00640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 0x7f4c7406e2f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4c9415f2c0 tx=0x7f4c80009110 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:45.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:45.837+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c9415a020 con 0x7f4c940cf280 2024-09-10T10:55:46.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.021+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4c94162720 con 0x7f4c940cf280 2024-09-10T10:55:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.819+0000 7f4c8aa00640 1 -- 172.21.15.28:0/1251853262 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v147) ==== 138+0+0 (secure 0 0 0) 0x7f4c9414ce30 con 0x7f4c940cf280 2024-09-10T10:55:46.822 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-09-10T10:55:46.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.822+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 msgr2=0x7f4c7406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:46.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.822+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 0x7f4c7406e2f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4c9415f2c0 tx=0x7f4c80009110 comp rx=0 tx=0).stop 2024-09-10T10:55:46.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.822+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 msgr2=0x7f4c9414c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:46.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.822+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f4c8402a220 tx=0x7f4c840046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 shutdown_connections 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c7406be30 0x7f4c7406e2f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c94162c60 0x7f4c940b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c94162340 0x7f4c9415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 --2- 172.21.15.28:0/1251853262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c940cf280 0x7f4c9414c8f0 secure :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f4c8402a220 tx=0x7f4c840046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 >> 172.21.15.28:0/1251853262 conn(0x7f4c9408bfc0 msgr2=0x7f4c94161830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.823+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 shutdown_connections 2024-09-10T10:55:46.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:46.824+0000 7f4c99400640 1 -- 172.21.15.28:0/1251853262 wait complete. 2024-09-10T10:55:46.848 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T10:55:47.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.012+0000 7f6516000640 1 Processor -- start 2024-09-10T10:55:47.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.012+0000 7f6516000640 1 -- start start 2024-09-10T10:55:47.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.013+0000 7f6516000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f651015eaa0 0x7f65100ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f6516000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f651015eef0 0x7f65100cd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f6516000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f651015f440 con 0x7f65100b4ae0 2024-09-10T10:55:47.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f6516000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f651015f720 con 0x7f651015eaa0 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f6516000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f65100b31e0 con 0x7f651015eef0 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f651015eaa0 0x7f65100ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f651015eaa0 0x7f65100ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33138/0 (socket says 172.21.15.28:33138) 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 -- 172.21.15.28:0/1244792635 learned_addr learned my addr 172.21.15.28:0/1244792635 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650fe00640 1 --2- 172.21.15.28:0/1244792635 >> v2:172.21.15.138:3300/0 conn(0x7f651015eef0 0x7f65100cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 -- 172.21.15.28:0/1244792635 >> v2:172.21.15.138:3300/0 conn(0x7f651015eef0 msgr2=0x7f65100cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 --2- 172.21.15.28:0/1244792635 >> v2:172.21.15.138:3300/0 conn(0x7f651015eef0 0x7f65100cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 -- 172.21.15.28:0/1244792635 >> v1:172.21.15.138:6789/0 conn(0x7f65100b4ae0 legacy=0x7f65100b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:47.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.014+0000 7f650ea00640 1 -- 172.21.15.28:0/1244792635 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65100b34c0 con 0x7f651015eaa0 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.015+0000 7f650ea00640 1 --2- 172.21.15.28:0/1244792635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f651015eaa0 0x7f65100ccae0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a013833341c9b721 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f650e000640 1 -- 172.21.15.28:0/1244792635 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc037070 con 0x7f651015eaa0 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f650e000640 1 -- 172.21.15.28:0/1244792635 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc032440 con 0x7f651015eaa0 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f650e000640 1 -- 172.21.15.28:0/1244792635 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc032810 con 0x7f651015eaa0 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 -- 172.21.15.28:0/1244792635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f651015eaa0 msgr2=0x7f65100ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 --2- 172.21.15.28:0/1244792635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f651015eaa0 0x7f65100ccae0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009870 tx=0x7f64fc02a270 comp rx=0 tx=0).stop 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 -- 172.21.15.28:0/1244792635 shutdown_connections 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 --2- 172.21.15.28:0/1244792635 >> v2:172.21.15.138:3300/0 conn(0x7f651015eef0 0x7f65100cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 --2- 172.21.15.28:0/1244792635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f651015eaa0 0x7f65100ccae0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 -- 172.21.15.28:0/1244792635 >> 172.21.15.28:0/1244792635 conn(0x7f651006bdf0 msgr2=0x7f651006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:47.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 -- 172.21.15.28:0/1244792635 shutdown_connections 2024-09-10T10:55:47.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.016+0000 7f6516000640 1 -- 172.21.15.28:0/1244792635 wait complete. 2024-09-10T10:55:47.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.018+0000 7f6516000640 1 Processor -- start 2024-09-10T10:55:47.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.018+0000 7f6516000640 1 -- start start 2024-09-10T10:55:47.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651015eaa0 0x7f6510160350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651015eef0 0x7f65100b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6510154cd0 con 0x7f65100b4ae0 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65100d0a30 con 0x7f651015eaa0 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f6516000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6510154e50 con 0x7f651015eef0 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f650f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f650f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33150/0 (socket says 172.21.15.28:33150) 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.019+0000 7f650f400640 1 -- 172.21.15.28:0/665178238 learned_addr learned my addr 172.21.15.28:0/665178238 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650ea00640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651015eaa0 0x7f6510160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:47.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 -- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651015eef0 msgr2=0x7f65100b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651015eef0 0x7f65100b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 -- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651015eaa0 msgr2=0x7f6510160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651015eaa0 0x7f6510160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 -- 172.21.15.28:0/665178238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65101f8980 con 0x7f65100b4ae0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f650f400640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f650000c430 tx=0x7f650000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500011790 con 0x7f65100b4ae0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6500017070 con 0x7f65100b4ae0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500011bb0 con 0x7f65100b4ae0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65100aedf0 con 0x7f65100b4ae0 2024-09-10T10:55:47.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.020+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65100af3b0 con 0x7f65100b4ae0 2024-09-10T10:55:47.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.021+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65100b39e0 con 0x7f65100b4ae0 2024-09-10T10:55:47.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.023+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f650001c460 con 0x7f65100b4ae0 2024-09-10T10:55:47.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.023+0000 7f6507400640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 0x7f64f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:47.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.023+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5619+0+0 (secure 0 0 0) 0x7f650008acb0 con 0x7f65100b4ae0 2024-09-10T10:55:47.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.024+0000 7f650ea00640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 0x7f64f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:47.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.025+0000 7f650ea00640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 0x7f64f006e2f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f64fc034000 tx=0x7f64fc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:47.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.029+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f65000596a0 con 0x7f65100b4ae0 2024-09-10T10:55:47.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.212+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f651006faa0 con 0x7f65100b4ae0 2024-09-10T10:55:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.823+0000 7f6507400640 1 -- 172.21.15.28:0/665178238 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v148) ==== 170+0+0 (secure 0 0 0) 0x7f6500058fa0 con 0x7f65100b4ae0 2024-09-10T10:55:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-09-10T10:55:47.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.826+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 msgr2=0x7f64f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.826+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 0x7f64f006e2f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f64fc034000 tx=0x7f64fc02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.826+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 msgr2=0x7f6510154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.826+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f650000c430 tx=0x7f650000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 shutdown_connections 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f64f006be30 0x7f64f006e2f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f651015eef0 0x7f65100b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f651015eaa0 0x7f6510160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 --2- 172.21.15.28:0/665178238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65100b4ae0 0x7f6510154150 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:47.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.827+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 >> 172.21.15.28:0/665178238 conn(0x7f651006bdf0 msgr2=0x7f65101584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.830+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 shutdown_connections 2024-09-10T10:55:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:47.830+0000 7f6516000640 1 -- 172.21.15.28:0/665178238 wait complete. 2024-09-10T10:55:47.866 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T10:55:48.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.025+0000 7f2a9b200640 1 Processor -- start 2024-09-10T10:55:48.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.026+0000 7f2a9b200640 1 -- start start 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2a9c0d2ac0 0x7f2a9c06e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 0x7f2a9c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9c0b1170 con 0x7f2a9c161ac0 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9c0ceba0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c06dfb0 con 0x7f2a9c0b3640 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a99e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2a9c161ac0 0x7f2a9c0d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47206/0 (socket says 172.21.15.28:47206) 2024-09-10T10:55:48.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a99e00640 1 -- 172.21.15.28:0/1900731458 learned_addr learned my addr 172.21.15.28:0/1900731458 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a91400640 1 --2- 172.21.15.28:0/1900731458 >> v2:172.21.15.138:3300/0 conn(0x7f2a9c0d2ac0 0x7f2a9c06e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a9a800640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 0x7f2a9c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.027+0000 7f2a99400640 1 -- 172.21.15.28:0/1900731458 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136660568 0 0) 0x7f2a9c0b1170 con 0x7f2a9c161ac0 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a99400640 1 -- 172.21.15.28:0/1900731458 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a80003520 con 0x7f2a9c161ac0 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a91400640 1 -- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 msgr2=0x7f2a9c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:48.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a91400640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 0x7f2a9c0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a91400640 1 -- 172.21.15.28:0/1900731458 >> v1:172.21.15.28:6789/0 conn(0x7f2a9c161ac0 legacy=0x7f2a9c0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a91400640 1 -- 172.21.15.28:0/1900731458 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0b1450 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.028+0000 7f2a9a800640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 0x7f2a9c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.029+0000 7f2a91400640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c06e6c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a840097d0 tx=0x7f2a8402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7dca084abed914c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.029+0000 7f2a99400640 1 -- 172.21.15.28:0/1900731458 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037070 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.029+0000 7f2a99400640 1 -- 172.21.15.28:0/1900731458 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84032440 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.029+0000 7f2a99400640 1 -- 172.21.15.28:0/1900731458 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a840327f0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 -- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0d2ac0 msgr2=0x7f2a9c06e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c06e6c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a840097d0 tx=0x7f2a8402a250 comp rx=0 tx=0).stop 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 -- 172.21.15.28:0/1900731458 shutdown_connections 2024-09-10T10:55:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c0b3640 0x7f2a9c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 --2- 172.21.15.28:0/1900731458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c06e6c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 -- 172.21.15.28:0/1900731458 >> 172.21.15.28:0/1900731458 conn(0x7f2a9c08bfc0 msgr2=0x7f2a9c0d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 -- 172.21.15.28:0/1900731458 shutdown_connections 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.030+0000 7f2a9b200640 1 -- 172.21.15.28:0/1900731458 wait complete. 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.032+0000 7f2a9b200640 1 Processor -- start 2024-09-10T10:55:48.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.032+0000 7f2a9b200640 1 -- start start 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0b3640 0x7f2a9c15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c161ac0 0x7f2a9c0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c161ea0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c0b1a10 con 0x7f2a9c0b3640 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9c162020 con 0x7f2a9c161ac0 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33162/0 (socket says 172.21.15.28:33162) 2024-09-10T10:55:48.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 -- 172.21.15.28:0/466250110 learned_addr learned my addr 172.21.15.28:0/466250110 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a99e00640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0b3640 0x7f2a9c15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a9a800640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c161ac0 0x7f2a9c0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 -- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c161ac0 msgr2=0x7f2a9c0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c161ac0 0x7f2a9c0cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 -- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0b3640 msgr2=0x7f2a9c15a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0b3640 0x7f2a9c15a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.033+0000 7f2a91400640 1 -- 172.21.15.28:0/466250110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c0a1bc0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a91400640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037040 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84042070 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8403b890 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c0a1ea0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.034+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c0b5ae0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.035+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c0b2e40 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.036+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2a8403ba30 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.038+0000 7f2a92a00640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 0x7f2a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.038+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 6001+0+0 (secure 0 0 0) 0x7f2a840ab6a0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.038+0000 7f2a99e00640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 0x7f2a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:48.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.039+0000 7f2a99e00640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 0x7f2a7806e2f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c001200 tx=0x7f2a8c000bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.043+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a9c0b2e40 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.267+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2a9c06e6c0 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.852+0000 7f2a92a00640 1 -- 172.21.15.28:0/466250110 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v337) ==== 174+0+0 (secure 0 0 0) 0x7f2a840a9e20 con 0x7f2a9c0d2ac0 2024-09-10T10:55:48.857 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 15 and data pool 16 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.866+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 msgr2=0x7f2a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.866+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 0x7f2a7806e2f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c001200 tx=0x7f2a8c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.866+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 msgr2=0x7f2a9c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.866+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a840046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 shutdown_connections 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2a7806be30 0x7f2a7806e2f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a9c161ac0 0x7f2a9c0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a9c0d2ac0 0x7f2a9c0b6320 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 --2- 172.21.15.28:0/466250110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a9c0b3640 0x7f2a9c15a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:48.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.867+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 >> 172.21.15.28:0/466250110 conn(0x7f2a9c08bfc0 msgr2=0x7f2a9c0d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:48.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.868+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 shutdown_connections 2024-09-10T10:55:48.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:48.871+0000 7f2a9b200640 1 -- 172.21.15.28:0/466250110 wait complete. 2024-09-10T10:55:48.901 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:49.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.057+0000 7fa7f8c00640 1 Processor -- start 2024-09-10T10:55:49.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.058+0000 7fa7f8c00640 1 -- start start 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f8c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa7f4072320 0x7fa7f40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f8c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7f415e1f0 0x7fa7f406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f8c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f4146140 con 0x7fa7f4072de0 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f8c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f40cd740 con 0x7fa7f415e1f0 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f8c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f40cda20 con 0x7fa7f4072320 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7f415e1f0 0x7fa7f406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7f415e1f0 0x7fa7f406e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33174/0 (socket says 172.21.15.28:33174) 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f2800640 1 -- 172.21.15.28:0/2534645497 learned_addr learned my addr 172.21.15.28:0/2534645497 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.059+0000 7fa7f1e00640 1 --2- 172.21.15.28:0/2534645497 >> v2:172.21.15.138:3300/0 conn(0x7fa7f4072320 0x7fa7f40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.060+0000 7fa7f2800640 1 -- 172.21.15.28:0/2534645497 >> v2:172.21.15.138:3300/0 conn(0x7fa7f4072320 msgr2=0x7fa7f40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.060+0000 7fa7f2800640 1 --2- 172.21.15.28:0/2534645497 >> v2:172.21.15.138:3300/0 conn(0x7fa7f4072320 0x7fa7f40cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.060+0000 7fa7f2800640 1 -- 172.21.15.28:0/2534645497 >> v1:172.21.15.138:6789/0 conn(0x7fa7f4072de0 legacy=0x7fa7f406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:49.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.060+0000 7fa7f2800640 1 -- 172.21.15.28:0/2534645497 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f40cdd00 con 0x7fa7f415e1f0 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f2800640 1 --2- 172.21.15.28:0/2534645497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f415e1f0 0x7fa7f406e7c0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40b31e0 tx=0x7fa7ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf34dea35360135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f1400640 1 -- 172.21.15.28:0/2534645497 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec036070 con 0x7fa7f415e1f0 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f1400640 1 -- 172.21.15.28:0/2534645497 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ec031600 con 0x7fa7f415e1f0 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f1400640 1 -- 172.21.15.28:0/2534645497 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec0319d0 con 0x7fa7f415e1f0 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f8c00640 1 -- 172.21.15.28:0/2534645497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f415e1f0 msgr2=0x7fa7f406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.061+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/2534645497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f415e1f0 0x7fa7f406e7c0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40b31e0 tx=0x7fa7ec02a6a0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 -- 172.21.15.28:0/2534645497 shutdown_connections 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/2534645497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f415e1f0 0x7fa7f406e7c0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/2534645497 >> v2:172.21.15.138:3300/0 conn(0x7fa7f4072320 0x7fa7f40cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 -- 172.21.15.28:0/2534645497 >> 172.21.15.28:0/2534645497 conn(0x7fa7f4067d70 msgr2=0x7fa7f406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:49.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 -- 172.21.15.28:0/2534645497 shutdown_connections 2024-09-10T10:55:49.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.062+0000 7fa7f8c00640 1 -- 172.21.15.28:0/2534645497 wait complete. 2024-09-10T10:55:49.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.063+0000 7fa7f8c00640 1 Processor -- start 2024-09-10T10:55:49.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.063+0000 7fa7f8c00640 1 -- start start 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 0x7fa7f41e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 0x7fa7f41e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f40cd520 con 0x7fa7f4072320 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f40cd3a0 con 0x7fa7f4072de0 2024-09-10T10:55:49.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.064+0000 7fa7f8c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f406ed00 con 0x7fa7f415e1f0 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33188/0 (socket says 172.21.15.28:33188) 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 -- 172.21.15.28:0/309590587 learned_addr learned my addr 172.21.15.28:0/309590587 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7e9400640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 0x7fa7f41e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f2800640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 0x7fa7f41e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 -- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 msgr2=0x7fa7f41e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 0x7fa7f41e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 -- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 msgr2=0x7fa7f41e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 0x7fa7f41e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 -- 172.21.15.28:0/309590587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f41e5990 con 0x7fa7f4072320 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f2800640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 0x7fa7f41e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:49.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7e9400640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 0x7fa7f41e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:49.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.065+0000 7fa7f1e00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9b0 tx=0x7fa7e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.066+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4017070 con 0x7fa7f4072320 2024-09-10T10:55:49.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.066+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e400c9a0 con 0x7fa7f4072320 2024-09-10T10:55:49.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.066+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e400cd90 con 0x7fa7f4072320 2024-09-10T10:55:49.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.066+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f41e4160 con 0x7fa7f4072320 2024-09-10T10:55:49.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.066+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f41e4720 con 0x7fa7f4072320 2024-09-10T10:55:49.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.067+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f4072700 con 0x7fa7f4072320 2024-09-10T10:55:49.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.068+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa7e401a070 con 0x7fa7f4072320 2024-09-10T10:55:49.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.069+0000 7fa7eaa00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 0x7fa7d006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.069+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7fa7e408b110 con 0x7fa7f4072320 2024-09-10T10:55:49.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.069+0000 7fa7e9400640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 0x7fa7d006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.070+0000 7fa7e9400640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 0x7fa7d006e2f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa7f41e7e40 tx=0x7fa7dc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.073+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7e4059800 con 0x7fa7f4072320 2024-09-10T10:55:49.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.258+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7f40cf470 con 0x7fa7f4072320 2024-09-10T10:55:49.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.260+0000 7fa7eaa00640 1 -- 172.21.15.28:0/309590587 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16997 (secure 0 0 0) 0x7fa7e4059100 con 0x7fa7f4072320 2024-09-10T10:55:49.262 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:49.262 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":149,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:48.269306+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:55:46.065844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T10:55:47.256070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 msgr2=0x7fa7d006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 0x7fa7d006e2f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fa7f41e7e40 tx=0x7fa7dc009040 comp rx=0 tx=0).stop 2024-09-10T10:55:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 msgr2=0x7fa7f41e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400e9b0 tx=0x7fa7e400ee70 comp rx=0 tx=0).stop 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 shutdown_connections 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa7d006be30 0x7fa7d006e2f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7f415e1f0 0x7fa7f41e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7f4072de0 0x7fa7f41e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 --2- 172.21.15.28:0/309590587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7f4072320 0x7fa7f41e8350 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 >> 172.21.15.28:0/309590587 conn(0x7fa7f4067d70 msgr2=0x7fa7f40b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 shutdown_connections 2024-09-10T10:55:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.262+0000 7fa7f8c00640 1 -- 172.21.15.28:0/309590587 wait complete. 2024-09-10T10:55:49.280 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.434+0000 7f52b7e00640 1 Processor -- start 2024-09-10T10:55:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.434+0000 7f52b7e00640 1 -- start start 2024-09-10T10:55:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.435+0000 7f52b7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 0x7f52b00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.435+0000 7f52b7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b006fdb0 con 0x7f52b00b5f80 2024-09-10T10:55:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.435+0000 7f52b7e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b0070090 con 0x7f52b00afd70 2024-09-10T10:55:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.435+0000 7f52b7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f52b0070370 con 0x7f52b00b63d0 2024-09-10T10:55:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.435+0000 7f52b4a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f52b00b5f80 0x7f52b00cd650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47220/0 (socket says 172.21.15.28:47220) 2024-09-10T10:55:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52b4a00640 1 -- 172.21.15.28:0/896465212 learned_addr learned my addr 172.21.15.28:0/896465212 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52b5e00640 1 --2- 172.21.15.28:0/896465212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 0x7f52b00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 449549445 0 0) 0x7f52b006fdb0 con 0x7f52b00b5f80 2024-09-10T10:55:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5298003520 con 0x7f52b00b5f80 2024-09-10T10:55:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084177240 0 0) 0x7f52b0070090 con 0x7f52b00afd70 2024-09-10T10:55:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52b006fdb0 con 0x7f52b00afd70 2024-09-10T10:55:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52b5e00640 1 -- 172.21.15.28:0/896465212 >> v1:172.21.15.138:6789/0 conn(0x7f52b00afd70 legacy=0x7f52b0160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52b5e00640 1 -- 172.21.15.28:0/896465212 >> v1:172.21.15.28:6789/0 conn(0x7f52b00b5f80 legacy=0x7f52b00cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.436+0000 7f52b5e00640 1 -- 172.21.15.28:0/896465212 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0070650 con 0x7f52b00b63d0 2024-09-10T10:55:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.437+0000 7f52b5e00640 1 --2- 172.21.15.28:0/896465212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 0x7f52b00cdd60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009dc0 tx=0x7f52a802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72ac7e8feaa2037f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8007740 con 0x7f52b00b63d0 2024-09-10T10:55:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a80078e0 con 0x7f52b00b63d0 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52afe00640 1 -- 172.21.15.28:0/896465212 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8007cb0 con 0x7f52b00b63d0 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 -- 172.21.15.28:0/896465212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 msgr2=0x7f52b00cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 --2- 172.21.15.28:0/896465212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 0x7f52b00cdd60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009dc0 tx=0x7f52a802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 -- 172.21.15.28:0/896465212 shutdown_connections 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 --2- 172.21.15.28:0/896465212 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00b63d0 0x7f52b00cdd60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 -- 172.21.15.28:0/896465212 >> 172.21.15.28:0/896465212 conn(0x7f52b006bc10 msgr2=0x7f52b006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 -- 172.21.15.28:0/896465212 shutdown_connections 2024-09-10T10:55:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.438+0000 7f52b7e00640 1 -- 172.21.15.28:0/896465212 wait complete. 2024-09-10T10:55:49.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.440+0000 7f52b7e00640 1 Processor -- start 2024-09-10T10:55:49.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.440+0000 7f52b7e00640 1 -- start start 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00afd70 0x7f52b015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b00b5f80 0x7f52b01f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f52b00b0150 con 0x7f52b00b63d0 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52b006fa80 con 0x7f52b00b5f80 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b7e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f52b0160060 con 0x7f52b00afd70 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33204/0 (socket says 172.21.15.28:33204) 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b5e00640 1 -- 172.21.15.28:0/3327420403 learned_addr learned my addr 172.21.15.28:0/3327420403 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b5400640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00afd70 0x7f52b015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.441+0000 7f52b4a00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b00b5f80 0x7f52b01f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 -- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00afd70 msgr2=0x7f52b015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00afd70 0x7f52b015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 -- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b00b5f80 msgr2=0x7f52b01f5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b00b5f80 0x7f52b01f5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 -- 172.21.15.28:0/3327420403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b015c5f0 con 0x7f52b00b63d0 2024-09-10T10:55:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b5e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ee0 tx=0x7f52a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a803f070 con 0x7f52b00b63d0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8031bc0 con 0x7f52b00b63d0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8009450 con 0x7f52b00b63d0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b015c8d0 con 0x7f52b00b63d0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.442+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b015ce90 con 0x7f52b00b63d0 2024-09-10T10:55:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.443+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0160820 con 0x7f52b00b63d0 2024-09-10T10:55:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.445+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52a803b410 con 0x7f52b00b63d0 2024-09-10T10:55:49.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.446+0000 7f52ad600640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 0x7f529006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.446+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f52a80ac570 con 0x7f52b00b63d0 2024-09-10T10:55:49.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.446+0000 7f52b5400640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 0x7f529006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.447+0000 7f52b5400640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 0x7f529006e2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007fd0 tx=0x7f52a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.450+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f52a807a730 con 0x7f52b00b63d0 2024-09-10T10:55:49.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.635+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52b015e8e0 con 0x7f52b00b63d0 2024-09-10T10:55:49.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.636+0000 7f52ad600640 1 -- 172.21.15.28:0/3327420403 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+16997 (secure 0 0 0) 0x7f52a807ea30 con 0x7f52b00b63d0 2024-09-10T10:55:49.639 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:49.639 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":149,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:48.269306+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:55:46.065844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T10:55:47.256070+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.638+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 msgr2=0x7f529006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.638+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 0x7f529006e2f0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f52a4007fd0 tx=0x7f52a4009040 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.638+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 msgr2=0x7f52b01f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.638+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009ee0 tx=0x7f52a8004a40 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 shutdown_connections 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f529006be30 0x7f529006e2f0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52b00b63d0 0x7f52b01f5b40 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52b00b5f80 0x7f52b01f5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 --2- 172.21.15.28:0/3327420403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f52b00afd70 0x7f52b015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 >> 172.21.15.28:0/3327420403 conn(0x7f52b006bc10 msgr2=0x7f52b015d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 shutdown_connections 2024-09-10T10:55:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.639+0000 7f52b7e00640 1 -- 172.21.15.28:0/3327420403 wait complete. 2024-09-10T10:55:49.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T10:55:49.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.814+0000 7f70a1800640 1 Processor -- start 2024-09-10T10:55:49.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.814+0000 7f70a1800640 1 -- start start 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f70a1800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f709c15e900 0x7f709c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f70a1800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f709c072050 0x7f709c072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f70a1800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c0b4620 con 0x7f709c0b4a20 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f70a1800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0729c0 con 0x7f709c15e900 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f70a1800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0ce100 con 0x7f709c072050 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f709c15e900 0x7f709c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f709c15e900 0x7f709c0b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33206/0 (socket says 172.21.15.28:33206) 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 -- 172.21.15.28:0/3503144885 learned_addr learned my addr 172.21.15.28:0/3503144885 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709b400640 1 --2- 172.21.15.28:0/3503144885 >> v2:172.21.15.138:3300/0 conn(0x7f709c072050 0x7f709c072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 -- 172.21.15.28:0/3503144885 >> v2:172.21.15.138:3300/0 conn(0x7f709c072050 msgr2=0x7f709c072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 --2- 172.21.15.28:0/3503144885 >> v2:172.21.15.138:3300/0 conn(0x7f709c072050 0x7f709c072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 -- 172.21.15.28:0/3503144885 >> v1:172.21.15.28:6789/0 conn(0x7f709c0b4a20 legacy=0x7f709c0ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.816+0000 7f709aa00640 1 -- 172.21.15.28:0/3503144885 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0ce3a0 con 0x7f709c15e900 2024-09-10T10:55:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.817+0000 7f709aa00640 1 --2- 172.21.15.28:0/3503144885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c15e900 0x7f709c0b4020 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f709c071dd0 tx=0x7f708c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36662be281fe5212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f709a000640 1 -- 172.21.15.28:0/3503144885 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c037070 con 0x7f709c15e900 2024-09-10T10:55:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f709a000640 1 -- 172.21.15.28:0/3503144885 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708c032440 con 0x7f709c15e900 2024-09-10T10:55:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f709a000640 1 -- 172.21.15.28:0/3503144885 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c032810 con 0x7f709c15e900 2024-09-10T10:55:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 -- 172.21.15.28:0/3503144885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c15e900 msgr2=0x7f709c0b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 --2- 172.21.15.28:0/3503144885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c15e900 0x7f709c0b4020 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f709c071dd0 tx=0x7f708c02a200 comp rx=0 tx=0).stop 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 -- 172.21.15.28:0/3503144885 shutdown_connections 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 --2- 172.21.15.28:0/3503144885 >> v2:172.21.15.138:3300/0 conn(0x7f709c072050 0x7f709c072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 --2- 172.21.15.28:0/3503144885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c15e900 0x7f709c0b4020 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 -- 172.21.15.28:0/3503144885 >> 172.21.15.28:0/3503144885 conn(0x7f709c06bdf0 msgr2=0x7f709c06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 -- 172.21.15.28:0/3503144885 shutdown_connections 2024-09-10T10:55:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.818+0000 7f70a1800640 1 -- 172.21.15.28:0/3503144885 wait complete. 2024-09-10T10:55:49.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.820+0000 7f70a1800640 1 Processor -- start 2024-09-10T10:55:49.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.820+0000 7f70a1800640 1 -- start start 2024-09-10T10:55:49.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f709c072050 0x7f709c153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f709c15e900 0x7f709c15bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0b4e50 con 0x7f709c0b4a20 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0ce960 con 0x7f709c072050 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f70a1800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f709c0cee80 con 0x7f709c15e900 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f7093e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f7093e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33220/0 (socket says 172.21.15.28:33220) 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f7093e00640 1 -- 172.21.15.28:0/81002079 learned_addr learned my addr 172.21.15.28:0/81002079 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f709b400640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f709c15e900 0x7f709c15bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.821+0000 7f709aa00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f709c072050 0x7f709c153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 -- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f709c15e900 msgr2=0x7f709c15bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f709c15e900 0x7f709c15bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 -- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f709c072050 msgr2=0x7f709c153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:49.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f709c072050 0x7f709c153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:49.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 -- 172.21.15.28:0/81002079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c1f3ce0 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7093e00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f708400c430 tx=0x7f708400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70840052d0 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7084005490 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c1f3fc0 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.822+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7084007260 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.823+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c1f4580 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.824+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709c15aaa0 con 0x7f709c0b4a20 2024-09-10T10:55:49.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.825+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7084007400 con 0x7f709c0b4a20 2024-09-10T10:55:49.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.826+0000 7f7092a00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 0x7f707806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:49.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.826+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 6101+0+0 (secure 0 0 0) 0x7f708408a560 con 0x7f709c0b4a20 2024-09-10T10:55:49.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.827+0000 7f709aa00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 0x7f707806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:49.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.827+0000 7f709aa00640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 0x7f707806e2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f708c034000 tx=0x7f708c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:49.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:49.830+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f709c15aaa0 con 0x7f709c0b4a20 2024-09-10T10:55:50.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.059+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f709c072480 con 0x7f709c0b4a20 2024-09-10T10:55:50.823 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:55:50.822+0000 7fe8aba00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:55:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.865+0000 7f7092a00640 1 -- 172.21.15.28:0/81002079 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v340) ==== 118+0+0 (secure 0 0 0) 0x7f7084058690 con 0x7f709c0b4a20 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.867+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 msgr2=0x7f707806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.867+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 0x7f707806e2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f708c034000 tx=0x7f708c02e000 comp rx=0 tx=0).stop 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.867+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 msgr2=0x7f709c15b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.867+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f708400c430 tx=0x7f708400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 shutdown_connections 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f707806be30 0x7f707806e2f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f709c15e900 0x7f709c15bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f709c0b4a20 0x7f709c15b720 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:50.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 --2- 172.21.15.28:0/81002079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f709c072050 0x7f709c153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:50.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 >> 172.21.15.28:0/81002079 conn(0x7f709c06bdf0 msgr2=0x7f709c15a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:50.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 shutdown_connections 2024-09-10T10:55:50.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:50.868+0000 7f70a1800640 1 -- 172.21.15.28:0/81002079 wait complete. 2024-09-10T10:55:50.889 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T10:55:50.889 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T10:55:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.050+0000 7f4bb2c00640 1 Processor -- start 2024-09-10T10:55:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.050+0000 7f4bb2c00640 1 -- start start 2024-09-10T10:55:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.051+0000 7f4bb2c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 0x7f4bac0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.051+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 0x7f4bac0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.051+0000 7f4bb2c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bac0b3f20 con 0x7f4bac15eec0 2024-09-10T10:55:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.051+0000 7f4bb2c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4bac15f440 con 0x7f4bac0b4ae0 2024-09-10T10:55:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.051+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac15f720 con 0x7f4bac15eaa0 2024-09-10T10:55:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bb0c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4bac15eec0 0x7f4bac154cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47240/0 (socket says 172.21.15.28:47240) 2024-09-10T10:55:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bb0c00640 1 -- 172.21.15.28:0/1591250037 learned_addr learned my addr 172.21.15.28:0/1591250037 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bab400640 1 --2- 172.21.15.28:0/1591250037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 0x7f4bac0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4babe00640 1 --2- 172.21.15.28:0/1591250037 >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 0x7f4bac0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897182325 0 0) 0x7f4bac0b3f20 con 0x7f4bac15eec0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b90003520 con 0x7f4bac15eec0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4138249807 0 0) 0x7f4b90003520 con 0x7f4bac15eec0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bab400640 1 -- 172.21.15.28:0/1591250037 >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 msgr2=0x7f4bac0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bab400640 1 --2- 172.21.15.28:0/1591250037 >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 0x7f4bac0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bab400640 1 -- 172.21.15.28:0/1591250037 >> v1:172.21.15.28:6789/0 conn(0x7f4bac15eec0 legacy=0x7f4bac154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4bab400640 1 -- 172.21.15.28:0/1591250037 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac0b31e0 con 0x7f4bac15eaa0 2024-09-10T10:55:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.052+0000 7f4babe00640 1 --2- 172.21.15.28:0/1591250037 >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 0x7f4bac0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:55:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bab400640 1 --2- 172.21.15.28:0/1591250037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 0x7f4bac0ccae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b9802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0db8423741625f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98037070 con 0x7f4bac15eaa0 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9802aba0 con 0x7f4bac15eaa0 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4baaa00640 1 -- 172.21.15.28:0/1591250037 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40019f0 con 0x7f4bac15eaa0 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 -- 172.21.15.28:0/1591250037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 msgr2=0x7f4bac0ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/1591250037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 0x7f4bac0ccae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b980097a0 tx=0x7f4b9802a110 comp rx=0 tx=0).stop 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 -- 172.21.15.28:0/1591250037 shutdown_connections 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/1591250037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac15eaa0 0x7f4bac0ccae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/1591250037 >> v2:172.21.15.138:3300/0 conn(0x7f4bac0b4ae0 0x7f4bac0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.054+0000 7f4bb2c00640 1 -- 172.21.15.28:0/1591250037 >> 172.21.15.28:0/1591250037 conn(0x7f4bac06bdf0 msgr2=0x7f4bac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.055+0000 7f4bb2c00640 1 -- 172.21.15.28:0/1591250037 shutdown_connections 2024-09-10T10:55:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.055+0000 7f4bb2c00640 1 -- 172.21.15.28:0/1591250037 wait complete. 2024-09-10T10:55:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.056+0000 7f4bb2c00640 1 Processor -- start 2024-09-10T10:55:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.056+0000 7f4bb2c00640 1 -- start start 2024-09-10T10:55:51.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac0b4ae0 0x7f4bac0ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bac15eec0 0x7f4bac0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0d09a0 con 0x7f4bac15eaa0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bac0d0820 con 0x7f4bac15eec0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bb2c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4bac0cd020 con 0x7f4bac0b4ae0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33236/0 (socket says 172.21.15.28:33236) 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.057+0000 7f4bab400640 1 -- 172.21.15.28:0/3253719895 learned_addr learned my addr 172.21.15.28:0/3253719895 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4babe00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac0b4ae0 0x7f4bac0ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bb0c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bac15eec0 0x7f4bac0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 -- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac0b4ae0 msgr2=0x7f4bac0ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac0b4ae0 0x7f4bac0ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 -- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bac15eec0 msgr2=0x7f4bac0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bac15eec0 0x7f4bac0ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 -- 172.21.15.28:0/3253719895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bac157d60 con 0x7f4bac15eaa0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.058+0000 7f4bab400640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f4b980098c0 tx=0x7f4b980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.059+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98041070 con 0x7f4bac15eaa0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.059+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b9803c660 con 0x7f4bac15eaa0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.059+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b98037040 con 0x7f4bac15eaa0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.059+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bac06e870 con 0x7f4bac15eaa0 2024-09-10T10:55:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.059+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bac06ee30 con 0x7f4bac15eaa0 2024-09-10T10:55:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.060+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bac068f50 con 0x7f4bac15eaa0 2024-09-10T10:55:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.061+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4b98044070 con 0x7f4bac15eaa0 2024-09-10T10:55:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.061+0000 7f4ba3e00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.062+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f4b980ab740 con 0x7f4bac15eaa0 2024-09-10T10:55:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.062+0000 7f4babe00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.063+0000 7f4babe00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 0x7f4b8c06e320 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4bac15c540 tx=0x7f4b9c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:51.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.066+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b98079870 con 0x7f4bac15eaa0 2024-09-10T10:55:51.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.286+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4bac0cd3f0 con 0x7f4bac15eaa0 2024-09-10T10:55:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.867+0000 7f4ba3e00640 1 -- 172.21.15.28:0/3253719895 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v341) ==== 122+0+0 (secure 0 0 0) 0x7f4b9807db70 con 0x7f4bac15eaa0 2024-09-10T10:55:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 msgr2=0x7f4b8c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 0x7f4b8c06e320 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4bac15c540 tx=0x7f4b9c009190 comp rx=0 tx=0).stop 2024-09-10T10:55:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 msgr2=0x7f4bac155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f4b980098c0 tx=0x7f4b980046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 shutdown_connections 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4b8c06be60 0x7f4b8c06e320 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bac15eec0 0x7f4bac0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bac15eaa0 0x7f4bac155970 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 --2- 172.21.15.28:0/3253719895 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4bac0b4ae0 0x7f4bac0ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 >> 172.21.15.28:0/3253719895 conn(0x7f4bac06bdf0 msgr2=0x7f4bac06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 shutdown_connections 2024-09-10T10:55:51.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:51.870+0000 7f4bb2c00640 1 -- 172.21.15.28:0/3253719895 wait complete. 2024-09-10T10:55:51.891 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T10:55:51.891 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.050+0000 7f09e3200640 1 Processor -- start 2024-09-10T10:55:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.050+0000 7f09e3200640 1 -- start start 2024-09-10T10:55:52.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f09dc0d0dd0 0x7f09dc0d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09dc0cf000 con 0x7f09dc15e660 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc06d3e0 con 0x7f09dc0d1f90 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f09dc0b37d0 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e2800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33240/0 (socket says 172.21.15.28:33240) 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.051+0000 7f09e1400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f09dc15e660 0x7f09dc06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47248/0 (socket says 172.21.15.28:47248) 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e2800640 1 -- 172.21.15.28:0/3858290401 learned_addr learned my addr 172.21.15.28:0/3858290401 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e1e00640 1 --2- 172.21.15.28:0/3858290401 >> v2:172.21.15.138:3300/0 conn(0x7f09dc0d0dd0 0x7f09dc0d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e2800640 1 -- 172.21.15.28:0/3858290401 >> v2:172.21.15.138:3300/0 conn(0x7f09dc0d0dd0 msgr2=0x7f09dc0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e2800640 1 --2- 172.21.15.28:0/3858290401 >> v2:172.21.15.138:3300/0 conn(0x7f09dc0d0dd0 0x7f09dc0d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e2800640 1 -- 172.21.15.28:0/3858290401 >> v1:172.21.15.28:6789/0 conn(0x7f09dc15e660 legacy=0x7f09dc06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.052+0000 7f09e2800640 1 -- 172.21.15.28:0/3858290401 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc0d16f0 con 0x7f09dc0d1f90 2024-09-10T10:55:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.053+0000 7f09e2800640 1 --2- 172.21.15.28:0/3858290401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b6fb0 tx=0x7f09d802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c785ff693e71ec80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.053+0000 7f09e0a00640 1 -- 172.21.15.28:0/3858290401 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8036070 con 0x7f09dc0d1f90 2024-09-10T10:55:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.053+0000 7f09e0a00640 1 -- 172.21.15.28:0/3858290401 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d8031600 con 0x7f09dc0d1f90 2024-09-10T10:55:52.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.053+0000 7f09e0a00640 1 -- 172.21.15.28:0/3858290401 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80319d0 con 0x7f09dc0d1f90 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 -- 172.21.15.28:0/3858290401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d1f90 msgr2=0x7f09dc0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 --2- 172.21.15.28:0/3858290401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b6fb0 tx=0x7f09d802a730 comp rx=0 tx=0).stop 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 -- 172.21.15.28:0/3858290401 shutdown_connections 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 --2- 172.21.15.28:0/3858290401 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0cf7c0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 --2- 172.21.15.28:0/3858290401 >> v2:172.21.15.138:3300/0 conn(0x7f09dc0d0dd0 0x7f09dc0d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 -- 172.21.15.28:0/3858290401 >> 172.21.15.28:0/3858290401 conn(0x7f09dc08bfc0 msgr2=0x7f09dc0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:52.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 -- 172.21.15.28:0/3858290401 shutdown_connections 2024-09-10T10:55:52.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.054+0000 7f09e3200640 1 -- 172.21.15.28:0/3858290401 wait complete. 2024-09-10T10:55:52.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.056+0000 7f09e3200640 1 Processor -- start 2024-09-10T10:55:52.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.056+0000 7f09e3200640 1 -- start start 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09dc15e660 0x7f09dc0b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc0d2c60 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc0cf330 con 0x7f09dc0d1f90 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f09dc0d2f40 con 0x7f09dc15e660 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33246/0 (socket says 172.21.15.28:33246) 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 -- 172.21.15.28:0/4275928655 learned_addr learned my addr 172.21.15.28:0/4275928655 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1400640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e2800640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09dc15e660 0x7f09dc0b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 -- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09dc15e660 msgr2=0x7f09dc0b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09dc15e660 0x7f09dc0b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 -- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 msgr2=0x7f09dc0c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1e00640 1 -- 172.21.15.28:0/4275928655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc14deb0 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.057+0000 7f09e1400640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09e1e00640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9e0 tx=0x7f09d000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0017070 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d000c970 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc14e190 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d000cd60 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.058+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc14e750 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.059+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc0b35a0 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.060+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f09d001a070 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.061+0000 7f09d6000640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 0x7f09c006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.061+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f09d008b410 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.061+0000 7f09e1400640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 0x7f09c006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.062+0000 7f09e1400640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 0x7f09c006e2f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f09cc007fd0 tx=0x7f09cc007d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.066+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f09dc0b35a0 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.286+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09dc0d3220 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.287+0000 7f09d6000640 1 -- 172.21.15.28:0/4275928655 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5234 (secure 0 0 0) 0x7f09d0059b90 con 0x7f09dc0d0dd0 2024-09-10T10:55:52.288 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:52.288 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":341,"btime":"2024-09-10T10:55:51:866579+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:55:51.288071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9505":{"gid":9505,"name":"a","rank":0,"incarnation":337,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/354907385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":354907385},{"type":"v1","addr":"172.21.15.28:6835","nonce":354907385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9505,"qdb_cluster":[9505]},"id":7}]} 2024-09-10T10:55:52.288 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 341 2024-09-10T10:55:52.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 msgr2=0x7f09c006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 0x7f09c006e2f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f09cc007fd0 tx=0x7f09cc007d60 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 msgr2=0x7f09dc14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9e0 tx=0x7f09d000eea0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 shutdown_connections 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f09c006be30 0x7f09c006e2f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f09dc15e660 0x7f09dc0b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09dc0d1f90 0x7f09dc0c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 --2- 172.21.15.28:0/4275928655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09dc0d0dd0 0x7f09dc14d3a0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 >> 172.21.15.28:0/4275928655 conn(0x7f09dc08bfc0 msgr2=0x7f09dc06ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 shutdown_connections 2024-09-10T10:55:52.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.289+0000 7f09e3200640 1 -- 172.21.15.28:0/4275928655 wait complete. 2024-09-10T10:55:52.306 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:55:52.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.461+0000 7f6692000640 1 Processor -- start 2024-09-10T10:55:52.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.461+0000 7f6692000640 1 -- start start 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f668c15af50 0x7f668c072de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15b330 0x7f668c073320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f668c0b42b0 0x7f668c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f668c06dc70 con 0x7f668c15af50 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f668c06df10 con 0x7f668c0b42b0 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6692000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f668c06e1f0 con 0x7f668c15b330 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6690c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f668c15af50 0x7f668c072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6690c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f668c15af50 0x7f668c072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33250/0 (socket says 172.21.15.28:33250) 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.462+0000 7f6690c00640 1 -- 172.21.15.28:0/2144525068 learned_addr learned my addr 172.21.15.28:0/2144525068 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6691600640 1 --2- 172.21.15.28:0/2144525068 >> v2:172.21.15.138:3300/0 conn(0x7f668c0b42b0 0x7f668c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f668be00640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15b330 0x7f668c073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6690c00640 1 -- 172.21.15.28:0/2144525068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15b330 msgr2=0x7f668c073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6690c00640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15b330 0x7f668c073320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6690c00640 1 -- 172.21.15.28:0/2144525068 >> v2:172.21.15.138:3300/0 conn(0x7f668c0b42b0 msgr2=0x7f668c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6690c00640 1 --2- 172.21.15.28:0/2144525068 >> v2:172.21.15.138:3300/0 conn(0x7f668c0b42b0 0x7f668c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.463+0000 7f6690c00640 1 -- 172.21.15.28:0/2144525068 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c06e4d0 con 0x7f668c15af50 2024-09-10T10:55:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.464+0000 7f6690c00640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15af50 0x7f668c072de0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b23a0 tx=0x7f668002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219ac179e7bdbf63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.464+0000 7f668b400640 1 -- 172.21.15.28:0/2144525068 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680037070 con 0x7f668c15af50 2024-09-10T10:55:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.464+0000 7f668b400640 1 -- 172.21.15.28:0/2144525068 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6680032440 con 0x7f668c15af50 2024-09-10T10:55:52.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.464+0000 7f668b400640 1 -- 172.21.15.28:0/2144525068 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680032810 con 0x7f668c15af50 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 -- 172.21.15.28:0/2144525068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15af50 msgr2=0x7f668c072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15af50 0x7f668c072de0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b23a0 tx=0x7f668002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 -- 172.21.15.28:0/2144525068 shutdown_connections 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 --2- 172.21.15.28:0/2144525068 >> v2:172.21.15.138:3300/0 conn(0x7f668c0b42b0 0x7f668c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15b330 0x7f668c073320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 --2- 172.21.15.28:0/2144525068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15af50 0x7f668c072de0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 -- 172.21.15.28:0/2144525068 >> 172.21.15.28:0/2144525068 conn(0x7f668c06c240 msgr2=0x7f668c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 -- 172.21.15.28:0/2144525068 shutdown_connections 2024-09-10T10:55:52.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.465+0000 7f6692000640 1 -- 172.21.15.28:0/2144525068 wait complete. 2024-09-10T10:55:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.466+0000 7f6692000640 1 Processor -- start 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.466+0000 7f6692000640 1 -- start start 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c0b42b0 0x7f668c1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15af50 0x7f668c167960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0adbc0 con 0x7f668c15b330 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0b4c70 con 0x7f668c0b42b0 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.467+0000 7f6692000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f668c0add40 con 0x7f668c15af50 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33260/0 (socket says 172.21.15.28:33260) 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 -- 172.21.15.28:0/870616326 learned_addr learned my addr 172.21.15.28:0/870616326 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6690c00640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c0b42b0 0x7f668c1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f668be00640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15af50 0x7f668c167960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 -- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15af50 msgr2=0x7f668c167960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15af50 0x7f668c167960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 -- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c0b42b0 msgr2=0x7f668c1495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c0b42b0 0x7f668c1495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 -- 172.21.15.28:0/870616326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c0cc950 con 0x7f668c15b330 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.468+0000 7f6691600640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f6684007e90 tx=0x7f668400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.469+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684014420 con 0x7f668c15b330 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.469+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6684019070 con 0x7f668c15b330 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.469+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c0ccc30 con 0x7f668c15b330 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.469+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684014820 con 0x7f668c15b330 2024-09-10T10:55:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.469+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c09d5e0 con 0x7f668c15b330 2024-09-10T10:55:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.470+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c09d7b0 con 0x7f668c15b330 2024-09-10T10:55:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.471+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f66840149c0 con 0x7f668c15b330 2024-09-10T10:55:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.472+0000 7f6688c00640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 0x7f666c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.472+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f668408b720 con 0x7f668c15b330 2024-09-10T10:55:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.472+0000 7f6690c00640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 0x7f666c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.473+0000 7f6690c00640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 0x7f666c06e2f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6680034000 tx=0x7f668002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.476+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66840598f0 con 0x7f668c15b330 2024-09-10T10:55:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.660+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f668c15ea00 con 0x7f668c15b330 2024-09-10T10:55:52.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.661+0000 7f6688c00640 1 -- 172.21.15.28:0/870616326 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16984 (secure 0 0 0) 0x7f668405dbf0 con 0x7f668c15b330 2024-09-10T10:55:52.663 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:52.663 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":150,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:55:49.853586+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:55:46.065844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T10:55:47.256070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:55:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 msgr2=0x7f666c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 0x7f666c06e2f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6680034000 tx=0x7f668002e000 comp rx=0 tx=0).stop 2024-09-10T10:55:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 msgr2=0x7f668c14fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f6684007e90 tx=0x7f668400e4b0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 shutdown_connections 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f666c06be30 0x7f666c06e2f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f668c15b330 0x7f668c14fc10 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f668c15af50 0x7f668c167960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 --2- 172.21.15.28:0/870616326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f668c0b42b0 0x7f668c1495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 >> 172.21.15.28:0/870616326 conn(0x7f668c06c240 msgr2=0x7f668c0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 shutdown_connections 2024-09-10T10:55:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.664+0000 7f6692000640 1 -- 172.21.15.28:0/870616326 wait complete. 2024-09-10T10:55:52.681 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:55:52.682 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:55:52.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.837+0000 7f35b5a00640 1 Processor -- start 2024-09-10T10:55:52.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.837+0000 7f35b5a00640 1 -- start start 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35b5a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35b0072dc0 0x7f35b00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35b5a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35b00b44c0 con 0x7f35b00b3c90 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35b5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35b015ead0 con 0x7f35b00afd60 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35b5a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f35b00b0140 con 0x7f35b0072dc0 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35ae000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35b0072dc0 0x7f35b00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35af400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f35b00b3c90 0x7f35b006f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47262/0 (socket says 172.21.15.28:47262) 2024-09-10T10:55:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.839+0000 7f35af400640 1 -- 172.21.15.28:0/1622566916 learned_addr learned my addr 172.21.15.28:0/1622566916 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.840+0000 7f35ae000640 1 -- 172.21.15.28:0/1622566916 >> v1:172.21.15.138:6789/0 conn(0x7f35b00afd60 legacy=0x7f35b00726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.840+0000 7f35ae000640 1 -- 172.21.15.28:0/1622566916 >> v1:172.21.15.28:6789/0 conn(0x7f35b00b3c90 legacy=0x7f35b006f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.840+0000 7f35ae000640 1 -- 172.21.15.28:0/1622566916 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b015edb0 con 0x7f35b0072dc0 2024-09-10T10:55:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.841+0000 7f35ae000640 1 --2- 172.21.15.28:0/1622566916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b0072dc0 0x7f35b00b3750 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f359c0097a0 tx=0x7f359c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110452922180e74f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.841+0000 7f35ad600640 1 -- 172.21.15.28:0/1622566916 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c037070 con 0x7f35b0072dc0 2024-09-10T10:55:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.841+0000 7f35ad600640 1 -- 172.21.15.28:0/1622566916 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c032440 con 0x7f35b0072dc0 2024-09-10T10:55:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.841+0000 7f35ad600640 1 -- 172.21.15.28:0/1622566916 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c032810 con 0x7f35b0072dc0 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 -- 172.21.15.28:0/1622566916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b0072dc0 msgr2=0x7f35b00b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 --2- 172.21.15.28:0/1622566916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b0072dc0 0x7f35b00b3750 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f359c0097a0 tx=0x7f359c02a230 comp rx=0 tx=0).stop 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 -- 172.21.15.28:0/1622566916 shutdown_connections 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 --2- 172.21.15.28:0/1622566916 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b0072dc0 0x7f35b00b3750 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 -- 172.21.15.28:0/1622566916 >> 172.21.15.28:0/1622566916 conn(0x7f35b006c030 msgr2=0x7f35b006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 -- 172.21.15.28:0/1622566916 shutdown_connections 2024-09-10T10:55:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.842+0000 7f35b5a00640 1 -- 172.21.15.28:0/1622566916 wait complete. 2024-09-10T10:55:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.843+0000 7f35b5a00640 1 Processor -- start 2024-09-10T10:55:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.843+0000 7f35b5a00640 1 -- start start 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35b0072dc0 0x7f35b01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 0x7f35b01f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35b015f890 con 0x7f35b00afd60 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35b015f370 con 0x7f35b00b3c90 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.844+0000 7f35b5a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35b00b40e0 con 0x7f35b0072dc0 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33280/0 (socket says 172.21.15.28:33280) 2024-09-10T10:55:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 -- 172.21.15.28:0/337369284 learned_addr learned my addr 172.21.15.28:0/337369284 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35af400640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 0x7f35b01f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35aea00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35b0072dc0 0x7f35b01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 -- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35b0072dc0 msgr2=0x7f35b01536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35b0072dc0 0x7f35b01536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 -- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 msgr2=0x7f35b01f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 0x7f35b01f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 -- 172.21.15.28:0/337369284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b00b2390 con 0x7f35b00afd60 2024-09-10T10:55:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35af400640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 0x7f35b01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.845+0000 7f35ae000640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f359c0098c0 tx=0x7f359c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.846+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c037040 con 0x7f35b00afd60 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.846+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c030770 con 0x7f35b00afd60 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.846+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b00b2da0 con 0x7f35b00afd60 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.846+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c030b40 con 0x7f35b00afd60 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.846+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b0155a40 con 0x7f35b00afd60 2024-09-10T10:55:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.847+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b00b3750 con 0x7f35b00afd60 2024-09-10T10:55:52.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.848+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f359c044020 con 0x7f35b00afd60 2024-09-10T10:55:52.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.848+0000 7f35a6a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 0x7f359006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:52.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.849+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f359c0ab630 con 0x7f35b00afd60 2024-09-10T10:55:52.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.849+0000 7f35aea00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 0x7f359006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:52.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.850+0000 7f35aea00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 0x7f359006e2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000f60 tx=0x7f35a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:52.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:52.853+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f359c032e20 con 0x7f35b00afd60 2024-09-10T10:55:53.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.034+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35b006c6c0 con 0x7f359006be30 2024-09-10T10:55:53.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.042+0000 7f35a6a00640 1 -- 172.21.15.28:0/337369284 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f35b006c6c0 con 0x7f359006be30 2024-09-10T10:55:53.046 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:53.047 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:55:53.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 msgr2=0x7f359006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 0x7f359006e2f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f35a0000f60 tx=0x7f35a0009040 comp rx=0 tx=0).stop 2024-09-10T10:55:53.049 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 msgr2=0x7f35b01530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f359c0098c0 tx=0x7f359c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 shutdown_connections 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f359006be30 0x7f359006e2f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35b00b3c90 0x7f35b01f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35b00afd60 0x7f35b01530b0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 --2- 172.21.15.28:0/337369284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35b0072dc0 0x7f35b01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.045+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 >> 172.21.15.28:0/337369284 conn(0x7f35b006c030 msgr2=0x7f35b0070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.046+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 shutdown_connections 2024-09-10T10:55:53.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.046+0000 7f35b5a00640 1 -- 172.21.15.28:0/337369284 wait complete. 2024-09-10T10:55:53.062 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-10T10:55:52.674111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290788,"kb_used_data":16284,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683932,"statfs":{"total":773094113280,"available":772796346368,"internally_reserved":0,"allocated":16674816,"data_stored":8983737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.852121"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865805+0000","last_change":"2024-09-10T10:55:48.867834+0000","last_active":"2024-09-10T10:55:49.865805+0000","last_peered":"2024-09-10T10:55:49.865805+0000","last_clean":"2024-09-10T10:55:49.865805+0000","last_became_active":"2024-09-10T10:55:48.867563+0000","last_became_peered":"2024-09-10T10:55:48.867563+0000","last_unstale":"2024-09-10T10:55:49.865805+0000","last_undegraded":"2024-09-10T10:55:49.865805+0000","last_fullsized":"2024-09-10T10:55:49.865805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:00.429383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947062+0000","last_change":"2024-09-10T10:55:48.855439+0000","last_active":"2024-09-10T10:55:49.947062+0000","last_peered":"2024-09-10T10:55:49.947062+0000","last_clean":"2024-09-10T10:55:49.947062+0000","last_became_active":"2024-09-10T10:55:48.855159+0000","last_became_peered":"2024-09-10T10:55:48.855159+0000","last_unstale":"2024-09-10T10:55:49.947062+0000","last_undegraded":"2024-09-10T10:55:49.947062+0000","last_fullsized":"2024-09-10T10:55:49.947062+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:12.344418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.834144+0000","last_change":"2024-09-10T10:55:47.834144+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.834144+0000","last_undegraded":"2024-09-10T10:55:47.834144+0000","last_fullsized":"2024-09-10T10:55:47.834144+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864724+0000","last_change":"2024-09-10T10:55:48.853911+0000","last_active":"2024-09-10T10:55:49.864724+0000","last_peered":"2024-09-10T10:55:49.864724+0000","last_clean":"2024-09-10T10:55:49.864724+0000","last_became_active":"2024-09-10T10:55:48.853601+0000","last_became_peered":"2024-09-10T10:55:48.853601+0000","last_unstale":"2024-09-10T10:55:49.864724+0000","last_undegraded":"2024-09-10T10:55:49.864724+0000","last_fullsized":"2024-09-10T10:55:49.864724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:51.207539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865887+0000","last_change":"2024-09-10T10:55:48.860012+0000","last_active":"2024-09-10T10:55:49.865887+0000","last_peered":"2024-09-10T10:55:49.865887+0000","last_clean":"2024-09-10T10:55:49.865887+0000","last_became_active":"2024-09-10T10:55:48.859059+0000","last_became_peered":"2024-09-10T10:55:48.859059+0000","last_unstale":"2024-09-10T10:55:49.865887+0000","last_undegraded":"2024-09-10T10:55:49.865887+0000","last_fullsized":"2024-09-10T10:55:49.865887+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.465847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872011+0000","last_change":"2024-09-10T10:55:48.872626+0000","last_active":"2024-09-10T10:55:49.872011+0000","last_peered":"2024-09-10T10:55:49.872011+0000","last_clean":"2024-09-10T10:55:49.872011+0000","last_became_active":"2024-09-10T10:55:48.872334+0000","last_became_peered":"2024-09-10T10:55:48.872334+0000","last_unstale":"2024-09-10T10:55:49.872011+0000","last_undegraded":"2024-09-10T10:55:49.872011+0000","last_fullsized":"2024-09-10T10:55:49.872011+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:51.489910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871869+0000","last_change":"2024-09-10T10:55:48.856346+0000","last_active":"2024-09-10T10:55:49.871869+0000","last_peered":"2024-09-10T10:55:49.871869+0000","last_clean":"2024-09-10T10:55:49.871869+0000","last_became_active":"2024-09-10T10:55:48.856069+0000","last_became_peered":"2024-09-10T10:55:48.856069+0000","last_unstale":"2024-09-10T10:55:49.871869+0000","last_undegraded":"2024-09-10T10:55:49.871869+0000","last_fullsized":"2024-09-10T10:55:49.871869+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961154+0000","last_change":"2024-09-10T10:55:48.876456+0000","last_active":"2024-09-10T10:55:49.961154+0000","last_peered":"2024-09-10T10:55:49.961154+0000","last_clean":"2024-09-10T10:55:49.961154+0000","last_became_active":"2024-09-10T10:55:48.876149+0000","last_became_peered":"2024-09-10T10:55:48.876149+0000","last_unstale":"2024-09-10T10:55:49.961154+0000","last_undegraded":"2024-09-10T10:55:49.961154+0000","last_fullsized":"2024-09-10T10:55:49.961154+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947262+0000","last_change":"2024-09-10T10:55:48.863492+0000","last_active":"2024-09-10T10:55:49.947262+0000","last_peered":"2024-09-10T10:55:49.947262+0000","last_clean":"2024-09-10T10:55:49.947262+0000","last_became_active":"2024-09-10T10:55:48.863048+0000","last_became_peered":"2024-09-10T10:55:48.863048+0000","last_unstale":"2024-09-10T10:55:49.947262+0000","last_undegraded":"2024-09-10T10:55:49.947262+0000","last_fullsized":"2024-09-10T10:55:49.947262+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:30.871617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961998+0000","last_change":"2024-09-10T10:55:48.866566+0000","last_active":"2024-09-10T10:55:49.961998+0000","last_peered":"2024-09-10T10:55:49.961998+0000","last_clean":"2024-09-10T10:55:49.961998+0000","last_became_active":"2024-09-10T10:55:48.866256+0000","last_became_peered":"2024-09-10T10:55:48.866256+0000","last_unstale":"2024-09-10T10:55:49.961998+0000","last_undegraded":"2024-09-10T10:55:49.961998+0000","last_fullsized":"2024-09-10T10:55:49.961998+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:16.476043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862921+0000","last_change":"2024-09-10T10:55:48.853498+0000","last_active":"2024-09-10T10:55:49.862921+0000","last_peered":"2024-09-10T10:55:49.862921+0000","last_clean":"2024-09-10T10:55:49.862921+0000","last_became_active":"2024-09-10T10:55:48.853209+0000","last_became_peered":"2024-09-10T10:55:48.853209+0000","last_unstale":"2024-09-10T10:55:49.862921+0000","last_undegraded":"2024-09-10T10:55:49.862921+0000","last_fullsized":"2024-09-10T10:55:49.862921+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:37.984075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872273+0000","last_change":"2024-09-10T10:55:48.865380+0000","last_active":"2024-09-10T10:55:49.872273+0000","last_peered":"2024-09-10T10:55:49.872273+0000","last_clean":"2024-09-10T10:55:49.872273+0000","last_became_active":"2024-09-10T10:55:48.865102+0000","last_became_peered":"2024-09-10T10:55:48.865102+0000","last_unstale":"2024-09-10T10:55:49.872273+0000","last_undegraded":"2024-09-10T10:55:49.872273+0000","last_fullsized":"2024-09-10T10:55:49.872273+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:39.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865722+0000","last_change":"2024-09-10T10:55:48.863226+0000","last_active":"2024-09-10T10:55:49.865722+0000","last_peered":"2024-09-10T10:55:49.865722+0000","last_clean":"2024-09-10T10:55:49.865722+0000","last_became_active":"2024-09-10T10:55:48.862762+0000","last_became_peered":"2024-09-10T10:55:48.862762+0000","last_unstale":"2024-09-10T10:55:49.865722+0000","last_undegraded":"2024-09-10T10:55:49.865722+0000","last_fullsized":"2024-09-10T10:55:49.865722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:24.148383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872189+0000","last_change":"2024-09-10T10:55:48.864947+0000","last_active":"2024-09-10T10:55:49.872189+0000","last_peered":"2024-09-10T10:55:49.872189+0000","last_clean":"2024-09-10T10:55:49.872189+0000","last_became_active":"2024-09-10T10:55:48.864649+0000","last_became_peered":"2024-09-10T10:55:48.864649+0000","last_unstale":"2024-09-10T10:55:49.872189+0000","last_undegraded":"2024-09-10T10:55:49.872189+0000","last_fullsized":"2024-09-10T10:55:49.872189+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:53.436925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946934+0000","last_change":"2024-09-10T10:55:48.862215+0000","last_active":"2024-09-10T10:55:49.946934+0000","last_peered":"2024-09-10T10:55:49.946934+0000","last_clean":"2024-09-10T10:55:49.946934+0000","last_became_active":"2024-09-10T10:55:48.861706+0000","last_became_peered":"2024-09-10T10:55:48.861706+0000","last_unstale":"2024-09-10T10:55:49.946934+0000","last_undegraded":"2024-09-10T10:55:49.946934+0000","last_fullsized":"2024-09-10T10:55:49.946934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:24.201115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960292+0000","last_change":"2024-09-10T10:55:48.879362+0000","last_active":"2024-09-10T10:55:49.960292+0000","last_peered":"2024-09-10T10:55:49.960292+0000","last_clean":"2024-09-10T10:55:49.960292+0000","last_became_active":"2024-09-10T10:55:48.875054+0000","last_became_peered":"2024-09-10T10:55:48.875054+0000","last_unstale":"2024-09-10T10:55:49.960292+0000","last_undegraded":"2024-09-10T10:55:49.960292+0000","last_fullsized":"2024-09-10T10:55:49.960292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.836013+0000","last_change":"2024-09-10T10:55:47.836013+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.836013+0000","last_undegraded":"2024-09-10T10:55:47.836013+0000","last_fullsized":"2024-09-10T10:55:47.836013+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947217+0000","last_change":"2024-09-10T10:55:48.861625+0000","last_active":"2024-09-10T10:55:49.947217+0000","last_peered":"2024-09-10T10:55:49.947217+0000","last_clean":"2024-09-10T10:55:49.947217+0000","last_became_active":"2024-09-10T10:55:48.860626+0000","last_became_peered":"2024-09-10T10:55:48.860626+0000","last_unstale":"2024-09-10T10:55:49.947217+0000","last_undegraded":"2024-09-10T10:55:49.947217+0000","last_fullsized":"2024-09-10T10:55:49.947217+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:19.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865795+0000","last_change":"2024-09-10T10:55:48.855667+0000","last_active":"2024-09-10T10:55:49.865795+0000","last_peered":"2024-09-10T10:55:49.865795+0000","last_clean":"2024-09-10T10:55:49.865795+0000","last_became_active":"2024-09-10T10:55:48.854331+0000","last_became_peered":"2024-09-10T10:55:48.854331+0000","last_unstale":"2024-09-10T10:55:49.865795+0000","last_undegraded":"2024-09-10T10:55:49.865795+0000","last_fullsized":"2024-09-10T10:55:49.865795+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:11.006767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960683+0000","last_change":"2024-09-10T10:55:48.874514+0000","last_active":"2024-09-10T10:55:49.960683+0000","last_peered":"2024-09-10T10:55:49.960683+0000","last_clean":"2024-09-10T10:55:49.960683+0000","last_became_active":"2024-09-10T10:55:48.874243+0000","last_became_peered":"2024-09-10T10:55:48.874243+0000","last_unstale":"2024-09-10T10:55:49.960683+0000","last_undegraded":"2024-09-10T10:55:49.960683+0000","last_fullsized":"2024-09-10T10:55:49.960683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.639323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835270+0000","last_change":"2024-09-10T10:55:47.835270+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835270+0000","last_undegraded":"2024-09-10T10:55:47.835270+0000","last_fullsized":"2024-09-10T10:55:47.835270+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960995+0000","last_change":"2024-09-10T10:55:48.878105+0000","last_active":"2024-09-10T10:55:49.960995+0000","last_peered":"2024-09-10T10:55:49.960995+0000","last_clean":"2024-09-10T10:55:49.960995+0000","last_became_active":"2024-09-10T10:55:48.876598+0000","last_became_peered":"2024-09-10T10:55:48.876598+0000","last_unstale":"2024-09-10T10:55:49.960995+0000","last_undegraded":"2024-09-10T10:55:49.960995+0000","last_fullsized":"2024-09-10T10:55:49.960995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:47.718665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961652+0000","last_change":"2024-09-10T10:55:48.876273+0000","last_active":"2024-09-10T10:55:49.961652+0000","last_peered":"2024-09-10T10:55:49.961652+0000","last_clean":"2024-09-10T10:55:49.961652+0000","last_became_active":"2024-09-10T10:55:48.875814+0000","last_became_peered":"2024-09-10T10:55:48.875814+0000","last_unstale":"2024-09-10T10:55:49.961652+0000","last_undegraded":"2024-09-10T10:55:49.961652+0000","last_fullsized":"2024-09-10T10:55:49.961652+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:23.371793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961329+0000","last_change":"2024-09-10T10:55:48.869413+0000","last_active":"2024-09-10T10:55:49.961329+0000","last_peered":"2024-09-10T10:55:49.961329+0000","last_clean":"2024-09-10T10:55:49.961329+0000","last_became_active":"2024-09-10T10:55:48.869149+0000","last_became_peered":"2024-09-10T10:55:48.869149+0000","last_unstale":"2024-09-10T10:55:49.961329+0000","last_undegraded":"2024-09-10T10:55:49.961329+0000","last_fullsized":"2024-09-10T10:55:49.961329+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:38.451801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960863+0000","last_change":"2024-09-10T10:55:48.866804+0000","last_active":"2024-09-10T10:55:49.960863+0000","last_peered":"2024-09-10T10:55:49.960863+0000","last_clean":"2024-09-10T10:55:49.960863+0000","last_became_active":"2024-09-10T10:55:48.866480+0000","last_became_peered":"2024-09-10T10:55:48.866480+0000","last_unstale":"2024-09-10T10:55:49.960863+0000","last_undegraded":"2024-09-10T10:55:49.960863+0000","last_fullsized":"2024-09-10T10:55:49.960863+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:31.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960874+0000","last_change":"2024-09-10T10:55:48.872545+0000","last_active":"2024-09-10T10:55:49.960874+0000","last_peered":"2024-09-10T10:55:49.960874+0000","last_clean":"2024-09-10T10:55:49.960874+0000","last_became_active":"2024-09-10T10:55:48.872150+0000","last_became_peered":"2024-09-10T10:55:48.872150+0000","last_unstale":"2024-09-10T10:55:49.960874+0000","last_undegraded":"2024-09-10T10:55:49.960874+0000","last_fullsized":"2024-09-10T10:55:49.960874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960690+0000","last_change":"2024-09-10T10:55:48.873569+0000","last_active":"2024-09-10T10:55:49.960690+0000","last_peered":"2024-09-10T10:55:49.960690+0000","last_clean":"2024-09-10T10:55:49.960690+0000","last_became_active":"2024-09-10T10:55:48.872446+0000","last_became_peered":"2024-09-10T10:55:48.872446+0000","last_unstale":"2024-09-10T10:55:49.960690+0000","last_undegraded":"2024-09-10T10:55:49.960690+0000","last_fullsized":"2024-09-10T10:55:49.960690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:58.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872284+0000","last_change":"2024-09-10T10:55:48.866629+0000","last_active":"2024-09-10T10:55:49.872284+0000","last_peered":"2024-09-10T10:55:49.872284+0000","last_clean":"2024-09-10T10:55:49.872284+0000","last_became_active":"2024-09-10T10:55:48.866351+0000","last_became_peered":"2024-09-10T10:55:48.866351+0000","last_unstale":"2024-09-10T10:55:49.872284+0000","last_undegraded":"2024-09-10T10:55:49.872284+0000","last_fullsized":"2024-09-10T10:55:49.872284+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:20.523256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961962+0000","last_change":"2024-09-10T10:55:48.874303+0000","last_active":"2024-09-10T10:55:49.961962+0000","last_peered":"2024-09-10T10:55:49.961962+0000","last_clean":"2024-09-10T10:55:49.961962+0000","last_became_active":"2024-09-10T10:55:48.873605+0000","last_became_peered":"2024-09-10T10:55:48.873605+0000","last_unstale":"2024-09-10T10:55:49.961962+0000","last_undegraded":"2024-09-10T10:55:49.961962+0000","last_fullsized":"2024-09-10T10:55:49.961962+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:43.380426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862682+0000","last_change":"2024-09-10T10:55:48.851514+0000","last_active":"2024-09-10T10:55:49.862682+0000","last_peered":"2024-09-10T10:55:49.862682+0000","last_clean":"2024-09-10T10:55:49.862682+0000","last_became_active":"2024-09-10T10:55:48.851231+0000","last_became_peered":"2024-09-10T10:55:48.851231+0000","last_unstale":"2024-09-10T10:55:49.862682+0000","last_undegraded":"2024-09-10T10:55:49.862682+0000","last_fullsized":"2024-09-10T10:55:49.862682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:21.797943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871910+0000","last_change":"2024-09-10T10:55:48.860423+0000","last_active":"2024-09-10T10:55:49.871910+0000","last_peered":"2024-09-10T10:55:49.871910+0000","last_clean":"2024-09-10T10:55:49.871910+0000","last_became_active":"2024-09-10T10:55:48.860101+0000","last_became_peered":"2024-09-10T10:55:48.860101+0000","last_unstale":"2024-09-10T10:55:49.871910+0000","last_undegraded":"2024-09-10T10:55:49.871910+0000","last_fullsized":"2024-09-10T10:55:49.871910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:15.280350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960768+0000","last_change":"2024-09-10T10:55:48.861221+0000","last_active":"2024-09-10T10:55:49.960768+0000","last_peered":"2024-09-10T10:55:49.960768+0000","last_clean":"2024-09-10T10:55:49.960768+0000","last_became_active":"2024-09-10T10:55:48.860949+0000","last_became_peered":"2024-09-10T10:55:48.860949+0000","last_unstale":"2024-09-10T10:55:49.960768+0000","last_undegraded":"2024-09-10T10:55:49.960768+0000","last_fullsized":"2024-09-10T10:55:49.960768+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:07.147500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.833598+0000","last_change":"2024-09-10T10:55:47.833598+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.833598+0000","last_undegraded":"2024-09-10T10:55:47.833598+0000","last_fullsized":"2024-09-10T10:55:47.833598+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961416+0000","last_change":"2024-09-10T10:55:47.855582+0000","last_active":"2024-09-10T10:55:49.961416+0000","last_peered":"2024-09-10T10:55:49.961416+0000","last_clean":"2024-09-10T10:55:49.961416+0000","last_became_active":"2024-09-10T10:55:47.855156+0000","last_became_peered":"2024-09-10T10:55:47.855156+0000","last_unstale":"2024-09-10T10:55:49.961416+0000","last_undegraded":"2024-09-10T10:55:49.961416+0000","last_fullsized":"2024-09-10T10:55:49.961416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:30.174489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961811+0000","last_change":"2024-09-10T10:55:48.876356+0000","last_active":"2024-09-10T10:55:49.961811+0000","last_peered":"2024-09-10T10:55:49.961811+0000","last_clean":"2024-09-10T10:55:49.961811+0000","last_became_active":"2024-09-10T10:55:48.875734+0000","last_became_peered":"2024-09-10T10:55:48.875734+0000","last_unstale":"2024-09-10T10:55:49.961811+0000","last_undegraded":"2024-09-10T10:55:49.961811+0000","last_fullsized":"2024-09-10T10:55:49.961811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:45.252064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851021+0000","last_change":"2024-09-10T10:55:47.846423+0000","last_active":"2024-09-10T10:55:47.851021+0000","last_peered":"2024-09-10T10:55:47.851021+0000","last_clean":"2024-09-10T10:55:47.851021+0000","last_became_active":"2024-09-10T10:55:47.842576+0000","last_became_peered":"2024-09-10T10:55:47.842576+0000","last_unstale":"2024-09-10T10:55:47.851021+0000","last_undegraded":"2024-09-10T10:55:47.851021+0000","last_fullsized":"2024-09-10T10:55:47.851021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.847568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946997+0000","last_change":"2024-09-10T10:55:48.864892+0000","last_active":"2024-09-10T10:55:49.946997+0000","last_peered":"2024-09-10T10:55:49.946997+0000","last_clean":"2024-09-10T10:55:49.946997+0000","last_became_active":"2024-09-10T10:55:48.864545+0000","last_became_peered":"2024-09-10T10:55:48.864545+0000","last_unstale":"2024-09-10T10:55:49.946997+0000","last_undegraded":"2024-09-10T10:55:49.946997+0000","last_fullsized":"2024-09-10T10:55:49.946997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:45.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947156+0000","last_change":"2024-09-10T10:55:47.848829+0000","last_active":"2024-09-10T10:55:49.947156+0000","last_peered":"2024-09-10T10:55:49.947156+0000","last_clean":"2024-09-10T10:55:49.947156+0000","last_became_active":"2024-09-10T10:55:47.848545+0000","last_became_peered":"2024-09-10T10:55:47.848545+0000","last_unstale":"2024-09-10T10:55:49.947156+0000","last_undegraded":"2024-09-10T10:55:49.947156+0000","last_fullsized":"2024-09-10T10:55:49.947156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:26.415458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862744+0000","last_change":"2024-09-10T10:55:48.853317+0000","last_active":"2024-09-10T10:55:49.862744+0000","last_peered":"2024-09-10T10:55:49.862744+0000","last_clean":"2024-09-10T10:55:49.862744+0000","last_became_active":"2024-09-10T10:55:48.853020+0000","last_became_peered":"2024-09-10T10:55:48.853020+0000","last_unstale":"2024-09-10T10:55:49.862744+0000","last_undegraded":"2024-09-10T10:55:49.862744+0000","last_fullsized":"2024-09-10T10:55:49.862744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:20.610135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961781+0000","last_change":"2024-09-10T10:55:47.847504+0000","last_active":"2024-09-10T10:55:49.961781+0000","last_peered":"2024-09-10T10:55:49.961781+0000","last_clean":"2024-09-10T10:55:49.961781+0000","last_became_active":"2024-09-10T10:55:47.847070+0000","last_became_peered":"2024-09-10T10:55:47.847070+0000","last_unstale":"2024-09-10T10:55:49.961781+0000","last_undegraded":"2024-09-10T10:55:49.961781+0000","last_fullsized":"2024-09-10T10:55:49.961781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:04.988364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865922+0000","last_change":"2024-09-10T10:55:48.867784+0000","last_active":"2024-09-10T10:55:49.865922+0000","last_peered":"2024-09-10T10:55:49.865922+0000","last_clean":"2024-09-10T10:55:49.865922+0000","last_became_active":"2024-09-10T10:55:48.867508+0000","last_became_peered":"2024-09-10T10:55:48.867508+0000","last_unstale":"2024-09-10T10:55:49.865922+0000","last_undegraded":"2024-09-10T10:55:49.865922+0000","last_fullsized":"2024-09-10T10:55:49.865922+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:48.476999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862859+0000","last_change":"2024-09-10T10:55:47.845137+0000","last_active":"2024-09-10T10:55:49.862859+0000","last_peered":"2024-09-10T10:55:49.862859+0000","last_clean":"2024-09-10T10:55:49.862859+0000","last_became_active":"2024-09-10T10:55:47.844834+0000","last_became_peered":"2024-09-10T10:55:47.844834+0000","last_unstale":"2024-09-10T10:55:49.862859+0000","last_undegraded":"2024-09-10T10:55:49.862859+0000","last_fullsized":"2024-09-10T10:55:49.862859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:32.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865918+0000","last_change":"2024-09-10T10:55:48.864028+0000","last_active":"2024-09-10T10:55:49.865918+0000","last_peered":"2024-09-10T10:55:49.865918+0000","last_clean":"2024-09-10T10:55:49.865918+0000","last_became_active":"2024-09-10T10:55:48.863229+0000","last_became_peered":"2024-09-10T10:55:48.863229+0000","last_unstale":"2024-09-10T10:55:49.865918+0000","last_undegraded":"2024-09-10T10:55:49.865918+0000","last_fullsized":"2024-09-10T10:55:49.865918+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:18.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960308+0000","last_change":"2024-09-10T10:55:47.843083+0000","last_active":"2024-09-10T10:55:49.960308+0000","last_peered":"2024-09-10T10:55:49.960308+0000","last_clean":"2024-09-10T10:55:49.960308+0000","last_became_active":"2024-09-10T10:55:47.839631+0000","last_became_peered":"2024-09-10T10:55:47.839631+0000","last_unstale":"2024-09-10T10:55:49.960308+0000","last_undegraded":"2024-09-10T10:55:49.960308+0000","last_fullsized":"2024-09-10T10:55:49.960308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:18.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.863052+0000","last_change":"2024-09-10T10:55:48.864786+0000","last_active":"2024-09-10T10:55:49.863052+0000","last_peered":"2024-09-10T10:55:49.863052+0000","last_clean":"2024-09-10T10:55:49.863052+0000","last_became_active":"2024-09-10T10:55:48.864460+0000","last_became_peered":"2024-09-10T10:55:48.864460+0000","last_unstale":"2024-09-10T10:55:49.863052+0000","last_undegraded":"2024-09-10T10:55:49.863052+0000","last_fullsized":"2024-09-10T10:55:49.863052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:05.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960378+0000","last_change":"2024-09-10T10:55:47.846337+0000","last_active":"2024-09-10T10:55:49.960378+0000","last_peered":"2024-09-10T10:55:49.960378+0000","last_clean":"2024-09-10T10:55:49.960378+0000","last_became_active":"2024-09-10T10:55:47.845507+0000","last_became_peered":"2024-09-10T10:55:47.845507+0000","last_unstale":"2024-09-10T10:55:49.960378+0000","last_undegraded":"2024-09-10T10:55:49.960378+0000","last_fullsized":"2024-09-10T10:55:49.960378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:11.338379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865877+0000","last_change":"2024-09-10T10:55:48.864808+0000","last_active":"2024-09-10T10:55:49.865877+0000","last_peered":"2024-09-10T10:55:49.865877+0000","last_clean":"2024-09-10T10:55:49.865877+0000","last_became_active":"2024-09-10T10:55:48.862818+0000","last_became_peered":"2024-09-10T10:55:48.862818+0000","last_unstale":"2024-09-10T10:55:49.865877+0000","last_undegraded":"2024-09-10T10:55:49.865877+0000","last_fullsized":"2024-09-10T10:55:49.865877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:44.808359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:47.853115+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:47.852824+0000","last_became_peered":"2024-09-10T10:55:47.852824+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:57.389591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864710+0000","last_change":"2024-09-10T10:55:48.865231+0000","last_active":"2024-09-10T10:55:49.864710+0000","last_peered":"2024-09-10T10:55:49.864710+0000","last_clean":"2024-09-10T10:55:49.864710+0000","last_became_active":"2024-09-10T10:55:48.862955+0000","last_became_peered":"2024-09-10T10:55:48.862955+0000","last_unstale":"2024-09-10T10:55:49.864710+0000","last_undegraded":"2024-09-10T10:55:49.864710+0000","last_fullsized":"2024-09-10T10:55:49.864710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.358498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872584+0000","last_change":"2024-09-10T10:55:47.843342+0000","last_active":"2024-09-10T10:55:49.872584+0000","last_peered":"2024-09-10T10:55:49.872584+0000","last_clean":"2024-09-10T10:55:49.872584+0000","last_became_active":"2024-09-10T10:55:47.842788+0000","last_became_peered":"2024-09-10T10:55:47.842788+0000","last_unstale":"2024-09-10T10:55:49.872584+0000","last_undegraded":"2024-09-10T10:55:49.872584+0000","last_fullsized":"2024-09-10T10:55:49.872584+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:00.612235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865943+0000","last_change":"2024-09-10T10:55:48.869535+0000","last_active":"2024-09-10T10:55:49.865943+0000","last_peered":"2024-09-10T10:55:49.865943+0000","last_clean":"2024-09-10T10:55:49.865943+0000","last_became_active":"2024-09-10T10:55:48.869284+0000","last_became_peered":"2024-09-10T10:55:48.869284+0000","last_unstale":"2024-09-10T10:55:49.865943+0000","last_undegraded":"2024-09-10T10:55:49.865943+0000","last_fullsized":"2024-09-10T10:55:49.865943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:44.081502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961991+0000","last_change":"2024-09-10T10:55:47.851219+0000","last_active":"2024-09-10T10:55:49.961991+0000","last_peered":"2024-09-10T10:55:49.961991+0000","last_clean":"2024-09-10T10:55:49.961991+0000","last_became_active":"2024-09-10T10:55:47.849193+0000","last_became_peered":"2024-09-10T10:55:47.849193+0000","last_unstale":"2024-09-10T10:55:49.961991+0000","last_undegraded":"2024-09-10T10:55:49.961991+0000","last_fullsized":"2024-09-10T10:55:49.961991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:50.483870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961814+0000","last_change":"2024-09-10T10:55:48.868003+0000","last_active":"2024-09-10T10:55:49.961814+0000","last_peered":"2024-09-10T10:55:49.961814+0000","last_clean":"2024-09-10T10:55:49.961814+0000","last_became_active":"2024-09-10T10:55:48.867694+0000","last_became_peered":"2024-09-10T10:55:48.867694+0000","last_unstale":"2024-09-10T10:55:49.961814+0000","last_undegraded":"2024-09-10T10:55:49.961814+0000","last_fullsized":"2024-09-10T10:55:49.961814+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:37.123756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872366+0000","last_change":"2024-09-10T10:55:47.847591+0000","last_active":"2024-09-10T10:55:49.872366+0000","last_peered":"2024-09-10T10:55:49.872366+0000","last_clean":"2024-09-10T10:55:49.872366+0000","last_became_active":"2024-09-10T10:55:47.847299+0000","last_became_peered":"2024-09-10T10:55:47.847299+0000","last_unstale":"2024-09-10T10:55:49.872366+0000","last_undegraded":"2024-09-10T10:55:49.872366+0000","last_fullsized":"2024-09-10T10:55:49.872366+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.943592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961027+0000","last_change":"2024-09-10T10:55:48.875294+0000","last_active":"2024-09-10T10:55:49.961027+0000","last_peered":"2024-09-10T10:55:49.961027+0000","last_clean":"2024-09-10T10:55:49.961027+0000","last_became_active":"2024-09-10T10:55:48.873851+0000","last_became_peered":"2024-09-10T10:55:48.873851+0000","last_unstale":"2024-09-10T10:55:49.961027+0000","last_undegraded":"2024-09-10T10:55:49.961027+0000","last_fullsized":"2024-09-10T10:55:49.961027+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:07.268035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862812+0000","last_change":"2024-09-10T10:55:47.850868+0000","last_active":"2024-09-10T10:55:49.862812+0000","last_peered":"2024-09-10T10:55:49.862812+0000","last_clean":"2024-09-10T10:55:49.862812+0000","last_became_active":"2024-09-10T10:55:47.850457+0000","last_became_peered":"2024-09-10T10:55:47.850457+0000","last_unstale":"2024-09-10T10:55:49.862812+0000","last_undegraded":"2024-09-10T10:55:49.862812+0000","last_fullsized":"2024-09-10T10:55:49.862812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:45.564987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947047+0000","last_change":"2024-09-10T10:55:48.861530+0000","last_active":"2024-09-10T10:55:49.947047+0000","last_peered":"2024-09-10T10:55:49.947047+0000","last_clean":"2024-09-10T10:55:49.947047+0000","last_became_active":"2024-09-10T10:55:48.860648+0000","last_became_peered":"2024-09-10T10:55:48.860648+0000","last_unstale":"2024-09-10T10:55:49.947047+0000","last_undegraded":"2024-09-10T10:55:49.947047+0000","last_fullsized":"2024-09-10T10:55:49.947047+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:36.041255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865960+0000","last_change":"2024-09-10T10:55:47.843477+0000","last_active":"2024-09-10T10:55:49.865960+0000","last_peered":"2024-09-10T10:55:49.865960+0000","last_clean":"2024-09-10T10:55:49.865960+0000","last_became_active":"2024-09-10T10:55:47.841371+0000","last_became_peered":"2024-09-10T10:55:47.841371+0000","last_unstale":"2024-09-10T10:55:49.865960+0000","last_undegraded":"2024-09-10T10:55:49.865960+0000","last_fullsized":"2024-09-10T10:55:49.865960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:45.636397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947145+0000","last_change":"2024-09-10T10:55:48.865788+0000","last_active":"2024-09-10T10:55:49.947145+0000","last_peered":"2024-09-10T10:55:49.947145+0000","last_clean":"2024-09-10T10:55:49.947145+0000","last_became_active":"2024-09-10T10:55:48.865486+0000","last_became_peered":"2024-09-10T10:55:48.865486+0000","last_unstale":"2024-09-10T10:55:49.947145+0000","last_undegraded":"2024-09-10T10:55:49.947145+0000","last_fullsized":"2024-09-10T10:55:49.947145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:56.003393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961925+0000","last_change":"2024-09-10T10:55:47.845313+0000","last_active":"2024-09-10T10:55:49.961925+0000","last_peered":"2024-09-10T10:55:49.961925+0000","last_clean":"2024-09-10T10:55:49.961925+0000","last_became_active":"2024-09-10T10:55:47.844999+0000","last_became_peered":"2024-09-10T10:55:47.844999+0000","last_unstale":"2024-09-10T10:55:49.961925+0000","last_undegraded":"2024-09-10T10:55:49.961925+0000","last_fullsized":"2024-09-10T10:55:49.961925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:07.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961293+0000","last_change":"2024-09-10T10:55:48.877412+0000","last_active":"2024-09-10T10:55:49.961293+0000","last_peered":"2024-09-10T10:55:49.961293+0000","last_clean":"2024-09-10T10:55:49.961293+0000","last_became_active":"2024-09-10T10:55:48.876383+0000","last_became_peered":"2024-09-10T10:55:48.876383+0000","last_unstale":"2024-09-10T10:55:49.961293+0000","last_undegraded":"2024-09-10T10:55:49.961293+0000","last_fullsized":"2024-09-10T10:55:49.961293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:41.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865237+0000","last_change":"2024-09-10T10:55:47.834691+0000","last_active":"2024-09-10T10:55:49.865237+0000","last_peered":"2024-09-10T10:55:49.865237+0000","last_clean":"2024-09-10T10:55:49.865237+0000","last_became_active":"2024-09-10T10:55:47.834400+0000","last_became_peered":"2024-09-10T10:55:47.834400+0000","last_unstale":"2024-09-10T10:55:49.865237+0000","last_undegraded":"2024-09-10T10:55:49.865237+0000","last_fullsized":"2024-09-10T10:55:49.865237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:52.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835555+0000","last_change":"2024-09-10T10:55:47.835555+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835555+0000","last_undegraded":"2024-09-10T10:55:47.835555+0000","last_fullsized":"2024-09-10T10:55:47.835555+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862623+0000","last_change":"2024-09-10T10:55:47.843176+0000","last_active":"2024-09-10T10:55:49.862623+0000","last_peered":"2024-09-10T10:55:49.862623+0000","last_clean":"2024-09-10T10:55:49.862623+0000","last_became_active":"2024-09-10T10:55:47.842914+0000","last_became_peered":"2024-09-10T10:55:47.842914+0000","last_unstale":"2024-09-10T10:55:49.862623+0000","last_undegraded":"2024-09-10T10:55:49.862623+0000","last_fullsized":"2024-09-10T10:55:49.862623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:46.876235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961424+0000","last_change":"2024-09-10T10:55:48.874049+0000","last_active":"2024-09-10T10:55:49.961424+0000","last_peered":"2024-09-10T10:55:49.961424+0000","last_clean":"2024-09-10T10:55:49.961424+0000","last_became_active":"2024-09-10T10:55:48.873744+0000","last_became_peered":"2024-09-10T10:55:48.873744+0000","last_unstale":"2024-09-10T10:55:49.961424+0000","last_undegraded":"2024-09-10T10:55:49.961424+0000","last_fullsized":"2024-09-10T10:55:49.961424+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:25.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872621+0000","last_change":"2024-09-10T10:55:47.845166+0000","last_active":"2024-09-10T10:55:49.872621+0000","last_peered":"2024-09-10T10:55:49.872621+0000","last_clean":"2024-09-10T10:55:49.872621+0000","last_became_active":"2024-09-10T10:55:47.844675+0000","last_became_peered":"2024-09-10T10:55:47.844675+0000","last_unstale":"2024-09-10T10:55:49.872621+0000","last_undegraded":"2024-09-10T10:55:49.872621+0000","last_fullsized":"2024-09-10T10:55:49.872621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:26.167518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960321+0000","last_change":"2024-09-10T10:55:48.872087+0000","last_active":"2024-09-10T10:55:49.960321+0000","last_peered":"2024-09-10T10:55:49.960321+0000","last_clean":"2024-09-10T10:55:49.960321+0000","last_became_active":"2024-09-10T10:55:48.871832+0000","last_became_peered":"2024-09-10T10:55:48.871832+0000","last_unstale":"2024-09-10T10:55:49.960321+0000","last_undegraded":"2024-09-10T10:55:49.960321+0000","last_fullsized":"2024-09-10T10:55:49.960321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:58.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960717+0000","last_change":"2024-09-10T10:55:47.850066+0000","last_active":"2024-09-10T10:55:49.960717+0000","last_peered":"2024-09-10T10:55:49.960717+0000","last_clean":"2024-09-10T10:55:49.960717+0000","last_became_active":"2024-09-10T10:55:47.849775+0000","last_became_peered":"2024-09-10T10:55:47.849775+0000","last_unstale":"2024-09-10T10:55:49.960717+0000","last_undegraded":"2024-09-10T10:55:49.960717+0000","last_fullsized":"2024-09-10T10:55:49.960717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:29.865577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961193+0000","last_change":"2024-09-10T10:55:48.860009+0000","last_active":"2024-09-10T10:55:49.961193+0000","last_peered":"2024-09-10T10:55:49.961193+0000","last_clean":"2024-09-10T10:55:49.961193+0000","last_became_active":"2024-09-10T10:55:48.859725+0000","last_became_peered":"2024-09-10T10:55:48.859725+0000","last_unstale":"2024-09-10T10:55:49.961193+0000","last_undegraded":"2024-09-10T10:55:49.961193+0000","last_fullsized":"2024-09-10T10:55:49.961193+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:06.818156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872362+0000","last_change":"2024-09-10T10:55:47.846489+0000","last_active":"2024-09-10T10:55:49.872362+0000","last_peered":"2024-09-10T10:55:49.872362+0000","last_clean":"2024-09-10T10:55:49.872362+0000","last_became_active":"2024-09-10T10:55:47.846067+0000","last_became_peered":"2024-09-10T10:55:47.846067+0000","last_unstale":"2024-09-10T10:55:49.872362+0000","last_undegraded":"2024-09-10T10:55:49.872362+0000","last_fullsized":"2024-09-10T10:55:49.872362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:28.004930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865650+0000","last_change":"2024-09-10T10:55:48.857694+0000","last_active":"2024-09-10T10:55:49.865650+0000","last_peered":"2024-09-10T10:55:49.865650+0000","last_clean":"2024-09-10T10:55:49.865650+0000","last_became_active":"2024-09-10T10:55:48.857007+0000","last_became_peered":"2024-09-10T10:55:48.857007+0000","last_unstale":"2024-09-10T10:55:49.865650+0000","last_undegraded":"2024-09-10T10:55:49.865650+0000","last_fullsized":"2024-09-10T10:55:49.865650+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:15.966260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.850937+0000","last_change":"2024-09-10T10:55:47.846071+0000","last_active":"2024-09-10T10:55:47.850937+0000","last_peered":"2024-09-10T10:55:47.850937+0000","last_clean":"2024-09-10T10:55:47.850937+0000","last_became_active":"2024-09-10T10:55:47.845052+0000","last_became_peered":"2024-09-10T10:55:47.845052+0000","last_unstale":"2024-09-10T10:55:47.850937+0000","last_undegraded":"2024-09-10T10:55:47.850937+0000","last_fullsized":"2024-09-10T10:55:47.850937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:32.064815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:48.868962+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:48.868602+0000","last_became_peered":"2024-09-10T10:55:48.868602+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:37.194259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.853061+0000","last_change":"2024-09-10T10:55:47.846363+0000","last_active":"2024-09-10T10:55:47.853061+0000","last_peered":"2024-09-10T10:55:47.853061+0000","last_clean":"2024-09-10T10:55:47.853061+0000","last_became_active":"2024-09-10T10:55:47.846001+0000","last_became_peered":"2024-09-10T10:55:47.846001+0000","last_unstale":"2024-09-10T10:55:47.853061+0000","last_undegraded":"2024-09-10T10:55:47.853061+0000","last_fullsized":"2024-09-10T10:55:47.853061+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:59.581099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961231+0000","last_change":"2024-09-10T10:55:48.861960+0000","last_active":"2024-09-10T10:55:49.961231+0000","last_peered":"2024-09-10T10:55:49.961231+0000","last_clean":"2024-09-10T10:55:49.961231+0000","last_became_active":"2024-09-10T10:55:48.861620+0000","last_became_peered":"2024-09-10T10:55:48.861620+0000","last_unstale":"2024-09-10T10:55:49.961231+0000","last_undegraded":"2024-09-10T10:55:49.961231+0000","last_fullsized":"2024-09-10T10:55:49.961231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:27.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872025+0000","last_change":"2024-09-10T10:55:47.837440+0000","last_active":"2024-09-10T10:55:49.872025+0000","last_peered":"2024-09-10T10:55:49.872025+0000","last_clean":"2024-09-10T10:55:49.872025+0000","last_became_active":"2024-09-10T10:55:47.836741+0000","last_became_peered":"2024-09-10T10:55:47.836741+0000","last_unstale":"2024-09-10T10:55:49.872025+0000","last_undegraded":"2024-09-10T10:55:49.872025+0000","last_fullsized":"2024-09-10T10:55:49.872025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:58.662475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865678+0000","last_change":"2024-09-10T10:55:48.867356+0000","last_active":"2024-09-10T10:55:49.865678+0000","last_peered":"2024-09-10T10:55:49.865678+0000","last_clean":"2024-09-10T10:55:49.865678+0000","last_became_active":"2024-09-10T10:55:48.866816+0000","last_became_peered":"2024-09-10T10:55:48.866816+0000","last_unstale":"2024-09-10T10:55:49.865678+0000","last_undegraded":"2024-09-10T10:55:49.865678+0000","last_fullsized":"2024-09-10T10:55:49.865678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:17.379036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961500+0000","last_change":"2024-09-10T10:55:47.845337+0000","last_active":"2024-09-10T10:55:49.961500+0000","last_peered":"2024-09-10T10:55:49.961500+0000","last_clean":"2024-09-10T10:55:49.961500+0000","last_became_active":"2024-09-10T10:55:47.845077+0000","last_became_peered":"2024-09-10T10:55:47.845077+0000","last_unstale":"2024-09-10T10:55:49.961500+0000","last_undegraded":"2024-09-10T10:55:49.961500+0000","last_fullsized":"2024-09-10T10:55:49.961500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:14.274310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960779+0000","last_change":"2024-09-10T10:55:48.864373+0000","last_active":"2024-09-10T10:55:49.960779+0000","last_peered":"2024-09-10T10:55:49.960779+0000","last_clean":"2024-09-10T10:55:49.960779+0000","last_became_active":"2024-09-10T10:55:48.864041+0000","last_became_peered":"2024-09-10T10:55:48.864041+0000","last_unstale":"2024-09-10T10:55:49.960779+0000","last_undegraded":"2024-09-10T10:55:49.960779+0000","last_fullsized":"2024-09-10T10:55:49.960779+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:39.301780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.852322+0000","last_change":"2024-09-10T10:55:47.850713+0000","last_active":"2024-09-10T10:55:47.852322+0000","last_peered":"2024-09-10T10:55:47.852322+0000","last_clean":"2024-09-10T10:55:47.852322+0000","last_became_active":"2024-09-10T10:55:47.849850+0000","last_became_peered":"2024-09-10T10:55:47.849850+0000","last_unstale":"2024-09-10T10:55:47.852322+0000","last_undegraded":"2024-09-10T10:55:47.852322+0000","last_fullsized":"2024-09-10T10:55:47.852322+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:40.025775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862722+0000","last_change":"2024-09-10T10:55:48.859546+0000","last_active":"2024-09-10T10:55:49.862722+0000","last_peered":"2024-09-10T10:55:49.862722+0000","last_clean":"2024-09-10T10:55:49.862722+0000","last_became_active":"2024-09-10T10:55:48.859164+0000","last_became_peered":"2024-09-10T10:55:48.859164+0000","last_unstale":"2024-09-10T10:55:49.862722+0000","last_undegraded":"2024-09-10T10:55:49.862722+0000","last_fullsized":"2024-09-10T10:55:49.862722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:29.838709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854948+0000","last_change":"2024-09-10T10:55:47.853293+0000","last_active":"2024-09-10T10:55:47.854948+0000","last_peered":"2024-09-10T10:55:47.854948+0000","last_clean":"2024-09-10T10:55:47.854948+0000","last_became_active":"2024-09-10T10:55:47.852967+0000","last_became_peered":"2024-09-10T10:55:47.852967+0000","last_unstale":"2024-09-10T10:55:47.854948+0000","last_undegraded":"2024-09-10T10:55:47.854948+0000","last_fullsized":"2024-09-10T10:55:47.854948+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:50.182209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961880+0000","last_change":"2024-09-10T10:55:48.874714+0000","last_active":"2024-09-10T10:55:49.961880+0000","last_peered":"2024-09-10T10:55:49.961880+0000","last_clean":"2024-09-10T10:55:49.961880+0000","last_became_active":"2024-09-10T10:55:48.874256+0000","last_became_peered":"2024-09-10T10:55:48.874256+0000","last_unstale":"2024-09-10T10:55:49.961880+0000","last_undegraded":"2024-09-10T10:55:49.961880+0000","last_fullsized":"2024-09-10T10:55:49.961880+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:04.561493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862937+0000","last_change":"2024-09-10T10:55:47.852303+0000","last_active":"2024-09-10T10:55:49.862937+0000","last_peered":"2024-09-10T10:55:49.862937+0000","last_clean":"2024-09-10T10:55:49.862937+0000","last_became_active":"2024-09-10T10:55:47.851277+0000","last_became_peered":"2024-09-10T10:55:47.851277+0000","last_unstale":"2024-09-10T10:55:49.862937+0000","last_undegraded":"2024-09-10T10:55:49.862937+0000","last_fullsized":"2024-09-10T10:55:49.862937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:43.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960557+0000","last_change":"2024-09-10T10:55:48.876162+0000","last_active":"2024-09-10T10:55:49.960557+0000","last_peered":"2024-09-10T10:55:49.960557+0000","last_clean":"2024-09-10T10:55:49.960557+0000","last_became_active":"2024-09-10T10:55:48.875834+0000","last_became_peered":"2024-09-10T10:55:48.875834+0000","last_unstale":"2024-09-10T10:55:49.960557+0000","last_undegraded":"2024-09-10T10:55:49.960557+0000","last_fullsized":"2024-09-10T10:55:49.960557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:12.003746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.866001+0000","last_change":"2024-09-10T10:55:47.844106+0000","last_active":"2024-09-10T10:55:49.866001+0000","last_peered":"2024-09-10T10:55:49.866001+0000","last_clean":"2024-09-10T10:55:49.866001+0000","last_became_active":"2024-09-10T10:55:47.843763+0000","last_became_peered":"2024-09-10T10:55:47.843763+0000","last_unstale":"2024-09-10T10:55:49.866001+0000","last_undegraded":"2024-09-10T10:55:49.866001+0000","last_fullsized":"2024-09-10T10:55:49.866001+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:19.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864866+0000","last_change":"2024-09-10T10:55:48.857933+0000","last_active":"2024-09-10T10:55:49.864866+0000","last_peered":"2024-09-10T10:55:49.864866+0000","last_clean":"2024-09-10T10:55:49.864866+0000","last_became_active":"2024-09-10T10:55:48.857414+0000","last_became_peered":"2024-09-10T10:55:48.857414+0000","last_unstale":"2024-09-10T10:55:49.864866+0000","last_undegraded":"2024-09-10T10:55:49.864866+0000","last_fullsized":"2024-09-10T10:55:49.864866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:52.855868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862873+0000","last_change":"2024-09-10T10:55:47.846380+0000","last_active":"2024-09-10T10:55:49.862873+0000","last_peered":"2024-09-10T10:55:49.862873+0000","last_clean":"2024-09-10T10:55:49.862873+0000","last_became_active":"2024-09-10T10:55:47.846123+0000","last_became_peered":"2024-09-10T10:55:47.846123+0000","last_unstale":"2024-09-10T10:55:49.862873+0000","last_undegraded":"2024-09-10T10:55:49.862873+0000","last_fullsized":"2024-09-10T10:55:49.862873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.465904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865669+0000","last_change":"2024-09-10T10:55:48.859593+0000","last_active":"2024-09-10T10:55:49.865669+0000","last_peered":"2024-09-10T10:55:49.865669+0000","last_clean":"2024-09-10T10:55:49.865669+0000","last_became_active":"2024-09-10T10:55:48.859045+0000","last_became_peered":"2024-09-10T10:55:48.859045+0000","last_unstale":"2024-09-10T10:55:49.865669+0000","last_undegraded":"2024-09-10T10:55:49.865669+0000","last_fullsized":"2024-09-10T10:55:49.865669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:39.427524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854980+0000","last_change":"2024-09-10T10:55:47.853193+0000","last_active":"2024-09-10T10:55:47.854980+0000","last_peered":"2024-09-10T10:55:47.854980+0000","last_clean":"2024-09-10T10:55:47.854980+0000","last_became_active":"2024-09-10T10:55:47.852861+0000","last_became_peered":"2024-09-10T10:55:47.852861+0000","last_unstale":"2024-09-10T10:55:47.854980+0000","last_undegraded":"2024-09-10T10:55:47.854980+0000","last_fullsized":"2024-09-10T10:55:47.854980+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:43.046220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872528+0000","last_change":"2024-09-10T10:55:48.870415+0000","last_active":"2024-09-10T10:55:49.872528+0000","last_peered":"2024-09-10T10:55:49.872528+0000","last_clean":"2024-09-10T10:55:49.872528+0000","last_became_active":"2024-09-10T10:55:48.869966+0000","last_became_peered":"2024-09-10T10:55:48.869966+0000","last_unstale":"2024-09-10T10:55:49.872528+0000","last_undegraded":"2024-09-10T10:55:49.872528+0000","last_fullsized":"2024-09-10T10:55:49.872528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960416+0000","last_change":"2024-09-10T10:55:47.850818+0000","last_active":"2024-09-10T10:55:49.960416+0000","last_peered":"2024-09-10T10:55:49.960416+0000","last_clean":"2024-09-10T10:55:49.960416+0000","last_became_active":"2024-09-10T10:55:47.850541+0000","last_became_peered":"2024-09-10T10:55:47.850541+0000","last_unstale":"2024-09-10T10:55:49.960416+0000","last_undegraded":"2024-09-10T10:55:49.960416+0000","last_fullsized":"2024-09-10T10:55:49.960416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:54.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947271+0000","last_change":"2024-09-10T10:55:48.862694+0000","last_active":"2024-09-10T10:55:49.947271+0000","last_peered":"2024-09-10T10:55:49.947271+0000","last_clean":"2024-09-10T10:55:49.947271+0000","last_became_active":"2024-09-10T10:55:48.862367+0000","last_became_peered":"2024-09-10T10:55:48.862367+0000","last_unstale":"2024-09-10T10:55:49.947271+0000","last_undegraded":"2024-09-10T10:55:49.947271+0000","last_fullsized":"2024-09-10T10:55:49.947271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851952+0000","last_change":"2024-09-10T10:55:47.846160+0000","last_active":"2024-09-10T10:55:47.851952+0000","last_peered":"2024-09-10T10:55:47.851952+0000","last_clean":"2024-09-10T10:55:47.851952+0000","last_became_active":"2024-09-10T10:55:47.845758+0000","last_became_peered":"2024-09-10T10:55:47.845758+0000","last_unstale":"2024-09-10T10:55:47.851952+0000","last_undegraded":"2024-09-10T10:55:47.851952+0000","last_fullsized":"2024-09-10T10:55:47.851952+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:13.218467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871975+0000","last_change":"2024-09-10T10:55:48.867075+0000","last_active":"2024-09-10T10:55:49.871975+0000","last_peered":"2024-09-10T10:55:49.871975+0000","last_clean":"2024-09-10T10:55:49.871975+0000","last_became_active":"2024-09-10T10:55:48.859669+0000","last_became_peered":"2024-09-10T10:55:48.859669+0000","last_unstale":"2024-09-10T10:55:49.871975+0000","last_undegraded":"2024-09-10T10:55:49.871975+0000","last_fullsized":"2024-09-10T10:55:49.871975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:05.994404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864565+0000","last_change":"2024-09-10T10:55:47.847392+0000","last_active":"2024-09-10T10:55:49.864565+0000","last_peered":"2024-09-10T10:55:49.864565+0000","last_clean":"2024-09-10T10:55:49.864565+0000","last_became_active":"2024-09-10T10:55:47.847116+0000","last_became_peered":"2024-09-10T10:55:47.847116+0000","last_unstale":"2024-09-10T10:55:49.864565+0000","last_undegraded":"2024-09-10T10:55:49.864565+0000","last_fullsized":"2024-09-10T10:55:49.864565+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:38.621928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":416,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960208+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:55:49.960208+0000","last_peered":"2024-09-10T10:55:49.960208+0000","last_clean":"2024-09-10T10:55:49.960208+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:55:49.960208+0000","last_undegraded":"2024-09-10T10:55:49.960208+0000","last_fullsized":"2024-09-10T10:55:49.960208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1941504,"data_stored":978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28936,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342904,"statfs":{"total":96636764160,"available":96607133696,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1937408,"data_stored":977391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1941504,"data_stored":977469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41972,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329868,"statfs":{"total":96636764160,"available":96593784832,"internally_reserved":0,"allocated":2523136,"data_stored":1561125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1937408,"data_stored":977939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":2502656,"data_stored":1553866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:55:53.066 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:55:53.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.258+0000 7fc4f1e00640 1 Processor -- start 2024-09-10T10:55:53.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.258+0000 7fc4f1e00640 1 -- start start 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4f1e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4f1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4ec072700 0x7fc4ec15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4f1e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec146140 con 0x7fc4ec072320 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4f1e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ec0cd740 con 0x7fc4ec072700 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4f1e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec0cda20 con 0x7fc4ec072de0 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eb400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4ec072700 0x7fc4ec15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eaa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc4ec072320 0x7fc4ec0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47274/0 (socket says 172.21.15.28:47274) 2024-09-10T10:55:53.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eaa00640 1 -- 172.21.15.28:0/2814580445 learned_addr learned my addr 172.21.15.28:0/2814580445 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4e3e00640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eb400640 1 -- 172.21.15.28:0/2814580445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 msgr2=0x7fc4ec15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eb400640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec15d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eb400640 1 -- 172.21.15.28:0/2814580445 >> v1:172.21.15.28:6789/0 conn(0x7fc4ec072320 legacy=0x7fc4ec0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4eb400640 1 -- 172.21.15.28:0/2814580445 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0cdd00 con 0x7fc4ec072700 2024-09-10T10:55:53.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.260+0000 7fc4e3e00640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:53.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.261+0000 7fc4eb400640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072700 0x7fc4ec15d720 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009d90 tx=0x7fc4e402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ff700f3431bbe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4ea000640 1 -- 172.21.15.28:0/2814580445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4031600 con 0x7fc4ec072700 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4ea000640 1 -- 172.21.15.28:0/2814580445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e40317a0 con 0x7fc4ec072700 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4ea000640 1 -- 172.21.15.28:0/2814580445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4031b70 con 0x7fc4ec072700 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 -- 172.21.15.28:0/2814580445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072700 msgr2=0x7fc4ec15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072700 0x7fc4ec15d720 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009d90 tx=0x7fc4e402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 -- 172.21.15.28:0/2814580445 shutdown_connections 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072700 0x7fc4ec15d720 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/2814580445 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec15d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 -- 172.21.15.28:0/2814580445 >> 172.21.15.28:0/2814580445 conn(0x7fc4ec067d70 msgr2=0x7fc4ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 -- 172.21.15.28:0/2814580445 shutdown_connections 2024-09-10T10:55:53.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.262+0000 7fc4f1e00640 1 -- 172.21.15.28:0/2814580445 wait complete. 2024-09-10T10:55:53.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.264+0000 7fc4f1e00640 1 Processor -- start 2024-09-10T10:55:53.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.264+0000 7fc4f1e00640 1 -- start start 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec072700 0x7fc4ec1e7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec1e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec15dde0 con 0x7fc4ec072320 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec15dc60 con 0x7fc4ec072700 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4f1e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec15df60 con 0x7fc4ec072de0 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4eaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4eaa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33310/0 (socket says 172.21.15.28:33310) 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.265+0000 7fc4eaa00640 1 -- 172.21.15.28:0/1024036939 learned_addr learned my addr 172.21.15.28:0/1024036939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 -- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 msgr2=0x7fc4ec1e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec1e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 -- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec072700 msgr2=0x7fc4ec1e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec072700 0x7fc4ec1e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 -- 172.21.15.28:0/1024036939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec0c7d90 con 0x7fc4ec072320 2024-09-10T10:55:53.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4eaa00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c400 tx=0x7fc4dc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc017070 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc005ce0 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0072f0 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec0c8070 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.266+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec0c8630 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.267+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec15e340 con 0x7fc4ec072320 2024-09-10T10:55:53.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.268+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4dc01a070 con 0x7fc4ec072320 2024-09-10T10:55:53.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.269+0000 7fc4e3400640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 0x7fc4c806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.269+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7fc4dc08a490 con 0x7fc4ec072320 2024-09-10T10:55:53.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.270+0000 7fc4e3e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 0x7fc4c806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.270+0000 7fc4e3e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 0x7fc4c806e2f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4006fd0 tx=0x7fc4d4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.273+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc4dc058c10 con 0x7fc4ec072320 2024-09-10T10:55:53.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.452+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4ec0c6300 con 0x7fc4c806be30 2024-09-10T10:55:53.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.459+0000 7fc4e3400640 1 -- 172.21.15.28:0/1024036939 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fc4ec0c6300 con 0x7fc4c806be30 2024-09-10T10:55:53.463 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 msgr2=0x7fc4c806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 0x7fc4c806e2f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4006fd0 tx=0x7fc4d4008040 comp rx=0 tx=0).stop 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 msgr2=0x7fc4ec15bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00c400 tx=0x7fc4dc00c8c0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 shutdown_connections 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc4c806be30 0x7fc4c806e2f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec072de0 0x7fc4ec1e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec072700 0x7fc4ec1e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 --2- 172.21.15.28:0/1024036939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4ec072320 0x7fc4ec15bde0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 >> 172.21.15.28:0/1024036939 conn(0x7fc4ec067d70 msgr2=0x7fc4ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 shutdown_connections 2024-09-10T10:55:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.463+0000 7fc4f1e00640 1 -- 172.21.15.28:0/1024036939 wait complete. 2024-09-10T10:55:53.479 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-10T10:55:52.674111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290788,"kb_used_data":16284,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683932,"statfs":{"total":773094113280,"available":772796346368,"internally_reserved":0,"allocated":16674816,"data_stored":8983737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.852121"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865805+0000","last_change":"2024-09-10T10:55:48.867834+0000","last_active":"2024-09-10T10:55:49.865805+0000","last_peered":"2024-09-10T10:55:49.865805+0000","last_clean":"2024-09-10T10:55:49.865805+0000","last_became_active":"2024-09-10T10:55:48.867563+0000","last_became_peered":"2024-09-10T10:55:48.867563+0000","last_unstale":"2024-09-10T10:55:49.865805+0000","last_undegraded":"2024-09-10T10:55:49.865805+0000","last_fullsized":"2024-09-10T10:55:49.865805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:00.429383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947062+0000","last_change":"2024-09-10T10:55:48.855439+0000","last_active":"2024-09-10T10:55:49.947062+0000","last_peered":"2024-09-10T10:55:49.947062+0000","last_clean":"2024-09-10T10:55:49.947062+0000","last_became_active":"2024-09-10T10:55:48.855159+0000","last_became_peered":"2024-09-10T10:55:48.855159+0000","last_unstale":"2024-09-10T10:55:49.947062+0000","last_undegraded":"2024-09-10T10:55:49.947062+0000","last_fullsized":"2024-09-10T10:55:49.947062+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:12.344418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.834144+0000","last_change":"2024-09-10T10:55:47.834144+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.834144+0000","last_undegraded":"2024-09-10T10:55:47.834144+0000","last_fullsized":"2024-09-10T10:55:47.834144+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864724+0000","last_change":"2024-09-10T10:55:48.853911+0000","last_active":"2024-09-10T10:55:49.864724+0000","last_peered":"2024-09-10T10:55:49.864724+0000","last_clean":"2024-09-10T10:55:49.864724+0000","last_became_active":"2024-09-10T10:55:48.853601+0000","last_became_peered":"2024-09-10T10:55:48.853601+0000","last_unstale":"2024-09-10T10:55:49.864724+0000","last_undegraded":"2024-09-10T10:55:49.864724+0000","last_fullsized":"2024-09-10T10:55:49.864724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:51.207539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865887+0000","last_change":"2024-09-10T10:55:48.860012+0000","last_active":"2024-09-10T10:55:49.865887+0000","last_peered":"2024-09-10T10:55:49.865887+0000","last_clean":"2024-09-10T10:55:49.865887+0000","last_became_active":"2024-09-10T10:55:48.859059+0000","last_became_peered":"2024-09-10T10:55:48.859059+0000","last_unstale":"2024-09-10T10:55:49.865887+0000","last_undegraded":"2024-09-10T10:55:49.865887+0000","last_fullsized":"2024-09-10T10:55:49.865887+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.465847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872011+0000","last_change":"2024-09-10T10:55:48.872626+0000","last_active":"2024-09-10T10:55:49.872011+0000","last_peered":"2024-09-10T10:55:49.872011+0000","last_clean":"2024-09-10T10:55:49.872011+0000","last_became_active":"2024-09-10T10:55:48.872334+0000","last_became_peered":"2024-09-10T10:55:48.872334+0000","last_unstale":"2024-09-10T10:55:49.872011+0000","last_undegraded":"2024-09-10T10:55:49.872011+0000","last_fullsized":"2024-09-10T10:55:49.872011+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:51.489910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871869+0000","last_change":"2024-09-10T10:55:48.856346+0000","last_active":"2024-09-10T10:55:49.871869+0000","last_peered":"2024-09-10T10:55:49.871869+0000","last_clean":"2024-09-10T10:55:49.871869+0000","last_became_active":"2024-09-10T10:55:48.856069+0000","last_became_peered":"2024-09-10T10:55:48.856069+0000","last_unstale":"2024-09-10T10:55:49.871869+0000","last_undegraded":"2024-09-10T10:55:49.871869+0000","last_fullsized":"2024-09-10T10:55:49.871869+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961154+0000","last_change":"2024-09-10T10:55:48.876456+0000","last_active":"2024-09-10T10:55:49.961154+0000","last_peered":"2024-09-10T10:55:49.961154+0000","last_clean":"2024-09-10T10:55:49.961154+0000","last_became_active":"2024-09-10T10:55:48.876149+0000","last_became_peered":"2024-09-10T10:55:48.876149+0000","last_unstale":"2024-09-10T10:55:49.961154+0000","last_undegraded":"2024-09-10T10:55:49.961154+0000","last_fullsized":"2024-09-10T10:55:49.961154+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947262+0000","last_change":"2024-09-10T10:55:48.863492+0000","last_active":"2024-09-10T10:55:49.947262+0000","last_peered":"2024-09-10T10:55:49.947262+0000","last_clean":"2024-09-10T10:55:49.947262+0000","last_became_active":"2024-09-10T10:55:48.863048+0000","last_became_peered":"2024-09-10T10:55:48.863048+0000","last_unstale":"2024-09-10T10:55:49.947262+0000","last_undegraded":"2024-09-10T10:55:49.947262+0000","last_fullsized":"2024-09-10T10:55:49.947262+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:30.871617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961998+0000","last_change":"2024-09-10T10:55:48.866566+0000","last_active":"2024-09-10T10:55:49.961998+0000","last_peered":"2024-09-10T10:55:49.961998+0000","last_clean":"2024-09-10T10:55:49.961998+0000","last_became_active":"2024-09-10T10:55:48.866256+0000","last_became_peered":"2024-09-10T10:55:48.866256+0000","last_unstale":"2024-09-10T10:55:49.961998+0000","last_undegraded":"2024-09-10T10:55:49.961998+0000","last_fullsized":"2024-09-10T10:55:49.961998+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:16.476043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862921+0000","last_change":"2024-09-10T10:55:48.853498+0000","last_active":"2024-09-10T10:55:49.862921+0000","last_peered":"2024-09-10T10:55:49.862921+0000","last_clean":"2024-09-10T10:55:49.862921+0000","last_became_active":"2024-09-10T10:55:48.853209+0000","last_became_peered":"2024-09-10T10:55:48.853209+0000","last_unstale":"2024-09-10T10:55:49.862921+0000","last_undegraded":"2024-09-10T10:55:49.862921+0000","last_fullsized":"2024-09-10T10:55:49.862921+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:37.984075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872273+0000","last_change":"2024-09-10T10:55:48.865380+0000","last_active":"2024-09-10T10:55:49.872273+0000","last_peered":"2024-09-10T10:55:49.872273+0000","last_clean":"2024-09-10T10:55:49.872273+0000","last_became_active":"2024-09-10T10:55:48.865102+0000","last_became_peered":"2024-09-10T10:55:48.865102+0000","last_unstale":"2024-09-10T10:55:49.872273+0000","last_undegraded":"2024-09-10T10:55:49.872273+0000","last_fullsized":"2024-09-10T10:55:49.872273+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:39.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865722+0000","last_change":"2024-09-10T10:55:48.863226+0000","last_active":"2024-09-10T10:55:49.865722+0000","last_peered":"2024-09-10T10:55:49.865722+0000","last_clean":"2024-09-10T10:55:49.865722+0000","last_became_active":"2024-09-10T10:55:48.862762+0000","last_became_peered":"2024-09-10T10:55:48.862762+0000","last_unstale":"2024-09-10T10:55:49.865722+0000","last_undegraded":"2024-09-10T10:55:49.865722+0000","last_fullsized":"2024-09-10T10:55:49.865722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:24.148383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872189+0000","last_change":"2024-09-10T10:55:48.864947+0000","last_active":"2024-09-10T10:55:49.872189+0000","last_peered":"2024-09-10T10:55:49.872189+0000","last_clean":"2024-09-10T10:55:49.872189+0000","last_became_active":"2024-09-10T10:55:48.864649+0000","last_became_peered":"2024-09-10T10:55:48.864649+0000","last_unstale":"2024-09-10T10:55:49.872189+0000","last_undegraded":"2024-09-10T10:55:49.872189+0000","last_fullsized":"2024-09-10T10:55:49.872189+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:53.436925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946934+0000","last_change":"2024-09-10T10:55:48.862215+0000","last_active":"2024-09-10T10:55:49.946934+0000","last_peered":"2024-09-10T10:55:49.946934+0000","last_clean":"2024-09-10T10:55:49.946934+0000","last_became_active":"2024-09-10T10:55:48.861706+0000","last_became_peered":"2024-09-10T10:55:48.861706+0000","last_unstale":"2024-09-10T10:55:49.946934+0000","last_undegraded":"2024-09-10T10:55:49.946934+0000","last_fullsized":"2024-09-10T10:55:49.946934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:24.201115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960292+0000","last_change":"2024-09-10T10:55:48.879362+0000","last_active":"2024-09-10T10:55:49.960292+0000","last_peered":"2024-09-10T10:55:49.960292+0000","last_clean":"2024-09-10T10:55:49.960292+0000","last_became_active":"2024-09-10T10:55:48.875054+0000","last_became_peered":"2024-09-10T10:55:48.875054+0000","last_unstale":"2024-09-10T10:55:49.960292+0000","last_undegraded":"2024-09-10T10:55:49.960292+0000","last_fullsized":"2024-09-10T10:55:49.960292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.836013+0000","last_change":"2024-09-10T10:55:47.836013+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.836013+0000","last_undegraded":"2024-09-10T10:55:47.836013+0000","last_fullsized":"2024-09-10T10:55:47.836013+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947217+0000","last_change":"2024-09-10T10:55:48.861625+0000","last_active":"2024-09-10T10:55:49.947217+0000","last_peered":"2024-09-10T10:55:49.947217+0000","last_clean":"2024-09-10T10:55:49.947217+0000","last_became_active":"2024-09-10T10:55:48.860626+0000","last_became_peered":"2024-09-10T10:55:48.860626+0000","last_unstale":"2024-09-10T10:55:49.947217+0000","last_undegraded":"2024-09-10T10:55:49.947217+0000","last_fullsized":"2024-09-10T10:55:49.947217+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:19.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865795+0000","last_change":"2024-09-10T10:55:48.855667+0000","last_active":"2024-09-10T10:55:49.865795+0000","last_peered":"2024-09-10T10:55:49.865795+0000","last_clean":"2024-09-10T10:55:49.865795+0000","last_became_active":"2024-09-10T10:55:48.854331+0000","last_became_peered":"2024-09-10T10:55:48.854331+0000","last_unstale":"2024-09-10T10:55:49.865795+0000","last_undegraded":"2024-09-10T10:55:49.865795+0000","last_fullsized":"2024-09-10T10:55:49.865795+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:11.006767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960683+0000","last_change":"2024-09-10T10:55:48.874514+0000","last_active":"2024-09-10T10:55:49.960683+0000","last_peered":"2024-09-10T10:55:49.960683+0000","last_clean":"2024-09-10T10:55:49.960683+0000","last_became_active":"2024-09-10T10:55:48.874243+0000","last_became_peered":"2024-09-10T10:55:48.874243+0000","last_unstale":"2024-09-10T10:55:49.960683+0000","last_undegraded":"2024-09-10T10:55:49.960683+0000","last_fullsized":"2024-09-10T10:55:49.960683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.639323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835270+0000","last_change":"2024-09-10T10:55:47.835270+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835270+0000","last_undegraded":"2024-09-10T10:55:47.835270+0000","last_fullsized":"2024-09-10T10:55:47.835270+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960995+0000","last_change":"2024-09-10T10:55:48.878105+0000","last_active":"2024-09-10T10:55:49.960995+0000","last_peered":"2024-09-10T10:55:49.960995+0000","last_clean":"2024-09-10T10:55:49.960995+0000","last_became_active":"2024-09-10T10:55:48.876598+0000","last_became_peered":"2024-09-10T10:55:48.876598+0000","last_unstale":"2024-09-10T10:55:49.960995+0000","last_undegraded":"2024-09-10T10:55:49.960995+0000","last_fullsized":"2024-09-10T10:55:49.960995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:47.718665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961652+0000","last_change":"2024-09-10T10:55:48.876273+0000","last_active":"2024-09-10T10:55:49.961652+0000","last_peered":"2024-09-10T10:55:49.961652+0000","last_clean":"2024-09-10T10:55:49.961652+0000","last_became_active":"2024-09-10T10:55:48.875814+0000","last_became_peered":"2024-09-10T10:55:48.875814+0000","last_unstale":"2024-09-10T10:55:49.961652+0000","last_undegraded":"2024-09-10T10:55:49.961652+0000","last_fullsized":"2024-09-10T10:55:49.961652+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:23.371793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961329+0000","last_change":"2024-09-10T10:55:48.869413+0000","last_active":"2024-09-10T10:55:49.961329+0000","last_peered":"2024-09-10T10:55:49.961329+0000","last_clean":"2024-09-10T10:55:49.961329+0000","last_became_active":"2024-09-10T10:55:48.869149+0000","last_became_peered":"2024-09-10T10:55:48.869149+0000","last_unstale":"2024-09-10T10:55:49.961329+0000","last_undegraded":"2024-09-10T10:55:49.961329+0000","last_fullsized":"2024-09-10T10:55:49.961329+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:38.451801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960863+0000","last_change":"2024-09-10T10:55:48.866804+0000","last_active":"2024-09-10T10:55:49.960863+0000","last_peered":"2024-09-10T10:55:49.960863+0000","last_clean":"2024-09-10T10:55:49.960863+0000","last_became_active":"2024-09-10T10:55:48.866480+0000","last_became_peered":"2024-09-10T10:55:48.866480+0000","last_unstale":"2024-09-10T10:55:49.960863+0000","last_undegraded":"2024-09-10T10:55:49.960863+0000","last_fullsized":"2024-09-10T10:55:49.960863+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:31.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960874+0000","last_change":"2024-09-10T10:55:48.872545+0000","last_active":"2024-09-10T10:55:49.960874+0000","last_peered":"2024-09-10T10:55:49.960874+0000","last_clean":"2024-09-10T10:55:49.960874+0000","last_became_active":"2024-09-10T10:55:48.872150+0000","last_became_peered":"2024-09-10T10:55:48.872150+0000","last_unstale":"2024-09-10T10:55:49.960874+0000","last_undegraded":"2024-09-10T10:55:49.960874+0000","last_fullsized":"2024-09-10T10:55:49.960874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960690+0000","last_change":"2024-09-10T10:55:48.873569+0000","last_active":"2024-09-10T10:55:49.960690+0000","last_peered":"2024-09-10T10:55:49.960690+0000","last_clean":"2024-09-10T10:55:49.960690+0000","last_became_active":"2024-09-10T10:55:48.872446+0000","last_became_peered":"2024-09-10T10:55:48.872446+0000","last_unstale":"2024-09-10T10:55:49.960690+0000","last_undegraded":"2024-09-10T10:55:49.960690+0000","last_fullsized":"2024-09-10T10:55:49.960690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:58.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872284+0000","last_change":"2024-09-10T10:55:48.866629+0000","last_active":"2024-09-10T10:55:49.872284+0000","last_peered":"2024-09-10T10:55:49.872284+0000","last_clean":"2024-09-10T10:55:49.872284+0000","last_became_active":"2024-09-10T10:55:48.866351+0000","last_became_peered":"2024-09-10T10:55:48.866351+0000","last_unstale":"2024-09-10T10:55:49.872284+0000","last_undegraded":"2024-09-10T10:55:49.872284+0000","last_fullsized":"2024-09-10T10:55:49.872284+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:20.523256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961962+0000","last_change":"2024-09-10T10:55:48.874303+0000","last_active":"2024-09-10T10:55:49.961962+0000","last_peered":"2024-09-10T10:55:49.961962+0000","last_clean":"2024-09-10T10:55:49.961962+0000","last_became_active":"2024-09-10T10:55:48.873605+0000","last_became_peered":"2024-09-10T10:55:48.873605+0000","last_unstale":"2024-09-10T10:55:49.961962+0000","last_undegraded":"2024-09-10T10:55:49.961962+0000","last_fullsized":"2024-09-10T10:55:49.961962+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:43.380426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862682+0000","last_change":"2024-09-10T10:55:48.851514+0000","last_active":"2024-09-10T10:55:49.862682+0000","last_peered":"2024-09-10T10:55:49.862682+0000","last_clean":"2024-09-10T10:55:49.862682+0000","last_became_active":"2024-09-10T10:55:48.851231+0000","last_became_peered":"2024-09-10T10:55:48.851231+0000","last_unstale":"2024-09-10T10:55:49.862682+0000","last_undegraded":"2024-09-10T10:55:49.862682+0000","last_fullsized":"2024-09-10T10:55:49.862682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:21.797943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871910+0000","last_change":"2024-09-10T10:55:48.860423+0000","last_active":"2024-09-10T10:55:49.871910+0000","last_peered":"2024-09-10T10:55:49.871910+0000","last_clean":"2024-09-10T10:55:49.871910+0000","last_became_active":"2024-09-10T10:55:48.860101+0000","last_became_peered":"2024-09-10T10:55:48.860101+0000","last_unstale":"2024-09-10T10:55:49.871910+0000","last_undegraded":"2024-09-10T10:55:49.871910+0000","last_fullsized":"2024-09-10T10:55:49.871910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:15.280350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960768+0000","last_change":"2024-09-10T10:55:48.861221+0000","last_active":"2024-09-10T10:55:49.960768+0000","last_peered":"2024-09-10T10:55:49.960768+0000","last_clean":"2024-09-10T10:55:49.960768+0000","last_became_active":"2024-09-10T10:55:48.860949+0000","last_became_peered":"2024-09-10T10:55:48.860949+0000","last_unstale":"2024-09-10T10:55:49.960768+0000","last_undegraded":"2024-09-10T10:55:49.960768+0000","last_fullsized":"2024-09-10T10:55:49.960768+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:07.147500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.833598+0000","last_change":"2024-09-10T10:55:47.833598+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.833598+0000","last_undegraded":"2024-09-10T10:55:47.833598+0000","last_fullsized":"2024-09-10T10:55:47.833598+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961416+0000","last_change":"2024-09-10T10:55:47.855582+0000","last_active":"2024-09-10T10:55:49.961416+0000","last_peered":"2024-09-10T10:55:49.961416+0000","last_clean":"2024-09-10T10:55:49.961416+0000","last_became_active":"2024-09-10T10:55:47.855156+0000","last_became_peered":"2024-09-10T10:55:47.855156+0000","last_unstale":"2024-09-10T10:55:49.961416+0000","last_undegraded":"2024-09-10T10:55:49.961416+0000","last_fullsized":"2024-09-10T10:55:49.961416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:30.174489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961811+0000","last_change":"2024-09-10T10:55:48.876356+0000","last_active":"2024-09-10T10:55:49.961811+0000","last_peered":"2024-09-10T10:55:49.961811+0000","last_clean":"2024-09-10T10:55:49.961811+0000","last_became_active":"2024-09-10T10:55:48.875734+0000","last_became_peered":"2024-09-10T10:55:48.875734+0000","last_unstale":"2024-09-10T10:55:49.961811+0000","last_undegraded":"2024-09-10T10:55:49.961811+0000","last_fullsized":"2024-09-10T10:55:49.961811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:45.252064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851021+0000","last_change":"2024-09-10T10:55:47.846423+0000","last_active":"2024-09-10T10:55:47.851021+0000","last_peered":"2024-09-10T10:55:47.851021+0000","last_clean":"2024-09-10T10:55:47.851021+0000","last_became_active":"2024-09-10T10:55:47.842576+0000","last_became_peered":"2024-09-10T10:55:47.842576+0000","last_unstale":"2024-09-10T10:55:47.851021+0000","last_undegraded":"2024-09-10T10:55:47.851021+0000","last_fullsized":"2024-09-10T10:55:47.851021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.847568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946997+0000","last_change":"2024-09-10T10:55:48.864892+0000","last_active":"2024-09-10T10:55:49.946997+0000","last_peered":"2024-09-10T10:55:49.946997+0000","last_clean":"2024-09-10T10:55:49.946997+0000","last_became_active":"2024-09-10T10:55:48.864545+0000","last_became_peered":"2024-09-10T10:55:48.864545+0000","last_unstale":"2024-09-10T10:55:49.946997+0000","last_undegraded":"2024-09-10T10:55:49.946997+0000","last_fullsized":"2024-09-10T10:55:49.946997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:45.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947156+0000","last_change":"2024-09-10T10:55:47.848829+0000","last_active":"2024-09-10T10:55:49.947156+0000","last_peered":"2024-09-10T10:55:49.947156+0000","last_clean":"2024-09-10T10:55:49.947156+0000","last_became_active":"2024-09-10T10:55:47.848545+0000","last_became_peered":"2024-09-10T10:55:47.848545+0000","last_unstale":"2024-09-10T10:55:49.947156+0000","last_undegraded":"2024-09-10T10:55:49.947156+0000","last_fullsized":"2024-09-10T10:55:49.947156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:26.415458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862744+0000","last_change":"2024-09-10T10:55:48.853317+0000","last_active":"2024-09-10T10:55:49.862744+0000","last_peered":"2024-09-10T10:55:49.862744+0000","last_clean":"2024-09-10T10:55:49.862744+0000","last_became_active":"2024-09-10T10:55:48.853020+0000","last_became_peered":"2024-09-10T10:55:48.853020+0000","last_unstale":"2024-09-10T10:55:49.862744+0000","last_undegraded":"2024-09-10T10:55:49.862744+0000","last_fullsized":"2024-09-10T10:55:49.862744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:20.610135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961781+0000","last_change":"2024-09-10T10:55:47.847504+0000","last_active":"2024-09-10T10:55:49.961781+0000","last_peered":"2024-09-10T10:55:49.961781+0000","last_clean":"2024-09-10T10:55:49.961781+0000","last_became_active":"2024-09-10T10:55:47.847070+0000","last_became_peered":"2024-09-10T10:55:47.847070+0000","last_unstale":"2024-09-10T10:55:49.961781+0000","last_undegraded":"2024-09-10T10:55:49.961781+0000","last_fullsized":"2024-09-10T10:55:49.961781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:04.988364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865922+0000","last_change":"2024-09-10T10:55:48.867784+0000","last_active":"2024-09-10T10:55:49.865922+0000","last_peered":"2024-09-10T10:55:49.865922+0000","last_clean":"2024-09-10T10:55:49.865922+0000","last_became_active":"2024-09-10T10:55:48.867508+0000","last_became_peered":"2024-09-10T10:55:48.867508+0000","last_unstale":"2024-09-10T10:55:49.865922+0000","last_undegraded":"2024-09-10T10:55:49.865922+0000","last_fullsized":"2024-09-10T10:55:49.865922+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:48.476999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862859+0000","last_change":"2024-09-10T10:55:47.845137+0000","last_active":"2024-09-10T10:55:49.862859+0000","last_peered":"2024-09-10T10:55:49.862859+0000","last_clean":"2024-09-10T10:55:49.862859+0000","last_became_active":"2024-09-10T10:55:47.844834+0000","last_became_peered":"2024-09-10T10:55:47.844834+0000","last_unstale":"2024-09-10T10:55:49.862859+0000","last_undegraded":"2024-09-10T10:55:49.862859+0000","last_fullsized":"2024-09-10T10:55:49.862859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:32.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865918+0000","last_change":"2024-09-10T10:55:48.864028+0000","last_active":"2024-09-10T10:55:49.865918+0000","last_peered":"2024-09-10T10:55:49.865918+0000","last_clean":"2024-09-10T10:55:49.865918+0000","last_became_active":"2024-09-10T10:55:48.863229+0000","last_became_peered":"2024-09-10T10:55:48.863229+0000","last_unstale":"2024-09-10T10:55:49.865918+0000","last_undegraded":"2024-09-10T10:55:49.865918+0000","last_fullsized":"2024-09-10T10:55:49.865918+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:18.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960308+0000","last_change":"2024-09-10T10:55:47.843083+0000","last_active":"2024-09-10T10:55:49.960308+0000","last_peered":"2024-09-10T10:55:49.960308+0000","last_clean":"2024-09-10T10:55:49.960308+0000","last_became_active":"2024-09-10T10:55:47.839631+0000","last_became_peered":"2024-09-10T10:55:47.839631+0000","last_unstale":"2024-09-10T10:55:49.960308+0000","last_undegraded":"2024-09-10T10:55:49.960308+0000","last_fullsized":"2024-09-10T10:55:49.960308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:18.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.863052+0000","last_change":"2024-09-10T10:55:48.864786+0000","last_active":"2024-09-10T10:55:49.863052+0000","last_peered":"2024-09-10T10:55:49.863052+0000","last_clean":"2024-09-10T10:55:49.863052+0000","last_became_active":"2024-09-10T10:55:48.864460+0000","last_became_peered":"2024-09-10T10:55:48.864460+0000","last_unstale":"2024-09-10T10:55:49.863052+0000","last_undegraded":"2024-09-10T10:55:49.863052+0000","last_fullsized":"2024-09-10T10:55:49.863052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:05.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960378+0000","last_change":"2024-09-10T10:55:47.846337+0000","last_active":"2024-09-10T10:55:49.960378+0000","last_peered":"2024-09-10T10:55:49.960378+0000","last_clean":"2024-09-10T10:55:49.960378+0000","last_became_active":"2024-09-10T10:55:47.845507+0000","last_became_peered":"2024-09-10T10:55:47.845507+0000","last_unstale":"2024-09-10T10:55:49.960378+0000","last_undegraded":"2024-09-10T10:55:49.960378+0000","last_fullsized":"2024-09-10T10:55:49.960378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:11.338379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865877+0000","last_change":"2024-09-10T10:55:48.864808+0000","last_active":"2024-09-10T10:55:49.865877+0000","last_peered":"2024-09-10T10:55:49.865877+0000","last_clean":"2024-09-10T10:55:49.865877+0000","last_became_active":"2024-09-10T10:55:48.862818+0000","last_became_peered":"2024-09-10T10:55:48.862818+0000","last_unstale":"2024-09-10T10:55:49.865877+0000","last_undegraded":"2024-09-10T10:55:49.865877+0000","last_fullsized":"2024-09-10T10:55:49.865877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:44.808359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:47.853115+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:47.852824+0000","last_became_peered":"2024-09-10T10:55:47.852824+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:57.389591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864710+0000","last_change":"2024-09-10T10:55:48.865231+0000","last_active":"2024-09-10T10:55:49.864710+0000","last_peered":"2024-09-10T10:55:49.864710+0000","last_clean":"2024-09-10T10:55:49.864710+0000","last_became_active":"2024-09-10T10:55:48.862955+0000","last_became_peered":"2024-09-10T10:55:48.862955+0000","last_unstale":"2024-09-10T10:55:49.864710+0000","last_undegraded":"2024-09-10T10:55:49.864710+0000","last_fullsized":"2024-09-10T10:55:49.864710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.358498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872584+0000","last_change":"2024-09-10T10:55:47.843342+0000","last_active":"2024-09-10T10:55:49.872584+0000","last_peered":"2024-09-10T10:55:49.872584+0000","last_clean":"2024-09-10T10:55:49.872584+0000","last_became_active":"2024-09-10T10:55:47.842788+0000","last_became_peered":"2024-09-10T10:55:47.842788+0000","last_unstale":"2024-09-10T10:55:49.872584+0000","last_undegraded":"2024-09-10T10:55:49.872584+0000","last_fullsized":"2024-09-10T10:55:49.872584+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:00.612235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865943+0000","last_change":"2024-09-10T10:55:48.869535+0000","last_active":"2024-09-10T10:55:49.865943+0000","last_peered":"2024-09-10T10:55:49.865943+0000","last_clean":"2024-09-10T10:55:49.865943+0000","last_became_active":"2024-09-10T10:55:48.869284+0000","last_became_peered":"2024-09-10T10:55:48.869284+0000","last_unstale":"2024-09-10T10:55:49.865943+0000","last_undegraded":"2024-09-10T10:55:49.865943+0000","last_fullsized":"2024-09-10T10:55:49.865943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:44.081502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961991+0000","last_change":"2024-09-10T10:55:47.851219+0000","last_active":"2024-09-10T10:55:49.961991+0000","last_peered":"2024-09-10T10:55:49.961991+0000","last_clean":"2024-09-10T10:55:49.961991+0000","last_became_active":"2024-09-10T10:55:47.849193+0000","last_became_peered":"2024-09-10T10:55:47.849193+0000","last_unstale":"2024-09-10T10:55:49.961991+0000","last_undegraded":"2024-09-10T10:55:49.961991+0000","last_fullsized":"2024-09-10T10:55:49.961991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:50.483870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961814+0000","last_change":"2024-09-10T10:55:48.868003+0000","last_active":"2024-09-10T10:55:49.961814+0000","last_peered":"2024-09-10T10:55:49.961814+0000","last_clean":"2024-09-10T10:55:49.961814+0000","last_became_active":"2024-09-10T10:55:48.867694+0000","last_became_peered":"2024-09-10T10:55:48.867694+0000","last_unstale":"2024-09-10T10:55:49.961814+0000","last_undegraded":"2024-09-10T10:55:49.961814+0000","last_fullsized":"2024-09-10T10:55:49.961814+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:37.123756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872366+0000","last_change":"2024-09-10T10:55:47.847591+0000","last_active":"2024-09-10T10:55:49.872366+0000","last_peered":"2024-09-10T10:55:49.872366+0000","last_clean":"2024-09-10T10:55:49.872366+0000","last_became_active":"2024-09-10T10:55:47.847299+0000","last_became_peered":"2024-09-10T10:55:47.847299+0000","last_unstale":"2024-09-10T10:55:49.872366+0000","last_undegraded":"2024-09-10T10:55:49.872366+0000","last_fullsized":"2024-09-10T10:55:49.872366+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.943592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961027+0000","last_change":"2024-09-10T10:55:48.875294+0000","last_active":"2024-09-10T10:55:49.961027+0000","last_peered":"2024-09-10T10:55:49.961027+0000","last_clean":"2024-09-10T10:55:49.961027+0000","last_became_active":"2024-09-10T10:55:48.873851+0000","last_became_peered":"2024-09-10T10:55:48.873851+0000","last_unstale":"2024-09-10T10:55:49.961027+0000","last_undegraded":"2024-09-10T10:55:49.961027+0000","last_fullsized":"2024-09-10T10:55:49.961027+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:07.268035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862812+0000","last_change":"2024-09-10T10:55:47.850868+0000","last_active":"2024-09-10T10:55:49.862812+0000","last_peered":"2024-09-10T10:55:49.862812+0000","last_clean":"2024-09-10T10:55:49.862812+0000","last_became_active":"2024-09-10T10:55:47.850457+0000","last_became_peered":"2024-09-10T10:55:47.850457+0000","last_unstale":"2024-09-10T10:55:49.862812+0000","last_undegraded":"2024-09-10T10:55:49.862812+0000","last_fullsized":"2024-09-10T10:55:49.862812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:45.564987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947047+0000","last_change":"2024-09-10T10:55:48.861530+0000","last_active":"2024-09-10T10:55:49.947047+0000","last_peered":"2024-09-10T10:55:49.947047+0000","last_clean":"2024-09-10T10:55:49.947047+0000","last_became_active":"2024-09-10T10:55:48.860648+0000","last_became_peered":"2024-09-10T10:55:48.860648+0000","last_unstale":"2024-09-10T10:55:49.947047+0000","last_undegraded":"2024-09-10T10:55:49.947047+0000","last_fullsized":"2024-09-10T10:55:49.947047+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:36.041255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865960+0000","last_change":"2024-09-10T10:55:47.843477+0000","last_active":"2024-09-10T10:55:49.865960+0000","last_peered":"2024-09-10T10:55:49.865960+0000","last_clean":"2024-09-10T10:55:49.865960+0000","last_became_active":"2024-09-10T10:55:47.841371+0000","last_became_peered":"2024-09-10T10:55:47.841371+0000","last_unstale":"2024-09-10T10:55:49.865960+0000","last_undegraded":"2024-09-10T10:55:49.865960+0000","last_fullsized":"2024-09-10T10:55:49.865960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:45.636397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947145+0000","last_change":"2024-09-10T10:55:48.865788+0000","last_active":"2024-09-10T10:55:49.947145+0000","last_peered":"2024-09-10T10:55:49.947145+0000","last_clean":"2024-09-10T10:55:49.947145+0000","last_became_active":"2024-09-10T10:55:48.865486+0000","last_became_peered":"2024-09-10T10:55:48.865486+0000","last_unstale":"2024-09-10T10:55:49.947145+0000","last_undegraded":"2024-09-10T10:55:49.947145+0000","last_fullsized":"2024-09-10T10:55:49.947145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:56.003393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961925+0000","last_change":"2024-09-10T10:55:47.845313+0000","last_active":"2024-09-10T10:55:49.961925+0000","last_peered":"2024-09-10T10:55:49.961925+0000","last_clean":"2024-09-10T10:55:49.961925+0000","last_became_active":"2024-09-10T10:55:47.844999+0000","last_became_peered":"2024-09-10T10:55:47.844999+0000","last_unstale":"2024-09-10T10:55:49.961925+0000","last_undegraded":"2024-09-10T10:55:49.961925+0000","last_fullsized":"2024-09-10T10:55:49.961925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:07.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961293+0000","last_change":"2024-09-10T10:55:48.877412+0000","last_active":"2024-09-10T10:55:49.961293+0000","last_peered":"2024-09-10T10:55:49.961293+0000","last_clean":"2024-09-10T10:55:49.961293+0000","last_became_active":"2024-09-10T10:55:48.876383+0000","last_became_peered":"2024-09-10T10:55:48.876383+0000","last_unstale":"2024-09-10T10:55:49.961293+0000","last_undegraded":"2024-09-10T10:55:49.961293+0000","last_fullsized":"2024-09-10T10:55:49.961293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:41.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865237+0000","last_change":"2024-09-10T10:55:47.834691+0000","last_active":"2024-09-10T10:55:49.865237+0000","last_peered":"2024-09-10T10:55:49.865237+0000","last_clean":"2024-09-10T10:55:49.865237+0000","last_became_active":"2024-09-10T10:55:47.834400+0000","last_became_peered":"2024-09-10T10:55:47.834400+0000","last_unstale":"2024-09-10T10:55:49.865237+0000","last_undegraded":"2024-09-10T10:55:49.865237+0000","last_fullsized":"2024-09-10T10:55:49.865237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:52.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835555+0000","last_change":"2024-09-10T10:55:47.835555+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835555+0000","last_undegraded":"2024-09-10T10:55:47.835555+0000","last_fullsized":"2024-09-10T10:55:47.835555+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862623+0000","last_change":"2024-09-10T10:55:47.843176+0000","last_active":"2024-09-10T10:55:49.862623+0000","last_peered":"2024-09-10T10:55:49.862623+0000","last_clean":"2024-09-10T10:55:49.862623+0000","last_became_active":"2024-09-10T10:55:47.842914+0000","last_became_peered":"2024-09-10T10:55:47.842914+0000","last_unstale":"2024-09-10T10:55:49.862623+0000","last_undegraded":"2024-09-10T10:55:49.862623+0000","last_fullsized":"2024-09-10T10:55:49.862623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:46.876235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961424+0000","last_change":"2024-09-10T10:55:48.874049+0000","last_active":"2024-09-10T10:55:49.961424+0000","last_peered":"2024-09-10T10:55:49.961424+0000","last_clean":"2024-09-10T10:55:49.961424+0000","last_became_active":"2024-09-10T10:55:48.873744+0000","last_became_peered":"2024-09-10T10:55:48.873744+0000","last_unstale":"2024-09-10T10:55:49.961424+0000","last_undegraded":"2024-09-10T10:55:49.961424+0000","last_fullsized":"2024-09-10T10:55:49.961424+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:25.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872621+0000","last_change":"2024-09-10T10:55:47.845166+0000","last_active":"2024-09-10T10:55:49.872621+0000","last_peered":"2024-09-10T10:55:49.872621+0000","last_clean":"2024-09-10T10:55:49.872621+0000","last_became_active":"2024-09-10T10:55:47.844675+0000","last_became_peered":"2024-09-10T10:55:47.844675+0000","last_unstale":"2024-09-10T10:55:49.872621+0000","last_undegraded":"2024-09-10T10:55:49.872621+0000","last_fullsized":"2024-09-10T10:55:49.872621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:26.167518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960321+0000","last_change":"2024-09-10T10:55:48.872087+0000","last_active":"2024-09-10T10:55:49.960321+0000","last_peered":"2024-09-10T10:55:49.960321+0000","last_clean":"2024-09-10T10:55:49.960321+0000","last_became_active":"2024-09-10T10:55:48.871832+0000","last_became_peered":"2024-09-10T10:55:48.871832+0000","last_unstale":"2024-09-10T10:55:49.960321+0000","last_undegraded":"2024-09-10T10:55:49.960321+0000","last_fullsized":"2024-09-10T10:55:49.960321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:58.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960717+0000","last_change":"2024-09-10T10:55:47.850066+0000","last_active":"2024-09-10T10:55:49.960717+0000","last_peered":"2024-09-10T10:55:49.960717+0000","last_clean":"2024-09-10T10:55:49.960717+0000","last_became_active":"2024-09-10T10:55:47.849775+0000","last_became_peered":"2024-09-10T10:55:47.849775+0000","last_unstale":"2024-09-10T10:55:49.960717+0000","last_undegraded":"2024-09-10T10:55:49.960717+0000","last_fullsized":"2024-09-10T10:55:49.960717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:29.865577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961193+0000","last_change":"2024-09-10T10:55:48.860009+0000","last_active":"2024-09-10T10:55:49.961193+0000","last_peered":"2024-09-10T10:55:49.961193+0000","last_clean":"2024-09-10T10:55:49.961193+0000","last_became_active":"2024-09-10T10:55:48.859725+0000","last_became_peered":"2024-09-10T10:55:48.859725+0000","last_unstale":"2024-09-10T10:55:49.961193+0000","last_undegraded":"2024-09-10T10:55:49.961193+0000","last_fullsized":"2024-09-10T10:55:49.961193+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:06.818156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872362+0000","last_change":"2024-09-10T10:55:47.846489+0000","last_active":"2024-09-10T10:55:49.872362+0000","last_peered":"2024-09-10T10:55:49.872362+0000","last_clean":"2024-09-10T10:55:49.872362+0000","last_became_active":"2024-09-10T10:55:47.846067+0000","last_became_peered":"2024-09-10T10:55:47.846067+0000","last_unstale":"2024-09-10T10:55:49.872362+0000","last_undegraded":"2024-09-10T10:55:49.872362+0000","last_fullsized":"2024-09-10T10:55:49.872362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:28.004930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865650+0000","last_change":"2024-09-10T10:55:48.857694+0000","last_active":"2024-09-10T10:55:49.865650+0000","last_peered":"2024-09-10T10:55:49.865650+0000","last_clean":"2024-09-10T10:55:49.865650+0000","last_became_active":"2024-09-10T10:55:48.857007+0000","last_became_peered":"2024-09-10T10:55:48.857007+0000","last_unstale":"2024-09-10T10:55:49.865650+0000","last_undegraded":"2024-09-10T10:55:49.865650+0000","last_fullsized":"2024-09-10T10:55:49.865650+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:15.966260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.850937+0000","last_change":"2024-09-10T10:55:47.846071+0000","last_active":"2024-09-10T10:55:47.850937+0000","last_peered":"2024-09-10T10:55:47.850937+0000","last_clean":"2024-09-10T10:55:47.850937+0000","last_became_active":"2024-09-10T10:55:47.845052+0000","last_became_peered":"2024-09-10T10:55:47.845052+0000","last_unstale":"2024-09-10T10:55:47.850937+0000","last_undegraded":"2024-09-10T10:55:47.850937+0000","last_fullsized":"2024-09-10T10:55:47.850937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:32.064815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:48.868962+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:48.868602+0000","last_became_peered":"2024-09-10T10:55:48.868602+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:37.194259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.853061+0000","last_change":"2024-09-10T10:55:47.846363+0000","last_active":"2024-09-10T10:55:47.853061+0000","last_peered":"2024-09-10T10:55:47.853061+0000","last_clean":"2024-09-10T10:55:47.853061+0000","last_became_active":"2024-09-10T10:55:47.846001+0000","last_became_peered":"2024-09-10T10:55:47.846001+0000","last_unstale":"2024-09-10T10:55:47.853061+0000","last_undegraded":"2024-09-10T10:55:47.853061+0000","last_fullsized":"2024-09-10T10:55:47.853061+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:59.581099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961231+0000","last_change":"2024-09-10T10:55:48.861960+0000","last_active":"2024-09-10T10:55:49.961231+0000","last_peered":"2024-09-10T10:55:49.961231+0000","last_clean":"2024-09-10T10:55:49.961231+0000","last_became_active":"2024-09-10T10:55:48.861620+0000","last_became_peered":"2024-09-10T10:55:48.861620+0000","last_unstale":"2024-09-10T10:55:49.961231+0000","last_undegraded":"2024-09-10T10:55:49.961231+0000","last_fullsized":"2024-09-10T10:55:49.961231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:27.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872025+0000","last_change":"2024-09-10T10:55:47.837440+0000","last_active":"2024-09-10T10:55:49.872025+0000","last_peered":"2024-09-10T10:55:49.872025+0000","last_clean":"2024-09-10T10:55:49.872025+0000","last_became_active":"2024-09-10T10:55:47.836741+0000","last_became_peered":"2024-09-10T10:55:47.836741+0000","last_unstale":"2024-09-10T10:55:49.872025+0000","last_undegraded":"2024-09-10T10:55:49.872025+0000","last_fullsized":"2024-09-10T10:55:49.872025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:58.662475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865678+0000","last_change":"2024-09-10T10:55:48.867356+0000","last_active":"2024-09-10T10:55:49.865678+0000","last_peered":"2024-09-10T10:55:49.865678+0000","last_clean":"2024-09-10T10:55:49.865678+0000","last_became_active":"2024-09-10T10:55:48.866816+0000","last_became_peered":"2024-09-10T10:55:48.866816+0000","last_unstale":"2024-09-10T10:55:49.865678+0000","last_undegraded":"2024-09-10T10:55:49.865678+0000","last_fullsized":"2024-09-10T10:55:49.865678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:17.379036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961500+0000","last_change":"2024-09-10T10:55:47.845337+0000","last_active":"2024-09-10T10:55:49.961500+0000","last_peered":"2024-09-10T10:55:49.961500+0000","last_clean":"2024-09-10T10:55:49.961500+0000","last_became_active":"2024-09-10T10:55:47.845077+0000","last_became_peered":"2024-09-10T10:55:47.845077+0000","last_unstale":"2024-09-10T10:55:49.961500+0000","last_undegraded":"2024-09-10T10:55:49.961500+0000","last_fullsized":"2024-09-10T10:55:49.961500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:14.274310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960779+0000","last_change":"2024-09-10T10:55:48.864373+0000","last_active":"2024-09-10T10:55:49.960779+0000","last_peered":"2024-09-10T10:55:49.960779+0000","last_clean":"2024-09-10T10:55:49.960779+0000","last_became_active":"2024-09-10T10:55:48.864041+0000","last_became_peered":"2024-09-10T10:55:48.864041+0000","last_unstale":"2024-09-10T10:55:49.960779+0000","last_undegraded":"2024-09-10T10:55:49.960779+0000","last_fullsized":"2024-09-10T10:55:49.960779+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:39.301780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.852322+0000","last_change":"2024-09-10T10:55:47.850713+0000","last_active":"2024-09-10T10:55:47.852322+0000","last_peered":"2024-09-10T10:55:47.852322+0000","last_clean":"2024-09-10T10:55:47.852322+0000","last_became_active":"2024-09-10T10:55:47.849850+0000","last_became_peered":"2024-09-10T10:55:47.849850+0000","last_unstale":"2024-09-10T10:55:47.852322+0000","last_undegraded":"2024-09-10T10:55:47.852322+0000","last_fullsized":"2024-09-10T10:55:47.852322+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:40.025775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862722+0000","last_change":"2024-09-10T10:55:48.859546+0000","last_active":"2024-09-10T10:55:49.862722+0000","last_peered":"2024-09-10T10:55:49.862722+0000","last_clean":"2024-09-10T10:55:49.862722+0000","last_became_active":"2024-09-10T10:55:48.859164+0000","last_became_peered":"2024-09-10T10:55:48.859164+0000","last_unstale":"2024-09-10T10:55:49.862722+0000","last_undegraded":"2024-09-10T10:55:49.862722+0000","last_fullsized":"2024-09-10T10:55:49.862722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:29.838709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854948+0000","last_change":"2024-09-10T10:55:47.853293+0000","last_active":"2024-09-10T10:55:47.854948+0000","last_peered":"2024-09-10T10:55:47.854948+0000","last_clean":"2024-09-10T10:55:47.854948+0000","last_became_active":"2024-09-10T10:55:47.852967+0000","last_became_peered":"2024-09-10T10:55:47.852967+0000","last_unstale":"2024-09-10T10:55:47.854948+0000","last_undegraded":"2024-09-10T10:55:47.854948+0000","last_fullsized":"2024-09-10T10:55:47.854948+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:50.182209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961880+0000","last_change":"2024-09-10T10:55:48.874714+0000","last_active":"2024-09-10T10:55:49.961880+0000","last_peered":"2024-09-10T10:55:49.961880+0000","last_clean":"2024-09-10T10:55:49.961880+0000","last_became_active":"2024-09-10T10:55:48.874256+0000","last_became_peered":"2024-09-10T10:55:48.874256+0000","last_unstale":"2024-09-10T10:55:49.961880+0000","last_undegraded":"2024-09-10T10:55:49.961880+0000","last_fullsized":"2024-09-10T10:55:49.961880+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:04.561493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862937+0000","last_change":"2024-09-10T10:55:47.852303+0000","last_active":"2024-09-10T10:55:49.862937+0000","last_peered":"2024-09-10T10:55:49.862937+0000","last_clean":"2024-09-10T10:55:49.862937+0000","last_became_active":"2024-09-10T10:55:47.851277+0000","last_became_peered":"2024-09-10T10:55:47.851277+0000","last_unstale":"2024-09-10T10:55:49.862937+0000","last_undegraded":"2024-09-10T10:55:49.862937+0000","last_fullsized":"2024-09-10T10:55:49.862937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:43.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960557+0000","last_change":"2024-09-10T10:55:48.876162+0000","last_active":"2024-09-10T10:55:49.960557+0000","last_peered":"2024-09-10T10:55:49.960557+0000","last_clean":"2024-09-10T10:55:49.960557+0000","last_became_active":"2024-09-10T10:55:48.875834+0000","last_became_peered":"2024-09-10T10:55:48.875834+0000","last_unstale":"2024-09-10T10:55:49.960557+0000","last_undegraded":"2024-09-10T10:55:49.960557+0000","last_fullsized":"2024-09-10T10:55:49.960557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:12.003746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.866001+0000","last_change":"2024-09-10T10:55:47.844106+0000","last_active":"2024-09-10T10:55:49.866001+0000","last_peered":"2024-09-10T10:55:49.866001+0000","last_clean":"2024-09-10T10:55:49.866001+0000","last_became_active":"2024-09-10T10:55:47.843763+0000","last_became_peered":"2024-09-10T10:55:47.843763+0000","last_unstale":"2024-09-10T10:55:49.866001+0000","last_undegraded":"2024-09-10T10:55:49.866001+0000","last_fullsized":"2024-09-10T10:55:49.866001+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:19.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864866+0000","last_change":"2024-09-10T10:55:48.857933+0000","last_active":"2024-09-10T10:55:49.864866+0000","last_peered":"2024-09-10T10:55:49.864866+0000","last_clean":"2024-09-10T10:55:49.864866+0000","last_became_active":"2024-09-10T10:55:48.857414+0000","last_became_peered":"2024-09-10T10:55:48.857414+0000","last_unstale":"2024-09-10T10:55:49.864866+0000","last_undegraded":"2024-09-10T10:55:49.864866+0000","last_fullsized":"2024-09-10T10:55:49.864866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:52.855868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862873+0000","last_change":"2024-09-10T10:55:47.846380+0000","last_active":"2024-09-10T10:55:49.862873+0000","last_peered":"2024-09-10T10:55:49.862873+0000","last_clean":"2024-09-10T10:55:49.862873+0000","last_became_active":"2024-09-10T10:55:47.846123+0000","last_became_peered":"2024-09-10T10:55:47.846123+0000","last_unstale":"2024-09-10T10:55:49.862873+0000","last_undegraded":"2024-09-10T10:55:49.862873+0000","last_fullsized":"2024-09-10T10:55:49.862873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.465904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865669+0000","last_change":"2024-09-10T10:55:48.859593+0000","last_active":"2024-09-10T10:55:49.865669+0000","last_peered":"2024-09-10T10:55:49.865669+0000","last_clean":"2024-09-10T10:55:49.865669+0000","last_became_active":"2024-09-10T10:55:48.859045+0000","last_became_peered":"2024-09-10T10:55:48.859045+0000","last_unstale":"2024-09-10T10:55:49.865669+0000","last_undegraded":"2024-09-10T10:55:49.865669+0000","last_fullsized":"2024-09-10T10:55:49.865669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:39.427524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854980+0000","last_change":"2024-09-10T10:55:47.853193+0000","last_active":"2024-09-10T10:55:47.854980+0000","last_peered":"2024-09-10T10:55:47.854980+0000","last_clean":"2024-09-10T10:55:47.854980+0000","last_became_active":"2024-09-10T10:55:47.852861+0000","last_became_peered":"2024-09-10T10:55:47.852861+0000","last_unstale":"2024-09-10T10:55:47.854980+0000","last_undegraded":"2024-09-10T10:55:47.854980+0000","last_fullsized":"2024-09-10T10:55:47.854980+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:43.046220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872528+0000","last_change":"2024-09-10T10:55:48.870415+0000","last_active":"2024-09-10T10:55:49.872528+0000","last_peered":"2024-09-10T10:55:49.872528+0000","last_clean":"2024-09-10T10:55:49.872528+0000","last_became_active":"2024-09-10T10:55:48.869966+0000","last_became_peered":"2024-09-10T10:55:48.869966+0000","last_unstale":"2024-09-10T10:55:49.872528+0000","last_undegraded":"2024-09-10T10:55:49.872528+0000","last_fullsized":"2024-09-10T10:55:49.872528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960416+0000","last_change":"2024-09-10T10:55:47.850818+0000","last_active":"2024-09-10T10:55:49.960416+0000","last_peered":"2024-09-10T10:55:49.960416+0000","last_clean":"2024-09-10T10:55:49.960416+0000","last_became_active":"2024-09-10T10:55:47.850541+0000","last_became_peered":"2024-09-10T10:55:47.850541+0000","last_unstale":"2024-09-10T10:55:49.960416+0000","last_undegraded":"2024-09-10T10:55:49.960416+0000","last_fullsized":"2024-09-10T10:55:49.960416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:54.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947271+0000","last_change":"2024-09-10T10:55:48.862694+0000","last_active":"2024-09-10T10:55:49.947271+0000","last_peered":"2024-09-10T10:55:49.947271+0000","last_clean":"2024-09-10T10:55:49.947271+0000","last_became_active":"2024-09-10T10:55:48.862367+0000","last_became_peered":"2024-09-10T10:55:48.862367+0000","last_unstale":"2024-09-10T10:55:49.947271+0000","last_undegraded":"2024-09-10T10:55:49.947271+0000","last_fullsized":"2024-09-10T10:55:49.947271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851952+0000","last_change":"2024-09-10T10:55:47.846160+0000","last_active":"2024-09-10T10:55:47.851952+0000","last_peered":"2024-09-10T10:55:47.851952+0000","last_clean":"2024-09-10T10:55:47.851952+0000","last_became_active":"2024-09-10T10:55:47.845758+0000","last_became_peered":"2024-09-10T10:55:47.845758+0000","last_unstale":"2024-09-10T10:55:47.851952+0000","last_undegraded":"2024-09-10T10:55:47.851952+0000","last_fullsized":"2024-09-10T10:55:47.851952+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:13.218467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871975+0000","last_change":"2024-09-10T10:55:48.867075+0000","last_active":"2024-09-10T10:55:49.871975+0000","last_peered":"2024-09-10T10:55:49.871975+0000","last_clean":"2024-09-10T10:55:49.871975+0000","last_became_active":"2024-09-10T10:55:48.859669+0000","last_became_peered":"2024-09-10T10:55:48.859669+0000","last_unstale":"2024-09-10T10:55:49.871975+0000","last_undegraded":"2024-09-10T10:55:49.871975+0000","last_fullsized":"2024-09-10T10:55:49.871975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:05.994404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864565+0000","last_change":"2024-09-10T10:55:47.847392+0000","last_active":"2024-09-10T10:55:49.864565+0000","last_peered":"2024-09-10T10:55:49.864565+0000","last_clean":"2024-09-10T10:55:49.864565+0000","last_became_active":"2024-09-10T10:55:47.847116+0000","last_became_peered":"2024-09-10T10:55:47.847116+0000","last_unstale":"2024-09-10T10:55:49.864565+0000","last_undegraded":"2024-09-10T10:55:49.864565+0000","last_fullsized":"2024-09-10T10:55:49.864565+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:38.621928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":416,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960208+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:55:49.960208+0000","last_peered":"2024-09-10T10:55:49.960208+0000","last_clean":"2024-09-10T10:55:49.960208+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:55:49.960208+0000","last_undegraded":"2024-09-10T10:55:49.960208+0000","last_fullsized":"2024-09-10T10:55:49.960208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1941504,"data_stored":978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28936,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342904,"statfs":{"total":96636764160,"available":96607133696,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1937408,"data_stored":977391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1941504,"data_stored":977469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41972,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329868,"statfs":{"total":96636764160,"available":96593784832,"internally_reserved":0,"allocated":2523136,"data_stored":1561125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1937408,"data_stored":977939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":2502656,"data_stored":1553866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:55:53.534 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.689+0000 7fbd17800640 1 Processor -- start 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.690+0000 7fbd17800640 1 -- start start 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd17800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 0x7fbd1015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd17800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd10072700 0x7fbd1015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd17800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd10146140 con 0x7fbd10072320 2024-09-10T10:55:53.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd17800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd100cd740 con 0x7fbd10072700 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd17800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd100cda20 con 0x7fbd10072de0 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd15800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd10072700 0x7fbd1015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd14e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbd10072320 0x7fbd100cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47284/0 (socket says 172.21.15.28:47284) 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.691+0000 7fbd14e00640 1 -- 172.21.15.28:0/3985860073 learned_addr learned my addr 172.21.15.28:0/3985860073 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd0fe00640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 0x7fbd1015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd15800640 1 -- 172.21.15.28:0/3985860073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 msgr2=0x7fbd1015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd15800640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 0x7fbd1015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd15800640 1 -- 172.21.15.28:0/3985860073 >> v1:172.21.15.28:6789/0 conn(0x7fbd10072320 legacy=0x7fbd100cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd15800640 1 -- 172.21.15.28:0/3985860073 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd100cdd00 con 0x7fbd10072700 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd0fe00640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 0x7fbd1015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.692+0000 7fbd0f400640 1 -- 172.21.15.28:0/3985860073 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755130650 0 0) 0x7fbd10146140 con 0x7fbd10072320 2024-09-10T10:55:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd15800640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072700 0x7fbd1015d720 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009d90 tx=0x7fbd0802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b237e77c1fbbd3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd0f400640 1 -- 172.21.15.28:0/3985860073 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08031600 con 0x7fbd10072700 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd0f400640 1 -- 172.21.15.28:0/3985860073 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd080317a0 con 0x7fbd10072700 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd0f400640 1 -- 172.21.15.28:0/3985860073 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08031b70 con 0x7fbd10072700 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd17800640 1 -- 172.21.15.28:0/3985860073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072700 msgr2=0x7fbd1015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.693+0000 7fbd17800640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072700 0x7fbd1015d720 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009d90 tx=0x7fbd0802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 -- 172.21.15.28:0/3985860073 shutdown_connections 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072700 0x7fbd1015d720 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 --2- 172.21.15.28:0/3985860073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072de0 0x7fbd1015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 -- 172.21.15.28:0/3985860073 >> 172.21.15.28:0/3985860073 conn(0x7fbd10067d70 msgr2=0x7fbd1006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 -- 172.21.15.28:0/3985860073 shutdown_connections 2024-09-10T10:55:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.694+0000 7fbd17800640 1 -- 172.21.15.28:0/3985860073 wait complete. 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.695+0000 7fbd17800640 1 Processor -- start 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.695+0000 7fbd17800640 1 -- start start 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd10072320 0x7fbd1015bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072700 0x7fbd1015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1015dde0 con 0x7fbd10072de0 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1015dc60 con 0x7fbd10072320 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.696+0000 7fbd17800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd1015df60 con 0x7fbd10072700 2024-09-10T10:55:53.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33342/0 (socket says 172.21.15.28:33342) 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 -- 172.21.15.28:0/2307055605 learned_addr learned my addr 172.21.15.28:0/2307055605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd0fe00640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072700 0x7fbd1015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd14e00640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd10072320 0x7fbd1015bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 -- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072700 msgr2=0x7fbd1015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072700 0x7fbd1015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 -- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd10072320 msgr2=0x7fbd1015bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd10072320 0x7fbd1015bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 -- 172.21.15.28:0/2307055605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd101f7660 con 0x7fbd10072de0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.697+0000 7fbd15800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009d90 tx=0x7fbd08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.698+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08040070 con 0x7fbd10072de0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.698+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd08030be0 con 0x7fbd10072de0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.698+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd101f7900 con 0x7fbd10072de0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.698+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08009450 con 0x7fbd10072de0 2024-09-10T10:55:53.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.698+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101f7ec0 con 0x7fbd10072de0 2024-09-10T10:55:53.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.699+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd1015e390 con 0x7fbd10072de0 2024-09-10T10:55:53.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.700+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd08039040 con 0x7fbd10072de0 2024-09-10T10:55:53.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.701+0000 7fbd0cc00640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 0x7fbcf006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:53.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.701+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7fbd080ac020 con 0x7fbd10072de0 2024-09-10T10:55:53.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.701+0000 7fbd14e00640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 0x7fbcf006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:53.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.702+0000 7fbd14e00640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 0x7fbcf006e2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000f60 tx=0x7fbd04007d50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:53.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.705+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbd0807a1e0 con 0x7fbd10072de0 2024-09-10T10:55:53.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.886+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd1015b920 con 0x7fbcf006be30 2024-09-10T10:55:53.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.893+0000 7fbd0cc00640 1 -- 172.21.15.28:0/2307055605 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7fbd04004c70 con 0x7fbcf006be30 2024-09-10T10:55:53.896 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:53.898 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:55:53.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 msgr2=0x7fbcf006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 0x7fbcf006e2f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000f60 tx=0x7fbd04007d50 comp rx=0 tx=0).stop 2024-09-10T10:55:53.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 msgr2=0x7fbd101e32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:53.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009d90 tx=0x7fbd08004a40 comp rx=0 tx=0).stop 2024-09-10T10:55:53.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 shutdown_connections 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbcf006be30 0x7fbcf006e2f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd10072de0 0x7fbd101e32b0 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd10072700 0x7fbd1015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 --2- 172.21.15.28:0/2307055605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd10072320 0x7fbd1015bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 >> 172.21.15.28:0/2307055605 conn(0x7fbd10067d70 msgr2=0x7fbd100b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 shutdown_connections 2024-09-10T10:55:53.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:53.897+0000 7fbd17800640 1 -- 172.21.15.28:0/2307055605 wait complete. 2024-09-10T10:55:53.914 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-10T10:55:52.674111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290788,"kb_used_data":16284,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683932,"statfs":{"total":773094113280,"available":772796346368,"internally_reserved":0,"allocated":16674816,"data_stored":8983737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.852121"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865805+0000","last_change":"2024-09-10T10:55:48.867834+0000","last_active":"2024-09-10T10:55:49.865805+0000","last_peered":"2024-09-10T10:55:49.865805+0000","last_clean":"2024-09-10T10:55:49.865805+0000","last_became_active":"2024-09-10T10:55:48.867563+0000","last_became_peered":"2024-09-10T10:55:48.867563+0000","last_unstale":"2024-09-10T10:55:49.865805+0000","last_undegraded":"2024-09-10T10:55:49.865805+0000","last_fullsized":"2024-09-10T10:55:49.865805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:00.429383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947062+0000","last_change":"2024-09-10T10:55:48.855439+0000","last_active":"2024-09-10T10:55:49.947062+0000","last_peered":"2024-09-10T10:55:49.947062+0000","last_clean":"2024-09-10T10:55:49.947062+0000","last_became_active":"2024-09-10T10:55:48.855159+0000","last_became_peered":"2024-09-10T10:55:48.855159+0000","last_unstale":"2024-09-10T10:55:49.947062+0000","last_undegraded":"2024-09-10T10:55:49.947062+0000","last_fullsized":"2024-09-10T10:55:49.947062+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:12.344418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.834144+0000","last_change":"2024-09-10T10:55:47.834144+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.834144+0000","last_undegraded":"2024-09-10T10:55:47.834144+0000","last_fullsized":"2024-09-10T10:55:47.834144+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864724+0000","last_change":"2024-09-10T10:55:48.853911+0000","last_active":"2024-09-10T10:55:49.864724+0000","last_peered":"2024-09-10T10:55:49.864724+0000","last_clean":"2024-09-10T10:55:49.864724+0000","last_became_active":"2024-09-10T10:55:48.853601+0000","last_became_peered":"2024-09-10T10:55:48.853601+0000","last_unstale":"2024-09-10T10:55:49.864724+0000","last_undegraded":"2024-09-10T10:55:49.864724+0000","last_fullsized":"2024-09-10T10:55:49.864724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:51.207539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865887+0000","last_change":"2024-09-10T10:55:48.860012+0000","last_active":"2024-09-10T10:55:49.865887+0000","last_peered":"2024-09-10T10:55:49.865887+0000","last_clean":"2024-09-10T10:55:49.865887+0000","last_became_active":"2024-09-10T10:55:48.859059+0000","last_became_peered":"2024-09-10T10:55:48.859059+0000","last_unstale":"2024-09-10T10:55:49.865887+0000","last_undegraded":"2024-09-10T10:55:49.865887+0000","last_fullsized":"2024-09-10T10:55:49.865887+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.465847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872011+0000","last_change":"2024-09-10T10:55:48.872626+0000","last_active":"2024-09-10T10:55:49.872011+0000","last_peered":"2024-09-10T10:55:49.872011+0000","last_clean":"2024-09-10T10:55:49.872011+0000","last_became_active":"2024-09-10T10:55:48.872334+0000","last_became_peered":"2024-09-10T10:55:48.872334+0000","last_unstale":"2024-09-10T10:55:49.872011+0000","last_undegraded":"2024-09-10T10:55:49.872011+0000","last_fullsized":"2024-09-10T10:55:49.872011+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:51.489910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871869+0000","last_change":"2024-09-10T10:55:48.856346+0000","last_active":"2024-09-10T10:55:49.871869+0000","last_peered":"2024-09-10T10:55:49.871869+0000","last_clean":"2024-09-10T10:55:49.871869+0000","last_became_active":"2024-09-10T10:55:48.856069+0000","last_became_peered":"2024-09-10T10:55:48.856069+0000","last_unstale":"2024-09-10T10:55:49.871869+0000","last_undegraded":"2024-09-10T10:55:49.871869+0000","last_fullsized":"2024-09-10T10:55:49.871869+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961154+0000","last_change":"2024-09-10T10:55:48.876456+0000","last_active":"2024-09-10T10:55:49.961154+0000","last_peered":"2024-09-10T10:55:49.961154+0000","last_clean":"2024-09-10T10:55:49.961154+0000","last_became_active":"2024-09-10T10:55:48.876149+0000","last_became_peered":"2024-09-10T10:55:48.876149+0000","last_unstale":"2024-09-10T10:55:49.961154+0000","last_undegraded":"2024-09-10T10:55:49.961154+0000","last_fullsized":"2024-09-10T10:55:49.961154+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947262+0000","last_change":"2024-09-10T10:55:48.863492+0000","last_active":"2024-09-10T10:55:49.947262+0000","last_peered":"2024-09-10T10:55:49.947262+0000","last_clean":"2024-09-10T10:55:49.947262+0000","last_became_active":"2024-09-10T10:55:48.863048+0000","last_became_peered":"2024-09-10T10:55:48.863048+0000","last_unstale":"2024-09-10T10:55:49.947262+0000","last_undegraded":"2024-09-10T10:55:49.947262+0000","last_fullsized":"2024-09-10T10:55:49.947262+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:30.871617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961998+0000","last_change":"2024-09-10T10:55:48.866566+0000","last_active":"2024-09-10T10:55:49.961998+0000","last_peered":"2024-09-10T10:55:49.961998+0000","last_clean":"2024-09-10T10:55:49.961998+0000","last_became_active":"2024-09-10T10:55:48.866256+0000","last_became_peered":"2024-09-10T10:55:48.866256+0000","last_unstale":"2024-09-10T10:55:49.961998+0000","last_undegraded":"2024-09-10T10:55:49.961998+0000","last_fullsized":"2024-09-10T10:55:49.961998+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:16.476043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862921+0000","last_change":"2024-09-10T10:55:48.853498+0000","last_active":"2024-09-10T10:55:49.862921+0000","last_peered":"2024-09-10T10:55:49.862921+0000","last_clean":"2024-09-10T10:55:49.862921+0000","last_became_active":"2024-09-10T10:55:48.853209+0000","last_became_peered":"2024-09-10T10:55:48.853209+0000","last_unstale":"2024-09-10T10:55:49.862921+0000","last_undegraded":"2024-09-10T10:55:49.862921+0000","last_fullsized":"2024-09-10T10:55:49.862921+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:37.984075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872273+0000","last_change":"2024-09-10T10:55:48.865380+0000","last_active":"2024-09-10T10:55:49.872273+0000","last_peered":"2024-09-10T10:55:49.872273+0000","last_clean":"2024-09-10T10:55:49.872273+0000","last_became_active":"2024-09-10T10:55:48.865102+0000","last_became_peered":"2024-09-10T10:55:48.865102+0000","last_unstale":"2024-09-10T10:55:49.872273+0000","last_undegraded":"2024-09-10T10:55:49.872273+0000","last_fullsized":"2024-09-10T10:55:49.872273+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:39.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865722+0000","last_change":"2024-09-10T10:55:48.863226+0000","last_active":"2024-09-10T10:55:49.865722+0000","last_peered":"2024-09-10T10:55:49.865722+0000","last_clean":"2024-09-10T10:55:49.865722+0000","last_became_active":"2024-09-10T10:55:48.862762+0000","last_became_peered":"2024-09-10T10:55:48.862762+0000","last_unstale":"2024-09-10T10:55:49.865722+0000","last_undegraded":"2024-09-10T10:55:49.865722+0000","last_fullsized":"2024-09-10T10:55:49.865722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:24.148383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872189+0000","last_change":"2024-09-10T10:55:48.864947+0000","last_active":"2024-09-10T10:55:49.872189+0000","last_peered":"2024-09-10T10:55:49.872189+0000","last_clean":"2024-09-10T10:55:49.872189+0000","last_became_active":"2024-09-10T10:55:48.864649+0000","last_became_peered":"2024-09-10T10:55:48.864649+0000","last_unstale":"2024-09-10T10:55:49.872189+0000","last_undegraded":"2024-09-10T10:55:49.872189+0000","last_fullsized":"2024-09-10T10:55:49.872189+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:53.436925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946934+0000","last_change":"2024-09-10T10:55:48.862215+0000","last_active":"2024-09-10T10:55:49.946934+0000","last_peered":"2024-09-10T10:55:49.946934+0000","last_clean":"2024-09-10T10:55:49.946934+0000","last_became_active":"2024-09-10T10:55:48.861706+0000","last_became_peered":"2024-09-10T10:55:48.861706+0000","last_unstale":"2024-09-10T10:55:49.946934+0000","last_undegraded":"2024-09-10T10:55:49.946934+0000","last_fullsized":"2024-09-10T10:55:49.946934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:24.201115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960292+0000","last_change":"2024-09-10T10:55:48.879362+0000","last_active":"2024-09-10T10:55:49.960292+0000","last_peered":"2024-09-10T10:55:49.960292+0000","last_clean":"2024-09-10T10:55:49.960292+0000","last_became_active":"2024-09-10T10:55:48.875054+0000","last_became_peered":"2024-09-10T10:55:48.875054+0000","last_unstale":"2024-09-10T10:55:49.960292+0000","last_undegraded":"2024-09-10T10:55:49.960292+0000","last_fullsized":"2024-09-10T10:55:49.960292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.836013+0000","last_change":"2024-09-10T10:55:47.836013+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.836013+0000","last_undegraded":"2024-09-10T10:55:47.836013+0000","last_fullsized":"2024-09-10T10:55:47.836013+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947217+0000","last_change":"2024-09-10T10:55:48.861625+0000","last_active":"2024-09-10T10:55:49.947217+0000","last_peered":"2024-09-10T10:55:49.947217+0000","last_clean":"2024-09-10T10:55:49.947217+0000","last_became_active":"2024-09-10T10:55:48.860626+0000","last_became_peered":"2024-09-10T10:55:48.860626+0000","last_unstale":"2024-09-10T10:55:49.947217+0000","last_undegraded":"2024-09-10T10:55:49.947217+0000","last_fullsized":"2024-09-10T10:55:49.947217+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:19.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865795+0000","last_change":"2024-09-10T10:55:48.855667+0000","last_active":"2024-09-10T10:55:49.865795+0000","last_peered":"2024-09-10T10:55:49.865795+0000","last_clean":"2024-09-10T10:55:49.865795+0000","last_became_active":"2024-09-10T10:55:48.854331+0000","last_became_peered":"2024-09-10T10:55:48.854331+0000","last_unstale":"2024-09-10T10:55:49.865795+0000","last_undegraded":"2024-09-10T10:55:49.865795+0000","last_fullsized":"2024-09-10T10:55:49.865795+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:11.006767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960683+0000","last_change":"2024-09-10T10:55:48.874514+0000","last_active":"2024-09-10T10:55:49.960683+0000","last_peered":"2024-09-10T10:55:49.960683+0000","last_clean":"2024-09-10T10:55:49.960683+0000","last_became_active":"2024-09-10T10:55:48.874243+0000","last_became_peered":"2024-09-10T10:55:48.874243+0000","last_unstale":"2024-09-10T10:55:49.960683+0000","last_undegraded":"2024-09-10T10:55:49.960683+0000","last_fullsized":"2024-09-10T10:55:49.960683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.639323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835270+0000","last_change":"2024-09-10T10:55:47.835270+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835270+0000","last_undegraded":"2024-09-10T10:55:47.835270+0000","last_fullsized":"2024-09-10T10:55:47.835270+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960995+0000","last_change":"2024-09-10T10:55:48.878105+0000","last_active":"2024-09-10T10:55:49.960995+0000","last_peered":"2024-09-10T10:55:49.960995+0000","last_clean":"2024-09-10T10:55:49.960995+0000","last_became_active":"2024-09-10T10:55:48.876598+0000","last_became_peered":"2024-09-10T10:55:48.876598+0000","last_unstale":"2024-09-10T10:55:49.960995+0000","last_undegraded":"2024-09-10T10:55:49.960995+0000","last_fullsized":"2024-09-10T10:55:49.960995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:47.718665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961652+0000","last_change":"2024-09-10T10:55:48.876273+0000","last_active":"2024-09-10T10:55:49.961652+0000","last_peered":"2024-09-10T10:55:49.961652+0000","last_clean":"2024-09-10T10:55:49.961652+0000","last_became_active":"2024-09-10T10:55:48.875814+0000","last_became_peered":"2024-09-10T10:55:48.875814+0000","last_unstale":"2024-09-10T10:55:49.961652+0000","last_undegraded":"2024-09-10T10:55:49.961652+0000","last_fullsized":"2024-09-10T10:55:49.961652+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:23.371793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961329+0000","last_change":"2024-09-10T10:55:48.869413+0000","last_active":"2024-09-10T10:55:49.961329+0000","last_peered":"2024-09-10T10:55:49.961329+0000","last_clean":"2024-09-10T10:55:49.961329+0000","last_became_active":"2024-09-10T10:55:48.869149+0000","last_became_peered":"2024-09-10T10:55:48.869149+0000","last_unstale":"2024-09-10T10:55:49.961329+0000","last_undegraded":"2024-09-10T10:55:49.961329+0000","last_fullsized":"2024-09-10T10:55:49.961329+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:38.451801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960863+0000","last_change":"2024-09-10T10:55:48.866804+0000","last_active":"2024-09-10T10:55:49.960863+0000","last_peered":"2024-09-10T10:55:49.960863+0000","last_clean":"2024-09-10T10:55:49.960863+0000","last_became_active":"2024-09-10T10:55:48.866480+0000","last_became_peered":"2024-09-10T10:55:48.866480+0000","last_unstale":"2024-09-10T10:55:49.960863+0000","last_undegraded":"2024-09-10T10:55:49.960863+0000","last_fullsized":"2024-09-10T10:55:49.960863+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:31.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960874+0000","last_change":"2024-09-10T10:55:48.872545+0000","last_active":"2024-09-10T10:55:49.960874+0000","last_peered":"2024-09-10T10:55:49.960874+0000","last_clean":"2024-09-10T10:55:49.960874+0000","last_became_active":"2024-09-10T10:55:48.872150+0000","last_became_peered":"2024-09-10T10:55:48.872150+0000","last_unstale":"2024-09-10T10:55:49.960874+0000","last_undegraded":"2024-09-10T10:55:49.960874+0000","last_fullsized":"2024-09-10T10:55:49.960874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960690+0000","last_change":"2024-09-10T10:55:48.873569+0000","last_active":"2024-09-10T10:55:49.960690+0000","last_peered":"2024-09-10T10:55:49.960690+0000","last_clean":"2024-09-10T10:55:49.960690+0000","last_became_active":"2024-09-10T10:55:48.872446+0000","last_became_peered":"2024-09-10T10:55:48.872446+0000","last_unstale":"2024-09-10T10:55:49.960690+0000","last_undegraded":"2024-09-10T10:55:49.960690+0000","last_fullsized":"2024-09-10T10:55:49.960690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:58.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872284+0000","last_change":"2024-09-10T10:55:48.866629+0000","last_active":"2024-09-10T10:55:49.872284+0000","last_peered":"2024-09-10T10:55:49.872284+0000","last_clean":"2024-09-10T10:55:49.872284+0000","last_became_active":"2024-09-10T10:55:48.866351+0000","last_became_peered":"2024-09-10T10:55:48.866351+0000","last_unstale":"2024-09-10T10:55:49.872284+0000","last_undegraded":"2024-09-10T10:55:49.872284+0000","last_fullsized":"2024-09-10T10:55:49.872284+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:20.523256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961962+0000","last_change":"2024-09-10T10:55:48.874303+0000","last_active":"2024-09-10T10:55:49.961962+0000","last_peered":"2024-09-10T10:55:49.961962+0000","last_clean":"2024-09-10T10:55:49.961962+0000","last_became_active":"2024-09-10T10:55:48.873605+0000","last_became_peered":"2024-09-10T10:55:48.873605+0000","last_unstale":"2024-09-10T10:55:49.961962+0000","last_undegraded":"2024-09-10T10:55:49.961962+0000","last_fullsized":"2024-09-10T10:55:49.961962+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:43.380426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862682+0000","last_change":"2024-09-10T10:55:48.851514+0000","last_active":"2024-09-10T10:55:49.862682+0000","last_peered":"2024-09-10T10:55:49.862682+0000","last_clean":"2024-09-10T10:55:49.862682+0000","last_became_active":"2024-09-10T10:55:48.851231+0000","last_became_peered":"2024-09-10T10:55:48.851231+0000","last_unstale":"2024-09-10T10:55:49.862682+0000","last_undegraded":"2024-09-10T10:55:49.862682+0000","last_fullsized":"2024-09-10T10:55:49.862682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:21.797943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871910+0000","last_change":"2024-09-10T10:55:48.860423+0000","last_active":"2024-09-10T10:55:49.871910+0000","last_peered":"2024-09-10T10:55:49.871910+0000","last_clean":"2024-09-10T10:55:49.871910+0000","last_became_active":"2024-09-10T10:55:48.860101+0000","last_became_peered":"2024-09-10T10:55:48.860101+0000","last_unstale":"2024-09-10T10:55:49.871910+0000","last_undegraded":"2024-09-10T10:55:49.871910+0000","last_fullsized":"2024-09-10T10:55:49.871910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:15.280350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960768+0000","last_change":"2024-09-10T10:55:48.861221+0000","last_active":"2024-09-10T10:55:49.960768+0000","last_peered":"2024-09-10T10:55:49.960768+0000","last_clean":"2024-09-10T10:55:49.960768+0000","last_became_active":"2024-09-10T10:55:48.860949+0000","last_became_peered":"2024-09-10T10:55:48.860949+0000","last_unstale":"2024-09-10T10:55:49.960768+0000","last_undegraded":"2024-09-10T10:55:49.960768+0000","last_fullsized":"2024-09-10T10:55:49.960768+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:07.147500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.833598+0000","last_change":"2024-09-10T10:55:47.833598+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.833598+0000","last_undegraded":"2024-09-10T10:55:47.833598+0000","last_fullsized":"2024-09-10T10:55:47.833598+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961416+0000","last_change":"2024-09-10T10:55:47.855582+0000","last_active":"2024-09-10T10:55:49.961416+0000","last_peered":"2024-09-10T10:55:49.961416+0000","last_clean":"2024-09-10T10:55:49.961416+0000","last_became_active":"2024-09-10T10:55:47.855156+0000","last_became_peered":"2024-09-10T10:55:47.855156+0000","last_unstale":"2024-09-10T10:55:49.961416+0000","last_undegraded":"2024-09-10T10:55:49.961416+0000","last_fullsized":"2024-09-10T10:55:49.961416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:30.174489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961811+0000","last_change":"2024-09-10T10:55:48.876356+0000","last_active":"2024-09-10T10:55:49.961811+0000","last_peered":"2024-09-10T10:55:49.961811+0000","last_clean":"2024-09-10T10:55:49.961811+0000","last_became_active":"2024-09-10T10:55:48.875734+0000","last_became_peered":"2024-09-10T10:55:48.875734+0000","last_unstale":"2024-09-10T10:55:49.961811+0000","last_undegraded":"2024-09-10T10:55:49.961811+0000","last_fullsized":"2024-09-10T10:55:49.961811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:45.252064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851021+0000","last_change":"2024-09-10T10:55:47.846423+0000","last_active":"2024-09-10T10:55:47.851021+0000","last_peered":"2024-09-10T10:55:47.851021+0000","last_clean":"2024-09-10T10:55:47.851021+0000","last_became_active":"2024-09-10T10:55:47.842576+0000","last_became_peered":"2024-09-10T10:55:47.842576+0000","last_unstale":"2024-09-10T10:55:47.851021+0000","last_undegraded":"2024-09-10T10:55:47.851021+0000","last_fullsized":"2024-09-10T10:55:47.851021+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:26.847568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946997+0000","last_change":"2024-09-10T10:55:48.864892+0000","last_active":"2024-09-10T10:55:49.946997+0000","last_peered":"2024-09-10T10:55:49.946997+0000","last_clean":"2024-09-10T10:55:49.946997+0000","last_became_active":"2024-09-10T10:55:48.864545+0000","last_became_peered":"2024-09-10T10:55:48.864545+0000","last_unstale":"2024-09-10T10:55:49.946997+0000","last_undegraded":"2024-09-10T10:55:49.946997+0000","last_fullsized":"2024-09-10T10:55:49.946997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:45.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947156+0000","last_change":"2024-09-10T10:55:47.848829+0000","last_active":"2024-09-10T10:55:49.947156+0000","last_peered":"2024-09-10T10:55:49.947156+0000","last_clean":"2024-09-10T10:55:49.947156+0000","last_became_active":"2024-09-10T10:55:47.848545+0000","last_became_peered":"2024-09-10T10:55:47.848545+0000","last_unstale":"2024-09-10T10:55:49.947156+0000","last_undegraded":"2024-09-10T10:55:49.947156+0000","last_fullsized":"2024-09-10T10:55:49.947156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:26.415458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862744+0000","last_change":"2024-09-10T10:55:48.853317+0000","last_active":"2024-09-10T10:55:49.862744+0000","last_peered":"2024-09-10T10:55:49.862744+0000","last_clean":"2024-09-10T10:55:49.862744+0000","last_became_active":"2024-09-10T10:55:48.853020+0000","last_became_peered":"2024-09-10T10:55:48.853020+0000","last_unstale":"2024-09-10T10:55:49.862744+0000","last_undegraded":"2024-09-10T10:55:49.862744+0000","last_fullsized":"2024-09-10T10:55:49.862744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:20.610135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961781+0000","last_change":"2024-09-10T10:55:47.847504+0000","last_active":"2024-09-10T10:55:49.961781+0000","last_peered":"2024-09-10T10:55:49.961781+0000","last_clean":"2024-09-10T10:55:49.961781+0000","last_became_active":"2024-09-10T10:55:47.847070+0000","last_became_peered":"2024-09-10T10:55:47.847070+0000","last_unstale":"2024-09-10T10:55:49.961781+0000","last_undegraded":"2024-09-10T10:55:49.961781+0000","last_fullsized":"2024-09-10T10:55:49.961781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:04.988364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865922+0000","last_change":"2024-09-10T10:55:48.867784+0000","last_active":"2024-09-10T10:55:49.865922+0000","last_peered":"2024-09-10T10:55:49.865922+0000","last_clean":"2024-09-10T10:55:49.865922+0000","last_became_active":"2024-09-10T10:55:48.867508+0000","last_became_peered":"2024-09-10T10:55:48.867508+0000","last_unstale":"2024-09-10T10:55:49.865922+0000","last_undegraded":"2024-09-10T10:55:49.865922+0000","last_fullsized":"2024-09-10T10:55:49.865922+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:48.476999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862859+0000","last_change":"2024-09-10T10:55:47.845137+0000","last_active":"2024-09-10T10:55:49.862859+0000","last_peered":"2024-09-10T10:55:49.862859+0000","last_clean":"2024-09-10T10:55:49.862859+0000","last_became_active":"2024-09-10T10:55:47.844834+0000","last_became_peered":"2024-09-10T10:55:47.844834+0000","last_unstale":"2024-09-10T10:55:49.862859+0000","last_undegraded":"2024-09-10T10:55:49.862859+0000","last_fullsized":"2024-09-10T10:55:49.862859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:32.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865918+0000","last_change":"2024-09-10T10:55:48.864028+0000","last_active":"2024-09-10T10:55:49.865918+0000","last_peered":"2024-09-10T10:55:49.865918+0000","last_clean":"2024-09-10T10:55:49.865918+0000","last_became_active":"2024-09-10T10:55:48.863229+0000","last_became_peered":"2024-09-10T10:55:48.863229+0000","last_unstale":"2024-09-10T10:55:49.865918+0000","last_undegraded":"2024-09-10T10:55:49.865918+0000","last_fullsized":"2024-09-10T10:55:49.865918+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:18.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960308+0000","last_change":"2024-09-10T10:55:47.843083+0000","last_active":"2024-09-10T10:55:49.960308+0000","last_peered":"2024-09-10T10:55:49.960308+0000","last_clean":"2024-09-10T10:55:49.960308+0000","last_became_active":"2024-09-10T10:55:47.839631+0000","last_became_peered":"2024-09-10T10:55:47.839631+0000","last_unstale":"2024-09-10T10:55:49.960308+0000","last_undegraded":"2024-09-10T10:55:49.960308+0000","last_fullsized":"2024-09-10T10:55:49.960308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:18.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.863052+0000","last_change":"2024-09-10T10:55:48.864786+0000","last_active":"2024-09-10T10:55:49.863052+0000","last_peered":"2024-09-10T10:55:49.863052+0000","last_clean":"2024-09-10T10:55:49.863052+0000","last_became_active":"2024-09-10T10:55:48.864460+0000","last_became_peered":"2024-09-10T10:55:48.864460+0000","last_unstale":"2024-09-10T10:55:49.863052+0000","last_undegraded":"2024-09-10T10:55:49.863052+0000","last_fullsized":"2024-09-10T10:55:49.863052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:05.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960378+0000","last_change":"2024-09-10T10:55:47.846337+0000","last_active":"2024-09-10T10:55:49.960378+0000","last_peered":"2024-09-10T10:55:49.960378+0000","last_clean":"2024-09-10T10:55:49.960378+0000","last_became_active":"2024-09-10T10:55:47.845507+0000","last_became_peered":"2024-09-10T10:55:47.845507+0000","last_unstale":"2024-09-10T10:55:49.960378+0000","last_undegraded":"2024-09-10T10:55:49.960378+0000","last_fullsized":"2024-09-10T10:55:49.960378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:11.338379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865877+0000","last_change":"2024-09-10T10:55:48.864808+0000","last_active":"2024-09-10T10:55:49.865877+0000","last_peered":"2024-09-10T10:55:49.865877+0000","last_clean":"2024-09-10T10:55:49.865877+0000","last_became_active":"2024-09-10T10:55:48.862818+0000","last_became_peered":"2024-09-10T10:55:48.862818+0000","last_unstale":"2024-09-10T10:55:49.865877+0000","last_undegraded":"2024-09-10T10:55:49.865877+0000","last_fullsized":"2024-09-10T10:55:49.865877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:44.808359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:47.853115+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:47.852824+0000","last_became_peered":"2024-09-10T10:55:47.852824+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:57.389591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864710+0000","last_change":"2024-09-10T10:55:48.865231+0000","last_active":"2024-09-10T10:55:49.864710+0000","last_peered":"2024-09-10T10:55:49.864710+0000","last_clean":"2024-09-10T10:55:49.864710+0000","last_became_active":"2024-09-10T10:55:48.862955+0000","last_became_peered":"2024-09-10T10:55:48.862955+0000","last_unstale":"2024-09-10T10:55:49.864710+0000","last_undegraded":"2024-09-10T10:55:49.864710+0000","last_fullsized":"2024-09-10T10:55:49.864710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.358498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872584+0000","last_change":"2024-09-10T10:55:47.843342+0000","last_active":"2024-09-10T10:55:49.872584+0000","last_peered":"2024-09-10T10:55:49.872584+0000","last_clean":"2024-09-10T10:55:49.872584+0000","last_became_active":"2024-09-10T10:55:47.842788+0000","last_became_peered":"2024-09-10T10:55:47.842788+0000","last_unstale":"2024-09-10T10:55:49.872584+0000","last_undegraded":"2024-09-10T10:55:49.872584+0000","last_fullsized":"2024-09-10T10:55:49.872584+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:00.612235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865943+0000","last_change":"2024-09-10T10:55:48.869535+0000","last_active":"2024-09-10T10:55:49.865943+0000","last_peered":"2024-09-10T10:55:49.865943+0000","last_clean":"2024-09-10T10:55:49.865943+0000","last_became_active":"2024-09-10T10:55:48.869284+0000","last_became_peered":"2024-09-10T10:55:48.869284+0000","last_unstale":"2024-09-10T10:55:49.865943+0000","last_undegraded":"2024-09-10T10:55:49.865943+0000","last_fullsized":"2024-09-10T10:55:49.865943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:44.081502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961991+0000","last_change":"2024-09-10T10:55:47.851219+0000","last_active":"2024-09-10T10:55:49.961991+0000","last_peered":"2024-09-10T10:55:49.961991+0000","last_clean":"2024-09-10T10:55:49.961991+0000","last_became_active":"2024-09-10T10:55:47.849193+0000","last_became_peered":"2024-09-10T10:55:47.849193+0000","last_unstale":"2024-09-10T10:55:49.961991+0000","last_undegraded":"2024-09-10T10:55:49.961991+0000","last_fullsized":"2024-09-10T10:55:49.961991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:50.483870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961814+0000","last_change":"2024-09-10T10:55:48.868003+0000","last_active":"2024-09-10T10:55:49.961814+0000","last_peered":"2024-09-10T10:55:49.961814+0000","last_clean":"2024-09-10T10:55:49.961814+0000","last_became_active":"2024-09-10T10:55:48.867694+0000","last_became_peered":"2024-09-10T10:55:48.867694+0000","last_unstale":"2024-09-10T10:55:49.961814+0000","last_undegraded":"2024-09-10T10:55:49.961814+0000","last_fullsized":"2024-09-10T10:55:49.961814+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:37.123756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872366+0000","last_change":"2024-09-10T10:55:47.847591+0000","last_active":"2024-09-10T10:55:49.872366+0000","last_peered":"2024-09-10T10:55:49.872366+0000","last_clean":"2024-09-10T10:55:49.872366+0000","last_became_active":"2024-09-10T10:55:47.847299+0000","last_became_peered":"2024-09-10T10:55:47.847299+0000","last_unstale":"2024-09-10T10:55:49.872366+0000","last_undegraded":"2024-09-10T10:55:49.872366+0000","last_fullsized":"2024-09-10T10:55:49.872366+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.943592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961027+0000","last_change":"2024-09-10T10:55:48.875294+0000","last_active":"2024-09-10T10:55:49.961027+0000","last_peered":"2024-09-10T10:55:49.961027+0000","last_clean":"2024-09-10T10:55:49.961027+0000","last_became_active":"2024-09-10T10:55:48.873851+0000","last_became_peered":"2024-09-10T10:55:48.873851+0000","last_unstale":"2024-09-10T10:55:49.961027+0000","last_undegraded":"2024-09-10T10:55:49.961027+0000","last_fullsized":"2024-09-10T10:55:49.961027+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:07.268035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862812+0000","last_change":"2024-09-10T10:55:47.850868+0000","last_active":"2024-09-10T10:55:49.862812+0000","last_peered":"2024-09-10T10:55:49.862812+0000","last_clean":"2024-09-10T10:55:49.862812+0000","last_became_active":"2024-09-10T10:55:47.850457+0000","last_became_peered":"2024-09-10T10:55:47.850457+0000","last_unstale":"2024-09-10T10:55:49.862812+0000","last_undegraded":"2024-09-10T10:55:49.862812+0000","last_fullsized":"2024-09-10T10:55:49.862812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:45.564987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947047+0000","last_change":"2024-09-10T10:55:48.861530+0000","last_active":"2024-09-10T10:55:49.947047+0000","last_peered":"2024-09-10T10:55:49.947047+0000","last_clean":"2024-09-10T10:55:49.947047+0000","last_became_active":"2024-09-10T10:55:48.860648+0000","last_became_peered":"2024-09-10T10:55:48.860648+0000","last_unstale":"2024-09-10T10:55:49.947047+0000","last_undegraded":"2024-09-10T10:55:49.947047+0000","last_fullsized":"2024-09-10T10:55:49.947047+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:36.041255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865960+0000","last_change":"2024-09-10T10:55:47.843477+0000","last_active":"2024-09-10T10:55:49.865960+0000","last_peered":"2024-09-10T10:55:49.865960+0000","last_clean":"2024-09-10T10:55:49.865960+0000","last_became_active":"2024-09-10T10:55:47.841371+0000","last_became_peered":"2024-09-10T10:55:47.841371+0000","last_unstale":"2024-09-10T10:55:49.865960+0000","last_undegraded":"2024-09-10T10:55:49.865960+0000","last_fullsized":"2024-09-10T10:55:49.865960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:45.636397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947145+0000","last_change":"2024-09-10T10:55:48.865788+0000","last_active":"2024-09-10T10:55:49.947145+0000","last_peered":"2024-09-10T10:55:49.947145+0000","last_clean":"2024-09-10T10:55:49.947145+0000","last_became_active":"2024-09-10T10:55:48.865486+0000","last_became_peered":"2024-09-10T10:55:48.865486+0000","last_unstale":"2024-09-10T10:55:49.947145+0000","last_undegraded":"2024-09-10T10:55:49.947145+0000","last_fullsized":"2024-09-10T10:55:49.947145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:56.003393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961925+0000","last_change":"2024-09-10T10:55:47.845313+0000","last_active":"2024-09-10T10:55:49.961925+0000","last_peered":"2024-09-10T10:55:49.961925+0000","last_clean":"2024-09-10T10:55:49.961925+0000","last_became_active":"2024-09-10T10:55:47.844999+0000","last_became_peered":"2024-09-10T10:55:47.844999+0000","last_unstale":"2024-09-10T10:55:49.961925+0000","last_undegraded":"2024-09-10T10:55:49.961925+0000","last_fullsized":"2024-09-10T10:55:49.961925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:07.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961293+0000","last_change":"2024-09-10T10:55:48.877412+0000","last_active":"2024-09-10T10:55:49.961293+0000","last_peered":"2024-09-10T10:55:49.961293+0000","last_clean":"2024-09-10T10:55:49.961293+0000","last_became_active":"2024-09-10T10:55:48.876383+0000","last_became_peered":"2024-09-10T10:55:48.876383+0000","last_unstale":"2024-09-10T10:55:49.961293+0000","last_undegraded":"2024-09-10T10:55:49.961293+0000","last_fullsized":"2024-09-10T10:55:49.961293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:41.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865237+0000","last_change":"2024-09-10T10:55:47.834691+0000","last_active":"2024-09-10T10:55:49.865237+0000","last_peered":"2024-09-10T10:55:49.865237+0000","last_clean":"2024-09-10T10:55:49.865237+0000","last_became_active":"2024-09-10T10:55:47.834400+0000","last_became_peered":"2024-09-10T10:55:47.834400+0000","last_unstale":"2024-09-10T10:55:49.865237+0000","last_undegraded":"2024-09-10T10:55:49.865237+0000","last_fullsized":"2024-09-10T10:55:49.865237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:52.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":148,"state":"creating+peering","last_fresh":"2024-09-10T10:55:47.835555+0000","last_change":"2024-09-10T10:55:47.835555+0000","last_active":"2024-09-10T10:55:47.821270+0000","last_peered":"2024-09-10T10:55:47.821270+0000","last_clean":"2024-09-10T10:55:47.821270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T10:55:47.835555+0000","last_undegraded":"2024-09-10T10:55:47.835555+0000","last_fullsized":"2024-09-10T10:55:47.835555+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862623+0000","last_change":"2024-09-10T10:55:47.843176+0000","last_active":"2024-09-10T10:55:49.862623+0000","last_peered":"2024-09-10T10:55:49.862623+0000","last_clean":"2024-09-10T10:55:49.862623+0000","last_became_active":"2024-09-10T10:55:47.842914+0000","last_became_peered":"2024-09-10T10:55:47.842914+0000","last_unstale":"2024-09-10T10:55:49.862623+0000","last_undegraded":"2024-09-10T10:55:49.862623+0000","last_fullsized":"2024-09-10T10:55:49.862623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:46.876235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961424+0000","last_change":"2024-09-10T10:55:48.874049+0000","last_active":"2024-09-10T10:55:49.961424+0000","last_peered":"2024-09-10T10:55:49.961424+0000","last_clean":"2024-09-10T10:55:49.961424+0000","last_became_active":"2024-09-10T10:55:48.873744+0000","last_became_peered":"2024-09-10T10:55:48.873744+0000","last_unstale":"2024-09-10T10:55:49.961424+0000","last_undegraded":"2024-09-10T10:55:49.961424+0000","last_fullsized":"2024-09-10T10:55:49.961424+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:25.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872621+0000","last_change":"2024-09-10T10:55:47.845166+0000","last_active":"2024-09-10T10:55:49.872621+0000","last_peered":"2024-09-10T10:55:49.872621+0000","last_clean":"2024-09-10T10:55:49.872621+0000","last_became_active":"2024-09-10T10:55:47.844675+0000","last_became_peered":"2024-09-10T10:55:47.844675+0000","last_unstale":"2024-09-10T10:55:49.872621+0000","last_undegraded":"2024-09-10T10:55:49.872621+0000","last_fullsized":"2024-09-10T10:55:49.872621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:26.167518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960321+0000","last_change":"2024-09-10T10:55:48.872087+0000","last_active":"2024-09-10T10:55:49.960321+0000","last_peered":"2024-09-10T10:55:49.960321+0000","last_clean":"2024-09-10T10:55:49.960321+0000","last_became_active":"2024-09-10T10:55:48.871832+0000","last_became_peered":"2024-09-10T10:55:48.871832+0000","last_unstale":"2024-09-10T10:55:49.960321+0000","last_undegraded":"2024-09-10T10:55:49.960321+0000","last_fullsized":"2024-09-10T10:55:49.960321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:58.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960717+0000","last_change":"2024-09-10T10:55:47.850066+0000","last_active":"2024-09-10T10:55:49.960717+0000","last_peered":"2024-09-10T10:55:49.960717+0000","last_clean":"2024-09-10T10:55:49.960717+0000","last_became_active":"2024-09-10T10:55:47.849775+0000","last_became_peered":"2024-09-10T10:55:47.849775+0000","last_unstale":"2024-09-10T10:55:49.960717+0000","last_undegraded":"2024-09-10T10:55:49.960717+0000","last_fullsized":"2024-09-10T10:55:49.960717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:29.865577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961193+0000","last_change":"2024-09-10T10:55:48.860009+0000","last_active":"2024-09-10T10:55:49.961193+0000","last_peered":"2024-09-10T10:55:49.961193+0000","last_clean":"2024-09-10T10:55:49.961193+0000","last_became_active":"2024-09-10T10:55:48.859725+0000","last_became_peered":"2024-09-10T10:55:48.859725+0000","last_unstale":"2024-09-10T10:55:49.961193+0000","last_undegraded":"2024-09-10T10:55:49.961193+0000","last_fullsized":"2024-09-10T10:55:49.961193+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:06.818156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872362+0000","last_change":"2024-09-10T10:55:47.846489+0000","last_active":"2024-09-10T10:55:49.872362+0000","last_peered":"2024-09-10T10:55:49.872362+0000","last_clean":"2024-09-10T10:55:49.872362+0000","last_became_active":"2024-09-10T10:55:47.846067+0000","last_became_peered":"2024-09-10T10:55:47.846067+0000","last_unstale":"2024-09-10T10:55:49.872362+0000","last_undegraded":"2024-09-10T10:55:49.872362+0000","last_fullsized":"2024-09-10T10:55:49.872362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:28.004930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865650+0000","last_change":"2024-09-10T10:55:48.857694+0000","last_active":"2024-09-10T10:55:49.865650+0000","last_peered":"2024-09-10T10:55:49.865650+0000","last_clean":"2024-09-10T10:55:49.865650+0000","last_became_active":"2024-09-10T10:55:48.857007+0000","last_became_peered":"2024-09-10T10:55:48.857007+0000","last_unstale":"2024-09-10T10:55:49.865650+0000","last_undegraded":"2024-09-10T10:55:49.865650+0000","last_fullsized":"2024-09-10T10:55:49.865650+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:15.966260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.850937+0000","last_change":"2024-09-10T10:55:47.846071+0000","last_active":"2024-09-10T10:55:47.850937+0000","last_peered":"2024-09-10T10:55:47.850937+0000","last_clean":"2024-09-10T10:55:47.850937+0000","last_became_active":"2024-09-10T10:55:47.845052+0000","last_became_peered":"2024-09-10T10:55:47.845052+0000","last_unstale":"2024-09-10T10:55:47.850937+0000","last_undegraded":"2024-09-10T10:55:47.850937+0000","last_fullsized":"2024-09-10T10:55:47.850937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:06:32.064815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:48.868962+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:48.868602+0000","last_became_peered":"2024-09-10T10:55:48.868602+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:37.194259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.853061+0000","last_change":"2024-09-10T10:55:47.846363+0000","last_active":"2024-09-10T10:55:47.853061+0000","last_peered":"2024-09-10T10:55:47.853061+0000","last_clean":"2024-09-10T10:55:47.853061+0000","last_became_active":"2024-09-10T10:55:47.846001+0000","last_became_peered":"2024-09-10T10:55:47.846001+0000","last_unstale":"2024-09-10T10:55:47.853061+0000","last_undegraded":"2024-09-10T10:55:47.853061+0000","last_fullsized":"2024-09-10T10:55:47.853061+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:59.581099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961231+0000","last_change":"2024-09-10T10:55:48.861960+0000","last_active":"2024-09-10T10:55:49.961231+0000","last_peered":"2024-09-10T10:55:49.961231+0000","last_clean":"2024-09-10T10:55:49.961231+0000","last_became_active":"2024-09-10T10:55:48.861620+0000","last_became_peered":"2024-09-10T10:55:48.861620+0000","last_unstale":"2024-09-10T10:55:49.961231+0000","last_undegraded":"2024-09-10T10:55:49.961231+0000","last_fullsized":"2024-09-10T10:55:49.961231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:27.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872025+0000","last_change":"2024-09-10T10:55:47.837440+0000","last_active":"2024-09-10T10:55:49.872025+0000","last_peered":"2024-09-10T10:55:49.872025+0000","last_clean":"2024-09-10T10:55:49.872025+0000","last_became_active":"2024-09-10T10:55:47.836741+0000","last_became_peered":"2024-09-10T10:55:47.836741+0000","last_unstale":"2024-09-10T10:55:49.872025+0000","last_undegraded":"2024-09-10T10:55:49.872025+0000","last_fullsized":"2024-09-10T10:55:49.872025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:58.662475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865678+0000","last_change":"2024-09-10T10:55:48.867356+0000","last_active":"2024-09-10T10:55:49.865678+0000","last_peered":"2024-09-10T10:55:49.865678+0000","last_clean":"2024-09-10T10:55:49.865678+0000","last_became_active":"2024-09-10T10:55:48.866816+0000","last_became_peered":"2024-09-10T10:55:48.866816+0000","last_unstale":"2024-09-10T10:55:49.865678+0000","last_undegraded":"2024-09-10T10:55:49.865678+0000","last_fullsized":"2024-09-10T10:55:49.865678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:17.379036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961500+0000","last_change":"2024-09-10T10:55:47.845337+0000","last_active":"2024-09-10T10:55:49.961500+0000","last_peered":"2024-09-10T10:55:49.961500+0000","last_clean":"2024-09-10T10:55:49.961500+0000","last_became_active":"2024-09-10T10:55:47.845077+0000","last_became_peered":"2024-09-10T10:55:47.845077+0000","last_unstale":"2024-09-10T10:55:49.961500+0000","last_undegraded":"2024-09-10T10:55:49.961500+0000","last_fullsized":"2024-09-10T10:55:49.961500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:14.274310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960779+0000","last_change":"2024-09-10T10:55:48.864373+0000","last_active":"2024-09-10T10:55:49.960779+0000","last_peered":"2024-09-10T10:55:49.960779+0000","last_clean":"2024-09-10T10:55:49.960779+0000","last_became_active":"2024-09-10T10:55:48.864041+0000","last_became_peered":"2024-09-10T10:55:48.864041+0000","last_unstale":"2024-09-10T10:55:49.960779+0000","last_undegraded":"2024-09-10T10:55:49.960779+0000","last_fullsized":"2024-09-10T10:55:49.960779+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:39.301780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.852322+0000","last_change":"2024-09-10T10:55:47.850713+0000","last_active":"2024-09-10T10:55:47.852322+0000","last_peered":"2024-09-10T10:55:47.852322+0000","last_clean":"2024-09-10T10:55:47.852322+0000","last_became_active":"2024-09-10T10:55:47.849850+0000","last_became_peered":"2024-09-10T10:55:47.849850+0000","last_unstale":"2024-09-10T10:55:47.852322+0000","last_undegraded":"2024-09-10T10:55:47.852322+0000","last_fullsized":"2024-09-10T10:55:47.852322+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:22:40.025775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862722+0000","last_change":"2024-09-10T10:55:48.859546+0000","last_active":"2024-09-10T10:55:49.862722+0000","last_peered":"2024-09-10T10:55:49.862722+0000","last_clean":"2024-09-10T10:55:49.862722+0000","last_became_active":"2024-09-10T10:55:48.859164+0000","last_became_peered":"2024-09-10T10:55:48.859164+0000","last_unstale":"2024-09-10T10:55:49.862722+0000","last_undegraded":"2024-09-10T10:55:49.862722+0000","last_fullsized":"2024-09-10T10:55:49.862722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:29.838709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854948+0000","last_change":"2024-09-10T10:55:47.853293+0000","last_active":"2024-09-10T10:55:47.854948+0000","last_peered":"2024-09-10T10:55:47.854948+0000","last_clean":"2024-09-10T10:55:47.854948+0000","last_became_active":"2024-09-10T10:55:47.852967+0000","last_became_peered":"2024-09-10T10:55:47.852967+0000","last_unstale":"2024-09-10T10:55:47.854948+0000","last_undegraded":"2024-09-10T10:55:47.854948+0000","last_fullsized":"2024-09-10T10:55:47.854948+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:20:50.182209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961880+0000","last_change":"2024-09-10T10:55:48.874714+0000","last_active":"2024-09-10T10:55:49.961880+0000","last_peered":"2024-09-10T10:55:49.961880+0000","last_clean":"2024-09-10T10:55:49.961880+0000","last_became_active":"2024-09-10T10:55:48.874256+0000","last_became_peered":"2024-09-10T10:55:48.874256+0000","last_unstale":"2024-09-10T10:55:49.961880+0000","last_undegraded":"2024-09-10T10:55:49.961880+0000","last_fullsized":"2024-09-10T10:55:49.961880+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:04.561493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862937+0000","last_change":"2024-09-10T10:55:47.852303+0000","last_active":"2024-09-10T10:55:49.862937+0000","last_peered":"2024-09-10T10:55:49.862937+0000","last_clean":"2024-09-10T10:55:49.862937+0000","last_became_active":"2024-09-10T10:55:47.851277+0000","last_became_peered":"2024-09-10T10:55:47.851277+0000","last_unstale":"2024-09-10T10:55:49.862937+0000","last_undegraded":"2024-09-10T10:55:49.862937+0000","last_fullsized":"2024-09-10T10:55:49.862937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:43.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960557+0000","last_change":"2024-09-10T10:55:48.876162+0000","last_active":"2024-09-10T10:55:49.960557+0000","last_peered":"2024-09-10T10:55:49.960557+0000","last_clean":"2024-09-10T10:55:49.960557+0000","last_became_active":"2024-09-10T10:55:48.875834+0000","last_became_peered":"2024-09-10T10:55:48.875834+0000","last_unstale":"2024-09-10T10:55:49.960557+0000","last_undegraded":"2024-09-10T10:55:49.960557+0000","last_fullsized":"2024-09-10T10:55:49.960557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:12.003746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.866001+0000","last_change":"2024-09-10T10:55:47.844106+0000","last_active":"2024-09-10T10:55:49.866001+0000","last_peered":"2024-09-10T10:55:49.866001+0000","last_clean":"2024-09-10T10:55:49.866001+0000","last_became_active":"2024-09-10T10:55:47.843763+0000","last_became_peered":"2024-09-10T10:55:47.843763+0000","last_unstale":"2024-09-10T10:55:49.866001+0000","last_undegraded":"2024-09-10T10:55:49.866001+0000","last_fullsized":"2024-09-10T10:55:49.866001+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:19.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864866+0000","last_change":"2024-09-10T10:55:48.857933+0000","last_active":"2024-09-10T10:55:49.864866+0000","last_peered":"2024-09-10T10:55:49.864866+0000","last_clean":"2024-09-10T10:55:49.864866+0000","last_became_active":"2024-09-10T10:55:48.857414+0000","last_became_peered":"2024-09-10T10:55:48.857414+0000","last_unstale":"2024-09-10T10:55:49.864866+0000","last_undegraded":"2024-09-10T10:55:49.864866+0000","last_fullsized":"2024-09-10T10:55:49.864866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:52.855868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862873+0000","last_change":"2024-09-10T10:55:47.846380+0000","last_active":"2024-09-10T10:55:49.862873+0000","last_peered":"2024-09-10T10:55:49.862873+0000","last_clean":"2024-09-10T10:55:49.862873+0000","last_became_active":"2024-09-10T10:55:47.846123+0000","last_became_peered":"2024-09-10T10:55:47.846123+0000","last_unstale":"2024-09-10T10:55:49.862873+0000","last_undegraded":"2024-09-10T10:55:49.862873+0000","last_fullsized":"2024-09-10T10:55:49.862873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.465904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865669+0000","last_change":"2024-09-10T10:55:48.859593+0000","last_active":"2024-09-10T10:55:49.865669+0000","last_peered":"2024-09-10T10:55:49.865669+0000","last_clean":"2024-09-10T10:55:49.865669+0000","last_became_active":"2024-09-10T10:55:48.859045+0000","last_became_peered":"2024-09-10T10:55:48.859045+0000","last_unstale":"2024-09-10T10:55:49.865669+0000","last_undegraded":"2024-09-10T10:55:49.865669+0000","last_fullsized":"2024-09-10T10:55:49.865669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:39.427524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.854980+0000","last_change":"2024-09-10T10:55:47.853193+0000","last_active":"2024-09-10T10:55:47.854980+0000","last_peered":"2024-09-10T10:55:47.854980+0000","last_clean":"2024-09-10T10:55:47.854980+0000","last_became_active":"2024-09-10T10:55:47.852861+0000","last_became_peered":"2024-09-10T10:55:47.852861+0000","last_unstale":"2024-09-10T10:55:47.854980+0000","last_undegraded":"2024-09-10T10:55:47.854980+0000","last_fullsized":"2024-09-10T10:55:47.854980+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:43.046220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872528+0000","last_change":"2024-09-10T10:55:48.870415+0000","last_active":"2024-09-10T10:55:49.872528+0000","last_peered":"2024-09-10T10:55:49.872528+0000","last_clean":"2024-09-10T10:55:49.872528+0000","last_became_active":"2024-09-10T10:55:48.869966+0000","last_became_peered":"2024-09-10T10:55:48.869966+0000","last_unstale":"2024-09-10T10:55:49.872528+0000","last_undegraded":"2024-09-10T10:55:49.872528+0000","last_fullsized":"2024-09-10T10:55:49.872528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960416+0000","last_change":"2024-09-10T10:55:47.850818+0000","last_active":"2024-09-10T10:55:49.960416+0000","last_peered":"2024-09-10T10:55:49.960416+0000","last_clean":"2024-09-10T10:55:49.960416+0000","last_became_active":"2024-09-10T10:55:47.850541+0000","last_became_peered":"2024-09-10T10:55:47.850541+0000","last_unstale":"2024-09-10T10:55:49.960416+0000","last_undegraded":"2024-09-10T10:55:49.960416+0000","last_fullsized":"2024-09-10T10:55:49.960416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:54.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947271+0000","last_change":"2024-09-10T10:55:48.862694+0000","last_active":"2024-09-10T10:55:49.947271+0000","last_peered":"2024-09-10T10:55:49.947271+0000","last_clean":"2024-09-10T10:55:49.947271+0000","last_became_active":"2024-09-10T10:55:48.862367+0000","last_became_peered":"2024-09-10T10:55:48.862367+0000","last_unstale":"2024-09-10T10:55:49.947271+0000","last_undegraded":"2024-09-10T10:55:49.947271+0000","last_fullsized":"2024-09-10T10:55:49.947271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-09-10T10:55:47.851952+0000","last_change":"2024-09-10T10:55:47.846160+0000","last_active":"2024-09-10T10:55:47.851952+0000","last_peered":"2024-09-10T10:55:47.851952+0000","last_clean":"2024-09-10T10:55:47.851952+0000","last_became_active":"2024-09-10T10:55:47.845758+0000","last_became_peered":"2024-09-10T10:55:47.845758+0000","last_unstale":"2024-09-10T10:55:47.851952+0000","last_undegraded":"2024-09-10T10:55:47.851952+0000","last_fullsized":"2024-09-10T10:55:47.851952+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:13.218467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871975+0000","last_change":"2024-09-10T10:55:48.867075+0000","last_active":"2024-09-10T10:55:49.871975+0000","last_peered":"2024-09-10T10:55:49.871975+0000","last_clean":"2024-09-10T10:55:49.871975+0000","last_became_active":"2024-09-10T10:55:48.859669+0000","last_became_peered":"2024-09-10T10:55:48.859669+0000","last_unstale":"2024-09-10T10:55:49.871975+0000","last_undegraded":"2024-09-10T10:55:49.871975+0000","last_fullsized":"2024-09-10T10:55:49.871975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:05.994404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864565+0000","last_change":"2024-09-10T10:55:47.847392+0000","last_active":"2024-09-10T10:55:49.864565+0000","last_peered":"2024-09-10T10:55:49.864565+0000","last_clean":"2024-09-10T10:55:49.864565+0000","last_became_active":"2024-09-10T10:55:47.847116+0000","last_became_peered":"2024-09-10T10:55:47.847116+0000","last_unstale":"2024-09-10T10:55:49.864565+0000","last_undegraded":"2024-09-10T10:55:49.864565+0000","last_fullsized":"2024-09-10T10:55:49.864565+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:38.621928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":416,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960208+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:55:49.960208+0000","last_peered":"2024-09-10T10:55:49.960208+0000","last_clean":"2024-09-10T10:55:49.960208+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:55:49.960208+0000","last_undegraded":"2024-09-10T10:55:49.960208+0000","last_fullsized":"2024-09-10T10:55:49.960208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33,"ondisk_log_size":33,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1941504,"data_stored":978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28936,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342904,"statfs":{"total":96636764160,"available":96607133696,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1937408,"data_stored":977391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1941504,"data_stored":977469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41972,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329868,"statfs":{"total":96636764160,"available":96593784832,"internally_reserved":0,"allocated":2523136,"data_stored":1561125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1937408,"data_stored":977939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33636,"kb_used_data":2444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338204,"statfs":{"total":96636764160,"available":96602320896,"internally_reserved":0,"allocated":2502656,"data_stored":1553866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:PG 16.3d is not active+clean 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 148, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:55:47.834144+0000', 'last_change': '2024-09-10T10:55:47.834144+0000', 'last_active': '2024-09-10T10:55:47.821270+0000', 'last_peered': '2024-09-10T10:55:47.821270+0000', 'last_clean': '2024-09-10T10:55:47.821270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:55:47.834144+0000', 'last_undegraded': '2024-09-10T10:55:47.834144+0000', 'last_fullsized': '2024-09-10T10:55:47.834144+0000', 'mapping_epoch': 148, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 148, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_clean_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2f is not active+clean 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 148, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:55:47.836013+0000', 'last_change': '2024-09-10T10:55:47.836013+0000', 'last_active': '2024-09-10T10:55:47.821270+0000', 'last_peered': '2024-09-10T10:55:47.821270+0000', 'last_clean': '2024-09-10T10:55:47.821270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:55:47.836013+0000', 'last_undegraded': '2024-09-10T10:55:47.836013+0000', 'last_fullsized': '2024-09-10T10:55:47.836013+0000', 'mapping_epoch': 148, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 148, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_clean_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:PG 16.2b is not active+clean 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 148, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:55:47.835270+0000', 'last_change': '2024-09-10T10:55:47.835270+0000', 'last_active': '2024-09-10T10:55:47.821270+0000', 'last_peered': '2024-09-10T10:55:47.821270+0000', 'last_clean': '2024-09-10T10:55:47.821270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:55:47.835270+0000', 'last_undegraded': '2024-09-10T10:55:47.835270+0000', 'last_fullsized': '2024-09-10T10:55:47.835270+0000', 'mapping_epoch': 148, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 148, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_clean_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:PG 16.0 is not active+clean 2024-09-10T10:55:53.919 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 148, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:55:47.833598+0000', 'last_change': '2024-09-10T10:55:47.833598+0000', 'last_active': '2024-09-10T10:55:47.821270+0000', 'last_peered': '2024-09-10T10:55:47.821270+0000', 'last_clean': '2024-09-10T10:55:47.821270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:55:47.833598+0000', 'last_undegraded': '2024-09-10T10:55:47.833598+0000', 'last_fullsized': '2024-09-10T10:55:47.833598+0000', 'mapping_epoch': 148, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 148, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_clean_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:55:53.920 INFO:tasks.ceph.ceph_manager.ceph:PG 16.f is not active+clean 2024-09-10T10:55:53.920 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 148, 'state': 'creating+peering', 'last_fresh': '2024-09-10T10:55:47.835555+0000', 'last_change': '2024-09-10T10:55:47.835555+0000', 'last_active': '2024-09-10T10:55:47.821270+0000', 'last_peered': '2024-09-10T10:55:47.821270+0000', 'last_clean': '2024-09-10T10:55:47.821270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T10:55:47.835555+0000', 'last_undegraded': '2024-09-10T10:55:47.835555+0000', 'last_fullsized': '2024-09-10T10:55:47.835555+0000', 'mapping_epoch': 148, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 148, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'last_clean_scrub_stamp': '2024-09-10T10:55:47.821270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T10:55:53.920 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T10:55:54.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.109+0000 7fcad6200640 1 Processor -- start 2024-09-10T10:55:54.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.109+0000 7fcad6200640 1 -- start start 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.110+0000 7fcad6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0070890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.110+0000 7fcad6200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcad0070dd0 0x7fcad00d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.110+0000 7fcad6200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad006cb30 con 0x7fcad00d09e0 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.110+0000 7fcad6200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fcad00d0e30 con 0x7fcad0070dd0 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.110+0000 7fcad6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcad006bfb0 con 0x7fcad0147360 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcac7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcad0070dd0 0x7fcad00d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcacfe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcad00d09e0 0x7fcad006f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54276/0 (socket says 172.21.15.28:54276) 2024-09-10T10:55:54.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcacfe00640 1 -- 172.21.15.28:0/3369564574 learned_addr learned my addr 172.21.15.28:0/3369564574 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:54.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcacf400640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcac7e00640 1 -- 172.21.15.28:0/3369564574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 msgr2=0x7fcad0070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:54.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcac7e00640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0070890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcac7e00640 1 -- 172.21.15.28:0/3369564574 >> v1:172.21.15.28:6789/0 conn(0x7fcad00d09e0 legacy=0x7fcad006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcac7e00640 1 -- 172.21.15.28:0/3369564574 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad006c290 con 0x7fcad0070dd0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcacea00640 1 -- 172.21.15.28:0/3369564574 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2303716185 0 0) 0x7fcad006cb30 con 0x7fcad00d09e0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.111+0000 7fcacf400640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.112+0000 7fcac7e00640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad0070dd0 0x7fcad00d04a0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8294c47bdfeb25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.112+0000 7fcacea00640 1 -- 172.21.15.28:0/3369564574 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8037070 con 0x7fcad0070dd0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.112+0000 7fcacea00640 1 -- 172.21.15.28:0/3369564574 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab802ae50 con 0x7fcad0070dd0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.112+0000 7fcacea00640 1 -- 172.21.15.28:0/3369564574 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8032660 con 0x7fcad0070dd0 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 -- 172.21.15.28:0/3369564574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad0070dd0 msgr2=0x7fcad00d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad0070dd0 0x7fcad00d04a0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab802a100 comp rx=0 tx=0).stop 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 -- 172.21.15.28:0/3369564574 shutdown_connections 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad0070dd0 0x7fcad00d04a0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 --2- 172.21.15.28:0/3369564574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0070890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 -- 172.21.15.28:0/3369564574 >> 172.21.15.28:0/3369564574 conn(0x7fcad00b2f40 msgr2=0x7fcad00b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:54.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 -- 172.21.15.28:0/3369564574 shutdown_connections 2024-09-10T10:55:54.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.113+0000 7fcad6200640 1 -- 172.21.15.28:0/3369564574 wait complete. 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.115+0000 7fcad6200640 1 Processor -- start 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.115+0000 7fcad6200640 1 -- start start 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad0070dd0 0x7fcad01412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcad006bc10 con 0x7fcad00d09e0 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcad006c850 con 0x7fcad0070dd0 2024-09-10T10:55:54.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcad6200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcad006bd90 con 0x7fcad0147360 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55100/0 (socket says 172.21.15.28:55100) 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 -- 172.21.15.28:0/606838815 learned_addr learned my addr 172.21.15.28:0/606838815 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcacfe00640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcacf400640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad0070dd0 0x7fcad01412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 -- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 msgr2=0x7fcad0142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 -- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad0070dd0 msgr2=0x7fcad01412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad0070dd0 0x7fcad01412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcac7e00640 1 -- 172.21.15.28:0/606838815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad00b4a70 con 0x7fcad00d09e0 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.116+0000 7fcacfe00640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:54.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcac7e00640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8037040 con 0x7fcad00d09e0 2024-09-10T10:55:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab8004250 con 0x7fcad00d09e0 2024-09-10T10:55:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8003610 con 0x7fcad00d09e0 2024-09-10T10:55:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad00b1440 con 0x7fcad00d09e0 2024-09-10T10:55:54.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.117+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad00b5010 con 0x7fcad00d09e0 2024-09-10T10:55:54.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.118+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad00d04a0 con 0x7fcad00d09e0 2024-09-10T10:55:54.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.119+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcab80037b0 con 0x7fcad00d09e0 2024-09-10T10:55:54.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.120+0000 7fcac7400640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.120+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7fcab80abe40 con 0x7fcad00d09e0 2024-09-10T10:55:54.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.120+0000 7fcacf400640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.121+0000 7fcacf400640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 0x7fcaac06e2f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcad0145660 tx=0x7fcac0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.124+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcab807a770 con 0x7fcad00d09e0 2024-09-10T10:55:54.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.373+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fcad00b51e0 con 0x7fcad00d09e0 2024-09-10T10:55:54.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.374+0000 7fcac7400640 1 -- 172.21.15.28:0/606838815 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1305 (secure 0 0 0) 0x7fcab807a070 con 0x7fcad00d09e0 2024-09-10T10:55:54.375 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:54.375 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":752,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":150,"num_osds":8,"num_up_osds":8,"osd_up_since":1725965015,"num_in_osds":8,"osd_in_since":1725965005,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":92},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":592778,"bytes_used":297766912,"bytes_avail":772796346368,"bytes_total":773094113280,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2449,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":341,"btime":"2024-09-10T10:55:51:866579+0000","id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":9505}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":52,"modified":"2024-09-10T10:55:30.668343+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-10T10:55:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 msgr2=0x7fcaac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 0x7fcaac06e2f0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcad0145660 tx=0x7fcac0008040 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 msgr2=0x7fcad01423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab80046c0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 shutdown_connections 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fcaac06be30 0x7fcaac06e2f0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcad0147360 0x7fcad0142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcad00d09e0 0x7fcad01423e0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 --2- 172.21.15.28:0/606838815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcad0070dd0 0x7fcad01412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 >> 172.21.15.28:0/606838815 conn(0x7fcad00b2f40 msgr2=0x7fcad006f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 shutdown_connections 2024-09-10T10:55:54.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.376+0000 7fcad6200640 1 -- 172.21.15.28:0/606838815 wait complete. 2024-09-10T10:55:54.392 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T10:55:54.392 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:55:54.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.547+0000 7f3611c00640 1 Processor -- start 2024-09-10T10:55:54.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.548+0000 7f3611c00640 1 -- start start 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3611c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c072d30 0x7f360c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3611c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f360c06f700 0x7f360c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3611c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f360c0cdbe0 con 0x7f360c15b300 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3611c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f360c06fb50 con 0x7f360c06f700 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3611c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f360c0728a0 con 0x7f360c072d30 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f360b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f360c06f700 0x7f360c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f360b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f360c06f700 0x7f360c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55104/0 (socket says 172.21.15.28:55104) 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f360b400640 1 -- 172.21.15.28:0/3870117939 learned_addr learned my addr 172.21.15.28:0/3870117939 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.549+0000 7f3603e00640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c072d30 0x7f360c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.550+0000 7f360b400640 1 -- 172.21.15.28:0/3870117939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c072d30 msgr2=0x7f360c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.550+0000 7f360b400640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c072d30 0x7f360c06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.550+0000 7f360b400640 1 -- 172.21.15.28:0/3870117939 >> v1:172.21.15.138:6789/0 conn(0x7f360c15b300 legacy=0x7f360c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.550+0000 7f360b400640 1 -- 172.21.15.28:0/3870117939 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c0cd900 con 0x7f360c06f700 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f360b400640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c06f700 0x7f360c15b6e0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f3604009e50 tx=0x7f360402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5642956501b33d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f360a000640 1 -- 172.21.15.28:0/3870117939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604036070 con 0x7f360c06f700 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f360a000640 1 -- 172.21.15.28:0/3870117939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3604031600 con 0x7f360c06f700 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f360a000640 1 -- 172.21.15.28:0/3870117939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040319d0 con 0x7f360c06f700 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f3611c00640 1 -- 172.21.15.28:0/3870117939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c06f700 msgr2=0x7f360c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.551+0000 7f3611c00640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c06f700 0x7f360c15b6e0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f3604009e50 tx=0x7f360402a710 comp rx=0 tx=0).stop 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 -- 172.21.15.28:0/3870117939 shutdown_connections 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c06f700 0x7f360c15b6e0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 --2- 172.21.15.28:0/3870117939 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c072d30 0x7f360c06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 -- 172.21.15.28:0/3870117939 >> 172.21.15.28:0/3870117939 conn(0x7f360c067d70 msgr2=0x7f360c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 -- 172.21.15.28:0/3870117939 shutdown_connections 2024-09-10T10:55:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.552+0000 7f3611c00640 1 -- 172.21.15.28:0/3870117939 wait complete. 2024-09-10T10:55:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.553+0000 7f3611c00640 1 Processor -- start 2024-09-10T10:55:54.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.553+0000 7f3611c00640 1 -- start start 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 0x7f360c1e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c15b300 0x7f360c1e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f360c0cd5c0 con 0x7f360c072d30 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f360c072530 con 0x7f360c06f700 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.554+0000 7f3611c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f360c0cffe0 con 0x7f360c15b300 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55110/0 (socket says 172.21.15.28:55110) 2024-09-10T10:55:54.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 -- 172.21.15.28:0/447646538 learned_addr learned my addr 172.21.15.28:0/447646538 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f360b400640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c15b300 0x7f360c1e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f360aa00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 0x7f360c1e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 -- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c15b300 msgr2=0x7f360c1e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c15b300 0x7f360c1e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 -- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 msgr2=0x7f360c1e36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 0x7f360c1e36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 -- 172.21.15.28:0/447646538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c1e41b0 con 0x7f360c072d30 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f360aa00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 0x7f360c1e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:54.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3603e00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d800 tx=0x7f35f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f400b6c0 con 0x7f360c072d30 2024-09-10T10:55:54.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.555+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f4019070 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.556+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f400ba80 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.556+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c1e4490 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.556+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c1e4a50 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.557+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360c146460 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.558+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35f400bc20 con 0x7f360c072d30 2024-09-10T10:55:54.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.558+0000 7f3602a00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:54.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.559+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f35f408b5b0 con 0x7f360c072d30 2024-09-10T10:55:54.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.559+0000 7f360aa00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:54.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.560+0000 7f360aa00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 0x7f35e806e2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f360c1e7490 tx=0x7f35fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:54.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.563+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35f4059ca0 con 0x7f360c072d30 2024-09-10T10:55:54.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.742+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f360c0adf00 con 0x7f35e806be30 2024-09-10T10:55:54.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.748+0000 7f3602a00640 1 -- 172.21.15.28:0/447646538 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252489 (secure 0 0 0) 0x7f360c0adf00 con 0x7f35e806be30 2024-09-10T10:55:54.752 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:54.753 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:55:54.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 msgr2=0x7f35e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 0x7f35e806e2f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f360c1e7490 tx=0x7f35fc009040 comp rx=0 tx=0).stop 2024-09-10T10:55:54.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 msgr2=0x7f360c1e89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:54.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f35f400d800 tx=0x7f35f400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 shutdown_connections 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35e806be30 0x7f35e806e2f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f360c15b300 0x7f360c1e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f360c072d30 0x7f360c1e89a0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 --2- 172.21.15.28:0/447646538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c06f700 0x7f360c1e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 >> 172.21.15.28:0/447646538 conn(0x7f360c067d70 msgr2=0x7f360c1e6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 shutdown_connections 2024-09-10T10:55:54.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:54.752+0000 7f3611c00640 1 -- 172.21.15.28:0/447646538 wait complete. 2024-09-10T10:55:54.769 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-10T10:55:54.674678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290816,"kb_used_data":16316,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683904,"statfs":{"total":773094113280,"available":772796317696,"internally_reserved":0,"allocated":16707584,"data_stored":8997720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.849150"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865805+0000","last_change":"2024-09-10T10:55:48.867834+0000","last_active":"2024-09-10T10:55:49.865805+0000","last_peered":"2024-09-10T10:55:49.865805+0000","last_clean":"2024-09-10T10:55:49.865805+0000","last_became_active":"2024-09-10T10:55:48.867563+0000","last_became_peered":"2024-09-10T10:55:48.867563+0000","last_unstale":"2024-09-10T10:55:49.865805+0000","last_undegraded":"2024-09-10T10:55:49.865805+0000","last_fullsized":"2024-09-10T10:55:49.865805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:00.429383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947062+0000","last_change":"2024-09-10T10:55:48.855439+0000","last_active":"2024-09-10T10:55:49.947062+0000","last_peered":"2024-09-10T10:55:49.947062+0000","last_clean":"2024-09-10T10:55:49.947062+0000","last_became_active":"2024-09-10T10:55:48.855159+0000","last_became_peered":"2024-09-10T10:55:48.855159+0000","last_unstale":"2024-09-10T10:55:49.947062+0000","last_undegraded":"2024-09-10T10:55:49.947062+0000","last_fullsized":"2024-09-10T10:55:49.947062+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:12.344418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865175+0000","last_change":"2024-09-10T10:55:48.880778+0000","last_active":"2024-09-10T10:55:49.865175+0000","last_peered":"2024-09-10T10:55:49.865175+0000","last_clean":"2024-09-10T10:55:49.865175+0000","last_became_active":"2024-09-10T10:55:48.880328+0000","last_became_peered":"2024-09-10T10:55:48.880328+0000","last_unstale":"2024-09-10T10:55:49.865175+0000","last_undegraded":"2024-09-10T10:55:49.865175+0000","last_fullsized":"2024-09-10T10:55:49.865175+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:27.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864724+0000","last_change":"2024-09-10T10:55:48.853911+0000","last_active":"2024-09-10T10:55:49.864724+0000","last_peered":"2024-09-10T10:55:49.864724+0000","last_clean":"2024-09-10T10:55:49.864724+0000","last_became_active":"2024-09-10T10:55:48.853601+0000","last_became_peered":"2024-09-10T10:55:48.853601+0000","last_unstale":"2024-09-10T10:55:49.864724+0000","last_undegraded":"2024-09-10T10:55:49.864724+0000","last_fullsized":"2024-09-10T10:55:49.864724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:51.207539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865887+0000","last_change":"2024-09-10T10:55:48.860012+0000","last_active":"2024-09-10T10:55:49.865887+0000","last_peered":"2024-09-10T10:55:49.865887+0000","last_clean":"2024-09-10T10:55:49.865887+0000","last_became_active":"2024-09-10T10:55:48.859059+0000","last_became_peered":"2024-09-10T10:55:48.859059+0000","last_unstale":"2024-09-10T10:55:49.865887+0000","last_undegraded":"2024-09-10T10:55:49.865887+0000","last_fullsized":"2024-09-10T10:55:49.865887+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.465847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872011+0000","last_change":"2024-09-10T10:55:48.872626+0000","last_active":"2024-09-10T10:55:49.872011+0000","last_peered":"2024-09-10T10:55:49.872011+0000","last_clean":"2024-09-10T10:55:49.872011+0000","last_became_active":"2024-09-10T10:55:48.872334+0000","last_became_peered":"2024-09-10T10:55:48.872334+0000","last_unstale":"2024-09-10T10:55:49.872011+0000","last_undegraded":"2024-09-10T10:55:49.872011+0000","last_fullsized":"2024-09-10T10:55:49.872011+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:51.489910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871869+0000","last_change":"2024-09-10T10:55:48.856346+0000","last_active":"2024-09-10T10:55:49.871869+0000","last_peered":"2024-09-10T10:55:49.871869+0000","last_clean":"2024-09-10T10:55:49.871869+0000","last_became_active":"2024-09-10T10:55:48.856069+0000","last_became_peered":"2024-09-10T10:55:48.856069+0000","last_unstale":"2024-09-10T10:55:49.871869+0000","last_undegraded":"2024-09-10T10:55:49.871869+0000","last_fullsized":"2024-09-10T10:55:49.871869+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961154+0000","last_change":"2024-09-10T10:55:48.876456+0000","last_active":"2024-09-10T10:55:49.961154+0000","last_peered":"2024-09-10T10:55:49.961154+0000","last_clean":"2024-09-10T10:55:49.961154+0000","last_became_active":"2024-09-10T10:55:48.876149+0000","last_became_peered":"2024-09-10T10:55:48.876149+0000","last_unstale":"2024-09-10T10:55:49.961154+0000","last_undegraded":"2024-09-10T10:55:49.961154+0000","last_fullsized":"2024-09-10T10:55:49.961154+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947262+0000","last_change":"2024-09-10T10:55:48.863492+0000","last_active":"2024-09-10T10:55:49.947262+0000","last_peered":"2024-09-10T10:55:49.947262+0000","last_clean":"2024-09-10T10:55:49.947262+0000","last_became_active":"2024-09-10T10:55:48.863048+0000","last_became_peered":"2024-09-10T10:55:48.863048+0000","last_unstale":"2024-09-10T10:55:49.947262+0000","last_undegraded":"2024-09-10T10:55:49.947262+0000","last_fullsized":"2024-09-10T10:55:49.947262+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:30.871617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961998+0000","last_change":"2024-09-10T10:55:48.866566+0000","last_active":"2024-09-10T10:55:49.961998+0000","last_peered":"2024-09-10T10:55:49.961998+0000","last_clean":"2024-09-10T10:55:49.961998+0000","last_became_active":"2024-09-10T10:55:48.866256+0000","last_became_peered":"2024-09-10T10:55:48.866256+0000","last_unstale":"2024-09-10T10:55:49.961998+0000","last_undegraded":"2024-09-10T10:55:49.961998+0000","last_fullsized":"2024-09-10T10:55:49.961998+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:16.476043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862921+0000","last_change":"2024-09-10T10:55:48.853498+0000","last_active":"2024-09-10T10:55:49.862921+0000","last_peered":"2024-09-10T10:55:49.862921+0000","last_clean":"2024-09-10T10:55:49.862921+0000","last_became_active":"2024-09-10T10:55:48.853209+0000","last_became_peered":"2024-09-10T10:55:48.853209+0000","last_unstale":"2024-09-10T10:55:49.862921+0000","last_undegraded":"2024-09-10T10:55:49.862921+0000","last_fullsized":"2024-09-10T10:55:49.862921+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:37.984075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872273+0000","last_change":"2024-09-10T10:55:48.865380+0000","last_active":"2024-09-10T10:55:49.872273+0000","last_peered":"2024-09-10T10:55:49.872273+0000","last_clean":"2024-09-10T10:55:49.872273+0000","last_became_active":"2024-09-10T10:55:48.865102+0000","last_became_peered":"2024-09-10T10:55:48.865102+0000","last_unstale":"2024-09-10T10:55:49.872273+0000","last_undegraded":"2024-09-10T10:55:49.872273+0000","last_fullsized":"2024-09-10T10:55:49.872273+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:39.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865722+0000","last_change":"2024-09-10T10:55:48.863226+0000","last_active":"2024-09-10T10:55:49.865722+0000","last_peered":"2024-09-10T10:55:49.865722+0000","last_clean":"2024-09-10T10:55:49.865722+0000","last_became_active":"2024-09-10T10:55:48.862762+0000","last_became_peered":"2024-09-10T10:55:48.862762+0000","last_unstale":"2024-09-10T10:55:49.865722+0000","last_undegraded":"2024-09-10T10:55:49.865722+0000","last_fullsized":"2024-09-10T10:55:49.865722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:24.148383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872189+0000","last_change":"2024-09-10T10:55:48.864947+0000","last_active":"2024-09-10T10:55:49.872189+0000","last_peered":"2024-09-10T10:55:49.872189+0000","last_clean":"2024-09-10T10:55:49.872189+0000","last_became_active":"2024-09-10T10:55:48.864649+0000","last_became_peered":"2024-09-10T10:55:48.864649+0000","last_unstale":"2024-09-10T10:55:49.872189+0000","last_undegraded":"2024-09-10T10:55:49.872189+0000","last_fullsized":"2024-09-10T10:55:49.872189+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:53.436925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946934+0000","last_change":"2024-09-10T10:55:48.862215+0000","last_active":"2024-09-10T10:55:49.946934+0000","last_peered":"2024-09-10T10:55:49.946934+0000","last_clean":"2024-09-10T10:55:49.946934+0000","last_became_active":"2024-09-10T10:55:48.861706+0000","last_became_peered":"2024-09-10T10:55:48.861706+0000","last_unstale":"2024-09-10T10:55:49.946934+0000","last_undegraded":"2024-09-10T10:55:49.946934+0000","last_fullsized":"2024-09-10T10:55:49.946934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:24.201115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960292+0000","last_change":"2024-09-10T10:55:48.879362+0000","last_active":"2024-09-10T10:55:49.960292+0000","last_peered":"2024-09-10T10:55:49.960292+0000","last_clean":"2024-09-10T10:55:49.960292+0000","last_became_active":"2024-09-10T10:55:48.875054+0000","last_became_peered":"2024-09-10T10:55:48.875054+0000","last_unstale":"2024-09-10T10:55:49.960292+0000","last_undegraded":"2024-09-10T10:55:49.960292+0000","last_fullsized":"2024-09-10T10:55:49.960292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865218+0000","last_change":"2024-09-10T10:55:48.875765+0000","last_active":"2024-09-10T10:55:49.865218+0000","last_peered":"2024-09-10T10:55:49.865218+0000","last_clean":"2024-09-10T10:55:49.865218+0000","last_became_active":"2024-09-10T10:55:48.875506+0000","last_became_peered":"2024-09-10T10:55:48.875506+0000","last_unstale":"2024-09-10T10:55:49.865218+0000","last_undegraded":"2024-09-10T10:55:49.865218+0000","last_fullsized":"2024-09-10T10:55:49.865218+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:17.274642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947217+0000","last_change":"2024-09-10T10:55:48.861625+0000","last_active":"2024-09-10T10:55:49.947217+0000","last_peered":"2024-09-10T10:55:49.947217+0000","last_clean":"2024-09-10T10:55:49.947217+0000","last_became_active":"2024-09-10T10:55:48.860626+0000","last_became_peered":"2024-09-10T10:55:48.860626+0000","last_unstale":"2024-09-10T10:55:49.947217+0000","last_undegraded":"2024-09-10T10:55:49.947217+0000","last_fullsized":"2024-09-10T10:55:49.947217+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:19.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865795+0000","last_change":"2024-09-10T10:55:48.855667+0000","last_active":"2024-09-10T10:55:49.865795+0000","last_peered":"2024-09-10T10:55:49.865795+0000","last_clean":"2024-09-10T10:55:49.865795+0000","last_became_active":"2024-09-10T10:55:48.854331+0000","last_became_peered":"2024-09-10T10:55:48.854331+0000","last_unstale":"2024-09-10T10:55:49.865795+0000","last_undegraded":"2024-09-10T10:55:49.865795+0000","last_fullsized":"2024-09-10T10:55:49.865795+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:11.006767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960683+0000","last_change":"2024-09-10T10:55:48.874514+0000","last_active":"2024-09-10T10:55:49.960683+0000","last_peered":"2024-09-10T10:55:49.960683+0000","last_clean":"2024-09-10T10:55:49.960683+0000","last_became_active":"2024-09-10T10:55:48.874243+0000","last_became_peered":"2024-09-10T10:55:48.874243+0000","last_unstale":"2024-09-10T10:55:49.960683+0000","last_undegraded":"2024-09-10T10:55:49.960683+0000","last_fullsized":"2024-09-10T10:55:49.960683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.639323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864712+0000","last_change":"2024-09-10T10:55:48.866343+0000","last_active":"2024-09-10T10:55:49.864712+0000","last_peered":"2024-09-10T10:55:49.864712+0000","last_clean":"2024-09-10T10:55:49.864712+0000","last_became_active":"2024-09-10T10:55:48.866023+0000","last_became_peered":"2024-09-10T10:55:48.866023+0000","last_unstale":"2024-09-10T10:55:49.864712+0000","last_undegraded":"2024-09-10T10:55:49.864712+0000","last_fullsized":"2024-09-10T10:55:49.864712+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:17.096658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960995+0000","last_change":"2024-09-10T10:55:48.878105+0000","last_active":"2024-09-10T10:55:49.960995+0000","last_peered":"2024-09-10T10:55:49.960995+0000","last_clean":"2024-09-10T10:55:49.960995+0000","last_became_active":"2024-09-10T10:55:48.876598+0000","last_became_peered":"2024-09-10T10:55:48.876598+0000","last_unstale":"2024-09-10T10:55:49.960995+0000","last_undegraded":"2024-09-10T10:55:49.960995+0000","last_fullsized":"2024-09-10T10:55:49.960995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:47.718665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961652+0000","last_change":"2024-09-10T10:55:48.876273+0000","last_active":"2024-09-10T10:55:49.961652+0000","last_peered":"2024-09-10T10:55:49.961652+0000","last_clean":"2024-09-10T10:55:49.961652+0000","last_became_active":"2024-09-10T10:55:48.875814+0000","last_became_peered":"2024-09-10T10:55:48.875814+0000","last_unstale":"2024-09-10T10:55:49.961652+0000","last_undegraded":"2024-09-10T10:55:49.961652+0000","last_fullsized":"2024-09-10T10:55:49.961652+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:23.371793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961329+0000","last_change":"2024-09-10T10:55:48.869413+0000","last_active":"2024-09-10T10:55:49.961329+0000","last_peered":"2024-09-10T10:55:49.961329+0000","last_clean":"2024-09-10T10:55:49.961329+0000","last_became_active":"2024-09-10T10:55:48.869149+0000","last_became_peered":"2024-09-10T10:55:48.869149+0000","last_unstale":"2024-09-10T10:55:49.961329+0000","last_undegraded":"2024-09-10T10:55:49.961329+0000","last_fullsized":"2024-09-10T10:55:49.961329+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:38.451801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960863+0000","last_change":"2024-09-10T10:55:48.866804+0000","last_active":"2024-09-10T10:55:49.960863+0000","last_peered":"2024-09-10T10:55:49.960863+0000","last_clean":"2024-09-10T10:55:49.960863+0000","last_became_active":"2024-09-10T10:55:48.866480+0000","last_became_peered":"2024-09-10T10:55:48.866480+0000","last_unstale":"2024-09-10T10:55:49.960863+0000","last_undegraded":"2024-09-10T10:55:49.960863+0000","last_fullsized":"2024-09-10T10:55:49.960863+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:31.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960874+0000","last_change":"2024-09-10T10:55:48.872545+0000","last_active":"2024-09-10T10:55:49.960874+0000","last_peered":"2024-09-10T10:55:49.960874+0000","last_clean":"2024-09-10T10:55:49.960874+0000","last_became_active":"2024-09-10T10:55:48.872150+0000","last_became_peered":"2024-09-10T10:55:48.872150+0000","last_unstale":"2024-09-10T10:55:49.960874+0000","last_undegraded":"2024-09-10T10:55:49.960874+0000","last_fullsized":"2024-09-10T10:55:49.960874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960690+0000","last_change":"2024-09-10T10:55:48.873569+0000","last_active":"2024-09-10T10:55:49.960690+0000","last_peered":"2024-09-10T10:55:49.960690+0000","last_clean":"2024-09-10T10:55:49.960690+0000","last_became_active":"2024-09-10T10:55:48.872446+0000","last_became_peered":"2024-09-10T10:55:48.872446+0000","last_unstale":"2024-09-10T10:55:49.960690+0000","last_undegraded":"2024-09-10T10:55:49.960690+0000","last_fullsized":"2024-09-10T10:55:49.960690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:58.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872284+0000","last_change":"2024-09-10T10:55:48.866629+0000","last_active":"2024-09-10T10:55:49.872284+0000","last_peered":"2024-09-10T10:55:49.872284+0000","last_clean":"2024-09-10T10:55:49.872284+0000","last_became_active":"2024-09-10T10:55:48.866351+0000","last_became_peered":"2024-09-10T10:55:48.866351+0000","last_unstale":"2024-09-10T10:55:49.872284+0000","last_undegraded":"2024-09-10T10:55:49.872284+0000","last_fullsized":"2024-09-10T10:55:49.872284+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:20.523256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961962+0000","last_change":"2024-09-10T10:55:48.874303+0000","last_active":"2024-09-10T10:55:49.961962+0000","last_peered":"2024-09-10T10:55:49.961962+0000","last_clean":"2024-09-10T10:55:49.961962+0000","last_became_active":"2024-09-10T10:55:48.873605+0000","last_became_peered":"2024-09-10T10:55:48.873605+0000","last_unstale":"2024-09-10T10:55:49.961962+0000","last_undegraded":"2024-09-10T10:55:49.961962+0000","last_fullsized":"2024-09-10T10:55:49.961962+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:43.380426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862682+0000","last_change":"2024-09-10T10:55:48.851514+0000","last_active":"2024-09-10T10:55:49.862682+0000","last_peered":"2024-09-10T10:55:49.862682+0000","last_clean":"2024-09-10T10:55:49.862682+0000","last_became_active":"2024-09-10T10:55:48.851231+0000","last_became_peered":"2024-09-10T10:55:48.851231+0000","last_unstale":"2024-09-10T10:55:49.862682+0000","last_undegraded":"2024-09-10T10:55:49.862682+0000","last_fullsized":"2024-09-10T10:55:49.862682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:21.797943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871910+0000","last_change":"2024-09-10T10:55:48.860423+0000","last_active":"2024-09-10T10:55:49.871910+0000","last_peered":"2024-09-10T10:55:49.871910+0000","last_clean":"2024-09-10T10:55:49.871910+0000","last_became_active":"2024-09-10T10:55:48.860101+0000","last_became_peered":"2024-09-10T10:55:48.860101+0000","last_unstale":"2024-09-10T10:55:49.871910+0000","last_undegraded":"2024-09-10T10:55:49.871910+0000","last_fullsized":"2024-09-10T10:55:49.871910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:15.280350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960768+0000","last_change":"2024-09-10T10:55:48.861221+0000","last_active":"2024-09-10T10:55:49.960768+0000","last_peered":"2024-09-10T10:55:49.960768+0000","last_clean":"2024-09-10T10:55:49.960768+0000","last_became_active":"2024-09-10T10:55:48.860949+0000","last_became_peered":"2024-09-10T10:55:48.860949+0000","last_unstale":"2024-09-10T10:55:49.960768+0000","last_undegraded":"2024-09-10T10:55:49.960768+0000","last_fullsized":"2024-09-10T10:55:49.960768+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:07.147500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864260+0000","last_change":"2024-09-10T10:55:48.874991+0000","last_active":"2024-09-10T10:55:49.864260+0000","last_peered":"2024-09-10T10:55:49.864260+0000","last_clean":"2024-09-10T10:55:49.864260+0000","last_became_active":"2024-09-10T10:55:48.874681+0000","last_became_peered":"2024-09-10T10:55:48.874681+0000","last_unstale":"2024-09-10T10:55:49.864260+0000","last_undegraded":"2024-09-10T10:55:49.864260+0000","last_fullsized":"2024-09-10T10:55:49.864260+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:04.706274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961416+0000","last_change":"2024-09-10T10:55:47.855582+0000","last_active":"2024-09-10T10:55:49.961416+0000","last_peered":"2024-09-10T10:55:49.961416+0000","last_clean":"2024-09-10T10:55:49.961416+0000","last_became_active":"2024-09-10T10:55:47.855156+0000","last_became_peered":"2024-09-10T10:55:47.855156+0000","last_unstale":"2024-09-10T10:55:49.961416+0000","last_undegraded":"2024-09-10T10:55:49.961416+0000","last_fullsized":"2024-09-10T10:55:49.961416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:30.174489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961811+0000","last_change":"2024-09-10T10:55:48.876356+0000","last_active":"2024-09-10T10:55:49.961811+0000","last_peered":"2024-09-10T10:55:49.961811+0000","last_clean":"2024-09-10T10:55:49.961811+0000","last_became_active":"2024-09-10T10:55:48.875734+0000","last_became_peered":"2024-09-10T10:55:48.875734+0000","last_unstale":"2024-09-10T10:55:49.961811+0000","last_undegraded":"2024-09-10T10:55:49.961811+0000","last_fullsized":"2024-09-10T10:55:49.961811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:45.252064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864842+0000","last_change":"2024-09-10T10:55:47.846423+0000","last_active":"2024-09-10T10:55:49.864842+0000","last_peered":"2024-09-10T10:55:49.864842+0000","last_clean":"2024-09-10T10:55:49.864842+0000","last_became_active":"2024-09-10T10:55:47.842576+0000","last_became_peered":"2024-09-10T10:55:47.842576+0000","last_unstale":"2024-09-10T10:55:49.864842+0000","last_undegraded":"2024-09-10T10:55:49.864842+0000","last_fullsized":"2024-09-10T10:55:49.864842+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:37.344579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946997+0000","last_change":"2024-09-10T10:55:48.864892+0000","last_active":"2024-09-10T10:55:49.946997+0000","last_peered":"2024-09-10T10:55:49.946997+0000","last_clean":"2024-09-10T10:55:49.946997+0000","last_became_active":"2024-09-10T10:55:48.864545+0000","last_became_peered":"2024-09-10T10:55:48.864545+0000","last_unstale":"2024-09-10T10:55:49.946997+0000","last_undegraded":"2024-09-10T10:55:49.946997+0000","last_fullsized":"2024-09-10T10:55:49.946997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:45.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947156+0000","last_change":"2024-09-10T10:55:47.848829+0000","last_active":"2024-09-10T10:55:49.947156+0000","last_peered":"2024-09-10T10:55:49.947156+0000","last_clean":"2024-09-10T10:55:49.947156+0000","last_became_active":"2024-09-10T10:55:47.848545+0000","last_became_peered":"2024-09-10T10:55:47.848545+0000","last_unstale":"2024-09-10T10:55:49.947156+0000","last_undegraded":"2024-09-10T10:55:49.947156+0000","last_fullsized":"2024-09-10T10:55:49.947156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:26.415458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862744+0000","last_change":"2024-09-10T10:55:48.853317+0000","last_active":"2024-09-10T10:55:49.862744+0000","last_peered":"2024-09-10T10:55:49.862744+0000","last_clean":"2024-09-10T10:55:49.862744+0000","last_became_active":"2024-09-10T10:55:48.853020+0000","last_became_peered":"2024-09-10T10:55:48.853020+0000","last_unstale":"2024-09-10T10:55:49.862744+0000","last_undegraded":"2024-09-10T10:55:49.862744+0000","last_fullsized":"2024-09-10T10:55:49.862744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:20.610135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961781+0000","last_change":"2024-09-10T10:55:47.847504+0000","last_active":"2024-09-10T10:55:49.961781+0000","last_peered":"2024-09-10T10:55:49.961781+0000","last_clean":"2024-09-10T10:55:49.961781+0000","last_became_active":"2024-09-10T10:55:47.847070+0000","last_became_peered":"2024-09-10T10:55:47.847070+0000","last_unstale":"2024-09-10T10:55:49.961781+0000","last_undegraded":"2024-09-10T10:55:49.961781+0000","last_fullsized":"2024-09-10T10:55:49.961781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:04.988364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865922+0000","last_change":"2024-09-10T10:55:48.867784+0000","last_active":"2024-09-10T10:55:49.865922+0000","last_peered":"2024-09-10T10:55:49.865922+0000","last_clean":"2024-09-10T10:55:49.865922+0000","last_became_active":"2024-09-10T10:55:48.867508+0000","last_became_peered":"2024-09-10T10:55:48.867508+0000","last_unstale":"2024-09-10T10:55:49.865922+0000","last_undegraded":"2024-09-10T10:55:49.865922+0000","last_fullsized":"2024-09-10T10:55:49.865922+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:48.476999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862859+0000","last_change":"2024-09-10T10:55:47.845137+0000","last_active":"2024-09-10T10:55:49.862859+0000","last_peered":"2024-09-10T10:55:49.862859+0000","last_clean":"2024-09-10T10:55:49.862859+0000","last_became_active":"2024-09-10T10:55:47.844834+0000","last_became_peered":"2024-09-10T10:55:47.844834+0000","last_unstale":"2024-09-10T10:55:49.862859+0000","last_undegraded":"2024-09-10T10:55:49.862859+0000","last_fullsized":"2024-09-10T10:55:49.862859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:32.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865918+0000","last_change":"2024-09-10T10:55:48.864028+0000","last_active":"2024-09-10T10:55:49.865918+0000","last_peered":"2024-09-10T10:55:49.865918+0000","last_clean":"2024-09-10T10:55:49.865918+0000","last_became_active":"2024-09-10T10:55:48.863229+0000","last_became_peered":"2024-09-10T10:55:48.863229+0000","last_unstale":"2024-09-10T10:55:49.865918+0000","last_undegraded":"2024-09-10T10:55:49.865918+0000","last_fullsized":"2024-09-10T10:55:49.865918+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:18.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960308+0000","last_change":"2024-09-10T10:55:47.843083+0000","last_active":"2024-09-10T10:55:49.960308+0000","last_peered":"2024-09-10T10:55:49.960308+0000","last_clean":"2024-09-10T10:55:49.960308+0000","last_became_active":"2024-09-10T10:55:47.839631+0000","last_became_peered":"2024-09-10T10:55:47.839631+0000","last_unstale":"2024-09-10T10:55:49.960308+0000","last_undegraded":"2024-09-10T10:55:49.960308+0000","last_fullsized":"2024-09-10T10:55:49.960308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:18.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.863052+0000","last_change":"2024-09-10T10:55:48.864786+0000","last_active":"2024-09-10T10:55:49.863052+0000","last_peered":"2024-09-10T10:55:49.863052+0000","last_clean":"2024-09-10T10:55:49.863052+0000","last_became_active":"2024-09-10T10:55:48.864460+0000","last_became_peered":"2024-09-10T10:55:48.864460+0000","last_unstale":"2024-09-10T10:55:49.863052+0000","last_undegraded":"2024-09-10T10:55:49.863052+0000","last_fullsized":"2024-09-10T10:55:49.863052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:05.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960378+0000","last_change":"2024-09-10T10:55:47.846337+0000","last_active":"2024-09-10T10:55:49.960378+0000","last_peered":"2024-09-10T10:55:49.960378+0000","last_clean":"2024-09-10T10:55:49.960378+0000","last_became_active":"2024-09-10T10:55:47.845507+0000","last_became_peered":"2024-09-10T10:55:47.845507+0000","last_unstale":"2024-09-10T10:55:49.960378+0000","last_undegraded":"2024-09-10T10:55:49.960378+0000","last_fullsized":"2024-09-10T10:55:49.960378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:11.338379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865877+0000","last_change":"2024-09-10T10:55:48.864808+0000","last_active":"2024-09-10T10:55:49.865877+0000","last_peered":"2024-09-10T10:55:49.865877+0000","last_clean":"2024-09-10T10:55:49.865877+0000","last_became_active":"2024-09-10T10:55:48.862818+0000","last_became_peered":"2024-09-10T10:55:48.862818+0000","last_unstale":"2024-09-10T10:55:49.865877+0000","last_undegraded":"2024-09-10T10:55:49.865877+0000","last_fullsized":"2024-09-10T10:55:49.865877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:44.808359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:47.853115+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:47.852824+0000","last_became_peered":"2024-09-10T10:55:47.852824+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:57.389591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864710+0000","last_change":"2024-09-10T10:55:48.865231+0000","last_active":"2024-09-10T10:55:49.864710+0000","last_peered":"2024-09-10T10:55:49.864710+0000","last_clean":"2024-09-10T10:55:49.864710+0000","last_became_active":"2024-09-10T10:55:48.862955+0000","last_became_peered":"2024-09-10T10:55:48.862955+0000","last_unstale":"2024-09-10T10:55:49.864710+0000","last_undegraded":"2024-09-10T10:55:49.864710+0000","last_fullsized":"2024-09-10T10:55:49.864710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.358498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872584+0000","last_change":"2024-09-10T10:55:47.843342+0000","last_active":"2024-09-10T10:55:49.872584+0000","last_peered":"2024-09-10T10:55:49.872584+0000","last_clean":"2024-09-10T10:55:49.872584+0000","last_became_active":"2024-09-10T10:55:47.842788+0000","last_became_peered":"2024-09-10T10:55:47.842788+0000","last_unstale":"2024-09-10T10:55:49.872584+0000","last_undegraded":"2024-09-10T10:55:49.872584+0000","last_fullsized":"2024-09-10T10:55:49.872584+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:00.612235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865943+0000","last_change":"2024-09-10T10:55:48.869535+0000","last_active":"2024-09-10T10:55:49.865943+0000","last_peered":"2024-09-10T10:55:49.865943+0000","last_clean":"2024-09-10T10:55:49.865943+0000","last_became_active":"2024-09-10T10:55:48.869284+0000","last_became_peered":"2024-09-10T10:55:48.869284+0000","last_unstale":"2024-09-10T10:55:49.865943+0000","last_undegraded":"2024-09-10T10:55:49.865943+0000","last_fullsized":"2024-09-10T10:55:49.865943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:44.081502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961991+0000","last_change":"2024-09-10T10:55:47.851219+0000","last_active":"2024-09-10T10:55:49.961991+0000","last_peered":"2024-09-10T10:55:49.961991+0000","last_clean":"2024-09-10T10:55:49.961991+0000","last_became_active":"2024-09-10T10:55:47.849193+0000","last_became_peered":"2024-09-10T10:55:47.849193+0000","last_unstale":"2024-09-10T10:55:49.961991+0000","last_undegraded":"2024-09-10T10:55:49.961991+0000","last_fullsized":"2024-09-10T10:55:49.961991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:50.483870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961814+0000","last_change":"2024-09-10T10:55:48.868003+0000","last_active":"2024-09-10T10:55:49.961814+0000","last_peered":"2024-09-10T10:55:49.961814+0000","last_clean":"2024-09-10T10:55:49.961814+0000","last_became_active":"2024-09-10T10:55:48.867694+0000","last_became_peered":"2024-09-10T10:55:48.867694+0000","last_unstale":"2024-09-10T10:55:49.961814+0000","last_undegraded":"2024-09-10T10:55:49.961814+0000","last_fullsized":"2024-09-10T10:55:49.961814+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:37.123756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872366+0000","last_change":"2024-09-10T10:55:47.847591+0000","last_active":"2024-09-10T10:55:49.872366+0000","last_peered":"2024-09-10T10:55:49.872366+0000","last_clean":"2024-09-10T10:55:49.872366+0000","last_became_active":"2024-09-10T10:55:47.847299+0000","last_became_peered":"2024-09-10T10:55:47.847299+0000","last_unstale":"2024-09-10T10:55:49.872366+0000","last_undegraded":"2024-09-10T10:55:49.872366+0000","last_fullsized":"2024-09-10T10:55:49.872366+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.943592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961027+0000","last_change":"2024-09-10T10:55:48.875294+0000","last_active":"2024-09-10T10:55:49.961027+0000","last_peered":"2024-09-10T10:55:49.961027+0000","last_clean":"2024-09-10T10:55:49.961027+0000","last_became_active":"2024-09-10T10:55:48.873851+0000","last_became_peered":"2024-09-10T10:55:48.873851+0000","last_unstale":"2024-09-10T10:55:49.961027+0000","last_undegraded":"2024-09-10T10:55:49.961027+0000","last_fullsized":"2024-09-10T10:55:49.961027+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:07.268035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862812+0000","last_change":"2024-09-10T10:55:47.850868+0000","last_active":"2024-09-10T10:55:49.862812+0000","last_peered":"2024-09-10T10:55:49.862812+0000","last_clean":"2024-09-10T10:55:49.862812+0000","last_became_active":"2024-09-10T10:55:47.850457+0000","last_became_peered":"2024-09-10T10:55:47.850457+0000","last_unstale":"2024-09-10T10:55:49.862812+0000","last_undegraded":"2024-09-10T10:55:49.862812+0000","last_fullsized":"2024-09-10T10:55:49.862812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:45.564987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947047+0000","last_change":"2024-09-10T10:55:48.861530+0000","last_active":"2024-09-10T10:55:49.947047+0000","last_peered":"2024-09-10T10:55:49.947047+0000","last_clean":"2024-09-10T10:55:49.947047+0000","last_became_active":"2024-09-10T10:55:48.860648+0000","last_became_peered":"2024-09-10T10:55:48.860648+0000","last_unstale":"2024-09-10T10:55:49.947047+0000","last_undegraded":"2024-09-10T10:55:49.947047+0000","last_fullsized":"2024-09-10T10:55:49.947047+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:36.041255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865960+0000","last_change":"2024-09-10T10:55:47.843477+0000","last_active":"2024-09-10T10:55:49.865960+0000","last_peered":"2024-09-10T10:55:49.865960+0000","last_clean":"2024-09-10T10:55:49.865960+0000","last_became_active":"2024-09-10T10:55:47.841371+0000","last_became_peered":"2024-09-10T10:55:47.841371+0000","last_unstale":"2024-09-10T10:55:49.865960+0000","last_undegraded":"2024-09-10T10:55:49.865960+0000","last_fullsized":"2024-09-10T10:55:49.865960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:45.636397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947145+0000","last_change":"2024-09-10T10:55:48.865788+0000","last_active":"2024-09-10T10:55:49.947145+0000","last_peered":"2024-09-10T10:55:49.947145+0000","last_clean":"2024-09-10T10:55:49.947145+0000","last_became_active":"2024-09-10T10:55:48.865486+0000","last_became_peered":"2024-09-10T10:55:48.865486+0000","last_unstale":"2024-09-10T10:55:49.947145+0000","last_undegraded":"2024-09-10T10:55:49.947145+0000","last_fullsized":"2024-09-10T10:55:49.947145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:56.003393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961925+0000","last_change":"2024-09-10T10:55:47.845313+0000","last_active":"2024-09-10T10:55:49.961925+0000","last_peered":"2024-09-10T10:55:49.961925+0000","last_clean":"2024-09-10T10:55:49.961925+0000","last_became_active":"2024-09-10T10:55:47.844999+0000","last_became_peered":"2024-09-10T10:55:47.844999+0000","last_unstale":"2024-09-10T10:55:49.961925+0000","last_undegraded":"2024-09-10T10:55:49.961925+0000","last_fullsized":"2024-09-10T10:55:49.961925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:07.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961293+0000","last_change":"2024-09-10T10:55:48.877412+0000","last_active":"2024-09-10T10:55:49.961293+0000","last_peered":"2024-09-10T10:55:49.961293+0000","last_clean":"2024-09-10T10:55:49.961293+0000","last_became_active":"2024-09-10T10:55:48.876383+0000","last_became_peered":"2024-09-10T10:55:48.876383+0000","last_unstale":"2024-09-10T10:55:49.961293+0000","last_undegraded":"2024-09-10T10:55:49.961293+0000","last_fullsized":"2024-09-10T10:55:49.961293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:41.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865237+0000","last_change":"2024-09-10T10:55:47.834691+0000","last_active":"2024-09-10T10:55:49.865237+0000","last_peered":"2024-09-10T10:55:49.865237+0000","last_clean":"2024-09-10T10:55:49.865237+0000","last_became_active":"2024-09-10T10:55:47.834400+0000","last_became_peered":"2024-09-10T10:55:47.834400+0000","last_unstale":"2024-09-10T10:55:49.865237+0000","last_undegraded":"2024-09-10T10:55:49.865237+0000","last_fullsized":"2024-09-10T10:55:49.865237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:52.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865294+0000","last_change":"2024-09-10T10:55:48.875693+0000","last_active":"2024-09-10T10:55:49.865294+0000","last_peered":"2024-09-10T10:55:49.865294+0000","last_clean":"2024-09-10T10:55:49.865294+0000","last_became_active":"2024-09-10T10:55:48.875098+0000","last_became_peered":"2024-09-10T10:55:48.875098+0000","last_unstale":"2024-09-10T10:55:49.865294+0000","last_undegraded":"2024-09-10T10:55:49.865294+0000","last_fullsized":"2024-09-10T10:55:49.865294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:33.767751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862623+0000","last_change":"2024-09-10T10:55:47.843176+0000","last_active":"2024-09-10T10:55:49.862623+0000","last_peered":"2024-09-10T10:55:49.862623+0000","last_clean":"2024-09-10T10:55:49.862623+0000","last_became_active":"2024-09-10T10:55:47.842914+0000","last_became_peered":"2024-09-10T10:55:47.842914+0000","last_unstale":"2024-09-10T10:55:49.862623+0000","last_undegraded":"2024-09-10T10:55:49.862623+0000","last_fullsized":"2024-09-10T10:55:49.862623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:46.876235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961424+0000","last_change":"2024-09-10T10:55:48.874049+0000","last_active":"2024-09-10T10:55:49.961424+0000","last_peered":"2024-09-10T10:55:49.961424+0000","last_clean":"2024-09-10T10:55:49.961424+0000","last_became_active":"2024-09-10T10:55:48.873744+0000","last_became_peered":"2024-09-10T10:55:48.873744+0000","last_unstale":"2024-09-10T10:55:49.961424+0000","last_undegraded":"2024-09-10T10:55:49.961424+0000","last_fullsized":"2024-09-10T10:55:49.961424+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:25.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872621+0000","last_change":"2024-09-10T10:55:47.845166+0000","last_active":"2024-09-10T10:55:49.872621+0000","last_peered":"2024-09-10T10:55:49.872621+0000","last_clean":"2024-09-10T10:55:49.872621+0000","last_became_active":"2024-09-10T10:55:47.844675+0000","last_became_peered":"2024-09-10T10:55:47.844675+0000","last_unstale":"2024-09-10T10:55:49.872621+0000","last_undegraded":"2024-09-10T10:55:49.872621+0000","last_fullsized":"2024-09-10T10:55:49.872621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:26.167518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960321+0000","last_change":"2024-09-10T10:55:48.872087+0000","last_active":"2024-09-10T10:55:49.960321+0000","last_peered":"2024-09-10T10:55:49.960321+0000","last_clean":"2024-09-10T10:55:49.960321+0000","last_became_active":"2024-09-10T10:55:48.871832+0000","last_became_peered":"2024-09-10T10:55:48.871832+0000","last_unstale":"2024-09-10T10:55:49.960321+0000","last_undegraded":"2024-09-10T10:55:49.960321+0000","last_fullsized":"2024-09-10T10:55:49.960321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:58.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960717+0000","last_change":"2024-09-10T10:55:47.850066+0000","last_active":"2024-09-10T10:55:49.960717+0000","last_peered":"2024-09-10T10:55:49.960717+0000","last_clean":"2024-09-10T10:55:49.960717+0000","last_became_active":"2024-09-10T10:55:47.849775+0000","last_became_peered":"2024-09-10T10:55:47.849775+0000","last_unstale":"2024-09-10T10:55:49.960717+0000","last_undegraded":"2024-09-10T10:55:49.960717+0000","last_fullsized":"2024-09-10T10:55:49.960717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:29.865577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961193+0000","last_change":"2024-09-10T10:55:48.860009+0000","last_active":"2024-09-10T10:55:49.961193+0000","last_peered":"2024-09-10T10:55:49.961193+0000","last_clean":"2024-09-10T10:55:49.961193+0000","last_became_active":"2024-09-10T10:55:48.859725+0000","last_became_peered":"2024-09-10T10:55:48.859725+0000","last_unstale":"2024-09-10T10:55:49.961193+0000","last_undegraded":"2024-09-10T10:55:49.961193+0000","last_fullsized":"2024-09-10T10:55:49.961193+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:06.818156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872362+0000","last_change":"2024-09-10T10:55:47.846489+0000","last_active":"2024-09-10T10:55:49.872362+0000","last_peered":"2024-09-10T10:55:49.872362+0000","last_clean":"2024-09-10T10:55:49.872362+0000","last_became_active":"2024-09-10T10:55:47.846067+0000","last_became_peered":"2024-09-10T10:55:47.846067+0000","last_unstale":"2024-09-10T10:55:49.872362+0000","last_undegraded":"2024-09-10T10:55:49.872362+0000","last_fullsized":"2024-09-10T10:55:49.872362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:28.004930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865650+0000","last_change":"2024-09-10T10:55:48.857694+0000","last_active":"2024-09-10T10:55:49.865650+0000","last_peered":"2024-09-10T10:55:49.865650+0000","last_clean":"2024-09-10T10:55:49.865650+0000","last_became_active":"2024-09-10T10:55:48.857007+0000","last_became_peered":"2024-09-10T10:55:48.857007+0000","last_unstale":"2024-09-10T10:55:49.865650+0000","last_undegraded":"2024-09-10T10:55:49.865650+0000","last_fullsized":"2024-09-10T10:55:49.865650+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:15.966260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864738+0000","last_change":"2024-09-10T10:55:47.846071+0000","last_active":"2024-09-10T10:55:49.864738+0000","last_peered":"2024-09-10T10:55:49.864738+0000","last_clean":"2024-09-10T10:55:49.864738+0000","last_became_active":"2024-09-10T10:55:47.845052+0000","last_became_peered":"2024-09-10T10:55:47.845052+0000","last_unstale":"2024-09-10T10:55:49.864738+0000","last_undegraded":"2024-09-10T10:55:49.864738+0000","last_fullsized":"2024-09-10T10:55:49.864738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:41.842448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:48.868962+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:48.868602+0000","last_became_peered":"2024-09-10T10:55:48.868602+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:37.194259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864393+0000","last_change":"2024-09-10T10:55:47.846363+0000","last_active":"2024-09-10T10:55:49.864393+0000","last_peered":"2024-09-10T10:55:49.864393+0000","last_clean":"2024-09-10T10:55:49.864393+0000","last_became_active":"2024-09-10T10:55:47.846001+0000","last_became_peered":"2024-09-10T10:55:47.846001+0000","last_unstale":"2024-09-10T10:55:49.864393+0000","last_undegraded":"2024-09-10T10:55:49.864393+0000","last_fullsized":"2024-09-10T10:55:49.864393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:52.524852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961231+0000","last_change":"2024-09-10T10:55:48.861960+0000","last_active":"2024-09-10T10:55:49.961231+0000","last_peered":"2024-09-10T10:55:49.961231+0000","last_clean":"2024-09-10T10:55:49.961231+0000","last_became_active":"2024-09-10T10:55:48.861620+0000","last_became_peered":"2024-09-10T10:55:48.861620+0000","last_unstale":"2024-09-10T10:55:49.961231+0000","last_undegraded":"2024-09-10T10:55:49.961231+0000","last_fullsized":"2024-09-10T10:55:49.961231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:27.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872025+0000","last_change":"2024-09-10T10:55:47.837440+0000","last_active":"2024-09-10T10:55:49.872025+0000","last_peered":"2024-09-10T10:55:49.872025+0000","last_clean":"2024-09-10T10:55:49.872025+0000","last_became_active":"2024-09-10T10:55:47.836741+0000","last_became_peered":"2024-09-10T10:55:47.836741+0000","last_unstale":"2024-09-10T10:55:49.872025+0000","last_undegraded":"2024-09-10T10:55:49.872025+0000","last_fullsized":"2024-09-10T10:55:49.872025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:58.662475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865678+0000","last_change":"2024-09-10T10:55:48.867356+0000","last_active":"2024-09-10T10:55:49.865678+0000","last_peered":"2024-09-10T10:55:49.865678+0000","last_clean":"2024-09-10T10:55:49.865678+0000","last_became_active":"2024-09-10T10:55:48.866816+0000","last_became_peered":"2024-09-10T10:55:48.866816+0000","last_unstale":"2024-09-10T10:55:49.865678+0000","last_undegraded":"2024-09-10T10:55:49.865678+0000","last_fullsized":"2024-09-10T10:55:49.865678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:17.379036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961500+0000","last_change":"2024-09-10T10:55:47.845337+0000","last_active":"2024-09-10T10:55:49.961500+0000","last_peered":"2024-09-10T10:55:49.961500+0000","last_clean":"2024-09-10T10:55:49.961500+0000","last_became_active":"2024-09-10T10:55:47.845077+0000","last_became_peered":"2024-09-10T10:55:47.845077+0000","last_unstale":"2024-09-10T10:55:49.961500+0000","last_undegraded":"2024-09-10T10:55:49.961500+0000","last_fullsized":"2024-09-10T10:55:49.961500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:14.274310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960779+0000","last_change":"2024-09-10T10:55:48.864373+0000","last_active":"2024-09-10T10:55:49.960779+0000","last_peered":"2024-09-10T10:55:49.960779+0000","last_clean":"2024-09-10T10:55:49.960779+0000","last_became_active":"2024-09-10T10:55:48.864041+0000","last_became_peered":"2024-09-10T10:55:48.864041+0000","last_unstale":"2024-09-10T10:55:49.960779+0000","last_undegraded":"2024-09-10T10:55:49.960779+0000","last_fullsized":"2024-09-10T10:55:49.960779+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:39.301780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865474+0000","last_change":"2024-09-10T10:55:47.850713+0000","last_active":"2024-09-10T10:55:49.865474+0000","last_peered":"2024-09-10T10:55:49.865474+0000","last_clean":"2024-09-10T10:55:49.865474+0000","last_became_active":"2024-09-10T10:55:47.849850+0000","last_became_peered":"2024-09-10T10:55:47.849850+0000","last_unstale":"2024-09-10T10:55:49.865474+0000","last_undegraded":"2024-09-10T10:55:49.865474+0000","last_fullsized":"2024-09-10T10:55:49.865474+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:17:18.172214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862722+0000","last_change":"2024-09-10T10:55:48.859546+0000","last_active":"2024-09-10T10:55:49.862722+0000","last_peered":"2024-09-10T10:55:49.862722+0000","last_clean":"2024-09-10T10:55:49.862722+0000","last_became_active":"2024-09-10T10:55:48.859164+0000","last_became_peered":"2024-09-10T10:55:48.859164+0000","last_unstale":"2024-09-10T10:55:49.862722+0000","last_undegraded":"2024-09-10T10:55:49.862722+0000","last_fullsized":"2024-09-10T10:55:49.862722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:29.838709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"150'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865073+0000","last_change":"2024-09-10T10:55:47.853293+0000","last_active":"2024-09-10T10:55:49.865073+0000","last_peered":"2024-09-10T10:55:49.865073+0000","last_clean":"2024-09-10T10:55:49.865073+0000","last_became_active":"2024-09-10T10:55:47.852967+0000","last_became_peered":"2024-09-10T10:55:47.852967+0000","last_unstale":"2024-09-10T10:55:49.865073+0000","last_undegraded":"2024-09-10T10:55:49.865073+0000","last_fullsized":"2024-09-10T10:55:49.865073+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:09.128890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961880+0000","last_change":"2024-09-10T10:55:48.874714+0000","last_active":"2024-09-10T10:55:49.961880+0000","last_peered":"2024-09-10T10:55:49.961880+0000","last_clean":"2024-09-10T10:55:49.961880+0000","last_became_active":"2024-09-10T10:55:48.874256+0000","last_became_peered":"2024-09-10T10:55:48.874256+0000","last_unstale":"2024-09-10T10:55:49.961880+0000","last_undegraded":"2024-09-10T10:55:49.961880+0000","last_fullsized":"2024-09-10T10:55:49.961880+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:04.561493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862937+0000","last_change":"2024-09-10T10:55:47.852303+0000","last_active":"2024-09-10T10:55:49.862937+0000","last_peered":"2024-09-10T10:55:49.862937+0000","last_clean":"2024-09-10T10:55:49.862937+0000","last_became_active":"2024-09-10T10:55:47.851277+0000","last_became_peered":"2024-09-10T10:55:47.851277+0000","last_unstale":"2024-09-10T10:55:49.862937+0000","last_undegraded":"2024-09-10T10:55:49.862937+0000","last_fullsized":"2024-09-10T10:55:49.862937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:43.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960557+0000","last_change":"2024-09-10T10:55:48.876162+0000","last_active":"2024-09-10T10:55:49.960557+0000","last_peered":"2024-09-10T10:55:49.960557+0000","last_clean":"2024-09-10T10:55:49.960557+0000","last_became_active":"2024-09-10T10:55:48.875834+0000","last_became_peered":"2024-09-10T10:55:48.875834+0000","last_unstale":"2024-09-10T10:55:49.960557+0000","last_undegraded":"2024-09-10T10:55:49.960557+0000","last_fullsized":"2024-09-10T10:55:49.960557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:12.003746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.866001+0000","last_change":"2024-09-10T10:55:47.844106+0000","last_active":"2024-09-10T10:55:49.866001+0000","last_peered":"2024-09-10T10:55:49.866001+0000","last_clean":"2024-09-10T10:55:49.866001+0000","last_became_active":"2024-09-10T10:55:47.843763+0000","last_became_peered":"2024-09-10T10:55:47.843763+0000","last_unstale":"2024-09-10T10:55:49.866001+0000","last_undegraded":"2024-09-10T10:55:49.866001+0000","last_fullsized":"2024-09-10T10:55:49.866001+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:19.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864866+0000","last_change":"2024-09-10T10:55:48.857933+0000","last_active":"2024-09-10T10:55:49.864866+0000","last_peered":"2024-09-10T10:55:49.864866+0000","last_clean":"2024-09-10T10:55:49.864866+0000","last_became_active":"2024-09-10T10:55:48.857414+0000","last_became_peered":"2024-09-10T10:55:48.857414+0000","last_unstale":"2024-09-10T10:55:49.864866+0000","last_undegraded":"2024-09-10T10:55:49.864866+0000","last_fullsized":"2024-09-10T10:55:49.864866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:52.855868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862873+0000","last_change":"2024-09-10T10:55:47.846380+0000","last_active":"2024-09-10T10:55:49.862873+0000","last_peered":"2024-09-10T10:55:49.862873+0000","last_clean":"2024-09-10T10:55:49.862873+0000","last_became_active":"2024-09-10T10:55:47.846123+0000","last_became_peered":"2024-09-10T10:55:47.846123+0000","last_unstale":"2024-09-10T10:55:49.862873+0000","last_undegraded":"2024-09-10T10:55:49.862873+0000","last_fullsized":"2024-09-10T10:55:49.862873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.465904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865669+0000","last_change":"2024-09-10T10:55:48.859593+0000","last_active":"2024-09-10T10:55:49.865669+0000","last_peered":"2024-09-10T10:55:49.865669+0000","last_clean":"2024-09-10T10:55:49.865669+0000","last_became_active":"2024-09-10T10:55:48.859045+0000","last_became_peered":"2024-09-10T10:55:48.859045+0000","last_unstale":"2024-09-10T10:55:49.865669+0000","last_undegraded":"2024-09-10T10:55:49.865669+0000","last_fullsized":"2024-09-10T10:55:49.865669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:39.427524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864454+0000","last_change":"2024-09-10T10:55:47.853193+0000","last_active":"2024-09-10T10:55:49.864454+0000","last_peered":"2024-09-10T10:55:49.864454+0000","last_clean":"2024-09-10T10:55:49.864454+0000","last_became_active":"2024-09-10T10:55:47.852861+0000","last_became_peered":"2024-09-10T10:55:47.852861+0000","last_unstale":"2024-09-10T10:55:49.864454+0000","last_undegraded":"2024-09-10T10:55:49.864454+0000","last_fullsized":"2024-09-10T10:55:49.864454+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:34.277448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872528+0000","last_change":"2024-09-10T10:55:48.870415+0000","last_active":"2024-09-10T10:55:49.872528+0000","last_peered":"2024-09-10T10:55:49.872528+0000","last_clean":"2024-09-10T10:55:49.872528+0000","last_became_active":"2024-09-10T10:55:48.869966+0000","last_became_peered":"2024-09-10T10:55:48.869966+0000","last_unstale":"2024-09-10T10:55:49.872528+0000","last_undegraded":"2024-09-10T10:55:49.872528+0000","last_fullsized":"2024-09-10T10:55:49.872528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960416+0000","last_change":"2024-09-10T10:55:47.850818+0000","last_active":"2024-09-10T10:55:49.960416+0000","last_peered":"2024-09-10T10:55:49.960416+0000","last_clean":"2024-09-10T10:55:49.960416+0000","last_became_active":"2024-09-10T10:55:47.850541+0000","last_became_peered":"2024-09-10T10:55:47.850541+0000","last_unstale":"2024-09-10T10:55:49.960416+0000","last_undegraded":"2024-09-10T10:55:49.960416+0000","last_fullsized":"2024-09-10T10:55:49.960416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:54.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947271+0000","last_change":"2024-09-10T10:55:48.862694+0000","last_active":"2024-09-10T10:55:49.947271+0000","last_peered":"2024-09-10T10:55:49.947271+0000","last_clean":"2024-09-10T10:55:49.947271+0000","last_became_active":"2024-09-10T10:55:48.862367+0000","last_became_peered":"2024-09-10T10:55:48.862367+0000","last_unstale":"2024-09-10T10:55:49.947271+0000","last_undegraded":"2024-09-10T10:55:49.947271+0000","last_fullsized":"2024-09-10T10:55:49.947271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864269+0000","last_change":"2024-09-10T10:55:47.846160+0000","last_active":"2024-09-10T10:55:49.864269+0000","last_peered":"2024-09-10T10:55:49.864269+0000","last_clean":"2024-09-10T10:55:49.864269+0000","last_became_active":"2024-09-10T10:55:47.845758+0000","last_became_peered":"2024-09-10T10:55:47.845758+0000","last_unstale":"2024-09-10T10:55:49.864269+0000","last_undegraded":"2024-09-10T10:55:49.864269+0000","last_fullsized":"2024-09-10T10:55:49.864269+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:35.560191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871975+0000","last_change":"2024-09-10T10:55:48.867075+0000","last_active":"2024-09-10T10:55:49.871975+0000","last_peered":"2024-09-10T10:55:49.871975+0000","last_clean":"2024-09-10T10:55:49.871975+0000","last_became_active":"2024-09-10T10:55:48.859669+0000","last_became_peered":"2024-09-10T10:55:48.859669+0000","last_unstale":"2024-09-10T10:55:49.871975+0000","last_undegraded":"2024-09-10T10:55:49.871975+0000","last_fullsized":"2024-09-10T10:55:49.871975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:05.994404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864565+0000","last_change":"2024-09-10T10:55:47.847392+0000","last_active":"2024-09-10T10:55:49.864565+0000","last_peered":"2024-09-10T10:55:49.864565+0000","last_clean":"2024-09-10T10:55:49.864565+0000","last_became_active":"2024-09-10T10:55:47.847116+0000","last_became_peered":"2024-09-10T10:55:47.847116+0000","last_unstale":"2024-09-10T10:55:49.864565+0000","last_undegraded":"2024-09-10T10:55:49.864565+0000","last_fullsized":"2024-09-10T10:55:49.864565+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:38.621928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":416,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960208+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:55:49.960208+0000","last_peered":"2024-09-10T10:55:49.960208+0000","last_clean":"2024-09-10T10:55:49.960208+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:55:49.960208+0000","last_undegraded":"2024-09-10T10:55:49.960208+0000","last_fullsized":"2024-09-10T10:55:49.960208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1941504,"data_stored":978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28936,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342904,"statfs":{"total":96636764160,"available":96607133696,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1937408,"data_stored":977391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1941504,"data_stored":977469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509589,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41972,"kb_used_data":2464,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329868,"statfs":{"total":96636764160,"available":96593784832,"internally_reserved":0,"allocated":2523136,"data_stored":1561125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1937408,"data_stored":977939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33664,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338176,"statfs":{"total":96636764160,"available":96602292224,"internally_reserved":0,"allocated":2535424,"data_stored":1567849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:55:57.773 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:55:57.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.941+0000 7fa28b200640 1 Processor -- start 2024-09-10T10:55:57.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.941+0000 7fa28b200640 1 -- start start 2024-09-10T10:55:57.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa28c15e660 0x7fa28c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa28c0cf000 con 0x7fa28c0d1f90 2024-09-10T10:55:57.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa28c0d29a0 con 0x7fa28c0d0dd0 2024-09-10T10:55:57.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa28c0b37d0 con 0x7fa28c15e660 2024-09-10T10:55:57.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa28c0d1f90 0x7fa28c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54284/0 (socket says 172.21.15.28:54284) 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.942+0000 7fa28a800640 1 -- 172.21.15.28:0/3215975288 learned_addr learned my addr 172.21.15.28:0/3215975288 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa289400640 1 --2- 172.21.15.28:0/3215975288 >> v2:172.21.15.138:3300/0 conn(0x7fa28c15e660 0x7fa28c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673509024 0 0) 0x7fa28c0cf000 con 0x7fa28c0d1f90 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa270003520 con 0x7fa28c0d1f90 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3001698372 0 0) 0x7fa270003520 con 0x7fa28c0d1f90 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 >> v2:172.21.15.138:3300/0 conn(0x7fa28c15e660 msgr2=0x7fa28c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 --2- 172.21.15.28:0/3215975288 >> v2:172.21.15.138:3300/0 conn(0x7fa28c15e660 0x7fa28c06ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 >> v1:172.21.15.138:6789/0 conn(0x7fa28c0d0dd0 legacy=0x7fa28c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c0d2c60 con 0x7fa28c0d1f90 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa2840019f0 con 0x7fa28c0d1f90 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 >> v1:172.21.15.28:6789/0 conn(0x7fa28c0d1f90 legacy=0x7fa28c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:57.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa270003a00 0x7fa270003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 0x7fa270004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700051c0 0x7fa27000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa27000c090 con 0x7fa270004800 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa27000c240 con 0x7fa2700051c0 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.943+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa27000c440 con 0x7fa270003a00 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 0x7fa270004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289e00640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa270003a00 0x7fa270003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa28a800640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700051c0 0x7fa27000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 -- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa270003a00 msgr2=0x7fa270003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa270003a00 0x7fa270003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 -- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700051c0 msgr2=0x7fa27000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2700051c0 0x7fa27000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa289400640 1 -- 172.21.15.28:0/3215975288 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa270003520 con 0x7fa270004800 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.944+0000 7fa28a800640 1 -- 172.21.15.28:0/3215975288 reap_dead start 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.945+0000 7fa289400640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 0x7fa270004c80 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fa278009a90 tx=0x7fa270005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecaec6b7da73110 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.945+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278036070 con 0x7fa270004800 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.945+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa278031400 con 0x7fa270004800 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.945+0000 7fa288a00640 1 -- 172.21.15.28:0/3215975288 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa278009690 con 0x7fa270004800 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 -- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 msgr2=0x7fa270004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:57.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 0x7fa270004c80 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fa278009a90 tx=0x7fa270005830 comp rx=0 tx=0).stop 2024-09-10T10:55:57.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 -- 172.21.15.28:0/3215975288 shutdown_connections 2024-09-10T10:55:57.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 --2- 172.21.15.28:0/3215975288 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa270004800 0x7fa270004c80 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 -- 172.21.15.28:0/3215975288 >> 172.21.15.28:0/3215975288 conn(0x7fa28c08bfc0 msgr2=0x7fa28c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:57.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 -- 172.21.15.28:0/3215975288 shutdown_connections 2024-09-10T10:55:57.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.946+0000 7fa28b200640 1 -- 172.21.15.28:0/3215975288 wait complete. 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.948+0000 7fa28b200640 1 Processor -- start 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.948+0000 7fa28b200640 1 -- start start 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270004800 0x7fa28c0b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa28c1e5b80 0x7fa28c14d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c0d3220 con 0x7fa28c14dc30 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa28c0cf330 con 0x7fa270004800 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa28c0d33a0 con 0x7fa28c1e5b80 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55138/0 (socket says 172.21.15.28:55138) 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa28a800640 1 -- 172.21.15.28:0/1273975152 learned_addr learned my addr 172.21.15.28:0/1273975152 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa289400640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa28c1e5b80 0x7fa28c14d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.949+0000 7fa289e00640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270004800 0x7fa28c0b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 -- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa28c1e5b80 msgr2=0x7fa28c14d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa28c1e5b80 0x7fa28c14d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 -- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270004800 msgr2=0x7fa28c0b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270004800 0x7fa28c0b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 -- 172.21.15.28:0/1273975152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c0c7e30 con 0x7fa28c14dc30 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28a800640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0d0850 tx=0x7fa284007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa284017020 con 0x7fa28c14dc30 2024-09-10T10:55:57.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa28400aa70 con 0x7fa28c14dc30 2024-09-10T10:55:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2840133f0 con 0x7fa28c14dc30 2024-09-10T10:55:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28c0c8110 con 0x7fa28c14dc30 2024-09-10T10:55:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.950+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28c1e6000 con 0x7fa28c14dc30 2024-09-10T10:55:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.952+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa28c1e5190 con 0x7fa28c14dc30 2024-09-10T10:55:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.952+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa28401a020 con 0x7fa28c14dc30 2024-09-10T10:55:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.953+0000 7fa282000640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 0x7fa26c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.953+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7fa28408b670 con 0x7fa28c14dc30 2024-09-10T10:55:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.954+0000 7fa289e00640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 0x7fa26c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.954+0000 7fa289e00640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 0x7fa26c06e2f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:57.958+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa284021020 con 0x7fa28c14dc30 2024-09-10T10:55:58.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.140+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa28c14e0b0 con 0x7fa26c06be30 2024-09-10T10:55:58.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.147+0000 7fa282000640 1 -- 172.21.15.28:0/1273975152 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252404 (secure 0 0 0) 0x7fa28c14e0b0 con 0x7fa26c06be30 2024-09-10T10:55:58.151 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:58.152 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.150+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 msgr2=0x7fa26c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.150+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 0x7fa26c06e2f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa27c000c00 tx=0x7fa27c009040 comp rx=0 tx=0).stop 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.150+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 msgr2=0x7fa28c1481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.150+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fa28c0d0850 tx=0x7fa284007b10 comp rx=0 tx=0).stop 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 shutdown_connections 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa26c06be30 0x7fa26c06e2f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa28c14dc30 0x7fa28c1481e0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa28c1e5b80 0x7fa28c14d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 --2- 172.21.15.28:0/1273975152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270004800 0x7fa28c0b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 >> 172.21.15.28:0/1273975152 conn(0x7fa28c08bfc0 msgr2=0x7fa28c0b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:58.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 shutdown_connections 2024-09-10T10:55:58.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.151+0000 7fa28b200640 1 -- 172.21.15.28:0/1273975152 wait complete. 2024-09-10T10:55:58.168 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":515,"stamp":"2024-09-10T10:55:56.675179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":290832,"kb_used_data":16328,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683888,"statfs":{"total":773094113280,"available":772796301312,"internally_reserved":0,"allocated":16719872,"data_stored":9004412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002044"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865805+0000","last_change":"2024-09-10T10:55:48.867834+0000","last_active":"2024-09-10T10:55:49.865805+0000","last_peered":"2024-09-10T10:55:49.865805+0000","last_clean":"2024-09-10T10:55:49.865805+0000","last_became_active":"2024-09-10T10:55:48.867563+0000","last_became_peered":"2024-09-10T10:55:48.867563+0000","last_unstale":"2024-09-10T10:55:49.865805+0000","last_undegraded":"2024-09-10T10:55:49.865805+0000","last_fullsized":"2024-09-10T10:55:49.865805+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:29:00.429383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947062+0000","last_change":"2024-09-10T10:55:48.855439+0000","last_active":"2024-09-10T10:55:49.947062+0000","last_peered":"2024-09-10T10:55:49.947062+0000","last_clean":"2024-09-10T10:55:49.947062+0000","last_became_active":"2024-09-10T10:55:48.855159+0000","last_became_peered":"2024-09-10T10:55:48.855159+0000","last_unstale":"2024-09-10T10:55:49.947062+0000","last_undegraded":"2024-09-10T10:55:49.947062+0000","last_fullsized":"2024-09-10T10:55:49.947062+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:12.344418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865175+0000","last_change":"2024-09-10T10:55:48.880778+0000","last_active":"2024-09-10T10:55:49.865175+0000","last_peered":"2024-09-10T10:55:49.865175+0000","last_clean":"2024-09-10T10:55:49.865175+0000","last_became_active":"2024-09-10T10:55:48.880328+0000","last_became_peered":"2024-09-10T10:55:48.880328+0000","last_unstale":"2024-09-10T10:55:49.865175+0000","last_undegraded":"2024-09-10T10:55:49.865175+0000","last_fullsized":"2024-09-10T10:55:49.865175+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:46:27.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864724+0000","last_change":"2024-09-10T10:55:48.853911+0000","last_active":"2024-09-10T10:55:49.864724+0000","last_peered":"2024-09-10T10:55:49.864724+0000","last_clean":"2024-09-10T10:55:49.864724+0000","last_became_active":"2024-09-10T10:55:48.853601+0000","last_became_peered":"2024-09-10T10:55:48.853601+0000","last_unstale":"2024-09-10T10:55:49.864724+0000","last_undegraded":"2024-09-10T10:55:49.864724+0000","last_fullsized":"2024-09-10T10:55:49.864724+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:51.207539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865887+0000","last_change":"2024-09-10T10:55:48.860012+0000","last_active":"2024-09-10T10:55:49.865887+0000","last_peered":"2024-09-10T10:55:49.865887+0000","last_clean":"2024-09-10T10:55:49.865887+0000","last_became_active":"2024-09-10T10:55:48.859059+0000","last_became_peered":"2024-09-10T10:55:48.859059+0000","last_unstale":"2024-09-10T10:55:49.865887+0000","last_undegraded":"2024-09-10T10:55:49.865887+0000","last_fullsized":"2024-09-10T10:55:49.865887+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:15:38.465847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872011+0000","last_change":"2024-09-10T10:55:48.872626+0000","last_active":"2024-09-10T10:55:49.872011+0000","last_peered":"2024-09-10T10:55:49.872011+0000","last_clean":"2024-09-10T10:55:49.872011+0000","last_became_active":"2024-09-10T10:55:48.872334+0000","last_became_peered":"2024-09-10T10:55:48.872334+0000","last_unstale":"2024-09-10T10:55:49.872011+0000","last_undegraded":"2024-09-10T10:55:49.872011+0000","last_fullsized":"2024-09-10T10:55:49.872011+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:51.489910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871869+0000","last_change":"2024-09-10T10:55:48.856346+0000","last_active":"2024-09-10T10:55:49.871869+0000","last_peered":"2024-09-10T10:55:49.871869+0000","last_clean":"2024-09-10T10:55:49.871869+0000","last_became_active":"2024-09-10T10:55:48.856069+0000","last_became_peered":"2024-09-10T10:55:48.856069+0000","last_unstale":"2024-09-10T10:55:49.871869+0000","last_undegraded":"2024-09-10T10:55:49.871869+0000","last_fullsized":"2024-09-10T10:55:49.871869+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961154+0000","last_change":"2024-09-10T10:55:48.876456+0000","last_active":"2024-09-10T10:55:49.961154+0000","last_peered":"2024-09-10T10:55:49.961154+0000","last_clean":"2024-09-10T10:55:49.961154+0000","last_became_active":"2024-09-10T10:55:48.876149+0000","last_became_peered":"2024-09-10T10:55:48.876149+0000","last_unstale":"2024-09-10T10:55:49.961154+0000","last_undegraded":"2024-09-10T10:55:49.961154+0000","last_fullsized":"2024-09-10T10:55:49.961154+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947262+0000","last_change":"2024-09-10T10:55:48.863492+0000","last_active":"2024-09-10T10:55:49.947262+0000","last_peered":"2024-09-10T10:55:49.947262+0000","last_clean":"2024-09-10T10:55:49.947262+0000","last_became_active":"2024-09-10T10:55:48.863048+0000","last_became_peered":"2024-09-10T10:55:48.863048+0000","last_unstale":"2024-09-10T10:55:49.947262+0000","last_undegraded":"2024-09-10T10:55:49.947262+0000","last_fullsized":"2024-09-10T10:55:49.947262+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:30.871617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961998+0000","last_change":"2024-09-10T10:55:48.866566+0000","last_active":"2024-09-10T10:55:49.961998+0000","last_peered":"2024-09-10T10:55:49.961998+0000","last_clean":"2024-09-10T10:55:49.961998+0000","last_became_active":"2024-09-10T10:55:48.866256+0000","last_became_peered":"2024-09-10T10:55:48.866256+0000","last_unstale":"2024-09-10T10:55:49.961998+0000","last_undegraded":"2024-09-10T10:55:49.961998+0000","last_fullsized":"2024-09-10T10:55:49.961998+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:23:16.476043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862921+0000","last_change":"2024-09-10T10:55:48.853498+0000","last_active":"2024-09-10T10:55:49.862921+0000","last_peered":"2024-09-10T10:55:49.862921+0000","last_clean":"2024-09-10T10:55:49.862921+0000","last_became_active":"2024-09-10T10:55:48.853209+0000","last_became_peered":"2024-09-10T10:55:48.853209+0000","last_unstale":"2024-09-10T10:55:49.862921+0000","last_undegraded":"2024-09-10T10:55:49.862921+0000","last_fullsized":"2024-09-10T10:55:49.862921+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:37.984075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872273+0000","last_change":"2024-09-10T10:55:48.865380+0000","last_active":"2024-09-10T10:55:49.872273+0000","last_peered":"2024-09-10T10:55:49.872273+0000","last_clean":"2024-09-10T10:55:49.872273+0000","last_became_active":"2024-09-10T10:55:48.865102+0000","last_became_peered":"2024-09-10T10:55:48.865102+0000","last_unstale":"2024-09-10T10:55:49.872273+0000","last_undegraded":"2024-09-10T10:55:49.872273+0000","last_fullsized":"2024-09-10T10:55:49.872273+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:39.627968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865722+0000","last_change":"2024-09-10T10:55:48.863226+0000","last_active":"2024-09-10T10:55:49.865722+0000","last_peered":"2024-09-10T10:55:49.865722+0000","last_clean":"2024-09-10T10:55:49.865722+0000","last_became_active":"2024-09-10T10:55:48.862762+0000","last_became_peered":"2024-09-10T10:55:48.862762+0000","last_unstale":"2024-09-10T10:55:49.865722+0000","last_undegraded":"2024-09-10T10:55:49.865722+0000","last_fullsized":"2024-09-10T10:55:49.865722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:06:24.148383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872189+0000","last_change":"2024-09-10T10:55:48.864947+0000","last_active":"2024-09-10T10:55:49.872189+0000","last_peered":"2024-09-10T10:55:49.872189+0000","last_clean":"2024-09-10T10:55:49.872189+0000","last_became_active":"2024-09-10T10:55:48.864649+0000","last_became_peered":"2024-09-10T10:55:48.864649+0000","last_unstale":"2024-09-10T10:55:49.872189+0000","last_undegraded":"2024-09-10T10:55:49.872189+0000","last_fullsized":"2024-09-10T10:55:49.872189+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:53.436925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946934+0000","last_change":"2024-09-10T10:55:48.862215+0000","last_active":"2024-09-10T10:55:49.946934+0000","last_peered":"2024-09-10T10:55:49.946934+0000","last_clean":"2024-09-10T10:55:49.946934+0000","last_became_active":"2024-09-10T10:55:48.861706+0000","last_became_peered":"2024-09-10T10:55:48.861706+0000","last_unstale":"2024-09-10T10:55:49.946934+0000","last_undegraded":"2024-09-10T10:55:49.946934+0000","last_fullsized":"2024-09-10T10:55:49.946934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:24.201115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960292+0000","last_change":"2024-09-10T10:55:48.879362+0000","last_active":"2024-09-10T10:55:49.960292+0000","last_peered":"2024-09-10T10:55:49.960292+0000","last_clean":"2024-09-10T10:55:49.960292+0000","last_became_active":"2024-09-10T10:55:48.875054+0000","last_became_peered":"2024-09-10T10:55:48.875054+0000","last_unstale":"2024-09-10T10:55:49.960292+0000","last_undegraded":"2024-09-10T10:55:49.960292+0000","last_fullsized":"2024-09-10T10:55:49.960292+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865218+0000","last_change":"2024-09-10T10:55:48.875765+0000","last_active":"2024-09-10T10:55:49.865218+0000","last_peered":"2024-09-10T10:55:49.865218+0000","last_clean":"2024-09-10T10:55:49.865218+0000","last_became_active":"2024-09-10T10:55:48.875506+0000","last_became_peered":"2024-09-10T10:55:48.875506+0000","last_unstale":"2024-09-10T10:55:49.865218+0000","last_undegraded":"2024-09-10T10:55:49.865218+0000","last_fullsized":"2024-09-10T10:55:49.865218+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:07:17.274642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947217+0000","last_change":"2024-09-10T10:55:48.861625+0000","last_active":"2024-09-10T10:55:49.947217+0000","last_peered":"2024-09-10T10:55:49.947217+0000","last_clean":"2024-09-10T10:55:49.947217+0000","last_became_active":"2024-09-10T10:55:48.860626+0000","last_became_peered":"2024-09-10T10:55:48.860626+0000","last_unstale":"2024-09-10T10:55:49.947217+0000","last_undegraded":"2024-09-10T10:55:49.947217+0000","last_fullsized":"2024-09-10T10:55:49.947217+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:19.400514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865795+0000","last_change":"2024-09-10T10:55:48.855667+0000","last_active":"2024-09-10T10:55:49.865795+0000","last_peered":"2024-09-10T10:55:49.865795+0000","last_clean":"2024-09-10T10:55:49.865795+0000","last_became_active":"2024-09-10T10:55:48.854331+0000","last_became_peered":"2024-09-10T10:55:48.854331+0000","last_unstale":"2024-09-10T10:55:49.865795+0000","last_undegraded":"2024-09-10T10:55:49.865795+0000","last_fullsized":"2024-09-10T10:55:49.865795+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:41:11.006767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960683+0000","last_change":"2024-09-10T10:55:48.874514+0000","last_active":"2024-09-10T10:55:49.960683+0000","last_peered":"2024-09-10T10:55:49.960683+0000","last_clean":"2024-09-10T10:55:49.960683+0000","last_became_active":"2024-09-10T10:55:48.874243+0000","last_became_peered":"2024-09-10T10:55:48.874243+0000","last_unstale":"2024-09-10T10:55:49.960683+0000","last_undegraded":"2024-09-10T10:55:49.960683+0000","last_fullsized":"2024-09-10T10:55:49.960683+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:19:05.639323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864712+0000","last_change":"2024-09-10T10:55:48.866343+0000","last_active":"2024-09-10T10:55:49.864712+0000","last_peered":"2024-09-10T10:55:49.864712+0000","last_clean":"2024-09-10T10:55:49.864712+0000","last_became_active":"2024-09-10T10:55:48.866023+0000","last_became_peered":"2024-09-10T10:55:48.866023+0000","last_unstale":"2024-09-10T10:55:49.864712+0000","last_undegraded":"2024-09-10T10:55:49.864712+0000","last_fullsized":"2024-09-10T10:55:49.864712+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:45:17.096658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960995+0000","last_change":"2024-09-10T10:55:48.878105+0000","last_active":"2024-09-10T10:55:49.960995+0000","last_peered":"2024-09-10T10:55:49.960995+0000","last_clean":"2024-09-10T10:55:49.960995+0000","last_became_active":"2024-09-10T10:55:48.876598+0000","last_became_peered":"2024-09-10T10:55:48.876598+0000","last_unstale":"2024-09-10T10:55:49.960995+0000","last_undegraded":"2024-09-10T10:55:49.960995+0000","last_fullsized":"2024-09-10T10:55:49.960995+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:14:47.718665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961652+0000","last_change":"2024-09-10T10:55:48.876273+0000","last_active":"2024-09-10T10:55:49.961652+0000","last_peered":"2024-09-10T10:55:49.961652+0000","last_clean":"2024-09-10T10:55:49.961652+0000","last_became_active":"2024-09-10T10:55:48.875814+0000","last_became_peered":"2024-09-10T10:55:48.875814+0000","last_unstale":"2024-09-10T10:55:49.961652+0000","last_undegraded":"2024-09-10T10:55:49.961652+0000","last_fullsized":"2024-09-10T10:55:49.961652+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:23.371793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961329+0000","last_change":"2024-09-10T10:55:48.869413+0000","last_active":"2024-09-10T10:55:49.961329+0000","last_peered":"2024-09-10T10:55:49.961329+0000","last_clean":"2024-09-10T10:55:49.961329+0000","last_became_active":"2024-09-10T10:55:48.869149+0000","last_became_peered":"2024-09-10T10:55:48.869149+0000","last_unstale":"2024-09-10T10:55:49.961329+0000","last_undegraded":"2024-09-10T10:55:49.961329+0000","last_fullsized":"2024-09-10T10:55:49.961329+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:58:38.451801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960863+0000","last_change":"2024-09-10T10:55:48.866804+0000","last_active":"2024-09-10T10:55:49.960863+0000","last_peered":"2024-09-10T10:55:49.960863+0000","last_clean":"2024-09-10T10:55:49.960863+0000","last_became_active":"2024-09-10T10:55:48.866480+0000","last_became_peered":"2024-09-10T10:55:48.866480+0000","last_unstale":"2024-09-10T10:55:49.960863+0000","last_undegraded":"2024-09-10T10:55:49.960863+0000","last_fullsized":"2024-09-10T10:55:49.960863+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:31.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960874+0000","last_change":"2024-09-10T10:55:48.872545+0000","last_active":"2024-09-10T10:55:49.960874+0000","last_peered":"2024-09-10T10:55:49.960874+0000","last_clean":"2024-09-10T10:55:49.960874+0000","last_became_active":"2024-09-10T10:55:48.872150+0000","last_became_peered":"2024-09-10T10:55:48.872150+0000","last_unstale":"2024-09-10T10:55:49.960874+0000","last_undegraded":"2024-09-10T10:55:49.960874+0000","last_fullsized":"2024-09-10T10:55:49.960874+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:55.012311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960690+0000","last_change":"2024-09-10T10:55:48.873569+0000","last_active":"2024-09-10T10:55:49.960690+0000","last_peered":"2024-09-10T10:55:49.960690+0000","last_clean":"2024-09-10T10:55:49.960690+0000","last_became_active":"2024-09-10T10:55:48.872446+0000","last_became_peered":"2024-09-10T10:55:48.872446+0000","last_unstale":"2024-09-10T10:55:49.960690+0000","last_undegraded":"2024-09-10T10:55:49.960690+0000","last_fullsized":"2024-09-10T10:55:49.960690+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:21:58.851593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872284+0000","last_change":"2024-09-10T10:55:48.866629+0000","last_active":"2024-09-10T10:55:49.872284+0000","last_peered":"2024-09-10T10:55:49.872284+0000","last_clean":"2024-09-10T10:55:49.872284+0000","last_became_active":"2024-09-10T10:55:48.866351+0000","last_became_peered":"2024-09-10T10:55:48.866351+0000","last_unstale":"2024-09-10T10:55:49.872284+0000","last_undegraded":"2024-09-10T10:55:49.872284+0000","last_fullsized":"2024-09-10T10:55:49.872284+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:20.523256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961962+0000","last_change":"2024-09-10T10:55:48.874303+0000","last_active":"2024-09-10T10:55:49.961962+0000","last_peered":"2024-09-10T10:55:49.961962+0000","last_clean":"2024-09-10T10:55:49.961962+0000","last_became_active":"2024-09-10T10:55:48.873605+0000","last_became_peered":"2024-09-10T10:55:48.873605+0000","last_unstale":"2024-09-10T10:55:49.961962+0000","last_undegraded":"2024-09-10T10:55:49.961962+0000","last_fullsized":"2024-09-10T10:55:49.961962+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:43.380426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862682+0000","last_change":"2024-09-10T10:55:48.851514+0000","last_active":"2024-09-10T10:55:49.862682+0000","last_peered":"2024-09-10T10:55:49.862682+0000","last_clean":"2024-09-10T10:55:49.862682+0000","last_became_active":"2024-09-10T10:55:48.851231+0000","last_became_peered":"2024-09-10T10:55:48.851231+0000","last_unstale":"2024-09-10T10:55:49.862682+0000","last_undegraded":"2024-09-10T10:55:49.862682+0000","last_fullsized":"2024-09-10T10:55:49.862682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:21.797943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871910+0000","last_change":"2024-09-10T10:55:48.860423+0000","last_active":"2024-09-10T10:55:49.871910+0000","last_peered":"2024-09-10T10:55:49.871910+0000","last_clean":"2024-09-10T10:55:49.871910+0000","last_became_active":"2024-09-10T10:55:48.860101+0000","last_became_peered":"2024-09-10T10:55:48.860101+0000","last_unstale":"2024-09-10T10:55:49.871910+0000","last_undegraded":"2024-09-10T10:55:49.871910+0000","last_fullsized":"2024-09-10T10:55:49.871910+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:15.280350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960768+0000","last_change":"2024-09-10T10:55:48.861221+0000","last_active":"2024-09-10T10:55:49.960768+0000","last_peered":"2024-09-10T10:55:49.960768+0000","last_clean":"2024-09-10T10:55:49.960768+0000","last_became_active":"2024-09-10T10:55:48.860949+0000","last_became_peered":"2024-09-10T10:55:48.860949+0000","last_unstale":"2024-09-10T10:55:49.960768+0000","last_undegraded":"2024-09-10T10:55:49.960768+0000","last_fullsized":"2024-09-10T10:55:49.960768+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:45:07.147500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864260+0000","last_change":"2024-09-10T10:55:48.874991+0000","last_active":"2024-09-10T10:55:49.864260+0000","last_peered":"2024-09-10T10:55:49.864260+0000","last_clean":"2024-09-10T10:55:49.864260+0000","last_became_active":"2024-09-10T10:55:48.874681+0000","last_became_peered":"2024-09-10T10:55:48.874681+0000","last_unstale":"2024-09-10T10:55:49.864260+0000","last_undegraded":"2024-09-10T10:55:49.864260+0000","last_fullsized":"2024-09-10T10:55:49.864260+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:04.706274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"149'4","reported_seq":18,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961416+0000","last_change":"2024-09-10T10:55:47.855582+0000","last_active":"2024-09-10T10:55:49.961416+0000","last_peered":"2024-09-10T10:55:49.961416+0000","last_clean":"2024-09-10T10:55:49.961416+0000","last_became_active":"2024-09-10T10:55:47.855156+0000","last_became_peered":"2024-09-10T10:55:47.855156+0000","last_unstale":"2024-09-10T10:55:49.961416+0000","last_undegraded":"2024-09-10T10:55:49.961416+0000","last_fullsized":"2024-09-10T10:55:49.961416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:30.174489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961811+0000","last_change":"2024-09-10T10:55:48.876356+0000","last_active":"2024-09-10T10:55:49.961811+0000","last_peered":"2024-09-10T10:55:49.961811+0000","last_clean":"2024-09-10T10:55:49.961811+0000","last_became_active":"2024-09-10T10:55:48.875734+0000","last_became_peered":"2024-09-10T10:55:48.875734+0000","last_unstale":"2024-09-10T10:55:49.961811+0000","last_undegraded":"2024-09-10T10:55:49.961811+0000","last_fullsized":"2024-09-10T10:55:49.961811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:38:45.252064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864842+0000","last_change":"2024-09-10T10:55:47.846423+0000","last_active":"2024-09-10T10:55:49.864842+0000","last_peered":"2024-09-10T10:55:49.864842+0000","last_clean":"2024-09-10T10:55:49.864842+0000","last_became_active":"2024-09-10T10:55:47.842576+0000","last_became_peered":"2024-09-10T10:55:47.842576+0000","last_unstale":"2024-09-10T10:55:49.864842+0000","last_undegraded":"2024-09-10T10:55:49.864842+0000","last_fullsized":"2024-09-10T10:55:49.864842+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:37.344579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.946997+0000","last_change":"2024-09-10T10:55:48.864892+0000","last_active":"2024-09-10T10:55:49.946997+0000","last_peered":"2024-09-10T10:55:49.946997+0000","last_clean":"2024-09-10T10:55:49.946997+0000","last_became_active":"2024-09-10T10:55:48.864545+0000","last_became_peered":"2024-09-10T10:55:48.864545+0000","last_unstale":"2024-09-10T10:55:49.946997+0000","last_undegraded":"2024-09-10T10:55:49.946997+0000","last_fullsized":"2024-09-10T10:55:49.946997+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:45.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947156+0000","last_change":"2024-09-10T10:55:47.848829+0000","last_active":"2024-09-10T10:55:49.947156+0000","last_peered":"2024-09-10T10:55:49.947156+0000","last_clean":"2024-09-10T10:55:49.947156+0000","last_became_active":"2024-09-10T10:55:47.848545+0000","last_became_peered":"2024-09-10T10:55:47.848545+0000","last_unstale":"2024-09-10T10:55:49.947156+0000","last_undegraded":"2024-09-10T10:55:49.947156+0000","last_fullsized":"2024-09-10T10:55:49.947156+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:26.415458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862744+0000","last_change":"2024-09-10T10:55:48.853317+0000","last_active":"2024-09-10T10:55:49.862744+0000","last_peered":"2024-09-10T10:55:49.862744+0000","last_clean":"2024-09-10T10:55:49.862744+0000","last_became_active":"2024-09-10T10:55:48.853020+0000","last_became_peered":"2024-09-10T10:55:48.853020+0000","last_unstale":"2024-09-10T10:55:49.862744+0000","last_undegraded":"2024-09-10T10:55:49.862744+0000","last_fullsized":"2024-09-10T10:55:49.862744+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:22:20.610135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961781+0000","last_change":"2024-09-10T10:55:47.847504+0000","last_active":"2024-09-10T10:55:49.961781+0000","last_peered":"2024-09-10T10:55:49.961781+0000","last_clean":"2024-09-10T10:55:49.961781+0000","last_became_active":"2024-09-10T10:55:47.847070+0000","last_became_peered":"2024-09-10T10:55:47.847070+0000","last_unstale":"2024-09-10T10:55:49.961781+0000","last_undegraded":"2024-09-10T10:55:49.961781+0000","last_fullsized":"2024-09-10T10:55:49.961781+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:04.988364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865922+0000","last_change":"2024-09-10T10:55:48.867784+0000","last_active":"2024-09-10T10:55:49.865922+0000","last_peered":"2024-09-10T10:55:49.865922+0000","last_clean":"2024-09-10T10:55:49.865922+0000","last_became_active":"2024-09-10T10:55:48.867508+0000","last_became_peered":"2024-09-10T10:55:48.867508+0000","last_unstale":"2024-09-10T10:55:49.865922+0000","last_undegraded":"2024-09-10T10:55:49.865922+0000","last_fullsized":"2024-09-10T10:55:49.865922+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:43:48.476999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862859+0000","last_change":"2024-09-10T10:55:47.845137+0000","last_active":"2024-09-10T10:55:49.862859+0000","last_peered":"2024-09-10T10:55:49.862859+0000","last_clean":"2024-09-10T10:55:49.862859+0000","last_became_active":"2024-09-10T10:55:47.844834+0000","last_became_peered":"2024-09-10T10:55:47.844834+0000","last_unstale":"2024-09-10T10:55:49.862859+0000","last_undegraded":"2024-09-10T10:55:49.862859+0000","last_fullsized":"2024-09-10T10:55:49.862859+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:04:32.174543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865918+0000","last_change":"2024-09-10T10:55:48.864028+0000","last_active":"2024-09-10T10:55:49.865918+0000","last_peered":"2024-09-10T10:55:49.865918+0000","last_clean":"2024-09-10T10:55:49.865918+0000","last_became_active":"2024-09-10T10:55:48.863229+0000","last_became_peered":"2024-09-10T10:55:48.863229+0000","last_unstale":"2024-09-10T10:55:49.865918+0000","last_undegraded":"2024-09-10T10:55:49.865918+0000","last_fullsized":"2024-09-10T10:55:49.865918+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:18.602516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960308+0000","last_change":"2024-09-10T10:55:47.843083+0000","last_active":"2024-09-10T10:55:49.960308+0000","last_peered":"2024-09-10T10:55:49.960308+0000","last_clean":"2024-09-10T10:55:49.960308+0000","last_became_active":"2024-09-10T10:55:47.839631+0000","last_became_peered":"2024-09-10T10:55:47.839631+0000","last_unstale":"2024-09-10T10:55:49.960308+0000","last_undegraded":"2024-09-10T10:55:49.960308+0000","last_fullsized":"2024-09-10T10:55:49.960308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:42:18.394474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.863052+0000","last_change":"2024-09-10T10:55:48.864786+0000","last_active":"2024-09-10T10:55:49.863052+0000","last_peered":"2024-09-10T10:55:49.863052+0000","last_clean":"2024-09-10T10:55:49.863052+0000","last_became_active":"2024-09-10T10:55:48.864460+0000","last_became_peered":"2024-09-10T10:55:48.864460+0000","last_unstale":"2024-09-10T10:55:49.863052+0000","last_undegraded":"2024-09-10T10:55:49.863052+0000","last_fullsized":"2024-09-10T10:55:49.863052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:53:05.869771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960378+0000","last_change":"2024-09-10T10:55:47.846337+0000","last_active":"2024-09-10T10:55:49.960378+0000","last_peered":"2024-09-10T10:55:49.960378+0000","last_clean":"2024-09-10T10:55:49.960378+0000","last_became_active":"2024-09-10T10:55:47.845507+0000","last_became_peered":"2024-09-10T10:55:47.845507+0000","last_unstale":"2024-09-10T10:55:49.960378+0000","last_undegraded":"2024-09-10T10:55:49.960378+0000","last_fullsized":"2024-09-10T10:55:49.960378+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:11.338379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865877+0000","last_change":"2024-09-10T10:55:48.864808+0000","last_active":"2024-09-10T10:55:49.865877+0000","last_peered":"2024-09-10T10:55:49.865877+0000","last_clean":"2024-09-10T10:55:49.865877+0000","last_became_active":"2024-09-10T10:55:48.862818+0000","last_became_peered":"2024-09-10T10:55:48.862818+0000","last_unstale":"2024-09-10T10:55:49.865877+0000","last_undegraded":"2024-09-10T10:55:49.865877+0000","last_fullsized":"2024-09-10T10:55:49.865877+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:26:44.808359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:47.853115+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:47.852824+0000","last_became_peered":"2024-09-10T10:55:47.852824+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:57.389591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864710+0000","last_change":"2024-09-10T10:55:48.865231+0000","last_active":"2024-09-10T10:55:49.864710+0000","last_peered":"2024-09-10T10:55:49.864710+0000","last_clean":"2024-09-10T10:55:49.864710+0000","last_became_active":"2024-09-10T10:55:48.862955+0000","last_became_peered":"2024-09-10T10:55:48.862955+0000","last_unstale":"2024-09-10T10:55:49.864710+0000","last_undegraded":"2024-09-10T10:55:49.864710+0000","last_fullsized":"2024-09-10T10:55:49.864710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:18.358498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872584+0000","last_change":"2024-09-10T10:55:47.843342+0000","last_active":"2024-09-10T10:55:49.872584+0000","last_peered":"2024-09-10T10:55:49.872584+0000","last_clean":"2024-09-10T10:55:49.872584+0000","last_became_active":"2024-09-10T10:55:47.842788+0000","last_became_peered":"2024-09-10T10:55:47.842788+0000","last_unstale":"2024-09-10T10:55:49.872584+0000","last_undegraded":"2024-09-10T10:55:49.872584+0000","last_fullsized":"2024-09-10T10:55:49.872584+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:07:00.612235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865943+0000","last_change":"2024-09-10T10:55:48.869535+0000","last_active":"2024-09-10T10:55:49.865943+0000","last_peered":"2024-09-10T10:55:49.865943+0000","last_clean":"2024-09-10T10:55:49.865943+0000","last_became_active":"2024-09-10T10:55:48.869284+0000","last_became_peered":"2024-09-10T10:55:48.869284+0000","last_unstale":"2024-09-10T10:55:49.865943+0000","last_undegraded":"2024-09-10T10:55:49.865943+0000","last_fullsized":"2024-09-10T10:55:49.865943+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:00:44.081502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961991+0000","last_change":"2024-09-10T10:55:47.851219+0000","last_active":"2024-09-10T10:55:49.961991+0000","last_peered":"2024-09-10T10:55:49.961991+0000","last_clean":"2024-09-10T10:55:49.961991+0000","last_became_active":"2024-09-10T10:55:47.849193+0000","last_became_peered":"2024-09-10T10:55:47.849193+0000","last_unstale":"2024-09-10T10:55:49.961991+0000","last_undegraded":"2024-09-10T10:55:49.961991+0000","last_fullsized":"2024-09-10T10:55:49.961991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:12:50.483870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961814+0000","last_change":"2024-09-10T10:55:48.868003+0000","last_active":"2024-09-10T10:55:49.961814+0000","last_peered":"2024-09-10T10:55:49.961814+0000","last_clean":"2024-09-10T10:55:49.961814+0000","last_became_active":"2024-09-10T10:55:48.867694+0000","last_became_peered":"2024-09-10T10:55:48.867694+0000","last_unstale":"2024-09-10T10:55:49.961814+0000","last_undegraded":"2024-09-10T10:55:49.961814+0000","last_fullsized":"2024-09-10T10:55:49.961814+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:20:37.123756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"149'3","reported_seq":17,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872366+0000","last_change":"2024-09-10T10:55:47.847591+0000","last_active":"2024-09-10T10:55:49.872366+0000","last_peered":"2024-09-10T10:55:49.872366+0000","last_clean":"2024-09-10T10:55:49.872366+0000","last_became_active":"2024-09-10T10:55:47.847299+0000","last_became_peered":"2024-09-10T10:55:47.847299+0000","last_unstale":"2024-09-10T10:55:49.872366+0000","last_undegraded":"2024-09-10T10:55:49.872366+0000","last_fullsized":"2024-09-10T10:55:49.872366+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:44.943592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961027+0000","last_change":"2024-09-10T10:55:48.875294+0000","last_active":"2024-09-10T10:55:49.961027+0000","last_peered":"2024-09-10T10:55:49.961027+0000","last_clean":"2024-09-10T10:55:49.961027+0000","last_became_active":"2024-09-10T10:55:48.873851+0000","last_became_peered":"2024-09-10T10:55:48.873851+0000","last_unstale":"2024-09-10T10:55:49.961027+0000","last_undegraded":"2024-09-10T10:55:49.961027+0000","last_fullsized":"2024-09-10T10:55:49.961027+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:07.268035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"149'6","reported_seq":20,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862812+0000","last_change":"2024-09-10T10:55:47.850868+0000","last_active":"2024-09-10T10:55:49.862812+0000","last_peered":"2024-09-10T10:55:49.862812+0000","last_clean":"2024-09-10T10:55:49.862812+0000","last_became_active":"2024-09-10T10:55:47.850457+0000","last_became_peered":"2024-09-10T10:55:47.850457+0000","last_unstale":"2024-09-10T10:55:49.862812+0000","last_undegraded":"2024-09-10T10:55:49.862812+0000","last_fullsized":"2024-09-10T10:55:49.862812+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:45.564987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947047+0000","last_change":"2024-09-10T10:55:48.861530+0000","last_active":"2024-09-10T10:55:49.947047+0000","last_peered":"2024-09-10T10:55:49.947047+0000","last_clean":"2024-09-10T10:55:49.947047+0000","last_became_active":"2024-09-10T10:55:48.860648+0000","last_became_peered":"2024-09-10T10:55:48.860648+0000","last_unstale":"2024-09-10T10:55:49.947047+0000","last_undegraded":"2024-09-10T10:55:49.947047+0000","last_fullsized":"2024-09-10T10:55:49.947047+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:28:36.041255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865960+0000","last_change":"2024-09-10T10:55:47.843477+0000","last_active":"2024-09-10T10:55:49.865960+0000","last_peered":"2024-09-10T10:55:49.865960+0000","last_clean":"2024-09-10T10:55:49.865960+0000","last_became_active":"2024-09-10T10:55:47.841371+0000","last_became_peered":"2024-09-10T10:55:47.841371+0000","last_unstale":"2024-09-10T10:55:49.865960+0000","last_undegraded":"2024-09-10T10:55:49.865960+0000","last_fullsized":"2024-09-10T10:55:49.865960+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:45.636397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947145+0000","last_change":"2024-09-10T10:55:48.865788+0000","last_active":"2024-09-10T10:55:49.947145+0000","last_peered":"2024-09-10T10:55:49.947145+0000","last_clean":"2024-09-10T10:55:49.947145+0000","last_became_active":"2024-09-10T10:55:48.865486+0000","last_became_peered":"2024-09-10T10:55:48.865486+0000","last_unstale":"2024-09-10T10:55:49.947145+0000","last_undegraded":"2024-09-10T10:55:49.947145+0000","last_fullsized":"2024-09-10T10:55:49.947145+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:56.003393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961925+0000","last_change":"2024-09-10T10:55:47.845313+0000","last_active":"2024-09-10T10:55:49.961925+0000","last_peered":"2024-09-10T10:55:49.961925+0000","last_clean":"2024-09-10T10:55:49.961925+0000","last_became_active":"2024-09-10T10:55:47.844999+0000","last_became_peered":"2024-09-10T10:55:47.844999+0000","last_unstale":"2024-09-10T10:55:49.961925+0000","last_undegraded":"2024-09-10T10:55:49.961925+0000","last_fullsized":"2024-09-10T10:55:49.961925+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:07.475226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961293+0000","last_change":"2024-09-10T10:55:48.877412+0000","last_active":"2024-09-10T10:55:49.961293+0000","last_peered":"2024-09-10T10:55:49.961293+0000","last_clean":"2024-09-10T10:55:49.961293+0000","last_became_active":"2024-09-10T10:55:48.876383+0000","last_became_peered":"2024-09-10T10:55:48.876383+0000","last_unstale":"2024-09-10T10:55:49.961293+0000","last_undegraded":"2024-09-10T10:55:49.961293+0000","last_fullsized":"2024-09-10T10:55:49.961293+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:10:41.647291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865237+0000","last_change":"2024-09-10T10:55:47.834691+0000","last_active":"2024-09-10T10:55:49.865237+0000","last_peered":"2024-09-10T10:55:49.865237+0000","last_clean":"2024-09-10T10:55:49.865237+0000","last_became_active":"2024-09-10T10:55:47.834400+0000","last_became_peered":"2024-09-10T10:55:47.834400+0000","last_unstale":"2024-09-10T10:55:49.865237+0000","last_undegraded":"2024-09-10T10:55:49.865237+0000","last_fullsized":"2024-09-10T10:55:49.865237+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:40:52.430885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865294+0000","last_change":"2024-09-10T10:55:48.875693+0000","last_active":"2024-09-10T10:55:49.865294+0000","last_peered":"2024-09-10T10:55:49.865294+0000","last_clean":"2024-09-10T10:55:49.865294+0000","last_became_active":"2024-09-10T10:55:48.875098+0000","last_became_peered":"2024-09-10T10:55:48.875098+0000","last_unstale":"2024-09-10T10:55:49.865294+0000","last_undegraded":"2024-09-10T10:55:49.865294+0000","last_fullsized":"2024-09-10T10:55:49.865294+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:33.767751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"149'8","reported_seq":22,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862623+0000","last_change":"2024-09-10T10:55:47.843176+0000","last_active":"2024-09-10T10:55:49.862623+0000","last_peered":"2024-09-10T10:55:49.862623+0000","last_clean":"2024-09-10T10:55:49.862623+0000","last_became_active":"2024-09-10T10:55:47.842914+0000","last_became_peered":"2024-09-10T10:55:47.842914+0000","last_unstale":"2024-09-10T10:55:49.862623+0000","last_undegraded":"2024-09-10T10:55:49.862623+0000","last_fullsized":"2024-09-10T10:55:49.862623+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:34:46.876235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961424+0000","last_change":"2024-09-10T10:55:48.874049+0000","last_active":"2024-09-10T10:55:49.961424+0000","last_peered":"2024-09-10T10:55:49.961424+0000","last_clean":"2024-09-10T10:55:49.961424+0000","last_became_active":"2024-09-10T10:55:48.873744+0000","last_became_peered":"2024-09-10T10:55:48.873744+0000","last_unstale":"2024-09-10T10:55:49.961424+0000","last_undegraded":"2024-09-10T10:55:49.961424+0000","last_fullsized":"2024-09-10T10:55:49.961424+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:16:25.276511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872621+0000","last_change":"2024-09-10T10:55:47.845166+0000","last_active":"2024-09-10T10:55:49.872621+0000","last_peered":"2024-09-10T10:55:49.872621+0000","last_clean":"2024-09-10T10:55:49.872621+0000","last_became_active":"2024-09-10T10:55:47.844675+0000","last_became_peered":"2024-09-10T10:55:47.844675+0000","last_unstale":"2024-09-10T10:55:49.872621+0000","last_undegraded":"2024-09-10T10:55:49.872621+0000","last_fullsized":"2024-09-10T10:55:49.872621+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:58:26.167518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960321+0000","last_change":"2024-09-10T10:55:48.872087+0000","last_active":"2024-09-10T10:55:49.960321+0000","last_peered":"2024-09-10T10:55:49.960321+0000","last_clean":"2024-09-10T10:55:49.960321+0000","last_became_active":"2024-09-10T10:55:48.871832+0000","last_became_peered":"2024-09-10T10:55:48.871832+0000","last_unstale":"2024-09-10T10:55:49.960321+0000","last_undegraded":"2024-09-10T10:55:49.960321+0000","last_fullsized":"2024-09-10T10:55:49.960321+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:02:58.954198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"149'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960717+0000","last_change":"2024-09-10T10:55:47.850066+0000","last_active":"2024-09-10T10:55:49.960717+0000","last_peered":"2024-09-10T10:55:49.960717+0000","last_clean":"2024-09-10T10:55:49.960717+0000","last_became_active":"2024-09-10T10:55:47.849775+0000","last_became_peered":"2024-09-10T10:55:47.849775+0000","last_unstale":"2024-09-10T10:55:49.960717+0000","last_undegraded":"2024-09-10T10:55:49.960717+0000","last_fullsized":"2024-09-10T10:55:49.960717+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:29.865577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961193+0000","last_change":"2024-09-10T10:55:48.860009+0000","last_active":"2024-09-10T10:55:49.961193+0000","last_peered":"2024-09-10T10:55:49.961193+0000","last_clean":"2024-09-10T10:55:49.961193+0000","last_became_active":"2024-09-10T10:55:48.859725+0000","last_became_peered":"2024-09-10T10:55:48.859725+0000","last_unstale":"2024-09-10T10:55:49.961193+0000","last_undegraded":"2024-09-10T10:55:49.961193+0000","last_fullsized":"2024-09-10T10:55:49.961193+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:42:06.818156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872362+0000","last_change":"2024-09-10T10:55:47.846489+0000","last_active":"2024-09-10T10:55:49.872362+0000","last_peered":"2024-09-10T10:55:49.872362+0000","last_clean":"2024-09-10T10:55:49.872362+0000","last_became_active":"2024-09-10T10:55:47.846067+0000","last_became_peered":"2024-09-10T10:55:47.846067+0000","last_unstale":"2024-09-10T10:55:49.872362+0000","last_undegraded":"2024-09-10T10:55:49.872362+0000","last_fullsized":"2024-09-10T10:55:49.872362+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:30:28.004930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865650+0000","last_change":"2024-09-10T10:55:48.857694+0000","last_active":"2024-09-10T10:55:49.865650+0000","last_peered":"2024-09-10T10:55:49.865650+0000","last_clean":"2024-09-10T10:55:49.865650+0000","last_became_active":"2024-09-10T10:55:48.857007+0000","last_became_peered":"2024-09-10T10:55:48.857007+0000","last_unstale":"2024-09-10T10:55:49.865650+0000","last_undegraded":"2024-09-10T10:55:49.865650+0000","last_fullsized":"2024-09-10T10:55:49.865650+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:53:15.966260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864738+0000","last_change":"2024-09-10T10:55:47.846071+0000","last_active":"2024-09-10T10:55:49.864738+0000","last_peered":"2024-09-10T10:55:49.864738+0000","last_clean":"2024-09-10T10:55:49.864738+0000","last_became_active":"2024-09-10T10:55:47.845052+0000","last_became_peered":"2024-09-10T10:55:47.845052+0000","last_unstale":"2024-09-10T10:55:49.864738+0000","last_undegraded":"2024-09-10T10:55:49.864738+0000","last_fullsized":"2024-09-10T10:55:49.864738+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:39:41.842448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961121+0000","last_change":"2024-09-10T10:55:48.868962+0000","last_active":"2024-09-10T10:55:49.961121+0000","last_peered":"2024-09-10T10:55:49.961121+0000","last_clean":"2024-09-10T10:55:49.961121+0000","last_became_active":"2024-09-10T10:55:48.868602+0000","last_became_peered":"2024-09-10T10:55:48.868602+0000","last_unstale":"2024-09-10T10:55:49.961121+0000","last_undegraded":"2024-09-10T10:55:49.961121+0000","last_fullsized":"2024-09-10T10:55:49.961121+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:23:37.194259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864393+0000","last_change":"2024-09-10T10:55:47.846363+0000","last_active":"2024-09-10T10:55:49.864393+0000","last_peered":"2024-09-10T10:55:49.864393+0000","last_clean":"2024-09-10T10:55:49.864393+0000","last_became_active":"2024-09-10T10:55:47.846001+0000","last_became_peered":"2024-09-10T10:55:47.846001+0000","last_unstale":"2024-09-10T10:55:49.864393+0000","last_undegraded":"2024-09-10T10:55:49.864393+0000","last_fullsized":"2024-09-10T10:55:49.864393+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:52.524852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961231+0000","last_change":"2024-09-10T10:55:48.861960+0000","last_active":"2024-09-10T10:55:49.961231+0000","last_peered":"2024-09-10T10:55:49.961231+0000","last_clean":"2024-09-10T10:55:49.961231+0000","last_became_active":"2024-09-10T10:55:48.861620+0000","last_became_peered":"2024-09-10T10:55:48.861620+0000","last_unstale":"2024-09-10T10:55:49.961231+0000","last_undegraded":"2024-09-10T10:55:49.961231+0000","last_fullsized":"2024-09-10T10:55:49.961231+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:27.421498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872025+0000","last_change":"2024-09-10T10:55:47.837440+0000","last_active":"2024-09-10T10:55:49.872025+0000","last_peered":"2024-09-10T10:55:49.872025+0000","last_clean":"2024-09-10T10:55:49.872025+0000","last_became_active":"2024-09-10T10:55:47.836741+0000","last_became_peered":"2024-09-10T10:55:47.836741+0000","last_unstale":"2024-09-10T10:55:49.872025+0000","last_undegraded":"2024-09-10T10:55:49.872025+0000","last_fullsized":"2024-09-10T10:55:49.872025+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:58.662475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865678+0000","last_change":"2024-09-10T10:55:48.867356+0000","last_active":"2024-09-10T10:55:49.865678+0000","last_peered":"2024-09-10T10:55:49.865678+0000","last_clean":"2024-09-10T10:55:49.865678+0000","last_became_active":"2024-09-10T10:55:48.866816+0000","last_became_peered":"2024-09-10T10:55:48.866816+0000","last_unstale":"2024-09-10T10:55:49.865678+0000","last_undegraded":"2024-09-10T10:55:49.865678+0000","last_fullsized":"2024-09-10T10:55:49.865678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:39:17.379036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961500+0000","last_change":"2024-09-10T10:55:47.845337+0000","last_active":"2024-09-10T10:55:49.961500+0000","last_peered":"2024-09-10T10:55:49.961500+0000","last_clean":"2024-09-10T10:55:49.961500+0000","last_became_active":"2024-09-10T10:55:47.845077+0000","last_became_peered":"2024-09-10T10:55:47.845077+0000","last_unstale":"2024-09-10T10:55:49.961500+0000","last_undegraded":"2024-09-10T10:55:49.961500+0000","last_fullsized":"2024-09-10T10:55:49.961500+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:14.274310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960779+0000","last_change":"2024-09-10T10:55:48.864373+0000","last_active":"2024-09-10T10:55:49.960779+0000","last_peered":"2024-09-10T10:55:49.960779+0000","last_clean":"2024-09-10T10:55:49.960779+0000","last_became_active":"2024-09-10T10:55:48.864041+0000","last_became_peered":"2024-09-10T10:55:48.864041+0000","last_unstale":"2024-09-10T10:55:49.960779+0000","last_undegraded":"2024-09-10T10:55:49.960779+0000","last_fullsized":"2024-09-10T10:55:49.960779+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:39.301780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865474+0000","last_change":"2024-09-10T10:55:47.850713+0000","last_active":"2024-09-10T10:55:49.865474+0000","last_peered":"2024-09-10T10:55:49.865474+0000","last_clean":"2024-09-10T10:55:49.865474+0000","last_became_active":"2024-09-10T10:55:47.849850+0000","last_became_peered":"2024-09-10T10:55:47.849850+0000","last_unstale":"2024-09-10T10:55:49.865474+0000","last_undegraded":"2024-09-10T10:55:49.865474+0000","last_fullsized":"2024-09-10T10:55:49.865474+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:17:18.172214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862722+0000","last_change":"2024-09-10T10:55:48.859546+0000","last_active":"2024-09-10T10:55:49.862722+0000","last_peered":"2024-09-10T10:55:49.862722+0000","last_clean":"2024-09-10T10:55:49.862722+0000","last_became_active":"2024-09-10T10:55:48.859164+0000","last_became_peered":"2024-09-10T10:55:48.859164+0000","last_unstale":"2024-09-10T10:55:49.862722+0000","last_undegraded":"2024-09-10T10:55:49.862722+0000","last_fullsized":"2024-09-10T10:55:49.862722+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:01:29.838709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"150'1","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865073+0000","last_change":"2024-09-10T10:55:47.853293+0000","last_active":"2024-09-10T10:55:49.865073+0000","last_peered":"2024-09-10T10:55:49.865073+0000","last_clean":"2024-09-10T10:55:49.865073+0000","last_became_active":"2024-09-10T10:55:47.852967+0000","last_became_peered":"2024-09-10T10:55:47.852967+0000","last_unstale":"2024-09-10T10:55:49.865073+0000","last_undegraded":"2024-09-10T10:55:49.865073+0000","last_fullsized":"2024-09-10T10:55:49.865073+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:03:09.128890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.961880+0000","last_change":"2024-09-10T10:55:48.874714+0000","last_active":"2024-09-10T10:55:49.961880+0000","last_peered":"2024-09-10T10:55:49.961880+0000","last_clean":"2024-09-10T10:55:49.961880+0000","last_became_active":"2024-09-10T10:55:48.874256+0000","last_became_peered":"2024-09-10T10:55:48.874256+0000","last_unstale":"2024-09-10T10:55:49.961880+0000","last_undegraded":"2024-09-10T10:55:49.961880+0000","last_fullsized":"2024-09-10T10:55:49.961880+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:04.561493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862937+0000","last_change":"2024-09-10T10:55:47.852303+0000","last_active":"2024-09-10T10:55:49.862937+0000","last_peered":"2024-09-10T10:55:49.862937+0000","last_clean":"2024-09-10T10:55:49.862937+0000","last_became_active":"2024-09-10T10:55:47.851277+0000","last_became_peered":"2024-09-10T10:55:47.851277+0000","last_unstale":"2024-09-10T10:55:49.862937+0000","last_undegraded":"2024-09-10T10:55:49.862937+0000","last_fullsized":"2024-09-10T10:55:49.862937+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:43.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960557+0000","last_change":"2024-09-10T10:55:48.876162+0000","last_active":"2024-09-10T10:55:49.960557+0000","last_peered":"2024-09-10T10:55:49.960557+0000","last_clean":"2024-09-10T10:55:49.960557+0000","last_became_active":"2024-09-10T10:55:48.875834+0000","last_became_peered":"2024-09-10T10:55:48.875834+0000","last_unstale":"2024-09-10T10:55:49.960557+0000","last_undegraded":"2024-09-10T10:55:49.960557+0000","last_fullsized":"2024-09-10T10:55:49.960557+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:59:12.003746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.866001+0000","last_change":"2024-09-10T10:55:47.844106+0000","last_active":"2024-09-10T10:55:49.866001+0000","last_peered":"2024-09-10T10:55:49.866001+0000","last_clean":"2024-09-10T10:55:49.866001+0000","last_became_active":"2024-09-10T10:55:47.843763+0000","last_became_peered":"2024-09-10T10:55:47.843763+0000","last_unstale":"2024-09-10T10:55:49.866001+0000","last_undegraded":"2024-09-10T10:55:49.866001+0000","last_fullsized":"2024-09-10T10:55:49.866001+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:19.517216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864866+0000","last_change":"2024-09-10T10:55:48.857933+0000","last_active":"2024-09-10T10:55:49.864866+0000","last_peered":"2024-09-10T10:55:49.864866+0000","last_clean":"2024-09-10T10:55:49.864866+0000","last_became_active":"2024-09-10T10:55:48.857414+0000","last_became_peered":"2024-09-10T10:55:48.857414+0000","last_unstale":"2024-09-10T10:55:49.864866+0000","last_undegraded":"2024-09-10T10:55:49.864866+0000","last_fullsized":"2024-09-10T10:55:49.864866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:44:52.855868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.862873+0000","last_change":"2024-09-10T10:55:47.846380+0000","last_active":"2024-09-10T10:55:49.862873+0000","last_peered":"2024-09-10T10:55:49.862873+0000","last_clean":"2024-09-10T10:55:49.862873+0000","last_became_active":"2024-09-10T10:55:47.846123+0000","last_became_peered":"2024-09-10T10:55:47.846123+0000","last_unstale":"2024-09-10T10:55:49.862873+0000","last_undegraded":"2024-09-10T10:55:49.862873+0000","last_fullsized":"2024-09-10T10:55:49.862873+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:16.465904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.865669+0000","last_change":"2024-09-10T10:55:48.859593+0000","last_active":"2024-09-10T10:55:49.865669+0000","last_peered":"2024-09-10T10:55:49.865669+0000","last_clean":"2024-09-10T10:55:49.865669+0000","last_became_active":"2024-09-10T10:55:48.859045+0000","last_became_peered":"2024-09-10T10:55:48.859045+0000","last_unstale":"2024-09-10T10:55:49.865669+0000","last_undegraded":"2024-09-10T10:55:49.865669+0000","last_fullsized":"2024-09-10T10:55:49.865669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:54:39.427524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864454+0000","last_change":"2024-09-10T10:55:47.853193+0000","last_active":"2024-09-10T10:55:49.864454+0000","last_peered":"2024-09-10T10:55:49.864454+0000","last_clean":"2024-09-10T10:55:49.864454+0000","last_became_active":"2024-09-10T10:55:47.852861+0000","last_became_peered":"2024-09-10T10:55:47.852861+0000","last_unstale":"2024-09-10T10:55:49.864454+0000","last_undegraded":"2024-09-10T10:55:49.864454+0000","last_fullsized":"2024-09-10T10:55:49.864454+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:06:34.277448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.872528+0000","last_change":"2024-09-10T10:55:48.870415+0000","last_active":"2024-09-10T10:55:49.872528+0000","last_peered":"2024-09-10T10:55:49.872528+0000","last_clean":"2024-09-10T10:55:49.872528+0000","last_became_active":"2024-09-10T10:55:48.869966+0000","last_became_peered":"2024-09-10T10:55:48.869966+0000","last_unstale":"2024-09-10T10:55:49.872528+0000","last_undegraded":"2024-09-10T10:55:49.872528+0000","last_fullsized":"2024-09-10T10:55:49.872528+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:32:46.642437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960416+0000","last_change":"2024-09-10T10:55:47.850818+0000","last_active":"2024-09-10T10:55:49.960416+0000","last_peered":"2024-09-10T10:55:49.960416+0000","last_clean":"2024-09-10T10:55:49.960416+0000","last_became_active":"2024-09-10T10:55:47.850541+0000","last_became_peered":"2024-09-10T10:55:47.850541+0000","last_unstale":"2024-09-10T10:55:49.960416+0000","last_undegraded":"2024-09-10T10:55:49.960416+0000","last_fullsized":"2024-09-10T10:55:49.960416+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:08:54.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.947271+0000","last_change":"2024-09-10T10:55:48.862694+0000","last_active":"2024-09-10T10:55:49.947271+0000","last_peered":"2024-09-10T10:55:49.947271+0000","last_clean":"2024-09-10T10:55:49.947271+0000","last_became_active":"2024-09-10T10:55:48.862367+0000","last_became_peered":"2024-09-10T10:55:48.862367+0000","last_unstale":"2024-09-10T10:55:49.947271+0000","last_undegraded":"2024-09-10T10:55:49.947271+0000","last_fullsized":"2024-09-10T10:55:49.947271+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:08.481266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864269+0000","last_change":"2024-09-10T10:55:47.846160+0000","last_active":"2024-09-10T10:55:49.864269+0000","last_peered":"2024-09-10T10:55:49.864269+0000","last_clean":"2024-09-10T10:55:49.864269+0000","last_became_active":"2024-09-10T10:55:47.845758+0000","last_became_peered":"2024-09-10T10:55:47.845758+0000","last_unstale":"2024-09-10T10:55:49.864269+0000","last_undegraded":"2024-09-10T10:55:49.864269+0000","last_fullsized":"2024-09-10T10:55:49.864269+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:35.560191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.871975+0000","last_change":"2024-09-10T10:55:48.867075+0000","last_active":"2024-09-10T10:55:49.871975+0000","last_peered":"2024-09-10T10:55:49.871975+0000","last_clean":"2024-09-10T10:55:49.871975+0000","last_became_active":"2024-09-10T10:55:48.859669+0000","last_became_peered":"2024-09-10T10:55:48.859669+0000","last_unstale":"2024-09-10T10:55:49.871975+0000","last_undegraded":"2024-09-10T10:55:49.871975+0000","last_fullsized":"2024-09-10T10:55:49.871975+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:47.821270+0000","last_clean_scrub_stamp":"2024-09-10T10:55:47.821270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:45:05.994404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"149'2","reported_seq":16,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.864565+0000","last_change":"2024-09-10T10:55:47.847392+0000","last_active":"2024-09-10T10:55:49.864565+0000","last_peered":"2024-09-10T10:55:49.864565+0000","last_clean":"2024-09-10T10:55:49.864565+0000","last_became_active":"2024-09-10T10:55:47.847116+0000","last_became_peered":"2024-09-10T10:55:47.847116+0000","last_unstale":"2024-09-10T10:55:49.864565+0000","last_undegraded":"2024-09-10T10:55:49.864565+0000","last_fullsized":"2024-09-10T10:55:49.864565+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:55:46.815230+0000","last_clean_scrub_stamp":"2024-09-10T10:55:46.815230+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:13:38.621928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":416,"reported_epoch":150,"state":"active+clean","last_fresh":"2024-09-10T10:55:49.960208+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:55:49.960208+0000","last_peered":"2024-09-10T10:55:49.960208+0000","last_clean":"2024-09-10T10:55:49.960208+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:55:49.960208+0000","last_undegraded":"2024-09-10T10:55:49.960208+0000","last_fullsized":"2024-09-10T10:55:49.960208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1941504,"data_stored":978009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509592,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28936,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342904,"statfs":{"total":96636764160,"available":96607133696,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1937408,"data_stored":977391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41408,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330432,"statfs":{"total":96636764160,"available":96594362368,"internally_reserved":0,"allocated":1945600,"data_stored":978969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37252,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334588,"statfs":{"total":96636764160,"available":96598618112,"internally_reserved":0,"allocated":1941504,"data_stored":977469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41988,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329852,"statfs":{"total":96636764160,"available":96593768448,"internally_reserved":0,"allocated":2535424,"data_stored":1567817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37248,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334592,"statfs":{"total":96636764160,"available":96598622208,"internally_reserved":0,"allocated":1937408,"data_stored":977939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33664,"kb_used_data":2476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338176,"statfs":{"total":96636764160,"available":96602292224,"internally_reserved":0,"allocated":2535424,"data_stored":1567849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:55:58.171 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:55:58.172 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T10:55:58.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.371+0000 7f7a68e00640 1 Processor -- start 2024-09-10T10:55:58.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.371+0000 7f7a68e00640 1 -- start start 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.372+0000 7f7a68e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a6415e410 0x7f7a640b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a68e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7a6406bc10 0x7f7a6406c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a68e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a640ae240 con 0x7f7a640b40c0 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a68e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7a6406c580 con 0x7f7a6406bc10 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a68e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a640ae520 con 0x7f7a6415e410 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a59400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7a640b40c0 0x7f7a640aff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54288/0 (socket says 172.21.15.28:54288) 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a59400640 1 -- 172.21.15.28:0/1623584108 learned_addr learned my addr 172.21.15.28:0/1623584108 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a62800640 1 --2- 172.21.15.28:0/1623584108 >> v2:172.21.15.138:3300/0 conn(0x7f7a6406bc10 0x7f7a6406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a61e00640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a6415e410 0x7f7a640b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418088390 0 0) 0x7f7a640ae240 con 0x7f7a640b40c0 2024-09-10T10:55:58.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.373+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a48003520 con 0x7f7a640b40c0 2024-09-10T10:55:58.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.374+0000 7f7a62800640 1 -- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a6415e410 msgr2=0x7f7a640b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:58.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.374+0000 7f7a62800640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a6415e410 0x7f7a640b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.374+0000 7f7a62800640 1 -- 172.21.15.28:0/1623584108 >> v1:172.21.15.28:6789/0 conn(0x7f7a640b40c0 legacy=0x7f7a640aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.374+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 631901080 0 0) 0x7f7a48003520 con 0x7f7a640b40c0 2024-09-10T10:55:58.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.374+0000 7f7a62800640 1 -- 172.21.15.28:0/1623584108 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a640ae800 con 0x7f7a6406bc10 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.375+0000 7f7a62800640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a6406c040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009d90 tx=0x7f7a5c02a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20136340b1655658 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.375+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c001670 con 0x7f7a6406bc10 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.375+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c007540 con 0x7f7a6406bc10 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.375+0000 7f7a61400640 1 -- 172.21.15.28:0/1623584108 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0078f0 con 0x7f7a6406bc10 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 -- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 msgr2=0x7f7a6406c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a6406c040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009d90 tx=0x7f7a5c02a560 comp rx=0 tx=0).stop 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 -- 172.21.15.28:0/1623584108 shutdown_connections 2024-09-10T10:55:58.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a6406c040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 --2- 172.21.15.28:0/1623584108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a6415e410 0x7f7a640b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 -- 172.21.15.28:0/1623584108 >> 172.21.15.28:0/1623584108 conn(0x7f7a6409aa30 msgr2=0x7f7a640b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 -- 172.21.15.28:0/1623584108 shutdown_connections 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.376+0000 7f7a68e00640 1 -- 172.21.15.28:0/1623584108 wait complete. 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.377+0000 7f7a68e00640 1 Processor -- start 2024-09-10T10:55:58.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 -- start start 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a64148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a640b40c0 0x7f7a641536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6415e7f0 con 0x7f7a6415e410 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a640b4510 con 0x7f7a6406bc10 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.378+0000 7f7a68e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6415e970 con 0x7f7a640b40c0 2024-09-10T10:55:58.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55146/0 (socket says 172.21.15.28:55146) 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 -- 172.21.15.28:0/602305397 learned_addr learned my addr 172.21.15.28:0/602305397 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a59400640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a640b40c0 0x7f7a641536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a61e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a64148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 -- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a640b40c0 msgr2=0x7f7a641536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a640b40c0 0x7f7a641536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 -- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 msgr2=0x7f7a64148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a64148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.379+0000 7f7a62800640 1 -- 172.21.15.28:0/602305397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a64155f80 con 0x7f7a6415e410 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a62800640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02ab50 tx=0x7f7a5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c009450 con 0x7f7a6415e410 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c004240 con 0x7f7a6415e410 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0045f0 con 0x7f7a6415e410 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a64156260 con 0x7f7a6415e410 2024-09-10T10:55:58.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.380+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a64156820 con 0x7f7a6415e410 2024-09-10T10:55:58.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.381+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a6406c040 con 0x7f7a6415e410 2024-09-10T10:55:58.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.382+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a5c03b410 con 0x7f7a6415e410 2024-09-10T10:55:58.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.383+0000 7f7a5aa00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 0x7f7a4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.383+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f7a5c0ac130 con 0x7f7a6415e410 2024-09-10T10:55:58.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.384+0000 7f7a61e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 0x7f7a4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.384+0000 7f7a61e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 0x7f7a4006e2f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009b40 tx=0x7f7a54009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.388+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a5c03f030 con 0x7f7a6415e410 2024-09-10T10:55:58.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.639+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7a640b1630 con 0x7f7a6415e410 2024-09-10T10:55:58.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.642+0000 7f7a5aa00640 1 -- 172.21.15.28:0/602305397 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f7a5c07a390 con 0x7f7a6415e410 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQCiIuBmLe57ERAA3yXVR2nOf3IuybS6Ad2WcA== 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T10:55:58.643 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-09-10T10:55:58.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 msgr2=0x7f7a4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 0x7f7a4006e2f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009b40 tx=0x7f7a54009190 comp rx=0 tx=0).stop 2024-09-10T10:55:58.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 msgr2=0x7f7a641552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02ab50 tx=0x7f7a5c004a40 comp rx=0 tx=0).stop 2024-09-10T10:55:58.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 shutdown_connections 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a4006be30 0x7f7a4006e2f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a6415e410 0x7f7a641552b0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a640b40c0 0x7f7a641536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 --2- 172.21.15.28:0/602305397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a6406bc10 0x7f7a64148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 >> 172.21.15.28:0/602305397 conn(0x7f7a6409aa30 msgr2=0x7f7a640b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 shutdown_connections 2024-09-10T10:55:58.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.644+0000 7f7a68e00640 1 -- 172.21.15.28:0/602305397 wait complete. 2024-09-10T10:55:58.661 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:55:58.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.817+0000 7f6d63000640 1 Processor -- start 2024-09-10T10:55:58.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.817+0000 7f6d63000640 1 -- start start 2024-09-10T10:55:58.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d63000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d5c15e660 0x7f6d5c06ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d63000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0cf000 con 0x7f6d5c0d1f90 2024-09-10T10:55:58.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d63000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d5c0d29a0 con 0x7f6d5c0d0dd0 2024-09-10T10:55:58.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d63000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d5c0b37d0 con 0x7f6d5c15e660 2024-09-10T10:55:58.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d62600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6d5c0d1f90 0x7f6d5c0cf7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54298/0 (socket says 172.21.15.28:54298) 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d62600640 1 -- 172.21.15.28:0/2166262813 learned_addr learned my addr 172.21.15.28:0/2166262813 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.818+0000 7f6d61200640 1 --2- 172.21.15.28:0/2166262813 >> v2:172.21.15.138:3300/0 conn(0x7f6d5c15e660 0x7f6d5c06ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.819+0000 7f6d5be00640 1 -- 172.21.15.28:0/2166262813 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542135868 0 0) 0x7f6d5c0cf000 con 0x7f6d5c0d1f90 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.819+0000 7f6d5be00640 1 -- 172.21.15.28:0/2166262813 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d44003520 con 0x7f6d5c0d1f90 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.819+0000 7f6d61200640 1 -- 172.21.15.28:0/2166262813 >> v1:172.21.15.138:6789/0 conn(0x7f6d5c0d0dd0 legacy=0x7f6d5c0d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.819+0000 7f6d61200640 1 -- 172.21.15.28:0/2166262813 >> v1:172.21.15.28:6789/0 conn(0x7f6d5c0d1f90 legacy=0x7f6d5c0cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.819+0000 7f6d61200640 1 -- 172.21.15.28:0/2166262813 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c0d2c60 con 0x7f6d5c15e660 2024-09-10T10:55:58.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.820+0000 7f6d61200640 1 --2- 172.21.15.28:0/2166262813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c15e660 0x7f6d5c06ccd0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097a0 tx=0x7f6d4802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb775c48c4bcd596 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.820+0000 7f6d5be00640 1 -- 172.21.15.28:0/2166262813 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d48037070 con 0x7f6d5c15e660 2024-09-10T10:55:58.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.820+0000 7f6d5be00640 1 -- 172.21.15.28:0/2166262813 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d48032440 con 0x7f6d5c15e660 2024-09-10T10:55:58.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.820+0000 7f6d5be00640 1 -- 172.21.15.28:0/2166262813 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d480327f0 con 0x7f6d5c15e660 2024-09-10T10:55:58.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 -- 172.21.15.28:0/2166262813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c15e660 msgr2=0x7f6d5c06ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 --2- 172.21.15.28:0/2166262813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c15e660 0x7f6d5c06ccd0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6d480097a0 tx=0x7f6d4802a230 comp rx=0 tx=0).stop 2024-09-10T10:55:58.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 -- 172.21.15.28:0/2166262813 shutdown_connections 2024-09-10T10:55:58.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 --2- 172.21.15.28:0/2166262813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c15e660 0x7f6d5c06ccd0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 -- 172.21.15.28:0/2166262813 >> 172.21.15.28:0/2166262813 conn(0x7f6d5c08bfc0 msgr2=0x7f6d5c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 -- 172.21.15.28:0/2166262813 shutdown_connections 2024-09-10T10:55:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.821+0000 7f6d63000640 1 -- 172.21.15.28:0/2166262813 wait complete. 2024-09-10T10:55:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.823+0000 7f6d63000640 1 Processor -- start 2024-09-10T10:55:58.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.823+0000 7f6d63000640 1 -- start start 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 0x7f6d5c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5c0d1f90 0x7f6d5c0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d3220 con 0x7f6d5c15e660 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5c0cf330 con 0x7f6d5c0d0dd0 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d63000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d5c0d33a0 con 0x7f6d5c0d1f90 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d62600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d62600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55160/0 (socket says 172.21.15.28:55160) 2024-09-10T10:55:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d62600640 1 -- 172.21.15.28:0/2571632089 learned_addr learned my addr 172.21.15.28:0/2571632089 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d61200640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5c0d1f90 0x7f6d5c0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d61c00640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 0x7f6d5c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d62600640 1 -- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5c0d1f90 msgr2=0x7f6d5c0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.824+0000 7f6d62600640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5c0d1f90 0x7f6d5c0ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d62600640 1 -- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 msgr2=0x7f6d5c154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d62600640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 0x7f6d5c154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d62600640 1 -- 172.21.15.28:0/2571632089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5c154870 con 0x7f6d5c15e660 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d61c00640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 0x7f6d5c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d62600640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400cb40 tx=0x7f6d5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54017070 con 0x7f6d5c15e660 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5400ac40 con 0x7f6d5c15e660 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54012620 con 0x7f6d5c15e660 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5c154b50 con 0x7f6d5c15e660 2024-09-10T10:55:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.825+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d5c155110 con 0x7f6d5c15e660 2024-09-10T10:55:58.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.826+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d5c15dea0 con 0x7f6d5c15e660 2024-09-10T10:55:58.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.827+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d5401a070 con 0x7f6d5c15e660 2024-09-10T10:55:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.828+0000 7f6d59600640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 0x7f6d3c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.829+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f6d5408ad60 con 0x7f6d5c15e660 2024-09-10T10:55:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.829+0000 7f6d61c00640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 0x7f6d3c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.830+0000 7f6d61c00640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 0x7f6d3c06e320 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6d50007fd0 tx=0x7f6d50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:58.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:58.833+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d5c15dea0 con 0x7f6d5c15e660 2024-09-10T10:55:59.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.055+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d5c0d1b20 con 0x7f6d5c15e660 2024-09-10T10:55:59.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.055+0000 7f6d59600640 1 -- 172.21.15.28:0/2571632089 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5234 (secure 0 0 0) 0x7f6d54058f20 con 0x7f6d5c15e660 2024-09-10T10:55:59.057 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:55:59.057 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":341,"btime":"2024-09-10T10:55:51:866579+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:55:51.288071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9505":{"gid":9505,"name":"a","rank":0,"incarnation":337,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/354907385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":354907385},{"type":"v1","addr":"172.21.15.28:6835","nonce":354907385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9505,"qdb_cluster":[9505]},"id":7}]} 2024-09-10T10:55:59.057 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 341 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.057+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 msgr2=0x7f6d3c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.057+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 0x7f6d3c06e320 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6d50007fd0 tx=0x7f6d50009040 comp rx=0 tx=0).stop 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.057+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 msgr2=0x7f6d5c158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.057+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400cb40 tx=0x7f6d5400a060 comp rx=0 tx=0).stop 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 shutdown_connections 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d3c06be60 0x7f6d3c06e320 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d5c15e660 0x7f6d5c158370 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5c0d1f90 0x7f6d5c0ce3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 --2- 172.21.15.28:0/2571632089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5c0d0dd0 0x7f6d5c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 >> 172.21.15.28:0/2571632089 conn(0x7f6d5c08bfc0 msgr2=0x7f6d5c0d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 shutdown_connections 2024-09-10T10:55:59.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.058+0000 7f6d63000640 1 -- 172.21.15.28:0/2571632089 wait complete. 2024-09-10T10:55:59.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 341, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:55:48.269671+0000', 'modified': '2024-09-10T10:55:51.288071+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9505': {'gid': 9505, 'name': 'a', 'rank': 0, 'incarnation': 337, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6835/354907385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 354907385}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 354907385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9505, 'qdb_cluster': [9505]} 2024-09-10T10:55:59.478 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:55:59.478 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:55:59.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.635+0000 7f6f25a00640 1 Processor -- start 2024-09-10T10:55:59.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.635+0000 7f6f25a00640 1 -- start start 2024-09-10T10:55:59.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 0x7f6f200720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f25a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6f200725f0 0x7f6f200aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f25a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f2006f480 con 0x7f6f20147230 2024-09-10T10:55:59.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f25a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6f200b34d0 con 0x7f6f200725f0 2024-09-10T10:55:59.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f25a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f200ae9e0 con 0x7f6f20147e90 2024-09-10T10:55:59.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f1ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6f20147230 0x7f6f2006e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:34360/0 (socket says 172.21.15.28:34360) 2024-09-10T10:55:59.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f1ea00640 1 -- 172.21.15.28:0/3440828969 learned_addr learned my addr 172.21.15.28:0/3440828969 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:59.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f16000640 1 --2- 172.21.15.28:0/3440828969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 0x7f6f200720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.637+0000 7f6f1f400640 1 --2- 172.21.15.28:0/3440828969 >> v2:172.21.15.138:3300/0 conn(0x7f6f200725f0 0x7f6f200aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.638+0000 7f6f1e000640 1 -- 172.21.15.28:0/3440828969 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962034321 0 0) 0x7f6f2006f480 con 0x7f6f20147230 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.638+0000 7f6f16000640 1 -- 172.21.15.28:0/3440828969 >> v2:172.21.15.138:3300/0 conn(0x7f6f200725f0 msgr2=0x7f6f200aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.638+0000 7f6f16000640 1 --2- 172.21.15.28:0/3440828969 >> v2:172.21.15.138:3300/0 conn(0x7f6f200725f0 0x7f6f200aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.638+0000 7f6f16000640 1 -- 172.21.15.28:0/3440828969 >> v1:172.21.15.138:6789/0 conn(0x7f6f20147230 legacy=0x7f6f2006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.638+0000 7f6f16000640 1 -- 172.21.15.28:0/3440828969 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2006e900 con 0x7f6f20147e90 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.639+0000 7f6f16000640 1 --2- 172.21.15.28:0/3440828969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 0x7f6f200720b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6f080097a0 tx=0x7f6f0802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556271408e6a1c53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.639+0000 7f6f1e000640 1 -- 172.21.15.28:0/3440828969 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08037070 con 0x7f6f20147e90 2024-09-10T10:55:59.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.639+0000 7f6f1e000640 1 -- 172.21.15.28:0/3440828969 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f08032440 con 0x7f6f20147e90 2024-09-10T10:55:59.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.639+0000 7f6f1e000640 1 -- 172.21.15.28:0/3440828969 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08032810 con 0x7f6f20147e90 2024-09-10T10:55:59.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 -- 172.21.15.28:0/3440828969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 msgr2=0x7f6f200720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 --2- 172.21.15.28:0/3440828969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 0x7f6f200720b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6f080097a0 tx=0x7f6f0802a180 comp rx=0 tx=0).stop 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 -- 172.21.15.28:0/3440828969 shutdown_connections 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 --2- 172.21.15.28:0/3440828969 >> v2:172.21.15.138:3300/0 conn(0x7f6f200725f0 0x7f6f200aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 --2- 172.21.15.28:0/3440828969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147e90 0x7f6f200720b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 -- 172.21.15.28:0/3440828969 >> 172.21.15.28:0/3440828969 conn(0x7f6f20067d70 msgr2=0x7f6f2006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 -- 172.21.15.28:0/3440828969 shutdown_connections 2024-09-10T10:55:59.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.640+0000 7f6f25a00640 1 -- 172.21.15.28:0/3440828969 wait complete. 2024-09-10T10:55:59.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.641+0000 7f6f25a00640 1 Processor -- start 2024-09-10T10:55:59.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.641+0000 7f6f25a00640 1 -- start start 2024-09-10T10:55:59.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 0x7f6f200cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147230 0x7f6f200c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2006eec0 con 0x7f6f20147e90 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20072a40 con 0x7f6f200725f0 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.642+0000 7f6f25a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2006f1a0 con 0x7f6f20147230 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 0x7f6f200cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 0x7f6f200cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:48998/0 (socket says 172.21.15.28:48998) 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1ea00640 1 -- 172.21.15.28:0/2540893592 learned_addr learned my addr 172.21.15.28:0/2540893592 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55166/0 (socket says 172.21.15.28:55166) 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f16000640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147230 0x7f6f200c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 -- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147230 msgr2=0x7f6f200c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147230 0x7f6f200c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 -- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 msgr2=0x7f6f200cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 0x7f6f200cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.643+0000 7f6f1f400640 1 -- 172.21.15.28:0/2540893592 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f20134b00 con 0x7f6f20147e90 2024-09-10T10:55:59.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.644+0000 7f6f1f400640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800efc0 tx=0x7f6f1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.645+0000 7f6f17400640 1 -- 172.21.15.28:0/2540893592 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18018020 con 0x7f6f20147e90 2024-09-10T10:55:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.645+0000 7f6f17400640 1 -- 172.21.15.28:0/2540893592 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18014480 con 0x7f6f20147e90 2024-09-10T10:55:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.645+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f20134de0 con 0x7f6f20147e90 2024-09-10T10:55:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.645+0000 7f6f17400640 1 -- 172.21.15.28:0/2540893592 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18014870 con 0x7f6f20147e90 2024-09-10T10:55:59.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.645+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201353a0 con 0x7f6f20147e90 2024-09-10T10:55:59.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.647+0000 7f6f17400640 1 -- 172.21.15.28:0/2540893592 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f18005d10 con 0x7f6f20147e90 2024-09-10T10:55:59.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.648+0000 7f6f17400640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.648+0000 7f6f17400640 1 -- 172.21.15.28:0/2540893592 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f6f1808bdb0 con 0x7f6f20147e90 2024-09-10T10:55:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.649+0000 7f6f28870740 1 Processor -- start 2024-09-10T10:55:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.649+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.649+0000 7f6f28870740 1 -- start start 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.649+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 0x7f6efc06e2f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f10007b10 tx=0x7f6f10007a60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c5cf0 0x55847840b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c7480 0x5584783c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5584783243b0 con 0x5584783c34b0 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558478329410 con 0x5584783c7480 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5584782d1620 con 0x5584783c5cf0 2024-09-10T10:55:59.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55172/0 (socket says 172.21.15.28:55172) 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 -- 172.21.15.28:0/2265877180 learned_addr learned my addr 172.21.15.28:0/2265877180 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c5cf0 0x55847840b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f1f400640 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c7480 0x5584783c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 -- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c5cf0 msgr2=0x55847840b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c5cf0 0x55847840b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 -- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c7480 msgr2=0x5584783c0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c7480 0x5584783c0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.650+0000 7f6f16000640 1 -- 172.21.15.28:0/2265877180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847803c2d0 con 0x5584783c34b0 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.651+0000 7f6f16000640 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098c0 tx=0x7f6f0802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaeb66685d938f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.651+0000 7f6f0f400640 1 -- 172.21.15.28:0/2265877180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08037070 con 0x5584783c34b0 2024-09-10T10:55:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.651+0000 7f6f0f400640 1 -- 172.21.15.28:0/2265877180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f08038040 con 0x5584783c34b0 2024-09-10T10:55:59.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.651+0000 7f6f0f400640 1 -- 172.21.15.28:0/2265877180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004430 con 0x5584783c34b0 2024-09-10T10:55:59.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- 172.21.15.28:0/2265877180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 msgr2=0x5584783c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f6f080098c0 tx=0x7f6f0802c000 comp rx=0 tx=0).stop 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- 172.21.15.28:0/2265877180 shutdown_connections 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c7480 0x5584783c0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c34b0 0x5584783c60d0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 --2- 172.21.15.28:0/2265877180 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c5cf0 0x55847840b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- 172.21.15.28:0/2265877180 >> 172.21.15.28:0/2265877180 conn(0x5584783c2740 msgr2=0x55847840a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- 172.21.15.28:0/2265877180 shutdown_connections 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- 172.21.15.28:0/2265877180 wait complete. 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.652+0000 7f6f28870740 1 -- start start 2024-09-10T10:55:59.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.653+0000 7f6f28870740 1 Processor -- start 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.653+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x5584783ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.653+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x558478407fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f28870740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f28870740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5584782e4780 con 0x5584783e6d80 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584783253d0 con 0x5584783c5cf0 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5584783c1be0 con 0x5584783c7480 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f28870740 10 client.0 _ll_register_callbacks cb 0x5584783225d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55178/0 (socket says 172.21.15.28:55178) 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 -- 172.21.15.28:0/1663238618 learned_addr learned my addr 172.21.15.28:0/1663238618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x5584783ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f16000640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x558478407fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 -- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 msgr2=0x558478407fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x558478407fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 -- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 msgr2=0x5584783ef7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x5584783ef7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 -- 172.21.15.28:0/1663238618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584781066b0 con 0x5584783e6d80 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x5584783ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f6f180624e0 tx=0x7f6f1805a020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b103f9c235d956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.654+0000 7f6f03e00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1805d040 con 0x5584783e6d80 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f08038040 con 0x5584783e6d80 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f6f180603e0 con 0x5584783e6d80 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=341) ==== 24+0+0 (secure 0 0 0) 0x7f6f180605e0 con 0x5584783e6d80 2024-09-10T10:55:59.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 10 client.10653 fetch_fsmap learned FSMap version 341 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=341,monmap=2+}) -- 0x5584783f0410 con 0x5584783e6d80 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f08038040 con 0x5584783e6d80 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 341) ==== 3502+0+0 (secure 0 0 0) 0x7f6f18055c30 con 0x5584783e6d80 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 10 client.10653 fetch_fsmap finished waiting for FSMap version 341 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 10 client.10653 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 20 client.10653 populate_metadata read hostname 'smithi028' 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x5584783e5600 0x5584783c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 4 client.10653 mds_command: new command op to 9505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.655+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 --> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558478336b40 con 0x5584783e5600 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.656+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x5584783e5600 0x5584783c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.656+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x5584783e5600 0x5584783c0400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae381ace91f22b16 server_cookie=7fd1736d209e319 in_seq=0 out_seq=0 2024-09-10T10:55:59.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.656+0000 7f6f03e00640 10 client.10653 ms_handle_connect on v2:172.21.15.28:6833/354907385 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.657+0000 7f6f03e00640 1 -- 172.21.15.28:0/1663238618 <== mds.0 v2:172.21.15.28:6833/354907385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558478336b40 con 0x5584783e5600 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.657+0000 7f6f03e00640 10 client.10653 handle_command_reply: tid=0 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.657+0000 7f6f28870740 1 client.10653 shutdown 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.657+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 msgr2=0x5584783c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.657+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f6f180624e0 tx=0x7f6f1805a020 comp rx=0 tx=0).stop 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 shutdown_connections 2024-09-10T10:55:59.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x5584783e5600 0x5584783c0400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x5584783c4ac0 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x558478407fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 --2- 172.21.15.28:0/1663238618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x5584783ef7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f03e00640 0 client.10653 ms_handle_reset on v2:172.21.15.28:6833/354907385 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 >> 172.21.15.28:0/1663238618 conn(0x5584783c34b0 msgr2=0x55847840de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 shutdown_connections 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 1 -- 172.21.15.28:0/1663238618 wait complete. 2024-09-10T10:55:59.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.658+0000 7f6f28870740 20 client.10653 trim_cache size 0 max 16384 2024-09-10T10:55:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 Processor -- start 2024-09-10T10:55:59.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 -- start start 2024-09-10T10:55:59.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x55847840e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x5584783c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201ffca0 con 0x5584783e6d80 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201370c0 con 0x5584783c5cf0 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.677+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f201ff7d0 con 0x5584783c7480 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55180/0 (socket says 172.21.15.28:55180) 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 -- 172.21.15.28:0/1088842848 learned_addr learned my addr 172.21.15.28:0/1088842848 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f16000640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x5584783c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x55847840e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 -- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 msgr2=0x5584783c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x5584783c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 -- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 msgr2=0x55847840e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x55847840e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 -- 172.21.15.28:0/1088842848 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584783e7180 con 0x5584783e6d80 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f16000640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x5584783c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x55847840e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f1f400640 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009d30 tx=0x7f6f18071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0405f6843718439 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.678+0000 7f6f0ea00640 1 -- 172.21.15.28:0/1088842848 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18075030 con 0x5584783e6d80 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f0ea00640 1 -- 172.21.15.28:0/1088842848 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18088070 con 0x5584783e6d80 2024-09-10T10:55:59.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f0ea00640 1 -- 172.21.15.28:0/1088842848 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18059a20 con 0x5584783e6d80 2024-09-10T10:55:59.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 -- 172.21.15.28:0/1088842848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 msgr2=0x55847842f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009d30 tx=0x7f6f18071000 comp rx=0 tx=0).stop 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 -- 172.21.15.28:0/1088842848 shutdown_connections 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783e6d80 0x55847842f0e0 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x5584783c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 --2- 172.21.15.28:0/1088842848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783c5cf0 0x55847840e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 -- 172.21.15.28:0/1088842848 >> 172.21.15.28:0/1088842848 conn(0x5584783c34b0 msgr2=0x5584783c1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 -- 172.21.15.28:0/1088842848 shutdown_connections 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.679+0000 7f6f28870740 1 -- 172.21.15.28:0/1088842848 wait complete. 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.680+0000 7f6f28870740 1 -- start start 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.680+0000 7f6f28870740 1 Processor -- start 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x55847841f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783e6d80 0x55847841fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5584783e5480 con 0x5584783c5cf0 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584783f06f0 con 0x5584783e6d80 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5584783e5ad0 con 0x5584783c7480 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f28870740 10 client.0 _ll_register_callbacks cb 0x7f6f1800cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55186/0 (socket says 172.21.15.28:55186) 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1ea00640 1 -- 172.21.15.28:0/3175369207 learned_addr learned my addr 172.21.15.28:0/3175369207 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1f400640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783e6d80 0x55847841fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f16000640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x55847841f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1ea00640 1 -- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 msgr2=0x55847841f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.681+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x55847841f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f1ea00640 1 -- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783e6d80 msgr2=0x55847841fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:55:59.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783e6d80 0x55847841fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f1ea00640 1 -- 172.21.15.28:0/3175369207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558478408e00 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f1ea00640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f6f10001a50 tx=0x7f6f10007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802df35690cfe987 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f03e00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f100133f0 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f6f1002da30 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f18088070 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=341) ==== 24+0+0 (secure 0 0 0) 0x7f6f10038050 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f28870740 10 client.10659 fetch_fsmap learned FSMap version 341 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.682+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=341,monmap=2+}) -- 0x5584783bbd50 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f18088070 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 341) ==== 3502+0+0 (secure 0 0 0) 0x7f6f1002dc30 con 0x5584783c5cf0 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 10 client.10659 fetch_fsmap finished waiting for FSMap version 341 2024-09-10T10:55:59.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 10 client.10659 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 20 client.10659 populate_metadata read hostname 'smithi028' 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x55847842f8b0 0x5584783ed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 4 client.10659 mds_command: new command op to 9505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 --> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] -- command(tid 0: {"prefix": "status"}) -- 0x558478330e90 con 0x55847842f8b0 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.683+0000 7f6f16000640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x55847842f8b0 0x5584783ed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f03e00640 10 client.10659 ms_handle_connect on v2:172.21.15.28:6833/354907385 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f16000640 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x55847842f8b0 0x5584783ed6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ccd9178a4de6e2e server_cookie=4be6a9977976e0bf in_seq=0 out_seq=0 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f03e00640 1 -- 172.21.15.28:0/3175369207 <== mds.0 v2:172.21.15.28:6833/354907385 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x558478330e90 con 0x55847842f8b0 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f03e00640 10 client.10659 handle_command_reply: tid=0 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 client.10659 shutdown 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 msgr2=0x558478409450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f6f10001a50 tx=0x7f6f10007da0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 shutdown_connections 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:6833/354907385,v1:172.21.15.28:6835/354907385] conn(0x55847842f8b0 0x5584783ed6d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584783e6d80 0x55847841fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5584783c7480 0x55847841f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 --2- 172.21.15.28:0/3175369207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5584783c5cf0 0x558478409450 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 >> 172.21.15.28:0/3175369207 conn(0x5584783c34b0 msgr2=0x5584783c46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f03e00640 0 client.10659 ms_handle_reset on v2:172.21.15.28:6833/354907385 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 shutdown_connections 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 1 -- 172.21.15.28:0/3175369207 wait complete. 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.684+0000 7f6f28870740 20 client.10659 trim_cache size 0 max 16384 2024-09-10T10:55:59.690 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 9505, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 10.825894864, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 341, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 150, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 150, 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 15.52648048 2024-09-10T10:55:59.691 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 msgr2=0x7f6efc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 0x7f6efc06e2f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f10007b10 tx=0x7f6f10007a60 comp rx=0 tx=0).stop 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 msgr2=0x7f6f200c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800efc0 tx=0x7f6f1800c490 comp rx=0 tx=0).stop 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 shutdown_connections 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6efc06be30 0x7f6efc06e2f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f20147e90 0x7f6f200c6610 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f20147230 0x7f6f200c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 --2- 172.21.15.28:0/2540893592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f200725f0 0x7f6f200cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 >> 172.21.15.28:0/2540893592 conn(0x7f6f20067d70 msgr2=0x7f6f2006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 shutdown_connections 2024-09-10T10:55:59.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.685+0000 7f6f25a00640 1 -- 172.21.15.28:0/2540893592 wait complete. 2024-09-10T10:55:59.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:55:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.856+0000 7f7103c00640 1 Processor -- start 2024-09-10T10:55:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.856+0000 7f7103c00640 1 -- start start 2024-09-10T10:55:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.857+0000 7f7103c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70fc072700 0x7f70fc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.857+0000 7f7103c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70fc146140 con 0x7f70fc072de0 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.857+0000 7f7103c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70fc0cd740 con 0x7f70fc072320 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.857+0000 7f7103c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f70fc0cda20 con 0x7f70fc072700 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f70fbe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f70fc072de0 0x7f70fc15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54310/0 (socket says 172.21.15.28:54310) 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f70fbe00640 1 -- 172.21.15.28:0/641711074 learned_addr learned my addr 172.21.15.28:0/641711074 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:55:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f7101c00640 1 --2- 172.21.15.28:0/641711074 >> v2:172.21.15.28:3300/0 conn(0x7f70fc072700 0x7f70fc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f7101c00640 1 -- 172.21.15.28:0/641711074 >> v1:172.21.15.138:6789/0 conn(0x7f70fc072320 legacy=0x7f70fc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f7101c00640 1 -- 172.21.15.28:0/641711074 >> v1:172.21.15.28:6789/0 conn(0x7f70fc072de0 legacy=0x7f70fc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f70fb400640 1 -- 172.21.15.28:0/641711074 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860319472 0 0) 0x7f70fc146140 con 0x7f70fc072de0 2024-09-10T10:55:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.858+0000 7f7101c00640 1 -- 172.21.15.28:0/641711074 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc0cdd00 con 0x7f70fc072700 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.859+0000 7f7101c00640 1 --2- 172.21.15.28:0/641711074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc15d8f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009d90 tx=0x7f70f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6898b70ee1d34adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.859+0000 7f70fb400640 1 -- 172.21.15.28:0/641711074 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4007740 con 0x7f70fc072700 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f70fb400640 1 -- 172.21.15.28:0/641711074 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f40078e0 con 0x7f70fc072700 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f70fb400640 1 -- 172.21.15.28:0/641711074 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f4007cb0 con 0x7f70fc072700 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 -- 172.21.15.28:0/641711074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 msgr2=0x7f70fc15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 --2- 172.21.15.28:0/641711074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc15d8f0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009d90 tx=0x7f70f402a660 comp rx=0 tx=0).stop 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 -- 172.21.15.28:0/641711074 shutdown_connections 2024-09-10T10:55:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 --2- 172.21.15.28:0/641711074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc15d8f0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 -- 172.21.15.28:0/641711074 >> 172.21.15.28:0/641711074 conn(0x7f70fc067d70 msgr2=0x7f70fc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:55:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 -- 172.21.15.28:0/641711074 shutdown_connections 2024-09-10T10:55:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.860+0000 7f7103c00640 1 -- 172.21.15.28:0/641711074 wait complete. 2024-09-10T10:55:59.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.862+0000 7f7103c00640 1 Processor -- start 2024-09-10T10:55:59.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.862+0000 7f7103c00640 1 -- start start 2024-09-10T10:55:59.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70fc072320 0x7f70fc0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70fc072de0 0x7f70fc1e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc15dfb0 con 0x7f70fc072700 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70fc15de30 con 0x7f70fc072320 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7103c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f70fc15e130 con 0x7f70fc072de0 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f70fbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55194/0 (socket says 172.21.15.28:55194) 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f70fbe00640 1 -- 172.21.15.28:0/1104305395 learned_addr learned my addr 172.21.15.28:0/1104305395 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7101200640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70fc072320 0x7f70fc0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.863+0000 7f7101c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70fc072de0 0x7f70fc1e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 -- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70fc072de0 msgr2=0x7f70fc1e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70fc072de0 0x7f70fc1e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 -- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70fc072320 msgr2=0x7f70fc0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:55:59.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70fc072320 0x7f70fc0b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 -- 172.21.15.28:0/1104305395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70fc1f7a60 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70fbe00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f70e800c3f0 tx=0x7f70e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e80074e0 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e80047e0 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8004bd0 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70fc1f7d40 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.864+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70fc1e7880 con 0x7f70fc072700 2024-09-10T10:55:59.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.865+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70fc15c800 con 0x7f70fc072700 2024-09-10T10:55:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.866+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f70e8011070 con 0x7f70fc072700 2024-09-10T10:55:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.867+0000 7f70f8c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 0x7f70dc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:55:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.867+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f70e808ab10 con 0x7f70fc072700 2024-09-10T10:55:59.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.868+0000 7f7101200640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 0x7f70dc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:55:59.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.868+0000 7f7101200640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 0x7f70dc06e2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000f60 tx=0x7f70f0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:55:59.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:55:59.871+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70fc15c800 con 0x7f70fc072700 2024-09-10T10:56:00.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.090+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70fc0cce60 con 0x7f70fc072700 2024-09-10T10:56:00.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.091+0000 7f70f8c00640 1 -- 172.21.15.28:0/1104305395 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v341) ==== 53+0+73 (secure 0 0 0) 0x7f70e8058cd0 con 0x7f70fc072700 2024-09-10T10:56:00.092 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.092+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 msgr2=0x7f70dc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.092+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 0x7f70dc06e2f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f70f0000f60 tx=0x7f70f0009040 comp rx=0 tx=0).stop 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 msgr2=0x7f70fc0a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f70e800c3f0 tx=0x7f70e800c8b0 comp rx=0 tx=0).stop 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 shutdown_connections 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f70dc06be30 0x7f70dc06e2f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:00.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f70fc072de0 0x7f70fc1e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70fc072700 0x7f70fc0a9490 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 --2- 172.21.15.28:0/1104305395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70fc072320 0x7f70fc0b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 >> 172.21.15.28:0/1104305395 conn(0x7f70fc067d70 msgr2=0x7f70fc1e7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 shutdown_connections 2024-09-10T10:56:00.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00.093+0000 7f7103c00640 1 -- 172.21.15.28:0/1104305395 wait complete. 2024-09-10T10:56:00.110 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:56:00.110 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:56:00.110 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:56:00.110 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:56:00.111 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:00.111 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:56:00.111 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:56:00.111 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:56:00.111 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:00.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.111 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.147 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 41945sec preferred_lft 41945sec 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.148 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:56:00.149 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.150 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:56:00.150 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:56:00.150 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:56:00.150 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:00.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.150 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:56:00.241 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:56:00.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.244 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:56:00.280 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:56:00.280 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:56:00.280 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:56:00.280 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:56:00.280 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:56:00.280 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:56:00.280 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:56:00.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:56:00.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:56:00.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:56:00.281 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:56:00.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:56:00.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:00.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.478 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:56:00.513 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:56:00.513 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:56:00.513 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:00.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.514 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:56:00.611 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:56:00.611 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:00.646 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:00.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:56:00.646 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:00.720 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:56:00.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.720 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:00.850 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:00.851 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:00.851 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:00.866 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:00.866 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:00.867 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:00.867 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:56:00.867 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:56:00.888 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:56:00.889 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:56:00.890 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:56:00.890 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:56:00.890 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:56:00.890 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:56:00.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:56:00.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:00.893 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:00.949 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:56:01.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:56:01.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:01.047 INFO:teuthology.orchestra.run.smithi028.stdout:client10671 2024-09-10T10:56:01.048 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:01.048 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:01.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.083 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir1) 2024-09-10T10:56:01.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.166 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir1/file1) 2024-09-10T10:56:01.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.244 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir2) 2024-09-10T10:56:01.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.322 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir2/file2) 2024-09-10T10:56:01.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.403 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-10T10:56:01.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.480 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir1/file1 testdir/hardlink1) 2024-09-10T10:56:01.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:01.563 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir2/file2 testdir/hardlink2) 2024-09-10T10:56:01.640 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:56:01.640 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:56:01.641 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:56:01.641 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:56:01.641 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir1/file1").st_ino) 2024-09-10T10:56:01.641 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:56:01.781 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2024-09-10T10:56:01.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:01.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:56:01.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:56:01.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:56:01.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:56:01.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2/file2").st_ino) 2024-09-10T10:56:01.794 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:56:01.933 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628279 2024-09-10T10:56:01.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir2").st_ino) 2024-09-10T10:56:01.945 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-09-10T10:56:02.085 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628278 2024-09-10T10:56:02.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:02.097 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:02.097 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:02.172 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:02.172 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:02.247 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:56:02.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.442+0000 7f6ae4600640 1 Processor -- start 2024-09-10T10:56:02.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.442+0000 7f6ae4600640 1 -- start start 2024-09-10T10:56:02.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.443+0000 7f6ae4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc147230 0x7f6adc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.443+0000 7f6ae4600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6adc0727c0 0x7f6adc0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.443+0000 7f6ae4600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6adc06f480 con 0x7f6adc147e90 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.443+0000 7f6ae4600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6adc0b34d0 con 0x7f6adc0727c0 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.443+0000 7f6ae4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc0ae9e0 con 0x7f6adc147230 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.444+0000 7f6ae1c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc147230 0x7f6adc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.444+0000 7f6ae2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6adc0727c0 0x7f6adc0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.444+0000 7f6ae1200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6adc147e90 0x7f6adc0720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:34366/0 (socket says 172.21.15.28:34366) 2024-09-10T10:56:02.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.444+0000 7f6ae1200640 1 -- 172.21.15.28:0/351159805 learned_addr learned my addr 172.21.15.28:0/351159805 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:02.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.445+0000 7f6ae2600640 1 -- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc147230 msgr2=0x7f6adc06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:02.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.445+0000 7f6ae2600640 1 --2- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc147230 0x7f6adc06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.445+0000 7f6ae2600640 1 -- 172.21.15.28:0/351159805 >> v1:172.21.15.138:6789/0 conn(0x7f6adc147e90 legacy=0x7f6adc0720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.445+0000 7f6ae2600640 1 -- 172.21.15.28:0/351159805 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc06e900 con 0x7f6adc0727c0 2024-09-10T10:56:02.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.446+0000 7f6ae2600640 1 --2- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc0727c0 0x7f6adc0aee10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d90 tx=0x7f6ad402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef6d627e2f08ba26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.446+0000 7f6adbe00640 1 -- 172.21.15.28:0/351159805 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4007740 con 0x7f6adc0727c0 2024-09-10T10:56:02.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.446+0000 7f6adbe00640 1 -- 172.21.15.28:0/351159805 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad40078e0 con 0x7f6adc0727c0 2024-09-10T10:56:02.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.446+0000 7f6adbe00640 1 -- 172.21.15.28:0/351159805 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4007cb0 con 0x7f6adc0727c0 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 -- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc0727c0 msgr2=0x7f6adc0aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 --2- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc0727c0 0x7f6adc0aee10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009d90 tx=0x7f6ad402a660 comp rx=0 tx=0).stop 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 -- 172.21.15.28:0/351159805 shutdown_connections 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 --2- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc0727c0 0x7f6adc0aee10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 --2- 172.21.15.28:0/351159805 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc147230 0x7f6adc06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 -- 172.21.15.28:0/351159805 >> 172.21.15.28:0/351159805 conn(0x7f6adc067d70 msgr2=0x7f6adc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:02.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 -- 172.21.15.28:0/351159805 shutdown_connections 2024-09-10T10:56:02.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.447+0000 7f6ae4600640 1 -- 172.21.15.28:0/351159805 wait complete. 2024-09-10T10:56:02.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.448+0000 7f6ae4600640 1 Processor -- start 2024-09-10T10:56:02.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.448+0000 7f6ae4600640 1 -- start start 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc0727c0 0x7f6adc0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc147230 0x7f6adc0aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc06f1a0 con 0x7f6adc147e90 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc06eec0 con 0x7f6adc147230 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.449+0000 7f6ae4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc06e5f0 con 0x7f6adc0727c0 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55196/0 (socket says 172.21.15.28:55196) 2024-09-10T10:56:02.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 -- 172.21.15.28:0/4020891854 learned_addr learned my addr 172.21.15.28:0/4020891854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae1200640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc147230 0x7f6adc0aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae1c00640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc0727c0 0x7f6adc0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 -- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc0727c0 msgr2=0x7f6adc0ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc0727c0 0x7f6adc0ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 -- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc147230 msgr2=0x7f6adc0aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc147230 0x7f6adc0aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 -- 172.21.15.28:0/4020891854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc0cf2f0 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.450+0000 7f6ae2600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009eb0 tx=0x7f6ad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.451+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4040070 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.451+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4031bc0 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.451+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4009450 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.451+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc0cf5d0 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.451+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc0cec70 con 0x7f6adc147e90 2024-09-10T10:56:02.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.452+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc147610 con 0x7f6adc147e90 2024-09-10T10:56:02.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.453+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6ad4039040 con 0x7f6adc147e90 2024-09-10T10:56:02.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.454+0000 7f6ad9600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 0x7f6abc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.454+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f6ad40ac840 con 0x7f6adc147e90 2024-09-10T10:56:02.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.454+0000 7f6ae1c00640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 0x7f6abc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.455+0000 7f6ae1c00640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 0x7f6abc06e2f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ad0 tx=0x7f6ad0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.458+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6ad407b050 con 0x7f6adc147e90 2024-09-10T10:56:02.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.654+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6adc0b4db0 con 0x7f6adc147e90 2024-09-10T10:56:02.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.654+0000 7f6ad9600640 1 -- 172.21.15.28:0/4020891854 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v150) ==== 80+0+0 (secure 0 0 0) 0x7f6ad407a950 con 0x7f6adc147e90 2024-09-10T10:56:02.656 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-09-10T10:56:02.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 msgr2=0x7f6abc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 0x7f6abc06e2f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009ad0 tx=0x7f6ad0008040 comp rx=0 tx=0).stop 2024-09-10T10:56:02.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 msgr2=0x7f6adc0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009eb0 tx=0x7f6ad4004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 shutdown_connections 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6abc06be30 0x7f6abc06e2f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6adc147e90 0x7f6adc0c84a0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6adc147230 0x7f6adc0aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 --2- 172.21.15.28:0/4020891854 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6adc0727c0 0x7f6adc0ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 >> 172.21.15.28:0/4020891854 conn(0x7f6adc067d70 msgr2=0x7f6adc145bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 shutdown_connections 2024-09-10T10:56:02.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.657+0000 7f6ae4600640 1 -- 172.21.15.28:0/4020891854 wait complete. 2024-09-10T10:56:02.673 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:56:02.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:02.674 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:02.759 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:02.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:02.760 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:56:02.812 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:02.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.967+0000 7f0592e00640 1 Processor -- start 2024-09-10T10:56:02.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.968+0000 7f0592e00640 1 -- start start 2024-09-10T10:56:02.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0592e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f058c072d30 0x7f058c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0592e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 0x7f058c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0592e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058c0cdbe0 con 0x7f058c15b300 2024-09-10T10:56:02.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0592e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f058c0728a0 con 0x7f058c072d30 2024-09-10T10:56:02.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0592e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f058c06fb30 con 0x7f058c06f700 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.969+0000 7f0590e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 0x7f058c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f058be00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f058c15b300 0x7f058c0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:34376/0 (socket says 172.21.15.28:34376) 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f058be00640 1 -- 172.21.15.28:0/2359701243 learned_addr learned my addr 172.21.15.28:0/2359701243 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f0583400640 1 --2- 172.21.15.28:0/2359701243 >> v2:172.21.15.138:3300/0 conn(0x7f058c072d30 0x7f058c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f0590e00640 1 -- 172.21.15.28:0/2359701243 >> v2:172.21.15.138:3300/0 conn(0x7f058c072d30 msgr2=0x7f058c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f0590e00640 1 --2- 172.21.15.28:0/2359701243 >> v2:172.21.15.138:3300/0 conn(0x7f058c072d30 0x7f058c06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f0590e00640 1 -- 172.21.15.28:0/2359701243 >> v1:172.21.15.138:6789/0 conn(0x7f058c15b300 legacy=0x7f058c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.970+0000 7f0590e00640 1 -- 172.21.15.28:0/2359701243 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c0cd900 con 0x7f058c06f700 2024-09-10T10:56:02.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.971+0000 7f0590e00640 1 --2- 172.21.15.28:0/2359701243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 0x7f058c15b6e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0584009d90 tx=0x7f058402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b6aa1093d456b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.971+0000 7f058b400640 1 -- 172.21.15.28:0/2359701243 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584031600 con 0x7f058c06f700 2024-09-10T10:56:02.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.971+0000 7f058b400640 1 -- 172.21.15.28:0/2359701243 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05840317a0 con 0x7f058c06f700 2024-09-10T10:56:02.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.971+0000 7f058b400640 1 -- 172.21.15.28:0/2359701243 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584031b70 con 0x7f058c06f700 2024-09-10T10:56:02.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 -- 172.21.15.28:0/2359701243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 msgr2=0x7f058c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 --2- 172.21.15.28:0/2359701243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 0x7f058c15b6e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0584009d90 tx=0x7f058402a5d0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 -- 172.21.15.28:0/2359701243 shutdown_connections 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 --2- 172.21.15.28:0/2359701243 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c06f700 0x7f058c15b6e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 --2- 172.21.15.28:0/2359701243 >> v2:172.21.15.138:3300/0 conn(0x7f058c072d30 0x7f058c06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 -- 172.21.15.28:0/2359701243 >> 172.21.15.28:0/2359701243 conn(0x7f058c067d70 msgr2=0x7f058c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 -- 172.21.15.28:0/2359701243 shutdown_connections 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.972+0000 7f0592e00640 1 -- 172.21.15.28:0/2359701243 wait complete. 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.974+0000 7f0592e00640 1 Processor -- start 2024-09-10T10:56:02.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.974+0000 7f0592e00640 1 -- start start 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.974+0000 7f0592e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 0x7f058c0a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0592e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 0x7f058c15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0592e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 0x7f058c15d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0592e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f058c0cd5c0 con 0x7f058c15b300 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0592e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f058c072530 con 0x7f058c06f700 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0592e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f058c0cffe0 con 0x7f058c072d30 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0590e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 0x7f058c15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 0x7f058c0a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0590e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 0x7f058c15d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55204/0 (socket says 172.21.15.28:55204) 2024-09-10T10:56:02.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0590e00640 1 -- 172.21.15.28:0/1051890655 learned_addr learned my addr 172.21.15.28:0/1051890655 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:02.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0583400640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 0x7f058c15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 -- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 msgr2=0x7f058c15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 0x7f058c15b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 -- 172.21.15.28:0/1051890655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 msgr2=0x7f058c15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 0x7f058c15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f058be00640 1 -- 172.21.15.28:0/1051890655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058c15c0b0 con 0x7f058c06f700 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.975+0000 7f0583400640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 0x7f058c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f058be00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 0x7f058c0a9490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c370 tx=0x7f057c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c017070 con 0x7f058c06f700 2024-09-10T10:56:02.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c005d00 con 0x7f058c06f700 2024-09-10T10:56:02.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c007310 con 0x7f058c06f700 2024-09-10T10:56:02.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058c15c390 con 0x7f058c06f700 2024-09-10T10:56:02.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.976+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058c15c950 con 0x7f058c06f700 2024-09-10T10:56:02.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.977+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058c06f1c0 con 0x7f058c06f700 2024-09-10T10:56:02.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.978+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f057c01a070 con 0x7f058c06f700 2024-09-10T10:56:02.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.979+0000 7f0588c00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 0x7f056806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:02.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.979+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7f057c08a380 con 0x7f058c06f700 2024-09-10T10:56:02.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.979+0000 7f0583400640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 0x7f056806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:02.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.980+0000 7f0583400640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 0x7f056806e2f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f058c0b3e90 tx=0x7f0574009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:02.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:02.984+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f057c058420 con 0x7f058c06f700 2024-09-10T10:56:03.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.206+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f058c06bcb0 con 0x7f058c06f700 2024-09-10T10:56:03.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.207+0000 7f0588c00640 1 -- 172.21.15.28:0/1051890655 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 341 v341) ==== 95+0+5234 (secure 0 0 0) 0x7f057c05c720 con 0x7f058c06f700 2024-09-10T10:56:03.209 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:03.209 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":341,"btime":"2024-09-10T10:55:51:866579+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":341,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:55:51.288071+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9505":{"gid":9505,"name":"a","rank":0,"incarnation":337,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6835/354907385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":354907385},{"type":"v1","addr":"172.21.15.28:6835","nonce":354907385}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9505,"qdb_cluster":[9505]},"id":7}]} 2024-09-10T10:56:03.209 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 341 2024-09-10T10:56:03.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 msgr2=0x7f056806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 0x7f056806e2f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f058c0b3e90 tx=0x7f0574009190 comp rx=0 tx=0).stop 2024-09-10T10:56:03.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 msgr2=0x7f058c0a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 0x7f058c0a9490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f057c00c370 tx=0x7f057c00c830 comp rx=0 tx=0).stop 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 shutdown_connections 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f056806be30 0x7f056806e2f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058c15b300 0x7f058c15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f058c072d30 0x7f058c15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 --2- 172.21.15.28:0/1051890655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f058c06f700 0x7f058c0a9490 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 >> 172.21.15.28:0/1051890655 conn(0x7f058c067d70 msgr2=0x7f058c159260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 shutdown_connections 2024-09-10T10:56:03.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.210+0000 7f0592e00640 1 -- 172.21.15.28:0/1051890655 wait complete. 2024-09-10T10:56:03.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T10:56:03.411 INFO:teuthology.orchestra.run.smithi028.stdout:{"message":"","return_code":0} 2024-09-10T10:56:03.412 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T10:56:03.412 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:56:03.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.567+0000 7ff62b200640 1 Processor -- start 2024-09-10T10:56:03.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.567+0000 7ff62b200640 1 -- start start 2024-09-10T10:56:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.568+0000 7ff62b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 0x7ff62c1629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.568+0000 7ff62b200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff62c15e990 0x7ff62c15edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.568+0000 7ff62b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff62c15f410 con 0x7ff62c06f520 2024-09-10T10:56:03.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.568+0000 7ff62b200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff62c15f6f0 con 0x7ff62c15e990 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.568+0000 7ff62b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff62c161f10 con 0x7ff62c073350 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff62a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff62c06f520 0x7ff62c071cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:54322/0 (socket says 172.21.15.28:54322) 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff62a800640 1 -- 172.21.15.28:0/3528886428 learned_addr learned my addr 172.21.15.28:0/3528886428 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff621400640 1 --2- 172.21.15.28:0/3528886428 >> v2:172.21.15.138:3300/0 conn(0x7ff62c15e990 0x7ff62c15edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff629e00640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 0x7ff62c1629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff629400640 1 -- 172.21.15.28:0/3528886428 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010789339 0 0) 0x7ff62c15f410 con 0x7ff62c06f520 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff629400640 1 -- 172.21.15.28:0/3528886428 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff610003520 con 0x7ff62c06f520 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff621400640 1 -- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 msgr2=0x7ff62c1629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff621400640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 0x7ff62c1629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff621400640 1 -- 172.21.15.28:0/3528886428 >> v1:172.21.15.28:6789/0 conn(0x7ff62c06f520 legacy=0x7ff62c071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff621400640 1 -- 172.21.15.28:0/3528886428 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1621f0 con 0x7ff62c15e990 2024-09-10T10:56:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.569+0000 7ff629e00640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 0x7ff62c1629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff621400640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c15e990 0x7ff62c15edd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff6140097a0 tx=0x7ff61402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca0e5d6dc1f969e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff629400640 1 -- 172.21.15.28:0/3528886428 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614037070 con 0x7ff62c15e990 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff629400640 1 -- 172.21.15.28:0/3528886428 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff614032440 con 0x7ff62c15e990 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff629400640 1 -- 172.21.15.28:0/3528886428 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6140327f0 con 0x7ff62c15e990 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff62b200640 1 -- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c15e990 msgr2=0x7ff62c15edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff62b200640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c15e990 0x7ff62c15edd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff6140097a0 tx=0x7ff61402a180 comp rx=0 tx=0).stop 2024-09-10T10:56:03.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.571+0000 7ff62b200640 1 -- 172.21.15.28:0/3528886428 shutdown_connections 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.572+0000 7ff62b200640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c15e990 0x7ff62c15edd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.572+0000 7ff62b200640 1 --2- 172.21.15.28:0/3528886428 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c073350 0x7ff62c1629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.572+0000 7ff62b200640 1 -- 172.21.15.28:0/3528886428 >> 172.21.15.28:0/3528886428 conn(0x7ff62c06c240 msgr2=0x7ff62c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.572+0000 7ff62b200640 1 -- 172.21.15.28:0/3528886428 shutdown_connections 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.572+0000 7ff62b200640 1 -- 172.21.15.28:0/3528886428 wait complete. 2024-09-10T10:56:03.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.573+0000 7ff62b200640 1 Processor -- start 2024-09-10T10:56:03.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.573+0000 7ff62b200640 1 -- start start 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f520 0x7ff62c0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c073350 0x7ff62c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c0b3290 con 0x7ff62c15e990 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c0b3110 con 0x7ff62c073350 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.574+0000 7ff62b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff62c06f9d0 con 0x7ff62c06f520 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55216/0 (socket says 172.21.15.28:55216) 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 -- 172.21.15.28:0/849006049 learned_addr learned my addr 172.21.15.28:0/849006049 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:03.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff629e00640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f520 0x7ff62c0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff621400640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c073350 0x7ff62c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 -- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f520 msgr2=0x7ff62c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f520 0x7ff62c0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 -- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c073350 msgr2=0x7ff62c159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c073350 0x7ff62c159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 -- 172.21.15.28:0/849006049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c0cb9b0 con 0x7ff62c15e990 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.575+0000 7ff62a800640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7ff62400cb40 tx=0x7ff62400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.576+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624017070 con 0x7ff62c15e990 2024-09-10T10:56:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.576+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62400aca0 con 0x7ff62c15e990 2024-09-10T10:56:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.576+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624012620 con 0x7ff62c15e990 2024-09-10T10:56:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.576+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c15a480 con 0x7ff62c15e990 2024-09-10T10:56:03.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.576+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c0cbf30 con 0x7ff62c15e990 2024-09-10T10:56:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.577+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c06fd90 con 0x7ff62c15e990 2024-09-10T10:56:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.578+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff62401a070 con 0x7ff62c15e990 2024-09-10T10:56:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.579+0000 7ff622a00640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 0x7ff60806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.579+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 6101+0+0 (secure 0 0 0) 0x7ff62408aed0 con 0x7ff62c15e990 2024-09-10T10:56:03.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.579+0000 7ff629e00640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 0x7ff60806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.580+0000 7ff629e00640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 0x7ff60806e320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009560 tx=0x7ff61c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.583+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff624059650 con 0x7ff62c15e990 2024-09-10T10:56:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.805+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff62c06ccd0 con 0x7ff62c15e990 2024-09-10T10:56:03.806 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:56:03.806+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:56:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.808+0000 7ff622a00640 1 -- 172.21.15.28:0/849006049 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v342) ==== 161+0+0 (secure 0 0 0) 0x7ff624058f50 con 0x7ff62c15e990 2024-09-10T10:56:03.809 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:56:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 msgr2=0x7ff60806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 0x7ff60806e320 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009560 tx=0x7ff61c008040 comp rx=0 tx=0).stop 2024-09-10T10:56:03.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 msgr2=0x7ff62c09c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7ff62400cb40 tx=0x7ff62400a060 comp rx=0 tx=0).stop 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 shutdown_connections 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff60806be60 0x7ff60806e320 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff62c15e990 0x7ff62c09c6d0 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff62c073350 0x7ff62c159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 --2- 172.21.15.28:0/849006049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff62c06f520 0x7ff62c0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 >> 172.21.15.28:0/849006049 conn(0x7ff62c06c240 msgr2=0x7ff62c0acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 shutdown_connections 2024-09-10T10:56:03.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.811+0000 7ff62b200640 1 -- 172.21.15.28:0/849006049 wait complete. 2024-09-10T10:56:03.822 INFO:tasks.ceph.mds.a.smithi028.stderr: -6560> 2024-09-10T10:55:50.822+0000 7fe8aba00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:56:03.828 DEBUG:teuthology.orchestra.run.smithi028:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-10T10:56:03.876 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:56:03.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 -- start start 2024-09-10T10:56:03.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 Processor -- start 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618bf324c00 0x5618bf3a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618bf325c00 0x5618bf3a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c0106600 con 0x5618bf325800 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618c0106480 con 0x5618bf324c00 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.909+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5618c0106300 con 0x5618bf325c00 2024-09-10T10:56:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42798/0 (socket says 172.21.15.28:42798) 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 -- 172.21.15.28:0/1546880530 learned_addr learned my addr 172.21.15.28:0/1546880530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4da0000640 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618bf324c00 0x5618bf3a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4da0a00640 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618bf325c00 0x5618bf3a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 -- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618bf325c00 msgr2=0x5618bf3a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618bf325c00 0x5618bf3a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 -- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618bf324c00 msgr2=0x5618bf3a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618bf324c00 0x5618bf3a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.910+0000 7f4d9f600640 1 -- 172.21.15.28:0/1546880530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bf35a780 con 0x5618bf325800 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9f600640 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x5618c013d830 tx=0x5618c00c82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bff43f9f7945bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9ce00640 1 -- 172.21.15.28:0/1546880530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34a680 con 0x5618bf325800 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9ce00640 1 -- 172.21.15.28:0/1546880530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5618bf350700 con 0x5618bf325800 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9ce00640 1 -- 172.21.15.28:0/1546880530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34ab60 con 0x5618bf325800 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618bf35a3c0 con 0x5618bf325800 2024-09-10T10:56:03.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5618bf35b680 con 0x5618bf325800 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4da1f88280 4 waiting for MDS map... 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9ce00640 1 -- 172.21.15.28:0/1546880530 <== mon.0 v2:172.21.15.28:3300/0 4 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x5618c0206600 con 0x5618bf325800 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4d9ce00640 1 -- 172.21.15.28:0/1546880530 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 342) ==== 2893+0+0 (secure 0 0 0) 0x5618bf37f180 con 0x5618bf325800 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4da1f88280 4 Got MDS map 342 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.911+0000 7f4da1f88280 10 main: JournalTool::main 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.912+0000 7f4da1f88280 4 main: JournalTool: connecting to RADOS... 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.912+0000 7f4da1f88280 1 Processor -- start 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.912+0000 7f4da1f88280 1 -- start start 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c0106180 con 0x5618c01e9400 2024-09-10T10:56:03.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618bf386480 con 0x5618c01e9c00 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5618bf386900 con 0x5618c01e9800 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42812/0 (socket says 172.21.15.28:42812) 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da0a00640 1 -- 172.21.15.28:0/2101605900 learned_addr learned my addr 172.21.15.28:0/2101605900 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4da0000640 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.913+0000 7f4d9f600640 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 -- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 msgr2=0x5618bf3a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 -- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 msgr2=0x5618bf3a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 -- 172.21.15.28:0/2101605900 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bf35b4a0 con 0x5618c01e9400 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4da0a00640 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x5618c020a570 tx=0x5618c0215800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbcfdee1a959245 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4d9a600640 1 -- 172.21.15.28:0/2101605900 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34a680 con 0x5618c01e9400 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4d9a600640 1 -- 172.21.15.28:0/2101605900 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5618bf350a80 con 0x5618c01e9400 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.914+0000 7f4d9a600640 1 -- 172.21.15.28:0/2101605900 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34ab60 con 0x5618c01e9400 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 -- 172.21.15.28:0/2101605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 msgr2=0x5618bf3a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x5618c020a570 tx=0x5618c0215800 comp rx=0 tx=0).stop 2024-09-10T10:56:03.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 -- 172.21.15.28:0/2101605900 shutdown_connections 2024-09-10T10:56:03.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 --2- 172.21.15.28:0/2101605900 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a1600 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 -- 172.21.15.28:0/2101605900 >> 172.21.15.28:0/2101605900 conn(0x5618c01e9000 msgr2=0x5618bf40a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 -- 172.21.15.28:0/2101605900 shutdown_connections 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 -- 172.21.15.28:0/2101605900 wait complete. 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.915+0000 7f4da1f88280 1 Processor -- start 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 -- start start 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c0107380 con 0x5618c01e9400 2024-09-10T10:56:03.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618c0106180 con 0x5618c01e9c00 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5618c0107500 con 0x5618c01e9800 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.916+0000 7f4da0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42826/0 (socket says 172.21.15.28:42826) 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 -- 172.21.15.28:0/1423461370 learned_addr learned my addr 172.21.15.28:0/1423461370 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4d9f600640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0000640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 -- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 msgr2=0x5618bf3a2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 -- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 msgr2=0x5618bf3a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 -- 172.21.15.28:0/1423461370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bf35a780 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da0a00640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x5618c020b6e0 tx=0x5618c00c8180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4d98800640 1 -- 172.21.15.28:0/1423461370 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34a4e0 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4d98800640 1 -- 172.21.15.28:0/1423461370 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5618bf350c40 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5618bf35a3c0 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.917+0000 7f4d98800640 1 -- 172.21.15.28:0/1423461370 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34a680 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.918+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618bf35a000 con 0x5618c01e9400 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.918+0000 7f4da1f88280 4 main: JournalTool: resolving pool 15 2024-09-10T10:56:03.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.919+0000 7f4d98800640 1 -- 172.21.15.28:0/1423461370 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x5618c026ec00 con 0x5618c01e9400 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.920+0000 7f4d98800640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 0x5618bf3a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.920+0000 7f4d98800640 1 -- 172.21.15.28:0/1423461370 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x5618c026ca00 con 0x5618c01e9400 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.920+0000 7f4da1f88280 4 main: JournalTool: creating IoCtx.. 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.920+0000 7f4da1f88280 4 main: Executing for rank 0 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.920+0000 7f4da0000640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 0x5618bf3a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.921+0000 7f4da1f88280 1 -- start start 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.921+0000 7f4da0000640 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 0x5618bf3a3700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x5618c00b6150 tx=0x5618c00c6960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.921+0000 7f4da1f88280 1 Processor -- start 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c026b400 0x5618bf3a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c026b800 0x5618bf3a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 0x5618bf3a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5618c0107800 con 0x5618c026bc00 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618c0107980 con 0x5618c026b800 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da1f88280 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5618bf386900 con 0x5618c026b400 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4d9f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 0x5618bf3a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4d9f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 0x5618bf3a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42830/0 (socket says 172.21.15.28:42830) 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 learned_addr learned my addr 172.21.15.28:0/3291145174 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.922+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c026b400 0x5618bf3a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c026b800 0x5618bf3a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c026b400 msgr2=0x5618bf3a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c026b400 0x5618bf3a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c026b800 msgr2=0x5618bf3a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c026b800 0x5618bf3a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618bf35a960 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c026b400 0x5618bf3a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c026b800 0x5618bf3a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d9f600640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 0x5618bf3a1600 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x5618c02a6cc0 tx=0x5618c00c6a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2924fcd1d210f68f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d94c00640 1 -- 172.21.15.28:0/3291145174 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618bf34a820 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d94c00640 1 -- 172.21.15.28:0/3291145174 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5618bf3501c0 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618bf35b4a0 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5618bf35a780 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da1f88280 4 waiting for MDS map... 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d94c00640 1 -- 172.21.15.28:0/3291145174 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5618c02afba0 con 0x5618c026bc00 2024-09-10T10:56:03.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d94c00640 1 -- 172.21.15.28:0/3291145174 <== mon.0 v2:172.21.15.28:3300/0 4 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x5618bf708200 con 0x5618c026bc00 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4d94c00640 1 -- 172.21.15.28:0/3291145174 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 342) ==== 2893+0+0 (secure 0 0 0) 0x5618bf37f500 con 0x5618c026bc00 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.923+0000 7f4da1f88280 4 Got MDS map 342 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.924+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5618bf70b800 0x5618bf722000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.924+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151) -- 0x5618bf70bc00 con 0x5618bf70b800 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.924+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5618bf70b800 0x5618bf722000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.924+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5618bf70b800 0x5618bf722000 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=903594791a793571 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.925+0000 7f4da0a00640 1 -- 172.21.15.28:0/3291145174 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5618c00be000 con 0x5618bf70b800 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.925+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5618bf72a000 0x5618bf722580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.925+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72a400 con 0x5618bf72a000 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.925+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5618bf72a000 0x5618bf722580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5618bf72a000 0x5618bf722580 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e5599d21a7534d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv22 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4d92e00640 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72bc00 con 0x5618bf72a000 2024-09-10T10:56:03.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4d92e00640 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e151) -- 0x5618bf734000 con 0x5618bf72a000 2024-09-10T10:56:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv19 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.931 INFO:teuthology.orchestra.run.smithi028.stdout:old journal was 4226680~2030 2024-09-10T10:56:03.931 INFO:teuthology.orchestra.run.smithi028.stdout:new journal start will be 8388608 (4159898 bytes past old end) 2024-09-10T10:56:03.931 INFO:teuthology.orchestra.run.smithi028.stdout:writing journal head 2024-09-10T10:56:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.926+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:5 15.14 15:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72a800 con 0x5618bf72a000 2024-09-10T10:56:03.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v151'23 uv23 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.932 INFO:teuthology.orchestra.run.smithi028.stdout:writing EResetJournal entry 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:6 15.1b 15:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72ac00 con 0x5618bf72a000 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x5618bf72b000 0x5618bf722b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:7 15.b 15:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72b400 con 0x5618bf72b000 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5618bf72b800 0x5618bf723080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- osd_op(unknown.0.0:8 15.1f 15:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf734000 con 0x5618bf72b800 2024-09-10T10:56:03.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5618bf734800 0x5618bf723600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- osd_op(unknown.0.0:9 15.1a 15:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf734c00 con 0x5618bf734800 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.928+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- osd_op(unknown.0.0:10 15.1e 15:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf735000 con 0x5618bf72b000 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5618bf72a400 0x5618bf723b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- osd_op(unknown.0.0:11 15.1d 15:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf72bc00 con 0x5618bf72a400 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4d9f600640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5618bf734800 0x5618bf723600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x5618bf72b000 0x5618bf722b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5618bf72b800 0x5618bf723080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.929+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5618bf72a400 0x5618bf723b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.930+0000 7f4d9f600640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5618bf734800 0x5618bf723600 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=83c80478f21d88b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.930+0000 7f4da0000640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5618bf72b800 0x5618bf723080 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=82b9bbbd54c44d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.930+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 5 ==== osd_op_reply(6 200.00000002 [delete] v151'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.930+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5618bf72a400 0x5618bf723b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=bf252237c1415d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.930+0000 7f4da0a00640 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x5618bf72b000 0x5618bf722b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=42cc77bcf0615d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.931+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== osd_op_reply(9 200.00000005 [delete] v151'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be780 con 0x5618bf734800 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.931+0000 7f4da0a00640 1 -- 172.21.15.28:0/3291145174 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== osd_op_reply(11 200.00000007 [delete] v151'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be000 con 0x5618bf72a400 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.932+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== osd_op_reply(8 200.00000004 [delete] v151'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72b800 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.932+0000 7f4da0a00640 1 -- 172.21.15.28:0/3291145174 <== osd.5 v2:172.21.15.138:6802/2049132176 1 ==== osd_op_reply(10 200.00000006 [delete] v151'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be000 con 0x5618bf72b000 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.932+0000 7f4da0a00640 1 -- 172.21.15.28:0/3291145174 <== osd.5 v2:172.21.15.138:6802/2049132176 2 ==== osd_op_reply(7 200.00000003 [delete] v151'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5618c00be000 con 0x5618bf72b000 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.932+0000 7f4d92e00640 1 -- 172.21.15.28:0/3291145174 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- osd_op(unknown.0.0:12 15.1b 15:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151) -- 0x5618bf734400 con 0x5618bf72a000 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da0000640 1 -- 172.21.15.28:0/3291145174 <== osd.3 v2:172.21.15.28:6816/2614769734 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v151'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5618c00be500 con 0x5618bf72a000 2024-09-10T10:56:03.936 INFO:teuthology.orchestra.run.smithi028.stdout:done 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5618bf72a400 msgr2=0x5618bf723b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x5618bf72a400 0x5618bf723b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5618bf734800 msgr2=0x5618bf723600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x5618bf734800 0x5618bf723600 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5618bf70b800 msgr2=0x5618bf722000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x5618bf70b800 0x5618bf722000 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5618bf72a000 msgr2=0x5618bf722580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x5618bf72a000 0x5618bf722580 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x5618bf72b000 msgr2=0x5618bf722b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x5618bf72b000 0x5618bf722b00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5618bf72b800 msgr2=0x5618bf723080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x5618bf72b800 0x5618bf723080 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 msgr2=0x5618bf3a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4da1f88280 1 --2- 172.21.15.28:0/3291145174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c026bc00 0x5618bf3a1600 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x5618c02a6cc0 tx=0x5618c00c6a20 comp rx=0 tx=0).stop 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 reap_dead start 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 reap_dead start 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.933+0000 7f4d9f600640 1 -- 172.21.15.28:0/3291145174 reap_dead start 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 shutdown_connections 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 >> 172.21.15.28:0/3291145174 conn(0x5618c026b000 msgr2=0x5618bf40a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 shutdown_connections 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/3291145174 wait complete. 2024-09-10T10:56:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 msgr2=0x5618bf3a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 0x5618bf3a3700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x5618c00b6150 tx=0x5618c00c6960 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 msgr2=0x5618bf3a3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x5618c020b6e0 tx=0x5618c00c8180 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 shutdown_connections 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x5618c026ac00 0x5618bf3a3700 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618c01e9c00 0x5618bf3a2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618c01e9800 0x5618bf3a2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1423461370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618c01e9400 0x5618bf3a3180 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 >> 172.21.15.28:0/1423461370 conn(0x5618c01e9000 msgr2=0x5618bf40a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 shutdown_connections 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.934+0000 7f4da1f88280 1 -- 172.21.15.28:0/1423461370 wait complete. 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 msgr2=0x5618bf3a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x5618c013d830 tx=0x5618c00c82a0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 shutdown_connections 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5618bf325c00 0x5618bf3a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5618bf325800 0x5618bf3a0b00 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 --2- 172.21.15.28:0/1546880530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618bf324c00 0x5618bf3a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 >> 172.21.15.28:0/1546880530 conn(0x5618bf325000 msgr2=0x5618bf40b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 shutdown_connections 2024-09-10T10:56:03.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:03.935+0000 7f4da1f88280 1 -- 172.21.15.28:0/1546880530 wait complete. 2024-09-10T10:56:03.943 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.114533, result: old journal was 4226680~2030 new journal start will be 8388608 (4159898 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-10T10:56:03.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T10:56:04.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.101+0000 7fd884a00640 1 Processor -- start 2024-09-10T10:56:04.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.101+0000 7fd884a00640 1 -- start start 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd884a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 0x7fd8800b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd884a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8800b4620 con 0x7fd88015e900 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd884a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8800ce100 con 0x7fd8800b4020 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd884a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8800b4f60 con 0x7fd880072760 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd87de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd88015e900 0x7fd8800ad020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35344/0 (socket says 172.21.15.28:35344) 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd87de00640 1 -- 172.21.15.28:0/927950897 learned_addr learned my addr 172.21.15.28:0/927950897 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.102+0000 7fd87e800640 1 --2- 172.21.15.28:0/927950897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 0x7fd8800b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807716580 0 0) 0x7fd8800b4620 con 0x7fd88015e900 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd864005730 con 0x7fd88015e900 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898651757 0 0) 0x7fd8800ce100 con 0x7fd8800b4020 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8800b4620 con 0x7fd8800b4020 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87e800640 1 -- 172.21.15.28:0/927950897 >> v1:172.21.15.138:6789/0 conn(0x7fd8800b4020 legacy=0x7fd880072050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87e800640 1 -- 172.21.15.28:0/927950897 >> v1:172.21.15.28:6789/0 conn(0x7fd88015e900 legacy=0x7fd8800ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.103+0000 7fd87e800640 1 -- 172.21.15.28:0/927950897 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8800ce3a0 con 0x7fd880072760 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.104+0000 7fd87e800640 1 --2- 172.21.15.28:0/927950897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 0x7fd8800b4a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd878009dc0 tx=0x7fd87802a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc21c33e48d91d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.104+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87802ae60 con 0x7fd880072760 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.104+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd878007740 con 0x7fd880072760 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.104+0000 7fd87d400640 1 -- 172.21.15.28:0/927950897 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878007b10 con 0x7fd880072760 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 -- 172.21.15.28:0/927950897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 msgr2=0x7fd8800b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 --2- 172.21.15.28:0/927950897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 0x7fd8800b4a20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd878009dc0 tx=0x7fd87802a640 comp rx=0 tx=0).stop 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 -- 172.21.15.28:0/927950897 shutdown_connections 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 --2- 172.21.15.28:0/927950897 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd880072760 0x7fd8800b4a20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 -- 172.21.15.28:0/927950897 >> 172.21.15.28:0/927950897 conn(0x7fd88006bdf0 msgr2=0x7fd88006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 -- 172.21.15.28:0/927950897 shutdown_connections 2024-09-10T10:56:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.105+0000 7fd884a00640 1 -- 172.21.15.28:0/927950897 wait complete. 2024-09-10T10:56:04.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.106+0000 7fd884a00640 1 Processor -- start 2024-09-10T10:56:04.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 -- start start 2024-09-10T10:56:04.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880072760 0x7fd8801468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 0x7fd880147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8800cee80 con 0x7fd88015e900 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8800ce960 con 0x7fd880072760 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.107+0000 7fd884a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd8800cf000 con 0x7fd8800b4020 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42846/0 (socket says 172.21.15.28:42846) 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 -- 172.21.15.28:0/3716925874 learned_addr learned my addr 172.21.15.28:0/3716925874 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87de00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880072760 0x7fd8801468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd875400640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 0x7fd880147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 -- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 msgr2=0x7fd880147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 0x7fd880147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 -- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880072760 msgr2=0x7fd8801468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880072760 0x7fd8801468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd87e800640 1 -- 172.21.15.28:0/3716925874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd880148620 con 0x7fd88015e900 2024-09-10T10:56:04.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.108+0000 7fd875400640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 0x7fd880147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:04.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd87e800640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fd87802ac30 tx=0x7fd878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878040070 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd87802ae60 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd878031be0 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd880148900 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.109+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd88014a680 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.110+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd88015aa20 con 0x7fd88015e900 2024-09-10T10:56:04.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.111+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd878039040 con 0x7fd88015e900 2024-09-10T10:56:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.112+0000 7fd876a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 0x7fd85c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.112+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fd8780ac520 con 0x7fd88015e900 2024-09-10T10:56:04.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.112+0000 7fd87de00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 0x7fd85c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.113+0000 7fd87de00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 0x7fd85c06e2f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd870000f60 tx=0x7fd870007d70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:04.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.117+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd87807ace0 con 0x7fd88015e900 2024-09-10T10:56:04.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.338+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd88014a4a0 con 0x7fd88015e900 2024-09-10T10:56:04.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.812+0000 7fd876a00640 1 -- 172.21.15.28:0/3716925874 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v343) ==== 162+0+0 (secure 0 0 0) 0x7fd88014a4a0 con 0x7fd88015e900 2024-09-10T10:56:04.813 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T10:56:04.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 msgr2=0x7fd85c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 0x7fd85c06e2f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fd870000f60 tx=0x7fd870007d70 comp rx=0 tx=0).stop 2024-09-10T10:56:04.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 msgr2=0x7fd88015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7fd87802ac30 tx=0x7fd878004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:04.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 shutdown_connections 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd85c06be30 0x7fd85c06e2f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd88015e900 0x7fd88015cf40 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd8800b4020 0x7fd880147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 --2- 172.21.15.28:0/3716925874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880072760 0x7fd8801468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 >> 172.21.15.28:0/3716925874 conn(0x7fd88006bdf0 msgr2=0x7fd880072050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 shutdown_connections 2024-09-10T10:56:04.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.816+0000 7fd884a00640 1 -- 172.21.15.28:0/3716925874 wait complete. 2024-09-10T10:56:04.837 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:04.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.996+0000 7f7a55e00640 1 Processor -- start 2024-09-10T10:56:04.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.996+0000 7f7a55e00640 1 -- start start 2024-09-10T10:56:04.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a55e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7a500d1540 0x7f7a5006dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:04.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a55e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a500d1970 con 0x7f7a5015e740 2024-09-10T10:56:04.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a55e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a500d0780 con 0x7f7a5009b690 2024-09-10T10:56:04.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a55e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7a5015eb20 con 0x7f7a500d1540 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a4e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7a500d1540 0x7f7a5006dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a4ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7a5015e740 0x7f7a5006d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35348/0 (socket says 172.21.15.28:35348) 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.997+0000 7f7a4ea00640 1 -- 172.21.15.28:0/4171845566 learned_addr learned my addr 172.21.15.28:0/4171845566 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.998+0000 7f7a4e000640 1 -- 172.21.15.28:0/4171845566 >> v1:172.21.15.138:6789/0 conn(0x7f7a5009b690 legacy=0x7f7a50070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.998+0000 7f7a4e000640 1 -- 172.21.15.28:0/4171845566 >> v1:172.21.15.28:6789/0 conn(0x7f7a5015e740 legacy=0x7f7a5006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:04.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.998+0000 7f7a4e000640 1 -- 172.21.15.28:0/4171845566 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500d0a60 con 0x7f7a500d1540 2024-09-10T10:56:05.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.998+0000 7f7a4d600640 1 -- 172.21.15.28:0/4171845566 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181276689 0 0) 0x7f7a500d1970 con 0x7f7a5015e740 2024-09-10T10:56:05.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.999+0000 7f7a4e000640 1 --2- 172.21.15.28:0/4171845566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a5006dbd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0fad3e4c306536 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:05.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.999+0000 7f7a4d600640 1 -- 172.21.15.28:0/4171845566 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c037070 con 0x7f7a500d1540 2024-09-10T10:56:05.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.999+0000 7f7a4d600640 1 -- 172.21.15.28:0/4171845566 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c032640 con 0x7f7a500d1540 2024-09-10T10:56:05.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:04.999+0000 7f7a4d600640 1 -- 172.21.15.28:0/4171845566 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c032a10 con 0x7f7a500d1540 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 -- 172.21.15.28:0/4171845566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 msgr2=0x7f7a5006dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 --2- 172.21.15.28:0/4171845566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a5006dbd0 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c02a2c0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 -- 172.21.15.28:0/4171845566 shutdown_connections 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 --2- 172.21.15.28:0/4171845566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a5006dbd0 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 -- 172.21.15.28:0/4171845566 >> 172.21.15.28:0/4171845566 conn(0x7f7a500d23a0 msgr2=0x7f7a500d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 -- 172.21.15.28:0/4171845566 shutdown_connections 2024-09-10T10:56:05.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.000+0000 7f7a55e00640 1 -- 172.21.15.28:0/4171845566 wait complete. 2024-09-10T10:56:05.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.002+0000 7f7a55e00640 1 Processor -- start 2024-09-10T10:56:05.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.002+0000 7f7a55e00640 1 -- start start 2024-09-10T10:56:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a5009b690 0x7f7a500b7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a5015e740 0x7f7a500cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5009baa0 con 0x7f7a500d1540 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500d1020 con 0x7f7a5015e740 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a55e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5006cd20 con 0x7f7a5009b690 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42864/0 (socket says 172.21.15.28:42864) 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 -- 172.21.15.28:0/1623741380 learned_addr learned my addr 172.21.15.28:0/1623741380 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4f400640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a5015e740 0x7f7a500cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4ea00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a5009b690 0x7f7a500b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 -- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a5009b690 msgr2=0x7f7a500b7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a5009b690 0x7f7a500b7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 -- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a5015e740 msgr2=0x7f7a500cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a5015e740 0x7f7a500cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.003+0000 7f7a4e000640 1 -- 172.21.15.28:0/1623741380 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500b6660 con 0x7f7a500d1540 2024-09-10T10:56:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a4e000640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c037050 con 0x7f7a500d1540 2024-09-10T10:56:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a3c038040 con 0x7f7a500d1540 2024-09-10T10:56:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c004490 con 0x7f7a500d1540 2024-09-10T10:56:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a500b6940 con 0x7f7a500d1540 2024-09-10T10:56:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.004+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a500b6f00 con 0x7f7a500d1540 2024-09-10T10:56:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.005+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5006d0b0 con 0x7f7a500d1540 2024-09-10T10:56:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.006+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7a3c030900 con 0x7f7a500d1540 2024-09-10T10:56:05.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.007+0000 7f7a47400640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 0x7f7a3006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:05.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.007+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f7a3c0abd60 con 0x7f7a500d1540 2024-09-10T10:56:05.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.007+0000 7f7a4ea00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 0x7f7a3006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.008+0000 7f7a4ea00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 0x7f7a3006e2f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000f60 tx=0x7f7a40007d20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.011+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7a3c07a5b0 con 0x7f7a500d1540 2024-09-10T10:56:05.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.231+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a5015df60 con 0x7f7a500d1540 2024-09-10T10:56:05.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.231+0000 7f7a47400640 1 -- 172.21.15.28:0/1623741380 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 344 v344) ==== 95+0+5235 (secure 0 0 0) 0x7f7a3c079eb0 con 0x7f7a500d1540 2024-09-10T10:56:05.232 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:05.232 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":344,"btime":"2024-09-10T10:56:04:815999+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":344,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:04.815980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:replay","state_seq":1,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T10:56:05.233 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 344 2024-09-10T10:56:05.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 msgr2=0x7f7a3006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 0x7f7a3006e2f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7a40000f60 tx=0x7f7a40007d20 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 msgr2=0x7f7a50157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0097a0 tx=0x7f7a3c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 shutdown_connections 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7a3006be30 0x7f7a3006e2f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a5015e740 0x7f7a500cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7a500d1540 0x7f7a50157cb0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 --2- 172.21.15.28:0/1623741380 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7a5009b690 0x7f7a500b7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 >> 172.21.15.28:0/1623741380 conn(0x7f7a500d23a0 msgr2=0x7f7a5006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 shutdown_connections 2024-09-10T10:56:05.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:05.233+0000 7f7a55e00640 1 -- 172.21.15.28:0/1623741380 wait complete. 2024-09-10T10:56:05.653 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 344, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:55:48.269671+0000', 'modified': '2024-09-10T10:56:04.815980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 151, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10686}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10686': {'gid': 10686, 'name': 'a', 'rank': 0, 'incarnation': 344, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.28:6835/2598694071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2598694071}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2598694071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:56:05.653 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10686:up:replay 2024-09-10T10:56:06.654 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:06.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.822+0000 7f9db2600640 1 Processor -- start 2024-09-10T10:56:06.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.822+0000 7f9db2600640 1 -- start start 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.823+0000 7f9db2600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9dac162060 0x7f9dac0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.823+0000 7f9db2600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9dac0b64b0 0x7f9dac0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.823+0000 7f9db2600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dac06d9f0 con 0x7f9dac161c80 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.823+0000 7f9db2600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9dac06dcd0 con 0x7f9dac162060 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.823+0000 7f9db2600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9dac06dfb0 con 0x7f9dac0b64b0 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9dab400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9dac161c80 0x7f9dac06e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35360/0 (socket says 172.21.15.28:35360) 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9dab400640 1 -- 172.21.15.28:0/1924123605 learned_addr learned my addr 172.21.15.28:0/1924123605 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daaa00640 1 --2- 172.21.15.28:0/1924123605 >> v2:172.21.15.28:3300/0 conn(0x7f9dac162060 0x7f9dac0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9dabe00640 1 --2- 172.21.15.28:0/1924123605 >> v2:172.21.15.138:3300/0 conn(0x7f9dac0b64b0 0x7f9dac0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daaa00640 1 -- 172.21.15.28:0/1924123605 >> v2:172.21.15.138:3300/0 conn(0x7f9dac0b64b0 msgr2=0x7f9dac0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:06.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daaa00640 1 --2- 172.21.15.28:0/1924123605 >> v2:172.21.15.138:3300/0 conn(0x7f9dac0b64b0 0x7f9dac0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daaa00640 1 -- 172.21.15.28:0/1924123605 >> v1:172.21.15.28:6789/0 conn(0x7f9dac161c80 legacy=0x7f9dac06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daaa00640 1 -- 172.21.15.28:0/1924123605 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac0d01f0 con 0x7f9dac162060 2024-09-10T10:56:06.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.824+0000 7f9daa000640 1 -- 172.21.15.28:0/1924123605 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 107600724 0 0) 0x7f9dac06d9f0 con 0x7f9dac161c80 2024-09-10T10:56:06.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.826+0000 7f9daaa00640 1 --2- 172.21.15.28:0/1924123605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac162060 0x7f9dac0b5f70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f9dac08d750 tx=0x7f9d9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5356af6b72d413a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9daa000640 1 -- 172.21.15.28:0/1924123605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98037070 con 0x7f9dac162060 2024-09-10T10:56:06.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9daa000640 1 -- 172.21.15.28:0/1924123605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98032440 con 0x7f9dac162060 2024-09-10T10:56:06.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9daa000640 1 -- 172.21.15.28:0/1924123605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98032810 con 0x7f9dac162060 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 -- 172.21.15.28:0/1924123605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac162060 msgr2=0x7f9dac0b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 --2- 172.21.15.28:0/1924123605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac162060 0x7f9dac0b5f70 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f9dac08d750 tx=0x7f9d9802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 -- 172.21.15.28:0/1924123605 shutdown_connections 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 --2- 172.21.15.28:0/1924123605 >> v2:172.21.15.138:3300/0 conn(0x7f9dac0b64b0 0x7f9dac0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 --2- 172.21.15.28:0/1924123605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac162060 0x7f9dac0b5f70 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.827+0000 7f9db2600640 1 -- 172.21.15.28:0/1924123605 >> 172.21.15.28:0/1924123605 conn(0x7f9dac0d44f0 msgr2=0x7f9dac0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.828+0000 7f9db2600640 1 -- 172.21.15.28:0/1924123605 shutdown_connections 2024-09-10T10:56:06.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.828+0000 7f9db2600640 1 -- 172.21.15.28:0/1924123605 wait complete. 2024-09-10T10:56:06.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.829+0000 7f9db2600640 1 Processor -- start 2024-09-10T10:56:06.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.829+0000 7f9db2600640 1 -- start start 2024-09-10T10:56:06.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dac161c80 0x7f9dac0b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dac162060 0x7f9dac160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac0cf1e0 con 0x7f9dac0b64b0 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac0d2aa0 con 0x7f9dac161c80 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9db2600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac0cf360 con 0x7f9dac162060 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.830+0000 7f9dab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9daaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dac161c80 0x7f9dac0b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42882/0 (socket says 172.21.15.28:42882) 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 -- 172.21.15.28:0/2746181663 learned_addr learned my addr 172.21.15.28:0/2746181663 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:06.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dabe00640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dac162060 0x7f9dac160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 -- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dac162060 msgr2=0x7f9dac160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dac162060 0x7f9dac160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 -- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dac161c80 msgr2=0x7f9dac0b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dac161c80 0x7f9dac0b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 -- 172.21.15.28:0/2746181663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac0c8e40 con 0x7f9dac0b64b0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.831+0000 7f9dab400640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c4d0 tx=0x7f9d9c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.832+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c018070 con 0x7f9dac0b64b0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.832+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c007500 con 0x7f9dac0b64b0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.832+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c004a20 con 0x7f9dac0b64b0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.832+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac0c9120 con 0x7f9dac0b64b0 2024-09-10T10:56:06.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.832+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac0c96e0 con 0x7f9dac0b64b0 2024-09-10T10:56:06.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.833+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac0c9ac0 con 0x7f9dac0b64b0 2024-09-10T10:56:06.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.834+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d9c004bc0 con 0x7f9dac0b64b0 2024-09-10T10:56:06.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.835+0000 7f9da3400640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:06.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.835+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f9d9c08a300 con 0x7f9dac0b64b0 2024-09-10T10:56:06.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.836+0000 7f9daaa00640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:06.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.837+0000 7f9daaa00640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d98034000 tx=0x7f9d9802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:06.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:06.840+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9dac0c9ac0 con 0x7f9dac0b64b0 2024-09-10T10:56:07.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.059+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dac0b5f70 con 0x7f9dac0b64b0 2024-09-10T10:56:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.060+0000 7f9da3400640 1 -- 172.21.15.28:0/2746181663 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 346 v346) ==== 95+0+5235 (secure 0 0 0) 0x7f9d9c08a0c0 con 0x7f9dac0b64b0 2024-09-10T10:56:07.061 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:07.061 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":346,"btime":"2024-09-10T10:56:06:819718+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":346,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:05.821325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T10:56:07.061 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 346 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 msgr2=0x7f9d8c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f9d98034000 tx=0x7f9d9802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 msgr2=0x7f9dac1f3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00c4d0 tx=0x7f9d9c00c990 comp rx=0 tx=0).stop 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 shutdown_connections 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f9d8c06be30 0x7f9d8c06e2f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:07.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9dac162060 0x7f9dac160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dac161c80 0x7f9dac0b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 --2- 172.21.15.28:0/2746181663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9dac0b64b0 0x7f9dac1f3d10 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 >> 172.21.15.28:0/2746181663 conn(0x7f9dac0d44f0 msgr2=0x7f9dac0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 shutdown_connections 2024-09-10T10:56:07.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:07.062+0000 7f9db2600640 1 -- 172.21.15.28:0/2746181663 wait complete. 2024-09-10T10:56:07.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 346, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:55:48.269671+0000', 'modified': '2024-09-10T10:56:05.821325+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 151, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10686}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10686': {'gid': 10686, 'name': 'a', 'rank': 0, 'incarnation': 344, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.28:6835/2598694071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2598694071}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2598694071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T10:56:07.482 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10686:up:rejoin 2024-09-10T10:56:08.483 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:08.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.649+0000 7f6791c00640 1 Processor -- start 2024-09-10T10:56:08.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.649+0000 7f6791c00640 1 -- start start 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.650+0000 7f6791c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f678c147e90 0x7f678c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.650+0000 7f6791c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c072a00 con 0x7f678c0725f0 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.650+0000 7f6791c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f678c06f480 con 0x7f678c147230 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.650+0000 7f6791c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f678c0b34d0 con 0x7f678c147e90 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f678c147e90 0x7f678c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f678c0725f0 0x7f678c0aee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35366/0 (socket says 172.21.15.28:35366) 2024-09-10T10:56:08.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678b400640 1 -- 172.21.15.28:0/643630519 learned_addr learned my addr 172.21.15.28:0/643630519 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678a000640 1 -- 172.21.15.28:0/643630519 >> v1:172.21.15.138:6789/0 conn(0x7f678c147230 legacy=0x7f678c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678a000640 1 -- 172.21.15.28:0/643630519 >> v1:172.21.15.28:6789/0 conn(0x7f678c0725f0 legacy=0x7f678c0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f6789600640 1 -- 172.21.15.28:0/643630519 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17009637 0 0) 0x7f6784001490 con 0x7f678c0725f0 2024-09-10T10:56:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.651+0000 7f678a000640 1 -- 172.21.15.28:0/643630519 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c06e900 con 0x7f678c147e90 2024-09-10T10:56:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.652+0000 7f678a000640 1 --2- 172.21.15.28:0/643630519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147e90 0x7f678c0720b0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f67780097a0 tx=0x7f677802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3401e1a97f05b024 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.652+0000 7f6789600640 1 -- 172.21.15.28:0/643630519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778037070 con 0x7f678c147e90 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6789600640 1 -- 172.21.15.28:0/643630519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6778032440 con 0x7f678c147e90 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6789600640 1 -- 172.21.15.28:0/643630519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778032810 con 0x7f678c147e90 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 -- 172.21.15.28:0/643630519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147e90 msgr2=0x7f678c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 --2- 172.21.15.28:0/643630519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147e90 0x7f678c0720b0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7f67780097a0 tx=0x7f677802a1e0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 -- 172.21.15.28:0/643630519 shutdown_connections 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 --2- 172.21.15.28:0/643630519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147e90 0x7f678c0720b0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 -- 172.21.15.28:0/643630519 >> 172.21.15.28:0/643630519 conn(0x7f678c067d70 msgr2=0x7f678c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 -- 172.21.15.28:0/643630519 shutdown_connections 2024-09-10T10:56:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.653+0000 7f6791c00640 1 -- 172.21.15.28:0/643630519 wait complete. 2024-09-10T10:56:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.655+0000 7f6791c00640 1 Processor -- start 2024-09-10T10:56:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.655+0000 7f6791c00640 1 -- start start 2024-09-10T10:56:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c0725f0 0x7f678c15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c147e90 0x7f678c0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f678c06eec0 con 0x7f678c147230 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f678c06f1a0 con 0x7f678c0725f0 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f6791c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f678c147610 con 0x7f678c147e90 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f678a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f678a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42892/0 (socket says 172.21.15.28:42892) 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f678a000640 1 -- 172.21.15.28:0/370099076 learned_addr learned my addr 172.21.15.28:0/370099076 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f678aa00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c0725f0 0x7f678c15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.656+0000 7f678b400640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c147e90 0x7f678c0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 -- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c147e90 msgr2=0x7f678c0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c147e90 0x7f678c0b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 -- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c0725f0 msgr2=0x7f678c15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c0725f0 0x7f678c15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 -- 172.21.15.28:0/370099076 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c13e290 con 0x7f678c147230 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f678a000640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f67780097a0 tx=0x7f67780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778037040 con 0x7f678c147230 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6778004250 con 0x7f678c147230 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c13e570 con 0x7f678c147230 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.657+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778032e20 con 0x7f678c147230 2024-09-10T10:56:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.658+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c0b13e0 con 0x7f678c147230 2024-09-10T10:56:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.659+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c072090 con 0x7f678c147230 2024-09-10T10:56:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.660+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6778030770 con 0x7f678c147230 2024-09-10T10:56:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.661+0000 7f6782a00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:08.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.661+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f67780ac1e0 con 0x7f678c147230 2024-09-10T10:56:08.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.661+0000 7f678aa00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:08.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.662+0000 7f678aa00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f677c004f60 tx=0x7f677c007e30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:08.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.665+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f677807a2c0 con 0x7f678c147230 2024-09-10T10:56:08.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.887+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f678c1452b0 con 0x7f678c147230 2024-09-10T10:56:08.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.887+0000 7f6782a00640 1 -- 172.21.15.28:0/370099076 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5244 (secure 0 0 0) 0x7f677807e5c0 con 0x7f678c147230 2024-09-10T10:56:08.889 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:08.889 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":347,"btime":"2024-09-10T10:56:07:825830+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:07.825823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10686,"qdb_cluster":[10686]},"id":7}]} 2024-09-10T10:56:08.889 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 347 2024-09-10T10:56:08.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 msgr2=0x7f676c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f677c004f60 tx=0x7f677c007e30 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 msgr2=0x7f678c13b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7f67780097a0 tx=0x7f67780046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 shutdown_connections 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f676c06be30 0x7f676c06e2f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f678c147e90 0x7f678c0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f678c147230 0x7f678c13b7d0 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 --2- 172.21.15.28:0/370099076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f678c0725f0 0x7f678c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.889+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 >> 172.21.15.28:0/370099076 conn(0x7f678c067d70 msgr2=0x7f678c0aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.890+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 shutdown_connections 2024-09-10T10:56:08.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:08.890+0000 7f6791c00640 1 -- 172.21.15.28:0/370099076 wait complete. 2024-09-10T10:56:09.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 347, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T10:55:48.269671+0000', 'modified': '2024-09-10T10:56:07.825823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 151, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10686}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10686': {'gid': 10686, 'name': 'a', 'rank': 0, 'incarnation': 344, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.28:6835/2598694071', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6833', 'nonce': 2598694071}, {'type': 'v1', 'addr': '172.21.15.28:6835', 'nonce': 2598694071}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 10686, 'qdb_cluster': [10686]} 2024-09-10T10:56:09.309 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T10:56:09.309 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T10:56:09.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.467+0000 7fb7b8400640 1 Processor -- start 2024-09-10T10:56:09.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.467+0000 7fb7b8400640 1 -- start start 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b8400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b8400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b006f480 con 0x7fb7b0147e90 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b8400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7b006e900 con 0x7fb7b00727c0 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b8400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b00b34d0 con 0x7fb7b0147230 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b5000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb7b0147e90 0x7fb7b00720b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35376/0 (socket says 172.21.15.28:35376) 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b5000640 1 -- 172.21.15.28:0/1568899940 learned_addr learned my addr 172.21.15.28:0/1568899940 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:09.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/1568899940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1119751215 0 0) 0x7fb7b006f480 con 0x7fb7b0147e90 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.469+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb798003520 con 0x7fb7b0147e90 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.470+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 470607863 0 0) 0x7fb7b006e900 con 0x7fb7b00727c0 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.470+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7b006f480 con 0x7fb7b00727c0 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.470+0000 7fb7b5a00640 1 -- 172.21.15.28:0/1568899940 >> v1:172.21.15.138:6789/0 conn(0x7fb7b00727c0 legacy=0x7fb7b00aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.470+0000 7fb7b5a00640 1 -- 172.21.15.28:0/1568899940 >> v1:172.21.15.28:6789/0 conn(0x7fb7b0147e90 legacy=0x7fb7b00720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.470+0000 7fb7b5a00640 1 -- 172.21.15.28:0/1568899940 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b006ebe0 con 0x7fb7b0147230 2024-09-10T10:56:09.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/1568899940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b006e0b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40097d0 tx=0x7fb7a402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3b44cf7d5b84df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4037070 con 0x7fb7b0147230 2024-09-10T10:56:09.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a4032440 con 0x7fb7b0147230 2024-09-10T10:56:09.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7afe00640 1 -- 172.21.15.28:0/1568899940 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4032810 con 0x7fb7b0147230 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7b8400640 1 -- 172.21.15.28:0/1568899940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 msgr2=0x7fb7b006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.471+0000 7fb7b8400640 1 --2- 172.21.15.28:0/1568899940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b006e0b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb7a40097d0 tx=0x7fb7a402a210 comp rx=0 tx=0).stop 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.472+0000 7fb7b8400640 1 -- 172.21.15.28:0/1568899940 shutdown_connections 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.472+0000 7fb7b8400640 1 --2- 172.21.15.28:0/1568899940 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b006e0b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.472+0000 7fb7b8400640 1 -- 172.21.15.28:0/1568899940 >> 172.21.15.28:0/1568899940 conn(0x7fb7b0067d70 msgr2=0x7fb7b006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.472+0000 7fb7b8400640 1 -- 172.21.15.28:0/1568899940 shutdown_connections 2024-09-10T10:56:09.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.472+0000 7fb7b8400640 1 -- 172.21.15.28:0/1568899940 wait complete. 2024-09-10T10:56:09.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.473+0000 7fb7b8400640 1 Processor -- start 2024-09-10T10:56:09.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.473+0000 7fb7b8400640 1 -- start start 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 0x7fb7b00c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b01e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b006f1a0 con 0x7fb7b0147e90 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b006e5f0 con 0x7fb7b00727c0 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b8400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b0147610 con 0x7fb7b0147230 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42898/0 (socket says 172.21.15.28:42898) 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.474+0000 7fb7b6400640 1 -- 172.21.15.28:0/4186473073 learned_addr learned my addr 172.21.15.28:0/4186473073 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 0x7fb7b00c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b5000640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b01e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 -- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 msgr2=0x7fb7b01e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b01e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 -- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 msgr2=0x7fb7b00c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 0x7fb7b00c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 -- 172.21.15.28:0/4186473073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b01e6490 con 0x7fb7b0147e90 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 0x7fb7b00c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:09.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b6400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c7a0 tx=0x7fb7a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7ad600640 1 -- 172.21.15.28:0/4186473073 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8003b70 con 0x7fb7b0147e90 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7ad600640 1 -- 172.21.15.28:0/4186473073 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8003d10 con 0x7fb7b0147e90 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b00c5ad0 con 0x7fb7b0147e90 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.475+0000 7fb7ad600640 1 -- 172.21.15.28:0/4186473073 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800aa90 con 0x7fb7b0147e90 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.476+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b00c6090 con 0x7fb7b0147e90 2024-09-10T10:56:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.478+0000 7fb7ad600640 1 -- 172.21.15.28:0/4186473073 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb7a800ac30 con 0x7fb7b0147e90 2024-09-10T10:56:09.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.479+0000 7fb7ad600640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 0x7fb79006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.479+0000 7fb7ad600640 1 -- 172.21.15.28:0/4186473073 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fb7a808afe0 con 0x7fb7b0147e90 2024-09-10T10:56:09.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.479+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 0x7fb79006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.479+0000 7fb7bb3f2740 1 Processor -- start 2024-09-10T10:56:09.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.479+0000 7fb7bb3f2740 1 -- start start 2024-09-10T10:56:09.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 0x7fb79006e2f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4034000 tx=0x7fb7a402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x557868a0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x5578689c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5578689263b0 con 0x5578689c54b0 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55786892b410 con 0x5578689c9480 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5578688d3620 con 0x5578689c7cf0 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.480+0000 7fb7b5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42912/0 (socket says 172.21.15.28:42912) 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 -- 172.21.15.28:0/3101372265 learned_addr learned my addr 172.21.15.28:0/3101372265 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x557868a0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b6400640 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x5578689c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 -- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 msgr2=0x557868a0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x557868a0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 -- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 msgr2=0x5578689c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x5578689c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 -- 172.21.15.28:0/3101372265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55786863e2d0 con 0x5578689c54b0 2024-09-10T10:56:09.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7b5000640 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001570 tx=0x7fb79c001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d780218107aca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.481+0000 7fb7a2000640 1 -- 172.21.15.28:0/3101372265 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c019070 con 0x5578689c54b0 2024-09-10T10:56:09.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7a2000640 1 -- 172.21.15.28:0/3101372265 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c005850 con 0x5578689c54b0 2024-09-10T10:56:09.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7a2000640 1 -- 172.21.15.28:0/3101372265 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c005c00 con 0x5578689c54b0 2024-09-10T10:56:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/3101372265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 msgr2=0x5578689c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001570 tx=0x7fb79c001a30 comp rx=0 tx=0).stop 2024-09-10T10:56:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/3101372265 shutdown_connections 2024-09-10T10:56:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x5578689c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c54b0 0x5578689c80d0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/3101372265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x557868a0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/3101372265 >> 172.21.15.28:0/3101372265 conn(0x5578689c4740 msgr2=0x557868a0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/3101372265 shutdown_connections 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.482+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/3101372265 wait complete. 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.483+0000 7fb7bb3f2740 1 -- start start 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.483+0000 7fb7bb3f2740 1 Processor -- start 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x5578689e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689e8cb0 0x5578689c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5578688e6780 con 0x5578689c9480 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5578689273d0 con 0x5578689e8cb0 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5578689c3be0 con 0x5578689c7cf0 2024-09-10T10:56:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7bb3f2740 10 client.0 _ll_register_callbacks cb 0x5578689245d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7b5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7b5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42914/0 (socket says 172.21.15.28:42914) 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7b5000640 1 -- 172.21.15.28:0/2322291680 learned_addr learned my addr 172.21.15.28:0/2322291680 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.484+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x5578689e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b6400640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689e8cb0 0x5578689c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 -- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 msgr2=0x5578689e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x5578689e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 -- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689e8cb0 msgr2=0x5578689c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689e8cb0 0x5578689c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 -- 172.21.15.28:0/2322291680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578687086b0 con 0x5578689c9480 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x5578689e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001a70 tx=0x7fb79c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf0610a9f256ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb796a00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79c023060 con 0x5578689c9480 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fb79c0070d0 con 0x5578689c9480 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.485+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb79c005850 con 0x5578689c9480 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=347) ==== 24+0+0 (secure 0 0 0) 0x7fb79c029050 con 0x5578689c9480 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 10 client.10749 fetch_fsmap learned FSMap version 347 2024-09-10T10:56:09.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=347,monmap=2+}) -- 0x557868a1d1f0 con 0x5578689c9480 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb79c005850 con 0x5578689c9480 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 347) ==== 3502+0+0 (secure 0 0 0) 0x7fb79c02c020 con 0x5578689c9480 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 10 client.10749 fetch_fsmap finished waiting for FSMap version 347 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 10 client.10749 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 20 client.10749 populate_metadata read hostname 'smithi028' 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689c1bb0 0x5578689c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 4 client.10749 mds_command: new command op to 10686 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 --> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557868938b40 con 0x5578689c1bb0 2024-09-10T10:56:09.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.486+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689c1bb0 0x5578689c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.487+0000 7fb796a00640 10 client.10749 ms_handle_connect on v2:172.21.15.28:6833/2598694071 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.487+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689c1bb0 0x5578689c1f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121e7d4656eb9e1e server_cookie=b53e6d4455f86a67 in_seq=0 out_seq=0 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb796a00640 1 -- 172.21.15.28:0/2322291680 <== mds.0 v2:172.21.15.28:6833/2598694071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557868938b40 con 0x5578689c1bb0 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb796a00640 10 client.10749 handle_command_reply: tid=0 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 client.10749 shutdown 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 msgr2=0x5578689e8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001a70 tx=0x7fb79c011000 comp rx=0 tx=0).stop 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 shutdown_connections 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689c1bb0 0x5578689c1f90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689e8cb0 0x5578689c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c9480 0x5578689e8770 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2322291680 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c7cf0 0x5578689e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb796a00640 0 client.10749 ms_handle_reset on v2:172.21.15.28:6833/2598694071 2024-09-10T10:56:09.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 >> 172.21.15.28:0/2322291680 conn(0x5578689c54b0 msgr2=0x557868a0bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 shutdown_connections 2024-09-10T10:56:09.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.488+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2322291680 wait complete. 2024-09-10T10:56:09.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.489+0000 7fb7bb3f2740 20 client.10749 trim_cache size 0 max 16384 2024-09-10T10:56:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.507+0000 7fb7bb3f2740 1 Processor -- start 2024-09-10T10:56:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 -- start start 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c7cf0 0x557868a0abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c9480 0x557868a30790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b0200080 con 0x5578689e8cb0 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00ca230 con 0x5578689c7cf0 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.508+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb7b01ffa30 con 0x5578689c9480 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42930/0 (socket says 172.21.15.28:42930) 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 -- 172.21.15.28:0/1783332781 learned_addr learned my addr 172.21.15.28:0/1783332781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 -- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c9480 msgr2=0x557868a30790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c9480 0x557868a30790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 -- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c7cf0 msgr2=0x557868a0abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c7cf0 0x557868a0abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 -- 172.21.15.28:0/1783332781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557868a1cf50 con 0x5578689e8cb0 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7b6400640 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8055520 tx=0x7fb7a8055360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137a2fa5a7b3cbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7a1600640 1 -- 172.21.15.28:0/1783332781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80597c0 con 0x5578689e8cb0 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7a1600640 1 -- 172.21.15.28:0/1783332781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a80590c0 con 0x5578689e8cb0 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.509+0000 7fb7a1600640 1 -- 172.21.15.28:0/1783332781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a805d470 con 0x5578689e8cb0 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/1783332781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 msgr2=0x557868a30cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8055520 tx=0x7fb7a8055360 comp rx=0 tx=0).stop 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/1783332781 shutdown_connections 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689e8cb0 0x557868a30cd0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689c9480 0x557868a30790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/1783332781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c7cf0 0x557868a0abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/1783332781 >> 172.21.15.28:0/1783332781 conn(0x5578689c54b0 msgr2=0x5578689c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/1783332781 shutdown_connections 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.510+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/1783332781 wait complete. 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.511+0000 7fb7bb3f2740 1 -- start start 2024-09-10T10:56:09.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.511+0000 7fb7bb3f2740 1 Processor -- start 2024-09-10T10:56:09.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x557868a30d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 0x557868a8b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5578689e7190 con 0x5578689c7cf0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557868a1d530 con 0x5578689c9480 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x557868a20be0 con 0x5578689e8cb0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7bb3f2740 10 client.0 _ll_register_callbacks cb 0x7fb79c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7b5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7b5a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42946/0 (socket says 172.21.15.28:42946) 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7b5a00640 1 -- 172.21.15.28:0/2174147685 learned_addr learned my addr 172.21.15.28:0/2174147685 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x557868a30d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.512+0000 7fb7b6400640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 0x557868a8b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 -- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 msgr2=0x557868a8b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 0x557868a8b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 -- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 msgr2=0x557868a30d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x557868a30d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 -- 172.21.15.28:0/2174147685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557868a09730 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b6400640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 0x557868a8b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7b5a00640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fb7a402a250 tx=0x7fb7a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9621f940c860b555 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb796a00640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4037040 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fb7a403e660 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a80590c0 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=347) ==== 24+0+0 (secure 0 0 0) 0x7fb7a405c050 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7bb3f2740 10 client.10755 fetch_fsmap learned FSMap version 347 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.513+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=347,monmap=2+}) -- 0x557868a31280 con 0x5578689c7cf0 2024-09-10T10:56:09.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a80590c0 con 0x5578689c7cf0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 347) ==== 3502+0+0 (secure 0 0 0) 0x7fb7a405f020 con 0x5578689c7cf0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 10 client.10755 fetch_fsmap finished waiting for FSMap version 347 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 10 client.10755 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 20 client.10755 populate_metadata read hostname 'smithi028' 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689f25e0 0x557868a1aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 4 client.10755 mds_command: new command op to 10686 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 --> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] -- command(tid 0: {"prefix": "status"}) -- 0x557868932e90 con 0x5578689f25e0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.514+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689f25e0 0x557868a1aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7b5000640 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689f25e0 0x557868a1aff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e76f0d98ba3490c server_cookie=f7426215f1656c1f in_seq=0 out_seq=0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb796a00640 10 client.10755 ms_handle_connect on v2:172.21.15.28:6833/2598694071 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb796a00640 1 -- 172.21.15.28:0/2174147685 <== mds.0 v2:172.21.15.28:6833/2598694071 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x557868932e90 con 0x5578689f25e0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb796a00640 10 client.10755 handle_command_reply: tid=0 2024-09-10T10:56:09.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 client.10755 shutdown 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 msgr2=0x557868a0fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fb7a402a250 tx=0x7fb7a40046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 shutdown_connections 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:6833/2598694071,v1:172.21.15.28:6835/2598694071] conn(0x5578689f25e0 0x557868a1aff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5578689e8cb0 0x557868a8b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5578689c9480 0x557868a30d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 --2- 172.21.15.28:0/2174147685 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5578689c7cf0 0x557868a0fb90 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb796a00640 0 client.10755 ms_handle_reset on v2:172.21.15.28:6833/2598694071 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 >> 172.21.15.28:0/2174147685 conn(0x5578689c54b0 msgr2=0x5578689e83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 shutdown_connections 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 1 -- 172.21.15.28:0/2174147685 wait complete. 2024-09-10T10:56:09.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.515+0000 7fb7bb3f2740 20 client.10755 trim_cache size 0 max 16384 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "f658987a-dd14-4573-bcd3-ee2a383c9fe0", 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 10686, 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 4.694645103, 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 347, 2024-09-10T10:56:09.521 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 151, 2024-09-10T10:56:09.522 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 151, 2024-09-10T10:56:09.522 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 5.6341110490000004 2024-09-10T10:56:09.522 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 msgr2=0x7fb79006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 0x7fb79006e2f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4034000 tx=0x7fb7a402e000 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 msgr2=0x7fb7b015de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800c7a0 tx=0x7fb7a800cc60 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 shutdown_connections 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb79006be30 0x7fb79006e2f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7b0147e90 0x7fb7b015de90 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb7b0147230 0x7fb7b01e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 --2- 172.21.15.28:0/4186473073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7b00727c0 0x7fb7b00c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 >> 172.21.15.28:0/4186473073 conn(0x7fb7b0067d70 msgr2=0x7fb7b00af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 shutdown_connections 2024-09-10T10:56:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.516+0000 7fb7b8400640 1 -- 172.21.15.28:0/4186473073 wait complete. 2024-09-10T10:56:09.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T10:56:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.686+0000 7f6659e00640 1 Processor -- start 2024-09-10T10:56:09.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.686+0000 7f6659e00640 1 -- start start 2024-09-10T10:56:09.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.687+0000 7f6659e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6654072de0 0x7f665415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.687+0000 7f6659e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6654146140 con 0x7f6654072700 2024-09-10T10:56:09.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.687+0000 7f6659e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66540cd740 con 0x7f6654072320 2024-09-10T10:56:09.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.687+0000 7f6659e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f66540cda20 con 0x7f6654072de0 2024-09-10T10:56:09.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6653400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6654072700 0x7f665415d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35382/0 (socket says 172.21.15.28:35382) 2024-09-10T10:56:09.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6653400640 1 -- 172.21.15.28:0/3498374419 learned_addr learned my addr 172.21.15.28:0/3498374419 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6652000640 1 --2- 172.21.15.28:0/3498374419 >> v2:172.21.15.138:3300/0 conn(0x7f6654072de0 0x7f665415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066333326 0 0) 0x7f6654146140 con 0x7f6654072700 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6638003520 con 0x7f6654072700 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139403846 0 0) 0x7f66540cd740 con 0x7f6654072320 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6654146140 con 0x7f6654072320 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6652000640 1 -- 172.21.15.28:0/3498374419 >> v1:172.21.15.138:6789/0 conn(0x7f6654072320 legacy=0x7f66540cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6652000640 1 -- 172.21.15.28:0/3498374419 >> v1:172.21.15.28:6789/0 conn(0x7f6654072700 legacy=0x7f665415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.688+0000 7f6652000640 1 -- 172.21.15.28:0/3498374419 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66540cdd00 con 0x7f6654072de0 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6652000640 1 --2- 172.21.15.28:0/3498374419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072de0 0x7f665415d1e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f66400097a0 tx=0x7f664002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e196c7fb5e0441f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640037070 con 0x7f6654072de0 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640032440 con 0x7f6654072de0 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6651600640 1 -- 172.21.15.28:0/3498374419 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640032810 con 0x7f6654072de0 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6659e00640 1 -- 172.21.15.28:0/3498374419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072de0 msgr2=0x7f665415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.690+0000 7f6659e00640 1 --2- 172.21.15.28:0/3498374419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072de0 0x7f665415d1e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f66400097a0 tx=0x7f664002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.691+0000 7f6659e00640 1 -- 172.21.15.28:0/3498374419 shutdown_connections 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.691+0000 7f6659e00640 1 --2- 172.21.15.28:0/3498374419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072de0 0x7f665415d1e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.691+0000 7f6659e00640 1 -- 172.21.15.28:0/3498374419 >> 172.21.15.28:0/3498374419 conn(0x7f6654067d70 msgr2=0x7f665406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.691+0000 7f6659e00640 1 -- 172.21.15.28:0/3498374419 shutdown_connections 2024-09-10T10:56:09.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.691+0000 7f6659e00640 1 -- 172.21.15.28:0/3498374419 wait complete. 2024-09-10T10:56:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.692+0000 7f6659e00640 1 Processor -- start 2024-09-10T10:56:09.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.692+0000 7f6659e00640 1 -- start start 2024-09-10T10:56:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072320 0x7f66540ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6654072de0 0x7f66540b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f665415dfb0 con 0x7f6654072700 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f665415de30 con 0x7f6654072320 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6659e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f665415e130 con 0x7f6654072de0 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.693+0000 7f6652000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42960/0 (socket says 172.21.15.28:42960) 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 -- 172.21.15.28:0/1585100463 learned_addr learned my addr 172.21.15.28:0/1585100463 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6653400640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6654072de0 0x7f66540b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652a00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072320 0x7f66540ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 -- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6654072de0 msgr2=0x7f66540b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6654072de0 0x7f66540b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 -- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072320 msgr2=0x7f66540ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072320 0x7f66540ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 -- 172.21.15.28:0/1585100463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66540af2d0 con 0x7f6654072700 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.694+0000 7f6652000640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f6640035040 tx=0x7f66400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.695+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640041030 con 0x7f6654072700 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.695+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664003d4e0 con 0x7f6654072700 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.695+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640037040 con 0x7f6654072700 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.695+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66540aeff0 con 0x7f6654072700 2024-09-10T10:56:09.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.695+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66540aa960 con 0x7f6654072700 2024-09-10T10:56:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.696+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66540cce60 con 0x7f6654072700 2024-09-10T10:56:09.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.697+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6640044070 con 0x7f6654072700 2024-09-10T10:56:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.698+0000 7f664b400640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 0x7f663406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.698+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f66400ab5b0 con 0x7f6654072700 2024-09-10T10:56:09.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.698+0000 7f6652a00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 0x7f663406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:09.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.699+0000 7f6652a00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 0x7f663406e2f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f66440011e0 tx=0x7f6644009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:09.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.702+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f66400796a0 con 0x7f6654072700 2024-09-10T10:56:09.823 INFO:tasks.ceph.mds.a.smithi028.stderr:2024-09-10T10:56:09.822+0000 7f317ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:56:09.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.924+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66540b1f20 con 0x7f6654072700 2024-09-10T10:56:09.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.925+0000 7f664b400640 1 -- 172.21.15.28:0/1585100463 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v347) ==== 53+0+73 (secure 0 0 0) 0x7f664007d9a0 con 0x7f6654072700 2024-09-10T10:56:09.926 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 msgr2=0x7f663406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 0x7f663406e2f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f66440011e0 tx=0x7f6644009040 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 msgr2=0x7f6654141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f6640035040 tx=0x7f66400046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 shutdown_connections 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f663406be30 0x7f663406e2f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6654072de0 0x7f66540b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6654072700 0x7f6654141e40 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 --2- 172.21.15.28:0/1585100463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6654072320 0x7f66540ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 >> 172.21.15.28:0/1585100463 conn(0x7f6654067d70 msgr2=0x7f6654144210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 shutdown_connections 2024-09-10T10:56:09.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:09.927+0000 7f6659e00640 1 -- 172.21.15.28:0/1585100463 wait complete. 2024-09-10T10:56:09.944 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T10:56:09.944 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T10:56:09.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-09-10T10:56:09.944 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T10:56:09.944 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:09.945 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T10:56:09.945 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T10:56:09.945 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T10:56:09.945 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:09.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:09.945 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-09-10T10:56:09.980 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T10:56:09.981 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 41935sec preferred_lft 41935sec 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ec86:c6ff:fe4c:e00f/64 scope link 2024-09-10T10:56:09.982 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether f6:1d:8d:16:49:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::f41d:8dff:fe16:4997/64 scope link 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 36:2a:a6:69:aa:7b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::342a:a6ff:fe69:aa7b/64 scope link 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-09-10T10:56:09.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:09.983 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T10:56:10.074 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-09-10T10:56:10.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:10.077 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-09-10T10:56:10.113 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-09-10T10:56:10.113 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T10:56:10.113 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T10:56:10.113 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T10:56:10.114 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T10:56:10.114 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-09-10T10:56:10.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T10:56:10.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:10.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:10.312 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:56:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:56:10.348 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:56:10.349 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:10.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:10.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-09-10T10:56:10.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:56:10.445 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:10.479 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T10:56:10.480 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T10:56:10.480 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:10.554 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-10T10:56:10.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:10.555 DEBUG:teuthology.orchestra.run.smithi028:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:10.684 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:10.684 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:10.684 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:10.699 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:10.699 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:10.699 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:10.699 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:56:10.700 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:56:10.721 INFO:teuthology.orchestra.run.smithi028.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-10T10:56:10.721 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-10T10:56:10.721 INFO:teuthology.orchestra.run.smithi028.stdout:invalid new device string format 2024-09-10T10:56:10.721 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:3300,172.21.15.138:3300,172.21.15.138:3301" 2024-09-10T10:56:10.721 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-10T10:56:10.722 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: resolved to: "172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790" 2024-09-10T10:56:10.722 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.28:6789,172.21.15.138:6789,172.21.15.138:6790:/ 2024-09-10T10:56:10.722 INFO:teuthology.orchestra.run.smithi028.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f658987a-dd14-4573-bcd3-ee2a383c9fe0" will pass to kernel 2024-09-10T10:56:10.723 INFO:teuthology.orchestra.run.smithi028.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T10:56:10.723 INFO:teuthology.orchestra.run.smithi028.stdout: You just mounted a file system that supports labels which does not 2024-09-10T10:56:10.723 INFO:teuthology.orchestra.run.smithi028.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T10:56:10.723 INFO:teuthology.orchestra.run.smithi028.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T10:56:10.723 INFO:teuthology.orchestra.run.smithi028.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T10:56:10.726 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T10:56:10.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:10.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:10.782 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T10:56:10.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T10:56:10.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T10:56:10.876 INFO:teuthology.orchestra.run.smithi028.stdout:client9730 2024-09-10T10:56:10.876 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:10.876 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:10.912 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-10T10:56:11.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.029+0000 7fd9f2444f40 1 Processor -- start 2024-09-10T10:56:11.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.029+0000 7fd9f2444f40 1 -- start start 2024-09-10T10:56:11.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8e81e70 0x5602c8f24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602c8f24a70 0x5602c8f29980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c8e9a530 con 0x7fd9d4002820 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602c8df7b30 con 0x5602c8f24a70 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5602c8f27ee0 con 0x5602c8e81e70 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42970/0 (socket says 172.21.15.28:42970) 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 -- 172.21.15.28:0/748803069 learned_addr learned my addr 172.21.15.28:0/748803069 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9eec00640 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8e81e70 0x5602c8f24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9f0000640 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602c8f24a70 0x5602c8f29980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 -- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8e81e70 msgr2=0x5602c8f24530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8e81e70 0x5602c8f24530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 -- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602c8f24a70 msgr2=0x5602c8f29980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:11.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602c8f24a70 0x5602c8f29980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.030+0000 7fd9ef600640 1 -- 172.21.15.28:0/748803069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c8f2bb40 con 0x7fd9d4002820 2024-09-10T10:56:11.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9ef600640 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009760 tx=0x7fd9e0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ab2b3648996f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9ed800640 1 -- 172.21.15.28:0/748803069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0008870 con 0x7fd9d4002820 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9ed800640 1 -- 172.21.15.28:0/748803069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e001a070 con 0x7fd9d4002820 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9ed800640 1 -- 172.21.15.28:0/748803069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0008c10 con 0x7fd9d4002820 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9f2444f40 1 -- 172.21.15.28:0/748803069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 msgr2=0x5602c8f23e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.031+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009760 tx=0x7fd9e0005a00 comp rx=0 tx=0).stop 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 -- 172.21.15.28:0/748803069 shutdown_connections 2024-09-10T10:56:11.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5602c8f24a70 0x5602c8f29980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8e81e70 0x5602c8f24530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/748803069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd9d4002820 0x5602c8f23e40 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 -- 172.21.15.28:0/748803069 >> 172.21.15.28:0/748803069 conn(0x5602c8e9e0a0 msgr2=0x5602c8e4e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 -- 172.21.15.28:0/748803069 shutdown_connections 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 -- 172.21.15.28:0/748803069 wait complete. 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 Processor -- start 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.032+0000 7fd9f2444f40 1 -- start start 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 0x5602c8f19ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8f24a70 0x5602c8f25850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4002820 0x5602c8f25d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c8e98070 con 0x5602c8e81e70 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5602c8f2bee0 con 0x7fd9d4002820 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9f2444f40 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5602c8f26860 con 0x5602c8f24a70 2024-09-10T10:56:11.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9ef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 0x5602c8f19ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.033+0000 7fd9ef600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 0x5602c8f19ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42978/0 (socket says 172.21.15.28:42978) 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 -- 172.21.15.28:0/3653221992 learned_addr learned my addr 172.21.15.28:0/3653221992 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9eec00640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8f24a70 0x5602c8f25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9f0000640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4002820 0x5602c8f25d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 -- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8f24a70 msgr2=0x5602c8f25850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8f24a70 0x5602c8f25850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 -- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4002820 msgr2=0x5602c8f25d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4002820 0x5602c8f25d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 -- 172.21.15.28:0/3653221992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c8f1a8a0 con 0x5602c8e81e70 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9eec00640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5602c8f24a70 0x5602c8f25850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9f0000640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9d4002820 0x5602c8f25d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:11.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.034+0000 7fd9ef600640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 0x5602c8f19ff0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009ce0 tx=0x7fd9e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.035+0000 7fd9e7e00640 1 -- 172.21.15.28:0/3653221992 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0028670 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.035+0000 7fd9e7e00640 1 -- 172.21.15.28:0/3653221992 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e0023070 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.035+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602c8e93280 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.035+0000 7fd9e7e00640 1 -- 172.21.15.28:0/3653221992 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0028a50 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.035+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602c8f1ab40 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.036+0000 7fd9e7e00640 1 -- 172.21.15.28:0/3653221992 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd9e0028bf0 con 0x5602c8e81e70 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.037+0000 7fd9e7e00640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9d006a810 0x7fd9d006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.037+0000 7fd9e7e00640 1 -- 172.21.15.28:0/3653221992 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fd9e008c090 con 0x5602c8e81e70 2024-09-10T10:56:11.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.038+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x5602c8e814d0 0x5602c8fca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.038+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e151) -- 0x5602c8e92e10 con 0x5602c8e814d0 2024-09-10T10:56:11.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.038+0000 7fd9eec00640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9d006a810 0x7fd9d006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.038+0000 7fd9f0000640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x5602c8e814d0 0x5602c8fca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.038+0000 7fd9eec00640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9d006a810 0x7fd9d006ccd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc056000 tx=0x7fd9dc050000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f0000640 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x5602c8e814d0 0x5602c8fca2e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f0000640 1 -- 172.21.15.28:0/3653221992 <== osd.4 v2:172.21.15.138:6810/1083898337 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (91) out=91b] v0'0 uv6 ondisk = 0) ==== 164+0+91 (crc 0 0 0) 0x7fd9e8010900 con 0x5602c8e814d0 2024-09-10T10:56:11.041 INFO:teuthology.orchestra.run.smithi028.stdout:Ufile1dir1 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x5602c8e814d0 msgr2=0x5602c8fca2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x5602c8e814d0 0x5602c8fca2e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9d006a810 msgr2=0x7fd9d006ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd9d006a810 0x7fd9d006ccd0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc056000 tx=0x7fd9dc050000 comp rx=0 tx=0).stop 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 msgr2=0x5602c8f19ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f2444f40 1 --2- 172.21.15.28:0/3653221992 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5602c8e81e70 0x5602c8f19ff0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009ce0 tx=0x7fd9e0002fb0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.039+0000 7fd9f0000640 1 -- 172.21.15.28:0/3653221992 reap_dead start 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.040+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 shutdown_connections 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.040+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 >> 172.21.15.28:0/3653221992 conn(0x5602c8e9e0a0 msgr2=0x5602c8e4f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.040+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 shutdown_connections 2024-09-10T10:56:11.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.040+0000 7fd9f2444f40 1 -- 172.21.15.28:0/3653221992 wait complete. 2024-09-10T10:56:11.045 DEBUG:teuthology.orchestra.run.smithi028:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-10T10:56:11.082 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:11.093 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:11.190 INFO:teuthology.orchestra.run.smithi028.stderr:error: stray data at end of buffer, offset 91 2024-09-10T10:56:11.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:56:11.202 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T10:56:11.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:56:11.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T10:56:11.403 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T10:56:11.403 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:56:11.403 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:11.403 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:11.438 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:11.439 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:11.514 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T10:56:11.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.710+0000 7f6f0b200640 1 Processor -- start 2024-09-10T10:56:11.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.710+0000 7f6f0b200640 1 -- start start 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.711+0000 7f6f0b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f040b47b0 0x7f6f0406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.711+0000 7f6f0b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f0409aa30 0x7f6f0409aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.711+0000 7f6f0b200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0406ddd0 con 0x7f6f041629e0 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.711+0000 7f6f0b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0409b3e0 con 0x7f6f040b47b0 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.711+0000 7f6f0b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0406e0b0 con 0x7f6f0409aa30 2024-09-10T10:56:11.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f040b47b0 0x7f6f0406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6f040b47b0 0x7f6f0406e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42986/0 (socket says 172.21.15.28:42986) 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 -- 172.21.15.28:0/3141550148 learned_addr learned my addr 172.21.15.28:0/3141550148 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f0a800640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f0409aa30 0x7f6f0409aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 -- 172.21.15.28:0/3141550148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f0409aa30 msgr2=0x7f6f0409aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f0409aa30 0x7f6f0409aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 -- 172.21.15.28:0/3141550148 >> v1:172.21.15.138:6789/0 conn(0x7f6f041629e0 legacy=0x7f6f04070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.712+0000 7f6f09400640 1 -- 172.21.15.28:0/3141550148 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f040d1b60 con 0x7f6f040b47b0 2024-09-10T10:56:11.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.713+0000 7f6f09400640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f040b47b0 0x7f6f0406e9d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009870 tx=0x7f6ef402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e6990196f7b9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.713+0000 7f6f08a00640 1 -- 172.21.15.28:0/3141550148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4037070 con 0x7f6f040b47b0 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f08a00640 1 -- 172.21.15.28:0/3141550148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ef4032440 con 0x7f6f040b47b0 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f08a00640 1 -- 172.21.15.28:0/3141550148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4032810 con 0x7f6f040b47b0 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 -- 172.21.15.28:0/3141550148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f040b47b0 msgr2=0x7f6f0406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f040b47b0 0x7f6f0406e9d0 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009870 tx=0x7f6ef402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 -- 172.21.15.28:0/3141550148 shutdown_connections 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f0409aa30 0x7f6f0409aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3141550148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f040b47b0 0x7f6f0406e9d0 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 -- 172.21.15.28:0/3141550148 >> 172.21.15.28:0/3141550148 conn(0x7f6f040b3c90 msgr2=0x7f6f040b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 -- 172.21.15.28:0/3141550148 shutdown_connections 2024-09-10T10:56:11.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.714+0000 7f6f0b200640 1 -- 172.21.15.28:0/3141550148 wait complete. 2024-09-10T10:56:11.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.716+0000 7f6f0b200640 1 Processor -- start 2024-09-10T10:56:11.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.716+0000 7f6f0b200640 1 -- start start 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 0x7f6f0415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f040b47b0 0x7f6f0415e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04162f40 con 0x7f6f041629e0 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04162dc0 con 0x7f6f0409aa30 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0b200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f040d0170 con 0x7f6f040b47b0 2024-09-10T10:56:11.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f09e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 0x7f6f0415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f09400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f040b47b0 0x7f6f0415e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42998/0 (socket says 172.21.15.28:42998) 2024-09-10T10:56:11.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 -- 172.21.15.28:0/3182908060 learned_addr learned my addr 172.21.15.28:0/3182908060 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:11.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f09e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 0x7f6f0415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.28:33118/0 (socket says 172.21.15.28:33118) 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 -- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f040b47b0 msgr2=0x7f6f0415e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f040b47b0 0x7f6f0415e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 -- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 msgr2=0x7f6f0415fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.717+0000 7f6f0a800640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 0x7f6f0415fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6f0a800640 1 -- 172.21.15.28:0/3182908060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f040c5e70 con 0x7f6f041629e0 2024-09-10T10:56:11.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6f0a800640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009dd0 tx=0x7f6f0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00018020 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f00014480 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f00014870 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f040c6150 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.718+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f040c6710 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.719+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0409aea0 con 0x7f6f041629e0 2024-09-10T10:56:11.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.720+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f00005d10 con 0x7f6f041629e0 2024-09-10T10:56:11.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.721+0000 7f6efe000640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 0x7f6ee806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:11.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.721+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f6f0008ba90 con 0x7f6f041629e0 2024-09-10T10:56:11.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.721+0000 7f6f09e00640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 0x7f6ee806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:11.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.722+0000 7f6f09e00640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 0x7f6ee806e2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f60 tx=0x7f6ef8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:11.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.725+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f00022070 con 0x7f6f041629e0 2024-09-10T10:56:11.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.928+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6f040c68e0 con 0x7f6f041629e0 2024-09-10T10:56:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.929+0000 7f6efe000640 1 -- 172.21.15.28:0/3182908060 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v151) ==== 80+0+120 (secure 0 0 0) 0x7f6f00010040 con 0x7f6f041629e0 2024-09-10T10:56:11.930 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6835/354907385 2024-09-11T10:56:03.806379+0000 2024-09-10T10:56:11.930 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.15.28:6833/354907385 2024-09-11T10:56:03.806379+0000 2024-09-10T10:56:11.930 INFO:teuthology.orchestra.run.smithi028.stderr:listed 2 entries 2024-09-10T10:56:11.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 msgr2=0x7f6ee806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 0x7f6ee806e2f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f60 tx=0x7f6ef8009040 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 msgr2=0x7f6f0409fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f6f00009dd0 tx=0x7f6f0000c490 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 shutdown_connections 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6ee806be30 0x7f6ee806e2f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f041629e0 0x7f6f0409fa80 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f040b47b0 0x7f6f0415e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 --2- 172.21.15.28:0/3182908060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f0409aa30 0x7f6f0415fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 >> 172.21.15.28:0/3182908060 conn(0x7f6f040b3c90 msgr2=0x7f6f040d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 shutdown_connections 2024-09-10T10:56:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:11.931+0000 7f6f0b200640 1 -- 172.21.15.28:0/3182908060 wait complete. 2024-09-10T10:56:11.949 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:56:11.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:11.949 DEBUG:teuthology.orchestra.run.smithi028:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:56:12.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:12.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:12.012 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:56:12.048 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:56:12.048 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:56:12.048 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:12.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:56:12.088 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:56:12.088 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:12.088 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:12.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:56:12.124 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T10:56:12.124 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:56:12.124 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:12.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:56:12.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:12.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.321+0000 7f6d2aa00640 1 Processor -- start 2024-09-10T10:56:12.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.321+0000 7f6d2aa00640 1 -- start start 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d2aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6d240d0dd0 0x7f6d240d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d2aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 0x7f6d240cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d2aa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d240cf000 con 0x7f6d2415e660 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d2aa00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2406d3e0 con 0x7f6d240d0dd0 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d2aa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d240b37d0 con 0x7f6d240d1f90 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d29600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6d240d0dd0 0x7f6d240d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d28c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6d2415e660 0x7f6d2406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35400/0 (socket says 172.21.15.28:35400) 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.322+0000 7f6d28c00640 1 -- 172.21.15.28:0/2074358237 learned_addr learned my addr 172.21.15.28:0/2074358237 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d2a000640 1 --2- 172.21.15.28:0/2074358237 >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 0x7f6d240cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d29600640 1 -- 172.21.15.28:0/2074358237 >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 msgr2=0x7f6d240cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d29600640 1 --2- 172.21.15.28:0/2074358237 >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 0x7f6d240cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d29600640 1 -- 172.21.15.28:0/2074358237 >> v1:172.21.15.28:6789/0 conn(0x7f6d2415e660 legacy=0x7f6d2406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d23e00640 1 -- 172.21.15.28:0/2074358237 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 348721007 0 0) 0x7f6d240cf000 con 0x7f6d2415e660 2024-09-10T10:56:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d29600640 1 -- 172.21.15.28:0/2074358237 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240d16f0 con 0x7f6d240d0dd0 2024-09-10T10:56:12.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.323+0000 7f6d2a000640 1 --2- 172.21.15.28:0/2074358237 >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 0x7f6d240cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.324+0000 7f6d29600640 1 --2- 172.21.15.28:0/2074358237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d0dd0 0x7f6d240d11b0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f6d2408cb90 tx=0x7f6d1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c86448eb4870d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.324+0000 7f6d23e00640 1 -- 172.21.15.28:0/2074358237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18037070 con 0x7f6d240d0dd0 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.324+0000 7f6d23e00640 1 -- 172.21.15.28:0/2074358237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d18032440 con 0x7f6d240d0dd0 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.324+0000 7f6d23e00640 1 -- 172.21.15.28:0/2074358237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18032810 con 0x7f6d240d0dd0 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 -- 172.21.15.28:0/2074358237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d0dd0 msgr2=0x7f6d240d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/2074358237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d0dd0 0x7f6d240d11b0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f6d2408cb90 tx=0x7f6d1802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 -- 172.21.15.28:0/2074358237 shutdown_connections 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/2074358237 >> v2:172.21.15.138:3300/0 conn(0x7f6d240d1f90 0x7f6d240cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/2074358237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d0dd0 0x7f6d240d11b0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 -- 172.21.15.28:0/2074358237 >> 172.21.15.28:0/2074358237 conn(0x7f6d2408bfc0 msgr2=0x7f6d240d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 -- 172.21.15.28:0/2074358237 shutdown_connections 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.325+0000 7f6d2aa00640 1 -- 172.21.15.28:0/2074358237 wait complete. 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.326+0000 7f6d2aa00640 1 Processor -- start 2024-09-10T10:56:12.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.327+0000 7f6d2aa00640 1 -- start start 2024-09-10T10:56:12.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.327+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d240d0dd0 0x7f6d240cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.327+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.327+0000 7f6d2aa00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d2415e660 0x7f6d240b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6d240d2c60 con 0x7f6d240d1f90 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d240cf330 con 0x7f6d240d0dd0 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d2aa00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d240d2f40 con 0x7f6d2415e660 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43008/0 (socket says 172.21.15.28:43008) 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 -- 172.21.15.28:0/3232167878 learned_addr learned my addr 172.21.15.28:0/3232167878 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d2a000640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d2415e660 0x7f6d240b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d29600640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d240d0dd0 0x7f6d240cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 -- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d2415e660 msgr2=0x7f6d240b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d2415e660 0x7f6d240b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 -- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d240d0dd0 msgr2=0x7f6d240cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d240d0dd0 0x7f6d240cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.328+0000 7f6d28c00640 1 -- 172.21.15.28:0/3232167878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240cd6f0 con 0x7f6d240d1f90 2024-09-10T10:56:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d28c00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c410 tx=0x7f6d1000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100074e0 con 0x7f6d240d1f90 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d100047e0 con 0x7f6d240d1f90 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004bd0 con 0x7f6d240d1f90 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2415d240 con 0x7f6d240d1f90 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.329+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d240cce50 con 0x7f6d240d1f90 2024-09-10T10:56:12.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.330+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d240d3220 con 0x7f6d240d1f90 2024-09-10T10:56:12.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.331+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d10011070 con 0x7f6d240d1f90 2024-09-10T10:56:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.332+0000 7f6d21600640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.332+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f6d1008aaf0 con 0x7f6d240d1f90 2024-09-10T10:56:12.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.333+0000 7f6d29600640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.333+0000 7f6d29600640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6d18034000 tx=0x7f6d1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.337+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d10058c60 con 0x7f6d240d1f90 2024-09-10T10:56:12.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.557+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d240d0530 con 0x7f6d240d1f90 2024-09-10T10:56:12.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.558+0000 7f6d21600640 1 -- 172.21.15.28:0/3232167878 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5244 (secure 0 0 0) 0x7f6d1005cf60 con 0x7f6d240d1f90 2024-09-10T10:56:12.559 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:12.559 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":347,"btime":"2024-09-10T10:56:07:825830+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:07.825823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10686,"qdb_cluster":[10686]},"id":7}]} 2024-09-10T10:56:12.560 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 347 2024-09-10T10:56:12.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 msgr2=0x7f6d0406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 0x7f6d0406e2f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6d18034000 tx=0x7f6d1802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 msgr2=0x7f6d2415cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000c410 tx=0x7f6d1000c8d0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 shutdown_connections 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6d0406be30 0x7f6d0406e2f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d2415e660 0x7f6d240b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6d240d1f90 0x7f6d2415cd00 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 --2- 172.21.15.28:0/3232167878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d240d0dd0 0x7f6d240cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 >> 172.21.15.28:0/3232167878 conn(0x7f6d2408bfc0 msgr2=0x7f6d2415d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 shutdown_connections 2024-09-10T10:56:12.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.560+0000 7f6d2aa00640 1 -- 172.21.15.28:0/3232167878 wait complete. 2024-09-10T10:56:12.577 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:12.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.732+0000 7fbe6a800640 1 Processor -- start 2024-09-10T10:56:12.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.732+0000 7fbe6a800640 1 -- start start 2024-09-10T10:56:12.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.733+0000 7fbe6a800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 0x7fbe640cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.733+0000 7fbe6a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe64072de0 0x7fbe6406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe6a800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe64146140 con 0x7fbe64072700 2024-09-10T10:56:12.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe6a800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbe640cd740 con 0x7fbe64072de0 2024-09-10T10:56:12.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe6a800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbe640cda20 con 0x7fbe64072320 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe5be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe64072de0 0x7fbe6406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe63e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbe64072700 0x7fbe6406e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35412/0 (socket says 172.21.15.28:35412) 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe63e00640 1 -- 172.21.15.28:0/3807736650 learned_addr learned my addr 172.21.15.28:0/3807736650 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe63400640 1 --2- 172.21.15.28:0/3807736650 >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 0x7fbe640cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe5be00640 1 -- 172.21.15.28:0/3807736650 >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 msgr2=0x7fbe640cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe5be00640 1 --2- 172.21.15.28:0/3807736650 >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 0x7fbe640cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe5be00640 1 -- 172.21.15.28:0/3807736650 >> v1:172.21.15.28:6789/0 conn(0x7fbe64072700 legacy=0x7fbe6406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe5be00640 1 -- 172.21.15.28:0/3807736650 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640cdd00 con 0x7fbe64072de0 2024-09-10T10:56:12.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.734+0000 7fbe63400640 1 --2- 172.21.15.28:0/3807736650 >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 0x7fbe640cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:12.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.735+0000 7fbe5be00640 1 --2- 172.21.15.28:0/3807736650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072de0 0x7fbe6406e0b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b31e0 tx=0x7fbe4c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8639e1b1eb3ba8ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe62a00640 1 -- 172.21.15.28:0/3807736650 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c037070 con 0x7fbe64072de0 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe62a00640 1 -- 172.21.15.28:0/3807736650 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c032440 con 0x7fbe64072de0 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe62a00640 1 -- 172.21.15.28:0/3807736650 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c02ae60 con 0x7fbe64072de0 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 -- 172.21.15.28:0/3807736650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072de0 msgr2=0x7fbe6406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 --2- 172.21.15.28:0/3807736650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072de0 0x7fbe6406e0b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b31e0 tx=0x7fbe4c02a110 comp rx=0 tx=0).stop 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 -- 172.21.15.28:0/3807736650 shutdown_connections 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 --2- 172.21.15.28:0/3807736650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072de0 0x7fbe6406e0b0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 --2- 172.21.15.28:0/3807736650 >> v2:172.21.15.138:3300/0 conn(0x7fbe64072320 0x7fbe640cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 -- 172.21.15.28:0/3807736650 >> 172.21.15.28:0/3807736650 conn(0x7fbe64067d70 msgr2=0x7fbe6406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 -- 172.21.15.28:0/3807736650 shutdown_connections 2024-09-10T10:56:12.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.736+0000 7fbe6a800640 1 -- 172.21.15.28:0/3807736650 wait complete. 2024-09-10T10:56:12.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.738+0000 7fbe6a800640 1 Processor -- start 2024-09-10T10:56:12.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.738+0000 7fbe6a800640 1 -- start start 2024-09-10T10:56:12.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe64072320 0x7fbe640b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64072de0 0x7fbe640ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640cd520 con 0x7fbe64072700 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640cd3a0 con 0x7fbe64072de0 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe6a800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6406ed00 con 0x7fbe64072320 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43024/0 (socket says 172.21.15.28:43024) 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 -- 172.21.15.28:0/1982141656 learned_addr learned my addr 172.21.15.28:0/1982141656 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe63e00640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64072de0 0x7fbe640ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe63400640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe64072320 0x7fbe640b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 -- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe64072320 msgr2=0x7fbe640b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe64072320 0x7fbe640b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 -- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64072de0 msgr2=0x7fbe640ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64072de0 0x7fbe640ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.739+0000 7fbe5be00640 1 -- 172.21.15.28:0/1982141656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe641f54f0 con 0x7fbe64072700 2024-09-10T10:56:12.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe5be00640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009890 tx=0x7fbe4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c037040 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe4c042070 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c030bd0 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe641f57d0 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.740+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641f5d90 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.741+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe640cce60 con 0x7fbe64072700 2024-09-10T10:56:12.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.742+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbe4c047460 con 0x7fbe64072700 2024-09-10T10:56:12.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.743+0000 7fbe5b400640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 0x7fbe4006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:12.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.743+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fbe4c0abc90 con 0x7fbe64072700 2024-09-10T10:56:12.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.743+0000 7fbe63400640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 0x7fbe4006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:12.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.744+0000 7fbe63400640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 0x7fbe4006e2f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a98c0 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:12.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.747+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe4c032e20 con 0x7fbe64072700 2024-09-10T10:56:12.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.975+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe6406bdb0 con 0x7fbe64072700 2024-09-10T10:56:12.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.976+0000 7fbe5b400640 1 -- 172.21.15.28:0/1982141656 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5244 (secure 0 0 0) 0x7fbe4c07a450 con 0x7fbe64072700 2024-09-10T10:56:12.977 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:12.977 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":347,"btime":"2024-09-10T10:56:07:825830+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:07.825823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10686,"qdb_cluster":[10686]},"id":7}]} 2024-09-10T10:56:12.977 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 347 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 msgr2=0x7fbe4006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 0x7fbe4006e2f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbe640a98c0 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 msgr2=0x7fbe640ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009890 tx=0x7fbe4c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 shutdown_connections 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fbe4006be30 0x7fbe4006e2f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe64072de0 0x7fbe640ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe64072700 0x7fbe640ac280 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 --2- 172.21.15.28:0/1982141656 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbe64072320 0x7fbe640b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 >> 172.21.15.28:0/1982141656 conn(0x7fbe64067d70 msgr2=0x7fbe640c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.978+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 shutdown_connections 2024-09-10T10:56:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:12.979+0000 7fbe6a800640 1 -- 172.21.15.28:0/1982141656 wait complete. 2024-09-10T10:56:12.996 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:56:13.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.150+0000 7f21f4600640 1 Processor -- start 2024-09-10T10:56:13.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.150+0000 7f21f4600640 1 -- start start 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072320 0x7f21ec0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f4600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21ec072de0 0x7f21ec06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f4600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21ec146140 con 0x7f21ec072700 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f4600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f21ec0cd740 con 0x7f21ec072de0 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec0cda20 con 0x7f21ec072320 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21ec072de0 0x7f21ec06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f2600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21ec072700 0x7f21ec06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35428/0 (socket says 172.21.15.28:35428) 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f2600640 1 -- 172.21.15.28:0/2939613556 learned_addr learned my addr 172.21.15.28:0/2939613556 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:13.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.152+0000 7f21f1c00640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072320 0x7f21ec0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.153+0000 7f21ebe00640 1 -- 172.21.15.28:0/2939613556 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175414804 0 0) 0x7f21ec146140 con 0x7f21ec072700 2024-09-10T10:56:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.153+0000 7f21f1200640 1 -- 172.21.15.28:0/2939613556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072320 msgr2=0x7f21ec0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.153+0000 7f21f1200640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072320 0x7f21ec0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.153+0000 7f21f1200640 1 -- 172.21.15.28:0/2939613556 >> v1:172.21.15.28:6789/0 conn(0x7f21ec072700 legacy=0x7f21ec06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.153+0000 7f21f1200640 1 -- 172.21.15.28:0/2939613556 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec0cdd00 con 0x7f21ec072de0 2024-09-10T10:56:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21f1200640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072de0 0x7f21ec06e0b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f21d80097a0 tx=0x7f21d802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f67f00283f7531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21ebe00640 1 -- 172.21.15.28:0/2939613556 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8037070 con 0x7f21ec072de0 2024-09-10T10:56:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21ebe00640 1 -- 172.21.15.28:0/2939613556 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21d802ae50 con 0x7f21ec072de0 2024-09-10T10:56:13.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21ebe00640 1 -- 172.21.15.28:0/2939613556 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d8032660 con 0x7f21ec072de0 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21f4600640 1 -- 172.21.15.28:0/2939613556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072de0 msgr2=0x7f21ec06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.154+0000 7f21f4600640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072de0 0x7f21ec06e0b0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7f21d80097a0 tx=0x7f21d802a100 comp rx=0 tx=0).stop 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 -- 172.21.15.28:0/2939613556 shutdown_connections 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072de0 0x7f21ec06e0b0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 --2- 172.21.15.28:0/2939613556 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072320 0x7f21ec0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 -- 172.21.15.28:0/2939613556 >> 172.21.15.28:0/2939613556 conn(0x7f21ec067d70 msgr2=0x7f21ec06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 -- 172.21.15.28:0/2939613556 shutdown_connections 2024-09-10T10:56:13.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.155+0000 7f21f4600640 1 -- 172.21.15.28:0/2939613556 wait complete. 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.156+0000 7f21f4600640 1 Processor -- start 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.156+0000 7f21f4600640 1 -- start start 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec072700 0x7f21ec15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072de0 0x7f21ec15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0cd520 con 0x7f21ec072320 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec0cd3a0 con 0x7f21ec072700 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.157+0000 7f21f4600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec06ed00 con 0x7f21ec072de0 2024-09-10T10:56:13.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43036/0 (socket says 172.21.15.28:43036) 2024-09-10T10:56:13.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 -- 172.21.15.28:0/1286464481 learned_addr learned my addr 172.21.15.28:0/1286464481 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:13.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f2600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072de0 0x7f21ec15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1200640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec072700 0x7f21ec15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 -- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072de0 msgr2=0x7f21ec15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072de0 0x7f21ec15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 -- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec072700 msgr2=0x7f21ec15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec072700 0x7f21ec15b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 -- 172.21.15.28:0/1286464481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec1f6070 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.158+0000 7f21f1c00640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e8a0 tx=0x7f21e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.159+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.159+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21e000c940 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.159+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000cd30 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.159+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec1348d0 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.159+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1f5590 con 0x7f21ec072320 2024-09-10T10:56:13.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.160+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21ec0cce60 con 0x7f21ec072320 2024-09-10T10:56:13.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.161+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21e001a070 con 0x7f21ec072320 2024-09-10T10:56:13.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.162+0000 7f21e9600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 0x7f21cc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.162+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f21e008aed0 con 0x7f21ec072320 2024-09-10T10:56:13.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.162+0000 7f21f1200640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 0x7f21cc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.163+0000 7f21f1200640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 0x7f21cc06e320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.166+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f21e0059040 con 0x7f21ec072320 2024-09-10T10:56:13.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.351+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21ec06bf90 con 0x7f21ec072320 2024-09-10T10:56:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.352+0000 7f21e9600640 1 -- 172.21.15.28:0/1286464481 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+17111 (secure 0 0 0) 0x7f21e005d340 con 0x7f21ec072320 2024-09-10T10:56:13.353 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:13.354 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":151,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:56:03.806396+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:55:46.065844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T10:55:47.256070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/354907385":"2024-09-11T10:56:03.806379+0000","172.21.15.28:6833/354907385":"2024-09-11T10:56:03.806379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 msgr2=0x7f21cc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 0x7f21cc06e320 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f21d8034000 tx=0x7f21d802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 msgr2=0x7f21ec1e4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f21e000e8a0 tx=0x7f21e000ed60 comp rx=0 tx=0).stop 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 shutdown_connections 2024-09-10T10:56:13.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f21cc06be60 0x7f21cc06e320 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f21ec072de0 0x7f21ec15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21ec072700 0x7f21ec15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 --2- 172.21.15.28:0/1286464481 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21ec072320 0x7f21ec1e4840 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 >> 172.21.15.28:0/1286464481 conn(0x7f21ec067d70 msgr2=0x7f21ec0c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 shutdown_connections 2024-09-10T10:56:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.355+0000 7f21f4600640 1 -- 172.21.15.28:0/1286464481 wait complete. 2024-09-10T10:56:13.371 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:56:13.372 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:13.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.526+0000 7f85c4000640 1 Processor -- start 2024-09-10T10:56:13.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.526+0000 7f85c4000640 1 -- start start 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 0x7f85bc15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c4000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85bc146140 con 0x7f85bc072320 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c4000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85bc0cd740 con 0x7f85bc072de0 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc0cda20 con 0x7f85bc072700 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c1600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f85bc072320 0x7f85bc0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35434/0 (socket says 172.21.15.28:35434) 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c1600640 1 -- 172.21.15.28:0/2152519110 learned_addr learned my addr 172.21.15.28:0/2152519110 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:13.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85c2000640 1 --2- 172.21.15.28:0/2152519110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 0x7f85bc15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3937780489 0 0) 0x7f85bc146140 con 0x7f85bc072320 2024-09-10T10:56:13.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.528+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85a4003520 con 0x7f85bc072320 2024-09-10T10:56:13.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.529+0000 7f85c2000640 1 -- 172.21.15.28:0/2152519110 >> v1:172.21.15.138:6789/0 conn(0x7f85bc072de0 legacy=0x7f85bc15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.529+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7944930 0 0) 0x7f85bc0cd740 con 0x7f85bc072de0 2024-09-10T10:56:13.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.529+0000 7f85c2000640 1 -- 172.21.15.28:0/2152519110 >> v1:172.21.15.28:6789/0 conn(0x7f85bc072320 legacy=0x7f85bc0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.529+0000 7f85c2000640 1 -- 172.21.15.28:0/2152519110 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc0cdd00 con 0x7f85bc072700 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.530+0000 7f85c2000640 1 --2- 172.21.15.28:0/2152519110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 0x7f85bc15d8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009dc0 tx=0x7f85b402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82c933bcc3db193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.530+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b402ae60 con 0x7f85bc072700 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.530+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b4007740 con 0x7f85bc072700 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.530+0000 7f85bbe00640 1 -- 172.21.15.28:0/2152519110 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b4007b10 con 0x7f85bc072700 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.530+0000 7f85c4000640 1 -- 172.21.15.28:0/2152519110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 msgr2=0x7f85bc15d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 --2- 172.21.15.28:0/2152519110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 0x7f85bc15d8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009dc0 tx=0x7f85b402a640 comp rx=0 tx=0).stop 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 -- 172.21.15.28:0/2152519110 shutdown_connections 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 --2- 172.21.15.28:0/2152519110 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072700 0x7f85bc15d8f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 -- 172.21.15.28:0/2152519110 >> 172.21.15.28:0/2152519110 conn(0x7f85bc067d70 msgr2=0x7f85bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:13.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 -- 172.21.15.28:0/2152519110 shutdown_connections 2024-09-10T10:56:13.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.531+0000 7f85c4000640 1 -- 172.21.15.28:0/2152519110 wait complete. 2024-09-10T10:56:13.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.532+0000 7f85c4000640 1 Processor -- start 2024-09-10T10:56:13.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.532+0000 7f85c4000640 1 -- start start 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 0x7f85bc0ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072de0 0x7f85bc0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc15dfb0 con 0x7f85bc072700 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc15de30 con 0x7f85bc072320 2024-09-10T10:56:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc15e130 con 0x7f85bc072de0 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.533+0000 7f85c0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:43042/0 (socket says 172.21.15.28:43042) 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 -- 172.21.15.28:0/758660700 learned_addr learned my addr 172.21.15.28:0/758660700 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c2000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072de0 0x7f85bc0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c1600640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 0x7f85bc0ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 -- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072de0 msgr2=0x7f85bc0ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072de0 0x7f85bc0ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 -- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 msgr2=0x7f85bc0ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 0x7f85bc0ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 -- 172.21.15.28:0/758660700 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc0a9610 con 0x7f85bc072700 2024-09-10T10:56:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c1600640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 0x7f85bc0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c0c00640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c440 tx=0x7f85a800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8017070 con 0x7f85bc072700 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a8005480 con 0x7f85bc072700 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8003830 con 0x7f85bc072700 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.534+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc0caab0 con 0x7f85bc072700 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.535+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc0cb070 con 0x7f85bc072700 2024-09-10T10:56:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.536+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc0a97e0 con 0x7f85bc072700 2024-09-10T10:56:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.537+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85a801a070 con 0x7f85bc072700 2024-09-10T10:56:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.537+0000 7f85b9600640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 0x7f859c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.538+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f85a808a2e0 con 0x7f85bc072700 2024-09-10T10:56:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.538+0000 7f85c1600640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 0x7f859c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.539+0000 7f85c1600640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 0x7f859c06e2f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005880 tx=0x7f85b0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.542+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f85a8058b40 con 0x7f85bc072700 2024-09-10T10:56:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.762+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85bc0cce60 con 0x7f85bc072700 2024-09-10T10:56:13.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.763+0000 7f85b9600640 1 -- 172.21.15.28:0/758660700 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5244 (secure 0 0 0) 0x7f85a8058440 con 0x7f85bc072700 2024-09-10T10:56:13.764 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:13.764 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":347,"btime":"2024-09-10T10:56:07:825830+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:07.825823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10686,"qdb_cluster":[10686]},"id":7}]} 2024-09-10T10:56:13.764 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 347 2024-09-10T10:56:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 msgr2=0x7f859c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 0x7f859c06e2f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005880 tx=0x7f85b0009040 comp rx=0 tx=0).stop 2024-09-10T10:56:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 msgr2=0x7f85bc15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f85a800c440 tx=0x7f85a800c900 comp rx=0 tx=0).stop 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 shutdown_connections 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f859c06be30 0x7f859c06e2f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85bc072de0 0x7f85bc0ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85bc072700 0x7f85bc15b8c0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 --2- 172.21.15.28:0/758660700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85bc072320 0x7f85bc0ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 >> 172.21.15.28:0/758660700 conn(0x7f85bc067d70 msgr2=0x7f85bc1449b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 shutdown_connections 2024-09-10T10:56:13.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.765+0000 7f85c4000640 1 -- 172.21.15.28:0/758660700 wait complete. 2024-09-10T10:56:13.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:13.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.936+0000 7f610ca00640 1 Processor -- start 2024-09-10T10:56:13.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.937+0000 7f610ca00640 1 -- start start 2024-09-10T10:56:13.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 0x7f61080cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61080cd3f0 0x7f610806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6108072d30 0x7f610806f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f61080728a0 con 0x7f61080cd3f0 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f61080cdbe0 con 0x7f610815b300 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f610ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61080cd900 con 0x7f6108072d30 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f60fd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61080cd3f0 0x7f610806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f60fd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61080cd3f0 0x7f610806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34748/0 (socket says 172.21.15.28:34748) 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f60fd400640 1 -- 172.21.15.28:0/1208524728 learned_addr learned my addr 172.21.15.28:0/1208524728 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f6105e00640 1 --2- 172.21.15.28:0/1208524728 >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 0x7f61080cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.938+0000 7f6106800640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6108072d30 0x7f610806f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 -- 172.21.15.28:0/1208524728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6108072d30 msgr2=0x7f610806f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6108072d30 0x7f610806f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 -- 172.21.15.28:0/1208524728 >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 msgr2=0x7f61080cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 --2- 172.21.15.28:0/1208524728 >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 0x7f61080cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 -- 172.21.15.28:0/1208524728 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6108072250 con 0x7f61080cd3f0 2024-09-10T10:56:13.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f6105e00640 1 --2- 172.21.15.28:0/1208524728 >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 0x7f61080cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.939+0000 7f60fd400640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61080cd3f0 0x7f610806f1c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f6108070250 tx=0x7f60f002c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234edb748f18b4b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f6105400640 1 -- 172.21.15.28:0/1208524728 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0038070 con 0x7f61080cd3f0 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f6105400640 1 -- 172.21.15.28:0/1208524728 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f0031710 con 0x7f61080cd3f0 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f6105400640 1 -- 172.21.15.28:0/1208524728 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0031ae0 con 0x7f61080cd3f0 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 -- 172.21.15.28:0/1208524728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61080cd3f0 msgr2=0x7f610806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61080cd3f0 0x7f610806f1c0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f6108070250 tx=0x7f60f002c3f0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 -- 172.21.15.28:0/1208524728 shutdown_connections 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6108072d30 0x7f610806f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 --2- 172.21.15.28:0/1208524728 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61080cd3f0 0x7f610806f1c0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.940+0000 7f610ca00640 1 --2- 172.21.15.28:0/1208524728 >> v2:172.21.15.138:3300/0 conn(0x7f610815b300 0x7f61080cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.941+0000 7f610ca00640 1 -- 172.21.15.28:0/1208524728 >> 172.21.15.28:0/1208524728 conn(0x7f6108067d70 msgr2=0x7f610806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.941+0000 7f610ca00640 1 -- 172.21.15.28:0/1208524728 shutdown_connections 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.941+0000 7f610ca00640 1 -- 172.21.15.28:0/1208524728 wait complete. 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.942+0000 7f610ca00640 1 Processor -- start 2024-09-10T10:56:13.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.942+0000 7f610ca00640 1 -- start start 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61080cd3f0 0x7f61081456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f610815b300 0x7f6108157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61080d0160 con 0x7f6108072d30 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61080cffe0 con 0x7f61080cd3f0 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f610ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f61080b3c40 con 0x7f610815b300 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f6105e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f6105e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34764/0 (socket says 172.21.15.28:34764) 2024-09-10T10:56:13.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.943+0000 7f6105e00640 1 -- 172.21.15.28:0/3609183427 learned_addr learned my addr 172.21.15.28:0/3609183427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f60fd400640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61080cd3f0 0x7f61081456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6106800640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f610815b300 0x7f6108157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 -- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f610815b300 msgr2=0x7f6108157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f610815b300 0x7f6108157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 -- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61080cd3f0 msgr2=0x7f61081456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61080cd3f0 0x7f61081456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:13.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 -- 172.21.15.28:0/3609183427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610815c7b0 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.944+0000 7f6105e00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f60f800d800 tx=0x7f60f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.945+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8004030 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.945+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f80041d0 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.945+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6108155e30 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.945+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80096a0 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.945+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081e4c70 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.946+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6108158c60 con 0x7f6108072d30 2024-09-10T10:56:13.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.947+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f60f8009840 con 0x7f6108072d30 2024-09-10T10:56:13.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.948+0000 7f60fea00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:13.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.948+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f60f808b270 con 0x7f6108072d30 2024-09-10T10:56:13.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.948+0000 7f60fd400640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:13.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.949+0000 7f60fd400640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 0x7f60e406e2f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f60f0035000 tx=0x7f60f0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:13.952+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f60f8059350 con 0x7f6108072d30 2024-09-10T10:56:14.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.174+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6108158980 con 0x7f6108072d30 2024-09-10T10:56:14.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.175+0000 7f60fea00640 1 -- 172.21.15.28:0/3609183427 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 347 v347) ==== 95+0+5244 (secure 0 0 0) 0x7f60f805d650 con 0x7f6108072d30 2024-09-10T10:56:14.176 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:14.176 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":347,"btime":"2024-09-10T10:56:07:825830+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336}],"filesystems":[{"mdsmap":{"epoch":347,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T10:55:48.269671+0000","modified":"2024-09-10T10:56:07.825823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":151,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":10686},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10686":{"gid":10686,"name":"a","rank":0,"incarnation":344,"state":"up:active","state_seq":4,"addr":"172.21.15.28:6835/2598694071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":2598694071},{"type":"v1","addr":"172.21.15.28:6835","nonce":2598694071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":10686,"qdb_cluster":[10686]},"id":7}]} 2024-09-10T10:56:14.177 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 347 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 msgr2=0x7f60e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 0x7f60e406e2f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f60f0035000 tx=0x7f60f0008030 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 msgr2=0x7f610815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f60f800d800 tx=0x7f60f800dcc0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 shutdown_connections 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f60e406be30 0x7f60e406e2f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f610815b300 0x7f6108157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61080cd3f0 0x7f61081456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 --2- 172.21.15.28:0/3609183427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6108072d30 0x7f610815c270 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.177+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 >> 172.21.15.28:0/3609183427 conn(0x7f6108067d70 msgr2=0x7f610806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:14.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.178+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 shutdown_connections 2024-09-10T10:56:14.180 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.178+0000 7f610ca00640 1 -- 172.21.15.28:0/3609183427 wait complete. 2024-09-10T10:56:14.195 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:56:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.349+0000 7fe3d2c00640 1 Processor -- start 2024-09-10T10:56:14.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.350+0000 7fe3d2c00640 1 -- start start 2024-09-10T10:56:14.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3d2c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3cc15b300 0x7fe3cc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3d2c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe3cc0cd3f0 0x7fe3cc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3d2c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3cc0cdbe0 con 0x7fe3cc072d30 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3d2c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe3cc0728a0 con 0x7fe3cc15b300 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3d2c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe3cc0cd900 con 0x7fe3cc0cd3f0 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3cbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3cc15b300 0x7fe3cc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3cbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3cc15b300 0x7fe3cc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34778/0 (socket says 172.21.15.28:34778) 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3cbe00640 1 -- 172.21.15.28:0/3282366971 learned_addr learned my addr 172.21.15.28:0/3282366971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.351+0000 7fe3c3400640 1 --2- 172.21.15.28:0/3282366971 >> v2:172.21.15.138:3300/0 conn(0x7fe3cc0cd3f0 0x7fe3cc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.352+0000 7fe3cbe00640 1 -- 172.21.15.28:0/3282366971 >> v2:172.21.15.138:3300/0 conn(0x7fe3cc0cd3f0 msgr2=0x7fe3cc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.352+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/3282366971 >> v2:172.21.15.138:3300/0 conn(0x7fe3cc0cd3f0 0x7fe3cc06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.352+0000 7fe3cbe00640 1 -- 172.21.15.28:0/3282366971 >> v1:172.21.15.138:6789/0 conn(0x7fe3cc072d30 legacy=0x7fe3cc0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:14.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.352+0000 7fe3cbe00640 1 -- 172.21.15.28:0/3282366971 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc072250 con 0x7fe3cc15b300 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/3282366971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc0cceb0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0b57c0 tx=0x7fe3bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a922b2d1258b5e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3cb400640 1 -- 172.21.15.28:0/3282366971 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc036070 con 0x7fe3cc15b300 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3cb400640 1 -- 172.21.15.28:0/3282366971 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc031600 con 0x7fe3cc15b300 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3cb400640 1 -- 172.21.15.28:0/3282366971 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0319d0 con 0x7fe3cc15b300 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 -- 172.21.15.28:0/3282366971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 msgr2=0x7fe3cc0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/3282366971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc0cceb0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0b57c0 tx=0x7fe3bc02a6f0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 -- 172.21.15.28:0/3282366971 shutdown_connections 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/3282366971 >> v2:172.21.15.138:3300/0 conn(0x7fe3cc0cd3f0 0x7fe3cc06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/3282366971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc0cceb0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.353+0000 7fe3d2c00640 1 -- 172.21.15.28:0/3282366971 >> 172.21.15.28:0/3282366971 conn(0x7fe3cc067d70 msgr2=0x7fe3cc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:14.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.354+0000 7fe3d2c00640 1 -- 172.21.15.28:0/3282366971 shutdown_connections 2024-09-10T10:56:14.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.354+0000 7fe3d2c00640 1 -- 172.21.15.28:0/3282366971 wait complete. 2024-09-10T10:56:14.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.355+0000 7fe3d2c00640 1 Processor -- start 2024-09-10T10:56:14.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.355+0000 7fe3d2c00640 1 -- start start 2024-09-10T10:56:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 0x7fe3cc15c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 0x7fe3cc1e3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc0d0160 con 0x7fe3cc15b300 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc0cffe0 con 0x7fe3cc0cd3f0 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d2c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe3cc06f700 con 0x7fe3cc072d30 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34782/0 (socket says 172.21.15.28:34782) 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.356+0000 7fe3d0c00640 1 -- 172.21.15.28:0/276460739 learned_addr learned my addr 172.21.15.28:0/276460739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3c3400640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 0x7fe3cc1e3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 0x7fe3cc15c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 -- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 msgr2=0x7fe3cc15c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 0x7fe3cc15c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 -- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 msgr2=0x7fe3cc1e3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 0x7fe3cc1e3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 -- 172.21.15.28:0/276460739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc1e7be0 con 0x7fe3cc15b300 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 0x7fe3cc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3c3400640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 0x7fe3cc1e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3d0c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c410 tx=0x7fe3c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011790 con 0x7fe3cc15b300 2024-09-10T10:56:14.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4017070 con 0x7fe3cc15b300 2024-09-10T10:56:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.357+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4011bb0 con 0x7fe3cc15b300 2024-09-10T10:56:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.358+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc15c8d0 con 0x7fe3cc15b300 2024-09-10T10:56:14.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.358+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc1e7ee0 con 0x7fe3cc15b300 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.359+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3cc06fac0 con 0x7fe3cc15b300 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.359+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe3c401c460 con 0x7fe3cc15b300 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.360+0000 7fe3c8c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 0x7fe3a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.360+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7fe3c408ac10 con 0x7fe3cc15b300 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.361+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 0x7fe3a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.361+0000 7fe3cbe00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 0x7fe3a806e2f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc033000 tx=0x7fe3bc009000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.365+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3c4021020 con 0x7fe3cc15b300 2024-09-10T10:56:14.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.553+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3cc06eec0 con 0x7fe3cc15b300 2024-09-10T10:56:14.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.554+0000 7fe3c8c00640 1 -- 172.21.15.28:0/276460739 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+17111 (secure 0 0 0) 0x7fe3c40593d0 con 0x7fe3cc15b300 2024-09-10T10:56:14.556 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:14.556 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":151,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:56:03.806396+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-10T10:55:46.065844+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-10T10:55:47.256070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/354907385":"2024-09-11T10:56:03.806379+0000","172.21.15.28:6833/354907385":"2024-09-11T10:56:03.806379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:56:14.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 msgr2=0x7fe3a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 0x7fe3a806e2f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc033000 tx=0x7fe3bc009000 comp rx=0 tx=0).stop 2024-09-10T10:56:14.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 msgr2=0x7fe3cc151910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7fe3c400c410 tx=0x7fe3c400c8d0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 shutdown_connections 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fe3a806be30 0x7fe3a806e2f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3cc15b300 0x7fe3cc151910 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3cc0cd3f0 0x7fe3cc1e3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 --2- 172.21.15.28:0/276460739 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3cc072d30 0x7fe3cc15c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 >> 172.21.15.28:0/276460739 conn(0x7fe3cc067d70 msgr2=0x7fe3cc0cab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 shutdown_connections 2024-09-10T10:56:14.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.557+0000 7fe3d2c00640 1 -- 172.21.15.28:0/276460739 wait complete. 2024-09-10T10:56:14.575 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T10:56:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.731+0000 7f4fafc00640 1 Processor -- start 2024-09-10T10:56:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.731+0000 7f4fafc00640 1 -- start start 2024-09-10T10:56:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.732+0000 7f4fafc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.732+0000 7f4fafc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4fa815e1f0 0x7f4fa806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fafc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fa8146140 con 0x7f4fa8072de0 2024-09-10T10:56:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fafc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa80cd740 con 0x7f4fa815e1f0 2024-09-10T10:56:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fafc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa80cda20 con 0x7f4fa8072320 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fad200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fad200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:54812/0 (socket says 172.21.15.28:54812) 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fad200640 1 -- 172.21.15.28:0/3560582293 learned_addr learned my addr 172.21.15.28:0/3560582293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.733+0000 7f4fadc00640 1 --2- 172.21.15.28:0/3560582293 >> v2:172.21.15.138:3300/0 conn(0x7f4fa815e1f0 0x7f4fa806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.734+0000 7f4fad200640 1 -- 172.21.15.28:0/3560582293 >> v2:172.21.15.138:3300/0 conn(0x7f4fa815e1f0 msgr2=0x7f4fa806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.734+0000 7f4fad200640 1 --2- 172.21.15.28:0/3560582293 >> v2:172.21.15.138:3300/0 conn(0x7f4fa815e1f0 0x7f4fa806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.734+0000 7f4fad200640 1 -- 172.21.15.28:0/3560582293 >> v1:172.21.15.138:6789/0 conn(0x7f4fa8072de0 legacy=0x7f4fa806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.734+0000 7f4fad200640 1 -- 172.21.15.28:0/3560582293 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa80cdd00 con 0x7f4fa8072320 2024-09-10T10:56:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fad200640 1 --2- 172.21.15.28:0/3560582293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc2a7ec123297d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fa7e00640 1 -- 172.21.15.28:0/3560582293 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98037070 con 0x7f4fa8072320 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fa7e00640 1 -- 172.21.15.28:0/3560582293 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f98032440 con 0x7f4fa8072320 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fa7e00640 1 -- 172.21.15.28:0/3560582293 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98032810 con 0x7f4fa8072320 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fafc00640 1 -- 172.21.15.28:0/3560582293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 msgr2=0x7f4fa80cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.735+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3560582293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009870 tx=0x7f4f9802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 -- 172.21.15.28:0/3560582293 shutdown_connections 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3560582293 >> v2:172.21.15.138:3300/0 conn(0x7f4fa815e1f0 0x7f4fa806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3560582293 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa8072320 0x7f4fa80cce60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 -- 172.21.15.28:0/3560582293 >> 172.21.15.28:0/3560582293 conn(0x7f4fa8067d70 msgr2=0x7f4fa806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 -- 172.21.15.28:0/3560582293 shutdown_connections 2024-09-10T10:56:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.736+0000 7f4fafc00640 1 -- 172.21.15.28:0/3560582293 wait complete. 2024-09-10T10:56:14.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.737+0000 7f4fafc00640 1 Processor -- start 2024-09-10T10:56:14.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.737+0000 7f4fafc00640 1 -- start start 2024-09-10T10:56:14.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 0x7f4fa815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa815e1f0 0x7f4fa80cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80cd520 con 0x7f4fa8072de0 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80cd3a0 con 0x7f4fa8072320 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4fafc00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa806ed00 con 0x7f4fa815e1f0 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.738+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34784/0 (socket says 172.21.15.28:34784) 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 -- 172.21.15.28:0/3611581987 learned_addr learned my addr 172.21.15.28:0/3611581987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4fad200640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 0x7f4fa815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4fadc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa815e1f0 0x7f4fa80cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 -- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa815e1f0 msgr2=0x7f4fa80cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa815e1f0 0x7f4fa80cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 -- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 msgr2=0x7f4fa815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 0x7f4fa815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 -- 172.21.15.28:0/3611581987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa8141ff0 con 0x7f4fa8072de0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4fad200640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 0x7f4fa815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.739+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c440 tx=0x7f4f90001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.740+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f900052d0 con 0x7f4fa8072de0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.740+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f90005470 con 0x7f4fa8072de0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.740+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa815c390 con 0x7f4fa8072de0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.740+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90007240 con 0x7f4fa8072de0 2024-09-10T10:56:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.740+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa8142570 con 0x7f4fa8072de0 2024-09-10T10:56:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.741+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa8072700 con 0x7f4fa8072de0 2024-09-10T10:56:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.742+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4f900073e0 con 0x7f4fa8072de0 2024-09-10T10:56:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.743+0000 7f4fa5600640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 0x7f4f8406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.743+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 6187+0+0 (secure 0 0 0) 0x7f4f9008a4c0 con 0x7f4fa8072de0 2024-09-10T10:56:14.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.743+0000 7f4fad200640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 0x7f4f8406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:14.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.744+0000 7f4fad200640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 0x7f4f8406e2f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4f98034000 tx=0x7f4f9802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:14.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.747+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f900585b0 con 0x7f4fa8072de0 2024-09-10T10:56:14.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.967+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4fa80cea50 con 0x7f4fa8072de0 2024-09-10T10:56:14.969 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-09-10T10:56:14.968+0000 7fcc5f400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:56:14.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.970+0000 7f4fa5600640 1 -- 172.21.15.28:0/3611581987 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v348) ==== 161+0+0 (secure 0 0 0) 0x7f4f9005c8b0 con 0x7f4fa8072de0 2024-09-10T10:56:14.971 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:56:14.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 msgr2=0x7f4f8406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 0x7f4f8406e2f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4f98034000 tx=0x7f4f9802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:14.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 msgr2=0x7f4fa80cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:14.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c440 tx=0x7f4f90001e70 comp rx=0 tx=0).stop 2024-09-10T10:56:14.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 shutdown_connections 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4f8406be30 0x7f4f8406e2f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4fa815e1f0 0x7f4fa80cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa8072de0 0x7f4fa80cbfb0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 --2- 172.21.15.28:0/3611581987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fa8072320 0x7f4fa815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 >> 172.21.15.28:0/3611581987 conn(0x7f4fa8067d70 msgr2=0x7f4fa80c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 shutdown_connections 2024-09-10T10:56:14.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:14.973+0000 7f4fafc00640 1 -- 172.21.15.28:0/3611581987 wait complete. 2024-09-10T10:56:14.979 INFO:tasks.ceph.mds.a.smithi028.stderr: -438> 2024-09-10T10:56:09.822+0000 7f317ca00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T10:56:14.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:56:15.021 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-09-10T10:56:15.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.150+0000 7f9427200640 1 Processor -- start 2024-09-10T10:56:15.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.151+0000 7f9427200640 1 -- start start 2024-09-10T10:56:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9427200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9420155930 0x7f94200addd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9427200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94200d0a80 0x7f942015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9427200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942015c1e0 con 0x7f942015b720 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9427200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f942015c4c0 con 0x7f9420155930 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9427200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f94200b3340 con 0x7f94200d0a80 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9420155930 0x7f94200addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9420155930 0x7f94200addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34810/0 (socket says 172.21.15.28:34810) 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 -- 172.21.15.28:0/42247495 learned_addr learned my addr 172.21.15.28:0/42247495 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f9425200640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94200d0a80 0x7f942015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 -- 172.21.15.28:0/42247495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94200d0a80 msgr2=0x7f942015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94200d0a80 0x7f942015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 -- 172.21.15.28:0/42247495 >> v1:172.21.15.138:6789/0 conn(0x7f942015b720 legacy=0x7f94200d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:15.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.152+0000 7f941fe00640 1 -- 172.21.15.28:0/42247495 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94200b3620 con 0x7f9420155930 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.153+0000 7f941fe00640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9420155930 0x7f94200addd0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9410009870 tx=0x7f941002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93e1f5df6058a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f941f400640 1 -- 172.21.15.28:0/42247495 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410037070 con 0x7f9420155930 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f941f400640 1 -- 172.21.15.28:0/42247495 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9410032440 con 0x7f9420155930 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f941f400640 1 -- 172.21.15.28:0/42247495 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410032810 con 0x7f9420155930 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 -- 172.21.15.28:0/42247495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9420155930 msgr2=0x7f94200addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9420155930 0x7f94200addd0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f9410009870 tx=0x7f941002a270 comp rx=0 tx=0).stop 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 -- 172.21.15.28:0/42247495 shutdown_connections 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94200d0a80 0x7f942015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 --2- 172.21.15.28:0/42247495 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9420155930 0x7f94200addd0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 -- 172.21.15.28:0/42247495 >> 172.21.15.28:0/42247495 conn(0x7f942006bdf0 msgr2=0x7f942006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:15.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 -- 172.21.15.28:0/42247495 shutdown_connections 2024-09-10T10:56:15.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.154+0000 7f9427200640 1 -- 172.21.15.28:0/42247495 wait complete. 2024-09-10T10:56:15.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.156+0000 7f9427200640 1 Processor -- start 2024-09-10T10:56:15.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.156+0000 7f9427200640 1 -- start start 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94200d0a80 0x7f9420153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9420155930 0x7f942009cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94200ae490 con 0x7f942015b720 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94200ae310 con 0x7f94200d0a80 2024-09-10T10:56:15.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9427200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f94200b3b40 con 0x7f9420155930 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9425200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9425200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34822/0 (socket says 172.21.15.28:34822) 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9425200640 1 -- 172.21.15.28:0/1203014780 learned_addr learned my addr 172.21.15.28:0/1203014780 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f9417400640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9420155930 0x7f942009cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.157+0000 7f941fe00640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94200d0a80 0x7f9420153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 -- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9420155930 msgr2=0x7f942009cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9420155930 0x7f942009cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 -- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94200d0a80 msgr2=0x7f9420153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94200d0a80 0x7f9420153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 -- 172.21.15.28:0/1203014780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94201543a0 con 0x7f942015b720 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9425200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f9418007e70 tx=0x7f941800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418014620 con 0x7f942015b720 2024-09-10T10:56:15.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9418019070 con 0x7f942015b720 2024-09-10T10:56:15.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418014a20 con 0x7f942015b720 2024-09-10T10:56:15.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942009d4f0 con 0x7f942015b720 2024-09-10T10:56:15.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.158+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942009dd90 con 0x7f942015b720 2024-09-10T10:56:15.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.160+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942009d6c0 con 0x7f942015b720 2024-09-10T10:56:15.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.160+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9418014bc0 con 0x7f942015b720 2024-09-10T10:56:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.161+0000 7f941cc00640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 0x7f93fc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:15.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.162+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6273+0+0 (secure 0 0 0) 0x7f941808b800 con 0x7f942015b720 2024-09-10T10:56:15.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.162+0000 7f941fe00640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 0x7f93fc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:15.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.163+0000 7f941fe00640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 0x7f93fc06e320 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9410034000 tx=0x7f941002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:15.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.166+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9418059920 con 0x7f942015b720 2024-09-10T10:56:15.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.386+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f942015bbb0 con 0x7f942015b720 2024-09-10T10:56:15.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:15.997+0000 7f941cc00640 1 -- 172.21.15.28:0/1203014780 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v349) ==== 104+0+0 (secure 0 0 0) 0x7f9418060140 con 0x7f942015b720 2024-09-10T10:56:16.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 msgr2=0x7f93fc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 0x7f93fc06e320 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9410034000 tx=0x7f941002e000 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 msgr2=0x7f9420072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f9418007e70 tx=0x7f941800e4b0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 shutdown_connections 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f93fc06be60 0x7f93fc06e320 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942015b720 0x7f9420072570 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9420155930 0x7f942009cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 --2- 172.21.15.28:0/1203014780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94200d0a80 0x7f9420153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 >> 172.21.15.28:0/1203014780 conn(0x7f942006bdf0 msgr2=0x7f94200b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.000+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 shutdown_connections 2024-09-10T10:56:16.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.001+0000 7f9427200640 1 -- 172.21.15.28:0/1203014780 wait complete. 2024-09-10T10:56:16.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:56:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.226+0000 7f444d000640 1 Processor -- start 2024-09-10T10:56:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.226+0000 7f444d000640 1 -- start start 2024-09-10T10:56:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f444d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4448161a90 0x7f44480ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f444d000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4448161e70 con 0x7f444806c7b0 2024-09-10T10:56:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f444d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44480b4270 con 0x7f44480b0120 2024-09-10T10:56:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f444d000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f44480b2770 con 0x7f4448161a90 2024-09-10T10:56:16.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f4445400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f444806c7b0 0x7f44480b5ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49082/0 (socket says 172.21.15.28:49082) 2024-09-10T10:56:16.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f4445e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4448161a90 0x7f44480ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:16.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.227+0000 7f4445400640 1 -- 172.21.15.28:0/3856636441 learned_addr learned my addr 172.21.15.28:0/3856636441 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.228+0000 7f4445e00640 1 -- 172.21.15.28:0/3856636441 >> v1:172.21.15.138:6789/0 conn(0x7f44480b0120 legacy=0x7f444806bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.228+0000 7f4445e00640 1 -- 172.21.15.28:0/3856636441 >> v1:172.21.15.28:6789/0 conn(0x7f444806c7b0 legacy=0x7f44480b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.228+0000 7f4445e00640 1 -- 172.21.15.28:0/3856636441 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44480b2a50 con 0x7f4448161a90 2024-09-10T10:56:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.229+0000 7f4445e00640 1 --2- 172.21.15.28:0/3856636441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f44480ae1b0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f44380097a0 tx=0x7f443802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7d8fed32f28101 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.229+0000 7f4444a00640 1 -- 172.21.15.28:0/3856636441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438037070 con 0x7f4448161a90 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.229+0000 7f4444a00640 1 -- 172.21.15.28:0/3856636441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4438032440 con 0x7f4448161a90 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.229+0000 7f4444a00640 1 -- 172.21.15.28:0/3856636441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438032810 con 0x7f4448161a90 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 -- 172.21.15.28:0/3856636441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 msgr2=0x7f44480ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 --2- 172.21.15.28:0/3856636441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f44480ae1b0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f44380097a0 tx=0x7f443802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 -- 172.21.15.28:0/3856636441 shutdown_connections 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 --2- 172.21.15.28:0/3856636441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f44480ae1b0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 -- 172.21.15.28:0/3856636441 >> 172.21.15.28:0/3856636441 conn(0x7f44480b66f0 msgr2=0x7f44480b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 -- 172.21.15.28:0/3856636441 shutdown_connections 2024-09-10T10:56:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.230+0000 7f444d000640 1 -- 172.21.15.28:0/3856636441 wait complete. 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.231+0000 7f444d000640 1 Processor -- start 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.231+0000 7f444d000640 1 -- start start 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806c7b0 0x7f444809b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 0x7f4448149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f444806c4a0 con 0x7f4448161a90 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f444806c320 con 0x7f444806c7b0 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f444d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44480b0800 con 0x7f44480b0120 2024-09-10T10:56:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.232+0000 7f4446800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34846/0 (socket says 172.21.15.28:34846) 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 -- 172.21.15.28:0/26707707 learned_addr learned my addr 172.21.15.28:0/26707707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4445e00640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806c7b0 0x7f444809b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4445400640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 0x7f4448149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 -- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 msgr2=0x7f4448149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 0x7f4448149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 -- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806c7b0 msgr2=0x7f444809b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806c7b0 0x7f444809b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 -- 172.21.15.28:0/26707707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44481536e0 con 0x7f4448161a90 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4445400640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 0x7f4448149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:16.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f4446800640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f444000c7a0 tx=0x7f444000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:16.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440003b70 con 0x7f4448161a90 2024-09-10T10:56:16.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4440003d30 con 0x7f4448161a90 2024-09-10T10:56:16.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444000aa90 con 0x7f4448161a90 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.233+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4448158480 con 0x7f4448161a90 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.234+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4448158a40 con 0x7f4448161a90 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.235+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44480b72a0 con 0x7f4448161a90 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.235+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f444000ac30 con 0x7f4448161a90 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.236+0000 7f443e000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 0x7f442806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:16.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.236+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 6273+0+0 (secure 0 0 0) 0x7f444008b060 con 0x7f4448161a90 2024-09-10T10:56:16.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.237+0000 7f4445e00640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 0x7f442806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:16.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.237+0000 7f4445e00640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 0x7f442806e2f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4438034000 tx=0x7f443802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:16.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.240+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f44400597c0 con 0x7f4448161a90 2024-09-10T10:56:16.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:16.422+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f44480b1b10 con 0x7f4448161a90 2024-09-10T10:56:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.007+0000 7f443e000640 1 -- 172.21.15.28:0/26707707 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) ==== 181+0+0 (secure 0 0 0) 0x7f44400590c0 con 0x7f4448161a90 2024-09-10T10:56:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:56:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 msgr2=0x7f442806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 0x7f442806e2f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f4438034000 tx=0x7f443802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 msgr2=0x7f4448156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f444000c7a0 tx=0x7f444000cc60 comp rx=0 tx=0).stop 2024-09-10T10:56:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 shutdown_connections 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f442806be30 0x7f442806e2f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4448161a90 0x7f4448156370 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f44480b0120 0x7f4448149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 --2- 172.21.15.28:0/26707707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f444806c7b0 0x7f444809b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 >> 172.21.15.28:0/26707707 conn(0x7f44480b66f0 msgr2=0x7f4448070000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.011+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 shutdown_connections 2024-09-10T10:56:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.012+0000 7f444d000640 1 -- 172.21.15.28:0/26707707 wait complete. 2024-09-10T10:56:17.033 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:56:17.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.192+0000 7f7137c00640 1 Processor -- start 2024-09-10T10:56:17.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.192+0000 7f7137c00640 1 -- start start 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f7137c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7130156740 0x7f71300b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f7137c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71300b30e0 con 0x7f71300b4af0 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f7137c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71300b33c0 con 0x7f71300cfea0 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f7137c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f713006edc0 con 0x7f7130156740 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f712fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f71300b4af0 0x7f71300cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49090/0 (socket says 172.21.15.28:49090) 2024-09-10T10:56:17.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f712fe00640 1 -- 172.21.15.28:0/252782904 learned_addr learned my addr 172.21.15.28:0/252782904 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:17.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f7135200640 1 --2- 172.21.15.28:0/252782904 >> v2:172.21.15.138:3300/0 conn(0x7f7130156740 0x7f71300b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:17.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890603053 0 0) 0x7f71300b30e0 con 0x7f71300b4af0 2024-09-10T10:56:17.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.194+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7118003520 con 0x7f71300b4af0 2024-09-10T10:56:17.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.195+0000 7f7135200640 1 -- 172.21.15.28:0/252782904 >> v1:172.21.15.138:6789/0 conn(0x7f71300cfea0 legacy=0x7f71300ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.195+0000 7f7135200640 1 -- 172.21.15.28:0/252782904 >> v1:172.21.15.28:6789/0 conn(0x7f71300b4af0 legacy=0x7f71300cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.195+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 513827023 0 0) 0x7f7118003520 con 0x7f71300b4af0 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.195+0000 7f7135200640 1 -- 172.21.15.28:0/252782904 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71300b36a0 con 0x7f7130156740 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.196+0000 7f7135200640 1 --2- 172.21.15.28:0/252782904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7130156740 0x7f71300b45b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f71240097a0 tx=0x7f712402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9aeb344123e694f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.196+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7124037070 con 0x7f7130156740 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.196+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f712402ac50 con 0x7f7130156740 2024-09-10T10:56:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.196+0000 7f712f400640 1 -- 172.21.15.28:0/252782904 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c001670 con 0x7f7130156740 2024-09-10T10:56:17.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 -- 172.21.15.28:0/252782904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7130156740 msgr2=0x7f71300b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 --2- 172.21.15.28:0/252782904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7130156740 0x7f71300b45b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f71240097a0 tx=0x7f712402a1c0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 -- 172.21.15.28:0/252782904 shutdown_connections 2024-09-10T10:56:17.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 --2- 172.21.15.28:0/252782904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7130156740 0x7f71300b45b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 -- 172.21.15.28:0/252782904 >> 172.21.15.28:0/252782904 conn(0x7f713006bdf0 msgr2=0x7f713006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:17.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 -- 172.21.15.28:0/252782904 shutdown_connections 2024-09-10T10:56:17.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.197+0000 7f7137c00640 1 -- 172.21.15.28:0/252782904 wait complete. 2024-09-10T10:56:17.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.198+0000 7f7137c00640 1 Processor -- start 2024-09-10T10:56:17.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.199+0000 7f7137c00640 1 -- start start 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.199+0000 7f7137c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71300b4af0 0x7f713014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.199+0000 7f7137c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7137c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7130156740 0x7f71300b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7137c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71300d0890 con 0x7f71300cfea0 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7137c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71300b3c60 con 0x7f71300b4af0 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7137c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f71300d0a10 con 0x7f7130156740 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34860/0 (socket says 172.21.15.28:34860) 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 -- 172.21.15.28:0/1361486808 learned_addr learned my addr 172.21.15.28:0/1361486808 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7135200640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71300b4af0 0x7f713014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:17.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f7135c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7130156740 0x7f71300b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 -- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7130156740 msgr2=0x7f71300b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7130156740 0x7f71300b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 -- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71300b4af0 msgr2=0x7f713014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71300b4af0 0x7f713014f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 -- 172.21.15.28:0/1361486808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713015edf0 con 0x7f71300cfea0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.200+0000 7f712fe00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c750 tx=0x7f711c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.201+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c017070 con 0x7f71300cfea0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.201+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711c005ad0 con 0x7f71300cfea0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.201+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713015f0d0 con 0x7f71300cfea0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.201+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c007070 con 0x7f71300cfea0 2024-09-10T10:56:17.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.201+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713015f690 con 0x7f71300cfea0 2024-09-10T10:56:17.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.202+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130068f50 con 0x7f71300cfea0 2024-09-10T10:56:17.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.203+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f711c01a070 con 0x7f71300cfea0 2024-09-10T10:56:17.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.204+0000 7f712cc00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:17.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.204+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5835+0+0 (secure 0 0 0) 0x7f711c08a4d0 con 0x7f71300cfea0 2024-09-10T10:56:17.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.204+0000 7f7135200640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:17.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.205+0000 7f7135200640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 0x7f711006e2f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:17.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.208+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f711c058d60 con 0x7f71300cfea0 2024-09-10T10:56:17.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:17.400+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7130156c40 con 0x7f71300cfea0 2024-09-10T10:56:18.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.009+0000 7f712cc00640 1 -- 172.21.15.28:0/1361486808 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) ==== 169+0+0 (secure 0 0 0) 0x7f711c058660 con 0x7f71300cfea0 2024-09-10T10:56:18.011 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-09-10T10:56:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 msgr2=0x7f711006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 0x7f711006e2f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7124034000 tx=0x7f712402e000 comp rx=0 tx=0).stop 2024-09-10T10:56:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 msgr2=0x7f713015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7f711c00c750 tx=0x7f711c00cc10 comp rx=0 tx=0).stop 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 shutdown_connections 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7130156740 0x7f71300b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71300cfea0 0x7f713015e370 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 --2- 172.21.15.28:0/1361486808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71300b4af0 0x7f713014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.013+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 >> 172.21.15.28:0/1361486808 conn(0x7f713006bdf0 msgr2=0x7f71300ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:18.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.014+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 shutdown_connections 2024-09-10T10:56:18.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.014+0000 7f7137c00640 1 -- 172.21.15.28:0/1361486808 wait complete. 2024-09-10T10:56:18.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-10T10:56:18.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.194+0000 7f1021c00640 1 Processor -- start 2024-09-10T10:56:18.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.194+0000 7f1021c00640 1 -- start start 2024-09-10T10:56:18.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f101c15e630 0x7f101c0cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f101c0d2230 0x7f101c0d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f101c0d00c0 con 0x7f101c0d4cd0 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f101c0d03a0 con 0x7f101c0d2230 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f101c06ce20 con 0x7f101c15e630 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f101b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f101c0d4cd0 0x7f101c06d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49100/0 (socket says 172.21.15.28:49100) 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f101b400640 1 -- 172.21.15.28:0/2795220997 learned_addr learned my addr 172.21.15.28:0/2795220997 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.195+0000 7f1021200640 1 --2- 172.21.15.28:0/2795220997 >> v2:172.21.15.28:3300/0 conn(0x7f101c0d2230 0x7f101c0d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f101be00640 1 --2- 172.21.15.28:0/2795220997 >> v2:172.21.15.138:3300/0 conn(0x7f101c15e630 0x7f101c0cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f1021200640 1 -- 172.21.15.28:0/2795220997 >> v2:172.21.15.138:3300/0 conn(0x7f101c15e630 msgr2=0x7f101c0cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:18.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f1021200640 1 --2- 172.21.15.28:0/2795220997 >> v2:172.21.15.138:3300/0 conn(0x7f101c15e630 0x7f101c0cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f1021200640 1 -- 172.21.15.28:0/2795220997 >> v1:172.21.15.28:6789/0 conn(0x7f101c0d4cd0 legacy=0x7f101c06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f1021200640 1 -- 172.21.15.28:0/2795220997 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c06d100 con 0x7f101c0d2230 2024-09-10T10:56:18.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.196+0000 7f101aa00640 1 -- 172.21.15.28:0/2795220997 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476403270 0 0) 0x7f101c0d00c0 con 0x7f101c0d4cd0 2024-09-10T10:56:18.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.197+0000 7f1021200640 1 --2- 172.21.15.28:0/2795220997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c0d2230 0x7f101c0d2680 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f101c08e550 tx=0x7f101402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93375200dcf699d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.197+0000 7f101aa00640 1 -- 172.21.15.28:0/2795220997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014007740 con 0x7f101c0d2230 2024-09-10T10:56:18.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.197+0000 7f101aa00640 1 -- 172.21.15.28:0/2795220997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10140078e0 con 0x7f101c0d2230 2024-09-10T10:56:18.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.197+0000 7f101aa00640 1 -- 172.21.15.28:0/2795220997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014007cb0 con 0x7f101c0d2230 2024-09-10T10:56:18.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 -- 172.21.15.28:0/2795220997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c0d2230 msgr2=0x7f101c0d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 --2- 172.21.15.28:0/2795220997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c0d2230 0x7f101c0d2680 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f101c08e550 tx=0x7f101402a5f0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 -- 172.21.15.28:0/2795220997 shutdown_connections 2024-09-10T10:56:18.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 --2- 172.21.15.28:0/2795220997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c0d2230 0x7f101c0d2680 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 --2- 172.21.15.28:0/2795220997 >> v2:172.21.15.138:3300/0 conn(0x7f101c15e630 0x7f101c0cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 -- 172.21.15.28:0/2795220997 >> 172.21.15.28:0/2795220997 conn(0x7f101c08bfc0 msgr2=0x7f101c0d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:18.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 -- 172.21.15.28:0/2795220997 shutdown_connections 2024-09-10T10:56:18.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.198+0000 7f1021c00640 1 -- 172.21.15.28:0/2795220997 wait complete. 2024-09-10T10:56:18.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.200+0000 7f1021c00640 1 Processor -- start 2024-09-10T10:56:18.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.200+0000 7f1021c00640 1 -- start start 2024-09-10T10:56:18.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101c0d2230 0x7f101c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 0x7f101c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f101c15eb90 con 0x7f101c15e630 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f101c15ea10 con 0x7f101c0d4cd0 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f101c0d5140 con 0x7f101c0d2230 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34886/0 (socket says 172.21.15.28:34886) 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 -- 172.21.15.28:0/1525630746 learned_addr learned my addr 172.21.15.28:0/1525630746 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f101b400640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 0x7f101c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f101be00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101c0d2230 0x7f101c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 -- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101c0d2230 msgr2=0x7f101c152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101c0d2230 0x7f101c152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 -- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 msgr2=0x7f101c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 0x7f101c0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f1021200640 1 -- 172.21.15.28:0/1525630746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c161250 con 0x7f101c15e630 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.201+0000 7f101b400640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 0x7f101c0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1021200640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f101402abe0 tx=0x7f1014004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014040070 con 0x7f101c15e630 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1014031bc0 con 0x7f101c15e630 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c1609b0 con 0x7f101c15e630 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1014009450 con 0x7f101c15e630 2024-09-10T10:56:18.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.202+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c160f70 con 0x7f101c15e630 2024-09-10T10:56:18.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.203+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c0cfa90 con 0x7f101c15e630 2024-09-10T10:56:18.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.204+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1014039040 con 0x7f101c15e630 2024-09-10T10:56:18.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.205+0000 7f1013e00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 0x7f0ffc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.205+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f10140ac580 con 0x7f101c15e630 2024-09-10T10:56:18.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.206+0000 7f101be00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 0x7f0ffc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.206+0000 7f101be00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 0x7f0ffc06e320 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f101c06e2b0 tx=0x7f100c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.209+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f101407b040 con 0x7f101c15e630 2024-09-10T10:56:18.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.406+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f101c0b3a70 con 0x7f101c15e630 2024-09-10T10:56:18.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.410+0000 7f1013e00640 1 -- 172.21.15.28:0/1525630746 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f101407a940 con 0x7f101c15e630 2024-09-10T10:56:18.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 msgr2=0x7f0ffc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 0x7f0ffc06e320 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f101c06e2b0 tx=0x7f100c008040 comp rx=0 tx=0).stop 2024-09-10T10:56:18.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 msgr2=0x7f101c0b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f101402abe0 tx=0x7f1014004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 shutdown_connections 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f0ffc06be60 0x7f0ffc06e320 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101c15e630 0x7f101c0b67b0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f101c0d4cd0 0x7f101c0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 --2- 172.21.15.28:0/1525630746 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f101c0d2230 0x7f101c152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 >> 172.21.15.28:0/1525630746 conn(0x7f101c08bfc0 msgr2=0x7f101c0d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 shutdown_connections 2024-09-10T10:56:18.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.412+0000 7f1021c00640 1 -- 172.21.15.28:0/1525630746 wait complete. 2024-09-10T10:56:18.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors' 2024-09-10T10:56:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.583+0000 7fa717200640 1 Processor -- start 2024-09-10T10:56:18.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.583+0000 7fa717200640 1 -- start start 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d0dd0 0x7fa7100d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa71015e660 0x7fa71006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7100d1f90 0x7fa7100cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa7100b37d0 con 0x7fa7100d1f90 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa7100cf330 con 0x7fa71015e660 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.584+0000 7fa717200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100cf000 con 0x7fa7100d0dd0 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7100d1f90 0x7fa7100cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa7100d1f90 0x7fa7100cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34890/0 (socket says 172.21.15.28:34890) 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 -- 172.21.15.28:0/4115958396 learned_addr learned my addr 172.21.15.28:0/4115958396 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa715e00640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d0dd0 0x7fa7100d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa715400640 1 --2- 172.21.15.28:0/4115958396 >> v2:172.21.15.138:3300/0 conn(0x7fa71015e660 0x7fa71006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 -- 172.21.15.28:0/4115958396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d0dd0 msgr2=0x7fa7100d11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d0dd0 0x7fa7100d11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 -- 172.21.15.28:0/4115958396 >> v2:172.21.15.138:3300/0 conn(0x7fa71015e660 msgr2=0x7fa71006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 --2- 172.21.15.28:0/4115958396 >> v2:172.21.15.138:3300/0 conn(0x7fa71015e660 0x7fa71006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.585+0000 7fa716800640 1 -- 172.21.15.28:0/4115958396 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100d16f0 con 0x7fa7100d1f90 2024-09-10T10:56:18.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.586+0000 7fa716800640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d1f90 0x7fa7100cf7c0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b3dd0 tx=0x7fa70c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c762732903887b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.586+0000 7fa714a00640 1 -- 172.21.15.28:0/4115958396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c036070 con 0x7fa7100d1f90 2024-09-10T10:56:18.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.586+0000 7fa714a00640 1 -- 172.21.15.28:0/4115958396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c031600 con 0x7fa7100d1f90 2024-09-10T10:56:18.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.586+0000 7fa714a00640 1 -- 172.21.15.28:0/4115958396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0319d0 con 0x7fa7100d1f90 2024-09-10T10:56:18.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 -- 172.21.15.28:0/4115958396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d1f90 msgr2=0x7fa7100cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d1f90 0x7fa7100cf7c0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b3dd0 tx=0x7fa70c02a6a0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 -- 172.21.15.28:0/4115958396 shutdown_connections 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d1f90 0x7fa7100cf7c0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 --2- 172.21.15.28:0/4115958396 >> v2:172.21.15.138:3300/0 conn(0x7fa71015e660 0x7fa71006ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 --2- 172.21.15.28:0/4115958396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d0dd0 0x7fa7100d11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 -- 172.21.15.28:0/4115958396 >> 172.21.15.28:0/4115958396 conn(0x7fa71008bfc0 msgr2=0x7fa7100d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 -- 172.21.15.28:0/4115958396 shutdown_connections 2024-09-10T10:56:18.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.587+0000 7fa717200640 1 -- 172.21.15.28:0/4115958396 wait complete. 2024-09-10T10:56:18.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.589+0000 7fa717200640 1 Processor -- start 2024-09-10T10:56:18.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.589+0000 7fa717200640 1 -- start start 2024-09-10T10:56:18.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 0x7fa7100b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 0x7fa7100b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100d2f40 con 0x7fa7100d0dd0 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7100d2c60 con 0x7fa71015e660 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa717200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa7100d3220 con 0x7fa7100d1f90 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34900/0 (socket says 172.21.15.28:34900) 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 -- 172.21.15.28:0/1469128342 learned_addr learned my addr 172.21.15.28:0/1469128342 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:18.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 -- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 msgr2=0x7fa7100b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa716800640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 0x7fa7100b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715400640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 0x7fa7100b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 0x7fa7100b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 -- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 msgr2=0x7fa7100b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 0x7fa7100b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715e00640 1 -- 172.21.15.28:0/1469128342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100b6c90 con 0x7fa7100d0dd0 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa716800640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 0x7fa7100b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:18.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.590+0000 7fa715400640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 0x7fa7100b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa715e00640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d800 tx=0x7fa70400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704018020 con 0x7fa7100d0dd0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7040096a0 con 0x7fa7100d0dd0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704009a70 con 0x7fa7100d0dd0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100cc3e0 con 0x7fa7100d0dd0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.591+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100cc9a0 con 0x7fa7100d0dd0 2024-09-10T10:56:18.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.592+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7100d2440 con 0x7fa7100d0dd0 2024-09-10T10:56:18.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.593+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa70401b020 con 0x7fa7100d0dd0 2024-09-10T10:56:18.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.594+0000 7fa70a000640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:18.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.594+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fa70408b060 con 0x7fa7100d0dd0 2024-09-10T10:56:18.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.594+0000 7fa715400640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:18.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.595+0000 7fa715400640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 0x7fa6f406e2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa7101581a0 tx=0x7fa700008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:18.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.598+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa704022030 con 0x7fa7100d0dd0 2024-09-10T10:56:18.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:18.848+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]} v 0) -- 0x7fa7100d1b20 con 0x7fa7100d0dd0 2024-09-10T10:56:19.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.019+0000 7fa70a000640 1 -- 172.21.15.28:0/1469128342 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_damage.TestDamage.test_open_ino_errors"]}]=0 v646) ==== 135+0+0 (secure 0 0 0) 0x7fa704059a90 con 0x7fa7100d0dd0 2024-09-10T10:56:19.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 msgr2=0x7fa6f406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 0x7fa6f406e2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fa7101581a0 tx=0x7fa700008040 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 msgr2=0x7fa7100b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d800 tx=0x7fa70400dcc0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 shutdown_connections 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa6f406be30 0x7fa6f406e2f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa71015e660 0x7fa7100b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa7100d1f90 0x7fa7100b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 --2- 172.21.15.28:0/1469128342 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7100d0dd0 0x7fa7100b6750 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.021+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 >> 172.21.15.28:0/1469128342 conn(0x7fa71008bfc0 msgr2=0x7fa7100b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.022+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 shutdown_connections 2024-09-10T10:56:19.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:19.022+0000 7fa717200640 1 -- 172.21.15.28:0/1469128342 wait complete. 2024-09-10T10:56:19.041 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated ... ERROR 2024-09-10T10:56:19.041 INFO:tasks.cephfs_test_runner: 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner:====================================================================== 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner:ERROR: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/test_damage.py", line 510, in test_open_ino_errors 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: self.fs.read_backtrace(file1_ino) 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1508, in read_backtrace 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: return self._read_data_xattr(ino_no, "parent", "inode_backtrace_t", pool) 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1464, in _read_data_xattr 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: return json.loads(self.dencoder(obj_type, obj_blob).strip()) 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1267, in dencoder 2024-09-10T10:56:19.042 INFO:tasks.cephfs_test_runner: p = self.mon_manager.controller.run(args=args, stdin=BytesIO(obj_blob), stdout=BytesIO()) 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: r.wait() 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi028 with status 1: 'ceph-dencoder type inode_backtrace_t import - decode dump_json' 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner:Ran 6 tests in 710.386s 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner: 2024-09-10T10:56:19.043 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner:====================================================================== 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner:ERROR: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner:That errors encountered during opening inos are properly propagated 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/test_damage.py", line 510, in test_open_ino_errors 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: self.fs.read_backtrace(file1_ino) 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1508, in read_backtrace 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: return self._read_data_xattr(ino_no, "parent", "inode_backtrace_t", pool) 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1464, in _read_data_xattr 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: return json.loads(self.dencoder(obj_type, obj_blob).strip()) 2024-09-10T10:56:19.044 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/filesystem.py", line 1267, in dencoder 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: p = self.mon_manager.controller.run(args=args, stdin=BytesIO(obj_blob), stdout=BytesIO()) 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: r.wait() 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi028 with status 1: 'ceph-dencoder type inode_backtrace_t import - decode dump_json' 2024-09-10T10:56:19.045 INFO:tasks.cephfs_test_runner: 2024-09-10T10:56:19.045 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:56:19.798 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4af3294f138848799d961e6845529271 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage) 2024-09-10T10:56:19.800 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T10:56:19.812 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:56:19.885 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:56:19.886 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:19.886 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:19.928 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:56:19.929 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-09-10T10:56:19.929 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:20.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T10:56:20.007 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:56:20.007 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:20.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:56:20.048 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-10T10:56:20.048 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:56:20.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T10:56:20.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.126 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:56:20.165 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:56:20.165 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T10:56:20.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.167 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:56:20.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.268 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T10:56:20.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-10T10:56:20.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.834 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-09-10T10:56:20.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.870 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-09-10T10:56:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-09-10T10:56:20.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:56:20.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:20.972 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:56:21.013 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T10:56:21.013 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T10:56:21.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:21.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T10:56:21.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:21.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T10:56:21.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:21.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-09-10T10:56:21.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:21.653 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-09-10T10:56:21.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:56:21.689 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-09-10T10:56:21.788 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-09-10T10:56:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:56:21.791 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T10:56:21.800 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T10:56:21.801 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:56:21.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.966+0000 7ff5c2200640 1 Processor -- start 2024-09-10T10:56:21.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.966+0000 7ff5c2200640 1 -- start start 2024-09-10T10:56:21.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.967+0000 7ff5c2200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 0x7ff5bc165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.967+0000 7ff5c2200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0cdbe0 con 0x7ff5bc072d30 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.967+0000 7ff5c2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5bc0cd900 con 0x7ff5bc15b300 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.967+0000 7ff5c2200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0728a0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.967+0000 7ff5baa00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff5bc072d30 0x7ff5bc06f1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49106/0 (socket says 172.21.15.28:49106) 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5baa00640 1 -- 172.21.15.28:0/2241932147 learned_addr learned my addr 172.21.15.28:0/2241932147 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:21.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5bbe00640 1 --2- 172.21.15.28:0/2241932147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 0x7ff5bc165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:21.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342394499 0 0) 0x7ff5bc0cdbe0 con 0x7ff5bc072d30 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5a0003520 con 0x7ff5bc072d30 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2828513747 0 0) 0x7ff5bc0cd900 con 0x7ff5bc15b300 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5bc0cdbe0 con 0x7ff5bc15b300 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5bbe00640 1 -- 172.21.15.28:0/2241932147 >> v1:172.21.15.138:6789/0 conn(0x7ff5bc15b300 legacy=0x7ff5bc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5bbe00640 1 -- 172.21.15.28:0/2241932147 >> v1:172.21.15.28:6789/0 conn(0x7ff5bc072d30 legacy=0x7ff5bc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.968+0000 7ff5bbe00640 1 -- 172.21.15.28:0/2241932147 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc072250 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.969+0000 7ff5bbe00640 1 --2- 172.21.15.28:0/2241932147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 0x7ff5bc165de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009dc0 tx=0x7ff5b402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2478fcfbf46ebd8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:21.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b402ae60 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b4007740 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5ba000640 1 -- 172.21.15.28:0/2241932147 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4007b10 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 -- 172.21.15.28:0/2241932147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 msgr2=0x7ff5bc165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 --2- 172.21.15.28:0/2241932147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 0x7ff5bc165de0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4009dc0 tx=0x7ff5b402a640 comp rx=0 tx=0).stop 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 -- 172.21.15.28:0/2241932147 shutdown_connections 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 --2- 172.21.15.28:0/2241932147 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc06f8d0 0x7ff5bc165de0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 -- 172.21.15.28:0/2241932147 >> 172.21.15.28:0/2241932147 conn(0x7ff5bc067d70 msgr2=0x7ff5bc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:21.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 -- 172.21.15.28:0/2241932147 shutdown_connections 2024-09-10T10:56:21.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.970+0000 7ff5c2200640 1 -- 172.21.15.28:0/2241932147 wait complete. 2024-09-10T10:56:21.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.972+0000 7ff5c2200640 1 Processor -- start 2024-09-10T10:56:21.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.972+0000 7ff5c2200640 1 -- start start 2024-09-10T10:56:21.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5bc072d30 0x7ff5bc157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc15b300 0x7ff5bc1f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0cffe0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc0cd5c0 con 0x7ff5bc072d30 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5c2200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc0d0160 con 0x7ff5bc15b300 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:21.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34908/0 (socket says 172.21.15.28:34908) 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.973+0000 7ff5bb400640 1 -- 172.21.15.28:0/4164524345 learned_addr learned my addr 172.21.15.28:0/4164524345 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bbe00640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc15b300 0x7ff5bc1f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5baa00640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5bc072d30 0x7ff5bc157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 -- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc15b300 msgr2=0x7ff5bc1f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc15b300 0x7ff5bc1f42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 -- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5bc072d30 msgr2=0x7ff5bc157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5bc072d30 0x7ff5bc157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:21.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 -- 172.21.15.28:0/4164524345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1580d0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5bb400640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c500 tx=0x7ff5ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.974+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0052d0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.975+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ac005490 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.975+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc1583b0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.975+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac012870 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.975+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc0ae6c0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.976+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc0b3bc0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.977+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff5ac012a10 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.977+0000 7ff5b3400640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 0x7ff59c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:21.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.978+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7ff5ac08a8f0 con 0x7ff5bc06f8d0 2024-09-10T10:56:21.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.978+0000 7ff5baa00640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 0x7ff59c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:21.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.979+0000 7ff5baa00640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 0x7ff59c06e2f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0cfbc0 tx=0x7ff5a8007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:21.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:21.982+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff5ac058ce0 con 0x7ff5bc06f8d0 2024-09-10T10:56:22.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.161+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5bc0b3e50 con 0x7ff59c06be30 2024-09-10T10:56:22.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.162+0000 7ff5b3400640 1 -- 172.21.15.28:0/4164524345 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13462 (secure 0 0 0) 0x7ff5bc0b3e50 con 0x7ff59c06be30 2024-09-10T10:56:22.165 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:22.165 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:56:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 msgr2=0x7ff59c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 0x7ff59c06e2f0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0cfbc0 tx=0x7ff5a8007d30 comp rx=0 tx=0).stop 2024-09-10T10:56:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 msgr2=0x7ff5bc0aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00c500 tx=0x7ff5ac00c9c0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 shutdown_connections 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff59c06be30 0x7ff59c06e2f0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff5bc15b300 0x7ff5bc1f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5bc072d30 0x7ff5bc157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 --2- 172.21.15.28:0/4164524345 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5bc06f8d0 0x7ff5bc0aebe0 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 >> 172.21.15.28:0/4164524345 conn(0x7ff5bc067d70 msgr2=0x7ff5bc15de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 shutdown_connections 2024-09-10T10:56:22.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.165+0000 7ff5c2200640 1 -- 172.21.15.28:0/4164524345 wait complete. 2024-09-10T10:56:22.182 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-10T10:56:20.682018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291036,"kb_used_data":16568,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683684,"statfs":{"total":773094113280,"available":772796092416,"internally_reserved":0,"allocated":16965632,"data_stored":9162676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36199,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-83,"num_read_kb":-85,"num_write":-94,"num_write_kb":-69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":420,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T10:56:14.975759+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:56:14.975759+0000","last_peered":"2024-09-10T10:56:14.975759+0000","last_clean":"2024-09-10T10:56:14.975759+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:56:14.975759+0000","last_undegraded":"2024-09-10T10:56:14.975759+0000","last_fullsized":"2024-09-10T10:56:14.975759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1953792,"data_stored":984482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28972,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1990656,"data_stored":1018908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1961984,"data_stored":990439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41452,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330388,"statfs":{"total":96636764160,"available":96594317312,"internally_reserved":0,"allocated":1994752,"data_stored":1019356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509596,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1953792,"data_stored":983942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509594,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42012,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329828,"statfs":{"total":96636764160,"available":96593743872,"internally_reserved":0,"allocated":2564096,"data_stored":1581297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509595,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":990971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33720,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338120,"statfs":{"total":96636764160,"available":96602234880,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:56:22.183 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:56:22.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.337+0000 7f35e5000640 1 Processor -- start 2024-09-10T10:56:22.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.337+0000 7f35e5000640 1 -- start start 2024-09-10T10:56:22.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35e5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35e5000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35e0072700 0x7f35e015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35e5000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35e0146140 con 0x7f35e0072320 2024-09-10T10:56:22.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35e5000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f35e00cd740 con 0x7f35e0072700 2024-09-10T10:56:22.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35e5000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35e00cda20 con 0x7f35e0072de0 2024-09-10T10:56:22.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35dde00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f35e0072320 0x7f35e00cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:49118/0 (socket says 172.21.15.28:49118) 2024-09-10T10:56:22.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35de800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35e0072700 0x7f35e015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.338+0000 7f35dde00640 1 -- 172.21.15.28:0/2281056639 learned_addr learned my addr 172.21.15.28:0/2281056639 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:22.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35d5400640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35dd400640 1 -- 172.21.15.28:0/2281056639 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1781593764 0 0) 0x7f35e0146140 con 0x7f35e0072320 2024-09-10T10:56:22.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35dd400640 1 -- 172.21.15.28:0/2281056639 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35c4003520 con 0x7f35e0072320 2024-09-10T10:56:22.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35de800640 1 -- 172.21.15.28:0/2281056639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 msgr2=0x7f35e015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35de800640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35de800640 1 -- 172.21.15.28:0/2281056639 >> v1:172.21.15.28:6789/0 conn(0x7f35e0072320 legacy=0x7f35e00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.339+0000 7f35de800640 1 -- 172.21.15.28:0/2281056639 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e00cdd00 con 0x7f35e0072700 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.340+0000 7f35de800640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072700 0x7f35e015d720 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009d90 tx=0x7f35d802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed8494fd80bf213 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.340+0000 7f35dd400640 1 -- 172.21.15.28:0/2281056639 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8031600 con 0x7f35e0072700 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.340+0000 7f35dd400640 1 -- 172.21.15.28:0/2281056639 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d80317a0 con 0x7f35e0072700 2024-09-10T10:56:22.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.340+0000 7f35dd400640 1 -- 172.21.15.28:0/2281056639 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8031b50 con 0x7f35e0072700 2024-09-10T10:56:22.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 -- 172.21.15.28:0/2281056639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072700 msgr2=0x7f35e015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072700 0x7f35e015d720 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009d90 tx=0x7f35d802a5d0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 -- 172.21.15.28:0/2281056639 shutdown_connections 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072700 0x7f35e015d720 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 --2- 172.21.15.28:0/2281056639 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 -- 172.21.15.28:0/2281056639 >> 172.21.15.28:0/2281056639 conn(0x7f35e0067d70 msgr2=0x7f35e006bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 -- 172.21.15.28:0/2281056639 shutdown_connections 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.341+0000 7f35e5000640 1 -- 172.21.15.28:0/2281056639 wait complete. 2024-09-10T10:56:22.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.342+0000 7f35e5000640 1 Processor -- start 2024-09-10T10:56:22.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 -- start start 2024-09-10T10:56:22.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e0072700 0x7f35e013c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e013c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35e015dde0 con 0x7f35e0072320 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.343+0000 7f35e5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35e015dc60 con 0x7f35e0072700 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35e5000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35e015df60 con 0x7f35e0072de0 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34930/0 (socket says 172.21.15.28:34930) 2024-09-10T10:56:22.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 -- 172.21.15.28:0/3745457063 learned_addr learned my addr 172.21.15.28:0/3745457063 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35d5400640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e0072700 0x7f35e013c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35de800640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e013c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 -- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 msgr2=0x7f35e013c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e013c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 -- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e0072700 msgr2=0x7f35e013c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e0072700 0x7f35e013c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 -- 172.21.15.28:0/3745457063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e013cdb0 con 0x7f35e0072320 2024-09-10T10:56:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35de800640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e013c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.344+0000 7f35dde00640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c430 tx=0x7f35d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.345+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0017070 con 0x7f35e0072320 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.345+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d0005ce0 con 0x7f35e0072320 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.345+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d00072f0 con 0x7f35e0072320 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.345+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e01e33e0 con 0x7f35e0072320 2024-09-10T10:56:22.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.345+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e01e39a0 con 0x7f35e0072320 2024-09-10T10:56:22.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.346+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e015e390 con 0x7f35e0072320 2024-09-10T10:56:22.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.347+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f35d001a070 con 0x7f35e0072320 2024-09-10T10:56:22.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.348+0000 7f35d6a00640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.348+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f35d008a1a0 con 0x7f35e0072320 2024-09-10T10:56:22.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.348+0000 7f35d5400640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.349+0000 7f35d5400640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 0x7f35bc06e2f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f35e01f5320 tx=0x7f35c8009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.352+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35d0058610 con 0x7f35e0072320 2024-09-10T10:56:22.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.532+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35e0145950 con 0x7f35bc06be30 2024-09-10T10:56:22.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.534+0000 7f35d6a00640 1 -- 172.21.15.28:0/3745457063 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13462 (secure 0 0 0) 0x7f35e0145950 con 0x7f35bc06be30 2024-09-10T10:56:22.535 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:22.536 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 msgr2=0x7f35bc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 0x7f35bc06e2f0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f35e01f5320 tx=0x7f35c8009190 comp rx=0 tx=0).stop 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 msgr2=0x7f35e013dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f35d000c430 tx=0x7f35d000c8f0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 shutdown_connections 2024-09-10T10:56:22.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f35bc06be30 0x7f35bc06e2f0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35e0072de0 0x7f35e013c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e0072700 0x7f35e013c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 --2- 172.21.15.28:0/3745457063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35e0072320 0x7f35e013dbb0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.535+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 >> 172.21.15.28:0/3745457063 conn(0x7f35e0067d70 msgr2=0x7f35e006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.536+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 shutdown_connections 2024-09-10T10:56:22.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.536+0000 7f35e5000640 1 -- 172.21.15.28:0/3745457063 wait complete. 2024-09-10T10:56:22.554 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-10T10:56:20.682018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":291036,"kb_used_data":16568,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683684,"statfs":{"total":773094113280,"available":772796092416,"internally_reserved":0,"allocated":16965632,"data_stored":9162676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36199,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-83,"num_read_kb":-85,"num_write":-94,"num_write_kb":-69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001529"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":420,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-09-10T10:56:14.975759+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:56:14.975759+0000","last_peered":"2024-09-10T10:56:14.975759+0000","last_clean":"2024-09-10T10:56:14.975759+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:56:14.975759+0000","last_undegraded":"2024-09-10T10:56:14.975759+0000","last_fullsized":"2024-09-10T10:56:14.975759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509594,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1953792,"data_stored":984482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28972,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1990656,"data_stored":1018908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33104,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338736,"statfs":{"total":96636764160,"available":96602865664,"internally_reserved":0,"allocated":1961984,"data_stored":990439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41452,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330388,"statfs":{"total":96636764160,"available":96594317312,"internally_reserved":0,"allocated":1994752,"data_stored":1019356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509596,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1953792,"data_stored":983942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509594,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42012,"kb_used_data":2504,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329828,"statfs":{"total":96636764160,"available":96593743872,"internally_reserved":0,"allocated":2564096,"data_stored":1581297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509595,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":990971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33720,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338120,"statfs":{"total":96636764160,"available":96602234880,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:56:22.554 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T10:56:22.555 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:56:22.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.708+0000 7f8d10a00640 1 Processor -- start 2024-09-10T10:56:22.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.708+0000 7f8d10a00640 1 -- start start 2024-09-10T10:56:22.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.709+0000 7f8d10a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d10a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d10a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d0c146140 con 0x7f8d0c072de0 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d10a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8d0c0cd740 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d10a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c0cda20 con 0x7f8d0c072320 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34940/0 (socket says 172.21.15.28:34940) 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 -- 172.21.15.28:0/727940116 learned_addr learned my addr 172.21.15.28:0/727940116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:22.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d09e00640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 -- 172.21.15.28:0/727940116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 msgr2=0x7f8d0c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:22.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c0cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 -- 172.21.15.28:0/727940116 >> v1:172.21.15.138:6789/0 conn(0x7f8d0c072de0 legacy=0x7f8d0c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.710+0000 7f8d0a800640 1 -- 172.21.15.28:0/727940116 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c0cdd00 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.711+0000 7f8d0a800640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a25ac2695fd3c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d09400640 1 -- 172.21.15.28:0/727940116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04036070 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d09400640 1 -- 172.21.15.28:0/727940116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04031600 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d09400640 1 -- 172.21.15.28:0/727940116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d040319d0 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 -- 172.21.15.28:0/727940116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 msgr2=0x7f8d0c06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e50 tx=0x7f8d0402a680 comp rx=0 tx=0).stop 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 -- 172.21.15.28:0/727940116 shutdown_connections 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c06e7c0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 --2- 172.21.15.28:0/727940116 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c0cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 -- 172.21.15.28:0/727940116 >> 172.21.15.28:0/727940116 conn(0x7f8d0c067d70 msgr2=0x7f8d0c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 -- 172.21.15.28:0/727940116 shutdown_connections 2024-09-10T10:56:22.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.712+0000 7f8d10a00640 1 -- 172.21.15.28:0/727940116 wait complete. 2024-09-10T10:56:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.714+0000 7f8d10a00640 1 Processor -- start 2024-09-10T10:56:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.714+0000 7f8d10a00640 1 -- start start 2024-09-10T10:56:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 0x7f8d0c0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0cd520 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d0c0cd3a0 con 0x7f8d0c072de0 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d10a00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d0c06ed00 con 0x7f8d0c072320 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d0a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34946/0 (socket says 172.21.15.28:34946) 2024-09-10T10:56:22.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.715+0000 7f8d0a800640 1 -- 172.21.15.28:0/189884720 learned_addr learned my addr 172.21.15.28:0/189884720 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d09e00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d03e00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 0x7f8d0c0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 -- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 msgr2=0x7f8d0c13fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c13fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 -- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 msgr2=0x7f8d0c0c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 0x7f8d0c0c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 -- 172.21.15.28:0/189884720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d0c0c7110 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d03e00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 0x7f8d0c0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d0a800640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f8d04034040 tx=0x7f8d04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.716+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04036050 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.717+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d04042070 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.717+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04009510 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.717+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d0c0c7390 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.717+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d0c0c7950 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.718+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d0c072700 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.719+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d04007bf0 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.719+0000 7f8d02000640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 0x7f8ce806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:22.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.719+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f8d040abaa0 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.720+0000 7f8d09e00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 0x7f8ce806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:22.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.721+0000 7f8d09e00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 0x7f8ce806e2f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1f52c0 tx=0x7f8cfc009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:22.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.723+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d0c0c7110 con 0x7f8d0c15e1f0 2024-09-10T10:56:22.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.906+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d0c0c7cf0 con 0x7f8ce806be30 2024-09-10T10:56:22.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.908+0000 7f8d02000640 1 -- 172.21.15.28:0/189884720 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13500 (secure 0 0 0) 0x7f8d0c0c7cf0 con 0x7f8ce806be30 2024-09-10T10:56:22.909 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:22.910 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:56:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.909+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 msgr2=0x7f8ce806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.909+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 0x7f8ce806e2f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c1f52c0 tx=0x7f8cfc009190 comp rx=0 tx=0).stop 2024-09-10T10:56:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.909+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 msgr2=0x7f8d0c13f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f8d04034040 tx=0x7f8d04004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:22.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 shutdown_connections 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ce806be30 0x7f8ce806e2f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d0c15e1f0 0x7f8d0c13f410 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d0c072de0 0x7f8d0c0c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 --2- 172.21.15.28:0/189884720 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d0c072320 0x7f8d0c13fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 >> 172.21.15.28:0/189884720 conn(0x7f8d0c067d70 msgr2=0x7f8d0c0c5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 shutdown_connections 2024-09-10T10:56:22.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:22.910+0000 7f8d10a00640 1 -- 172.21.15.28:0/189884720 wait complete. 2024-09-10T10:56:22.927 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-10T10:56:22.682580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291196,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683524,"statfs":{"total":773094113280,"available":772795928576,"internally_reserved":0,"allocated":17113088,"data_stored":9204040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-36199,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-83,"num_read_kb":-85,"num_write":-94,"num_write_kb":-69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.710451"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":423,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-10T10:56:18.390316+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:56:18.390316+0000","last_peered":"2024-09-10T10:56:18.390316+0000","last_clean":"2024-09-10T10:56:18.390316+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:56:18.390316+0000","last_undegraded":"2024-09-10T10:56:18.390316+0000","last_fullsized":"2024-09-10T10:56:18.390316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:57:31.477479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509597,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28972,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41456,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330384,"statfs":{"total":96636764160,"available":96594313216,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42032,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329808,"statfs":{"total":96636764160,"available":96593723392,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33720,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338120,"statfs":{"total":96636764160,"available":96602234880,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:56:22.928 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:56:23.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.081+0000 7f7e7ae00640 1 Processor -- start 2024-09-10T10:56:23.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.081+0000 7f7e7ae00640 1 -- start start 2024-09-10T10:56:23.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.082+0000 7f7e7ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740b69d0 0x7f7e74162a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.082+0000 7f7e7ae00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e740af280 0x7f7e740afa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.082+0000 7f7e7ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7e740affb0 0x7f7e74154d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e7ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7e740b6db0 con 0x7f7e740affb0 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e7ae00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7e7415eb60 con 0x7f7e740af280 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e7ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7415ee40 con 0x7f7e740b69d0 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7e740affb0 0x7f7e74154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7e740affb0 0x7f7e74154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34958/0 (socket says 172.21.15.28:34958) 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 -- 172.21.15.28:0/4147895605 learned_addr learned my addr 172.21.15.28:0/4147895605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e73e00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740b69d0 0x7f7e74162a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e6be00640 1 --2- 172.21.15.28:0/4147895605 >> v2:172.21.15.138:3300/0 conn(0x7f7e740af280 0x7f7e740afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 -- 172.21.15.28:0/4147895605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740b69d0 msgr2=0x7f7e74162a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:23.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740b69d0 0x7f7e74162a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 -- 172.21.15.28:0/4147895605 >> v2:172.21.15.138:3300/0 conn(0x7f7e740af280 msgr2=0x7f7e740afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:23.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 --2- 172.21.15.28:0/4147895605 >> v2:172.21.15.138:3300/0 conn(0x7f7e740af280 0x7f7e740afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.083+0000 7f7e78e00640 1 -- 172.21.15.28:0/4147895605 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7415f120 con 0x7f7e740affb0 2024-09-10T10:56:23.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.084+0000 7f7e78e00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740affb0 0x7f7e74154d00 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7b682d0dfef713 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:23.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e73400640 1 -- 172.21.15.28:0/4147895605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c036070 con 0x7f7e740affb0 2024-09-10T10:56:23.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e73400640 1 -- 172.21.15.28:0/4147895605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e6c031600 con 0x7f7e740affb0 2024-09-10T10:56:23.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e73400640 1 -- 172.21.15.28:0/4147895605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0319d0 con 0x7f7e740affb0 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 -- 172.21.15.28:0/4147895605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740affb0 msgr2=0x7f7e74154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740affb0 0x7f7e74154d00 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c02a5b0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 -- 172.21.15.28:0/4147895605 shutdown_connections 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740affb0 0x7f7e74154d00 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/4147895605 >> v2:172.21.15.138:3300/0 conn(0x7f7e740af280 0x7f7e740afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/4147895605 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740b69d0 0x7f7e74162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 -- 172.21.15.28:0/4147895605 >> 172.21.15.28:0/4147895605 conn(0x7f7e7406bc10 msgr2=0x7f7e7406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:23.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 -- 172.21.15.28:0/4147895605 shutdown_connections 2024-09-10T10:56:23.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.085+0000 7f7e7ae00640 1 -- 172.21.15.28:0/4147895605 wait complete. 2024-09-10T10:56:23.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.087+0000 7f7e7ae00640 1 Processor -- start 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.087+0000 7f7e7ae00640 1 -- start start 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740affb0 0x7f7e74153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e740b69d0 0x7f7e7415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74155240 con 0x7f7e740af280 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e740af6e0 con 0x7f7e740b69d0 2024-09-10T10:56:23.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e7ae00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e741553c0 con 0x7f7e740affb0 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e73e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34968/0 (socket says 172.21.15.28:34968) 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e73e00640 1 -- 172.21.15.28:0/53614062 learned_addr learned my addr 172.21.15.28:0/53614062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.088+0000 7f7e6be00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740affb0 0x7f7e74153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e78e00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e740b69d0 0x7f7e7415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 -- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740affb0 msgr2=0x7f7e74153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:23.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740affb0 0x7f7e74153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 -- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e740b69d0 msgr2=0x7f7e7415d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e740b69d0 0x7f7e7415d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 -- 172.21.15.28:0/53614062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e74160180 con 0x7f7e740af280 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e73e00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400d5f0 tx=0x7f7e6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e640125f0 con 0x7f7e740af280 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e64019070 con 0x7f7e740af280 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e74160460 con 0x7f7e740af280 2024-09-10T10:56:23.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e640129d0 con 0x7f7e740af280 2024-09-10T10:56:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.089+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e74160a20 con 0x7f7e740af280 2024-09-10T10:56:23.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.090+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e74155820 con 0x7f7e740af280 2024-09-10T10:56:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.091+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e64012b70 con 0x7f7e740af280 2024-09-10T10:56:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.092+0000 7f7e70c00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 0x7f7e5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.092+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f7e6408b1e0 con 0x7f7e740af280 2024-09-10T10:56:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.092+0000 7f7e6be00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 0x7f7e5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:23.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.093+0000 7f7e6be00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 0x7f7e5006e2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7e741618b0 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:23.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.096+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e64059c10 con 0x7f7e740af280 2024-09-10T10:56:23.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.283+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e7406ee30 con 0x7f7e740af280 2024-09-10T10:56:23.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.284+0000 7f7e70c00640 1 -- 172.21.15.28:0/53614062 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+13467 (secure 0 0 0) 0x7f7e64059510 con 0x7f7e740af280 2024-09-10T10:56:23.286 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:23.286 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":154,"fsid":"f658987a-dd14-4573-bcd3-ee2a383c9fe0","created":"2024-09-10T10:43:21.616313+0000","modified":"2024-09-10T10:56:18.008806+0000","last_up_change":"2024-09-10T10:43:35.402905+0000","last_in_change":"2024-09-10T10:43:25.476785+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T10:43:36.538469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"897214ac-815c-46c9-bd71-6557ec7b9859","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6809","nonce":1098003867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6811","nonce":1098003867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6815","nonce":1098003867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":1098003867},{"type":"v1","addr":"172.21.15.28:6813","nonce":1098003867}]},"public_addr":"172.21.15.28:6809/1098003867","cluster_addr":"172.21.15.28:6811/1098003867","heartbeat_back_addr":"172.21.15.28:6815/1098003867","heartbeat_front_addr":"172.21.15.28:6813/1098003867","state":["exists","up"]},{"osd":1,"uuid":"300c9348-8f79-4a46-a010-b71ef4e61396","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6801","nonce":2223305111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6803","nonce":2223305111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6807","nonce":2223305111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":2223305111},{"type":"v1","addr":"172.21.15.28:6805","nonce":2223305111}]},"public_addr":"172.21.15.28:6801/2223305111","cluster_addr":"172.21.15.28:6803/2223305111","heartbeat_back_addr":"172.21.15.28:6807/2223305111","heartbeat_front_addr":"172.21.15.28:6805/2223305111","state":["exists","up"]},{"osd":2,"uuid":"c87c3518-ad7b-47e2-8405-81c0509fd89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6825","nonce":1014290416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6827","nonce":1014290416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6831","nonce":1014290416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":1014290416},{"type":"v1","addr":"172.21.15.28:6829","nonce":1014290416}]},"public_addr":"172.21.15.28:6825/1014290416","cluster_addr":"172.21.15.28:6827/1014290416","heartbeat_back_addr":"172.21.15.28:6831/1014290416","heartbeat_front_addr":"172.21.15.28:6829/1014290416","state":["exists","up"]},{"osd":3,"uuid":"ea9fa499-68d2-496b-8b75-8d0e15cc4be3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6817","nonce":2614769734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6819","nonce":2614769734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6823","nonce":2614769734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":2614769734},{"type":"v1","addr":"172.21.15.28:6821","nonce":2614769734}]},"public_addr":"172.21.15.28:6817/2614769734","cluster_addr":"172.21.15.28:6819/2614769734","heartbeat_back_addr":"172.21.15.28:6823/2614769734","heartbeat_front_addr":"172.21.15.28:6821/2614769734","state":["exists","up"]},{"osd":4,"uuid":"9d933671-197e-4151-ad30-dacffb5143bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6811","nonce":1083898337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6813","nonce":1083898337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6817","nonce":1083898337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1083898337},{"type":"v1","addr":"172.21.15.138:6815","nonce":1083898337}]},"public_addr":"172.21.15.138:6811/1083898337","cluster_addr":"172.21.15.138:6813/1083898337","heartbeat_back_addr":"172.21.15.138:6817/1083898337","heartbeat_front_addr":"172.21.15.138:6815/1083898337","state":["exists","up"]},{"osd":5,"uuid":"53157853-f5c4-47c2-b63c-09dd6c2a964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6803","nonce":2049132176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6805","nonce":2049132176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6809","nonce":2049132176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2049132176},{"type":"v1","addr":"172.21.15.138:6807","nonce":2049132176}]},"public_addr":"172.21.15.138:6803/2049132176","cluster_addr":"172.21.15.138:6805/2049132176","heartbeat_back_addr":"172.21.15.138:6809/2049132176","heartbeat_front_addr":"172.21.15.138:6807/2049132176","state":["exists","up"]},{"osd":6,"uuid":"7a541dc7-d8ae-4619-991d-49c2a3caffd4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6819","nonce":1283254914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6821","nonce":1283254914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6825","nonce":1283254914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1283254914},{"type":"v1","addr":"172.21.15.138:6823","nonce":1283254914}]},"public_addr":"172.21.15.138:6819/1283254914","cluster_addr":"172.21.15.138:6821/1283254914","heartbeat_back_addr":"172.21.15.138:6825/1283254914","heartbeat_front_addr":"172.21.15.138:6823/1283254914","state":["exists","up"]},{"osd":7,"uuid":"bb5c2ad5-c112-419b-8ad5-e415abfc4b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6827","nonce":4029970444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6829","nonce":4029970444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6833","nonce":4029970444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":4029970444},{"type":"v1","addr":"172.21.15.138:6831","nonce":4029970444}]},"public_addr":"172.21.15.138:6827/4029970444","cluster_addr":"172.21.15.138:6829/4029970444","heartbeat_back_addr":"172.21.15.138:6833/4029970444","heartbeat_front_addr":"172.21.15.138:6831/4029970444","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.043060+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.791562+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.528105+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.889226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.297984+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:31.698320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.238935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T10:43:32.071969+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6835/2598694071":"2024-09-11T10:56:14.969031+0000","172.21.15.28:6833/2598694071":"2024-09-11T10:56:14.969031+0000","172.21.15.28:6835/354907385":"2024-09-11T10:56:03.806379+0000","172.21.15.28:6833/354907385":"2024-09-11T10:56:03.806379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:56:23.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 msgr2=0x7f7e5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:23.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 0x7f7e5006e2f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f7e741618b0 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 msgr2=0x7f7e7415fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400d5f0 tx=0x7f7e6400dab0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 shutdown_connections 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7e5006be30 0x7f7e5006e2f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e740b69d0 0x7f7e7415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e740affb0 0x7f7e74153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 --2- 172.21.15.28:0/53614062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e740af280 0x7f7e7415fa80 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:23.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 >> 172.21.15.28:0/53614062 conn(0x7f7e7406bc10 msgr2=0x7f7e740b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:23.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 shutdown_connections 2024-09-10T10:56:23.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:23.286+0000 7f7e7ae00640 1 -- 172.21.15.28:0/53614062 wait complete. 2024-09-10T10:56:24.305 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T10:56:24.305 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.475+0000 7f8b03000640 1 Processor -- start 2024-09-10T10:56:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.475+0000 7f8b03000640 1 -- start start 2024-09-10T10:56:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.476+0000 7f8b03000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8afc0af940 0x7f8afc1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.476+0000 7f8b03000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.476+0000 7f8b03000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8afc0b3960 con 0x7f8afc072070 2024-09-10T10:56:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.476+0000 7f8b03000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8afc147f60 con 0x7f8afc0af940 2024-09-10T10:56:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.476+0000 7f8b03000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc144bf0 con 0x7f8afc0b3460 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8af3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8af3400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:47618/0 (socket says 172.21.15.28:47618) 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8af3400640 1 -- 172.21.15.28:0/708757152 learned_addr learned my addr 172.21.15.28:0/708757152 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8afbe00640 1 --2- 172.21.15.28:0/708757152 >> v2:172.21.15.138:3300/0 conn(0x7f8afc0af940 0x7f8afc1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8afbe00640 1 -- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 msgr2=0x7f8afc0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:24.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.477+0000 7f8afbe00640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.478+0000 7f8afbe00640 1 -- 172.21.15.28:0/708757152 >> v1:172.21.15.138:6789/0 conn(0x7f8afc072070 legacy=0x7f8afc0b4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.478+0000 7f8afbe00640 1 -- 172.21.15.28:0/708757152 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc0b3c40 con 0x7f8afc0af940 2024-09-10T10:56:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.478+0000 7f8af3400640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:56:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8afbe00640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc0af940 0x7f8afc1446b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0097a0 tx=0x7f8aec02a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55bb156b040b5077 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8afb400640 1 -- 172.21.15.28:0/708757152 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec037070 con 0x7f8afc0af940 2024-09-10T10:56:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8afb400640 1 -- 172.21.15.28:0/708757152 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aec02ae40 con 0x7f8afc0af940 2024-09-10T10:56:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8afb400640 1 -- 172.21.15.28:0/708757152 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec032660 con 0x7f8afc0af940 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8b03000640 1 -- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc0af940 msgr2=0x7f8afc1446b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.479+0000 7f8b03000640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc0af940 0x7f8afc1446b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0097a0 tx=0x7f8aec02a0f0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 -- 172.21.15.28:0/708757152 shutdown_connections 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 --2- 172.21.15.28:0/708757152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc0af940 0x7f8afc1446b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 -- 172.21.15.28:0/708757152 >> 172.21.15.28:0/708757152 conn(0x7f8afc067d70 msgr2=0x7f8afc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 -- 172.21.15.28:0/708757152 shutdown_connections 2024-09-10T10:56:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.480+0000 7f8b03000640 1 -- 172.21.15.28:0/708757152 wait complete. 2024-09-10T10:56:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.481+0000 7f8b03000640 1 Processor -- start 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.481+0000 7f8b03000640 1 -- start start 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc072070 0x7f8afc156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 0x7f8afc15ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc15b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0b4200 con 0x7f8afc0af940 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8afc0b4780 con 0x7f8afc072070 2024-09-10T10:56:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.482+0000 7f8b03000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8afc0afd20 con 0x7f8afc0b3460 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 0x7f8afc15ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 0x7f8afc15ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34632/0 (socket says 172.21.15.28:34632) 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 -- 172.21.15.28:0/3587486299 learned_addr learned my addr 172.21.15.28:0/3587486299 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8afbe00640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc072070 0x7f8afc156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8b01000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc15b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 -- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 msgr2=0x7f8afc15b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc15b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 -- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc072070 msgr2=0x7f8afc156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8afc072070 0x7f8afc156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8afc160810 con 0x7f8afc0af940 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8b01000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8afc0b3460 0x7f8afc15b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.483+0000 7f8af3400640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 0x7f8afc15ad00 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400e9d0 tx=0x7f8ae400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.484+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4017070 con 0x7f8afc0af940 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.484+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ae400ca60 con 0x7f8afc0af940 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.484+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400ce10 con 0x7f8afc0af940 2024-09-10T10:56:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.484+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8afc145260 con 0x7f8afc0af940 2024-09-10T10:56:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.484+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8afc145820 con 0x7f8afc0af940 2024-09-10T10:56:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.485+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8afc1446b0 con 0x7f8afc0af940 2024-09-10T10:56:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.486+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ae401a070 con 0x7f8afc0af940 2024-09-10T10:56:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8af8c00640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ad806be30 0x7f8ad806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f8ae408b380 con 0x7f8afc0af940 2024-09-10T10:56:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8af8c00640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f8ad8072370 0x7f8ad80747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ad8074e80 con 0x7f8ad8072370 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f8ae408b730 con 0x7f8afc0af940 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8afbe00640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ad806be30 0x7f8ad806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.487+0000 7f8b01000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f8ad8072370 0x7f8ad80747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.488+0000 7f8b01000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f8ad8072370 0x7f8ad80747b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.488+0000 7f8afbe00640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ad806be30 0x7f8ad806e2f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8aec034000 tx=0x7f8aec02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.488+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== osd.0 v2:172.21.15.28:6808/1098003867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad8074e80 con 0x7f8ad8072370 2024-09-10T10:56:24.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 --> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8afc0b4d40 con 0x7f8ad8072370 2024-09-10T10:56:24.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8af8c00640 1 -- 172.21.15.28:0/3587486299 <== osd.0 v2:172.21.15.28:6808/1098003867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8afc0b4d40 con 0x7f8ad8072370 2024-09-10T10:56:24.504 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:24.504 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:24.504 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f8ad8072370 msgr2=0x7f8ad80747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:6808/1098003867,v1:172.21.15.28:6809/1098003867] conn(0x7f8ad8072370 0x7f8ad80747b0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ad806be30 msgr2=0x7f8ad806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f8ad806be30 0x7f8ad806e2f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f8aec034000 tx=0x7f8aec02e000 comp rx=0 tx=0).stop 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 msgr2=0x7f8afc15ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b03000640 1 --2- 172.21.15.28:0/3587486299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8afc0af940 0x7f8afc15ad00 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400e9d0 tx=0x7f8ae400ee90 comp rx=0 tx=0).stop 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.502+0000 7f8b01000640 1 -- 172.21.15.28:0/3587486299 reap_dead start 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.503+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 shutdown_connections 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.503+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 >> 172.21.15.28:0/3587486299 conn(0x7f8afc067d70 msgr2=0x7f8afc0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:24.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.503+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 shutdown_connections 2024-09-10T10:56:24.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.503+0000 7f8b03000640 1 -- 172.21.15.28:0/3587486299 wait complete. 2024-09-10T10:56:24.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T10:56:24.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.675+0000 7ff339400640 1 Processor -- start 2024-09-10T10:56:24.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.676+0000 7ff339400640 1 -- start start 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff339400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff334072de0 0x7ff33415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff339400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff334072700 0x7ff33415d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff339400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff334146140 con 0x7ff334072320 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff339400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff3340cd740 con 0x7ff334072700 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff339400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff3340cda20 con 0x7ff334072de0 2024-09-10T10:56:24.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff332800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff334072320 0x7ff3340cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51226/0 (socket says 172.21.15.28:51226) 2024-09-10T10:56:24.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff332800640 1 -- 172.21.15.28:0/506952172 learned_addr learned my addr 172.21.15.28:0/506952172 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:24.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff333200640 1 --2- 172.21.15.28:0/506952172 >> v2:172.21.15.28:3300/0 conn(0x7ff334072700 0x7ff33415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff32be00640 1 --2- 172.21.15.28:0/506952172 >> v2:172.21.15.138:3300/0 conn(0x7ff334072de0 0x7ff33415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.677+0000 7ff331e00640 1 -- 172.21.15.28:0/506952172 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787269832 0 0) 0x7ff334146140 con 0x7ff334072320 2024-09-10T10:56:24.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.678+0000 7ff331e00640 1 -- 172.21.15.28:0/506952172 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff318003520 con 0x7ff334072320 2024-09-10T10:56:24.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.678+0000 7ff32be00640 1 -- 172.21.15.28:0/506952172 >> v2:172.21.15.28:3300/0 conn(0x7ff334072700 msgr2=0x7ff33415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:24.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.678+0000 7ff32be00640 1 --2- 172.21.15.28:0/506952172 >> v2:172.21.15.28:3300/0 conn(0x7ff334072700 0x7ff33415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.678+0000 7ff32be00640 1 -- 172.21.15.28:0/506952172 >> v1:172.21.15.28:6789/0 conn(0x7ff334072320 legacy=0x7ff3340cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.678+0000 7ff32be00640 1 -- 172.21.15.28:0/506952172 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3340cdd00 con 0x7ff334072de0 2024-09-10T10:56:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.679+0000 7ff32be00640 1 --2- 172.21.15.28:0/506952172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff33415d1e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff33406f8c0 tx=0x7ff31c02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644ada3bcb5136d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.679+0000 7ff331e00640 1 -- 172.21.15.28:0/506952172 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c037070 con 0x7ff334072de0 2024-09-10T10:56:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.679+0000 7ff331e00640 1 -- 172.21.15.28:0/506952172 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff31c032440 con 0x7ff334072de0 2024-09-10T10:56:24.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.679+0000 7ff331e00640 1 -- 172.21.15.28:0/506952172 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c02ae60 con 0x7ff334072de0 2024-09-10T10:56:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 -- 172.21.15.28:0/506952172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 msgr2=0x7ff33415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 --2- 172.21.15.28:0/506952172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff33415d1e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff33406f8c0 tx=0x7ff31c02a110 comp rx=0 tx=0).stop 2024-09-10T10:56:24.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 -- 172.21.15.28:0/506952172 shutdown_connections 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 --2- 172.21.15.28:0/506952172 >> v2:172.21.15.28:3300/0 conn(0x7ff334072700 0x7ff33415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 --2- 172.21.15.28:0/506952172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff33415d1e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 -- 172.21.15.28:0/506952172 >> 172.21.15.28:0/506952172 conn(0x7ff334067d70 msgr2=0x7ff33406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 -- 172.21.15.28:0/506952172 shutdown_connections 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.680+0000 7ff339400640 1 -- 172.21.15.28:0/506952172 wait complete. 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.681+0000 7ff339400640 1 Processor -- start 2024-09-10T10:56:24.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.681+0000 7ff339400640 1 -- start start 2024-09-10T10:56:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff334072700 0x7ff3340b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff3340b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff33415dde0 con 0x7ff334072320 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff33415dc60 con 0x7ff334072de0 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.682+0000 7ff339400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff33415df60 con 0x7ff334072700 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34642/0 (socket says 172.21.15.28:34642) 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 -- 172.21.15.28:0/3233952712 learned_addr learned my addr 172.21.15.28:0/3233952712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:24.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff333200640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff3340b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff32be00640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff334072700 0x7ff3340b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 -- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff334072700 msgr2=0x7ff3340b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff334072700 0x7ff3340b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 -- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 msgr2=0x7ff3340b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff3340b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 -- 172.21.15.28:0/3233952712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334140690 con 0x7ff334072320 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.683+0000 7ff332800640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c430 tx=0x7ff32400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.684+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324017070 con 0x7ff334072320 2024-09-10T10:56:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.684+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff324005ce0 con 0x7ff334072320 2024-09-10T10:56:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.684+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3341e6130 con 0x7ff334072320 2024-09-10T10:56:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.684+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3240072f0 con 0x7ff334072320 2024-09-10T10:56:24.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.684+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3341e66f0 con 0x7ff334072320 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.685+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33415e320 con 0x7ff334072320 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.686+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff32401a070 con 0x7ff334072320 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.687+0000 7ff32aa00640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.687+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7ff32408a1a0 con 0x7ff334072320 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.687+0000 7ff32be00640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:24.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.688+0000 7ff32be00640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 0x7ff31006e2f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff31c034000 tx=0x7ff31c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:24.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.691+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff324058610 con 0x7ff334072320 2024-09-10T10:56:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.878+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7ff33415c330 con 0x7ff31006be30 2024-09-10T10:56:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.879+0000 7ff32aa00640 1 -- 172.21.15.28:0/3233952712 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff33415c330 con 0x7ff31006be30 2024-09-10T10:56:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 msgr2=0x7ff31006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 0x7ff31006e2f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff31c034000 tx=0x7ff31c02e000 comp rx=0 tx=0).stop 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 msgr2=0x7ff334140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7ff32400c430 tx=0x7ff32400c8f0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 shutdown_connections 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ff31006be30 0x7ff31006e2f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff334072de0 0x7ff3340b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff334072700 0x7ff3340b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 --2- 172.21.15.28:0/3233952712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff334072320 0x7ff334140150 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.881+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 >> 172.21.15.28:0/3233952712 conn(0x7ff334067d70 msgr2=0x7ff33406e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.882+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 shutdown_connections 2024-09-10T10:56:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:24.882+0000 7ff339400640 1 -- 172.21.15.28:0/3233952712 wait complete. 2024-09-10T10:56:24.899 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T10:56:24.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:25.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.057+0000 7f4c8ca00640 1 Processor -- start 2024-09-10T10:56:25.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.057+0000 7f4c8ca00640 1 -- start start 2024-09-10T10:56:25.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c8ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4c880d2950 0x7f4c880d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c8ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c880b56a0 0x7f4c880b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c8ca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c880b1030 con 0x7f4c880d2d30 2024-09-10T10:56:25.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c8ca00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4c880b1c00 con 0x7f4c880b56a0 2024-09-10T10:56:25.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c8ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4c880b12d0 con 0x7f4c880d2950 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c85e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c880b56a0 0x7f4c880b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c85e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c880b56a0 0x7f4c880b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34658/0 (socket says 172.21.15.28:34658) 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.058+0000 7f4c85e00640 1 -- 172.21.15.28:0/2471280487 learned_addr learned my addr 172.21.15.28:0/2471280487 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.059+0000 7f4c86800640 1 --2- 172.21.15.28:0/2471280487 >> v2:172.21.15.138:3300/0 conn(0x7f4c880d2950 0x7f4c880d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.059+0000 7f4c85e00640 1 -- 172.21.15.28:0/2471280487 >> v2:172.21.15.138:3300/0 conn(0x7f4c880d2950 msgr2=0x7f4c880d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.059+0000 7f4c85e00640 1 --2- 172.21.15.28:0/2471280487 >> v2:172.21.15.138:3300/0 conn(0x7f4c880d2950 0x7f4c880d4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.059+0000 7f4c85e00640 1 -- 172.21.15.28:0/2471280487 >> v1:172.21.15.138:6789/0 conn(0x7f4c880d2d30 legacy=0x7f4c880b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.059+0000 7f4c85e00640 1 -- 172.21.15.28:0/2471280487 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c880b15b0 con 0x7f4c880b56a0 2024-09-10T10:56:25.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.060+0000 7f4c85e00640 1 --2- 172.21.15.28:0/2471280487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880b3c50 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f4c880b6e80 tx=0x7f4c7402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14759d4219a25e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.060+0000 7f4c85400640 1 -- 172.21.15.28:0/2471280487 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74037070 con 0x7f4c880b56a0 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.060+0000 7f4c85400640 1 -- 172.21.15.28:0/2471280487 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c74032440 con 0x7f4c880b56a0 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.060+0000 7f4c85400640 1 -- 172.21.15.28:0/2471280487 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74032810 con 0x7f4c880b56a0 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2471280487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 msgr2=0x7f4c880b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2471280487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880b3c50 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f4c880b6e80 tx=0x7f4c7402a200 comp rx=0 tx=0).stop 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2471280487 shutdown_connections 2024-09-10T10:56:25.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2471280487 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880b3c50 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2471280487 >> v2:172.21.15.138:3300/0 conn(0x7f4c880d2950 0x7f4c880d4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2471280487 >> 172.21.15.28:0/2471280487 conn(0x7f4c8808bfc0 msgr2=0x7f4c880d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2471280487 shutdown_connections 2024-09-10T10:56:25.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.061+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2471280487 wait complete. 2024-09-10T10:56:25.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.062+0000 7f4c8ca00640 1 Processor -- start 2024-09-10T10:56:25.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 -- start start 2024-09-10T10:56:25.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880d2950 0x7f4c8815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880d2d30 0x7f4c880ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880b48a0 con 0x7f4c880b56a0 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880d5180 con 0x7f4c880d2d30 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.063+0000 7f4c8ca00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c880b4a20 con 0x7f4c880d2950 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34668/0 (socket says 172.21.15.28:34668) 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 -- 172.21.15.28:0/2748465094 learned_addr learned my addr 172.21.15.28:0/2748465094 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c87200640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880d2d30 0x7f4c880ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c85e00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880d2950 0x7f4c8815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 -- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880d2950 msgr2=0x7f4c8815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c880d2950 0x7f4c8815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 -- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880d2d30 msgr2=0x7f4c880ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c880d2d30 0x7f4c880ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c881f3e70 con 0x7f4c880b56a0 2024-09-10T10:56:25.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.064+0000 7f4c86800640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880c8e50 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800e870 tx=0x7f4c7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78017070 con 0x7f4c880b56a0 2024-09-10T10:56:25.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7800c940 con 0x7f4c880b56a0 2024-09-10T10:56:25.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7800cd30 con 0x7f4c880b56a0 2024-09-10T10:56:25.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c880c95c0 con 0x7f4c880b56a0 2024-09-10T10:56:25.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c880cc4e0 con 0x7f4c880b56a0 2024-09-10T10:56:25.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.065+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c880c99b0 con 0x7f4c880b56a0 2024-09-10T10:56:25.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.067+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c7801a070 con 0x7f4c880b56a0 2024-09-10T10:56:25.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c7ea00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f4c7808af40 con 0x7f4c880b56a0 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c7ea00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f4c68072370 0x7f4c680747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c78007e50 con 0x7f4c68072370 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f4c7808b2f0 con 0x7f4c880b56a0 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c85e00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.068+0000 7f4c87200640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f4c68072370 0x7f4c680747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.069+0000 7f4c87200640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f4c68072370 0x7f4c680747b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.069+0000 7f4c85e00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4c74034000 tx=0x7f4c7402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.070+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== osd.1 v2:172.21.15.28:6800/2223305111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c78007e50 con 0x7f4c68072370 2024-09-10T10:56:25.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.083+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 --> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4c880d4c40 con 0x7f4c68072370 2024-09-10T10:56:25.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.083+0000 7f4c7ea00640 1 -- 172.21.15.28:0/2748465094 <== osd.1 v2:172.21.15.28:6800/2223305111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4c880d4c40 con 0x7f4c68072370 2024-09-10T10:56:25.085 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:25.086 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:25.086 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f4c68072370 msgr2=0x7f4c680747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:6800/2223305111,v1:172.21.15.28:6801/2223305111] conn(0x7f4c68072370 0x7f4c680747b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 msgr2=0x7f4c6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4c6806be30 0x7f4c6806e2f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4c74034000 tx=0x7f4c7402e000 comp rx=0 tx=0).stop 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 msgr2=0x7f4c880c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 --2- 172.21.15.28:0/2748465094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c880b56a0 0x7f4c880c8e50 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800e870 tx=0x7f4c7800ed30 comp rx=0 tx=0).stop 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c87200640 1 -- 172.21.15.28:0/2748465094 reap_dead start 2024-09-10T10:56:25.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 shutdown_connections 2024-09-10T10:56:25.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 >> 172.21.15.28:0/2748465094 conn(0x7f4c8808bfc0 msgr2=0x7f4c8806e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 shutdown_connections 2024-09-10T10:56:25.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.084+0000 7f4c8ca00640 1 -- 172.21.15.28:0/2748465094 wait complete. 2024-09-10T10:56:25.100 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T10:56:25.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.257+0000 7f5b29000640 1 Processor -- start 2024-09-10T10:56:25.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.257+0000 7f5b29000640 1 -- start start 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.258+0000 7f5b29000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5b24072320 0x7f5b240cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b29000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b24146140 con 0x7f5b24072de0 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b29000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b2406eed0 con 0x7f5b2415e1f0 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b29000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5b240cd740 con 0x7f5b24072320 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b1be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5b24072de0 0x7f5b2406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51234/0 (socket says 172.21.15.28:51234) 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b1be00640 1 -- 172.21.15.28:0/3400733279 learned_addr learned my addr 172.21.15.28:0/3400733279 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21e00640 1 --2- 172.21.15.28:0/3400733279 >> v2:172.21.15.138:3300/0 conn(0x7f5b24072320 0x7f5b240cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21400640 1 -- 172.21.15.28:0/3400733279 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550373158 0 0) 0x7f5b24146140 con 0x7f5b24072de0 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21400640 1 -- 172.21.15.28:0/3400733279 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b08003520 con 0x7f5b24072de0 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21e00640 1 -- 172.21.15.28:0/3400733279 >> v1:172.21.15.138:6789/0 conn(0x7f5b2415e1f0 legacy=0x7f5b2406e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21e00640 1 -- 172.21.15.28:0/3400733279 >> v1:172.21.15.28:6789/0 conn(0x7f5b24072de0 legacy=0x7f5b2406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.259+0000 7f5b21e00640 1 -- 172.21.15.28:0/3400733279 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b240cda20 con 0x7f5b24072320 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.260+0000 7f5b21e00640 1 --2- 172.21.15.28:0/3400733279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24072320 0x7f5b240cce60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b140097a0 tx=0x7f5b1402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6424c9a1c5fbaa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b21400640 1 -- 172.21.15.28:0/3400733279 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14037070 con 0x7f5b24072320 2024-09-10T10:56:25.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b21400640 1 -- 172.21.15.28:0/3400733279 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b14032440 con 0x7f5b24072320 2024-09-10T10:56:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b21400640 1 -- 172.21.15.28:0/3400733279 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b140327f0 con 0x7f5b24072320 2024-09-10T10:56:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 -- 172.21.15.28:0/3400733279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24072320 msgr2=0x7f5b240cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 --2- 172.21.15.28:0/3400733279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24072320 0x7f5b240cce60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5b140097a0 tx=0x7f5b1402a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 -- 172.21.15.28:0/3400733279 shutdown_connections 2024-09-10T10:56:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 --2- 172.21.15.28:0/3400733279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24072320 0x7f5b240cce60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 -- 172.21.15.28:0/3400733279 >> 172.21.15.28:0/3400733279 conn(0x7f5b24067d70 msgr2=0x7f5b2406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 -- 172.21.15.28:0/3400733279 shutdown_connections 2024-09-10T10:56:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.261+0000 7f5b29000640 1 -- 172.21.15.28:0/3400733279 wait complete. 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.263+0000 7f5b29000640 1 Processor -- start 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.263+0000 7f5b29000640 1 -- start start 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b24072de0 0x7f5b2415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2415e1f0 0x7f5b2415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b240cd3a0 con 0x7f5b24072320 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2415c9e0 con 0x7f5b2415e1f0 2024-09-10T10:56:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b29000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b240cd520 con 0x7f5b24072de0 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34684/0 (socket says 172.21.15.28:34684) 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.264+0000 7f5b21e00640 1 -- 172.21.15.28:0/3675097984 learned_addr learned my addr 172.21.15.28:0/3675097984 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b1be00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b24072de0 0x7f5b2415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b22800640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2415e1f0 0x7f5b2415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 -- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b24072de0 msgr2=0x7f5b2415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b24072de0 0x7f5b2415b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 -- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2415e1f0 msgr2=0x7f5b2415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2415e1f0 0x7f5b2415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 -- 172.21.15.28:0/3675097984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b240cbbd0 con 0x7f5b24072320 2024-09-10T10:56:25.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b21e00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f5b14035040 tx=0x7f5b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14037050 con 0x7f5b24072320 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b14043070 con 0x7f5b24072320 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14006e10 con 0x7f5b24072320 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.265+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b240cbeb0 con 0x7f5b24072320 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.266+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b240cc470 con 0x7f5b24072320 2024-09-10T10:56:25.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.266+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b240cf4e0 con 0x7f5b24072320 2024-09-10T10:56:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.267+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5b14032810 con 0x7f5b24072320 2024-09-10T10:56:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.268+0000 7f5b1a000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 0x7f5b0006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.268+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f5b140ab810 con 0x7f5b24072320 2024-09-10T10:56:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.269+0000 7f5b1be00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 0x7f5b0006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.269+0000 7f5b1be00640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 0x7f5b0006e2f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5b240ad310 tx=0x7f5b0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.272+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b240cf4e0 con 0x7f5b24072320 2024-09-10T10:56:25.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.456+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5b2406bf90 con 0x7f5b0006be30 2024-09-10T10:56:25.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.456+0000 7f5b1a000640 1 -- 172.21.15.28:0/3675097984 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5b2406bf90 con 0x7f5b0006be30 2024-09-10T10:56:25.458 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 msgr2=0x7f5b0006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 0x7f5b0006e2f0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5b240ad310 tx=0x7f5b0c009040 comp rx=0 tx=0).stop 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 msgr2=0x7f5b240ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f5b14035040 tx=0x7f5b140046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 shutdown_connections 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f5b0006be30 0x7f5b0006e2f0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2415e1f0 0x7f5b2415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b24072de0 0x7f5b2415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 --2- 172.21.15.28:0/3675097984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b24072320 0x7f5b240ac930 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.458+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 >> 172.21.15.28:0/3675097984 conn(0x7f5b24067d70 msgr2=0x7f5b24143c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.459+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 shutdown_connections 2024-09-10T10:56:25.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.459+0000 7f5b29000640 1 -- 172.21.15.28:0/3675097984 wait complete. 2024-09-10T10:56:25.476 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T10:56:25.477 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.634+0000 7fd2da400640 1 Processor -- start 2024-09-10T10:56:25.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.635+0000 7fd2da400640 1 -- start start 2024-09-10T10:56:25.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2da400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd2d415e650 0x7fd2d40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2da400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2da400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d40d2790 con 0x7fd2d40cfae0 2024-09-10T10:56:25.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2da400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40b3e40 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2da400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40cff50 con 0x7fd2d415e650 2024-09-10T10:56:25.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2d2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2d2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34690/0 (socket says 172.21.15.28:34690) 2024-09-10T10:56:25.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2d2a00640 1 -- 172.21.15.28:0/774876713 learned_addr learned my addr 172.21.15.28:0/774876713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.636+0000 7fd2d3400640 1 --2- 172.21.15.28:0/774876713 >> v2:172.21.15.138:3300/0 conn(0x7fd2d415e650 0x7fd2d40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.637+0000 7fd2d2a00640 1 -- 172.21.15.28:0/774876713 >> v2:172.21.15.138:3300/0 conn(0x7fd2d415e650 msgr2=0x7fd2d40cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.637+0000 7fd2d2a00640 1 --2- 172.21.15.28:0/774876713 >> v2:172.21.15.138:3300/0 conn(0x7fd2d415e650 0x7fd2d40cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.637+0000 7fd2d2a00640 1 -- 172.21.15.28:0/774876713 >> v1:172.21.15.138:6789/0 conn(0x7fd2d40cfae0 legacy=0x7fd2d41622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.637+0000 7fd2d2a00640 1 -- 172.21.15.28:0/774876713 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40d2a30 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2d2a00640 1 --2- 172.21.15.28:0/774876713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fd2d408ca10 tx=0x7fd2c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3607b5ad9033075f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2d2000640 1 -- 172.21.15.28:0/774876713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0037070 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2d2000640 1 -- 172.21.15.28:0/774876713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c0032440 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2d2000640 1 -- 172.21.15.28:0/774876713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0032810 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2da400640 1 -- 172.21.15.28:0/774876713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 msgr2=0x7fd2d40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.638+0000 7fd2da400640 1 --2- 172.21.15.28:0/774876713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7fd2d408ca10 tx=0x7fd2c002a270 comp rx=0 tx=0).stop 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 -- 172.21.15.28:0/774876713 shutdown_connections 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 --2- 172.21.15.28:0/774876713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40cf5a0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 --2- 172.21.15.28:0/774876713 >> v2:172.21.15.138:3300/0 conn(0x7fd2d415e650 0x7fd2d40cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 -- 172.21.15.28:0/774876713 >> 172.21.15.28:0/774876713 conn(0x7fd2d408bfc0 msgr2=0x7fd2d40d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 -- 172.21.15.28:0/774876713 shutdown_connections 2024-09-10T10:56:25.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.639+0000 7fd2da400640 1 -- 172.21.15.28:0/774876713 wait complete. 2024-09-10T10:56:25.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.640+0000 7fd2da400640 1 Processor -- start 2024-09-10T10:56:25.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.640+0000 7fd2da400640 1 -- start start 2024-09-10T10:56:25.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2d40cfae0 0x7fd2d415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d415e650 0x7fd2d4154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40d32d0 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40d2ff0 con 0x7fd2d415e650 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2da400640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40d5040 con 0x7fd2d40cfae0 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.641+0000 7fd2d3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40b9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34706/0 (socket says 172.21.15.28:34706) 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 -- 172.21.15.28:0/4029930563 learned_addr learned my addr 172.21.15.28:0/4029930563 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d2a00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2d40cfae0 0x7fd2d415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3e00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d415e650 0x7fd2d4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 -- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2d40cfae0 msgr2=0x7fd2d415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd2d40cfae0 0x7fd2d415ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 -- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d415e650 msgr2=0x7fd2d4154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d415e650 0x7fd2d4154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40ca530 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.642+0000 7fd2d3400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40b9610 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400e9b0 tx=0x7fd2c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.643+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4017070 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.643+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c400c9a0 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.643+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400cd90 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.643+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d40c8a00 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.643+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d40c9d30 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.644+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2d415ea30 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.645+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd2c401a070 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.645+0000 7fd2cb400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2b406be30 0x7fd2b406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fd2c408ad60 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2cb400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7fd2b4072370 0x7fd2b40747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2b4074e80 con 0x7fd2b4072370 2024-09-10T10:56:25.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7fd2c408d380 con 0x7fd2d40cf0f0 2024-09-10T10:56:25.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2d2a00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2b406be30 0x7fd2b406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.646+0000 7fd2d3e00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7fd2b4072370 0x7fd2b40747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.647+0000 7fd2d2a00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2b406be30 0x7fd2b406e2f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.647+0000 7fd2d3e00640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7fd2b4072370 0x7fd2b40747b0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.648+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== osd.2 v2:172.21.15.28:6824/1014290416 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2b4074e80 con 0x7fd2b4072370 2024-09-10T10:56:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.661+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 --> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd2d40b6c50 con 0x7fd2b4072370 2024-09-10T10:56:25.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.661+0000 7fd2cb400640 1 -- 172.21.15.28:0/4029930563 <== osd.2 v2:172.21.15.28:6824/1014290416 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd2d40b6c50 con 0x7fd2b4072370 2024-09-10T10:56:25.663 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:25.663 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:25.664 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7fd2b4072370 msgr2=0x7fd2b40747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:6824/1014290416,v1:172.21.15.28:6825/1014290416] conn(0x7fd2b4072370 0x7fd2b40747b0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2b406be30 msgr2=0x7fd2b406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fd2b406be30 0x7fd2b406e2f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0034000 tx=0x7fd2c002e000 comp rx=0 tx=0).stop 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 msgr2=0x7fd2d40b9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 --2- 172.21.15.28:0/4029930563 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2d40cf0f0 0x7fd2d40b9610 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400e9b0 tx=0x7fd2c400ee70 comp rx=0 tx=0).stop 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2d3e00640 1 -- 172.21.15.28:0/4029930563 reap_dead start 2024-09-10T10:56:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 shutdown_connections 2024-09-10T10:56:25.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 >> 172.21.15.28:0/4029930563 conn(0x7fd2d408bfc0 msgr2=0x7fd2d40b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 shutdown_connections 2024-09-10T10:56:25.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.662+0000 7fd2da400640 1 -- 172.21.15.28:0/4029930563 wait complete. 2024-09-10T10:56:25.679 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T10:56:25.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.834+0000 7f4cf3e00640 1 Processor -- start 2024-09-10T10:56:25.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.834+0000 7f4cf3e00640 1 -- start start 2024-09-10T10:56:25.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cec0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf3e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4cec0cf330 0x7f4cec06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf3e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cec0cf980 con 0x7f4cec0d1420 2024-09-10T10:56:25.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf3e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4cec161e00 con 0x7f4cec0cf330 2024-09-10T10:56:25.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec06d050 con 0x7f4cec161a20 2024-09-10T10:56:25.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.835+0000 7f4cf0a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4cec0d1420 0x7f4cec0cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51246/0 (socket says 172.21.15.28:51246) 2024-09-10T10:56:25.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cf0a00640 1 -- 172.21.15.28:0/1631414105 learned_addr learned my addr 172.21.15.28:0/1631414105 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/1631414105 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0cf330 0x7f4cec06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cf1400640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cec0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3609068169 0 0) 0x7f4cec0cf980 con 0x7f4cec0d1420 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4cd4003520 con 0x7f4cec0d1420 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 317243271 0 0) 0x7f4cd4003520 con 0x7f4cec0d1420 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 msgr2=0x7f4cec0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cec0d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0cf330 msgr2=0x7f4cec06d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 --2- 172.21.15.28:0/1631414105 >> v2:172.21.15.138:3300/0 conn(0x7f4cec0cf330 0x7f4cec06d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec0d1800 con 0x7f4cec0d1420 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7f4cd8001670 con 0x7f4cec0d1420 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 >> v1:172.21.15.28:6789/0 conn(0x7f4cec0d1420 legacy=0x7f4cec0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cd40039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cd4003f50 0x7f4cd4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4004d40 0x7f4cd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec161e00 con 0x7f4cd4003f50 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec06d050 con 0x7f4cd4004d40 2024-09-10T10:56:25.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.836+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd400b9c0 con 0x7f4cec161a20 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cd4003f50 0x7f4cd4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf1400640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cd40039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4004d40 0x7f4cd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 -- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 msgr2=0x7f4cd40039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec161a20 0x7f4cd40039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 -- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4004d40 msgr2=0x7f4cd40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4004d40 0x7f4cd40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf0a00640 1 -- 172.21.15.28:0/1631414105 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd4003520 con 0x7f4cd4003f50 2024-09-10T10:56:25.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.837+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4004d40 0x7f4cd40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:25.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.838+0000 7f4cf0a00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cd4003f50 0x7f4cd4004800 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c240 tx=0x7f4cd4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2699e96feb2d707 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.838+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8007d50 con 0x7f4cd4003f50 2024-09-10T10:56:25.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.838+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cd802e070 con 0x7f4cd4003f50 2024-09-10T10:56:25.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.838+0000 7f4cebe00640 1 -- 172.21.15.28:0/1631414105 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8005d70 con 0x7f4cd4003f50 2024-09-10T10:56:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 -- 172.21.15.28:0/1631414105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cd4003f50 msgr2=0x7f4cd4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/1631414105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cd4003f50 0x7f4cd4004800 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800c240 tx=0x7f4cd4005830 comp rx=0 tx=0).stop 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf1e00640 1 -- 172.21.15.28:0/1631414105 reap_dead start 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 -- 172.21.15.28:0/1631414105 shutdown_connections 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 -- 172.21.15.28:0/1631414105 >> 172.21.15.28:0/1631414105 conn(0x7f4cec09d850 msgr2=0x7f4cec08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 -- 172.21.15.28:0/1631414105 shutdown_connections 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.839+0000 7f4cf3e00640 1 -- 172.21.15.28:0/1631414105 wait complete. 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.841+0000 7f4cf3e00640 1 Processor -- start 2024-09-10T10:56:25.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.841+0000 7f4cf3e00640 1 -- start start 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4003f50 0x7f4cec14f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec0b77f0 0x7f4cec09e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec06d2f0 con 0x7f4cec0b7c70 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec0cff00 con 0x7f4cd4003f50 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf3e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec0b1040 con 0x7f4cec0b77f0 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34724/0 (socket says 172.21.15.28:34724) 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 -- 172.21.15.28:0/3525306736 learned_addr learned my addr 172.21.15.28:0/3525306736 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf0a00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec0b77f0 0x7f4cec09e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1400640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4003f50 0x7f4cec14f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 -- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec0b77f0 msgr2=0x7f4cec09e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec0b77f0 0x7f4cec09e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 -- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4003f50 msgr2=0x7f4cec14f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4003f50 0x7f4cec14f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:25.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.842+0000 7f4cf1e00640 1 -- 172.21.15.28:0/3525306736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec0a0260 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4cf1e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f4cec08d720 tx=0x7f4ce4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4010d80 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce4003990 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4003d80 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec0a0540 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.843+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec0a0b00 con 0x7f4cec0b7c70 2024-09-10T10:56:25.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.844+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cec068e40 con 0x7f4cec0b7c70 2024-09-10T10:56:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.845+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4ce4005d10 con 0x7f4cec0b7c70 2024-09-10T10:56:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.846+0000 7f4ce9600640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 0x7f4ccc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.846+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f4ce408b530 con 0x7f4cec0b7c70 2024-09-10T10:56:25.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.846+0000 7f4cf1400640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 0x7f4ccc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.847+0000 7f4cf1400640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 0x7f4ccc06e320 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009a20 tx=0x7f4ce0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:25.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:25.850+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4cec068e40 con 0x7f4cec0b7c70 2024-09-10T10:56:26.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.033+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f4cec0a2170 con 0x7f4ccc06be60 2024-09-10T10:56:26.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.034+0000 7f4ce9600640 1 -- 172.21.15.28:0/3525306736 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4cec0a2170 con 0x7f4ccc06be60 2024-09-10T10:56:26.036 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.036+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 msgr2=0x7f4ccc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.036+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 0x7f4ccc06e320 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009a20 tx=0x7f4ce0008040 comp rx=0 tx=0).stop 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.036+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 msgr2=0x7f4cec1f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.036+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f4cec08d720 tx=0x7f4ce4010590 comp rx=0 tx=0).stop 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 shutdown_connections 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f4ccc06be60 0x7f4ccc06e320 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4cec0b7c70 0x7f4cec1f7c40 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4cec0b77f0 0x7f4cec09e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 --2- 172.21.15.28:0/3525306736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4cd4003f50 0x7f4cec14f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 >> 172.21.15.28:0/3525306736 conn(0x7f4cec09d850 msgr2=0x7f4cec160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 shutdown_connections 2024-09-10T10:56:26.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.037+0000 7f4cf3e00640 1 -- 172.21.15.28:0/3525306736 wait complete. 2024-09-10T10:56:26.055 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T10:56:26.056 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:26.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.213+0000 7f50be200640 1 Processor -- start 2024-09-10T10:56:26.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.214+0000 7f50be200640 1 -- start start 2024-09-10T10:56:26.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50be200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f50b8072070 0x7f50b806ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50be200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b80ce7f0 0x7f50b815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50be200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b806f130 con 0x7f50b815c430 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50be200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f50b806f410 con 0x7f50b8072070 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50be200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50b815c9e0 con 0x7f50b80ce7f0 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50b7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f50b8072070 0x7f50b806ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50afe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f50b815c430 0x7f50b8072450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51248/0 (socket says 172.21.15.28:51248) 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50afe00640 1 -- 172.21.15.28:0/2845532852 learned_addr learned my addr 172.21.15.28:0/2845532852 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50b7e00640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b80ce7f0 0x7f50b815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.215+0000 7f50b7400640 1 -- 172.21.15.28:0/2845532852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b80ce7f0 msgr2=0x7f50b815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:26.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.216+0000 7f50b7400640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b80ce7f0 0x7f50b815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.216+0000 7f50b7400640 1 -- 172.21.15.28:0/2845532852 >> v1:172.21.15.28:6789/0 conn(0x7f50b815c430 legacy=0x7f50b8072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.216+0000 7f50b7400640 1 -- 172.21.15.28:0/2845532852 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b815ccc0 con 0x7f50b8072070 2024-09-10T10:56:26.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50b7400640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b806ea60 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097a0 tx=0x7f50a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dca6fd133bef475 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50b6a00640 1 -- 172.21.15.28:0/2845532852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8037070 con 0x7f50b8072070 2024-09-10T10:56:26.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50b6a00640 1 -- 172.21.15.28:0/2845532852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a8032440 con 0x7f50b8072070 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50b6a00640 1 -- 172.21.15.28:0/2845532852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8032810 con 0x7f50b8072070 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 -- 172.21.15.28:0/2845532852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 msgr2=0x7f50b806ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b806ea60 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f50a80097a0 tx=0x7f50a802a180 comp rx=0 tx=0).stop 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 -- 172.21.15.28:0/2845532852 shutdown_connections 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b80ce7f0 0x7f50b815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 --2- 172.21.15.28:0/2845532852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b806ea60 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.217+0000 7f50be200640 1 -- 172.21.15.28:0/2845532852 >> 172.21.15.28:0/2845532852 conn(0x7f50b8067d70 msgr2=0x7f50b806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.218+0000 7f50be200640 1 -- 172.21.15.28:0/2845532852 shutdown_connections 2024-09-10T10:56:26.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.218+0000 7f50be200640 1 -- 172.21.15.28:0/2845532852 wait complete. 2024-09-10T10:56:26.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.219+0000 7f50be200640 1 Processor -- start 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.219+0000 7f50be200640 1 -- start start 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50b80ce7f0 0x7f50b814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b815c430 0x7f50b8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f50b815da00 con 0x7f50b8072070 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50b80cec00 con 0x7f50b80ce7f0 2024-09-10T10:56:26.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50be200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50b815db80 con 0x7f50b815c430 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.220+0000 7f50b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b814e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34744/0 (socket says 172.21.15.28:34744) 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 -- 172.21.15.28:0/349555684 learned_addr learned my addr 172.21.15.28:0/349555684 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 -- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b815c430 msgr2=0x7f50b8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50b815c430 0x7f50b8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 -- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50b80ce7f0 msgr2=0x7f50b814f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50b80ce7f0 0x7f50b814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b81548b0 con 0x7f50b8072070 2024-09-10T10:56:26.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50b7400640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b814e300 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f50a8035040 tx=0x7f50a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8037040 con 0x7f50b8072070 2024-09-10T10:56:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a8030770 con 0x7f50b8072070 2024-09-10T10:56:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a8030b80 con 0x7f50b8072070 2024-09-10T10:56:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.221+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b8154b90 con 0x7f50b8072070 2024-09-10T10:56:26.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.222+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b80add90 con 0x7f50b8072070 2024-09-10T10:56:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.222+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f50b80ad8a0 con 0x7f50b8072070 2024-09-10T10:56:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.223+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50a8044020 con 0x7f50b8072070 2024-09-10T10:56:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.224+0000 7f50af400640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f509406be30 0x7f509406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.224+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f50a80ab320 con 0x7f50b8072070 2024-09-10T10:56:26.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50af400640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f5094072370 0x7f50940747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50a8008eb0 con 0x7f5094072370 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f50a80b2050 con 0x7f50b8072070 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50afe00640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f509406be30 0x7f509406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50b7e00640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f5094072370 0x7f50940747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.225+0000 7f50afe00640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f509406be30 0x7f509406e2f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f60 tx=0x7f50a0007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.226+0000 7f50b7e00640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f5094072370 0x7f50940747b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.226+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== osd.3 v2:172.21.15.28:6816/2614769734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50a8008eb0 con 0x7f5094072370 2024-09-10T10:56:26.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.239+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 --> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f50b815d4c0 con 0x7f5094072370 2024-09-10T10:56:26.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50af400640 1 -- 172.21.15.28:0/349555684 <== osd.3 v2:172.21.15.28:6816/2614769734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f50b815d4c0 con 0x7f5094072370 2024-09-10T10:56:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:26.242 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:26.242 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:26.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f5094072370 msgr2=0x7f50940747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:6816/2614769734,v1:172.21.15.28:6817/2614769734] conn(0x7f5094072370 0x7f50940747b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f509406be30 msgr2=0x7f509406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f509406be30 0x7f509406e2f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f50a0000f60 tx=0x7f50a0007a80 comp rx=0 tx=0).stop 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 msgr2=0x7f50b814e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 --2- 172.21.15.28:0/349555684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f50b8072070 0x7f50b814e300 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f50a8035040 tx=0x7f50a80046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50b7e00640 1 -- 172.21.15.28:0/349555684 reap_dead start 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 shutdown_connections 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 >> 172.21.15.28:0/349555684 conn(0x7f50b8067d70 msgr2=0x7f50b806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 shutdown_connections 2024-09-10T10:56:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.240+0000 7f50be200640 1 -- 172.21.15.28:0/349555684 wait complete. 2024-09-10T10:56:26.260 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T10:56:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.415+0000 7fc0d0800640 1 Processor -- start 2024-09-10T10:56:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.416+0000 7fc0d0800640 1 -- start start 2024-09-10T10:56:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0d0800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072320 0x7fc0c80cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0d0800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0c8072de0 0x7fc0c806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0d0800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0c8146140 con 0x7fc0c8072700 2024-09-10T10:56:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0d0800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc0c80cd740 con 0x7fc0c8072de0 2024-09-10T10:56:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0d0800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c80cda20 con 0x7fc0c8072320 2024-09-10T10:56:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0cd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0c8072de0 0x7fc0c806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0cd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0c8072de0 0x7fc0c806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34746/0 (socket says 172.21.15.28:34746) 2024-09-10T10:56:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0cd400640 1 -- 172.21.15.28:0/2936159937 learned_addr learned my addr 172.21.15.28:0/2936159937 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.417+0000 7fc0cde00640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072320 0x7fc0c80cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.418+0000 7fc0cd400640 1 -- 172.21.15.28:0/2936159937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072320 msgr2=0x7fc0c80cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.418+0000 7fc0cd400640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072320 0x7fc0c80cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.418+0000 7fc0cd400640 1 -- 172.21.15.28:0/2936159937 >> v1:172.21.15.138:6789/0 conn(0x7fc0c8072700 legacy=0x7fc0c806e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.418+0000 7fc0cd400640 1 -- 172.21.15.28:0/2936159937 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c80cdd00 con 0x7fc0c8072de0 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.418+0000 7fc0cca00640 1 -- 172.21.15.28:0/2936159937 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200418774 0 0) 0x7fc0c8146140 con 0x7fc0c8072700 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0cd400640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072de0 0x7fc0c806e0b0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009870 tx=0x7fc0b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3d5159be5c09b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0cca00640 1 -- 172.21.15.28:0/2936159937 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8037070 con 0x7fc0c8072de0 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0cca00640 1 -- 172.21.15.28:0/2936159937 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0b8032440 con 0x7fc0c8072de0 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0cca00640 1 -- 172.21.15.28:0/2936159937 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8032810 con 0x7fc0c8072de0 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0d0800640 1 -- 172.21.15.28:0/2936159937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072de0 msgr2=0x7fc0c806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.419+0000 7fc0d0800640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072de0 0x7fc0c806e0b0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009870 tx=0x7fc0b802a1c0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 -- 172.21.15.28:0/2936159937 shutdown_connections 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072de0 0x7fc0c806e0b0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 --2- 172.21.15.28:0/2936159937 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072320 0x7fc0c80cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 -- 172.21.15.28:0/2936159937 >> 172.21.15.28:0/2936159937 conn(0x7fc0c8067d70 msgr2=0x7fc0c806bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 -- 172.21.15.28:0/2936159937 shutdown_connections 2024-09-10T10:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.420+0000 7fc0d0800640 1 -- 172.21.15.28:0/2936159937 wait complete. 2024-09-10T10:56:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.421+0000 7fc0d0800640 1 Processor -- start 2024-09-10T10:56:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.421+0000 7fc0d0800640 1 -- start start 2024-09-10T10:56:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c8072700 0x7fc0c81e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072de0 0x7fc0c81e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80cd520 con 0x7fc0c8072320 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c80cd3a0 con 0x7fc0c8072700 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0d0800640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c806ed00 con 0x7fc0c8072de0 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0cde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.422+0000 7fc0cde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34758/0 (socket says 172.21.15.28:34758) 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 -- 172.21.15.28:0/4238361610 learned_addr learned my addr 172.21.15.28:0/4238361610 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cd400640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c8072700 0x7fc0c81e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0ce800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072de0 0x7fc0c81e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 -- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072de0 msgr2=0x7fc0c81e9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072de0 0x7fc0c81e9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 -- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c8072700 msgr2=0x7fc0c81e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c8072700 0x7fc0c81e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 -- 172.21.15.28:0/4238361610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c8131cd0 con 0x7fc0c8072320 2024-09-10T10:56:26.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0cde00640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00e8a0 tx=0x7fc0bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc017070 con 0x7fc0c8072320 2024-09-10T10:56:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0bc00c940 con 0x7fc0c8072320 2024-09-10T10:56:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc00cd30 con 0x7fc0c8072320 2024-09-10T10:56:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.423+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c8131fb0 con 0x7fc0c8072320 2024-09-10T10:56:26.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.424+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c8132570 con 0x7fc0c8072320 2024-09-10T10:56:26.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.425+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c80cce60 con 0x7fc0c8072320 2024-09-10T10:56:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.425+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0bc01a070 con 0x7fc0c8072320 2024-09-10T10:56:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.427+0000 7fc0c2000640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 0x7fc0ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.427+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fc0bc08af80 con 0x7fc0c8072320 2024-09-10T10:56:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.427+0000 7fc0cd400640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 0x7fc0ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.428+0000 7fc0cd400640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 0x7fc0ac06e2f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.431+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0bc059360 con 0x7fc0c8072320 2024-09-10T10:56:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.618+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc0c8131a80 con 0x7fc0ac06be30 2024-09-10T10:56:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.619+0000 7fc0c2000640 1 -- 172.21.15.28:0/4238361610 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc0c8131a80 con 0x7fc0ac06be30 2024-09-10T10:56:26.621 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T10:56:26.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 msgr2=0x7fc0ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 0x7fc0ac06e2f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8034000 tx=0x7fc0b802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:26.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 msgr2=0x7fc0c81e6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00e8a0 tx=0x7fc0bc00ed60 comp rx=0 tx=0).stop 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 shutdown_connections 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fc0ac06be30 0x7fc0ac06e2f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc0c8072de0 0x7fc0c81e9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c8072700 0x7fc0c81e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 --2- 172.21.15.28:0/4238361610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0c8072320 0x7fc0c81e6e90 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 >> 172.21.15.28:0/4238361610 conn(0x7fc0c8067d70 msgr2=0x7fc0c80b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 shutdown_connections 2024-09-10T10:56:26.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.621+0000 7fc0d0800640 1 -- 172.21.15.28:0/4238361610 wait complete. 2024-09-10T10:56:26.638 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T10:56:26.639 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:26.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.798+0000 7fb09d000640 1 Processor -- start 2024-09-10T10:56:26.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.798+0000 7fb09d000640 1 -- start start 2024-09-10T10:56:26.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.799+0000 7fb09d000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0980cf590 0x7fb098072300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.799+0000 7fb09d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb098072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.799+0000 7fb09d000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09815b660 con 0x7fb0980aee90 2024-09-10T10:56:26.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.799+0000 7fb09d000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb098158e50 con 0x7fb0980cf590 2024-09-10T10:56:26.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.799+0000 7fb09d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb09815c270 con 0x7fb0980cfa00 2024-09-10T10:56:26.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb096a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb0980cf590 0x7fb098072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb097400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb0980aee90 0x7fb0980af270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.28:56442/0 (socket says 172.21.15.28:56442) 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb097400640 1 -- 172.21.15.28:0/4083061513 learned_addr learned my addr 172.21.15.28:0/4083061513 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb097e00640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb098072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb096a00640 1 -- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 msgr2=0x7fb098072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb096a00640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb098072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.800+0000 7fb096a00640 1 -- 172.21.15.28:0/4083061513 >> v1:172.21.15.138:6789/0 conn(0x7fb0980aee90 legacy=0x7fb0980af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.801+0000 7fb096a00640 1 -- 172.21.15.28:0/4083061513 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09815b940 con 0x7fb0980cf590 2024-09-10T10:56:26.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb096a00640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb098072300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb0840097a0 tx=0x7fb08402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318de9eb71b9d6b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb096000640 1 -- 172.21.15.28:0/4083061513 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084037070 con 0x7fb0980cf590 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb096000640 1 -- 172.21.15.28:0/4083061513 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb08402ae50 con 0x7fb0980cf590 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb096000640 1 -- 172.21.15.28:0/4083061513 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb084032660 con 0x7fb0980cf590 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb09d000640 1 -- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 msgr2=0x7fb098072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.802+0000 7fb09d000640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb098072300 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb0840097a0 tx=0x7fb08402a100 comp rx=0 tx=0).stop 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 -- 172.21.15.28:0/4083061513 shutdown_connections 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb098072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 --2- 172.21.15.28:0/4083061513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb098072300 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 -- 172.21.15.28:0/4083061513 >> 172.21.15.28:0/4083061513 conn(0x7fb09806bdf0 msgr2=0x7fb09806c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 -- 172.21.15.28:0/4083061513 shutdown_connections 2024-09-10T10:56:26.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.803+0000 7fb09d000640 1 -- 172.21.15.28:0/4083061513 wait complete. 2024-09-10T10:56:26.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.804+0000 7fb09d000640 1 Processor -- start 2024-09-10T10:56:26.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.804+0000 7fb09d000640 1 -- start start 2024-09-10T10:56:26.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 0x7fb09814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb0981531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb0981536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb098073ed0 con 0x7fb0980aee90 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09815bf00 con 0x7fb0980cf590 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb09d000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0980b4ad0 con 0x7fb0980cfa00 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.805+0000 7fb097400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 0x7fb09814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 0x7fb09814dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34766/0 (socket says 172.21.15.28:34766) 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 -- 172.21.15.28:0/3733239306 learned_addr learned my addr 172.21.15.28:0/3733239306 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb096a00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb0981531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097e00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb0981536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 -- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 msgr2=0x7fb0981536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb0981536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 -- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 msgr2=0x7fb0981531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb0981531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb098152100 con 0x7fb0980aee90 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097e00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0980cfa00 0x7fb0981536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb096a00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0980cf590 0x7fb0981531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:26.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb097400640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 0x7fb09814dc50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c430 tx=0x7fb08800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.806+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088017070 con 0x7fb0980aee90 2024-09-10T10:56:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.807+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb088005ce0 con 0x7fb0980aee90 2024-09-10T10:56:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.807+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880072f0 con 0x7fb0980aee90 2024-09-10T10:56:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.807+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0981523e0 con 0x7fb0980aee90 2024-09-10T10:56:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.807+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0981529a0 con 0x7fb0980aee90 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.808+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0980af270 con 0x7fb0980aee90 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.809+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb08801a070 con 0x7fb0980aee90 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.809+0000 7fb08f400640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806be30 0x7fb07806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fb08808a1e0 con 0x7fb0980aee90 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb08f400640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7fb078072370 0x7fb0780747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb078074e80 con 0x7fb078072370 2024-09-10T10:56:26.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7fb088091050 con 0x7fb0980aee90 2024-09-10T10:56:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb096a00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806be30 0x7fb07806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.810+0000 7fb097e00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7fb078072370 0x7fb0780747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.811+0000 7fb096a00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806be30 0x7fb07806e2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb084034000 tx=0x7fb08402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.811+0000 7fb097e00640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7fb078072370 0x7fb0780747b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.812+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== osd.4 v2:172.21.15.138:6810/1083898337 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb078074e80 con 0x7fb078072370 2024-09-10T10:56:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.825+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 --> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb0980ad870 con 0x7fb078072370 2024-09-10T10:56:26.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.826+0000 7fb08f400640 1 -- 172.21.15.28:0/3733239306 <== osd.4 v2:172.21.15.138:6810/1083898337 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb0980ad870 con 0x7fb078072370 2024-09-10T10:56:26.829 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:26.829 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:26.829 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7fb078072370 msgr2=0x7fb0780747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6810/1083898337,v1:172.21.15.138:6811/1083898337] conn(0x7fb078072370 0x7fb0780747b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806be30 msgr2=0x7fb07806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fb07806be30 0x7fb07806e2f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb084034000 tx=0x7fb08402e000 comp rx=0 tx=0).stop 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 msgr2=0x7fb09814dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 --2- 172.21.15.28:0/3733239306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0980aee90 0x7fb09814dc50 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7fb08800c430 tx=0x7fb08800c8f0 comp rx=0 tx=0).stop 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb097e00640 1 -- 172.21.15.28:0/3733239306 reap_dead start 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 shutdown_connections 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 >> 172.21.15.28:0/3733239306 conn(0x7fb09806bdf0 msgr2=0x7fb09806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 shutdown_connections 2024-09-10T10:56:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.827+0000 7fb09d000640 1 -- 172.21.15.28:0/3733239306 wait complete. 2024-09-10T10:56:26.843 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T10:56:26.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.997+0000 7f2c3c000640 1 Processor -- start 2024-09-10T10:56:27.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.998+0000 7f2c3c000640 1 -- start start 2024-09-10T10:56:27.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3c000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c34147e90 0x7f2c340720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3c000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2c340725f0 0x7f2c340aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3c000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c34072a00 con 0x7f2c34147230 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3c000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2c340b34d0 con 0x7f2c34147e90 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3c000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2c340ae9e0 con 0x7f2c340725f0 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c39600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2c34147230 0x7f2c3406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51258/0 (socket says 172.21.15.28:51258) 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c39600640 1 -- 172.21.15.28:0/2445980445 learned_addr learned my addr 172.21.15.28:0/2445980445 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c38c00640 1 --2- 172.21.15.28:0/2445980445 >> v2:172.21.15.28:3300/0 conn(0x7f2c34147e90 0x7f2c340720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c3a000640 1 --2- 172.21.15.28:0/2445980445 >> v2:172.21.15.138:3300/0 conn(0x7f2c340725f0 0x7f2c340aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:26.999+0000 7f2c33e00640 1 -- 172.21.15.28:0/2445980445 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 295259157 0 0) 0x7f2c28001110 con 0x7f2c34147230 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.000+0000 7f2c38c00640 1 -- 172.21.15.28:0/2445980445 >> v2:172.21.15.138:3300/0 conn(0x7f2c340725f0 msgr2=0x7f2c340aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.000+0000 7f2c38c00640 1 --2- 172.21.15.28:0/2445980445 >> v2:172.21.15.138:3300/0 conn(0x7f2c340725f0 0x7f2c340aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.000+0000 7f2c38c00640 1 -- 172.21.15.28:0/2445980445 >> v1:172.21.15.28:6789/0 conn(0x7f2c34147230 legacy=0x7f2c3406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.000+0000 7f2c38c00640 1 -- 172.21.15.28:0/2445980445 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3406f480 con 0x7f2c34147e90 2024-09-10T10:56:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c38c00640 1 --2- 172.21.15.28:0/2445980445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340720b0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2c34148320 tx=0x7f2c2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a33330045e8695c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c33e00640 1 -- 172.21.15.28:0/2445980445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20037070 con 0x7f2c34147e90 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c33e00640 1 -- 172.21.15.28:0/2445980445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c20032440 con 0x7f2c34147e90 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c33e00640 1 -- 172.21.15.28:0/2445980445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20032810 con 0x7f2c34147e90 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 -- 172.21.15.28:0/2445980445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 msgr2=0x7f2c340720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 --2- 172.21.15.28:0/2445980445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340720b0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f2c34148320 tx=0x7f2c2002a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 -- 172.21.15.28:0/2445980445 shutdown_connections 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 --2- 172.21.15.28:0/2445980445 >> v2:172.21.15.138:3300/0 conn(0x7f2c340725f0 0x7f2c340aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 --2- 172.21.15.28:0/2445980445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340720b0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 -- 172.21.15.28:0/2445980445 >> 172.21.15.28:0/2445980445 conn(0x7f2c34067d70 msgr2=0x7f2c3406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.001+0000 7f2c3c000640 1 -- 172.21.15.28:0/2445980445 shutdown_connections 2024-09-10T10:56:27.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.002+0000 7f2c3c000640 1 -- 172.21.15.28:0/2445980445 wait complete. 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.003+0000 7f2c3c000640 1 Processor -- start 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.003+0000 7f2c3c000640 1 -- start start 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 0x7f2c341419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 0x7f2c341f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406eec0 con 0x7f2c34147e90 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3406ebe0 con 0x7f2c340725f0 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3c000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3406f1a0 con 0x7f2c34147230 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34790/0 (socket says 172.21.15.28:34790) 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c3a000640 1 -- 172.21.15.28:0/1537093874 learned_addr learned my addr 172.21.15.28:0/1537093874 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.004+0000 7f2c39600640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 0x7f2c341419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c38c00640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 0x7f2c341f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 -- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 msgr2=0x7f2c341f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 0x7f2c341f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 -- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 msgr2=0x7f2c341419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 0x7f2c341419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 -- 172.21.15.28:0/1537093874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c341612d0 con 0x7f2c34147e90 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c38c00640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 0x7f2c341f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c39600640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 0x7f2c341419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3a000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00efc0 tx=0x7f2c2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c018020 con 0x7f2c34147e90 2024-09-10T10:56:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c014480 con 0x7f2c34147e90 2024-09-10T10:56:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c014870 con 0x7f2c34147e90 2024-09-10T10:56:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.005+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c340c6260 con 0x7f2c34147e90 2024-09-10T10:56:27.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.006+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c340c6820 con 0x7f2c34147e90 2024-09-10T10:56:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.006+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c34147610 con 0x7f2c34147e90 2024-09-10T10:56:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.007+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2c2c005d10 con 0x7f2c34147e90 2024-09-10T10:56:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.008+0000 7f2c31600640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 0x7f2c1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.008+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f2c2c08b940 con 0x7f2c34147e90 2024-09-10T10:56:27.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.009+0000 7f2c39600640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 0x7f2c1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.009+0000 7f2c39600640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 0x7f2c1406e320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000f60 tx=0x7f2c28007a10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.013+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c2c022070 con 0x7f2c34147e90 2024-09-10T10:56:27.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.195+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f2c341f57d0 con 0x7f2c1406be60 2024-09-10T10:56:27.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.197+0000 7f2c31600640 1 -- 172.21.15.28:0/1537093874 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2c341f57d0 con 0x7f2c1406be60 2024-09-10T10:56:27.198 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T10:56:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 msgr2=0x7f2c1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 0x7f2c1406e320 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000f60 tx=0x7f2c28007a10 comp rx=0 tx=0).stop 2024-09-10T10:56:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 msgr2=0x7f2c340c87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00efc0 tx=0x7f2c2c00c490 comp rx=0 tx=0).stop 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 shutdown_connections 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2c1406be60 0x7f2c1406e320 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c34147e90 0x7f2c340c87c0 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c34147230 0x7f2c341f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 --2- 172.21.15.28:0/1537093874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c340725f0 0x7f2c341419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 >> 172.21.15.28:0/1537093874 conn(0x7f2c34067d70 msgr2=0x7f2c34145420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 shutdown_connections 2024-09-10T10:56:27.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.198+0000 7f2c3c000640 1 -- 172.21.15.28:0/1537093874 wait complete. 2024-09-10T10:56:27.216 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T10:56:27.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.374+0000 7f2eb3200640 1 Processor -- start 2024-09-10T10:56:27.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.374+0000 7f2eb3200640 1 -- start start 2024-09-10T10:56:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb3200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2eb40d2950 0x7f2eb40d4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb40b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb40b1030 con 0x7f2eb40d2d30 2024-09-10T10:56:27.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb3200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2eb40b1c00 con 0x7f2eb40d2950 2024-09-10T10:56:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb40b12d0 con 0x7f2eb40b56a0 2024-09-10T10:56:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb2800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2eb40d2d30 0x7f2eb40b4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51268/0 (socket says 172.21.15.28:51268) 2024-09-10T10:56:27.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.375+0000 7f2eb2800640 1 -- 172.21.15.28:0/159675894 learned_addr learned my addr 172.21.15.28:0/159675894 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1400640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb40b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/159675894 >> v2:172.21.15.138:3300/0 conn(0x7f2eb40d2950 0x7f2eb40d4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb0a00640 1 -- 172.21.15.28:0/159675894 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099279035 0 0) 0x7f2eb40b1030 con 0x7f2eb40d2d30 2024-09-10T10:56:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb0a00640 1 -- 172.21.15.28:0/159675894 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e98003520 con 0x7f2eb40d2d30 2024-09-10T10:56:27.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1e00640 1 -- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 msgr2=0x7f2eb40b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb40b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1e00640 1 -- 172.21.15.28:0/159675894 >> v1:172.21.15.28:6789/0 conn(0x7f2eb40d2d30 legacy=0x7f2eb40b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.376+0000 7f2eb1e00640 1 -- 172.21.15.28:0/159675894 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb40b15b0 con 0x7f2eb40d2950 2024-09-10T10:56:27.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.377+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2950 0x7f2eb40d4c40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40097a0 tx=0x7f2ea402a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5b6047aab2e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb0a00640 1 -- 172.21.15.28:0/159675894 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4037070 con 0x7f2eb40d2950 2024-09-10T10:56:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb0a00640 1 -- 172.21.15.28:0/159675894 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea402ae40 con 0x7f2eb40d2950 2024-09-10T10:56:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb0a00640 1 -- 172.21.15.28:0/159675894 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4032660 con 0x7f2eb40d2950 2024-09-10T10:56:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 -- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2950 msgr2=0x7f2eb40d4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2950 0x7f2eb40d4c40 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40097a0 tx=0x7f2ea402a0f0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 -- 172.21.15.28:0/159675894 shutdown_connections 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb40b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 --2- 172.21.15.28:0/159675894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2950 0x7f2eb40d4c40 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 -- 172.21.15.28:0/159675894 >> 172.21.15.28:0/159675894 conn(0x7f2eb408bfc0 msgr2=0x7f2eb40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 -- 172.21.15.28:0/159675894 shutdown_connections 2024-09-10T10:56:27.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.378+0000 7f2eb3200640 1 -- 172.21.15.28:0/159675894 wait complete. 2024-09-10T10:56:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.380+0000 7f2eb3200640 1 Processor -- start 2024-09-10T10:56:27.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.380+0000 7f2eb3200640 1 -- start start 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb41f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 0x7f2eb40cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2d30 0x7f2eb41f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb40b48a0 con 0x7f2eb40d2950 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb40d5180 con 0x7f2eb40d2d30 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb3200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb40b4a20 con 0x7f2eb40b56a0 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 0x7f2eb40cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 0x7f2eb40cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34800/0 (socket says 172.21.15.28:34800) 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb1400640 1 -- 172.21.15.28:0/3386677612 learned_addr learned my addr 172.21.15.28:0/3386677612 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb41f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.381+0000 7f2eb2800640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2d30 0x7f2eb41f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 -- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 msgr2=0x7f2eb41f4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2eb40b56a0 0x7f2eb41f4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 -- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2d30 msgr2=0x7f2eb41f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2d30 0x7f2eb41f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb40cb960 con 0x7f2eb40d2950 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb2800640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2eb40d2d30 0x7f2eb41f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb1400640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 0x7f2eb40cca00 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e870 tx=0x7f2ea000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea0017070 con 0x7f2eb40d2950 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea000c940 con 0x7f2eb40d2950 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea000cd30 con 0x7f2eb40d2950 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.382+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb40cbc40 con 0x7f2eb40d2950 2024-09-10T10:56:27.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.383+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb40cc200 con 0x7f2eb40d2950 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.383+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2eb40cc5e0 con 0x7f2eb40d2950 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.385+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2ea001a070 con 0x7f2eb40d2950 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.385+0000 7f2eaa000640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e9406be30 0x7f2e9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.385+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f2ea008b3b0 con 0x7f2eb40d2950 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eaa000640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f2e94072370 0x7f2e940747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e94074e80 con 0x7f2e94072370 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f2ea008b760 con 0x7f2eb40d2950 2024-09-10T10:56:27.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e9406be30 0x7f2e9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eb2800640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f2e94072370 0x7f2e940747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.386+0000 7f2eb1e00640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e9406be30 0x7f2e9406e2f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4034000 tx=0x7f2ea402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.387+0000 7f2eb2800640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f2e94072370 0x7f2e940747b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.388+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== osd.5 v2:172.21.15.138:6802/2049132176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e94074e80 con 0x7f2e94072370 2024-09-10T10:56:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.401+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 --> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2eb40d4c40 con 0x7f2e94072370 2024-09-10T10:56:27.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eaa000640 1 -- 172.21.15.28:0/3386677612 <== osd.5 v2:172.21.15.138:6802/2049132176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2eb40d4c40 con 0x7f2e94072370 2024-09-10T10:56:27.404 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:27.404 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:27.404 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f2e94072370 msgr2=0x7f2e940747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6802/2049132176,v1:172.21.15.138:6803/2049132176] conn(0x7f2e94072370 0x7f2e940747b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e9406be30 msgr2=0x7f2e9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2e9406be30 0x7f2e9406e2f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2ea4034000 tx=0x7f2ea402e000 comp rx=0 tx=0).stop 2024-09-10T10:56:27.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 msgr2=0x7f2eb40cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 --2- 172.21.15.28:0/3386677612 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2eb40d2950 0x7f2eb40cca00 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f2ea000e870 tx=0x7f2ea000ed30 comp rx=0 tx=0).stop 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb2800640 1 -- 172.21.15.28:0/3386677612 reap_dead start 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 shutdown_connections 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 >> 172.21.15.28:0/3386677612 conn(0x7f2eb408bfc0 msgr2=0x7f2eb40b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 shutdown_connections 2024-09-10T10:56:27.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.402+0000 7f2eb3200640 1 -- 172.21.15.28:0/3386677612 wait complete. 2024-09-10T10:56:27.419 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T10:56:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.574+0000 7f800be00640 1 Processor -- start 2024-09-10T10:56:27.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.574+0000 7f800be00640 1 -- start start 2024-09-10T10:56:27.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f800be00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 0x7f80040d4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f800be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80040b6690 0x7f800406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f800be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80040d3690 con 0x7f80040d30f0 2024-09-10T10:56:27.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f800be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f800406d530 con 0x7f80040b6690 2024-09-10T10:56:27.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f800be00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f80040d3970 con 0x7f800415e5f0 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.575+0000 7f8009e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80040b6690 0x7f800406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80040b6690 0x7f800406d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34808/0 (socket says 172.21.15.28:34808) 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 -- 172.21.15.28:0/2078678062 learned_addr learned my addr 172.21.15.28:0/2078678062 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 -- 172.21.15.28:0/2078678062 >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 msgr2=0x7f80040d4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009400640 1 --2- 172.21.15.28:0/2078678062 >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 0x7f80040d4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 --2- 172.21.15.28:0/2078678062 >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 0x7f80040d4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 -- 172.21.15.28:0/2078678062 >> v1:172.21.15.138:6789/0 conn(0x7f80040d30f0 legacy=0x7f80040b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:27.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009e00640 1 -- 172.21.15.28:0/2078678062 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800406cee0 con 0x7f80040b6690 2024-09-10T10:56:27.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.576+0000 7f8009400640 1 --2- 172.21.15.28:0/2078678062 >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 0x7f80040d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.577+0000 7f8009e00640 1 --2- 172.21.15.28:0/2078678062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80040b6690 0x7f800406d9c0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009e50 tx=0x7f7ffc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17559b998d94376 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.577+0000 7f8003e00640 1 -- 172.21.15.28:0/2078678062 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc036070 con 0x7f80040b6690 2024-09-10T10:56:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.577+0000 7f8003e00640 1 -- 172.21.15.28:0/2078678062 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc031600 con 0x7f80040b6690 2024-09-10T10:56:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.577+0000 7f8003e00640 1 -- 172.21.15.28:0/2078678062 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0319d0 con 0x7f80040b6690 2024-09-10T10:56:27.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 -- 172.21.15.28:0/2078678062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80040b6690 msgr2=0x7f800406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 --2- 172.21.15.28:0/2078678062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80040b6690 0x7f800406d9c0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009e50 tx=0x7f7ffc02a730 comp rx=0 tx=0).stop 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 -- 172.21.15.28:0/2078678062 shutdown_connections 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 --2- 172.21.15.28:0/2078678062 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80040b6690 0x7f800406d9c0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 --2- 172.21.15.28:0/2078678062 >> v2:172.21.15.138:3300/0 conn(0x7f800415e5f0 0x7f80040d4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 -- 172.21.15.28:0/2078678062 >> 172.21.15.28:0/2078678062 conn(0x7f800408bfc0 msgr2=0x7f80040d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 -- 172.21.15.28:0/2078678062 shutdown_connections 2024-09-10T10:56:27.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.578+0000 7f800be00640 1 -- 172.21.15.28:0/2078678062 wait complete. 2024-09-10T10:56:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.580+0000 7f800be00640 1 Processor -- start 2024-09-10T10:56:27.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.580+0000 7f800be00640 1 -- start start 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.580+0000 7f800be00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 0x7f800415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f800be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80040d30f0 0x7f80040b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f800be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f800be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80040d5310 con 0x7f800415e5f0 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f800be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f800415e9d0 con 0x7f80040d30f0 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f800be00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f800406df00 con 0x7f80040b6690 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34824/0 (socket says 172.21.15.28:34824) 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 -- 172.21.15.28:0/426882707 learned_addr learned my addr 172.21.15.28:0/426882707 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8008a00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80040d30f0 0x7f80040b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009400640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 0x7f800415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 -- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 msgr2=0x7f800415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 0x7f800415cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 -- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80040d30f0 msgr2=0x7f80040b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80040d30f0 0x7f80040b5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009e00640 1 -- 172.21.15.28:0/426882707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800415d0f0 con 0x7f800415e5f0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.581+0000 7f8009400640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 0x7f800415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f8009e00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc034040 tx=0x7f7ffc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc036050 con 0x7f800415e5f0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc042070 con 0x7f800415e5f0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc009510 con 0x7f800415e5f0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041e9ec0 con 0x7f800415e5f0 2024-09-10T10:56:27.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.582+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041ea480 con 0x7f800415e5f0 2024-09-10T10:56:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.583+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800406d9c0 con 0x7f800415e5f0 2024-09-10T10:56:27.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.584+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ffc007bb0 con 0x7f800415e5f0 2024-09-10T10:56:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.585+0000 7f8001600640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 0x7f7fe406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.585+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f7ffc0abca0 con 0x7f800415e5f0 2024-09-10T10:56:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.585+0000 7f8009400640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 0x7f7fe406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.586+0000 7f8009400640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 0x7f7fe406e320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f80040b54c0 tx=0x7f7ff8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.589+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f800415d0f0 con 0x7f800415e5f0 2024-09-10T10:56:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.773+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f80040d0a50 con 0x7f7fe406be60 2024-09-10T10:56:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.774+0000 7f8001600640 1 -- 172.21.15.28:0/426882707 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f80040d0a50 con 0x7f7fe406be60 2024-09-10T10:56:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T10:56:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 msgr2=0x7f7fe406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 0x7f7fe406e320 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f80040b54c0 tx=0x7f7ff8008040 comp rx=0 tx=0).stop 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 msgr2=0x7f80040b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc034040 tx=0x7f7ffc004a40 comp rx=0 tx=0).stop 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 shutdown_connections 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f7fe406be60 0x7f7fe406e320 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f800415e5f0 0x7f80040b5e90 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80040d30f0 0x7f80040b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 --2- 172.21.15.28:0/426882707 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f80040b6690 0x7f800415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.776+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 >> 172.21.15.28:0/426882707 conn(0x7f800408bfc0 msgr2=0x7f80040b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.777+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 shutdown_connections 2024-09-10T10:56:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.777+0000 7f800be00640 1 -- 172.21.15.28:0/426882707 wait complete. 2024-09-10T10:56:27.794 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T10:56:27.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.953+0000 7f6cd2e00640 1 Processor -- start 2024-09-10T10:56:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.953+0000 7f6cd2e00640 1 -- start start 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd2e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ccc0b30a0 0x7f6ccc162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd2e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc0b1030 con 0x7f6ccc0d2950 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd2e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc0b12d0 con 0x7f6ccc0b56a0 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd2e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc0b1c00 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd1a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6ccc0d2950 0x7f6ccc0d4c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51278/0 (socket says 172.21.15.28:51278) 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.954+0000 7f6cd1a00640 1 -- 172.21.15.28:0/2274097858 learned_addr learned my addr 172.21.15.28:0/2274097858 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6cd2400640 1 --2- 172.21.15.28:0/2274097858 >> v2:172.21.15.138:3300/0 conn(0x7f6ccc0b30a0 0x7f6ccc162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3055756978 0 0) 0x7f6ccc0b1030 con 0x7f6ccc0d2950 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cb4003520 con 0x7f6ccc0d2950 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1532834905 0 0) 0x7f6ccc0b12d0 con 0x7f6ccc0b56a0 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ccc0b1030 con 0x7f6ccc0b56a0 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3952361535 0 0) 0x7f6cb4003520 con 0x7f6ccc0d2950 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6cd2400640 1 -- 172.21.15.28:0/2274097858 >> v1:172.21.15.138:6789/0 conn(0x7f6ccc0b56a0 legacy=0x7f6ccc1624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6cd2400640 1 -- 172.21.15.28:0/2274097858 >> v1:172.21.15.28:6789/0 conn(0x7f6ccc0d2950 legacy=0x7f6ccc0d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.955+0000 7f6cd2400640 1 -- 172.21.15.28:0/2274097858 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0b15b0 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.956+0000 7f6cd2400640 1 --2- 172.21.15.28:0/2274097858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc162bc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49c7fff3abcae92a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0001690 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4007540 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6ccbe00640 1 -- 172.21.15.28:0/2274097858 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4007910 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 -- 172.21.15.28:0/2274097858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b30a0 msgr2=0x7f6ccc162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 --2- 172.21.15.28:0/2274097858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc162bc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc402a5f0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 -- 172.21.15.28:0/2274097858 shutdown_connections 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 --2- 172.21.15.28:0/2274097858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc162bc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 -- 172.21.15.28:0/2274097858 >> 172.21.15.28:0/2274097858 conn(0x7f6ccc08bfc0 msgr2=0x7f6ccc0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 -- 172.21.15.28:0/2274097858 shutdown_connections 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.957+0000 7f6cd2e00640 1 -- 172.21.15.28:0/2274097858 wait complete. 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.959+0000 7f6cd2e00640 1 Processor -- start 2024-09-10T10:56:27.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.959+0000 7f6cd2e00640 1 -- start start 2024-09-10T10:56:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b56a0 0x7f6ccc159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ccc0d2950 0x7f6ccc1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc163280 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc163100 con 0x7f6ccc0b56a0 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc0d2d30 con 0x7f6ccc0d2950 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd1a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34834/0 (socket says 172.21.15.28:34834) 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd1a00640 1 -- 172.21.15.28:0/3049604868 learned_addr learned my addr 172.21.15.28:0/3049604868 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd1000640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b56a0 0x7f6ccc159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.960+0000 7f6cd2400640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ccc0d2950 0x7f6ccc1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 -- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ccc0d2950 msgr2=0x7f6ccc1f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6ccc0d2950 0x7f6ccc1f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 -- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b56a0 msgr2=0x7f6ccc159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b56a0 0x7f6ccc159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc1f5280 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1000640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ccc0b56a0 0x7f6ccc159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd1a00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc0cb3f0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c750 tx=0x7f6cc000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0018070 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc00072d0 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc00047e0 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1f5520 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.961+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1f5ae0 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.962+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6ccc0d4c40 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.963+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6cc001b070 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.964+0000 7f6cc9600640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6cac06be30 0x7f6cac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.964+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f6cc008a140 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.964+0000 7f6cc9600640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f6cac072370 0x7f6cac0747b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.964+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cc001be50 con 0x7f6cac072370 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.964+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f6cc0091050 con 0x7f6ccc0b30a0 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.965+0000 7f6cd1000640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6cac06be30 0x7f6cac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.965+0000 7f6cd2400640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f6cac072370 0x7f6cac0747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:27.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.965+0000 7f6cd1000640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6cac06be30 0x7f6cac06e2f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0b1320 tx=0x7f6cb8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.965+0000 7f6cd2400640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f6cac072370 0x7f6cac0747b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:27.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.967+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== osd.6 v2:172.21.15.138:6818/1283254914 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cc001be50 con 0x7f6cac072370 2024-09-10T10:56:27.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.980+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 --> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6ccc0b9d10 con 0x7f6cac072370 2024-09-10T10:56:27.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cc9600640 1 -- 172.21.15.28:0/3049604868 <== osd.6 v2:172.21.15.138:6818/1283254914 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6ccc0b9d10 con 0x7f6cac072370 2024-09-10T10:56:27.983 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:27.984 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:27.984 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:27.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f6cac072370 msgr2=0x7f6cac0747b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6818/1283254914,v1:172.21.15.138:6819/1283254914] conn(0x7f6cac072370 0x7f6cac0747b0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:27.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6cac06be30 msgr2=0x7f6cac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6cac06be30 0x7f6cac06e2f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0b1320 tx=0x7f6cb8009040 comp rx=0 tx=0).stop 2024-09-10T10:56:27.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 msgr2=0x7f6ccc0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:27.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 --2- 172.21.15.28:0/3049604868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ccc0b30a0 0x7f6ccc0cb3f0 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c750 tx=0x7f6cc000cc10 comp rx=0 tx=0).stop 2024-09-10T10:56:27.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2400640 1 -- 172.21.15.28:0/3049604868 reap_dead start 2024-09-10T10:56:27.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 shutdown_connections 2024-09-10T10:56:27.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 >> 172.21.15.28:0/3049604868 conn(0x7f6ccc08bfc0 msgr2=0x7f6ccc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:27.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 shutdown_connections 2024-09-10T10:56:27.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:27.981+0000 7f6cd2e00640 1 -- 172.21.15.28:0/3049604868 wait complete. 2024-09-10T10:56:27.999 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T10:56:28.155 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.154+0000 7fba82c00640 1 Processor -- start 2024-09-10T10:56:28.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.154+0000 7fba82c00640 1 -- start start 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba82c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fba7c072320 0x7fba7c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba82c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c146140 con 0x7fba7c15e1f0 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba82c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7c06eed0 con 0x7fba7c072de0 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba82c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fba7c0cd740 con 0x7fba7c072320 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba80c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fba7c15e1f0 0x7fba7c06e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51280/0 (socket says 172.21.15.28:51280) 2024-09-10T10:56:28.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba80c00640 1 -- 172.21.15.28:0/2282979554 learned_addr learned my addr 172.21.15.28:0/2282979554 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:28.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba7be00640 1 --2- 172.21.15.28:0/2282979554 >> v2:172.21.15.138:3300/0 conn(0x7fba7c072320 0x7fba7c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751489703 0 0) 0x7fba7c146140 con 0x7fba7c15e1f0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.155+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba60003520 con 0x7fba7c15e1f0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537136797 0 0) 0x7fba7c06eed0 con 0x7fba7c072de0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba7c146140 con 0x7fba7c072de0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7be00640 1 -- 172.21.15.28:0/2282979554 >> v1:172.21.15.138:6789/0 conn(0x7fba7c072de0 legacy=0x7fba7c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3635245355 0 0) 0x7fba60003520 con 0x7fba7c15e1f0 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7be00640 1 -- 172.21.15.28:0/2282979554 >> v1:172.21.15.28:6789/0 conn(0x7fba7c15e1f0 legacy=0x7fba7c06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.156+0000 7fba7be00640 1 -- 172.21.15.28:0/2282979554 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0cda20 con 0x7fba7c072320 2024-09-10T10:56:28.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.157+0000 7fba7be00640 1 --2- 172.21.15.28:0/2282979554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c072320 0x7fba7c0cce60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097a0 tx=0x7fba6c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96e5e3f9c21f3a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.157+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037070 con 0x7fba7c072320 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.157+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c02abc0 con 0x7fba7c072320 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.157+0000 7fba7b400640 1 -- 172.21.15.28:0/2282979554 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba740019f0 con 0x7fba7c072320 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 -- 172.21.15.28:0/2282979554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c072320 msgr2=0x7fba7c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 --2- 172.21.15.28:0/2282979554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c072320 0x7fba7c0cce60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097a0 tx=0x7fba6c02a130 comp rx=0 tx=0).stop 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 -- 172.21.15.28:0/2282979554 shutdown_connections 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 --2- 172.21.15.28:0/2282979554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c072320 0x7fba7c0cce60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 -- 172.21.15.28:0/2282979554 >> 172.21.15.28:0/2282979554 conn(0x7fba7c067d70 msgr2=0x7fba7c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 -- 172.21.15.28:0/2282979554 shutdown_connections 2024-09-10T10:56:28.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.158+0000 7fba82c00640 1 -- 172.21.15.28:0/2282979554 wait complete. 2024-09-10T10:56:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.159+0000 7fba82c00640 1 Processor -- start 2024-09-10T10:56:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 -- start start 2024-09-10T10:56:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba7c072de0 0x7fba7c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 0x7fba7c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c0cd3a0 con 0x7fba7c072320 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c15c9e0 con 0x7fba7c15e1f0 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.160+0000 7fba82c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba7c0cd520 con 0x7fba7c072de0 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34844/0 (socket says 172.21.15.28:34844) 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 -- 172.21.15.28:0/328961670 learned_addr learned my addr 172.21.15.28:0/328961670 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:28.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba73e00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba7c072de0 0x7fba7c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba80c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 0x7fba7c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 -- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba7c072de0 msgr2=0x7fba7c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba7c072de0 0x7fba7c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 -- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 msgr2=0x7fba7c0cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 0x7fba7c0cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba7be00640 1 -- 172.21.15.28:0/328961670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c0b3670 con 0x7fba7c072320 2024-09-10T10:56:28.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.161+0000 7fba80c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 0x7fba7c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:28.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba7be00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fba6c035040 tx=0x7fba6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037050 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c043070 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c0b3910 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c006e10 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.162+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c0b3ed0 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.163+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7c072700 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.164+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba6c03ca10 con 0x7fba7c072320 2024-09-10T10:56:28.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.165+0000 7fba78c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 0x7fba5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.165+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fba6c0ab840 con 0x7fba7c072320 2024-09-10T10:56:28.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.165+0000 7fba73e00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 0x7fba5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.166+0000 7fba73e00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 0x7fba5806e2f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fba64007fd0 tx=0x7fba64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.170+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fba6c07a420 con 0x7fba7c072320 2024-09-10T10:56:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.353+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fba7c1458f0 con 0x7fba5806be30 2024-09-10T10:56:28.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.354+0000 7fba78c00640 1 -- 172.21.15.28:0/328961670 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fba7c1458f0 con 0x7fba5806be30 2024-09-10T10:56:28.357 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T10:56:28.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 msgr2=0x7fba5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 0x7fba5806e2f0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fba64007fd0 tx=0x7fba64009040 comp rx=0 tx=0).stop 2024-09-10T10:56:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 msgr2=0x7fba7c140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fba6c035040 tx=0x7fba6c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 shutdown_connections 2024-09-10T10:56:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fba5806be30 0x7fba5806e2f0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba7c15e1f0 0x7fba7c0cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba7c072de0 0x7fba7c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 --2- 172.21.15.28:0/328961670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba7c072320 0x7fba7c140da0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 >> 172.21.15.28:0/328961670 conn(0x7fba7c067d70 msgr2=0x7fba7c0c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 shutdown_connections 2024-09-10T10:56:28.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.356+0000 7fba82c00640 1 -- 172.21.15.28:0/328961670 wait complete. 2024-09-10T10:56:28.375 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T10:56:28.375 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T10:56:28.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.535+0000 7f2e02200640 1 Processor -- start 2024-09-10T10:56:28.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.535+0000 7f2e02200640 1 -- start start 2024-09-10T10:56:28.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2e02200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2e02200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2dfc15ecf0 0x7f2dfc06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2e02200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dfc072070 con 0x7f2dfc0cce40 2024-09-10T10:56:28.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2e02200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2dfc15f260 con 0x7f2dfc15ecf0 2024-09-10T10:56:28.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2e02200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc15f540 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.28:47762/0 (socket says 172.21.15.28:47762) 2024-09-10T10:56:28.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2dfb400640 1 -- 172.21.15.28:0/2871398027 learned_addr learned my addr 172.21.15.28:0/2871398027 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.536+0000 7f2df2a00640 1 --2- 172.21.15.28:0/2871398027 >> v2:172.21.15.138:3300/0 conn(0x7f2dfc15ecf0 0x7f2dfc06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.537+0000 7f2dfb400640 1 -- 172.21.15.28:0/2871398027 >> v2:172.21.15.138:3300/0 conn(0x7f2dfc15ecf0 msgr2=0x7f2dfc06f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.537+0000 7f2dfb400640 1 --2- 172.21.15.28:0/2871398027 >> v2:172.21.15.138:3300/0 conn(0x7f2dfc15ecf0 0x7f2dfc06f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.537+0000 7f2dfb400640 1 -- 172.21.15.28:0/2871398027 >> v1:172.21.15.138:6789/0 conn(0x7f2dfc0cce40 legacy=0x7f2dfc072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.537+0000 7f2dfb400640 1 -- 172.21.15.28:0/2871398027 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc15f820 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.537+0000 7f2dfaa00640 1 -- 172.21.15.28:0/2871398027 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843638183 0 0) 0x7f2dfc072070 con 0x7f2dfc0cce40 2024-09-10T10:56:28.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.538+0000 7f2dfb400640 1 --2- 172.21.15.28:0/2871398027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f52d7ef12a8c97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.538+0000 7f2dfaa00640 1 -- 172.21.15.28:0/2871398027 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec037070 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.538+0000 7f2dfaa00640 1 -- 172.21.15.28:0/2871398027 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec032440 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.538+0000 7f2dfaa00640 1 -- 172.21.15.28:0/2871398027 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec032810 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 -- 172.21.15.28:0/2871398027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 msgr2=0x7f2dfc0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 --2- 172.21.15.28:0/2871398027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec02a1c0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 -- 172.21.15.28:0/2871398027 shutdown_connections 2024-09-10T10:56:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 --2- 172.21.15.28:0/2871398027 >> v2:172.21.15.138:3300/0 conn(0x7f2dfc15ecf0 0x7f2dfc06f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 --2- 172.21.15.28:0/2871398027 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15b0f0 0x7f2dfc0b3c40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 -- 172.21.15.28:0/2871398027 >> 172.21.15.28:0/2871398027 conn(0x7f2dfc067d70 msgr2=0x7f2dfc06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 -- 172.21.15.28:0/2871398027 shutdown_connections 2024-09-10T10:56:28.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.539+0000 7f2e02200640 1 -- 172.21.15.28:0/2871398027 wait complete. 2024-09-10T10:56:28.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.541+0000 7f2e02200640 1 Processor -- start 2024-09-10T10:56:28.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.541+0000 7f2e02200640 1 -- start start 2024-09-10T10:56:28.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.541+0000 7f2e02200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 0x7f2dfc14cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2e02200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dfc15b0f0 0x7f2dfc15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2e02200640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15ecf0 0x7f2dfc15dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2e02200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc15b4d0 con 0x7f2dfc0cce40 2024-09-10T10:56:28.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2e02200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc0b4180 con 0x7f2dfc15b0f0 2024-09-10T10:56:28.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2e02200640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2dfc0afa00 con 0x7f2dfc15ecf0 2024-09-10T10:56:28.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 0x7f2dfc14cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 0x7f2dfc14cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34852/0 (socket says 172.21.15.28:34852) 2024-09-10T10:56:28.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 -- 172.21.15.28:0/160420790 learned_addr learned my addr 172.21.15.28:0/160420790 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:28.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2df2a00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dfc15b0f0 0x7f2dfc15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfbe00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15ecf0 0x7f2dfc15dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 -- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15ecf0 msgr2=0x7f2dfc15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15ecf0 0x7f2dfc15dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 -- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dfc15b0f0 msgr2=0x7f2dfc15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2dfc15b0f0 0x7f2dfc15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfb400640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc1e7a60 con 0x7f2dfc0cce40 2024-09-10T10:56:28.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.542+0000 7f2dfbe00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2dfc15ecf0 0x7f2dfc15dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:28.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2dfb400640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 0x7f2dfc14cdf0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec041020 con 0x7f2dfc0cce40 2024-09-10T10:56:28.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dec030640 con 0x7f2dfc0cce40 2024-09-10T10:56:28.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec037040 con 0x7f2dfc0cce40 2024-09-10T10:56:28.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc1e7d40 con 0x7f2dfc0cce40 2024-09-10T10:56:28.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.543+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1e8300 con 0x7f2dfc0cce40 2024-09-10T10:56:28.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.544+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2dfc0b3c40 con 0x7f2dfc0cce40 2024-09-10T10:56:28.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.545+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2dec044020 con 0x7f2dfc0cce40 2024-09-10T10:56:28.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df3e00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dd806be60 0x7f2dd806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f2dec0ab5c0 con 0x7f2dfc0cce40 2024-09-10T10:56:28.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df3e00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f2dd80723a0 0x7f2dd80747e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dec02cea0 con 0x7f2dd80723a0 2024-09-10T10:56:28.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=154) ==== 24+0+0 (secure 0 0 0) 0x7f2dec0b2050 con 0x7f2dfc0cce40 2024-09-10T10:56:28.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.546+0000 7f2df2a00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dd806be60 0x7f2dd806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.547+0000 7f2dfbe00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f2dd80723a0 0x7f2dd80747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.547+0000 7f2df2a00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dd806be60 0x7f2dd806e320 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b240 tx=0x7f2de400b3a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.547+0000 7f2dfbe00640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f2dd80723a0 0x7f2dd80747e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.548+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== osd.7 v2:172.21.15.138:6826/4029970444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2dec02cea0 con 0x7f2dd80723a0 2024-09-10T10:56:28.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.562+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 --> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2dfc06e2e0 con 0x7f2dd80723a0 2024-09-10T10:56:28.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2df3e00640 1 -- 172.21.15.28:0/160420790 <== osd.7 v2:172.21.15.138:6826/4029970444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2dfc06e2e0 con 0x7f2dd80723a0 2024-09-10T10:56:28.565 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-09-10T10:56:28.565 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T10:56:28.566 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-09-10T10:56:28.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f2dd80723a0 msgr2=0x7f2dd80747e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6826/4029970444,v1:172.21.15.138:6827/4029970444] conn(0x7f2dd80723a0 0x7f2dd80747e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dd806be60 msgr2=0x7f2dd806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f2dd806be60 0x7f2dd806e320 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b240 tx=0x7f2de400b3a0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 msgr2=0x7f2dfc14cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 --2- 172.21.15.28:0/160420790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2dfc0cce40 0x7f2dfc14cdf0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009870 tx=0x7f2dec0046c0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2dfbe00640 1 -- 172.21.15.28:0/160420790 reap_dead start 2024-09-10T10:56:28.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 shutdown_connections 2024-09-10T10:56:28.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 >> 172.21.15.28:0/160420790 conn(0x7f2dfc067d70 msgr2=0x7f2dfc0b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 shutdown_connections 2024-09-10T10:56:28.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.564+0000 7f2e02200640 1 -- 172.21.15.28:0/160420790 wait complete. 2024-09-10T10:56:28.580 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T10:56:28.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.736+0000 7fade6c00640 1 Processor -- start 2024-09-10T10:56:28.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.736+0000 7fade6c00640 1 -- start start 2024-09-10T10:56:28.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.737+0000 7fade6c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fade0161c80 0x7fade006e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7fade6c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fade006e870 con 0x7fade0162060 2024-09-10T10:56:28.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7fade6c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fade006d9f0 con 0x7fade00b6680 2024-09-10T10:56:28.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7fade6c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fade006dcd0 con 0x7fade0161c80 2024-09-10T10:56:28.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddf400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fade0162060 0x7fade00b5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51292/0 (socket says 172.21.15.28:51292) 2024-09-10T10:56:28.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fade0161c80 0x7fade006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddf400640 1 -- 172.21.15.28:0/4148568105 learned_addr learned my addr 172.21.15.28:0/4148568105 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:28.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddfe00640 1 -- 172.21.15.28:0/4148568105 >> v1:172.21.15.138:6789/0 conn(0x7fade00b6680 legacy=0x7fade00cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:28.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddfe00640 1 -- 172.21.15.28:0/4148568105 >> v1:172.21.15.28:6789/0 conn(0x7fade0162060 legacy=0x7fade00b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddfe00640 1 -- 172.21.15.28:0/4148568105 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade006dfb0 con 0x7fade0161c80 2024-09-10T10:56:28.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.738+0000 7faddea00640 1 -- 172.21.15.28:0/4148568105 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625939179 0 0) 0x7fadcc001110 con 0x7fade0162060 2024-09-10T10:56:28.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.739+0000 7faddfe00640 1 --2- 172.21.15.28:0/4148568105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade006e330 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fadd00097a0 tx=0x7fadd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdded1ea67d67d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7faddea00640 1 -- 172.21.15.28:0/4148568105 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0037070 con 0x7fade0161c80 2024-09-10T10:56:28.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7faddea00640 1 -- 172.21.15.28:0/4148568105 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadd0032440 con 0x7fade0161c80 2024-09-10T10:56:28.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7faddea00640 1 -- 172.21.15.28:0/4148568105 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0032810 con 0x7fade0161c80 2024-09-10T10:56:28.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 -- 172.21.15.28:0/4148568105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 msgr2=0x7fade006e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 --2- 172.21.15.28:0/4148568105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade006e330 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fadd00097a0 tx=0x7fadd002a1e0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 -- 172.21.15.28:0/4148568105 shutdown_connections 2024-09-10T10:56:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 --2- 172.21.15.28:0/4148568105 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade006e330 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 -- 172.21.15.28:0/4148568105 >> 172.21.15.28:0/4148568105 conn(0x7fade00d44f0 msgr2=0x7fade00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 -- 172.21.15.28:0/4148568105 shutdown_connections 2024-09-10T10:56:28.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.740+0000 7fade6c00640 1 -- 172.21.15.28:0/4148568105 wait complete. 2024-09-10T10:56:28.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.742+0000 7fade6c00640 1 Processor -- start 2024-09-10T10:56:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.742+0000 7fade6c00640 1 -- start start 2024-09-10T10:56:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 0x7fade0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 0x7fade00cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fade00b6b30 con 0x7fade0161c80 2024-09-10T10:56:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fade00d2aa0 con 0x7fade0162060 2024-09-10T10:56:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7fade6c00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fade00b6cb0 con 0x7fade00b6680 2024-09-10T10:56:28.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7faddf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7faddf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34862/0 (socket says 172.21.15.28:34862) 2024-09-10T10:56:28.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.743+0000 7faddf400640 1 -- 172.21.15.28:0/2775587412 learned_addr learned my addr 172.21.15.28:0/2775587412 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:28.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddfe00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 0x7fade0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fade4c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 0x7fade00cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 -- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 msgr2=0x7fade0159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 0x7fade0159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 -- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 msgr2=0x7fade00cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 0x7fade00cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 -- 172.21.15.28:0/2775587412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade0159f90 con 0x7fade0161c80 2024-09-10T10:56:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddfe00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 0x7fade0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fade4c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 0x7fade00cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7faddf400640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c750 tx=0x7fadcc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc018070 con 0x7fade0161c80 2024-09-10T10:56:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadcc0072d0 con 0x7fade0161c80 2024-09-10T10:56:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade00cb930 con 0x7fade0161c80 2024-09-10T10:56:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.744+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0047e0 con 0x7fade0161c80 2024-09-10T10:56:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.745+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade00b92c0 con 0x7fade0161c80 2024-09-10T10:56:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.746+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade006e330 con 0x7fade0161c80 2024-09-10T10:56:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.747+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fadcc01b070 con 0x7fade0161c80 2024-09-10T10:56:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.747+0000 7fadd7e00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 0x7fadc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.748+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fadcc08a110 con 0x7fade0161c80 2024-09-10T10:56:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.748+0000 7faddfe00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 0x7fadc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.749+0000 7faddfe00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 0x7fadc006e2f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fadd0034000 tx=0x7fadd002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.752+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fade0159f90 con 0x7fade0161c80 2024-09-10T10:56:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.940+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fade00cbc10 con 0x7fadc006be30 2024-09-10T10:56:28.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.941+0000 7fadd7e00640 1 -- 172.21.15.28:0/2775587412 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fade00cbc10 con 0x7fadc006be30 2024-09-10T10:56:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T10:56:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 msgr2=0x7fadc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 0x7fadc006e2f0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fadd0034000 tx=0x7fadd002e000 comp rx=0 tx=0).stop 2024-09-10T10:56:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 msgr2=0x7fade00cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c750 tx=0x7fadcc00cc10 comp rx=0 tx=0).stop 2024-09-10T10:56:28.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 shutdown_connections 2024-09-10T10:56:28.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fadc006be30 0x7fadc006e2f0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fade0162060 0x7fade00cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fade0161c80 0x7fade00cdfa0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 --2- 172.21.15.28:0/2775587412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fade00b6680 0x7fade0159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.943+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 >> 172.21.15.28:0/2775587412 conn(0x7fade00d44f0 msgr2=0x7fade00b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.944+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 shutdown_connections 2024-09-10T10:56:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:28.944+0000 7fade6c00640 1 -- 172.21.15.28:0/2775587412 wait complete. 2024-09-10T10:56:28.960 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:56:29.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.115+0000 7fa3abe00640 1 Processor -- start 2024-09-10T10:56:29.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.115+0000 7fa3abe00640 1 -- start start 2024-09-10T10:56:29.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.116+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa3a406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.116+0000 7fa3abe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a40b49a0 con 0x7fa3a4162ad0 2024-09-10T10:56:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.116+0000 7fa3abe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a40afa20 con 0x7fa3a40720b0 2024-09-10T10:56:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3abe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a40afce0 con 0x7fa3a40cfca0 2024-09-10T10:56:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a8a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3a4162ad0 0x7fa3a40b4290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:51296/0 (socket says 172.21.15.28:51296) 2024-09-10T10:56:29.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a8a00640 1 -- 172.21.15.28:0/309958316 learned_addr learned my addr 172.21.15.28:0/309958316 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:29.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a9e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa3a406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153181996 0 0) 0x7fa3a40b49a0 con 0x7fa3a4162ad0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa38c003520 con 0x7fa3a4162ad0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778534792 0 0) 0x7fa3a40afa20 con 0x7fa3a40720b0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.117+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3a40b49a0 con 0x7fa3a40720b0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 401804549 0 0) 0x7fa38c003520 con 0x7fa3a4162ad0 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 msgr2=0x7fa3a406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa3a406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 >> v1:172.21.15.138:6789/0 conn(0x7fa3a40720b0 legacy=0x7fa3a4072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a40b34d0 con 0x7fa3a4162ad0 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2967702216 0 0) 0x7fa390001690 con 0x7fa3a4162ad0 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 >> v1:172.21.15.28:6789/0 conn(0x7fa3a4162ad0 legacy=0x7fa3a40b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38c003b70 0x7fa38c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa38c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa38c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a40afce0 con 0x7fa38c004800 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c00ba70 con 0x7fa38c003b70 2024-09-10T10:56:29.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c00bc70 con 0x7fa3a40cfca0 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa38c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a9400640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38c003b70 0x7fa38c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a9e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa38c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 -- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 msgr2=0x7fa38c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa38c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 -- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38c003b70 msgr2=0x7fa38c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:29.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa38c003b70 0x7fa38c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a8a00640 1 -- 172.21.15.28:0/309958316 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c003520 con 0x7fa38c004800 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.118+0000 7fa3a9e00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40cfca0 0x7fa38c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.119+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa38c004c80 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fa390007fb0 tx=0x7fa38c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a701aa1a20a0c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390007c00 con 0x7fa38c004800 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa390005ad0 con 0x7fa38c004800 2024-09-10T10:56:29.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3a3e00640 1 -- 172.21.15.28:0/309958316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa390007070 con 0x7fa38c004800 2024-09-10T10:56:29.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 -- 172.21.15.28:0/309958316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 msgr2=0x7fa38c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 --2- 172.21.15.28:0/309958316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa38c004c80 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fa390007fb0 tx=0x7fa38c005830 comp rx=0 tx=0).stop 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3a9e00640 1 -- 172.21.15.28:0/309958316 reap_dead start 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 -- 172.21.15.28:0/309958316 shutdown_connections 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 -- 172.21.15.28:0/309958316 >> 172.21.15.28:0/309958316 conn(0x7fa3a4067d70 msgr2=0x7fa3a406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 -- 172.21.15.28:0/309958316 shutdown_connections 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.120+0000 7fa3abe00640 1 -- 172.21.15.28:0/309958316 wait complete. 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.122+0000 7fa3abe00640 1 Processor -- start 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.122+0000 7fa3abe00640 1 -- start start 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40b0020 0x7fa3a40af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a41f8960 0x7fa3a4158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a40b3d70 con 0x7fa38c004800 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a40b3a90 con 0x7fa3a41f8960 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3abe00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a406f6f0 con 0x7fa3a40b0020 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3a9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3a9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34888/0 (socket says 172.21.15.28:34888) 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3a9400640 1 -- 172.21.15.28:0/3066095606 learned_addr learned my addr 172.21.15.28:0/3066095606 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:29.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40b0020 0x7fa3a40af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.123+0000 7fa3a9e00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a41f8960 0x7fa3a4158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 -- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40b0020 msgr2=0x7fa3a40af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40b0020 0x7fa3a40af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 -- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a41f8960 msgr2=0x7fa3a4158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a41f8960 0x7fa3a4158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 -- 172.21.15.28:0/3066095606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a40af590 con 0x7fa38c004800 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a9400640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fa39800f7d0 tx=0x7fa39800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39801a070 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39800fe80 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398007050 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a40b0570 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.124+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a40b0b30 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.125+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a4072490 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.126+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3980071f0 con 0x7fa38c004800 2024-09-10T10:56:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.127+0000 7fa3a1600640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 0x7fa38406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.128+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7fa39808a530 con 0x7fa38c004800 2024-09-10T10:56:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.128+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 0x7fa38406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.129+0000 7fa3a8a00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 0x7fa38406e320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa390032000 tx=0x7fa39002d000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.132+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa398058f60 con 0x7fa38c004800 2024-09-10T10:56:29.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.310+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3a415dd00 con 0x7fa38406be60 2024-09-10T10:56:29.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.312+0000 7fa3a1600640 1 -- 172.21.15.28:0/3066095606 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13339 (secure 0 0 0) 0x7fa3a415dd00 con 0x7fa38406be60 2024-09-10T10:56:29.313 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:29.313 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:56:29.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 msgr2=0x7fa38406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 0x7fa38406e320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa390032000 tx=0x7fa39002d000 comp rx=0 tx=0).stop 2024-09-10T10:56:29.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 msgr2=0x7fa3a415bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:29.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7fa39800f7d0 tx=0x7fa39800fc90 comp rx=0 tx=0).stop 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 shutdown_connections 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7fa38406be60 0x7fa38406e320 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3a41f8960 0x7fa3a4158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa3a40b0020 0x7fa3a40af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 --2- 172.21.15.28:0/3066095606 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa38c004800 0x7fa3a415bec0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 >> 172.21.15.28:0/3066095606 conn(0x7fa3a4067d70 msgr2=0x7fa3a406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 shutdown_connections 2024-09-10T10:56:29.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:29.314+0000 7fa3abe00640 1 -- 172.21.15.28:0/3066095606 wait complete. 2024-09-10T10:56:29.332 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":535,"stamp":"2024-09-10T10:56:28.684023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291196,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683524,"statfs":{"total":773094113280,"available":772795928576,"internally_reserved":0,"allocated":17113088,"data_stored":9204040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.672362"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":424,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-10T10:56:18.390358+0000","last_change":"2024-09-10T10:43:38.464512+0000","last_active":"2024-09-10T10:56:18.390358+0000","last_peered":"2024-09-10T10:56:18.390358+0000","last_clean":"2024-09-10T10:56:18.390358+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:56:18.390358+0000","last_undegraded":"2024-09-10T10:56:18.390358+0000","last_fullsized":"2024-09-10T10:56:18.390358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T10:43:37.448763+0000","last_clean_scrub_stamp":"2024-09-10T10:43:37.448763+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509598,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28972,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41456,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330384,"statfs":{"total":96636764160,"available":96594313216,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509598,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509596,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42032,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329808,"statfs":{"total":96636764160,"available":96593723392,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33720,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338120,"statfs":{"total":96636764160,"available":96602234880,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:56:29.334 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T10:43:37.448763+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=43, tm_sec=37, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=10, tm_min=56, tm_sec=23, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T10:56:29.335 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T10:56:41.187 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:41.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:49.336 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T10:56:49.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.504+0000 7f3c4a600640 1 Processor -- start 2024-09-10T10:56:49.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.504+0000 7f3c4a600640 1 -- start start 2024-09-10T10:56:49.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.505+0000 7f3c4a600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3c4415b300 0x7f3c440cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.505+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c4415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c4a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c440cdbe0 con 0x7f3c440cd3f0 2024-09-10T10:56:49.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c4a600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3c440728a0 con 0x7f3c4415b300 2024-09-10T10:56:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c4a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c440cd900 con 0x7f3c4406f8d0 2024-09-10T10:56:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3c4415b300 0x7f3c440cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3c4415b300 0x7f3c440cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53644/0 (socket says 172.21.15.28:53644) 2024-09-10T10:56:49.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 -- 172.21.15.28:0/1793218848 learned_addr learned my addr 172.21.15.28:0/1793218848 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43e00640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c4415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 -- 172.21.15.28:0/1793218848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 msgr2=0x7f3c4415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c4415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 -- 172.21.15.28:0/1793218848 >> v1:172.21.15.138:6789/0 conn(0x7f3c440cd3f0 legacy=0x7f3c4406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.506+0000 7f3c43400640 1 -- 172.21.15.28:0/1793218848 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c44072250 con 0x7f3c4415b300 2024-09-10T10:56:49.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c43400640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c4415b300 0x7f3c440cceb0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400bb10 tx=0x7f3c3402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5aa7abaa04f04a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c42a00640 1 -- 172.21.15.28:0/1793218848 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34036020 con 0x7f3c4415b300 2024-09-10T10:56:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c42a00640 1 -- 172.21.15.28:0/1793218848 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c34032820 con 0x7f3c4415b300 2024-09-10T10:56:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c42a00640 1 -- 172.21.15.28:0/1793218848 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c34032bf0 con 0x7f3c4415b300 2024-09-10T10:56:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c4a600640 1 -- 172.21.15.28:0/1793218848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c4415b300 msgr2=0x7f3c440cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.508+0000 7f3c4a600640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c4415b300 0x7f3c440cceb0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400bb10 tx=0x7f3c3402c920 comp rx=0 tx=0).stop 2024-09-10T10:56:49.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 -- 172.21.15.28:0/1793218848 shutdown_connections 2024-09-10T10:56:49.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c4415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 --2- 172.21.15.28:0/1793218848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c4415b300 0x7f3c440cceb0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 -- 172.21.15.28:0/1793218848 >> 172.21.15.28:0/1793218848 conn(0x7f3c44067d70 msgr2=0x7f3c4406bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 -- 172.21.15.28:0/1793218848 shutdown_connections 2024-09-10T10:56:49.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.509+0000 7f3c4a600640 1 -- 172.21.15.28:0/1793218848 wait complete. 2024-09-10T10:56:49.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.510+0000 7f3c4a600640 1 Processor -- start 2024-09-10T10:56:49.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.511+0000 7f3c4a600640 1 -- start start 2024-09-10T10:56:49.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c441550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c4415b300 0x7f3c441426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440d0160 con 0x7f3c440cd3f0 2024-09-10T10:56:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c440cffe0 con 0x7f3c4415b300 2024-09-10T10:56:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c4a600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3c44072d30 con 0x7f3c4406f8d0 2024-09-10T10:56:49.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53660/0 (socket says 172.21.15.28:53660) 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 -- 172.21.15.28:0/257717498 learned_addr learned my addr 172.21.15.28:0/257717498 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c43e00640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c4415b300 0x7f3c441426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c43400640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c441550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 -- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 msgr2=0x7f3c441550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c441550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 -- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c4415b300 msgr2=0x7f3c441426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c4415b300 0x7f3c441426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.512+0000 7f3c3be00640 1 -- 172.21.15.28:0/257717498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c44142bf0 con 0x7f3c440cd3f0 2024-09-10T10:56:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c3be00640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c430 tx=0x7f3c2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0052d0 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2c005490 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c007260 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c44155c00 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.513+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c440cc470 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.514+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44158f70 con 0x7f3c440cd3f0 2024-09-10T10:56:49.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.515+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c2c007400 con 0x7f3c440cd3f0 2024-09-10T10:56:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.516+0000 7f3c3b400640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 0x7f3c2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.516+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f3c2c08a300 con 0x7f3c440cd3f0 2024-09-10T10:56:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.517+0000 7f3c43400640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 0x7f3c2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.517+0000 7f3c43400640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 0x7f3c2006e2f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3c34037000 tx=0x7f3c34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.521+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3c2c058d30 con 0x7f3c440cd3f0 2024-09-10T10:56:49.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.704+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 --> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c44158c90 con 0x7f3c2006be30 2024-09-10T10:56:49.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.705+0000 7f3c3b400640 1 -- 172.21.15.28:0/257717498 <== mgr.4102 v2:172.21.15.138:6800/980174962 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13385 (secure 0 0 0) 0x7f3c44158c90 con 0x7f3c2006be30 2024-09-10T10:56:49.706 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:49.707 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-09-10T10:56:49.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.707+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 msgr2=0x7f3c2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 0x7f3c2006e2f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3c34037000 tx=0x7f3c34008040 comp rx=0 tx=0).stop 2024-09-10T10:56:49.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 msgr2=0x7f3c441e4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00c430 tx=0x7f3c2c00c8f0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 shutdown_connections 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f3c2006be30 0x7f3c2006e2f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c4415b300 0x7f3c441426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c440cd3f0 0x7f3c441e4660 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 --2- 172.21.15.28:0/257717498 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3c4406f8d0 0x7f3c441550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 >> 172.21.15.28:0/257717498 conn(0x7f3c44067d70 msgr2=0x7f3c440af500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 shutdown_connections 2024-09-10T10:56:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.708+0000 7f3c4a600640 1 -- 172.21.15.28:0/257717498 wait complete. 2024-09-10T10:56:49.724 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-09-10T10:56:48.688373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":291180,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754683540,"statfs":{"total":773094113280,"available":772795944960,"internally_reserved":0,"allocated":17113088,"data_stored":9204040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002487"},"pg_stats":[{"pgid":"1.0","version":"154'119","reported_seq":434,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-10T10:56:25.964673+0000","last_change":"2024-09-10T10:56:25.964737+0000","last_active":"2024-09-10T10:56:25.964673+0000","last_peered":"2024-09-10T10:56:25.964673+0000","last_clean":"2024-09-10T10:56:25.964673+0000","last_became_active":"2024-09-10T10:43:38.464258+0000","last_became_peered":"2024-09-10T10:43:38.464258+0000","last_unstale":"2024-09-10T10:56:25.964673+0000","last_undegraded":"2024-09-10T10:56:25.964673+0000","last_fullsized":"2024-09-10T10:56:25.964673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"154'119","last_scrub_stamp":"2024-09-10T10:56:25.964582+0000","last_deep_scrub":"154'119","last_deep_scrub_stamp":"2024-09-10T10:56:25.964582+0000","last_clean_scrub_stamp":"2024-09-10T10:56:25.964582+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:17.570836+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509600,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28972,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33132,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":41456,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330384,"statfs":{"total":96636764160,"available":96594313216,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509602,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37292,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509600,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42028,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94329812,"statfs":{"total":96636764160,"available":96593727488,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509601,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37296,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1990656,"data_stored":1002913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33708,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338132,"statfs":{"total":96636764160,"available":96602247168,"internally_reserved":0,"allocated":2584576,"data_stored":1593281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T10:56:49.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T10:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.876+0000 7ffbc5600640 1 Processor -- start 2024-09-10T10:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.876+0000 7ffbc5600640 1 -- start start 2024-09-10T10:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbc5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbc5600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 0x7ffbc00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbc5600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc00b1170 con 0x7ffbc0161ac0 2024-09-10T10:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbc5600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc00ceba0 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbc5600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc006dfb0 con 0x7ffbc00b3640 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbbf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbbf400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53668/0 (socket says 172.21.15.28:53668) 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.877+0000 7ffbbf400640 1 -- 172.21.15.28:0/744218708 learned_addr learned my addr 172.21.15.28:0/744218708 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbc4c00640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 0x7ffbc00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbbf400640 1 -- 172.21.15.28:0/744218708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 msgr2=0x7ffbc00b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbbf400640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 0x7ffbc00b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbbf400640 1 -- 172.21.15.28:0/744218708 >> v1:172.21.15.138:6789/0 conn(0x7ffbc0161ac0 legacy=0x7ffbc00d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbbf400640 1 -- 172.21.15.28:0/744218708 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc00b1450 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.878+0000 7ffbc4c00640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 0x7ffbc00b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.879+0000 7ffbbf400640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00c7e20 tx=0x7ffbac02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba1b0c6ba3f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.879+0000 7ffbbea00640 1 -- 172.21.15.28:0/744218708 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac037070 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.879+0000 7ffbbea00640 1 -- 172.21.15.28:0/744218708 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac032440 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.879+0000 7ffbbea00640 1 -- 172.21.15.28:0/744218708 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac032810 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 -- 172.21.15.28:0/744218708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc00d2ac0 msgr2=0x7ffbc006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00c7e20 tx=0x7ffbac02a2a0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 -- 172.21.15.28:0/744218708 shutdown_connections 2024-09-10T10:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00b3640 0x7ffbc00b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 --2- 172.21.15.28:0/744218708 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc00d2ac0 0x7ffbc006e6c0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 -- 172.21.15.28:0/744218708 >> 172.21.15.28:0/744218708 conn(0x7ffbc008bfc0 msgr2=0x7ffbc00d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:49.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 -- 172.21.15.28:0/744218708 shutdown_connections 2024-09-10T10:56:49.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.880+0000 7ffbc5600640 1 -- 172.21.15.28:0/744218708 wait complete. 2024-09-10T10:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.882+0000 7ffbc5600640 1 Processor -- start 2024-09-10T10:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.882+0000 7ffbc5600640 1 -- start start 2024-09-10T10:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.882+0000 7ffbc5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc00b3640 0x7ffbc0157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.882+0000 7ffbc5600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00d2ac0 0x7ffbc01f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.882+0000 7ffbc5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc5600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc0161ea0 con 0x7ffbc0161ac0 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc5600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc00b1a10 con 0x7ffbc00b3640 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc5600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc0162020 con 0x7ffbc00d2ac0 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53676/0 (socket says 172.21.15.28:53676) 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 -- 172.21.15.28:0/2283713238 learned_addr learned my addr 172.21.15.28:0/2283713238 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbbfe00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc00b3640 0x7ffbc0157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbbf400640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00d2ac0 0x7ffbc01f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 -- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00d2ac0 msgr2=0x7ffbc01f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00d2ac0 0x7ffbc01f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 -- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc00b3640 msgr2=0x7ffbc0157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc00b3640 0x7ffbc0157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:49.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.883+0000 7ffbc4c00640 1 -- 172.21.15.28:0/2283713238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc0157ca0 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbc4c00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800cef0 tx=0x7ffbb800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb801a070 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb800d860 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800dc50 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc00cc120 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.884+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc00b51a0 con 0x7ffbc0161ac0 2024-09-10T10:56:49.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.885+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc006e6c0 con 0x7ffbc0161ac0 2024-09-10T10:56:49.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.886+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbb801d070 con 0x7ffbc0161ac0 2024-09-10T10:56:49.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.887+0000 7ffbb7e00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 0x7ffba006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:49.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.887+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7ffbb808b680 con 0x7ffbc0161ac0 2024-09-10T10:56:49.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.887+0000 7ffbbfe00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 0x7ffba006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:49.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.888+0000 7ffbbfe00640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 0x7ffba006e320 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0006fd0 tx=0x7ffbb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:49.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:49.891+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffbb805a170 con 0x7ffbc0161ac0 2024-09-10T10:56:50.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.077+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7ffbc00b8630 con 0x7ffbc0161ac0 2024-09-10T10:56:50.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.081+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v30)=0 v30) ==== 123+0+0 (secure 0 0 0) 0x7ffbb8059a70 con 0x7ffbc0161ac0 2024-09-10T10:56:50.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.082+0000 7ffbb7e00640 1 -- 172.21.15.28:0/2283713238 <== mon.0 v2:172.21.15.28:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ffbb805de20 con 0x7ffbc0161ac0 2024-09-10T10:56:50.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 msgr2=0x7ffba006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 0x7ffba006e320 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0006fd0 tx=0x7ffbb0008040 comp rx=0 tx=0).stop 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 msgr2=0x7ffbc00cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800cef0 tx=0x7ffbb800a710 comp rx=0 tx=0).stop 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 shutdown_connections 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7ffba006be60 0x7ffba006e320 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbc0161ac0 0x7ffbc00cfcb0 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffbc00d2ac0 0x7ffbc01f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 --2- 172.21.15.28:0/2283713238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffbc00b3640 0x7ffbc0157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 >> 172.21.15.28:0/2283713238 conn(0x7ffbc008bfc0 msgr2=0x7ffbc006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:50.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 shutdown_connections 2024-09-10T10:56:50.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.084+0000 7ffbc5600640 1 -- 172.21.15.28:0/2283713238 wait complete. 2024-09-10T10:56:50.101 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:56:50.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T10:56:50.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.256+0000 7f6b33e00640 1 Processor -- start 2024-09-10T10:56:50.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.256+0000 7f6b33e00640 1 -- start start 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b33e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b2c072de0 0x7f6b2c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b33e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b2c146140 con 0x7f6b2c072320 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b33e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b2c0cd740 con 0x7f6b2c072700 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b33e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6b2c0cda20 con 0x7f6b2c072de0 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b31400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6b2c072320 0x7f6b2c0cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42126/0 (socket says 172.21.15.28:42126) 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b30a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b2c072de0 0x7f6b2c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:50.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b31400640 1 -- 172.21.15.28:0/505883380 learned_addr learned my addr 172.21.15.28:0/505883380 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-09-10T10:56:50.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b30a00640 1 -- 172.21.15.28:0/505883380 >> v1:172.21.15.138:6789/0 conn(0x7f6b2c072700 legacy=0x7f6b2c15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b30a00640 1 -- 172.21.15.28:0/505883380 >> v1:172.21.15.28:6789/0 conn(0x7f6b2c072320 legacy=0x7f6b2c0cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.259+0000 7f6b30a00640 1 -- 172.21.15.28:0/505883380 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c0cdd00 con 0x7f6b2c072de0 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.258+0000 7f6b2be00640 1 -- 172.21.15.28:0/505883380 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266236296 0 0) 0x7f6b2c146140 con 0x7f6b2c072320 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.259+0000 7f6b30a00640 1 --2- 172.21.15.28:0/505883380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072de0 0x7f6b2c15d1e0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097a0 tx=0x7f6b1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9e703aad3d72c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b2be00640 1 -- 172.21.15.28:0/505883380 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18037070 con 0x7f6b2c072de0 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b2be00640 1 -- 172.21.15.28:0/505883380 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b18032440 con 0x7f6b2c072de0 2024-09-10T10:56:50.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b2be00640 1 -- 172.21.15.28:0/505883380 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18032830 con 0x7f6b2c072de0 2024-09-10T10:56:50.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 -- 172.21.15.28:0/505883380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072de0 msgr2=0x7f6b2c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 --2- 172.21.15.28:0/505883380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072de0 0x7f6b2c15d1e0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6b180097a0 tx=0x7f6b1802a1a0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 -- 172.21.15.28:0/505883380 shutdown_connections 2024-09-10T10:56:50.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 --2- 172.21.15.28:0/505883380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072de0 0x7f6b2c15d1e0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 -- 172.21.15.28:0/505883380 >> 172.21.15.28:0/505883380 conn(0x7f6b2c067d70 msgr2=0x7f6b2c06bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:50.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 -- 172.21.15.28:0/505883380 shutdown_connections 2024-09-10T10:56:50.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.260+0000 7f6b33e00640 1 -- 172.21.15.28:0/505883380 wait complete. 2024-09-10T10:56:50.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.262+0000 7f6b33e00640 1 Processor -- start 2024-09-10T10:56:50.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.262+0000 7f6b33e00640 1 -- start start 2024-09-10T10:56:50.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c072700 0x7f6b2c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 0x7f6b2c0c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c15dfb0 con 0x7f6b2c072320 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c15de30 con 0x7f6b2c072700 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b33e00640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6b2c15e130 con 0x7f6b2c072de0 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b31400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b31400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53692/0 (socket says 172.21.15.28:53692) 2024-09-10T10:56:50.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.263+0000 7f6b31400640 1 -- 172.21.15.28:0/3398193395 learned_addr learned my addr 172.21.15.28:0/3398193395 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b30a00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c072700 0x7f6b2c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 0x7f6b2c0c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 -- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 msgr2=0x7f6b2c0c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 0x7f6b2c0c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 -- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c072700 msgr2=0x7f6b2c0c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c072700 0x7f6b2c0c7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 -- 172.21.15.28:0/3398193395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c1303d0 con 0x7f6b2c072320 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 0x7f6b2c0c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:56:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b31400640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c4a0 tx=0x7f6b2000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:50.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20018070 con 0x7f6b2c072320 2024-09-10T10:56:50.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6b200074e0 con 0x7f6b2c072320 2024-09-10T10:56:50.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20004a40 con 0x7f6b2c072320 2024-09-10T10:56:50.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c1306b0 con 0x7f6b2c072320 2024-09-10T10:56:50.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.264+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c130c70 con 0x7f6b2c072320 2024-09-10T10:56:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.265+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b2c0cce60 con 0x7f6b2c072320 2024-09-10T10:56:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.266+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6b20004be0 con 0x7f6b2c072320 2024-09-10T10:56:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.267+0000 7f6b29600640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 0x7f6b0c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:56:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.267+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5421+0+0 (secure 0 0 0) 0x7f6b2008a240 con 0x7f6b2c072320 2024-09-10T10:56:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.268+0000 7f6b30a00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 0x7f6b0c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:56:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.268+0000 7f6b30a00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 0x7f6b0c06e2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6b18034000 tx=0x7f6b1802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:56:50.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.271+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b20058be0 con 0x7f6b2c072320 2024-09-10T10:56:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.493+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b2c0ce360 con 0x7f6b2c072320 2024-09-10T10:56:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.493+0000 7f6b29600640 1 -- 172.21.15.28:0/3398193395 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 349 v349) ==== 95+0+3923 (secure 0 0 0) 0x7f6b200584e0 con 0x7f6b2c072320 2024-09-10T10:56:50.495 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-09-10T10:56:50.495 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":349,"btime":"2024-09-10T10:56:15:995836+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8417,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6836/2708625682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":2708625682},{"type":"v1","addr":"172.21.15.138:6836","nonce":2708625682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":335},{"gid":8423,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6834/3313599104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":3313599104},{"type":"v1","addr":"172.21.15.28:6834","nonce":3313599104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":9499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/1453896370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6835","nonce":1453896370},{"type":"v1","addr":"172.21.15.138:6837","nonce":1453896370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":336},{"gid":10833,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/3846725455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6833","nonce":3846725455},{"type":"v1","addr":"172.21.15.28:6835","nonce":3846725455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":349}],"filesystems":[]} 2024-09-10T10:56:50.495 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 349 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 msgr2=0x7f6b0c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 0x7f6b0c06e2f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f6b18034000 tx=0x7f6b1802e000 comp rx=0 tx=0).stop 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 msgr2=0x7f6b2c142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c4a0 tx=0x7f6b2000c960 comp rx=0 tx=0).stop 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 shutdown_connections 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:6800/980174962,v1:172.21.15.138:6801/980174962] conn(0x7f6b0c06be30 0x7f6b0c06e2f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6b2c072de0 0x7f6b2c0c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6b2c072700 0x7f6b2c0c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 --2- 172.21.15.28:0/3398193395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b2c072320 0x7f6b2c142cf0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:56:50.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.495+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 >> 172.21.15.28:0/3398193395 conn(0x7f6b2c067d70 msgr2=0x7f6b2c06f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:56:50.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.496+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 shutdown_connections 2024-09-10T10:56:50.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-09-10T10:56:50.496+0000 7f6b33e00640 1 -- 172.21.15.28:0/3398193395 wait complete. 2024-09-10T10:56:50.512 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T10:56:50.512 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T10:56:50.513 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T10:56:50.513 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.527 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T10:56:50.527 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T10:56:50.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.581 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T10:56:50.581 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T10:56:50.581 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.598 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T10:56:50.598 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T10:56:50.598 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.656 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T10:56:50.656 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T10:56:50.656 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T10:56:50.656 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.702 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T10:56:50.703 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T10:56:50.703 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.743 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T10:56:50.744 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T10:56:50.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.830 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T10:56:50.831 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T10:56:50.831 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.881 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T10:56:50.882 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T10:56:50.882 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.933 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T10:56:50.934 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T10:56:50.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:50.994 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T10:56:50.994 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T10:56:50.995 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.044 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T10:56:51.044 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T10:56:51.045 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.098 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T10:56:51.098 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T10:56:51.099 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T10:56:51.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.161 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T10:56:51.162 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T10:56:51.162 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.229 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T10:56:51.229 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T10:56:51.230 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T10:56:51.230 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.251 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T10:56:51.251 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T10:56:51.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.270 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T10:56:51.270 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T10:56:51.270 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T10:56:51.329 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T10:56:51.329 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T10:56:51.330 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'Error loading MDS rank' | egrep -v 'missing journal object' | egrep -v 'Error recovering journal' | egrep -v 'error decoding table object' | egrep -v 'failed to read JournalPointer' | egrep -v 'Corrupt directory entry' | egrep -v 'Corrupt fnode header' | egrep -v 'corrupt sessionmap header' | egrep -v 'Corrupt dentry' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_READ_ONLY | egrep -v 'force file system read-only' | egrep -v 'with standby daemon mds' | egrep -v 'MDS abort because newly corrupt dentry' | head -n 1 2024-09-10T10:56:51.425 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:51.426 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T10:56:51.661 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:51.662 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T10:56:51.796 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:51.796 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T10:56:51.906 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:56:51.907 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T10:56:52.042 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:56:52.043 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T10:56:52.343 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:56:52.344 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T10:56:52.465 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:56:52.466 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T10:56:52.565 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:56:52.566 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T10:56:52.672 INFO:tasks.ceph:Archiving mon data... 2024-09-10T10:56:52.673 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/data/mon.a.tgz 2024-09-10T10:56:52.674 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-09-10T10:56:52.711 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.Hy8ZHRJH4B 2024-09-10T10:56:52.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Hy8ZHRJH4B 2024-09-10T10:56:52.952 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.Hy8ZHRJH4B 2024-09-10T10:56:53.045 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.Hy8ZHRJH4B is 648KB 2024-09-10T10:56:53.140 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.Hy8ZHRJH4B 2024-09-10T10:56:53.177 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/data/mon.b.tgz 2024-09-10T10:56:53.177 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-10T10:56:53.213 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.me1wJ1qYSg 2024-09-10T10:56:53.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.me1wJ1qYSg 2024-09-10T10:56:53.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.me1wJ1qYSg 2024-09-10T10:56:53.550 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.me1wJ1qYSg is 670KB 2024-09-10T10:56:53.643 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.me1wJ1qYSg 2024-09-10T10:56:53.678 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/data/mon.c.tgz 2024-09-10T10:56:53.678 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-10T10:56:53.753 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.K5Bw7AhQNd 2024-09-10T10:56:53.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.K5Bw7AhQNd 2024-09-10T10:56:53.982 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.K5Bw7AhQNd 2024-09-10T10:56:54.075 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.K5Bw7AhQNd is 671KB 2024-09-10T10:56:54.166 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.K5Bw7AhQNd 2024-09-10T10:56:54.201 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T10:56:54.201 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:56:54.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T10:56:54.334 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:56:54.369 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T10:56:54.369 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T10:56:54.370 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi028/crash 2024-09-10T10:56:54.371 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:56:54.431 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi138/crash 2024-09-10T10:56:54.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T10:56:54.525 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T10:56:54.525 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:54.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:54.627 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:54.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T10:56:54.738 INFO:tasks.ceph:Compressing logs... 2024-09-10T10:56:54.738 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:56:54.742 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:56:54.793 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T10:56:54.794 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2024-09-10T10:56:54.795 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84113.log 2024-09-10T10:56:54.796 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69160.log 2024-09-10T10:56:54.796 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2024-09-10T10:56:54.797 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36121.log 2024-09-10T10:56:54.797 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35921.log 2024-09-10T10:56:54.798 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73008.log 2024-09-10T10:56:54.798 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84113.log.gz 2024-09-10T10:56:54.798 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55144.log 88.3% 2024-09-10T10:56:54.798 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69160.log.gz 2024-09-10T10:56:54.802 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36121.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T10:56:54.802 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41433.log 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35921.log.gz 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.41433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41433.log.gz 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T10:56:54.803 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36081.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70836.log 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55144.log.gz 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.73008.log.gz 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2024-09-10T10:56:54.804 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70836.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-09-10T10:56:54.805 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.69727.log 2024-09-10T10:56:54.805 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70836.log.gz 2024-09-10T10:56:54.805 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2024-09-10T10:56:54.805 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2024-09-10T10:56:54.810 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36041.log 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41832.log 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36041.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36041.log.gz 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.41832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41832.log.gz 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35961.log 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-09-10T10:56:54.811 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35961.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35961.log.gz -5 2024-09-10T10:56:54.812 INFO:teuthology.orchestra.run.smithi138.stderr: --verbose -- /var/log/ceph/ceph-client.3.41828.log 2024-09-10T10:56:54.812 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36201.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T10:56:54.812 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-09-10T10:56:54.812 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.41828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41828.log.gz 2024-09-10T10:56:54.812 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T10:56:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T10:56:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:56:54.815 INFO:teuthology.orchestra.run.smithi138.stderr: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:56:54.818 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50201.log 2024-09-10T10:56:54.818 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.69727.log.gz 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67069.log 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79307.log 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50201.log.gz 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67069.log.gz 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52947.log 2024-09-10T10:56:54.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54353.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.79307.log.gz 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60273.log 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44861.log 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52947.log.gz 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71066.log 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60273.log.gz 2024-09-10T10:56:54.820 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85210.log 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44861.log.gz 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71066.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71066.log.gz 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53325.log 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77636.log 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85210.log.gz 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62946.log 2024-09-10T10:56:54.821 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53325.log.gz 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73153.log 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77636.log.gz 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77317.log 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.62946.log.gz 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.73153.log.gz 2024-09-10T10:56:54.822 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60821.log 2024-09-10T10:56:54.823 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-09-10T10:56:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46475.log 2024-09-10T10:56:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.60821.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60821.log.gz 2024-09-10T10:56:54.823 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87569.log 2024-09-10T10:56:54.823 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.77317.log.gz 2024-09-10T10:56:54.825 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T10:56:54.825 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T10:56:54.825 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36001.log 2024-09-10T10:56:54.825 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.36001.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36001.log.gz 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58896.log 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46475.log.gz 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87569.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.54472.log 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87569.log.gz 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80777.log 2024-09-10T10:56:54.832 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58896.log.gz 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80073.log 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54472.log.gz 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86771.log 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80777.log.gz 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.80073.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80073.log.gz 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87065.log 2024-09-10T10:56:54.833 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45618.log 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.86771.log.gz 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87065.log.gz 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59369.log 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45618.log.gz 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45786.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2024-09-10T10:56:54.834 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44379.log 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76686.log 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59369.log.gz 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43576.log 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44379.log.gz 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80077.log 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76686.log.gz 2024-09-10T10:56:54.835 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.43576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43576.log.gz 2024-09-10T10:56:54.836 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53073.log 2024-09-10T10:56:54.836 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.80077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80077.log.gz 2024-09-10T10:56:54.836 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2024-09-10T10:56:54.836 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80204.log 2024-09-10T10:56:54.836 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53073.log.gz 2024-09-10T10:56:54.837 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T10:56:54.837 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41414.log 2024-09-10T10:56:54.837 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36161.log 2024-09-10T10:56:54.837 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.41414.log: 92.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:56:54.838 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41414.log.gz 2024-09-10T10:56:54.838 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36161.log.gz 2024-09-10T10:56:54.843 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64003.log 2024-09-10T10:56:54.843 INFO:teuthology.orchestra.run.smithi028.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2024-09-10T10:56:54.843 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45029.log 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80204.log.gz 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64003.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64003.log.gz 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50495.log 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64162.log 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69237.log 2024-09-10T10:56:54.844 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50495.log.gz 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49363.log 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48695.log 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69237.log.gz 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.64162.log.gz 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49363.log.gz 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81791.log 2024-09-10T10:56:54.845 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61478.log 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81791.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48695.log.gz 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85460.log 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81791.log.gz 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68322.log 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61478.log.gz 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85233.log 2024-09-10T10:56:54.846 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85460.log.gz 2024-09-10T10:56:54.847 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2024-09-10T10:56:54.847 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68322.log.gz 2024-09-10T10:56:54.851 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83461.log 2024-09-10T10:56:54.851 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47541.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.85233.log.gz 2024-09-10T10:56:54.851 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81419.log 2024-09-10T10:56:54.851 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64466.log 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.83461.log.gz 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78141.log 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81419.log.gz 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63092.log 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64466.log.gz 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78141.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78141.log.gz 2024-09-10T10:56:54.852 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67293.log 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49871.log 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63092.log.gz 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56264.log 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.67293.log.gz 2024-09-10T10:56:54.853 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49871.log.gz 2024-09-10T10:56:54.855 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63565.log 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56264.log.gz 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87107.log 2024-09-10T10:56:54.857 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50243.log 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63565.log.gz 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83800.log 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87107.log.gz 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50243.log: gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50243.log.gz -- 2024-09-10T10:56:54.858 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.42797.log 2024-09-10T10:56:54.859 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71946.log 2024-09-10T10:56:54.860 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83800.log.gz 2024-09-10T10:56:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73583.log 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.71946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71946.log.gz 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48517.log 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59579.log 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73583.log.gz 2024-09-10T10:56:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.48517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48517.log.gz 2024-09-10T10:56:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48329.log 2024-09-10T10:56:54.863 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57130.log 2024-09-10T10:56:54.863 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59579.log.gz 2024-09-10T10:56:54.863 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76298.log 88.6% 2024-09-10T10:56:54.863 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48329.log.gz 2024-09-10T10:56:54.864 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68597.log 2024-09-10T10:56:54.864 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57130.log.gz 2024-09-10T10:56:54.865 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54514.log 2024-09-10T10:56:54.865 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76298.log.gz 2024-09-10T10:56:54.865 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85819.log 2024-09-10T10:56:54.865 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68597.log.gz 2024-09-10T10:56:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68439.log 2024-09-10T10:56:54.866 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54514.log.gz 2024-09-10T10:56:54.866 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66745.log 2024-09-10T10:56:54.867 INFO:teuthology.orchestra.run.smithi028.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.85819.log.gz 2024-09-10T10:56:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57552.log 2024-09-10T10:56:54.867 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68439.log.gz 2024-09-10T10:56:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69297.log 2024-09-10T10:56:54.868 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66745.log.gz 2024-09-10T10:56:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57552.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.57552.log.gz -5 2024-09-10T10:56:54.869 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69667.log 2024-09-10T10:56:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86855.log 2024-09-10T10:56:54.870 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69297.log.gz 2024-09-10T10:56:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64914.log 2024-09-10T10:56:54.871 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69667.log.gz 2024-09-10T10:56:54.872 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76966.log 2024-09-10T10:56:54.872 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86855.log.gz 2024-09-10T10:56:54.872 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64914.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.64914.log.gz --verbose 2024-09-10T10:56:54.873 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.75911.log 2024-09-10T10:56:54.873 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66219.log 2024-09-10T10:56:54.873 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76966.log.gz 2024-09-10T10:56:54.874 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75682.log 2024-09-10T10:56:54.874 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75911.log.gz 2024-09-10T10:56:54.875 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55387.log 2024-09-10T10:56:54.875 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66219.log.gz 2024-09-10T10:56:54.876 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75682.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75682.log.gz 2024-09-10T10:56:54.876 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2024-09-10T10:56:54.877 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74724.log 2024-09-10T10:56:54.877 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55387.log.gz 2024-09-10T10:56:54.878 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72943.log 2024-09-10T10:56:54.878 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2024-09-10T10:56:54.878 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52416.log 2024-09-10T10:56:54.879 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74724.log.gz 2024-09-10T10:56:54.879 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72943.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.72943.log.gz 2024-09-10T10:56:54.880 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85071.log 2024-09-10T10:56:54.880 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76788.log 2024-09-10T10:56:54.880 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52416.log.gz 2024-09-10T10:56:54.881 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62422.log 2024-09-10T10:56:54.881 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85071.log.gz 2024-09-10T10:56:54.882 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59920.log 2024-09-10T10:56:54.882 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76788.log.gz 2024-09-10T10:56:54.883 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62422.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62422.log.gz 2024-09-10T10:56:54.883 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72438.log 2024-09-10T10:56:54.884 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59075.log 2024-09-10T10:56:54.884 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59920.log.gz 2024-09-10T10:56:54.885 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-09-10T10:56:54.885 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72438.log.gz 2024-09-10T10:56:54.886 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64531.log 2024-09-10T10:56:54.886 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59075.log.gz 2024-09-10T10:56:54.887 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46181.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-09-10T10:56:54.887 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55553.log 2024-09-10T10:56:54.887 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57216.log 2024-09-10T10:56:54.888 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55102.log 2024-09-10T10:56:54.888 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55553.log.gz 2024-09-10T10:56:54.889 INFO:teuthology.orchestra.run.smithi028.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.64531.log.gz 2024-09-10T10:56:54.890 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58094.log 2024-09-10T10:56:54.890 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57216.log.gz 2024-09-10T10:56:54.890 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55102.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T10:56:54.891 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55102.log.gz 2024-09-10T10:56:54.891 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69029.log 2024-09-10T10:56:54.891 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.1.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58094.log.gz 2024-09-10T10:56:54.892 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55186.log 2024-09-10T10:56:54.892 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.69029.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69029.log.gz 2024-09-10T10:56:54.893 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84025.log 2024-09-10T10:56:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52037.log 2024-09-10T10:56:54.899 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55186.log.gz 2024-09-10T10:56:54.899 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75178.log 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84025.log.gz 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52037.log.gz 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66272.log 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75178.log.gz 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42593.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2024-09-10T10:56:54.900 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78519.log 2024-09-10T10:56:54.901 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60441.log 2024-09-10T10:56:54.901 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66272.log.gz 2024-09-10T10:56:54.902 INFO:teuthology.orchestra.run.smithi138.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T10:56:54.903 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82180.log 2024-09-10T10:56:54.903 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78519.log.gz 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56366.log 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60441.log.gz 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82180.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82180.log.gz 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58368.log 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66614.log 2024-09-10T10:56:54.904 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56366.log.gz 2024-09-10T10:56:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41819.log 2024-09-10T10:56:54.905 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58368.log.gz 2024-09-10T10:56:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.66614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66614.log.gz 2024-09-10T10:56:54.906 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40581.log 2024-09-10T10:56:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85502.log 2024-09-10T10:56:54.907 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41819.log.gz 2024-09-10T10:56:54.907 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78912.log 2024-09-10T10:56:54.908 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40581.log.gz 2024-09-10T10:56:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62274.log 2024-09-10T10:56:54.908 INFO:teuthology.orchestra.run.smithi028.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.85502.log.gz 2024-09-10T10:56:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69550.log 2024-09-10T10:56:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62274.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.78912.log.gz 2024-09-10T10:56:54.910 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75060.log 2024-09-10T10:56:54.910 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62274.log.gz 2024-09-10T10:56:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61627.log 2024-09-10T10:56:54.911 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69550.log.gz 2024-09-10T10:56:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58685.log 2024-09-10T10:56:54.912 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75060.log.gz 2024-09-10T10:56:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87233.log 2024-09-10T10:56:54.913 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.61627.log.gz 2024-09-10T10:56:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65834.log 2024-09-10T10:56:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87233.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58685.log.gz 2024-09-10T10:56:54.915 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87233.log.gz 2024-09-10T10:56:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69901.log 2024-09-10T10:56:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.65834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42253.log 2024-09-10T10:56:54.916 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65834.log.gz 2024-09-10T10:56:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2024-09-10T10:56:54.918 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69901.log.gz 2024-09-10T10:56:54.919 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68364.log 2024-09-10T10:56:54.919 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42253.log.gz 2024-09-10T10:56:54.919 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49008.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2024-09-10T10:56:54.919 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71243.log 2024-09-10T10:56:54.920 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40749.log 2024-09-10T10:56:54.920 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68364.log.gz 2024-09-10T10:56:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71243.log.gz 2024-09-10T10:56:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50753.log 2024-09-10T10:56:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68541.log 2024-09-10T10:56:54.922 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40749.log.gz 2024-09-10T10:56:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.50753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50753.log.gz 2024-09-10T10:56:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61441.log 2024-09-10T10:56:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-09-10T10:56:54.924 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.68541.log.gz 2024-09-10T10:56:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69625.log 2024-09-10T10:56:54.925 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61441.log.gz 2024-09-10T10:56:54.926 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-09-10T10:56:54.927 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-09-10T10:56:54.927 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74049.log 2024-09-10T10:56:54.927 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69625.log.gz 2024-09-10T10:56:54.928 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48831.log 2024-09-10T10:56:54.928 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-09-10T10:56:54.929 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62983.log 2024-09-10T10:56:54.930 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74049.log.gz 2024-09-10T10:56:54.931 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59758.log 2024-09-10T10:56:54.931 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48831.log.gz 2024-09-10T10:56:54.931 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56138.log 2024-09-10T10:56:54.932 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.62983.log.gz 2024-09-10T10:56:54.932 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76112.log 2024-09-10T10:56:54.933 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59758.log.gz 2024-09-10T10:56:54.933 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56138.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56138.log.gz 2024-09-10T10:56:54.933 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56493.log 2024-09-10T10:56:54.934 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78435.log 2024-09-10T10:56:54.935 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76112.log.gz 2024-09-10T10:56:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56493.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56493.log.gz 2024-09-10T10:56:54.936 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78318.log 2024-09-10T10:56:54.937 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72227.log 2024-09-10T10:56:54.937 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78435.log.gz 2024-09-10T10:56:54.938 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78318.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78318.log.gz 2024-09-10T10:56:54.938 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67228.log 2024-09-10T10:56:54.939 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-09-10T10:56:54.939 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72227.log.gz 2024-09-10T10:56:54.940 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56577.log 2024-09-10T10:56:54.940 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67228.log.gz 2024-09-10T10:56:54.941 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66163.log 2024-09-10T10:56:54.941 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-09-10T10:56:54.942 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56577.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.56577.log.gz 2024-09-10T10:56:54.942 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47291.log 2024-09-10T10:56:54.943 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45534.log 2024-09-10T10:56:54.943 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66163.log.gz 2024-09-10T10:56:54.944 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78225.log 2024-09-10T10:56:54.944 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47291.log.gz 2024-09-10T10:56:54.945 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84281.log 2024-09-10T10:56:54.945 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45534.log.gz 2024-09-10T10:56:54.946 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80458.log 2024-09-10T10:56:54.947 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84281.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84281.log.gz 2024-09-10T10:56:54.947 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79166.log 2024-09-10T10:56:54.947 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.78225.log.gz 2024-09-10T10:56:54.948 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74335.log 2024-09-10T10:56:54.948 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80458.log.gz 2024-09-10T10:56:54.949 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.79166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79166.log.gz 2024-09-10T10:56:54.949 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85756.log 2024-09-10T10:56:54.950 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44253.log 2024-09-10T10:56:54.950 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74335.log.gz 2024-09-10T10:56:54.951 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.85756.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.85756.log.gz -5 2024-09-10T10:56:54.951 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59327.log 2024-09-10T10:56:54.952 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64199.log 2024-09-10T10:56:54.952 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44253.log.gz 2024-09-10T10:56:54.953 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47887.log 2024-09-10T10:56:54.953 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59327.log.gz 2024-09-10T10:56:54.954 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64078.log 2024-09-10T10:56:54.954 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64199.log.gz 2024-09-10T10:56:54.955 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69358.log 2024-09-10T10:56:54.955 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64078.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47887.log.gz 2024-09-10T10:56:54.956 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61775.log 2024-09-10T10:56:54.956 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64078.log.gz 2024-09-10T10:56:54.957 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-09-10T10:56:54.957 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.69358.log.gz 2024-09-10T10:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52458.log 2024-09-10T10:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.61775.log.gz 2024-09-10T10:56:54.959 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36081.log: gzip -5 --verbose -- 92.4% /var/log/ceph/ceph-client.admin.83983.log -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-09-10T10:56:54.959 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60525.log 2024-09-10T10:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52458.log.gz 2024-09-10T10:56:54.961 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87317.log 2024-09-10T10:56:54.961 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83983.log.gz 2024-09-10T10:56:54.962 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72713.log 2024-09-10T10:56:54.962 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60525.log.gz 2024-09-10T10:56:54.963 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87317.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.87317.log.gz 2024-09-10T10:56:54.963 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81691.log 2024-09-10T10:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82939.log 2024-09-10T10:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72713.log.gz 2024-09-10T10:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85568.log 2024-09-10T10:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81691.log.gz 2024-09-10T10:56:54.966 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61330.log 2024-09-10T10:56:54.966 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82939.log.gz 2024-09-10T10:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85568.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85568.log.gz 2024-09-10T10:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42083.log 2024-09-10T10:56:54.968 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67827.log 2024-09-10T10:56:54.969 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.61330.log.gz 2024-09-10T10:56:54.969 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42083.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.42083.log.gz --verbose 2024-09-10T10:56:54.970 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.71454.log 2024-09-10T10:56:54.970 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55281.log 2024-09-10T10:56:54.971 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67827.log.gz 2024-09-10T10:56:54.972 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61404.log 2024-09-10T10:56:54.972 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71454.log.gz 2024-09-10T10:56:54.972 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61738.log 2024-09-10T10:56:54.973 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55281.log.gz 2024-09-10T10:56:54.974 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79819.log 2024-09-10T10:56:54.974 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.61404.log.gz 2024-09-10T10:56:54.975 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44505.log 2024-09-10T10:56:54.975 INFO:teuthology.orchestra.run.smithi028.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.61738.log.gz 2024-09-10T10:56:54.976 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77965.log 2024-09-10T10:56:54.976 INFO:teuthology.orchestra.run.smithi028.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.79819.log.gz 2024-09-10T10:56:54.977 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64255.log 2024-09-10T10:56:54.977 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44505.log.gz 2024-09-10T10:56:54.978 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45702.log 2024-09-10T10:56:54.978 INFO:teuthology.orchestra.run.smithi028.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.77965.log.gz 2024-09-10T10:56:54.979 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77828.log 2024-09-10T10:56:54.979 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64255.log.gz 2024-09-10T10:56:54.980 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45702.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.45702.log.gz 2024-09-10T10:56:54.980 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.80800.log 2024-09-10T10:56:54.981 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.77828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77828.log.gz 2024-09-10T10:56:54.981 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30050.log 2024-09-10T10:56:54.982 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61590.log 2024-09-10T10:56:54.982 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.tmp-client.admin.30050.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30050.log.gz 2024-09-10T10:56:54.983 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-09-10T10:56:54.983 INFO:teuthology.orchestra.run.smithi028.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.80800.log.gz 2024-09-10T10:56:54.984 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2024-09-10T10:56:54.984 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61590.log.gz 2024-09-10T10:56:54.985 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71308.log 2024-09-10T10:56:54.985 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-09-10T10:56:54.986 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40665.log 2024-09-10T10:56:54.986 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2024-09-10T10:56:54.987 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57300.log 2024-09-10T10:56:54.988 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77463.log 2024-09-10T10:56:54.988 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40665.log.gz 2024-09-10T10:56:54.988 INFO:teuthology.orchestra.run.smithi028.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.71308.log.gz 2024-09-10T10:56:54.989 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86265.log 2024-09-10T10:56:54.989 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57300.log.gz 2024-09-10T10:56:54.990 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-09-10T10:56:54.990 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77463.log.gz 2024-09-10T10:56:54.991 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67438.log 2024-09-10T10:56:54.991 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86265.log.gz 2024-09-10T10:56:54.992 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60004.log 2024-09-10T10:56:54.993 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-09-10T10:56:54.993 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83149.log 2024-09-10T10:56:54.994 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67438.log.gz 2024-09-10T10:56:54.994 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45744.log 2024-09-10T10:56:54.995 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60004.log.gz 2024-09-10T10:56:54.995 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83149.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83149.log.gz -5 2024-09-10T10:56:54.995 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64694.log 2024-09-10T10:56:54.996 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65473.log 2024-09-10T10:56:54.997 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45744.log.gz 2024-09-10T10:56:54.997 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54598.log 2024-09-10T10:56:54.998 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64694.log.gz 2024-09-10T10:56:54.999 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76906.log 2024-09-10T10:56:55.000 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65473.log.gz 2024-09-10T10:56:55.000 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54598.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54598.log.gz 2024-09-10T10:56:55.000 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63919.log 2024-09-10T10:56:55.001 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78715.log 2024-09-10T10:56:55.001 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76906.log.gz 2024-09-10T10:56:55.001 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87023.log 2024-09-10T10:56:55.002 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63919.log.gz 2024-09-10T10:56:55.003 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68172.log 2024-09-10T10:56:55.003 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.78715.log.gz 2024-09-10T10:56:55.004 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87023.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.87023.log.gz 2024-09-10T10:56:55.004 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70700.log 2024-09-10T10:56:55.005 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84155.log 2024-09-10T10:56:55.005 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68172.log.gz 2024-09-10T10:56:55.006 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70700.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.70700.log.gz -- 2024-09-10T10:56:55.006 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.41675.log 2024-09-10T10:56:55.007 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82012.log 2024-09-10T10:56:55.007 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.84155.log.gz 2024-09-10T10:56:55.008 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41675.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.86813.log -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-09-10T10:56:55.008 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.009 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52290.log 2024-09-10T10:56:55.009 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82012.log.gz 2024-09-10T10:56:55.010 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86813.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86813.log.gzgzip 2024-09-10T10:56:55.010 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-09-10T10:56:55.011 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62459.log 2024-09-10T10:56:55.011 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52290.log.gz 2024-09-10T10:56:55.012 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42265.log 2024-09-10T10:56:55.012 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-09-10T10:56:55.013 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74437.log 2024-09-10T10:56:55.014 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62459.log.gz 2024-09-10T10:56:55.014 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59033.log 2024-09-10T10:56:55.015 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42265.log.gz 2024-09-10T10:56:55.015 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-09-10T10:56:55.016 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74437.log.gz 2024-09-10T10:56:55.016 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66426.log 2024-09-10T10:56:55.017 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59033.log.gz 2024-09-10T10:56:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82339.log 2024-09-10T10:56:55.018 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-09-10T10:56:55.018 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66426.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66426.log.gz 2024-09-10T10:56:55.019 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45660.log 2024-09-10T10:56:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57342.log 2024-09-10T10:56:55.020 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.82339.log.gz 2024-09-10T10:56:55.020 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45660.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.45660.log.gz 2024-09-10T10:56:55.021 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62311.log 2024-09-10T10:56:55.021 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2024-09-10T10:56:55.022 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57342.log.gz 2024-09-10T10:56:55.023 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46349.log 2024-09-10T10:56:55.023 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62311.log.gz 2024-09-10T10:56:55.023 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47132.log 2024-09-10T10:56:55.024 INFO:teuthology.orchestra.run.smithi138.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T10:56:55.024 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46349.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gz 2024-09-10T10:56:55.025 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose 88.5% -- /var/log/ceph/ceph-client.admin.46055.log -- replaced with /var/log/ceph/ceph-client.admin.46349.log.gz 2024-09-10T10:56:55.025 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.026 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-09-10T10:56:55.026 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47132.log.gz 2024-09-10T10:56:55.027 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71514.log 2024-09-10T10:56:55.027 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-09-10T10:56:55.028 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53983.log 2024-09-10T10:56:55.028 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-09-10T10:56:55.028 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70658.log 2024-09-10T10:56:55.029 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71514.log.gz 2024-09-10T10:56:55.030 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51564.log 2024-09-10T10:56:55.030 INFO:teuthology.orchestra.run.smithi028.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53983.log.gz 2024-09-10T10:56:55.031 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-09-10T10:56:55.031 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70658.log.gz 2024-09-10T10:56:55.032 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51564.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51564.log.gz 2024-09-10T10:56:55.032 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57702.log 2024-09-10T10:56:55.033 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58453.log 2024-09-10T10:56:55.033 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57702.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz 2024-09-10T10:56:55.034 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72090.log 2024-09-10T10:56:55.034 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57702.log.gz 2024-09-10T10:56:55.035 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56451.log 2024-09-10T10:56:55.035 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58453.log.gz 2024-09-10T10:56:55.036 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76728.log 2024-09-10T10:56:55.036 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72090.log.gz 2024-09-10T10:56:55.037 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53493.log 2024-09-10T10:56:55.037 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56451.log.gz 2024-09-10T10:56:55.038 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51893.log 2024-09-10T10:56:55.038 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.76728.log.gz 2024-09-10T10:56:55.039 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53493.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53493.log.gz 2024-09-10T10:56:55.039 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72765.log 2024-09-10T10:56:55.040 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75803.log 2024-09-10T10:56:55.040 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-09-10T10:56:55.041 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59537.log 2024-09-10T10:56:55.041 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72765.log.gz 2024-09-10T10:56:55.042 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43256.log 2024-09-10T10:56:55.043 INFO:teuthology.orchestra.run.smithi028.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.75803.log.gz 2024-09-10T10:56:55.043 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59537.log.gz 2024-09-10T10:56:55.043 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40899.log 2024-09-10T10:56:55.044 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64796.log 2024-09-10T10:56:55.045 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43256.log.gz 2024-09-10T10:56:55.045 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66042.log 2024-09-10T10:56:55.045 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40899.log.gz 2024-09-10T10:56:55.046 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59838.log 2024-09-10T10:56:55.047 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64796.log.gz 2024-09-10T10:56:55.047 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42264.log 2024-09-10T10:56:55.047 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66042.log.gz 2024-09-10T10:56:55.049 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40144.log 2024-09-10T10:56:55.050 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59838.log.gz 2024-09-10T10:56:55.050 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76340.log 2024-09-10T10:56:55.050 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42264.log.gz 2024-09-10T10:56:55.051 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51456.log 2024-09-10T10:56:55.051 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40144.log.gz 2024-09-10T10:56:55.051 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62163.log 87.6% 2024-09-10T10:56:55.052 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76340.log.gz 2024-09-10T10:56:55.052 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55842.log 2024-09-10T10:56:55.053 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62163.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.51456.log.gz 2024-09-10T10:56:55.054 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76070.log 2024-09-10T10:56:55.054 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62163.log.gz 2024-09-10T10:56:55.055 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50155.log 2024-09-10T10:56:55.055 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76070.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55842.log.gz 2024-09-10T10:56:55.056 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76070.log.gz 2024-09-10T10:56:55.056 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40310.log 2024-09-10T10:56:55.057 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49405.log 2024-09-10T10:56:55.057 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50155.log.gz 2024-09-10T10:56:55.058 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46859.log 2024-09-10T10:56:55.058 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40310.log.gz 2024-09-10T10:56:55.059 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49405.log: gzip -5 89.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.49405.log.gz -- 2024-09-10T10:56:55.059 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.68481.log 2024-09-10T10:56:55.060 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.46859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46859.log.gz 2024-09-10T10:56:55.060 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82497.log 2024-09-10T10:56:55.061 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76400.log 2024-09-10T10:56:55.061 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68481.log.gz 2024-09-10T10:56:55.061 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80837.log 2024-09-10T10:56:55.062 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82497.log.gz 2024-09-10T10:56:55.063 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48521.log 2024-09-10T10:56:55.063 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.76400.log.gz 2024-09-10T10:56:55.064 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72555.log 2024-09-10T10:56:55.064 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.80837.log.gz 2024-09-10T10:56:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.48521.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48521.log.gz 2024-09-10T10:56:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-09-10T10:56:55.066 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-09-10T10:56:55.066 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72555.log.gz 2024-09-10T10:56:55.067 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41999.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T10:56:55.067 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-09-10T10:56:55.069 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87191.log 2024-09-10T10:56:55.069 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-09-10T10:56:55.070 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46009.log 2024-09-10T10:56:55.071 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87191.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.87191.log.gz 2024-09-10T10:56:55.071 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51606.log 2024-09-10T10:56:55.072 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44694.log 2024-09-10T10:56:55.072 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46009.log.gz 2024-09-10T10:56:55.073 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51606.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.51606.log.gz -5 2024-09-10T10:56:55.074 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.81834.log 2024-09-10T10:56:55.074 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T10:56:55.074 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-09-10T10:56:55.075 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44694.log.gz 2024-09-10T10:56:55.075 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53451.log 2024-09-10T10:56:55.075 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81834.log.gz 2024-09-10T10:56:55.077 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56535.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-09-10T10:56:55.077 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53793.log 2024-09-10T10:56:55.077 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73255.log 2024-09-10T10:56:55.077 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53451.log.gz 2024-09-10T10:56:55.078 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.53793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53793.log.gz 2024-09-10T10:56:55.078 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61812.log 2024-09-10T10:56:55.079 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-09-10T10:56:55.079 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.73255.log.gz 2024-09-10T10:56:55.081 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43595.log 2024-09-10T10:56:55.081 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61812.log.gz 2024-09-10T10:56:55.081 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69783.log 2024-09-10T10:56:55.081 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-09-10T10:56:55.083 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.43595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43595.log.gz 2024-09-10T10:56:55.084 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78771.log 2024-09-10T10:56:55.084 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41759.log 2024-09-10T10:56:55.084 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69783.log.gz 2024-09-10T10:56:55.085 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78771.log.gz 2024-09-10T10:56:55.086 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67270.log 2024-09-10T10:56:55.086 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70938.log 2024-09-10T10:56:55.086 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41759.log.gz 2024-09-10T10:56:55.087 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82128.log 2024-09-10T10:56:55.087 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67270.log.gz 2024-09-10T10:56:55.088 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82441.log 2024-09-10T10:56:55.088 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.70938.log.gz 2024-09-10T10:56:55.088 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82128.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82128.log.gz 2024-09-10T10:56:55.088 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.85372.log 2024-09-10T10:56:55.089 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74227.log 2024-09-10T10:56:55.090 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.82441.log.gz 2024-09-10T10:56:55.091 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85372.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85372.log.gz 2024-09-10T10:56:55.091 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55060.log 2024-09-10T10:56:55.091 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43007.log 2024-09-10T10:56:55.092 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74227.log.gz 2024-09-10T10:56:55.092 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55060.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55060.log.gz 2024-09-10T10:56:55.092 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56054.log 2024-09-10T10:56:55.093 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73779.log 2024-09-10T10:56:55.093 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43007.log.gz 2024-09-10T10:56:55.094 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56054.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56054.log.gz 2024-09-10T10:56:55.095 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61293.log 2024-09-10T10:56:55.095 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-09-10T10:56:55.096 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73779.log.gz 2024-09-10T10:56:55.096 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80641.log 2024-09-10T10:56:55.097 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61293.log.gz 2024-09-10T10:56:55.101 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73045.log 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80641.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.80641.log.gz 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87149.log 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72597.log 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.73045.log.gz 2024-09-10T10:56:55.102 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87149.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87149.log.gz 2024-09-10T10:56:55.103 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55741.log 2024-09-10T10:56:55.103 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58991.log 2024-09-10T10:56:55.105 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72597.log.gz 2024-09-10T10:56:55.106 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.55741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55741.log.gz 2024-09-10T10:56:55.106 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70373.log 2024-09-10T10:56:55.106 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82381.log 2024-09-10T10:56:55.106 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58991.log.gz 2024-09-10T10:56:55.107 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70373.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70373.log.gz 2024-09-10T10:56:55.107 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47475.log 2024-09-10T10:56:55.112 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71646.log 2024-09-10T10:56:55.112 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82381.log.gz 2024-09-10T10:56:55.113 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44784.log 2024-09-10T10:56:55.113 INFO:teuthology.orchestra.run.smithi028.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47475.log.gz 2024-09-10T10:56:55.113 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45372.log 2024-09-10T10:56:55.114 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44784.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.71646.log.gz 2024-09-10T10:56:55.114 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44784.log.gz 2024-09-10T10:56:55.114 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86939.log 2024-09-10T10:56:55.114 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64568.log 2024-09-10T10:56:55.115 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45372.log: /var/log/ceph/ceph-client.admin.86939.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45372.log.gz -5 2024-09-10T10:56:55.115 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74125.log 2024-09-10T10:56:55.115 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86939.log.gz 2024-09-10T10:56:55.116 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56703.log 2024-09-10T10:56:55.116 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64568.log.gz 2024-09-10T10:56:55.125 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41956.log 2024-09-10T10:56:55.125 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74125.log.gz 2024-09-10T10:56:55.126 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54766.log 2024-09-10T10:56:55.126 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56703.log.gz 2024-09-10T10:56:55.126 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41956.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41956.log.gz 2024-09-10T10:56:55.126 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65083.log 2024-09-10T10:56:55.127 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50113.log 2024-09-10T10:56:55.127 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54766.log.gz 2024-09-10T10:56:55.127 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65185.log 2024-09-10T10:56:55.127 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65083.log.gz 2024-09-10T10:56:55.128 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61886.log 2024-09-10T10:56:55.128 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50113.log.gz 2024-09-10T10:56:55.128 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65185.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65185.log.gz 2024-09-10T10:56:55.129 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70144.log 2024-09-10T10:56:55.129 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84323.log 2024-09-10T10:56:55.129 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.61886.log.gz 2024-09-10T10:56:55.129 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78262.log 2024-09-10T10:56:55.130 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84323.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.70144.log.gz 2024-09-10T10:56:55.130 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84323.log.gz 2024-09-10T10:56:55.130 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57468.log 2024-09-10T10:56:55.131 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70878.log 2024-09-10T10:56:55.131 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78262.log.gz 2024-09-10T10:56:55.131 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62089.log 2024-09-10T10:56:55.131 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57468.log.gz 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70289.log 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70878.log.gz 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54700.log 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.62089.log.gz 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70289.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70289.log.gzgzip 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36436.log 2024-09-10T10:56:55.138 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44421.log 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36436.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54700.log.gz 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36436.log.gz 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86223.log 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70181.log 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44421.log.gz 2024-09-10T10:56:55.139 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70237.log 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86223.log.gz 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83419.log 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.70181.log.gz 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70237.log.gz 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54430.log 2024-09-10T10:56:55.140 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83419.log.gz 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46097.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz /var/log/ceph/ceph-client.admin.74167.log 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54430.log.gz 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74167.log: gzip -5 89.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.74167.log.gz -- 2024-09-10T10:56:55.141 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.68135.log 2024-09-10T10:56:55.142 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T10:56:55.144 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44036.log 2024-09-10T10:56:55.144 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz 2024-09-10T10:56:55.144 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84407.log 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.2.44036.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44036.log.gz 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.68135.log.gz 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78063.log 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66360.log 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84407.log.gz 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52374.log 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78063.log.gz 2024-09-10T10:56:55.145 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84239.log 2024-09-10T10:56:55.146 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66360.log.gz 2024-09-10T10:56:55.147 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81556.log 2024-09-10T10:56:55.147 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52374.log.gz 2024-09-10T10:56:55.147 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84239.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84239.log.gz 2024-09-10T10:56:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70760.log 2024-09-10T10:56:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47194.log 2024-09-10T10:56:55.148 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81556.log.gz 2024-09-10T10:56:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79567.log 2024-09-10T10:56:55.149 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70760.log.gz 2024-09-10T10:56:55.149 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86181.log 2024-09-10T10:56:55.150 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47194.log.gz 2024-09-10T10:56:55.150 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83191.log 2024-09-10T10:56:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86181.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.79567.log.gz 2024-09-10T10:56:55.151 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.86181.log.gz 2024-09-10T10:56:55.151 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72807.log 2024-09-10T10:56:55.152 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60315.log 2024-09-10T10:56:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83191.log.gz 2024-09-10T10:56:55.153 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57384.log 2024-09-10T10:56:55.153 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72807.log.gz 2024-09-10T10:56:55.154 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-09-10T10:56:55.154 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60315.log.gz 2024-09-10T10:56:55.155 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73947.log 2024-09-10T10:56:55.155 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57384.log.gz 2024-09-10T10:56:55.156 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80365.log 2024-09-10T10:56:55.156 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-09-10T10:56:55.157 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52815.log 2024-09-10T10:56:55.157 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73947.log.gz 2024-09-10T10:56:55.158 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46855.log 2024-09-10T10:56:55.159 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71942.log 2024-09-10T10:56:55.159 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52815.log.gz 2024-09-10T10:56:55.160 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.80365.log.gz 2024-09-10T10:56:55.160 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.46855.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.46855.log.gz 2024-09-10T10:56:55.160 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61219.log 2024-09-10T10:56:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.71942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71942.log.gz 2024-09-10T10:56:55.161 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49531.log 2024-09-10T10:56:55.162 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62644.log 2024-09-10T10:56:55.162 INFO:teuthology.orchestra.run.smithi028.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.61219.log.gz 2024-09-10T10:56:55.163 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49531.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.49531.log.gz 2024-09-10T10:56:55.163 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-09-10T10:56:55.164 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75859.log 2024-09-10T10:56:55.164 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62644.log.gz 2024-09-10T10:56:55.165 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64120.log 2024-09-10T10:56:55.165 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-09-10T10:56:55.166 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75859.log: gzip -5 --verbose 88.4% -- -- replaced with /var/log/ceph/ceph-client.admin.75859.log.gz 2024-09-10T10:56:55.166 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.70994.log 2024-09-10T10:56:55.167 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79265.log 2024-09-10T10:56:55.167 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64120.log.gz 2024-09-10T10:56:55.168 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70994.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70994.log.gz 2024-09-10T10:56:55.168 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80557.log 2024-09-10T10:56:55.169 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42262.log 2024-09-10T10:56:55.170 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79265.log.gz 2024-09-10T10:56:55.170 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80557.log.gz 2024-09-10T10:56:55.170 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73719.log 2024-09-10T10:56:55.171 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69321.log 2024-09-10T10:56:55.172 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42262.log.gz 2024-09-10T10:56:55.172 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49748.log 2024-09-10T10:56:55.172 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73719.log.gz 2024-09-10T10:56:55.173 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74513.log 2024-09-10T10:56:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49748.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.69321.log.gz 2024-09-10T10:56:55.175 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36347.log 2024-09-10T10:56:55.175 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49748.log.gz 2024-09-10T10:56:55.176 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73839.log 2024-09-10T10:56:55.176 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74513.log.gz 2024-09-10T10:56:55.177 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.36347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58284.log 2024-09-10T10:56:55.177 INFO:teuthology.orchestra.run.smithi028.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36347.log.gz 2024-09-10T10:56:55.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62015.log 2024-09-10T10:56:55.178 INFO:teuthology.orchestra.run.smithi028.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.73839.log.gz 2024-09-10T10:56:55.179 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72251.log 2024-09-10T10:56:55.179 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58284.log.gz 2024-09-10T10:56:55.180 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40833.log 2024-09-10T10:56:55.180 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62015.log.gz 2024-09-10T10:56:55.181 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-09-10T10:56:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40833.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.72251.log.gz 2024-09-10T10:56:55.182 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81598.log 2024-09-10T10:56:55.183 INFO:teuthology.orchestra.run.smithi028.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.40833.log.gz 2024-09-10T10:56:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74944.log 2024-09-10T10:56:55.183 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-09-10T10:56:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76634.log 2024-09-10T10:56:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74944.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.81598.log.gz 2024-09-10T10:56:55.185 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74944.log.gz 2024-09-10T10:56:55.185 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78870.log 2024-09-10T10:56:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42839.log 2024-09-10T10:56:55.186 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76634.log.gz 2024-09-10T10:56:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77022.log 2024-09-10T10:56:55.187 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78870.log.gz 2024-09-10T10:56:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77176.log 2024-09-10T10:56:55.188 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42839.log.gz 2024-09-10T10:56:55.189 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62570.log 2024-09-10T10:56:55.189 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77022.log.gz 2024-09-10T10:56:55.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73489.log 2024-09-10T10:56:55.190 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77176.log.gz 2024-09-10T10:56:55.191 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77116.log 2024-09-10T10:56:55.191 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62570.log.gz 2024-09-10T10:56:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73489.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T10:56:55.192 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73489.log.gz 2024-09-10T10:56:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45071.log 2024-09-10T10:56:55.194 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77116.log.gz 2024-09-10T10:56:55.194 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79713.log 2024-09-10T10:56:55.195 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45071.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45071.log.gz 2024-09-10T10:56:55.196 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80599.log 2024-09-10T10:56:55.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57991.log 2024-09-10T10:56:55.197 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79713.log.gz 2024-09-10T10:56:55.198 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80599.log.gz 2024-09-10T10:56:55.198 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70448.log 2024-09-10T10:56:55.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63702.log 2024-09-10T10:56:55.199 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57991.log.gz 2024-09-10T10:56:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70448.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70448.log.gz 2024-09-10T10:56:55.200 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65420.log 2024-09-10T10:56:55.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74190.log 2024-09-10T10:56:55.201 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63702.log.gz 2024-09-10T10:56:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65420.log.gz 2024-09-10T10:56:55.202 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69985.log 2024-09-10T10:56:55.203 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79660.log 2024-09-10T10:56:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69985.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69985.log.gz 2024-09-10T10:56:55.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85414.log 2024-09-10T10:56:55.205 INFO:teuthology.orchestra.run.smithi028.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.74190.log.gz 2024-09-10T10:56:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-09-10T10:56:55.206 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79660.log.gz 2024-09-10T10:56:55.206 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85414.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85414.log.gz 2024-09-10T10:56:55.206 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68733.log 2024-09-10T10:56:55.207 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74555.log 2024-09-10T10:56:55.208 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2024-09-10T10:56:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68733.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68733.log.gz 2024-09-10T10:56:55.208 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72480.log 2024-09-10T10:56:55.209 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53409.log 2024-09-10T10:56:55.209 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74555.log.gz 2024-09-10T10:56:55.210 INFO:teuthology.orchestra.run.smithi138.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T10:56:55.211 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72480.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72480.log.gz 2024-09-10T10:56:55.211 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80402.log 2024-09-10T10:56:55.212 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62496.log 2024-09-10T10:56:55.212 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53409.log.gz 2024-09-10T10:56:55.213 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78678.log 2024-09-10T10:56:55.213 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.80402.log.gz 2024-09-10T10:56:55.213 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77252.log 2024-09-10T10:56:55.214 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62496.log.gz 2024-09-10T10:56:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75766.log 2024-09-10T10:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77252.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.78678.log.gz 2024-09-10T10:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68775.log 2024-09-10T10:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77252.log.gz 2024-09-10T10:56:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77570.log 2024-09-10T10:56:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75766.log: /var/log/ceph/ceph-client.admin.68775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71712.log 2024-09-10T10:56:55.218 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.68775.log.gz 2024-09-10T10:56:55.219 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.75766.log.gz 2024-09-10T10:56:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42582.log 2024-09-10T10:56:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71712.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.77570.log.gz 2024-09-10T10:56:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55745.log 2024-09-10T10:56:55.220 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71712.log.gz 2024-09-10T10:56:55.221 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69690.log 2024-09-10T10:56:55.222 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42582.log.gz 2024-09-10T10:56:55.222 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.55745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55745.log.gz 2024-09-10T10:56:55.222 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86981.log 2024-09-10T10:56:55.223 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75286.log 2024-09-10T10:56:55.224 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69690.log: /var/log/ceph/ceph-client.admin.86981.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86981.log.gz 2024-09-10T10:56:55.225 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40354.log 2024-09-10T10:56:55.225 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82699.log 2024-09-10T10:56:55.226 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.69690.log.gz 2024-09-10T10:56:55.226 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.75286.log.gz 2024-09-10T10:56:55.227 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59201.log 88.2% 2024-09-10T10:56:55.227 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40354.log.gz 2024-09-10T10:56:55.227 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81100.log 2024-09-10T10:56:55.228 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82699.log.gz 2024-09-10T10:56:55.229 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59201.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59201.log.gz 2024-09-10T10:56:55.229 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45828.log 2024-09-10T10:56:55.230 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59453.log 2024-09-10T10:56:55.230 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81100.log.gz 2024-09-10T10:56:55.231 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45828.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.45828.log.gz 2024-09-10T10:56:55.231 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.0.60825.log 2024-09-10T10:56:55.232 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59453.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59453.log.gz 2024-09-10T10:56:55.232 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-10T10:56:55.233 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.60825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77528.log 2024-09-10T10:56:55.233 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60825.log.gz 2024-09-10T10:56:55.234 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78389.log 2024-09-10T10:56:55.234 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-10T10:56:55.236 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49489.log 2024-09-10T10:56:55.236 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77528.log.gz 2024-09-10T10:56:55.237 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78389.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78389.log.gz 2024-09-10T10:56:55.237 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58033.log 2024-09-10T10:56:55.238 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73541.log 2024-09-10T10:56:55.238 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58033.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49489.log.gz 2024-09-10T10:56:55.239 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45277.log 2024-09-10T10:56:55.239 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58033.log.gz 2024-09-10T10:56:55.240 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71604.log 2024-09-10T10:56:55.240 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73541.log.gz 2024-09-10T10:56:55.242 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45277.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45277.log.gz 2024-09-10T10:56:55.242 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80281.log 2024-09-10T10:56:55.242 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48263.log 2024-09-10T10:56:55.243 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71604.log.gz 2024-09-10T10:56:55.244 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50369.log 2024-09-10T10:56:55.244 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80281.log.gz 2024-09-10T10:56:55.245 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73433.log 2024-09-10T10:56:55.245 INFO:teuthology.orchestra.run.smithi028.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48263.log.gz 2024-09-10T10:56:55.246 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64937.log 2024-09-10T10:56:55.246 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50369.log.gz/var/log/ceph/ceph-client.admin.73433.log: 2024-09-10T10:56:55.247 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69033.log 2024-09-10T10:56:55.247 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73433.log.gz 2024-09-10T10:56:55.248 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41862.log 2024-09-10T10:56:55.249 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.69033.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.64937.log.gz 2024-09-10T10:56:55.250 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69033.log.gz 2024-09-10T10:56:55.250 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85148.log 2024-09-10T10:56:55.251 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83716.log 2024-09-10T10:56:55.251 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41862.log.gz 2024-09-10T10:56:55.252 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85148.log.gz 2024-09-10T10:56:55.252 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75478.log 2024-09-10T10:56:55.253 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52122.log 2024-09-10T10:56:55.254 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83716.log.gz 2024-09-10T10:56:55.254 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.75478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75478.log.gz 2024-09-10T10:56:55.254 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59159.log 2024-09-10T10:56:55.255 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80511.log 2024-09-10T10:56:55.256 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52122.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52122.log.gz 2024-09-10T10:56:55.256 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59159.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.62923.log -- replaced with /var/log/ceph/ceph-client.admin.59159.log.gz 2024-09-10T10:56:55.256 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.257 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-10T10:56:55.258 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80511.log.gz 2024-09-10T10:56:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62923.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62923.log.gz 2024-09-10T10:56:55.258 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75474.log 2024-09-10T10:56:55.259 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81635.log 2024-09-10T10:56:55.260 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-10T10:56:55.260 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.75474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75474.log.gz 2024-09-10T10:56:55.260 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72288.log 2024-09-10T10:56:55.261 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59663.log 2024-09-10T10:56:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81635.log: /var/log/ceph/ceph-client.admin.72288.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.81635.log.gz 2024-09-10T10:56:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72867.log 89.3% 2024-09-10T10:56:55.262 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72288.log.gz 2024-09-10T10:56:55.263 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63763.log 2024-09-10T10:56:55.264 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59663.log.gz 2024-09-10T10:56:55.264 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72867.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72867.log.gz 2024-09-10T10:56:55.265 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65261.log 2024-09-10T10:56:55.265 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63763.log: -- /var/log/ceph/ceph-client.admin.70513.log 2024-09-10T10:56:55.266 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65261.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63763.log.gz 2024-09-10T10:56:55.267 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65261.log.gz 2024-09-10T10:56:55.267 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54241.log 2024-09-10T10:56:55.267 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50957.log 2024-09-10T10:56:55.268 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.54241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54241.log.gz 2024-09-10T10:56:55.269 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86897.log 2024-09-10T10:56:55.269 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.70513.log.gz 2024-09-10T10:56:55.270 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45215.log 2024-09-10T10:56:55.270 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50957.log: /var/log/ceph/ceph-client.admin.86897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50957.log.gz 2024-09-10T10:56:55.271 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86897.log.gz 2024-09-10T10:56:55.271 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71754.log 2024-09-10T10:56:55.272 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60048.log 2024-09-10T10:56:55.272 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45215.log.gz 2024-09-10T10:56:55.273 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71754.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71754.log.gz 2024-09-10T10:56:55.273 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52571.log 2024-09-10T10:56:55.274 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53535.log 2024-09-10T10:56:55.274 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60048.log.gz 2024-09-10T10:56:55.275 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52571.log.gz 2024-09-10T10:56:55.275 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46601.log 2024-09-10T10:56:55.276 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42263.log 2024-09-10T10:56:55.277 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46601.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53535.log.gz 2024-09-10T10:56:55.277 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T10:56:55.278 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46601.log.gz 2024-09-10T10:56:55.278 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61079.log 2024-09-10T10:56:55.278 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42263.log.gz 2024-09-10T10:56:55.279 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52164.log 2024-09-10T10:56:55.280 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.61079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78477.log 2024-09-10T10:56:55.280 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61079.log.gz 2024-09-10T10:56:55.281 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.52164.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.61923.log 2024-09-10T10:56:55.282 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52164.log.gz 2024-09-10T10:56:55.283 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62237.log 2024-09-10T10:56:55.283 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78477.log.gz 2024-09-10T10:56:55.283 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70490.log 2024-09-10T10:56:55.284 INFO:teuthology.orchestra.run.smithi028.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.61923.log.gz 2024-09-10T10:56:55.285 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85318.log 2024-09-10T10:56:55.285 INFO:teuthology.orchestra.run.smithi028.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.62237.log.gz 2024-09-10T10:56:55.286 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87527.log 2024-09-10T10:56:55.286 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70490.log.gz 2024-09-10T10:56:55.287 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86139.log 2024-09-10T10:56:55.287 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85318.log.gz 2024-09-10T10:56:55.288 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67480.log 2024-09-10T10:56:55.288 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87527.log.gz 2024-09-10T10:56:55.289 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74902.log 2024-09-10T10:56:55.290 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86139.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86139.log.gz 2024-09-10T10:56:55.291 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87359.log 2024-09-10T10:56:55.291 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67480.log.gz 2024-09-10T10:56:55.292 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74902.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.74902.log.gz /var/log/ceph/ceph-client.admin.47387.log 2024-09-10T10:56:55.292 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.292 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82072.log 2024-09-10T10:56:55.293 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87359.log.gz 2024-09-10T10:56:55.294 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47387.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47387.log.gz 2024-09-10T10:56:55.294 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54658.log 2024-09-10T10:56:55.295 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82896.log 2024-09-10T10:56:55.295 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82072.log.gz 2024-09-10T10:56:55.296 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61164.log 2024-09-10T10:56:55.296 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54658.log.gz 2024-09-10T10:56:55.297 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52206.log 2024-09-10T10:56:55.297 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.82896.log.gz 2024-09-10T10:56:55.298 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57258.log 2024-09-10T10:56:55.299 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52206.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.61164.log.gz 2024-09-10T10:56:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-09-10T10:56:55.300 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52206.log.gz 2024-09-10T10:56:55.300 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70550.log 2024-09-10T10:56:55.301 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57258.log.gz 2024-09-10T10:56:55.306 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49668.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83846.log 2024-09-10T10:56:55.306 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50561.log 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.70550.log.gz 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80716.log 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83846.log.gz 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50561.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56409.log 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50561.log.gz 2024-09-10T10:56:55.307 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73101.log 2024-09-10T10:56:55.308 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80716.log.gz 2024-09-10T10:56:55.308 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56409.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.56409.log.gz -5 2024-09-10T10:56:55.309 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.86536.log 2024-09-10T10:56:55.309 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70331.log 2024-09-10T10:56:55.309 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73101.log.gz 2024-09-10T10:56:55.310 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86536.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86536.log.gz 2024-09-10T10:56:55.310 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65303.log 2024-09-10T10:56:55.310 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.70331.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54049.log 2024-09-10T10:56:55.311 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70331.log.gz 2024-09-10T10:56:55.312 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65303.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65303.log.gz 2024-09-10T10:56:55.312 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65838.log 2024-09-10T10:56:55.312 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57426.log 2024-09-10T10:56:55.313 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54049.log.gz 2024-09-10T10:56:55.313 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.65838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65838.log.gz 2024-09-10T10:56:55.314 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64736.log 2024-09-10T10:56:55.315 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57426.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57426.log.gz 2024-09-10T10:56:55.315 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58537.log 2024-09-10T10:56:55.316 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44295.log 2024-09-10T10:56:55.317 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1%/var/log/ceph/ceph-client.admin.58537.log: -- replaced with /var/log/ceph/ceph-client.admin.64736.log.gz 2024-09-10T10:56:55.317 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 88.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58537.log.gz 2024-09-10T10:56:55.317 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.62607.log 2024-09-10T10:56:55.318 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84197.log 2024-09-10T10:56:55.319 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44295.log.gz 2024-09-10T10:56:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61256.log 2024-09-10T10:56:55.320 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62607.log.gz 2024-09-10T10:56:55.320 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87443.log 2024-09-10T10:56:55.321 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.84197.log.gz 2024-09-10T10:56:55.322 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43049.log 2024-09-10T10:56:55.322 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61256.log.gz 2024-09-10T10:56:55.323 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57636.log 2024-09-10T10:56:55.323 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87443.log.gz 2024-09-10T10:56:55.324 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45113.log 2024-09-10T10:56:55.324 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43049.log.gz 2024-09-10T10:56:55.325 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83556.log 2024-09-10T10:56:55.325 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57636.log.gz 2024-09-10T10:56:55.326 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76518.log 2024-09-10T10:56:55.326 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45113.log.gz 2024-09-10T10:56:55.328 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84491.log 2024-09-10T10:56:55.328 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83556.log.gz 2024-09-10T10:56:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76518.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.76518.log.gz 2024-09-10T10:56:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70060.log 2024-09-10T10:56:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42600.log 2024-09-10T10:56:55.330 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84491.log: /var/log/ceph/ceph-client.admin.70060.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.84491.log.gz 2024-09-10T10:56:55.331 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70060.log.gz 2024-09-10T10:56:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60483.log 2024-09-10T10:56:55.331 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56306.log 2024-09-10T10:56:55.332 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42600.log.gz 2024-09-10T10:56:55.333 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45870.log 2024-09-10T10:56:55.333 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60483.log.gz 2024-09-10T10:56:55.334 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67111.log 2024-09-10T10:56:55.334 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56306.log.gz 2024-09-10T10:56:55.335 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45870.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T10:56:55.335 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45870.log.gz 2024-09-10T10:56:55.336 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77354.log 2024-09-10T10:56:55.336 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67111.log.gz 2024-09-10T10:56:55.337 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78201.log 2024-09-10T10:56:55.339 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86307.log 2024-09-10T10:56:55.339 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.77354.log.gz 2024-09-10T10:56:55.340 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78201.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78201.log.gz -5 2024-09-10T10:56:55.340 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77294.log 2024-09-10T10:56:55.341 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49321.log 2024-09-10T10:56:55.341 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.86307.log.gz 2024-09-10T10:56:55.343 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51977.log 2024-09-10T10:56:55.343 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77294.log.gz 2024-09-10T10:56:55.344 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42250.log 2024-09-10T10:56:55.344 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49321.log.gz 2024-09-10T10:56:55.345 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68228.log 2024-09-10T10:56:55.345 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51977.log.gz 2024-09-10T10:56:55.346 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75953.log 2024-09-10T10:56:55.346 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42250.log.gz 2024-09-10T10:56:55.348 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53283.log 2024-09-10T10:56:55.348 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68228.log.gz 2024-09-10T10:56:55.349 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44032.log 2024-09-10T10:56:55.349 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75953.log.gz 2024-09-10T10:56:55.350 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69508.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53283.log.gz 2024-09-10T10:56:55.350 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.351 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.2.44032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44032.log.gz 2024-09-10T10:56:55.352 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53031.log 2024-09-10T10:56:55.353 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40048.log 2024-09-10T10:56:55.353 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69508.log.gz 2024-09-10T10:56:55.354 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53031.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53031.log.gz 2024-09-10T10:56:55.355 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86410.log 2024-09-10T10:56:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40048.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40048.log.gz 2024-09-10T10:56:55.356 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50327.log 2024-09-10T10:56:55.357 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62348.log 2024-09-10T10:56:55.357 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86410.log.gz 2024-09-10T10:56:55.358 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50327.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67718.log 2024-09-10T10:56:55.359 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50327.log.gz 2024-09-10T10:56:55.360 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79604.log 2024-09-10T10:56:55.360 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62348.log.gz 2024-09-10T10:56:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67718.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.67718.log.gz 2024-09-10T10:56:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50285.log 2024-09-10T10:56:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79604.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.79604.log.gz 2024-09-10T10:56:55.363 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50453.log 2024-09-10T10:56:55.364 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62385.log 2024-09-10T10:56:55.364 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50285.log.gz 2024-09-10T10:56:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77923.log 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50453.log.gz 2024-09-10T10:56:55.366 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62385.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62385.log.gz 2024-09-10T10:56:55.367 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61664.log 2024-09-10T10:56:55.369 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2024-09-10T10:56:55.369 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77923.log.gz 2024-09-10T10:56:55.370 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61664.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61664.log.gz 2024-09-10T10:56:55.370 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84365.log 2024-09-10T10:56:55.371 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51276.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-09-10T10:56:55.372 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45576.log 2024-09-10T10:56:55.373 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62792.log 2024-09-10T10:56:55.373 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84365.log.gz 2024-09-10T10:56:55.374 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45576.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45576.log.gz 2024-09-10T10:56:55.375 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-09-10T10:56:55.376 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76172.log 2024-09-10T10:56:55.376 INFO:teuthology.orchestra.run.smithi028.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.62792.log.gz 2024-09-10T10:56:55.377 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64349.log 2024-09-10T10:56:55.377 INFO:teuthology.orchestra.run.smithi028.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-09-10T10:56:55.378 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76172.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.76172.log.gz 2024-09-10T10:56:55.379 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64349.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64349.log.gz 2024-09-10T10:56:55.380 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61701.log 2024-09-10T10:56:55.381 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61849.log 2024-09-10T10:56:55.383 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60227.log 2024-09-10T10:56:55.383 INFO:teuthology.orchestra.run.smithi028.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.61701.log.gz 2024-09-10T10:56:55.384 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61849.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.61849.log.gz 2024-09-10T10:56:55.385 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60227.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60227.log.gz 2024-09-10T10:56:55.385 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61515.log 2024-09-10T10:56:55.386 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60092.log 2024-09-10T10:56:55.388 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75605.log 2024-09-10T10:56:55.388 INFO:teuthology.orchestra.run.smithi028.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.61515.log.gz 2024-09-10T10:56:55.389 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71345.log 2024-09-10T10:56:55.389 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60092.log.gz 2024-09-10T10:56:55.390 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75605.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75605.log.gz 2024-09-10T10:56:55.391 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.71345.log: --verbose -- /var/log/ceph/ceph-client.admin.83636.log 2024-09-10T10:56:55.392 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.71345.log.gz 2024-09-10T10:56:55.392 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83275.log 2024-09-10T10:56:55.394 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83107.log 2024-09-10T10:56:55.394 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83636.log.gz 2024-09-10T10:56:55.395 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83275.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83275.log.gz 2024-09-10T10:56:55.396 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83107.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83107.log.gz 2024-09-10T10:56:55.397 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60567.log 2024-09-10T10:56:55.398 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T10:56:55.399 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71168.log 2024-09-10T10:56:55.400 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mgr.y.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60567.log.gz 2024-09-10T10:56:55.400 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47729.log 2024-09-10T10:56:55.402 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74283.log 2024-09-10T10:56:55.402 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71168.log.gz 2024-09-10T10:56:55.403 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.47729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47729.log.gz 2024-09-10T10:56:55.404 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75995.log 2024-09-10T10:56:55.406 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74283.log.gz 2024-09-10T10:56:55.406 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40707.log 2024-09-10T10:56:55.407 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75995.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75995.log.gz 2024-09-10T10:56:55.408 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72657.log 2024-09-10T10:56:55.409 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40707.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40707.log.gz 2024-09-10T10:56:55.410 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73373.log 2024-09-10T10:56:55.411 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72657.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.72657.log.gz 2024-09-10T10:56:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73373.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T10:56:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82794.log 2024-09-10T10:56:55.413 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73373.log.gz 2024-09-10T10:56:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41325.log 2024-09-10T10:56:55.415 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67892.log 2024-09-10T10:56:55.415 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82794.log.gz 2024-09-10T10:56:55.417 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42259.log 2024-09-10T10:56:55.419 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67892.log.gz 2024-09-10T10:56:55.419 INFO:teuthology.orchestra.run.smithi028.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.41325.log.gzgzip 2024-09-10T10:56:55.419 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.81876.log/var/log/ceph/ceph-client.admin.42259.log: 2024-09-10T10:56:55.419 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42259.log.gz 2024-09-10T10:56:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81034.log 2024-09-10T10:56:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67386.log 2024-09-10T10:56:55.421 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81876.log.gz 2024-09-10T10:56:55.422 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73989.log 2024-09-10T10:56:55.423 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0%/var/log/ceph/ceph-client.admin.67386.log: -- replaced with /var/log/ceph/ceph-client.admin.81034.log.gz 2024-09-10T10:56:55.424 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66126.log 2024-09-10T10:56:55.424 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67386.log.gz 2024-09-10T10:56:55.425 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45958.log 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73989.log.gz 2024-09-10T10:56:55.425 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44169.log 2024-09-10T10:56:55.429 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45958.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.66126.log.gz 2024-09-10T10:56:55.429 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45958.log.gz 2024-09-10T10:56:55.429 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44169.log: -- /var/log/ceph/ceph-client.admin.49236.log 2024-09-10T10:56:55.429 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44169.log.gz 2024-09-10T10:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81292.log 2024-09-10T10:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67774.log 2024-09-10T10:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49236.log.gz 2024-09-10T10:56:55.432 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.81292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81292.log.gz 2024-09-10T10:56:55.433 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63961.log 2024-09-10T10:56:55.434 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55961.log 2024-09-10T10:56:55.434 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67774.log.gz 2024-09-10T10:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71114.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63961.log.gz 2024-09-10T10:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.436 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60143.log 2024-09-10T10:56:55.437 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55961.log.gz 2024-09-10T10:56:55.438 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87724.log 2024-09-10T10:56:55.438 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71114.log.gz 2024-09-10T10:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60143.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60143.log.gz 2024-09-10T10:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76578.log 2024-09-10T10:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87724.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87724.log.gz 2024-09-10T10:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83758.log 2024-09-10T10:56:55.442 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82836.log 2024-09-10T10:56:55.442 INFO:teuthology.orchestra.run.smithi028.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.76578.log.gz 2024-09-10T10:56:55.444 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83758.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.83758.log.gz 2024-09-10T10:56:55.444 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82836.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T10:56:55.445 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82836.log.gz 2024-09-10T10:56:55.445 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53367.log 2024-09-10T10:56:55.447 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84533.log 2024-09-10T10:56:55.448 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53367.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53367.log.gz 2024-09-10T10:56:55.449 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61553.log 2024-09-10T10:56:55.450 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84533.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.84533.log.gz 2024-09-10T10:56:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61553.log: 87.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T10:56:55.452 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.61553.log.gz 2024-09-10T10:56:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2024-09-10T10:56:55.453 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-09-10T10:56:55.454 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44337.log 2024-09-10T10:56:55.455 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2024-09-10T10:56:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42881.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-09-10T10:56:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58775.log 2024-09-10T10:56:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44337.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44337.log.gz 2024-09-10T10:56:55.458 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62200.log 2024-09-10T10:56:55.459 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57594.log 2024-09-10T10:56:55.460 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58775.log.gz 2024-09-10T10:56:55.461 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62200.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62200.log.gz 2024-09-10T10:56:55.461 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83317.log/var/log/ceph/ceph-client.admin.57594.log: 2024-09-10T10:56:55.462 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57594.log.gz 2024-09-10T10:56:55.462 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40449.log 2024-09-10T10:56:55.464 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65364.log 2024-09-10T10:56:55.464 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83317.log.gz 2024-09-10T10:56:55.465 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40449.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-09-10T10:56:55.466 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84449.log 2024-09-10T10:56:55.467 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53157.log 2024-09-10T10:56:55.467 INFO:teuthology.orchestra.run.smithi028.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.65364.log.gz 2024-09-10T10:56:55.468 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51216.log 2024-09-10T10:56:55.468 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84449.log.gz 2024-09-10T10:56:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53157.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53157.log.gz 2024-09-10T10:56:55.470 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.51216.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54976.log 2024-09-10T10:56:55.471 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51216.log.gz 2024-09-10T10:56:55.471 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53906.log 2024-09-10T10:56:55.473 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69943.log 2024-09-10T10:56:55.473 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54976.log.gz 2024-09-10T10:56:55.474 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83233.log 2024-09-10T10:56:55.474 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53906.log.gz 2024-09-10T10:56:55.476 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62681.log 2024-09-10T10:56:55.476 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69943.log.gz 2024-09-10T10:56:55.477 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81744.log 2024-09-10T10:56:55.477 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83233.log.gz 2024-09-10T10:56:55.478 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62681.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62681.log.gz 2024-09-10T10:56:55.479 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53241.log 2024-09-10T10:56:55.479 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.81744.log.gz 2024-09-10T10:56:55.480 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65646.log 2024-09-10T10:56:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55228.log 2024-09-10T10:56:55.482 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53241.log.gz 2024-09-10T10:56:55.483 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65646.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65646.log.gz 2024-09-10T10:56:55.483 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86055.log 2024-09-10T10:56:55.484 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55228.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55228.log.gz 2024-09-10T10:56:55.485 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63039.log 2024-09-10T10:56:55.486 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68280.log 2024-09-10T10:56:55.486 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86055.log.gz 2024-09-10T10:56:55.487 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63039.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63039.log.gz 2024-09-10T10:56:55.488 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-09-10T10:56:55.488 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68280.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68280.log.gz 2024-09-10T10:56:55.489 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62718.log 2024-09-10T10:56:55.490 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44903.log 2024-09-10T10:56:55.490 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-09-10T10:56:55.491 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62718.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62718.log.gz 2024-09-10T10:56:55.492 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56661.log 2024-09-10T10:56:55.493 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44903.log.gz 2024-09-10T10:56:55.493 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86452.log 2024-09-10T10:56:55.495 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61141.log 2024-09-10T10:56:55.495 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56661.log.gz 2024-09-10T10:56:55.496 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86452.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86452.log.gz 2024-09-10T10:56:55.497 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64872.log 2024-09-10T10:56:55.497 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61141.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61141.log.gz 2024-09-10T10:56:55.498 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75113.log 2024-09-10T10:56:55.499 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50002.log 2024-09-10T10:56:55.499 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64872.log.gz 2024-09-10T10:56:55.501 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75113.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40791.log 2024-09-10T10:56:55.501 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75113.log.gz 2024-09-10T10:56:55.502 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50002.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83065.log 2024-09-10T10:56:55.502 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50002.log.gz 2024-09-10T10:56:55.503 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40791.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40791.log.gz 2024-09-10T10:56:55.504 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83023.log 2024-09-10T10:56:55.505 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83065.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42601.log 2024-09-10T10:56:55.505 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83065.log.gz 2024-09-10T10:56:55.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67330.log 2024-09-10T10:56:55.509 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83023.log.gz 2024-09-10T10:56:55.509 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42601.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42601.log.gz 2024-09-10T10:56:55.510 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.67330.log: --verbose -- /var/log/ceph/ceph-client.admin.40268.log 2024-09-10T10:56:55.510 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67330.log.gz 2024-09-10T10:56:55.510 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85962.log 2024-09-10T10:56:55.511 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74377.log 2024-09-10T10:56:55.511 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40268.log.gz 2024-09-10T10:56:55.513 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.85962.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85962.log.gz/var/log/ceph/ceph-client.admin.74377.log: 2024-09-10T10:56:55.513 INFO:teuthology.orchestra.run.smithi028.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T10:56:55.514 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74377.log.gz 2024-09-10T10:56:55.514 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79777.log 2024-09-10T10:56:55.515 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64508.log 2024-09-10T10:56:55.516 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79777.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79777.log.gz 2024-09-10T10:56:55.516 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.72985.log 2024-09-10T10:56:55.517 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64508.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64508.log.gz 2024-09-10T10:56:55.517 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50749.log 2024-09-10T10:56:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72344.log 2024-09-10T10:56:55.519 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72985.log.gz 2024-09-10T10:56:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.50749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50749.log.gz 2024-09-10T10:56:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68051.log 2024-09-10T10:56:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82981.log 2024-09-10T10:56:55.521 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72344.log: /var/log/ceph/ceph-client.admin.68051.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72344.log.gz 2024-09-10T10:56:55.522 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73195.log 2024-09-10T10:56:55.522 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68051.log.gz 2024-09-10T10:56:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60357.log 2024-09-10T10:56:55.523 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82981.log.gz 2024-09-10T10:56:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73195.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73195.log.gz 2024-09-10T10:56:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78978.log 2024-09-10T10:56:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77410.log 2024-09-10T10:56:55.526 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.60357.log.gz 2024-09-10T10:56:55.526 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80992.log 2024-09-10T10:56:55.526 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78978.log.gz 2024-09-10T10:56:55.527 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77410.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.86494.log -- replaced with /var/log/ceph/ceph-client.admin.77410.log.gz 2024-09-10T10:56:55.528 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.528 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57002.log 2024-09-10T10:56:55.529 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.80992.log.gz 2024-09-10T10:56:55.529 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86494.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52861.log 2024-09-10T10:56:55.530 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86494.log.gz 2024-09-10T10:56:55.531 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56180.log 2024-09-10T10:56:55.531 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57002.log.gz 2024-09-10T10:56:55.532 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52861.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52861.log.gz 2024-09-10T10:56:55.532 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-09-10T10:56:55.533 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-09-10T10:56:55.533 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56180.log.gz 2024-09-10T10:56:55.534 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47433.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-09-10T10:56:55.534 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56745.log 2024-09-10T10:56:55.535 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86097.log 2024-09-10T10:56:55.535 INFO:teuthology.orchestra.run.smithi028.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-09-10T10:56:55.536 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56745.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.56745.log.gz -- 2024-09-10T10:56:55.537 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.0.79170.log 2024-09-10T10:56:55.537 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49279.log 2024-09-10T10:56:55.538 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86097.log.gz 2024-09-10T10:56:55.538 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.79170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79170.log.gz 2024-09-10T10:56:55.539 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75742.log 2024-09-10T10:56:55.539 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87401.log 2024-09-10T10:56:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49279.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49279.log.gz/var/log/ceph/ceph-client.admin.75742.log: 2024-09-10T10:56:55.540 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-09-10T10:56:55.541 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75742.log.gz 2024-09-10T10:56:55.541 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66618.log 2024-09-10T10:56:55.542 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87401.log.gz 2024-09-10T10:56:55.542 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49050.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-09-10T10:56:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73643.log 2024-09-10T10:56:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.66618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53789.log 2024-09-10T10:56:55.544 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66618.log.gz 2024-09-10T10:56:55.544 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73643.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.73643.log.gz 2024-09-10T10:56:55.545 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.87485.log 2024-09-10T10:56:55.546 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71285.log 2024-09-10T10:56:55.546 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53789.log.gz 2024-09-10T10:56:55.547 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61002.log 2024-09-10T10:56:55.547 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87485.log.gz 2024-09-10T10:56:55.548 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82404.log 2024-09-10T10:56:55.548 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71285.log.gz 2024-09-10T10:56:55.549 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63819.log 2024-09-10T10:56:55.549 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61002.log.gz 2024-09-10T10:56:55.550 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67017.log 2024-09-10T10:56:55.551 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63819.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48179.log 2024-09-10T10:56:55.551 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63819.log.gz 2024-09-10T10:56:55.552 INFO:teuthology.orchestra.run.smithi028.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.82404.log.gz 2024-09-10T10:56:55.552 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81288.log 2024-09-10T10:56:55.553 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67017.log.gz 2024-09-10T10:56:55.554 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-09-10T10:56:55.554 INFO:teuthology.orchestra.run.smithi028.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48179.log.gz 2024-09-10T10:56:55.555 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.81288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63180.log 2024-09-10T10:56:55.555 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.81288.log.gz 2024-09-10T10:56:55.556 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45155.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-09-10T10:56:55.556 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-09-10T10:56:55.557 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67616.log 2024-09-10T10:56:55.558 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46139.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.63180.log.gz 2024-09-10T10:56:55.558 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48036.log 2024-09-10T10:56:55.558 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-09-10T10:56:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54556.log 2024-09-10T10:56:55.560 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67616.log.gz 2024-09-10T10:56:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48036.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48036.log.gz 2024-09-10T10:56:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76135.log 2024-09-10T10:56:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.54556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47345.log 2024-09-10T10:56:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54556.log.gz 2024-09-10T10:56:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52905.log 2024-09-10T10:56:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47345.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76135.log.gz 2024-09-10T10:56:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74967.log 2024-09-10T10:56:55.564 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47345.log.gz 2024-09-10T10:56:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51935.log 2024-09-10T10:56:55.566 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52905.log.gz 2024-09-10T10:56:55.566 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76476.log 2024-09-10T10:56:55.567 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51935.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.74967.log.gz 2024-09-10T10:56:55.567 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56840.log 2024-09-10T10:56:55.568 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51935.log.gz 2024-09-10T10:56:55.568 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81496.log 2024-09-10T10:56:55.569 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76476.log.gz 2024-09-10T10:56:55.569 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56840.log.gz 2024-09-10T10:56:55.570 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74615.log 2024-09-10T10:56:55.570 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42599.log 2024-09-10T10:56:55.571 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81496.log.gz 2024-09-10T10:56:55.572 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74615.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.74615.log.gz 2024-09-10T10:56:55.572 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82264.log 2024-09-10T10:56:55.572 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47055.log 2024-09-10T10:56:55.573 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42599.log.gz 2024-09-10T10:56:55.574 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82264.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82264.log.gz 2024-09-10T10:56:55.574 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-09-10T10:56:55.574 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82615.log 2024-09-10T10:56:55.575 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47055.log.gz 2024-09-10T10:56:55.576 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44616.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-09-10T10:56:55.576 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57890.log 2024-09-10T10:56:55.577 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76228.log 2024-09-10T10:56:55.577 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.82615.log.gz 2024-09-10T10:56:55.578 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.57890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57890.log.gz 2024-09-10T10:56:55.578 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54237.log 2024-09-10T10:56:55.579 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60633.log 2024-09-10T10:56:55.579 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76228.log: /var/log/ceph/ceph-client.0.54237.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76228.log.gz 2024-09-10T10:56:55.579 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54237.log.gz 2024-09-10T10:56:55.580 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73895.log 2024-09-10T10:56:55.581 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.60633.log: --verbose -- /var/log/ceph/ceph-client.admin.73802.log 2024-09-10T10:56:55.582 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60633.log.gz 2024-09-10T10:56:55.582 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56222.log 2024-09-10T10:56:55.582 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73895.log.gz 2024-09-10T10:56:55.583 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44211.log 2024-09-10T10:56:55.585 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65327.log 2024-09-10T10:56:55.585 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56222.log.gz 2024-09-10T10:56:55.585 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.73802.log.gz 2024-09-10T10:56:55.586 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68504.log 2024-09-10T10:56:55.586 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44211.log.gz 2024-09-10T10:56:55.586 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76864.log 2024-09-10T10:56:55.588 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-09-10T10:56:55.588 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.65327.log.gz 2024-09-10T10:56:55.589 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76864.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.68504.log.gz 2024-09-10T10:56:55.589 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72396.log 2024-09-10T10:56:55.589 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.76864.log.gz 2024-09-10T10:56:55.590 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53601.log 2024-09-10T10:56:55.590 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-09-10T10:56:55.591 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82222.log 2024-09-10T10:56:55.591 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72396.log.gz 2024-09-10T10:56:55.593 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65125.log 2024-09-10T10:56:55.593 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53601.log.gz 2024-09-10T10:56:55.594 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82222.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.82222.log.gz 2024-09-10T10:56:55.594 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84787.log 2024-09-10T10:56:55.594 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74578.log 2024-09-10T10:56:55.595 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65125.log.gz 2024-09-10T10:56:55.595 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.84787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84787.log.gz 2024-09-10T10:56:55.596 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78824.log 2024-09-10T10:56:55.597 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67934.log 2024-09-10T10:56:55.598 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74578.log: /var/log/ceph/ceph-client.admin.78824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59962.log 2024-09-10T10:56:55.598 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78824.log.gz 2024-09-10T10:56:55.599 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.74578.log.gz 2024-09-10T10:56:55.599 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78594.log 2024-09-10T10:56:55.600 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.67934.log.gz 2024-09-10T10:56:55.600 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59962.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61978.log 2024-09-10T10:56:55.600 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59962.log.gz 2024-09-10T10:56:55.601 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.78594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58411.log 2024-09-10T10:56:55.602 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.78594.log.gz 2024-09-10T10:56:55.603 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42594.log 2024-09-10T10:56:55.603 INFO:teuthology.orchestra.run.smithi028.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.61978.log.gz 2024-09-10T10:56:55.603 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87642.log 2024-09-10T10:56:55.604 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.58411.log.gz 2024-09-10T10:56:55.605 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65538.log 2024-09-10T10:56:55.605 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42594.log.gz 2024-09-10T10:56:55.606 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87682.log 2024-09-10T10:56:55.606 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87642.log.gz 2024-09-10T10:56:55.607 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55323.log 2024-09-10T10:56:55.607 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65538.log.gz 2024-09-10T10:56:55.608 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87682.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76541.log 2024-09-10T10:56:55.608 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87682.log.gz 2024-09-10T10:56:55.609 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76929.log 2024-09-10T10:56:55.609 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55323.log.gz 2024-09-10T10:56:55.610 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63246.log 2024-09-10T10:56:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.76929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58161.log 2024-09-10T10:56:55.613 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.76541.log.gz 2024-09-10T10:56:55.613 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63246.log: gzip -5 --verbose -- 92.4% -- replaced with /var/log/ceph/ceph-client.admin.76929.log.gz 2024-09-10T10:56:55.614 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63246.log.gz 2024-09-10T10:56:55.614 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.52989.log 2024-09-10T10:56:55.614 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55429.log 2024-09-10T10:56:55.614 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58161.log.gz 2024-09-10T10:56:55.615 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51721.log 2024-09-10T10:56:55.616 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52989.log.gz 2024-09-10T10:56:55.616 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56096.log 2024-09-10T10:56:55.617 INFO:teuthology.orchestra.run.smithi028.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55429.log.gz 2024-09-10T10:56:55.618 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57894.log 2024-09-10T10:56:55.618 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51721.log.gz 2024-09-10T10:56:55.619 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62881.log 2024-09-10T10:56:55.619 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56096.log.gz 2024-09-10T10:56:55.620 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.57894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57894.log.gz 2024-09-10T10:56:55.620 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80893.log 2024-09-10T10:56:55.621 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58495.log 2024-09-10T10:56:55.621 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62881.log.gz/var/log/ceph/ceph-client.admin.80893.log: 2024-09-10T10:56:55.622 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.80893.log.gz 2024-09-10T10:56:55.622 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63138.log 2024-09-10T10:56:55.623 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66318.log 2024-09-10T10:56:55.624 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7%/var/log/ceph/ceph-client.admin.63138.log: -- replaced with /var/log/ceph/ceph-client.admin.58495.log.gz 2024-09-10T10:56:55.624 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63138.log.gz 2024-09-10T10:56:55.624 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79543.log 2024-09-10T10:56:55.625 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41556.log 2024-09-10T10:56:55.626 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66318.log.gz 2024-09-10T10:56:55.626 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79543.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79543.log.gz 2024-09-10T10:56:55.626 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69836.log 2024-09-10T10:56:55.627 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41556.log: /var/log/ceph/ceph-client.admin.50411.log 2024-09-10T10:56:55.628 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41556.log.gz 2024-09-10T10:56:55.628 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69836.log.gz 2024-09-10T10:56:55.629 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84071.log 2024-09-10T10:56:55.629 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58621.log 2024-09-10T10:56:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50411.log: /var/log/ceph/ceph-client.admin.84071.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50411.log.gz 2024-09-10T10:56:55.631 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84071.log.gz 2024-09-10T10:56:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46667.log 2024-09-10T10:56:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70901.log 2024-09-10T10:56:55.632 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58621.log.gz 2024-09-10T10:56:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80341.log 2024-09-10T10:56:55.633 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2024-09-10T10:56:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42923.log 2024-09-10T10:56:55.635 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53199.log 2024-09-10T10:56:55.635 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.70901.log.gz 2024-09-10T10:56:55.635 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80341.log.gz 2024-09-10T10:56:55.636 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42923.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40226.log 2024-09-10T10:56:55.636 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42923.log.gz 2024-09-10T10:56:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51851.log 2024-09-10T10:56:55.638 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53199.log.gz 2024-09-10T10:56:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51522.log 2024-09-10T10:56:55.639 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40226.log.gz 2024-09-10T10:56:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68112.log 2024-09-10T10:56:55.640 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51851.log.gz 2024-09-10T10:56:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60399.log 2024-09-10T10:56:55.641 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51522.log.gz 2024-09-10T10:56:55.642 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56619.log 2024-09-10T10:56:55.642 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68112.log.gz 2024-09-10T10:56:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59621.log 2024-09-10T10:56:55.643 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60399.log.gz 2024-09-10T10:56:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56619.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56619.log.gz 2024-09-10T10:56:55.644 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77824.log 2024-09-10T10:56:55.645 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53115.log 2024-09-10T10:56:55.646 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59621.log.gz 2024-09-10T10:56:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.77824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77824.log.gz 2024-09-10T10:56:55.646 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64391.log 2024-09-10T10:56:55.648 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.53115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67681.log 2024-09-10T10:56:55.648 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53115.log.gz 2024-09-10T10:56:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60185.log 2024-09-10T10:56:55.649 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64391.log.gz 2024-09-10T10:56:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.67681.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71401.log 2024-09-10T10:56:55.651 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.60185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49447.log 2024-09-10T10:56:55.651 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60185.log.gz 2024-09-10T10:56:55.652 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.67681.log.gz 2024-09-10T10:56:55.652 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83941.log 2024-09-10T10:56:55.652 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71401.log.gz 2024-09-10T10:56:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65580.log 2024-09-10T10:56:55.653 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49447.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49447.log.gz 2024-09-10T10:56:55.654 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59285.log 2024-09-10T10:56:55.654 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83941.log.gz 2024-09-10T10:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67153.log 2024-09-10T10:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.65580.log.gz 2024-09-10T10:56:55.656 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85760.log 2024-09-10T10:56:55.656 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59285.log.gz 2024-09-10T10:56:55.657 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55018.log 2024-09-10T10:56:55.658 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67153.log.gz 2024-09-10T10:56:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.85760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85760.log.gz 2024-09-10T10:56:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84791.log 2024-09-10T10:56:55.660 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.55018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61367.log 2024-09-10T10:56:55.660 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55018.log.gz 2024-09-10T10:56:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.84791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.84791.log.gz 2024-09-10T10:56:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82550.log 2024-09-10T10:56:55.661 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62533.log 2024-09-10T10:56:55.663 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.61367.log: /var/log/ceph/ceph-client.admin.82550.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.61367.log.gz 2024-09-10T10:56:55.663 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82550.log.gz 2024-09-10T10:56:55.663 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67976.log 2024-09-10T10:56:55.664 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70606.log 2024-09-10T10:56:55.664 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62533.log.gz 2024-09-10T10:56:55.665 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47733.log 2024-09-10T10:56:55.665 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67976.log.gz 2024-09-10T10:56:55.666 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70606.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67658.log 2024-09-10T10:56:55.666 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70606.log.gz 2024-09-10T10:56:55.667 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.47733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68650.log 2024-09-10T10:56:55.667 INFO:teuthology.orchestra.run.smithi028.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47733.log.gz 2024-09-10T10:56:55.668 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74826.log 2024-09-10T10:56:55.668 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67658.log.gz 2024-09-10T10:56:55.669 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68650.log.gz 2024-09-10T10:56:55.670 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73396.log 2024-09-10T10:56:55.670 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41498.log 2024-09-10T10:56:55.671 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74826.log.gz 2024-09-10T10:56:55.672 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66900.log 2024-09-10T10:56:55.673 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-09-10T10:56:55.673 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.73396.log.gz 2024-09-10T10:56:55.673 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41498.log.gz 2024-09-10T10:56:55.674 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63642.log 2024-09-10T10:56:55.675 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66900.log.gz 2024-09-10T10:56:55.675 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51360.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-09-10T10:56:55.675 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57174.log 2024-09-10T10:56:55.676 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57510.log 2024-09-10T10:56:55.677 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63642.log.gz 2024-09-10T10:56:55.677 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49913.log 2024-09-10T10:56:55.677 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57174.log.gz 2024-09-10T10:56:55.678 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40398.log 2024-09-10T10:56:55.679 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57510.log.gz 2024-09-10T10:56:55.679 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49913.log.gz 2024-09-10T10:56:55.679 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77074.log 2024-09-10T10:56:55.680 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56920.log 2024-09-10T10:56:55.681 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40398.log.gz 2024-09-10T10:56:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.77074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52651.log 2024-09-10T10:56:55.682 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77074.log.gz 2024-09-10T10:56:55.682 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.56920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-09-10T10:56:55.683 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56920.log.gz 2024-09-10T10:56:55.684 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52651.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.52651.log.gz -5 2024-09-10T10:56:55.684 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46391.log 2024-09-10T10:56:55.685 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47845.log 2024-09-10T10:56:55.685 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-09-10T10:56:55.686 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46391.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46391.log.gz 2024-09-10T10:56:55.686 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-09-10T10:56:55.687 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.47845.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.86729.log 2024-09-10T10:56:55.688 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47845.log.gz 2024-09-10T10:56:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-09-10T10:56:55.688 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52080.log 2024-09-10T10:56:55.689 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63872.log 2024-09-10T10:56:55.690 INFO:teuthology.orchestra.run.smithi028.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86729.log.gz 2024-09-10T10:56:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52080.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52080.log.gz 2024-09-10T10:56:55.690 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57086.log 2024-09-10T10:56:55.691 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.63872.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63726.log 2024-09-10T10:56:55.692 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57086.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63872.log.gz 2024-09-10T10:56:55.693 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57086.log.gz 2024-09-10T10:56:55.693 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64974.log 2024-09-10T10:56:55.693 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.63726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72167.log 2024-09-10T10:56:55.695 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87275.log 2024-09-10T10:56:55.695 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.64974.log.gz 2024-09-10T10:56:55.696 INFO:teuthology.orchestra.run.smithi028.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.63726.log.gz 2024-09-10T10:56:55.696 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63434.log 2024-09-10T10:56:55.696 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72167.log.gz 2024-09-10T10:56:55.697 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.87275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65030.log 2024-09-10T10:56:55.697 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87275.log.gz 2024-09-10T10:56:55.698 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.63434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63434.log.gz 2024-09-10T10:56:55.698 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66924.log 2024-09-10T10:56:55.699 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.65030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79483.log 2024-09-10T10:56:55.699 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65030.log.gz 2024-09-10T10:56:55.700 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79885.log 2024-09-10T10:56:55.701 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79483.log: 92.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-09-10T10:56:55.702 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66924.log.gz 2024-09-10T10:56:55.702 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79483.log.gz 2024-09-10T10:56:55.703 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79885.log: gzip -5 --verbose 88.0% -- /var/log/ceph/ceph-client.admin.58579.log -- replaced with /var/log/ceph/ceph-client.admin.79885.log.gz 2024-09-10T10:56:55.703 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.704 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86367.log 2024-09-10T10:56:55.704 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-09-10T10:56:55.705 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82035.log 2024-09-10T10:56:55.705 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58579.log.gz 2024-09-10T10:56:55.706 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.86367.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86367.log.gz 2024-09-10T10:56:55.706 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49176.log 2024-09-10T10:56:55.707 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75004.log 2024-09-10T10:56:55.708 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.49176.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49176.log.gz 2024-09-10T10:56:55.708 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59243.log 2024-09-10T10:56:55.713 INFO:teuthology.orchestra.run.smithi028.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.82035.log.gz 2024-09-10T10:56:55.713 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5/var/log/ceph/ceph-client.admin.75004.log: --verbose -- /var/log/ceph/ceph-client.admin.68841.log 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59243.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.75004.log.gz 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84599.log 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59243.log.gz 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.68841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75220.log 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68841.log.gz 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81951.log 2024-09-10T10:56:55.714 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84599.log.gz 2024-09-10T10:56:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.75220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51034.log 2024-09-10T10:56:55.715 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.81951.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.75220.log.gz 2024-09-10T10:56:55.716 INFO:teuthology.orchestra.run.smithi028.stderr:gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81951.log.gz -5 2024-09-10T10:56:55.716 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.74766.log 2024-09-10T10:56:55.717 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40623.log 2024-09-10T10:56:55.717 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51034.log.gz 2024-09-10T10:56:55.717 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74766.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66961.log 2024-09-10T10:56:55.718 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74766.log.gz 2024-09-10T10:56:55.718 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79405.log 2024-09-10T10:56:55.719 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40623.log.gz 2024-09-10T10:56:55.719 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69466.log 2024-09-10T10:56:55.720 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66961.log.gz 2024-09-10T10:56:55.721 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.79405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70121.log 2024-09-10T10:56:55.721 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79405.log.gz 2024-09-10T10:56:55.722 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51414.log 2024-09-10T10:56:55.722 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69466.log.gz 2024-09-10T10:56:55.723 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.70121.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70121.log.gz 2024-09-10T10:56:55.723 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71562.log 2024-09-10T10:56:55.723 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.51414.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59495.log 2024-09-10T10:56:55.725 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51414.log.gz 2024-09-10T10:56:55.725 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.71562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78636.log 2024-09-10T10:56:55.725 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71562.log.gz 2024-09-10T10:56:55.726 INFO:teuthology.orchestra.run.smithi028.stderr:gzip/var/log/ceph/ceph-client.admin.59495.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50915.log 2024-09-10T10:56:55.726 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5%/var/log/ceph/ceph-client.admin.78636.log: -- replaced with /var/log/ceph/ceph-client.admin.59495.log.gz 2024-09-10T10:56:55.727 INFO:teuthology.orchestra.run.smithi028.stderr:gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78636.log.gz -5 2024-09-10T10:56:55.727 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T10:56:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-09-10T10:56:55.729 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50915.log.gz 2024-09-10T10:56:55.729 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52332.log 2024-09-10T10:56:55.731 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.50053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52731.log 2024-09-10T10:56:55.731 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-09-10T10:56:55.732 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52332.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52332.log.gz 2024-09-10T10:56:55.732 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58326.log 2024-09-10T10:56:55.733 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.52731.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52731.log.gz 2024-09-10T10:56:55.733 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59117.log 2024-09-10T10:56:55.735 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66102.log 2024-09-10T10:56:55.735 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58326.log.gz 2024-09-10T10:56:55.736 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59117.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59117.log.gz 2024-09-10T10:56:55.736 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T10:56:55.737 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66102.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44463.log 2024-09-10T10:56:55.738 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66102.log.gz 2024-09-10T10:56:55.739 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43728.log 2024-09-10T10:56:55.740 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44463.log.gz 2024-09-10T10:56:55.741 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62755.log 2024-09-10T10:56:55.741 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43728.log.gz 2024-09-10T10:56:55.742 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59411.log 2024-09-10T10:56:55.743 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62755.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62755.log.gz 2024-09-10T10:56:55.744 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82657.log 2024-09-10T10:56:55.745 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.59411.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59411.log.gz 2024-09-10T10:56:55.746 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51112.log 2024-09-10T10:56:55.747 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82657.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.82657.log.gz 2024-09-10T10:56:55.748 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83377.log 2024-09-10T10:56:55.748 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.51112.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51112.log.gz 2024-09-10T10:56:55.749 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-09-10T10:56:55.750 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83377.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83377.log.gz 2024-09-10T10:56:55.751 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58243.log 2024-09-10T10:56:55.752 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46265.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-09-10T10:56:55.753 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62052.log 2024-09-10T10:56:55.754 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.58243.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58243.log.gz 2024-09-10T10:56:55.755 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-09-10T10:56:55.756 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62052.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62052.log.gz 2024-09-10T10:56:55.757 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57044.log 2024-09-10T10:56:55.758 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46559.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-09-10T10:56:55.758 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83890.log 2024-09-10T10:56:55.759 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.57044.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57044.log.gz 2024-09-10T10:56:55.760 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64307.log 2024-09-10T10:56:55.761 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83890.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83890.log.gz 2024-09-10T10:56:55.761 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2024-09-10T10:56:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64307.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64307.log.gz 2024-09-10T10:56:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66822.log 2024-09-10T10:56:55.765 INFO:teuthology.orchestra.run.smithi028.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2024-09-10T10:56:55.766 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.66822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66822.log.gz 2024-09-10T10:56:55.767 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-09-10T10:56:55.768 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73331.log 2024-09-10T10:56:55.769 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42965.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-09-10T10:56:55.770 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65965.log 2024-09-10T10:56:55.771 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.73331.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.73331.log.gz 2024-09-10T10:56:55.772 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.65965.log: /var/log/ceph/ceph-client.admin.64642.log 2024-09-10T10:56:55.773 INFO:teuthology.orchestra.run.smithi028.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.65965.log.gz 2024-09-10T10:56:55.773 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54808.log 2024-09-10T10:56:55.775 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.64642.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64642.log.gz 2024-09-10T10:56:55.775 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39946.log/var/log/ceph/ceph-client.admin.54808.log: 2024-09-10T10:56:55.776 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54808.log.gz 2024-09-10T10:56:55.777 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72620.log 2024-09-10T10:56:55.778 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39946.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39946.log.gz 2024-09-10T10:56:55.779 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63438.log 2024-09-10T10:56:55.781 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.72620.log: /var/log/ceph/ceph-client.0.63438.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.72620.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.63438.log.gz 2024-09-10T10:56:55.781 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:55.782 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62126.log 2024-09-10T10:56:55.783 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80946.log 2024-09-10T10:56:55.785 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.62126.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.62126.log.gz 2024-09-10T10:56:55.785 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.80946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69414.log 2024-09-10T10:56:55.786 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80946.log.gz 2024-09-10T10:56:55.787 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74671.log 2024-09-10T10:56:55.788 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.69414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69414.log.gz 2024-09-10T10:56:55.789 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67540.log 2024-09-10T10:56:55.790 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.74671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74671.log.gz 2024-09-10T10:56:55.791 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.67540.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67540.log.gz 2024-09-10T10:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T10:56:56.216 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T10:56:56.219 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T10:56:56.517 INFO:teuthology.orchestra.run.smithi138.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T10:56:56.585 INFO:teuthology.orchestra.run.smithi028.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T10:56:56.761 INFO:teuthology.orchestra.run.smithi028.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T10:56:56.837 INFO:teuthology.orchestra.run.smithi138.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T10:56:56.840 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-10T10:56:56.841 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m2.066s 2024-09-10T10:56:56.841 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m6.991s 2024-09-10T10:56:56.841 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.681s 2024-09-10T10:56:56.866 INFO:teuthology.orchestra.run.smithi028.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T10:56:56.927 INFO:teuthology.orchestra.run.smithi028.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T10:56:57.613 INFO:teuthology.orchestra.run.smithi028.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T10:56:57.616 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-09-10T10:56:57.616 INFO:teuthology.orchestra.run.smithi028.stderr:real 0m2.845s 2024-09-10T10:56:57.616 INFO:teuthology.orchestra.run.smithi028.stderr:user 0m8.844s 2024-09-10T10:56:57.616 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m2.805s 2024-09-10T10:56:57.617 INFO:tasks.ceph:Archiving logs... 2024-09-10T10:56:57.618 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi028/log 2024-09-10T10:56:57.619 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:56:59.495 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi138/log 2024-09-10T10:56:59.496 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:57:00.553 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:57:00.562 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:57:00.562 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:57:00.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:57:00.679 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:57:00.688 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:57:00.688 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:57:00.693 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:57:00.724 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:57:00.726 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:57:00.732 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:57:00.732 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-09-10T10:57:00.732 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -347us[ -347us] +/- 43ms 2024-09-10T10:57:00.732 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +142us[ +142us] +/- 70ms 2024-09-10T10:57:00.732 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +473us[ +474us] +/- 71ms 2024-09-10T10:57:00.733 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -349us[ -335us] +/- 43ms 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +124us[ +138us] +/- 70ms 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +478us[ +478us] +/- 71ms 2024-09-10T10:57:00.734 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:57:00.735 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:57:00.743 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:57:00.744 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:57:00.752 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:57:00.920 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:57:01.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:57:01.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:57:01.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:57:01.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:57:01.158 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:57:01.168 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:57:01.176 INFO:teuthology.task.internal:Duration was 1271.293666 seconds 2024-09-10T10:57:01.176 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:57:01.188 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:57:01.188 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:57:01.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:57:01.275 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:57:01.279 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:57:01.773 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:57:01.774 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-09-10T10:57:01.774 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:57:01.824 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-09-10T10:57:01.825 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:57:01.875 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:57:01.876 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:57:01.880 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:57:01.938 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:57:01.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:57:01.943 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:57:02.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:57:02.129 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:57:02.129 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:57:02.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:57:02.241 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:57:02.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:57:02.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:57:02.323 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-09-10T10:57:02.327 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-09-10T10:57:02.356 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:57:02.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:57:02.390 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:57:02.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:57:02.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:57:02.433 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:57:02.433 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi028 2024-09-10T10:57:02.433 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:57:02.498 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899045/remote/smithi138 2024-09-10T10:57:02.498 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:57:02.561 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:57:02.561 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:57:02.565 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:57:02.600 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:57:02.611 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:57:02.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:57:02.622 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:57:02.622 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:57:02.642 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:57:02.672 INFO:teuthology.orchestra.run.smithi028.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:57 /home/ubuntu/cephtest 2024-09-10T10:57:02.676 INFO:teuthology.orchestra.run.smithi138.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:57 /home/ubuntu/cephtest 2024-09-10T10:57:02.678 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:57:02.688 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:57:02.711 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/damage} duration: 1271.2936658859253 failure_reason: 'Test failure: test_open_ino_errors (tasks.cephfs.test_damage.TestDamage)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4af3294f138848799d961e6845529271 status: fail success: false 2024-09-10T10:57:02.711 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:57:02.793 INFO:teuthology.run:FAIL